How to evaluate a digital product agency before signing a big contract

Commissioning a large digital product build is one of the heaviest decisions an organisation makes. Not because agencies are untrustworthy, but because the wrong choice costs months of work, a significant budget, and internal political capital to unwind.

At Livewall, we regularly sit across from clients who started a project with an agency that looked excellent on paper. Great website, impressive portfolio, smooth sales conversations. And yet it went wrong. Sometimes in the technology, sometimes in the collaboration, sometimes as early as the briefing.

This article is not about finding the cheapest or the largest digital product agency. It is about the questions that reveal real quality long before you sign anything.

Look at what they actually built, not what they show you

A portfolio tells you what an agency wants you to believe. Always ask to see the working products behind the beautiful screens. Which cases are live and accessible? Can you actually use the application?

Also ask: who built this? Agencies grow and shrink. The team that built Sportvisunie or Dumpert is not necessarily the team that will work on your project. Ask for the names and roles of the people who will actually be assigned to your work.

A strong agency will let you speak directly with the technical lead, the UX designer, and the project manager. Not just the account manager who closes the contract.

How do they handle ambiguity and change?

Digital products change during the build. Always. User research reveals new insights, stakeholders shift priorities, technical choices prove suboptimal in retrospect. How an agency handles that uncertainty matters more than their technical credentials.

Ask directly: how do you respond when we need a fundamental scope change halfway through? An honest agency will explain how they discuss, price, and adapt to that. An agency that says they can handle anything without describing the process deserves extra scrutiny.

Also ask about their approach to rapid prototyping. Do they work prototype-first, where something working is the starting point rather than a 100-page specification? Or do they only start building once everything is documented? The first approach handles change better. The second generates paperwork that goes stale quickly.

At Livewall, we always start small: something that works, something you can test. Then we grow it based on what we learn.

What is their view on technology?

An agency that solves every brief with the same technology stack is an agency fitting clients into its own mould, not the other way around. Ask why they are proposing the technology they recommend. What alternatives did they consider? What are the downsides of their preference?

Also pay attention to how they talk about AI. An agency that does not mention AI is behind the curve. An agency that treats AI as the answer to everything does not understand it either. The interesting agencies can explain precisely where AI accelerates their web application development and where human judgment remains essential.

Our sister label Mach8 focuses on AI automation and AI-first products. We bring that in when it is genuinely relevant, but we always start with the question: what does this solve for the user?

A technology choice without a sharp answer to that question is not a choice. It is a guess.

How do they measure success?

This is the question that reveals the most about an agency and gets asked the least. After the pitch, ask: how will you know whether this project succeeded? Not at handover, but six months later.

An agency that answers with delivery dates and bug-free releases is thinking in outputs. An agency that answers with usage data, retention, business impact, and goals you have defined is thinking in outcomes. That distinction shapes the entire collaboration.

Also ask: do you continue working after go-live, or is that a separate commercial engagement? The best digital products are never finished. They grow based on data and user behaviour. Scale-up development matters as much as the initial build. An agency that does not raise this in conversation treats your product as a project with an endpoint.

The commercial signals that actually matter

A big pitch with many slides and a fixed project price sounds appealing. But a fixed price on a complex digital product build is almost always a fiction. The agency has priced in the uncertainty, or they will recover the loss later through change requests.

Ask instead how they handle scope discussions. Do they have a transparent system for tracking and pricing changes? Do they work on weekly goals with a small, focused team?

At Livewall, we work with small teams of no more than three per project, with weekly goals. That keeps collaboration direct and progress visible. No elaborate reporting, just a product you can see growing every week.

Also ask about references they would rather not give you. Ask about a project that did not go well and how they resolved it. An agency that cannot name a failure has never thought honestly about one.

The questions you should actually ask

To make this practical: these are the questions that Livewall finds most valuable when clients ask them of us.

Who specifically will work on my project? Ask for names, backgrounds, and availability. Not a general team profile.

Can we see an existing product you built and speak with the client? References chosen by the agency are less useful than a conversation you initiate yourself.

What does the first four weeks of the project look like? A concrete answer here reveals more about working style than any methodology presentation.

What does your team do if you fundamentally disagree with our direction? Agencies that always agree are more dangerous than agencies that push back.

How does handover work if we ever switch agencies? The way an agency answers this tells you everything about how they approach technical debt and ownership.

These questions are not meant to trip agencies up. They are meant to start the conversation you need in order to make a good choice. A good digital product agency will welcome them.