How to Evaluate an AI Development Partner for Your Firm

Most firms evaluating an AI development partner are doing it for the first time. They do not have a framework for it. They know roughly what they want to build, they have had a few conversations, and now they need to decide who to work with and whether the scope and price make sense.

This piece gives you the questions to ask - and more importantly, the answers that should make you pause.

The questions worth asking

Do they understand the work your firm does? Ask them to describe a typical engagement in your sector - what a week looks like for a mid-level person at a firm like yours. Vague answers are a signal. A developer who has never worked in or around professional services will describe the work at the level of the org chart. Someone who actually understands it will describe the specific friction points: the formats that clients expect, the review cycles, the judgment calls that happen before anything goes out the door.

How do they scope discovery? Discovery should be a structured process with a defined output - not a free-form series of calls that eventually turns into a proposal. Ask what the discovery phase produces. A discovery document that specifies the workflow, the edge cases, the output format, and the quality criteria is the foundation of a build that actually works. If the answer is that discovery is just a few conversations before they start building, that is worth noting.

What does the handoff actually include? This is the question most buyers forget to ask until it matters. When the engagement ends, do you get the code? Documentation written for someone non-technical? Access to your own environment, or theirs? If the tool lives on their infrastructure and requires their ongoing involvement to keep running, you have not bought a tool - you have bought a dependency. Ask specifically: if we stopped working together tomorrow, would the tool keep running, and could we modify it without you?

Who owns the work? Read the contract. IP ownership should be stated clearly - that everything built for you belongs to you, not to the builder. Some vendors retain rights to the underlying approach or methodology. That is sometimes reasonable for generic infrastructure; it is not reasonable for logic that encodes your firm's specific workflows and judgment. The question of ownership is not a negotiating tactic - it is a first-order consideration for anything you are going to run your work through.

Have they built something like this before? Not AI projects generally - projects for professional services firms with similar workflow complexity. There is a significant difference between someone who has built consumer applications and someone who has built tools for the specific kind of work your firm does. The edge cases in a contract review tool for a law firm are not the same as the edge cases in a consumer productivity app. Ask for examples from your sector. If they cannot provide any, ask how they plan to learn what they do not know about your work.

What does iteration look like after launch? You will want to change things. The tool will behave in ways that require adjustment. The prompt logic will need refinement based on real-world output. How does that work? Is there a support period? A fixed number of revision hours? An ongoing relationship? The answer tells you both what to expect and whether they have thought seriously about the post-launch reality.

Red flags

The no-discovery flag deserves extra attention. A vendor who wants to skip directly to building is either optimizing for their own speed or does not understand that the discovery document is what makes the build work. The most common source of expensive rework in AI development engagements is a specification that was too thin - requirements gathered in a few calls rather than a structured process. Most AI development engagements that fail fail here.

The SaaS flag is subtler. Some vendors frame their offering as building you a custom tool, but the delivery is a configured version of their platform - something that runs in their infrastructure and requires their ongoing involvement. That is a product, not a build. It may be the right choice for some firms, but it is a different thing from owning a tool outright. Know which one you are buying.

The conversation that tells you the most

The single most revealing thing a vendor can do in an early conversation is push back on your initial idea. Not to be difficult - but because they can see something you cannot. They have built similar things before. They know that the way you have described the problem is missing a constraint, or that the thing you think you want is not quite the thing that will solve the problem.

A vendor who accepts your brief uncritically and starts talking about timeline and technology is telling you something. Either they are in early sales mode and will get into the harder questions later, or they do not have enough domain context to know what to push back on. Neither is reassuring.

The vendor you want is the one who asks whether you have seen the edge case where X happens, or who tells you that in their experience firms like yours always underestimate Y. That kind of conversation is uncomfortable sometimes. It also produces better tools.

For a detailed look at how a well-structured engagement actually runs - from discovery through handoff - the 303 engagement walkthrough describes what that looks like in practice.