How to Measure Whether Your Firm's AI Adoption Is Actually Working
Moving beyond "people are using it" to metrics that actually tell you something.
"People are using it" is not a metric. It describes activity, not progress. A firm where everyone uses AI to get mediocre output they have to rewrite is "using AI" in the same sense as a firm where people consistently produce better work faster. Those aren't the same outcome.
Measuring AI adoption requires moving from activity metrics (are people using it?) to outcome metrics (is the use producing anything different?). The distinction matters because activity metrics don't tell you whether to do more of the same or change course.
What not to measure
Metrics that feel concrete but aren't
Number of people using AI tools
Usage headcount tells you about access and habit formation, not about whether the use is producing value. A firm where 100% of staff use AI daily to get output they throw away is not more advanced than one where 40% use it to consistently improve their work.
Sessions or queries per month
Volume of AI interactions is an activity metric. High volume might mean people are getting value from AI. It might mean they're generating bad output repeatedly and trying again. The number doesn't distinguish between the two.
Satisfaction survey scores
People tend to report positive sentiment about AI tools regardless of whether the tools are changing their work. "I find it useful" often means "I tried it a few times and it wasn't terrible." Satisfaction surveys are too coarse to tell you whether AI use is compounding or stagnating.
What to measure instead
Four signals that actually indicate progress
Prompt library size and growth rate
The clearest signal that AI use is compounding is whether the prompt library is growing. A library that grows month over month means people are using AI, finding what works, and contributing it back. A library that stagnates means individual use is happening but nothing is becoming institutional. Track entries added per month and retrieval frequency - a library nobody opens isn't working.
Time to first usable draft on specific deliverables
Pick two or three deliverable types your firm produces repeatedly - client memos, research briefs, proposal sections. Measure how long it takes to get to a usable first draft before and after AI adoption for those specific tasks. This is more work to measure than headcount, but it's the only way to know whether AI is changing throughput on the actual work.
Prompt standard compliance
If you've established a shared prompt standard, spot-check whether people are using it. Pull five random prompts from your team's recent work and check whether they include role, context, task, format, and constraints - or some principled subset of those. The question isn't whether they wrote the perfect prompt. It's whether the habit is forming.
The new-hire calibration
A reliable qualitative signal: when a new person joins your firm, how long does it take them to be using AI effectively for their work? If it takes weeks or months of self-discovery, your AI infrastructure isn't working. If they can access the prompt library, read the standards, and be productive with AI within their first week, the institutional knowledge is compounding.
The baseline problem
Why most firms can't measure AI impact
The reason firms default to activity metrics is that they didn't measure the baseline before they started. If you don't know how long a client memo took to draft six months ago, you can't measure whether AI changed that. If you didn't document how prompts were being written before training, you can't evaluate whether standards improved.
Going forward: before any AI initiative or training, pick two specific deliverable types, time how long they take, and record what a typical prompt looks like. Even rough baselines make post-adoption measurement meaningful. Without them, you're evaluating by feel - which tends to drift toward whatever story people already believe about whether AI is working.