Measuring AI ROI honestly

Every AI vendor has an ROI calculator, and they all produce impressive numbers. The problem is that most of them measure what's easy to claim rather than what actually changed in your business. If you want to know whether an AI project was worth it, you need a simple, honest method, and you need to set it up before the project starts.

Why AI ROI is easy to get wrong

A few traps catch almost everyone:

• No baseline. If you didn't measure how long something took before, you can't know how much time you saved.
• Counting time saved as money saved. Hours freed up only become value if they're used for something worthwhile.
• Ignoring the review step. If AI drafts something in seconds but a person spends ten minutes checking and fixing it, the saving is smaller than it looks.
• Measuring the demo, not the rollout. Results from a handful of hand-picked examples rarely match real, messy volume.
• Leaving out ongoing costs. Subscriptions, usage fees, maintenance and training time continue after launch.

Step 1: Record a baseline before you start

Pick the workflow and measure it as it runs today. You don't need a stopwatch study. A week or two of rough tracking is usually enough:

• Volume: how many times the task happens per week or month.
• Time: roughly how long each one takes, including waiting and handoffs.
• Quality: how often mistakes happen, and what it costs to fix them.
• Speed: how long customers or colleagues wait for the result.

Write it down and have the process owner agree it's fair. This baseline is the most valuable number in the whole project.

Step 2: Decide what success means

Before building anything, agree on one or two measures that matter and a threshold for success. For example: "Reduce average handling time per invoice by at least a third, with no increase in error rate." Or: "Respond to new inquiries within five minutes during business hours instead of the current typical wait of several hours."

Choosing the measure up front stops everyone from searching for a flattering number afterwards.

Step 3: Count the full cost

Add up everything it takes to get the result, not just the software:

The review line matters. If a person needs to check every output, that time is a real, ongoing cost. It often shrinks as the system improves and trust grows, so measure it rather than assuming.

Step 4: Measure after launch, the same way

Once the new process has been running on real work for a few weeks, measure it using the same method as your baseline. Compare like with like: the same kind of work, similar volume, the same definition of "done."

Measure quality as carefully as speed. A process that's twice as fast but produces more errors may not be an improvement, especially if errors reach customers.

Step 5: Be honest about where the time went

Suppose the project saves your team a meaningful number of hours each week. The key question is what happened to those hours. Common, legitimate answers include:

• The team handled more volume without hiring.
• People spent more time on higher-value work, such as sales, advising or quality control.
• Backlogs cleared and response times improved.
• Overtime or temporary help was reduced.

If you can't point to any of these, the time may have been absorbed without much benefit. That's worth knowing, and it's a management question as much as a technology one.

Value that's harder to count

Some benefits are real but hard to put a number on: faster responses to customers, fewer errors reaching clients, less tedious work for staff, and more consistent processes. Don't inflate these into precise dollar figures. Instead, describe them honestly and, where possible, track a simple indicator, like response time or the number of corrections needed.

Be equally honest about downsides, such as new failure modes, dependence on a vendor or extra steps for staff.

A simple ROI summary

At the end of a pilot or the first few months, a one-page summary should answer:

1. What did the workflow look like before? (baseline)
2. What does it look like now, measured the same way?
3. What did it cost to get here, and what does it cost to keep running?
4. What happened to the time or capacity that was freed up?
5. What got worse or needs watching?
6. Should we continue, expand, adjust or stop?

This is less exciting than a vendor's calculator, but it's a number you can defend to your leadership team, and it tells you where to invest next.

Measurement is built into every project we do. If you want help setting up a baseline or evaluating a tool you already have, book a free call.