Jay Pozo

Articles

Cheap AI Outputs Can Be Expensive Workflows

Measure AI by the cost of an accepted outcome. Include review, correction, integration, and operating work before deciding whether cheap output saves money.

Measure the cost of an AI workflow through to a result someone can use. Generating the output is only part of the work.

A reply can appear in seconds. Someone still has to check the reply and correct it before sending. I compare the complete task against an agreed quality standard before counting a business saving.

How do you calculate the cost of an AI workflow?

It depends on the outcome and its quality requirements. For a draft someone must check, include review and correction. For an automated action, include monitoring and recovery. Compare the same task boundary across alternatives, with setup shown separately from recurring costs. Keep consequential errors visible alongside the financial comparison.

As of October 2026, McKinsey reports that about 20% of respondents faced AI operating costs that constrained use.

Its survey covered 1,719 participants across 97 nations, with national responses weighted by GDP contribution. These are self-reported findings about AI use, rather than costs for a particular workflow.

For your own costs, start with the result you need. I define what counts as accepted, then record the effort needed to get there.

What counts as an accepted outcome?

First, define finished. A generated reply is ready to read. A reply ready to send has passed the checks the task requires. I want to measure through to that point.

Take a hypothetical customer-support assistant. Suppose the reply must answer the question accurately using current account information. The operator also needs to explain the answer. That gives us a standard to test.

I count rewrites and time spent checking another system. If our task ends when the customer receives help, I follow what happens after the message is sent too.

I apply the same standard to the alternatives. Manual replies can need correction too. The comparison has to include the checks each approach requires.

I define acceptance with the person responsible for the result. They can explain which mistakes are minor and which leave a reply unusable. That detail matters when we assess the output.

Which costs belong in the comparison?

Once the task is clear, I separate the costs like this:

Cost category What to include
Initial setup Connecting tools and preparing evaluation material
Each attempt Model use, retrieval, retries, and tool calls
Completing the task Review and correction
Continuing operation Monitoring and maintenance

This is a proposed planning structure, rather than a measured industry cost model.

I choose a planning period and state the expected workload. Then I check what happens with fewer cases, or more difficult ones. The conclusion should show those assumptions.

Setup stays visible. So does the specialist attention a pilot needs to keep running. A low per-case figure tells us less if the ordinary team needs that help every day.

How do you compare the options fairly?

I use this planning formula:

Cost per accepted outcome = total attributable cost / outcomes meeting the agreed quality standard

This is an accounting aid, rather than a research statistic. The numerator includes the cost of rejected attempts. The denominator includes only accepted results within the chosen scope.

I apply it to a credible alternative too. An improved manual process or a template gives us something useful to compare. Use similar cases and record any differences in conditions.

Google's Rules of ML recommends establishing metrics before formalising the system. For this comparison, define acceptance and record completion effort before choosing the implementation.

I keep the rejection reasons beside the costs. Routine requests may be cheap while exceptions take considerable effort. I want to see that difference before estimating a wider rollout.

Why does human review change the economics?

Review is work. The operator has to decide whether they can use the answer. Reading a fluent reply may be quick. Checking its claims can require another set of records.

For our hypothetical assistant, I show the relevant account information beside the proposed answer. Then I watch whether that helps the operator check it. The interface is part of the comparison.

As of October 2026, NIST's generative AI risk profile describes confidently false output and human interaction risks. Its risk categories give you questions to investigate in your application.

I test review directly, including cases with mistakes. Does the operator catch the error? Can they correct it before acting? Those are separate questions from whether the generated reply reads well.

If checking means reconstructing the answer from scratch, I count that effort. It is part of producing an accepted result.

Does faster work mean lower business costs?

Saved time gives the team capacity. Turning that capacity into lower spending or more completed work takes another step. I ask what the organisation can actually do with it.

Suppose the hypothetical support team handles replies faster but still waits for account corrections from another department. The complete customer outcome may barely change.

Can the team use the time to resolve more requests? Can it improve another part of the service? I put that assumption in the business case so we can check it.

Avoid treating every saved minute as cash immediately recovered. Show the operational change needed to turn the time into a benefit the business can use.

What should you test before expanding the workflow?

Run a bounded evaluation with the people expected to operate it:

  1. Define accepted outcomes and collect permitted examples of the task.
  2. Compare a simpler baseline with the proposed AI assistance.
  3. Record completion effort and explain rejected results. Include retries in the effort.
  4. Test exceptions and recovery, including who handles unresolved cases.

I track elapsed time separately from active effort. Less labour and a faster customer outcome are different benefits. Measure both if both matter to the decision.

Repeat the comparison when a material change affects quality or cost. Keep an owner for the evaluation material and the operating process.

The task and its boundaries should come from discovery, as described in Before the PRD. For simpler options, see Do You Need AI, or a Better Form?.

We connect success metrics to Requirements and Iterative development in the process. You can see the available offers or book a call with the task you are trying to make cheaper.

I count the work through to acceptance. That is where the cost comparison needs to end.

What else should you know about AI workflow costs?

What is cost per accepted outcome?

It is total attributable cost divided by the number of results meeting an agreed quality standard. Include failed attempts and correction in the cost.

Does a lower model price make the workflow cheaper?

It reduces one cost component. Check whether quality, retries, review, or maintenance change before expecting a lower cost for the complete task.

Should human review count as an AI cost?

Count review in every option being compared. Include existing checks and any additional checking introduced by AI, using the same task boundary.

How do we compare a pilot with normal operation?

Use representative cases and ordinary operators. Record setup separately, include continuing support, and test what happens when the usual operator is unavailable.

Frequently asked questions

What is cost per accepted outcome?

It is total attributable cost divided by the number of results meeting an agreed quality standard. Include failed attempts and correction in the cost.

Does a lower model price make the workflow cheaper?

It reduces one cost component. Check whether quality, retries, review, or maintenance change before expecting a lower cost for the complete task.

Should human review count as an AI cost?

Count review in every option being compared. Include existing checks and any additional checking introduced by AI, using the same task boundary.

How do we compare a pilot with normal operation?

Use representative cases and ordinary operators. Record setup separately, include continuing support, and test what happens when the usual operator is unavailable.

Four ways to start

  • Technical adviser

    I check the software a developer or agency builds for you, at any stage. I can step in before you sign or long after they hand it over.

    Starts at USD 2,000 · 10 hours

Check your agency before you sign · Free PDF

Book a call