How to check an AI savings claim, step by step

A claim, for this purpose, is one sentence that states a number about what AI does to cost, time, headcount or output, presented as fact. Copy it word for word, with the page it appeared on and the attribution printed beside it. A paraphrase quietly chooses a verb and a denominator for the original, and those are the two things the later steps test.

Then notice what kind of number the publisher says it is. A figure the publisher labels as its own forecast, with words such as could, estimated or potentially, has already told you it is not presented as fact. In the audit, labelling an estimate as an estimate was the clearest sourcing practice observed, and it took the claim out of scope.

The claim names a firm, a report, or nothing at all. Open the document itself and write down its title, publisher and date. A blog post quoting a report is not the report, and an aggregator quoting the blog is a step further away. If nobody can produce a title, a publisher and a date, the number has no source you can open, and that is a finding in itself.

Three shapes recur. The first is a firm's name used as a citation: a figure, a name in brackets, and no title, year or link. A name is not a citation, and a figure that arrives with a firm attached but no document has not been shown to come from the firm. The second is no attribution printed at all, which was the commonest shape in the audit, 24 of 43 claims: a number with nothing behind it. The third is an attribution that falls apart at the second hop, where the intermediary named as the source does not contain the number either. Following a citation one hop can feel like verification and not be.

Read the text around the claim before deciding it has no source. A source printed in the next sentence, under a chart or in a caption is easy to miss, and reporting a sourced figure as unsourced is a false finding about whoever published it.

Open the file and search it for the figure in each form it can be written, such as 20%, 20 percent and twenty percent. If the figure is present, you read the sentence it sits in. If it is absent, you have a finding, but only if the search could have found it.

That condition is the part most checks skip. Before accepting an absence, show that the search is live: the extraction returned the document's real text, and it found other numbers in the same file. In the ledger's first entry, a named and linked report was extracted in full and searched. The range attached to it occurred zero times, while the same extraction returned 75 distinct percentage figures, which is what makes the zero mean something. A search that returns nothing from a file it cannot read proves nothing, and the ledger's rule for that case is that a failed probe produces unverifiable, never not found.

The first claim to fail this check on the ledger was on this site. Entry A1 credited a range to a real report that did not contain it, and the ledger's own first label let a smaller, real figure from the same report stand in for the claim. On 2026-10-06 the claim audit re-ran the probe, the entry was relabelled not found, and a dated correction was printed above the tally. The method applies to whoever published the number.

A number being present in a document is not a verdict. In one audited document the figure appears only as chart axis labels, and the claim credited to it still fails. Presence has to be read; only absence is settled by the search and its control.

Then read the verb. Expect, forecast, predict, by 2029 and could are not measured. The real figure behind the failed range in entry A1 is smaller and describes what surveyed organisations expect within 18 months, not what anyone measured, and the source's own sentence says so.

The sharpest version is a forecast re-tensed as a result. Two ledger entries trace to a single prediction about the end of the decade, re-attributed to a different firm and written in the present tense. The ledger calls the tense change the worst of the transformations it names, because a prediction is being used to underwrite a budget decision this quarter.

Productivity, revenue, cost, headcount and elapsed time are five different things, and they get swapped for each other freely. A study that counts issues resolved per hour has measured throughput, not money. Whether extra throughput becomes a saving depends on what happens next: if the queue was the constraint it shows up as revenue, if headcount holds flat while volume grows into the slack it shows up as cost per contact, and if nothing downstream changes it shows up nowhere.

Then the denominator. A percentage of personnel expense, of total operating cost and of one process's cost are three different numbers that look identical in a sentence. A range can also be assembled from measurements that were never the same measurement. Ledger entry A5 traces only to vendors selling finance automation, each stating a different range against a different denominator, such as time per invoice, month end close time and share of manual work. No study produces the blended figure.

Last, the boundaries: one industry, one function, one company, one survey. A result measured in a single support organisation is another company's published result in its own terms. It is not a percentage to apply to your business.

A figure is checkable when the page that states it also states the sample, the period and who was asked. The audit found that rare: two publishers in the frame printed a method and a sample for their own survey beside the figures, and every figure traceable to one of those statements is marked verified. Everything else about an own-survey figure, however precise, could not be checked by anyone.

A company measuring its own product is evidence, but the weakest kind, and the kind that dominates search results on this subject. A printed method is what makes it checkable at all.

Check the version as well. The same study can exist as a working paper and as a journal article with different figures, and people mix numbers across drafts without saying so. Cite the version of record, and when you meet a quoted figure, find out which version the quoter read.

End each claim with one of four verdicts: verified, verified with correction, not found or unverifiable. Pair it with a disposition, which is publish, publish with the correction, or do not publish, and attach the caveat that must travel with the figure. Usually the caveat is the verb: this is what surveyed executives expect.

Not found means the document was read and the search was live. Unverifiable means you could not establish one or the other, because the document was not located or it refused you. Several publishers answer automated requests with an error. Record the status and stop. Do not retry, change headers or work around the block. A page that refused the request produces unverifiable, and it never supports a claim of absence.

A benchmark is a number produced by a company that is not yours, doing work that is not yours, counted by someone who was not you, under conditions nobody wrote down. Even a verified one is a weak guess about your business. What you can collect before anything is built is four numbers on the task itself: volume per month, taken from the system that already records it; time per unit, including the waiting and the chasing, timed over twenty units rather than estimated for one; the fully loaded hourly cost of whoever does it; and what breaks, meaning error rate, rework rate and how often it is late.

Multiply the first three and you have the current cost of that task in your business, a quantity no published percentage can compete with because it is measured on the real thing. Estimate it on your own numbers first, then measure the same four the same way, from the same system, one month and three months after launch. The difference is the result. Agree the baseline in writing before the build starts, and read it from a system that was already running, not from a spreadsheet built to make the case. Faster and wrong is not a saving.