Optimization Theater: Five Tells That Your AI Dashboard Is Lying to You

Optimization theater is the practice of improving a measurement rather than an outcome. It is rarely dishonest. It is what happens when a team is asked to show progress on a schedule that does not match the speed of real progress, so the measure bends toward what is available.

The cost is not the wasted spend. It is the credibility damage when a board eventually asks the follow-up question and nobody can answer it. Here are the five tells and the counter-question for each.

Tell one: the metric did not exist before the deployment

A new metric appears alongside the new tool. Time saved. Content pieces generated. AI-assisted interactions. None of these existed in last year's operating review, and none of them appear in any plan the board approved.

Counter-question: what was this number twelve months ago? If it cannot be computed, the metric cannot show improvement, only existence.

Tell two: the numerator moved and the denominator vanished

Meetings booked went up. Meetings booked per thousand contacts touched went down, and that ratio is not on the slide. Reply rate improved on a list one tenth the size. Conversion improved on a segment quietly narrowed.

Counter-question: what is the denominator, and how did it change over the same period? Rate metrics without a stated base are the single most common form of theater.

Tell three: the cohort changed between the before and the after

The pilot ran with three volunteers who wanted it to work. The comparison group is the rest of the team, who did not. Or the pilot covered enterprise accounts and the baseline covered all segments. Selection does the work the tool was supposed to do.

Counter-question: were the two groups comparable before the deployment, and can you show one pre-period metric where they matched?

Tell four: activity replaced outcome

The report is full of things that happened and empty of things that changed. Emails sent, summaries generated, calls transcribed, seats activated. This is the failure mode a well-known operator post described precisely: definitions you made up, presented instead of revenue you produced.

Counter-question: which line in the revenue plan moved, by how much, and over what period? If the answer requires two steps of inference, treat it as unproven. The discipline is the same one in reporting AI impact to the board.

Tell five: the comparison window was chosen after the fact

Results are shown against the weakest available quarter, or against a period that included a hiring gap, a pricing change or a seasonal trough. Nobody agreed the window in advance because agreeing it in advance would have removed the option to pick.

Counter-question: when was the comparison window agreed, and by whom? Write it into the pilot charter before the pilot starts and this tell disappears permanently.

Why smart teams do this

Three structural pressures, none of them malicious.

1. Reporting cadence outruns effect size. Real revenue effects take two to four quarters. Board decks come quarterly. Something has to fill the gap.
2. No baseline was captured. Once the before is gone, every honest analyst is stuck constructing a comparison, and constructed comparisons drift toward the flattering one.
3. The sponsor is also the evaluator. When the person who approved the spend also grades it, incentives do the rest without anyone deciding to mislead.

The structural fix is to separate the sponsor from the evaluator and to capture the baseline before the tool is switched on. The measured shape of this gap across the category is in the proof gap research.

The one-page format that ends it

One page per AI initiative, same fields every time, reviewed on the same cadence as pipeline.

• The claim. One sentence, naming the business metric.
• The baseline. The value before deployment, with the date it was captured.
• The current value. With the denominator stated.
• The comparison window. Agreed date, agreed by name.
• The counterfactual. What else changed in the period that could explain the movement.
• Cost. Licence, data, human hours, remediation.
• Kill criteria status. Triggered, near, or clear.

Any initiative that cannot fill this page is not ready to be reported. That is not a bureaucratic standard, it is the minimum evidence a sceptic needs, and the sceptic in your board meeting is the one whose opinion decides the next budget.

Related: Reporting AI impact to the board · Pipeline truth test · Optimization theater framework · Skill: detecting optimization theater