Estimated vs Reconciled

Why it matters

Every agent carries an opinion of its own work. It reports high confidence, its evals pass, and it marks the run a success. That is the estimated number. The reconciled number is what survives contact with reality, once the outcome has actually settled. When the two disagree, the estimate was optimistic, and the size of the disagreement is the honest measure of how much you can trust the system's self report.

Take a collections agent that decides which overdue accounts to chase and how. Across a quarter it estimates it resolved 95% of the accounts it worked. When those accounts are reconciled against what customers actually paid, the real figure is 70%. The agent was overconfident by 25 points. Nothing in the estimate was visibly broken. The accounts it counted as resolved simply did not pay, and only the reconciled view shows it.

Business context

Every operating review runs on numbers a team reports about its own performance, and an autonomous system is no different: it reports the version that flatters. When those self-reported numbers drive staffing, forecasts, and where the next investment goes, a persistent overconfidence quietly misallocates real resources. A leader who can set the estimated figure beside the reconciled one is managing to reality; a leader who cannot is managing to a report the system wrote about itself.

Where it fits

This comparison is the readout of reconciliation. The estimated side comes from the agent; the reconciled side comes from the settled business outcome. A persistent gap between them is exactly the Confidence Gap made numeric.

When it applies

Any time a system scores its own work and the true result arrives later. Until you can put the two numbers side by side, you are managing to the estimate, and the estimate is the one number that can be wrong in your favor without anyone noticing.

Common misunderstandings

It is not a margin of error. The gap is directional, not random noise. The estimate tends to run optimistic in the system's own favor.

It is not fixed by recalibrating confidence. Lowering the reported number does not close the gap unless the reconciled result actually improves.

It is not visible in the estimate alone. The estimate can look healthy on its own. The gap only appears once the reconciled number sits next to it.

Related concepts

In the product