How Provy Works
Provy connects to the agent runs you already produce and tells you whether they actually worked. Three things go in: your runs, the contract for what a good run looks like, and the real outcome once it lands. Provy grades each run, reconciles what it looked like against what really happened, and, when a clean-looking run still missed, names the agent behind it. Three things come out: a trust score, the Ledger of reconciled value, and the causes with a verified fix. This guide walks the whole path in plain terms, with one diagram.
Every observability tool tells you what your agents ran. Provy tells you whether the run actually worked, by comparing it to the real outcome. You send the runs you already produce, you say once what a good run looks like, and you connect the outcome your own records already hold. Provy does the rest: it grades each run, reconciles the estimate against reality, and when a clean-looking run still failed it names the agent behind it. You get back one trust score, a Ledger of what the workflow is worth once outcomes are reconciled, and the specific cause to fix with proof that the fix held.
- Three inputs: your agent runs, a success contract, and the real outcome. Provy invents none of them.
- Two steps: grade each run against the contract, then reconcile the estimate against reality.
- Three outputs: a trust score, the Ledger of reconciled value, and fixes proven by the real outcome.
- It is generic. A trading desk sees dollars; a support team sees resolved contacts. No agent name or metric is hardcoded.
What goes in
Provy reads three things, and it produces none of them itself.
Your agent runs. The steps, tool calls, model calls, and decisions your agents already make. You send them with one line of SDK code or a REST call, from any framework and any model. Nothing about your agents changes, and you can start in shadow mode so there is no production risk while you watch.
A success contract. A plain list of the conditions a good run has to meet, for example "the payment matched the approved amount" or "the ticket was resolved without a callback." You describe success in words and Provy drafts the conditions, mapped to the signals your agents already report. This is said once per workflow, not per run.
The real outcome. What actually happened, connected from the system that already holds it: the settled trade, the closed ticket, the paid claim. Provy does not watch your business. It compares the run to the outcome you connect.
What Provy does
Between the inputs and the outputs are two steps.
1. Grade. As each run finishes, Provy scores it against the contract, condition by condition. A run that meets every condition looks good; a run that misses one did not succeed, even if it ran cleanly. This grade is an estimate. It is what the run looked like at the time, before the outcome is known.
2. Reconcile. When the real outcome arrives, Provy lines it up against the estimate for the same work item. The estimate either matched reality or diverged from it. Reconciliation is the heart of Provy: it is the difference between "the run looked fine" and "the run was right." Until an outcome reconciles, a number stays an estimate and is labeled as one.
The case that matters most is the run that passed every check and still produced the wrong outcome, a silent miss. When that happens, Provy points you to the step behind it, so a fleet-level number becomes a specific thing to fix, and it checks that the fix holds against the next real outcomes. When a run holds no clear cause, Provy says so plainly rather than guessing.
What comes out
These produce three answers, each on its own surface.
A trust score. On the Command Center, one 0 to 1000 score answers "can you rely on this fleet," as a weighted blend of six factors, with a calibration bar showing how the estimate compared to what proved out. A positive band is shown only after enough outcomes have reconciled to earn it, never a green "Trusted" on unconfirmed runs.
The Ledger. The exec value page. It states what the workflow is worth once outcomes are reconciled: what was delivered, what was lost, what is still at risk, and the net confirmed value. A trading desk sees dollars from the real profit and loss; a support team sees resolved contacts, and dollars too once it sets what a wrong outcome costs. The Ledger also names which agent is behind the loss and projects where the value is heading at the current rate.
The causes and the fix. Workflow Health lines up every agent and every tool it calls in one view, worst first, so you see where trouble concentrates before you dig in. It tracks two separate questions: did each call work, and, separately, was that call ever the reason a real outcome came out wrong. A tool can succeed every time and still be the reason behind a miss, when its result was ignored, went stale, or fell back to a default further down the run. Provy keeps the two apart rather than folding them into one score, then groups the misses into causes, each with the responsible step and a verify-first fix. A fix counts only when the real outcome improves on the runs that follow, not when a check turns green. Provy watches the next runs and marks the fix improved, no change, or worse.
It works for any pipeline
Nothing in Provy is tied to a domain. It never hardcodes an agent name, a pipeline shape, or a business metric. A run is a set of decisions; an outcome is an opaque result the tenant connects; a contract is a list of conditions mapped to signals the agents report. The same product serves a six-agent trading pipeline whose outcome is a profit-and-loss number and a three-agent support pipeline whose outcome is a label of resolved or escalated. The Ledger prices the first in dollars directly and the second in dollars only once the team sets what a wrong outcome costs, seeded from a cited industry figure for support and left blank everywhere the figure would be a guess.
What Provy does not claim
Provy compares a pre-written contract against the settled outcome the tenant connects. It does not observe your systems, and it does not fabricate an outcome that has not arrived. A prediction with no reported outcome stays unresolved rather than being scored as a pass or a fail. When a miss has no cause visible in the run, Provy says so instead of guessing one. The honesty is the point: a number you can defend is worth more than a number that looks good.
Notes & references
This guide is the plain-language overview. The deeper papers develop the ideas it rests on: Decision Assurance for the discipline, Why Observability Is Not Enough for the gap it fills, and Outcome Intelligence for measuring the result rather than the activity. The mechanics of the reconcile step have their own concept page, Reconciliation.
Version history
- v1.0 · July 2026 · First edition of the how-it-works field guide.
- v1.1 · July 2026 · Expanded the Workflow Health section: call health and outcome involvement are tracked as two separate signals, not one.
How to cite
See it on your own runs
Send a week of agent runs, say what a good one looks like, connect the outcome your records already hold, and watch the estimate reconcile against reality.
Get a demo