Outcome Drift
The slow slide where an agent keeps running the same way but stops getting the same results.
Outcome Drift is the gradual divergence between what an AI system used to achieve and what it achieves now, as models, data, and the world change.
Why it matters
An agent that was right at launch does not stay right by default. The model behind it gets swapped or updated. The data it sees shifts. The customers, the vendors, the regulations, and the competition all move underneath it. None of this changes the agent's code, so nothing looks broken. What changes is the result. A system can hold its behavior perfectly steady and still get worse, because the target it was tuned against has moved.
A support triage agent routes tickets to the right queue. At launch it lands the right queue nine times in ten. Over the next two quarters the product ships new features, customers start asking about things the agent has never seen, and the mix of incoming issues shifts. The routing rules never changed, but the share of tickets that end up in the right hands quietly falls. The drift is in the outcome, not in the run.
Business context
Drift is a governance problem disguised as a quiet quarter. Because nothing breaks, an agent can degrade for months while every status report stays green, and the loss shows up only in a slowly rising exception queue or a falling conversion number no one connects back to the automation. For a leader the risk is complacency: the systems most trusted to run unattended are the ones where a falling result goes unnoticed longest.
Where it fits
Outcome Drift is the reason Continuous Verification exists. A one time check confirms a system worked once; only repeated reconciliation catches the slide afterward. Left unwatched, drift is how a system accumulates silent failures without any single one setting off an alarm.
When it applies
Any system that runs long enough for its world to change, which is nearly all of them. Drift matters most where the behavior is stable and unattended, because that is exactly where no one is looking for a problem. The steadier the run, the more a falling result depends on someone measuring the outcome to be caught at all.
Common misunderstandings
It is not model drift. Model or data drift describes the inputs changing; outcome drift is about the result falling, whatever the cause.
It is not a broken system. The agent can run exactly as designed and still drift, because the world it was tuned against has moved.
It is not caught by a launch test. A check at go-live proves the system worked then, not that it still works now.
Related concepts
In the product
Related insights
Further reading
- Provy Research. Continuous Verification.
- Provy Research. AI failure modes.