Tool-call drift
Tool-call drift is the difference between the tool calls an agent made in a recorded session and the calls it makes when that session is replayed after a change. A call can be added, dropped, reordered, or made with different arguments. Drift is a signal, not automatically a defect.
Last updated 2 August 2026
The four shapes it takes
| Shape | What you see | What it usually means |
|---|---|---|
| Added call | The replay calls a tool the recording never did | A new instruction, a newly available tool, or the agent compensating for something it can no longer find |
| Dropped call | A call in the recording never happens on replay | The agent now believes it already has the answer, often because the prompt told it so |
| Reordered calls | The same calls in a different sequence | Usually harmless between reads; between writes it can change the outcome |
| Argument drift | The same call with different arguments | The interesting one: same shape, different amount, id, filter, or limit |
Drift is a signal, not a verdict
If you changed the prompt to make the agent verify an address before shipping, an added address lookup is the change working. Drift only becomes a defect when it is a difference nobody asked for, or when its blast radius is larger than the change that produced it.
That judgment is a human one, which is why a replay diff is worth reading rather than only passing or failing. A gate that treats every divergence as a failure gets switched off within a week.
Argument drift is the one to read closely
Added and dropped calls are loud. Argument drift is quiet: the same refund call, the same order id, a different amount. Or the same query with a row limit that quietly went from a hundred to ten, so the agent now reasons over a truncated result and sounds exactly as confident as before.
These are the regressions that reach production, because every surface-level check still passes. The call was made, it succeeded, the answer reads well.
Establish the noise floor first
Some drift has nothing to do with your change. Replay the same session twice against the same version and compare the two: whatever differs is that session’s noise floor, produced by sampling, retrieval order, timestamps, or a tool that returns rows in an unstable order. Measure it before gating anything on drift, or the first red build will be the model being a model.
A session with a high noise floor is usually telling you something real: an underspecified prompt, or a tool whose output the agent has to guess at.
How it differs from the terms it gets confused with
| Term | What it describes |
|---|---|
| Tool-call drift | A replayed session diverging from its recording after a deliberate change |
| Hallucination | The model inventing a fact, a tool, or an argument value with no basis in what it was given |
| Flakiness | The same version behaving differently run to run, with no change involved |
| Data drift | The distribution of real inputs moving away from what a model was built for |
| Schema drift | A tool or API changing its contract underneath the agent |
They interact. Schema drift usually surfaces as tool-call drift first, because the agent starts calling a tool differently the moment the tool starts answering differently.
Reading a drift diff
Mirrors replays a recorded session against a change and diffs it call by call, so drift is what the Replay screen shows rather than something you reconstruct from two log files. Nothing about the concept needs Mirrors: the same information is in any pair of traces. The work is lining them up so the difference is the only thing you are looking at.
Related terms
- Agent regression test: A replay of a recorded session against a change, checking that behavior a team already relies on still holds.
- Seeded environment: Fabricated data with the shape of the real thing: same schema and relationships, invented values.