Mirrors vs Braintrust

What Braintrust is good at

• The experiment loop is the cleanest in the category. Dataset, task, scorers, run, compare against the last run, and the autoevals library ships the scorers so you are not writing a judge from scratch on day one.
• Brainstore, their purpose-built store for traces, stays fast to query at volumes where general-purpose stores stop being pleasant.
• Loop, an in-product agent that helps write scorers and work through results, which is a real answer to the fact that most teams stall on defining what good looks like.
• Unlimited seats and projects on every tier including the free one, so nobody has to ration access to the tool that tells them whether the change was good.
• Human review workflows and CI integration, plus an $80M Series B led by ICONIQ in February 2026, which is a reasonable proxy for how long the platform will be around.

Side by side

Prices and limits checked against braintrust.dev on 2 August 2026. Because the metering units differ (data processed and scores, against sandbox time here), the pricing row is a description of two models rather than a like-for-like number.

The actual difference

Braintrust is the scoring layer. You bring outputs, it tells you whether they got better or worse, and it does that with less ceremony than anything else here. It is deliberately agnostic about where the outputs came from.

Mirrors is where they come from. It reads your traces, your code, or your API docs and stands up the systems the agent calls: a running service per tool, with a schema, fabricated seed records, and state that persists through a session and resets between runs. That is what makes a replay repeatable, and repeatability is the precondition for the comparison Braintrust is so good at.

So the two products are not really answering the same question. Braintrust asks whether this output was good. Mirrors asks where the agent can act while you find out.

Braintrust's objection, in their words

Braintrust does not think building the environment is a good use of a team's time, and they published the argument. From "How to eval stateful agents", 26 June 2026:

"A customer support agent that's supposed to look up orders and issue refunds needs an actual order database to look things up in and an actual refund system to issue refunds through."

"Building a perfect replica of your production environment for eval purposes is expensive to set up and difficult to maintain. Every time the real environment changes, your eval environment needs to change too. Teams that try this method often spend more time maintaining their eval infrastructure than actually improving their agents."

Their advice follows from that, and it is three words: "Don't chase perfect replay."

We agree with the first quote completely. That is the premise of this company: the agent needs an actual order database and an actual refund system, or the test is not a test. The disagreement is about who builds it.

The post is describing a replica a team writes and maintains by hand, and about that it is right. A hand-built fake rots. Every schema change in the real system becomes a ticket against the fake one, nobody files it, and six months later the eval passes against a system that no longer exists. We have watched it happen too.

Mirrors is not hand-built. The environment is constructed from your traces, your code, or your API docs, and rebuilt from those same sources when they change, which is precisely why it does not accumulate the maintenance debt the post describes. And the drift is measurable rather than assumed: the replay view diffs each recorded production call against its counterpart in the environment, so you can see where the copy has stopped matching before you trust it with a merge gate.

On the third quote we do not disagree at all. A perfect replica is the wrong goal, and chasing one is how teams end up maintaining infrastructure instead of improving agents. What a regression test needs is an environment faithful enough on the paths the agent actually takes, plus an honest measurement of where it is not. That is what we build.

Read the Braintrust post in full

When to use Braintrust instead

If you already have somewhere safe for your agent to run, or your agent only reads and never writes, Braintrust is the better buy and it is not close. The eval loop is sharper than ours, the scorer library is broader, and unlimited seats means the whole team can look at the same experiment.

Use it also if the volume of data you process is the thing you want to pay for rather than the time an agent spends executing, or if Loop and Brainstore are solving a problem you have today. And if you accept their argument that the environment is not worth building, then Braintrust plus a careful staging setup is the coherent version of that position: you should not buy Mirrors half-convinced.

Most teams run both

They compose the way the objection above implies they should. Mirrors produces the run: the agent acting on a rebuilt environment where a wrong refund is safe to make and can be made a hundred times. Braintrust grades those runs and tells you whether the change helped.

A team running both writes its scorers in Braintrust once, then points them at output that came from an environment nobody has to keep in sync by hand.

What this page compares, and what it does not

These five are the observability and evaluation platforms a team already has open when it asks how to test an agent, which is why they are the pages we wrote. They are not the whole field. Tools that do stand something in for an agent's counterpart, LangWatch's Scenario and Laminar's agent rollouts among them, are outside this comparison, and nothing here is a claim about them.

Standing in for a live dependency is also not a new idea. Record-and-replay HTTP mocking, VCR-style cassettes, predates LLM agents by about fifteen years and is free. What Mirrors adds is the construction and the state: the environment is built from your traces, code, or docs instead of hand-written, and it is a seeded, stateful backend you can reset between runs instead of a fixed transcript of responses.