Did that change break your agent, or is it just noise?
DeltaProof checks AI agents in CI. It runs each case as many times as the statistics need, checks every run against rules you write, and answers PASS, FAIL or INCONCLUSIVE, with signed evidence anyone can verify offline.
Every run of an agent differs a little. The question is whether a change moved it further than chance would.
A bad change, caught and fixed.
A new prompt makes a refund agent skip the order lookup on some multi-item refunds. Here's what DeltaProof does about it.
The gate fails and says exactly why: which rule broke, in how many runs, and on which kind of input.
Pass, fail, or not yet.
Most checks force a yes or a no. DeltaProof says when it doesn't know, instead of guessing.
No real change.
The differences are within what the model does on its own.
Something really broke.
You get the rule it broke, the failing runs and the inputs that trigger it.
Not enough evidence yet.
It says so. Run more trials or raise the budget, and ask again.
Measured, and you can re-run it.
Seeded benchmark: 6 synthetic agents × 7 kinds of regression, deterministic and reproducible from the repository. Dollar figures are priced token counts, not invoices. Verdicts are evidence with stated uncertainty, not guarantees.
Put your next agent change on trial.
Open source under Apache-2.0. Install it, write down the rules your agent must follow, and gate your next pull request.