Every release compared with the last approved one.
Run suites on demand, on a schedule or from a pipeline, against many agents and projects at once. Every run records the version, model and environment; repeated trials give confidence intervals, so a regression is a real change and not model noise. Load tests replay whole conversations and score quality under load.
Runs & regression · included from the Team plan
What it does
- On demand, scheduled (with blackout windows and dependencies) or triggered by CI/CD, a knowledge change or a model change
- Many agents and projects in parallel, within each agent’s concurrency limit
- Baselines per agent version; test-by-test comparison of any two runs with confidence intervals
- Drill down from a run to the exact response, score, reason, claim and source
- Load, stress, spike and soak tests: p50/p95/p99, time to first token, errors, task completion and step efficiency by user load
- Quality gates return pass, warn or fail to the pipeline; an override needs an approver and a reason
Works with
- Evaluation
Scores you can defend, by meaning — not by wording.
See runs & regression with your data
We’ll connect a test environment of your agent and a few of your policy documents for the demo.
Request a demo