Know your AI agents are right — before your members do.
Sepal Assure tests chatbots, virtual assistants and task agents against the documents they’re supposed to answer from. It scores every answer by meaning, checks each claim and citation, compares every release with the last approved one, and keeps watching once the agent is live.
| Agent | Version | Score | Pass rate |
|---|---|---|---|
| Ask Bluebonnet Copilot Studio | 2.3.0 | 98.7 | 100.0% |
| Benefits Estimate Agent AI Foundry | 1.2.0 | 99.8 | 100.0% |
| HR Assist Copilot Studio | 1.5.0 | 95.4 | 100.0% |
| IT Service Desk Agent ServiceNow Now Assist | 3.0.0 | 99.8 | 100.0% |
| Appointment Scheduler Vendor platform | 5.2 | 92.4 | 100.0% |
| Since August | 143 runs · 1,468 conversations tested | ||
Wrong answers sound exactly like right ones.
An agent that says members have 60 days to pay for service credit, when the policy says 90, reads perfectly. Keyword checks miss it, and a model asked “is this good?” usually says yes. Sepal Assure checks the answer the way your experts would: against the source, claim by claim.
- Meaning, not wording
Paraphrases match; a changed number, a missing condition or a fact from the wrong year doesn’t. Each score comes with its reason.
- Every claim and every citation
Each factual sentence is checked against your approved documents, and each link must resolve, be current, support the sentence and be one this user may see.
- Judges you can trust — or not
Model judges from a different family than the agent are calibrated against your SMEs’ labels, and only decide where they agree. Everywhere else, a person decides.
- Real changes, not noise
Repeated trials give confidence intervals, so a regression is a real change in the agent — not a different roll of the dice.
From your documents to a release decision
- Generate tests from approved sources
Connect SharePoint or upload policies. Sepal drafts questions, expected answers, paraphrases, Spanish versions and attacks from each passage.
- Experts approve them
Nothing generated runs until a subject-matter expert approves, edits or rejects it. Tests are flagged again when their source changes.
- Run on demand, on a schedule or from the pipeline
Single questions, scripted and simulated conversations, task agents with expected tool calls, fairness pairs and load tests.
- Gate the release
Compare with the baseline case by case, hand the release review to your approvers with the evidence, and record any accepted risk.
- Keep watching in production
Score a masked sample of real conversations, run a daily canary, and catch the day a provider changes the model without telling you.
Everything in one platform
Eleven modules on one database, one set of permissions and one audit trail. On the right, the first plan that includes each one.
Build & test
Agents & environmentsTeamEvery agent, every platform, tested the same way.Knowledge sourcesTeamTest against the sources of truth your agents use.Test designTeamGenerate tests from your documents. Experts approve them.Evaluate
EvaluationTeamScores you can defend, by meaning — not by wording.Runs & regressionTeamEvery release compared with the last approved one.Operate
Production monitoringBusinessKnow when quality drifts — even when nobody released anything.Findings & releasesTeamEvidence for every release decision. The decision stays yours.Govern
AI governanceBusinessTesting evidence that answers to NIST AI RMF and Texas TRAIGA.Insights
Dashboards & reportsTeamFor QA teams, product owners and leadership.Administration
AdministrationTeamConfigure it yourself.IntegrationsTeamFits the tools your QA and delivery teams already use.Every agent, wherever it was built.
Agents you build, agents your vendors supply, and agents you can only reach through a chat window are tested the same way, with the checks each level of visibility allows.
- Copilot Studio, Teams and Bot Framework
Conversations over Direct Line, transcripts from Dataverse, and topic analytics alongside your test results.
- Azure OpenAI, AI Foundry, Vertex AI and Bedrock
Test deployments and agents directly through their APIs, or read OpenTelemetry traces for tool calls and retrieval.
- Vendor and closed-box agents
A browser session or transcript import is enough to test what members see, even when you can’t see inside.
- Inside your network
A runner you host reaches agents that aren’t on the internet; only results leave your network.
More than an evaluation tool.
Developer evaluation platforms stop at the score. Public agencies also need the people, the approvals and the paperwork. Sepal Assure adds the pieces that usually end up in a spreadsheet.
- Governance evidence, collected for you
An AI inventory and controls for NIST AI RMF, the Texas Responsible AI Governance Act, OWASP Top 10 for LLMs and ISO/IEC 42001 — with evidence linked from real runs, reviews and monitors.
- Content health
Your documents disagree with each other more than you think. Sepal finds the contradictions, stale pages and unanswered questions before an agent repeats them.
- Release reviews your approvers can read
Gate results, the comparison with the baseline, open findings and accepted risks, routed by risk tier and recorded for the auditors.
- Findings in the tools you use
ServiceNow, OpenText ALM, Azure DevOps or Jira tickets with the failing answer attached, and a fix verified only when the test passes again.
See it with your own agent.
Point us at a test environment and a handful of policy documents, and we’ll show you what your agent gets wrong — in a week.