Sepal Assure

Know your AI agents are right — before your members do.

Sepal Assure tests chatbots, virtual assistants and task agents against the documents they’re supposed to answer from. It scores every answer by meaning, checks each claim and citation, compares every release with the last approved one, and keeps watching once the agent is live.

Agent quality in productionSample organization · computed November 12, 2026
AgentVersionScorePass rate
Ask Bluebonnet
Copilot Studio
2.3.098.7100.0%
Benefits Estimate Agent
AI Foundry
1.2.099.8100.0%
HR Assist
Copilot Studio
1.5.095.4100.0%
IT Service Desk Agent
ServiceNow Now Assist
3.0.099.8100.0%
Appointment Scheduler
Vendor platform
5.292.4100.0%
Since August143 runs · 1,468 conversations tested
Quality scoreCandidate 2.4.0: 94.7 vs 98.7 baseline · 3 regressions caught before release

Wrong answers sound exactly like right ones.

An agent that says members have 60 days to pay for service credit, when the policy says 90, reads perfectly. Keyword checks miss it, and a model asked “is this good?” usually says yes. Sepal Assure checks the answer the way your experts would: against the source, claim by claim.

  • Meaning, not wording

    Paraphrases match; a changed number, a missing condition or a fact from the wrong year doesn’t. Each score comes with its reason.

  • Every claim and every citation

    Each factual sentence is checked against your approved documents, and each link must resolve, be current, support the sentence and be one this user may see.

  • Judges you can trust — or not

    Model judges from a different family than the agent are calibrated against your SMEs’ labels, and only decide where they agree. Everywhere else, a person decides.

  • Real changes, not noise

    Repeated trials give confidence intervals, so a regression is a real change in the agent — not a different roll of the dice.

From your documents to a release decision

  1. Generate tests from approved sources

    Connect SharePoint or upload policies. Sepal drafts questions, expected answers, paraphrases, Spanish versions and attacks from each passage.

  2. Experts approve them

    Nothing generated runs until a subject-matter expert approves, edits or rejects it. Tests are flagged again when their source changes.

  3. Run on demand, on a schedule or from the pipeline

    Single questions, scripted and simulated conversations, task agents with expected tool calls, fairness pairs and load tests.

  4. Gate the release

    Compare with the baseline case by case, hand the release review to your approvers with the evidence, and record any accepted risk.

  5. Keep watching in production

    Score a masked sample of real conversations, run a daily canary, and catch the day a provider changes the model without telling you.

Everything in one platform

Eleven modules on one database, one set of permissions and one audit trail. On the right, the first plan that includes each one.

Build & test

Agents & environmentsTeamEvery agent, every platform, tested the same way.Knowledge sourcesTeamTest against the sources of truth your agents use.Test designTeamGenerate tests from your documents. Experts approve them.

Evaluate

EvaluationTeamScores you can defend, by meaning — not by wording.Runs & regressionTeamEvery release compared with the last approved one.

Operate

Production monitoringBusinessKnow when quality drifts — even when nobody released anything.Findings & releasesTeamEvidence for every release decision. The decision stays yours.

Govern

AI governanceBusinessTesting evidence that answers to NIST AI RMF and Texas TRAIGA.

Insights

Dashboards & reportsTeamFor QA teams, product owners and leadership.

Administration

AdministrationTeamConfigure it yourself.IntegrationsTeamFits the tools your QA and delivery teams already use.

Every agent, wherever it was built.

Agents you build, agents your vendors supply, and agents you can only reach through a chat window are tested the same way, with the checks each level of visibility allows.

See how Sepal Assure connects

  • Copilot Studio, Teams and Bot Framework

    Conversations over Direct Line, transcripts from Dataverse, and topic analytics alongside your test results.

  • Azure OpenAI, AI Foundry, Vertex AI and Bedrock

    Test deployments and agents directly through their APIs, or read OpenTelemetry traces for tool calls and retrieval.

  • Vendor and closed-box agents

    A browser session or transcript import is enough to test what members see, even when you can’t see inside.

  • Inside your network

    A runner you host reaches agents that aren’t on the internet; only results leave your network.

More than an evaluation tool.

Developer evaluation platforms stop at the score. Public agencies also need the people, the approvals and the paperwork. Sepal Assure adds the pieces that usually end up in a spreadsheet.

  • Governance evidence, collected for you

    An AI inventory and controls for NIST AI RMF, the Texas Responsible AI Governance Act, OWASP Top 10 for LLMs and ISO/IEC 42001 — with evidence linked from real runs, reviews and monitors.

  • Content health

    Your documents disagree with each other more than you think. Sepal finds the contradictions, stale pages and unanswered questions before an agent repeats them.

  • Release reviews your approvers can read

    Gate results, the comparison with the baseline, open findings and accepted risks, routed by risk tier and recorded for the auditors.

  • Findings in the tools you use

    ServiceNow, OpenText ALM, Azure DevOps or Jira tickets with the failing answer attached, and a fix verified only when the test passes again.

See it with your own agent.

Point us at a test environment and a handful of policy documents, and we’ll show you what your agent gets wrong — in a week.

Request a demoTry the sample organization