Skip to main content
Unlisted page
This page is unlisted. Search engines will not index it, and only users having a direct link can access it.
Version: 11.3.0

Bring-Your-Own Judges

The Rule-of-Two breaker is mechanical — it decides from trust, sensitivity, and the sink, and it does not read the meaning of a message. Judges are the optional layer that does read meaning: a judge asks a model a bounded set of questions about a request — does this content look like it is trying to steer the agent? does this proposed action actually serve what the person asked for? — and returns calibrated scores.

A judge is a signal, never the decision

A judge never decides an outcome on its own. It returns scores; your thresholds and the breaker decide what to do with them. And when a judge cannot answer — it is unavailable, misconfigured, or times out — it abstains; it never silently lets a call through. Judges can only add caution, never remove it.

Judges are off by default, so adopting input guardrails adds no latency and changes no behavior until you choose to bring one in.

Bring Your Own​

You choose the judge. AgentPass ships a default HYPR judge so there is sensible behavior out of the box, and lets you register your own from any of these kinds:

Judge kindWhat it is
TypeSafe / JevHYPR's own judgment models (the default)
OpenAI-schemaA provider that answers against a strict JSON schema
Anthropic toolA provider that answers by calling a required tool
Signed webhookYour own endpoint, called with a signed request

A judge's credential is always held as a reference to a secret, never as a literal value on the judge's configuration — so a secret never appears on the record, in a log, or in an export.

Bench, Shadow, Promote​

A judge is not trusted with live guardrail signal the moment you register it. It earns that through a lifecycle you drive from the Judges console (under the Guardrails navigation):

  1. Register the judge and point it at its credential.
  2. Probe it to confirm it is reachable and answers in the expected shape.
  3. Bench it against a shipped golden set — a labeled collection of injection and benign examples — which reports how well it separates the two, so you can compare a candidate against what you run today on the same cases.
  4. Promote it. Only a promoted judge may back live signal, and promotion freezes the exact configuration that was benched, so what you measured is what runs.
  5. Shadow and monitor. Run a judge alongside your current one without it affecting decisions, and watch how it behaves on real traffic before you rely on it. A promoted judge that becomes persistently unavailable is stepped down automatically rather than being left in place answering nothing.

You bind a judge per category of question — for example, injection, drift, or classification — so you can run different judges for different jobs, and keep a shadow alongside a primary for each.

Judges console showing a registered judge with its bench results and the promote and shadow controls

Deterministic Injection Detection​

Alongside the judges, AgentPass runs a deterministic injection detector — pattern-based recognition of known injection shapes — which contributes its own signal without calling any model. Like a judge, it is a signal: it informs and can add caution, but it does not override the mechanical breaker or lower a session's sensitivity.

The Model Governance Owner​

Because a judge is a model making security-relevant calls, who may change one is governed separately from the rest of guardrails. Input guardrails add a Model Governance Owner role for the person who owns judge configuration, with view and manage permissions dedicated to model governance. Administrators hold these permissions; the Security Analyst and Auditor roles can view judge configuration but not change it.