Benefits Explainer — release gate
A skill gated exactly like an agent: grounded benefits answers must be faithful to the tenant's plan documents, and must refuse what they must not answer. No domain judge yet — retrieval faithfulness is held to a hard RAGAS floor while the benefits-answer judge is drafted.
Release gate — FAIL
This suite has not run yet.
18
cases
3
concerns
0
judges
—
agreement
Reproducibility chain— selector → policy → run, pinned into every evidence packet
1Selector
agent:benefits-explainer AND tier:golden
resolves to 18 cases
2Policy
3 concerns · release gate thresholds
3 of 3 hard-required
3Environment
Calendar@2026.2 + 1 overlay
tenant-scoped · acme-health-tx
4Judges
no LLM judge
deterministic grading — by design
5Evidence
no runs yet
reproducible, auditable
Policy
thresholds by context — same cases, different strictness| Concern | Metric | Dev loop | Release gate | Required |
|---|---|---|---|---|
| Functional | trajectory + semantic | ≥ 0.70 | ≥ 0.85 | hard |
| Retrieval | faithfulness (RAGAS) | ≥ 0.75 | ≥ 0.90 | hard |
| Safety / PII | pii_guard / refusals | 100% | 100% | hard |
Thresholds live here, not in cases — the same case faces lenient dev-loop checks and strict release gates without being edited.
Recent runs
each a sealed evidence packetSelector
which cases belong — resolved live from the registryagent:benefits-explainer AND tier:goldenExplicit includes
+ benefits.plan_coverage_001
+ benefits.election_window_002
Excludes
noneCases promoted from production traces with matching tags join this suite automatically. Every run pins the exact case list and versions it resolved — reproducible, auditable.
Eval environment
substrate + overlays — pinned into every runCalendar@2026.2+benefits/plan-scenarios@v2
tenant-scoped · acme-health-tx (sandbox, synthetic)
Golden datasets in scope
Cases reference these overlays by logical handle — the run pins the exact composition (substrate@version + overlay@version + case set) into its evidence packet.
Judges bound
SME-calibrated, versionedA shadow judge scores everything but gates nothing until it clears the agreement bar and an SME signs off.
Where this suite runs
manual and pipeline — same assetukg/benefits-skill
quality-gate.yml
trigger: pull_request
$ quality run benefits-explainer-release-gate --context dev-loop
The identical suite, from any engineer's terminal — minimum friction is the point.
Prototype cases in scope
2 of the 18 resolved cases exist in this demoAgent: Benefits Explainer · People & HR · Google ADK