2026-W26 · daily
Jun 22 – 28 · online run — an evidence packet satisfying EU AI Act Art. 72 post-market monitoring, and it flows into the Governance ledger.
Watch window — BREACH
Groundedness fell below the release-gate bar — see the alert below.
205
traces
5
rollups
breach
verdict
Pinned
everything this online run scored againstwindow
2026-W26 · daily — Jun 22 – 28
scope
portfolio sample — hr-policy-advisor + payroll-copilot (the two judge-covered agents this window)
sample
205 traces sampled — 130 via Phoenix · 75 via Langfuse
guardrail config
guard-pack@prod-2026.06 — in sync with offline safety@v2
agent build
hr-policy-advisor 1.8.1 · prompt v14 · payroll-copilot 0.9.3 · prompt v10 — both unchanged through the arc below
Same pinning discipline as a sealed run — window, sample, judge versions, and guardrail config are all versioned, auditable assets (§57).
Score scoreboard
native units, per concern — never blended| Concern | Metric | Result | Bar | Verdict |
|---|---|---|---|---|
| Retrieval | groundedness | 0.88 first window below the bar — the state-policy jurisdiction pages migrated 2026-06-23; the live retrieval index picked up the structural change before the offline golden corpus did | ≥ 0.90 | fail |
| Safety / PII | pii incidents shipped | 0 | = 0 | pass |
| Numeric | oracle corrections | 2.8/day avg | alert ≥ 5/day | pass |
| Functional | online-eval pass rate | 89% | ≥ 90% | fail |
| Safety / PII | refusal correctness | 0.97 | ≥ 0.95 | pass |
Native units by design (§12) — 0.93 groundedness, 0 pii incidents, "3 today" oracle corrections. Flattening these to one score is how a breach hides.
Groundedness trend
vs the hr-policy-release-gate bar, across recent windowsW26 and W27 fell below the bar two windows running — that sustained pattern, not either window alone, is what triggers the alert (§58).
Alert & rollback
alert_2026w27_groundednessgroundedness (retrieval)2026-W26 then 2026-W27 · values 0.88 → 0.85 vs bar ≥ 0.90
detected
2026-W27 close · 2026-07-05 22:14
paged
hr-policy-advisor on-call, PagerDuty (payroll-intelligence & HR platform rotation)
acknowledged
acknowledged in 6 min — ginny.w
root cause
The state-policy jurisdiction pages migrated 2026-06-23. The live retrieval index picked up the structural change immediately; the offline golden corpus — and the release-gate suite that runs against it — hadn't. Grounded citations broke on leave-carryover answers for two windows running. This is exactly what a watch window catches that a release gate, run only against the pinned offline corpus, cannot.
action
Staged rollback — Production retrieval pinned back to the pre-migration index snapshot (2026-06-20) pending a corpus reconciliation. hr-policy-advisor's agent build (1.8.1 · prompt v14) was never implicated and stayed unchanged throughout.
No synchronous blocking beyond guardrails; no auto-de-provision on judge-scored metrics. Guardrails kept blocking/redacting inline the entire time (pii incidents stayed at 0) — this alert paged the team and rolled back, it did not silently drop traffic.
Representative traces from this window feed the trace inbox via the same four feeds as the activity stream — guardrail, online-eval, drift, and end-user feedback.