run_5528
WFM Time-Off — release gate · pull-request · 2026-07-07 10:24
Run — PASS
24 of 24 clean on functional, numeric, and action — the gate is green. Fairness is simply absent: no cohort suite is bound (the §54 named-missing artifact).
24
cases
3
concerns
$1.34
cost
1.9 s
p95
4m 02s
duration
Sealed evidence packet
reproducible — every version pinnedagent build
wfm-assistant 2.3.1 · gemini-2.5-pro · prompt v9
suite policy
wfm-timeoff-release-gate · policy@2026.07
routing table
aag/routing-table@2026.06 — concern → backend, centrally owned
grader packs
functional@v3numeric@v1action@v1
dataset
Calendar@2026.2 + wfm/timeoff-scenarios@v1 · 24 cases pinned
sandbox tenant
acme-mfg-oh (sandbox, synthetic)
50 GB substrate + 4 KB overlay = a reproducible eval environment — re-runnable a year from now, byte-identical (§24).
Concern scoreboard
the suite policy matrix with actuals filled in| Concern | Metric | Result | Threshold | Verdict |
|---|---|---|---|---|
| Numeric | numeric_exact | 18 / 18 checks exact | = 100% | pass |
| Action | side-effect asserts | 14 / 14 asserts · confirmations present | = 100% | pass |
| Functional | trajectory + semantic | IN_ORDER ✓ · 0.90 avg semantic | ≥ 0.85 | pass |
Native units by design — RAGAS 0.91, IN_ORDER ✓, 22.5 h == 22.5 h. Flattening to a 0–100 score is how failures hide (§12).
Case × concern
24 cases · failures pinned firstNM
AC
FN
✓ pass · ✕ fail · – not exercised by this case · NM Numeric · AC Action · FN Functional
0 failures — nothing to diagnose. The packet still matters: it is the evidence Governance cites.