Efficiency

Cost

What the product costs to run and what the platform costs to assure it — measured on every run, gated by suite policy, and always shown next to the quality verdict, never alone. Latency, throughput, and availability live in Performance.

Performance

Two cost families, kept deliberately separate. Serving economics is what the product costs to run (per agent). Eval spend is what the platform costs to assure it (per suite). Conflating them is how dashboards get misread — they are never summed into one number.

Family 1

Agent serving economics

what the product costs to run — per agent, per interaction

Per-agent serving cost

native units — dollars, tokens (§12)

online watch windows

Agent	Cost / interaction	Volume / wk	Annual @ scale	Posture
Payroll Copilot development · 4.8k tok	$0.032	1.2M / wk projected	$2.0M / yr	advisory
HR Policy Advisor pre-production · 3.9k tok	$0.028	620k / wk projected	$0.90M / yr	advisory
WFM Assistant production · 2.4k tok	$0.019	1.8M / wk production	$1.78M / yr	hard
People Navigator production · 0.9k tok	$0.006	2.4M / wk production	$0.75M / yr	hard

Per-interaction cost is read off runs (§11); annual figures are projected at each agent's target volume. Scaled production and real-time surfaces opt into hard cost posture — dev and pre-production stay advisory (§76). Latency percentiles, throughput & availability live in Performance now →

Family 2

Eval spend

what the platform costs to assure it — per suite, framed as ROI

Per-suite eval spend & what it protects

platform cost — never the same number as serving cost

Suite	Backend	Judge tok	Sandbox	Weekly	Protects — the ROI
Payroll Copilot — release gate Payroll Intelligence · 9 runs / wk · ~$2.28 ea	$11.20	$6.40	$2.90	$20.50	31 / 31 regulated pay figures verified per run24 exfil probes resisted~180 SME-min / wk saved (§38)
HR Policy Advisor — release gate People Experience · 7 runs / wk · ~$4.62 ea	$18.60	$9.10	$4.60	$32.30	18 / 18 accrual checks verified per run48 adversarial probes resisted~240 SME-min / wk saved (§38)
WFM Assistant — dev loop Workforce Core · 22 runs / wk · ~$1.12 ea	$13.80	$5.20	$5.60	$24.60	44 / 44 balance checks · 9 action asserts per runcohort parity watched (Fairlearn)~90 SME-min / wk saved (§38)

$77.40 / wkall suites · ~$4.0k / yr guarding $4.7M / yr of serving cost

~$4k a year of eval spend guards $4.7M a year of agent serving cost — and the regulated numerics, exfil resistance, and SME hours behind it. Eval spend is ROI, not overhead.

The trade-off view

a change is always cost Δ + quality verdict, together — never cost alone

Ships

Payroll Copilot · Prompt cache + trimmed system context

proposed build pc-opt-7f2 · prompt cache + trimmed context

Caches the static policy preamble and drops ~1.1k tokens of redundant context per turn. A pure serving-cost optimization — no change to tools, model, or numerics.

Cost & latency

$0.032$0.028−12%/ interaction

Annual @ scale

$2.0M / yr → $1.75M / yr

−$250k / yr

p95 latency

2.3 s → 2.1 s

−0.2 s p95

Quality gateall green

IN_ORDER ✓ · 0.90 avg semantic

≥ 0.85pass

31 / 31 checks exact

= 100%pass

0 incidents · 9 / 9 refusals correct

= 100%pass

24 / 24 probes resisted (1.00)

≥ 0.95pass

−12% cost, quality held — every hard gate green. Ships. Cost improvements are welcome; they still clear the same bar.

Baseline run_5488 — measured against run_5488 · 0.9.3 · p95 2.3 s

Blocked

Payroll Copilot · Swap the reasoner for a smaller distilled model

proposed build cm-lite-3b · smaller distilled reasoner

Cuts serving cost 41% and even trims p95 — functional, safety, and adversarial all still green. The smaller model writes fluent, semantically-fine answers; the deterministic oracle catches the wrong arithmetic on regulated pay figures. Regulated numerics are never judged by an LLM (§9).

Cost & latency

$0.032$0.019−41%/ interaction

Annual @ scale

$2.0M / yr → $1.19M / yr

−$810k / yr

p95 latency

2.3 s → 1.9 s

−0.4 s p95 — faster, too

Quality gategate red

IN_ORDER ✓ · 0.89 avg semantic

≥ 0.85pass

27 / 31 checks exact

= 100%fail

0 incidents · 9 / 9 refusals correct

= 100%pass

24 / 24 probes resisted (1.00)

≥ 0.95pass

−41% cost, −$810k a year, faster — and blocked by a single row. numeric_exact fell 31 → 27 on regulated pay figures; that hard gate is non-negotiable. Cost never buys its way past it.

Baseline run_5488 — measured against run_5488 · 0.9.3 · p95 2.3 s

Cost-regression policy rows

cost & latency are concerns — same table, same hard / advisory flag (§76)

advisory by default · hard opt-in

Suite / metric	Threshold	Actual	Flag	Verdict
WFM Assistant — dev loop p95 latency	≤ 2.0 s	1.8 s	hard	pass
Δcost vs baseline	≤ +15%	+2%	hard	pass
cost / interaction	≤ $0.025	$0.019	advisory	pass
Payroll Copilot — release gate p95 latency	≤ 3.0 s	2.3 s	advisory	pass
cost / interaction	≤ $0.040	$0.032	advisory	pass
Δcost vs baseline	≤ +15%	−12%	advisory	pass
HR Policy Advisor — release gate p95 latency	≤ 3.5 s	2.7 s	advisory	pass
cost / interaction	≤ $0.035	$0.028	advisory	pass
Δcost vs baseline	≤ +20%	+6%	advisory	pass

These rows live in the suite policy alongside every quality concern — dollars, seconds, and tokens in native units. A cost regression fails the gate exactly the way a numeric regression does. These are release-gate thresholds; the operational latency budgets they mirror are watched on Performance.

Baseline & re-baseline ledger

baseline = last green release-gate run on main — no rolling averages (§77)

Current baseline

run_5488 green

Payroll Copilot · payroll-copilot 0.9.3 · Payroll Copilot — release gate

cost / interaction

$0.032

p95

2.3 s

sealed

2026-07-06 11:50

Last green release-gate run merged to main (#168). Fixed until an attributed re-baseline moves it — no rolling average, so drift never ratchets the floor.

Re-baseline ledger — deliberate increases, attributed

rb_5411+10% cost, deliberate & attributed2026-06-24 15:40

Mandatory source-citation grounding added for EU AI Act Art. 15 traceability — a deliberate token-cost increase. Re-baselined so Δcost-vs-baseline measures against the new intended floor, not the pre-citation build.

oldpayroll-copilot 0.9.2 · $0.029 / interactionnewrun_5488 · payroll-copilot 0.9.3 · $0.032 / interaction

by cho.c · approved minerva.m (AI Governance Officer)

A deliberate cost increase lands here like a break-glass override (§16) — who, when, justification, old → new. Drift alone can never move the baseline.