Portable Evals/cases

benefits.election_window_002

validv1goldenauthored

When can I change my benefits elections? — Benefits Explainer (Google ADK)

ukg/benefits-skill · evals/cases/benefits.election_window_002.yaml@ 6b2e903PR #54 · reviewed

One asset → 2 backends— routed centrally by AAG · each grader runs on exactly one engine, so nothing is scored twice

portable-v1 · authored once

benefits.election_window_002

“When can I change my benefits elections?”

2 concerns · 1 YAML asset · ukg/benefits-skill

Functional

→ ADK eval

functional@v3 · in-sync

Retrieval

→ RAGAS

retrieval@v2 · in-sync

The assertion

During annual open enrollment, or within 30 days of a qualifying life event.

must_cite

ENROLL-TX-2026 §1.4

Golden context

ENROLL-TX-2026 §1.4

Elections may be changed during the annual open-enrollment window or within 30 days of a qualifying life event (marriage, birth, loss of other coverage).

expected_tools · in_order

get_employee_context → search_plan_documents

Compiled → 2 targetsbenefits.election_window_002.functional.configbenefits.election_window_002.retrieval.config all in-sync

Portable eval case

portable-v1

data + graders · thresholds live in suite policy

input

“When can I change my benefits elections?”

entities

overlay handles — never raw substrate IDs

employees.enrolled_dental_tx→ E-77120 · via benefits/plan-scenarios@v2 on Calendar@2026.2

params

{"as_of":"2026-07-09"}

tenant

acme-health-tx (sandbox, synthetic)

expected_tools

order: in_order

1get_employee_context
2search_plan_documents{"topic":"election_window"}

retrieval_context

golden chunks

ENROLL-TX-2026 §1.4
Elections may be changed during the annual open-enrollment window or within 30 days of a qualifying life event (marriage, birth, loss of other coverage).

expected_output

During annual open enrollment, or within 30 days of a qualifying life event.

must_cite

ENROLL-TX-2026 §1.4

metadata

pillar:peopletier:goldenbenefitsgroundedsensitivity: syntheticpersona: employee

datasets

pinned compositions this case belongs to

benefits-plan-golden

kind

drives grader packs

FunctionalTool trajectory, response match, and semantic regression bulk.RetrievalFaithfulness and context quality for grounded answers.

Graders

inherited from AAG packs via kind: — each routed

Functionalfunctional@v3inherited→ ADK eval

tool_trajectoryin-order tool matchExpected tools called, in order, with matching args.

response_semanticLLM judgeProse semantically matches the golden answer.

response_lexicalROUGE-1Lexical overlap with the golden answer.

Retrievalretrieval@v2inherited→ RAGAS

groundednessclaim decompositionEvery claim in the answer is supported by retrieved policy chunks.

context_precisionchunk relevanceRetrieved chunks are relevant to the question.

context_recallchunk coverageThe golden citations appear in the retrieved set.

Routed, not replicated. Each grader goes to exactly one backend. Nothing is forced onto a backend that cannot express it, and the same case is never scored twice for the same thing.

Routing

concern → backend · centrally owned by AAG

2 targets

Concern	Backend	Why this backend
Functional	→ ADK eval	Native to the agent runtime — owns the regression bulk (trajectory, ROUGE, semantic judge).
Retrieval	→ RAGAS	Purpose-built faithfulness and context metrics; semantic match stays on ADK so nothing is scored twice.

Compiled output

2 routed targets

in-sync

Artifacts generated per routed backend (full compiled view shown on the hero case, hr.policy_carryover_004):

FNbenefits.election_window_002.functional.config→ ADK evalin-sync

RTbenefits.election_window_002.retrieval.config→ RAGASin-sync