why scripted checks aren't enough
A true story: the refund that every ordering check passed.
We watched a frontier model, confused by poorly-named tools, pass a customer id where an order id belonged. The policy check errored. The model issued the refund anyway — then checked the policy again afterwards. A hand-written trajectory assertion — "was the policy checked before the refund?" — passed this run: a check did precede the refund. Its answer was just never valid. The induced spec caught it, because the spec knows what state the decision requires, not merely what order calls happen in. That trace ships in the repo as a permanent test. And the gate blocks it live:
node examples/refund/bad-agent.mjs # reenacts the captured failure — no LLM needed → get_order({"order_id":"ORD-1002"}) → check_refund_policy({"order_id":"CUST-1004"}) ← the real captured mistake {"error":"order CUST-1004 not found"} ✗ issue_refund — blocked: gate guard does not hold: (refund_policy.max_amount > 0.25) the refund never executed — backend state: {"refunds":[]}