Design the eval before the interface

1. 01A golden set of at least twenty real inputs exists — pulled from actual user data, not written by the team.Inputs the team invents are inputs the team already handles. Users are weirder than your imagination.
2. 02The golden set includes the ugly cases: ambiguous asks, hostile phrasing, wrong-language input, and the 40-page document.A golden set of happy paths is a demo with a spreadsheet attached.
3. 03The team has reviewed the ten worst outputs together, in one room, on one screen.Not a summary of failures — the actual failures. Shared discomfort is the point.
4. 04Each output in the eval is graded against written criteria a new hire could apply without asking questions.“Good response” is not a rubric. “Cites the source document; under 100 words; no invented figures” is.
5. 05The eval’s failure direction matches the go/no-go decision: if we chose precision, the eval punishes false confidence hardest.An eval that rewards answer rate while the design promises caution is measuring a different product.
6. 06The latency budget is stated in milliseconds and has been tested on production infrastructure, not a laptop.“Fast enough” is not a budget. “p95 under 3,000 ms on prod” is a sentence you can lose an argument with.
7. 07A regression plan exists: every prompt or model change reruns the golden set before it ships.Prompt edits are code changes. A one-word tweak can silently break twelve behaviors.
8. 08A designer can run the full eval without an engineer in the room.If the eval needs a terminal and a favor, design decisions will be made without it.
9. 09The eval scores refusals and hedges, not just answers — we know how often the model declines and whether it should have.
10. 10We know the score at which we ship, the score at which we hold, and who decided those numbers.Thresholds set after seeing results are results, not thresholds.
11. 11The golden set has an owner and a refresh cadence — real inputs from production feed it after launch.An eval frozen at launch measures the product you shipped, not the one users have.

Discussion prompts

• If the eval and a stakeholder’s gut disagree about whether an output is good, which one wins — and has that ever actually happened here?
• Who wrote our grading criteria, and have two people independently graded the same ten outputs to see if they agree?
• What behavior are we not measuring because it is hard to measure, and what happens if that’s the thing users notice first?
• When the ten worst outputs went up on the screen, who in the room was surprised — and why were the rest of us not sharing what we knew?
• Would this feature pass its own eval today? If nobody can answer in one sentence, what is the interface being designed against?
• What is the smallest prompt change that ever shipped without a rerun of the golden set, and how would we even know?