When the answer is wrong

1. 01We have collected at least ten real wrong outputs from this feature and reviewed them as a team.Design from actual failures, not imagined ones. They are weirder than you think.
2. 02The system says “I can’t do that” or “I’m not sure” somewhere — we have seen it happen.A system that has never declined is a system that is bluffing somewhere.
3. 03Confidence language is calibrated: hedged claims hedge, and firm claims are backed.Uniform confidence across all outputs is a lie in both directions.
4. 04Every factual claim the user might act on carries a checkable source or a visible gap.Citations the user can open, not decorative footnote numbers.
5. 05A user can verify the output faster than they could have done the task themselves.If verification costs more than authorship, the feature has negative value.
6. 06Undo exists for every AI-applied change, and it is one action, not an archaeology dig.
7. 07The user can see exactly what the AI changed, diffed against what was there before.
8. 08Correcting a wrong answer is cheaper than re-prompting from scratch.Edit-in-place, regenerate-this-part, or fix-and-continue — not “try again.”
9. 09There is a path to a human or a non-AI fallback when the model fails repeatedly.Graceful degradation, not an infinite retry loop with apologies.
10. 10The error states have real copy, written for this feature, reviewed by a writer.“Something went wrong” is the copy equivalent of shrugging at a paying customer.

Discussion prompts

• What is the most plausible wrong answer this feature could give — the one that looks right? Who catches it, and when?
• Show me the screen where the system admits uncertainty. If no one can, why are we confident it exists?
• What does a user lose if they trust a wrong output for a week before noticing? Can they get it back?
• Are we designing recovery for the user who notices the error, or also for the one who doesn’t?
• If a wrong answer from this feature ended up in a screenshot on social media, which missing safeguard would we wish we had shipped?