When It’s Wrong

“The AI got it wrong” is a useless bug report, because it collapses four distinct failures that need four distinct UX responses. A hallucination is confident fabrication — the invented citation, the API method that doesn’t exist, the case law that was never decided. A misunderstanding is a correct answer to the wrong question: the model parsed “book the earlier flight” as the earlier of two options you weren’t considering. A capability miss is a request outside what the system can actually do — asking a text model to count the words it just generated, or a support bot to issue a refund it has no tool for. A context failure is the model working from stale, missing, or wrong material: the retrieval that came back empty, the document that never attached, the memory that expired.

Each one demands different design. Hallucinations need verification affordances at the point of consumption, because the user cannot detect them from fluency alone. Misunderstandings need the system to show its interpretation — Cursor restating a plan before executing it — so the mismatch surfaces before the work does. Capability misses need honest scoping up front; they are failures of the contract, not the model. Context failures need legible context: visible attachment chips, “searched 4 files” disclosures, anything that lets a person answer “what was it actually looking at?”

Instrument your product to tell these apart. A team that logs everything as “bad response” will fix the wrong thing.

AI’s defining failure mode is that wrong answers arrive in the same confident prose as right ones. A crashed app tells you it crashed; a hallucinating model hands you polished fiction. That symmetry means honesty cannot be left to the model’s tone — it has to be built into the surface. The design goal is simple to state and hard to ship: the user should be able to tell, at the moment of reading, how much skepticism this particular answer deserves.

Calibrated confidence is the first lever. The system usually knows when it’s on thin ice — retrieval came back sparse, the question sits outside the corpus, the tool call failed and it’s answering from memory. Surface that as differentiated presentation, not a uniform authoritative voice. Perplexity leads with sources precisely so a thin answer looks thin. The early AI search summaries that recited absurdities with encyclopedic confidence did more brand damage than any outage, because they taught users the voice means nothing.

The second lever is admitted limits. ChatGPT stating its knowledge cutoff, Claude declining to guess at a paywalled article’s contents, a support bot saying “I can’t process refunds, but here’s who can” — each is a failure, and each builds trust, because the alternative in every case is a fabrication discovered later. An honest “I don’t know” is the cheapest trust-building interaction your product will ever ship. Hide the miss and the user finds it anyway — and then re-audits everything you ever told them.

The number that governs whether people keep an AI in their workflow is not accuracy — it’s recovery cost. Developers tolerate Copilot’s mediocre completions all day because rejecting one costs a keystroke. The same person abandons an AI email assistant after a single wrong send, because that miss can’t be recovered at all. A 90%-accurate tool with five-second recovery beats a 98%-accurate tool with ten-minute recovery, every time, in every domain.

Two mechanics do most of the work. First, refinement over restart: when the output misses, the unit of correction should be the delta — “shorter, and keep the second paragraph” — not a blank-slate rewrite of the whole prompt. ChatGPT and Claude both let you edit a prior message and branch from it, preserving everything that was already right. Second, reversibility for anything that touched real state: Cursor checkpoints AI edits so a bad run rolls back wholesale; Notion AI writes into documents with full version history one menu away. Where undo is impossible — the sent email, the executed payment — the lesson runs backward: that action needed an approval gate, because recovery was never going to be available.

Audit your product with one question: when the AI is wrong, what do the next fifteen seconds look like? If the answer is “start over,” you haven’t finished designing the feature.

Systems fail binarily by default: it worked, or a wall — “something went wrong.” Good AI products replace the wall with a ladder, a designed sequence of degraded-but-useful states between full success and total failure. Web search is down? Answer from training data and say so: “I couldn’t search, so treat recent details with suspicion.” The model is overloaded? Fall back to a smaller model and label the downgrade. Retrieval found nothing? Say what was searched and offer to broaden, instead of confabulating from nothing. Each rung preserves partial value and — critically — tells the person exactly how to weight what they got.

Design the ladder deliberately, rung by rung, before launch. For each capability, write down: what’s the best response when this dependency fails? What does the system say about the downgrade? What’s the rung below that? Teams that skip this exercise ship the wall, because the wall is what happens when nobody decided otherwise. The pattern shows up in mature products everywhere: Claude noting when a tool call failed mid-run and continuing with what it has, Copilot falling back to local context when the index is stale, Maps degrading from live traffic to static routes without pretending otherwise.

The bottom rung of every ladder is a graceful handoff — to a human, to a manual flow, to a plain search box. An AI feature with no bottom rung holds the user’s task hostage to the model’s uptime, and users remember hostage situations.

When an AI product fails, the user assigns blame somewhere — to themselves, to the model, or to you — and that assignment is a design outcome, not an accident. The blank prompt box quietly teaches self-blame: the output missed, so I must have prompted it wrong. That feels safe for the product and is corrosive in practice, because users who blame themselves don’t iterate — they leave, quietly, convinced the tool is for someone cleverer. The opposite framing, an unaccountable system that shrugs (“AI can make mistakes”), reads as the product blaming the weather.

The forgivable framing is shared work-in-progress. Errors presented as drafts to steer — “here’s my attempt; tell me what to change” — recruit the user as an editor instead of putting them on trial as a prompter. It’s the difference between a junior colleague showing you a first pass and a vending machine dispensing the wrong item. People forgive colleagues; they kick vending machines. Copilot’s ghost text nails this register: a suggestion is an offer, dismissed without ceremony, and its wrongness never feels like an indictment of anyone.

Two rules follow. Never let boilerplate do the apologizing — a disclaimer under every response is the product disclaiming responsibility in advance, and users read it exactly that way. And when the failure genuinely was the system’s — a context failure, a tool outage — say so specifically. “I lost the attachment; please re-add it” costs you a moment of pride and buys back the user’s willingness to try again.

Every correction a user makes is a labeled example of your product failing — the most valuable data your team can collect, offered free, at the exact moment of failure. The user who edits the AI’s draft, rejects the suggested fix, re-asks the question with clarification, or rolls back an agent’s change has just told you precisely what wrong looks like. Most products let that signal evaporate into session logs nobody reads. The disciplined ones pipe it straight back into the eval suite you built in chapter two: today’s correction becomes tomorrow’s golden-set case, and the failure class gets a regression test before it gets a recurrence.

This is why verification affordances and feedback mechanisms are secretly the same investment. Citations, diffs, and visible reasoning don’t just help users catch errors — they generate structured evidence of which errors users caught, and where. Cursor’s accept/reject on every edit is simultaneously a recovery affordance and a per-suggestion labeling pipeline. A thumbs-down with no follow-through is theater; a correction that visibly changes system behavior is a flywheel.

Close the loop organizationally, not just technically. Someone on the team should own the correction stream the way someone owns the crash log — triaging misses into the taxonomy from this chapter’s first section, promoting the recurring ones into evals, and killing the failure classes that keep paying out. Products that treat errors as embarrassments plateau. Products that treat them as the roadmap compound.