Legibility Is the Product

A black-box AI product works fine right up until the moment trust actually matters. While the answers are correct, nobody asks how they were made. The first time an answer is wrong — and there is always a first time — the person has exactly two options: catch it themselves through independent expertise, or discover it downstream when the fabricated statute doesn’t exist or the summarized email said the opposite of what it claimed. The interface offered them no third option, because it showed nothing that could be checked.

The damage is not scoped to the wrong answer. A hidden failure, discovered later, retroactively poisons every answer that came before it — the person has no way to know which of the last fifty outputs were also wrong, so they discount all of them. This is why AI products lose trust in cliffs, not slopes. The lawyers sanctioned in 2023 for filing ChatGPT-invented case citations became the category’s cautionary tale not because the model hallucinated — models hallucinate — but because nothing in the interface distinguished a retrieved case from a composed one.

The uncomfortable implication for designers: your error rate is partly the model team’s problem, but your users’ ability to catch errors is entirely yours. A legible system with a 5% error rate outperforms an opaque one at 2%, because caught errors cost minutes and hidden errors cost the relationship. Legibility is how you make a fallible system safe to rely on.

There are two ways to make an AI system legible, and they are not equal. Explanation asks the model to justify its conclusion after the fact — “I recommended this because…” — which is generating more prose with the same machinery that produced the possibly-wrong answer. Process shows what actually happened: the searches run, the files read, the commands executed, the sources consulted. People are strikingly good at auditing a process (“it searched the marketing docs, but this is a legal question”) and strikingly bad at auditing a bare conclusion, however eloquently it defends itself.

Agent products converged on process legibility because nothing else survived contact with real work. Claude Code and Cursor stream a working transcript — every file read, every command run, every edit made — and that transcript is what makes delegation tolerable: you watch the agent head into the wrong directory in minute one and stop it, instead of excavating the wreckage in minute twenty. Deep-research modes in ChatGPT, Gemini, and Perplexity narrate their plan and progress across multi-minute runs for the same reason. A silent ten-minute spinner is indistinguishable from a hang, and an unexplained conclusion after ten silent minutes is indistinguishable from a guess.

The design discipline is choosing the altitude of the transcript. Raw tool-call logs are process, technically, but they audit like a core dump. The transcript worth rendering is the reasoning-shaped one: what was consulted, what was assumed, what was skipped, what failed and got retried. Log what happened; narrate what mattered.

Perplexity’s core insight was economic, not technical: an answer with sources and the same answer without them are different products at different prices. The cited answer transfers the burden of proof from the person to the interface — any claim can be audited in one click — while the uncited answer demands faith and offers nothing to inspect. Google’s AI Overviews, Bing Copilot, and ChatGPT’s search mode all landed on inline citations for the same reason. In answer engines, the citations are the product; the prose is packaging.

But a citation is only worth what it costs to verify. The failure modes are well documented by now: citations that link to pages that don’t support the claim, footnotes stacked at the end of a paragraph so no individual sentence is attributable, sources cherry-picked by the same retrieval that shaped the answer. A decorative citation is worse than none, because it borrows the credibility of the form while defeating its function — people click through once, find the mismatch, and stop clicking forever. If you render citations, claim-level attribution is the bar: this sentence, this source, this passage.

The generalization beyond search: every AI product has a citation economy, whether it renders footnotes or not. A coding assistant’s diff is a citation — the claim is “this change does what you asked,” and the diff is the checkable evidence. Count the clicks between any claim your product makes and the evidence for it. More than one is friction; no path at all is faith.

The obvious solution to overconfident AI — print a confidence score — fails in practice. Token probabilities don’t map to factual reliability, people read “87%” with a precision the number doesn’t possess, and a percentage on every sentence turns prose into a hazard map nobody can read. Model self-reports are worse: a system that hallucinates facts will also hallucinate its confidence about them. Numeric confidence is legibility theater with extra decimal places.

What works is calibrated language backed by real signals the system actually has. Retrieval came back thin. The question falls outside the indexed corpus. Two sources disagree. The request is past the knowledge cutoff. Each of those is a fact about the process, not a vibe about the answer, and each supports honest differentiated presentation: “here’s a well-sourced answer” reads differently from “here’s my best guess — verify before using it.” ChatGPT stating its cutoff when asked about recent events, or Claude declining to guess at a paywalled article’s contents, are confidence signals expressed as behavior rather than numbers — and they’re legible precisely because they’re grounded in something checkable.

The discipline is reserving hedges for real uncertainty. A system that hedges everything is as uncalibrated as one that hedges nothing; both give the reader zero information about where to spend skepticism. Audit your product’s language the way you’d audit a colleague’s: does its “probably” mean anything? If every answer carries the same disclaimer boilerplate, you’ve shipped a legal notice, not a confidence signal.

Total transparency is its own kind of opacity. An interface that shows every token of reasoning, every tool call, and every intermediate draft has technically hidden nothing — and communicated nothing, because the one line that matters is buried in forty that don’t. Legibility is not volume. It is structure: the answer up front, the evidence one gesture away, the full trace available to whoever needs to audit it.

The shipped consensus is a three-layer shape. ChatGPT collapses its reasoning behind a “thought for 40 seconds” disclosure — headline visible, receipts on click. Claude Code summarizes each agent step in a line and expands to full tool output on demand. Perplexity leads with the synthesized answer and holds sources in numbered references you pull only when a claim smells off. In every case the default view is sized for the common case — reading and trusting — while the expanded view serves the rarer, higher-stakes case: verifying. Streaming plays a supporting role here; watching the answer compose token by token is honest about its generative nature, but streaming is the texture of legibility, not the substance. The substance is what you can open.

The design test: can a skeptical user get from any claim to its supporting evidence in one interaction, without that evidence taxing the ninety percent of users who never look? If auditing requires scrolling a wall of logs, you’ve failed the auditor. If the audit trail dominates the default view, you’ve failed everyone else.

Every legibility pattern in this chapter has a counterfeit. Fake progress stages that animate while nothing runs. “Thinking” copy invented by a copywriter rather than derived from computation. Shimmer states choreographed to make a fast answer feel considered, or a stalled one feel busy. Citations rendered for authority rather than attribution. Each of these borrows the visual language of shown work without the work — and each is a bet that users won’t check. They check.

The unmasking is always cheap. Someone notices the “searching your documents…” stage appears even when the answer required no search, or that the progress narration is identical across wildly different tasks, or that the confident sourcing dissolves on the first click-through. And because the counterfeit shares an interface with the genuine article, the discovery doesn’t just discredit the fake — it poisons every legitimate transcript, citation, and confidence signal you render afterward. The user can no longer tell your real work from your performance, so they discount both. Trust theater is a loan against your honest surfaces, and the interest rate is brutal.

The rule is absolute in a way few design rules are: show real work or show a spinner. A plain spinner is honest about revealing nothing. A performance of diligence is a lie with production values, and it converts your best trust asset — the legible process — into evidence for the prosecution. If the pipeline doesn’t emit real events, don’t invent them; instrument the pipeline.