The Model Is a Material

Deterministic software let designers work from a single happy path because the software had one. Given the same input, the button did the same thing, forever. A model does not. The same prompt produces different phrasings, different structures, occasionally different answers — and the distribution shifts with the model version, the context length, even the surrounding conversation. Designing for a model means designing for a distribution of outputs, not an output.

This is not a defect to be engineered away; it is the property that makes the material useful. The same flexibility that lets a model summarize a contract it has never seen is the flexibility that lets it get the summary wrong. Wood grain is why wood is beautiful and why it splits. The design question is never “how do we make it deterministic” — it is “where does variance help, where does it hurt, and what does the interface do about each.”

In practice this changes your artifacts. A static mock of an AI feature is a lie of omission — it shows one draw from the distribution and implies it is the product. Teams that handle the material well design in ranges: the great response, the mediocre one, the confidently wrong one, the refusal. If your design review only ever sees the great one, you have reviewed a poster, not a product.

One demo proves nothing. A cherry-picked transcript tells you the ceiling of the feature; it tells you nothing about the floor, and the floor is what users remember. The instrument that tells you both is an eval: a fixed set of representative inputs — a golden set — run against the system, graded against what a good answer looks like, every time anything changes.

Designers should be able to read one the way they read analytics. When the ML team says “the summarizer scores 87%,” the design questions write themselves: 87% of what inputs? Graded by whom, against what rubric? What do the 13% look like — mildly awkward or dangerously wrong? An aggregate score is a compression of exactly the information a designer needs, and you decompress it by reading the failures, not the average.

Better still, contribute to the golden set. Designers are usually the people closest to real user inputs — the vague ones, the typo-ridden ones, the ones that smuggle in two questions. An eval built only from clean engineering-authored inputs will pass while the product fails. When Anthropic and OpenAI publish model cards, the interesting pages are the eval tables and their caveats; internalize that habit at product scale. The eval is where the team’s definition of “good” is actually written down — if design didn’t help write it, design delegated the most important spec in the product.

A feature that works 95% of the time sounds shippable until you do the arithmetic: at a thousand uses a day, that is fifty daily failures, and every one lands on a person who does not know they drew the short straw. The pass rate tells you how often you fail; the failure distribution tells you how it feels — and the second number is the design brief.

Not all misses cost the same. A summary that is stiffly worded is a shrug; a summary that inverts a “not” in a medical note is a catastrophe at the same eval score. So sort your failures by consequence, not frequency. Then design the interface for the miss: calibrated confidence so shaky answers look shaky, graceful degradation so a failed retrieval produces a labeled partial answer instead of a silent fabrication. Copilot’s ghost text is the canonical example of interface absorbing failure — a wrong suggestion costs one keystroke to reject, so a high miss rate is survivable that a modal “here is your code” would not be.

The uncomfortable discipline: the 5% is not an edge case, it is a customer segment. Someone hits it every day. Design their experience with the same intent you design the happy path, because they are the users deciding whether your product is trustworthy — the delighted 95% were never the ones at risk of leaving.

Three properties of the material show up directly in the interface, and pretending they belong to engineering produces designs that cannot be built as drawn. Latency first: model responses take seconds to minutes, and the gap between “instant” and “eventually” is an interaction design problem. Streaming is the load-bearing answer — tokens render as they generate, dead wait becomes read time, and the person can abort a bad answer three seconds in rather than thirty. ChatGPT, Claude, and Perplexity all stream because nothing else makes conversational latency tolerable; deep-research modes narrate multi-minute progress because a silent spinner is indistinguishable from a hang.

Cost second: every token in and out is metered, which makes design decisions into unit-economics decisions. Auto-summarizing every document a user opens versus summarizing on request can differ by orders of magnitude in spend for the same perceived feature. Model choice is the other lever — fast cheap models for autocomplete, slow expensive ones for deep work — and products like ChatGPT surface that trade-off as user-facing modes rather than hiding it.

Context third: models have a finite window, and what fills it is a design decision disguised as a technical one. Which document, how much history, whose preferences get included determines what the model can possibly know — Cursor’s @-mentions and Claude’s file attachments make context assembly a visible user act precisely because implicit context guessing fails illegibly.

Somewhere in your product is a system prompt: the standing instructions that tell the model what it is, how to behave, what to refuse, and what tone to take. It is the single highest-leverage piece of writing in the product — it shapes every response every user ever sees — and in most organizations no designer has read it. That is like shipping an app where nobody on the design team has seen the UI copy.

The system prompt is where the personality you specced actually gets implemented. If the product brief says “concise and warm” and the outputs are verbose and stiff, the bug is probably in the prompt, and the fix is a writing task, not a modeling task. Anthropic publishes Claude’s system prompts, and reading them is instructive: they are long, specific, full of behavioral edge cases — when to refuse, how to handle ambiguity, what format to default to. That is design specification by another name, maintained with the care of design documentation.

Treat it accordingly. Version it, review changes to it the way you review copy changes, and eval it — a one-line tweak to a system prompt can shift behavior across every feature at once, which is exactly why it deserves a golden-set run before it ships. Content designers are the natural owners here; the teams that figured this out early simply gave their best writer commit access to the prompt.

You cannot develop material intuition from meetings. Chefs taste; designers working with models need a kitchen — a low-friction place to run the same input against multiple models and settings and read the outputs side by side. Every major provider ships one: Anthropic’s and OpenAI’s developer consoles both offer playgrounds where you can set a system prompt, vary the input, and compare runs. None of this requires writing code.

What you learn in an hour of side-by-side comparison would take a quarter to absorb from dashboards. That the small fast model handles your product’s most common request nearly as well as the flagship — a cost decision design can now argue with evidence. That your feature’s prompt collapses when the input contains a table. That one model hedges everything and another never hedges at all, which is a voice decision, not a benchmark decision. The choice of model is partly a design choice, and you can only make it if you have handled the candidates.

Make it a practice, not an event. Ten real user inputs — the messy ones from support tickets, not the polished ones from the launch deck — run through your actual system prompt on the models under consideration, once a month. It is the cheapest research program in your organization, and it converts you from a consumer of the ML team’s conclusions into a participant in them.

With deterministic software, QA answers “does the new thing work.” With a model in the loop, every change — a prompt edit, a model upgrade, a retrieval tweak — reshuffles the entire output distribution. The new version can be better on average and worse in ways your users depended on. Model upgrades that improved benchmarks while breaking beloved behaviors have burned nearly every AI product’s community at least once; “the update made it worse” threads are the genre’s most reliable churn signal.

The tool for this is the regression eval: run the golden set on the old version and the new one, and diff the failures. Not the scores — the failures. A change that moves 87% to 89% but breaks a previously-solid behavior that support tickets prove people rely on is a regression wearing an improvement’s clothes. Design belongs in this review because design owns the judgment call the numbers cannot make: which behaviors are load-bearing for trust.

And when behavior does change, say so in the open. Silent model swaps convert every user’s hard-won intuition about the product into misinformation overnight — the person who learned exactly how to phrase requests wakes up to a stranger. A changelog entry, an in-product note, a beta channel for the new model: these are cheap. Rebuilding the trust of users who discovered the change by being burned by it is not.