PTN—40
Feedback loops
Collect user feedback at the moment of reaction, then visibly change behavior because of it.
Every AI product ships thumbs up/down. Almost none close the loop. The buttons collect training data for a model retrain months away, the user sees nothing change, and within a week they stop clicking. A feedback loop is only a loop if the person can perceive the return path: the correction takes effect now, in this session, on this kind of output — or the interface says honestly that it will take time and where it went.
The strongest feedback mechanisms don’t look like feedback mechanisms. An edit to generated text, a rejected suggestion, a regenerate with a steering note — these are corrections captured in the flow of work, with far richer signal than a binary rating. Cursor and Copilot learn more from which completions get accepted than any thumbs widget could gather, because acceptance is feedback people give without deciding to give it.
When you do ask explicitly, ask small and specific. “Was this helpful?” after every response is a tax; a targeted “Too long?” chip after a visibly long answer is a conversation. And whatever you collect, spend some of it locally: if someone downvotes a verbose answer and the very next answer is shorter, they’ve learned that feedback matters here. That lesson is worth more than the data point.
When to use
- Output quality varies and the system can actually act on corrections.
- People already edit, accept, or reject output — implicit signal is on the table.
- You want personalization but would rather earn it from behavior than infer it silently.
- A response class keeps missing (tone, length, format) and a targeted prompt could catch it.
Design considerations
- 01Capture implicit signals first — edits, accepts, rejects, regenerates — before adding explicit widgets.
- 02On a downvote, ask one specific follow-up (“inaccurate / too long / wrong tone”), then stop.
- 03Apply corrections within the session where possible; “Got it — keeping answers shorter” is the loop made audible.
- 04Tell people where feedback goes when it can’t act immediately, in one honest sentence.
- 05Never let a rating interrupt the task; feedback affordances sit beside output, not in front of it.
- 06Track whether feedback rates decay — a falling thumbs rate is the loop telling you it’s open.
Pitfalls
- ✕The write-only thumbs. Feedback disappears into a dashboard nobody reads, users notice nothing changes, and participation collapses — then teams conclude “users don’t give feedback.”
- ✕Over-rotating on one signal: a single downvote flips the assistant into a different personality, which is as trust-destroying as ignoring feedback entirely.
- ✕Modal surveys after every interaction — you’re charging people time to help you, mid-task.
- ✕Treating regenerate clicks as neutral. A regenerate is a soft downvote; products that ignore it are discarding their highest-volume quality signal.
In the wild
- GitHub Copilot
- Treats completion acceptance and rejection as its core quality signal — feedback captured entirely inside the act of coding, no widget required.
- Cursor
- Accept/reject on inline diffs makes every AI edit a feedback event; rejected changes vanish cleanly, so saying “no” costs nothing.
- ChatGPT
- Pairs thumbs-down with a quick reason picker, and regenerate offers a comparison between responses — turning a rating into a preference judgment.
- Midjourney
- The upscale/variation grid is a feedback loop as the primary interface: choosing one image over three siblings is a preference signal that immediately shapes the next generation.