WS—04 · Over time
Before you let it remember
Memory, personalization, and model updates are where AI products quietly betray their users. Run this session before shipping anything that persists across sessions or changes behavior based on history. The bar: the user can always answer “why is it acting like this?”
Checks
Discussion prompts
- Would our users be comfortable reading their own memory file out loud in a meeting? If not, why are we comfortable storing it?
- What happens to a user’s trust the first time the product remembers something they didn’t know it heard?
- When we upgrade the model and outputs change tone overnight, what tells the user it was us and not them?
- Whose interests does this personalization serve when the user’s preference and our engagement metric disagree?
- What does this feature look like after two years of accumulated memory — sharper, or haunted by outdated context?
- If a user asked “why did it do that?” about an adapted behavior, how many of us could answer from the UI alone?