What is agent memory?

The TL;DR#

Large language models (LLMs) are stateless: they don't remember anything between conversations.

Agent memory is a system outside the model that writes things down and hands them back later.

• Memory can be as simple as files that you keep updated, and pass back to the model manually.
• Memory comes in four flavors: working (what's in front of the model right now), episodic (what happened), semantic (facts) and procedural (how to do things).
• Most AI labs and commonly used products now include some type of built-in memory mechanism.
• The challenge is when to forget memories that have stopped being true.

In three years of memory engineering, AI memory went from "copy-paste your own notes" to background processes that rewrite what your assistant knows about you while you sleep. AI labs call it “dreaming” - it’s super challenging to get right, and is a major problem to be solved.

Geniuses with goldfish memories#

Every LLM has an embarrassing secret: it remembers nothing.

A model that spent three hours helping you debug your app yesterday has no idea who you are as soon as you open a new chat window.

Models are frozen at the end of their training. It’s why you can surprise them with new information if they aren’t enabled with web search tools. The only thing they can "see" is the context window, the text sent to them with each request. Close that tab and the relationship resets…unless they have agent memory.

Wen Sang, co-founder of Genspark, described today's AI models on my podcast as "geniuses with goldfish memories" and I think that’s apt.

It takes memory systems engineering to give them the information they need to “remember” you.

If Claude knows you're vegetarian, it’s not because anything changed inside the model. Somewhere, a system wrote down "user is vegetarian," and it quietly slips that note into your future conversations.

Richmond Alake, who coined the discipline memory engineering, breaks it into four verbs that run as a loop:

1. Retain: capture what matters from a conversation or task, and write it down.
2. Recall: when a new request comes in, find the memories that matter.
3. Reuse: put them in front of the model so they shape the next answer.
4. Refine: keep them true. Correct what changed, merge duplicates, and let stale memories expire.

Most memory systems are good at the first three. Refine (keeping memories true) is where they usually break. Your assistant keeps reminding you to buy flights you already booked or flags a deadline you shipped last week. Those notes were true once, but they went stale, and nothing updated the system: perhaps another chat took care of your flight booking, but didn’t tell the memory your other sessions read from.

Richmond, who leads AI developer experience at Oracle, thinks solving memory (and when to change it) is where the next big fight in AI is happening. In April he told me: "Memory is the last battleground for anyone that operates in the AI stack." Labs, database companies and startups all want to be the place your AI's memories live.

The four kinds of memory#

AI researchers borrowed four categories from cognitive science to describe what agents store (the CoALA framework is the usual reference). A handy way to keep them straight:

• Working memory = scratch pad. It's whatever is in the context window right now: your current conversation, the file the agent just opened.
• Episodic memory = diary. What happened, and when: "we tried this database migration on Tuesday and it failed." Usually stored as transcripts or summaries the agent can search later.
• Semantic memory = textbook. Facts about you and your world: "vegetarian," "works at Google," "the staging database is called latex-2."
• Procedural memory = muscle memory. How to do things: instructions, preferences, skills. "Always run the tests before committing." In coding agents this is literally a text file of rules (like Claude Code's CLAUDE.md) loaded into every session. However, like humans, agents don’t always remember to follow these rules.

These categories are important to remember, because each goes stale at different speeds. A workflow rule can stay true for months. A fact like "the launch is on Friday" is stale, well, after Friday. A memory system that treats them all the same will misbehave in creative ways.

How we got here: a recent history#

The copy-paste era: Early ChatGPT forgot everything between chats. Power users kept their own notes and pasted them into each new conversation. You were the memory, like your own personal elephant.

RAG, memory as search (2020 onward): Retrieval-augmented generation chops documents and old conversations into chunks, turns them into numbers that capture meaning (embeddings), and stores them in a vector database. When a question comes in, it searches for similar chunks and hands them to the model. RAG is still everywhere today, but searching a pile of notes isn't the same as knowing something.

MemGPT, memory as an operating system (October 2023): Researchers at UC Berkeley borrowed a trick from how computers manage RAM - give the model a small "main memory" that's always in view, plus bigger storage it can page information in and out of by itself. That paper became the company Letta, and the idea of agent-managed, tiered memory shows up in nearly every serious system since.

Memory becomes a product (February 2024): ChatGPT shipped a visible list of facts it had learned about you. Two problems showed up fast - you had to manage the list yourself, and facts went stale: "user is traveling to Bengaluru" kept loading long after you got home.

Dreaming (2026): Now the big labs run background passes that review and rewrite memories between sessions. Anthropic introduced dreaming for developers' agents in May. OpenAI shipped Dreaming for ChatGPT in June. And capture is spreading beyond chat: OpenAI's Computer History (August 2026, opt-in, Mac only) turns activity in the apps and websites you allow into memories, stored on your machine as plain text files.

How to forget#

Most memory systems are great at adding things and mediocre at retiring them.

Shlok Khemani, who spent a year reverse-engineering how ChatGPT and Claude remember, saw it go wrong in his own profile. He used ChatGPT to weigh a vacation, Thailand versus Turkey, and went to Thailand. His profile recorded trips to both, on overlapping dates, because it summarized his deliberations and nothing ever checked what actually happened.

If I’m not careful, my agents will soon think my wife and I have 5 or more cats, given how much time we’ve spent considering kittens on our local humane society’s website.

We have to treat consideration differently than doing, and we need to ensure our AI get the updates they need when something gets done. Two weeks after I gave my agents persistent memory, one agent told me a contract I needed to get signed was still pending - but it’d been signed days ago, and nobody had thought to update the agent’s memory.

Researchers are now measuring this:

• On a 2026 benchmark called STALE, where later information quietly invalidates an earlier fact, the best model tested scored only 55.2% overall.
• In another study (TEPA), when facts completely reversed, agents with simple "just keep adding" memory scored 0.21. Agents with no memory at all scored 0.31. Remembering the wrong thing proved worse than remembering nothing.

A lot of the interesting design work for AI memories is now in forgetting, which isn't the same as deleting. Deleting erases a memory, and that's the right call when someone asks you to remove their data. Forgetting is closer to updating an address book: you cross out the old address and write the new one next to it, so you know where someone lives now and where they used to. Forgetting is a policy you set, and it usually means adding information, not removing it:

• Expiry dates for facts. Mark when a fact was true, and replace it (keeping the history) when something newer and more authoritative comes along.
• Fading. Rank memories by how recent, important and relevant they are, and let low scorers drop out of view.
• A review pass. Periodically check stored memories against the real source (the approval system, not your old notes about it) and propose fixes, with evidence attached. This is what the labs' dreaming passes do, and you can build a basic version yourself.

Should you build it or buy it?#

Sooner or later you'll want a memory in something you're building. Two paths:

Buy a memory layer. Platforms such as Mem0, Zep, Walrus Memory, and Letta handle extracting facts from conversations, storing them and retrieving the right ones, each with a different approach (Zep, for example, models facts with validity windows over time). Buy when memory is core to your product and you need it working for many users on day one.

Build it with files. The unglamorous option goes further than you'd expect. For a single user, a folder of text files (one per topic) plus a small index the agent reads at startup gets you capture and recall in an afternoon. You can open it in a text editor, track changes with git, and take it to a different model. Coding agents settled on this for a reason, and OpenAI's newest memory feature writes plain text files too. I built an open source template for my own army of coding agents.

Files break at predictable points: many users, permissions, audits, scale. Most personal and internal agents will never reach a breaking point.

Whichever route you take, check whether your memories can leave. Many memory systems make it hard to move memory between models. Memory you can't move ties you to whichever model or tool you started with, so make it portable.

A quick decision guide#

• Just you, one agent, your own machine? Text files plus an index, tracked in git and you’ll be good to go.
• A product where many users expect memory? A memory platform, or a database you own if memory is what makes your product special.
• Long-running agents that act on their own? Agent-managed memory plus a background review pass.
• Regulated or audited? A database with a record of where every fact came from and when it was valid. Loose files won't survive an audit.

Whatever you pick, make sure you set a policy of how facts expire, how they get replaced, and how you’ll review changes. If you don’t, you’re at the mercy of whatever your LLM confidently (and likely incorrectly) decides for you.