Agent Memory vs. RAG: What AI Agents Remember and How They Know

In October, we tested a new paid feature in an SEO tool. We decided it was overhyped and canceled the trial we had planned. The AI agent we work with saved that in its memory as a single line: what we rejected, when, and why.

Without that line, the agent would have brought the same feature up again in the next report. From its point of view, it's still an interesting new option. With the line, it checks whether we've already said no before it suggests anything. We call this the memory of refusals.

That's what good agent memory looks like. The question isn't how much an agent remembers. It's whether every memory has a source, a date, and an owner. Memory without a source is just a guess. Repeat a guess a few times, and it starts to pass for fact.

This guide explains how agent memory differs from RAG and from ChatGPT's memory, when memory causes damage, and what GDPR says about it. It also shows the four rules we use to run our agents' memory, with real entries. At the end, you'll find a prompt you can use today to clean up your own assistant's memory.

What is agent memory?

Agent memory is the set of notes an AI agent saves from conversations and tasks so it doesn't start from scratch next time. It's not the same as the company documents an agent searches (RAG). It's also not the same as what the agent sees inside a single conversation (the context window).

The word "agent" covers two things here. One is a chat assistant, like ChatGPT or Claude, that a person works with. The other is an agent that does tasks on its own, such as answering customers or preparing reports. Both can have memory, but it works a little differently.

The third column matters most. A document has someone in the company who keeps it up to date. No one in the company owns the agent's notes.

Why does an AI agent forget after every conversation?

Because a language model has no memory between conversations, and every chat starts with a blank page. Anything the agent "remembers" has to be handed to it again. A person can paste it into a message. A system can pull it from documents. Or the agent can read it from its own notes.

Agent memory is the third route. The agent writes a note outside the model and reads it in the next conversation. The model doesn't learn anything along the way. It just reads what someone wrote down earlier. That's why the quality of memory depends on the quality of the notes, not on how smart the model is.

What is RAG, and how is it different from agent memory?

RAG (retrieval-augmented generation) is a method where the agent searches your company's documents before it answers. Then it answers the question based on the passages it found. Put simply, RAG is a filing cabinet the company organizes, and memory is a notebook the agent keeps for itself.

The difference is who writes. RAG holds what someone in the company chose to write down, like a price list, a procedure, or an offer. Memory holds whatever the agent decided was worth keeping while it worked. Sometimes that's a good call, and sometimes it isn't.

Plenty of companies assume that more complex memory makes a better agent. A study from October 2026 suggests otherwise. The authors compared complex memory systems with a simple search over the record of past conversations. The simple approach won in 6 of 7 test settings. It's a preprint, meaning it hasn't been peer-reviewed yet, and the margin was sometimes small. Still, the takeaway for a business is practical: don't start with complex memory. Start by putting what you already have in order.

We cover how a company manages the knowledge its agents rely on in our article on the context layer.

Your memory, your project's memory, your company's memory: what's the difference?

Not every company has one shared memory. Each employee has their own ChatGPT or Claude account, works in their own chats, and has their own memory. One person doesn't use another person's memory. That's not a flaw. It's three different levels, and they differ in who sees the entries and who maintains them.

You get the first two levels in off-the-shelf tools today. In Claude, each project has its own memory. On team plans, the owner of the company account can't view or edit employees' memories. Memory is off by default there, and each employee turns it on for themselves. So everyone can shape their own memory to work better, and no one else touches it.

The third level has to be built. When an agent answers customers on its own, it can't rely on one employee's memory. It needs a shared source where every piece of information has an owner and a date. Otherwise, it'll tell a salesperson something it picked up from a conversation with the bookkeeper.

How do we run our agents' memory?

We work with AI agents every day, writing code, articles, and reports with them. Every session starts with no knowledge of the last one, so the agent needs memory. Along the way, we settled on four rules. None of them needs an expensive tool, because they're all about where an entry comes from and how long it stays valid.

1. A map, not a pile. At the start of every task, the agent reads a short index that says "doing X, check Y." It loads everything else only when it needs it. The order never changes: settled answers first, then project status, then the raw source. That's how we pick up topics from earlier sessions without explaining everything again.

2. Entry weight. Every entry has a weight: an owner's decision, a verified fact, a note, or the agent's guess. When two entries conflict, the higher weight wins, and with equal weight, the newer one wins. The agent's guess never overrides the boss's decision.

3. An evidence count. A lesson from one event goes into memory as a hypothesis. Each time it's confirmed again, we add the date. It becomes a rule only after two independent confirmations, so a single slip doesn't change how we work.

4. A validity window. Changing a fact doesn't erase the old entry. The new version gets a "valid from" date, the old one gets an end date, and both stay in the history. The agent then says "the price valid from that date" instead of quoting a number with no time attached.

Here are two real entries from our memory, rewritten without jargon:

The first entry is the memory of refusals from the start of this article. The agent also remembers what we don't do and why. The second entry has been confirmed twice since it was first saved, so it's now a rule, not a one-off remark. This article follows it too.

When does agent memory cause harm?

Memory causes harm when the agent trusts an entry that's no longer true, or never was. We see four situations like that.

An outdated entry is the most common case. A company changes its prices. For the next month, the agent keeps quoting the old rate to customers because it picked it up in a conversation. A contradicting entry can be more dangerous because no one notices it. The returns policy says 14 days. But the agent remembers that "we make an exception for regular customers," and no one ever approved that.

Researchers studied the last two situations in 2026. With someone else's entries, one simple thing helps: recording who the entry belongs to. They tested this on 305 pairs of runs. An agent that had owner information on each entry used someone else's login credentials once instead of 92 times.

Planted entries are harder. In another test, a single manipulated web page poisoned an agent's memory, and the fake entry kept working in later tasks. The attack succeeded in up to a third of attempts. Both studies are preprints, and both ran in test environments, not in real companies. Still, the direction is clear.

For more on where wrong answers from models come from, see our article on why AI hallucinates.

What can an agent remember about a customer, and how do you delete it under GDPR?

An agent can remember anything said in a conversation: a customer's name, contract terms, a health issue mentioned in passing. If that's personal data, a memory entry falls under GDPR just like a CRM record does. This applies to companies in the EU and to anyone who serves customers there. Outside the EU, check what your local privacy law requires.

GDPR boils down to three simple questions here. What purpose did the agent save this for? Did it need to, or is this more than the work requires? How long will the entry be kept? On top of that, Article 17 gives people the right to have their data erased. A customer can ask the company to delete their data, including what's in the agent's memory.

Here's a catch few people know about. In ChatGPT and Claude, memory is a separate set of data, apart from chat history. OpenAI and Anthropic both state that deleting a chat doesn't automatically delete the memories created from it. An employee deletes the chat where they pasted a customer's contract, and the memory from that chat stays.

So a deletion request takes several steps:

1. Delete the memory in the assistant's memory settings. Deleting the chat doesn't cover it.
2. Delete the chats where the customer's data came up.
3. Delete files and disconnect apps the assistant could have pulled that data from. ChatGPT, for example, can use saved files and connected apps.

OpenAI also notes that it may keep logs of deleted memories for up to 30 days. It's worth telling the customer that up front.

With a custom-built agent, it's not the model provider who decides where memory lives and who can see it. That's up to whoever deploys the agent. Ask about it before the agent starts talking to customers. For more on where data you type into AI tools ends up, see where your company data goes when using AI.

What belongs in a document, and what should agent memory keep?

In our view, anything that has to be right every time belongs in a document: your offer, your prices, what the company won't do, and how it talks. A document like that is visible, someone owns it, and they know when they last changed it.

Agent memory gets the rest: preferences and corrections. For example, that a customer prefers answers in bullet points, or that we rejected an idea in October. Always with a date and a note of who it came from.

That's how we build company memory: a person owns the source, and the agent only reads from it and adds dated corrections. We describe the same split in our free company-context template. An AI tool's memory builds up on its own, bit by bit, from conversations. So keep what has to be right every time in a file, not in memory.

Where should a business start?

Not with picking a tool, but with three simple steps.

1. Pick one area of knowledge. For example, customer questions about delivery times or your returns policy. One area, not the whole company.
2. List the sources and their owners. Which document holds the truth in this area, and who updates it. Without an owner, a document turns into a guess too.
3. Only then turn on agent memory. With the four rules: a map, weights, an evidence count, and a validity window. Once a month, someone reviews what the agent has saved.

If you use ChatGPT or Claude every day, you can start even smaller, with your own memory. Paste this prompt into your assistant:

If you have a lot of conversations, go topic by topic afterward. For example, customers, pricing, and how you work, each on its own:

OpenAI itself points out that you can ask ChatGPT what it remembers about you. Keep in mind that you'll get an overview, not a complete list. In Claude, you'll find the full memory in settings, and in a chat you can ask it to remember, change, or forget something. The result will help you see which entries have no source. Move the confirmed ones into your own document, and fix or delete the rest in memory settings.

If you want to do the same for your whole company, let's start with one area of knowledge. That's how we work on company memory.

Frequently asked questions

Does ChatGPT remember chats even if you delete them?

It can. ChatGPT stores saved memories separately from chat history, so deleting a chat doesn't automatically delete a memory created from it. To remove that information, delete the memory too. OpenAI notes it may keep logs of deleted memories for up to 30 days.

Does an AI agent learn from my data?

Memory isn't training. The model stays the same, and the agent saves notes outside it and reads them in the next conversation. Whether a provider uses your chats to train its models is a separate question, covered in its privacy terms. For more, see where your company data goes when using AI.

Can't I just paste all my documents into the chat?

For one conversation, sometimes. The agent sees everything you paste, but it's gone when the chat ends, and next time you have to paste it again. Memory and sources someone keeps up to date carry that knowledge forward.

Does Claude have memory, and who can see it?

Yes. Each project in Claude has its own memory, and you can see the full memory in settings. On team plans, the account owner can't view or edit employees' memories, and memory stays off until each person turns it on.

Do you need a vector database for agent memory?

Not to start. Simple setups are enough at first, which the October 2026 study also suggests. What matters more than the technology is whether every entry has a source and a date.

What happens to memory if we switch tools or model providers?

It depends on where the memory lives. If it's in your own files, you take it with you. If it's with the provider, check before you start whether you can export it.

Who in the company is responsible for what the agent "knows"?

Whoever owns the source. The agent can save corrections, but a named person is responsible for the document.