Agent memory: scratchpad vs long-term store

A goldfish and a default agent have the same problem. Nothing survives past what's directly in front of them. Close the session, and every constraint you spent ten messages carefully establishing is just gone, the agent starts the next run exactly as blank as the first one.

The previous post covered the loop that runs within a single task. This one is about what, if anything, survives after that task ends.

Short-term memory is just the context window

Within one run, an agent's "memory" is whatever's sitting in its context window, the running scratchpad of everything said and done so far in this session. That's not really memory in any persistent sense, it's closer to working memory, and it disappears the moment the session does.

This is fine for a lot of tasks. A single conversation, a single multi-step job that completes in one sitting, doesn't need anything beyond the scratchpad it already has. The problem only shows up when a task needs to remember something across sessions, a user's stated preference, a fact learned last week, a decision made yesterday that shouldn't need re-explaining today.

Long-term memory means an actual store

For anything that needs to survive past one session, you need somewhere to write it down and somewhere to read it back from later. Usually that's a vector database for anything you'll want to search by meaning later, or a plain key-value store for anything with a fixed, known shape, like "user's preferred language" or "last five orders."

The mechanism is simple. At some point during or after a run, decide something is worth remembering, write it to the store. At the start of a new run, pull back whatever's relevant to the current task and put it into the context window before the agent starts working. The agent never "remembers" anything on its own, it's re-fed its own past notes every time, the same way you'd read your own journal before starting the day.

What actually needs to persist

The temptation is to save everything, every message, every tool call, every result, on the theory that more memory is strictly better. It isn't. A store stuffed with low-value noise makes retrieval worse, the same failure mode as a RAG index full of near-duplicate chunks, and it costs money and latency to write and search through all of it.

What's usually worth keeping: durable facts about the user or task that won't change soon, corrections the user explicitly made ("no, I meant X" is exactly the kind of thing worth remembering so you don't ask again), and outcomes of decisions that would be expensive to redo. What's usually not worth keeping: the routine back-and-forth of a session that resolved cleanly and has no bearing on anything future.

The failure mode on the other side

It's possible to remember too eagerly in a different way too: an agent that treats a one-off, context-specific instruction as a permanent rule. If someone says "skip the intro for this one email," that's about this email. Saved as a permanent preference, it silently changes behavior on every future email, and nobody asked for that. Deciding what's a durable fact versus a one-time instruction is a judgment call the system has to make deliberately, it doesn't happen automatically just because you built a memory store.

One agent, given a working loop and something worth remembering, can do a lot. The next question is what happens once one agent stops being enough for the task.