Agent Memory

Agent memory is the set of mechanisms that let an LLM-driven agent hold onto information across turns, sessions, and tasks rather than forgetting everything once its context fills. A language model is stateless between calls, and its context window is finite; durable memory must therefore live outside the model — in files, databases, or vector stores — and be selectively pulled back in when relevant. How an agent writes, retrieves, and prunes that external record largely determines whether it behaves like a colleague who learns or a goldfish with tools.

Short-term versus long-term

Short-term memory is the agent's working memory: the tokens currently inside the context window — the live conversation, recent tool results, intermediate reasoning for the task at hand. It is fast, complete, and strictly bounded; when a long session overflows it, agents summarize or compact older turns to make room, trading fidelity for continuity. Long-term memory persists outside the model, commonly in a vector database or a structured store, and is queried at decision time so that only the few facts relevant to the current step re-enter the window — the same retrieval discipline that powers RAG. The interplay is the craft: a memory system that retrieves too little makes the agent amnesiac; one that retrieves too much floods the window and buries the signal.

Categories of long-term memory

Practitioners borrow cognitive-science labels. Semantic memory holds general facts: your fleet's models, your pool credentials' structure, your preferences. Episodic memory records specific past events and their outcomes — this repair worked, that command bricked a control board — see episodic vs semantic memory for the distinction in depth. Procedural memory captures how to act: learned rules, checklists, and habits, sometimes folded into the agent's instructions themselves. A mature memory system manages all three, and — just as importantly — decides what to forget: stale facts actively mislead, so expiry, contradiction-checking, and consolidation (merging episodes into general lessons) are as much a part of memory management as storage.

Write policies and retrieval scoring

Two design decisions dominate real implementations. The first is the write policy: what earns a place in long-term memory — explicit "remember this" instructions, automatic summaries checkpointed as the window fills, or reflections distilled after a task ends. Plain-text memory files that the agent reads at startup are a popular, auditable form of procedural memory precisely because a human can open and edit them. The second is retrieval scoring: mature systems rank candidate memories by a blend of semantic relevance, recency, and importance rather than similarity alone — a pattern popularized by the "generative agents" research — so an old-but-critical lesson can outrank a fresh-but-trivial one. Get both policies right and the agent improves with age; get either wrong and it drowns in its own diary.

Memory as sovereign territory

Memory is what lets an agentic workflow accumulate experience instead of starting cold on every run — and that accumulation is precisely why its location matters. An agent that has worked alongside you for months holds a dense behavioral record: your infrastructure, your holdings, your habits, your mistakes. Hosted agent platforms keep that record on someone else's servers, subject to someone else's retention policy, subpoenas, and breaches. Keeping memory stores on your own hardware — the same instinct as self-custody for keys — means the most personal database an AI will ever build about you never leaves your control, and remains portable across whatever models you run next. The model is replaceable; the memory is yours. D-Central documents memory architecture as a core concern for locally run AI, because an agent's memory is either an asset you own or a dossier you rent back.