AI agent memory is not a single database or a model that permanently learns from every conversation. It is an application design: the system decides what to keep, where to keep it, when to retrieve it, and what to place in the model’s limited context for the next decision.
The useful distinction is between working memory, which supports the task happening now; episodic memory, which records selected past events and outcomes; and semantic memory, which stores facts that should remain useful outside one event. An agent may use all three, but each needs a different write rule, retrieval method, retention period, and deletion path.
This terminology is a design aid rather than a universal product standard. The implementation guidance below is provider-neutral. Relevant claims were checked against official LangGraph, Anthropic, and OpenAI documentation on 2026-07-24; links appear where those current platform details matter.
What agent memory actually means
A language model only reasons over the information available in its current input. The surrounding application creates continuity by carrying state forward or retrieving stored records. Memory therefore has two layers:
- Persistence: data survives beyond the request that created it.
- Recall: the application selects some of that data and returns it to the model at the right time.
Saving everything solves only persistence. A transcript archive that is never searched is storage, while an unrelated result from vector search is recall without reliable memory. Do not confuse memory type with duration: working state can be checkpointed for days, while a semantic preference may expire sooner. Also distinguish semantic memory—facts and concepts—from semantic search, a retrieval technique based on meaning. LangGraph’s official overview makes the same distinction (official LangGraph memory overview, verified 2026-07-24).
The model’s context window is the final assembly area, not the memory system itself. It can contain recent messages, instructions, tool results, retrieved memories, and external documents. The context window guide explains why this budget needs explicit allocation.
Working, episodic, and semantic memory compared
The three categories overlap in some implementations, so classify a record by how it will be used rather than by its storage engine.
| Memory type | Primary purpose | Typical records | Sensible write trigger | Retrieval pattern | Main failure risk |
|---|---|---|---|---|---|
| Working | Continue the current task | Recent turns, plan state, tool results, pending approvals | The active workflow needs the value on a later step | Thread or run ID, usually exact and ordered | Old or excessive context displaces current evidence |
| Episodic | Reuse lessons from a past event | Situation, action, outcome, correction, trace link | An outcome is validated, unusual, or instructive | Similar task plus metadata and scope filters | A superficially similar episode is applied in the wrong conditions |
| Semantic | Reuse a fact across events | Confirmed preferences, ownership, terminology, system facts | A fact is authoritative or explicitly confirmed | Exact, relational, semantic, or hybrid lookup | A stale or inferred fact is treated as current truth |
Retention should follow the record’s purpose and sensitivity, not this label alone. A working checkpoint may need to outlive a short semantic preference, and a high-risk episode may require a shorter lifetime than an ordinary project fact.
Working memory: state for the current task
Working memory contains the information needed to continue the active conversation or workflow. Typical fields include the recent message history, the current goal, a task plan, tool results, unresolved questions, temporary calculations, and identifiers for pending actions.
Keep this state structured where possible. Instead of asking the model to rediscover a pending approval from a transcript, store approval_status: "pending" beside the action and its immutable ID. Transcripts supply nuance; structured state should control execution.
Working memory often needs checkpoints. If a tool call fails or a person pauses the workflow, the system should resume from the last confirmed state without repeating completed side effects. That is especially important for agents that send messages, modify records, or trigger jobs. The human approval gate guide covers safe pause-and-resume behavior.
Do not keep an unbounded transcript in every model call. Older turns can be summarized, filtered, or moved into longer-term storage, but the summary must preserve current commitments, open constraints, and identifiers. Anthropic’s official context-engineering guidance describes context as a finite resource that must be curated as an agent accumulates messages, tool results, and external data (official Anthropic context-engineering guide, verified 2026-07-24).
Session features are implementation conveniences, not substitutes for a retention design. For example, the OpenAI Agents SDK can maintain conversation history across runs and lets applications limit retrieved history or clear a session. Its documentation also warns against combining an SDK session with server-managed continuation identifiers in the same run (official OpenAI Agents SDK sessions guide, verified 2026-07-24). Whichever mechanism you use, choose one source of conversational truth and document where it persists.
Episodic memory: selected events and outcomes
Episodic memory records what happened in a particular run: the situation, the action taken, the result, and any feedback. It answers questions such as “How did we solve a similar incident?” or “Which step caused this workflow to fail last time?”
A useful episode is more than a raw transcript. Store:
- a stable episode ID and timestamp;
- the task and relevant starting conditions;
- actions and tool versions;
- the observed outcome;
- human corrections or approval decisions;
- links to the trace or artifacts;
- scope, owner, retention class, and confidence.
Episodes can support few-shot retrieval: when a new task resembles an earlier one, the system retrieves a successful or cautionary example. Retrieval should consider task structure and constraints, not just topical similarity. A troubleshooting episode for a staging database may be dangerously misleading in production even when the error text is identical.
Do not turn every interaction into an episode. Keep corrected mistakes, validated procedures, unusual failures, or successful decisions under recognizable conditions. Aggregate repeated ordinary events or promote their lesson into a semantic fact.
Semantic memory: durable facts and relationships
Semantic memory stores facts intended to remain meaningful outside one event. Examples include a user’s confirmed communication preference, an organization’s approved terminology, a service ownership map, or the fact that a project uses a particular deployment region.
Store facts as explicit records rather than prose blobs. A practical record includes subject, predicate, value, source, created_at, last_confirmed_at, confidence, scope, and expires_at. Add a version or supersession link so a new fact can replace an old one without destroying history.
Separate observations from conclusions. “The user selected concise responses in three sessions” is evidence; “the user always wants short answers” is a generalization. The latter should not silently become a permanent profile. High-impact facts should come from an authoritative system or explicit confirmation, not model inference alone.
When semantic records conflict, prefer the correct scope and stronger provenance, show uncertainty, and retain superseded records for audit until their retention period ends.
Storage and retrieval choices
Choose storage by access pattern rather than by the word “memory.” A relational database is often suitable for typed facts, ownership, versions, consent, and deletion. A document or object store fits larger episodes and artifacts. A vector index helps find conceptually similar text, while keyword or metadata search is better for exact names, dates, IDs, and policy labels. Many systems need a hybrid.
Retrieval should proceed in gates:
- establish the authenticated user, tenant, and task scope;
- apply authorization and retention filters;
- generate candidates with exact, semantic, or hybrid search;
- rerank for relevance, recency, confidence, and source quality;
- fit only the strongest records into a memory budget;
- attach provenance so the agent or reviewer can inspect the source.
Never rely on vector similarity as authorization. Filter by tenant and permissions before model use, then verify access again when resolving sources. The multi-tenant RAG security guide applies the same defense-in-depth principle.
Keep working state and long-term memory logically separate even if they share infrastructure. LangGraph, for example, documents thread-scoped checkpoints for short-term state and stores with custom namespaces for memory shared across threads (official LangGraph persistence guide, verified 2026-07-24). That is one implementation, not a requirement; the important boundary is which records may cross sessions, users, or agents.
Privacy, retention, and deletion
Memory increases privacy risk because it turns temporary conversation data into a reusable profile. Before writing a record, decide why it is needed, who may read it, how long it should exist, and how the subject can correct or delete it. Sensitive data should be opt-in or prohibited unless the use case clearly requires it.
Build deletion as a data operation, not an instruction to the model. A deletion request may need to remove the primary record, vector representation, derived summary, caches, queued background jobs, and replicas, while retaining only a minimal audit marker where policy requires one. Maintain an index from the subject or source record to every derived memory so erasure is testable.
Provider-managed conversation state is not the same as application-owned memory. As one current example, OpenAI’s official data-controls table says the Responses API has a 30-day application-state retention period by default, subject to endpoint and data-control conditions (official OpenAI data controls, verified 2026-07-24). Treat that as a provider-specific fact to recheck, not as a general retention rule. Your own database, logs, backups, and connected tools each need separate policies.
Managed agent products can expose different controls. Amazon Bedrock AgentCore, for example, allows raw short-term events to be retained for up to 365 days (official AgentCore memory creation guide, verified 2026-07-24). Its deletion documentation also notes that deleting a raw event does not delete structured long-term records derived from it; those records require a separate deletion operation (official AgentCore event deletion guide, verified 2026-07-24). These are product-specific controls, not general defaults, and application logs, traces, and exported copies still need their own deletion paths.
Encrypt memory, restrict service access, redact secrets from traces, and log reads as well as writes. Test deletion with a synthetic identity and confirm that retrieval cannot recover the removed fact.
When memory makes an agent worse
More memory can reduce performance. Stale preferences can override a new request. Similar episodes can anchor the agent to the wrong solution. Long summaries can crowd out current evidence. Poisoned or unauthorized records can steer tool use, and contradictory facts can make outputs inconsistent.
Use a write threshold and a retrieval threshold. A record should be stored only when its expected future value exceeds its privacy and maintenance cost. It should be retrieved only when it is relevant, authorized, sufficiently current, and supported by provenance. Give current explicit instructions priority over remembered preferences, and let users inspect and correct durable records.
Evaluate with ablation tests: run representative tasks with no memory, working memory only, and each long-term class enabled. Measure task success, wrong personalization, retrieval precision, latency, token use, and deletion correctness. Classify failures by write, search, reranking, context assembly, or execution.
Frequently asked questions
Does an AI model remember previous conversations by itself?
Usually not in the sense meant here. Continuity generally comes from the application or provider storing conversation state and supplying selected information to the model again. Check the exact product’s storage and retention controls rather than inferring them from the model name.
Is a vector database required for agent memory?
No. Structured working state may fit a key-value or relational store, while exact facts often benefit from metadata or relational queries. A vector index is useful when meaning-based similarity improves retrieval, and hybrid search can combine semantic recall with exact-term matching.
What should an agent save to long-term memory?
Save information with a clear future use, reliable source, defined scope, and deletion policy. Good candidates include explicitly confirmed preferences, validated procedures, and meaningful outcomes. Avoid automatically promoting every message or model inference into a durable fact.
How long should agent memory be retained?
There is no universal duration. Set retention by purpose, sensitivity, legal or contractual requirements, correction needs, and the cost of stale data. Use the LLM data retention checklist to map provider state, application databases, logs, caches, and backups separately.
How do you know whether memory improves the agent?
Compare the same representative tasks with memory disabled and enabled. Measure task success alongside wrong personalization, retrieval precision, latency, token use, and deletion correctness. Traces help diagnose failures, but they are not the evaluation itself; see the agent observability guide.
A practical starting point
The safest starting point is deliberately small: structured working state, a few explicitly confirmed semantic facts, and episodic records only for validated successes or meaningful failures. Add memory when tests show that it improves a concrete task. The goal is not for an agent to remember everything; it is for the system to recall the right evidence without carrying forward the wrong assumptions.