Memory Architecture: Retrieval, Serialization, and Consolidation
Memory Architecture: Retrieval, Serialization, and Consolidation
Memory is useless if the agent cannot find the right information at the right time. This post covers the three retrieval models we see in production systems, the serialization strategies that make retrieval work, and the consolidation processes that keep memory fresh.
Three Retrieval Models
The retrieval strategy you choose determines what kinds of queries your agent can answer efficiently. Each model has different strengths.
Filesystem Retrieval (Anthropic Memory Stores)
Anthropic's Memory Stores treat memory as a filesystem. The agent reads and writes files using ordinary tools - bash, read, write, edit, glob, grep. There are no dedicated memory tools.
The constraints create the retrieval discipline: max 8 stores per session, 100KB per memory, and Anthropic recommends many small files over a few large ones.
This model is best for structured entity lookups. The agent can glob for files matching a user ID, grep for a keyword across notes, or read a specific memory by path. The retrieval surface is bounded by the store count and size limits, which prevents the unbounded search problem.
Anthropic's production patterns: use memory for per-customer or per-entity facts, scope memory so one user's data never leaks into another session, and expire entries when their underlying source of truth changes.
Neural Memory Retrieval (Google Titans)
Google's Titans architecture eliminates explicit retrieval entirely. Instead of the agent searching for relevant memories, the model updates its own internal parameters as data streams in.
The core mechanism is a surprise metric: unexpected information triggers stronger memory encoding. Combined with momentum (considering both momentary and recent context) and adaptive weight decay (forgetting), the model builds an internal representation of the entire context.
Retrieval is implicit - the model has already internalized what it needs. This works well for long-context reasoning where the relevant information is diffuse and hard to anticipate. Titans handles contexts over 2 million tokens.
But there is a trade-off: implicit memory is not inspectable, auditable, or correctable in the way explicit stores are. You cannot verify what the model remembered or delete incorrect information.
Graph Retrieval (MAGMA and GAM)
The ACL 2026 MAGMA paper proposes a middle ground: represent each memory item across orthogonal semantic, temporal, causal, and entity graphs. Retrieval becomes policy-guided traversal over these relational views.
When an agent needs to find relevant memories, it starts from the current context (entities mentioned, time frame, causal chain) and traverses the graph to reach connected memories. The traversal is guided by learned policies that adapt to the query type.
The GAM paper adds a hierarchical structure: an event progression graph for ongoing dialogue and a topic associative network for consolidated knowledge. Retrieval bridges these two layers using a multi-factor strategy that integrates temporal, confidence, and role-based signals.
This model excels at relationship discovery - finding connections between entities that are not obvious from keywords or vector similarity.
Serialization Strategies
How you serialize memory determines what queries you can run efficiently.
For filesystem stores, structured formats (JSON with explicit fields) enable field-level filtering. Anthropic's memory stores accept any text format, but structured formats make retrieval more precise.
For neural memory, serialization is not a separate concern - the model decides what to encode based on the surprise metric. But this also means you cannot control the encoding format or inspect the stored memories.
For graph memory, serialization is a triple store: (entity, relationship, entity) with type, direction, weight, and confidence score. Each dimension (semantic, temporal, causal, entity) gets its own graph with its own schema.
Consolidation
Memories decay. Without consolidation, old information crowds out new information.
In Anthropic's model, consolidation is the agent's responsibility - it can rewrite, summarize, or archive its own memories using file tools. The version history preserves the original for rollback.
In Titans, consolidation is automatic - the surprise metric and adaptive weight decay continuously manage what to retain.
In graph architectures, consolidation is a dedicated process that runs periodically. GAM's approach is most explicit: an episodic buffering phase captures ongoing dialogue in a local graph, and a semantic consolidation phase integrates completed narrative units into the global network only upon semantic shifts. This prevents memory contamination - a phenomenon where established nodes become structurally isolated and forgotten.
All three approaches converge on the same insight: memory systems fail silently. An agent that retrieves no relevant memory will simply proceed without it. The only defense is monitoring retrieval hit rates and consolidating proactively.
Read the full Context Engineering series: orioverse.com/blog/what-is-context-engineering