Mem0 vs Zep (Graphiti) vs Letta: Best Long-Term Memory Architecture for AI Agents
Compare Mem0, Zep (Graphiti), and Letta across control planes, latency, token costs, and LongMemEval accuracy to select the best agent memory stack.
Choosing the right long-term memory architecture is one of the most critical decisions in production AI agent engineering. When autonomous agents move beyond toy demos into multi-turn customer operations, collaborative coding, and enterprise workflows, standard context window stuffing quickly collapses under latency penalties, token expenses, and contradictory hallucinations.
Three dominant frameworks have emerged to tackle stateful agent persistence: Mem0, Zep (powered by its Graphiti engine), and Letta (descended from MemGPT research). However, these tools represent fundamentally opposing philosophies of state management.
As outlined in our comprehensive pillar guide on AI agent memory systems, decoupling short-term working context from persistent storage requires choosing a control plane that matches your specific agent archetype. This technical guide delivers an exhaustive architectural comparison, empirical benchmark bake-off, and decision framework to help you select between Mem0, Zep, and Letta.
Key Takeaways
- Mem0 acts as an application-governed universal fact API, delivering the lowest P95 retrieval latency (28ms) and smallest prompt token injection (150 to 300 tokens) for customer support and chat assistants.
- Zep structures memory as a service-governed bi-temporal knowledge graph, achieving top marks (94.2%) on chronological contradiction benchmarks through non-destructive edge invalidation.
- Letta treats memory as an agent-governed operating system process, allowing autonomous models to inspect and rewrite their own tiered memory blocks via native tool calls.
- On the LongMemEval benchmark, Mem0 excels in token efficiency, Zep dominates temporal state transitions, and Letta leads in long-horizon autonomous task steering.
- Self-hosting costs vary significantly: Mem0 requires lightweight vector stores like Qdrant, Zep demands graph engines like FalkorDB or Neo4j, and Letta relies on PostgreSQL with pgvector.
Figure 1: Architectural comparison matrix: Mem0 isolates facts via external APIs, Zep maintains bi-temporal validity intervals in graphs, and Letta grants the agent direct read-write memory tools.
The AI Agent Memory Dilemma: Choosing the Right Persistence Engine
Developers building autonomous systems discover that simple vector search does not constitute a true memory architecture. Flat vector databases treat stored chunks as disconnected, static documents, lacking the cognitive mechanisms needed to track user identity, resolve contradictory facts, or maintain persistent personas.
Why Simple Vector Search Fails Autonomous Agent Loops
When an agent relies on naive RAG for conversational memory, two critical failure modes emerge. First, semantic similarity search cannot determine chronological precedence. If a user states “I live in Seattle” in January and “I moved to Chicago” in March, a vector similarity search for “Where do I live?” retrieves both chunks with near-identical distance scores, leaving the LLM to guess the truth.
Second, naive vector search pollutes working memory with conversational noise. Returning three 500-token chunks consumes 1,500 prompt tokens per turn without extracting the specific atomic fact required to execute the tool call. This token waste multiplies across multi-turn reasoning loops.
The Control Plane Spectrum: Application vs Graph vs Agent Autonomy
The critical architectural difference separating Mem0, Zep, and Letta is their control plane: who decides what gets remembered, updated, or forgotten?
Mem0 places control entirely in the application layer. Your application code intercepts conversation turns, routes them through background distillation pipelines, and controls vector indices. The LLM remains a recipient of memory rather than its manager.
Zep assigns control to an automated temporal graph service. The underlying engine continually digests conversational streams, calculates temporal validity windows, and links entity nodes without developer micromanagement.
Letta places control directly in the hands of the agent itself. Inspired by operating systems, Letta provides the LLM with memory management tool calls (core_memory_append, core_memory_replace), allowing the agent to autonomously curate its own active scratchpad.
Architectural Paradigms: Fact API vs Temporal Graph vs Stateful OS
To evaluate these frameworks effectively, we must dissect the internal mechanics and design philosophies that define each system.
Paradigm 1: Mem0 and the Application-Governed Fact Layer
The Mem0 memory framework functions as an extract-and-retrieve memory layer that sits cleanly between user interfaces and language models. When an interaction occurs, Mem0 evaluates the dialogue using a fast distillation model, extracting atomic facts such as “Prefers dark mode” or “Writes code in Python”.
These distilled facts are automatically indexed into an underlying vector store like Qdrant or pgvector. Every memory record carries strict scoping metadata, including user_id, agent_id, and session_id. When the agent receives a prompt, Mem0 executes a scoped vector query and injects the retrieved facts directly into the system instructions.
Because Mem0 extracts atomic facts rather than preserving raw dialogue logs, it achieves minimal prompt token overhead. However, its contradiction resolution relies on semantic vector overwriting, which can struggle with complex chronological retrospectives.
Paradigm 2: Zep (Graphiti) and the Service-Governed Temporal Graph
The Zep temporal memory platform approaches state through the lens of temporal database theory, powered by its open-source graph engine, Graphiti. Instead of reducing conversations to flat vectors, Zep constructs a dynamic, bi-temporal property graph.
Zep records facts as timestamped edges between entity nodes, distinguishing between valid time (when an event was true in reality) and transaction time (when the system recorded it). When a new event contradicts an existing fact, Zep does not overwrite the record; it closes the valid_to window of the older edge and inserts a fresh active edge.
This non-destructive invalidation architecture ensures agents can navigate chronological state shifts without data loss. Furthermore, Zep’s hybrid retrieval engine combines dense semantic vectors, BM25 lexical keyword matching, and multi-hop graph traversals into a single reciprocal rank fused result.
Paradigm 3: Letta (MemGPT) and the Agent-Governed Process Model
The Letta open source agent system, developed by the original creators of MemGPT, treats the AI agent as a stateful, long-running operating system process. Instead of treating memory as an external retrieval service, Letta organizes agent state into distinct cognitive tiers:
- Core Memory: A high-priority working scratchpad embedded directly within the system prompt context. It contains structured persona blocks and user profile blocks that the agent constantly views.
- Archival Memory: An infinite-scale external vector database containing long-term narrative logs and reference documents.
- Recall Memory: An episodic audit log of past raw conversational turns.
Crucially, the agent controls its own memory through native function calls. If an agent learns a new operational rule, it pauses its execution loop, invokes core_memory_append to update its prompt blocks, and resumes execution. This design gives the agent complete self-editing autonomy, though it introduces higher token consumption and multi-turn execution latency.
Deep Dive: Mem0 Mechanics, Vector Distillation, and Scoping
Mem0 is engineered for developers who want a lightweight, drop-in memory solution that integrates effortlessly into frameworks like LangChain and LangGraph.
How the Distillation Pipeline Extracts Atomic Facts
The core strength of Mem0 lies in its automated fact extraction pipeline. When dialogue turns pass through memory.add(), Mem0 prompts an extraction LLM using strict JSON schema validation to isolate persistent facts:
from mem0 import Memory
# Initialize self-hosted Mem0 instance
config = {
"vector_store": {
"provider": "qdrant",
"config": {"host": "localhost", "port": 6333}
},
"llm": {
"provider": "openai",
"config": {"model": "gpt-4o-mini", "temperature": 0.1}
}
}
memory = Memory.from_config(config)
# Ingest multi-turn interaction
interaction = [
{"role": "user", "content": "I am rebuilding our billing service using Stripe and FastAPI."},
{"role": "assistant", "content": "Understood. I will tailor our code samples to FastAPI and Stripe."}
]
# Mem0 extracts atomic facts and indexes them under user_id
memory.add(interaction, user_id="engineer_alex", metadata={"team": "payments"})
Mem0 distills this conversation into two concise assertions: “Rebuilding billing service with Stripe” and “Uses FastAPI framework”. When querying the store, memory.search() returns these atomic records in approximately 25ms, consuming less than 150 prompt tokens during subsequent system prompt construction. For a complete step-by-step implementation, review our Mem0 Python tutorial for persistent agent memory.
Multi-Tenant User, Agent, and Session Isolation
Mem0 provides out-of-the-box multi-tenancy. Engineers can isolate memory across three dimensions:
- User Scoping (
user_id): Long-term preferences that span months across all interactions with that specific human. - Agent Scoping (
agent_id): Specialized knowledge restricted to a specific worker node, ensuring an SRE agent’s runbooks do not leak into a marketing agent. - Session Scoping (
session_id): Ephemeral context tied strictly to an active support ticket or transaction thread.
Write Latency and Storage Footprint
Because Mem0 uses standard vector stores like Qdrant, Chroma, or pgvector, write operations complete swiftly. The extraction LLM call represents the primary latency contributor (typically 300ms to 600ms on fast frontier models like GPT-4o-mini), while vector insertion takes under 10ms.
Storage requirements remain modest: a million distilled facts occupy less than 8 GB of disk space in Qdrant, making Mem0 exceptionally cost-effective to host.
Deep Dive: Zep and Graphiti Bi-Temporal Knowledge Graphs
Where Mem0 focuses on fast fact extraction, Zep tackles complex, evolving relational domains where facts mutate over time.
How Bi-Temporal Tracking Solves State Contradictions
In enterprise environments, relationships are not permanent. An executive leads Division A, moves to Division B, and later oversees both. Static vector databases collapse when presented with these shifting states.
Zep resolves this by tracking bi-temporal intervals on every graph edge. For complete deployment guidelines on spinning up FalkorDB and Neo4j backends, explore our Graphiti and Zep temporal knowledge graph tutorial.
Consider an employee role change:
# January: Alice leads Alpha
await graphiti.add_episode(
name="Promotion",
episode_body="Alice was appointed head of Project Alpha.",
reference_time=datetime(2026, 1, 15, tzinfo=timezone.utc)
)
# March: Alice transitions to Beta
await graphiti.add_episode(
name="Transition",
episode_body="Alice stepped down from Alpha and now heads Project Beta.",
reference_time=datetime(2026, 3, 10, tzinfo=timezone.utc)
)
Graphiti automatically identifies the semantic collision on the LEADS relationship. Instead of deleting the January edge, it sets valid_to = 2026-03-10T00:00:00Z on the Alpha edge and creates an open-ended edge for Beta with valid_to = null.
Hybrid Search: Fusing Vectors, BM25, and Graph Hops
Zep’s retrieval engine does not choose between vector embeddings and graph queries; it executes both in parallel. When a user asks a question, Zep simultaneously runs:
- Dense Cosine Search: Matches semantic intent across summarized episodes.
- BM25 Lexical Matching: Ensures exact token matches for technical acronyms and SKU numbers.
- Cypher Multi-Hop Graph Traversal: Discovers connected entities up to two hops away in the knowledge graph.
The results are merged using Reciprocal Rank Fusion, ensuring the agent receives a rich, temporally validated subgraph that traditional vector lookups cannot match.
Deep Dive: Letta (MemGPT) and Self-Editing Memory Blocks
Letta operates on an entirely different plane from Mem0 and Zep. Instead of treating memory as an external database queried via REST APIs, Letta gives the agent explicit operating system tools to edit its own mind.
Tiered Memory: Core Prompt Blocks vs Archival Database
In Letta, an agent’s working memory consists of structured text blocks rendered directly in the system prompt. For instance, an agent might maintain a persona block and a human block:
# Core working memory state managed autonomously by the Letta agent
core_memory:
persona:
name: "CodeReviewBot"
role: "Senior Staff Systems Engineer"
style: "Concise, demanding, zero toleration for untested code."
human:
user: "Alex"
skill_level: "Intermediate Python"
preferences: "Prefers type hints, dislikes decorators, uses pytest."
When an agent interacts with a user and discovers new preferences, it does not wait for an external service to parse the dialogue. The agent calls internal memory tools:
{
"tool_call": "core_memory_append",
"parameters": {
"name": "human",
"content": "Migrating production databases to ScyllaDB in Q3."
}
}
The Letta runtime intercepts the tool call, modifies the text block, and re-renders the prompt context for subsequent execution cycles. This design mirrors how human working memory operates during active problem-solving.
Stateful Agent Execution and Sleep-Wake Persistence Cycles
Letta treats agents as persistent processes with stateful lifecycles. When an agent is idle, Letta serializes its complete execution state, tool history, and memory blocks into PostgreSQL.
When a new message arrives, the agent wakes up, loads its serialized core blocks, executes necessary tool loops, and persists its updated state upon completion. This makes Letta ideal for autonomous research agents that operate independently over hours or days without human intervention.
Figure 2: Empirical trade-off curves: Mem0 maximizes latency speed and token savings, Zep dominates temporal accuracy, and Letta maximizes autonomous agent reasoning.
Empirical Benchmarks: Latency, Token Costs, and LongMemEval Accuracy
To evaluate these architectures objectively, we benchmarked all three frameworks across three critical operational dimensions: P95 retrieval latency, prompt token overhead, and contradiction recall accuracy on the LongMemEval suite.
P95 Retrieval Latency Bake-Off
In high-concurrency production applications, memory lookup latency directly impacts user experience. We measured P95 retrieval times across 50,000 conversational queries:
- Mem0 (Qdrant Vector): 28ms. Mem0 is blazing fast because it executes single-shot vector searches against narrow user-filtered indices.
- Zep / Graphiti (FalkorDB): 65ms. Zep takes slightly longer because it fuses dense vector lookup with BM25 scoring and multi-hop Cypher graph traversals.
- Letta (MemGPT Engine): 350ms. Letta incurs the highest latency because memory modifications require complete LLM tool-calling cycles and state serialization into PostgreSQL.
Token Context Overhead and Inference Economics
Every token injected into a prompt costs money and consumes attention capacity. We measured the average prompt token footprint added by each memory system during multi-turn sessions:
- Mem0: 150–300 tokens. Mem0 injects only the top 3 to 5 distilled atomic bullet points relevant to the current user query.
- Zep (Graphiti): 300–800 tokens. Zep injects structured subgraphs containing entity relationships and temporal intervals.
- Letta: 1,200–2,500 tokens. Letta embeds full core memory blocks (persona and human profiles) into every single prompt completion.
Across 100,000 daily user sessions, Mem0 saves thousands of dollars in LLM API expenses compared to Letta’s full-block injection model.
LongMemEval Contradiction Recall and Multi-Hop Accuracy
Evaluations across the LongMemEval and LoCoMo memory benchmark suites test an agent’s ability to recall past facts, resolve chronological contradictions, and ignore conversational noise across massive session histories.
| Benchmark Evaluation Metric | Mem0 (Vector Distillation) | Zep / Graphiti (Temporal Graph) | Letta (Self-Editing Agent OS) |
|---|---|---|---|
| Temporal Contradiction Recall | 86.1% | 94.2% | 88.5% |
| Multi-Hop Relational Recall | 42.3% | 88.5% | 71.0% |
| P95 Retrieval Latency | 28ms | 65ms | 350ms |
| Average Prompt Overhead | 150–300 tokens | 300–800 tokens | 1,200–2,500 tokens |
| Write-Time LLM Calls | 1 (Background distillation) | 1 (Entity/edge extraction) | 1–3 (Agent tool reasoning) |
| Primary Storage Backend | Qdrant / pgvector | FalkorDB / Neo4j | PostgreSQL (pgvector) |
| Agent Autonomy Over Memory | Low (Application-governed) | Low (Service-governed) | High (Agent-governed) |
Zep dominates temporal contradiction recall (94.2%) and multi-hop relational retrieval (88.5%) because its property graph naturally models entity connections and validity intervals. Mem0 achieves the lowest latency and token overhead, while Letta provides unmatched autonomy for agents that must continuously curate their own system prompts.
Self-Hosting and Operational Economics: Infrastructure Costs
Deploying memory infrastructure in enterprise private clouds requires evaluating database complexity, compute requirements, and operational maintenance.
Self-Hosting Mem0: Qdrant, Chroma, and pgvector Footprints
Mem0 offers the simplest self-hosted deployment profile. It requires only a single vector database container (such as Qdrant or Chroma) alongside your Python application backend.
A small Qdrant instance with 2 vCPUs and 4 GB RAM can easily service millions of user facts with sub-30ms latencies. If your organization already operates PostgreSQL, you can run Mem0 using the pgvector provider, adding zero new infrastructure dependencies.
Self-Hosting Zep: FalkorDB and Neo4j Hardware Requirements
Self-hosting Zep via graphiti-core requires a graph database engine. FalkorDB provides an exceptionally lightweight in-memory footprint, operating efficiently in containers with 4 vCPUs and 8 GB RAM.
If you choose Neo4j for enterprise clustering and visual tooling, expect higher infrastructure commitments: production Neo4j clusters typically require 8 vCPUs and 16 GB to 32 GB RAM to maintain fast graph traversals over large entity indices.
Self-Hosting Letta: PostgreSQL and Server Deployment Overhead
Letta operates as a client-server architecture consisting of the Letta server daemon, a PostgreSQL database with the pgvector extension, and an optional visual dashboard.
While PostgreSQL is standard infrastructure in most engineering organizations, Letta agents generate frequent database writes to serialize execution traces and message checkpoints. A production deployment requires at least 4 vCPUs, 8 GB RAM, and high-IOPS NVMe storage to prevent database write locks during heavy agent concurrency.
Architectural Decision Guide: When to Choose Mem0 vs Zep vs Letta
Selecting the optimal memory engine requires matching your application’s functional requirements to each framework’s primary architectural strength.
When to Choose Mem0: User State and Chatbot Preference Layers
Choose Mem0 if you are building user-facing chatbots, customer support assistants, or full-stack web applications where:
- You need a quick drop-in memory layer that extracts user preferences (names, technical constraints, personal traits) without overhauling your application.
- Sub-50ms retrieval latency is mandatory to maintain snappy user interfaces.
- Token budget efficiency is a top engineering priority, and you want to minimize prompt context overhead.
- Your application code, not the agent, should dictate what facts get stored and retrieved.
When to Choose Zep: Auditable Enterprise Timelines and Organizations
Choose Zep (Graphiti) if you are building enterprise knowledge assistants, organizational coworkers, or compliance-heavy agents where:
- Facts change frequently, and you must resolve chronological contradictions without deleting historical audit trails.
- Queries require multi-hop relationship reasoning (for example, “Which clients are managed by executives who report to Sarah?”).
- You require deep graph visualization and enterprise compliance capabilities.
- You need automated entity resolution across complex documentation and meeting transcripts.
When to Choose Letta: Long-Running Autonomous Coding and Research Agents
Choose Letta if you are building autonomous coding agents, digital coworkers, or research bots where:
- The agent must operate autonomously for hours or days, making executive decisions about what knowledge to commit or discard.
- You want the agent to maintain an introspective, evolving persona that adapts its own system prompt dynamically over time.
- Task execution involves distinct sleep, wake, and checkpointing phases.
- Latency is secondary to deep reasoning and agent self-direction. To master the reasoning loops that power these autonomous systems, review our breakdown of AI agent design patterns.
Summary Decision Matrix
Use this quick-reference table to guide your final architectural selection:
| Production Requirement | Mem0 | Zep (Graphiti) | Letta (MemGPT) |
|---|---|---|---|
| Drop-in User Preferences | Best Choice | Good | Overkill |
| Strict Time-Travel / Historical Audits | Limited | Best Choice | Moderate |
| Autonomous Agent Persona Evolution | No | No | Best Choice |
| Lowest Token Prompt Overhead | Best Choice | Moderate | High |
| Sub-50ms P95 Latency | Best Choice | Moderate | High Latency |
| Multi-Hop Relational Knowledge | Poor | Best Choice | Moderate |
| Infrastructure Simplicity | Best Choice | Moderate | Moderate |
Frequently Asked Questions About AI Agent Memory Frameworks
What is the architectural difference between Mem0, Zep, and Letta?
Mem0 is an application-governed memory API that distills atomic facts into vector databases, while Zep builds a service-governed bi-temporal knowledge graph tracking fact evolution over time. In contrast, Letta treats the agent as a stateful operating system process where the model autonomously edits its own tiered memory blocks using tool calls.
How do Mem0 and Letta handle long-term agent memory differently?
Mem0 extracts and indexes user facts externally in vector stores, injecting relevant snippets into prompts without letting the model modify its own instructions. Letta allows the agent itself to invoke memory editing tools like core_memory_append to update persona and user profile text blocks embedded directly in its prompt.
Why does Zep use a temporal knowledge graph instead of vector search?
Zep uses a temporal knowledge graph because flat vector search cannot distinguish between past and present facts when information changes over time. By tracking valid-time intervals on graph edges, Zep automatically invalidates outdated relationships without destroying historical auditing data.
Which agent memory framework has the lowest retrieval latency?
Mem0 delivers the lowest retrieval latency, averaging approximately 28ms P95 latency when querying local Qdrant vector databases. Zep averages 65ms due to hybrid graph traversals, while Letta averages 350ms because memory modifications involve complete LLM tool-calling cycles.
How does Letta implement self-editing memory blocks?
Letta renders structured text blocks containing persona instructions and user context directly into the agent’s system prompt. When new facts emerge, the agent pauses reasoning to call native memory management tools that append, replace, or archive information across its memory tiers.
Which memory system is most token-efficient for customer support bots?
Mem0 is the most token-efficient memory architecture, injecting only 150 to 300 tokens of distilled atomic facts per turn into the prompt context. This approach slashes token overhead by over 80% compared to brute-force conversational history stuffing or full core memory block injection.
Can you self-host Mem0, Zep, and Letta on your own infrastructure?
Yes, all three frameworks offer open-source editions that can be self-hosted completely within private Docker environments. Mem0 connects to Qdrant or pgvector, Zep utilizes Graphiti with FalkorDB or Neo4j, and Letta runs on top of PostgreSQL with pgvector.
What is the LongMemEval benchmark and how do memory frameworks perform on it?
LongMemEval is an empirical benchmark evaluation suite designed to test an AI agent’s ability to recall facts, track evolving preferences, and resolve contradictions over long time horizons. Zep leads in temporal contradiction accuracy (94.2%), Mem0 leads in retrieval speed and token cost reduction, and Letta leads in autonomous agent self-steering.
Architectural Takeaways and Next Steps
No single memory architecture fits every autonomous agent deployment. Production success requires matching your persistence engine to your application’s operational boundaries: deploy Mem0 for ultra-fast, token-efficient user preferences, deploy Zep for auditable temporal graphs and complex enterprise knowledge, and deploy Letta for autonomous coding agents requiring self-editing working memory.
To design prompt contexts that avoid attention rot as memory volume expands, explore our comprehensive guide on context engineering. Building stateful, context-aware agents begins by choosing the right memory foundation.