The trade-off, in the open.
Give an agent the whole conversation and it answers well — but it re-reads, and re-pays for, everything on every question, and eventually the transcript no longer fits. A memory layer reads a small fraction of that. The real question is how much answer quality you give up to get there. Here is Memomee's answer, measured against a reader that sees everything and against the open-source alternatives — the same test for every system, and every number from a real measurement.
Memomee is a memory governance and retrieval control plane for long-running agents — the kind that write code, run research, or drive workflows across hours and sessions. If yours run long enough to branch, get interrupted, or drift, this page is the evidence. If you only need short-session chat history, you don't need it.
The decision, in one screen.
Four ways to give an agent memory, on one fair test — same benchmarks, same answer model, same grader, same questions; only the memory strategy changes. If you're evaluating alternatives, start here; the evidence follows.
What we're comparing · and why these four
Paste the whole conversation into the model on every question.
Best for: Short chats that still fit the window.
Embed raw turns; retrieve the top-k by similarity.
Best for: Stateless lookup over documents.
An open-source memory system with its own extract-and-store pipeline.
Best for: A drop-in OSS chat-memory layer.
Typed derive → govern → retrieve → assemble — a control plane, not a store.
Best for: Long-running agents that branch, drift, or need governance.
Two bookends — the full-context ceiling and the RAG floor — plus the leading open-source memory system, so every comparison is against a meaningful reference, not a strawman. Weighing all six memory approaches, not just these four? Choosing memory for AI agents walks the full landscape.
Which should you reach for?
| If you're building… | Reach for |
|---|---|
| Conversations that fit a context window | Full context |
| One-shot lookup over documents | RAG |
| Agents running for hours or across sessions | Memomee |
| Governance — approval, audit, forgetting, isolation | Memomee |
| Lowest token cost at long context | Memomee |
Memomee at a glance
Four findings
- 01Near-parity recall
Finds the evidence an answer needs about as often as a reader that sees everything — recall@k 0.97–1.00.
- 025–11× fewer tokens
Reads a compact ~0.2–2.4k-token context, not the whole (and growing) transcript.
- 03Governance nobody else publishes
Approval, audit, supersession, forgetting, zero cross-user leaks — re-verified on every release.
- 04Honest trade-offs
A full-context reader still scores higher on hard tasks; we show the gap rather than hide it.
The rest of this page is the evidence behind those four — head-to-head, the governance, and where each system wins; the twelve-benchmark sweep and methodology live on /benchmarks. Jump to the recommendation for the call.
Three ideas, then the numbers explain themselves.
The benchmarks below come from the memory-research literature. Three plain-language ideas make every table on this page readable.
The accuracy ceiling: hand the model the entire conversation on every question. It answers well — but it re-reads (and re-pays for) everything, and it breaks the moment the conversation outgrows the context window. It's the bar to measure against, not something you'd ship.
Did retrieval find the right evidence? The share of questions where the fact an answer needs actually reached the model. When recall is high but the answer score is lower, the gap is in writing the answer — not in the memory.
Every system here is answered and graded by the same AI, over the same questions, across three independent runs. Only the memory strategy changes — so the comparison is fair. That is exactly what public leaderboards usually are not.
Most of the answer, a fraction of the tokens.
What matters in production is answer quality set against how much text you read to get it. The full-context reader scores highest — top of the chart — because it reads everything. Memomee lands just below it while reading a small fraction of the tokens: most of the answer, at a fraction of the cost.
What the chart shows: Memomee reaches most of the full-context reader's answer quality while reading 5–11× fewer tokens per question on these two tests. Reading few tokens isn't the whole story on its own — the RAG baseline on LoCoMo reads almost nothing (197 tokens) yet trails on answer quality (though that gap is within run-to-run noise). What you want is the top-left corner: high answer quality and low token cost.
- LongMemEval — Memomee scores 0.600 reading 1,203 tokens: about 83% of the full-context reader's answer quality for roughly a fifth of the tokens, +0.067 over both mem0 and the RAG baseline (which tie at 0.533) — a lead it holds across every seed here.
- LoCoMo — 0.472 reading 2,410 tokens: about 81% of the reader's answer quality for roughly a tenth of the tokens, edging the RAG baseline (+0.012, a statistical tie) and clearing mem0 by +0.13.
Where memomee lands
answer score · same test, 4 systemsmemomee +0.067 over both open-source systems; −0.120 vs the full-context reader.
memomee +0.13 over mem0, +0.012 over RAG (a tie); −0.111 vs the full-context reader.
Bars scale to a 0.80 score ceiling · full-context reads the whole transcript · mem0 shown lighter to distinguish it from the RAG baseline.
RAG baseline — a deliberately plain vector-retrieval RAG, the "just embeddings" floor. It embeds every raw conversation turn with the same Voyage embedder Memomee's own vector search uses, then for each question retrieves the top 5 turns by cosine similarity and hands them to the same answer model. No extraction, no summarization, no query planning, no graph, no fact store — chunk-and-embed over turns. Because it shares Memomee's embedder and the held-constant answer-writer and grader, the gap to it isolates what Memomee's derive-and-plan pipeline adds on top of plain semantic search.
mem0 (open source) — the mem0 memory system on its own native pipeline (gpt-4o-mini extraction + its own embedder). A full system-vs-system comparison; its memories aren't turn-tagged, so recall@k isn't computed for it.
LongMemEval
3 runs · 50 questions each| System | Answer score | recall@k | Read tokens |
|---|---|---|---|
| Memomee | 0.600 ± 0.028 | 0.97 | 1,203 |
| RAG baseline | 0.533 ± 0.034 | 0.92 | 1,114 |
| mem0 (open source) | 0.533 ± 0.019 | — | 412 |
| Full-context · reads everything | 0.720 ± 0.043 | 1.00 | 6,150 |
Memomee vs. the RAG baseline +0.067 · vs. the full-context reader −0.120 · reads ≈ 5.1× fewer tokens.
LoCoMo
3 runs · 50 questions each| System | Answer score | recall@k | Read tokens |
|---|---|---|---|
| Memomee | 0.472 ± 0.050 | 1.00 | 2,410 |
| RAG baseline | 0.459 ± 0.040 | 0.78 | 197 |
| mem0 (open source) | 0.341 ± 0.029 | — | 578 |
| Full-context · reads everything | 0.582 ± 0.011 | 1.00 | 26,529 |
Memomee vs. the RAG baseline +0.012 · vs. the full-context reader −0.111 · reads ≈ 11× fewer tokens.
What the numbers do and don't say.
Memomee beats the RAG baseline on average (+0.067 on LongMemEval, +0.012 on LoCoMo) and clears mem0 by a clear margin (+0.07 and +0.13). But the RAG lead sits inside run-to-run noise — the spread is about the size of the lead, and on LoCoMo the two are effectively tied. Memomee edges RAG; it doesn't dominate it.
A reader handed the entire transcript scores higher on both head-to-head tests — Memomee trails by about 0.11–0.12 — and on most of the wider sweep. We show that gap rather than hide it. Most of it comes down to how memory is written — Memomee distills each conversation into typed facts with an LLM before the reader ever sees them, and the quality of that extraction is what narrows the gap.
Whatever the answer score, retrieval finds the right evidence about as reliably as the full-context reader — recall@k of 0.97–1.00 — while reading a compact assembled context: 5–11× fewer tokens on the head-to-head tests. That token discount is the point.
The same held-constant setup for all four systems: DeepSeek writes and grades every answer, over three runs of 50 questions each. Memomee and the RAG baseline share one semantic-search embedder; mem0 runs its own native pipeline (gpt-4o-mini extraction) — a system-vs-system comparison, not a component swap. Compare differences within a test, not raw scores across different tests.
Everyone claims to win LoCoMo.
Almost none of those numbers can be compared. Swap the AI that writes the answers, or the one that grades them, and the score moves more than the memory system ever does. So these are vendor claims, each tagged with the setup it came from — not results we can honestly place next to ours.
| System | Published claim | Benchmark / setup | Why it isn't comparable |
|---|---|---|---|
| Mem0 | 66% · 68% (graph) | LoCoMo (categories 1–4) | gpt-4o-mini answer model, gpt-4o grader; no standard LoCoMo grader |
| Mem0 | 92.5% | LoCoMo · gpt-5 | different answer model; actively contested (Mem0 ↔ Zep) |
| Zep | 63.8% · 71.2% | LongMemEval | gpt-4o-mini / gpt-4o answer model — the answer model differs |
| Letta / MemGPT | 74% | LoCoMo | non-standard grader; session-vs-turn ingestion |
| Supermemory | 81.6% · 84.6% | LongMemEval · gpt-4o / gpt-5 | different answer model (81.6% gpt-4o, 84.6% gpt-5); 84.6% disputed |
| Memobase | 75.8% | LoCoMo | unstandardized grader; disputed |
| LangMem | 51.2% | LoCoMo | third-party run, different setup |
This is why we only compare inside our own setup — the same answer-writer, grader, and search for every system. We never line our score up against a number in this table: a different answer model, a home-grown LoCoMo grader (there is no standard one), or session-by-session vs. turn-by-turn ingestion each move the result more than the memory layer does. Several of these figures are disputed by other vendors, too.
Twelve benchmarks, not two.
The two head-to-heads above aren't the whole story. Across the wider sweep — six long-conversation stress tests, three multi-hop question sets, conversational memory, and a million-token tier — retrieval keeps finding the right evidence (recall@k near 1.0), Memomee edges the full-context reader on a few saturated venues and trails on most, always at a compact ~0.2–2.4k-token read-context. The full tables, the MemoryAgentBench MCC placement, and the methodology live on the benchmarks page — the research paper to this page's decision brief.
From a firehose of events to a small, governed context.
The numbers above come out of one pipeline. It turns everything an agent says and does into compact, typed, governed memory — then reads back only the few facts a question needs. Two things fall out of that design: the token discount you saw in the charts, and governance the store-and-search alternatives can't offer.
Event ledger
Every message, tool call, and result is appended to an immutable log — the raw truth, never edited in place. Corrections arrive as new events, so history stays reconstructable.
Derive
An LLM distills each event into typed memories — facts, decisions, resources, procedures, preferences — instead of keeping raw turns. This is what makes the context small: you read a few distilled facts, not the whole conversation.
Govern
Behaviour-changing memories — procedures, preferences, learned lessons — are born pending and cannot shape an answer until they're explicitly approved and audited. Facts stay freely retrievable; the rules that change how an agent acts are gated. This is the control plane.
Retrieve
For each query a hybrid fan-out searches every substrate at once — lexical, vector, graph, and the typed stores — to gather high-recall candidates. Recall first; precision comes next.
Assemble
Scope, freshness, supersession, and contradictions are resolved before ranking; a reranker and a conflict-collapse step produce one compact, typed, provenance-backed context block. This is where the token discount is made — a tight context, not the transcript.
Answer
The agent answers from that compact block, and a retrieval trace records exactly what was read and what was filtered — so every answer is auditable back to its evidence.
Steps 02 and 05 do the work: the agent reads a compact, roughly constant ~0.2–2.4k-token block of distilled facts instead of a transcript that grows without bound. That's the 5–11× token discount — and why memomee still answers when the conversation no longer fits a context window at all.
Step 03 is the part nobody else publishes: a memory that would change how the agent behaves is captured but inert until it's approved, every answer carries a provenance trace, superseded facts leave a trail, and one user's memory can never surface in another's.
Want the real diagram, stage by stage? How it works walks the full pipeline — every box clickable.
The real difference is governance.
Accurate answers at a token discount are table stakes. What actually sets Memomee apart is what happens after retrieval — the rules that decide which memory is allowed to shape an answer, and the checks that prove those rules hold: behavior-changing memory born pending until approved, facts superseded with lineage, abstention on conflict, zero cross-user leaks, governed forgetting. These aren't third-party benchmarks; they're guarantees, re-verified automatically on every release with zero tolerance for a regression. Most memory products don't publish them because they don't have them.
What a control plane does that a store doesn't.
Those guarantees exist because Memomee is a control plane, not a store. Storing and searching memory is the easy part; deciding what should survive, change, be forgotten, or shape the next answer is the part the alternatives skip.
| Memomee | Managed memory APIs | Agent frameworks | Graph engines | DIY stacks | |
|---|---|---|---|---|---|
| Resumable working state | yes | no | partial | no | partial |
| Append-only event ledger | yes | no | no | no | partial |
| Fact supersession & lineage | yes | no | no | partial | no |
| Contradiction warnings | yes | no | no | no | no |
| Retrieval trace + filtered counts | yes | no | no | no | no |
| Workflow continuity, not store | yes | no | partial | no | no |
| Framework-agnostic | yes | yes | no | yes | yes |
Examples per category — managed memory APIs: Cloudflare Agent Memory, Mem0 · agent frameworks: Letta, LangGraph + LangMem · graph engines: Zep, Graphiti · DIY: vector DB + summary loop.
Memomee isn't for everyone.
The honest close to a fair comparison: the right choice depends on what you're building. Here is when each approach wins.
…if your conversations comfortably fit the model's window and re-reading everything isn't a cost problem.
…if you need stateless retrieval over documents — not evolving, per-user, governed memory.
…if your agents run long enough to branch, get interrupted, or contradict themselves — and you need to govern what memory is allowed to shape an answer.
If you picked Memomee, it earns its place on long-running agents by reading a fraction of the tokens at near-parity recall and by governing what memory is allowed to shape an answer — the part the alternatives skip. If you picked full context or RAG, that's the right call for your workload, and this page did its job.
What we don't claim
- The best raw recall. Memomee finds the right evidence about as often as a reader that sees everything — it does not out-retrieve a system that reads the whole transcript.
- A broad win on answer quality over full context. A full-context reader still scores higher on both head-to-head tests (Memomee trails ~0.11–0.12) and on most of the twelve-benchmark sweep; Memomee edges it on only a few (PersonaMem, BEAM, and BEAM's million-token tier — where a full transcript no longer fits) and matches it on ConvoMem, at a large token discount.
- That memory broadly beats full context. Our own stress tests disprove that.
- What we do claim: parity on finding the right evidence, an edge over open-source memory (RAG, mem0) in a fair test — decisive over mem0, within run-to-run noise over RAG — a 5–11× token discount on long conversations, and governance the alternatives don't offer — knowing when to abstain, replacing facts with a trail, and never leaking one user's memory into another's.
Methodology: same-setup head-to-head, three runs (seeds 11 / 12 / 13), 50 questions each, the same answer-writer and grader across all four systems, turn-by-turn ingestion (Memomee and the RAG baseline share one Voyage embedder; mem0 runs its own native gpt-4o-mini stack). Recall here is measured at the evidence-session level (did the right session reach the model); a stricter turn-level measure runs lower and is reported in full on the benchmarks page. The 5–11× token range is the two head-to-heads (≈ 5.1× LongMemEval, ≈ 11× LoCoMo fewer read-tokens than the full-context reader); across the wider benchmark set memomee assembles a compact, roughly constant ~0.2–2.4k-token read-context per question regardless of transcript length. Every "read tokens" figure on this page is the assembled read-context (read_tokens), one metric page-wide. The vs-RAG and vs-full-context deltas are computed from unrounded per-seed means, so they can differ by 0.001 from subtracting the rounded arm scores shown. The full sweep, variance, and evidence tiers: /benchmarks. Full documentation is coming to docs.memomee.ai.