AI agent memory systems have a human equivalent. Think of these levels like human memory: start at complete amnesia and move toward a mind that learns, reflects, and grows on its own. This page is the plain-English overview. The technical taxonomy, the deep dives on 15 memory systems, and the comparison table live in the repository.
HEAD 2 HEAD - the 6 levels
What it is, how it works, and the catch you pay for it.
Total Amnesia
A calculator
How it works
- Hit
2 + 2 = 4, then hit clear. It answers your question, forgets you immediately, and treats your next message as if you just met.
The catch
- Can't hold a conversation.
Short-Term / Working Memory
A whiteboard in a meeting room
How it works
- Remembers everything said in the current chat, right in front of its eyes.
The catch
- The board fills up quickly. Once full, old notes get erased.
The Executive Summary
Meeting minutes
How it works
- When the whiteboard gets full, an assistant writes a quick 1-paragraph summary so the chat can keep going without running out of room.
The catch
- Small details, exact numbers, and quotes get lost in translation.
The Filing Cabinet
A searchable archive
How it works
- The agent saves past chats into a database. When you mention a past topic, it searches for matching keywords or ideas and pulls the file out.
The catch
- It can pull outdated notes, or find files that sound similar but don't actually answer your question.
The Address Book & Mind Map
A structured profile
How it works
- Instead of just dumping raw notes, it neatly organizes facts: "Alice is Bob's manager," "Prefers Python over JS," "Updated project deadline to Friday."
The catch
- Needs careful upkeep so facts don't clash or get messy.
Deep Learning & Reflection
Sleep and dream consolidation
How it works
- During downtime, the agent reviews past experiences on its own: resolving conflicts, dropping trivia, and extracting lessons such as "Last time I tried X, it failed; do Y instead".
The catch
- Can convince itself of wrong assumptions if it reflects on flawed data.
In Plain English
Amnesia
Pure reflex. One question in, one answer out, zero memory.
Working Memory
Remembers the chat while you are typing, but only up to a fixed limit.
Summaries
Keeps longer conversations alive by constantly turning old turns into bullet points.
Searchable Archive
Can dig into past conversations from weeks ago using a search bar.
Organized Profile
Actually understands relationships, tracking who is who and how facts connect over time.
Self-Evolving Mind
Doesn't just store what you say, it learns habits, fixes its own mistakes, and gets smarter the more you work with it.
| Level | Medium | What persists |
|---|---|---|
| L0 Amnesia | Nothing | Only the current answer. The next message starts from zero. |
| L1 Working | Context window | The current chat, until the window fills up. |
| L2 Summary | Rolling summaries | A compressed version of older turns. |
| L3 Episodic | Vector store or logs | Past conversations that can be searched later. |
| L4 Semantic | Profiles, facts, graphs | What is true about the user and the world. |
| L5 Reflective | Background consolidation | Lessons and self-edits that change future behavior. |
Memory as a village
Part of the memory lab
This is why there are so many memory-related frameworks: every level above L2 is a different engineering trade-off between recall quality, latency, token cost, and operational pain. This lab maps those trade-offs.
The repository holds the technical side: a six-level taxonomy in the README, a comparison
table across 15 systems, and one deep dive per system in mds/.
First lab result: in a value-recall task, a note written in one session
was applied exactly in the next: 5/5 warm runs passed, against 0/5 in each no-memory arm.
The first model could not write code, so the lab now compares two MoE and two dense local
models. All four active models pass the same service gate 5/5 warm against 0/5 cold.
The two dense models also pass the 5-repeat coding transfer 5/5 warm against 0/10 cold
under a 30-call budget; the MoE control cannot replace the fixture function. The
protocol is in lab/README.md.
Second lab result: the same note, in a store with 12 unrelated newer notes. A recency digest lost it, and warm runs failed 0/5. A local store with CPU embeddings found it (5/5), and so did TencentDB Agent Memory's keyword store (5/5). On this task, retrieval mattered more than the size of the memory.
Third lab result: memory held an old service block and a newer correction. In warm runs, file recency with 12 newer notes, pure semantic similarity, and TencentDB L0 all left the stale values in place 0/5. A same-key supersede rule kept the newest block (5/5). Without the newer noise, file recency also passed 5/5. Freshness is a retrieval policy, not a property of the store type.
Fourth lab result: a partial correction has only some of the keys. The exact-key rule did not fire: the older four-key handoff stayed on top and warm runs passed 0/5. A read-time merge that takes the newest value for each overlapping key composed the current four-key block and passed 5/5.
Fifth lab result: Mem0 kept a stale service block and its correction. Search ranked the correction first by 0.0006, and a later model answered with the current values. Mem0 did not supersede the old block, so the margin is fragile. The TencentDB L1-L3 detour is a negative result: its upstream extraction prompt is Chinese, the local model echoed a template, and L1 stayed empty. The English-prompt Mem0 walkthrough is in lab/sessions/01-mem0-extraction.md.
Sixth lab result: the same Mem0 store was then wired into the blind agent task. With a stale service block, a newer correction, and 12 unrelated newer notes, warm runs passed 5/5 while both cold arms passed 0/5. Every arm made one tool call. Mem0 still kept the stale block, so the pass came from the ranker and from showing only the top note, not from a supersede rule. The five-repeat walkthrough now covers both the probe and this transfer: lab/sessions/01-mem0-extraction.md.
Seventh lab result: a read-time supersede filter removed the stale block that Mem0 had retained. The policy parses key=value tokens from both the config seed and Mem0's sentence-shaped extraction, keeps the newest note per key set, and does not rewrite the store. The same five-repeat blind task stayed at warm 5/5 and cold 0/5, with one tool call per arm, and the stale block was absent from all five warm memory prompts. This is an adapter policy, not proof that Mem0 natively supersedes old memories.
Eighth lab result: a partial Mem0 correction exposed extraction loss.
Mem0 rewrote port=7314 and region=eu-north-1 as prose, mixing old
and new values. The merge policy had no keys to group, so warm passed 0/5. With
MEM0_INFER=0 raw notes, merge composed the four-key block and warm passed 5/5.
Extraction and retrieval policy are separate failure points.
Ninth lab result: serving concurrency is not the same as parallel decode. On one llama.cpp slot, blazedit_5k at concurrency 2 kept output near 196 tokens per second. TTFT p50 rose from 187 ms to 2,770 ms and request latency p50 from 2.61 s to 5.18 s, because the second client waited for the slot. More clients did not add throughput.
Tenth lab result: a real memory prefix is cheap at the prompt boundary, but not free. A Gemma 4 inventory memory block added 123 input tokens to a 77-token cold prompt, so the warm prompt was 2.6 times longer. TTFT p50 rose only 7.6 ms, from 33.3 ms to 40.9 ms, and request latency p50 rose 9.7 ms, from 574.3 ms to 584.0 ms. The 128 decode tokens dominate the request. The prefix is not where the serving money goes.
Eleventh lab result: when the same memory prefix is replayed, llama.cpp reuses almost all of it. After the first request in each arm, 82 of 86 cold prompt tokens and 205 of 209 warm prompt tokens came from cache, and repeated-request TTFT fell to about 10 to 13 ms. This is a best-case cache test: 9 of 10 requests repeated the same prompt, so it does not measure a production cache hit rate.
Related project
- mcpwith.codemode.cc - MCP code mode: how LLM agents run code inside MCP servers, contrasted with three older harness patterns.