Hybrid Memory: Vector + Graph + KV
Hybrid memory runs three stores in parallel, vector for semantic similarity, KV for fast fact lookup, graph for entity-relationship reasoning, with a scoring layer that fuses them on retrieval. This is a widely used production pattern for external memory; Mem0 (Chhikara et al., 2025) is one reference implementation.
Learning Objectives
- Explain why a single store (vector only, graph only, KV only) is insufficient for agent memory.
- Name Mem0's three parallel stores and what each one optimizes for.
- Describe Mem0's fusion scoring, relevance, importance, recency, and why it is a weighted sum, not a hierarchy.
- Implement a toy three-store memory in stdlib with an
add()that writes to all three and asearch()that fuses results.
The Problem
One store is wrong for one of three query classes:
- Semantic similarity"what did we discuss about agent drift last week?" Vector wins; KV and graph miss.
- Fact lookup"what is the user's phone number?" KV wins; vector is wasteful, graph is overkill.
- Relationship reasoning"which customers share the same billing entity?" Graph wins; vector and KV cannot answer.
Production agents issue all three in one session. A single-store memory is always wrong for two of them. Mem0's contribution is wiring all three behind a single add/search surface with a scoring function that fuses them.
The Concept
Three stores in parallel
Mem0 (arXiv:2504.19413, April 2025) on add(text, user_id, metadata):
- Extract candidate facts from the text (an LLM-driven step).
- Write each fact to the vector store (embedding) for semantic search.
- Write each fact to the KV store keyed on (user_id, fact_type, entity) for $O(1)$ lookup.
- Write each fact to the graph store (Mem0g) as typed edges for relationship queries.
On search(query, user_id):
- Vector store returns top-k by embedding cosine.
- KV store returns direct hits keyed on query-derived (user_id, type, entity).
- Graph store returns subgraph reachable from query entities.
- A scoring layer fuses the three.
Fusion scoring
score = w_relevance * relevance(q, record)
+ w_importance * importance(record)
+ w_recency * recency(record)- Relevancevector cosine, KV exact match, graph path weight.
- Importancetagged at write time or learned (some facts matter more: names, IDs, policies).
- Recencyexponential decay over time since last write or read.
Weights are tuned per product. Higher w_recency for chat agents; higher w_importance for compliance agents; higher w_relevance for retrieval agents.
Mem0g and temporal reasoning
Mem0g adds a conflict detector. When a new fact contradicts an existing edge, the existing edge is marked invalid but not deleted. Temporal queries ("what was the user's city in March?") traverse the valid-at-time subgraph.
This is the compliance-grade behavior Letta's invalidation pattern generalizes.
Benchmark numbers
The Mem0 paper reports (2025):
- LoCoMo (long-form conversation memory): 91.6
- LongMemEval (long-horizon episodic memory): 93.4
- BEAM 1M (1M-token memory benchmark): 64.1
Comparison baselines (full-context 128k LLM, flat vector store, flat KV) all lose by 10+ points. Benchmarks alone don't justify choice, operational shape does, but the numbers show the fusion design is not a rounding error.
Scope taxonomy
Mem0 splits memory by scope:
- User memorypersists across sessions, keyed on
user_id. - Session memorypersists within one thread.
- Agent memoryper-agent instance state.
Every write picks one scope. Retrieval can query across scopes with per-scope weights. Mixing scopes without thought is how you get "the assistant told Alice about Bob's project" incidents.
Where this pattern goes wrong
- Embedding drift. Vector results that look right on the first hundred queries degrade as the corpus grows. Add periodic re-embedding of the top-N-used records.
- KV schema creep.
(user_id, type, entity)looks simple until every team adds their owntype. Audit the type set quarterly. - Graph explosion. One noisy extractor adds 50 edges per message. Cap graph writes per
addcall; drop low-confidence edges.
Build It
code/main.py implements the three-store pattern in stdlib:
VectorStorenaive token-overlap similarity as an embedding stand-in.KVStoredict keyed on(user_id, fact_type, entity).GraphStoretyped edges (subject, relation, object, valid).Mem0top-level facade withadd()search()fusion scoring, and scope-aware retrieval.- A worked trace on a multi-user, multi-session conversation.
Run it:
python3 code/main.pyThe output shows three separate recall paths plus the fused top-k. Flip the scoring weights at the top of main() and watch the ranking change.
Use It
- Mem0 (Apache 2.0)production-ready. Self-host with Postgres + Qdrant + Neo4j, or use the managed cloud.
- Lettathree-tier core/recall/archival; bring your own vector and graph backends.
- Zepcommercial alternative with temporal KG and fact extraction.
- Custom buildswhen you need exact control over the extractor (compliance) or fusion weights (voice agents where recency dominates).
Ship It
outputs/skill-hybrid-memory.md generates a three-store memory scaffold with a fusion scorer, scope taxonomy, and temporal invalidation wired in.
Exercises
- Replace the toy vector similarity with a real embedding model (sentence-transformers, Ollama, OpenAI embeddings). Measure recall@10 on a synthetic long conversation. Does the ranking drift over 1000 writes?
- Add a temporal query:
search(query, as_of=timestamp). Return only records valid at or before that time. Which store needs the most work? - Implement a conflict detector: if an incoming fact contradicts a graph edge, invalidate the old edge and log both. Test on "user lives in Berlin" -> "user lives in Lisbon."
- Port the fusion scorer to include a
user_feedbackdimension (thumbs-up on retrieved records). How do you prevent gaming (the agent only returns records it already liked)? - Read the Mem0 docs (
docs.mem0.ai). Port the toy tomem0client calls. Compare retrieval quality on the same 20 test queries.
Key Terms
| Term | What people say | What it actually means |
|---|---|---|
| Hybrid memory | "Vector plus graph plus KV" | Three stores written in parallel, fused on retrieval |
| Fact extraction | "Memory ingestion" | LLM step that breaks text into (entity, relation, fact) tuples |
| Fusion scoring | "Relevance ranking" | Weighted sum of relevance, importance, recency |
| Scope | "Memory namespace" | user / session / agent, determines who sees what |
| Mem0g | "Memory graph" | Typed edges with temporal validity for relationship queries |
| Temporal invalidation | "Soft delete" | Mark contradicted edges invalid; never delete |
| Embedding drift | "Retrieval rot" | Vector quality degrades as corpus grows; re-embed periodically |
Further Reading
- Chhikara et al., Mem0 (arXiv:2504.19413)the original paper
- Mem0 docsproduction API, SDKs, managed cloud
- Packer et al., MemGPT (arXiv:2310.08560)the virtual-context predecessor
- Letta, Memory Blocks blogthe three-tier sibling design
Check what you learned
7 questions. Answer each one to see why.
Which query class does a KV store handle best?
The full code
Every file for this lesson. Copy it, run it, then change it.
"""Mem0-shaped hybrid memory: vector + KV + graph with fusion scoring.
Stdlib only. Vector store uses token-overlap as an embedding stand-in.
Scope taxonomy: user / session / agent. Fusion: relevance + importance + recency.
"""
from __future__ import annotations
import time
from dataclasses import dataclass, field
from typing import Any
@dataclass
class Record:
rid: str
text: str
scope: str
user_id: str
session_id: str
importance: float = 0.5
ts: float = field(default_factory=time.time)
tags: tuple[str, ...] = ()
class VectorStore:
def __init__(self) -> None:
self._records: dict[str, Record] = {}
def add(self, record: Record) -> None:
self._records[record.rid] = record
def search(self, query: str, top_k: int = 5) -> list[tuple[float, Record]]:
q_tokens = set(query.lower().split())
scored: list[tuple[float, Record]] = []
for record in self._records.values():
r_tokens = set(record.text.lower().split())
if not r_tokens:
continue
overlap = len(q_tokens & r_tokens)
if overlap == 0:
continue
score = overlap / (len(q_tokens | r_tokens))
scored.append((score, record))
scored.sort(key=lambda x: -x[0])
return scored[:top_k]
@dataclass(frozen=True)
class KVKey:
user_id: str
fact_type: str
entity: str
class KVStore:
def __init__(self) -> None:
self._map: dict[KVKey, Record] = {}
def put(self, key: KVKey, record: Record) -> None:
self._map[key] = record
def get(self, key: KVKey) -> Record | None:
return self._map.get(key)
def by_user(self, user_id: str) -> list[Record]:
return [r for k, r in self._map.items() if k.user_id == user_id]
@dataclass
class Edge:
subject: str
relation: str
obj: str
valid: bool = True
ts: float = field(default_factory=time.time)
class GraphStore:
def __init__(self) -> None:
self._edges: list[Edge] = []
def add_edge(self, subject: str, relation: str, obj: str) -> None:
for edge in self._edges:
if edge.valid and edge.subject == subject and edge.relation == relation:
edge.valid = False
self._edges.append(Edge(subject=subject, relation=relation, obj=obj))
def neighbors(self, subject: str, valid_only: bool = True) -> list[Edge]:
return [e for e in self._edges
if e.subject == subject and (e.valid or not valid_only)]
def all_edges(self) -> list[Edge]:
return list(self._edges)
@dataclass
class Mem0Config:
w_relevance: float = 0.6
w_importance: float = 0.2
w_recency: float = 0.2
recency_halflife_s: float = 86400.0
class Mem0:
def __init__(self, config: Mem0Config | None = None) -> None:
self.vector = VectorStore()
self.kv = KVStore()
self.graph = GraphStore()
self.config = config or Mem0Config()
self._counter = 0
def add(self, text: str, *, user_id: str, session_id: str = "s0",
scope: str = "user", importance: float = 0.5,
tags: tuple[str, ...] = (),
kv_triples: tuple[tuple[str, str], ...] = (),
graph_triples: tuple[tuple[str, str, str], ...] = ()) -> str:
self._counter += 1
rid = f"m{self._counter:03d}"
record = Record(rid=rid, text=text, scope=scope, user_id=user_id,
session_id=session_id, importance=importance, tags=tags)
self.vector.add(record)
for fact_type, entity in kv_triples:
self.kv.put(KVKey(user_id=user_id, fact_type=fact_type, entity=entity), record)
for subject, relation, obj in graph_triples:
self.graph.add_edge(subject, relation, obj)
return rid
def _recency_score(self, record: Record, now: float) -> float:
elapsed = max(0.0, now - record.ts)
half = self.config.recency_halflife_s
return 0.5 ** (elapsed / half) if half > 0 else 1.0
def search(self, query: str, *, user_id: str,
scope: str | None = None, top_k: int = 5) -> list[tuple[float, Record]]:
now = time.time()
vector_hits = self.vector.search(query, top_k=top_k * 3)
fused: dict[str, tuple[float, Record]] = {}
for rel, record in vector_hits:
if scope is not None and record.scope != scope:
continue
if record.user_id != user_id and record.scope == "user":
continue
recency = self._recency_score(record, now)
score = (self.config.w_relevance * rel
+ self.config.w_importance * record.importance
+ self.config.w_recency * recency)
fused[record.rid] = (score, record)
for record in self.kv.by_user(user_id):
if record.rid in fused:
continue
recency = self._recency_score(record, now)
score = (self.config.w_relevance * 0.4
+ self.config.w_importance * record.importance
+ self.config.w_recency * recency)
fused[record.rid] = (score, record)
ordered = sorted(fused.values(), key=lambda x: -x[0])
return ordered[:top_k]
def main() -> None:
print("=" * 70)
print("MEM0 HYBRID MEMORY — Phase 14, Lesson 09")
print("=" * 70)
mem = Mem0()
mem.add(
"ava prefers citation-heavy, terse writing over tutorial style",
user_id="ava", session_id="s001",
importance=0.7, tags=("preference", "writing"),
kv_triples=(("writing_style", "terse_citation_heavy"),),
)
mem.add(
"ava is building a 30-lesson curriculum on agent engineering",
user_id="ava", session_id="s001",
importance=0.9, tags=("project",),
kv_triples=(("project", "agent_curriculum"),),
graph_triples=(("ava", "owns_project", "agent_curriculum"),),
)
mem.add(
"ava lives in Berlin",
user_id="ava", session_id="s001",
importance=0.6, tags=("profile",),
kv_triples=(("city", "Berlin"),),
graph_triples=(("ava", "lives_in", "Berlin"),),
)
mem.add(
"ava moved to Lisbon last month",
user_id="ava", session_id="s002",
importance=0.8, tags=("profile", "update"),
kv_triples=(("city", "Lisbon"),),
graph_triples=(("ava", "lives_in", "Lisbon"),),
)
mem.add(
"bob requested a refund for invoice 4711",
user_id="bob", session_id="s010",
importance=0.9, tags=("billing",),
kv_triples=(("refund_request", "4711"),),
)
print("\nvector-only recall for 'writing style preferences'")
for score, record in mem.vector.search("writing style preferences", top_k=3):
print(f" {score:.3f} {record.rid} {record.text}")
print("\ngraph recall for entities linked to 'ava'")
for edge in mem.graph.neighbors("ava", valid_only=False):
status = "VALID " if edge.valid else "INVALID"
print(f" [{status}] {edge.subject} --{edge.relation}--> {edge.obj}")
print("\nKV recall for ava all facts")
for record in mem.kv.by_user("ava"):
print(f" {record.rid} {record.text}")
print("\nfused top-3 for ava, query 'where does ava live'")
for score, record in mem.search("where does ava live", user_id="ava", top_k=3):
print(f" {score:.3f} {record.rid} {record.text}")
print("\nfused top-3 for ava, query 'what is she building'")
for score, record in mem.search("what is ava building", user_id="ava", top_k=3):
print(f" {score:.3f} {record.rid} {record.text}")
print("\nscope isolation: bob's refund does not leak to ava's search")
hits = mem.search("refund invoice", user_id="ava", top_k=5)
print(f" ava results: {len(hits)} (expect 0 user-scoped hits from bob)")
for score, record in hits:
print(f" {score:.3f} {record.user_id} {record.text}")
print()
print("fusion: relevance + importance + recency. per-product weight tuning.")
if __name__ == "__main__":
main()