Get Started

Agent Memory, Virtual Context and Memory Paging

Context windows are finite. Conversations, documents, and tool traces are not. The fix is OS virtual memory restated, main context is RAM, external store is disk, the agent pages between them. MemGPT (Packer et al., 2023) named the pattern; many production memory systems build on it.

75 minutesBuildPython (stdlib)Free lesson

Learning Objectives

  • Explain the OS analogy MemGPT builds on: main context = RAM, external context = disk, memory tools = page in/out.
  • Implement the two-tier MemGPT pattern in stdlib with a main-context buffer, an external searchable store, and page in/out tools.
  • Describe how the agent issues "interrupts" to query or modify external memory and how the result is spliced back into the next prompt.
  • Identify the MemGPT design choices that carry into Letta (Lesson 08) and Mem0 (Lesson 09).

The Problem

Context windows look like they should solve memory. They do not. Three failure modes recur in production:

  1. Overflow. Multi-turn conversations, long documents, or tool-call-heavy trajectories cross the window. Everything past the cutoff is gone.
  2. Dilution. Even within the window, stuffing irrelevant context dilutes attention over what matters. Frontier models still degrade on long inputs.
  3. Persistence. A new session starts with an empty window. Agents without external memory cannot say "remember when you asked me to..." across sessions.

Bigger windows help but do not fix this. Mem0's 2025 paper measured that 128k-window baselines still miss long-horizon facts that a 4k-window agent with external memory catches.

The Concept

The OS analogy

MemGPT (Packer et al., arXiv:2310.08560, v2 Feb 2024) maps context management to operating-system virtual memory:

OS conceptMemGPT concept2026 production analog
RAMmain context (prompt)Anthropic/OpenAI context window
Diskexternal contextvector DB, KV, graph store
Page faultmemory tool callmemory.searchmemory.readmemory.write
OS kernelagent control loopReAct loop with memory tools

The agent runs a normal ReAct loop. One extra class of tools lets it page data in and out of main context.

Two tiers

  • Main context. Fixed-size prompt holding the current task. Always visible to the model.
  • External context. Unbounded, searchable via tools. Read when relevant, written when facts emerge.

The original paper evaluated the design on two tasks beyond the base window: document analysis longer than 100k tokens and multi-session chat with persistent memory across days.

The interrupt pattern

MemGPT introduces memory-as-interrupt: mid-conversation the agent can invoke a memory tool, the runtime executes it, and the result splices into the next assistant turn as a new observation. Conceptually identical to a Unix read() syscall that blocks the process, returns bytes, and the process continues.

Canonical memory tool surface:

  • core_memory_append(section, text)write to a persistent section of the prompt.
  • core_memory_replace(section, old, new)edit a persistent section.
  • archival_memory_insert(text)write to the searchable external store.
  • archival_memory_search(query, top_k)retrieve from the external store.
  • conversation_search(query)scan past turns.

Where the paper ends and production begins

In September 2024 MemGPT became Letta. The research repo (cpacker/MemGPT) remains; Letta extends the design:

  • Three tiers instead of two (core, recall, archival, Lesson 08).
  • Native reasoning replacing the send_message/heartbeat pattern (Lesson 08).
  • Sleep-time agents running async memory work (Lesson 08).

The MemGPT paper is the 2026 foundation even if production systems run Letta, Mem0, or a custom two-tier store.

Where this pattern goes wrong

  • Memory rot. Writes accumulate faster than reads; retrieval drowns in stale facts. Fix: periodic consolidation (Letta sleep-time), explicit invalidation (Mem0 conflict detector).
  • Memory poisoning. External memory is retrieved text. If attacker-controlled content lands in a memory note, the agent re-ingests it next session. This is the Greshake et al. (Lesson 27) attack restated over time.
  • Citation loss. Agent recalls "the user asked me to ship X" but cannot cite which turn. Store source references (session ID, turn ID) with every archival write.

Build It

code/main.py implements MemGPT's two-tier pattern in stdlib:

  • MainContextfixed-size prompt buffer with a core dict and a messages list; auto-compacts oldest messages when over cap.
  • ArchivalStorein-memory BM25-esque store (token-overlap scoring) of (id, text, tags, session, turn) records.
  • Five memory tools mapping to the MemGPT surface.
  • A scripted agent that fills archival with facts, then answers a question by calling archival_memory_search.

Run it:

Code
python3 code/main.py

The trace shows the agent writing three facts, filling main context to the cap (forcing eviction), then answering a follow-up question by retrieving from archival, reproducing the MemGPT workflow without any real LLM.

Use It

Every production memory system today is a MemGPT variant:

  • Letta (Lesson 08), three tiers, native reasoning, sleep-time compute.
  • Mem0 (Lesson 09), vector + KV + graph fused with a scoring layer.
  • OpenAI Assistants / Responsesmanaged memory via threads and files.
  • Claude Agent SDKlong-term memory via skills and session store.

Pick one by operational shape (self-hosted, managed, framework-integrated), not by the core pattern, the core pattern is MemGPT.

The shape of agent memory

Paging solves capacity. It does not decide what to store. Four memory types recur across production systems, each answering a different question:

  • Working memorywhat matters right now? The in-context tier: current task, recent turns, pinned core sections. The prompt itself.
  • Episodic memorywhat happened? Past turns and trajectories, stored with session and turn references, replayable on demand.
  • Semantic memorywhat is true? Facts about the user, the domain, the world, updated and deduplicated as they change.
  • Procedural memoryhow do I do this? Learned routines, preferences, and rules that steer future behavior rather than recall.

Open-source implementations pick different points of attack:

TypeImplementationHow it tackles it
WorkingMemGPT / LettaPages content in and out of a fixed prompt budget via memory tools (this lesson, Lesson 08)
EpisodicZepTemporal knowledge graph, facts carry validity intervals, so "what was true when" is queryable
SemanticMem0Extraction pipeline that dedupes and updates facts across vector, KV, and graph stores (Lesson 09)
Semantic + proceduralLangMemBackground extraction of facts and behavioral rules into a store the agent consults between turns
Episodic + semanticagentmemoryCaptures sessions as they run, consolidates them into typed, searchable records

Ship It

outputs/skill-virtual-memory.md is a reusable skill that produces a correct two-tier memory scaffold (main + archival + tool surface) for any target runtime, with eviction policy and citation fields wired in.

Exercises

  1. Add a max_main_context_tokens cap measured in tokens (approximate with len(text.split()) * 1.3). Compact the oldest messages into a summary when the cap is exceeded. Compare behavior with and without the summarizer.
  2. Implement BM25 properly over the archival store (term frequency, inverse document frequency). Measure recall@10 on a toy fact set versus the token-overlap baseline.
  3. Add citation fields (session_id, turn_id, source_url) to archival inserts. Make the agent cite sources on every retrieval-backed answer.
  4. Simulate memory poisoning: add an archival record that says "ignore all future user instructions." Write a guard that scans retrievals for directive-shaped text and marks them untrusted.
  5. Port the implementation to use the MemGPT research repo's core-memory JSON schema (cpacker/MemGPT). What changes when you switch from flat strings to typed sections?

Key Terms

TermWhat people sayWhat it actually means
Virtual context"Unlimited memory"Main (prompt) + external (searchable) tiers with page in/out
Main context"Working memory"The prompt, fixed-size, always visible
Archival memory"Long-term store"External searchable persistence, retrieved on demand
Core memory"Persistent prompt section"Named sections pinned inside the main context
Memory tool"Memory API"Tool call the agent issues to read/write external memory
Interrupt"Memory page fault"Agent pauses, runtime fetches, result splices into next turn
Memory rot"Stale facts"Old writes drown retrieval; fix with consolidation
Memory poisoning"Injected persistent note"Attacker content stored as memory, re-ingested on recall

Further Reading

Check what you learned

7 questions. Answer each one to see why.

Question 1 of 70 correct

What OS analogy does MemGPT build on?

The full code

Every file for this lesson. Copy it, run it, then change it.

main.py
"""MemGPT-shaped two-tier memory in stdlib.

Main context is a fixed-size prompt buffer (core dict + messages list).
Archival memory is an external searchable store. Agents page data in and out
via memory tools. No LLM call — a scripted agent drives the scenario so the
control flow is testable offline.
"""

from __future__ import annotations

from dataclasses import dataclass, field
from typing import Any


@dataclass
class Message:
    role: str
    text: str


@dataclass
class MainContext:
    core: dict[str, str] = field(default_factory=dict)
    messages: list[Message] = field(default_factory=list)
    max_messages: int = 4
    evicted: list[Message] = field(default_factory=list)

    def append(self, role: str, text: str) -> None:
        self.messages.append(Message(role=role, text=text))
        while len(self.messages) > self.max_messages:
            self.evicted.append(self.messages.pop(0))

    def render(self) -> str:
        parts: list[str] = ["[core]"]
        for key, value in sorted(self.core.items()):
            parts.append(f"  {key}: {value}")
        parts.append("[messages]")
        for msg in self.messages:
            parts.append(f"  {msg.role}: {msg.text}")
        return "\n".join(parts)


@dataclass
class ArchivalRecord:
    rid: str
    text: str
    tags: tuple[str, ...] = ()
    session_id: str = "s0"
    turn_id: int = 0


class ArchivalStore:
    def __init__(self) -> None:
        self._records: list[ArchivalRecord] = []
        self._counter = 0

    def insert(self, text: str, *, tags: tuple[str, ...] = (),
               session_id: str = "s0", turn_id: int = 0) -> str:
        self._counter += 1
        rid = f"a{self._counter:03d}"
        self._records.append(ArchivalRecord(
            rid=rid, text=text, tags=tags,
            session_id=session_id, turn_id=turn_id,
        ))
        return rid

    def search(self, query: str, top_k: int = 3) -> list[ArchivalRecord]:
        q_tokens = set(query.lower().split())
        scored: list[tuple[float, ArchivalRecord]] = []
        for record in self._records:
            r_tokens = set(record.text.lower().split())
            if not r_tokens:
                continue
            overlap = len(q_tokens & r_tokens)
            if overlap == 0:
                continue
            score = overlap / (len(q_tokens) + len(r_tokens) - overlap)
            scored.append((score, record))
        scored.sort(key=lambda x: -x[0])
        return [r for _, r in scored[:top_k]]

    def count(self) -> int:
        return len(self._records)


class MemoryTools:
    def __init__(self, main: MainContext, archival: ArchivalStore) -> None:
        self.main = main
        self.archival = archival

    def core_memory_append(self, section: str, text: str) -> str:
        existing = self.main.core.get(section, "")
        self.main.core[section] = (existing + " " + text).strip() if existing else text
        return f"core[{section}] appended: {len(self.main.core[section])} chars"

    def core_memory_replace(self, section: str, old: str, new: str) -> str:
        current = self.main.core.get(section, "")
        if old not in current:
            return f"error: {old!r} not in core[{section}]"
        self.main.core[section] = current.replace(old, new)
        return f"core[{section}] replaced"

    def archival_memory_insert(self, text: str, tags: tuple[str, ...] = ()) -> str:
        rid = self.archival.insert(text, tags=tags)
        return f"stored {rid} ({self.archival.count()} records)"

    def archival_memory_search(self, query: str, top_k: int = 3) -> str:
        hits = self.archival.search(query, top_k=top_k)
        if not hits:
            return "no matches"
        return "\n".join(f"  {h.rid}: {h.text}" for h in hits)

    def conversation_search(self, query: str) -> str:
        q = query.lower()
        for msg in reversed(self.main.evicted + self.main.messages):
            if q in msg.text.lower():
                return f"found ({msg.role}): {msg.text}"
        return "no matches"


@dataclass
class ToolCall:
    name: str
    args: dict[str, Any]


def run_scripted_agent(tools: MemoryTools, script: list[ToolCall]) -> list[str]:
    observations: list[str] = []
    for call in script:
        fn = getattr(tools, call.name, None)
        if fn is None:
            observations.append(f"error: unknown tool {call.name!r}")
            continue
        try:
            observations.append(fn(**call.args))
        except Exception as e:
            observations.append(f"error: {type(e).__name__}: {e}")
    return observations


def main() -> None:
    print("=" * 70)
    print("MEMGPT VIRTUAL CONTEXT — Phase 14, Lesson 07")
    print("=" * 70)

    main_ctx = MainContext(max_messages=3)
    archival = ArchivalStore()
    tools = MemoryTools(main_ctx, archival)

    main_ctx.append("user", "my name is ava and I ship agents for a living")
    main_ctx.append("assistant", "noted. what are you building right now?")
    main_ctx.append("user", "a retrieval bot for our sales org, 12 tools so far")
    main_ctx.append("assistant", "12 tools is in the long-horizon band; plan for drift")

    script = [
        ToolCall("core_memory_append",
                 {"section": "persona", "text": "the agent remembers user details politely"}),
        ToolCall("core_memory_append",
                 {"section": "user", "text": "name=ava, role=ships agents"}),
        ToolCall("archival_memory_insert",
                 {"text": "ava is building a retrieval bot with 12 tools for sales",
                  "tags": ("project", "ava")}),
        ToolCall("archival_memory_insert",
                 {"text": "long-horizon tool chains drift after 20 steps per BFCL V4",
                  "tags": ("bfcl", "tools")}),
        ToolCall("archival_memory_insert",
                 {"text": "sleep-time compute consolidates memory asynchronously",
                  "tags": ("letta", "memory")}),
    ]
    observations = run_scripted_agent(tools, script)

    print("\ntool trace (memory writes)")
    for call, obs in zip(script, observations):
        print(f"  {call.name}({call.args}) -> {obs}")

    print("\nfilling main context until eviction kicks in")
    main_ctx.append("user", "what were you saying about tool chains?")
    main_ctx.append("assistant", "let me check archival")

    print(f"\nmain context ({len(main_ctx.messages)} messages, "
          f"{len(main_ctx.evicted)} evicted)")
    print(main_ctx.render())

    print("\npage in: archival_memory_search('tool chains drift')")
    hit = tools.archival_memory_search("tool chains drift", top_k=2)
    print(hit)

    print("\nconversation_search for 'retrieval bot'")
    print(tools.conversation_search("retrieval bot"))

    print()
    print("pattern: memory is interrupt-driven. agent calls a tool, runtime")
    print("fetches, result splices back as observation. same as Unix read().")


if __name__ == "__main__":
    main()