Memory Blocks and Sleep-Time Compute
Discrete functional memory blocks the model can edit directly, and a sleep-time agent that consolidates memory asynchronously while the primary agent is idle. These two ideas are how you scale memory beyond one conversation.
Learning Objectives
- Name the three memory tiers Letta uses (core, recall, archival) and the role of each.
- Explain the memory-block pattern: Human block, Persona block, and user-defined blocks as first-class typed objects.
- Describe what sleep-time compute is, why it sits off the critical path, and why it can run a stronger model than the primary agent.
- Implement a scripted two-agent loop where a primary agent serves responses and a sleep-time agent consolidates blocks between turns.
The Problem
MemGPT (Lesson 07) solved the virtual-memory control flow. Three production problems emerged:
- Latency. Every memory operation sits on the critical path. If the agent has to prune, summarize, or reconcile while the user waits, tail latency blows up.
- Memory rot. Writes accumulate. Contradicted facts stay. Retrieval drowns in stale content.
- Structure loss. A flat archival store cannot express "the Human block is always in the prompt; the Persona block is always in the prompt; the Task block swaps per session."
Letta (letta.com) is the platform name the original MemGPT project adopted in 2024, the paper's pattern keeps the MemGPT name, and the 2026 Letta V1 rewrite is a later, separate step. Memory blocks make structure explicit; sleep-time compute moves consolidation off the critical path.
The Concept
Three tiers
| Tier | Scope | Where it lives | Written by |
|---|---|---|---|
| Core | Always visible | Inside the main prompt | Agent tool call + sleep-time rewrites |
| Recall | Conversation history | Retrievable | Automatic turn logging |
| Archival | Arbitrary facts | Vector + KV + graph | Agent tool call + sleep-time ingest |
Core is the MemGPT core. Recall is the conversation buffer with its evicted tail. Archival is the external store. The split cleans up MemGPT's two-tier overloading.
Memory blocks
A block is a typed, persistent, editable section of the core tier. The original MemGPT paper defined two:
- Human blockfacts about the user (name, role, preferences, goals).
- Persona blockthe agent's self-concept (identity, tone, constraints).
Letta generalizes to arbitrary user-defined blocks: a Task block for the current goal, a Project block for codebase facts, a Safety block for hard constraints. Each block has an idlabelvaluelimit (character cap), description (so the model knows when to edit it).
Blocks are editable via the tool surface:
block_append(label, text)block_replace(label, old, new)block_read(label)block_summarize(label)condense a block that is near its limit.
Sleep-time compute
The 2025 Letta addition: run a second agent in background, off the critical path. Sleep-time agents process conversation transcripts and codebase context, write learned_context into shared blocks, and consolidate or invalidate archival records.
Properties that fall out:
- No latency cost. Primary responses do not wait for memory ops.
- Stronger model allowed. The sleep-time agent can be a more expensive, slower model because it is not latency-constrained.
- Natural consolidation window. Dedup, summarize, invalidate contradicted facts when the user is not waiting.
The shape matches how humans work: you do the task, you sleep on it, the long-term memory settles overnight.
Native reasoning
Letta V1 (letta_v1_agent2026) deprecates send_message/heartbeat and inline Thought: tokens in favor of native reasoning. The Responses API (OpenAI) and the Messages API with extended thinking (Anthropic) emit reasoning on a separate channel, passed through turns (encrypted across providers in production). The control loop is still ReAct. The thought trace is structural, not prompt-shaped.
Where this pattern goes wrong
- Block bloat. Infinite
block_appendhits the limit fast. Wire a block summarizer before the write that pushes over the cap. - Silent drift. Sleep-time agent rewrites a block and the primary agent never notices. Version blocks and surface diffs in the trace.
- Poisoned consolidation. Sleep-time agent processes attacker-reachable content into core. Lesson 27 applies to the sleep-time surface too.
Build It
code/main.py implements:
Blockid, label, value, limit, description.BlockStoreCRUD +near_limit(label)helper.- Two scripted agents,
PrimaryAgentserves a turn,SleepTimeAgentconsolidates between turns. - A trace that shows a three-turn conversation with block writes, plus a sleep-time pass that summarizes a block and invalidates a stale fact.
Run it:
python3 code/main.pyThe transcript shows the split: primary turns are fast and produce raw writes; the sleep pass compacts and cleans up.
Use It
- Letta (letta.com) for the reference implementation. Self-host or managed cloud.
- Claude Agent SDK skills as block-shaped knowledge, a skill is a named, versioned, retrievable block of instructions the agent loads on demand.
- Custom builds for teams that want control over the storage backend. Use the Letta API contract so you can migrate later.
Ship It
outputs/skill-memory-blocks.md generates a Letta-shaped block system with sleep-time hooks for any runtime, including safety rules and citation wiring.
Exercises
- Add a
block_summarizetool that replaces the block value with a model-generated summary whennear_limitreturns true. Which trigger threshold minimizes both summarization calls and block overflow? - Implement sleep-time dedup over archival: two records whose text has >90% token overlap collapse to one. Do it only in the sleep pass, never on the critical path.
- Version blocks. On every write record the old value and a diff. Expose
block_history(label)so operators can debug "why did the agent forget X." - Treat sleep-time agents as untrusted writers. When they touch the Persona or Safety block, require a second-agent review before committing.
- Port the example to use the Letta API (
letta_v1_agent). What changes in the block schema, and how does native reasoning alter the trace shape?
Key Terms
| Term | What people say | What it actually means |
|---|---|---|
| Memory block | "Editable prompt section" | Typed, persistent, LLM-editable segment of core memory |
| Human block | "User memory" | Facts about the user, pinned in core |
| Persona block | "Agent identity" | Self-concept, tone, constraints, pinned in core |
| Sleep-time compute | "Async memory work" | Second agent doing consolidation off the critical path |
| Core / Recall / Archival | "Tiers" | Three-layer memory split: always-visible / conversation / external |
| Block limit | "Cap" | Character limit per block; forces summarization |
| Native reasoning | "Thinking channel" | Provider-level reasoning output, not prompt-level Thought: |
| Learned context | "Sleep output" | Facts the sleep-time agent writes into shared blocks |
Further Reading
- Letta, Memory Blocks blogthe block pattern
- Letta, Sleep-time Compute blogasync consolidation
- Letta, Rearchitecting the Agent Loopnative reasoning rewrite
- Packer et al., MemGPT (arXiv:2310.08560)the origin
Check what you learned
7 questions. Answer each one to see why.
What are Letta's three memory tiers?
The full code
Every file for this lesson. Copy it, run it, then change it.
"""Letta-shaped memory blocks with a sleep-time consolidation agent.
Primary agent writes raw facts during turns. Sleep-time agent runs between
turns, off the critical path, and consolidates blocks. Scripted so it runs
offline.
"""
from __future__ import annotations
from dataclasses import dataclass, field
from typing import Any
@dataclass
class Block:
label: str
value: str = ""
limit: int = 300
description: str = ""
version: int = 0
history: list[str] = field(default_factory=list)
def append(self, text: str) -> str:
old = self.value
self.value = (self.value + " " + text).strip() if self.value else text
self.version += 1
self.history.append(old)
return f"{self.label} v{self.version} ({len(self.value)}/{self.limit})"
def replace(self, old: str, new: str) -> str:
if old not in self.value:
return f"error: {old!r} not in {self.label}"
prev = self.value
self.value = self.value.replace(old, new)
self.version += 1
self.history.append(prev)
return f"{self.label} v{self.version} replaced"
def rewrite(self, new: str) -> str:
prev = self.value
self.value = new
self.version += 1
self.history.append(prev)
return f"{self.label} v{self.version} rewritten ({len(self.value)}/{self.limit})"
def near_limit(self, threshold: float = 0.8) -> bool:
return len(self.value) >= int(self.limit * threshold)
class BlockStore:
def __init__(self) -> None:
self._blocks: dict[str, Block] = {}
def create(self, label: str, description: str, limit: int = 300) -> Block:
block = Block(label=label, description=description, limit=limit)
self._blocks[label] = block
return block
def get(self, label: str) -> Block | None:
return self._blocks.get(label)
def labels(self) -> list[str]:
return sorted(self._blocks)
def render(self) -> str:
lines: list[str] = []
for label in self.labels():
block = self._blocks[label]
lines.append(f"[{block.label} v{block.version} "
f"{len(block.value)}/{block.limit}]")
lines.append(f" {block.value}")
return "\n".join(lines)
@dataclass
class ArchivalRecord:
rid: str
text: str
valid: bool = True
class Archival:
def __init__(self) -> None:
self._records: list[ArchivalRecord] = []
self._counter = 0
def insert(self, text: str) -> str:
self._counter += 1
rid = f"a{self._counter:03d}"
self._records.append(ArchivalRecord(rid=rid, text=text))
return rid
def invalidate(self, rid: str) -> bool:
for record in self._records:
if record.rid == rid:
record.valid = False
return True
return False
def valid_records(self) -> list[ArchivalRecord]:
return [r for r in self._records if r.valid]
def all_records(self) -> list[ArchivalRecord]:
return list(self._records)
class PrimaryAgent:
"""Handles turns. Writes raw facts fast; never summarizes or consolidates."""
def __init__(self, blocks: BlockStore, archival: Archival) -> None:
self.blocks = blocks
self.archival = archival
self.trace: list[str] = []
def turn(self, user_text: str, writes: list[tuple[str, str, str]]) -> str:
self.trace.append(f"user: {user_text}")
for kind, label_or_text, payload in writes:
if kind == "block_append":
block = self.blocks.get(label_or_text)
if block is not None:
self.trace.append(f" block_append -> {block.append(payload)}")
elif kind == "archival_insert":
rid = self.archival.insert(payload)
self.trace.append(f" archival_insert -> {rid}")
response = f"response to: {user_text}"
self.trace.append(f"assistant: {response}")
return response
class SleepTimeAgent:
"""Off the critical path. Summarizes near-limit blocks, invalidates
contradicted archival records, no user latency cost.
"""
def __init__(self, blocks: BlockStore, archival: Archival) -> None:
self.blocks = blocks
self.archival = archival
self.trace: list[str] = []
def run(self, contradictions: list[tuple[str, str]]) -> None:
self.trace.append("sleep-time pass start")
for label in self.blocks.labels():
block = self.blocks.get(label)
if block is None:
continue
if block.near_limit():
summary = _summarize(block.value, block.limit // 2)
result = block.rewrite(summary)
self.trace.append(f" consolidate {label}: {result}")
for claim, reason in contradictions:
for record in self.archival.all_records():
if record.valid and claim.lower() in record.text.lower():
self.archival.invalidate(record.rid)
self.trace.append(
f" invalidate {record.rid} ({reason}): {record.text[:50]}..."
)
self.trace.append("sleep-time pass end")
def _summarize(text: str, target_len: int) -> str:
sentences = [s.strip() for s in text.split(".") if s.strip()]
if not sentences:
return text[:target_len]
picked: list[str] = []
total = 0
for sentence in sentences:
if total + len(sentence) + 1 > target_len:
break
picked.append(sentence)
total += len(sentence) + 2
return ". ".join(picked) + "."
def main() -> None:
print("=" * 70)
print("LETTA MEMORY BLOCKS + SLEEP-TIME COMPUTE — Phase 14, Lesson 08")
print("=" * 70)
blocks = BlockStore()
blocks.create("human", "facts about the user", limit=180)
blocks.create("persona", "the agent's self-concept", limit=160)
blocks.create("task", "the current task scope", limit=220)
archival = Archival()
primary = PrimaryAgent(blocks, archival)
sleep = SleepTimeAgent(blocks, archival)
primary.turn(
"my name is ava, I ship agents for a living, I live in Berlin",
[("block_append", "human", "name=ava role=ships_agents city=Berlin")],
)
primary.turn(
"today help me plan a 30-lesson curriculum on agent engineering",
[
("block_append", "task", "plan 30-lesson agent curriculum, target senior eng"),
("archival_insert", "",
"ava prefers concise, citation-heavy writing over tutorial-style"),
],
)
primary.turn(
"I moved to Lisbon last month; update your notes",
[
("block_append", "human", "city=Lisbon (updated from Berlin)"),
("archival_insert", "",
"ava lives in Berlin - old address, outdated"),
],
)
primary.turn(
"also the curriculum target is senior and staff engineers, not junior",
[("block_append", "task",
"audience=senior+staff eng, cite arXiv and first-party framework docs")],
)
print("\nprimary turns (writes are fast and raw)")
for line in primary.trace:
print(f" {line}")
print("\nblocks after primary phase (pre-consolidation)")
print(blocks.render())
sleep.run(contradictions=[
("ava lives in Berlin",
"human block updated city to Lisbon; Berlin archival claim is stale"),
])
print("\nsleep-time trace")
for line in sleep.trace:
print(f" {line}")
print("\nblocks after sleep-time (consolidated)")
print(blocks.render())
print("\narchival state")
for record in archival.all_records():
status = "VALID " if record.valid else "INVALID"
print(f" {record.rid} [{status}] {record.text}")
print()
print("key property: primary-turn latency is unchanged by consolidation.")
print("sleep-time can run a stronger, slower model — it is off the path.")
if __name__ == "__main__":
main()