Skill Libraries and Lifelong Learning (Voyager)
Voyager (Wang et al., TMLR 2024) treats executable code as a skill. Skills are named, retrievable, composable, and refined by environment feedback. This is the reference architecture for Claude Agent SDK skills, skillkit, and the 2026 skill-library pattern.
Learning Objectives
- Name Voyager's three components, automatic curriculum, skill library, iterative prompting, and the role of each.
- Explain why Voyager makes the action space code, not primitive commands.
- Implement a stdlib skill library with registration, retrieval, composition, and failure-driven refinement.
- Map Voyager's pattern onto the 2026 Claude Agent SDK skills and the skillkit ecosystem.
The Problem
Agents that rebuild every capability from scratch in every session do three things wrong:
- Waste tokens. Every task re-elicits the same reasoning.
- Lose progress. A correction learned in session A doesn't transfer to session B.
- Fail on long-horizon composition. Complex tasks need capability hierarchies; one-shot prompts cannot express them.
Voyager's answer: treat each reusable capability as a named chunk of code stored in a library, retrievable by similarity, composable with other skills, and refined by execution feedback.
The Concept
Three components
Voyager (arXiv:2305.16291) structures an agent around:
- Automatic curriculum. A curiosity-driven proposer picks the next task based on the agent's current skill set and environment state. Exploration is bottom-up.
- Skill library. Each skill is executable code. New skills are added when a task succeeds. Skills are retrieved by query-to-description similarity.
- Iterative prompting mechanism. On failure, the agent receives execution errors, environment feedback, and self-verification output, then refines the skill.
The Minecraft evaluation (Wang et al., 2024): 3.3x more unique items, 8.5x faster stone tools, 6.4x faster iron tools, 2.3x longer map traversal versus baselines. The numbers are Minecraft-specific, but the pattern transfers.
Action space = code
Most agents emit primitive commands. Voyager emits JavaScript functions. A skill is:
async function craftIronPickaxe(bot) {
await mineIron(bot, 3);
await mineStick(bot, 2);
await placeCraftingTable(bot);
await craft(bot, 'iron_pickaxe');
}Composed from sub-skills. Stored keyed on description and embedding. Retrieved as a program, not a prompt.
This is the 2026 Claude Agent SDK skill: a named, retrievable chunk of code plus instructions the agent loads on demand.
Skill retrieval
New task "make a diamond pickaxe." Agent:
- Embeds the task description.
- Queries the skill library for top-k similar skills.
- Retrieves
craftIronPickaxemineDiamondplaceCraftingTableetc. - Composes the new skill from retrieved primitives + new logic.
This is the pattern MCP resources (Phase 13) and Agent SDK skills implement: retrieval over a knowledge/code surface, scoped to the current task.
Iterative refinement
Voyager's feedback loop:
- Agent writes a skill.
- Skill runs against the environment.
- One of three signals returns:
successerror(with stack trace),self-verification failure. - Agent rewrites the skill using the signal as context.
- Loop until success or max rounds.
This is Self-Refine (Lesson 05) applied to code generation with environment-grounded verification. CRITIC (Lesson 05) is the same pattern with external tools as the verifier.
Curriculum and exploration
Voyager's curriculum module proposes tasks like "build a shelter near the lake" based on what the agent has and what it has not yet done. The proposer uses the environment state + skill inventory to pick a task just above current capability, the exploration sweet spot.
For production agents this translates to a "what's missing" operator: given the current skill library and a domain, what skills are we not yet covering? Teams typically implement this manually as curriculum review.
Where this pattern goes wrong
- Skill library rot. Same skill added 10 times with slightly different descriptions. Add deduplication on write; retrieval returns only one.
- Composed-skill drift. Parent skill depends on a child that was refined. Version skills; a parent pinned to v1 doesn't magically pick up v3.
- Retrieval quality. Vector retrieval over skill descriptions degrades as the library grows past a few hundred. Supplement with tag filters and hard constraints ("only skills with
category=tooling").
Build It
code/main.py implements a stdlib skill library:
Skillname, description, code (as string), version, tags, dependencies.SkillLibraryregister, search (token overlap), compose (topological sort of deps), and refine (version bump on update).- A scripted agent that registers three primitive skills, composes a fourth, hits a failure, and refines.
Run it:
python3 code/main.pyThe trace shows library writes, retrieval, composition, a failed execution, and a v2 refinement, Voyager's loop end to end.
Use It
- Claude Agent SDK skills (Anthropic), the 2026 reference: each skill has a description, code, and instructions; loaded on demand during an agent session.
- skillkit (npm: skillkit), cross-agent skill management for 32+ AI coding agents.
- Custom skill librariesdomain-specific (SQL skills for data agents, Terraform skills for infra agents). The Voyager pattern scales down.
- OpenAI Agents SDK
toolsat the low end; each tool is a lightweight skill.
Ship It
outputs/skill-skill-library.md generates a Voyager-shaped skill library with registration, retrieval, versioning, and refinement wired in for any target runtime.
Exercises
- Add a dependency-cycle detector to
compose(). What happens when skill A depends on B which depends on A? Error vs warning? - Implement per-skill version pinning. When a parent skill composes child
crafting@1a refinement tocrafting@2must not silently upgrade the parent. - Replace token-overlap retrieval with sentence-transformers embeddings (or a BM25 stdlib impl). Measure retrieval@5 on a 50-skill toy library.
- Add a "curriculum" agent: given the current library and a domain description, propose 5 missing skills. Call it weekly.
- Read Anthropic's Claude Agent SDK skill docs. Port the toy library to the SDK's skill schema. What changes about discoverability?
Key Terms
| Term | What people say | What it actually means |
|---|---|---|
| Skill | "Reusable capability" | Named chunk of code + description, retrievable by similarity |
| Skill library | "Agent memory of how-to" | Persistent store of skills, searchable and composable |
| Curriculum | "Task proposer" | Bottom-up goal generator driven by current capability gap |
| Composition | "Skill DAG" | Skills invoking skills; topologically sorted on execution |
| Iterative refinement | "Self-correcting loop" | Env feedback + errors + self-verification fold back into the next version |
| Action-space-as-code | "Programmatic actions" | Emit functions, not primitive commands, for temporally extended behavior |
| Dedup on write | "Skill collapse" | Near-duplicate descriptions collapse to one canonical skill |
Further Reading
- Wang et al., Voyager (arXiv:2305.16291)the original skill-library paper
- Claude Agent SDK overviewskills as the 2026 productization
- Anthropic, Building agents with the Claude Agent SDKskills and subagents in practice
- Madaan et al., Self-Refine (arXiv:2303.17651)the refinement loop underneath Voyager
Check what you learned
7 questions. Answer each one to see why.
What does Voyager treat as the action space?
The full code
Every file for this lesson. Copy it, run it, then change it.
"""Voyager-shaped skill library: register, retrieve, compose, refine.
Stdlib only. Action space is code; skills are retrievable and composable;
failures feed back into the next version.
"""
from __future__ import annotations
from dataclasses import dataclass, field
from typing import Any, Callable
@dataclass
class Skill:
name: str
description: str
code: str
fn: Callable[..., Any]
version: int = 1
tags: tuple[str, ...] = ()
depends_on: tuple[str, ...] = ()
history: list[str] = field(default_factory=list)
class SkillLibrary:
def __init__(self) -> None:
self._skills: dict[str, Skill] = {}
def register(self, skill: Skill, dedup: bool = True) -> str:
if dedup and skill.name in self._skills:
existing = self._skills[skill.name]
existing.history.append(existing.code)
existing.code = skill.code
existing.fn = skill.fn
existing.description = skill.description
existing.tags = skill.tags
existing.depends_on = skill.depends_on
existing.version += 1
return f"refined {skill.name} -> v{existing.version}"
self._skills[skill.name] = skill
return f"registered {skill.name} v{skill.version}"
def search(self, query: str, top_k: int = 3,
tag_filter: str | None = None) -> list[tuple[float, Skill]]:
q_tokens = set(query.lower().split())
scored: list[tuple[float, Skill]] = []
for skill in self._skills.values():
if tag_filter and tag_filter not in skill.tags:
continue
d_tokens = set(skill.description.lower().split())
if not d_tokens:
continue
overlap = len(q_tokens & d_tokens)
if overlap == 0:
continue
score = overlap / len(q_tokens | d_tokens)
scored.append((score, skill))
scored.sort(key=lambda x: -x[0])
return scored[:top_k]
def get(self, name: str) -> Skill | None:
return self._skills.get(name)
def topo_order(self, name: str) -> list[str]:
visited: set[str] = set()
order: list[str] = []
stack = [(name, False)]
while stack:
node, processed = stack.pop()
if processed:
order.append(node)
continue
if node in visited:
continue
visited.add(node)
stack.append((node, True))
skill = self._skills.get(node)
if skill is None:
continue
for dep in skill.depends_on:
if dep not in visited:
stack.append((dep, False))
return order
def execute(self, name: str, context: dict[str, Any] | None = None) -> dict[str, Any]:
if context is None:
context = {}
context.setdefault("log", [])
for skill_name in self.topo_order(name):
skill = self._skills.get(skill_name)
if skill is None:
context["log"].append(f"missing skill: {skill_name}")
context["failed"] = True
return context
try:
result = skill.fn(context)
context["log"].append(
f"ran {skill.name} v{skill.version}: {result}"
)
except Exception as e:
context["log"].append(
f"error in {skill.name} v{skill.version}: "
f"{type(e).__name__}: {e}"
)
context["failed"] = True
return context
context["failed"] = False
return context
def list_names(self) -> list[str]:
return sorted(self._skills)
def _mine(context: dict[str, Any]) -> str:
context["resources"] = context.get("resources", {})
context["resources"]["ore"] = context["resources"].get("ore", 0) + 3
return "+3 ore"
def _place_table(context: dict[str, Any]) -> str:
context["has_table"] = True
return "placed crafting table"
def _craft_iron_pick_v1(context: dict[str, Any]) -> str:
if not context.get("has_table"):
raise RuntimeError("no crafting table in context — cannot craft")
ore = context.get("resources", {}).get("ore", 0)
stick = context.get("resources", {}).get("stick", 0)
if ore < 3:
raise RuntimeError(f"need 3 ore, have {ore}")
if stick < 2:
raise RuntimeError(f"need 2 stick, have {stick}")
context["resources"]["ore"] -= 3
context["resources"]["stick"] -= 2
context["inventory"] = context.get("inventory", [])
context["inventory"].append("iron_pickaxe")
return "crafted iron_pickaxe"
def _craft_iron_pick_v2(context: dict[str, Any]) -> str:
if not context.get("has_table"):
return "skipped craft: no table yet"
ore = context.get("resources", {}).get("ore", 0)
stick = context.get("resources", {}).get("stick", 0)
if ore < 3 or stick < 2:
return f"skipped craft: ore={ore}, stick={stick}"
context["resources"]["ore"] -= 3
context["resources"]["stick"] -= 2
context["inventory"] = context.get("inventory", [])
context["inventory"].append("iron_pickaxe")
return "crafted iron_pickaxe"
def _gather_sticks(context: dict[str, Any]) -> str:
context["resources"] = context.get("resources", {})
context["resources"]["stick"] = context["resources"].get("stick", 0) + 2
return "+2 stick"
def main() -> None:
print("=" * 70)
print("VOYAGER SKILL LIBRARY — Phase 14, Lesson 10")
print("=" * 70)
lib = SkillLibrary()
print("\nphase 1: register primitive skills")
print(" " + lib.register(Skill(
name="mine_ore",
description="mine iron ore from nearby rock formations",
code="mine(3)",
fn=_mine,
tags=("gather", "ore"),
)))
print(" " + lib.register(Skill(
name="place_crafting_table",
description="place a crafting table at current position",
code="place_table()",
fn=_place_table,
tags=("setup", "crafting"),
)))
print(" " + lib.register(Skill(
name="gather_sticks",
description="gather sticks from tree or broken planks",
code="gather(2, stick)",
fn=_gather_sticks,
tags=("gather", "stick"),
)))
print("\nphase 2: compose a higher-order skill (v1)")
print(" " + lib.register(Skill(
name="craft_iron_pickaxe",
description="craft an iron pickaxe using ore and a crafting table",
code="mine_ore(); place_table(); craft('iron_pickaxe')",
fn=_craft_iron_pick_v1,
depends_on=("mine_ore", "place_crafting_table"),
tags=("craft", "tool"),
)))
print("\nphase 3: retrieval on 'I need a pickaxe'")
for score, skill in lib.search("I need a pickaxe"):
print(f" {score:.3f} {skill.name} v{skill.version}: {skill.description}")
print("\nphase 4: execute craft_iron_pickaxe (expect failure — missing sticks)")
context = lib.execute("craft_iron_pickaxe")
for line in context["log"]:
print(f" {line}")
print(f" failed: {context.get('failed')}")
print("\nphase 5: iterative refinement — rewrite as v2 with stick deps")
print(" " + lib.register(Skill(
name="craft_iron_pickaxe",
description="craft an iron pickaxe using ore, sticks, and a crafting table",
code="mine_ore(); gather_sticks(); place_table(); craft('iron_pickaxe')",
fn=_craft_iron_pick_v2,
depends_on=("mine_ore", "gather_sticks", "place_crafting_table"),
tags=("craft", "tool"),
)))
print("\nphase 6: re-execute (expect success)")
context = lib.execute("craft_iron_pickaxe")
for line in context["log"]:
print(f" {line}")
print(f" inventory: {context.get('inventory')}")
print(f" failed: {context.get('failed')}")
print("\nlibrary state")
for name in lib.list_names():
skill = lib.get(name)
assert skill is not None
print(f" {name} v{skill.version} deps={skill.depends_on} "
f"tags={skill.tags}")
print()
print("pattern: retrieve composable skills, execute, fold feedback into v2.")
print("same loop powers Claude Agent SDK skills and the skillkit registry.")
if __name__ == "__main__":
main()