Get Started

Capstone 09, Code Migration Agent (Repo-Level Language / Runtime Upgrade)

Amazon's MigrationBench (Java 8 to 17) and Google's App Engine Py2-to-Py3 migrator set the 2026 bar. Moderne's OpenRewrite does deterministic AST rewrites at scale. Grit targets the same problem with codemod-style DSL. The production pattern combines both: a deterministic substrate for safe rewrites plus an agent layer for the ambiguous cases, a sandbox for per-branch builds, and a test harness that flips green before the PR opens. The capstone is to migrate 50 real repos and publish a pass rate with a failure taxonomy.

30 hoursCapstonePython (agent), Java / Python (targets), TypeScript (dashboard)Free lesson

Problem

Large-scale code migration is one of the cleanest production applications of 2026 coding agents. The ground truth is obvious (does the test suite pass after the migration?), the rewards are real (a Java-8 fleet migration is a headcount-scale project), and the benchmarks are public (MigrationBench 50-repo subset). Moderne's OpenRewrite handles the deterministic side. The agent layer handles everything OpenRewrite recipes cannot: ambiguous rewrites, build-system drift, long-tail syntax, transitive dependency breakage.

You will build an agent that takes a Java 8 repo (or Python 2 repo) and produces a green-CI migrated branch. You will measure pass rate, test-coverage preservation, cost per repo, and build a failure taxonomy. The side-by-side against a deterministic-only baseline tells you where the agent's value actually lives.

Concept

The pipeline has two layers. The deterministic substrate (OpenRewrite for Java, libcst for Python) runs the bulk of mechanical rewrites safely: imports, method signatures, null-safety edits, try-with-resources, deprecated API replacements. It is fast and produces auditable diffs. The agent layer (OpenAI Agents SDK or LangGraph over Claude Opus 4.7 and GPT-5.4-Codex) handles cases the recipes cannot: build-file upgrades (Maven/Gradle/pyproject), transitive dependency conflicts, test flakes, custom annotations.

Each repo gets a Daytona sandbox with the target runtime preinstalled. The agent iterates: run build, classify failures, apply fix, rerun. Hard limits: 30 minutes per repo, $8 per repo, 20 agent turns. If all tests pass and the coverage delta is not negative, the branch opens a PR. If not, the repo gets filed under a failure class with evidence.

The failure taxonomy is the deliverable. Across 50 repos, what broke? Transitive deps? Custom annotations? Build tool version? Test flakes unrelated to migration? Each class gets a count and an exemplar diff. Future recipe authors can target the top three.

Architecture

Code
target repo
      |
      v
OpenRewrite / libcst deterministic recipes
   (safe, fast, auditable, ~70-80% of fixes)
      |
      v
Daytona sandbox per branch
      |
      v
agent loop (Claude Opus 4.7 / GPT-5.4-Codex):
   - run build -> capture failures
   - classify failures (build, test, lint)
   - apply fix (patch or retry recipe)
   - rerun
   - budget: 30 min, $8, 20 turns
      |
      v
test + coverage delta gate
      |
      v (passed)
open PR
      |
      v (failed)
file under failure class + attach repro

Stack

  • Deterministic substrate: OpenRewrite (Java) or libcst (Python)
  • Agent: OpenAI Agents SDK or LangGraph over Claude Opus 4.7 + GPT-5.4-Codex
  • Sandbox: Daytona devcontainers per branch, pre-installed target runtime (Java 17 / Python 3.12)
  • Build systems: Maven, Gradle, uv (Python)
  • Benchmarks: Amazon MigrationBench 50-repo subset (Java 8 to 17), Google App Engine Py2-to-Py3 repos
  • Test harness: parallel runner, coverage via Jacoco (Java) or coverage.py (Python)
  • Observability: Langfuse + trace bundle per repo with every diff chunk
  • Dashboard: failure-taxonomy dashboard with per-class counts and exemplar diffs

Build It

  1. Recipe pass. Run OpenRewrite (Java) or libcst (Python) recipes first. Catch the 70-80% of migrations that are mechanical. Commit as "recipe" commit.
  2. Build trial. Daytona sandbox: install target runtime, run the build. If green, skip to tests. If red, hand off to agent.
  3. Agent loop. LangGraph with tools: run_buildread_fileedit_filerun_testgit_diff. Agent classifies the failure (dep, syntax, test, build-tool) and applies a targeted fix. Rerun.
  4. Budget caps. 30 minutes wall-clock per repo, $8 cost, 20 agent turns. Any breach halts and files under "budget_exhausted" with the current diff.
  5. Test + coverage gate. After the build goes green, run the test suite. Compare coverage to the base repo. If coverage dropped more than 2%, file under "coverage_regression".
  6. PR open. On success, push the branch, open the PR with the diff and a summary of which recipes applied and which commits the agent authored.
  7. Failure taxonomy. For each failed repo, tag with a class: dep_upgrade_requiredbuild_tool_driftcustom_annotationtest_flakesyntax_edge_casebudget_exhausted. Build a dashboard.
  8. 50-repo run. Execute across the MigrationBench subset. Report per-class pass rate, cost-per-repo, coverage-preservation, and a compare-vs-deterministic-only baseline.

Use It

Code
$ migrate legacy-java-service --target java17
[recipe]   27 rewrites applied (JUnit 4->5, HashMap initializer, try-with-resources)
[build]    FAIL: cannot find symbol sun.misc.BASE64Encoder
[agent]    turn 1 classify: removed_jdk_api
[agent]    turn 2 apply: sun.misc.BASE64Encoder -> java.util.Base64
[build]    OK
[tests]    412/412 passing; coverage 84.1% -> 84.3%
[pr]       opened #1841  cost=$3.20  turns=4

Ship It

outputs/skill-migration-agent.md is the deliverable. Given a repo, it executes deterministic recipes then an agent loop to produce a green migrated branch, or files the repo under a taxonomy class.

WeightCriterionHow it is measured
25MigrationBench pass rate50-repo subset pass@1
20Test-coverage preservationMean coverage delta vs base
20Cost per migrated repo$/repo on passing runs
20Agent / deterministic-tool integrationFraction of fixes that OpenRewrite handled vs agent authored
15Failure analysis write-upTaxonomy completeness with exemplars
100

Exercises

  1. Run the migrate pipeline with OpenRewrite only (no agent). Compare pass rate to the full pipeline. Identify the cases where the agent alone is the difference.
  2. Implement a "lint-clean" check: after migration, run a style linter (spotless for Java, ruff for Python). Fail the PR if new lint errors appear. Measure the coverage-preserved-but-style-regressed rate.
  3. Add a "minimal-diff" optimizer: after the agent's branch passes tests, trim unnecessary changes with a second pass. Report diff-size reduction.
  4. Extend to a third migration: Node 18 to Node 22. Reuse the sandbox wrapping; swap the recipe layer for a custom codemod.
  5. Measure time-to-first-green-build (TTFGB) as a UX metric. Target: p50 under 10 minutes.

Key Terms

TermWhat people sayWhat it actually means
Deterministic substrate"Recipe engine"OpenRewrite / libcst: declarative AST rewrites with safety guarantees
Codemod"Code-modifying program"A rewrite rule that changes source code mechanically
Build drift"Tool version skew"Subtle Maven / Gradle / uv behavior changes between major versions
Failure class"Taxonomy bucket"A labeled reason a repo did not migrate: dep, syntax, test, build-tool, budget
Coverage delta"Coverage preservation"Change in test coverage % from base to migrated branch
Agent turn"Tool-call round"One plan -> act -> observe cycle in the agent loop
Budget exhaustion"Hit the ceiling"The repo consumed its 30-min / $8 / 20-turn limit without passing

Further Reading

Check what you learned

7 questions. Answer each one to see why.

Question 1 of 70 correct

Why does the pipeline combine a deterministic substrate with an agent layer rather than using just one?

The full code

Every file for this lesson. Copy it, run it, then change it.

main.py
"""Code migration agent — deterministic recipes + agent-loop fallback scaffold.

The hard architectural primitive is the two-layer structure: deterministic
recipe pass first (fast, auditable, safe), then agent loop for remaining
failures with a hard budget and a failure-classification step that feeds a
taxonomy dashboard. This scaffold implements both layers and runs a
50-repo simulation with mixed outcomes.

Run:  python main.py
"""

from __future__ import annotations

import random
from dataclasses import dataclass, field


# ---------------------------------------------------------------------------
# repo + failure taxonomy
# ---------------------------------------------------------------------------

FAILURE_CLASSES = [
    "dep_upgrade_required",
    "build_tool_drift",
    "custom_annotation",
    "test_flake",
    "syntax_edge_case",
    "budget_exhausted",
    "coverage_regression",
]


@dataclass
class Repo:
    name: str
    loc: int
    lang: str          # "java" | "python"
    hardness: float    # 0..1


@dataclass
class Attempt:
    repo: Repo
    recipe_applied: int = 0
    agent_turns: int = 0
    cost_usd: float = 0.0
    wall_min: float = 0.0
    status: str = "pending"  # "pass" | "fail"
    failure_class: str | None = None
    coverage_base: float = 80.0
    coverage_final: float = 80.0


# ---------------------------------------------------------------------------
# deterministic recipe pass  --  OpenRewrite / libcst stand-in
# ---------------------------------------------------------------------------

def run_recipes(repo: Repo) -> int:
    """Returns number of rewrites applied."""
    base = 20 + int(repo.loc / 500)
    return int(base * (1 - 0.2 * repo.hardness))


# ---------------------------------------------------------------------------
# agent loop  --  classify failure, apply fix, retry; budget-aware
# ---------------------------------------------------------------------------

BUDGET_MIN = 30.0
BUDGET_USD = 8.0
BUDGET_TURNS = 20


def agent_loop(attempt: Attempt, rng: random.Random) -> None:
    """Simulates the plan-act loop until pass or budget exhaustion."""
    # cost per turn drifts with hardness
    per_turn_min = 2.8 + attempt.repo.hardness * 2.0
    per_turn_usd = 0.45 + attempt.repo.hardness * 0.65

    # probability of passing per turn depends on hardness (0.02-0.18)
    turn_pass_p = max(0.02, 0.22 * (1 - attempt.repo.hardness * 0.95))

    while True:
        if attempt.agent_turns >= BUDGET_TURNS:
            attempt.status = "fail"
            attempt.failure_class = "budget_exhausted"
            return
        if attempt.wall_min >= BUDGET_MIN or attempt.cost_usd >= BUDGET_USD:
            attempt.status = "fail"
            attempt.failure_class = "budget_exhausted"
            return

        attempt.agent_turns += 1
        attempt.wall_min += per_turn_min
        attempt.cost_usd += per_turn_usd

        if rng.random() < turn_pass_p:
            # coverage check
            delta = rng.gauss(0.0, 0.6)
            attempt.coverage_final = attempt.coverage_base + delta
            if attempt.coverage_final < attempt.coverage_base - 2.0:
                attempt.status = "fail"
                attempt.failure_class = "coverage_regression"
                return
            attempt.status = "pass"
            return


# ---------------------------------------------------------------------------
# classification of stuck repos  --  bucket into taxonomy
# ---------------------------------------------------------------------------

def classify_failure(rng: random.Random) -> str:
    """Stand-in for the agent's failure classifier. Real implementation
    reads build logs and test output."""
    weights = {
        "dep_upgrade_required": 0.30,
        "build_tool_drift": 0.20,
        "custom_annotation": 0.18,
        "test_flake": 0.15,
        "syntax_edge_case": 0.17,
    }
    r = rng.random()
    acc = 0.0
    for cls, w in weights.items():
        acc += w
        if r <= acc:
            return cls
    return "syntax_edge_case"


# ---------------------------------------------------------------------------
# pipeline  --  recipes then agent then PR/file outcome
# ---------------------------------------------------------------------------

def migrate(repo: Repo, rng: random.Random) -> Attempt:
    attempt = Attempt(repo=repo)
    attempt.recipe_applied = run_recipes(repo)

    # easy repos often go straight to pass after recipes
    straight_through_p = 0.55 * (1 - repo.hardness)
    if rng.random() < straight_through_p:
        delta = rng.gauss(0.0, 0.4)
        attempt.coverage_final = attempt.coverage_base + delta
        attempt.status = "pass"
        attempt.wall_min = 3.0 + rng.random() * 4
        attempt.cost_usd = 0.30
        return attempt

    # otherwise run the agent loop
    agent_loop(attempt, rng)

    if attempt.status == "fail" and attempt.failure_class == "budget_exhausted":
        # classify root cause of why the budget was exhausted
        if rng.random() < 0.75:
            attempt.failure_class = classify_failure(rng)
    return attempt


# ---------------------------------------------------------------------------
# 50-repo simulation
# ---------------------------------------------------------------------------

def synth_bench(rng: random.Random) -> list[Repo]:
    bench: list[Repo] = []
    for i in range(50):
        lang = "java" if rng.random() < 0.6 else "python"
        hardness = min(0.95, max(0.05, rng.gauss(0.65, 0.18)))
        bench.append(Repo(name=f"repo-{i:02d}-{lang}",
                          loc=rng.randint(800, 40_000),
                          lang=lang,
                          hardness=hardness))
    return bench


def main() -> None:
    rng = random.Random(19)
    bench = synth_bench(rng)

    results: list[Attempt] = []
    for repo in bench:
        results.append(migrate(repo, rng))

    passed = [a for a in results if a.status == "pass"]
    failed = [a for a in results if a.status == "fail"]

    print(f"=== migration-bench run (50 repos) ===")
    print(f"passed : {len(passed):2d}  ({len(passed) / 50:.1%})")
    print(f"failed : {len(failed):2d}")

    print("\nfailure taxonomy:")
    taxonomy: dict[str, int] = {}
    for a in failed:
        taxonomy[a.failure_class or "unknown"] = taxonomy.get(a.failure_class or "unknown", 0) + 1
    for cls, n in sorted(taxonomy.items(), key=lambda x: -x[1]):
        print(f"  {cls:24s} {n}")

    if passed:
        mean_cost = sum(a.cost_usd for a in passed) / len(passed)
        mean_min = sum(a.wall_min for a in passed) / len(passed)
        mean_turns = sum(a.agent_turns for a in passed) / len(passed)
        mean_cov_delta = sum(a.coverage_final - a.coverage_base for a in passed) / len(passed)
        print("\npass-set metrics:")
        print(f"  mean $/repo     : ${mean_cost:.2f}")
        print(f"  mean wall min   : {mean_min:.1f}")
        print(f"  mean agent turns: {mean_turns:.1f}")
        print(f"  mean cov delta  : {mean_cov_delta:+.2f} points")


if __name__ == "__main__":
    main()