Get Started

Capstone Lesson 25: Verification Gates and the Observation Budget

An agent harness without a verification layer is a wish in a trenchcoat. This lesson builds the deterministic gate chain that decides whether a tool call is allowed to fire, how much of its output the agent is allowed to see, and when the loop has to stop because the agent has read too much. The chain is a function of small, named gates plus an observation ledger that tracks every token the model has been shown.

90 minutesBuildPython (stdlib)Free lesson

Learning Objectives

  • Build a VerificationGate protocol with a deterministic evaluate(call) method.
  • Compose budget, recency, whitelist, and regex gates into a chain with short-circuit semantics.
  • Track every observation through an ObservationLedger keyed by tool and turn.
  • Refuse a tool call when the cumulative observation budget would be exceeded.
  • Surface a structured GateDecision record that downstream observability can ingest.

The Problem

When an agent harness lets the model call tools freely, three classes of bug appear within the first hour of real use.

The first is unbounded observation. A grep across a 200K-line repo dumps half a million tokens of output into the next turn. The model sees one match per kilobyte and the rest of the context is wasted. The token bill is large and the agent is now worse, not better, at the task.

The second is stale recency. A long-running task accumulates fifty tool calls. The model rereads the first read_file from turn three as if it were live state. Edits made on turn forty-seven never show up because the prompt builder serialized the earliest observations first.

The third is privilege creep. A research task starts by calling web_searchthen somehow ends up running shell because the model invented a tool name and the harness defaulted to permissive. By the time anyone reads the trace, a junk file is sitting in /tmp and a curl ran against a private API.

A verification gate is the harness component that says no. It is not a model. It is not a judge. It is a deterministic function of (call, history, ledger) that returns either ALLOW or DENY with a reason. The reason is logged. The model is told. The loop continues or aborts.

The Concept

A gate is anything with an evaluate(call, ctx) -> GateDecision method. The chain is an ordered list. Evaluation short-circuits on the first deny. Order matters: cheap structural gates run before expensive token-counting gates.

This lesson ships four gates:

  • WhitelistGate. Allowed tool names are an explicit set. Anything outside is denied. This is the cheapest gate and runs first.
  • RegexGate. Tool arguments are matched against a regex. Useful for refusing shell calls with rm -rf in them, or HTTP calls to internal IPs. Pure on the call payload.
  • RecencyGate. The model only sees observations from the last N turns. Older observations are masked. The gate refuses a tool call whose result would extend an observation window that has already aged out.
  • BudgetGate. The cumulative tokens the model has read across the session has a ceiling. When the ledger says the ceiling is reached, every further tool call is denied.

The observation ledger is the bookkeeping. Every successful tool call writes one row: tool name, turn, tokens emitted, cumulative. The ledger answers two questions: how much has the model seen total, and how much has it seen of tool X. The budget gate reads the first. A per-tool budget gate, which you will write as an exercise, reads the second.

Architecture

The harness asks the chain. The chain either nods or refuses. If it nods, the tool runs, the ledger ticks, and the result is appended to the message store. If it refuses, the model is handed the refusal as a system message and the loop decides whether to retry or abort.

What you will build

The implementation is a single main.py plus tests.

  1. Observation and ToolCall dataclasses define the wire shapes.
  2. ObservationLedger records (turn, tool, tokens) rows and answers cumulative() and per_tool(name).
  3. GateDecision carries (allow, reason, gate_name).
  4. VerificationGate is the protocol. Each gate implements evaluate(call, ctx).
  5. GateChain wraps an ordered list. It calls each gate, returns the first deny, or returns allow if every gate passes.
  6. The demo runs a tiny synthetic agent loop. Three turns. The third turn trips the budget gate and the loop reports a clean refusal with a non-zero refusal count.

The token counter is intentionally a stupid len(text) // 4 heuristic. The point of this lesson is the gate plumbing, not the tokenizer. Drop in a real tokenizer in production.

Why the chain order matters

A deny is cheaper than an allow. WhitelistGate runs in $O(1)$ hash lookup. RegexGate runs in $O(\text{pattern} \cdot \text{argv})$. RecencyGate reads a small slice of the message store. BudgetGate reads the entire ledger. You order them by ascending cost so a denied call short-circuits before doing the expensive work.

You also order them by blast radius. Whitelist is the strongest claim: this tool is not in the contract. The regex gate is next: this argument is not in the contract. Recency comes after: the harness still cares but the call is structurally legal. Budget is last because, by definition, it only fires when everything else passed.

How this composes with the rest of Track A

The previous lessons gave you the loop, the tool registry, the message store, the prompt builder, and the model router. This lesson adds the layer between the model and the tools. Lesson 26 ships the sandbox that the dispatcher hands the tool call to once the gate chain says ALLOW. Lesson 27 ships the eval harness that records refusal counts as a quality signal. Lesson 28 wires the gate decisions into OpenTelemetry spans. Lesson 29 stitches the lot into a working coding agent.

Running it

Terminal
cd phases/19-capstone-projects/25-verification-gates-observation-budget
python3 code/main.py
python3 -m pytest code/tests/ -v

The demo prints a turn-by-turn trace including every gate decision and exits zero. The tests cover the ledger, each gate in isolation, the chain short-circuit, and the synthetic loop end-to-end.

Check what you learned

6 questions. Answer each one to see why.

Question 1 of 60 correct

Before reading the lesson: which of these is the strongest reason a harness needs a deterministic gate chain rather than a model-as-judge?

The full code

Every file for this lesson. Copy it, run it, then change it.

main.py
"""
Verification gates and observation budget for an agent harness.

See: phases/19-capstone-projects/25-verification-gates-observation-budget/docs/en.md
Concept refs:
  - Gate-chain pattern (cheapest deny first, allow last).
  - Observation budget as a deterministic stopping criterion.
The demo at the bottom runs a synthetic three-turn loop and exits zero.
"""

from __future__ import annotations

import json
import re
import sys
from dataclasses import dataclass, field
from typing import Callable, Iterable, Protocol


# ---------------------------------------------------------------------------
# Wire shapes
# ---------------------------------------------------------------------------


@dataclass(frozen=True)
class ToolCall:
    """A request from the model to invoke a tool."""

    turn: int
    tool: str
    argv: tuple[str, ...]
    payload: str = ""

    def to_dict(self) -> dict:
        return {
            "turn": self.turn,
            "tool": self.tool,
            "argv": list(self.argv),
            "payload": self.payload,
        }


@dataclass(frozen=True)
class Observation:
    """The text the model is shown after a tool call."""

    turn: int
    tool: str
    text: str
    tokens: int

    def to_dict(self) -> dict:
        return {
            "turn": self.turn,
            "tool": self.tool,
            "tokens": self.tokens,
        }


@dataclass(frozen=True)
class GateDecision:
    """A single gate's verdict."""

    allow: bool
    gate: str
    reason: str

    def to_dict(self) -> dict:
        return {"allow": self.allow, "gate": self.gate, "reason": self.reason}


# ---------------------------------------------------------------------------
# Token estimator
# ---------------------------------------------------------------------------


def estimate_tokens(text: str) -> int:
    """A deterministic, conservative stand-in for a real tokenizer.

    Real harnesses plug in tiktoken or the model's own tokenizer.
    The gate chain only cares that the counter is monotonic and deterministic.
    """

    if not text:
        return 0
    return max(1, len(text) // 4)


# ---------------------------------------------------------------------------
# Observation ledger
# ---------------------------------------------------------------------------


@dataclass
class ObservationLedger:
    """Append-only ledger of every observation the model has been shown."""

    rows: list[Observation] = field(default_factory=list)

    def record(self, obs: Observation) -> None:
        self.rows.append(obs)

    def cumulative(self) -> int:
        return sum(row.tokens for row in self.rows)

    def per_tool(self, name: str) -> int:
        return sum(row.tokens for row in self.rows if row.tool == name)

    def turns_seen(self) -> list[int]:
        return sorted({row.turn for row in self.rows})

    def latest_turn(self) -> int:
        return self.rows[-1].turn if self.rows else -1

    def snapshot(self) -> list[dict]:
        return [row.to_dict() for row in self.rows]


# ---------------------------------------------------------------------------
# Gate protocol
# ---------------------------------------------------------------------------


@dataclass
class GateContext:
    """Read-only context passed to every gate."""

    ledger: ObservationLedger
    current_turn: int
    history: tuple[ToolCall, ...] = ()


class VerificationGate(Protocol):
    name: str

    def evaluate(self, call: ToolCall, ctx: GateContext) -> GateDecision: ...


# ---------------------------------------------------------------------------
# Concrete gates
# ---------------------------------------------------------------------------


@dataclass
class WhitelistGate:
    """Refuse any tool not in the explicit allow-set. Cheapest gate."""

    allowed: frozenset[str]
    name: str = "whitelist"

    def evaluate(self, call: ToolCall, ctx: GateContext) -> GateDecision:
        if call.tool in self.allowed:
            return GateDecision(True, self.name, "tool in allow-set")
        return GateDecision(
            False,
            self.name,
            f"tool {call.tool!r} not in allow-set {sorted(self.allowed)}",
        )


@dataclass
class RegexGate:
    """Refuse a call whose argv joins to a string matching any refuse pattern."""

    refuse_patterns: tuple[re.Pattern[str], ...]
    name: str = "regex"

    @classmethod
    def from_strings(cls, patterns: Iterable[str], name: str = "regex") -> "RegexGate":
        compiled = tuple(re.compile(p) for p in patterns)
        return cls(refuse_patterns=compiled, name=name)

    def evaluate(self, call: ToolCall, ctx: GateContext) -> GateDecision:
        haystack = " ".join(call.argv) + " " + call.payload
        for pat in self.refuse_patterns:
            if pat.search(haystack):
                return GateDecision(
                    False, self.name, f"argv matched refuse pattern {pat.pattern!r}"
                )
        return GateDecision(True, self.name, "no refuse pattern matched")


@dataclass
class RecencyGate:
    """Refuse a call if the last observation is more than window turns old.

    The intent is to force a fresh read instead of relying on stale state.
    The first call in a session always passes.
    """

    window: int
    name: str = "recency"

    def evaluate(self, call: ToolCall, ctx: GateContext) -> GateDecision:
        last = ctx.ledger.latest_turn()
        if last < 0:
            return GateDecision(True, self.name, "no prior observations")
        gap = call.turn - last
        if gap > self.window:
            return GateDecision(
                False,
                self.name,
                f"observation gap {gap} turns exceeds window {self.window}",
            )
        return GateDecision(True, self.name, f"gap {gap} within window {self.window}")


@dataclass
class BudgetGate:
    """Refuse a call once the cumulative observation budget is exhausted.

    A single call cannot in advance know how many tokens its result will be.
    The gate is therefore evaluated against the ledger as it stands before the
    call, and the harness re-runs the cumulative check against the new ledger
    state after recording the observation.
    """

    max_tokens: int
    name: str = "budget"

    def evaluate(self, call: ToolCall, ctx: GateContext) -> GateDecision:
        used = ctx.ledger.cumulative()
        if used >= self.max_tokens:
            return GateDecision(
                False,
                self.name,
                f"observation budget exhausted: {used}/{self.max_tokens} tokens",
            )
        remaining = self.max_tokens - used
        return GateDecision(
            True, self.name, f"{remaining} tokens of budget remaining"
        )


@dataclass
class PerToolBudgetGate:
    """Optional gate: refuse if a single tool has consumed more than its share."""

    limits: dict[str, int]
    name: str = "per-tool-budget"

    def evaluate(self, call: ToolCall, ctx: GateContext) -> GateDecision:
        limit = self.limits.get(call.tool)
        if limit is None:
            return GateDecision(True, self.name, "tool has no per-tool budget")
        used = ctx.ledger.per_tool(call.tool)
        if used >= limit:
            return GateDecision(
                False,
                self.name,
                f"per-tool budget for {call.tool} exhausted: {used}/{limit}",
            )
        return GateDecision(
            True, self.name, f"per-tool {call.tool}: {limit - used} tokens remaining"
        )


# ---------------------------------------------------------------------------
# Gate chain
# ---------------------------------------------------------------------------


@dataclass
class ChainOutcome:
    """The full result of a chain evaluation: the per-gate decisions plus a final verdict."""

    decisions: list[GateDecision]

    @property
    def allow(self) -> bool:
        return all(d.allow for d in self.decisions)

    @property
    def deny_reason(self) -> str | None:
        for d in self.decisions:
            if not d.allow:
                return f"[{d.gate}] {d.reason}"
        return None

    def to_dict(self) -> dict:
        return {
            "allow": self.allow,
            "deny_reason": self.deny_reason,
            "decisions": [d.to_dict() for d in self.decisions],
        }


@dataclass
class GateChain:
    """Ordered list of gates evaluated with short-circuit on first deny."""

    gates: tuple[VerificationGate, ...]

    def evaluate(self, call: ToolCall, ctx: GateContext) -> ChainOutcome:
        decisions: list[GateDecision] = []
        for gate in self.gates:
            decision = gate.evaluate(call, ctx)
            decisions.append(decision)
            if not decision.allow:
                return ChainOutcome(decisions=decisions)
        return ChainOutcome(decisions=decisions)


# ---------------------------------------------------------------------------
# Mini synthetic agent loop for the demo
# ---------------------------------------------------------------------------


ToolFn = Callable[[ToolCall], str]


@dataclass
class LoopReport:
    """Audit record of a synthetic loop run."""

    turns: int
    allowed: int
    refused: int
    observations: list[Observation]
    decisions: list[ChainOutcome]

    def to_dict(self) -> dict:
        return {
            "turns": self.turns,
            "allowed": self.allowed,
            "refused": self.refused,
            "observations": [o.to_dict() for o in self.observations],
            "decisions": [d.to_dict() for d in self.decisions],
        }


def run_synthetic_loop(
    calls: list[ToolCall],
    chain: GateChain,
    tool_fns: dict[str, ToolFn],
) -> LoopReport:
    """Run a fixed sequence of tool calls through the chain.

    This is the harness skeleton in miniature. A real harness would consult
    the model for the next tool call; the gate-chain contract is identical.
    """

    ledger = ObservationLedger()
    decisions: list[ChainOutcome] = []
    observations: list[Observation] = []
    allowed = 0
    refused = 0

    history: list[ToolCall] = []

    for call in calls:
        ctx = GateContext(
            ledger=ledger, current_turn=call.turn, history=tuple(history)
        )
        outcome = chain.evaluate(call, ctx)
        decisions.append(outcome)
        history.append(call)
        if not outcome.allow:
            refused += 1
            continue
        fn = tool_fns.get(call.tool)
        if fn is None:
            refused += 1
            continue
        result = fn(call)
        obs = Observation(
            turn=call.turn,
            tool=call.tool,
            text=result,
            tokens=estimate_tokens(result),
        )
        ledger.record(obs)
        observations.append(obs)
        allowed += 1

    return LoopReport(
        turns=len(calls),
        allowed=allowed,
        refused=refused,
        observations=observations,
        decisions=decisions,
    )


# ---------------------------------------------------------------------------
# Demo wiring
# ---------------------------------------------------------------------------


def _demo_tools() -> dict[str, ToolFn]:
    """Three synthetic tools. read_file is verbose, list_dir is small, run_tests is structured."""

    def read_file(call: ToolCall) -> str:
        target = call.argv[0] if call.argv else "<missing>"
        return (
            f"# fake contents of {target}\n"
            + ("line of fake source code that is sixty bytes long " * 12)
        )

    def list_dir(call: ToolCall) -> str:
        return "main.py\nREADME.md\ntests/test_main.py\n"

    def run_tests(call: ToolCall) -> str:
        return json.dumps(
            {"status": "passed", "tests": 4, "duration_ms": 42}, indent=2
        )

    return {"read_file": read_file, "list_dir": list_dir, "run_tests": run_tests}


def build_default_chain(budget: int = 200) -> GateChain:
    """Wire the canonical four-gate chain in the order documented in en.md."""

    return GateChain(
        gates=(
            WhitelistGate(
                allowed=frozenset({"read_file", "list_dir", "run_tests"})
            ),
            RegexGate.from_strings(
                patterns=(
                    r"\brm\s+-rf\b",
                    r"\bsudo\b",
                    r"^/etc/",
                )
            ),
            RecencyGate(window=3),
            BudgetGate(max_tokens=budget),
        )
    )


def run_demo() -> int:
    """Self-terminating demo. Prints a JSON trace and exits zero."""

    chain = build_default_chain(budget=200)
    tools = _demo_tools()

    calls = [
        ToolCall(turn=1, tool="list_dir", argv=("./",)),
        ToolCall(turn=2, tool="read_file", argv=("main.py",)),
        ToolCall(turn=3, tool="read_file", argv=("README.md",)),
        ToolCall(turn=4, tool="run_tests", argv=("./",)),
        ToolCall(turn=5, tool="shell", argv=("rm", "-rf", "/")),
    ]

    report = run_synthetic_loop(calls, chain, tools)

    print("VERIFICATION GATE DEMO")
    print(f"turns={report.turns} allowed={report.allowed} refused={report.refused}")
    print("")
    for idx, (call, outcome) in enumerate(zip(calls, report.decisions)):
        verdict = "ALLOW" if outcome.allow else "DENY"
        print(f"  [{idx}] turn={call.turn} tool={call.tool} -> {verdict}")
        if not outcome.allow:
            print(f"        reason: {outcome.deny_reason}")
    print("")
    print(f"cumulative tokens observed: {sum(o.tokens for o in report.observations)}")
    print(f"observations recorded: {len(report.observations)}")

    if report.refused < 1:
        print("ERROR: demo expected at least one refusal", file=sys.stderr)
        return 1
    return 0


if __name__ == "__main__":
    sys.exit(run_demo())