Get Started

Failure Modes: Why Agents Break

MASFT (Berkeley, 2025) catalogs 14 multi-agent failure modes in 3 categories. Microsoft's Taxonomy documents how existing AI failures amplify in agentic settings. Industry field data converges on five recurring modes: hallucinated actions, scope creep, cascading errors, context loss, tool misuse.

60 minutesLearn + BuildPython (stdlib)Free lesson

Learning Objectives

  • Name MASFT's three failure categories and at least four specific modes in each.
  • Explain why agentic failure amplifies existing AI failure modes (bias, hallucination).
  • Describe the five industry-recurring modes and their mitigations.
  • Implement a stdlib detector that tags agent traces with failure-mode labels.

The Problem

Teams ship agents that work on 90% of traces. The 10% failures are not random noise, they fall into a small number of recurring categories. Once you can name them, you can monitor for them and fix them.

The Concept

MASFT (Berkeley, arXiv:2503.13657)

Multi-Agent System Failure Taxonomy. 14 failure modes clustered into 3 categories. Inter-annotator Cohen's Kappa 0.88, the categories are reliably distinguishable.

Central claim: failures are fundamental design flaws in multi-agent systems, not LLM limitations to be fixed with better base models.

Microsoft Taxonomy of Failure Mode in Agentic AI Systems

  • Existing AI failures (bias, hallucination, data leakage) amplify in agentic settings.
  • New failures emerge from autonomy: unintended action at scale, tool misuse, mission drift.
  • The whitepaper is the risk register for agentic products.

Characterizing Faults in Agentic AI (arXiv:2603.06847)

  • Failures arise from orchestration, internal state evolution, and environment interaction.
  • Not just "bad code" or "bad model output."

LLM Agent Hallucinations Survey (arXiv:2509.18970)

Two primary manifestations:

  1. Instruction-following Deviationagent doesn't follow the system prompt.
  2. Long-range Contextual Misuseagent forgets or misapplies context from earlier turns.

Sub-intention errors: Omission (missed step), Redundancy (repeated step), Disorder (out-of-order steps).

The five industry-recurring modes

Arize, Galileo, NimbleBrain 2024-2026 field analyses converge on:

  1. Hallucinated actions. Agent invokes a tool that doesn't exist or fabricates arguments.
  2. Scope creep. Agent expands task beyond the user's ask (creates extra PRs, sends extra emails).
  3. Cascading errors. One wrong call triggers downstream effects. A phantom SKU hallucination triggers four API calls, a multi-system incident.
  4. Context loss. Long-horizon tasks forget early-turn constraints.
  5. Tool misuse. Calls the right tool with wrong arguments, or the wrong tool entirely.

Cascading is the killer. Agents cannot distinguish "I failed" from "the task is impossible" and often hallucinate a success message on 400 errors to close the loop.

Mitigation: gates at every step

Automated verification gates at every step of a reasoning chain, checking factual grounding against environment state. Concretely:

  • Per-step safety classifier (Lesson 21).
  • Tool-call argument validation (Lesson 06).
  • Cross-check retrieved content against known facts (Lesson 05, CRITIC).
  • Detect success hallucination by re-probing state (was the file actually created?).

Where failure monitoring goes wrong

  • Tagging only crashes. Most agent failures produce valid-looking output. Need content-level checks.
  • No baseline. Drift detection needs a last-known-good; without it you cannot say "this is getting worse."
  • Over-alerting. Every failure produces a page. Cluster and rate-limit.

Build It

code/main.py implements a stdlib failure-mode tagger:

  • A synthetic trace dataset covering the five modes.
  • Detector functions per mode (signature patterns on tool calls, outputs, repeat actions).
  • A tagger that labels each trace and reports mode distribution.

Run it:

Code
python3 code/main.py

Output: per-trace labels + aggregate distribution, a cheap reproduction of what Phoenix's trace clustering surfaces.

Use It

  • Phoenix for production drift clustering (Lesson 24).
  • Langfuse for session replay + annotation.
  • Custom for domain-specific signatures your observability platform can't detect.

Ship It

outputs/skill-failure-detector.md generates failure-mode detectors tailored to your domain, wired to a trace store.

Exercises

  1. Add a detector for "success hallucination": agent returns success but the target state is unchanged.
  2. Tag 100 real traces from a product you've built. Which mode dominates? What's the cost of fixing it?
  3. Implement a "cascade radius" metric: given a failure at step N, how many downstream steps did it affect?
  4. Read MASFT's 14 failure modes. Pick three that apply to your product. Write detectors.
  5. Wire one detector into a CI job: fail the build if >=5% of traces tag a mode.

Key Terms

TermWhat people sayWhat it actually means
MASFT"Multi-agent failure taxonomy"Berkeley 14-mode categorization
Cascading error"Ripple failure"One early mistake propagates through N steps
Context loss"Forgot the constraint"Long-horizon turn drops early-turn facts
Tool misuse"Wrong tool / wrong args"Valid call, wrong invocation
Success hallucination"Faked completion"Agent claims success on a 400; state unchanged
Scope creep"Overreach"Agent does more than asked
Instruction-following deviation"Disobedience"Ignores system prompt or user constraint
Sub-intention errors"Plan bugs"Omission, redundancy, disorder in plan execution

Further Reading

Check what you learned

7 questions. Answer each one to see why.

Question 1 of 70 correct

What is MASFT's central claim?

The full code

Every file for this lesson. Copy it, run it, then change it.

main.py
"""Stdlib failure-mode tagger for agent traces.

Detects the five industry-recurring modes: hallucinated actions, scope creep,
cascading errors, context loss, tool misuse. Each detector returns a tag if
the trace matches; aggregate distribution mirrors Phoenix's trace clustering.
"""

from __future__ import annotations

from collections import Counter
from dataclasses import dataclass, field
from typing import Any


@dataclass
class TraceStep:
    kind: str
    name: str
    args: dict[str, Any] = field(default_factory=dict)
    status: str = "ok"
    result: str = ""


@dataclass
class Trace:
    tid: str
    user_request: str
    constraints: list[str]
    steps: list[TraceStep]
    final_success_claim: bool
    target_state_changed: bool


KNOWN_TOOLS = {"search", "read_file", "write_file", "list_dir"}


def detect_hallucinated_action(trace: Trace) -> str | None:
    for step in trace.steps:
        if step.kind == "tool_call" and step.name not in KNOWN_TOOLS:
            return "hallucinated_action"
    return None


def detect_scope_creep(trace: Trace) -> str | None:
    request = trace.user_request.lower()
    writes = [s for s in trace.steps
              if s.kind == "tool_call" and s.name == "write_file"]
    explicit_write_words = ("write", "create", "save", "update", "edit")
    wanted_write = any(w in request for w in explicit_write_words)
    if len(writes) > 0 and not wanted_write:
        return "scope_creep"
    return None


def detect_cascading_errors(trace: Trace) -> str | None:
    saw_error = False
    downstream_ops = 0
    for step in trace.steps:
        if step.kind == "tool_call" and step.status == "error":
            saw_error = True
            continue
        if saw_error and step.kind == "tool_call":
            downstream_ops += 1
    if saw_error and downstream_ops >= 2:
        return "cascading_errors"
    return None


def detect_context_loss(trace: Trace) -> str | None:
    for constraint in trace.constraints:
        con_l = constraint.lower()
        if "do not" in con_l:
            forbidden_token = con_l.split("do not")[-1].strip().split()[0]
            for step in trace.steps:
                if step.kind == "tool_call" and forbidden_token in str(step.args).lower():
                    return "context_loss"
    return None


def detect_tool_misuse(trace: Trace) -> str | None:
    tool_args_schema = {
        "read_file": {"path"},
        "write_file": {"path", "content"},
        "list_dir": {"path"},
        "search": {"query"},
    }
    for step in trace.steps:
        if step.kind != "tool_call":
            continue
        expected = tool_args_schema.get(step.name)
        if expected is None:
            continue
        if not expected.issubset(set(step.args.keys())):
            return "tool_misuse"
    return None


def detect_success_hallucination(trace: Trace) -> str | None:
    request = trace.user_request.lower()
    write_intent = any(w in request for w in
                       ("write", "create", "save", "update", "edit", "make"))
    if (write_intent and trace.final_success_claim
            and not trace.target_state_changed):
        return "success_hallucination"
    return None


DETECTORS = (
    detect_hallucinated_action,
    detect_scope_creep,
    detect_cascading_errors,
    detect_context_loss,
    detect_tool_misuse,
    detect_success_hallucination,
)


def tag(trace: Trace) -> list[str]:
    return [label for label in (d(trace) for d in DETECTORS) if label]


def main() -> None:
    print("=" * 70)
    print("AGENT FAILURE MODES — Phase 14, Lesson 26")
    print("=" * 70)

    traces = [
        Trace(tid="t001", user_request="find the config file",
              constraints=["do not modify any files"],
              steps=[
                  TraceStep("tool_call", "search", {"query": "config"}),
                  TraceStep("tool_call", "read_file", {"path": "config.yml"}),
              ],
              final_success_claim=True, target_state_changed=False),
        Trace(tid="t002", user_request="find the config file",
              constraints=["do not modify any files"],
              steps=[
                  TraceStep("tool_call", "search", {"query": "config"}),
                  TraceStep("tool_call", "write_file",
                            {"path": "config.yml", "content": "..."}),
              ],
              final_success_claim=True, target_state_changed=True),
        Trace(tid="t003", user_request="list project files",
              constraints=[],
              steps=[
                  TraceStep("tool_call", "magic_scanner",
                            {"path": "/"}),
              ],
              final_success_claim=True, target_state_changed=False),
        Trace(tid="t004", user_request="look up invoice 4711",
              constraints=[],
              steps=[
                  TraceStep("tool_call", "search",
                            {"query": "invoice 4711"}, status="error"),
                  TraceStep("tool_call", "read_file", {"path": "/tmp/foo"}),
                  TraceStep("tool_call", "write_file",
                            {"path": "/tmp/foo", "content": "fabricated"}),
                  TraceStep("tool_call", "list_dir", {"path": "/tmp"}),
              ],
              final_success_claim=True, target_state_changed=True),
        Trace(tid="t005", user_request="update readme with release notes",
              constraints=["do not modify src/"],
              steps=[
                  TraceStep("tool_call", "read_file", {"path": "README.md"}),
                  TraceStep("tool_call", "write_file",
                            {"path": "README.md", "content": "notes"}),
                  TraceStep("tool_call", "write_file",
                            {"path": "src/foo.py", "content": "also notes"}),
              ],
              final_success_claim=True, target_state_changed=True),
        Trace(tid="t006", user_request="read some file",
              constraints=[],
              steps=[
                  TraceStep("tool_call", "read_file",
                            {"file": "/tmp/foo"}),
              ],
              final_success_claim=False, target_state_changed=False),
        Trace(tid="t007", user_request="create a PR",
              constraints=[],
              steps=[
                  TraceStep("tool_call", "search", {"query": "PR template"}),
              ],
              final_success_claim=True, target_state_changed=False),
    ]

    distribution: Counter = Counter()
    print()
    for trace in traces:
        labels = tag(trace)
        distribution.update(labels)
        print(f"  {trace.tid}  user={trace.user_request[:40]!r}")
        print(f"    labels: {labels if labels else '[clean]'}")

    print("\naggregate distribution")
    for label, count in distribution.most_common():
        print(f"  {label}: {count}")
    print()
    print("gate at every step: classifier, argument validation, state probe.")
    print("cascading is the killer. detect early, stop the loop.")


if __name__ == "__main__":
    main()