Failure Modes: Why Agents Break
MASFT (Berkeley, 2025) catalogs 14 multi-agent failure modes in 3 categories. Microsoft's Taxonomy documents how existing AI failures amplify in agentic settings. Industry field data converges on five recurring modes: hallucinated actions, scope creep, cascading errors, context loss, tool misuse.
Learning Objectives
- Name MASFT's three failure categories and at least four specific modes in each.
- Explain why agentic failure amplifies existing AI failure modes (bias, hallucination).
- Describe the five industry-recurring modes and their mitigations.
- Implement a stdlib detector that tags agent traces with failure-mode labels.
The Problem
Teams ship agents that work on 90% of traces. The 10% failures are not random noise, they fall into a small number of recurring categories. Once you can name them, you can monitor for them and fix them.
The Concept
MASFT (Berkeley, arXiv:2503.13657)
Multi-Agent System Failure Taxonomy. 14 failure modes clustered into 3 categories. Inter-annotator Cohen's Kappa 0.88, the categories are reliably distinguishable.
Central claim: failures are fundamental design flaws in multi-agent systems, not LLM limitations to be fixed with better base models.
Microsoft Taxonomy of Failure Mode in Agentic AI Systems
- Existing AI failures (bias, hallucination, data leakage) amplify in agentic settings.
- New failures emerge from autonomy: unintended action at scale, tool misuse, mission drift.
- The whitepaper is the risk register for agentic products.
Characterizing Faults in Agentic AI (arXiv:2603.06847)
- Failures arise from orchestration, internal state evolution, and environment interaction.
- Not just "bad code" or "bad model output."
LLM Agent Hallucinations Survey (arXiv:2509.18970)
Two primary manifestations:
- Instruction-following Deviationagent doesn't follow the system prompt.
- Long-range Contextual Misuseagent forgets or misapplies context from earlier turns.
Sub-intention errors: Omission (missed step), Redundancy (repeated step), Disorder (out-of-order steps).
The five industry-recurring modes
Arize, Galileo, NimbleBrain 2024-2026 field analyses converge on:
- Hallucinated actions. Agent invokes a tool that doesn't exist or fabricates arguments.
- Scope creep. Agent expands task beyond the user's ask (creates extra PRs, sends extra emails).
- Cascading errors. One wrong call triggers downstream effects. A phantom SKU hallucination triggers four API calls, a multi-system incident.
- Context loss. Long-horizon tasks forget early-turn constraints.
- Tool misuse. Calls the right tool with wrong arguments, or the wrong tool entirely.
Cascading is the killer. Agents cannot distinguish "I failed" from "the task is impossible" and often hallucinate a success message on 400 errors to close the loop.
Mitigation: gates at every step
Automated verification gates at every step of a reasoning chain, checking factual grounding against environment state. Concretely:
- Per-step safety classifier (Lesson 21).
- Tool-call argument validation (Lesson 06).
- Cross-check retrieved content against known facts (Lesson 05, CRITIC).
- Detect success hallucination by re-probing state (was the file actually created?).
Where failure monitoring goes wrong
- Tagging only crashes. Most agent failures produce valid-looking output. Need content-level checks.
- No baseline. Drift detection needs a last-known-good; without it you cannot say "this is getting worse."
- Over-alerting. Every failure produces a page. Cluster and rate-limit.
Build It
code/main.py implements a stdlib failure-mode tagger:
- A synthetic trace dataset covering the five modes.
- Detector functions per mode (signature patterns on tool calls, outputs, repeat actions).
- A tagger that labels each trace and reports mode distribution.
Run it:
python3 code/main.pyOutput: per-trace labels + aggregate distribution, a cheap reproduction of what Phoenix's trace clustering surfaces.
Use It
- Phoenix for production drift clustering (Lesson 24).
- Langfuse for session replay + annotation.
- Custom for domain-specific signatures your observability platform can't detect.
Ship It
outputs/skill-failure-detector.md generates failure-mode detectors tailored to your domain, wired to a trace store.
Exercises
- Add a detector for "success hallucination": agent returns success but the target state is unchanged.
- Tag 100 real traces from a product you've built. Which mode dominates? What's the cost of fixing it?
- Implement a "cascade radius" metric: given a failure at step N, how many downstream steps did it affect?
- Read MASFT's 14 failure modes. Pick three that apply to your product. Write detectors.
- Wire one detector into a CI job: fail the build if >=5% of traces tag a mode.
Key Terms
| Term | What people say | What it actually means |
|---|---|---|
| MASFT | "Multi-agent failure taxonomy" | Berkeley 14-mode categorization |
| Cascading error | "Ripple failure" | One early mistake propagates through N steps |
| Context loss | "Forgot the constraint" | Long-horizon turn drops early-turn facts |
| Tool misuse | "Wrong tool / wrong args" | Valid call, wrong invocation |
| Success hallucination | "Faked completion" | Agent claims success on a 400; state unchanged |
| Scope creep | "Overreach" | Agent does more than asked |
| Instruction-following deviation | "Disobedience" | Ignores system prompt or user constraint |
| Sub-intention errors | "Plan bugs" | Omission, redundancy, disorder in plan execution |
Further Reading
- Cemri et al., MASFT (arXiv:2503.13657)14 failure modes, 3 categories
- Microsoft, Taxonomy of Failure Mode in Agentic AI Systemsrisk register
- Arize Phoenixdrift clustering in practice
- Anthropic, Building Effective Agentswhen simpler patterns avoid modes entirely
Check what you learned
7 questions. Answer each one to see why.
What is MASFT's central claim?
The full code
Every file for this lesson. Copy it, run it, then change it.
"""Stdlib failure-mode tagger for agent traces.
Detects the five industry-recurring modes: hallucinated actions, scope creep,
cascading errors, context loss, tool misuse. Each detector returns a tag if
the trace matches; aggregate distribution mirrors Phoenix's trace clustering.
"""
from __future__ import annotations
from collections import Counter
from dataclasses import dataclass, field
from typing import Any
@dataclass
class TraceStep:
kind: str
name: str
args: dict[str, Any] = field(default_factory=dict)
status: str = "ok"
result: str = ""
@dataclass
class Trace:
tid: str
user_request: str
constraints: list[str]
steps: list[TraceStep]
final_success_claim: bool
target_state_changed: bool
KNOWN_TOOLS = {"search", "read_file", "write_file", "list_dir"}
def detect_hallucinated_action(trace: Trace) -> str | None:
for step in trace.steps:
if step.kind == "tool_call" and step.name not in KNOWN_TOOLS:
return "hallucinated_action"
return None
def detect_scope_creep(trace: Trace) -> str | None:
request = trace.user_request.lower()
writes = [s for s in trace.steps
if s.kind == "tool_call" and s.name == "write_file"]
explicit_write_words = ("write", "create", "save", "update", "edit")
wanted_write = any(w in request for w in explicit_write_words)
if len(writes) > 0 and not wanted_write:
return "scope_creep"
return None
def detect_cascading_errors(trace: Trace) -> str | None:
saw_error = False
downstream_ops = 0
for step in trace.steps:
if step.kind == "tool_call" and step.status == "error":
saw_error = True
continue
if saw_error and step.kind == "tool_call":
downstream_ops += 1
if saw_error and downstream_ops >= 2:
return "cascading_errors"
return None
def detect_context_loss(trace: Trace) -> str | None:
for constraint in trace.constraints:
con_l = constraint.lower()
if "do not" in con_l:
forbidden_token = con_l.split("do not")[-1].strip().split()[0]
for step in trace.steps:
if step.kind == "tool_call" and forbidden_token in str(step.args).lower():
return "context_loss"
return None
def detect_tool_misuse(trace: Trace) -> str | None:
tool_args_schema = {
"read_file": {"path"},
"write_file": {"path", "content"},
"list_dir": {"path"},
"search": {"query"},
}
for step in trace.steps:
if step.kind != "tool_call":
continue
expected = tool_args_schema.get(step.name)
if expected is None:
continue
if not expected.issubset(set(step.args.keys())):
return "tool_misuse"
return None
def detect_success_hallucination(trace: Trace) -> str | None:
request = trace.user_request.lower()
write_intent = any(w in request for w in
("write", "create", "save", "update", "edit", "make"))
if (write_intent and trace.final_success_claim
and not trace.target_state_changed):
return "success_hallucination"
return None
DETECTORS = (
detect_hallucinated_action,
detect_scope_creep,
detect_cascading_errors,
detect_context_loss,
detect_tool_misuse,
detect_success_hallucination,
)
def tag(trace: Trace) -> list[str]:
return [label for label in (d(trace) for d in DETECTORS) if label]
def main() -> None:
print("=" * 70)
print("AGENT FAILURE MODES — Phase 14, Lesson 26")
print("=" * 70)
traces = [
Trace(tid="t001", user_request="find the config file",
constraints=["do not modify any files"],
steps=[
TraceStep("tool_call", "search", {"query": "config"}),
TraceStep("tool_call", "read_file", {"path": "config.yml"}),
],
final_success_claim=True, target_state_changed=False),
Trace(tid="t002", user_request="find the config file",
constraints=["do not modify any files"],
steps=[
TraceStep("tool_call", "search", {"query": "config"}),
TraceStep("tool_call", "write_file",
{"path": "config.yml", "content": "..."}),
],
final_success_claim=True, target_state_changed=True),
Trace(tid="t003", user_request="list project files",
constraints=[],
steps=[
TraceStep("tool_call", "magic_scanner",
{"path": "/"}),
],
final_success_claim=True, target_state_changed=False),
Trace(tid="t004", user_request="look up invoice 4711",
constraints=[],
steps=[
TraceStep("tool_call", "search",
{"query": "invoice 4711"}, status="error"),
TraceStep("tool_call", "read_file", {"path": "/tmp/foo"}),
TraceStep("tool_call", "write_file",
{"path": "/tmp/foo", "content": "fabricated"}),
TraceStep("tool_call", "list_dir", {"path": "/tmp"}),
],
final_success_claim=True, target_state_changed=True),
Trace(tid="t005", user_request="update readme with release notes",
constraints=["do not modify src/"],
steps=[
TraceStep("tool_call", "read_file", {"path": "README.md"}),
TraceStep("tool_call", "write_file",
{"path": "README.md", "content": "notes"}),
TraceStep("tool_call", "write_file",
{"path": "src/foo.py", "content": "also notes"}),
],
final_success_claim=True, target_state_changed=True),
Trace(tid="t006", user_request="read some file",
constraints=[],
steps=[
TraceStep("tool_call", "read_file",
{"file": "/tmp/foo"}),
],
final_success_claim=False, target_state_changed=False),
Trace(tid="t007", user_request="create a PR",
constraints=[],
steps=[
TraceStep("tool_call", "search", {"query": "PR template"}),
],
final_success_claim=True, target_state_changed=False),
]
distribution: Counter = Counter()
print()
for trace in traces:
labels = tag(trace)
distribution.update(labels)
print(f" {trace.tid} user={trace.user_request[:40]!r}")
print(f" labels: {labels if labels else '[clean]'}")
print("\naggregate distribution")
for label, count in distribution.most_common():
print(f" {label}: {count}")
print()
print("gate at every step: classifier, argument validation, state probe.")
print("cascading is the killer. detect early, stop the loop.")
if __name__ == "__main__":
main()