Get Started
Protect your app and website

Mission 10 of 10

Know what happened, cap the damage, and have a plan.

The first nine missions stop things going wrong. This one is for when they do anyway: logs, spend caps, alerts, backups and a written incident plan, then a capstone that checks your whole app.

75 minIntermediateFreeStep 1 free

What you'll do

Log calls and cap spend

Write structured logs with personal data stripped out, and put hard spend caps on every user and on the whole app.

Set alerts and backups

Get told when tool calls, denials or errors spike, and keep backups that survive ransomware.

Write your incident plan

Fill in a one-page plan while you're calm, then run the capstone checklist against your own app.

  • Automated dependency updates and uptime monitoring
  • A pass or fail score across all ten missions
Before you start
  • An app of your own, ideally the agent and web app you've been hardening through this track
  • Python 3.11+ and uv for the logging and alert scripts
  • A GitHub repo (for Dependabot/Renovate) and a place your app runs (for uptime monitoring)
  • Twenty minutes at the end to run the capstone honestly
Your progress0/6
01/06Structured logs of every tool and model call, PII redacted0 done
STEP 01 of 06~14 min

Structured logs of every tool and model call, PII redacted

You can't investigate, alert on, or prove anything without logs. And a log full of PII is a second breach.

Best practice

One structured (JSON) line per meaningful event, queryable later, with personal data and secrets stripped before they're written.

'Structured' is the key word: print('sent email to user') is useless at 3am; {"event":"tool_call","tool":"send_email","decision":"denied","rid":"a1b2"} can be filtered, counted and alerted on.

What to log

For an AI app: every model call (which model, token counts, latency) and every tool call (name, arguments, allowed or denied and why), tied together by a request id so you can follow one user's whole journey.

For any app: auth events (login success/failure, password reset), errors and 5xx responses, and any security-relevant decision. Not: successful reads of public data, or anything so chatty it drowns the signal.

The redaction rule

Logs get shipped to services, read by people, and kept for ages. They must not contain passwords, API keys, full card numbers, or more personal data than you need.

  1. Add this logger to your app. It redacts at the moment of logging, not later:

    log.py
    """log.py: structured JSON logs, one line per event, with PII and secrets redacted.
    Structured means machine-queryable: you can later ask "all tool calls that were denied in the last hour"."""
    import json, re, sys, time, uuid
    
    # Redaction: strip the things that must never land in a log file or log service.
    PATTERNS = [
        (re.compile(r"\b[\w.+-]+@[\w-]+\.[\w.-]+\b"), "<email>"),
        (re.compile(r"\b(?:\d[ -]*?){13,19}\b"), "<card>"),
        (re.compile(r"\b(sk-[A-Za-z0-9]{10,}|AKIA[0-9A-Z]{16}|ghp_[A-Za-z0-9]{36})\b"), "<secret>"),
        (re.compile(r"\b\d{4,}\b"), "<num>"),                    # account numbers, ids: coarse but safe
    ]
    
    def redact(value: str) -> str:
        for pat, repl in PATTERNS:
            value = pat.sub(repl, value)
        return value
    
    def log(event: str, **fields) -> None:
        safe = {k: (redact(v) if isinstance(v, str) else v) for k, v in fields.items()}
        line = {"ts": round(time.time(), 3), "event": event, **safe}
        sys.stdout.write(json.dumps(line) + "\n"); sys.stdout.flush()
    
    # Give each request an id so you can follow one user's whole journey across many log lines.
    def request_id() -> str:
        return uuid.uuid4().hex[:12]
    
    # Usage in the tool wrapper from the lock-down-tools mission:
    #   log("tool_call", rid=rid, tool=name, args=json.dumps(args), decision="allowed")
    #   log("tool_call", rid=rid, tool=name, args=json.dumps(args), decision="denied", reason=str(e))
    #   log("model_call", rid=rid, model=MODEL, input_tokens=usage.input_tokens, output_tokens=usage.output_tokens)
  2. Wire it into the tool wrapper from lock down tools and MCP: log every dispatch as allowed or denied, and log each model call's token usage.

    Now the denied lines are both your audit trail (who tried what) and, in the next steps, your attack signal (a surge of denials is an attack in progress).

    The request id ties a user's model calls and tool calls into one followable story.

What this closes

This closes OWASP A09, logging and monitoring failures, and its LLM-list cousin.

The most common version of that failure isn't 'no logs', it's 'logs that don't contain the one field you need, or contain so much PII you daren't share them'. Structured plus redacted fixes both.

Warning
Redaction is best-effort and coarse on purpose (the <num> rule over-redacts rather than risk leaking an account number). Prefer logging an id that maps to a user in your database over logging the user's email or name at all. The safest PII in a log is the PII that was never put there.
Checkpoint
Your app emits one JSON line per tool and model call, each with a request id, and a line containing an email or a key shows <email> / <secret> instead of the real value.
Stuck? Ask the community
Next mission · Agent engineeringAn AI agent in 60 lines of Python: the loop, two tools, and where it breaks
→