Get Started
Protect your app and website

Mission 9 of 10

Attack your own app on purpose, and fail the build.

Defences only count if they keep holding. You'll write an attack suite, grade it with a judge model, scan your own site with ZAP, and run both in CI so a regression fails the build.

75 minAdvancedA few cents per run on the Claude API, or free with a local judge modelStep 1 free

What you'll do

Write an attack suite

Put injection, policy, exfiltration and cost attacks in one YAML file, all aimed at your own app.

Grade it with a judge

Run a Python runner that fires each attack and has a judge model decide pass or fail.

Scan and automate in CI

Run an OWASP ZAP baseline scan of your own site, then put both harnesses in GitHub Actions.

  • Only ever pointed at systems you own or have permission to test
  • A habit of turning every real bug into a permanent test case
Before you start
  • The defended agent (agent_v2.py / the ToolBox) from lock down tools and MCP, or your own app with an endpoint to hit
  • Python 3.11+ and uv; an LLM for the judge (Claude API or local Ollama)
  • Docker, for the ZAP scan (or use the GitHub Action, which needs no local Docker)
  • A site you own to scan, and a GitHub repo if you want CI
Your progress0/5
01/05What red-teaming your own app means, and what it doesn't0 done
STEP 01 of 05~8 min

What red-teaming your own app means, and what it doesn't

Set the scope so this stays defensive: your systems, your policy, your build.

Before any code, get clear on what this mission is. Red-teaming here means one thing: running known attacks against your own app, on purpose and repeatedly, so you find the holes before an attacker does and so a code change that reopens one is caught automatically.

It's the same idea as testing your prompts like code and guardrails for LLM apps, aimed at security rather than quality.

Those missions teach the eval mechanics; this one assumes them and focuses on the attacks.

Two boundaries

These two keep it defensive and legal:

  • Your systems only. Every attack in the suite and every ZAP scan targets an app or site you own or have written permission to test. Running these against someone else's app is an offence under the Computer Misuse Act 1990, full stop.
  • Your policy, not the model's safety. When the suite tests 'policy bypass', it tests *your* stated rules: 'the support agent only discusses broadband', 'refunds are capped at £50', 'users see only their own orders'. It is not about defeating the model provider's safety guardrails, which is a different and out-of-scope activity. The distinction is simple: you're checking that your app enforces the limits you designed, exactly like checking that a form rejects a negative quantity.

The payoff

It's regression protection. The prompt-injection mission showed injection is probabilistic and a refactor can quietly make it worse.

A red-team suite that runs on every push turns 'I hope that's still holding' into a green or red tick. That's the whole reason to automate it rather than test by hand once.

Tip
Keep the suite in the same repo as the app so it versions with the code. An attack case and the fix that made it pass belong together in history; when someone asks 'why is this check here?', the commit shows the bug it caught.
Checkpoint
You can state the two boundaries (own systems, own policy) and name the one app you'll point this suite at. That app is the only target for everything that follows.
Stuck? Ask the community
Next mission · Protect your app and websiteProduction readiness: logs, spend caps, alerts, backups, and a one-page incident plan
→