Get Started
← Own your AI · Mission 23 of 31

Build an AI that uses your browser, with guardrails it can't talk its way past

Give an AI a real browser and it can compare prices, read pages and fill in forms for you. It will also follow instructions hidden in a web page if you let it. Build a small browser agent that runs on a local model, watch it get prompt-injected and stopped by code, then plug a real browser into Claude with Playwright MCP.

120 minIntermediateTools: Free with a local model. An API model for browser-use costs a few pence per taskStep 1 free
A working browser agenta local model reads the page and drives Playwright, one checked action at a time
Five guardrails in codeallow-list, untrusted page text, sensitive-field refusal, a confirm gate that shows you the form, and a loop guard
A practice high street to test onthree sample shops, a hidden prompt injection and a form that sends nothing
  • Playwright MCP set up for Claude Desktop, Claude Code or VS Code, locked to the sites you choose
  • The same job in browser-use, the popular agent library, for when you want a bigger brain
Before you start
  • Ollama with a model that follows JSON well. qwen3.5:4b (3.3GB) is used here: Run AI on your PC
  • uv from docs.astral.sh/uv ↗, and Python 3 for the practice site's web server
  • Node.js 20 or newer for Playwright MCP (optional step)
  • 16GB of RAM. A graphics card makes each step seconds instead of a minute
  • Helpful first: Give your offline AI real tools, which explains the tool-calling loop this builds on
01/08How a browser agent works, and what it may touch0 done
STEP 01 of 08~10 min

How a browser agent works, and what it may touch

Look, decide, act, check: four parts, and only one of them is the AI.

  1. A browser agent is a loopYour code reads the page (the address, the links, buttons and fields, the visible text) and hands a compact description to a model. The model replies with one action: click element 4, fill these fields, go back, or done. Your code checks that action against rules, does it with Playwright ↗ (the browser automation library), and goes round again.

  2. The model is the least trustworthy partIt misreads pages, claims it has done things it hasn't, and follows instructions it finds in page text. Everything that keeps you safe has to live in the code around it, not in the prompt. That's the whole design of this mission.

  3. Three ways to build one, all in this mission:

    • Your own small agent (steps 2 to 5): about 280 lines of Python you can read end to end, a local model through Ollama, and guardrails you can see.
    • Playwright MCP (step 6): Microsoft's server that gives Claude, VS Code or any MCP client a real browser as a set of tools.
    • browser-use (step 7): the open-source library most people reach for, best with a strong API model.

  4. Good first jobscomparing prices across a few shops, checking stock or opening times, collecting facts from pages into a table, filling a long form with details *you* give it, then stopping for you to check. Never let it buy, pay, log into your real accounts, solve CAPTCHAs, send messages without you reading them, or delete anything.

  5. Get the kitbrowser-agent.zip has the agent, the practice site, the browser-use script and the MCP config. kit.md has practice tasks with their correct answers and a guardrail checklist.

✓
CheckpointYou've unzipped the kit and can name the one part of the loop you can't trust.
Stuck? Ask the community
Next mission · Own your AIMake AI music on your own PC, songs with vocals included
→