
Mission 8 of 8
Deploy an agent that runs all day, with caps, logs and alerts.
An agent in your terminal is a demo. Here you'll wrap yours in a queue, a spend cap, logs, Docker and an alert, so it can run on a small server and tell you when it falls over.
What you'll do
Write a FastAPI service with a SQLite job table, a rate limit and a daily spend cap.
Package it with Docker and put it on a small server, reachable only privately.
Add a five-minute cron job that pushes a notification to your phone when health or spend looks wrong.
- Structured JSON logs with the job id on every line
- Secrets handled properly, with a spend limit set in two places
- Missions 3 and 6 done:
agent2.py,toolbox.py,tools.py,tools_v2.py,expenses.txt, the eval suite - Docker Desktop or Docker Engine locally. Docker without fear if it is new
- A VPS you can SSH into, hardened first: harden a server. Or a serverless host account for the alternative in step 5
pip install fastapi uvicornin the venv- Control LLM cost read once; this mission builds the cap, that one explains the levers behind the numbers
Your progress0/6
- 01The shape: an API that queues, a worker that runs, a store that remembers8 min
- 02Build it: store.py and service.py20 min
- 03Secrets, caps and limits: the numbers that keep it yours12 min
- 04Containerise it: non-root, health check, data on a volume14 min
- 05Put it on a server, privately18 min
- 06Health, alerts, and the two missions that come next18 min
The shape: an API that queues, a worker that runs, a store that remembers
Agent tasks take 10 to 60 seconds. HTTP requests should take 10 milliseconds. Separate them.
You've spent seven missions improving the agent. Before it goes on a server, decide its shape, because the obvious shape is the wrong one.
The mistake to avoid
The mistake every first deployment makes is to run the agent inside the request: POST /ask, wait 40 seconds, return the answer. It works until two people ask at once, or a tool hangs, or the client times out at 30 seconds and retries, and now you are paying for the same task twice.
The fix is as old as web development: accept the job, return an id immediately, do the work elsewhere, let the client poll or get called back.
The three parts
So the service has three parts:
- The API.
POST /tasksvalidates the input, checks the rate limit and the spend cap, writes a row, returns202 Acceptedwith an id.GET /tasks/{id}returns the row.GET /healthreturns whether the worker is alive and how much has been spent today. Nothing in the API calls the model. - The worker. A loop in a background thread: claim the oldest queued job, run
agent2.run(), price the usage, write the result and the spend, log it. One worker is fine to start; it is a thread, and the API stays responsive while it runs. Two workers later is amax_workerschange. - The store. SQLite, two tables:
jobsandspend. It is the queue, the result cache and the spend ledger. For one machine and thousands of jobs a day it is more than enough, and it survives restarts because it is a file on a volume.
This is the same shape as every task-queue system, and when you outgrow a thread and a SQLite file (many workers, several machines) the replacement is Redis or Postgres and a proper queue, with the same three parts. Distributed task queue builds that version.
agent2.run(task) returns the contract dict from mission 3; the service calls it and stores the dict. Everything you tested in mission 6 is what runs here, which is the entire reason for having built the contract.POST /tasks returns, in how long, and what it does not do.