Become an AI inference provider: sell your own model API
Every company calling an AI API pays someone to run a model on a GPU, and with good open-weight models that someone can be you. Serve a model with vLLM on a rented GPU and build the parts most tutorials skip: keys, rate limits, metering and billing. Then see why selling raw tokens is a trap, and package what firms pay for: private, UK-hosted, fixed-price AI.
- Offers that make margin: private deployments, UK-hosted APIs and hosted fine-tunes, with an SLA and usage policy
- A route onto OpenRouter: the provider requirements and a model listing in their format
- Comfort renting and driving a cloud GPU over SSH: Rent a cloud GPU first
- uv for the Python scripts: Put AI in your terminal
- Basic Linux server skills: Harden a server and Your own domain with HTTPS
- A card for GPU rental, and a budget you're happy to spend testing (a few afternoons of GPU time is enough to start)
- A business set up properly: Make it legal
Know what you're really selling
Tokens are a commodity. Privacy, location, a fixed bill and a person to call are not.
The stack is simpleAn open-weight model (Qwen, Gemma, Mistral and others publish weights you can run commercially) runs on a GPU under an inference server (vLLM is the standard). The server speaks the OpenAI API, so any app built for OpenAI works by changing two settings: the base URL and the key. You add a gateway that knows who each customer is, limits them and meters them. That's an AI API business.
Where the money isn'tBig providers sell popular open models at very low prices per million tokens, at huge scale and near-full utilisation. A single rented GPU that sits idle half the day can't beat them on raw price. Step 3 proves it with your own numbers.
Where the money isCustomers who need something the cheap shared APIs don't give them:
- Private deployments: a dedicated model server for one firm. Law, accounting, finance and health-adjacent firms want AI on client documents without sending them to a shared US service.
- UK data residency: some contracts require data to stay in the UK. Rent your GPUs in a UK data centre and say so.
- Niche fine-tuned models: a small model trained for one job (claims notes, property listings, one software product's support tickets), hosted for every company in that vertical. vLLM serves many fine-tuned adapters on one base model.
- A fixed monthly bill and a named person who answers when something breaks.
Careful- What it takes. Capital: a few hundred pounds of GPU time to test, then GPU rental you commit to once a customer signs (a 96GB card runs around $1,500 a month always-on). Price so the customer's fee covers the server from month one.
- Time: 10 to 20 hours a week while you set up and sell; a stable deployment needs a few hours a week of monitoring and updates.
- Skills: Linux servers, Python, reading logs, and explaining AI plainly to non-technical buyers.
- Paperwork: customer terms with an SLA and acceptable use policy (templates in the kit, reviewed by a solicitor), professional indemnity and cyber insurance, and a data processing agreement for each business customer.
- Time to first customer: commonly one to three months, because firms trial on their own documents before they commit.
- Who this suits: people who already run servers or build with AI APIs and like operations work: monitoring, uptime, upgrades, and explaining it all to a client.
Everything you'll use is in one kitDownload the kit (14KB): the gateway, the smoke test, the benchmark, the cost calculator, the production setup script, a Caddyfile, an OpenRouter model listing, the SLA and usage policy, offers and a 30-day plan.