Fine-tune small open models for companies that can't send data to OpenAI

Here's the gap. Companies with confidential text and a lot of it (claims handlers, law firms, accountants, insurers, lettings agencies, healthcare admin, any business whose own clients put "no third-party AI" in their contracts) watched everyone else get AI and were told no by their compliance team. Meanwhile the same job gets done by hand, every day: someone reading 400 emails to forward them to the right team.

The offer, in one line"I build a small AI model that does one of your repetitive text jobs, runs entirely on your own hardware, and comes with a test report that proves how accurate it is. Fixed price, optional monthly care."

Why a small model can do itA big general model is good at everything. Your client needs it to be good at one thing. Fine-tuning a small open model on a few hundred to a few thousand of their own examples teaches it exactly their labels, their fields and their edge cases. The evidence is solid: the LoRA Land study fine-tuned 310 models from 10 base models across 31 tasks ↗, and the 4-bit LoRA models beat their own base models by 34 points and GPT-4 by 10 points on average. A 2024 study on text classification found fine-tuned smaller models beat zero-shot GPT-3.5, GPT-4 and Claude Opus ↗ in every case they tested. Narrow, repetitive, labelled tasks are exactly where this works.

What clients actually buyThe obvious pitch ("save on API fees") is usually the wrong one. Sorting 30,000 tickets a month with a small API model like gpt-5-mini costs about $5 a month at OpenAI's published prices ↗. Nobody pays you £10k to save $5. They buy:

• Confidentiality. The data never leaves their building. For many firms this is the only reason, and it's enough.
• Control. A model they own doesn't change or disappear when a provider updates it. OpenAI's own optimisation guide now says it is winding down its fine-tuning platform ↗, closed to new users. Anyone who built on it needs a plan.
• Repeatability. The same input gives the same output, which matters for audits.
• Cost, but only at scale. It becomes a real argument with big documents, big models and big volumes. Step 5 does the maths.

When NOT to fine-tuneSay this before the client does, and you'll be trusted. OpenAI's guide says careful prompting "may be all you need" ↗, and Google's Vertex AI docs recommend starting with prompting ↗ and tuning only when you have labelled data (they suggest 100 examples or more) and prompting isn't enough. In practice:

• The answer depends on facts that change (prices, policies, stock, case law). Use retrieval, not fine-tuning: fine-tuning teaches a behaviour, not a library. Build your own "chat with my files" app covers retrieval.
• A local model plus a good prompt already gets it right. Try that first in the feasibility sprint. Pick the right AI model for your laptop helps you choose one.
• They have fewer than about 100 labelled examples and no one to label more. You'll be training on noise.
• The task isn't repetitive or the labels aren't agreed. If two of their staff disagree on a third of examples, no model will fix that.