Fine-tune a small AI on your own writing and run it in Ollama
Prompting can describe your style. Fine-tuning bakes it in. Turn a folder of your own writing into a training set, run a real LoRA fine-tune on your laptop's CPU to prove the whole pipeline, then train a 4B model with Unsloth on a free GPU and chat with it offline in Ollama.
ollama run- A style scorecard that shows whether it really learned your voice, not just whether the loss went down
- The GPU recipe: Unsloth cells for Qwen3-4B on a free Colab T4 or a rented card, exported straight to GGUF
- Ollama installed and working: Run AI on your PC
- uv, the Python launcher, from docs.astral.sh/uv ↗. It installs the right Python and packages per script
- 16GB of RAM for the CPU run, and about 6GB of free disk
- At least 100 passages of your own writing (blog posts, newsletters, long emails, reports). 300 or more for the GPU run
- For the 4B run: a Google account for Colab, or a GPU pod as in Rent a cloud GPU
Know what fine-tuning changes, and the plan
Fine-tuning teaches a model how to write, not what to know. Pick the right job for it.
Fine-tuning changes behaviourYou show the model hundreds of examples of a request and the reply you want, and it adjusts its weights until that kind of reply comes naturally. It's brilliant for voice, tone, structure and format: your newsletter style, your way of replying to customers, a fixed report layout. It's poor at teaching facts. For "answer questions about my documents", use retrieval instead: Chat with your documents.
LoRA makes it cheapInstead of retraining all the weights, LoRA freezes the model and trains small adapter matrices next to it. On the 0.6B model in this mission that's 10 million trainable parameters out of 606 million (1.67%). The same trick is what lets Unsloth ↗ train a 4B model on a free GPU.
The planYou'll do the whole pipeline twice:
- On your CPU, tiny and fast: dataset, a short LoRA run on Qwen3-0.6B, merge, convert to GGUF, load into Ollama, score it. This proves every step works on your machine with your data.
- On a GPU, for quality: the same dataset into Unsloth with Qwen3-4B-Instruct-2507, exported as a 4-bit GGUF you run in Ollama every day.
This mission's demo data is Jerome K. Jerome's *Idle Thoughts of an Idle Fellow* (1886), a public-domain book with a very recognisable voice: chatty, first person, long sentences full of semicolons and double hyphens. Once it works on Jerome, swap in your own writing.
Get the kitwrite-like-me.zip has every script in this mission. kit.md lists them with the full command sequence.
CarefulFine-tuned models can repeat their training text word for word. Only train on writing that's yours or that you have the rights to use, and strip out anything private first: addresses, client names, passwords, health details. Treat the finished model as if it were the dataset.