Cut your AI bill without cutting quality

Every paid LLM charges by the token, not by the request. A token is a chunk of text, very roughly 4 characters or about three quarters of a word in English, so tokenisation might be two tokens and the one. The model turns your text into tokens, works on tokens, and writes tokens back.

One call has two sides, billed at two different rates:

• Input tokens are everything you send: the system prompt, the instructions, any examples, the user's message and any documents you paste in. You usually pay less per input token.
• Output tokens are everything the model writes back. You usually pay more per output token, often several times more, and the model makes them one at a time, so they also drive most of the waiting.

There is a ceiling on the input side called the context window: the most tokens a model can take in one call (the prompt plus the space left for its answer). Go over it and the call is rejected or the oldest text is dropped. A bigger window lets you send more, but you still pay for every token you put in it, so a large window is not a reason to send more than you need.

So the whole bill for one call is this:

cost = input_tokens x input_price + output_tokens x output_price

Everything that follows is one of three moves: send fewer input tokens, make fewer output tokens, or make fewer and cheaper calls. To measure any of it you need a project. Make one with uv:

uv init --bare --python 3.13 cost
cd cost
uv add tiktoken ollama

tiktoken is the tokenizer OpenAI-style models use, so you can count tokens on your own PC with no account and no network. ollama is the Python client for the local models you installed, which you will use to measure real timings and the models' own token counts. Drop the kit files into this cost folder: count_tokens.py, cache.py, cost_calc.py, and the demo scripts trim_demo.py, cap_demo.py, cache_demo.py, batch_demo.py, route_demo.py, stream_demo.py. Or type each one as you reach it.

Confirm tiktoken works. This prints the token numbers that Hello, world! becomes for gpt-4o:

uv run python -c "import tiktoken; print(tiktoken.encoding_for_model('gpt-4o').encode('Hello, world!'))"

[13225, 11, 2375, 0]

Four numbers, so four tokens: one for Hello, one for the comma, one for world and one for !. Each number is that token's id in the model's vocabulary. That is all a tokenizer does: text in, a list of token ids out, and the length of that list is what you pay for.