Talk to your AI out loud, in real time, fully offline
Speak, pause, and a local AI answers in a natural voice; talk over it and it stops to listen. Build the whole voice loop yourself in one Python file: pause detection, Whisper, a streaming LLM, sentence-by-sentence speech and interruptions. Then tune it from 3.9 seconds to 1.7 seconds on a laptop with no graphics card.
- A test conversation you can replay without a microphone, to tune and compare settings
- Measured settings that cut response time from 3.9 to 1.7 seconds on a CPU
- Ollama with
gemma4:e2b: Run AI on your PC - uv from docs.astral.sh/uv ↗
- A microphone, and headphones (so it doesn't hear itself)
- 16GB of RAM and about 1GB of disk for the voice models
- Helpful background: Free AI voiceovers covers Kokoro voices; Replace Alexa puts a voice assistant around the whole house
How a real-time voice loop works
Five stages, and the trick that makes it feel quick: start speaking before the answer is finished.
The loopthe microphone streams 30-millisecond frames. Pause detection watches the volume and decides when you've finished (600 milliseconds of quiet, by default). faster-whisper turns your speech into text. Ollama streams the answer word by word. A splitter cuts the stream into sentences, and each sentence goes to text-to-speech (Kokoro or Piper) and out of the speaker.
The trick: don't wait for the whole answerAs soon as the first sentence is complete it's spoken, while the model is still writing the second. Time to the first sound is what makes a voice assistant feel alive, and it's the number this mission measures: from the moment you stop talking to the moment audio for the reply is ready.
Interruptions ("barge-in") are the other half. While it talks, the microphone keeps listening. If you speak for a quarter of a second, it stops playback, closes the stream from Ollama so the model stops writing, throws away any queued sentences, and treats what you're saying as the next turn.
Get the kitvoice-chat.zip has
talk.py(the whole pipeline) andmake_test_audio.py. kit.md has the download commands, every setting and a troubleshooting checklist.