Build an RC car that learns to drive itself by watching you
Bolt a Raspberry Pi 5 and a camera onto an RC car, drive ten laps of a taped track, and train a neural network that copies your steering from the picture alone. Then hand it the controls and watch it lap the track with nobody touching the sticks. Donkey Car 5.4.0 does the heavy lifting; you can test the whole training pipeline on your PC before the car even arrives.
- Your own steering model: about 800,000 parameters, trained on your laps, running 20 times a second on the Pi
- A kit: Pi setup script, starter config, parts list, track guide and two Python tools
- A Raspberry Pi 5 4GB (£105.60 at The Pi Hut), the official active cooler and a 64GB+ fast microSD card
- Raspberry Pi Camera Module 3 Wide (£33.60) plus the Pi 5 camera adapter cable (about £1)
- An RC car with a separate ESC and a 3-wire steering servo (the WLtoys 144010 is around £80 to £120), a PCA9685 servo board (£6 to £16) and a USB-C power bank
- A Bluetooth game controller (Xbox or PS4) is strongly recommended, and a smooth floor with room for a 15 to 25m loop
- Comfortable with SSH and a terminal: Terminal survival kit. For cloud training, Rent a cloud GPU
How a car learns to drive from your laps
Understand what the network learns, so every later step makes sense, and prove the training pipeline on your PC today.
Behavioural cloningYou drive. Twenty times a second the Pi saves a 160 by 120 camera frame together with your steering and throttle at that instant. A folder of these records is a tub. A small convolutional network then learns one job: given a picture, output the steering and throttle you would have used. On autopilot, the same network looks at each new frame and drives.
Why it worksThe tape lines are the only thing that changes in a predictable way between frames. After a few thousand examples the network has learnt that lines bending right mean steer right, and that a line drifting into the middle of the picture means you're off line. No maps, no rules, no hand-written code for bends.
The softwareDonkey Car ↗ is the open-source platform that runs it all: camera, controller, recording, training and autopilot. Version 5.4.0 (Python 3.12 and 3.13, Keras 3) is the one this mission pins, on the Pi and on your PC.
Prove it on your PC before the car arrivesInstall Donkey Car with the PC extras into its own Python environment. uv ↗ keeps it isolated:
Windows (PowerShell)uv venv --python 3.12 donkey-env .\donkey-env\Scripts\activate uv pip install "donkeycar[pc]==5.4.0" donkey createcar --path mycar cd mycarMac / Linuxuv venv --python 3.12 donkey-env source donkey-env/bin/activate uv pip install "donkeycar[pc]==5.4.0" donkey createcar --path mycar cd mycarThen grab the kit's make_sample_tub.py. It draws synthetic camera frames of a taped track with random bends and drift, labels each with the steering a good driver would use, and writes a real tub with Donkey's own writer:
Terminalpython make_sample_tub.py --out ./data/sample --n 4000Output1000 records 2000 records 3000 records 4000 records wrote 4000 records to ./data/sampleThe kitDownload the kit or open kit.md to see each file.
donkey createcar printed Donkey setup complete. and data/sample holds a manifest.json, catalog files and an images folder of 4,000 JPEGs.If it breaksuv can't find Python 3.12 + 1 more
uv can't find Python 3.12
uv python install 3.12 once, then repeat uv venv. Donkey Car 5.4.0 supports Python 3.12 and 3.13 only; 3.14 is refused by the package.The install fails on TensorFlow on a Mac
uv pip install "donkeycar[macos]==5.4.0", which pulls TensorFlow 2.19 plus the Metal plugin.