Self-Driving RC Car

overview

This was the final project for ELEC 424 at Rice in Spring 2026, built with my teammates Faye De Carlo, Rahul Kishore, and Trisha Rangi under the team name Rice Demon Slayers. The goal was to take a stock RC car and make it drive itself: follow a piece of blue tape down a hallway, stop at two red boxes along the way, and recognize when it had reached the end of the course.

Every decision on the car (steering angle, throttle, when to brake) is made on a Raspberry Pi 5 running real-time computer vision off a USB webcam. There is no IMU, no LIDAR, no GPS. Just a camera, a wheel encoder, and a 25 FPS control loop reading colored pixels off the floor.

hardware

The car itself is a hobby-grade RC chassis with the original receiver and stock servo. The Pi 5 sits on top, the webcam is mounted forward and slightly tilted down, and the LCD shows live telemetry (current state, steering error, encoder ticks) for debugging without having to plug in a monitor.

software stack

Everything runs as a single Python process on the Pi. OpenCV does the heavy lifting on lane detection and stop-sign color matching, and a YOLOv5 model runs alongside for general object detection demos. The original instructable we started from used deprecated GPIO libraries, so we migrated to lgpio and gpiod and wrote a custom kernel driver to talk to the optical encoder, which the existing userspace drivers couldn't read reliably on the Pi 5.

lane following

The camera captures at 160×120. That sounds laughably small, but at higher resolutions the control loop fell below the rate where the steering felt responsive, and the lane is just a blue stripe on the floor; 160×120 is enough pixels to find it. At this resolution we hold the control loop above 25 FPS.

Each frame: convert to HSV, mask for blue, find the centroid of the masked region in the bottom half of the frame, and use the horizontal offset of that centroid from the image center as the steering error.

PD controller

Steering is a straight PD loop on the centroid error. After a lot of tuning on the hallway carpet:

The proportional gain is calibrated so the servo hits its mechanical limit when the lane drifts about 50 pixels off-center, which is roughly a third of the frame width. That's aggressive enough to recover from a hard turn without ringing on the straights. The derivative term smooths out the jitter from the centroid bouncing between pixels frame to frame. We skipped integral entirely because the steering had no persistent bias to integrate out, and adding integral just made the car drift toward whichever wall it had most recently corrected away from.

stop detection

The course has two red boxes that the car has to stop at. Red is the annoying color in HSV because hue wraps around: red lives at both ends of the [0, 180] hue range. So the mask is two ranges OR'd together:

From there we threshold on contour area (anything under 1000 px² gets rejected as a false positive) and only run the stop check on every third frame so we keep the steering loop fast.

The first version of this code tried to stop the instant it saw red, and that turned out to be a disaster: a single noisy frame (red dust on the carpet, a colored shoe walking by, a reflection) would trigger a full stop in the middle of nowhere. The fix was a small state machine: after the first detection, count down 10 consecutive frames still seeing the box before committing to stop. After the stop, a 4-second cooldown prevents the same box from being recounted as we drive away from it.

what was hard

• The Pi 5 broke half our drivers. The instructable we forked from was written for older Pi hardware with the old GPIO library. On the Pi 5 we had to switch to lgpio and gpiod for general GPIO and write a custom kernel driver to get the optical encoder readable from userspace.
• Derivative noise. First version of the PD controller had Kd set way too high. The centroid jitters plus or minus 1 to 2 pixels per frame just from camera noise, and a high Kd turned that into audible servo chatter. Cutting Kd down to 0.004 was the fix.
• Steering trim. Mechanical alignment on the car was actually correct, so we left the software steering trim at exactly 0.0. Every time we tried to "help" with a software offset we made the lane following worse.
• Resolution vs. responsiveness. Higher capture resolution gave better lane detection but dropped the control loop below ~15 FPS, at which point the car would visibly oversteer on corners. 160×120 was the sweet spot that kept the loop above 25 FPS.

results

The car completes the full course end to end: follows the tape down a hallway, stops at the first red box, resumes, stops at the second red box, and recognizes the endpoint marker. Telemetry plots from the demo runs show stable error tracking and clean proportional/derivative response over hundreds of frames with no visible oscillation.

YOLOv5 also runs live alongside the lane controller as a demo, recognizing objects in front of the car without dropping the control loop below real-time.

Source and writeup are on the team's GitHub (linked from the Hackster page). MIT-licensed.