Core Electronics Forum

Face and Movement Tracking Pan-Tilt System with Raspberry Pi and OpenCV

Hey all, just put the finishing touches on the guide Face and Movement Tracking Pan-Tilt System with Raspberry Pi and OpenCV.

This will teach you exactly how to make the Pimoroni Picade Pan-Tilt Hat for the Raspberry Pi 4 Model B track faces. This means it will keep your face (and the action) always in the centre of the frame. The intention here was to not only create an easy-to-use face-tracking system with a Pan-Tilt Hat but also do so in a way that can be readily expanded upon no matter what systems or code additions you choose to use. So naturally, I took this tracking script to the next level by expanding on it to include movement detection (where it turns towards the centre of movement) and a patrol layer (which it reverts to when no faces or movements are seen). Both will make finding faces even more likely :slight_smile:.

This is another foray into the Open-CV landscape with Raspberry Pi (link here for other guides). Cameras in combination with machine learning create the most powerful sensor you can ever put on a Raspberry Pi Single Board Computer. And now we are making those cameras smart and mobile with artificial intelligence!

Read more


Hi Tim,

What a cool tutorial!

I’m really interested in adapting this to follow an object like my RC Helicopters - I have the ongoing problem of how to film them with reasonable zoom when flying by myself and not being able to follow the models with a simple tripod setup. Have tried a GoPro headmount but find it too heavy and distracting, and prefer to film directly from my phone for easy access too.

Would it be difficult to change the face tracking to track an object instead?


Running a layer of object detection instead of face tracking is definitely possible. Just gotta dive into the scripts and do some tweaking of the Python.

Start by getting Object Detection up and running on your Raspberry Pi. The trained Coco Library I have used here has 80 trained objects, ‘airplane’ being one of them. I’d be curious to see if it would as it stands detect your helicopter as an airplane. If so then you are off to the races.

If it doesn’t detect then that’ll be the first big hurdle. Doing a quick look around there is a team working on general UAV detection (UAVData: A dataset for unmanned aerial vehicle detection | SpringerLink) but they haven’t released the trained library yet. I’m sure there is a trained library somewhere that includes helicopters, I just haven’t found it yet.

Would make for a brilliant project, would love to see some RC Heli flips!


Another Thing worth metioning is you can create your own objects libraries by training a system yourself. Take a look at Edge Impulse as this is one potential way to do so. Here is a video of a fellow training the system on a full sized desktop to know what certain vegetables are from scratch, then running the system on a Raspberry Pi. Just replace the vegetables with your RC helicopter :slight_smile: . That should be a very good path to take to spring over that hurdle. Link to video.

I’m definitely going to learn more about Edge Impulse myself!

1 Like

Hi Tim
I’m running into problems with the facetracking script and hoping you can help me. This is the error I get in the Thonny Shell:

Python 3.7.3 (/usr/bin/python3)

Frame rate set to 40.000 fps
Traceback (most recent call last):
File “/home/pi/Downloads/”, line 65, in
faces = faceCascade.detectMultiScale(frame, 1.1, 3, 0, (10, 10))
cv2.error: OpenCV(4.4.0) /home/pi/opencv-4.4.0/modules/objdetect/src/cascadedetect.cpp:1689: error: (-215:Assertion failed) !empty() in function ‘detectMultiScale’

It also disables the camera somehow and I have to reboot to get it back. I’m new to this so it is probably something obvious.
Thanks in Advance.


How very interesting that it will disable the camera until reboot, I haven’t run into that issue before.

Lemme just do the quick triple check that you are; 1. Flashed and running the older ‘Buster’ Raspberry Pi OS. 2. Have the camera correctly connected and enabled in Raspberry Pi Configurations. 3. When you type | raspivid -t 0 | it opens up a live preview of what the camera is seeing on the desktop.

If so the next step would be to type and enter the following line into your terminal. If your still running into issues pop me some screen caps of your screen when you try to run the code :slight_smile: we’ll get it working together.

sudo apt-get install python-opencv python3-opencv opencv-data

1 Like

Yes to 1, 2 and 3. Then I ran the Python install, rebooted and bingo it’s all working.
Thanks for you help.


Hi I am having the same error come up when l run the code in thonny
faces = faceCascade.detectMultiScale(frame, 1.1, 3, 0, (10, 10))
cv2.error: OpenCV(4.4.0) /home/pi/opencv``-4.4.0/modules/objdetect/src/cascadedetect.cpp:1689: error: (-215:Assertion failed) !empty() in function ‘detectMultiScale

Im running os version 11 (bullseye) which could be the issue and l couldnt install python-opencv as it came back package not found. But the rest of the install completed ok. Any ideas? Thanks.