Face and Movement Tracking Pan-Tilt System with Raspberry Pi and OpenCV

Hey Adrian,

Very neat looking droid! Unfortunate to hear you now have more errors than before, definitely not ideal.

As sam mentioned, installing libpng12-dev may have overwritten the current libjasper1 and caused some issues. It wouldn’t hurt to send some of the error messages you are receiving.

All the links to download/flash the old Buster version of PiOS are in that guide that Sam sent.

It should be relatively straightforward if you’re still up for it. Once you have Buster installed it’ll become simply a matter of redoing the steps in the guide to set it up.

Don’t hesitate to let us know if you need help with the process!

1 Like

Hi Gents,
I’m inspired and felt I was letting the side down giving up on this.
As suggested, I installed the Buster OS and followed your process carefully,
and guess what… It works. So chuffed and really do appreciated your time and effort you folks have put in to this.
It seems though, I do have one more request… Predictably.
How can I reverse the servo’s direction, Any ideas?

Many Thanks
Adrian

3 Likes

Hi Adrian,

Glad to hear that you’re on your way to a working pan-tilt system. Just to clarify is the camera only tracking in one direction and not moving back? You aren’t reaching the hardware limits of your system or servos? Could you adjust the starting point or range in the script to move the camera further?

You’re so close to the finish line we’ll be ecstatic to see you cross it.

2 Likes

Ok, Is see why you would say that, what I haven’t mentioned to you is that I have modified the hardware, this is to fit inside a Star Wars Droid that I am making, the head in particular… My Bad.
I am still using the Pimoroni PT Hat, but the gimbal and servos have been removed and replaced with the Droids Pan and Tilt mech, which is inverted to the Pimoroni’s.
Any ideas to reverse the servo action?

2 Likes

Hey @Adrian284751,

Assuming you are using the code provided in this guide it should be fairly easy to invert the pan and tilt functions.

The following code is a slightly modified version of the code provided in the guide with these values inverted. Let me know if this works for you!

#!/usr/bin/env python
import cv2, sys, time, os
from pantilthat import *

os.system('sudo modprobe bcm2835-v4l2')
os.system('v4l2-ctl -p 40')

FRAME_W = 320
FRAME_H = 200

cam_pan = 40
cam_tilt = 20

cascPath = '/usr/share/opencv/lbpcascades/lbpcascade_frontalface.xml'
faceCascade = cv2.CascadeClassifier(cascPath)

cap = cv2.VideoCapture(0)
cap.set(cv2.CAP_PROP_FRAME_WIDTH,  320)
cap.set(cv2.CAP_PROP_FRAME_HEIGHT, 200)
time.sleep(2)

pan(cam_pan-90)
tilt(cam_tilt-90)
light_mode(WS2812)

def lights(r,g,b,w):
    for x in range(18):
        set_pixel_rgbw(x,r if x in [3,4] else 0,g if x in [3,4] else 0,b,w if x in [0,1,6,7] else 0)
    show()

lights(0,0,0,50)

while True:

    ret, frame = cap.read()
    frame = cv2.flip(frame, -1)
    
    if ret == False:
        print("Error getting image")
        continue

    gray = cv2.cvtColor(frame, cv2.COLOR_BGR2GRAY)
    gray = cv2.equalizeHist(gray)

    faces = faceCascade.detectMultiScale(frame, 1.1, 3, 0, (10, 10))
   
    lights(50 if len(faces) == 0 else 0, 50 if len(faces) > 0 else 0,0,50)

    for (x, y, w, h) in faces:
        cv2.rectangle(frame, (x, y), (x + w, y + h), (0, 255, 0), 4)

        #Get the centre of the face
        #Inverted operations by adding instead of subtracting
        x = x + (w/2)
        y = y + (h/2)

        #Correct relative to centre of image
        #Inverted operations by adding instead of subtracting
        #These changes result in the camera moving in the opposite direction
        turn_x  = float(x + (FRAME_W/2))
        turn_y  = float(y + (FRAME_H/2))

        #Convert to percentage offset
        turn_x  /= float(FRAME_W/2)
        turn_y  /= float(FRAME_H/2)

        #Scale offset to degrees
        #Inverted operation by negating to result in inverted offset
        turn_x   *= -2.5 # VFOV
        turn_y   *= -2.5 # HFOV
        cam_pan   = -turn_x
        cam_tilt  = turn_y

        cam_pan = max(0,min(180,cam_pan))
        cam_tilt = max(0,min(180,cam_tilt))

        # Update the servos
        pan(int(cam_pan-90))
        tilt(int(cam_tilt-90))

        break

    frame = cv2.resize(frame, (540,300))
    frame = cv2.flip(frame, 1)

    cv2.imshow('Video', frame)

    if cv2.waitKey(1) & 0xFF == ord('q'):
        break

# When everything is done, release the capture
cap.release()
cv2.destroyAllWindows()

Hope this helps! :slight_smile:

2 Likes

Hi Samuel,

Things are going very well with the RPI Pan Tilt Facial Tracking, and I have to say thanks for the input from you and the team.
https://youtu.be/T_fsh5O1HB8?si=4WM6Oez-keeps5bF

Kind Regards

Adrian

2 Likes