Home C++ Tracking an object with OpenCV, C++, and a webcam

Tracking an object with OpenCV, C++, and a webcam

by Sean Ziegler
Image of the tracker locating ball

If you want to automate a task with even a moderate level of complexity, chances are your “robot” will need to see. There are many ways to track an object with OpenCV, but one of the simplest is tracking by color.

OpenCV

OpenCV is arguably the most powerful (and the most popular) computer vision library on the market. It is available on Windows, Linux, Mac, and iOS/Android. It also has an interface for C++, Python, and Java.

The best part is: it’s free for both personal and commercial use. You can use it to identify objects, stitch together images, apply transformations and blurs, and a ton of other stuff that is way above my head.

Tennis Ball Tracking

What we’re going for is a very (and I mean very) simple implementation of computer vision, but it works for a single object on-screen with no similar colors. If you want to track multiple objects at once or have similar colors in the background, you would need to change the approach.

Setting up OpenCV

I will use OpenCV 3.4.3 Visual Studio 2019 Community for this project. Let’s run through how to set up OpenCV.

Go to the OpenCV releases page and choose the version you need. If you’re on Windows, you’ll be downloading an executable file. It’s actually not an installer, just a self-extracting archive.

I highly recommend extracting it to a folder such as C:\opencv and eliminating any spaces in your path since spaces can cause problems later.

Extracting OpenCV files.

We need to add OpenCV to our computer’s PATH environment variables. There are two equally valid ways of doing this: Using the windows system properties or the setx command. To do it with setx, open a command prompt and run:

setx /m OPENCV_DIR <where_openCV_extracted>\build\x64\vc15

Great, now we need to show Visual Studio the location of these files so it can compile and link them when we build our program. Open Visual Studio and right-click on your project (I’ll assume you know how to create a solution and project in Visual Studio). Click properties and then C/C++. You should see the screen below.

Adding OpenCV include directory.

Right-click on “Additional Include Directories” and add:

<where_openCV_extracted>\build\include>

Click on “Linker” and right-click on “Additional Library Directories”.

Adding OpenCV lib directory.
<where_openCV_extracted>\build\x64\vc15\lib>

Visual Studio should have all the files it needs for compiling and linking when we build our projects.

Writing the code

First, we have some boilerplate for importing libraries:

#include "openCV.h"
#include "track.h"

#define MINTRACKAREA 50

using namespace cv;

Now, let’s define a function that takes a stream from the webcam.

Mat Tracker::trackBall(VideoCapture cap){

Mat frame;
		
//Resize large images to reduce processing load
cap >> frame;

Okay, now we will take the image frame and convert it from RGB to HSV. HSV is a little easier to handle when we begin thresh-holding the colors of the tennis ball later.

        //Convert RGB to HSV colormap
//and apply Gaussain blur
Mat hsvFrame;
cvtColor(frame, hsvFrame, CV_RGB2HSV);

Applying a small 1 x 1 Gaussian blur will help reduce the noise in the image and improve the accuracy of our track.

blur(hsvFrame, hsvFrame, cv::Size(1, 1));

Okay, we are finally ready to threshold the image. The inRange function assigned any pixel in its range to a 1 and any pixel outside its range to a 0. This should result in a black-and-white picture of a tennis ball.

//Threshold 
Scalar lowerBound = cv::Scalar(55, 100, 50);
Scalar upperBound = cv::Scalar(90, 255, 255);
Mat threshFrame;
inRange(hsvFrame, lowerBound, upperBound, threshFrame);

We are well on our way. Now that we have a black-and-white image, we need to find the center of the ball. OpenCV includes a function known as moments that can automatically calculate the centroid of the binary image.

//Calculate X,Y centroid
Moments m = moments(threshFrame, false);
Point com(m.m10 / m.m00, m.m01 / m.m00);

Finally, let’s just draw a marker over the centroid and show the image.

//Draw crosshair
Scalar color = cv::Scalar(0, 0, 255);
drawMarker(frame, com, color, cv::MARKER_CROSS, 50, 5);

imshow("Tennis Ball", frame);
imshow("Thresholded Tennis Ball", threshFrame);

return threshFrame;
};

Awesome! This code will repeat every time the webcam sends a new frame. The result is a ball with a crosshair drawn right over the center.

Final Product

The final product.

We can make many improvements to this code in the future. It could use a mechanism to detect if the ball is too far or off-screen. Support for multiple colors would also be a significant improvement for the future.

You may also like

Leave a comment

This website uses cookies to improve your experience. We'll assume you're ok with this, but you can opt-out if you wish. Accept