‘Cause I got something for you. It is shiny, it is clean.

In which we use OpenCV to track a laser.

Building on my article about using HSV thresholding to isolate features in video frames, below is a video showing the use of OpenCV to track a red laser.  The majority of the algorithm is the same as presented in the previous article; the only addition is actually determining the area of interest after thresholding.  In order to preventing spoiling anyone's fun, I am not going to post the code as of now.

Hint: think about how to find the largest area in white after thresholding.

If you would like to compare algorithms or just want to know what I did, drop me a note via the contact form.

In preliminary testing, the algorithm seems robust enough to handle changing lighting conditions as well as differing background colors; however, the angle of reflection of the laser can cause issues.  For best results, the laser should be relatively perpendicular to the image plane.  As the laser approaches being parallel to the image plane, less laser light is reflected back to the camera.  The same applies to irregular objects or surfaces that scatter the laser at angles away from the camera.  This can be seen at the end of the video when the laser is reflected off the lamp shade.

Testing was done using a Logitech C920.  Depending on the laser used and ambient lighting conditions, turning off auto white balance may prevent the intensity of the laser from being interpreted as white.

As far as applications, I leave that to the reader.

Come together, right now...

In which we join files line by line.

Working in IT, there is no shortage of "grunt work" that is amenable to one off scripts or applications.  In an effort to prevent others (and possibly myself) from having to reimplement these, I am publishing these as useLESS tools.

useLESS tools: scripts or programs written to paper over the shortcomings or deficiencies in applications or processes.  You could use less, and they would be made useless, if you could reimplement everything in light of the experience gained by doing it the last time.

Today I give you our first useLESS tool - fjoin

At work I am currently involved in integrating with an external vendor.  As part of the process, I have been provided with pseudo realistic test data to process.  Being pseudo realistic, it doesn't really match with what our system expects and must be "massaged".  That is to say, nothing works.

In particular, I have two text files.  One text file has the IDs used by the vendor, and the other has the IDs our system expects.  I need a way to turn these into SQL update statements, and it needs to be repeatable so I can rerun it as testing progresses.  What to do?

Using notepad++ or you favorite editor, it is fairly easy to record a macro that inserts

UPDATE [TABLE] SET [FIELD] = '

before each line of one file and

' WHERE ID = 

at the beginning of each line in the other so that if you took line x from the first file and line x from the second file and joined them, you would get something like

UPDATE [TABLE] SET [FIELD] = 'OURID_123' WHERE ID = 42

The real trick is merging the two files line by line.  fjoin does just this.  It takes two or more files and joins them line by line stopping at the end of the shortest file.  The result is sent to the terminal so you can pipe this into another program or redirect to a file.

If you would like to create an executable from this code, you can use pyinstaller

pysinstaller -F fjoin.py

This will create a single (rather large) executable for you.

import sys

if len(sys.argv) == 1:
    print 'No files specified'
    sys.exit(0)

files = []
# skip the name of the program
for arg in sys.argv[1:]:
    try:
        files.append(open(arg))
    except IOError:
        print 'ERROR:', arg, 'does not exist'
        sys.exit(1)

reading = True
composite = []

while reading:
    for f in files:
        line = f.readline()
        # stop as soon as a file comes up short
        if not line:
            reading = False
            break
        else:
            # remove newline
            composite.append(line.rstrip('\n'))
    # only output if we joined across all files
    # in other words, no partial lines
    if len(composite) == len(files):
        print ''.join(composite)
    composite = []

for f in files:
    f.close()

fjoin.py (2.36 kb)

I said, "Do you speak-a my language?"

In which we learn how to turn a buffer of bytes into an image OpenCV can work with.

If you followed my previous post, you may have realized you can produce, with very few modifications, a stream of MJPEG (jpeg) images captured from the webcam. You may also be left wondering how to work with these images using OpenCV. It might actually be easier than you imagine.

The little bit of Python code below will display a single image. The code loads an image from a file into a buffer, converts the buffer into an OpenCV image, and displays the image. If you are receiving the image as a buffer already, you can forgo the loading step.

import cv2
from cv2 import cv
import numpy as np

# let's load a buffer from a jpeg file.
# this could come from the output of frameGrabber
with open("IMAGE.jpg", mode='rb') as file:
    fileContent = file.read()

# convert the binary buffer to a numpy array
# this is a requirement of the OpenCV Python binding
pic = np.fromstring(fileContent, np.int8)

# here is the real magic
# OpenCV can actually decode a wide variety of image formats
img = cv2.imdecode(pic, cv.CV_LOAD_IMAGE_COLOR)

cv2.namedWindow("image")
while True:
    cv2.imshow("image", img)
    
    key = cv2.waitKey(20)
    if key == 27: # exit on ESC        
        break
    
cv2.destroyAllWindows()

For those of you working in C, you can convert a buffer into an image with this (tested in Visual Studio 2012)

#include <stdio.h>
#include <malloc.h>
#include "opencv2\core\core_c.h"
#include "opencv2\highgui\highgui_c.h"

// used to load a jpeg image into a buffer
int load_buffer(const char *filename, char **result) 
{ 
	int size = 0;
	FILE *f = fopen(filename, "rb");
	if (f == NULL) 
	{ 
		*result = NULL;
		return -1; // -1 means file opening fail 
	} 
	fseek(f, 0, SEEK_END);
	size = ftell(f);
	fseek(f, 0, SEEK_SET);
	*result = (char *)malloc(size+1);
	if (size != fread(*result, sizeof(char), size, f)) 
	{ 
		free(*result);
		return -2; // -2 means file reading fail 
	} 
	fclose(f);
	(*result)[size] = 0;
	return size;
}

int main() 
{ 
	char *buffer; 
	int size;
	CvMat mat;
	IplImage *img;

	// load jpeg file  into buffer
	// this could come from the output of frameGrabber
	size =load_buffer("IMG.jpg", &buffer);
	if (size < 0) 
	{ 
		puts("Error loading file");
		return 1;
	}

	// create a cvMat from the buffer
	// note the params: height, width, and format
	mat = cvMat(1080, 1920, CV_8UC3, (void*)buffer);
	// magic sauce, decode the image
	img = cvDecodeImage(&mat, 1);

	// show the image
	cvShowImage("image", img );

	// wait for a key
	cvWaitKey(0);

	// release the image
	cvReleaseImage(&img);


	return 0;
}

For further information see the OpenCV Docs.

OpenCVjpeg.py (2.24 kb)
OpenCVjpeg.c (2.78 kb)

you gotta keep 'em separated

In which we threshold video frames using HSV values.

In my previous post, I provided a tool that allows one to determine the H, S, and V values of a pixel or pixels in an image for use in setting a threshold on an image or video frame.  Today, I will show you what these values are good for, namely setting a threshold on frames in a video or webcam stream.

You may download the Python code here HSVthresholder.
The code is copyright © 2013 Matthew Witherwax and released under the BSD license.

To use the application, simply supply a command line argument for either a video file or a webcam using -v.  In the case of a webcam, specify dev# where # is the number of the device.  If you only have one webcam, you should be able to pass dev0; otherwise, you may start at dev1 and increment until you find the webcam you would like to use.  In the case of a video file, the application will loop the video indefinitely until you exit with esc.  For other options, pass -h on the command line.

The general flow of the program is:

  1. Capture frame from video or webcam
  2. Dilate image
  3. Convert to HSV
  4. Threshold image
  5. Erode image
  6. Display image

 

You can find more information about dilation and erosion here [1] and the function inRange() used for thresholding here [2].

All the interesting bits in the code are documented with comments, but I would like to draw attention to a two operations with parameters you may wish to tune.

In both dilating and erosion, a structuring element is supplied.  Possible values to adjust are the element itself which may be one of cv2.MORPH_RECT, cv2.MORPH_CROSS, or cv2.MORPH_ELLIPSE as well as the size, the second argument to cv2.getStructuringElement.  In addition, you may adjust the number of times the operation is applied by adjusting the iterations parameter.  While tuning these parameters, you might find it beneficial to create a third named window to display the frame after the dilation so you can see what effects your changes have.

Dilating

# here we dilate the image so we can better threshold the colors
if self.dilate:
	element = cv2.getStructuringElement(cv2.MORPH_CROSS,(5,5))
	frame = cv2.dilate(frame, element, iterations=5)


Erosion

# here we erode the image to remove noise
if self.erode:
	element = cv2.getStructuringElement(cv2.MORPH_RECT,(5,5))
	frameHSV = cv2.erode(frameHSV, element, iterations=2)

And here are some screenshots that show the application isolating the red grip on a pen.  The red grip show up as white while the rest of the image is blacked out.  If you look closely, you will see some stray white dots.  Depending on your needs you may either ignore these or tune the parameters until they disappear.  Now that the grip has been isolated, you can track it, but that is a topic for another post.

[1] http://docs.opencv.org/doc/tutorials/imgproc/erosion_dilatation/erosion_dilatation.html
[2] http://docs.opencv.org/modules/core/doc/operations_on_arrays.html#cv2.inRange

HSVthresholder.py (6.38 kb)

Was that Imperial Red, Lust, or Crimson? ...I am pretty sure it was just red.

In which we find the H, S, and V values for an image's pixels.

Recently I have been working on an application that requires me to locate a small colored dot in an image or video frame.  In order to accomplish this, I have been using OpenCV.  For reasons outside the scope of this post, I am working with the images/frames in the HSV color space.  You can find more information about HSV in general and OpenCV's HSV implementation here [1].  There you will also find a nifty tool - ColorWheelHSV - to help you visualize OpenCV's HSV implementation.  However, if you are like me, you might find it difficult to determine proper H, S, and V values by eyeballing your target and the output of ColorWheelHSV.

Enter HSVpixelpicker.

The Python script below will allow you to determine the H, S, and V values of a given pixel in an image as OpenCV sees it.  Simply pass the image file name with -f on the command line, click on the pixel you are interested in, and the H, S, and V values will be printed to the console.  Using this tool, you can check the exact value of the pixel or pixels you are interested in.  The more samples you have to test, the better the idea you will get of the H, S, and V values you are dealing with.  You can then refer to ColorWheelHSV to select your ranges.

The code is copyright © 2013 Matthew Witherwax and released under the BSD license.

from optparse import OptionParser
import cv2
from cv2 import cv

class HSVpixelpicker:
    def __init__(self):
        # original image
        self.image = None
        # image converted to HSV
        self.imageHSV = None
        
        # create window to show image
        cv2.namedWindow('image')
        # wire click handler
        cv.SetMouseCallback('image',self.on_mouse, 0);
        
    # handles left clicking on the image
    # gets the pixel under the cursor and prints its HSV values
    def on_mouse(self, event,x,y,flag,param):
      if(event==cv.CV_EVENT_LBUTTONDOWN):
        pixel = self.imageHSV[y][x]
        print 'H:',pixel[0],'\tS:',pixel[1],'\tV:',pixel[2]
        
    def open_image(self, filename):
        self.image = cv2.imread(filename)
        self.imageHSV = cv2.cvtColor(self.image,cv2.COLOR_BGR2HSV)
        
    def show(self, filename):
        self.open_image(filename)
        while True:
            cv2.imshow('image', self.image)
            
            # show image until user presses the esc key
            if cv.WaitKey(10) == 27:
                break
            
        # clean up
        cv2.destroyAllWindows()

if __name__ == "__main__":
    parser = OptionParser()
    parser.add_option('-f', '--file', action='store',
        type='string', dest='filename')
    (options, args) = parser.parse_args()

    hsv_picker = HSVpixelpicker()
    hsv_picker.show(options.filename)

[1] http://www.shervinemami.info/colorConversion.html

HSVpixelpicker.py (3.03 kb)

Transcode FLV to MP4

On a recent business trip to Kuala Lumpur and Dubai, I had a 16 hour flight and thought I would watch some old Udacity videos.  I downloaded some lectures from Udacity[1] for offline viewing and discovered they were in FLV format.  Unfortunately, the media player built into Windows 8, Windows Media Player, does not play FLV videos out of the box.  There are many solutions to this problem from installing various codecs and alternative video players to transcoding.  I chose to do the latter, and this is my story.

I know there are many different programs that promise to take care of transcoding for you, but I prefer to install as few unknown/untrusted bits of software as possible.  As a quick method to bulk convert the files, I wrote the Python script below. You will need two things to run this script:

  1. Python - I wrote this using version 2.7.5 from the Anaconda package[2]
  2. ffmpeg[3]
from os import listdir, remove
from os.path import isfile, join
import subprocess

vid_path = '[PATH TO VIDEOS]'
ffmpeg_path = '[PATH TO ffmpeg.exe]'

vids = [f for f in listdir(vid_path) if isfile(join(vid_path,f))]

for vid in vids:
    file_name = vid[:-4]
    subprocess.call(ffmpeg_path + ' -i ' + join(vid_path,vid)
        + ' -qscale 0 -ar 22050 ' + join(vid_path,file_name)
        + '.mp4', shell=True) 
    remove(join(vid_path,vid))

This script is quick and dirty.  It simply loops over all files (hopefully just .flv videos are present) in the vid_path directory specified and uses ffmpeg to transcode them to mp4.  Be sure to set ffmpeg_path to the path of ffmpeg.exe on your system.  After a video is transcoded, it is deleted. One very import note: beware of shell=True in subprocess.call().  In some cases, this can be a security risk.  Please see [4].

Possible improvement opportunities for the reader:

  • make removing the original file conditional
  • only join vid_path and vid once
  • filter discovered files so that only .flv files (or the target type of your choice) are processed
  • traverse nested directories looking for files
  • research the arguments ffmpeg accepts and provide a way to change them


[1] Udacity
[2] Continuum Analytics
[3] ffmpeg.zeranoe
[4] subprocess.call()