I said, "Do you speak-a my language?"

In which we learn how to turn a buffer of bytes into an image OpenCV can work with.

If you followed my previous post, you may have realized you can produce, with very few modifications, a stream of MJPEG (jpeg) images captured from the webcam. You may also be left wondering how to work with these images using OpenCV. It might actually be easier than you imagine.

The little bit of Python code below will display a single image. The code loads an image from a file into a buffer, converts the buffer into an OpenCV image, and displays the image. If you are receiving the image as a buffer already, you can forgo the loading step.

import cv2
from cv2 import cv
import numpy as np

# let's load a buffer from a jpeg file.
# this could come from the output of frameGrabber
with open("IMAGE.jpg", mode='rb') as file:
    fileContent = file.read()

# convert the binary buffer to a numpy array
# this is a requirement of the OpenCV Python binding
pic = np.fromstring(fileContent, np.int8)

# here is the real magic
# OpenCV can actually decode a wide variety of image formats
img = cv2.imdecode(pic, cv.CV_LOAD_IMAGE_COLOR)

while True:
    cv2.imshow("image", img)
    key = cv2.waitKey(20)
    if key == 27: # exit on ESC        

For those of you working in C, you can convert a buffer into an image with this (tested in Visual Studio 2012)

#include <stdio.h>
#include <malloc.h>
#include "opencv2\core\core_c.h"
#include "opencv2\highgui\highgui_c.h"

// used to load a jpeg image into a buffer
int load_buffer(const char *filename, char **result) 
	int size = 0;
	FILE *f = fopen(filename, "rb");
	if (f == NULL) 
		*result = NULL;
		return -1; // -1 means file opening fail 
	fseek(f, 0, SEEK_END);
	size = ftell(f);
	fseek(f, 0, SEEK_SET);
	*result = (char *)malloc(size+1);
	if (size != fread(*result, sizeof(char), size, f)) 
		return -2; // -2 means file reading fail 
	(*result)[size] = 0;
	return size;

int main() 
	char *buffer; 
	int size;
	CvMat mat;
	IplImage *img;

	// load jpeg file  into buffer
	// this could come from the output of frameGrabber
	size =load_buffer("IMG.jpg", &buffer);
	if (size < 0) 
		puts("Error loading file");
		return 1;

	// create a cvMat from the buffer
	// note the params: height, width, and format
	mat = cvMat(1080, 1920, CV_8UC3, (void*)buffer);
	// magic sauce, decode the image
	img = cvDecodeImage(&mat, 1);

	// show the image
	cvShowImage("image", img );

	// wait for a key

	// release the image

	return 0;

For further information see the OpenCV Docs.

OpenCVjpeg.py (2.24 kb)
OpenCVjpeg.c (2.78 kb)