SERINDA – Camera object

I had issues where reinitializing VideoCapture was bombing the server when I’d go back to it from another page or instantiate more than one camera.  I wrote a quick Camera object for opencv4nodejs and tested it with my current code.

The current code stems from examples from the opencv4nodejs codebase and this project with this article.  There is nothing wrong with the code.  The examples were designed to run as one-offs.  I need them to run differently.

runVideoFaceDetection takes src which is the camera port, detectFaces is a method, and callback is another method.

grabFrames then creates the VideoCapture then on an interval it takes a frame from the capture and sends it back to the face detection where it will have a square drawn and sent back to the page.

grabFrames(videoFile, onFrame) {
    const cap = new cv.VideoCapture(videoFile);
	setInterval(() => {
	    let frame = cap.read();
	    // loop back to start on end of stream reached
	    if (frame.empty) {
	        cap.reset();
		frame = cap.read();
	    }
	    onFrame(frame);
	}, this.camInterval);
}

runVideoFaceDetection(src, detectFaces, callback) {
    this.grabFrames(src, (frame) => {
	const frameResized = frame.resizeToMax(320);

	// detect faces
	const faceRects = detectFaces(frameResized);
	if (faceRects.length) {
	     // draw detection
	     faceRects.forEach(faceRect => this.drawFaceBorder(frameResized, faceRect));
	}
        callback(cv.imencode('.jpg', frameResized));
    });
}

This would not work for what I needed to do.  I need to be able to open multiple cameras at a time.  Even run multiple filters such as a grid, colors, etc. So I changed the code above a little by making a Camera class in Javascript.  When I finish porting to TypeScript I might port it over.

I have a little more work to do to this class for passing in filters.  However, this works exactly as I need it to and doesn’t break the server.

The constructor sets up our camera object with what it needs.  I can set the webPort and resizeMax to whatever port I have the camera on and whatever resolution I want the video to display.  This means my main camera on my laptop is /0/ and my USB camera is /10 so I can set camera.webPort = 10; for the second camera and it works just fine.

I’ve also set the release in this class so the webcam turns off.  This is because otherwise the webcam will remain on always.  So I can call camera.stopCamera() when exiting a page or component.  If I want the camera to run passively, I can leave it on and still run filters.  It takes processor power, but I can still do that.

const cv = require('opencv4nodejs');

class Camera {
    constructor() {
        this.camFps = 10;
        this.camInterval = Math.ceil(1000 / this.camFps);
        this.webPort = -1;
        this.resizeMax = 320;
    }

    startCamera() {
        if (!this.cap) {
            this.cap = new cv.VideoCapture(this.webPort);
        }
    }

    //todo: rewrite to make it so filter and detectFaces are better callbacks
    intervalSet() {
        setInterval(() => {
            let frame = this.cap.read();
            // loop back to start on end of stream reached
            if (frame.empty) {
                this.cap.reset();
                frame = this.cap.read();
            }

            this.onFrame(frame);
        }, this.camInterval);
    }

    onFrame(frame) {
        const frameResized = frame.resizeToMax(this.resizeMax);

        if (this.filter) {
            this.filter(this.detectFaces, frameResized);
        }

        this.callback(cv.imencode('.jpg', frameResized));
    }

    getVideo(filter, detectFaces, callback) {
        this.callback = callback;
        this.filter = filter;
        this.detectFaces = detectFaces;

        this.intervalSet(this.cap);
    }

    stopCamera() {
        this.cap.release();
    }
}

module.exports = new Camera();

The new code for running on the backend is this:

faceRects(detectFaces, frameResized) {
   // detect faces
   const faceRects = detectFaces(frameResized);
   if (faceRects.length) {
      // draw detection
      faceRects.forEach(faceRect => this.drawFaceBorder(frameResized, faceRect));
   }
}

runVideoFaceDetection(src, detectFaces, callback) {
   camera.webPort = 0;
   camera.startCamera();
   camera.getVideo(this.faceRects, detectFaces, callback);
}

Where detectFaces and callback are methods passed from another javascript component (and also discussed above one of the items I need to fix in the camera class as I call the callback in the class and shouldn’t).  faceRects is the filter I’m using to overlay on the image.  For now, you can see the direction I’m heading.

I’ll clean this class up at some point so it’s better for others to use.  For now, it does what I need in the contexts that I’m using it.