PiBox » piboxd

Now that I've settled on mjpg-streamer for webcam playback, this won't be so easy. What has to happen is:

1. The inbound image from V4l2 has to be pre-processed before it is passed to the JPEG library.
2. The UVC driver for V4l2 may provide image processing capabilities with the V4L2_CID_HFLIP user command but its not clear if all webcams will support this. See the Linux USB Video Class (UVC) driver documentation
   in the linux kernel (Documentation/uvcvideo.txt).
3. The JPEG library doesn't provide image rotation/flip.

If V4L2_CID_HFLIP were added to input_cmd():input_uvc.c and it worked that would be the easiest solution. Here's a snippet that might show if it works or not though this may be v4l1 and not v4l2.

If not, that would leave processing internal to mjpg-streamer. It looks as though a processor could be added in cam_thread() at about input_uvc.c:710. Here a YUV image has been compressed into a jpeg buffer or a jpeg image has been read from the camera and placed in the buffer. In either case we have a jpeg image ready to be rotated. The question is: what library can I use to rotate a jpeg image?

There is an HTML/CSS option to do a transform that does rotate the image but it stops the stream. There may be a Javascript solution for this, however. And the mjpeg-streamer team has an apparent solution as well.

I compiled the attached program and ran it. The output shows the available v4l2 controls for my webcam, and that the HFLIP/VFLIP options are not available:

Control Brightness
Control Contrast
Control Saturation
Control White Balance Temperature, Auto
Control Gain
Control Power Line Frequency
Control White Balance Temperature
Control Sharpness
Control Backlight Compensation

So I can't use V4l2 to flip the image. At least not on every webcam. And I'll need a consistent solution for all webcams, which means some kind of Javascript rotation.

css transforms work on the desktop but I think the processing power required is too great for it to work on Android phones. I don't think I can use client side processing for this. It will have to be server side processing, and that probably means flipping the image inside of mjpeg_streamer.

mjpg_streamer works with mjpeg images from the cameras. It will only assume YUV from the camera if you tell it. If it is YUV, the image must be converted to MJPEG first. This slows the stream down considerably since it isn't hardware accelerated with the DSP on the Pi. And even if it was, I'd still probably have to do a transform on it and that would take more CPU cycles. It's just likely to be too slow.

Fortunately I found that the CSS transforms work just fine if I reduce the size of the image captured from the camera. I dropped it to 288x216 (320x240 would probably work too) but at 30FPS. Then I applied an X-axis rotation using CSS transforms. This is equivalent to a flip. I checked that flipping the camera over got a right-side up image. The trick is to apply the flip to the div container and not to the <img> tag. This also means the controls I added in the div with the image are flipped, so they have to be moved out of the div and properly aligned, probably by creating a top level container with two div's: one for the image and one for the controls.

I tested the flipped image on my phone. It was slow but worked. I tested it on my Galaxy Tab 2 and it worked just as well as on the desktop. So the problem with the phone is probably just hardware - not enough horsepower there.

I may want to provide additional tweak options that get passed to piboxd to change the size of the image and frame rate. I'd rather not have the client side doing scaling either in order to squeeze out as much performance on that side as possible.

1. Fix the webcam.tmpl to have a wrapping div with two divs, with one of them a new div for the controls.
2. Update the pibox.css to support the new div layouts.
3. Add a control to flip the image. This will have to change the css using Javascript on the image's wrapping div.
4. Possibly find some icons to use in the buttons. The fam-fam-fam icons would probably be sufficient.
5. If possible, add some scaling buttons and support in pibox.php for restarting the stream with new resolution settings in piboxd.

Fixed webcam.tmpl to use div wrappers to allow flipping the wrapper for the image without flipping the controls.
Added flip control to reload/quit options. Made them all real buttons that call javascript.
Flip top/bottom border stylings so the appearance of the video player border remains the same on flips.

I'll look for some icons to make the buttons prettier and look into adding a scaling button to make the image bigger or smaller.

For 0.7 I don't need scaling or fancy icons for buttons. What I have is good enough. I'm looking for a meaningful prototype for 1.0 so for now, this issue can be closed.