Feature #239

Add support for ffserver to stream mp4 (or switch to mjpg-streamer)

Added by Hammel about 7 years ago. Updated about 7 years ago.

Status:ClosedStart date:25 Oct 2013
Priority:ImmediateDue date:
Assignee:Hammel% Done:

100%

Category:04 - Root File System
Target version:0.7.0
Severity:01 - Critical

Description

Tablets don't do flash. So now I need to find another format to stream. MP4 seems fine, as tablets do HTML5 and can handle mp4 via jwplayer.

If I have to use mp4 for tablets, then there is no need for flash. crtmpserver is therefore not needed. At least I don't think it is.

According to jwplayer docs for html5 the best streaming solution is mp4 for mobile devices and flash for firefox on the desktop. I don't care about IE support as its not likely to be used in a mobile device or in development.

ffserver.conf - ffserver configuration that provides three types of streams (734 Bytes) Hammel, 25 Oct 2013 16:44

camff-new.sh Magnifier - ffmpeg script that feed ffserver (318 Bytes) Hammel, 25 Oct 2013 16:44

flash.html Magnifier - jwplayer that should place the flash stream, but doesn't (555 Bytes) Hammel, 25 Oct 2013 16:44

mp4.html Magnifier - jwplayer that should place the mp4 stream, but doesn't (468 Bytes) Hammel, 25 Oct 2013 16:44

webm.html Magnifier - jwplayer that should place the webm stream, but doesn't (471 Bytes) Hammel, 25 Oct 2013 16:44

flash.html Magnifier (378 Bytes) Hammel, 25 Oct 2013 22:50

android.html Magnifier - HTML5 page that selects video stream (414 Bytes) Hammel, 02 Nov 2013 14:24

camff-new.sh Magnifier - plays ffmpeg to ffserver (349 Bytes) Hammel, 02 Nov 2013 14:24

ffserver.conf - ffserver configuration, with multiple output streams (1004 Bytes) Hammel, 02 Nov 2013 14:24

webcam.html Magnifier - updated to call android.html (622 Bytes) Hammel, 02 Nov 2013 14:24

camff-2.sh Magnifier - Latest version of ffmpeg input stream (362 Bytes) Hammel, 03 Nov 2013 21:51

ffserver.conf - Latest version of ffserver.conf with support for mp4 (1.52 KB) Hammel, 03 Nov 2013 21:51

Associated revisions

Revision 6d612325
Added by Hammel about 7 years ago

RM #239: Clean up web interface to properly display webcam video.

Revision 30d2d3ae
Added by Hammel about 7 years ago

RM #239: Updated piboxd to support webcam playback via mjpg-streamer instead of ffmpeg.
Added Dia diagram showing message passing protocol between web (php) and piboxd.

History

#1 Updated by Hammel about 7 years ago

  • Description updated (diff)

#2 Updated by Hammel about 7 years ago

Multiple experiments on the desktop to test ffserver streaming. I've come up with an ffmpeg line that works, an ffserver.conf that works and three sample web pages with JWPLayer embedded that don't work, but should. All are attached.

What's interesting is that if you run mplayer on the ffserver stream (such as http://host:port/webcam.webm) it plays almost perfectly, with little to no delay. It's only when you stick the stupid browser on the end as your "player" that things get mucked up.

#3 Updated by Hammel about 7 years ago

  • % Done changed from 10 to 20

Got it working tonight on my laptop to the Galaxy tab - and I got it working with Flash without using JWPlayer or a browser. Instead, I tried to open the flv stream from ffserver directly in the "internet browser" on Android (I have no idea which browser this is except it isn't firefox and doesn't appear to be Google Chrome or Opera). When I tried to open this URL the browser asked me which app I wanted to use to view the video. I chose MXPlayer and it worked fine. In fact, it doesn't even lag. It has better latency than playing the stream under Google Chrome directly on the laptop on which the video is being streamed!

So now the question is how I do create a UI that takes the user to the correct URL so the browser will ask which app to use to play the video? I can do a redirect if the browser is on a tablet (based on screen size or OS?). I'm not sure why embedding this in a video tag wouldn't do the same thing, but it didn't.

Without a web page to start the stream I can't control the stream, re: disable it after the user leaves the page. I could start the stream and then kill it due to no-heartbeats after a longer timeout is set.

#4 Updated by Hammel about 7 years ago

I've attached a way to use a web page to redirect to the flv stream.

#5 Updated by Hammel about 7 years ago

  • % Done changed from 20 to 30

What needs to happen to complete this task.

  1. piboxd
    1. spawnStream() needs context arg, which determines which type of args to pass to ffmpeg
    2. handleStream() needs to pass context arg to spawnStream
      1. context arg is upper 16 bits of header passed from pibox.php
    3. spawnStream() should check if stream is already running.
      1. If so, drop request
    4. timer() should timeout based on context
      1. webcam: timeout after 10 minutes
      2. video: never timeout - end of playback (ffmpeg child dies) should cause session to end
  1. piboxwww
    1. pibox.php
      1. webcam() needs to look at "type" GET variable and set upper 16 bits of header
        1. android = 1
        2. ios = 2
        3. desktop = 3
      2. webcam() needs to read redirect template
        1. replace IPADDR tag with PiBox IP address
        2. return to caller to redirect to ffserver
    2. webcam.html calls pibox.php?function=webacam&type=<one of>
      1. android
      2. ios
      3. desktop
  1. ffserver pkg
    1. ffserver.conf
    2. ffserver init script

#6 Updated by Hammel about 7 years ago

  • % Done changed from 30 to 40

Code is implemented. Was missing index.html pointing to webcam.html but that's implemented now too.

First test shows that flash version still works fine. ffserver version doesn't appear to be working. Some notes:

  1. ffserver was started manually before the test from an ssh login.
  2. "Android" request does start the camera running (LED is lit on camera) which means ffmpeg is started
  3. Galaxy tab does not connect to the mp4 stream on ffserver (neither does phone, but that wasn't tested manually previously to show it worked)
  4. ffmpeg session was not killed by piboxd after 10 minutes. It's still running a day later.
  5. Killing ffmpeg/ffserver and restarting ffserver did not allow a new mp4 stream to start. I think piboxd prevents this due to new handling of webcam service request. Kill piboxd cleared this problem.
  6. After kill piboxd and making another android request (from my desktop) the camera started but then, after a short time, stopped. LED went off. ffserver and piboxd are still running. ffmpeg is an exited child waiting to be reaped.

#7 Updated by Hammel about 7 years ago

I'm manually testing streaming with ffmpeg/ffserver to android and nothing seems to work on my phone, a Kyocera with Android 4.0.4 (Ice Cream Sandwich).

Attached are my ffserver.conf and the ffmpeg command I use. I'm also attaching an HTML5 file that I use to select a video stream.

I'm wondering if the ffmpeg on my distro is too old (0.8.4). The latest stable is 1.2.4 and there is a 2.1 release as well. I've been through this before - see RM #192. I'll need to cross compile a release version of ffmpeg, then package it under /opt for testing. If it works better, then I may disable the ffmpeg build in buildroot if it doesn't affect any other packages.

I think I have the right config for ffserver and am feeding it with ffmpeg correctly. The question is how do I tell a browser (any browser) to play the stream on Android. The <video> tag should handle this, but I'm not getting any mp4 playback. It's possible that ffmpeg/ffserver doesn't encode it correctly with 0.8.4.

#8 Updated by Hammel about 7 years ago

I created an ffmpeg opkg metabuild last night and tested it. It appears somewhat better than before but I run into this problem when running the ffmpeg command to read from the webcam and pass to ffserver:

[libx264 0x21f1180] broken ffmpeg default settings detected
[libx264
0x21f1180] use an encoding preset (e.g. -vpre medium)
[libx264 0x21f1180] preset usage: -vpre <speed> -vpre <profile>
[libx264
0x21f1180] speed presets are listed in x264 --help
[libx264 @ 0x21f1180] profile is optional; x264 defaults to high

After much digging (which included a problem with firefox no longer allowing me access to google - a system update cleared it, I think, or maybe it was just the reboot), I found a discussion on the problem with a definitive answer. I need to use presets (via -vpre) defined by x264 and I can find those by running x264 --help. However, I didn't package x264 with ffmpeg in my metabuild, so I'm adding that now.

#9 Updated by Hammel about 7 years ago

Not much luck with the 2.1 version. I did manage to get an avi stream to play direct to ffplay running on my laptop, but nothing works to the android phone.

Here is some additional information that might help to stream mp4 over rtp without ffserver, which I got from this superuser.com discussion.
'
The attached configs seem to let the mp4 stream work on the RPi side. I might go back and try using video.js as a way to play the stream.

I should note that ffmpeg 2.1 doesn't seem to support the -profile, -preset, -tune options that the definitive answer said was in GIT back in March. The changes to the ffserver.conf got past that problem allowing mp4 streams to work, but it was just a lucky try - something I found by googling and I'm not sure where I got it. Also, none of this guide worked either.

One more thing: all this work with ffmpeg/ffserver is driving me nuts when crtmpserver says it can stream to android and iOS devices. Maybe I should just go back to that and see how I can configure it correctly for my needs.

#10 Updated by Hammel about 7 years ago

I've updated android.html and ffserver.conf on the target. The former has the defined format for the video/object/embed tags as defined by W3C schools. The ffserver.conf has streams for mpeg, mp4, flv and webm. However, when I try to run ffmpeg to feed ffserver it seg faults. I need to try running each stream in ffserver one at a time till I find out which one is causing ffmpeg to seg fault.

#11 Updated by Hammel about 7 years ago

  • % Done changed from 40 to 50
I managed to get a stream to my Android phone tonight. The tricks:
  1. Use MXPlayer and specify the network stream (skip going through a browser)
  2. Stream mpeg from ffserver
  3. Modified the ffmpeg command slightly (see target)

The stream was jumpy but didn't fall behind much. I think this can be fixed through a combination of ffmpeg buffer changes and options to the stream in ffserver.conf. But at least its streaming to Android.

So here is what I need to do next.
  1. Clean up ffmpeg cmd/ffserver.conf to get a clean stream, without as little lag and stutter as possible.
  2. Save those configs here so I don't lose them.
  3. Try to integrate video.js into a web page to play that stream.
  4. Clean up piboxwww based on
    1. Playing Flash to the desktop through crtmpserver (possibly replacing jwplayer with videojs)
    2. Playing this new config to Android either direct to the player or via videojs
    3. Providing a stub for handling iOS (yet to be tested)
    4. Make sure pibox.php is setup to request the correct streams from piboxd
    5. Remove any extraneous files in piboxwww
  5. Make sure piboxd is updated to play the correct streams

If video.js works then I'll be able to use the original setup to start/stop streams based on javascript control in web pages. If not, I'll have to use the timed control that I recently implemented in piboxd.

#12 Updated by Hammel about 7 years ago

  • % Done changed from 50 to 60

I got mjpeg-streamer cross compiled today and tested it on the desktop, android and iOS. It works perfectly on all three. I created a web page that links to it and that worked too. So I can create a CSS-wrapped page just like the front page and that will allow me to start/stop using javascript.

So ffmpeg is out and mjpeg-streamer is in for the webcam. I don't really think this is a viable option for movies, however. I'll still need to figure out how to use ffmpeg for that.

So now the steps to do are:
  1. Build a cross compiled version of mjpeg-stream. Sample with notes are in <cdw>/mjpeg-streamer. May need to convert it to autoconf to fix it.
  2. Clean up piboxwww with a new page for the webcam that just loads an <img> with the mjpeg-streamer stream. Wrap it in some CSS controls.
  3. Fix piboxd.php to play mjpeg-streamer for the webcam. Include a timeout on it of 3 minutes and have the javascript send heartbeats every 30 seconds.
  4. Get rid of extraneous files from piboxwww tree.

The good news is I don't need JWPlayer or video.js to play mjpeg streams. That simplifies the webcam playback at least. Here's a sample command line:

./mjpg_streamer -i "input_uvc.so -r 320x240 -f 15" -o "output_http.so -p 9090"

#13 Updated by Hammel about 7 years ago

  • Subject changed from Add support for ffserver to stream mp4 to Add support for ffserver to stream mp4 (or switch to mjpg-streamer)
  • Status changed from In Progress to Closed
  • % Done changed from 60 to 100

The mjpg-streamer project has been created and pushed upstream. It works just fine to stream the webcam video without excessive lag to desktops, android and iOS. The build includes a patch to allow the source tree to be cross compiled with the PiBox Development Platform. It's only dependency is libjpeg which is already in the rootfs.

The piboxwww package has been cleaned up to utilize the streaming from mjpg-streamer. This includes a proper template that includes Javascript code to stop the stream when we leave the page along with proper messaging to piboxd to start/stop the stream. Extrenous files have also been removed although I left examples of using JWPlayer in there in case I might want them for streaming movies as Flash.

Piboxd has been cleaned up to launch and manage the mjpg-streamer application instead of ffmpeg.

With all these changes, the webcam is now properly supported. This issue can be closed.

Also available in: Atom PDF