Add support for ffserver to stream mp4 (or switch to mjpg-streamer)
|Status:||Closed||Start date:||25 Oct 2013|
|Category:||04 - Root File System|
|Severity:||01 - Critical|
Tablets don't do flash. So now I need to find another format to stream. MP4 seems fine, as tablets do HTML5 and can handle mp4 via jwplayer.
If I have to use mp4 for tablets, then there is no need for flash. crtmpserver is therefore not needed. At least I don't think it is.
According to jwplayer docs for html5 the best streaming solution is mp4 for mobile devices and flash for firefox on the desktop. I don't care about IE support as its not likely to be used in a mobile device or in development.
#2 Updated by Hammel about 7 years ago
- File ffserver.conf added
- File camff-new.sh added
- File flash.html added
- File mp4.html added
- File webm.html added
- % Done changed from 0 to 10
Multiple experiments on the desktop to test ffserver streaming. I've come up with an ffmpeg line that works, an ffserver.conf that works and three sample web pages with JWPLayer embedded that don't work, but should. All are attached.
What's interesting is that if you run mplayer on the ffserver stream (such as http://host:port/webcam.webm) it plays almost perfectly, with little to no delay. It's only when you stick the stupid browser on the end as your "player" that things get mucked up.
#3 Updated by Hammel about 7 years ago
- % Done changed from 10 to 20
Got it working tonight on my laptop to the Galaxy tab - and I got it working with Flash without using JWPlayer or a browser. Instead, I tried to open the flv stream from ffserver directly in the "internet browser" on Android (I have no idea which browser this is except it isn't firefox and doesn't appear to be Google Chrome or Opera). When I tried to open this URL the browser asked me which app I wanted to use to view the video. I chose MXPlayer and it worked fine. In fact, it doesn't even lag. It has better latency than playing the stream under Google Chrome directly on the laptop on which the video is being streamed!
So now the question is how I do create a UI that takes the user to the correct URL so the browser will ask which app to use to play the video? I can do a redirect if the browser is on a tablet (based on screen size or OS?). I'm not sure why embedding this in a video tag wouldn't do the same thing, but it didn't.
Without a web page to start the stream I can't control the stream, re: disable it after the user leaves the page. I could start the stream and then kill it due to no-heartbeats after a longer timeout is set.
#5 Updated by Hammel about 7 years ago
- % Done changed from 20 to 30
What needs to happen to complete this task.
- spawnStream() needs context arg, which determines which type of args to pass to ffmpeg
- handleStream() needs to pass context arg to spawnStream
- context arg is upper 16 bits of header passed from pibox.php
- spawnStream() should check if stream is already running.
- If so, drop request
- timer() should timeout based on context
- webcam: timeout after 10 minutes
- video: never timeout - end of playback (ffmpeg child dies) should cause session to end
- webcam() needs to look at "type" GET variable and set upper 16 bits of header
- android = 1
- ios = 2
- desktop = 3
- webcam() needs to read redirect template
- replace IPADDR tag with PiBox IP address
- return to caller to redirect to ffserver
- webcam() needs to look at "type" GET variable and set upper 16 bits of header
- webcam.html calls pibox.php?function=webacam&type=<one of>
- ffserver pkg
- ffserver init script
#6 Updated by Hammel about 7 years ago
- % Done changed from 30 to 40
Code is implemented. Was missing index.html pointing to webcam.html but that's implemented now too.
First test shows that flash version still works fine. ffserver version doesn't appear to be working. Some notes:
- ffserver was started manually before the test from an ssh login.
- "Android" request does start the camera running (LED is lit on camera) which means ffmpeg is started
- Galaxy tab does not connect to the mp4 stream on ffserver (neither does phone, but that wasn't tested manually previously to show it worked)
- ffmpeg session was not killed by piboxd after 10 minutes. It's still running a day later.
- Killing ffmpeg/ffserver and restarting ffserver did not allow a new mp4 stream to start. I think piboxd prevents this due to new handling of webcam service request. Kill piboxd cleared this problem.
- After kill piboxd and making another android request (from my desktop) the camera started but then, after a short time, stopped. LED went off. ffserver and piboxd are still running. ffmpeg is an exited child waiting to be reaped.
#7 Updated by Hammel about 7 years ago
I'm manually testing streaming with ffmpeg/ffserver to android and nothing seems to work on my phone, a Kyocera with Android 4.0.4 (Ice Cream Sandwich).
Attached are my ffserver.conf and the ffmpeg command I use. I'm also attaching an HTML5 file that I use to select a video stream.
I'm wondering if the ffmpeg on my distro is too old (0.8.4). The latest stable is 1.2.4 and there is a 2.1 release as well. I've been through this before - see RM #192. I'll need to cross compile a release version of ffmpeg, then package it under /opt for testing. If it works better, then I may disable the ffmpeg build in buildroot if it doesn't affect any other packages.
I think I have the right config for ffserver and am feeding it with ffmpeg correctly. The question is how do I tell a browser (any browser) to play the stream on Android. The <video> tag should handle this, but I'm not getting any mp4 playback. It's possible that ffmpeg/ffserver doesn't encode it correctly with 0.8.4.
#8 Updated by Hammel about 7 years ago
I created an ffmpeg opkg metabuild last night and tested it. It appears somewhat better than before but I run into this problem when running the ffmpeg command to read from the webcam and pass to ffserver:
0x21f1180] broken ffmpeg default settings detected0x21f1180] use an encoding preset (e.g. -vpre medium)
0x21f1180] preset usage: -vpre <speed> -vpre <profile>0x21f1180] speed presets are listed in x264 --help
[libx264 @ 0x21f1180] profile is optional; x264 defaults to high
After much digging (which included a problem with firefox no longer allowing me access to google - a system update cleared it, I think, or maybe it was just the reboot), I found a discussion on the problem with a definitive answer. I need to use presets (via -vpre) defined by x264 and I can find those by running x264 --help. However, I didn't package x264 with ffmpeg in my metabuild, so I'm adding that now.
#9 Updated by Hammel about 7 years ago
Not much luck with the 2.1 version. I did manage to get an avi stream to play direct to ffplay running on my laptop, but nothing works to the android phone.
Here is some additional information that might help to stream mp4 over rtp without ffserver, which I got from this superuser.com discussion.
The attached configs seem to let the mp4 stream work on the RPi side. I might go back and try using video.js as a way to play the stream.
I should note that ffmpeg 2.1 doesn't seem to support the -profile, -preset, -tune options that the definitive answer said was in GIT back in March. The changes to the ffserver.conf got past that problem allowing mp4 streams to work, but it was just a lucky try - something I found by googling and I'm not sure where I got it. Also, none of this guide worked either.
One more thing: all this work with ffmpeg/ffserver is driving me nuts when crtmpserver says it can stream to android and iOS devices. Maybe I should just go back to that and see how I can configure it correctly for my needs.
#10 Updated by Hammel about 7 years ago
I've updated android.html and ffserver.conf on the target. The former has the defined format for the video/object/embed tags as defined by W3C schools. The ffserver.conf has streams for mpeg, mp4, flv and webm. However, when I try to run ffmpeg to feed ffserver it seg faults. I need to try running each stream in ffserver one at a time till I find out which one is causing ffmpeg to seg fault.
#11 Updated by Hammel about 7 years ago
- % Done changed from 40 to 50
- Use MXPlayer and specify the network stream (skip going through a browser)
- Stream mpeg from ffserver
- Modified the ffmpeg command slightly (see target)
The stream was jumpy but didn't fall behind much. I think this can be fixed through a combination of ffmpeg buffer changes and options to the stream in ffserver.conf. But at least its streaming to Android.So here is what I need to do next.
- Clean up ffmpeg cmd/ffserver.conf to get a clean stream, without as little lag and stutter as possible.
- Save those configs here so I don't lose them.
- Try to integrate video.js into a web page to play that stream.
- Clean up piboxwww based on
- Playing Flash to the desktop through crtmpserver (possibly replacing jwplayer with videojs)
- Playing this new config to Android either direct to the player or via videojs
- Providing a stub for handling iOS (yet to be tested)
- Make sure pibox.php is setup to request the correct streams from piboxd
- Remove any extraneous files in piboxwww
- Make sure piboxd is updated to play the correct streams
#12 Updated by Hammel about 7 years ago
- % Done changed from 50 to 60
So ffmpeg is out and mjpeg-streamer is in for the webcam. I don't really think this is a viable option for movies, however. I'll still need to figure out how to use ffmpeg for that.So now the steps to do are:
- Build a cross compiled version of mjpeg-stream. Sample with notes are in <cdw>/mjpeg-streamer. May need to convert it to autoconf to fix it.
- Clean up piboxwww with a new page for the webcam that just loads an <img> with the mjpeg-streamer stream. Wrap it in some CSS controls.
- Get rid of extraneous files from piboxwww tree.
The good news is I don't need JWPlayer or video.js to play mjpeg streams. That simplifies the webcam playback at least. Here's a sample command line:
./mjpg_streamer -i "input_uvc.so -r 320x240 -f 15" -o "output_http.so -p 9090"
#13 Updated by Hammel about 7 years ago
- Subject changed from Add support for ffserver to stream mp4 to Add support for ffserver to stream mp4 (or switch to mjpg-streamer)
- Status changed from In Progress to Closed
- % Done changed from 60 to 100
The mjpg-streamer project has been created and pushed upstream. It works just fine to stream the webcam video without excessive lag to desktops, android and iOS. The build includes a patch to allow the source tree to be cross compiled with the PiBox Development Platform. It's only dependency is libjpeg which is already in the rootfs.
Piboxd has been cleaned up to launch and manage the mjpg-streamer application instead of ffmpeg.
With all these changes, the webcam is now properly supported. This issue can be closed.