Feature #221
closed
Need daemon to handle launch/shutdown of ffmpeg
100%
Description
The PiBox media server web server needs to launch ffmpeg before it returns a page to the remote browser. Launching it is easy from php before the streaming starts, but we need to kill it when the browser leaves that page.
To do that we need javascript in the page served to perform an action. That action will cause the ffmpeg session associated with that page to stop.
The best way to handle this is to have a daemon that launches ffmpeg and associates a tag with it. PHP generates the tag and tells the daemon to launch ffmpeg. The tag is returned in the Javascript in the served page so that javascript can call a URL that is PHP that pulls the tag in the GET variables and tells the daemon to stop the ffmpeg session.
Updated by Hammel over 10 years ago
- % Done changed from 0 to 10
- php socket client tutorial: http://www.binarytides.com/php-socket-programming-tutorial/
- javascript example to handle unload event: http://stackoverflow.com/questions/653976/call-url-before-closing-of-browser-window
- Accept inbound XML messages
- Validate them
- Generate a UUID
- Return the UUID or NACK
The accepted XML is parsed and a data structure placed on a queue that is handled by a separate thread to manage child processes.
XML format is TBD. Using XML allows me to use a template file within the web server where tags can be replaced by PHP to specify what ffmpeg should be opening and how. libxml2 is already available in PiBox's rootfs.
I need a way to start an autoconf project. Some possible tutorials:- http://www.dwheeler.com/autotools/ - this seems to be the best, most recent and its a video tutorial.
- http://smalltalk.gnu.org/blog/bonzinip/all-you-should-really-know-about-autoconf-and-automake
- http://mij.oltrelinux.com/devel/autoconf-automake/
- http://markuskimius.wikidot.com/programming:tut:autotools/
- http://www.lrde.epita.fr/~adl/autotools.html
Updated by Hammel over 10 years ago
- % Done changed from 10 to 30
I've created a new project, piboxd, under the pibox umbrella on gitorious. An autoconf-based build has been created and I've started work on the core of the daemon. So far I've got command line parsing, logging, embedded build date/version, daemon mode and initial support for queueing of inbound messages. The server thread that accepts inbound messages also has simple nc-based unit tests.
I've got a preliminary design for the daemon including the simplistic protocol for messages to it. Messages are one-way, which means a request comes in and is processed by the daemon but is not acknowledged. This design has a block diagram in a Dia project which will be included in the source tree for piboxd.
After thinking about the original description of the daemon I've started to feel there might need to be one additional message from the browser (via javascript, probably through a PHP proxy request) that lets the daemon know the page is being viewed. If the daemon doesn't get this within a few seconds (it's very close to the browser, physically, in the trailer) then it should kill the process it just started. The javascript on the web page could repeat this heartbeat message periodically. This is a safeguard against abandoned streams. Additionally, the stream could have a time limit placed on it so it times out after a while. A timeout on the web cam would be useful. A timeout on streamed movies might not be.
Updated by Hammel over 10 years ago
- % Done changed from 30 to 40
Update on development of the daemon:
There is a small protocol for messaging between the web interface and the daemon. That protocol is structurally supported but not all features are implemented yet. Most importantly, support for starting and stopping a webcam stream is implemented. Unit tests are written for this functionality as well, though they don't verify that crtmpserver connects to the stream. They only test that launching ffmpeg using the correct command line is attempted.
What's next:- Add test for multiple webcam stream requests. Only one request for the webcam can be active at at time. This is implemented in the daemon, but not tested yet.
- Add negative tests for queue processor. That would be tests that send badly formatted stream requests.
- Add duration test for queue processor. That would be a test that starts and stops tests over an extended period of time to test long term stability.
- Implement heartbeat processing. This should update the specified tag's heartbeat field with a new timestamp. That timestamp is local - when the message was received by the daemon.
- Implement thread that handles automatic shutdown of a stream if no heartbeat is received within a configurable timeout period. Be sure to call shutdown for thread from main().
- Write init script for daemon.
- Add packaging (opkg) for daemon.
- Verify cross compile works.
After that, I cam return to the web interface to write the PHP code that calls the daemon and then the Javascript that provides the heartbeat call and the termination call to the PHP.
Updated by Hammel over 10 years ago
- Added test for multiple webcam streams.
- Added negative queue processor tests.
- Added timed test.
- Fixed minor bugs exposed by tests.
- Implement heartbeat processing. This should update the specified tag's heartbeat field with a new timestamp. That timestamp is local - when the message was received by the daemon.
- Implement thread that handles automatic shutdown of a stream if no heartbeat is received within a configurable timeout period. Be sure to call shutdown for thread from main().
- Write init script for daemon.
- Add packaging (opkg) for daemon.
- Verify cross compile works.
I should also run the daemon with Valgrind and the timed duration test to see if I'm leaking anything.
Updated by Hammel over 10 years ago
- % Done changed from 40 to 50
- Implemented heartbeat processing.
- Added thread to expire abandonded streams.
- Updated unit tests to support both.
- Write init script for daemon.
- Add packaging (opkg) for daemon.
- Verify cross compile works.
- Run the daemon with Valgrind and the timed duration test to see if it leaks
Updated by Hammel over 10 years ago
- % Done changed from 50 to 60
- Completed init script.
- Added packaging (opkg) targets.
- Verify cross compile works.
- Run the daemon with Valgrind and the timed duration test to see if it leaks
Updated by Hammel over 10 years ago
- % Done changed from 60 to 80
- Added cross compile script.
- Run the daemon with Valgrind and the timed duration test to see if it leaks
Updated by Hammel over 10 years ago
Running with valgrind found some problems. I'm fixing them piece by piece. There are a couple small leaks still left to identify, I think.
The command to run for basic valgrind coverage is:
valgrind --tool=memcheck --leak-check=yes --show-reachable=yes --num-callers=20 --track-fds=yes src/piboxd -v3
This is taken verbatim (except for the program being tested) from http://cs.ecs.baylor.edu/~donahoo/tools/valgrind/. I'm running the following minimal test to search for leaks, currently:
./server.sh -v -t procs -t proce
Updated by Hammel over 10 years ago
- % Done changed from 80 to 90
I've cleaned up all valgrind reported leaks. One interesting one is that valgrind reports a dlopen "still reachable" leak if I use pthread_exit() when my threads exit. To avoid this I now call return(0) instead since the thread itself will be cleaned up immediately afterward. That's because where this happens is when the daemon is exiting anyway.
So now valgrid runs cleanly running simple tests. I'm going to run the timed test and make sure there are no long term leaks. Once those (if any) are cleaned up I'm done with this version of the daemon.
Updated by Hammel over 10 years ago
Rats. There are some issues when the timed test is run. This may be due to either a long-run leak or there may be specific tests that generate these leaks. I need to run each unit test manually first, then the timed test to find if what's causing these. The output from valgrind follows.
==22581== 960 bytes in 12 blocks are indirectly lost in loss record 1 of 4 ==22581== at 0x4A074CD: malloc (vg_replace_malloc.c:236) ==22581== by 0x402D99: queueMsg (msgQueue.c:352) ==22581== by 0x402823: msgProcessor (msgProcessor.c:352) ==22581== by 0x3474207D8F: start_thread (in /lib64/libpthread-2.14.90.so) ==22581== ==22581== 1,035 bytes in 150 blocks are indirectly lost in loss record 2 of 4 ==22581== at 0x4A074CD: malloc (vg_replace_malloc.c:236) ==22581== by 0x4026A1: msgProcessor (msgProcessor.c:312) ==22581== by 0x3474207D8F: start_thread (in /lib64/libpthread-2.14.90.so) ==22581== ==22581== 1,131 (1,040 direct, 91 indirect) bytes in 13 blocks are definitely lost in loss record 3 of 4 ==22581== at 0x4A074CD: malloc (vg_replace_malloc.c:236) ==22581== by 0x402D99: queueMsg (msgQueue.c:352) ==22581== by 0x402823: msgProcessor (msgProcessor.c:352) ==22581== by 0x3474207D8F: start_thread (in /lib64/libpthread-2.14.90.so) ==22581== ==22581== 13,104 (11,200 direct, 1,904 indirect) bytes in 140 blocks are definitely lost in loss record 4 of 4 ==22581== at 0x4A074CD: malloc (vg_replace_malloc.c:236) ==22581== by 0x402E36: queueMsg (msgQueue.c:352) ==22581== by 0x402823: msgProcessor (msgProcessor.c:352) ==22581== by 0x3474207D8F: start_thread (in /lib64/libpthread-2.14.90.so) ==22581== ==22581== LEAK SUMMARY: ==22581== definitely lost: 12,240 bytes in 153 blocks ==22581== indirectly lost: 1,995 bytes in 162 blocks ==22581== possibly lost: 0 bytes in 0 blocks ==22581== still reachable: 0 bytes in 0 blocks ==22581== suppressed: 0 bytes in 0 blocks
Updated by Hammel over 10 years ago
- Status changed from In Progress to Closed
- % Done changed from 90 to 100
Updated unit tests to make it easier to run individual subtests. Fixed leaks exposed in various subtests as reported by valgrind. Timed tests of 300 seconds now reports no leaks.
The daemon is completed, though its not tested on the hardware to verify it correctly starts the webcam stream. However, any bugs uncovered in that testing will be opened as new bugs.
This issue can be closed.