https://redmine.graphics-muse.org/https://redmine.graphics-muse.org/favicon.ico?16278048512014-12-16T05:13:58ZGraphics Muse Issue TrackerPiBoid - Feature #384: Add support for interfacing with display via Androidhttps://redmine.graphics-muse.org/issues/384?journal_id=15092014-12-16T05:13:58ZHammelmjhammel@graphics-muse.org
<ul></ul>One way to handle this:
<ol>
<li>Android app connects to PiBox web server over https (see <a href="http://monkey-project.com/documentation/1.5/plugins/polarssl.html" class="external">monkey's secure ssl</a>)</li>
<li>A restful interface provides command input from the app</li>
<li>The restful interface passes the commands to appmgr</li>
<li>appmgr forwards the commands to the current app</li>
<li>The app can use the command or ignore it</li>
</ol>
<p>This should be implemented via a library of some kind so any app can use the same input mechanism with callbacks.</p>
<p>See <a href="http://stackoverflow.com/questions/1968416/how-to-do-http-authentication-in-android" class="external">handling http authentication</a> on stackoverflow.</p> PiBoid - Feature #384: Add support for interfacing with display via Androidhttps://redmine.graphics-muse.org/issues/384?journal_id=15482014-12-26T21:31:30ZHammelmjhammel@graphics-muse.org
<ul><li><strong>Status</strong> changed from <i>New</i> to <i>In Progress</i></li><li><strong>% Done</strong> changed from <i>0</i> to <i>10</i></li></ul><p>I now have adkt function (in androidtest) for building a test application. It contains information on how to work with Android from the command line.</p>
<p>Once I've completed <a href="http://developer.android.com/training/basics/firstapp/index.html" class="external">the tutorial</a> I'll need to create a Dia flowchart showing how the application will operate.</p> PiBoid - Feature #384: Add support for interfacing with display via Androidhttps://redmine.graphics-muse.org/issues/384?journal_id=15492014-12-27T18:31:45ZHammelmjhammel@graphics-muse.org
<ul><li><strong>% Done</strong> changed from <i>10</i> to <i>20</i></li></ul><p>Successfully built the <a href="http://developer.android.com/guide/topics/ui/layout/gridview.html" class="external">GridView tutorial</a>. This will become the baseline for the initial android app. Initially I called this PiDroid but a google search shows there are other apps/projects named PiDroid. So the official name will be <strong>PiBoid</strong>, which is PiBox for Android.</p>
<p>I'm going to clean up the code to make sure the app and package names are correct, then import to gitorious.</p>
<p>After that I'll create the Dia diagram for how this app should operate and interact with PiBox.</p>
<p>Note: when building the GridView tutorial I discovered that the sample images provided, which were in JPEG format, didn't work. I had to convert them to PNG. Just a personal note that the PiBox Android app should use PNGs and not JPEGs.</p> PiBoid - Feature #384: Add support for interfacing with display via Androidhttps://redmine.graphics-muse.org/issues/384?journal_id=15502014-12-27T22:34:31ZHammelmjhammel@graphics-muse.org
<ul><li><strong>Project</strong> changed from <i>PiBox</i> to <i>PiBoid</i></li><li><strong>Category</strong> deleted (<del><i>04 - Applications</i></del>)</li></ul> PiBoid - Feature #384: Add support for interfacing with display via Androidhttps://redmine.graphics-muse.org/issues/384?journal_id=15512014-12-27T22:34:59ZHammelmjhammel@graphics-muse.org
<ul><li><strong>Category</strong> set to <i>Design</i></li></ul> PiBoid - Feature #384: Add support for interfacing with display via Androidhttps://redmine.graphics-muse.org/issues/384?journal_id=15522014-12-27T23:43:46ZHammelmjhammel@graphics-muse.org
<ul></ul>Some important links for this project.
<ol>
<li><a href="http://developer.android.com/training/index.html" class="external">Android Training</a> is where to learn about android development.</li>
<li><a href="http://developer.android.com/reference/packages.html" class="external">Android Reference</a> has the API javadoc.</li>
<li><a href="http://developer.android.com/design/downloads/index.html" class="external">Android Downloads</a> like the Action Bar icon pack.
<ol>
<li><a href="http://developer.android.com/design/style/iconography.html" class="external">Android Iconography</a></li>
</ol>
</li>
<li><a href="http://stackoverflow.com/questions/3196511/custom-keyboard-built-in-an-application-on-android" class="external">Custom keyboard</a> discussion on stackoveflow talks about the pros/cons of using a custom input method vs the soft keyboard.</li>
<li>Soft Keyboard Examples
<ol>
<li><a href="http://code.tutsplus.com/tutorials/create-a-custom-keyboard-on-android--cms-22615" class="external">Keyboard example</a> shows how to use Android's <a href="http://developer.android.com/reference/android/inputmethodservice/Keyboard.html" class="external">Keyboard</a> and <a href="http://developer.android.com/reference/android/inputmethodservice/Keyboard.html" class="external">KeyboardView</a> to create custom keyboards.</li>
<li><a href="http://www.fampennings.nl/maarten/android/09keyboard/index.htm" class="external">Detailed example</a> with a clear tutorial including source code.</li>
<li><a href="https://github.com/rciovati/Android-KeyboardView-Example" class="external">Another example</a> on github, but the tutorial is in italian. However, the source is very short and easy to follow.</li>
<li><a href="http://code.google.com/p/shortyz/source/browse/app/src/main/res/xml/keyboard.xml" class="external">shortyz</a> has an example keyboard layout.</li>
<li><a href="https://github.com/alanszlosek/android-keyboard" class="external">alanszlosek</a> example on github (with screenshot)</li>
<li><a href="https://github.com/dhaval0122/keyboard_sample_2#readme" class="external">dhaval0122</a> example on github (no description or screenshot but extremely simple)</li>
</ol>
</li>
<li>Broadcast/Multicast reception
<ol>
<li><a href="http://developer.android.com/reference/android/net/wifi/WifiManager.MulticastLock.html" class="external">WifiManager.MulticastLock</a> is used to allow reception of multicast packets.</li>
<li><a href="http://codeisland.org/2012/udp-multicast-on-android/" class="external">Android Multicast Receiver</a> example</li>
<li><a href="http://codeisland.org/2012/udp-multicast-on-android/" class="external">UDP Multicast on Android</a> discusses sending as well as receiving.</li>
<li><a href="http://michieldemey.be/blog/network-discovery-using-udp-broadcast/" class="external">Network Discovery using UDP Broadcast</a> is for Java but is probably similar in Android.</li>
</ol>
</li>
<li><a href="http://www.tutorialspoint.com/android/android_user_interface_layouts.htm" class="external">Android Layouts</a></li>
<li><a href="http://www.coderzheaven.com/2011/03/12/creating-scrolling-listview-in-android/" class="external">Scrolling ListView</a>
<ol>
<li><a href="http://developer.android.com/guide/topics/ui/layout/listview.html" class="external">ListView API</a></li>
</ol></li>
</ol>
<p>It appears that using Multicast over wifi may not be as easy as using over wired networks. The initial implementation will therefore fallback to using remote hosts configured by the user.</p> PiBoid - Feature #384: Add support for interfacing with display via Androidhttps://redmine.graphics-muse.org/issues/384?journal_id=15612014-12-30T05:37:37ZHammelmjhammel@graphics-muse.org
<ul><li><strong>% Done</strong> changed from <i>20</i> to <i>30</i></li></ul><p>I now have a 3x3 grid I can use for the main navigation commands as well as menu options for selecting and adding remote devices to control.</p>
Next up:
<ol>
<li>Save devices to a file</li>
<li>Show the devices in the file</li>
<li>Allow deleting devices where we select them</li>
<li>Create icons for the 3x3 grid</li>
</ol> PiBoid - Feature #384: Add support for interfacing with display via Androidhttps://redmine.graphics-muse.org/issues/384?journal_id=15622014-12-30T22:53:54ZHammelmjhammel@graphics-muse.org
<ul></ul>Implemented:
<ol>
<li>Save devices to a file</li>
<li>Show the devices in the file</li>
</ol>
<p>Delete option is available but callback is not implemented yet.</p>
<p>After that, need icons from main window.</p> PiBoid - Feature #384: Add support for interfacing with display via Androidhttps://redmine.graphics-muse.org/issues/384?journal_id=15632014-12-31T19:09:51ZHammelmjhammel@graphics-muse.org
<ul></ul><p>All keypad icons are integrated into the app. I should probably test this on my phone to see if they fit.</p>
Next up
<ol>
<li>Prompt for credentials on connect request.
<ol>
<li>Fields: userid, password (hidden)</li>
<li>Buttons: login, cancel, show password
<ol>
<li>Login: submits request and verifies
<ol>
<li>Success Saves credentials and returns to home page </li>
<li>Failure: stays on credentials</li>
</ol>
</li>
<li>Cancel: returns to home page.</li>
<li>Show password: toggles password visibility</li>
</ol>
</li>
<li>Uses saved credentials but allows editing</li>
</ol>
</li>
<li>Front page should show which device it is connected to (text field above keypad)</li>
<li>Create custom soft keyboards, attach to keypad buttons and create keyhandler stubs
<ol>
<li>ABC</li>
<li>123</li>
<li>Special characters</li>
</ol></li>
</ol> PiBoid - Feature #384: Add support for interfacing with display via Androidhttps://redmine.graphics-muse.org/issues/384?journal_id=15652015-01-06T22:51:52ZHammelmjhammel@graphics-muse.org
<ul><li><strong>% Done</strong> changed from <i>30</i> to <i>40</i></li></ul><p>Userid/password prompt for login to remote host has been added and credentials are saved between uses. <br />Added the connected device to the ActionBar, when connected.<br />Created two custom keyboards: alphabetic and numeric/special keys. These have support for capturing their keycodes, which match ASCII values for the printed characters.</p>
Next up:
<ol>
<li>Perform the actual login to the remote device's web service.</li>
<li>Create a service that can send through the connection the keycodes as they are pressed.</li>
</ol>
<p>Note that the main keypad should map to the keycodes used by GTK+ for the arrow, Enter, Tab and ESC keys.</p> PiBoid - Feature #384: Add support for interfacing with display via Androidhttps://redmine.graphics-muse.org/issues/384?journal_id=15662015-01-09T16:51:02ZHammelmjhammel@graphics-muse.org
<ul><li><strong>File</strong> <a href="/attachments/68">piboid-main.png</a> <a class="icon-only icon-download" title="Download" href="/attachments/download/68/piboid-main.png">piboid-main.png</a> added</li><li><strong>File</strong> <a href="/attachments/69">piboid-numkeys.png</a> <a class="icon-only icon-download" title="Download" href="/attachments/download/69/piboid-numkeys.png">piboid-numkeys.png</a> added</li><li><strong>File</strong> <a href="/attachments/70">piboid-alphakeys.png</a> <a class="icon-only icon-download" title="Download" href="/attachments/download/70/piboid-alphakeys.png">piboid-alphakeys.png</a> added</li><li><strong>% Done</strong> changed from <i>40</i> to <i>50</i></li></ul><p>I've completed the core/stubs of the Android app through to the point of having it send keycodes from the keypad to the PiBox server. I can see the request arrive at the Monkey web server but don't have any code on the web server to handle it yet. That comes next as I develop the PiBox side of this project.</p>
<p>What's missing on the Android side is to handle all keystrokes from the keypad and the two soft keyboards. I've currently implemented some keypad keys as GDK keysyms, such as KP_Left or Tab (see /usr/include/gtk-2.0/gdk/gdkkeysyms.h). To continue this I'll need to create a lookup table for each keystroke on the keypad or soft keyboards. There are only about 50 or 60 keys total so this is just a bit of busy work.</p>
<p>Using the keysyms should make it easier to work with a library on the PiBox side to translate into keystrokes under GTK+/GDK.</p>
<p>I've attached a few screenshots showing the main keypad and the custom soft keyboards. They don't show the device configuration and login but those are simple text fields and buttons, nothing special there. Note that it's possible to switch software keyboards using the keypad if your device doesn't cover the keypad by the keyboards. Alternatively you can close the keyboards using the keyboard button at the bottom/center of each keyboard.</p>
<p><img src="https://redmine.graphics-muse.org/attachments/download/68/piboid-main.png" title="Main keypad" alt="Main keypad" /><br /><img src="https://redmine.graphics-muse.org/attachments/download/69/piboid-numkeys.png" title="Numeric/special keys keyboard" alt="Numeric/special keys keyboard" /><br /><img src="https://redmine.graphics-muse.org/attachments/download/70/piboid-alphakeys.png" title="Alphabetic keyboard." alt="Alphabetic keyboard." /></p> PiBoid - Feature #384: Add support for interfacing with display via Androidhttps://redmine.graphics-muse.org/issues/384?journal_id=15672015-01-09T18:44:34ZHammelmjhammel@graphics-muse.org
<ul></ul><p>All keycodes are now sent properly using GDK keysyms. I added a KeyCodes class to do the conversion from ASCII codes to the keysyms.</p>
<p>Now on to the PiBox side support.</p> PiBoid - Feature #384: Add support for interfacing with display via Androidhttps://redmine.graphics-muse.org/issues/384?journal_id=15682015-01-09T21:38:00ZHammelmjhammel@graphics-muse.org
<ul></ul><p>Implemented passthrough from piboxwww to AppMgr. This was pretty simple. Network throughput is pretty slow at my house but the router config here requires sending two floors down and then two floors back up. I suspect it will be a bit faster in production in a travel trailer.</p> PiBoid - Feature #384: Add support for interfacing with display via Androidhttps://redmine.graphics-muse.org/issues/384?journal_id=15692015-01-11T19:12:44ZHammelmjhammel@graphics-muse.org
<ul><li><strong>Target version</strong> changed from <i>0.10.0</i> to <i>0.12.0</i></li></ul><p>The PiBox side of this task involves creating a new app library that can handle inbound messages automatically and pass them to a registered app-specific handler. This turns out to be a big deal and can require lots of changes to all the apps and appmgr. It's a very good idea because it allows message passing at a low level using a very unrestrictive messaging protocol and can (or so I expect) integrate with GTK+ to make it easier for the Android app to apply keystrokes to the currently running app. Even nicer: if the app doesn't support the library then the remote simply has no effect. It doesn't break anything.</p>
<p>However, it's also apparent that so many changes potentially break many things and, at the moment, the current codebase is very stable. So I've decided to make the 0.10.0 release with the current stable codebase but without the Android app, add a new release just for that (and the new app library) and push out the other two releases (hardware focus and TV/music apps) currently scheduled. That way I'll have a nice stable release I can use to do demos or use to introduce people to the project.</p>
<p>Pushing this task to 0.11.0.</p> PiBoid - Feature #384: Add support for interfacing with display via Androidhttps://redmine.graphics-muse.org/issues/384?journal_id=15822015-01-14T21:22:39ZHammelmjhammel@graphics-muse.org
<ul><li><strong>Priority</strong> changed from <i>High</i> to <i>Immediate</i></li></ul> PiBoid - Feature #384: Add support for interfacing with display via Androidhttps://redmine.graphics-muse.org/issues/384?journal_id=15902015-01-28T18:39:04ZHammelmjhammel@graphics-muse.org
<ul></ul><p>There might be a way to simplify the PiBox app side of this: <a href="http://www.semicomplete.com/projects/xdotool/" class="external">xdotool</a>. This is a user space tool for faking keyboard input. There is a <a href="http://www.semicomplete.com/files/xdotool/docs/html/index.html" class="external">library</a> associated with this that can be linked from C programs.</p>
<p>I think the trick here will be that the primary navigation will be a different action than the specialized keyboards. The navigation will cause <a href="http://www.semicomplete.com/files/xdotool/docs/html/xdo_8h.html#ab5c1bfa3aa0e0b847149988227076aa5" class="external">xdo_keysequence()</a> to be called while the keyboards need to cause <a href="http://www.semicomplete.com/files/xdotool/docs/html/xdo_8h.html#a2eab7063dc284d3d67b483323bea7502" class="external">xdo_type()</a> to be called.</p>
<p>If this works then I can wrap this library in the pibox app library and have it receive remote commands and issue these xdo calls from the appmgr without having to modify the apps themselves. A very handy solution, if it works.</p>
<p>A <a href="http://www.doctort.org/adam/nerd-notes/x11-fake-keypress-event.html" class="external">simpler solution</a> that doesn't require an external library may be possible as well.</p> PiBoid - Feature #384: Add support for interfacing with display via Androidhttps://redmine.graphics-muse.org/issues/384?journal_id=15992015-03-22T02:05:47ZHammelmjhammel@graphics-muse.org
<ul><li><strong>Priority</strong> changed from <i>Immediate</i> to <i>Urgent</i></li></ul> PiBoid - Feature #384: Add support for interfacing with display via Androidhttps://redmine.graphics-muse.org/issues/384?journal_id=16602015-06-29T22:11:41ZHammelmjhammel@graphics-muse.org
<ul><li><strong>Priority</strong> changed from <i>Urgent</i> to <i>High</i></li><li><strong>Severity</strong> changed from <i>03 - Medium</i> to <i>01 - Critical</i></li></ul> PiBoid - Feature #384: Add support for interfacing with display via Androidhttps://redmine.graphics-muse.org/issues/384?journal_id=28642020-08-13T01:59:49ZHammelmjhammel@graphics-muse.org
<ul><li><strong>Priority</strong> changed from <i>High</i> to <i>Normal</i></li><li><strong>Severity</strong> changed from <i>01 - Critical</i> to <i>03 - Medium</i></li></ul> PiBoid - Feature #384: Add support for interfacing with display via Androidhttps://redmine.graphics-muse.org/issues/384?journal_id=35692023-05-21T03:57:30ZHammelmjhammel@graphics-muse.org
<ul><li><strong>Target version</strong> changed from <i>0.12.0</i> to <i>3.0 - Corrino</i></li></ul><p>It's unclear if this is really necessary anymore. But I'll hang onto it since so much work was done already.</p>