Add support for interfacing with display via Android
|Status:||In Progress||Start date:||09 Sep 2014|
|Target version:||PiBox - 0.12.0|
|Severity:||03 - Medium|
This essentially allows sending appmgr the command to run. Appmgr would need to reply that the app is started so the Adroid app can switch to a management tool specific to the app now running on PiBox.
RM #384: Initial UI layouts and flowcharting for PiBoid and its comm with PiBox Media systems.
RM #384: Added input field and button for adding a device via IP address. Changed the Settings menu option to Select a Device.
RM #384: Notify user if there are no configured devices yet. Also, code cleanup - tabs to spaces.
RM #384: Add dialog display to select state (connect, delete, cancel) for selected device.
RM #384: Remade arrow keys so their lighting is consistent. Added Home, Enter, Tab, Letters and Number keys.
RM #384: Updated to reflect current implementation and added Connect box between Select Device and Credentials to reflect expected implementation. Removed deprecated sample images that had been used as placeholders for the keypad.
RM #384: Removed ellipses from ABC and 123 buttons, added special characters to 123 button since it will support both types.
RM #384: Implemented basic login dialog with saved credentials, sans actual login to remote device.
RM #384: uncomment finish() call for SelectDevice activity when no devices are configured.
RM #384: Added a text view to display current connection, but it's kinda klunky looking at the moment.
RM #384: Fix keypad to fit on any display. Added sample special/numeric keyboard, but sans PiBoid customizations.
Aligned keyboard to bottom of any display on popup and support closing of keyboard.
RM #384: Converted example keyboard into numbers+plus special characters keyboard and updated design doc to reflect layout changes.
RM #384: Added alphabetic keyboard with a custom keyboard view that can change the coloring of the CAPS key depending on its state.
RM #384: Added a Comm class to handle submitting keycodes to PiBox via the www interface.
RM #384: Fixed crash when Selecting Device but no devices exist by passing back an Intent with RESULT_CANCELLED before calling finish().
RM #384: Added space key to numberic soft keyboard. Completed key handline for keypad and both soft keyboards. Add class to convert keypad/keyboard keycodes to GDK keysyms before submitting to remote PiBox device.
RM #384: Updated cdtools script to change from "piboxlib" to "libpibox" for the project name.
RM #384: Remove app-level functions (copied from appmgr originally) and integrate use of registration packet and its associated handler callback.
RM #384: Added library initialization routine utilizing new registration structure.
#1 Updated by Hammel over 6 years ago
- Android app connects to PiBox web server over https (see monkey's secure ssl)
- A restful interface provides command input from the app
- The restful interface passes the commands to appmgr
- appmgr forwards the commands to the current app
- The app can use the command or ignore it
This should be implemented via a library of some kind so any app can use the same input mechanism with callbacks.
See handling http authentication on stackoverflow.
#2 Updated by Hammel over 6 years ago
- Status changed from New to In Progress
- % Done changed from 0 to 10
I now have adkt function (in androidtest) for building a test application. It contains information on how to work with Android from the command line.
Once I've completed the tutorial I'll need to create a Dia flowchart showing how the application will operate.
#3 Updated by Hammel over 6 years ago
- % Done changed from 10 to 20
Successfully built the GridView tutorial. This will become the baseline for the initial android app. Initially I called this PiDroid but a google search shows there are other apps/projects named PiDroid. So the official name will be PiBoid, which is PiBox for Android.
I'm going to clean up the code to make sure the app and package names are correct, then import to gitorious.
After that I'll create the Dia diagram for how this app should operate and interact with PiBox.
Note: when building the GridView tutorial I discovered that the sample images provided, which were in JPEG format, didn't work. I had to convert them to PNG. Just a personal note that the PiBox Android app should use PNGs and not JPEGs.
#6 Updated by Hammel over 6 years ago
- Android Training is where to learn about android development.
- Android Reference has the API javadoc.
- Android Downloads like the Action Bar icon pack.
- Custom keyboard discussion on stackoveflow talks about the pros/cons of using a custom input method vs the soft keyboard.
- Soft Keyboard Examples
- Keyboard example shows how to use Android's Keyboard and KeyboardView to create custom keyboards.
- Detailed example with a clear tutorial including source code.
- Another example on github, but the tutorial is in italian. However, the source is very short and easy to follow.
- shortyz has an example keyboard layout.
- alanszlosek example on github (with screenshot)
- dhaval0122 example on github (no description or screenshot but extremely simple)
- Broadcast/Multicast reception
- Android Layouts
- Scrolling ListView
It appears that using Multicast over wifi may not be as easy as using over wired networks. The initial implementation will therefore fallback to using remote hosts configured by the user.
#7 Updated by Hammel over 6 years ago
- % Done changed from 20 to 30
I now have a 3x3 grid I can use for the main navigation commands as well as menu options for selecting and adding remote devices to control.Next up:
- Save devices to a file
- Show the devices in the file
- Allow deleting devices where we select them
- Create icons for the 3x3 grid
#9 Updated by Hammel over 6 years ago
All keypad icons are integrated into the app. I should probably test this on my phone to see if they fit.Next up
- Prompt for credentials on connect request.
- Fields: userid, password (hidden)
- Buttons: login, cancel, show password
- Login: submits request and verifies
- Success Saves credentials and returns to home page
- Failure: stays on credentials
- Cancel: returns to home page.
- Show password: toggles password visibility
- Login: submits request and verifies
- Uses saved credentials but allows editing
- Front page should show which device it is connected to (text field above keypad)
- Create custom soft keyboards, attach to keypad buttons and create keyhandler stubs
- Special characters
#10 Updated by Hammel over 6 years ago
- % Done changed from 30 to 40
Userid/password prompt for login to remote host has been added and credentials are saved between uses.
Added the connected device to the ActionBar, when connected.
Created two custom keyboards: alphabetic and numeric/special keys. These have support for capturing their keycodes, which match ASCII values for the printed characters.
- Perform the actual login to the remote device's web service.
- Create a service that can send through the connection the keycodes as they are pressed.
Note that the main keypad should map to the keycodes used by GTK+ for the arrow, Enter, Tab and ESC keys.
#11 Updated by Hammel over 6 years ago
- File piboid-main.png added
- File piboid-numkeys.png added
- File piboid-alphakeys.png added
- % Done changed from 40 to 50
I've completed the core/stubs of the Android app through to the point of having it send keycodes from the keypad to the PiBox server. I can see the request arrive at the Monkey web server but don't have any code on the web server to handle it yet. That comes next as I develop the PiBox side of this project.
What's missing on the Android side is to handle all keystrokes from the keypad and the two soft keyboards. I've currently implemented some keypad keys as GDK keysyms, such as KP_Left or Tab (see /usr/include/gtk-2.0/gdk/gdkkeysyms.h). To continue this I'll need to create a lookup table for each keystroke on the keypad or soft keyboards. There are only about 50 or 60 keys total so this is just a bit of busy work.
Using the keysyms should make it easier to work with a library on the PiBox side to translate into keystrokes under GTK+/GDK.
I've attached a few screenshots showing the main keypad and the custom soft keyboards. They don't show the device configuration and login but those are simple text fields and buttons, nothing special there. Note that it's possible to switch software keyboards using the keypad if your device doesn't cover the keypad by the keyboards. Alternatively you can close the keyboards using the keyboard button at the bottom/center of each keyboard.
#13 Updated by Hammel over 6 years ago
Implemented passthrough from piboxwww to AppMgr. This was pretty simple. Network throughput is pretty slow at my house but the router config here requires sending two floors down and then two floors back up. I suspect it will be a bit faster in production in a travel trailer.
#14 Updated by Hammel over 6 years ago
- Target version changed from 0.10.0 to 0.12.0
The PiBox side of this task involves creating a new app library that can handle inbound messages automatically and pass them to a registered app-specific handler. This turns out to be a big deal and can require lots of changes to all the apps and appmgr. It's a very good idea because it allows message passing at a low level using a very unrestrictive messaging protocol and can (or so I expect) integrate with GTK+ to make it easier for the Android app to apply keystrokes to the currently running app. Even nicer: if the app doesn't support the library then the remote simply has no effect. It doesn't break anything.
However, it's also apparent that so many changes potentially break many things and, at the moment, the current codebase is very stable. So I've decided to make the 0.10.0 release with the current stable codebase but without the Android app, add a new release just for that (and the new app library) and push out the other two releases (hardware focus and TV/music apps) currently scheduled. That way I'll have a nice stable release I can use to do demos or use to introduce people to the project.
Pushing this task to 0.11.0.
#16 Updated by Hammel over 6 years ago
I think the trick here will be that the primary navigation will be a different action than the specialized keyboards. The navigation will cause xdo_keysequence() to be called while the keyboards need to cause xdo_type() to be called.
If this works then I can wrap this library in the pibox app library and have it receive remote commands and issue these xdo calls from the appmgr without having to modify the apps themselves. A very handy solution, if it works.
A simpler solution that doesn't require an external library may be possible as well.