Feature #384
open
Add support for interfacing with display via Android
50%
Description
This essentially allows sending appmgr the command to run. Appmgr would need to reply that the app is started so the Adroid app can switch to a management tool specific to the app now running on PiBox.
Files
| piboid-main.png (143 KB) piboid-main.png | Main keypad | ||
| piboid-numkeys.png (144 KB) piboid-numkeys.png | Numeric/special keys keyboard | ||
| piboid-alphakeys.png (141 KB) piboid-alphakeys.png | Alphabetic keyboard. |
Updated by Hammel over 9 years ago
- Android app connects to PiBox web server over https (see monkey's secure ssl)
- A restful interface provides command input from the app
- The restful interface passes the commands to appmgr
- appmgr forwards the commands to the current app
- The app can use the command or ignore it
This should be implemented via a library of some kind so any app can use the same input mechanism with callbacks.
See handling http authentication on stackoverflow.
Updated by Hammel over 9 years ago
- Status changed from New to In Progress
- % Done changed from 0 to 10
I now have adkt function (in androidtest) for building a test application. It contains information on how to work with Android from the command line.
Once I've completed the tutorial I'll need to create a Dia flowchart showing how the application will operate.
Updated by Hammel over 9 years ago
- % Done changed from 10 to 20
Successfully built the GridView tutorial. This will become the baseline for the initial android app. Initially I called this PiDroid but a google search shows there are other apps/projects named PiDroid. So the official name will be PiBoid, which is PiBox for Android.
I'm going to clean up the code to make sure the app and package names are correct, then import to gitorious.
After that I'll create the Dia diagram for how this app should operate and interact with PiBox.
Note: when building the GridView tutorial I discovered that the sample images provided, which were in JPEG format, didn't work. I had to convert them to PNG. Just a personal note that the PiBox Android app should use PNGs and not JPEGs.
Updated by Hammel over 9 years ago
- Project changed from PiBox to PiBoid
- Category deleted (
04 - Applications)
Updated by Hammel over 9 years ago
- Android Training is where to learn about android development.
- Android Reference has the API javadoc.
- Android Downloads like the Action Bar icon pack.
- Custom keyboard discussion on stackoveflow talks about the pros/cons of using a custom input method vs the soft keyboard.
- Soft Keyboard Examples
- Keyboard example shows how to use Android's Keyboard and KeyboardView to create custom keyboards.
- Detailed example with a clear tutorial including source code.
- Another example on github, but the tutorial is in italian. However, the source is very short and easy to follow.
- shortyz has an example keyboard layout.
- alanszlosek example on github (with screenshot)
- dhaval0122 example on github (no description or screenshot but extremely simple)
- Broadcast/Multicast reception
- WifiManager.MulticastLock is used to allow reception of multicast packets.
- Android Multicast Receiver example
- UDP Multicast on Android discusses sending as well as receiving.
- Network Discovery using UDP Broadcast is for Java but is probably similar in Android.
- Android Layouts
- Scrolling ListView
It appears that using Multicast over wifi may not be as easy as using over wired networks. The initial implementation will therefore fallback to using remote hosts configured by the user.
Updated by Hammel over 9 years ago
- % Done changed from 20 to 30
I now have a 3x3 grid I can use for the main navigation commands as well as menu options for selecting and adding remote devices to control.
Next up:- Save devices to a file
- Show the devices in the file
- Allow deleting devices where we select them
- Create icons for the 3x3 grid
Updated by Hammel over 9 years ago
- Save devices to a file
- Show the devices in the file
Delete option is available but callback is not implemented yet.
After that, need icons from main window.
Updated by Hammel over 9 years ago
All keypad icons are integrated into the app. I should probably test this on my phone to see if they fit.
Next up- Prompt for credentials on connect request.
- Fields: userid, password (hidden)
- Buttons: login, cancel, show password
- Login: submits request and verifies
- Success Saves credentials and returns to home page
- Failure: stays on credentials
- Cancel: returns to home page.
- Show password: toggles password visibility
- Login: submits request and verifies
- Uses saved credentials but allows editing
- Front page should show which device it is connected to (text field above keypad)
- Create custom soft keyboards, attach to keypad buttons and create keyhandler stubs
- ABC
- 123
- Special characters
Updated by Hammel over 9 years ago
- % Done changed from 30 to 40
Userid/password prompt for login to remote host has been added and credentials are saved between uses.
Added the connected device to the ActionBar, when connected.
Created two custom keyboards: alphabetic and numeric/special keys. These have support for capturing their keycodes, which match ASCII values for the printed characters.
- Perform the actual login to the remote device's web service.
- Create a service that can send through the connection the keycodes as they are pressed.
Note that the main keypad should map to the keycodes used by GTK+ for the arrow, Enter, Tab and ESC keys.
Updated by Hammel over 9 years ago
- File piboid-main.png piboid-main.png added
- File piboid-numkeys.png piboid-numkeys.png added
- File piboid-alphakeys.png piboid-alphakeys.png added
- % Done changed from 40 to 50
I've completed the core/stubs of the Android app through to the point of having it send keycodes from the keypad to the PiBox server. I can see the request arrive at the Monkey web server but don't have any code on the web server to handle it yet. That comes next as I develop the PiBox side of this project.
What's missing on the Android side is to handle all keystrokes from the keypad and the two soft keyboards. I've currently implemented some keypad keys as GDK keysyms, such as KP_Left or Tab (see /usr/include/gtk-2.0/gdk/gdkkeysyms.h). To continue this I'll need to create a lookup table for each keystroke on the keypad or soft keyboards. There are only about 50 or 60 keys total so this is just a bit of busy work.
Using the keysyms should make it easier to work with a library on the PiBox side to translate into keystrokes under GTK+/GDK.
I've attached a few screenshots showing the main keypad and the custom soft keyboards. They don't show the device configuration and login but those are simple text fields and buttons, nothing special there. Note that it's possible to switch software keyboards using the keypad if your device doesn't cover the keypad by the keyboards. Alternatively you can close the keyboards using the keyboard button at the bottom/center of each keyboard.
Updated by Hammel over 9 years ago
All keycodes are now sent properly using GDK keysyms. I added a KeyCodes class to do the conversion from ASCII codes to the keysyms.
Now on to the PiBox side support.
Updated by Hammel over 9 years ago
Implemented passthrough from piboxwww to AppMgr. This was pretty simple. Network throughput is pretty slow at my house but the router config here requires sending two floors down and then two floors back up. I suspect it will be a bit faster in production in a travel trailer.
Updated by Hammel over 9 years ago
- Target version changed from 0.10.0 to 0.12.0
The PiBox side of this task involves creating a new app library that can handle inbound messages automatically and pass them to a registered app-specific handler. This turns out to be a big deal and can require lots of changes to all the apps and appmgr. It's a very good idea because it allows message passing at a low level using a very unrestrictive messaging protocol and can (or so I expect) integrate with GTK+ to make it easier for the Android app to apply keystrokes to the currently running app. Even nicer: if the app doesn't support the library then the remote simply has no effect. It doesn't break anything.
However, it's also apparent that so many changes potentially break many things and, at the moment, the current codebase is very stable. So I've decided to make the 0.10.0 release with the current stable codebase but without the Android app, add a new release just for that (and the new app library) and push out the other two releases (hardware focus and TV/music apps) currently scheduled. That way I'll have a nice stable release I can use to do demos or use to introduce people to the project.
Pushing this task to 0.11.0.
Updated by Hammel about 9 years ago
There might be a way to simplify the PiBox app side of this: xdotool. This is a user space tool for faking keyboard input. There is a library associated with this that can be linked from C programs.
I think the trick here will be that the primary navigation will be a different action than the specialized keyboards. The navigation will cause xdo_keysequence() to be called while the keyboards need to cause xdo_type() to be called.
If this works then I can wrap this library in the pibox app library and have it receive remote commands and issue these xdo calls from the appmgr without having to modify the apps themselves. A very handy solution, if it works.
A simpler solution that doesn't require an external library may be possible as well.
Updated by Hammel almost 9 years ago
- Priority changed from Urgent to High
- Severity changed from 03 - Medium to 01 - Critical
Updated by Hammel over 3 years ago
- Priority changed from High to Normal
- Severity changed from 01 - Critical to 03 - Medium