Project

General

Profile

Actions

Bug #951

closed

Keyboard not working with Media, Player or Kiosk modes

Added by Hammel about 1 year ago. Updated 9 months ago.

Status:
Closed
Priority:
Immediate
Assignee:
Category:
General
Target version:
Start date:
26 Mar 2023
Due date:
% Done:

100%

Estimated time:
Severity:
01 - Critical

Description

Tested on RPi 1 with desktop display using wired and wireless keyboards: arrow keys do not support navigation.
Tested on RPi 2 and 3 after RM #957 resolved: still broken.

This has to be fixed before the 2.0 release.

References:
  1. https://www.x.org/releases/current/doc/xorg-docs/input/XKB-Config.html
  2. https://developer-old.gnome.org/gtk2/2.24/

Related issues

Related to PiBox - Feature #1004: Drop ext4 image buildClosedHammel05 Jul 2023

Actions
Actions #1

Updated by Hammel about 1 year ago

  • Status changed from New to In Progress
  • % Done changed from 0 to 10
Might be due to two recently fixed problems
  1. RM #949: If a touchscreen is identified, then the launcher keyboard was disabled.
  2. RM #955: Recognized touchscreens was not working correctly.

So this needs to be retested with the updates and see if those fixes address this issue.

Actions #2

Updated by Hammel about 1 year ago

  • % Done changed from 10 to 20

Still broken with those fixes. But the whole keyboard mapping problem is broken too.

In pmsui, there is src/config/etc/X11/xinitrc, which includes a test for a specific USB vendor/product pair for the FAVI keyboards. If found, then xmodmap is used to map the keys. This isn't very flexible, to say the least.

The more modern way of handling keyboards is XKB. This sets the keyboard type in the xorg.conf. There are a variety of configurations (see X.org wiki doc on XKB) that can be seen in /usr/share/X11/xkb/rules/base. And these can be changed at runtime with setxkbdmap.

So there are a couple of things to update here.
  1. Add setxkbdmap to dev platform buildroot config.
  2. Change pmsui's xinitrc to read a config file and run setxkbmap instead of using xmodmap and checking for usb identifiers.
  3. Default to using pc104, us keyboard types, as in:
Section "InputClass" 
    Identifier "keyboard defaults" 
    MatchIsKeyboard "on" 

    Option "XkbModel" "pc104" 
    Option "XkbLayout" "us" 
    Option "XKbOptions" "" 
EndSection

But we don't do this in the xinitrc. Instead we use the setxkbdmap utility in the xinitrc in pmsui.

I think that may fix all my generic keyboard problems, though it's not international. Users can then change the config file, call it /etc/xkb.cfg, with a model/layout to use at boot time.

I need to manually try this to verify it works. I have an RPi1 running Kiosk that I can use to verify it.

Actions #3

Updated by Hammel about 1 year ago

  • Description updated (diff)
Actions #4

Updated by Hammel about 1 year ago

  • Subject changed from Keyboard not working with Kiosk mode to Keyboard not working with Media, Player or Kiosk modes
  • Description updated (diff)
Actions #5

Updated by Hammel about 1 year ago

I added a test configuration with pc104 to a conf file for X.org, but that didn't change anything. The X.org.log shows the configuration was accepted, however.

I tested with evtest using both the Sino Wealth and Favi keyboards. evtest reports key left and key right the same way and correctly. So the problem is something else.

Actions #6

Updated by Hammel about 1 year ago

Okay, I added debug code to see what keyval was being presented (because it was showing "NULL" when translated to GDK) and now it just works. Can't seem to get it to fail again.

I'll keep trying it with different models and reboots. Maybe this is an intermittent problem? Or maybe I just fixed it with the icon pre-scaling/color icons fixes?

Actions #7

Updated by Hammel about 1 year ago

Tried it on a RPi2 this time. The Sino keyboard only was plugged in and didn't work. Added X.org config and rebooted. Still didn't work.
Plugged in the Favi keyboard. Neither worked.
Rebooted. Both keyboards worked.
Removed Favi keyboard. Rebooted. Sino keyboard doesn't work.

So this may have something to do with the way the xinitrc is structured to support Favi keyboards. Try the following.
  1. Remove the favi keyboard config.
  2. Add setxkbmap to xinitrc without the favi config.
Actions #8

Updated by Hammel about 1 year ago

  • % Done changed from 20 to 50

Use of xkb didn't seem to help any, at least not without lots of customization. The technique used in pmsui's xinitrc is simpler, and as it turns out extensible.

That bit of code at the start that is looking specifically for FAVI usb ids should be changed to this:

# Load a keyboard file.  This might load more than one if there
# are multiple keyboards installed - be careful!
rm -f "/etc/X11/kbd/active" 
while read line; do
    usbid="$(echo ${line} | cut -f1 -d" ")" 
    kbdfile="$(echo ${line} | cut -f2 -d" ")" 
    kbd="$(lsusb | grep "${usbid}")" 
    if [ -n "${kbd}" ]
    then
        sh "/etc/X11/kbd/${kbdfile}" 
        echo "Keyboard: ${usbid} /etc/X11/kbd/${kbdfile}" >> "/etc/X11/kbd/active" 
    fi
done < /etc/X11/kbd/kbd.list

For this to work we need to add the following pieces to the dev platform (or maybe pmsui, but probably dev):
  1. mkdir /etc/X11/kbd
  2. new file: /etc/X11/kbd/kbd.list
  3. Add to kbd.list for each supported input device
    1. <usbid pair> <filename>
  4. Add to /etc/X11/kbd
    1. One file for each entry in kbd.list that contains the xmodmap's for the keys of interest (usually arrow keys)

Then I just need to run xev for each keyboard and get their keycodes for the arrow keys and create a new /etc/X11/kbd/<file> for that keyboard.

Yes, this is clumsy, but it's easy to extend for the DIY'ers and that's kinda what PiBox is all about.

One thing this doesn't do, as written: you can't plug in a keyboard after boot and have it work. The way around this problem is to make that new bit of code a shell script and tie it to mdev.conf's inputdev handling. That may or may not work. I may have to experiment with that.

Actions #9

Updated by Hammel about 1 year ago

So this will work to handle plug events for new keyboards.

  • Change /etc/mdev.conf from
    input/event[0-9]*   root:root 644   */lib/mdev/inputdev
    

    to
    input/event[0-9]*   root:root 640   @/usr/bin/keyboard.sh
    
  • /usr/bin/keyboard.sh should call the script that loads the xmodmaps (see previous comment on this issue) and make use of the following env vars passed from mdev
        ACTION      = add
        SEQNUM      = 939
        MAJOR       = 13
        MDEV        = input/event0
        DEVPATH     = /devices/platform/soc/3f980000.usb/usb1/1-1/1-1.2/1-1.2:1.0/0003:258A:0001.0005/input/input7/event0
        DEVNAME     = input/event0
        SUBSYSTEM   = input
        MINOR       = 64
        PHYSDEVPATH = 
        PHYSDEVBUS  = 
        PWD         = /dev
    

    The important one is extracting the usb id from DEVPATH, which are in reverse order:
ids="$(echo ${DEVPATH} | cut -f10 -d"/")" 
echo "USB ID: $(echo ${ids} | cut -f2 -d":"):$(echo ${ids}|cut -f1 -d":")" 

Another important issue is that the shell script can be called repeatedly with no guarantee that the first call completes before the second is made. I'll need a way to serialize this.

One final thing: the script can be called multiple times (multiple input/event* devices from a single keyboard). The script should be run each time but rather only once since the same USB IDs are used for each input event.

Update

  • The first step in this is to make sure that an mdev event can update the X server. The xmodmap frontend needs to set DISPLAY=unix:0.0 before calling the xmodmap backend script for the given usb id pair.
    • Just write the front end as a script and call it from /usr/bin/keyboard.sh on plug event for a keyboard and see if the arrow keys work. If so, then setting DISPLAY is all that's required.
  • Serializing could be done by having piboxd run the xmodmap frontend script. /usr/bin/keyboard.sh uses nc to send a new message type, MT_KEYBOARD, that causes piboxd to run the xmodmap front end.
  • piboxd can read /etc/X11/kdb/active to see if the usb id pair has already been run, skipping the mdev event if it has.
Actions #10

Updated by Hammel about 1 year ago

I've tested the two scripts I need:
  1. keyboard.sh - called via mdev to send a pibox protocol message (MT_SYS, MA_KEYB) to piboxd.
  2. keyboard-fe.sh - called by piboxd and xinitrc to configured keyboard arrow keys.

I've also added the code to piboxd to handle this new message type and added the new MA_KEYB action for MA_SYS to libpibox:pmsg.h.

Now I need to rebuild libpibox in the dev platform, then I can build piboxd. The former will require rebuilding the SD card so I'll just push a new version of piboxd with that. Since the shell scripts are in pmsui I'll also build and push a new version of that for testing.

So that means
  • Test compile libpibox
  • Push libpibox
  • Rebuild dev platform w/updated libpibox
  • Build piboxd
  • Build pmsui
  • Rebuild SD card
  • Push meta to SD card
  • Push piboxd to SD card
  • Push pmsui to SD card

Assuming this works then I should update the wiki with the new MA_KEYB action so I don't forget it later.

I'm still not happy with this solution - Xorg should really be doing this. Terminals recognize arrow keys. I'm not sure why Xorg doesn't.

Actions #11

Updated by Hammel 12 months ago

  • Status changed from In Progress to Closed
  • % Done changed from 50 to 100

Tested on Media System with all updates - works fine.
Tested on Player System - works fine.
Tested on Kiosk System - fails

All repo updates commited and pushed:
  1. pibox
  2. libpibox
  3. piboxd
  4. pmsui

The kiosk problem

The mdev.conf update breaks the launcher on touchscreens. The problem is that I changed the permissions on the input devices to 640 but for the touchscreen to work it needs to be 644.

Actions #12

Updated by Hammel 12 months ago

  • Status changed from Closed to In Progress
  • % Done changed from 100 to 90
Actions #13

Updated by Hammel 12 months ago

Change to mdev.conf pushed. Still needs a hardware test but I have little doubt it will work because I made the same change manually and it worked.

Actions #14

Updated by Hammel 12 months ago

  • Status changed from In Progress to Closed
  • % Done changed from 90 to 100

Verified working on kiosk.

All code committed and pushed.

Closing issue.

Actions #15

Updated by Hammel 9 months ago

  • Status changed from Closed to In Progress
  • Target version changed from 2.0 - Harkonnen to 3.0 - Corrino
  • % Done changed from 100 to 50

Re-opening on 3.0.

After updating all component builds to the latest that seem to work, I'm able to boot to the dev system. However, no keyboards work at all.

It seems that no keyboards work at all, not even alphabetic keys. The changes for this RM were pushed to pmsui, so that doesn't help here.

Also, the changes from this RM are not quite right because the user should be able to extend these even on the overlay/squashfs builds. The way to do that is to change keyboard-fe.sh to merge /etc/X11/kbd/kbd.list with /media/mmcblk0p3/opt/etc/kbd.list and then sort and update the list under /etc/X11/kbd/kbd.list.

Also, the keyboard maps under /etc/X11/kbd/ should be extended with whatever is found in /media/mmcblk0p3/opt/etc/kbd.

Actions #16

Updated by Hammel 9 months ago

Ugh. Found the problem. The x86-input-keyboard driver for X.org has been dropped for Linux systems. Buildroot therefore also dropped it. Instead, we should use xf86-input-libinput. However, Buildroot requires udev to build this. The ArchLinux libinput wiki page links to it's package page which doesn't list udev as a requirement.

So to get keyboards working under the latest X.org (which comes with Buildroot) I'll need to build xf86-input-libinput without udev requirement, assuming that's possible. There is a git gist for doing this with gentoo, sort of. But this isn't very helpful info.

A little digging into Buildroot shows you have to change the System Configuration->/dev Management to "Dynamic using devtmpfs + eudev" to enable udev options, of which xf86-input-libinput is one. But that would likely break everything else since it's mdev. See this online documentation for an explanation of the /dev Management options.

So the next option is to try and get libinput and xf86-input-libinput and try to build them manually against the staging tree without udev. If that works, then I can add pibox-specific mods to Buildroot to build these without udev.

If this doesn't work, I may not be able to use either X.org or the latest Buildroot. That's a major show stopper.

Note: it might be possible to enable "Dynamic using devtmpfs + eudev" to enable the those options to be built and then just test if they get used or even work with mdev. After all, mdev will be there because it's enabled in the Busybox config. It's just not clear that libinput and the X.org input driver will work without udev being enabled.

Actions #17

Updated by Hammel 9 months ago

  • % Done changed from 50 to 70

It looks like eudev (which is Buildroot's stripped down udev extracted from systemd) and mdev can co-exist. All that's required is to run the S10udev init script as S09udev and leave the existing S10mdev script in place. What happens is that the udev script tries to clear the hotplug handler in the kernel, so by running the mdev init script after the udev init script mdev can install itself as the hotplug handler. Even with mdev handling hotplug events, the keyboards and mice just work under X. I'm not quite sure how udev is getting the hotplug events (probably through changes to devtmpfs) but plugging in the Sino keyboard and the various Favi-style keyboards all just work now.

What this means

  1. I may be able to get rid of the special handlers for keyboards now. I know I just added them at the end of the 2.0 cycle but with eudev handling this for me I may not need it anymore.
  2. I need to make sure udev is not doing anything else under the hood on kernel events. I don't think it is because there are no rules under /etc/udev/rules.d.
  3. I need to use the Buildroot hack to enable eudev even though Busybox is providing mdev.

I tested this with keyboards and mice and also by plugging in a USB stick. They all worked fine. I did notice that repeatedly plugging and unplugging a keyboard would eventually stop all keyboards from working, but if I just plugged 3 keyboards in, one at a time, then they all worked including the FAVI keyboard with it's builtin mouse.

I also tested with the new mdev keyboard handler disabled and the keyboards just worked. So it looks like that isn't necessary anymore.

Side note: doing this allows the arrow keys to support history on the dev system.

Actions #18

Updated by Hammel 9 months ago

Actions #19

Updated by Hammel 9 months ago

  • Status changed from In Progress to Closed
  • % Done changed from 70 to 100

Changes tested, committed and pushed for PiBox. The updates to pmsui for keyboards will need to be removed separately.

Closing this issue (again).

Actions

Also available in: Atom PDF