PiBox » launcher

See RM #1003. This shows that I need separate Buildroot configs per HW type in order to enable options based on processor type.

From RM #1003:

I'd like to enable a Mesa driver for each RPi build. But that can't be enabled by default because of the Target Options->Target Architecture Variant option set in the single Buildroot config. This field needs to be set based on the processor.

Note that the RPi2 with the Cortex-A7 has not been available since 2016, so building for the Cortex-A53 (like the Rpi3) is reasonable.

It's unclear if the Cortex-A53 builds will work with the Cortex-A7, but I think they will. Since Buildroot is NOT generating the kernel or toolchain, it just uses this option to identify what other Target Package options to enable for selection. So I should just be able to have the three configs: rpi1, rpi2 (which includes rpi3) and rpi4.

So I first need to dup the buildroot config for each hardware build (rpi, rpi2 and rpi4). Then update them for the hardware platform. Then enable the correct OpenGL support.

I'm going to start with RPI3, as that seems the most likely to have the best opengl support. I need to review how to enable RPi-Userland in Buildroot and then integrate it into to PiBox (dropping the GLES build since that duplicates RPi-Userland, assuming they both build from master).

This is being tested under the rpi3 sandbox. Currently a full, new 3.0 build is ongoing.

When that completes, I can work on dup'ing the config file and configuring for rpi2 hardware.

Not done with building yet - I was missing the Mesa->OpenGL ES API support in the Buildroot config. Without it, cairo is built without opengl support. I can this by looking at <buildroot>/output/build/cairo/config.log:

$ ./configure --target=arm-buildroot-linux-gnueabihf --host=arm-buildroot-linux-gnueabihf --build=x86_64-pc-linux-gnu --prefix=/usr --exec-prefix=/usr --sysconfdir=/etc --localstatedir=/var --program-prefix= --disable-gtk-doc --disable-gtk-doc-html --disable-doc --disable-docs --disable-documentation --with-xmlto=no --with-fop=no --disable-dependency-tracking --enable-ipv6 --disable-nls --disable-static --enable-shared --enable-trace=no --enable-interpreter=no --disable-directfb --enable-ft --enable-gobject --disable-gl --disable-glesv2 --disable-vg --enable-xlib --enable-xcb --with-x --enable-xlib-xrender --enable-ps --enable-pdf --enable-png --enable-script --enable-svg --disable-tee --disable-xml

If I rebuild cairo with the OpenGL ES API enabled it looks like this.

$ ./configure --target=arm-buildroot-linux-gnueabihf --host=arm-buildroot-linux-gnueabihf --build=x86_64-pc-linux-gnu --prefix=/usr --exec-prefix=/usr --sysconfdir=/etc --localstatedir=/var --program-prefix= --disable-gtk-doc --disable-gtk-doc-html --disable-doc --disable-docs --disable-documentation --with-xmlto=no --with-fop=no --disable-dependency-tracking --enable-ipv6 --disable-nls --disable-static --enable-shared --enable-trace=no --enable-interpreter=no --disable-directfb --enable-ft --enable-gobject --disable-gl --enable-glesv2 --disable-vg --enable-xlib --enable-xcb --with-x --enable-xlib-xrender --enable-ps --enable-pdf --enable-png --enable-script --enable-svg --disable-tee --disable-xml

But this fails to build:

checking whether cairo's OpenGLESv2 surface backend feature could be enabled... no (glesv2.pc nor OpenGL ES 2.0 headers not found)
configure: error:  OpenGLESv2 surface backend feature could not be enabled
make[2]: *** [package/pkg-generic.mk:283: /home/mjhammel/src/ximba/raspberrypi3/bld/buildroot-2023.02.2/output/build/cairo-1.16.0/.stamp_configured] Error 1
make[1]: *** [configs/buildroot.mk:170: .buildroot] Error 2
make: *** [configs/buildroot.mk:146: buildroot-rebuild] Error 2

This might be because I need to rebuild Mesa to pickup the API update in the Buildroot config.

Did a test build with media system. This never booted fully into firstboot. I suspect something is wrong with the config.txt. I can disable the new items at the top and try again to see if it boots. The config.txt changes were never tested on rpi2 so they seem the most likely cause of the boot issues.

I should note that the toolchain and Buildroot builds are using the same ARM architecture settings. The kernel is using a Broadcom chip that should be specific to rpi2/rpi3 but I still need to verify that.

Tried with a 2nd SD card but the same problem showed up.

Cleaned the complete build tree and restarted. Ran into some errors in Buildroot, like libpibox needing to add tslib as a dependency in libpibox.mk and php failing with a "nothing" pragma directive that is not supported on arm-v8-a (which is what is being built for rpi2/rpi3). I've fixed those problems and Buildroot is running again. Waiting on it to complete.

Rebuilding everything fixed the boot problem. The RPi3 boots to X.org just fine. RPi2, however, fails to start X.org. I'll look into that later.

First I want to check if the Media System boots on RPi3 and, if so, does having Cairo using the openGL backend help with drawing in the Launcher.

Works fine on RPi3 with animation on PiNet. I might be able to speed up the animation even, but that's a task for another day (see RM #1032).

Next will be to check that the launcher is actually using the opengl backend. I might need to get an opengl surface for use by the cairo drawing routines.

I also need to determine why RPi2 X.org exits. The log doesn't show anything terrible but it is missing the glx extension. That may be the mesa->OpenGL GLX option since I don't see any X.org extensions as options available from Buildroot. Running S99UI manually:

root(tty0)$ /etc/init.d/S99UI start
Starting PiBox UI...
root(tty0)$ 
modprobe: FATAL: Module g2d_23 not found in directory /lib/modules/6.1.51-v7+
Illegal instruction
/usr/bin/xinit: connection to X server lost

waiting for X server to shut down (II) Server terminated successfully (0). Closing log file.

$ cat /etc/pibox-version
VERSION: PiBox 2.0
HW: rpi2
BUILD: Thu Sep  7 19:52:33 UTC 2023
MODEL: Raspberry Pi 2 Model B Rev 1.1

So I can either build for rpi, rpi2 Rev1.1, rpi2 Rev1.2/rpi3 and rpi4 or I can just fallback to A7 for all rpi2 and rpi3's. RPi2 (A53 version) will be EOL in 2026 so it's still worth supporting, but the Rev 1.1 may not be worth maintaining. It's also worth noting that RPi 3 is becoming available again after the supply chain issues but RPi 2 is not. So maybe I should just update all RPi 2's to RPi 3's. Also, the RPI 4 is now supported by PiBox and is available in quantity at $10 more than the RPi 3.

Also, the Pi Zero 2W is the same as the RPi3. So just dropping RPi2 would be the simplest solution, though I could build it with the older RPi1 models and see if that works.

RPi1 does not boot on RPi2. I only have 1 RPi2 rev 1.1 (I have one RPi2 rev1.2 that DOES work with the RPi3 build) now anyway, so maybe it's just time to retire RPi2 rev1.1 and move to just RPi1, RPi3 and RPi4. I'll order an RPi3 replacement for the RPi2 and that will be the end of the problem.

This gl-cairo-cube example is likely to be the code I need to review to get Cairo working with GTK+ and OpenGL. However, it requires adding GtkGlExt as a new package to Buildroot so that apps can have direct access to it.

I grabbed the source and put it under src/ximba/work/gl-cairo-cube. It doesn't build out of the box. I managed to get gtk2 to work with it but it's not finding gtk/gtkgl.h.

That code expects gtk+3 and gdk3. So that isn't going to work for me.

I might be able to do this with SDL instead of Cairo (or with Cairo).

• https://www.libsdl.org/projects/gtk-demo/
• https://discourse.libsdl.org/t/tutorial-a-better-way-to-embed-an-sdl-window-into-gtk-or-gnome-etc/5617
• https://lifeofpenguin.blogspot.com/2021/12/embedding-sdl-widget-in-gtk-application.html

If you get an SDL window in the GtkDrawingArea and use the SDL window with Cairo (see gl-cairo-simple) that might work. Although gl-cairo-simple is using GL to draw and I want to use Cairo to draw into OpenGL, for speed improvements and offloading display stuff from the CPU.

This one seems to work if I use either of

window_id = (void*)(intptr_t)gdk_x11_drawable_get_xid(gdk_window);
window_id = (void*)(intptr_t)GDK_WINDOW_XWINDOW(gdk_window);

instead of

window_id = (void*)(intptr_t)GDK_WINDOW_XID(gdk_window);

I would then need to get the SDL surface for use with Cairo.

Getting back to straight Cairo/GTK/GDK (no SDL), I've found that you can do this in Cairo.

        cairo_surface_t *surface =
            cairo_image_surface_create (CAIRO_FORMAT_ARGB32, 240, 80);
        cairo_t *cr =
            cairo_create (surface);

To get a surface that we know is OpenGL, we can use this example which uses the undocumented GL backend API cairo_gl_surface_create() along with a GLX API. This is, in fact, the exact example I was looking for that will definitively get me an OpenGL backend for Cairo drawings in the Gtk widget.

static cairo_surface_t*
create_source_surface(int width, int height)
{

    int rgba_attribs[] = {
        GLX_RGBA,
        GLX_RED_SIZE, 1,
        GLX_GREEN_SIZE, 1,
        GLX_BLUE_SIZE, 1,
        GLX_ALPHA_SIZE, 1,
        GLX_DOUBLEBUFFER,
        None };

    XVisualInfo *visinfo;
    GLXContext ctx;
    struct closure *arg;
    cairo_device_t *device;
    cairo_surface_t *surface;
    Display *dpy;

    dpy = XOpenDisplay(NULL);
    if (dpy == NULL)
        return NULL;

    visinfo = glXChooseVisual (dpy, DefaultScreen (dpy), rgba_attribs);
    if (visinfo == NULL) {
        XCloseDisplay (dpy);
        return NULL;
    }
    ctx = glXCreateContext (dpy, visinfo, NULL, True);
    XFree (visinfo);

    if (ctx == NULL) {
        XCloseDisplay (dpy);
        return NULL;
    }
    arg = (struct closure*)(malloc (sizeof (struct closure)));
    arg->dpy = dpy;
    arg->ctx = ctx;
    device = cairo_glx_device_create (dpy, ctx);
    if (cairo_device_set_user_data (device,
                                    (cairo_user_data_key_t *) cleanup,
                                    arg,
                                    cleanup))
    {
        cleanup (arg);
        return NULL;
    }

    surface = cairo_gl_surface_create (device,
                                       CAIRO_CONTENT_COLOR_ALPHA,
                                       width, height);
    cairo_device_destroy (device);
    return surface;
}

Now I need to integrate this example to launcher, including verification of the cairo_surface_type_t returned from cairo_surface_get_type() being CAIRO_SURFACE_TYPE_GL.

Addendum¶

I tried to build this example on Debian but it won't build because Debian is missing cairo-gl.h. However, that header is available in the 1.16 release in the Buildroot staging tree (it's under /usr/include/cairo/cairo-gl.h). So I can create an app package of this example to build against an RPi staging tree as a proof of concept that this code would work if ported to Launcher and/or PiNet.

Ported to an app build. That worked. Copied to a RPi 2 v1.2 and ran it. It works! It doesn't do anything, but the window opens to black and there are no errors on the command line.

Next step, add the code to test if it's an GL backend. That can written to stdout for this test. It would be nice to have the window do something too.

This doesn't actually work. The sample program fails to get an XVisualInfo pointer in create_source_surface(). It's not clear why this fails but it's a call to glxChooseVisual() that fails.

An alternative method is available from an SDL github archive in the SDL_x11opengl.c:X11_GL_CreateContext() function.

This simply isn't working. Another option, for animations at least, is to use dispmanx to overlay an animation over the app. For example, create a block for the animation in GTK and then use the dimensions of that block to launch an external dispmanx program to do the animation, with keyboard input staying with the GTK app.

• https://elinux.org/Raspberry_Pi_VideoCore_APIs
• https://github.com/fernandorpardo/DispmanX
• https://github.com/hex007/eop/tree/master - Eye of Pi image viewer written in DispmanX.
• https://github.com/fernandorpardo/DispmanX
• https://github.com/random-prj/dispmanx-demo - good, short demo!
• https://github.com/mdtrooper/triplebuffer_raspberry/tree/master
• https://github.com/AndrewFromMelbourne/raspidmx
• https://forums.raspberrypi.com/viewtopic.php?t=178133
• https://forums.raspberrypi.com/viewtopic.php?t=318902
• https://gist.github.com/Hermann-SW2 (dispmanX hello_pi demo updated with DrawCircle() instead of just a rectangular image)
  • https://github.com/Hermann-SW2/userland/tree/master/host_applications/linux/apps/hello_pi/i420toh264#i420toh264
• https://raspberrypi.stackexchange.com/questions/7092/display-a-test-pattern-by-writing-the-frame-buffer-and-then-clear-it - Simplest example yet, and offers insight into how to set the display rectangle and position for DispmanX