Feature #1033
open
Preload icons
20%
Description
do_carousel_drawing() loads images as it needs them. This can bog down moving through the list.
To fix this, readDB() should read the image at the time the db is being created. It should also scale the icons.
To help speed up start time, only the first 50 (or 100 or whatever) should be loaded. The rest will be loaded in the background with a g_idle_add() function.
This will simplify musicfe.c:do_carousel_drawing() by removing the image load section and thus speed up processing.
References:- https://www.baeldung.com/linux/gui
- https://www.cs.nmsu.edu/~joshagam/archive/cs574/3-dri.html
- https://unix.stackexchange.com/a/733390
- https://www.khronos.org/opengl/wiki/Platform_specifics:_Linux
- https://dri.freedesktop.org/wiki/
- https://en.wikipedia.org/wiki/Direct_Rendering_Manager
- https://en.wikipedia.org/wiki/Direct_Rendering_Infrastructure
Updated by Hammel over 1 year ago
This isn't as easy at it seems. Consider
- If there are 1000's of images, that could take 100's of MB of memory to pre-allocate the images.
- If we only allocate a few, which few do we allocate?
- What if the user searches for albums/artists - do we redo the preallocations? Won't that just slow things down too?
The way it works now is that 5 images are selected with 2 on either side of the active selection. Those five are loaded each time. It's relatively fast but could benefit from 2D acceleration (re: OpenGL again).
A short term alternative would be to cache the images and only load new ones on either end as the carousel spins. And don't allow the user to hit next/prev too fast - disable key presses while drawing. That may give the appearance of better performance.
For more help with OpenGL/etc, see the Adding OpenGL to Cairo issue. In particular, this discussion suggests that an X program can just create an empty window and provide the size and offsets to DispmanX to do the rendering. DispmanX will be displayed above the X window, which means you could interleave the two windows.
Updated by Hammel over 1 year ago
Side note: The cover art database is made up of png files because the genid3db.sh outputs them as png files. So I only need a PNG loader
Updated by Hammel over 1 year ago
Another side note: All HDMI connected boards have /dev/dri/card0. The RPi LCD and TFT displays do not have this device file. /dev/dri/card0 is the DRM interface. The reason it's missing on RPi LCD and TFT's is because either firstboot or lcdshow removes the dtoverlays for vc4-fkms-v3d*.
Updated by Hammel about 1 year ago
- Severity changed from 02 - High to 03 - Medium
Updated by Hammel about 1 year ago
- Status changed from New to In Progress
- % Done changed from 0 to 10
MusicFE is the initial target for this, since it has the simplest interface to update. There is a branch of in my sandbox called "feature/rm1033" for this work.
So the following links are the best to learn how to do this.
- Image Viewer from AndrewFromMelbourne
- Display a test pattern and clear it from StackExchange
- Userland triangle demo
Also: This forum post gives the details of the current state of display graphics on Raspberry Pi.
The triangle.c demo is actually just about all I want. The init_ogl() function is what maps OpenGL surfaces to DispmanX, then OpenGL commands are used to transpose images on the display. When DispManX is retired fully (on 32bit builds, which is all Pibox is using right now) the transition should be simple since it's just the mapping function that will need updating (and the exit function, come to think of it).
The trick in the init_ogl() is to use the size of the GTK+ window where the musicfe carousel is displayed instead of the full display. For experimentation, initially I'll use the full display just to make sure the code is working and keyboard input works.
The first thing to do is duplicate the triangle code and initialize it running with a timer, just to see it work inside of musicfe.
Updated by Hammel about 1 year ago
- Severity changed from 03 - Medium to 02 - High
Updated by Hammel about 1 year ago
Code is ported and ready to test. If I understand this correctly, the EGL will be displayed under the X, but I think that only works if I set the layer the EGL should be on with DispmanX.
Updated by Hammel about 1 year ago
- % Done changed from 10 to 20
That code didn't work for some reason. It didn't crash anything, but nothing displayed. So I switched to the clouds demo. This was even simpler code and it integrated easily into musicfe. I'm able to animate a cloud using DispmanX over the Cairo-based carousel. The difference between the two is that the former wrapped OpenGL in with DispmanX while the latter was strictly DispmanX.
I also read the forum post for the current state of graphics on Raspberry Pi. Since I rely on omxplayer, I'm required to stick to 32bit builds. And since I'm required to do that, it doesn't really hurt me to stick to using DispmanX. I did try the OpenGL code but as mentioned, it didn't work (might have been missing drivers?).
For now, I'm going to convert to using dispmanX and a Plex-style interface instead of the carousel. I will leave the Cairo-base carousel as a build option, however, should I decide to use this on non-Raspberry Pi hardware. I should also mirror the userland repo in case that goes away for some reason.
Updated by Hammel about 1 year ago
Need a step-by-step plan for this.
- Update demo to accept GTK+ window size and offset instead of using full screen.
- Refactor do_carousel_drawing()
- Get 5 images
- Only parse JSON data.
- pointer to data struct array
- Allocate Cairo images vs DispmanX resources
- Compute scaling and positioning
- Draw Cairo vs DispmanX
- Get 5 images
- Free/cleanup data struct array
Updated by Hammel about 1 year ago
Some things I'm learning while working on that plan.
- dispmanx has no native file loading.
- Will need to incorporate a png reader
- Each image in the carousel is a dispmanx resource
- Only create 1 new resource, no matter the direction the carousel moves
- vc_dispmanx_element_change_source() can swap sources, just remember are we moving down or up in the carousel.
This would really be better done with OpenGL but I couldn't get that to integrate - maybe I need to make sure I'm linking against -lbrcmGLESv2 -lbrcmEGL? But OpenGL doesn't load PNG files natively either.
Updated by Hammel about 1 year ago
- Priority changed from Immediate to Normal
It's unclear if doing this brings any added value right now.
Moving back down the priority list. I'll revisit later, if necessary.