PiBox with Docker
|Status:||In Progress||Start date:||30 Nov 2019|
|Target version:||PiBox - 3.0 - Corrino|
|Severity:||03 - Medium|
Getting containers on PiBox isn't very straight forward. I can compile container engines and clients but cross compiling them (because many of them use Go) might not be so simple. This may limit me, initially, to existing container engines and clients such as Docker (though I'd prefer to use LXD/LXC).
Docker has static ARM binaries that should work for RPi. I need to download them and test on the dev platform to see what happens.
If I want to get really clever I can create my own container infrastructure.
- Status changed from New to In Progress
- % Done changed from 0 to 20
There is a pidock project locally (not pushed). This has a docker component that pulls premade docker images from upstream repositories. I haven't run this in awhile so I need to retest this to make sure it still works, then check it in.
Before I do that I'm upgrading Buildroot to the latest long term release (2020.02.4). This has Docker support in it and can build Docker manually. I'd like to use that instead of pulling the upstream binaries. I may leave both options in the tree.
I'm also stripping PiBox of most of its rootfs constructs because I need very little in the base overlay other than what is required to run Docker. Then Docker can be used to run upstream containers.
- Tracker changed from Bug to Feature
Building Docker from Buildroot fails right now because of a problem downloading the libsodium archive. I can get to the archive from my browser but not via Buildroot.
I've disabled the Docker build in Buildroot to let it complete. Then I'll build the Docker component separately which will download the upstream pre-built binaries.
Building Docker from Buildroot failed. And after some experimentation I've changed my mind about how to do this.
I've integrated docker into the PiBox build twice now. The first merged lots of changes all at once and just got confusing. The second was cleaner - more commits of specific incremental changes - but in the end I didn't like doing it that way. The development platform doesn't need Docker. It shouldn't be a new hardware type either. It's just another user space component.
The correct way to do this is to make Docker an opkg, just like any other app. So I'm going to redo this as a 3rd party build, like omxplayer or mjpeg-streamer, and simply build an opkg with the appropriate bits. This is easier, and cleaner, than trying to merge it into the PiBox development platform build anyway.
Some of the changes I made in my 2nd attempt above are needed for other RM's however, so I'll cherry pick those as patches. No sense wasting that work.
Problems found in testing.
- check-config.sh doesn't work because the dev platform doesn't have bash.
- cgroups are not fully enabled in the kernel so the init script doesn't work.
- /proc/config.gz is not enabled by default. Use "modprobe configs" to load it - add that to modules.conf
I think enabling Bash and running check-config.sh would help.
Looks like the version of docker I downloaded, while a static arm binary, doesn't work on the RPi3. I get an "illegal instruction" from it. But then, I also get a segfault with curl. So I'm wondering if there is some other weird problem here.
Update: curl works if just retrieving the front page to google.com.
Ugh. This might be the reason:
bash-4.4# file /usr/bin/dockerd
/usr/bin/dockerd: ELF 32-bit LSB executable, ARM, EABI5 version 1 (GNU/Linux), statically linked, for GNU/Linux 3.2.0, Go BuildID=jHK_QBChLPFK5ZRuGt5x/_ghDqTccjM27JRHTjOh1/POv6fCAkLuZNd7OEiA8l/pKPAHl62MhHNeEAscq-H, BuildID[sha1]=adf3a13c53a3141edf105d06a95e88398d2f0227, not stripped
bash-4.4# file /usr/bin/pibox-network-config
/usr/bin/pibox-network-config: ELF 32-bit LSB executable, ARM, EABI5 version 1 (SYSV), dynamically linked, interpreter /lib/ld-linux-armhf.so.3, for GNU/Linux 4.14.99, stripped
The "for GNU/Linux" bit is what's different. Because I use a newer Linux in the toolchain I may need to build Docker in Buildroot. Because of this I need to modify PiBox, not just create an opkg for Docker.
Lowering priority/severity. I don't think I'll be coming back to this any time soon.