Action Item #511

Port PiBox to Raspberry Pi 2

Added by Hammel over 1 year ago. Updated over 1 year ago.

Status:ClosedStart date:19 Mar 2016
Priority:ImmediateDue date:
Assignee:Hammel% Done:

100%

Category:00 - Basic Build Issues
Target version:0.11.0
Severity:01 - Critical

Description

This is a matter of:
  1. Updating xcc to build for the new hardware
  2. Updating firmware configurations
  3. Building the kernel to support hardware changes where necessary
  4. Updating the root file system if necessary

Danny needs this to work on the Pi2's he bought. I had it working but didn't check it in. So now I need to create a branch for it to test the changes before merging with master.


Related issues

Related to PiBox - Testing #460: Review BananaPi as replacement for RaspberryPi In Progress 24 Jul 2015

Associated revisions

Revision a272ba09
Added by Hammel over 1 year ago

RM #511: New xcc config files for the RPI2. These are used by setting HW=rpi2 on the GNU Make command line, as in
"make HW=rpi2 rpifw-pkg kernel-pkg gles buildroot-pkg pkg-scripts"

Revision c5e3c447
Added by Hammel over 1 year ago

RM #511: Copy in all the firmware files instead of just the ones we need for the original Pi boards.

Revision 0ef551cb
Added by Hammel over 1 year ago

RM #511: Copy firmware subdirectories, which are recent additions upstream.

Revision 7df8064c
Added by Hammel over 1 year ago

RM #511: Add reference for rpi2 to HW env var.

Revision 8dfa141b
Added by Hammel over 1 year ago

RM #511: Don't create m4 directory as it is now included in the upstream repo for libpibox.

Revision 34c5b30a
Added by Hammel over 1 year ago

RM #511: Patches to bump libpng to 1.6.22.

Revision f228602b
Added by Hammel over 1 year ago

RM #511: Add removal of second patch from libpng for bump to 1.6.22.

Revision cb9fbd63
Added by Hammel over 1 year ago

RM #511: Remove mysql from dev platform. It isn't used in media server and doesn't build anymore.

Revision cccf3bcc
Added by Hammel over 1 year ago

RM #511: Bump vim to move to github from googlecode.

Revision 56cffd6a
Added by Hammel over 1 year ago

RM #511: Fix omx to support multiple toolchain/staging tree builds.

Revision 01889fc6
Added by Hammel over 1 year ago

RM #511: Add options for specifying toolchain and staging trees, defaulting to XID and STAGING if set. Added options to aide in debugging individual app builds.

Revision b85f4cf7
Added by Hammel over 1 year ago

RM #511: Allow setting toolchain, staging and opkg directories on the command line.

Revision 28ab7762
Added by Hammel over 1 year ago

RM #511: Minor tweek to getops to add usage statement.

Revision 94b931e1
Added by Hammel over 1 year ago

RM #511: Allow setting toolchain, staging and opkg directories on the command line.

Revision 77f32607
Added by Hammel over 1 year ago

RM #511: Allow setting toolchain, staging and opkg directories on the command line.

Revision 4b85b7a7
Added by Hammel over 1 year ago

RM #511: Allow setting toolchain, staging and opkg directories on the command line.

Revision 4b85b7a7
Added by Hammel over 1 year ago

RM #511: Allow setting toolchain, staging and opkg directories on the command line.

Revision 452aff42
Added by Hammel over 1 year ago

RM #511: Allow setting toolchain, staging and opkg directories on the command line.

Revision 0f852519
Added by Hammel over 1 year ago

RM #511: Set toolchain, staging and opkg directory configurations for all apps.

Revision 3c00b7a3
Added by Hammel over 1 year ago

RM #511: Fix gles to build for either rpi or rpi2 hardware.

Revision f3c34c82
Added by Hammel over 1 year ago

RM #511: Support rpi2's use of mainline upstream kernel.

Revision 78a1fe22
Added by Hammel over 1 year ago

RM #511: Fix gles to build for either rpi or rpi2 hardware.

Revision d7da48c0
Added by Hammel over 1 year ago

RM #511: Support alternative kernel name (kernel7.img) for rpi2 hardware.

Revision 3b58e2b7
Added by Hammel over 1 year ago

RM #511: Enable fbturbo as the default X server driver.

Revision 7fc6a855
Added by Hammel over 1 year ago

RM #511: Enable new framebuffer configs now that fbturbo is the default.

Revision 7bc77802
Added by Hammel over 1 year ago

RM #511: Copy dhcpd.conf to /etc/dhcp directory so it works in either location.

Revision 3ef378a1
Added by Hammel over 1 year ago

RM #511: Fix gles build on rpi1 after integration with rpi2 build.

Revision cb59a7da
Added by Hammel over 1 year ago

RM #511: Bump rpi1 kernel to 4.4.y.

Revision cd17aeb1
Added by Hammel over 1 year ago

RM #511: Switch to running X.org from inittab for media server/player for both rpi1 and rpi2.

Revision 8c112586
Added by Hammel over 1 year ago

RM #511: Return index from parse(). Added logging in spawnPlayer() to figure that out.

Revision 1f296a1f
Added by Hammel over 1 year ago

RM #511: Set new default for vtsrc and vttmp to 3 and 4, respectively, to support changes in kernel and inittab for upcoming 0.11.0 release.

Revision c428b1ea
Added by Hammel over 1 year ago

RM #511: Copy all firmware to boot partition. Only copy -s/-t files if specified. Fix up kernel files based on HW.

Revision a0bf5a82
Added by Hammel over 1 year ago

RM #511: Fix initialization of doFile to be 0, not 1.

History

#1 Updated by Hammel over 1 year ago

  • Status changed from New to In Progress
  • % Done changed from 0 to 10

I just created the mjhammel/rpi2 branch for the pibox repo. This has the new xcc configurations on it.

Next I'll probably have to fix the firmware targets to copy all firmware to the DOS partition, as noted in RM #460.

#2 Updated by Hammel over 1 year ago

  • % Done changed from 10 to 20

I pushed the change to grab all the firmware with rpifw-pkg target. It's in the mjhammel/rpi2 branch.

Haven't tested it yet.

#3 Updated by Hammel over 1 year ago

  • % Done changed from 20 to 50

I rebuilt today, just to get started again. The rebuild exposed bugs in libpibox and pnc builds which are now fixed. Both the rpi (galileo) and rpi2 (feynman) builds have completed and are ready for testing.

The rpi2 build in the mjhammel/rpi2 branch.

The rpi build is in master.

I need to
  1. Run metabuild on CentOS to get packages for the media server
  2. generate the rpi2 image on CentOS
  3. install and test on an rpi2 board

I also need to test the images on an rpi to make sure it's still working correctly.

#4 Updated by Hammel over 1 year ago

Ran metabuild on Fedora and CentOS. It failed in both places.

On Fedora it looks like this:

/opt/rpiTC/bin/arm-unknown-linux-gnueabi-g++ 
 --sysroot=/opt/rpiTC/arm-unknown-linux-gnueabi/sysroot -mfloat-abi=hard \
 -mcpu=arm1176jzf-s -fomit-frame-pointer -mabi=aapcs-linux -mtune=arm1176jzf-s \
 -mfpu=vfp -Wno-psabi -mno-apcs-stack-check -O3 -mstructure-size-boundary=32 \
 -mno-sched-prolog -march=armv6zk -std=c++0x -D__STDC_CONSTANT_MACROS \
 -D__STDC_LIMIT_MACROS -DTARGET_POSIX -DTARGET_LINUX -fPIC -DPIC -D_REENTRANT \
 -D_LARGEFILE64_SOURCE -D_FILE_OFFSET_BITS=64 -DHAVE_CMAKE_CONFIG \
 -D__VIDEOCORE4__ -U_FORTIFY_SOURCE -Wall -DHAVE_OMXLIB -DUSE_EXTERNAL_FFMPEG \
 -DHAVE_LIBAVCODEC_AVCODEC_H -DHAVE_LIBAVUTIL_OPT_H -DHAVE_LIBAVUTIL_MEM_H \
 -DHAVE_LIBAVUTIL_AVUTIL_H -DHAVE_LIBAVFORMAT_AVFORMAT_H \
 -DHAVE_LIBAVFILTER_AVFILTER_H -DHAVE_LIBSWRESAMPLE_SWRESAMPLE_H -DOMX \
 -DOMX_SKIP64BIT -ftree-vectorize -DUSE_EXTERNAL_OMX -DTARGET_RASPBERRY_PI \
 -DUSE_EXTERNAL_LIBBCM_HOST \
 -I/opt/rpiTC/arm-unknown-linux-gnueabi/sysroot/usr/include \
 -I/home/mjhammel/src/ximba/raspberrypi/bld/buildroot-2015.02/output/target/opt/vc/include \
 -I/home/mjhammel/src/ximba/raspberrypi/bld/buildroot-2015.02/output/target/opt/vc/include/interface/vcos/pthreads \
 -I/home/mjhammel/src/ximba/raspberrypi/bld/buildroot-2015.02/output/target/opt/vc/include/interface/vmcs_host \
 -I/home/mjhammel/src/ximba/raspberrypi/bld/buildroot-2015.02/output/staging/usr/lib/dbus-1.0/include \
 -isystem/home/mjhammel/src/ximba/raspberrypi/bld/buildroot-2015.02/output/staging/usr/include \
 -isystem/home/mjhammel/src/ximba/raspberrypi/bld/buildroot-2015.02/output/staging/usr/include/freetype2 \
 -I./ -Ilinux -Iffmpeg_compiled/usr/local/include/ -I /usr/include/dbus-1.0 -I \
 /usr/lib/arm-linux-gnueabihf/dbus-1.0/include -c utils/log.cpp -o utils/log.o \
 -Wno-deprecated-declarations
In file included from /opt/rpiTC/arm-unknown-linux-gnueabi/include/c++/4.8.5/random:38:0,
                 from /opt/rpiTC/arm-unknown-linux-gnueabi/include/c++/4.8.5/bits/stl_algo.h:65,
                 from /opt/rpiTC/arm-unknown-linux-gnueabi/include/c++/4.8.5/algorithm:62,
                 from ./utils/StdString.h:667,
                 from utils/log.cpp:26:
/opt/rpiTC/arm-unknown-linux-gnueabi/include/c++/4.8.5/cmath:1043:11: error: '::acoshl' has not been declared
   using ::acoshl;
           ^     
/opt/rpiTC/arm-unknown-linux-gnueabi/include/c++/4.8.5/cmath:1047:11: error: '::asinhl' has not been declared
   using ::asinhl;

On CentOS is looks like this:

/opt/rpiTC/bin/arm-unknown-linux-gnueabi-g++ 
 --sysroot=/opt/rpiTC/arm-unknown-linux-gnueabi/sysroot -mfloat-abi=hard \
 -mcpu=arm1176jzf-s -fomit-frame-pointer -mabi=aapcs-linux -mtune=arm1176jzf-s \
 -mfpu=vfp -Wno-psabi -mno-apcs-stack-check -O3 -mstructure-size-boundary=32 \
 -mno-sched-prolog -march=armv6zk -std=c++0x -D__STDC_CONSTANT_MACROS \
 -D__STDC_LIMIT_MACROS -DTARGET_POSIX -DTARGET_LINUX -fPIC -DPIC -D_REENTRANT \
 -D_LARGEFILE64_SOURCE -D_FILE_OFFSET_BITS=64 -DHAVE_CMAKE_CONFIG \
 -D__VIDEOCORE4__ -U_FORTIFY_SOURCE -Wall -DHAVE_OMXLIB -DUSE_EXTERNAL_FFMPEG \
 -DHAVE_LIBAVCODEC_AVCODEC_H -DHAVE_LIBAVUTIL_OPT_H -DHAVE_LIBAVUTIL_MEM_H \
 -DHAVE_LIBAVUTIL_AVUTIL_H -DHAVE_LIBAVFORMAT_AVFORMAT_H \
 -DHAVE_LIBAVFILTER_AVFILTER_H -DHAVE_LIBSWRESAMPLE_SWRESAMPLE_H -DOMX \
 -DOMX_SKIP64BIT -ftree-vectorize -DUSE_EXTERNAL_OMX -DTARGET_RASPBERRY_PI \
 -DUSE_EXTERNAL_LIBBCM_HOST \
 -I/opt/rpiTC/arm-unknown-linux-gnueabi/sysroot/usr/include \
 -I/home/mjhammel/src/ximba/raspberrypi2/bld/buildroot-2015.02/output/target/opt/vc/include \
 -I/home/mjhammel/src/ximba/raspberrypi2/bld/buildroot-2015.02/output/target/opt/vc/include/interface/vcos/pthreads \
 -I/home/mjhammel/src/ximba/raspberrypi2/bld/buildroot-2015.02/output/target/opt/vc/include/interface/vmcs_host \
 -I/home/mjhammel/src/ximba/raspberrypi2/bld/buildroot-2015.02/output/staging/usr/lib/dbus-1.0/include \
 -isystem/home/mjhammel/src/ximba/raspberrypi2/bld/buildroot-2015.02/output/staging/usr/include \
 -isystem/home/mjhammel/src/ximba/raspberrypi2/bld/buildroot-2015.02/output/staging/usr/include/freetype2 \
 -I./ -Ilinux -Iffmpeg_compiled/usr/local/include/ -I /usr/include/dbus-1.0 -I \
 /usr/lib/arm-linux-gnueabihf/dbus-1.0/include -c BitstreamConverter.cpp -o \
 BitstreamConverter.o -Wno-deprecated-declarations
In file included from DllAvFormat.h:27:0,
                 from BitstreamConverter.h:27,
                 from BitstreamConverter.cpp:26:
DllAvCodec.h:79:64: error: 'PixelFormat' has not been declared
   virtual int avpicture_fill(AVPicture *picture, uint8_t *ptr, PixelFormat pix_fmt, int width, int height)=0;

#5 Updated by Hammel over 1 year ago

I think the Fedora one is a mixture of headers from host and target. If so, that may also be the problem on the CentOS box.

#6 Updated by Hammel over 1 year ago

  • % Done changed from 50 to 60

Found the problem: I was mixing toolchains and staging trees. I found when I built on Fedora by specifying XI= and SD= that omxplayer built correctly with the master branch of PiBox. However, after some testing I found that /opt/rpiTC is not the original and may be the rpi2 toolchain. So it's fubar. Similar problem on CentOS. I also found that the oxmplayer configs were incorrectly forcing the use of /opt/rpiTC and that broke the build if the wrong staging tree was specified too.

So I fixed omxplayer on fedora to work with the master branch build of PiBox. Now I'm building the rpi2 branch from scratch (after merging with master) on both Fedora and CentOS. I'm finding minor problems in the build, like expired packages. But once these complete I can retry omxplayer with rpi2 on both distros.

#7 Updated by Hammel over 1 year ago

  • % Done changed from 60 to 80

Fixed dev platform build (pibox repo) and metabuild. The latter required updates to the cross.sh script in various app repos, along with specifying env vars on make command lines for other app repos.

The fedora build is complete, both disk image and app opkgs, and ready for testing on an RPi2. The CentOS omxplayer build still fails, but for a different reason now:

In file included from DllAvFormat.h:27:0,
                 from BitstreamConverter.h:27,
                 from BitstreamConverter.cpp:26:
DllAvCodec.h:79:64: error: 'PixelFormat' has not been declared
   virtual int avpicture_fill(AVPicture *picture, uint8_t *ptr, PixelFormat pix_fmt, int width, int height)=0;
                                                                ^
DllAvCodec.h:84:34: error: 'PixelFormat' has not been declared
   virtual int avpicture_get_size(PixelFormat pix_fmt, int width, int height)=0;
                                  ^
DllAvCodec.h:101:51: error: 'PixelFormat' has not been declared
   virtual int avpicture_alloc(AVPicture *picture, PixelFormat pix_fmt, int width, int height)=0;

It's unclear what this problem is. The two builds should be using the same headers and toolchains.

#8 Updated by Hammel over 1 year ago

Test boot on RPi2

Basically it worked if I used the prebuilt kernel provided in the firmware release. However....

  1. Firmware build doesn't copy all necessary files, I think. Just copy all files and then remove the prebuilt kernels.
  2. PiBox kernel is built incorrectly so doesn't boot. I need to use bcm2709_defconfig for RPi 2 and RPi 3.
  3. Kernel name will need to be kernel7.img, not kernel.img.
  4. Copy kernel in after copying in firmware.
  5. Audio doesn't play out of the combined audio/mic port. (audio plays fine through HDMI, which is the use case for the server but not the player).
  6. GLES build (pibox.cmake) has hard coded values for RPi 1 arch. This needs to be configured with tags so glex.mk can set replace the tags with the appropriate architecture based on the HW setting of the build.

That's it for now. I noticed that, when using the prebuilt kernel, that the stability of the system was suspect. psplash, for one, doesn't seem to work at all. I can just remove that one or make sure it's building against the right kernel perhaps.

The Media Player UI doesn't work correctly with a USB cabled keyboard but that is probably completely unrelated to this port. I'll have to try with a FAVI keyboard to be certain.

#9 Updated by Hammel over 1 year ago

My kernel boots and the system basically worked. However, I'm back to the "X.org keyboard doesn't work" problem. I still have S01Xearly and S98Xearly to run X once and then S99UI to start it a second time, which works on the original RPi B+. After some experimentation I found that moving S99UI to inittab runs as "wait" after all sysinit complete works correctly, allowing the keyboard to work again.

I still have a couple of other problems however.
  1. Keyboard navigation in videofe doesn't work for wired USB or FAVI keyboards. The ESC and Enter keys work, however. Sounds like a keymap problem.
    1. Fixed: set focus and default for view widget in videofe app.
  2. Exiting omxplayer kills the X server and we end up at the login prompt.
    1. Fixed: didn't kill X, just went to the wrong vt. Now vtsrc should be "3" instead of "2" in videofe.cfg.
  3. videofe's parse() doesn't handle filenames with spaces. This can be fixed by parsing the command, dropping the last arg (%s) and then replacing it with the filename.
    1. Fixed: parse only configured command, but not with filename, then do filename substitution in args[] list for %s in configured command.
  4. The default kernel is huge: 9M, not including driver modules. I'm sure this could be slimmed down.

I need to fix these and the issues above and push to the rpi branch. Then I'll look into the omx keyboard issue.

#10 Updated by Hammel over 1 year ago

  • Severity changed from 03 - Medium to 01 - Critical

#11 Updated by Hammel over 1 year ago

Except for the kernel image size, everything else is done. I used the RPi 2 on a camping trip this past week and it worked quite well. The VideoFE was more responsive than on the RPi 1.

I've merged these changes into a build for RPi 1 and test booted. This failed initially because I accidently built the latest kernel (4.4.y) instead of the original (3.14.y). Using the new kernel (and likely the new firmware) requires copying all the firmware files just as with RPi 2. Eventually I'll need to remove files to find out which ones are not required.

Once I got this booted the rootfs ran except for starting X. The init script fails with "illegal instruction", however running xinit manually works. So I need to dig through the init script to find out what has the illegal instruction.

What this tells me is that most of the changes for RPi 2 will work with RPi 1, even using the same kernel. The main difference is the toolchain. In the process of porting I found bugs with the various applications and with the meta build to make sure I was building against the correct staging trees.

So this is nearly done. I have to cleanup the RPi 1 build and then merge with upstream master. I still need to test the opkgs on the RPi 1 and that may show problems, however.

#12 Updated by Hammel over 1 year ago

The illegal instruction comes from /opt/vc/bin/tvservice which is being used to determine if the HDMI is plugged in or not.
The illegal instruction is likely do to the settings used for -mcpu and -mfpu. I currently use

CPU=generic-armv7-a
MFPU=vfpv4

But according to the userland forum (also see the raspberry pi forum) I should use this (for gcc 4.8 or later)

MCPU=cortex-a7
MFPU=neon-vfpv4

So I need to rebuild everything, since the toolchain needs to be rebuilt. Might be a good time to upgrade the kernel used for the toolchain too, as it's 3.14 and the new RPI/RPI2 kernel is 4.4. I'm already using gcc 4.8.5 (mainline, not linaro) so that should be fine with these updates.

#13 Updated by Hammel over 1 year ago

Ugh. So while that's an issue for performance, my issue with withe RPI 1 build (not RPI2) so that won't fix it. The problem might be with the kernel used in the toolchain for rpi1. What I need to do is bump down to the 3.14.y kernel for rpi1 and make sure that is being used and not the 4.4.y kernel used for rpi2.

#14 Updated by Hammel over 1 year ago

The problem was that the wrong cpu, fpu and arch were set in pibox.cmake for the gles (re: userland) component. Fixing that boots the rpi b+ normally.

However, I noticed that the inittab change is not in the core system. And that makes sense because it's only needed if you use the pmsui package. So I need to add a preinst script to the pmsui package that saves the old inittab and installs a new one and a postrm script that restores the saved inittab.

I also found that the videofe.cfg settings are not board specific. So I need to fix the install to handle setting the vtsrc correctly for each board type.

#15 Updated by Hammel over 1 year ago

Full build test for rpi1 and rpi2:
  1. With latest firmware and 4.4.y, mkinstall.sh needs to copy all firmware (except overylays directory) files to boot partition now. It can then remove the two kernel files before install the HW-specific kernel file.
  2. videofe now needs to be changed to have vtsrc set to 3 instead of 2. This is applicable to both HW builds.

After that I may be done with the port. Finally!

#16 Updated by Hammel over 1 year ago

  • Status changed from In Progress to Closed
  • % Done changed from 80 to 100

Verified fixes on both platforms. All changes committed and pushed.

Closing issue.

Also available in: Atom PDF