There is a posted issue on GitHub for packages that have moved hosts since Crosstool-NG 1.24.0 was released.
Review this and see if that helps fix the problems with isl and expat.

Fixed isl problem with a config file change.
Fixed expat problem by pre-fetching the archive - added new xcc-prefetch target for xcc builds.
Build now fails on cross-gdb build:

[ALL  ]    checking whether to use python... /usr/bin/python3.7
[ALL  ]    Traceback (most recent call last):
[ALL  ]      File "/home/mjhammel/src/ximba/raspberrypi4/src/../bld/crosstool-ng-1.24.0.bld/work/aarch64-xeon-linux-gnu/src/gdb/gdb/python/python-config.py", line 7, in <module>
[ALL  ]        from distutils import sysconfig
[ALL  ]    ImportError: cannot import name 'sysconfig' from 'distutils' (/usr/lib/python3.7/distutils/__init__.py)
[ERROR]    configure: error: failure running python-config --includes
[ERROR]    make[3]: *** [Makefile:8763: configure-gdb] Error 1

This may be fixed in the docker image (which is running with buster - re: 10.x) with the following:

sudo apt-get -y install python3.7-distutils
sudo apt-get -y install python3.7-dev

This is being tested now.

Those xcc changes worked. I have a toolchain for rk3399 now.

1. disable rpifw component in the build based on HW=xeon.
2. Use kernel 5.16.x for HW=xeo n
   1. k ernel.org is mostly there
   2. megi's kernel is recommended for PinePhone Pro but is 5.16 but may be more stable
3. Update fwobj and kfwobj configurations, probably to be xeon specific - done; needs verifying with kernel build
4. Find a suitable config for rk3399
   1. Probably from megi's kernel repo

Still need to finish Buildroot updates.
Buildroot is completed.

1. Add DTB pull from kernel build
   1. DTS: rk3399-pinephone-pro.dts
2. Review kernel news and kernel releases

1. for Levinboot (also from DeltaGem)
   1. Temporary binary download
2. for u-boot
   1. u-boot for Rockchip
   2. Rockchip partitioning
   3. Current Manjaro u-boot, which is based on upstream plus patches but which is not fully booting yet. It forms the basis for dsimic's u-boot which has additional patches not found in any git repo yet.

1. Check for sfdisk
2. Levinboot
   1. Check for fdtput, which comes from device-tree-compiler package on Debian
   2. Levinboot SD card setup
3. u-boot
   1. u-boot SD card setup
4. Nuke eMMC
5. Write eMMC with boot image - this can brick the phone!

1. Using levinboot to select kernel boot image

1. u-boot build does not boot Linux
   1. piboxRK u-boot build complete
   2. Fails to find SD card
2. Levinboot build does boot Linux, but resets during kernel init (before rootfs)

1. Strip rpi bits from build
2. Drop git history
3. create new GitLab repo
4. Push current build

Initial import of Xeon dev platform (PiBox based) done today.

• Build system runs cleanly.
• u-boot build can boot kernel
• Kernel is not mounting rootfs

Awaiting additional u-boot updates from upstream but still working boot processing since it's known that u-boot can boot kernel enough to get to rootfs.

The first step is to try and get a bootloader that generates both debug output and a shell prompt on the serial console.

Regression Test¶

Old build doesn't boot from pre-built images. I didn't try rebuilding but I think after a year it's not really worth it. Better to update all source repositories and rebuild.

Manjaro¶

1. boot.txt/boot.scr - configures gpio to vibrate and set LED on phone when booting.
2. dtbs for rockchip-pinephone-pro, which may be for either for u-boot or Linux (probably Linux).
3. initramfs-linux.img, which is gzipped cpio archive for initramfs. Contains useful tips on init processing.
4. u-boot.itb which is likely the device tree blob for u-boot.

It's also a flat structure, as in

Image
boot.scr
boot.txt
idbloader.img
initramfs-linux.img
u-boot.itb
-> dtbs

Pine64 u-boot wiki¶

The Pine64 u-boot wiki page appears to be recently updated and might hold the best instructions for building u-boot, but it suggests my default toolchain won't be sufficient. In general, I should bump my toolchain to at least gcc 10 (it's at 8.3 currently) in order to build with my own toolchain.

Alternatively, I can use-prebuilt toolchains from musl.cc, specifically or1k-linux-musl-cross. The musl.cc toolchains can be built using a platform agnostic script or musl-cross-make.

Note: musl is a replacement libc that still works with gcc.

NOTE : I think this is probably the best path to start with given I want to build my own toolchains, at least at some point.

Megous Repos¶

Megous has instructions on building u-boot on github that might be the easiest to follow. However, he's not using u-boot so there is nothing to compare my builds to.

Megous p-boot¶

An alternative to u-boot is p-boot, which is apparently specifically for pinephone/pinephone pro. This might prove simpler to build, or if I just want to punt building the bootloader I can just download a pre-build binary.

p-boot has the advantage of being ready-made with graphics in the bootloader, which means you get immediate feedback to the user of phone activity (it also lights up LEDs). This seems to be the bootloader being used by Pine64 for the phone at the factory now, though I'm not positive of that.

I'm working on rebuilding the toolchain with latest crosstool-ng and latest gcc/glib/etc. I've created a cdtools setup for xeon and pushed it to the repo. I've added a screen setup for xeon. The original work is in scr rpi4 . The current work is being done in scr xeon1 .

Once the toolchain is generated I'll start porting my xcc configs to use the Pine64 u-boot wiki information. I'm avoiding the musl toolchains for now ad those would require rewriting the Make rules for xcc. I'll test with crosstool-ng to start and if that doesn't generate a working u-boot then I'll switch to musl.

Update 1¶

Work can be found in scr xeon1.
Toolchain is built.
bootloader-get is updated to download the three git repos.
bootloader-unpack is next to update.
Note: this should all go into a branch called "pine64-wiki-uboot"

Update 2¶

The build of the bootloader is much different than usual u-boot, at least with other rk3399 platforms. Unfortunately, the build of the first component - ATF firmware - fails. I tried both the crust and master branches, as well as the v0.5 tag. They all fail with

make[1]: *** No rule to make target 'lib/el3_runtime/arm64/context_mgmt.c', needed by '/home/mjhammel/src/ximba/xeon/bld/bl31-v0.5/build/sun50i_a64/release/bl31/context_mgmt.o'.  Stop.

So this isn't working, at least not with my custom toolchain. I'll push this as the pine64_uboot-wiki branch for reference. Next I'll try Megous Repos. The first thing I'll try with Megous is to perform the bootloader build manually, without integrating with PiBox. If I can get that to boot (using their toolchain) then I know I should be able to reproduce with my build. The link to the Linaro toolchain is broken, but it's available from the Linaro website.

Bootloader update¶

I've got a toolchain built and am trying against branches of xeon configured for two of the three bootloader build references. Both references suggests a single binary be written to an SD card. Previous work on rk3399 had the idbloader.img written first followed by the u-boot/ATF combo so I need to resolve why there is a difference here.

1. Megous build completed.
   1. branch is megous
   2. It's not clear if the single binary generated by this build should be written to the SD card like the Pine64 Wiki suggests or if I need an idbloader.img, which is not generated by the build.
2. Pine64 Wiki build fails
   1. branch is pine64-uboot-wiki
   2. Fixed pibox-xeon build configuration but the Crust repository wants the binary toolchain. My toolchain doesn't accept some options, such as -msfimm and -mshftimm

Before trying the more complex Manjaro build I need to just try the single binary from Megous. I can also ask on the IRC channel for help.

I tried the megous build, without kernel or anything else. Nothing happened. It likely doesn't have console output enabled if it's working at all.

I tried a binary of p-boot, without kernel or anything else. Nothing happened. The build for p-boot fails if I use the ork1 or aarch64 musl cross toolchains. Also, the binaries provided assume you're trying to do your work on an arm64 platform which doesn't work for me.

So I'm on to the Manjaro build. The master branch has a commit that suggests lots of PPP work has been upstreamed , so I may not need to patch much.

If I use u-boot from Manjaro then theoretically I can use the standard rk3399 boot structure. Look at src/ximba/raspberrypi4/extras/distros/Manjaro/boot/phosh-beta30 for the files found in the first partition of the phosh SD card. It looks like this:

$ sudo fdisk -l /dev/sdg
GPT PMBR size mismatch (12941311 != 30535679) will be corrected by write.
The backup GPT table is not on the end of the device.
Disk /dev/sdg: 14.56 GiB, 15634268160 bytes, 30535680 sectors
Disk model: Card  Reader    
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disklabel type: gpt
Disk identifier: 2244B8F3-B854-4C8C-85CE-0FEDC7F7518A

Device      Start      End  Sectors   Size Type
/dev/sdg1   62500   500000   437501 213.6M Microsoft basic data
/dev/sdg2  500001 12941278 12441278   5.9G Linux filesystem

But the type looks weird, so here's another way to look at it.

$ sudo lsblk -f /dev/sdg 
NAME   FSTYPE FSVER LABEL      UUID                                 FSAVAIL FSUSE% MOUNTPOINT
sdg                                                                                
|-sdg1 vfat   FAT16 BOOT_MNJRO 1003-6743                             181.7M    15% /mnt/boot
`-sdg2 ext4   1.0   ROOT_MNJRO 1ff58e5c-324b-4315-8bf9-f1aa2ef1f556  161.2M    92% /mnt/disk

$ ll /mnt/boot
total 28100
-rwxr-xr-x 1 root root 19666952 Dec 10  2021 Image 
drwxr-xr-x 4 root root     4096 Dec 14  2021 dtbs 
drwxr-xr-x 2 root root     4096 Dec 14  2021 extlinux 
-rwxr-xr-x 1 root root   166570 Dec 12  2021 idbloader.img 
-rwxr-xr-x 1 root root  8028566 Dec 14  2021 initramfs-linux.img 
-rwxr-xr-x 1 root root   894448 Dec 12  2021 u-boot.itb 

And these are the following.

$ file *
Image:               Linux kernel ARM64 boot executable Image, little-endian, 4K pages
boot.scr:            u-boot legacy uImage, U-Boot boot script, Linux/ARM, Script File (Not compressed), 1449 bytes, Mon Mar 1
3 07:15:15 2023, Load Address: 0x00000000, Entry Point: 0x00000000, Header CRC: 0x1B10CEB3, Data CRC: 0xB65BF1CC
boot.txt:            ASCII text
dtbs:                directory
idbloader.img:       data
initramfs-linux.img: gzip compressed data, from Unix, original size modulo 2^32 18647040
u-boot.itb:          Device Tree Blob version 17, size=1408, boot CPU=0, string block size=131, DT structure block size=1216

boot.scr is generated from boot.txt via the ppp-uboot-mkscr script found in the u-boot repo.

Since I have the build instructions and the files on the boot partition I can do a test build of the bootloader, then write it to the SD card using one of the following methods.

1. https://opensource.rock-chips.com/wiki_Boot_option
2. https://wiki.pine64.org/wiki/RK3399_boot_sequence#Load_offsets
3. The method used on rk3399 at work

The second partition is the rootfs. I'll partition the rest of the card like I do with PiBox systems.

The first pass of the Manjaro u-boot build is complete. It generates a binary, but I've yet to try that binary.

Next step: Verify the u-boot components (idbloader.img, u-boot.itb) of the boot partition for Manjaro are available from the u-boot build. The verify the steps to creating the image.

Also: make Xeon versions of boot.txt and possibly ppp-uboot-mkscr.

The process I probably want, based on the u-boot README.rockchip, is this (the boot delay is set to 3 seconds for keyboard input):

Option 3: Package the image with TPL:

  - Write tpl+spl at 64th sector

    => sudo dd if=idbloader.img of=/dev/sdc seek=64

  - Write U-Boot proper at 16384 sector

    => sudo dd if=u-boot.itb of=/dev/sdc seek=16384
    => sync

Put this SD (or micro-SD) card into your board and reset it. You should see
something like:

U-Boot TPL board init
Trying to boot from BOOTROM
Returning to boot ROM...

U-Boot SPL board init
Trying to boot from MMC1

U-Boot 2019.07-rc1-00241-g5b3244767a (May 08 2019 - 10:51:06 +0530)

Model: Orange Pi RK3399 Board
DRAM:  2 GiB
MMC:   dwmmc@fe310000: 2, dwmmc@fe320000: 1, sdhci@fe330000: 0
Loading Environment from MMC... OK
In:    serial@ff1a0000
Out:   serial@ff1a0000
Err:   serial@ff1a0000
Model: Orange Pi RK3399 Board
Net:   eth0: ethernet@fe300000
Hit any key to stop autoboot:  0
=>

That didn't work either. What does work is to grab the tow-boot binary for PPP and write the shared.disk-image.img to the SD card. That booted directly into Manjaro, which is what I had written to the SD card previously. But after writing shared.disk-image.img to the SD card there is only one partition and it's not formatted. It looks like this:

$ sudo fdisk -l /dev/sdg
GPT PMBR size mismatch (26657 != 15523839) will be corrected by write.
The backup GPT table is not on the end of the device.
Disk /dev/sdg: 7.4 GiB, 7948206080 bytes, 15523840 sectors
Disk model: Card  Reader    
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disklabel type: gpt
Disk identifier: E0CA6E57-39B2-4482-9838-21E2785CD93D

Device     Start   End Sectors Size Type
/dev/sdg1     64 24639   24576  12M unknown

$ sudo lsblk -f /dev/sdg
NAME   FSTYPE FSVER LABEL UUID FSAVAIL FSUSE% MOUNTPOINT
sdg                                           
`-sdg1             

Since this booted into Manjaro that either means that Manjaro is embedded in that first partition or an unseen Manjaro partition I put on there previously is being used. Note that this first partition ends before sector 50000, which is where Manjaro's boot sector started when I tried to copy over Manjaro data to the SD card to test my own u-boot build. I can dd zero's to where that first partition was previously to really wipe it out and then try again. If it still boots to Manjaro then the tow-boot image has Manjaro embedded in it. And I don't want that. I suspect that is the case, though, based on the Shared Storage Strategy section of the Tow-Boot Getting Started Guide.

This all really sucks. I'm resisting using tow-boot because, while it's based on u-boot the build system has been changed to some esoteric thing (nix, think). I don't want to deal with that. u-boot should work - the PPP has a defconfig in the master branch of upstream u-boot. I prefer u-boot because I've worked on it before and I'm familiar with the code (at least a bit). Looking at tow-boot it certainly looks much simpler than u-boot but it's hard to tell exactly what it's doing.

I guess I can try partitioning the SD card with tow-boot installed, then adding Manjaro with customizations to the boot.txt that will tell me it's booting my stuff. If that works, then it's just a matter of pushing my kernels/rootfs to the partitions in place of Manjaro's. It's hacky, because I can't reproduce tow-boot yet. But I can try to build it myself.

In the meantime I can keep trying to get u-boot to load based on how we did it at work with our rk3399 board. It's possible I won't get serial console output in u-boot but I may be able to get it to boot my kernel at least, which would be sufficient for now.

Success¶

Xeon mksd and mkinstall have been updated to match the structure of the Manjaro SD card. A test build of the SD card with just the boot partition installed (no rootfs overlay yet) booted into the Xeon kernel and initramfs. This failed because the rootfs was not there, but also because the overlay and squashfs modules were not available either.

So this is major progress. I can now boot a kernel built by Xeon (using Megi's kernel tree via Tow-Boot) into a Xeon generated initramfs.

Next step is to debug the initramfs to make sure it fails only because the rootfs overlay/squashfs files are missing.

Note:¶

Tow-boot requires holding the volume-down key while booting in order to boot from the SD card VFAT partition. This will go away once Tow-Boot is written to eMMC or SPI, I think. For now, it's a development requirement.

And now I have the init script in the initramfs working all the way to attempting to mount the squashfs, which I haven't built yet.

Now I need to investigate what UI environment is being used on the phone for Manjaro. I can do this by booting the phone with Manjaro and then browsing the file system and running processes over the serial console. I'm hoping it's X.org and not wayland. Or maybe it's just a framebuffer based display.

Update

The Rockchip specs say the rk3399 is a T860, which Wikipedia says is a midgard GPU, so I need to look for midgard drivers.

• Login credentials: root/root
• Serial console is ttyS2

This is the relevant information about the display driver.

X.Org X Server 1.21.1.1
X Protocol Version 11, Revision 0
[     5.999] Current version of pixman: 0.40.0
[     6.014] (==) Using system config directory "/usr/share/X11/xorg.conf.d" 
[     6.021] (==) ModulePath set to "/usr/lib/xorg/modules" 
[     6.039] (II) Loading /usr/lib/xorg/modules/extensions/libglx.so
[     6.069] (II) Loading /usr/lib/xorg/modules/drivers/modesetting_drv.so
[     6.094] (II) Loading /usr/lib/xorg/modules/libglamoregl.so
[     7.208] (==) modeset(0): DPMS enabled
[     7.208] (II) modeset(0): [DRI2] Setup complete
[     7.208] (II) modeset(0): [DRI2]   DRI driver: rockchip
[     7.208] (II) modeset(0): [DRI2]   VDPAU driver: rockchip
[     7.238] (II) AIGLX: Loaded and initialized rockchip
[     7.476] (II) Loading /usr/lib/xorg/modules/input/libinput_drv.so

[root@manjaro-arm ~]# ls -l /usr/lib/xorg/modules
total 716
drwxr-xr-x 2 root root   4096 Dec 14  2021 drivers
drwxr-xr-x 2 root root   4096 Dec 14  2021 extensions
drwxr-xr-x 2 root root   4096 Dec 14  2021 input
-rwxr-xr-x 1 root root  92800 Nov 12 00:11 libexa.so
-rwxr-xr-x 1 root root  22744 Nov 12 00:11 libfbdevhw.so
-rwxr-xr-x 1 root root 204848 Nov 12 00:11 libglamoregl.so          <----
-rwxr-xr-x 1 root root 183120 Nov 12 00:11 libint10.so
-rwxr-xr-x 1 root root  10272 Nov 12 00:11 libshadowfb.so
-rwxr-xr-x 1 root root  34888 Nov 12 00:11 libshadow.so
-rwxr-xr-x 1 root root  31760 Nov 12 00:11 libvgahw.so
-rwxr-xr-x 1 root root 125800 Nov 12 00:11 libwfb.so
[root@manjaro-arm ~]# ls -l /usr/lib/xorg/modules/drivers
total 112
-rwxr-xr-x 1 root root 110992 Nov 12 00:11 modesetting_drv.so
[root@manjaro-arm ~]# ls -l /usr/lib/xorg/modules/extensions        <----
total 296
-rwxr-xr-x 1 root root 301968 Nov 12 00:11 libglx.so                <----
[root@manjaro-arm ~]# ls -l /usr/lib/xorg/modules/input
total 104
-rwxr-xr-x 1 root root 23040 Nov 12 00:11 inputtest_drv.so
-rwxr-xr-x 1 root root 79880 Nov 10 05:00 libinput_drv.so           <----

• libglamoregl.so: Must come from Mesa, I think. I enabled Mesa but I'm not seeing the library.
• /usr/lib/xorg/modules/extensions: This doesn't get created via the Xorg build. I see no options for "extensions".
• libglx.so: GLX is enabled with Mesa but the library isn't showing up in the target tree.
• libinput_drv.so: only available with udev builds, which I'm not doing.

I looked at the Mesa build and it looks like libglx may be embedded in libGL.so. So I don't need the extensions directory. But I don't know where libglamoregl.so comes from yet.

The Buildroot build just works. The X.org driver is the modesetting driver which is builtin. After some minor tweaks to the init scripts, the rootfs came up and I got the two xterm windows on the phone. So the phone now boots the Dev Platform. That's a big deal - I didn't think I'd get this far so quickly after getting towboot installed.

The next problem is the serial console. The serial console is required for initial dev platform work because the screen is too small and doesn't provide a keyboard (I could add bui-keyboard but that seems unnecessary with a serial console). Adding a usb keyboard might be possible if I use a USB-C to USB-A adapter or similar.

The serial console not working except in towboot. Towboot runs at 115000,8n1. The bootloader boot.txt from Manjaro sets it to 15000000 (probably 8n1). But there is no kernel output on the serial console nor can I get the getty to display. Manjaro also does not have any kernel boot messages but the getty does display.

There is no cmdline config in the defconfig. The rk3399-pinephone-pro.dts has the following line:

        stdout-path = "serial2:115200n8";

This suggest ttyS2 @ 115200,8n1. I' think I've tried that, but I'll try again. And then I'll see if I can change that to serial0 or serial1.

Note that serial2 is defined in rk3399.dtsi as uart2 which is disabled there. rk3399-pinephone-pro.dts enables uart2, but none of the other uarts. The Firefly wiki describes the rk3399 uarts and also uses uart2 for serial debug port (aka serial console). However, they suggest setting a serial debug port like this.

&uart4 {
       current-speed = <9600>;
       no-loopback-test;
       status = "okay";
};

The pinephone-pro DTS only enables the port but doesn't set the line speed property. Also, kernel documentation shows the chosen node like this:

/ {
    chosen {
        stdout-path = "/serial@f00:115200";
    };

    serial@f00 {
        compatible = "vendor,some-uart";
        reg = <0xf00 0x10>;
    };
};

Note that leading slash before the name "serial". That may not matter since the pinephone pro DTS is referencing the uart node via the serial2 alias.

In any case, it seems likely that the serial console is not working because of a mismatch between the selected port and line speed and the terminal program, in my case Minicom. That said, the boot.txt being used sets one console option as such:

console=/dev/ttyS2,115200n8

This seems correct which would leave just the Minicom configuration. Alternatively, I could use screen like so

screen /dev/ttyUSBx 115200