kobo_clara-custom-distro.md (11901B)
1 title: Kobo Clara HD Custom Linux Distro/RootFS 2 date: 2021-07-22 12:00 3 --- 4 5 These are just some notes I made when creating my own mini-distro after 6 wanting something more custom than just using buildroot or making the 7 official firmware more slim. For people other than me, I suggest 8 looking through (C)LFS or running postmarketOS instead once this 9 reader's pull request[1] gets integrated into upstream. 10 11 Two things that'll greatly help with this is having serial terminal 12 access with the four uart pins near the top right in the back of the 13 reader, near the uSD card slot (I don't connect the 5V pin as my reader 14 doesn't really turn on anything other than the power LED). I suggest 15 maybe soldering female pin headers to there to make your life easier 16 (you can later cut out a hole in the back cover or desolder the headers 17 once you're done). Other than that, I suggest installing QEMU with ARM 18 userspace to test programs that you have built or running them on a 19 separate ARM device like a Raspberry Pi. 20 21 ## Prelude 22 Ever since I learnt that the official firmware for the Clara was just 23 using a modified Linux kernel with busybox as coreutils and many other 24 libraries, I just knew that I had to minimize it. I also saw that it 25 was using glibc for it's libc, which I really dislike as statically 26 linking C programs against it was a pain in my experience, compared to 27 something like musl and uclibc. It's also much larger than them and I 28 don't use any of glibc extensions so it seemed like a waste of space to 29 me. 30 31 Initially when I replaced Nickel with Plato, I was able to shave about 32 100 MiB after I removed /usr/local (which contains Nickel, Qt and a few 33 other things), from 189 MiB to 74 MiB, but I still wanted to make it 34 smaller. 35 36 Using buildroot, I was able to get it under 2 MiB (!!) which was a 37 little less than half the size of an uncompressed armhf Alpine Linux 38 minirootfs (4.9M for 3.14). With Busybox, it was pretty much working 39 out of the box, with serial terminal access! But waiting around 15 40 minutes for the toolchain to build each time I wanted to change 41 something in the rootfs took way too long, although it could've been 42 minimized if I used ccache with a fairly large cache size. I still 43 found that it compiled and installed a lot of things I wouldn't be 44 using (particularly in /usr) even after disabling almost all of the 45 third-party packages. 46 47 I've uploaded the config file and the resulting rootfs for 48 buildroot 2021.05. The root password by default is changeme. 49 EDIT 2022-10-21: gone, build it yourself 50 51 Of course the rootfs I got from buildroot nor me making the official 52 firmware smaller is the point of this article, and the actual point is 53 making one yourself! (or rather what I did to make my own) 54 55 ## Cross-toolchain 56 For now as of July 22, 2021, I'm using my distro (Void Linux)'s 57 packaged cross toolchain for armhf musl, but eventually I would be 58 using my own. 59 60 I'm not compiling off of the device itself as it would be somewhat slow 61 for bigger programs, which is currently primarily the Linux kernel, 62 U-Boot, and the toolchain itself, considering that the ereader's CPU 63 (Freescale i.MX 6SLL) is a single core running up to 1 GHz. Including 64 the development tools and headers would also take up more space on the 65 device itself, and since the terminal can currently only be accessed 66 through it's serial/uart pins, I don't think it's ideal. 67 68 TODO: include steps to create own toolchain (probably based off of gcc 69 4.7.3 as that doesn't require c++) 70 71 ## Building the rootfs 72 Assuming you made a new filesystem on your rootfs's partition, it'll 73 likely be empty with no directories you'd expect to find on a regular 74 distro. So you'll just have to make them. 75 cd /path/to/rootfs 76 mkdir bin dev etc proc sbin 77 78 Your binaries would usually go in /bin, the uSD card, ttymxc0, and 79 other devices would go in /dev, felker init's default program/script to 80 execute is usually in /etc/rc, /proc is optional but I have it mounted 81 to see what is currently mounted through /proc/mounts (or mount(1) 82 without any arguments) as well as to see my disk usage through df(1). 83 /sbin is there to place the init in as /sbin/init is the default init 84 path the kernel looks at. 85 86 ## toybox 87 Now on to the main part of the distro, the userspace. I intend to keep 88 it fairly minimal so I've chosen to use toybox along with a slightly 89 modified version of felker (musl dev)'s init[2], as well as dash[3] as 90 the main shell since toybox doesn't include one as of 0.8.5 (though 91 it'll probably be there by 1.0). I'll also be statically linking all 92 the programs that'll be used so I wouldn't have to worry about shared 93 libraries not being included/copied over, and also including LTO for 94 slightly faster binaries. Originally, I tried going with sinit, sbase, 95 and ubase but I was having trouble getting serial terminal access with 96 getty to /dev/ttymxc0 (the default serial tty, at least with the 97 vendor kernel). I didn't have this problem with busybox's and toybox's 98 getty however. My config for toybox was also about 81K smaller than my 99 trimmed sbase-box and ubase-box (352K compared to 267K+166K) where I 100 removed programs that I won't use from ${BIN} in their respective 101 Makefiles. 102 EDIT 2022-10-21: also gone 103 104 First I suggest exporting some environment variables to set the 105 toolchain used as well as enabling static linking and LTO. 106 107 ``` 108 export CROSS_COMPILE="arm-linux-musleabihf-" # change to your cross-tc 109 export CC="${CROSS_COMPILE}gcc" 110 export LDFLAGS="--static" 111 export CFLAGS="-flto -static" 112 export ARCH=arm # for compiling the linux kernel 113 ``` 114 115 To compile toybox, get the source from 116 https://landley.net/toybox/downloads/ (or clone the upstream repo). 117 Then run make menuconfig (optionally with make defconfig before it) and 118 change it as you see fit. Personally, I disabled most of the programs I 119 wouldn't use and kept only the ones that'll help with fixing a problem. 120 Finally, make sure to run make. 121 122 ``` 123 make defconfig 124 make menuconfig 125 make 126 ``` 127 128 To move it to your rootfs and set it's symlinks, you could probably run 129 make install after setting PREFIX to your rootfs's /bin directory, but 130 I did it manually. 131 132 ``` 133 # automatic (didn't test, check README) 134 make PREFIX=/path/to/rootfs/bin/ install 135 136 # (semi?) manual 137 cp toybox /path/to/rootfs/bin 138 139 # add symlinks if doing manual and you want them 140 cd /path/to/rootfs/bin 141 for prog in $(qemu-arm ./toybox); do ln -s toybox "$prog"; done 142 ``` 143 144 ## dash 145 Also as of toybox 0.8.5, a shell still isn't included (probably would 146 be included by 1.0 according to scripts/install.sh as well as a few 147 other programs like gzip), so a separate shell would need to be built. 148 Any can be used but dash would be shown as an example as I was able to 149 get a static binary without too much trouble. 150 151 First obtain the source[3] and cd into its 152 untarred directory. Assuming your CC and CFLAGS are set, you can run 153 these steps: 154 155 ``` 156 autoreconf -fiv 157 ./configure --host=$CROSS_COMPILE --with-libedit 158 make 159 ${CROSS_COMPILE}strip src/dash 160 ``` 161 162 As this is going to be used as the main shell, I've decided to just 163 copy it to /bin/sh in the rootfs directory, though copying it there but 164 as /bin/dash and /bin/sh being symlinked to dash is also an option. 165 166 ``` 167 cp src/dash /path/to/rootfs/bin/sh 168 # or 169 cp src/dash /path/to/rootfs/bin 170 cd /path/to/rootfs/bin 171 ln -s dash sh 172 ``` 173 174 ## felker's init 175 The init is just a single file that you can get from felker's site[2] 176 or the gist on github[7]. I haven't had a good experience with the 177 default startup program (/etc/rc) as a shell script with execve() run 178 on it so I'd change it to execvp() and remove the third (specifies 179 environment). To compile and install the init, all you need to do is 180 run: 181 182 ``` 183 $CC $CFLAGS -o init init.c 184 cp init /path/to/rootfs/sbin 185 ``` 186 187 Instead of /etc/rc being a shell script, you can also make a C program 188 that does whatever you think is needed for a proper startup. I'll still 189 use a shell script though which is linked here. 190 EDIT 2022-10-21: you get the idea, it's gone. 191 192 ## /etc/passwd 193 Copying the rootfs's contents to your device's/uSD card's root 194 partition and then turning the device on should now work with a login 195 prompt shown in the serial terminal. However, you probably wouldn't be 196 able to login to any user. So you'll have to create a file at 197 /path/to/rootfs/etc/passwd. For an empty password to root, you can use 198 this, though I suggest setting a password as soon as you login: 199 200 ``` 201 # in rootfs's /etc/passwd 202 root::0:0:root:/root:/bin/sh 203 ``` 204 205 With the passwd file created/updated, you should now be able to login 206 to root after the rootfs is copied to your uSD card. Your rootfs so far 207 should now be around 550-560K, which is much much smaller than the 208 original firmware's, though it'll likely be much larger to maybe a few 209 megabytes once a proper reader software is added. 210 211 ## Custom Linux Kernel 212 WARNING: I haven't actually gotten the kernel to load in u-boot yet. It 213 just hangs in the "Starting kernel ..." step and the init doesn't get 214 loaded, so I'm assuming the kernel itself isn't either. If anyone out 215 there has gotten a custom kernel working in the Kobo Clara HD, please 216 send me an email or message on xmpp. 217 218 UPDATE Jul 28, 2021: Gave up on it as I just couldn't get any kernels I 219 built (both vendor and akemnade's mainline) to boot. But neither did 220 postmarketOS boot beyond the initial initramfs messages without the log 221 file being created. So I'll revisit this for later. 222 223 EDIT 2022-10-21: I have gotten this working, but have been unable to 224 get Plato build for musl, so I will have to either continue fighting 225 with the crab or create my own with fbink, as that still works. 226 Separate article on this later. 227 228 My next big step is compiling my own kernel for the Clara HD. With the 229 default configuration built for the vendor kernel, it appears to be 230 about 3M, so my goal is to build a kernel that is smaller than that 231 while retaining only the functionality that I need. I'm also not going 232 to include networking support as that is unneeded for my purposes, but 233 I suggest just keeping it if you're unsure. The wifi driver for the 234 Kobo Clara HD is available as an out-of-tree driver[8]. 235 236 You should first obtain the kernel source, with two main options, the 237 vendor kernel[9] and the mainline kernel (with akemnade's 238 patches)[10]. For the latter, you need to clone the repo and switch to 239 the latest kobo/drm-merged branch (kobo/merged-5.13 as of July 25, 240 2021). 241 242 After you've got them and assuming the CROSS_COMPILE and ARCH 243 environment variables are set, you'd want to configure the kernel. 244 245 I had a hard time compiling the vendor kernel with many things 246 disabled, so I've kept my config somewhat similar to the default 247 config. The config I used is available here (EDIT: dead). 248 249 ``` 250 make menuconfig 251 make zImage 252 ``` 253 254 Assuming it compiles properly and arch/arm/boot/zImage exists, all 255 that's needed to is to write it to your uSD card at the 1M offset. 256 dd if=/path/to/kernel/zImage of=/path/to/uSDdev bs=512 seek=2048 257 258 ## Custom U-Boot 259 I have not done this yet, nor really plan to, but if you do manage to 260 compile the Kobo's vendored u-boot source, then all you'd have to do to 261 install it is: 262 263 ``` 264 dd if=u-boot-file of=/dev/mmcblk0 bs=128k count=1 seek=6 265 ``` 266 267 If I remember correctly, this command was included in an older 268 firmware's startup script/rcS for updating udev, and it should still 269 work. 270 271 ## Links 272 [1]: https://gitlab.com/postmarketOS/pmaports/-/merge_requests/2334 273 [2]: https://ewontfix.com/14 274 [3]: https://git.kernel.org/pub/scm/utils/dash/dash.git 275 [4]: https://github.com/akemnade/linux/tree/kobo/merged-5.13 276 [5]: https://misc.andi.de1.cc/kobo/uboot-env.txt 277 [6]: https://misc.andi.de1.cc/kobo/ 278 [7]: https://gist.github.com/rofl0r/6168719/raw/183525e0f0007169a49392b21ceee5b507e3aee8/init.c 279 [8]: https://github.com/jwrdegoede/rtl8189ES_linux/tree/rtl8189fs 280 [9]: https://github.com/kobolabs/Kobo-Reader/blob/master/hw/imx6sll-clara/kernel.tar.bz2 281 [10]: https://github.com/akemnade/linux/tree/kobo/drm-merged-5.12