This resource provides the possibility of conducting a RISC-V full system simulation without a block device by leveraging [Linux's userspace support] (https://www.kernel.org/doc/html/latest/driver-api/early-userspace/early_userspace_support.html).
This document provides instructions to create a RISCV bootloader (berkeley bootloader (bbl)) and also points to the associated gem5 scripts to run riscv Linux full system simulations without using a disk image. The bootloader bbl is compiled with a Linux kernel, a device tree, and a workload. Similar to the riscv-fs resource, we'll also rely on BusyBox for basic Linux utilities, on UCanLinux for the configuration of the Linux kernel and the configuration of BusyBox, and on riscv-pk for building a proxy kernel.
riscv-fs-nodisk/ |___ gem5/ # gem5 source code (to be cloned here) | |___ riscv-gnu-toolchain/ # riscv tool chain for cross compilation | |___ riscv64-sample/ # UCanLinux source | |___ linux/ # linux source | |___ busybox/ # busybox source | |___ riscv-pk/ # riscv proxy kernel source (bbl) | |___ cpio/ # contains the .cpio files | |___ initdir/ # contains the structure of initramfs | |___ configs/ | |___ system # gem5 system config files | |___ run_riscv.py # gem5 run script | |___ README.md # This README file
When Linux kernel booting process takes place, initramfs, a root filesystem embedded into the kernel, will be loaded to memory. When initramfs is loaded, the kernel will try to execute one of the following scripts located in that filesystem, {/init, /sbin/init, /etc/init, /bin/init, /bin/sh}. Instead of using the default /init script, we will use our version of /init to execute the desired workload right after the early userspace is loaded.
Note: Since the initramfs decompressing process takes place while Linux kernel is booting (which means it will happen during the full system simulation), we'll try to minimize the size of the initramfs.
riscv-gnu-toolchainIn this step, we'll use GNU toolchain for RISC-V.
This step is necessary if you do not have basic libraries built for RISCV or if you're cross-compiling RISCV.
cd riscv-fs-nodisk/ git clone https://github.com/riscv-collab/riscv-gnu-toolchain --recursive cd riscv-gnu-toolchain git checkout 1a36b5dc44d71ab6a583db5f4f0062c2a4ad963b # --prefix parameter specifying the installation location ./configure --prefix=/opt/riscv make linux -j $(nproc)
To update the PATH environment variable so that the RISCV compilers can be found,
export PATH=$PATH:/opt/riscv/bin/
UCanLinux SourceThis repo contains a Linux configuration for RISCV at riscv64-sample/kernel.config and a BusyBox configuration at riscv64-sample/busybox.config.
# going back to base riscv-fs directory cd riscv-fs-nodisk/ git clone https://github.com/UCanLinux/riscv64-sample
busyboxMore information about Busybox is here.
cd riscv-fs-nodisk/ git clone git://busybox.net/busybox.git cd busybox git checkout 1_34_stable # checkout the a stable branch cp ../riscv64-sample/busybox.config .config yes "" | make CROSS_COMPILE=riscv64-unknown-linux-gnu- oldconfig make CROSS_COMPILE=riscv64-unknown-linux-gnu- all -j$(nproc) make CROSS_COMPILE=riscv64-unknown-linux-gnu- install
The files of interest are in busybox/_install/bin.
Linux kernelWe'll compiling the Linux kernel to get the linux/usr/gen_init_cpio, which would be used later.
cd riscv-fs-nodisk/ git clone --depth 1 --branch v5.10 https://git.kernel.org/pub/scm/linux/kernel/git/stable/linux.git cd linux cp ../riscv64-sample/kernel.config .config yes "" | make ARCH=riscv CROSS_COMPILE=riscv64-unknown-linux-gnu- oldconfig make ARCH=riscv CROSS_COMPILE=riscv64-unknown-linux-gnu- menuconfig # Go to "General setup --->" # Check on "Initial RAM filesystem and RAM disk (initramfs/initrd) support" make ARCH=riscv CROSS_COMPILE=riscv64-unknown-linux-gnu- all -j $(nproc)
cd riscv-fs-nodisk/ git clone https://gem5.googlesource.com/public/gem5 cd gem5/util/m5 scons build/riscv/out/m5
Note: the default cross-compiler is riscv64-unknown-linux-gnu-. To change the cross-compiler, you can set the cross-compiler using the scons sticky variable riscv.CROSS_COMPILE. For example,
scons riscv.CROSS_COMPILE=riscv64-linux-gnu- build/riscv/out/m5
initramfscd riscv-fs-nodisk/ mkdir cpio mkdir misc mkdir initdir
We'll use the riscv64-sample/initdir to define the structure of initramfs.
cd riscv-fs-nodisk/initdir cp -r ../busybox/_install/bin/ . mkdir lib cp /opt/riscv/sysroot/lib/ld-linux-riscv64-lp64d.so.1 lib/ # busybox' dependency cp /opt/riscv/sysroot/lib/libc.so.6 lib/ # busybox' dependency cp /opt/riscv/sysroot/lib/libm.so.6 lib/ # busybox' dependency cp /opt/riscv/sysroot/lib/libresolv.so.2 lib/ # busybox' dependency mkdir proc mkdir sys mkdir sbin cp ../gem5/util/m5/build/riscv/out/m5 sbin/m5 # replace m5 by the desired workload
Create initdir/init script with the following content,
#!/bin/busybox sh exec /sbin/init # script to execute the workload
Create initdir/sbin/init script with the following content,
#!/bin/busybox sh /sbin/m5 exit
Make the scripts executable,
chmod +x init chmod +x sbin/init
To create the cpio file of the initdir folder,
cd riscv-fs-nodisk/linux usr/gen_initramfs.sh -o ../cpio/disk.cpio ../initdir/ lsinitramfs ../cpio/disk.cpio # checking the file structure of the created cpio file
/dev/ folderBy default, initramfs would have a /dev/console and /dev/tty. Without these devices, we cannot see what is written to stdout and stderr.
The following commands will build a .cpio file with /dev/console and /dev/tty,
cd riscv-fs-nodisk/misc mkdir dev fakeroot -- mknod -m 622 dev/console c 5 1 fakeroot -- mknod -m 622 dev/tty c 5 0 fakeroot -- mknod -m 622 dev/ttyprintk c 5 3 fakeroot -- mknod -m 622 dev/null c 1 3 fakeroot -- find . -print0 | cpio --owner root:root --null -o --format=newc > ../cpio/dev.cpio cd ../ rm -r misc
Note: mknod -m 622 /dev/tty c 5 0 means we're creating /dev/tty with permission of 622. c means a character device being created, 5 is the major number, and 0 is the minor number. More information about the major/minor numbering is available at (https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/Documentation/admin-guide/devices.txt).
cd riscv-fs-nodisk/cpio cat disk.cpio dev.cpio > init.cpio
Linux Kernel with a customized initramfscd riscv-fs-nodisk/linux make ARCH=riscv CROSS_COMPILE=riscv64-unknown-linux-gnu- menuconfig # Go to "General setup --->" # Check on "Initial RAM filesystem and RAM disk (initramfs/initrd) support" # Change "Initramfs source file(s)" to the absoblute path of riscv-fs-nodisk/cpio/init.cpio make ARCH=riscv CROSS_COMPILE=riscv64-unknown-linux-gnu- all -j $(nproc)
The file of interest is at arch/riscv/boot/Image.
bbl with the Linux kernel as the payloadcd riscv-fs-nodisk/ git clone https://github.com/riscv/riscv-pk.git cd riscv-pk mkdir build cd build # configure bbl build ../configure --host=riscv64-unknown-linux-gnu --with-payload=../../linux/arch/riscv/boot/Image --prefix=/opt/riscv/ make -j$(nproc) chmod 755 bbl riscv64-unknown-linux-gnu-strip bbl cp bbl bbl-m5-exit
The desired bootloader is file is at riscv-fs-nodisk/riscv-pk/build/bbl or riscv-fs-nodisk/riscv-pk/build/bbl-m5-exit.
gem5/build/RISCV/gem5.opt configs/run_riscv.py bbl-m5-exit atomic 1