mbabaie | ea56865 | 2021-06-28 21:11:03 +0000 | [diff] [blame] | 1 | --- |
| 2 | title: RISC-V full system |
| 3 | tags: |
| 4 | - fullsystem |
| 5 | - riscv |
| 6 | layout: default |
| 7 | permalink: resources/riscv-fs |
| 8 | shortdoc: > |
| 9 | Resources to build a riscv disk image, a riscv boot loader and points to the gem5 scripts to run riscv Linux FS simulations. |
| 10 | author: ["Ayaz Akram"] |
| 11 | --- |
| 12 | |
Ayaz Akram | 9c86c41 | 2021-03-04 01:35:59 -0800 | [diff] [blame] | 13 | # RISCV Full System |
| 14 | |
| 15 | This document provides instructions to create a riscv disk image, a riscv boot loader (`berkeley bootloader (bbl)`) and also points to the associated gem5 scripts to run riscv Linux full system simulations. |
| 16 | The boot loader `bbl` is compiled with a Linux kernel and a device tree as well. |
| 17 | |
| 18 | The used disk image is based on [busybox](https://busybox.net/) and [UCanLinux](https://github.com/UCanLinux/). It is built using the instructions, mostly from [here](https://github.com/UCanLinux/riscv64-sample). |
| 19 | |
Ayaz Akram | 895efba | 2021-05-26 08:45:49 -0700 | [diff] [blame] | 20 | **Note:** All components are cross compiled on an x86 host using a riscv tool chain. We used `88b004d4c2a7d4e4f08b17ee32d2` commit of the riscv tool chain source while building the source (riscv gcc version 10.2.0). |
Ayaz Akram | 9c86c41 | 2021-03-04 01:35:59 -0800 | [diff] [blame] | 21 | |
| 22 | We assume the following directory structure while following the instructions in this README file: |
| 23 | |
| 24 | ``` |
| 25 | riscv-fs/ |
| 26 | |___ gem5/ # gem5 source code (to be cloned here) |
| 27 | | |
| 28 | |___ riscv-disk # built disk image will go here |
| 29 | | |
Ayaz Akram | 9c86c41 | 2021-03-04 01:35:59 -0800 | [diff] [blame] | 30 | |___ riscv-gnu-toolchain # riscv tool chain for cross compilation |
| 31 | | |
| 32 | |___ riscv64-sample # UCanLinux source |
| 33 | | |__linux # linux source |
| 34 | | |__busybox # busybox source |
| 35 | | |__riscv-pk # riscv proxy kernel source (bbl) |
| 36 | | |__RootFS # root file system for disk image |
| 37 | | |
Ayaz Akram | 9c86c41 | 2021-03-04 01:35:59 -0800 | [diff] [blame] | 38 | |___ README.md # This README file |
| 39 | ``` |
| 40 | |
| 41 | ## RISCV Toolchain |
| 42 | |
| 43 | We use `RISC-V GNU Compiler Toolchain`. To build the toolchain, follow the following instructions, assuming you are in the `riscv-fs` directory. |
| 44 | |
| 45 | ```sh |
| 46 | # install required libraries |
| 47 | sudo apt-get install -y autoconf automake autotools-dev curl python3 libmpc-dev libmpfr-dev libgmp-dev gawk build-essential bison flex texinfo gperf libtool patchutils bc zlib1g-dev libexpat-dev |
| 48 | |
| 49 | # clone riscv gnu toolchain source |
| 50 | git clone https://github.com/riscv/riscv-gnu-toolchain |
| 51 | cd riscv-gnu-toolchain |
Ayaz Akram | 895efba | 2021-05-26 08:45:49 -0700 | [diff] [blame] | 52 | git checkout 88b004d4c2a7d4e4f08b17ee32d2 |
Ayaz Akram | 9c86c41 | 2021-03-04 01:35:59 -0800 | [diff] [blame] | 53 | |
| 54 | # change the prefix to your directory |
| 55 | # of choice for installation of the |
| 56 | # toolchain |
| 57 | ./configure --prefix=/opt/riscv |
| 58 | |
| 59 | # build the toolchain |
| 60 | make linux -j$(nproc) |
| 61 | ``` |
| 62 | |
| 63 | Update the `PATH` environment variable so that the following instructions can figure out where to find the riscv toolchain. |
| 64 | |
| 65 | ```sh |
| 66 | export PATH=$PATH:/opt/riscv/bin/ |
| 67 | ``` |
| 68 | |
Hoa Nguyen | 24bbfc3 | 2021-07-19 15:49:52 -0700 | [diff] [blame] | 69 | **Note:** The above step is necessary and might cause errors while cross compiling different components for riscv if other methods are used to point to the toolchain. |
Ayaz Akram | 9c86c41 | 2021-03-04 01:35:59 -0800 | [diff] [blame] | 70 | |
| 71 | ## UCanLinux Source |
| 72 | |
| 73 | Clone the `UCanLinux source.` |
| 74 | |
| 75 | ```sh |
| 76 | # going back to base riscv-fs directory |
| 77 | cd ../ |
| 78 | |
| 79 | git clone https://github.com/UCanLinux/riscv64-sample |
| 80 | ``` |
| 81 | |
Ayaz Akram | 895efba | 2021-05-26 08:45:49 -0700 | [diff] [blame] | 82 | The following sections provide instructions to build both `bbl` and disk images. |
Ayaz Akram | 9c86c41 | 2021-03-04 01:35:59 -0800 | [diff] [blame] | 83 | |
| 84 | ## Linux Kernel |
| 85 | |
| 86 | Clone the latest LTS Linux kernel (v5.10): |
| 87 | |
| 88 | ```sh |
| 89 | cd riscv64-sample/ |
| 90 | git clone --depth 1 --branch v5.10 https://git.kernel.org/pub/scm/linux/kernel/git/stable/linux.git |
| 91 | ``` |
| 92 | |
| 93 | To configure and compile the kernel: |
| 94 | |
| 95 | ```sh |
| 96 | cd linux |
| 97 | |
| 98 | # copy the kernel config from the riscv64-sample |
| 99 | # directory (cloned previously) |
| 100 | |
| 101 | cp ../kernel.config .config |
| 102 | |
| 103 | # configure the kernel and build it |
| 104 | make ARCH=riscv CROSS_COMPILE=riscv64-unknown-linux-gnu- menuconfig |
| 105 | make ARCH=riscv CROSS_COMPILE=riscv64-unknown-linux-gnu- all -j$(nproc) |
| 106 | ``` |
| 107 | |
| 108 | This should generate a `vmlinux` image in the `linux` directory. |
Bobby R. Bruce | fa12e88 | 2022-07-29 14:03:56 -0700 | [diff] [blame] | 109 | A pre-built RISC-V 5.10 linux kernel can be downloaded [here](http://dist.gem5.org/dist/v22-0/kernels/riscv/static/vmlinux-5.10). |
Ayaz Akram | 9c86c41 | 2021-03-04 01:35:59 -0800 | [diff] [blame] | 110 | |
| 111 | ## Bootloader (bbl) |
| 112 | |
| 113 | To build the bootloader, clone the RISCV proxy kernel (`pk`) source, which is an application execution environment and contains the bbl source as well. |
| 114 | |
| 115 | ```sh |
| 116 | # going back to base riscv64-sample directory |
| 117 | cd ../ |
| 118 | git clone https://github.com/riscv/riscv-pk.git |
| 119 | |
| 120 | cd riscv-pk |
| 121 | |
| 122 | mkdir build |
| 123 | cd build |
| 124 | |
| 125 | apt-get install device-tree-compiler |
| 126 | |
Ayaz Akram | 895efba | 2021-05-26 08:45:49 -0700 | [diff] [blame] | 127 | # configure bbl build |
| 128 | ../configure --host=riscv64-unknown-linux-gnu --with-payload=../../linux/vmlinux --prefix=/opt/riscv/ |
Ayaz Akram | 9c86c41 | 2021-03-04 01:35:59 -0800 | [diff] [blame] | 129 | |
Ayaz Akram | 9c86c41 | 2021-03-04 01:35:59 -0800 | [diff] [blame] | 130 | make -j$(nproc) |
| 131 | |
| 132 | chmod 755 bbl |
| 133 | |
| 134 | # optional: strip the bbl binary |
| 135 | riscv64-unknown-linux-gnu-strip bbl |
| 136 | ``` |
| 137 | |
| 138 | This will produce a `bbl` bootloader binary with linux kernel in `riscv-pk/build` directory. |
Bobby R. Bruce | fa12e88 | 2022-07-29 14:03:56 -0700 | [diff] [blame] | 139 | A pre-built copy of this bootloard binary, with the linux kernel can be downloaded [here](http://dist.gem5.org/dist/v22-0/kernels/riscv/static/bootloader-vmlinux-5.10). |
Ayaz Akram | 9c86c41 | 2021-03-04 01:35:59 -0800 | [diff] [blame] | 140 | |
| 141 | ## Busy Box |
| 142 | |
| 143 | Clone and compile the busybox: |
| 144 | |
| 145 | ```sh |
| 146 | # going back to riscv64-sample directory |
| 147 | cd ../.. |
| 148 | git clone git://busybox.net/busybox.git |
| 149 | cd busybox |
| 150 | git checkout 1_30_stable # checkout the latest stable branch |
| 151 | make menuconfig |
Ayaz Akram | 895efba | 2021-05-26 08:45:49 -0700 | [diff] [blame] | 152 | cp ../busybox.config .config # optional |
Ayaz Akram | 9c86c41 | 2021-03-04 01:35:59 -0800 | [diff] [blame] | 153 | make menuconfig |
| 154 | make CROSS_COMPILE=riscv64-unknown-linux-gnu- all -j$(nproc) |
| 155 | make CROSS_COMPILE=riscv64-unknown-linux-gnu- install |
| 156 | ``` |
| 157 | |
| 158 | ## Root File System for Disk Image |
| 159 | |
| 160 | Next, we will be setting up a root file system: |
| 161 | |
| 162 | ```sh |
| 163 | # going back to riscv64-sample directory |
Ayaz Akram | 895efba | 2021-05-26 08:45:49 -0700 | [diff] [blame] | 164 | cd ../ |
Ayaz Akram | 9c86c41 | 2021-03-04 01:35:59 -0800 | [diff] [blame] | 165 | |
| 166 | mkdir RootFS |
| 167 | cd RootFS |
| 168 | cp -a ../skeleton/* . |
| 169 | |
| 170 | # copy linux tools/binaries from busbybox (created above) |
| 171 | cp -a ../busybox/_install/* . |
| 172 | |
| 173 | # install modules from linux kernel compiled above |
| 174 | cd ../linux/ |
| 175 | make ARCH=riscv CROSS_COMPILE=riscv64-unknown-linux-gnu- INSTALL_MOD_PATH=../RootFS modules_install |
| 176 | |
| 177 | # install libraries from the toolchain built above |
| 178 | cd ../RootFS |
| 179 | cp -a /opt/riscv/sysroot/lib . |
| 180 | |
| 181 | # create empty directories |
| 182 | mkdir dev home mnt proc sys tmp var |
| 183 | cd etc/network |
| 184 | mkdir if-down.d if-post-down.d if-pre-up.d if-up.d |
| 185 | |
| 186 | # build m5 util for riscv and move |
| 187 | # it to the root file system as well |
Ayaz Akram | 895efba | 2021-05-26 08:45:49 -0700 | [diff] [blame] | 188 | cd ../../../../ |
Ayaz Akram | 9c86c41 | 2021-03-04 01:35:59 -0800 | [diff] [blame] | 189 | cd gem5/util/m5 |
Ayaz Akram | 895efba | 2021-05-26 08:45:49 -0700 | [diff] [blame] | 190 | scons build/riscv/out/m5 |
| 191 | cp build/riscv/out/m5 ../../../riscv64-sample/RootFS/sbin/ |
Ayaz Akram | 9c86c41 | 2021-03-04 01:35:59 -0800 | [diff] [blame] | 192 | ``` |
| 193 | |
Hoa Nguyen | 0b9d579 | 2021-07-19 15:07:16 -0700 | [diff] [blame] | 194 | **Note**: the default cross-compiler is `riscv64-unknown-linux-gnu-`. To change the cross-compiler, you can set the cross-compiler using the scons sticky variable `riscv.CROSS_COMPILE`. For example, |
| 195 | ```sh |
| 196 | scons riscv.CROSS_COMPILE=riscv64-linux-gnu- build/riscv/out/m5 |
| 197 | ``` |
Ayaz Akram | 9c86c41 | 2021-03-04 01:35:59 -0800 | [diff] [blame] | 198 | ## Disk Image |
| 199 | |
| 200 | Create a disk of 512MB size. |
| 201 | |
| 202 | ```sh |
| 203 | cd ../../../ |
| 204 | dd if=/dev/zero of=riscv_disk bs=1M count=512 |
| 205 | ``` |
| 206 | |
| 207 | Making and mounting a root file system on the disk: |
| 208 | |
| 209 | ```sh |
| 210 | mkfs.ext2 -L riscv-rootfs riscv_disk |
| 211 | |
| 212 | sudo mkdir /mnt/rootfs |
| 213 | sudo mount riscv_disk /mnt/rootfs |
| 214 | |
Ayaz Akram | 895efba | 2021-05-26 08:45:49 -0700 | [diff] [blame] | 215 | sudo cp -a riscv64-sample/RootFS/* /mnt/rootfs |
Ayaz Akram | 9c86c41 | 2021-03-04 01:35:59 -0800 | [diff] [blame] | 216 | |
| 217 | sudo chown -R -h root:root /mnt/rootfs/ |
| 218 | df /mnt/rootfs |
Ayaz Akram | 9c86c41 | 2021-03-04 01:35:59 -0800 | [diff] [blame] | 219 | sudo umount /mnt/rootfs |
| 220 | ``` |
| 221 | |
| 222 | The disk image `riscv_disk` is ready to use. |
Bobby R. Bruce | fa12e88 | 2022-07-29 14:03:56 -0700 | [diff] [blame] | 223 | A pre-built, gzipped, disk image can be downloaded [here](http://dist.gem5.org/dist/v22-0/images/riscv/busybox/riscv-disk.img.gz). |
Ayaz Akram | 9c86c41 | 2021-03-04 01:35:59 -0800 | [diff] [blame] | 224 | |
| 225 | **Note:** If you need to resize the disk image once it is created, you can do the following: |
| 226 | |
| 227 | ```sh |
| 228 | e2fsck -f riscv_disk |
| 229 | resize2fs ./riscv_disk 512M |
| 230 | ``` |
| 231 | |
| 232 | Also, if it is required to change the contents of the disk image, it can be mounted as: |
| 233 | |
| 234 | ```sh |
| 235 | mount -o loop riscv_disk [some mount directory] |
| 236 | ``` |
| 237 | |
Bobby R. Bruce | 976dfb5 | 2022-03-30 11:11:27 -0700 | [diff] [blame] | 238 | ## Example Run Script |
Ayaz Akram | 9c86c41 | 2021-03-04 01:35:59 -0800 | [diff] [blame] | 239 | |
Bobby R. Bruce | 976dfb5 | 2022-03-30 11:11:27 -0700 | [diff] [blame] | 240 | An example configuration using this disk image with the boot loader can be found in `configs/example/gem5_library/riscv-fs.py` in the gem5 repository. |
Ayaz Akram | 9c86c41 | 2021-03-04 01:35:59 -0800 | [diff] [blame] | 241 | |
Bobby R. Bruce | 976dfb5 | 2022-03-30 11:11:27 -0700 | [diff] [blame] | 242 | To run this, you can execute the following within the gem5 repository: |
Ayaz Akram | 9c86c41 | 2021-03-04 01:35:59 -0800 | [diff] [blame] | 243 | |
| 244 | ```sh |
Bobby R. Bruce | 976dfb5 | 2022-03-30 11:11:27 -0700 | [diff] [blame] | 245 | scons build/RISCV/gem5.opt -j`nproc` |
| 246 | ./build/RISCV/gem5.opt configs/example/gem5_library/riscv-fs.py |
Ayaz Akram | 9c86c41 | 2021-03-04 01:35:59 -0800 | [diff] [blame] | 247 | ``` |
| 248 | |
Bobby R. Bruce | 976dfb5 | 2022-03-30 11:11:27 -0700 | [diff] [blame] | 249 | The gem5 stdlib will automatically download the resources as required. |
| 250 | Once the simulation has booted you can interact with the system's console via `telnet`: |
Ayaz Akram | 9c86c41 | 2021-03-04 01:35:59 -0800 | [diff] [blame] | 251 | |
| 252 | ```sh |
Hoa Nguyen | 24bbfc3 | 2021-07-19 15:49:52 -0700 | [diff] [blame] | 253 | telnet localhost <port> |
| 254 | ``` |
| 255 | |
| 256 | Another option is to use `m5term` provided by gem5. To compile and launch `m5term`, |
| 257 | ```sh |
| 258 | cd gem5/util/term |
| 259 | make # compiling |
| 260 | ./m5term localhost <port> # launching the terminal |
Ayaz Akram | 9c86c41 | 2021-03-04 01:35:59 -0800 | [diff] [blame] | 261 | ``` |
| 262 | |
Bobby R. Bruce | 976dfb5 | 2022-03-30 11:11:27 -0700 | [diff] [blame] | 263 | The linux has both `login` and `password` set as `root`. |