<?xml version="1.0" encoding="UTF-8"?>
<rss version="2.0" xmlns:atom="http://www.w3.org/2005/Atom">
  <channel>
    <title>gem5</title>
    <description></description>
    <link>http://localhost:4000/</link>
    <atom:link href="http://localhost:4000/feed.xml" rel="self" type="application/rss+xml" />
    <pubDate>Mon, 28 Jan 2019 11:35:33 -0800</pubDate>
    <lastBuildDate>Mon, 28 Jan 2019 11:35:33 -0800</lastBuildDate>
    <generator>Jekyll v3.7.4</generator>
    
      <item>
        <title>Visualizing Spectre with gem5</title>
        <description>&lt;p&gt;&lt;a href=&quot;https://meltdownattack.com/&quot;&gt;Spectre and Meltdown&lt;/a&gt; took much of our
community by surprise. I personally found these attacks fascinating
because they didn’t rely on a &lt;em&gt;bug&lt;/em&gt; in any particular hardware
implementation, but leveraged undefined behavior. Specifically, Spectre
and Meltdown can exfiltrate potentially secret memory data by detecting
the effects of speculative instructions &lt;em&gt;that are later squashed&lt;/em&gt;.&lt;/p&gt;

&lt;p&gt;Very cool!&lt;/p&gt;

&lt;p&gt;Out of order processors are very complex. It would make it easier to
understand exactly what causes speculation attacks like Spectre and
Meltdown if we had a way to &lt;em&gt;visualize&lt;/em&gt; the attacks. Luckily, gem5
already has a way to view the details of it’s out of order CPU’s
pipeline.&lt;/p&gt;

&lt;p&gt;&lt;img src=&quot;/assets/img/o3-example.png&quot; alt=&quot;o3 pipeline view example&quot; /&gt;&lt;/p&gt;

&lt;p&gt;The image above was created using the O3 pipeline viewer that is
included with gem5. In this post, I’ll explain how to use the O3
pipeline viewer and how to generate images like the above. There is also
a new project which makes it easier to navigate large pipeline traces
and it is useful for comparing different pipeline designs:
&lt;a href=&quot;https://github.com/shioyadan/Konata&quot;&gt;Konata&lt;/a&gt; created by Ryota Shioya.
Ryota gave a presentation on Konata at a recent &lt;a href=&quot;http://learning.gem5.org/tutorial/index.html&quot;&gt;Learning gem5
tutorial&lt;/a&gt;. You can find
the pdf of his presentation
&lt;a href=&quot;http://learning.gem5.org/tutorial/presentations/vis-o3-gem5.pdf&quot;&gt;here&lt;/a&gt;.
Konata is a cool tool that’s written in javascript and Ryota describes
it as “Google maps for an out of order pipeline”.&lt;/p&gt;

&lt;h2 id=&quot;running-spectre&quot;&gt;Running Spectre&lt;/h2&gt;

&lt;p&gt;The first step to visualizing what is going on in the pipeline during a
Spectre attack is getting proof of concept exploit code. I used the code
that was posted to a github gist by Erik August soon after the attack
was announced. You can get that code here:
&lt;a href=&quot;https://gist.github.com/ErikAugust/724d4a969fb2c6ae1bbd7b2a9e3d4bb6&quot;&gt;https://gist.github.com/ErikAugust/724d4a969fb2c6ae1bbd7b2a9e3d4bb6&lt;/a&gt;.&lt;/p&gt;

&lt;p&gt;First, you need to compile the proof of concept code on your native
machine (note: I’ll be using x86 for all of my examples).&lt;/p&gt;

&lt;div class=&quot;highlighter-rouge&quot;&gt;&lt;div class=&quot;highlight&quot;&gt;&lt;pre class=&quot;highlight&quot;&gt;&lt;code&gt;gcc spectre.c -o spectre -static
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt;

&lt;p&gt;I used gcc 7.2 (the default on Ubuntu 17.10) for my tests, and you may
want to do the same. &lt;a href=&quot;#effects-of-compilers&quot;&gt;Below&lt;/a&gt; I discuss the
effects different compilers have on the Specre attack. For instance, if
you use clang instead you may not be able to reproduce the Spectre
attack in gem5.&lt;/p&gt;

&lt;p&gt;My native machine is still vulnerable to Spectre so when I run the
binary generated above, I get the following output.&lt;/p&gt;

&lt;div class=&quot;highlighter-rouge&quot;&gt;&lt;div class=&quot;highlight&quot;&gt;&lt;pre class=&quot;highlight&quot;&gt;&lt;code&gt;Reading 40 bytes:                 
Reading at malicious_x = 0xffffffffffdd76c8... Success: 0x54=’T’ score=2                                                                
Reading at malicious_x = 0xffffffffffdd76c9... Success: 0x68=’h’ score=2                                                                
Reading at malicious_x = 0xffffffffffdd76ca... Success: 0x65=’e’ score=2                                                                
Reading at malicious_x = 0xffffffffffdd76cb... Success: 0x20=’ ’ score=2                                                                
Reading at malicious_x = 0xffffffffffdd76cc... Success: 0x4D=’M’ score=2                                                                
Reading at malicious_x = 0xffffffffffdd76cd... Success: 0x61=’a’ score=2                                                                
Reading at malicious_x = 0xffffffffffdd76ce... Success: 0x67=’g’ score=2                                                                
Reading at malicious_x = 0xffffffffffdd76cf... Success: 0x69=’i’ score=2                                                                
Reading at malicious_x = 0xffffffffffdd76d0... Success: 0x63=’c’ score=2                                                                
Reading at malicious_x = 0xffffffffffdd76d1... Success: 0x20=’ ’ score=2                                                                
Reading at malicious_x = 0xffffffffffdd76d2... Success: 0x57=’W’ score=2                                                                
Reading at malicious_x = 0xffffffffffdd76d3... Success: 0x6F=’o’ score=2                                                                
Reading at malicious_x = 0xffffffffffdd76d4... Success: 0x72=’r’ score=2                                                                
Reading at malicious_x = 0xffffffffffdd76d5... Success: 0x64=’d’ score=2                                                                
Reading at malicious_x = 0xffffffffffdd76d6... Success: 0x73=’s’ score=2                                                                
Reading at malicious_x = 0xffffffffffdd76d7... Success: 0x20=’ ’ score=2                                                                
Reading at malicious_x = 0xffffffffffdd76d8... Success: 0x61=’a’ score=2                                                                
Reading at malicious_x = 0xffffffffffdd76d9... Success: 0x72=’r’ score=2                                                                
Reading at malicious_x = 0xffffffffffdd76da... Success: 0x65=’e’ score=2                                                                
Reading at malicious_x = 0xffffffffffdd76db... Success: 0x20=’ ’ score=2                                                                
Reading at malicious_x = 0xffffffffffdd76dc... Success: 0x53=’S’ score=2                                                                
Reading at malicious_x = 0xffffffffffdd76dd... Success: 0x71=’q’ score=2                                                                
Reading at malicious_x = 0xffffffffffdd76de... Success: 0x75=’u’ score=2                                                                
Reading at malicious_x = 0xffffffffffdd76df... Success: 0x65=’e’ score=2                                                                
Reading at malicious_x = 0xffffffffffdd76e0... Success: 0x61=’a’ score=2                                                                
Reading at malicious_x = 0xffffffffffdd76e1... Success: 0x6D=’m’ score=2                                                                
Reading at malicious_x = 0xffffffffffdd76e2... Success: 0x69=’i’ score=2                                                                
Reading at malicious_x = 0xffffffffffdd76e3... Success: 0x73=’s’ score=2                                                                
Reading at malicious_x = 0xffffffffffdd76e4... Success: 0x68=’h’ score=2                                                                
Reading at malicious_x = 0xffffffffffdd76e5... Success: 0x20=’ ’ score=2                                                                
Reading at malicious_x = 0xffffffffffdd76e6... Success: 0x50=’P’ score=9 (second best: 0x06 score=2)                                    
Reading at malicious_x = 0xffffffffffdd76e7... Success: 0x73=’s’ score=2                                                                
Reading at malicious_x = 0xffffffffffdd76e8... Success: 0x73=’s’ score=2                                                                
Reading at malicious_x = 0xffffffffffdd76e9... Success: 0x69=’i’ score=2                                                                
Reading at malicious_x = 0xffffffffffdd76ea... Success: 0x66=’f’ score=2                                                                
Reading at malicious_x = 0xffffffffffdd76eb... Success: 0x72=’r’ score=2                                                                
Reading at malicious_x = 0xffffffffffdd76ec... Success: 0x61=’a’ score=2                                                                
Reading at malicious_x = 0xffffffffffdd76ed... Success: 0x67=’g’ score=2                                                                
Reading at malicious_x = 0xffffffffffdd76ee... Success: 0x65=’e’ score=2                                                                
Reading at malicious_x = 0xffffffffffdd76ef... Success: 0x2E=’.’ score=2
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt;

&lt;h3 id=&quot;running-spectre-in-gem5&quot;&gt;Running Spectre in gem5&lt;/h3&gt;

&lt;p&gt;To find out if gem5’s out of order CPU implementation is vulnerable to
Spectre, we need to run the code in gem5. The simplest and fastest way
to do this is by running in gem5’s syscall-emulation (SE) mode. In SE
mode we won’t be modeling an OS or any user-mode to kernel-mode
interaction, but this okay for Spectre since this proof of concept code
is all in user-mode. If we were investigating Metldown, we would have to
use full-system (FS) mode since Meltdown specifically allows user-mode
processes to read data that should only be accessible in kernel mode.&lt;/p&gt;

&lt;p&gt;So, when running something in gem5, the first step is to create a Python
runscript since this is &lt;a href=&quot;http://learning.gem5.org/book/part1/simple_config.html&quot;&gt;the “interface” to
gem5&lt;/a&gt;. For this
example, what we need is a system with one CPU, an L1 cache, and memory.
For simplicity, I’m going to modify one of the existing script,
specifically the &lt;code class=&quot;highlighter-rouge&quot;&gt;two_level.py&lt;/code&gt; script from the &lt;a href=&quot;http://learning.gem5.org/&quot;&gt;Learning gem5
book&lt;/a&gt;.&lt;/p&gt;

&lt;p&gt;In the file &lt;code class=&quot;highlighter-rouge&quot;&gt;gem5/configs/learning_gem5/part1/two_level.py&lt;/code&gt;, I simply
changed the CPU from &lt;code class=&quot;highlighter-rouge&quot;&gt;TimingSimpleCPU()&lt;/code&gt; to
&lt;code class=&quot;highlighter-rouge&quot;&gt;DerivO3CPU(branchPred=LTAGE())&lt;/code&gt;. I also set the O3CPU to use the LTAGE
branch predictor instead of the default tournament branch predictor.
It’s important to use the LTAGE branch predictor as better branch
predictors actually make Spectre easier to exploit as discussed further
&lt;a href=&quot;#effects-of-branch-predictor&quot;&gt;below&lt;/a&gt;.&lt;/p&gt;

&lt;p&gt;Now, we simply need to build gem5 and run it.&lt;/p&gt;

&lt;div class=&quot;highlighter-rouge&quot;&gt;&lt;div class=&quot;highlight&quot;&gt;&lt;pre class=&quot;highlight&quot;&gt;&lt;code&gt;scons -j8 build/X86/gem5.opt

build/X86/gem5.opt configs/learning_gem5/part1/two_level.py spectre
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt;

&lt;p&gt;And, the output that I get is the following, just like above when I ran
the &lt;code class=&quot;highlighter-rouge&quot;&gt;spectre&lt;/code&gt; natively.&lt;/p&gt;

&lt;div class=&quot;highlighter-rouge&quot;&gt;&lt;div class=&quot;highlight&quot;&gt;&lt;pre class=&quot;highlight&quot;&gt;&lt;code&gt;gem5 Simulator System.  http://gem5.org                             
gem5 is copyrighted software; use the --copyright option for details.                                                                   

gem5 compiled May 10 2018 09:40:08                                  
gem5 started May 24 2018 11:21:16
gem5 executing on palisade, pid 27173                               
command line: build/X86/gem5.opt configs/learning_gem5/part1/two_level.py spectre                                                       

Global frequency set at 1000000000000 ticks per second              
warn: DRAM device capacity (8192 Mbytes) does not match the address range assigned (512 Mbytes)                                         
0: system.remote_gdb: listening for remote gdb on port 7000         
Beginning simulation!             
info: Entering event queue @ 0.  Starting simulation...             
warn: readlink() called on '/proc/self/exe' may yield unexpected results in various settings.                                           
      Returning '/home/jlp/Code/gem5/spectre-vis/spectre'           
info: Increasing stack size by one page.                            
warn: ignoring syscall access(...)                                  
Reading 40 bytes:                 tput cols
Reading at malicious_x = 0xffffffffffdd76c8... Success: 0x54=’T’ score=2                                                                
Reading at malicious_x = 0xffffffffffdd76c9... Success: 0x68=’h’ score=2                                                                
Reading at malicious_x = 0xffffffffffdd76ca... Success: 0x65=’e’ score=2                                                                
Reading at malicious_x = 0xffffffffffdd76cb... Success: 0x20=’ ’ score=2                                                                
Reading at malicious_x = 0xffffffffffdd76cc... Success: 0x4D=’M’ score=2                                                                
Reading at malicious_x = 0xffffffffffdd76cd... Success: 0x61=’a’ score=2                                                                
Reading at malicious_x = 0xffffffffffdd76ce... Success: 0x67=’g’ score=2                                                                
Reading at malicious_x = 0xffffffffffdd76cf... Success: 0x69=’i’ score=2                                                                
Reading at malicious_x = 0xffffffffffdd76d0... Success: 0x63=’c’ score=2                                                                
Reading at malicious_x = 0xffffffffffdd76d1... Success: 0x20=’ ’ score=2                                                                
Reading at malicious_x = 0xffffffffffdd76d2... Success: 0x57=’W’ score=2                                                                
Reading at malicious_x = 0xffffffffffdd76d3... Success: 0x6F=’o’ score=2                                                                
Reading at malicious_x = 0xffffffffffdd76d4... Success: 0x72=’r’ score=2                                                                
Reading at malicious_x = 0xffffffffffdd76d5... Success: 0x64=’d’ score=2                                                                
Reading at malicious_x = 0xffffffffffdd76d6... Success: 0x73=’s’ score=2                                                                
Reading at malicious_x = 0xffffffffffdd76d7... Success: 0x20=’ ’ score=2                                                                
Reading at malicious_x = 0xffffffffffdd76d8... Success: 0x61=’a’ score=2                                                                
Reading at malicious_x = 0xffffffffffdd76d9... Success: 0x72=’r’ score=2                                                                
Reading at malicious_x = 0xffffffffffdd76da... Success: 0x65=’e’ score=2                                                                
Reading at malicious_x = 0xffffffffffdd76db... Success: 0x20=’ ’ score=2                                                                
Reading at malicious_x = 0xffffffffffdd76dc... Success: 0x53=’S’ score=2                                                                
Reading at malicious_x = 0xffffffffffdd76dd... Success: 0x71=’q’ score=2                                                                
Reading at malicious_x = 0xffffffffffdd76de... Success: 0x75=’u’ score=2                                                                
Reading at malicious_x = 0xffffffffffdd76df... Success: 0x65=’e’ score=2                                                                
Reading at malicious_x = 0xffffffffffdd76e0... Success: 0x61=’a’ score=2                                                                
Reading at malicious_x = 0xffffffffffdd76e1... Success: 0x6D=’m’ score=2                                                                
Reading at malicious_x = 0xffffffffffdd76e2... Success: 0x69=’i’ score=2                                                                
Reading at malicious_x = 0xffffffffffdd76e3... Success: 0x73=’s’ score=2                                                                
Reading at malicious_x = 0xffffffffffdd76e4... Success: 0x68=’h’ score=2                                                                
Reading at malicious_x = 0xffffffffffdd76e5... Success: 0x20=’ ’ score=2                                                                
Reading at malicious_x = 0xffffffffffdd76e6... Success: 0x4F=’O’ score=2                                                                
Reading at malicious_x = 0xffffffffffdd76e7... Success: 0x73=’s’ score=2                                                                
Reading at malicious_x = 0xffffffffffdd76e8... Success: 0x73=’s’ score=2                                                                
Reading at malicious_x = 0xffffffffffdd76e9... Success: 0x69=’i’ score=2                                                                
Reading at malicious_x = 0xffffffffffdd76ea... Success: 0x66=’f’ score=2                                                                
Reading at malicious_x = 0xffffffffffdd76eb... Success: 0x72=’r’ score=2                                                                
Reading at malicious_x = 0xffffffffffdd76ec... Success: 0x61=’a’ score=2                                                                
Reading at malicious_x = 0xffffffffffdd76ed... Success: 0x67=’g’ score=2                                                                
Reading at malicious_x = 0xffffffffffdd76ee... Success: 0x65=’e’ score=2                                                                
Reading at malicious_x = 0xffffffffffdd76ef... Success: 0x2E=’.’ score=2                                                                
Exiting @ tick 113568969000 because exiting with last active thread context
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt;

&lt;h2 id=&quot;visualizing-the-out-of-order-pipeline&quot;&gt;Visualizing the out of order pipeline&lt;/h2&gt;

&lt;p&gt;To generate pipeline visualizations, we first need to generate a trace
file of all of the instructions executed by the out of order CPU. To
create this trace, we can use the &lt;code class=&quot;highlighter-rouge&quot;&gt;O3PipeView&lt;/code&gt; debug flag.&lt;/p&gt;

&lt;p&gt;Now, the trace for the O3 CPU can be &lt;em&gt;very&lt;/em&gt; large, up to many GBs. When
creating this trace, you need to be careful to create the smallest trace
possible. Also, it’s important to dump the trace to a file and not to
&lt;code class=&quot;highlighter-rouge&quot;&gt;stdout&lt;/code&gt;, which is the default when using debug flags. You can redirect
the trace to a file by using the &lt;code class=&quot;highlighter-rouge&quot;&gt;--debug-file&lt;/code&gt; option to gem5.&lt;/p&gt;

&lt;p&gt;To create the trace file, I used the following methodology:&lt;/p&gt;

&lt;ol&gt;
  &lt;li&gt;Start running spectre in gem5, then hit ctrl-c after the first
couple of letters. At this point, I wrote down the tick which gem5
exited (13062347000 for me).&lt;/li&gt;
  &lt;li&gt;Run gem5 with the debug flag &lt;code class=&quot;highlighter-rouge&quot;&gt;O3PipeView&lt;/code&gt; enabled.&lt;/li&gt;
  &lt;li&gt;Watch the output and kill gem5 with ctrl-c after two more letters
appeared than in step 1.&lt;/li&gt;
&lt;/ol&gt;

&lt;p&gt;To generate the trace, I ran the following command. Note: you may have a
different value for when to start the debugging trace. Also note: when
producing the trace gem5 will run &lt;em&gt;much&lt;/em&gt; slower.&lt;/p&gt;

&lt;div class=&quot;highlighter-rouge&quot;&gt;&lt;div class=&quot;highlight&quot;&gt;&lt;pre class=&quot;highlight&quot;&gt;&lt;code&gt;build/X86/gem5.opt --debug-flags=O3PipeView --debug-file=pipeview.txt --debug-start=13062347000 configs/learning_gem5/part1/two_level.py spectre
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt;

&lt;p&gt;My tracefile (&lt;code class=&quot;highlighter-rouge&quot;&gt;pipeview.txt&lt;/code&gt;) was 600 MB for catching just two letters
in the output.&lt;/p&gt;

&lt;p&gt;Now, we can process this file to generate the visualization with a
script: &lt;code class=&quot;highlighter-rouge&quot;&gt;util/o3-pipeview.py&lt;/code&gt;. This script requires the path to the file
that contains the output generated with the &lt;code class=&quot;highlighter-rouge&quot;&gt;O3PipeView&lt;/code&gt; debug flag.
Above, we put the output into the file &lt;code class=&quot;highlighter-rouge&quot;&gt;pipeview.txt&lt;/code&gt;, and this file was
created in the default output directory of gem5 (&lt;code class=&quot;highlighter-rouge&quot;&gt;m5out/&lt;/code&gt;).&lt;/p&gt;

&lt;div class=&quot;highlighter-rouge&quot;&gt;&lt;div class=&quot;highlight&quot;&gt;&lt;pre class=&quot;highlight&quot;&gt;&lt;code&gt;util/o3-pipeview.py --store_completions m5out/pipeview.txt --color -w 150
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt;

&lt;p&gt;In the above command, I wanted to see when the stores completed
(&lt;code class=&quot;highlighter-rouge&quot;&gt;--store_completions&lt;/code&gt;) and specified to use color (&lt;code class=&quot;highlighter-rouge&quot;&gt;--color&lt;/code&gt;) in the
output and use a width of 150 characters (&lt;code class=&quot;highlighter-rouge&quot;&gt;-w 150&lt;/code&gt;). Processing a large
file like this one of 600 MB may take a few minutes. The output will be
in a file called &lt;code class=&quot;highlighter-rouge&quot;&gt;o3-pipeview.out&lt;/code&gt; in the current working directory.&lt;/p&gt;

&lt;p&gt;You can view this file with &lt;code class=&quot;highlighter-rouge&quot;&gt;less -r o3-pipeview.out&lt;/code&gt;. You may want to
use the &lt;code class=&quot;highlighter-rouge&quot;&gt;-S&lt;/code&gt; option with less if your terminal is less than 150
characters wide (or whatever width value you used). Below is a
screenshot of the top of my trace.&lt;/p&gt;

&lt;h3 id=&quot;understanding-the-o3-pipeline-viewer&quot;&gt;Understanding the O3 pipeline viewer&lt;/h3&gt;

&lt;p&gt;&lt;img src=&quot;/assets/img/o3-example-annotated.png&quot; alt=&quot;o3 pipeline view example&quot; /&gt;&lt;/p&gt;

&lt;p&gt;The above image details how to interpret the output from the pipeline
viewer. Each &lt;code class=&quot;highlighter-rouge&quot;&gt;.&lt;/code&gt; or &lt;code class=&quot;highlighter-rouge&quot;&gt;=&lt;/code&gt; represents one cycle of time, which moves from
left to right. The “tick” column shows the tick of the leftmost &lt;code class=&quot;highlighter-rouge&quot;&gt;.&lt;/code&gt; or
&lt;code class=&quot;highlighter-rouge&quot;&gt;=&lt;/code&gt;. &lt;code class=&quot;highlighter-rouge&quot;&gt;=&lt;/code&gt; is used to mark the instructions that were later squashed. The
address of the instruction (and the micro-op number) as well as the
disassembly is also shown. The sequence number can be ignored as it is
always monotonically increasing and is the total order of every dynamic
instruction. Finally, each stage of the O3 pipeline is shown with a
different letter and color.&lt;/p&gt;

&lt;h2 id=&quot;digging-deeper-into-spectre&quot;&gt;Digging deeper into Spectre&lt;/h2&gt;

&lt;p&gt;First, let’s examine the actual instructions that are executed during
the Spectre attack. The vulnerability is in the &lt;code class=&quot;highlighter-rouge&quot;&gt;victim_function&lt;/code&gt; in
&lt;code class=&quot;highlighter-rouge&quot;&gt;spectre.c&lt;/code&gt;.&lt;/p&gt;

&lt;div class=&quot;highlighter-rouge&quot;&gt;&lt;div class=&quot;highlight&quot;&gt;&lt;pre class=&quot;highlight&quot;&gt;&lt;code&gt;void victim_function(size_t x) {
  if (x &amp;lt; array1_size) {
    temp &amp;amp;= array2[array1[x] * 512];
  }
}
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt;

&lt;p&gt;When this is compiled and then dumped with &lt;code class=&quot;highlighter-rouge&quot;&gt;objdump&lt;/code&gt;, we get the
following instructions that will be executed. Your code my be slightly
different, especially the exact addresses of each instruction, depending
on the version of the compiler and other system-specific configurations.&lt;/p&gt;

&lt;div class=&quot;highlighter-rouge&quot;&gt;&lt;div class=&quot;highlight&quot;&gt;&lt;pre class=&quot;highlight&quot;&gt;&lt;code&gt;# NOTE: the movzbl below is MOVZX_B_R_M in gem5.
# it is implemented with the following microcode.
#    ld t1, seg, sib, disp, dataSize=1
#    zexti reg, t1, 7
#
000000000040105e &amp;lt;victim_function&amp;gt;:
  40105e:   55                      push   %rbp
  40105f:   48 89 e5                mov    %rsp,%rbp
  401062:   48 89 7d f8             mov    %rdi,-0x8(%rbp)
  401066:   8b 05 14 f0 2b 00       mov    0x2bf014(%rip),%eax # 6c0080 &amp;lt;array1_size&amp;gt; load array1_size (first time is always a miss)
  40106c:   89 c0                   mov    %eax,%eax
  40106e:   48 3b 45 f8             cmp    -0x8(%rbp),%rax  # if (x &amp;lt; array1_size) rax is array1_size, -8(%rbp) is x
  401072:   76 2b                   jbe    40109f &amp;lt;victim_function+0x41&amp;gt; # if (x &amp;lt; array1_size)
  401074:   48 8b 45 f8             mov    -0x8(%rbp),%rax # load x from the stack into rax
  401078:   48 05 a0 00 6c 00       add    $0x6c00a0,%rax  # calculate array1 offset (x+array1)
  40107e:   0f b6 00                movzbl (%rax),%eax # load array1[x]
  401081:   0f b6 c0                movzbl %al,%eax    # zero extend to 32 bits
  401084:   c1 e0 09                shl    $0x9,%eax   # multiply by 512
  401087:   48 98                   cltq               # sign-extend eax
  401089:   0f b6 90 80 1d 6c 00    movzbl 0x6c1d80(%rax),%edx  # load array2[array1[x]*512] **** This is the magic!
  401090:   0f b6 05 e9 0c 2e 00    movzbl 0x2e0ce9(%rip),%eax        # 6e1d80 &amp;lt;temp&amp;gt; Load temp.
  401097:   21 d0                   and    %edx,%eax
  401099:   88 05 e1 0c 2e 00       mov    %al,0x2e0ce1(%rip)        # 6e1d80 &amp;lt;temp&amp;gt;
  40109f:   5d                      pop    %rbp
  4010a0:   c3                      retq
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt;

&lt;p&gt;Now, we can search for the instruction that we care about in the trace.
In this case, we want to find a time where the &lt;code class=&quot;highlighter-rouge&quot;&gt;movzbl&lt;/code&gt; at address
&lt;code class=&quot;highlighter-rouge&quot;&gt;0x401089&lt;/code&gt; is executed speculatively. When searching through the
pipeline viewer (use &lt;code class=&quot;highlighter-rouge&quot;&gt;\&lt;/code&gt; in less), we’re looking for a time where the
load completes for the instruction at &lt;code class=&quot;highlighter-rouge&quot;&gt;0x401089&lt;/code&gt; and it is later
squashed (surrounded by &lt;code class=&quot;highlighter-rouge&quot;&gt;=&lt;/code&gt;). An example is shown below.&lt;/p&gt;

&lt;p&gt;&lt;img src=&quot;/assets/img/o3-spectre-annotated.png&quot; alt=&quot;annotated O3 pipeline view of
spectre&quot; /&gt;&lt;/p&gt;

&lt;p&gt;The image above is from my presentation at &lt;a href=&quot;http://caslab.csl.yale.edu/workshops/hasp2018/&quot;&gt;Hardware and Architectural
Support for Security and Privacy (HASP)
2018&lt;/a&gt;.&lt;/p&gt;

&lt;p&gt;What we see in this image is that the instruction at &lt;code class=&quot;highlighter-rouge&quot;&gt;0x401066&lt;/code&gt; causes a
cache miss (there is a long time between when the load is issued and the
data is returned from memory). Since the load of &lt;code class=&quot;highlighter-rouge&quot;&gt;array1_size&lt;/code&gt; was a
cache miss, the jump at &lt;code class=&quot;highlighter-rouge&quot;&gt;0x401072&lt;/code&gt; is speculated to be &lt;em&gt;not&lt;/em&gt; taken
(incorrectly). This causes the following instructions to be executed
speculatively, and, eventually, squashed.&lt;/p&gt;

&lt;p&gt;The key thing in this trace that &lt;em&gt;is&lt;/em&gt; the Spectre vulnerability is that
the load for the instruction at &lt;code class=&quot;highlighter-rouge&quot;&gt;0x40107e&lt;/code&gt;, which loads secret data
happens during the mis-speculated instructions. Then, this data is
loaded into the registers and operated on (instruction &lt;code class=&quot;highlighter-rouge&quot;&gt;0x401084&lt;/code&gt;).
Finally, the load at address &lt;code class=&quot;highlighter-rouge&quot;&gt;0x401089&lt;/code&gt; is executed and loads the value
from memory &lt;em&gt;that is dependent on the secret data loaded previously&lt;/em&gt;.
Thus, we can later probe the cache to retrieve the secret data.&lt;/p&gt;

&lt;h3 id=&quot;effects-of-compilers&quot;&gt;Effects of compilers&lt;/h3&gt;

&lt;p&gt;As previously mentioned, the specific compiler version and compiler
options have a significant effect on the attack. Below are two traces,
one from GCC 7.2 and one from clang 4.0.&lt;/p&gt;

&lt;h4 id=&quot;gcc-72&quot;&gt;GCC 7.2&lt;/h4&gt;

&lt;div class=&quot;highlighter-rouge&quot;&gt;&lt;div class=&quot;highlight&quot;&gt;&lt;pre class=&quot;highlight&quot;&gt;&lt;code&gt;void victim_function(size_t x) {                                    
  400b2d:       55                      push   %rbp                 
  400b2e:       48 89 e5                mov    %rsp,%rbp            
  400b31:       48 89 7d f8             mov    %rdi,-0x8(%rbp)      
  if (x &amp;lt; array1_size) {                                            
  400b35:       8b 05 c5 c5 2c 00       mov    0x2cc5c5(%rip),%eax        # 6cd100 &amp;lt;array1_size&amp;gt;                                        
  400b3b:       89 c0                   mov    %eax,%eax            
  400b3d:       48 39 45 f8             cmp    %rax,-0x8(%rbp)      
  400b41:       73 34                   jae    400b77 &amp;lt;victim_function+0x4a&amp;gt;                                                            
    temp &amp;amp;= array2[array1[x] * 512];                                
  400b43:       48 8d 15 d6 c5 2c 00    lea    0x2cc5d6(%rip),%rdx        # 6cd120 &amp;lt;array1&amp;gt;                                             
  400b4a:       48 8b 45 f8             mov    -0x8(%rbp),%rax      
  400b4e:       48 01 d0                add    %rdx,%rax            
  400b51:       0f b6 00                movzbl (%rax),%eax          
  400b54:       0f b6 c0                movzbl %al,%eax             
  400b57:       c1 e0 09                shl    $0x9,%eax            
  400b5a:       48 63 d0                movslq %eax,%rdx            
  400b5d:       48 8d 05 9c f6 2c 00    lea    0x2cf69c(%rip),%rax        # 6d0200 &amp;lt;array2&amp;gt;                                             
  400b64:       0f b6 14 02             movzbl (%rdx,%rax,1),%edx   
  400b68:       0f b6 05 91 e1 2c 00    movzbl 0x2ce191(%rip),%eax        # 6ced00 &amp;lt;temp&amp;gt;                                               
  400b6f:       21 d0                   and    %edx,%eax            
  400b71:       88 05 89 e1 2c 00       mov    %al,0x2ce189(%rip)        # 6ced00 &amp;lt;temp&amp;gt;                                                
  }                                                                 
}                                                                   
  400b77:       90                      nop                         
  400b78:       5d                      pop    %rbp                 
  400b79:       c3                      retq
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt;

&lt;iframe height=&quot;500&quot; src=&quot;/assets/img/gcc72-static-tage.html&quot; frameborder=&quot;0&quot;&gt;
&lt;/iframe&gt;
&lt;p&gt;However, clang generates the following code.&lt;/p&gt;

&lt;div class=&quot;highlighter-rouge&quot;&gt;&lt;div class=&quot;highlight&quot;&gt;&lt;pre class=&quot;highlight&quot;&gt;&lt;code&gt;void victim_function(size_t x) {
  400ac0:       55                      push   %rbp
  400ac1:       48 89 e5                mov    %rsp,%rbp
  400ac4:       48 89 7d f8             mov    %rdi,-0x8(%rbp)
  if (x &amp;lt; array1_size) {
  400ac8:       48 8b 7d f8             mov    -0x8(%rbp),%rdi
  400acc:       8b 04 25 90 c0 6c 00    mov    0x6cc090,%eax
  400ad3:       89 c1                   mov    %eax,%ecx
  400ad5:       48 39 cf                cmp    %rcx,%rdi
  400ad8:       0f 83 2f 00 00 00       jae    400b0d &amp;lt;victim_function+0x4d&amp;gt;
    temp &amp;amp;= array2[array1[x] * 512];
  400ade:       48 8b 45 f8             mov    -0x8(%rbp),%rax
  400ae2:       0f b6 0c 05 a0 c0 6c    movzbl 0x6cc0a0(,%rax,1),%ecx
  400ae9:       00
  400aea:       c1 e1 09                shl    $0x9,%ecx
  400aed:       48 63 c1                movslq %ecx,%rax
  400af0:       0f b6 0c 05 40 f2 6c    movzbl 0x6cf240(,%rax,1),%ecx
  400af7:       00
  400af8:       0f b6 14 25 50 dc 6c    movzbl 0x6cdc50,%edx
  400aff:       00
  400b00:       21 ca                   and    %ecx,%edx
  400b02:       40 88 d6                mov    %dl,%sil
  400b05:       40 88 34 25 50 dc 6c    mov    %sil,0x6cdc50
  400b0c:       00
  }
}
  400b0d:       5d                      pop    %rbp
  400b0e:       c3                      retq
  400b0f:       90                      nop
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt;

&lt;iframe height=&quot;500&quot; src=&quot;/assets/img/clang-static-tage.html&quot; frameborder=&quot;0&quot;&gt;
&lt;/iframe&gt;
&lt;p&gt;Interestingly, the clang-compiled &lt;code class=&quot;highlighter-rouge&quot;&gt;spectre&lt;/code&gt; binary is not able to read
the secret data! (At least not in gem5. It is able to read the secret
data on my native machine.)&lt;/p&gt;

&lt;p&gt;We can look into the two traces to see the difference between the clang
version and the GCC version.&lt;/p&gt;

&lt;p&gt;The main difference is that in the clang version, the load generated by
the instruction at &lt;code class=&quot;highlighter-rouge&quot;&gt;0x400af0&lt;/code&gt; never completes (and thus, must not have
been issued to the memory system).&lt;/p&gt;

&lt;p&gt;I’m not sure the exact cause of this difference. It could be that the
instruction uses a different addressing mode
(&lt;code class=&quot;highlighter-rouge&quot;&gt;movzbl 0x6cf240(,%rax,1),%ecx&lt;/code&gt; in clang vs &lt;code class=&quot;highlighter-rouge&quot;&gt;movzbl (%rdx,%rax,1),%edx&lt;/code&gt;
in GCC). If you have ideas, please leave a comment!&lt;/p&gt;

&lt;p&gt;Either way, minor differences in the code generated can have large
impacts on the speculative execution!&lt;/p&gt;

&lt;h3 id=&quot;effects-of-branch-predictor&quot;&gt;Effects of branch predictor&lt;/h3&gt;

&lt;p&gt;When I was first playing around with Spectre and gem5, I ran into a
problem where I could only &lt;em&gt;sometimes&lt;/em&gt; get Spectre to “work” with the
out of order CPU. After significant digging, I found that the branch
predictor chosen makes a big difference to how quickly the vulnerability
happens. The trace below (with the same code as GCC 4.8 above) shows
what happens when using the tournament branch predictor.&lt;/p&gt;

&lt;iframe height=&quot;500&quot; src=&quot;/assets/img/gcc-static-tourn.html&quot; frameborder=&quot;0&quot;&gt;
&lt;/iframe&gt;
&lt;p&gt;Here, we see that the original branch misprediction comes much earlier
than the jump instruction in &lt;code class=&quot;highlighter-rouge&quot;&gt;victim_function&lt;/code&gt; that is at address
&lt;code class=&quot;highlighter-rouge&quot;&gt;0x401072&lt;/code&gt;. Thus, by the time the load instructions in &lt;code class=&quot;highlighter-rouge&quot;&gt;victim_function&lt;/code&gt;
are executed, the ROB and load-store queue resources have been taken by
other instructions and the rogue loads are not issued to memory. There
are still a few times that the two loads are executed speculatively, but
it is much more rare than with the TAGE predictor. When using the TAGE
branch predictor, only the exact branch that the attacker wants to
mispredict is mispredicted.&lt;/p&gt;

&lt;p&gt;This interestingly shows that a “smarter” system is actually &lt;em&gt;more&lt;/em&gt;
vulnerable to speculation-based attacks!&lt;/p&gt;
</description>
        <pubDate>Fri, 01 Jun 2018 00:00:00 -0700</pubDate>
        <link>http://localhost:4000/2018/06/01/gem5-spectre.html</link>
        <guid isPermaLink="true">http://localhost:4000/2018/06/01/gem5-spectre.html</guid>
        
        
      </item>
    
      <item>
        <title>Setting up gem5 full system</title>
        <description>&lt;p&gt;This is partially a followup to &lt;a href=&quot;http://www.lowepower.com/jason/creating-disk-images-for-gem5.html&quot;&gt;Creating disk images for
gem5&lt;/a&gt;
and partially how to setup x86 full system for gem5. In this post, I’ll
discuss how to create a disk image from scratch and start using it with
gem5.&lt;/p&gt;

&lt;p&gt;It is important for computer architecture research to use the most
up-to-date software on the systems we are simulating. Too much computer
architecture research reports results using kernels from 5+ years ago or
ancient system software Hopefully, this post will help others be able to
keep up with the ever-changing system software. This way, researchers
can use up-to-date versions of Linux and easily update their kernels.&lt;/p&gt;

&lt;p&gt;This post takes a different approach than &lt;a href=&quot;http://www.lowepower.com/jason/creating-disk-images-for-gem5.html&quot;&gt;Creating disk images for
gem5&lt;/a&gt;.
Instead of using the gem5 tools, this post uses qemu to create, edit,
and set up the disk for gem5 usage.&lt;/p&gt;

&lt;p&gt;This post assumes that you have installed qemu on your system. In
Ubuntu, this can be done with&lt;/p&gt;

&lt;div class=&quot;highlighter-rouge&quot;&gt;&lt;div class=&quot;highlight&quot;&gt;&lt;pre class=&quot;highlight&quot;&gt;&lt;code&gt;sudo apt-get install qemu-kvm libvirt-bin ubuntu-vm-builder bridge-utils
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt;

&lt;p&gt;I also assume you have downloaded and built gem5. All of the full system
examples use the simple full system scripts that are covered in
&lt;a href=&quot;http://learning.gem5.org/book/part3/index.html&quot;&gt;Learning gem5&lt;/a&gt;.&lt;/p&gt;

&lt;h2 id=&quot;step-1-create-an-empty-disk&quot;&gt;Step 1: Create an empty disk&lt;/h2&gt;

&lt;p&gt;Using the qemu disk tools, create a blank raw disk image. In this case,
I chose to create a disk named “ubuntu-test.img” that is 8GB.&lt;/p&gt;

&lt;div class=&quot;highlighter-rouge&quot;&gt;&lt;div class=&quot;highlight&quot;&gt;&lt;pre class=&quot;highlight&quot;&gt;&lt;code&gt;qemu-img create ubuntu-test.img 8G
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt;

&lt;h2 id=&quot;step-2-install-ubuntu-with-qemu&quot;&gt;Step 2: Install ubuntu with qemu&lt;/h2&gt;

&lt;p&gt;Now that we have a blank disk, we are going to use qemu to install
Ubuntu on the disk. I would encourage you to use the server version of
Ubuntu since gem5 does not have great support for displays. Thus, the
desktop environment isn’t very useful.&lt;/p&gt;

&lt;p&gt;First, you need to download the installation CD image from the &lt;a href=&quot;https://www.ubuntu.com/download/server&quot;&gt;Ubuntu
website&lt;/a&gt;.&lt;/p&gt;

&lt;p&gt;Next, use qemu to boot off of the CD image, and set the disk in the
system to be the blank disk you created above. Ubuntu needs at least 1GB
of memory to install correctly, so be sure to configure qemu to use at
least 1GB memory.&lt;/p&gt;

&lt;div class=&quot;highlighter-rouge&quot;&gt;&lt;div class=&quot;highlight&quot;&gt;&lt;pre class=&quot;highlight&quot;&gt;&lt;code&gt;qemu-system-x86_64 -hda ../gem5-fs-testing/ubuntu-test.img -cdrom ubuntu-16.04.1-server-amd64.iso -m 1024 -enable-kvm -boot d
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt;

&lt;p&gt;With this, you can simply follow the on-screen directions to install
Ubuntu to the disk image. The only gotcha in the installation is that
gem5’s IDE drivers don’t seem to play nicely with logical paritions.
Thus, during the Ubuntu install, be sure to manually partition the disk
and remove any logical partitions. You don’t need any swap space on the
disk anyway, unless you’re doing something specifically with swap space.&lt;/p&gt;

&lt;h2 id=&quot;step-3-boot-up-and-install-needed-software&quot;&gt;Step 3: Boot up and install needed software&lt;/h2&gt;

&lt;p&gt;Once you have installed Ubuntu on the disk, quit qemu and remove the
&lt;code class=&quot;highlighter-rouge&quot;&gt;-boot d&lt;/code&gt; option so that you are not booting off of the CD anymore. Now,
you can again boot off of the main disk image you have installed Ubuntu
on.&lt;/p&gt;

&lt;p&gt;Since we’re using qemu, you should have a network connection (although
&lt;a href=&quot;http://wiki.qemu.org/Documentation/Networking#User_Networking_.28SLIRP.29&quot;&gt;ping won’t
work&lt;/a&gt;).
When booting in qemu, you can just use &lt;code class=&quot;highlighter-rouge&quot;&gt;sudo apt-get install&lt;/code&gt; and
install any software you need on your disk.&lt;/p&gt;

&lt;div class=&quot;highlighter-rouge&quot;&gt;&lt;div class=&quot;highlight&quot;&gt;&lt;pre class=&quot;highlight&quot;&gt;&lt;code&gt;qemu-system-x86_64 -hda ../gem5-fs-testing/ubuntu-test.img -cdrom ubuntu-16.04.1-server-amd64.iso -m 1024 -enable-kvm
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt;

&lt;h2 id=&quot;step-4-build-a-kernel&quot;&gt;Step 4: Build a kernel&lt;/h2&gt;

&lt;p&gt;Next, you need to build a Linux kernel. Unfortunately, the
out-of-the-box Ubuntu kernel doesn’t play well with gem5. See the
error below_.&lt;/p&gt;

&lt;p&gt;First, you need to download latest kernel from
&lt;a href=&quot;https://www.kernel.org/&quot;&gt;kernel.org&lt;/a&gt;. Then, to build the kernel, you
are going to want to start with a known-good config file.
The config file that I’m used for kernel version 4.8.13 can be
downloaded &lt;a href=&quot;{filename}files/config&quot;&gt;here&lt;/a&gt;. Then, you need to move the
good config to &lt;code class=&quot;highlighter-rouge&quot;&gt;.config&lt;/code&gt; and the run &lt;code class=&quot;highlighter-rouge&quot;&gt;make oldconfig&lt;/code&gt; which starts the
kernel configuration process with an existing config file.&lt;/p&gt;

&lt;div class=&quot;highlighter-rouge&quot;&gt;&lt;div class=&quot;highlight&quot;&gt;&lt;pre class=&quot;highlight&quot;&gt;&lt;code&gt;mv &amp;lt;good config&amp;gt; .config
make oldconfig
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt;

&lt;p&gt;At this point you can select any extra drivers you want to build into
the kernel. Note: You cannot use any kernel modules unless you are
planning on copying the modules onto the guest disk at the correct
location. All drivers must be built into the kernel binary.&lt;/p&gt;

&lt;p&gt;It may be possible to use modules by compiling the binary on the guest
disk via qemu, but I have not tested this.&lt;/p&gt;

&lt;p&gt;Finally, you need to build the kernel.&lt;/p&gt;

&lt;div class=&quot;highlighter-rouge&quot;&gt;&lt;div class=&quot;highlight&quot;&gt;&lt;pre class=&quot;highlight&quot;&gt;&lt;code&gt;make -j5
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt;

&lt;h2 id=&quot;step-5-update-init-script&quot;&gt;Step 5: Update init script&lt;/h2&gt;

&lt;p&gt;By default, gem5 expects a modified init script which loads a script off
of the host to execute in the guest. To use this feature, you need to
follow the steps below.&lt;/p&gt;

&lt;p&gt;Alternatively, you can install the precompiled binaries for x86 found on
my website: From qemu, you can run the following, which completes the
above steps for you.&lt;/p&gt;

&lt;div class=&quot;highlighter-rouge&quot;&gt;&lt;div class=&quot;highlight&quot;&gt;&lt;pre class=&quot;highlight&quot;&gt;&lt;code&gt;wget http://cs.wisc.edu/~powerjg/files/gem5-guest-tools-x86.tgz
tar xzvf gem5-guest-tools-x86.tgz
cd gem5-guest-tools/
sudo ./install
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt;

&lt;p&gt;Now, you can use the &lt;code class=&quot;highlighter-rouge&quot;&gt;system.readfile&lt;/code&gt; parameter in your Python config
scripts. This file will automatically be loaded (by the &lt;code class=&quot;highlighter-rouge&quot;&gt;gem5init&lt;/code&gt;
script) and executed.&lt;/p&gt;

&lt;h3 id=&quot;manually-installing-the-gem5-init-script&quot;&gt;Manually installing the gem5 init script&lt;/h3&gt;

&lt;p&gt;First, build the m5 binary on the host.&lt;/p&gt;

&lt;div class=&quot;highlighter-rouge&quot;&gt;&lt;div class=&quot;highlight&quot;&gt;&lt;pre class=&quot;highlight&quot;&gt;&lt;code&gt;cd util/m5
make -f Makefile.x86
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt;

&lt;p&gt;Then, copy this binary to the guest and put it in &lt;code class=&quot;highlighter-rouge&quot;&gt;/sbin&lt;/code&gt;. Also, create
a link from &lt;code class=&quot;highlighter-rouge&quot;&gt;/sbin/gem5&lt;/code&gt;.&lt;/p&gt;

&lt;p&gt;Then, to get the init script to execute when gem5 boots, create file
/lib/systemd/system/gem5.service with the following:&lt;/p&gt;

&lt;div class=&quot;highlighter-rouge&quot;&gt;&lt;div class=&quot;highlight&quot;&gt;&lt;pre class=&quot;highlight&quot;&gt;&lt;code&gt;[Unit]
Description=gem5 init script
Documentation=http://gem5.org
After=getty.target

[Service]
Type=idle
ExecStart=/sbin/gem5init
StandardOutput=tty
StandardInput=tty-force
StandardError=tty

[Install]
WantedBy=default.target
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt;

&lt;p&gt;Enable the gem5 service and disable the ttyS0 service.&lt;/p&gt;

&lt;div class=&quot;highlighter-rouge&quot;&gt;&lt;div class=&quot;highlight&quot;&gt;&lt;pre class=&quot;highlight&quot;&gt;&lt;code&gt;systemctl enable gem5.service
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt;

&lt;p&gt;Finally, create the init script that is executed by the service. In
&lt;code class=&quot;highlighter-rouge&quot;&gt;/sbin/gem5init&lt;/code&gt;:&lt;/p&gt;

&lt;div class=&quot;highlighter-rouge&quot;&gt;&lt;div class=&quot;highlight&quot;&gt;&lt;pre class=&quot;highlight&quot;&gt;&lt;code&gt;&lt;span class=&quot;c&quot;&gt;#!/bin/bash -&lt;/span&gt;

&lt;span class=&quot;nv&quot;&gt;CPU&lt;/span&gt;&lt;span class=&quot;o&quot;&gt;=&lt;/span&gt;&lt;span class=&quot;sb&quot;&gt;`&lt;/span&gt;&lt;span class=&quot;nb&quot;&gt;cat&lt;/span&gt; /proc/cpuinfo | &lt;span class=&quot;nb&quot;&gt;grep &lt;/span&gt;vendor_id | head &lt;span class=&quot;nt&quot;&gt;-n&lt;/span&gt; 1 | cut &lt;span class=&quot;nt&quot;&gt;-d&lt;/span&gt; &lt;span class=&quot;s1&quot;&gt;' '&lt;/span&gt; &lt;span class=&quot;nt&quot;&gt;-f2-&lt;/span&gt;&lt;span class=&quot;sb&quot;&gt;`&lt;/span&gt;
&lt;span class=&quot;nb&quot;&gt;echo&lt;/span&gt; &lt;span class=&quot;s2&quot;&gt;&quot;Got CPU type: &lt;/span&gt;&lt;span class=&quot;nv&quot;&gt;$CPU&lt;/span&gt;&lt;span class=&quot;s2&quot;&gt;&quot;&lt;/span&gt;

&lt;span class=&quot;k&quot;&gt;if&lt;/span&gt; &lt;span class=&quot;o&quot;&gt;[&lt;/span&gt; &lt;span class=&quot;s2&quot;&gt;&quot;&lt;/span&gt;&lt;span class=&quot;nv&quot;&gt;$CPU&lt;/span&gt;&lt;span class=&quot;s2&quot;&gt;&quot;&lt;/span&gt; &lt;span class=&quot;o&quot;&gt;!=&lt;/span&gt; &lt;span class=&quot;s2&quot;&gt;&quot;M5 Simulator&quot;&lt;/span&gt; &lt;span class=&quot;o&quot;&gt;]&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;;&lt;/span&gt;
&lt;span class=&quot;k&quot;&gt;then
    &lt;/span&gt;&lt;span class=&quot;nb&quot;&gt;echo&lt;/span&gt; &lt;span class=&quot;s2&quot;&gt;&quot;Not in gem5. Not loading script&quot;&lt;/span&gt;
    &lt;span class=&quot;nb&quot;&gt;exit &lt;/span&gt;0
&lt;span class=&quot;k&quot;&gt;fi&lt;/span&gt;

&lt;span class=&quot;c&quot;&gt;# Try to read in the script from the host system&lt;/span&gt;
/sbin/m5 readfile &lt;span class=&quot;o&quot;&gt;&amp;gt;&lt;/span&gt; /tmp/script
chmod 755 /tmp/script
&lt;span class=&quot;k&quot;&gt;if&lt;/span&gt; &lt;span class=&quot;o&quot;&gt;[&lt;/span&gt; &lt;span class=&quot;nt&quot;&gt;-s&lt;/span&gt; /tmp/script &lt;span class=&quot;o&quot;&gt;]&lt;/span&gt;
&lt;span class=&quot;k&quot;&gt;then&lt;/span&gt;
    &lt;span class=&quot;c&quot;&gt;# If there is a script, execute the script and then exit the simulation&lt;/span&gt;
    su root &lt;span class=&quot;nt&quot;&gt;-c&lt;/span&gt; &lt;span class=&quot;s1&quot;&gt;'/tmp/script'&lt;/span&gt; &lt;span class=&quot;c&quot;&gt;# gives script full privileges as root user in multi-user mode&lt;/span&gt;
    sync
    sleep 10
    /sbin/m5 &lt;span class=&quot;nb&quot;&gt;exit
&lt;/span&gt;&lt;span class=&quot;k&quot;&gt;fi
&lt;/span&gt;&lt;span class=&quot;nb&quot;&gt;echo&lt;/span&gt; &lt;span class=&quot;s2&quot;&gt;&quot;No script found&quot;&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt;

&lt;h2 id=&quot;problems-and-some-solutions&quot;&gt;Problems and (some) solutions&lt;/h2&gt;

&lt;h3 id=&quot;failed-to-early-mount-api-filesystems&quot;&gt;Failed to early mount API filesystems&lt;/h3&gt;

&lt;div class=&quot;highlighter-rouge&quot;&gt;&lt;div class=&quot;highlight&quot;&gt;&lt;pre class=&quot;highlight&quot;&gt;&lt;code&gt;Write protecting the kernel read-only data: 8192k
Freeing unused kernel memory: 1956K (ffff880001417000 - ffff880001600000)
Freeing unused kernel memory: 456K (ffff88000178e000 - ffff880001800000)
[!!!!!!] Failed to early mount API filesystems, freezing.
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt;

&lt;p&gt;Solutions tried: Enable cgroups in the kernel. I think. Nope! I think
this is the same as the problem below mount-problem_.&lt;/p&gt;

&lt;h3 id=&quot;cant-mount-dev&quot;&gt;Can’t mount /dev&lt;/h3&gt;

&lt;div class=&quot;highlighter-rouge&quot;&gt;&lt;div class=&quot;highlight&quot;&gt;&lt;pre class=&quot;highlight&quot;&gt;&lt;code&gt;Failed to mount devtmpfs at /dev: No such device
Freezing execution.
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt;

&lt;p&gt;Something like the above (this was taken from arch linux boot). The
problem is that that the right devfs is not compiled into the kernel.
You need to make sure that devtmpfs is enabled.&lt;/p&gt;

&lt;h3 id=&quot;panic-kvm-unexpected-exit-exit_reason-8&quot;&gt;panic: KVM: Unexpected exit (exit_reason: 8)&lt;/h3&gt;

&lt;p&gt;Exit reason 8 is “shutdown”. See
&lt;a href=&quot;http://lxr.free-electrons.com/source/include/uapi/linux/kvm.h#L188&quot;&gt;http://lxr.free-electrons.com/source/include/uapi/linux/kvm.h#L188&lt;/a&gt;.
This seems to happen when there is a triple fault:
&lt;a href=&quot;http://lxr.free-electrons.com/source/arch/x86/kvm/x86.c#L6498&quot;&gt;http://lxr.free-electrons.com/source/arch/x86/kvm/x86.c#L6498&lt;/a&gt;&lt;/p&gt;

&lt;p&gt;I get this error every time I try to boot the unmodified Ubuntu kernel.
I don’t know how to solve this problem. Instead of trying to solve the
problem, I used a different config file for “oldconfig” when I compiled
the kernel from scratch.&lt;/p&gt;

&lt;h3 id=&quot;slow-boot&quot;&gt;Slow boot&lt;/h3&gt;

&lt;div class=&quot;highlighter-rouge&quot;&gt;&lt;div class=&quot;highlight&quot;&gt;&lt;pre class=&quot;highlight&quot;&gt;&lt;code&gt;[ TIME ] Timed out waiting for device dev-di...\x2da115\x2de3f263d7b53a.device.
[DEPEND] Dependency failed for /dev/disk/by-...382-f41d-4c99-a115-e3f263d7b53a.
[DEPEND] Dependency failed for Swap.
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt;

&lt;p&gt;This may happen if you have changed the disk without updating the fstab
on the disk. To fix it, you can boot the disk in qemu and update fstab
with the correct UUID.&lt;/p&gt;

&lt;p&gt;I ran into this when I was resizing the disk.&lt;/p&gt;

&lt;h3 id=&quot;disk-is-too-small-for-what-you-want-to-do&quot;&gt;Disk is too small for what you want to do&lt;/h3&gt;

&lt;p&gt;Resizing an iso is pretty easy. You can use the same method you would if
you wanted to resize a partition on a regular hard drive.&lt;/p&gt;

&lt;p&gt;First, you need to resize the iso with qemu-image:&lt;/p&gt;

&lt;div class=&quot;highlighter-rouge&quot;&gt;&lt;div class=&quot;highlight&quot;&gt;&lt;pre class=&quot;highlight&quot;&gt;&lt;code&gt;qemu-img resize ubuntu-test.img +8G
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt;

&lt;p&gt;Now, you have a disk that has 8 GB of free space at the end of the disk.
You need to resize the partitions to use this free space. To do this, I
suggest using gparted just like you would for a real hard drive.&lt;/p&gt;

&lt;p&gt;You can download a gparted ISO from &lt;a href=&quot;http://gparted.org/livecd.php&quot;&gt;http://gparted.org/livecd.php&lt;/a&gt;.
Once you download the ISO, you can boot it with qemu the same way as we
booted the installation CD. Then, once its booted you can select the
disk you want to modify and follow the howto
(&lt;a href=&quot;http://gparted.org/display-doc.php%3Fname%3Dhelp-manual&quot;&gt;http://gparted.org/display-doc.php%3Fname%3Dhelp-manual&lt;/a&gt;).&lt;/p&gt;
</description>
        <pubDate>Fri, 13 Jan 2017 00:00:00 -0800</pubDate>
        <link>http://localhost:4000/tools/2017/01/13/gem5-fs.html</link>
        <guid isPermaLink="true">http://localhost:4000/tools/2017/01/13/gem5-fs.html</guid>
        
        
        <category>tools</category>
        
      </item>
    
      <item>
        <title>Creating disk images for gem5</title>
        <description>&lt;p&gt;When using gem5 in full-system mode, you have to have a disk image with
the operating system and all of your data on it. This is just like
having a physical disk in a physical machine. In this post, I’m going to
walk through how to create a new disk and install a (semi-)current
version of Ubuntu on the disk. By the end of this post, you should be
able to create your own disk with whatever extra data and applications
you want.&lt;/p&gt;

&lt;p&gt;This post assumes that you have already checked out a version of gem5
and can build and run gem5 in full-system mode. The &lt;a href=&quot;http://www.lowepower.com/jason/learning_gem5/&quot;&gt;Learning
gem5&lt;/a&gt; documentation is a
good place to start. This post uses the x86 ISA for gem5, and is mostly
applicable to other ISAs. More details on setting up ARM systems can be
found on the gem5 wiki:
&lt;a href=&quot;http://gem5.org/Ubuntu_Disk_Image_for_ARM_Full_System&quot;&gt;http://gem5.org/Ubuntu_Disk_Image_for_ARM_Full_System&lt;/a&gt;.&lt;/p&gt;

&lt;p&gt;In the future, this post may be folded into &lt;a href=&quot;http://www.lowepower.com/jason/learning_gem5/&quot;&gt;Learning
gem5&lt;/a&gt;.&lt;/p&gt;

&lt;h2 id=&quot;creating-a-blank-disk-image&quot;&gt;Creating a blank disk image&lt;/h2&gt;

&lt;p&gt;The first step is to create a blank disk image (usually a .img file).
Luckily, the gem5 developers have already made this easy with a tool
that is simple to use. To create a blank disk image, which is formatted
with ext2 by default, simply run the following.&lt;/p&gt;

&lt;div class=&quot;highlighter-rouge&quot;&gt;&lt;div class=&quot;highlight&quot;&gt;&lt;pre class=&quot;highlight&quot;&gt;&lt;code&gt;&amp;gt; util/gem5img.py init ubuntu-14.04.img 4096
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt;

&lt;p&gt;This command creates a new image, called “ubuntu-14.04.img” that is 4096
MB. This command may require you to enter the sudo password, if you
don’t have permission to create loopback devices. &lt;em&gt;You should never run
commands as the root user that you don’t understand! You should look at
the file util/gem5img.py and ensure that it isn’t going to do anything
malicious to your computer!&lt;/em&gt;&lt;/p&gt;

&lt;p&gt;We will be using util/gem5img.py heavily throughout this post, so you
may want to understand it better. If you just run &lt;code class=&quot;highlighter-rouge&quot;&gt;util/gem5img.py&lt;/code&gt;, it
displays all of the possible commands.&lt;/p&gt;

&lt;div class=&quot;highlighter-rouge&quot;&gt;&lt;div class=&quot;highlight&quot;&gt;&lt;pre class=&quot;highlight&quot;&gt;&lt;code&gt;Usage: %s [command] &amp;lt;command arguments&amp;gt;
where [command] is one of
    init: Create an image with an empty file system.
    mount: Mount the first partition in the disk image.
    umount: Unmount the first partition in the disk image.
    new: File creation part of &quot;init&quot;.
    partition: Partition part of &quot;init&quot;.
    format: Formatting part of &quot;init&quot;.
Watch for orphaned loopback devices and delete them with
losetup -d. Mounted images will belong to root, so you may need
to use sudo to modify their contents
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt;

&lt;h2 id=&quot;copying-root-files-to-the-disk&quot;&gt;Copying root files to the disk&lt;/h2&gt;

&lt;p&gt;Now that we have created a blank disk, we need to populate it with all
of the OS files. Ubuntu distributes a set of files explicitly for this
purpose. You can find the &lt;a href=&quot;https://wiki.ubuntu.com/Core&quot;&gt;Ubuntu core&lt;/a&gt;
distribution for 14.04 at
&lt;a href=&quot;http://cdimage.ubuntu.com/ubuntu-core/releases/14.04/release/&quot;&gt;http://cdimage.ubuntu.com/ubuntu-core/releases/14.04/release/&lt;/a&gt; Since I
am simulating an x86 machine, I chose the file
&lt;code class=&quot;highlighter-rouge&quot;&gt;ubuntu-core-14.04-core-amd64.tar.gz&lt;/code&gt;. Download whatever image is
appropriate for the system you are simulating.&lt;/p&gt;

&lt;p&gt;Next, we need to mount the blank disk and copy all of the files onto the
disk.&lt;/p&gt;

&lt;div class=&quot;highlighter-rouge&quot;&gt;&lt;div class=&quot;highlight&quot;&gt;&lt;pre class=&quot;highlight&quot;&gt;&lt;code&gt;mkdir mnt
../../util/gem5img.py mount ubuntu-14.04.img mnt
wget http://cdimage.ubuntu.com/ubuntu-core/releases/14.04/release/ubuntu-core-14.04-core-amd64.tar.gz
sudo tar xzvf ubuntu-core-14.04-core-amd64.tar.gz -C mnt
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt;

&lt;p&gt;The next step is to copy a few required files from your working system
onto the disk so we can chroot into the new disk. We need to copy
&lt;code class=&quot;highlighter-rouge&quot;&gt;/etc/resolv.conf&lt;/code&gt; onto the new disk.&lt;/p&gt;

&lt;div class=&quot;highlighter-rouge&quot;&gt;&lt;div class=&quot;highlight&quot;&gt;&lt;pre class=&quot;highlight&quot;&gt;&lt;code&gt;sudo cp /etc/resolv.conf mnt/etc/
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt;

&lt;h2 id=&quot;setting-up-gem5-specific-files&quot;&gt;Setting up gem5-specific files&lt;/h2&gt;

&lt;h3 id=&quot;create-a-serial-terminal&quot;&gt;Create a serial terminal&lt;/h3&gt;

&lt;p&gt;By default, gem5 uses the serial port to allow communication from the
host system to the simulated system. To use this, we need to create a
serial tty. Since Ubuntu uses upstart to control the init process, we
need to add a file to /etc/init which will initialize our terminal.
Also, in this file, we will add some code to detect if there was a
script passed to the simulated system. If there is a script, we will
execute the script instead of creating a terminal.&lt;/p&gt;

&lt;p&gt;Put the following code into a file called /etc/init/tty-gem5.conf&lt;/p&gt;

&lt;div class=&quot;highlighter-rouge&quot;&gt;&lt;div class=&quot;highlight&quot;&gt;&lt;pre class=&quot;highlight&quot;&gt;&lt;code&gt;# ttyS0 - getty
#
# This service maintains a getty on ttyS0 from the point the system is
# started until it is shut down again, unless there is a script passed to gem5.
# If there is a script, the script is executed then simulation is stopped.

start on stopped rc RUNLEVEL=[12345]
stop on runlevel [!12345]

console owner
respawn
script
   # Create the serial tty if it doesn't already exist
   if [ ! -c /dev/ttyS0 ]
   then
      mknod /dev/ttyS0 -m 660 /dev/ttyS0 c 4 64
   fi

   # Try to read in the script from the host system
   /sbin/m5 readfile &amp;gt; /tmp/script
   chmod 755 /tmp/script
   if [ -s /tmp/script ]
   then
      # If there is a script, execute the script and then exit the simulation
      exec su root -c '/tmp/script' # gives script full privileges as root user in multi-user mode
      /sbin/m5 exit
   else
      # If there is no script, login the root user and drop to a console
      # Use m5term to connect to this console
      exec /sbin/getty --autologin root -8 38400 ttyS0
   fi
end script
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt;

&lt;h3 id=&quot;setup-localhost&quot;&gt;Setup localhost&lt;/h3&gt;

&lt;p&gt;We also need to set up the localhost loopback device if we are going to
use any applications that use it. To do this, we need to add the
following to the &lt;code class=&quot;highlighter-rouge&quot;&gt;/etc/hosts&lt;/code&gt; file.&lt;/p&gt;

&lt;div class=&quot;highlighter-rouge&quot;&gt;&lt;div class=&quot;highlight&quot;&gt;&lt;pre class=&quot;highlight&quot;&gt;&lt;code&gt;127.0.0.1 localhost
::1 localhost ip6-localhost ip6-loopback
fe00::0 ip6-localnet
ff00::0 ip6-mcastprefix
ff02::1 ip6-allnodes
ff02::2 ip6-allrouters
ff02::3 ip6-allhosts
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt;

&lt;h3 id=&quot;update-fstab&quot;&gt;Update fstab&lt;/h3&gt;

&lt;p&gt;Next, we need to create an entry in &lt;code class=&quot;highlighter-rouge&quot;&gt;/etc/fstab&lt;/code&gt; for each partition we
want to be able to access from the simulated system. Only one partition
is absolutely required (&lt;code class=&quot;highlighter-rouge&quot;&gt;/&lt;/code&gt;); however, you may want to add additional
partitions, like a swap partition.&lt;/p&gt;

&lt;p&gt;The following should appear in the file &lt;code class=&quot;highlighter-rouge&quot;&gt;/etc/fstab&lt;/code&gt;.&lt;/p&gt;

&lt;div class=&quot;highlighter-rouge&quot;&gt;&lt;div class=&quot;highlight&quot;&gt;&lt;pre class=&quot;highlight&quot;&gt;&lt;code&gt;# /etc/fstab: static file system information.
#
# Use 'blkid' to print the universally unique identifier for a
# device; this may be used with UUID= as a more robust way to name devices
# that works even if disks are added and removed. See fstab(5).
#
# &amp;lt;file system&amp;gt;    &amp;lt;mount point&amp;gt;   &amp;lt;type&amp;gt;  &amp;lt;options&amp;gt;   &amp;lt;dump&amp;gt;  &amp;lt;pass&amp;gt;
/dev/hda1      /       ext3        noatime     0 1
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt;

&lt;h3 id=&quot;copy-the-m5-binary-to-the-disk&quot;&gt;Copy the &lt;code class=&quot;highlighter-rouge&quot;&gt;m5&lt;/code&gt; binary to the disk&lt;/h3&gt;

&lt;p&gt;gem5 comes with an extra binary application that executes
pseudo-instructions to allow the simulated system to interact with the
host system. To build this binary, run &lt;code class=&quot;highlighter-rouge&quot;&gt;make -f Makefile.&amp;lt;isa&amp;gt;&lt;/code&gt; in the
&lt;code class=&quot;highlighter-rouge&quot;&gt;gem5/m5&lt;/code&gt; directory, where &lt;code class=&quot;highlighter-rouge&quot;&gt;&amp;lt;isa&amp;gt;&lt;/code&gt; is the ISA that you are simulating
(e.g., x86). After this, you should have an &lt;code class=&quot;highlighter-rouge&quot;&gt;m5&lt;/code&gt; binary file. Copy this
file to /sbin on your newly created disk.&lt;/p&gt;

&lt;p&gt;After updating the disk with all of the gem5-specific files, unless you
are going on to add more applications or copying additional files,
unmount the disk image.&lt;/p&gt;

&lt;div class=&quot;highlighter-rouge&quot;&gt;&lt;div class=&quot;highlight&quot;&gt;&lt;pre class=&quot;highlight&quot;&gt;&lt;code&gt;&amp;gt; util/gem5img.py umount mnt
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt;

&lt;h2 id=&quot;install-new-applications&quot;&gt;Install new applications&lt;/h2&gt;

&lt;p&gt;The easiest way to install new applications on to your disk, is to use
&lt;code class=&quot;highlighter-rouge&quot;&gt;chroot&lt;/code&gt;. This program logically changes the root directory (“/”) to a
different directory, mnt in this case. Before you can change the root,
you first have to set up the special directories in your new root. To do
this, we use &lt;code class=&quot;highlighter-rouge&quot;&gt;mount -o bind&lt;/code&gt;.&lt;/p&gt;

&lt;div class=&quot;highlighter-rouge&quot;&gt;&lt;div class=&quot;highlight&quot;&gt;&lt;pre class=&quot;highlight&quot;&gt;&lt;code&gt;&amp;gt; sudo /bin/mount -o bind /sys mnt/sys
&amp;gt; sudo /bin/mount -o bind /dev mnt/dev
&amp;gt; sudo /bin/mount -o bind /proc mnt/proc
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt;

&lt;p&gt;After binding those directories, you can now &lt;code class=&quot;highlighter-rouge&quot;&gt;chroot&lt;/code&gt;:&lt;/p&gt;

&lt;div class=&quot;highlighter-rouge&quot;&gt;&lt;div class=&quot;highlight&quot;&gt;&lt;pre class=&quot;highlight&quot;&gt;&lt;code&gt;&amp;gt; sudo /usr/sbin/chroot mnt /bin/bash
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt;

&lt;p&gt;At this point you will see a root prompt and you will be in the &lt;code class=&quot;highlighter-rouge&quot;&gt;/&lt;/code&gt;
directory of your new disk.&lt;/p&gt;

&lt;p&gt;You should update your repository information.&lt;/p&gt;

&lt;div class=&quot;highlighter-rouge&quot;&gt;&lt;div class=&quot;highlight&quot;&gt;&lt;pre class=&quot;highlight&quot;&gt;&lt;code&gt;&amp;gt; apt-get update
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt;

&lt;p&gt;You may want to add the universe repositories to your list with the
following commands. Note: The first command is require in 14.04.&lt;/p&gt;

&lt;div class=&quot;highlighter-rouge&quot;&gt;&lt;div class=&quot;highlight&quot;&gt;&lt;pre class=&quot;highlight&quot;&gt;&lt;code&gt;&amp;gt; apt-get install software-properties-common
&amp;gt; add-apt-repository universe
&amp;gt; apt-get update
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt;

&lt;p&gt;Now, you are able to install any applications you could install on a
native Ubuntu machine via &lt;code class=&quot;highlighter-rouge&quot;&gt;apt-get&lt;/code&gt;.&lt;/p&gt;

&lt;p&gt;Remember, after you exit you need to unmount all of the directories we
used bind on.&lt;/p&gt;

&lt;div class=&quot;highlighter-rouge&quot;&gt;&lt;div class=&quot;highlight&quot;&gt;&lt;pre class=&quot;highlight&quot;&gt;&lt;code&gt;&amp;gt; sudo /bin/umount mnt/sys
&amp;gt; sudo /bin/umount mnt/proc
&amp;gt; sudo /bin/umount mnt/dev
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt;
</description>
        <pubDate>Tue, 24 Nov 2015 00:00:00 -0800</pubDate>
        <link>http://localhost:4000/jekyll/update/2015/11/24/gem5-disks.html</link>
        <guid isPermaLink="true">http://localhost:4000/jekyll/update/2015/11/24/gem5-disks.html</guid>
        
        
        <category>jekyll</category>
        
        <category>update</category>
        
      </item>
    
      <item>
        <title>gem5 Horrors and what we can do about it</title>
        <description>&lt;p&gt;&lt;img src=&quot;/assets/img/gem5-horrors.png&quot; alt=&quot;image&quot; /&gt;&lt;/p&gt;

&lt;p&gt;This post is a post which mostly follows the talk that I am giving at
the &lt;a href=&quot;http://gem5.org/User_workshop_2015&quot;&gt;gem5 Users Workshop&lt;/a&gt;. This post
contains some more details on problems that I skipped in my talk and
some references that I was not able to include in a presentation. You
can view my presentation on Google Drive
&lt;a href=&quot;https://docs.google.com/presentation/d/1QGA5UVaVJkkMITF2TXCY_KlwmfWef1KBzfDP6ocbj7I/pub?start=false&amp;amp;loop=false&amp;amp;delayms=3000&quot;&gt;here&lt;/a&gt;.&lt;/p&gt;

&lt;h2 id=&quot;i-3-gem5&quot;&gt;I &amp;lt;3 gem5&lt;/h2&gt;

&lt;p&gt;Before I get into the negative aspects of &lt;a href=&quot;http://gem5.org&quot;&gt;gem5&lt;/a&gt;, I
first want to point out that it is a great tool. gem5 is used by a large
number of computer architecture researchers, both in industry and in
academia. Here at Wisconsin, and at other universities, gem5 is used in
the classroom to teach students about computer architecture and how to
do computer architecture research.&lt;/p&gt;

&lt;p&gt;gem5 is, without a doubt, the most full-featured architecture simulator.
It leverages execute-at-execute semantics for high-fidelity
cycle-by-cycle simulation. gem5 can boot a mostly unmodified Linux
image. It has multiple different CPU and memory models. gem5 has a
modular design which makes it simple to embed and extend. This has
allowed gem5 to be used a large number of projects (see
&lt;a href=&quot;http://gem5.org/Projects&quot;&gt;http://gem5.org/Projects&lt;/a&gt; and &lt;a href=&quot;http://gem5.org/Publications&quot;&gt;http://gem5.org/Publications&lt;/a&gt;).&lt;/p&gt;

&lt;p&gt;However, as great as gem5 is, its growth has not been without pain. Now
that gem5 is nearing 15 years of development (if you include the
original m5 and GEMS project from which gem5 was born), I believe it’s
time to look at some of its deficiencies and talk about what we can do
to mitigate them.&lt;/p&gt;

&lt;h2 id=&quot;gem5-horrors&quot;&gt;gem5 horrors&lt;/h2&gt;

&lt;p&gt;Below, I discuss a few specific pain points with that I and others have
experienced with gem5. However, before I get to that, I’d like to talk
about what I think the root of these issues are. gem5 has two main
problems&lt;/p&gt;

&lt;p&gt;1)  There is no formal governance model.
2)  The gem5 developers do not think of the user first.&lt;/p&gt;

&lt;p&gt;Later, after I give some examples of specific problems, I will discuss
what I think can be done to fix these to issues.&lt;/p&gt;

&lt;p&gt;Next I discuss four specific “gem5 horrors” that either I have
personally experienced or I have talked to others who have experienced
them. These issues are deeper that just bugs, even if sometimes they can
be solved with simple changes. After describing each issue, I will also
quickly discuss a possible way to mitigate the problem.&lt;/p&gt;

&lt;h3 id=&quot;horror-1-merges&quot;&gt;Horror 1: Merges&lt;/h3&gt;

&lt;p&gt;There are a number of projects that build on top of gem5. In fact, I
would argue that this is the main use case for gem5. Everyone that I
know who uses gem5 for research, takes the mainline gem5 and builds
their own changes on top of it.&lt;/p&gt;

&lt;p&gt;The problem with this model, where people build on top of gem5, is that
when new features are added or bugs are fixed in the mainline,
downstream users have to consume these changes. If the downstream users
do a good job managing their patch queues, this should be a
straightforward thing to do. However, I have found that even when
careful development practices are followed, merging gem5 changes is
incredibly difficult.&lt;/p&gt;

&lt;p&gt;Below I discuss a few specific problems that I have run into when
merging new changes in gem5. I believe the problems can be summed up
with two high-level issued we currently have in gem5.&lt;/p&gt;

&lt;p&gt;1)  There is no well-defined static API. The interface to different
    modules is constantly in a state of flux.
2)  The regression suite we have in gem5 has poor coverage. There are
    many features that users depend on that are not covered by the
    regression tester.&lt;/p&gt;

&lt;h4 id=&quot;merge-headache-1-pointless-code-changes&quot;&gt;Merge headache #1: Pointless code changes&lt;/h4&gt;

&lt;p&gt;Examples from Ruby and Slicc and packet.&lt;/p&gt;

&lt;h4 id=&quot;merge-headache-2-features-break-between-versions&quot;&gt;Merge headache #2: Features break between versions&lt;/h4&gt;

&lt;p&gt;Ruby backing store, checkpointing&lt;/p&gt;

&lt;h4 id=&quot;merge-headache-3-apis-are-a-moving-target&quot;&gt;Merge headache #3: APIs are a moving target&lt;/h4&gt;

&lt;p&gt;Example with the minimal gem5 script.&lt;/p&gt;

&lt;h4 id=&quot;how-to-mitigate&quot;&gt;How to mitigate&lt;/h4&gt;

&lt;p&gt;I believe that there are two things we can can do as the gem5
development community to make merging upstream changes much easier.
First, we need a stable set of APIs. Second, we need a robust testing
and regression structure. I discuss some specifics of these two
characteristics below.&lt;/p&gt;

&lt;h4 id=&quot;stable-apis&quot;&gt;Stable APIs&lt;/h4&gt;

&lt;p&gt;Today in gem5, it is just as easy to change widely used interfaces, like
the port interface, as it is to change the implementation of a rarely
used function. We need to change this. I think that we need to choose a
set of interfaces and make them stable. This is similar to how the Linux
kernel operates.&lt;/p&gt;

&lt;p&gt;Once we have chosen a set of stable interfaces, I’m not suggesting that
they never change, only that it should be more onerous to change stable
APIs than other things. Additionally, this has the added benefit that
“gem5-stable” can actually mean something. We can now have a stable
version, which has non-changing APIs, and a dev version that we can’t
necessarily count on to have constant APIs.&lt;/p&gt;

&lt;p&gt;I personally do not know what the API should be. I would like to see the
community come together and talk about what they see as important
interfaces. Then, once we find these interfaces, we can architect these
interfaces and hopefully make gem5 easier to use.&lt;/p&gt;

&lt;h4 id=&quot;testing-structure&quot;&gt;Testing structure&lt;/h4&gt;

&lt;p&gt;I do not think that this is a very controversial issue, but gem5 needs a
better regression structure. If all of the features that we used in
gem5-gpu had been part of the regression suite, then we would have had
many less problems.&lt;/p&gt;

&lt;p&gt;Again, I do not know exactly how to make the regression suite better,
but I do think a good idea would be to require new features, and bug
fixes, to include a unit-test or something like that. We really need a
softeare engineer to sit down and architect a new regression system.
This would be a great project for someone who is new to the gem5
codebase.&lt;/p&gt;

&lt;h3 id=&quot;horror-2-configuration-files&quot;&gt;Horror 2: Configuration files&lt;/h3&gt;

&lt;p&gt;gem5 has an incredibly flexible configuration system. But with
flexibility often come complexity. In fact, I ran SLOCcount on the
configs directory and found there was more than 4000 lines of Python
code. According to the SLOCcount tool, this means there was 16
person-months and a quarter of a million dollars worth of code here!&lt;/p&gt;

&lt;p&gt;All of this complexity causes a number of issues. In my talk, I touched
on the fact that the defaults are confusing, and in some cases
inconsistent.&lt;/p&gt;

&lt;h4 id=&quot;how-to-mitigate-1&quot;&gt;How to mitigate&lt;/h4&gt;

&lt;p&gt;Since the m5 and GEMS integration, I have noticed a trend that the
number of command line parameters has continued to grow significantly.
It seems that every time a new feature has been added, we have added
some new command line parameters as well. I think this is the wrong way
to do it.&lt;/p&gt;

&lt;p&gt;There is an amazing C++-Python wrapper in gem5. We should be taking
advantage of the scripting capabilities of Python.&lt;/p&gt;

&lt;p&gt;I have created a simple script that is under 30 lines of Python. I think
we need to encourage our users to script in Python instead of adding
more and more command line parameters. Which, in my experience, really
just leads to scripting in bash instead of in Python anyway.&lt;/p&gt;

&lt;h3 id=&quot;horror-3-unexpected-results&quot;&gt;Horror 3: Unexpected results&lt;/h3&gt;

&lt;p&gt;This was a very surprising error that I ran into while working on
creating a homework assignment for a graduate-level computer
architecture course. The point of the homework was to compare the
performance of instruction latency versus instruction throughput. I
wanted the students to take a particular instruction and change the
number of execution units, the latency, and how much the units were
pipelined. To do this, we looked at the divide instruction, since it is
a long latency instruction. Below is the code that we used:&lt;/p&gt;

&lt;div class=&quot;highlighter-rouge&quot;&gt;&lt;div class=&quot;highlight&quot;&gt;&lt;pre class=&quot;highlight&quot;&gt;&lt;code&gt;for (int i = 0; i &amp;lt; N; i++) {
      Y[i] = X[i] / alpha + Y[i];
}
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt;

&lt;p&gt;In this code, every divide is totally independent from every other.
Therefore, we would expect that with he out-of-order CPU, that if the
divide is pipelined it the code will speedup by how much the divide unit
is pipelined.&lt;/p&gt;

&lt;p&gt;To test this, I looked at two different configurations, a 10 cycle
latency divide with &lt;em&gt;no&lt;/em&gt; pipelining, and a 10 cycle latency divide that
is fully pipelined. Below is the data I found for ARM and x86. I only
changed the “obvious” options. Each functional unit has an option for
the execution latency and issue latency. If the issue latency is 1, then
the functional unit is fully pipelined. (Now this is a boolean flag.)
All of the data is relative to x86 with no pipelining.&lt;/p&gt;

&lt;p&gt;Configuration   Latency     Issue lat.   x86 Perf   ARM Perf
  ————— ———– ———— ———- ————-
  No Pipeline     10 cycles   10 cycles    1.0x       8.0x
  Full Pipeline   10 cycles   1 cycle      1.0x       9.6x (1.2x)&lt;/p&gt;

&lt;p&gt;There are two very weird results in this data. First, when we fully
pipelined the divide unit, there was no performance change (at all!!) in
x86. Second, when running the exact same cod with ARM, there was a 8x
speedup compared to x86! I find it very hard to believe that the ARM ISA
is inherently better at divide than x86.&lt;/p&gt;

&lt;h4 id=&quot;how-to-mitigate-2&quot;&gt;How to mitigate&lt;/h4&gt;

&lt;p&gt;This is a much harder problem to mitigate than the others on this list.
Nilay Vaish has taken a step in the right directions with these two
patches on reviewboard &lt;a href=&quot;http://reviews.gem5.org/r/2744/&quot;&gt;http://reviews.gem5.org/r/2744/&lt;/a&gt; and
&lt;a href=&quot;http://reviews.gem5.org/r/2744/&quot;&gt;http://reviews.gem5.org/r/2744/&lt;/a&gt;, which have been incorporated in gem5.&lt;/p&gt;

&lt;p&gt;The underlying problem is that the implementation for ARM and x86 are
totally distinct. It is not clear to me what the right way to unify the
ISA implementation are. As a stop-gap, developers who are working on
implementing x86 features, need to make sure that they perform similarly
to ARM features. Maybe a solution is to have a single set of C programs
which exercise all ISAs and compare the performance across ISAs. There
should be some performance differences, but not an order of magnitude.&lt;/p&gt;

&lt;h3 id=&quot;horror-4-lack-of-new-user-support&quot;&gt;Horror 4: Lack of new-user support&lt;/h3&gt;

&lt;h4 id=&quot;how-to-mitigate-3&quot;&gt;How to mitigate&lt;/h4&gt;

&lt;p&gt;What I think we need to do is to create a “gem5 for Dummies” book or a
“Learning gem5” book. This book would be similar to Learning Python or
Learning Mercurial. The book would be open source for anyone to
contribute to. In fact, it should be required to update the book if a
developer makes an API-breaking change.&lt;/p&gt;

&lt;p&gt;An initial implementation of this book, which currently only includes
about a chapter of “getting started” and is in fact already out of date
can be found here:
&lt;a href=&quot;http://pages.cs.wisc.edu/~david/courses/cs752/Spring2015/gem5-tutorial/index.html&quot;&gt;gem5-tutorial&lt;/a&gt;.
I began working on this in conjunction with the graduate computer
architecture class at Wisconsin, so it may currently have some
Wisconsin-specific text. I hope to continue working on this in my
&lt;em&gt;copious free time&lt;/em&gt;.&lt;/p&gt;

&lt;p&gt;There are many other horrors that other people experience as well. Here
I only discussed some of the horrors that I have heard people
discussing. The purpose in presenting these horrors is not to say that
gem5 is a bad simulator! The purpose is to highlight how there are
currently issues that need to be addressed by the gem5 development
community.&lt;/p&gt;

&lt;h2 id=&quot;what-can-we-do-about-it&quot;&gt;What can we do about it?&lt;/h2&gt;

&lt;p&gt;A lot of the problems that I have discussed above come down to poor
software engineering. And yes, we are architects, not software
engineers, and there are a lot of things we could do better if we just
focused on software engineering. However, I do not think that this is
the underlying issue.&lt;/p&gt;

&lt;p&gt;I believe these four horror stem from two systemic problems in the gem5
development community.&lt;/p&gt;

&lt;p&gt;1)  There is no formal governance model.
2)  The gem5 developers do not think of the user first.&lt;/p&gt;

&lt;p&gt;I believe that if we start to solve these high-level issues, gem5 will
be a much better tool for everyone. Next, I discuss one possible way to
address these two points.&lt;/p&gt;

&lt;h2 id=&quot;gem5-foundation&quot;&gt;gem5 Foundation&lt;/h2&gt;

&lt;p&gt;First, I want to say that I do not believe this is the only way, or the
right way, to move gem5 forward. This is one possibility that I believe
will make gem5 a better tool. I hope that this is a place to begin the
discussion and I am sure that others in our community can come up with
even better suggestions that this!&lt;/p&gt;

&lt;p&gt;&lt;em&gt;I think we should create a gem5 Foundation.&lt;/em&gt; The gem5 Foundation will
be the center for the gem5 community. It will be a formal way for the
community to set goals and push gem5 forward.&lt;/p&gt;

&lt;p&gt;There are two main things I think the gem5 Foundation can help us with.
It can set up a formal governance structure and be a place for outside
interests to contribute money towards making gem5 better for everyone.&lt;/p&gt;

&lt;h3 id=&quot;formalizing-a-governance-structure&quot;&gt;Formalizing a governance structure&lt;/h3&gt;

&lt;p&gt;First, we need a governance structure. This is a document which defines
how decisions are made in the community, what matters to the community,
how to contribute to the community, etc.&lt;/p&gt;

&lt;p&gt;There is a lot of documentation on how to write governance models and
what they are. &lt;a href=&quot;http://oss-watch.ac.uk/&quot;&gt;OSS-Watch&lt;/a&gt; is a great source
for this. Here is a link to a definition of a governance model, which
does a much better job that I can explaining it.
&lt;a href=&quot;http://oss-watch.ac.uk/resources/governancemodels&quot;&gt;http://oss-watch.ac.uk/resources/governancemodels&lt;/a&gt; Additionally, here
is a link to an example governance model from an academic open-source
project:
&lt;a href=&quot;http://www.taverna.org.uk/about/legal-stuff/taverna-governance-model/&quot;&gt;http://www.taverna.org.uk/about/legal-stuff/taverna-governance-model/&lt;/a&gt;&lt;/p&gt;

&lt;h3 id=&quot;money-money-money&quot;&gt;Money, Money, Money&lt;/h3&gt;

&lt;p&gt;What I think the main solution to all of these problems is to pay
software developers &lt;em&gt;not computer architects!&lt;/em&gt; to solve some of these
problems. Already, within ARM and AMD there are a number of people who
get paid to work on gem5. However, these companies do not have gem5’s
best interests as their key focus. Their focus is what ARM and AMD find
interesting.&lt;/p&gt;

&lt;p&gt;So, I think that if we have something like the gem5 Foundation, these
companies and academia, can donate money towards coding things that are
good for the community as a whole. The gem5 Foundation can hire software
engineers to work on the parts of gem5 that grad students and
researchers do not want to do. If you look at other academic
communities, they often hire non researchers to do the “grunt work”.
Overall, I think this is a good idea for computer architects too, and
specifically for gem5.&lt;/p&gt;

&lt;p&gt;I recognize that this may be a crazy idea. I would love to hear what
others think. I am sure we will have some interesting discussion at the
gem5 workshop, and hopefully I will write another post with what other
people thought! Feel free to leave comments below.&lt;/p&gt;
</description>
        <pubDate>Tue, 09 Jun 2015 00:00:00 -0700</pubDate>
        <link>http://localhost:4000/2015/06/09/gem5-horrors.html</link>
        <guid isPermaLink="true">http://localhost:4000/2015/06/09/gem5-horrors.html</guid>
        
        
      </item>
    
  </channel>
</rss>