website: Update Learning gem5 Part 3
This has various updates based on new gem5 changes, fixes for markdown,
and general typo fixes.
Signed-off-by: Jason Lowe-Power <jason@lowepower.com>
Change-Id: Ibb71c667b2e0ec79bfcf003882439c2cb513e7d0
Reviewed-on: https://gem5-review.googlesource.com/c/public/gem5-website/+/62071
Reviewed-by: Bobby Bruce <bbruce@ucdavis.edu>
Maintainer: Bobby Bruce <bbruce@ucdavis.edu>
Tested-by: Bobby Bruce <bbruce@ucdavis.edu>
diff --git a/_pages/documentation/learning_gem5/part3/part3_00_MSIntro.md b/_pages/documentation/learning_gem5/part3/part3_00_MSIntro.md
index cd1f04b..a446073 100644
--- a/_pages/documentation/learning_gem5/part3/part3_00_MSIntro.md
+++ b/_pages/documentation/learning_gem5/part3/part3_00_MSIntro.md
@@ -8,13 +8,10 @@
---
-Introduction to Ruby
-====================
+## Introduction to Ruby
-Ruby comes from the [multifacet GEMS
-project](http://research.cs.wisc.edu/gems/). Ruby provides a detailed
-cache memory and cache coherence models as well as a detailed network
-model (Garnet).
+Ruby comes from the [multifacet GEMS project](http://research.cs.wisc.edu/gems/).
+Ruby provides a detailed cache memory and cache coherence models as well as a detailed network model (Garnet).
Ruby is flexible. It can model many different kinds of coherence
implementations, including broadcast, directory, token, region-based
diff --git a/_pages/documentation/learning_gem5/part3/part3_01_cache-intro.md b/_pages/documentation/learning_gem5/part3/part3_01_cache-intro.md
index b842e56..4fc4510 100644
--- a/_pages/documentation/learning_gem5/part3/part3_01_cache-intro.md
+++ b/_pages/documentation/learning_gem5/part3/part3_01_cache-intro.md
@@ -8,8 +8,7 @@
---
-MSI example cache protocol
-==========================
+## MSI example cache protocol
Before we implement a cache coherence protocol, it is important to have
a solid understanding of cache coherence. This section leans heavily on
@@ -25,7 +24,7 @@
Details for the protocol can be found in Section 8.2 of *A Primer on Memory Consistency and Cache Coherence* (pages 141-149).
It will be helpful to print out Section 8.2 to reference as you are implementing the protocol.
-You can download an excerpt of Sorin et al. that contains Section 8.2 [here](/_pages/static/external/Sorin_et-al_Excerpt_8.2.pdf).
+You can download the Second Edition [via this link](https://link.springer.com/content/pdf/10.1007/978-3-031-01764-3.pdf).
## First steps to writing a protocol
@@ -41,11 +40,9 @@
```python
Import('*')
-all_protocols.extend([
-'MSI',
-])
+main.Append(ALL_PROTOCOLS=['MSI'])
-protocol_dirs.append(str(Dir('.').abspath))
+main.Append(PROTOCOL_DIRS=[Dir('.')])
```
We do two things in this file. First, we register the name of our
@@ -57,10 +54,9 @@
the SLICC compiler.
You can download the `SConsopts` file
-[here](/_pages/static/scripts/part3/MSI_protocol/SConsopts).
+[here](https://gem5.googlesource.com/public/gem5/+/refs/heads/stable/src/learning_gem5/part3/SConsopts).
-Writing a state machine file
-----------------------------
+### Writing a state machine file
The next step, and most of the effort in writing a protocol, is to
create the state machine files. State machine files generally follow the
@@ -85,3 +81,5 @@
Transitions
: Specify actions to execute given a starting state and an event and
the final state. This is the meat of the state machine definition.
+
+Over the next few sections we will go over how to write each of these components of the protocol.
diff --git a/_pages/documentation/learning_gem5/part3/part3_02_cache-declarations.md b/_pages/documentation/learning_gem5/part3/part3_02_cache-declarations.md
index 986a790..8509fe2 100644
--- a/_pages/documentation/learning_gem5/part3/part3_02_cache-declarations.md
+++ b/_pages/documentation/learning_gem5/part3/part3_02_cache-declarations.md
@@ -8,9 +8,6 @@
---
-Declaring a state machine
-=========================
-
Let's start on our first state machine file! First, we will create the
L1 cache controller for our MSI protocol.
@@ -50,8 +47,7 @@
from the author of the protocol. It is important to be clear with these
descriptions since coherence protocols can get quite complicated.
-State machine parameters
-------------------------
+## State machine parameters
Proceeding the `machine()` declaration is a colon, after which all of
the parameters to the state machine are declared. These parameters are
@@ -181,8 +177,7 @@
mandatoryQueue = Param.MessageBuffer("")
```
-State declarations
-------------------
+## State declarations
The next part of the state machine is the state declaration. Here, we
are going to declare all of the stable and transient states for the
@@ -240,8 +235,7 @@
functional writes, all blocks are updated with new data if they have
busy, read-only, or read-write permission.
-Event declarations
-------------------
+## Event declarations
Next, we need to declare all of the events that are triggered by
incoming messages for this cache controller. These events come directly
@@ -275,8 +269,7 @@
}
```
-User-defined structures
------------------------
+## User-defined structures
Next, we need to define some structures that we will use in other places
in this controller. The first one we will define is `Entry`. This is the
@@ -334,8 +327,7 @@
The `external="yes"` tells SLICC to not look for the definition of this
structure. This is similar to declaring a variable `extern` in C/C++.
-Other declarations and definitions required
--------------------------------------------
+## Other declarations and definitions required
Finally, we are going to go through some boilerplate of declaring
variables, declaring functions in `AbstractController` that we will use
diff --git a/_pages/documentation/learning_gem5/part3/part3_03_cache-in-ports.md b/_pages/documentation/learning_gem5/part3/part3_03_cache-in-ports.md
index 5f5d1e3..5c54e87 100644
--- a/_pages/documentation/learning_gem5/part3/part3_03_cache-in-ports.md
+++ b/_pages/documentation/learning_gem5/part3/part3_03_cache-in-ports.md
@@ -8,9 +8,6 @@
---
-In port code blocks
-===================
-
After declaring all of the structures we need in the state machine file,
the first "functional" part of the file are the "in ports". This section
specifies what *events* to *trigger* on different incoming messages.
@@ -228,7 +225,7 @@
is specified as a `NetDest`, which is a bitmap of all the `MachineID` in
the system. This allows messages to be broadcast to a flexible set of
receivers. The message also has a size. You can find the possible
-message sizes in `src/mem/protocol/RubySlicc_Exports.sm`.
+message sizes in `src/mem/ruby/protocol/RubySlicc_Exports.sm`.
This message may also contain a data block and the number acks that are
expected. Thus, we can include these in the message definition as well.
@@ -240,7 +237,7 @@
written the functional access may fail.
You can download the complete `MSI-msg.sm` file
-[here](/_pages/static/scripts/part3/MSI_protocol/MSI-msg.sm).
+[here](https://gem5.googlesource.com/public/gem5/+/refs/heads/stable/src/learning_gem5/part3/MSI-msg.sm).
Now that we have defined the data in the response message, we can look
at how we choose which action to trigger in the `in_port` for response
diff --git a/_pages/documentation/learning_gem5/part3/part3_04_cache_actions.md b/_pages/documentation/learning_gem5/part3/part3_04_cache_actions.md
index e782a77..4e7c04d 100644
--- a/_pages/documentation/learning_gem5/part3/part3_04_cache_actions.md
+++ b/_pages/documentation/learning_gem5/part3/part3_04_cache_actions.md
@@ -7,9 +7,7 @@
author: Jason Lowe-Power
---
-
-Action code blocks
-==================
+## Action code blocks
The next section of the state machine file is the action blocks. The
action blocks are executed during a transition from one state to
diff --git a/_pages/documentation/learning_gem5/part3/part3_05_cache_transitions.md b/_pages/documentation/learning_gem5/part3/part3_05_cache_transitions.md
index 9cc1e9c..a0d7ac0 100644
--- a/_pages/documentation/learning_gem5/part3/part3_05_cache_transitions.md
+++ b/_pages/documentation/learning_gem5/part3/part3_05_cache_transitions.md
@@ -8,8 +8,7 @@
---
-Transition code blocks
-======================
+## Transition code blocks
Finally, we've reached the final section of the state machine file! This
section contains the details for all of the transitions between states
@@ -201,4 +200,4 @@
```
You can download the complete `MSI-cache.sm` file
-[here](/_pages/static/scripts/part3/MSI_protocol/MSI-cache.sm).
+[here](https://gem5.googlesource.com/public/gem5/+/refs/heads/stable/src/learning_gem5/part3/MSI-cache.sm).
diff --git a/_pages/documentation/learning_gem5/part3/part3_06_directory.md b/_pages/documentation/learning_gem5/part3/part3_06_directory.md
index 4a3d70b..ff427d1 100644
--- a/_pages/documentation/learning_gem5/part3/part3_06_directory.md
+++ b/_pages/documentation/learning_gem5/part3/part3_06_directory.md
@@ -8,9 +8,6 @@
---
-MSI Directory implementation
-============================
-
Implementing a directory controller is very similar to the L1 cache
controller, except using a different state machine table. The state
machine fore the directory can be found in Table 8.2 in Sorin et al.
@@ -36,6 +33,8 @@
MessageBuffer *responseFromCache, network="From", virtual_network="2",
vnet_type="response";
+MessageBuffer *requestToMemory;
+
MessageBuffer *responseFromMemory;
{
@@ -65,11 +64,10 @@
the L1 cache. These virtual network numbers are how the Ruby network
directs messages between controllers.
-There is also one more special message buffer: `responseFromMemory`.
+There is also two more special message buffers: `requestToMemory` and `responseFromMemory`.
This is similar to the `mandatoryQueue`, except instead of being like a
-slave port for CPUs it is like a master port. The `responseFromMemory`
-buffer will deliver response sent across the the memory port, as we will
-see below in the action section.
+responder port for CPUs it is like a requestor port. The `responseFromMemory` and `requestToMemory`
+buffers will deliver responses sent across the the memory port and send requests across the memory port, as we will see below in the action section.
After the parameters and message buffers, we need to declare all of the
states, events, and other local structures.
@@ -115,7 +113,7 @@
MemAck, desc="Ack from memory that write is complete";
}
-structure(Entry, desc="...", interface="AbstractEntry") {
+structure(Entry, desc="...", interface="AbstractCacheEntry", main="false") {
State DirState, desc="Directory state";
NetDest Sharers, desc="Sharers for this block";
NetDest Owner, desc="Owner of this block";
@@ -136,7 +134,10 @@
the sharers and the owner. This makes sense for the sharers, since we
want a full bitvector for all L1 caches that may be sharing the block.
The reason we also use a `NetDest` for the owner is to simply copy the
-structure into the message we send as a response as shown below.
+structure into the message we send as a response as shown below.D
+Note that we add one extra parameter to the `Entry` declaration: `main="false"`.
+This extra parameter tells the replacement policy that this `Entry` is special and should be ignored.
+In the `DirectoryMemory` we are tracking *all* of the backing memory locations, so there is no need for a replacement policy.
In this implementation, we use a few more transient states than in Table
8.2 in Sorin et al. to deal with the fact that the memory latency in
@@ -283,12 +284,11 @@
}
```
-The next part of the state machine file is the actions. First, we define
-actions for queuing memory reads and writes. For this, we will use a
-special function define in the `AbstractController`: `queueMemoryRead`.
-This function takes an address and converts it to a gem5 request and
-packet and sends it to across the port that is connected to this
-controller. We will see how to connect this port in the
+The next part of the state machine file is the actions.
+First, we define actions for sending memory reads and writes.
+For this, we will use the special `memQueue_out` port that we defined above.
+If we `enqueue` messages on this port, they will be translated into "normal" gem5 `PacketPtr`s and sent across the memory port defined in the configuration.
+We will see how to connect this port in the
configuration section \<MSI-config-section\>. Note that we need two
different actions to send data to memory for both requests and responses
since there are two different message buffers (virtual networks) that
@@ -297,7 +297,13 @@
```cpp
action(sendMemRead, "r", desc="Send a memory read request") {
peek(request_in, RequestMsg) {
- queueMemoryRead(in_msg.Requestor, address, toMemLatency);
+ enqueue(memQueue_out, MemoryMsg, toMemLatency) {
+ out_msg.addr := address;
+ out_msg.Type := MemoryRequestType:MEMORY_READ;
+ out_msg.Sender := in_msg.Requestor;
+ out_msg.MessageSize := MessageSizeType:Request_Control;
+ out_msg.Len := 0;
+ }
}
}
@@ -305,8 +311,14 @@
peek(request_in, RequestMsg) {
DPRINTF(RubySlicc, "Writing memory for %#x\n", address);
DPRINTF(RubySlicc, "Writing %s\n", in_msg.DataBlk);
- queueMemoryWrite(in_msg.Requestor, address, toMemLatency,
- in_msg.DataBlk);
+ enqueue(memQueue_out, MemoryMsg, toMemLatency) {
+ out_msg.addr := address;
+ out_msg.Type := MemoryRequestType:MEMORY_WB;
+ out_msg.Sender := in_msg.Requestor;
+ out_msg.MessageSize := MessageSizeType:Writeback_Data;
+ out_msg.DataBlk := in_msg.DataBlk;
+ out_msg.Len := 0;
+ }
}
}
@@ -314,9 +326,14 @@
peek(response_in, ResponseMsg) {
DPRINTF(RubySlicc, "Writing memory for %#x\n", address);
DPRINTF(RubySlicc, "Writing %s\n", in_msg.DataBlk);
- queueMemoryWrite(in_msg.Sender, address, toMemLatency,
- in_msg.DataBlk);
- }
+ enqueue(memQueue_out, MemoryMsg, toMemLatency) {
+ out_msg.addr := address;
+ out_msg.Type := MemoryRequestType:MEMORY_WB;
+ out_msg.Sender := in_msg.Sender;
+ out_msg.MessageSize := MessageSizeType:Writeback_Data;
+ out_msg.DataBlk := in_msg.DataBlk;
+ out_msg.Len := 0;
+ }
}
```
@@ -574,4 +591,4 @@
```
You can download the complete `MSI-dir.sm` file
-[here](/_pages/static/scripts/part3/MSI_protocol/MSI-dir.sm).
+[here](https://gem5.googlesource.com/public/gem5/+/refs/heads/stable/src/learning_gem5/part3/MSI-dir.sm).
diff --git a/_pages/documentation/learning_gem5/part3/part3_07_MSIbuilding.md b/_pages/documentation/learning_gem5/part3/part3_07_MSIbuilding.md
index b4bf80e..7ba83eb 100644
--- a/_pages/documentation/learning_gem5/part3/part3_07_MSIbuilding.md
+++ b/_pages/documentation/learning_gem5/part3/part3_07_MSIbuilding.md
@@ -8,18 +8,16 @@
---
-Compiling a SLICC protocol
-==========================
+## Building the MSI protocol
-The SLICC file
---------------
+### The SLICC file
Now that we have finished implementing the protocol, we need to compile
it. You can download the complete SLICC files below:
-- [MSI-cache.sm](/_pages/static/scripts/part3/MSI\_protocol/MSI-cache.sm)
-- [MSI-dir.sm](/_pages/static/scripts/part3/MSI\_protocol/MSI-dir.sm)
-- [MSI-msg.sm](/_pages/static/scripts/part3/MSI\_protocol/MSI-msg.sm)
+- [MSI-cache.sm](https://gem5.googlesource.com/public/gem5/+/refs/heads/stable/src/learning_gem5/part3/MSI-cache.sm)
+- [MSI-dir.sm](https://gem5.googlesource.com/public/gem5/+/refs/heads/stable/src/learning_gem5/part3/MSI-dir.sm)
+- [MSI-msg.sm](https://gem5.googlesource.com/public/gem5/+/refs/heads/stable/src/learning_gem5/part3/MSI-msg.sm)
Before building the protocol, we need to create one more file:
`MSI.slicc`. This file tells the SLICC compiler which state machine
@@ -44,10 +42,9 @@
```
You can download the fill file
-[here](/_pages/static/scripts/part3/MSI_protocol/MSI.slicc).
+[here](https://gem5.googlesource.com/public/gem5/+/refs/heads/stable/src/learning_gem5/part3/s/MSI.slicc).
-Compiling a protocol with SCons
--------------------------------
+### Compiling a protocol with SCons
Most SCons defaults (found in `build_opts/`) specify the protocol as
`MI_example`, an example, but poor performing protocol. Therefore, we
@@ -55,7 +52,7 @@
to specify the SCons options on the command line. The command line below
will build our new protocol with the X86 ISA.
-```
+```sh
scons build/X86_MSI/gem5.opt --default=X86 PROTOCOL=MSI SLICC_HTML=True
```
diff --git a/_pages/documentation/learning_gem5/part3/part3_08_configuration.md b/_pages/documentation/learning_gem5/part3/part3_08_configuration.md
index 275c08d..0a76256 100644
--- a/_pages/documentation/learning_gem5/part3/part3_08_configuration.md
+++ b/_pages/documentation/learning_gem5/part3/part3_08_configuration.md
@@ -8,8 +8,7 @@
---
-Configuring a simple Ruby system
-================================
+## A configuration script for the MSI protocol
First, create a new configuration directory in `configs/`. Just like all
gem5 configuration files, we will have a configuration run script. For
@@ -43,10 +42,9 @@
system and the memory controllers.
You can download the complete run script
-[here](_pages/static/scripts/part3/configs/simple_ruby.py).
+[here](https://gem5.googlesource.com/public/gem5/+/refs/heads/stable/configs/learning_gem5/part3/simple_ruby.py).
-Cache system configuration
---------------------------
+### Cache system configuration
Now, let's create a file `msi_caches.py`. In this file, we will create
four classes: `MyCacheSystem` which will inherit from `RubySystem`,
@@ -54,7 +52,7 @@
by SLICC from our two state machines, and `MyNetwork` which will inherit
from `SimpleNetwork`.
-### L1 Cache
+#### L1 Cache
Let's start with the `L1Cache`. First, we will inherit from
`L1Cache_Controller` since we named our L1 cache "L1Cache" in the state
@@ -150,7 +148,7 @@
self.responseFromDirOrSibling.slave = ruby_system.network.master
```
-### Directory
+#### Directory
Now, we can similarly implement the directory. There are three
differences from the L1 cache. First, we need to set the address ranges
@@ -202,7 +200,7 @@
self.responseFromMemory = MessageBuffer()
```
-### Ruby System
+#### Ruby System
Now, we can implement the Ruby system object. For this object, the
constructor is simple. It just checks the SCons variable `PROTOCOL` to
@@ -297,7 +295,7 @@
cpu.dtb.walker.port = self.sequencers[i].slave
```
-### Network
+#### Network
Finally, the last object we have to implement is the network. The
constructor is simple, but we need to declare an empty list for the list
@@ -342,4 +340,4 @@
```
You can download the complete `msi_caches.py` file
-[here](/_pages/static/scripts/part3/configs/msi_caches.py).
+[here](https://gem5.googlesource.com/public/gem5/+/refs/heads/stable/configs/learning_gem5/part3/msi_caches.py).
diff --git a/_pages/documentation/learning_gem5/part3/part3_09_running.md b/_pages/documentation/learning_gem5/part3/part3_09_running.md
index 50f89f1..6e9ae53 100644
--- a/_pages/documentation/learning_gem5/part3/part3_09_running.md
+++ b/_pages/documentation/learning_gem5/part3/part3_09_running.md
@@ -8,8 +8,7 @@
---
-Running the simple Ruby system
-==============================
+## Running the simple Ruby system
Now, we can run our system with the MSI protocol!
@@ -112,7 +111,7 @@
With the above code compiled as `threads`, we can run gem5!
-```
+```sh
build/MSI/gem5.opt configs/learning_gem5/part6/simple_ruby.py
```
@@ -120,6 +119,7 @@
are unimplemented syscalls in SE mode due to using pthreads and can be
safely ignored for this simple example.
+```termout
gem5 Simulator System. http://gem5.org
gem5 is copyrighted software; use the --copyright option for details.
@@ -168,3 +168,4 @@
warn: ignoring syscall madvise(...)
Validating...Success!
Exiting @ tick 9386342000 because exiting with last active thread context
+```
diff --git a/_pages/documentation/learning_gem5/part3/part3_10_MSIdebugging.md b/_pages/documentation/learning_gem5/part3/part3_10_MSIdebugging.md
index 7ff437d..d06c110 100644
--- a/_pages/documentation/learning_gem5/part3/part3_10_MSIdebugging.md
+++ b/_pages/documentation/learning_gem5/part3/part3_10_MSIdebugging.md
@@ -8,8 +8,6 @@
---
-Debugging SLICC Protocols
-=========================
In this section, I present the steps that I took while debugging the MSI
protocol implemented earlier in this chapter. Learning to debug
@@ -26,8 +24,7 @@
then the solution to the error, sometimes with some commentary of the
different tactics I tried to solve the error.
-General debugging tips
-----------------------
+## General debugging tips
Ruby has many useful debug flags. However, the most useful, by far, is
`ProtocolTrace`. Below, you will see several examples of using the
@@ -83,8 +80,7 @@
easier to debug your protocol with the random tester than with real
applications!
-Understanding Protocol Traces
------------------------------
+## Understanding Protocol Traces
Unfortunately, despite extensive effort to catch bugs in them, coherence
protocols (even heavily tested ones) will have bugs. Sometimes these
@@ -105,6 +101,7 @@
is happening. To start with, lets look at a small snippet of a protocol
trace (we will discuss the details of this trace further below):
+```protocoltrace
...
4541 0 L1Cache Replacement MI_A>MI_A [0x4ac0, line 0x4ac0]
4542 0 L1Cache PutAck MI_A>I [0x4ac0, line 0x4ac0]
@@ -118,37 +115,38 @@
5321 0 Seq Begin > [0x4aec, line 0x4ac0] ST
5322 0 L1Cache Store S>SM_AD [0x4ac0, line 0x4ac0]
5327 0 Directory GetM S>M_M [0x4ac0, line 0x4ac0]
+```
Every line in this trace has a set pattern in terms of what information
appears on that line. Specifically, the fields are:
-1. Current Tick: the tick the print is occurs in
-2. Machine Version: The number of the machine where this request is
- coming from. For example, if there are 4 L1 caches, then the numbers
- would be 0-3. Assuming you have 1 L1 Cache per core, you can think
- of this as representing the core the request is coming from.
-3. Component: which part of the system is doing the print. Generally,
- `Seq` is shorthand for Sequencer, `L1Cache` represents the L1 Cache,
- "Directory" represents the directory, and so on. For L1 caches and
- the directory, this represents the name of the machine type (i.e.,
- what is after "MachineType:" in the `machine()` definition).
-4. Action: what the component is doing. For example, "Begin" means the
- Sequencer has received a new request, "Done" means that the
- Sequencer is completing a previous request, and "DataDirNoAcks"
- means that our DataDirNoAcks event is being triggered.
-5. Transition (e.g., MI\_A\>MI\_A): what state transition this action
- is doing (format: "currentState\>nextState"). If no transition is
- happening, this is denoted with "\>".
-6. Address (e.g., [0x4ac0, line 0x4ac0]): the physical address of the
- request (format: [wordAddress, lineAddress]). This address will
- always be cache-block aligned except for requests from the
- `Sequencer` and `mandatoryQueue`.
-7. (Optional) Comments: optionally, there is one additional field to
- pass comments. For example, the "LD" , "ST", and "33 cycles" lines
- use this extra field to pass additional information to the trace --
- such as identifying the request as a load or store. For SLICC
- transitions, `APPEND_TRANSITION_COMMENT` often use this, as we
- [discussed previously](../cache-actions/).
+1. Current Tick: the tick the print is occurs in
+2. Machine Version: The number of the machine where this request is
+ coming from. For example, if there are 4 L1 caches, then the numbers
+ would be 0-3. Assuming you have 1 L1 Cache per core, you can think
+ of this as representing the core the request is coming from.
+3. Component: which part of the system is doing the print. Generally,
+ `Seq` is shorthand for Sequencer, `L1Cache` represents the L1 Cache,
+ "Directory" represents the directory, and so on. For L1 caches and
+ the directory, this represents the name of the machine type (i.e.,
+ what is after "MachineType:" in the `machine()` definition).
+4. Action: what the component is doing. For example, "Begin" means the
+ Sequencer has received a new request, "Done" means that the
+ Sequencer is completing a previous request, and "DataDirNoAcks"
+ means that our DataDirNoAcks event is being triggered.
+5. Transition (e.g., MI\_A\>MI\_A): what state transition this action
+ is doing (format: "currentState\>nextState"). If no transition is
+ happening, this is denoted with "\>".
+6. Address (e.g., [0x4ac0, line 0x4ac0]): the physical address of the
+ request (format: [wordAddress, lineAddress]). This address will
+ always be cache-block aligned except for requests from the
+ `Sequencer` and `mandatoryQueue`.
+7. (Optional) Comments: optionally, there is one additional field to
+ pass comments. For example, the "LD" , "ST", and "33 cycles" lines
+ use this extra field to pass additional information to the trace --
+ such as identifying the request as a load or store. For SLICC
+ transitions, `APPEND_TRANSITION_COMMENT` often use this, as we
+ [discussed previously](../cache-actions/).
Generally, spaces are used to separate each of these fields (the space
between the fields are added implicitly, you do not need to add them).
@@ -176,15 +174,18 @@
`Seq Begin` and `Seq Done` trace prints
come from (search for ProtocolTrace).
-Errors I ran into debugging MSI
--------------------------------
+## Errors I ran into debugging MSI
+```termout
gem5.opt: build/MSI/mem/ruby/system/Sequencer.cc:423: void Sequencer::readCallback(Addr, DataBlock&, bool, MachineType, Cycles, Cycles, Cycles): Assertion `m_readRequestTable.count(makeLineAddress(address))' failed.
+```
-I'm an idiot, it was that I called readCallback in externalStoreHit
+I'm made a silly mistake. It was that I called readCallback in externalStoreHit
instead of writeCallback. It's good to start simple!
+```termout
gem5.opt: build/MSI/mem/ruby/network/MessageBuffer.cc:220: Tick MessageBuffer::dequeue(Tick, bool): Assertion `isReady(current_time)' failed.
+```
I ran gem5 in GDB to get more information. Look at
L1Cache\_Controller::doTransitionWorker. The current transition is:
@@ -195,8 +196,10 @@
The problem is that the PutAck is on the forward network, not the
response network.
+```termout
panic: Invalid transition
system.caches.controllers0 time: 3594 addr: 3264 event: DataDirAcks state: IS_D
+```
Hmm. I think this shouldn't have happened. The needed acks should always
be 0 or you get data from the owner. Ah. So I implemented sendDataToReq
@@ -215,19 +218,21 @@
requestor is the owner do we include the number of sharers. Otherwise,
it doesn't matter at all and we just set the sharers to 0.
-
+```termout
panic: Invalid transition system.caches.controllers0 time: 5332
addr: 0x4ac0 event: Inv state: SM\_AD
+```
First, let's look at where Inv is triggered. If you get an invalidate...
only then. Maybe it's that we are on the sharer list and shouldn't be?
We can use protocol trace and grep to find what's going on.
-```
+```sh
build/MSI/gem5.opt --debug-flags=ProtocolTrace configs/learning_gem5/part6/ruby_test.py | grep 0x4ac0
```
+```termout
...
4541 0 L1Cache Replacement MI_A>MI_A [0x4ac0, line 0x4ac0]
4542 0 L1Cache PutAck MI_A>I [0x4ac0, line 0x4ac0]
@@ -241,6 +246,7 @@
5321 0 Seq Begin > [0x4aec, line 0x4ac0] ST
5322 0 L1Cache Store S>SM_AD [0x4ac0, line 0x4ac0]
5327 0 Directory GetM S>M_M [0x4ac0, line 0x4ac0]
+```
Maybe there is a sharer in the sharers list when there shouldn't be? We
can add a defensive assert in clearOwner and setOwner.
@@ -261,7 +267,9 @@
Now, I get the following error:
+```termout
panic: Runtime Error at MSI-dir.sm:301: assert failure.
+```
This is in setOwner. Well, actually this is OK since we need to have the
sharers still set until we count them to send the ack count to the
@@ -280,7 +288,9 @@
So, onto the next problem!
+```termout
panic: Deadlock detected: current_time: 56091 last_progress_time: 6090 difference: 50001 processor: 0
+```
Deadlocks are the worst kind of error. Whatever caused the deadlock is
ancient history (i.e., likely happened many cycles earlier), and often
@@ -290,6 +300,7 @@
the protocol trace into a file because it grows *very* big) I see that
there is an address that is trying to be replaced. Let's start there.
+```protocoltrace
56091 0 L1Cache Replacement SM_A>SM_A [0x5ac0, line 0x5ac0]
56091 0 L1Cache Replacement SM_A>SM_A [0x5ac0, line 0x5ac0]
56091 0 L1Cache Replacement SM_A>SM_A [0x5ac0, line 0x5ac0]
@@ -300,10 +311,12 @@
56091 0 L1Cache Replacement SM_A>SM_A [0x5ac0, line 0x5ac0]
56091 0 L1Cache Replacement SM_A>SM_A [0x5ac0, line 0x5ac0]
56091 0 L1Cache Replacement SM_A>SM_A [0x5ac0, line 0x5ac0]
+```
Before this replacement got stuck I see the following in the protocol
trace. Note: this is 50000 cycles in the past!
+```protocoltrace
...
5592 0 L1Cache Store S>SM_AD [0x5ac0, line 0x5ac0]
5597 0 Directory GetM S>M_M [0x5ac0, line 0x5ac0]
@@ -311,6 +324,7 @@
5641 0 Directory MemData M_M>M [0x5ac0, line 0x5ac0]
...
5646 0 L1Cache DataDirAcks SM_AD>SM_A [0x5ac0, line 0x5ac0]
+```
Ah! This clearly should not be DataDirAcks since we only have a single
CPU! So, we seem to not be subtracting properly. Going back to the
@@ -328,6 +342,7 @@
What I'm seeing at the end of the protocol trace is the following.
+```protocoltrace
144684 0 L1Cache Replacement MI_A>MI_A [0x5bc0, line 0x5bc0]
...
144685 0 Directory GetM MI_M>MI_M [0x54c0, line 0x54c0]
@@ -340,6 +355,7 @@
...
144687 0 Directory GetM MI_M>MI_M [0x54c0, line 0x54c0]
...
+```
This is repeated for a long time.
@@ -385,10 +401,12 @@
Then, I see the following output when running with ProtocolTrace and
RubySlicc.
+```gem5trace
118 0 Directory MemData M_M>M [0x400, line 0x400]
118: system.caches.controllers2: MSI-dir.sm:160: Owner [NetDest (16) 1 0 - - - 0 - - - - - - - - - - - - - ]
118 0 Directory GetM M>M [0x400, line 0x400]
118: system.caches.controllers2: MSI-dir.sm:160: Owner [NetDest (16) 1 1 - - - 0 - - - - - - - - - - - - - ]
+```
It looks like when we process the GetM when in state M we need to first
clear the owner before adding the new owner. The other options is in
@@ -397,7 +415,9 @@
Oooo! This is a new error!
+```termout
panic: Runtime Error at MSI-dir.sm:229: Unexpected message type..
+```
What is this message that fails? Let's use the RubyNetwork debug flag to
try to track down what message is causing this error. A few lines above
@@ -409,7 +429,9 @@
the first two 0's are for the CPUs, and the other 1 must be fore the
directory.
+```gem5trace
2285: PerfectSwitch-2: Message: [ResponseMsg: addr = [0x8c0, line 0x8c0] Type = InvAck Sender = L1Cache-1 Destination = [NetDest (16) 0 0 - - - 1 - - - - - - - - - - - - - ] DataBlk = [ 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0xb1 0xb2 0xb3 0xb4 0xca 0xcb 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 ] MessageSize = Control Acks = 0 ]
+```
This message has the type InvAck, which is clearly wrong! It seems that
we are setting the requestor wrong when we send the invalidate (Inv)
@@ -419,8 +441,10 @@
requestor. This was already correct for the FwdGetS/M, but I missed the
invalidate somehow. On to the next error!
+```termout
panic: Invalid transition
system.caches.controllers0 time: 2287 addr: 0x8c0 event: LastInvAck state: SM_AD
+```
This seems to be that I am not counting the acks correctly. It could
also be that the directory is much slower than the other caches at
@@ -449,10 +473,12 @@
corrupted in the ancient past. I believe the address is the last one in
the protocol trace.
+```termout
panic: Action/check failure: proc: 0 address: 19688 data: 0x779e6d0
byte\_number: 0 m\_value+byte\_number: 53 byte: 0 [19688, value: 53,
status: Check\_Pending, initiating node: 0, store\_count: 4]Time:
5843
+```
So, it could be something to do with ack counts, though I don't think
this is the issue. Either way, it's a good idea to annotate the protocol
@@ -468,7 +494,9 @@
}
```
+```protocoltrace
5737 1 L1Cache InvAck SM_AD>SM_AD [0x400, line 0x400] Acks: -1
+```
For these data issues, the debug flag RubyNetwork is useful because it
prints the value of the data blocks at every point it is in the network.
@@ -477,24 +505,28 @@
have valid data. In fact, if we go back in time some we see that there
was some non-zero elements.
+```protocoltrace
5382 1 L1Cache Inv S>I [0x4cc0, line 0x4cc0]
+```
- > 5383: PerfectSwitch-1: Message: [ResponseMsg: addr = [0x4cc0, line
- > 0x4cc0] Type = InvAck Sender = L1Cache-1 Destination = [NetDest (16) 1
- > 0 - - - 0 - - - - - - - - - - - - - ] DataBlk = [ 0x0 0x0 0x0 0x0 0x0
- > 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0
- > 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0
- > 0x0 0x35 0x36 0x37 0x61 0x6d 0x6e 0x6f 0x70 0x0 0x0 0x0 0x0 0x0 0x0
- > 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 ] MessageSize = Control Acks =
- > 0 ] ... ... ... 5389 0 Directory MemData M\_M\ >M [0x4cc0, line 0x4cc0]
- > 5390: PerfectSwitch-2: incoming: 0 5390: PerfectSwitch-2: Message:
- > [ResponseMsg: addr = [0x4cc0, line 0x4cc0] Type = Data Sender =
- > Directory-0 Destination = [NetDest (16) 1 0 - - - 0 - - - - - - - - -
- > - - - - ] DataBlk = [ 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0
- > 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0
- > 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0
- > 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0
- > 0x0 ] MessageSize = Data Acks = 1 ]
+```gem5trace
+ 5383: PerfectSwitch-1: Message: [ResponseMsg: addr = [0x4cc0, line
+ 0x4cc0] Type = InvAck Sender = L1Cache-1 Destination = [NetDest (16) 1
+ 0 - - - 0 - - - - - - - - - - - - - ] DataBlk = [ 0x0 0x0 0x0 0x0 0x0
+ 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0
+ 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0
+ 0x0 0x35 0x36 0x37 0x61 0x6d 0x6e 0x6f 0x70 0x0 0x0 0x0 0x0 0x0 0x0
+ 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 ] MessageSize = Control Acks =
+ 0 ] ... ... ... 5389 0 Directory MemData M\_M\ >M [0x4cc0, line 0x4cc0]
+ 5390: PerfectSwitch-2: incoming: 0 5390: PerfectSwitch-2: Message:
+ [ResponseMsg: addr = [0x4cc0, line 0x4cc0] Type = Data Sender =
+ Directory-0 Destination = [NetDest (16) 1 0 - - - 0 - - - - - - - - -
+ - - - - ] DataBlk = [ 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0
+ 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0
+ 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0
+ 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0
+ 0x0 ] MessageSize = Data Acks = 1 ]
+```
It seems that memory is not being updated correctly on the M-\>S
transition. After lots of digging and using the MemoryAccess debug flag
@@ -503,8 +535,10 @@
PutM, but not right for Data. We need to have another action to send
data from response queue!
+```termout
panic: Invalid transition
system.caches.controllers0 time: 44381 addr: 0x7c0 event: Inv state: SM_AD
+```
Invalid transition is my personal favorite kind of SLICC error. For this
error, you know exactly what address caused it, and it's very easy to
diff --git a/_pages/documentation/learning_gem5/part3/part3_11_simple-MI_example.md b/_pages/documentation/learning_gem5/part3/part3_11_simple-MI_example.md
index a00905b..69362d9 100644
--- a/_pages/documentation/learning_gem5/part3/part3_11_simple-MI_example.md
+++ b/_pages/documentation/learning_gem5/part3/part3_11_simple-MI_example.md
@@ -7,10 +7,6 @@
author: Jason Lowe-Power
---
-
-Configuring for a standard protocol
-===================================
-
You can easily adapt the simple example configurations from this part to
the other SLICC protocols in gem5. In this chapter, we will briefly look
at an example with `MI_example`, though this can be easily extended to
@@ -26,7 +22,7 @@
the classes needed for `MI_example`. There are only a couple of changes
from `MSI`, mostly due to different naming schemes. You can download the
file
-[here](/_pages/static/scripts/part3/configs/ruby_caches_MI_example.py).
+[here](https://gem5.googlesource.com/public/gem5/+/refs/heads/stable/configs/learning_gem5/part3/ruby_caches_MI_example.py).
```python
class MyCacheSystem(RubySystem):
