M5's new memory system (introduced in the first 2.0 beta release) was designed with the following goals:
For details on the new coherence protocol, introduced (along with a substantial cache model rewrite) in 2.0b4, see Coherence Protocol.
All objects that connect to the memory system inherit from MemObject
. This class adds the pure virtual functions getMasterPort(const std::string &name, PortID idx)
and getSlavePort(const std::string &name, PortID idx)
which returns a port corresponding to the given name and index. This interface is used to structurally connect the MemObjects together.
The next large part of the memory system is the idea of ports. Ports are used to interface memory objects to each other. They will always come in pairs, with a MasterPort and a SlavePort, and we refer to the other port object as the peer. These are used to make the design more modular. With ports a specific interface between every type of object doesn't have to be created. Every memory object has to have at least one port to be useful. A master module, such as a CPU, has one or more MasterPort instances. A slave module, such as a memory controller, has one or more SlavePorts. An interconnect component, such as a cache, bridge or bus, has both MasterPort and SlavePort instances.
There are two groups of functions in the port object. The send*
functions are called on the port by the object that owns that port. For example to send a packet in the memory system a CPU would call myPort->sendTimingReq(pkt)
to send a packet. Each send function has a corresponding recv function that is called on the ports peer. So the implementation of the sendTimingReq()
call above would simply be peer->recvTimingReq(pkt)
on the slave port. Using this method we only have one virtual function call penalty but keep generic ports that can connect together any memory system objects.
Master ports can send requests and receive responses, whereas slave ports receive requests and send responses. Due to the coherence protocol, a slave port can also send snoop requests and receive snoop responses, with the master port having the mirrored interface.
In Python, Ports are first-class attributes of simulation objects, much like Params. Two objects can specify that their ports should be connected using the assignment operator. Unlike a normal variable or parameter assignment, port connections are symmetric: A.port1 = B.port2
has the same meaning as B.port2 = A.port1
. The notion of master and slave ports exists in the Python objects as well, and a check is done when the ports are connected together.
Objects such as busses that have a potentially unlimited number of ports use “vector ports”. An assignment to a vector port appends the peer to a list of connections rather than overwriting a previous connection.
In C++, memory ports are connected together by the python code after all objects are instantiated.
A request object encapsulates the original request issued by a CPU or I/O device. The parameters of this request are persistent throughout the transaction, so a request object‘s fields are intended to be written at most once for a given request. There are a handful of constructors and update methods that allow subsets of the object’s fields to be written at different times (or not at all). Read access to all request fields is provided via accessor methods which verify that the data in the field being read is valid.
The fields in the request object are typically not available to devices in a real system, so they should normally be used only for statistics or debugging and not as architectural values.
Request object fields include:
A Packet is used to encapsulate a transfer between two objects in the memory system (e.g., the L1 and L2 cache). This is in contrast to a Request where a single Request travels all the way from the requester to the ultimate destination and back, possibly being conveyed by several different Packets along the way.
Read access to many packet fields is provided via accessor methods which verify that the data in the field being read is valid.
A packet contains the following all of which are accessed by accessors to be certain the data is valid:
dataStatic()
, dataDynamic()
, and dataDynamicArray()
which control if the data associated with the packet is freed when the packet is, not, with delete
, and with delete []
respectively.allocate()
and the data is freed when the packet is destroyed. (Always safe to call).getPtr()
get()
and set()
can be used to manipulate the data in the packet. The get() method does a guest-to-host endian conversion and the set method does a host-to-guest endian conversion.SenderState
pointer which is a virtual base opaque structure used to hold state associated with the packet but specific to the sending device (e.g., an MSHR). A pointer to this state is returned in the packet's response so that the sender can quickly look up the state needed to process it. A specific subclass would be derived from this to carry state specific to a particular sending device.CoherenceState
pointer which is a virtual base opaque structure used to hold coherence-related state. A specific subclass would be derived from this to carry state specific to a particular coherence protocol.There are three types of accesses supported by the ports.
Packet::intersect()
and fixPacket()
methods can help with this.The protocol for allocation and deallocation of Packet objects varies depending on the access type. (We're talking about low-level C++ new
/delete
issues here, not anything related to the coherence protocol.)
Packet::makeResponse()
method). There is no provision for having multiple responders to a single request. Since the response is always generated before sendAtomic()
or sendFunctional()
returns, the requester can allocate the Packet object statically or on the stack.delete
and then new
(and gain the convenience of using makeResponse()
). However, this optimization is optional, and the requester must not rely on receiving the same Packet object back in response to a request. Note that when the responder is not the target device (as in a cache-to-cache transfer), then the target device will still delete the request packet, and thus the responding cache must allocate a new Packet object for its response. Also, because the target device may delete the request packet immediately on delivery, any other memory device wishing to reference a broadcast packet past point where the packet is delivered must make a copy of that packet, as the pointer to the packet that is delivered cannot be relied upon to stay valid.Timing requests simulate a real memory system, so unlike functional and atomic accesses their response is not instantaneous. Because the timing requests are not instantaneous, flow control is needed. When a timing packet is sent via sendTiming()
the packet may or may not be accepted, which is signaled by returning true or false. If false is returned the object should not attempt to sent anymore packets until it receives a recvRetry()
call. At this time it should again try to call sendTiming()
; however the packet may again be rejected. Note: The original packet does not need to be resent, a higher priority packet can be sent instead. Once sendTiming()
returns true, the packet may still not be able to make it to its destination. For packets that require a response (i.e. pkt->needsResponse()
is true), any memory object can refuse to acknowledge the packet by changing its result to Nacked
and sending it back to its source. However, if it is a response packet, this can not be done. The true/false return is intended to be used for local flow control, while nacking is for global flow control. In both cases a response can not be nacked.
Ranges in the memory system are handled by having devices that are sensitive to an address range provide an implementation for getAddrRanges
in their slave port objects. This method returns an AddrRangeList
of addresses it responds to. When these ranges change (e.g. from PCI configuration taking place) the device should call sendRangeChange()
on its slave port so that the new ranges are propagated to the entire hierarchy. This is precisely what happens during init()
; all memory objects call sendRangeChange()
, and a flurry of range updates occur until everyones ranges have been propagated to all busses in the system.