commit c8c406bc305a6bf52304d5f158f5d8def0dea51f
author Alec Roelke <ar4jc@virginia.edu> Sun Apr 09 15:23:23 2017 -0400
committer Alec Roelke <ar4jc@virginia.edu> Mon May 15 19:30:14 2017 +0000
tree e0a014c7898055ce4ca13f00fe685c9fff0dce85
parent 8ced1bd0b0f9890992bcf9f517df5e33839621c7

cpu: fix problem with forwarding and locked load

If a (regular) store is followed closely enough by a locked load that
overlaps, the LSQ will forward the store's data to the locked load and
never tell the cache about the locked load.  As a result, the cache will
not lock the address and all future store-conditional requests on that
address will fail.  This patch fixes that by preventing forwarding if
the memory request is a locked load and adding another case to the LSQ
forwarding logic that delays the locked load request if a store in the
LSQ contains all or part of the data that is requested.

[Merge second and last if blocks because their bodies are the same.]

Change-Id: I895cc2b9570035267bdf6ae3fdc8a09049969841
Reviewed-on: https://gem5-review.googlesource.com/2400
Reviewed-by: Jason Lowe-Power <jason@lowepower.com>
Reviewed-by: Nikos Nikoleris <nikos.nikoleris@arm.com>
Reviewed-by: Tony Gutierrez <anthony.gutierrez@amd.com>
Maintainer: Jason Lowe-Power <jason@lowepower.com>

src/cpu/o3/lsq_unit.hh

diff --git a/src/cpu/o3/lsq_unit.hh b/src/cpu/o3/lsq_unit.hh
index 10d4966..9d88530 100644
--- a/src/cpu/o3/lsq_unit.hh
+++ b/src/cpu/o3/lsq_unit.hh
@@ -671,8 +671,9 @@
             (req->getVaddr() + req->getSize()) >
             storeQueue[store_idx].inst->effAddr;
 
-        // If the store's data has all of the data needed, we can forward.
-        if ((store_has_lower_limit && store_has_upper_limit)) {
+        // If the store's data has all of the data needed and the load isn't
+        // LLSC, we can forward.
+        if (store_has_lower_limit && store_has_upper_limit && !req->isLLSC()) {
             // Get shift amount for offset into the store's data.
             int shift_amt = req->getVaddr() - storeQueue[store_idx].inst->effAddr;
 
@@ -707,11 +708,18 @@
 
             ++lsqForwLoads;
             return NoFault;
-        } else if ((store_has_lower_limit && lower_load_has_store_part) ||
-                   (store_has_upper_limit && upper_load_has_store_part) ||
-                   (lower_load_has_store_part && upper_load_has_store_part)) {
+        } else if (
+                (!req->isLLSC() &&
+                 ((store_has_lower_limit && lower_load_has_store_part) ||
+                  (store_has_upper_limit && upper_load_has_store_part) ||
+                  (lower_load_has_store_part && upper_load_has_store_part))) ||
+                (req->isLLSC() &&
+                 ((store_has_lower_limit || upper_load_has_store_part) &&
+                  (store_has_upper_limit || lower_load_has_store_part)))) {
             // This is the partial store-load forwarding case where a store
-            // has only part of the load's data.
+            // has only part of the load's data and the load isn't LLSC or
+            // the load is LLSC and the store has all or part of the load's
+            // data
 
             // If it's already been written back, then don't worry about
             // stalling on it.