mem-ruby,configs: add a configurable GPU L3 to GPU_VIPER by Basemism · Pull Request #9 · hal-uw/gap-gem5

Basemism · 2026-09-02T14:07:13Z

Add an optional memory-side GPU L3 to the GPU directories in MOESI_AMD_Base. The feature is disabled by default through --use-gpu-l3

The default inclusive policy:

Retains L3 entries on ordinary GPU and DMA read hits.
Fills the L3 from memory responses.
Updates the L3 when a dirty TCC probe supplies newer data.
Writes dirty data through to memory, leaving clean L3 copies that can be silently evicted.
Handles DMA reads, writes, atomics, writebacks, and L3-hit completion paths without leaving stale copies.

This policy is inferred from the Infinity Cache organization described on page 10 of the AMD CDNA 3 Architecture white paper (https://www.amd.com/content/dam/amd/en/documents/instinct-tech-docs/white-papers/amd-cdna-3-white-paper.pdf)

--l3-exclusive retains an experimental victim-cache policy for configurations that need an exclusive hierarchy.

Also make TCC bank counts and per-slice L3 bank counts independently configurable through --tcc-num-banks and --l3-num-banks. Place the L3 set-index bits above the directory-selection bits and fix construction when --numa-high-bit is supplied.

Copilot

🟡 Changes recommended

There are correctness issues in --numa-high-bit handling for start_index_bit, and the L3Exclusive eviction path can drop dirty data (plus a confirmed argparse default type hazard now affecting new TCC bank usage).

Once you've addressed the issues Copilot identified, you can request another Copilot review.

Pull request overview

This PR adds an optional, configurable memory-side GPU L3 (Infinity Cache–like) behavior to the MOESI_AMD_Base GPU directory path used by GPU_VIPER, and extends GPU_VIPER configurability for directory/TCC/L3 banking and address-bit placement.

Changes:

Add GPU L3 retention vs. consume behavior (inclusive vs. experimental exclusive) in MOESI_AMD_Base-dir.sm, plus L3 fill behavior on relevant completion paths.
Add GPU_VIPER CLI options to enable GPU L3 and exclusive mode, and make TCC/L3 per-slice bank counts independently configurable.
Adjust L3 indexing bit placement intent by explicitly setting start_index_bit in GPU_VIPER directory construction.

File summaries

File	Description
src/mem/ruby/protocol/MOESI_AMD_Base-dir.sm	Implements GPU L3 retention/fill policies and adds `L3Exclusive` handling hooks in the directory SLICC machine.
configs/ruby/GPU_VIPER.py	Adds GPU L3 config flags, separates bank-count configurability, and sets L3 `start_index_bit` during directory construction.

Review details

Suppressed comments (3)

configs/ruby/GPU_VIPER.py:605

options.numa_high_bit changes which bits are used for directory interleaving (intlvHighBit=numa_bit). Setting L3CacheMemory.start_index_bit to block_size_bits + dir_bits can overlap with/underlap the actual directory-selection bits when --numa-high-bit is provided. To ensure the L3 set-index bits are always above the directory-selection bits, derive this from numa_bit (the current interleave high bit) instead.

        dir_cntrl = DirCntrl(noTCCdir=True, TCC_select_num_bits=TCC_bits)
        dir_cntrl.create(options, dir_ranges, ruby_system, system)
        dir_cntrl.L3CacheMemory.start_index_bit = block_size_bits + dir_bits
        dir_cntrl.number_of_TBEs = options.num_tbes

configs/ruby/GPU_VIPER.py:675

Same issue as the CPU-side directories: start_index_bit should be derived from the interleave high bit (numa_bit) so the L3 set-index bits remain above the directory-selection bits regardless of how the address is interleaved.

        dir_cntrl.create(options, [addr_range], ruby_system, system,
                         num_dirs=options.dgpu_num_dirs)
        dir_cntrl.L3CacheMemory.start_index_bit = block_size_bits + dir_bits
        dir_cntrl.number_of_TBEs = options.num_tbes

src/mem/ruby/protocol/MOESI_AMD_Base-dir.sm:1297

Same eviction-path issue as above: the silent-drop path should be limited to the non-exclusive GPU L3 policy. In L3Exclusive mode, evicted victim-cache entries may be dirty and need a writeback.

        if (L3CacheMemory.cacheAvail(address) == false) {
          Addr victim := L3CacheMemory.cacheProbe(address);
          if (GPUonly) {
            // GPU L3 (Infinity Cache) is read-only — evicted entries are
            // always clean, so silent drop is correct.

Files reviewed: 2/2 changed files
Comments generated: 3
Review effort level: Lite

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

Basemism · 2026-09-02T14:20:47Z

        if (L3CacheMemory.cacheAvail(address) == false) {
          Addr victim := L3CacheMemory.cacheProbe(address);
-          CacheEntry victim_entry := static_cast(CacheEntry, "pointer",
-                                                 L3CacheMemory.lookup(victim));
-          enqueue(memQueue_out, MemoryMsg, to_memory_controller_latency) {
-            out_msg.addr := victim;
-            out_msg.Type := MemoryRequestType:MEMORY_WB;
-            out_msg.Sender := machineID;
-            out_msg.MessageSize := MessageSizeType:Writeback_Data;
-            out_msg.DataBlk := victim_entry.DataBlk;
+          if (GPUonly) {
+            // GPU L3 (Infinity Cache) is read-only — evicted entries are
+            // always clean, so silent drop is correct.


VicDirty runs d_writeDataToMemory before al_allocateL3Block. The dirty data reaches memory before entering L3, so the resulting L3 copy is clean. Silent GPU-L3 eviction is safe in both policies.

how do you guarantee it reaches memory before entering the L3? Is there an ACK you wait on?

There is no ACK, but I don't think its needed (?) because the writeback is issued before any later memory access to that address. L3 holds only a clean replica, it never becomes the sole owner of dirty data.

Add a configurable GPU L3 cache to the GPU-only MOESI_AMD_Base directory and support both inclusive and exclusive allocation policies. Keep the feature disabled by default so existing GPU_VIPER configurations retain their current behavior. Handle L3 hits, fills, writebacks, atomics, DMA reads, and probe responses without leaving stale copies behind. Preserve inclusive L3 entries on ordinary GPU and DMA reads, invalidate or update an L3 entry when a dirty TCC probe supplies newer data, and retain the exclusive mode as an explicit option. Also compute directory and block index widths before processing --numa-high-bit so configurations using an explicit NUMA bit can construct the L3 cache safely.

mattsinc · 2026-09-11T06:19:31Z

@abmerop this is the patch we discussed earlier

mattsinc

@v-ramadas : I believe we'd found that the L2 should be WB, but I don't know if we found the same for the L3 (yet) or not? It is a little weird to me that the L2 would be WB but the L3 WT ... but @Basemism is indicating the documentation says to do it that way. Thoughts?

mattsinc · 2026-09-11T07:01:33Z

        self.size = MemorySize(size)
        self.assoc = assoc
-        self.replacement_policy = TreePLRURP()
+        self.replacement_policy = BRRIPRP()


should we update this to use the variant Jarvis integrated into the mainline where the config tells us what RP to use? Then we can just pick BRRIP from there, and it's much more extensible ...

mattsinc · 2026-09-11T07:03:21Z

-                256 / options.num_tccs
-            )  # number of data banks
-            self.tagArrayBanks = 256 / options.num_tccs  # number of tag banks
+            self.dataArrayBanks = options.tcc_num_banks  # number of data banks


@v-ramadas were you relying on the L2 banking policy not changing in any of your changes?

@Basemism since this doesn't seem to have anything to do with adding an L3, this should really be a separate PR.

Agreed. The TCC banking changes are not required for the L3. Ill prepare a pr for this

My changes don't really rely on the number of banks. I think this doesn't affect me.

mattsinc · 2026-09-11T07:04:30Z

    tagArrayBanks = 16

-    def create(self, options, ruby_system, system):
+    def create(self, options, ruby_system, system, num_dirs=None):


most of the changes here also seem unrelated to adding an L3?

Agreed. The TCC banking changes are not required for the L3. Ill prepare a pr for this

mattsinc · 2026-09-11T07:05:03Z

        self.tagAccessLatency = options.l3_tag_latency
        self.resourceStalls = False
-        self.replacement_policy = TreePLRURP()
+        self.replacement_policy = BRRIPRP()


same thing here -- we should use Jarvis' configurable RP code (like @Basemism you did for the ScalarCache) to make this easier to play around with.

mattsinc · 2026-09-11T07:05:40Z

        dir_cntrl.number_of_TBEs = options.num_tbes
-        dir_cntrl.useL3OnWT = False
+        dir_cntrl.GPUonly = True
+        dir_cntrl.useL3OnWT = options.use_gpu_l3


@Basemism do you support a WB L3 in this patch? If not, there probably needs to be a warning here.

Not currently, I'll prepare a warning shortly

mattsinc · 2026-09-11T07:06:21Z

        tbe.MemData := true;
-        L3CacheMemory.deallocate(address);
+        if (GPUonly && !L3Exclusive) {
+          // CDNA 3 Infinity Cache is a non-destructive memory-side cache,


this comment doesn't seem to have anything to do with setting MRU?

The comment describes the retain-on-hit policy, not setMRU() itself

mattsinc · 2026-09-11T07:07:07Z

-        }
-      }
-    }
+         out_msg.addr := address;


it's hard for me to tell, but it seems like a space was accidentally removed here?

Yes, its whitespace... I’ll restore it.

mattsinc · 2026-09-11T07:17:56Z

-            out_msg.MessageSize := MessageSizeType:Writeback_Data;
-            out_msg.DataBlk := victim_entry.DataBlk;
+          if (GPUonly) {
+            // GPU L3 (Infinity Cache) is read-only — evicted entries are


where did you find that it's read only?

"The AMD Infinity Cache is an entirely new and massive structure for the AMD CDNA 3 architecture that
boosts generational performance and efficiency by increasing cache bandwidth and reducing the number
of off-chip memory accesses. Typically, GPU caches are more closely aligned with and physically co-located
with memory controllers and that is especially true for the AMD CDNA 3 architecture. The AMD Infinity Cache
was carefully designed as a shared memory-side cache, meaning that it caches the contents of memory and
cannot hold dirty data evicted from a lower level cache. This has two significant benefits. First, the AMD
Infinity Cache doesn’t participate in coherency and does not have to absorb or handle any snoop traffic,
which significantly improves efficiency and reduces the latency of snooping from lower level caches. Second,
it can actually hold nominally uncacheable memory such as buffers for I/O."

https://www.amd.com/content/dam/amd/en/documents/instinct-tech-docs/white-papers/[amd-cdna-3-white-paper.pdf](https://www.amd.com/content/dam/amd/en/documents/instinct-tech-docs/white-papers/amd-cdna-3-white-paper.pdf)

@mattsinc, I agree with @Basemism; the L3 seems to be write-through, no write-allocate. It's main purpose seems to be amplify read BW. L3 is also not coherent since L2 is writeback. It mainly acts as a directory with some read data cached.

mattsinc · 2026-09-11T07:21:24Z

@Basemism also we'll have to figure out how to integrate the notion of scopes into your L3 code. @abmerop I know you have a patch on these we need to integrate.

About read-only L3, I'm a bit confused though, because Table 49 (https://www.amd.com/content/dam/amd/en/documents/instinct-tech-docs/instruction-set-architectures/amd-instinct-mi300-cdna3-instruction-set-architecture.pdf) seems to show that SC1/SC0 writes should hit and be kept in the L3, which clashes with the whitepaper @Basemism points to...

v-ramadas · 2026-09-11T23:02:39Z

@Basemism also we'll have to figure out how to integrate the notion of scopes into your L3 code. @abmerop I know you have a patch on these we need to integrate.

About read-only L3, I'm a bit confused though, because Table 49 (https://www.amd.com/content/dam/amd/en/documents/instinct-tech-docs/instruction-set-architectures/amd-instinct-mi300-cdna3-instruction-set-architecture.pdf) seems to show that SC1/SC0 writes should hit and be kept in the L3, which clashes with the whitepaper @Basemism points to...

The whitepaper says "The AMD Infinity Cache
was carefully designed as a shared memory-side cache, meaning that it caches the contents of memory and
cannot hold dirty data evicted from a lower level cache". I wonder if this means that L2 evictions are not cached in L3, but any write command with SC0+SC1 could still hit in the L3 provided a previous read operation cached the data there?

abmerop · 2026-09-16T18:24:26Z

+    // Write memory for data that must be persisted:
+    // WT/atomic updates, or dirty data returned by probes.
+    // Clean read data does not need writeback — memory already has it.
+    if (tbe.wtData || tbe.atomicData || tbe.Dirty) {


Is this an unrelated bug? For example, if !GPUonly it should still do this for dirty TBEs?

The old condition wrote clean data back to memory, but skipped dirty data. I think this is a general bug and is shared, and not GPU-only.

abmerop · 2026-09-16T18:33:03Z

  }

-  transition(BDW_M, MemData, U) {
+  transition(BDR_M, L3Hit, U) {L3TagArrayWrite, L3DataArrayWrite} {


Is this a missing transition, or is it require for this new support? Same with (BDW_M, L3Hit, U) and (BL2_M, L3Hit, U)

They were missing + required. I ran into these missing transitions while testing.

Basemism self-assigned this Sep 2, 2026

Basemism requested review from TomXia, mattsinc and v-ramadas and a lite review from Copilot September 2, 2026 14:07

Copilot started reviewing on behalf of Basemism September 2, 2026 14:08 View session

Copilot AI reviewed Sep 2, 2026

View reviewed changes

Basemism force-pushed the staging-basem/gpu-l3 branch from 1eaa94d to 16742eb Compare September 2, 2026 14:24

v-ramadas force-pushed the staging branch from 8ec0968 to adc9050 Compare September 9, 2026 19:47

Basemism force-pushed the staging-basem/gpu-l3 branch from 16742eb to 387a0a7 Compare September 10, 2026 13:57

Basemism force-pushed the staging-basem/gpu-l3 branch from 387a0a7 to 114ae95 Compare September 10, 2026 13:58

mattsinc requested a review from abmerop September 11, 2026 06:19

mattsinc requested changes Sep 11, 2026

View reviewed changes

Basemism force-pushed the staging-basem/gpu-l3 branch from 56d812f to 0405770 Compare September 11, 2026 09:36

Basemism added 2 commits September 11, 2026 11:37

configs: make L3 cache bank count configurable

49fd9cc

style: fix L3 memory request indentation

90e2138

Basemism force-pushed the staging-basem/gpu-l3 branch from 0405770 to 90e2138 Compare September 11, 2026 09:37

configs: warn that GPU L3 is write-through

54289b5

abmerop reviewed Sep 16, 2026

View reviewed changes

TomXia force-pushed the staging branch from 752ac80 to 998b8e3 Compare September 19, 2026 19:54

Conversation

Basemism commented Sep 2, 2026

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

🟡 Changes recommended

Suppressed comments (3)

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

mattsinc commented Sep 11, 2026

Uh oh!

mattsinc left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

v-ramadas Sep 11, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

mattsinc commented Sep 11, 2026

Uh oh!

v-ramadas commented Sep 11, 2026

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Basemism Sep 17, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

v-ramadas Sep 11, 2026 •

edited

Loading

Basemism Sep 17, 2026 •

edited

Loading