arch-amdgpu: improve page-walk caching and TLB coalescing by Basemism · Pull Request #2 · hal-uw/gap-gem5

Basemism · 2026-09-02T12:01:50Z

(Apologies for the incoming word vomit.)

This PR improves AMD GPU page-table walking and TLB request coalescing in two related parts:

Adds a neighbour page-walk cache for final-level PTEs returned in the same aligned fetch block as the requested PTE.
Adds page-size prediction and adaptive PTE-block coalescing.

PWC changes

The page-table walker now maintains two caches:

The existing PWC continues to cache intermediate page-table entries, but does not cache final-level PTEs. Final translations are already cached by the TLB, so storing them in the original PWC would duplicate entries and consume capacity intended for intermediate walk levels.
The neighbour PWC caches valid final-level PTEs returned in the same aligned memory fetch as the requested PTE. For example, a 128-byte page-table fetch contains sixteen 8-byte PTEs. The walker consumes the requested PTE immediately and inserts the other valid leaf PTEs into the neighbour PWC for later translations.

This makes use of data already returned by the page-table memory request. Translations for nearby pages may subsequently complete from the neighbour PWC without another memory access.

The page-table fetch width is configurable through pwc_fetch_bytes. Requests are aligned to that width, and the requested PTE is selected from the returned block. The configuration is validated to ensure that the width is a supported power-of-two multiple of the 8-byte PTE size.

Insertion is deferred until the walker determines whether an entry is intermediate or final-level:

Intermediate entries are inserted into the original PWC.
Valid neighbouring final-level entries are inserted into the neighbour PWC.
Invalid neighbouring PTEs are not inserted.

Page-table invalidations apply to both PWCs. The block-fetch and extraction behavior is implemented for both timing and functional page-table walks.

Separate statistics are provided for accesses, hits, misses, insertions, and invalidations in each PWC, along with the number of invalid neighbouring PTEs skipped.

TLB lookup changes

The TLB lookup path now checks every page size that the Vega page-table walker can install: 4 KiB, 2 MiB, 1 GiB (, and 512 though I haven't tested this path)

For each candidate page size, the virtual address is aligned to that page-size boundary before the TLB key is constructed. This allows any address within a large mapping to find the entry installed for the mapping’s base address.

Page-size prediction

Before translation completes, the coalescer must choose the address range over which requests may share a TLB lookup.

Previously, requests were initially grouped using a fixed assumed page size. If the completed translation returned a different size, the requests had to be removed and re-coalesced using the actual size. A workload that repeatedly used the other page size therefore repeated the same incorrect initial grouping for each lookup.

This PR adds a saturating predictor that chooses between 4 KiB and 2 MiB when constructing the initial coalescing group. The predictor is trained using the page size returned by completed translations.

If the prediction is incorrect, the affected requests are removed from the speculative group and re-coalesced using the actual page size. This preserves the previous fallback behavior while avoiding repeated re-coalescing when a workload exhibits stable page-size behavior.

Page sizes larger than the maximum supported coalescing granule are clamped to that granule. A 1 GiB or 512 GiB mapping therefore cannot serialize requests from the entire mapping behind a single coalescer entry.

Adaptive neighbour-PTE coalescing

Fetching adjacent leaf PTEs is most useful when nearby translation requests can share the resulting neighbour-PWC entry block.

The L3 TLB coalescer can widen a request group to cover the virtual pages represented by one pwc_fetch_bytes block. A second saturating predictor decides whether to apply this wider coalescing based on previous translation outcomes:

A TLB miss that performs a page-table memory fetch trains toward block-wide coalescing because the fetch supplies neighbouring leaf PTEs.
A direct TLB hit trains away from wider coalescing because no page walk or neighbour-PWC population was required.
A walk resolved by either PWC is distinguished from a walk that accesses page-table memory. A PWC hit therefore does not train the predictor as though it fetched and populated another block of neighbouring PTEs.

This is important because both a PWC hit and a page-table memory access pass through the walker, but only the memory access retrieves new neighbouring PTEs.

When speculation is incorrect, the implementation preserves request completion, retry handling, and coalescer FIFO accounting while requests are regrouped.

Copilot

Copilot was unable to review this pull request because the user who requested the review has reached their quota limit.

Extend the Vega page-table walker with a second page-walk cache for valid neighbouring final-level PTEs fetched in the same memory block. Keep the existing PWC for non-final walk levels and defer insertions until the walker knows whether an entry terminates the walk. Add pwc_fetch_bytes to control aligned page-table memory requests, propagate it to GPU, command-processor, and SDMA walkers, and validate that the width is a supported power-of-two multiple of a PTE. Track accesses, hits, misses, insertions, invalidations, and rejected invalid neighbours separately for the two caches.

Track Vega TLB lookup outcomes by page size and extend lookups to the large page sizes the walker can install. Probe additional page sizes during a walk and clamp page sizes used as coalescer keys so a giant mapping cannot serialize an unbounded address region. Add adaptive predictors for page size and coalescing distance, together with statistics for their decisions and outcomes. Preserve request completion and retry ordering while allowing coalescing granularity to follow observed TLB behavior.

mattsinc · 2026-09-11T06:23:56Z

@Basemism : at least from the tests we've been running, we've been seemingly seeing that there is no L3 TLB in Vega20 or MI300X GPUs. Are you seeing something different? Does this patch fundamentally rely on there being an L3 TLB? I suspect the same changes work regardless of if there is an L3 TLB or not?

@TomXia it would be good to get your feedback on this patch too, given that it will need to go on top of your changes.

mattsinc

I started going through this @Basemism , but either my browser is totally messed up, or the spacing is not correct in your PR. If you haven't, can you please run the checkin script that fixes spacing, etc.? This is really hard to read as is...

mattsinc · 2026-09-11T06:24:51Z

                        system.%s_tlb[%d].cpu_side_ports[0]"
                    % (name, index, name, index)
                )
+                # Give each Vega coalescer a handle to the TLB it feeds so the


before we integrate this into the mainline of gem5, Jason will almost definitely ask for a stblib version of this. It's fine to not worry about for today since we aren't using stdlib for the paper, but just an FYI

mattsinc · 2026-09-11T06:25:16Z

                        system.%s_tlb[%d].cpu_side_ports[0]"
                    % (name, index, name, index)
                )
+                # Give each Vega coalescer a handle to the TLB it feeds so the


per my comment on the PR itself, if there is no L3 TLB we probably want to move this to the L2 TLB?

I've already updated it for tlb l2, there may be some stale comments referencing l3. Will check

mattsinc · 2026-09-11T06:26:21Z

        assert(devmem);
        devmem->access(read);

+        // Extract the requested 8B entry from the fetched block.


I'm not sure if it's just my computer, but the spacing seems off on this whole file. Did you run this through the checkin scripts to ensure the gem5 coding style was followed?

mattsinc · 2026-09-11T06:27:05Z

+        pendingLineAddr = read->getAddr();
+        pendingLineIndex = lineIndex;
+        for (unsigned i = 0; i < walker->pwcFetchEntries; i++)
+            pendingLineEntries[i] = letoh(lineData[i]);


especially here -- either something is wrong with GitHub showing on my browser, or the spacing inside this for loop is wrong -- it looks like there is no indentation?

also what is letoh? Google says it's not in the C++ standard library, but I also don't see it anywhere in your patch ...

letoh is defined in src/sim/byteswap.hh it converts little-endian PTE words to host order.

mattsinc · 2026-09-11T06:29:54Z

+            pendingLineEntries[i] = letoh(lineData[i]);
+        pendingLineValid = true;
+
+        // Rewrite word 0 so stepWalk()'s getLE<uint64_t>() sees the


I'm not sure why rewriting word 0 is needed? Why does this affect the correct entry -- is this an LRU thing?

stepWalk() expects the PTE it is processing at the start of the packet data. For a wider aligned fetch, the PTE can be anywhere within that returned block.

We save the complete block for neighbour-PWC insertion. The requested PTE is placed at the packet’s expected position.

mattsinc · 2026-09-11T06:30:07Z

+
+        // Populate pending PWC metadata for deferred insertion in stepWalk()
+        Addr originalEntryAddr =
+            read->getAddr() + lineIndex * sizeof(uint64_t);


spacing definitely seems off here too

mattsinc · 2026-09-11T06:30:47Z

+    assert(idx < walker->pwcFetchEntries);
+    lineIndex = idx;
+    DPRINTF(GPUPTWalker, "Aligned PDE2 %#lx -> %#lx lineIndex %u\n",
+            pde2Addr, alignedAddr, lineIndex);


spacing issue here too

Basemism · 2026-09-11T08:25:40Z

@Basemism : at least from the tests we've been running, we've been seemingly seeing that there is no L3 TLB in Vega20 or MI300X GPUs. Are you seeing something different? Does this patch fundamentally rely on there being an L3 TLB? I suspect the same changes work regardless of if there is an L3 TLB or not?

@TomXia it would be good to get your feedback on this patch too, given that it will need to go on top of your changes.

I've already update the pr to align with Vishnus changes with 2 level tlb. It now activates line coalescing at tlb_level== 2. However, it is hard-coded to level 2 and would require a rebuild for if the tlb levels changes for future and historical GPU configs.

Basemism · 2026-09-11T08:54:00Z

I will recheck stale comments and spacing artefacts shortly

TomXia

Thanks @Basemism, I like this implementation, it is far more complete than mine. I would probably have done much the same given more time and a second shot at it.

Apart from the comments below, do you happen to have any memory performance numbers from before and after the patch? Happy to run the comparison on my side if you don't, so no worries.

TomXia

Thanks Basemism, I think these are functionally good to me.

mattsinc · 2026-09-13T20:15:17Z

I will recheck stale comments and spacing artefacts shortly

Sounds good, let me know when you are done with that and I'll reapprove

Basemism · 2026-09-14T08:14:53Z

I will recheck stale comments and spacing artefacts shortly

Sounds good, let me know when you are done with that and I'll reapprove

@mattsinc I think I'm done

Basemism · 2026-09-14T09:30:57Z

Thanks @Basemism, I like this implementation, it is far more complete than mine. I would probably have done much the same given more time and a second shot at it.

Apart from the comments below, do you happen to have any memory performance numbers from before and after the patch? Happy to run the comparison on my side if you don't, so no worries.

Thanks for the feedback, I really appreciate it. I do have some isolated performance stats, but they are fairly out of date and the workloads were not especially representative, so I’m not entirely happy with them.

It would be useful to compare pre-PWC, the current PWC impl. , and this improved version. I assume you already have numbers for the first two; if you could run the same suite for this version as well, that would be great.

Basemism requested review from TomXia, mattsinc and v-ramadas September 2, 2026 12:01

Basemism self-assigned this Sep 2, 2026

Basemism changed the title ~~Staging basem/pwc~~ arch-amdgpu: improve page-walk caching and TLB coalescing Sep 2, 2026

Basemism requested a balanced review from Copilot September 2, 2026 12:24

Copilot AI reviewed Sep 2, 2026

v-ramadas force-pushed the staging branch from 8ec0968 to adc9050 Compare September 9, 2026 19:47

Basemism force-pushed the staging-basem/pwc branch 2 times, most recently from 69d448f to 152067f Compare September 10, 2026 13:37

Basemism added 2 commits September 10, 2026 15:52

Basemism force-pushed the staging-basem/pwc branch from 152067f to f319f1c Compare September 10, 2026 13:52

mattsinc requested changes Sep 11, 2026

View reviewed changes

Basemism added 2 commits September 11, 2026 10:59

style: format page-walk caching changes

8630870

configs: clarify final-level GPU TLB comments

dcff9a1

Basemism requested a review from mattsinc September 11, 2026 09:03

TomXia reviewed Sep 12, 2026

View reviewed changes

Comment thread src/arch/amdgpu/vega/tlb.cc

TomXia reviewed Sep 12, 2026

View reviewed changes

Comment thread src/arch/amdgpu/vega/tlb.cc

TomXia reviewed Sep 12, 2026

View reviewed changes

Comment thread src/arch/amdgpu/vega/tlb_coalescer.hh Outdated

arch-amdgpu: derive line coalescing width from PWC fetch

6b0ec2f

TomXia approved these changes Sep 12, 2026

View reviewed changes

TomXia force-pushed the staging branch from 752ac80 to 998b8e3 Compare September 19, 2026 19:54

Conversation

Basemism commented Sep 2, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

PWC changes

TLB lookup changes

Page-size prediction

Adaptive neighbour-PTE coalescing

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Uh oh!

mattsinc commented Sep 11, 2026

Uh oh!

mattsinc left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Basemism commented Sep 11, 2026

Uh oh!

Basemism commented Sep 11, 2026

Uh oh!

TomXia left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

TomXia left a comment

Choose a reason for hiding this comment

Uh oh!

mattsinc commented Sep 13, 2026

Uh oh!

Basemism commented Sep 14, 2026

Uh oh!

Basemism commented Sep 14, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

Basemism commented Sep 2, 2026 •

edited

Loading