Skip to content

ci: pin Strix Halo jobs to the native runners - #146

Merged
rominf merged 1 commit into
mainfrom
ci-pin-strix-native-runners
Jul 28, 2026
Merged

ci: pin Strix Halo jobs to the native runners#146
rominf merged 1 commit into
mainfrom
ci-pin-strix-native-runners

Conversation

@rominf

@rominf rominf commented Jul 28, 2026

Copy link
Copy Markdown
Collaborator

Problem

"E2E tests (Strix Halo, Ubuntu)" has been intermittently failing on unrelated
PRs (#139, #140, #141), dying at its very first step with Permission denied
on mkdir. #143 passed only because it happened to be scheduled elsewhere.

Three self-hosted Strix Halo runners share the strix-halo label, and two of
them are Linux: the native Ubuntu host and a WSL host. The job's selector,
runs-on: [self-hosted, linux, strix-halo], matched both, so it landed on
whichever runner was free.

That job's setup hardcodes /home/ubuntu/actions-runner/... for HOME,
TMPDIR, and the pip cache — the native host's root partition is full and
that nvme is the only place with space. On the WSL host the runner executes
as a different user under a different home, so those mkdirs fail outright
and the job dies immediately. Failing runs report runner_name: strix-halo-wsl; the passing one reports strix-halo-ubuntu.

The nightly Strix job in nightly.yml copies the same hardcoded paths and the
same ambiguous selector, so it has the same latent failure — just less
visibly, since it is nightly and non-blocking.

Change

Add the native label to the runner selector for the Strix jobs, making
scheduling deterministic. The Windows jobs are already unambiguous in practice
(the WSL runner is labelled Linux, not Windows), but they are pinned too so
every Strix selector reads the same way and stays correct if the WSL host ever
gains a Windows label.

The comment above the Ubuntu job's env: block now notes that the hardcoded
paths are the reason for the pin, so the coupling is obvious to the next reader.

Notes

  • Requires the native label on strix-halo-ubuntu and strix-halo-windows;
    this is already applied and live.
  • Deliberately out of scope: making the hardcoded paths runner-agnostic, and
    adding any job for the WSL runner. Both are separate follow-ups.

The "E2E tests (Strix Halo, Ubuntu)" job intermittently failed at its very
first step with "Permission denied" on mkdir, red-X-ing unrelated PRs.

Three self-hosted Strix Halo runners share the `strix-halo` label, and two
of them are Linux: the native Ubuntu host and a WSL host. `runs-on:
[self-hosted, linux, strix-halo]` matched both, so the job was scheduled
onto whichever was free. Its setup hardcodes /home/ubuntu/actions-runner
paths — the only partition with free space on the native host — which do
not exist and are not writable on the WSL host, where the runner executes
as a different user under a different home. Landing on the native runner
passed; landing on WSL failed immediately. The same latent mismatch exists
in nightly.yml's Strix job, which copies the same paths.

Add the `native` label to the runner selector for these jobs so scheduling
is deterministic. The Windows jobs are already unambiguous in practice
(the WSL runner is labelled Linux, not Windows), but they are pinned too
so every Strix selector reads the same way and stays correct if the WSL
host ever gains a Windows label.

The hardcoded paths are left as-is here; making them runner-agnostic is a
separate change.

Signed-off-by: Roman Inflianskas <Roman.Inflianskas@amd.com>
@rominf
rominf added this pull request to the merge queue Jul 28, 2026
Merged via the queue into main with commit d2662d5 Jul 28, 2026
21 checks passed
@rominf
rominf deleted the ci-pin-strix-native-runners branch July 28, 2026 12:12
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants