metal: threadgroup-memory alignment and expert-index width fixes

[tarruda]

Overview

Extracted 2 metal fixes from #27879 as requested by @ggerganov

Requirements

• I have read and agree with the contributing guidelines
• AI usage disclosure: Yes

[ggerganov]

@tarruda Thanks. And are you sure this test fails on master? It passes on my end without the metal changes in the PR.

[tarruda]

Replied here

Seems like it requires some validation to be enabled for it to be detected on master

[ggerganov]

I have extracted this change into: #27951

[ggerganov]

This change does not seem necessary, so for now not merging it. That is unless the AI can demonstrate an issue with it that I am not seeing.

[tarruda]

@ggerganov in the interest of full disclosure (in case it was not clear from my previous PRs that touch any of the ML things): I have no idea what I'm doing here. I'm merely a GPT prompter when it comes to any ML/Metal related code. I . Entirely possible this is all just hallucination.

With that said, these issues were surfaced by GPT 5.6 Sol as metal correctness fixes that were uncovered by my Qwen4exp exploration. I have forwarded your comments to it, and here is what is told me:

Yes. ggerganov is correct that the added test passes on master, but that does not invalidate the fixes. The failures only appear with Metal API validation enabled.

   Change                          Unpatched, normal run           Unpatched with MTL_DEBUG_LAYER=1                    Patched with validation
  ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━  ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━  ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━  ━━━━━━━━━━━━━━━━━━━━━━━━━
   Threadgroup memory alignment    Qwen4Exp passes                 Aborts: length(4) must be a multiple of 16 bytes    Qwen4Exp passes
  ──────────────────────────────  ──────────────────────────────  ──────────────────────────────────────────────────  ─────────────────────────
   tiitg: ushort -> uint           Added MUL_MAT_ID test passes    Aborts: 65536 must be <= 65535 for tiitg            Test passes

  I reproduced these on an Apple M1 Ultra using Metal code identical to the PR’s master base.

  My assessment:

  - The 16-byte alignment fix is clearly justified. Qwen4Exp directly exposed an existing Metal API violation. Padding centrally is also consistent with another Metal caller that already documents
    the 16-byte requirement.

  - The uint fix is also effective, but it addresses Metal validation of a Qwen-sized dispatch rather than a demonstrated numerical error in normal execution. The local threadgroup only has 128
    threads, so this looks partly like validation-layer compatibility, but using the canonical wider type is harmless.

  - The PR’s current test is not a regression test under standard CI. Apple CI enables GGML_METAL_SHADER_DEBUG, but not Metal API validation. Consequently, that test passes both before and after the
    change.

  - The fixes should therefore be described as pre-existing Metal API/validation problems uncovered by Qwen4Exp, not as ordinary Qwen numerical-correctness failures.

  The strongest improvement would be a targeted test invocation with MTL_DEBUG_LAYER=1. Otherwise, ggerganov cannot reproduce the failure from the test as currently configured.

  All temporary diagnostic changes have been removed; only the previously existing untracked comparison report remains.

[ggerganov]

Ok, got it. I can see the error with MTL_DEBUG_LAYER=1. Seems benign, but might be useful to do it like this so that we can see other potential errors in the future. I'll consider what to do from here - thanks.

@ggerganov in the interest of full disclosure (in case it was not clear from my previous PRs that touch any of the ML things): I have no idea what I'm doing here. I'm merely a GPT prompter when it comes to any ML/Metal related code. I . Entirely possible this is all just hallucination.

Yes, no problem at all. I've already accepted that the AI is going to obsolete our job soon. Just trying to understand and learn a few more things while still can.

[tarruda]

I'll consider what to do from here - thanks.

Ok closing this one then. LMK if I can do anything else to help.

Yes, no problem at all. I've already accepted that the AI is going to obsolete our job soon. Just trying to understand and learn a few more things while still can.

😢

This is a side discussion, but no matter how good AI gets at programming GPU kernels, it still cannot always make good judgement calls, especially when the whole architecture of a system is involved.

When I'm prompting LLMs, I try to make it as combative to my ideas as possible, but I still see a lot of sycophancy, basic mistakes and overall slop, even on SOTA models like GPT 5.6 Sol.

Obviously I cannot predict the future, but with the current technology, it is hard to see how a frontier model will be able to obsolete a domain expert like you paired with a small local model for doing grunt work/investigations. So keep up the good work!