Skip to content

Qwen 3.5 MoE: Add Metal source transformations#18879

Open
manuelcandales wants to merge 3 commits intogh/manuelcandales/173/headfrom
gh/manuelcandales/174/head
Open

Qwen 3.5 MoE: Add Metal source transformations#18879
manuelcandales wants to merge 3 commits intogh/manuelcandales/173/headfrom
gh/manuelcandales/174/head

Conversation

@manuelcandales
Copy link
Copy Markdown
Contributor

Adds metal_source_transformations.py with module replacements for Metal:

  • FusedMoEExperts -> MetalMoEExperts (two metal::gather_qmv calls with
    SiLU gating, replacing torch.ops.triton.fused_moe)
  • GatedDeltaNet -> metal::gated_delta_rule custom op (replaces both the
    T=1 native path and T>1 Triton kernel)
  • FullAttention -> removes turboquant codepath, keeps standard SDPA
  • SparseMoE -> removes .float() cast on expert_weights

Also includes quantize_experts_metal() which quantizes expert weights to
MLX affine INT4 format (unsigned uint4 with scale + bias per group),
compatible with the Metal gather_qmv kernel.

Authored with Claude.

[ghstack-poisoned]
@manuelcandales manuelcandales requested a review from lucylq as a code owner April 14, 2026 16:25
@pytorch-bot
Copy link
Copy Markdown

pytorch-bot bot commented Apr 14, 2026

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/executorch/18879

Note: Links to docs will display an error until the docs builds have been completed.

❗ 1 Active SEVs

There are 1 currently active SEVs. If your PR is affected, please view them below:

✅ You can merge normally! (4 Unrelated Failures)

As of commit d80da37 with merge base 5707e2a (image):

FLAKY - The following jobs failed but were likely due to flakiness present on trunk:

BROKEN TRUNK - The following jobs failed but were present on the merge base:

👉 Rebase onto the `viable/strict` branch to avoid these failures

This comment was automatically generated by Dr. CI and updates every 15 minutes.

[ghstack-poisoned]
[ghstack-poisoned]
@manuelcandales manuelcandales requested review from mergennachin and metascroy and removed request for lucylq April 15, 2026 15:14
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant