Skip to content

[Raiden Weight Sync 2/7] Support inhomogeneous layer cycle interval in raiden_unscan - #5167

Merged
copybara-service[bot] merged 1 commit into
mainfrom
yixuann-m1-raiden-unscan
Sep 9, 2026
Merged

[Raiden Weight Sync 2/7] Support inhomogeneous layer cycle interval in raiden_unscan#5167
copybara-service[bot] merged 1 commit into
mainfrom
yixuann-m1-raiden-unscan

Conversation

@YixuanWang-99

@YixuanWang-99 YixuanWang-99 commented Sep 9, 2026

Copy link
Copy Markdown
Collaborator

Overview

Part of the stacked Raiden weight-sync enablement PR chain replacing #5089.

Stack:

  • [M7] [Raiden Weight Sync 1/7] Add cross-repo drift guard and FFI resolution tests #5166: Cross-repo drift guard
  • [M1] AI-Hypercomputer/maxtext (this PR): Inhomogeneous layer cycle support in raiden_unscan
  • [M2] AI-Hypercomputer/maxtext: MoE 128-lane layout and padding (pairs with Tunix T5)
  • [M3] AI-Hypercomputer/maxtext: Target-free streaming weight conversion
  • [M5] AI-Hypercomputer/maxtext: Shared expert gate weight_dtype fix
  • [M4] AI-Hypercomputer/maxtext: TrainingEngine Raiden FFI integration (pairs with Tunix T2a)
  • [M6] AI-Hypercomputer/maxtext: Rollout cleanup

Details

  • Supports cycle_interval in unscan_layers for models with periodic inhomogeneous layers (e.g. Qwen3.5 GDN).
  • Review thread addressed: When slot is None and cycle_interval > 1, raises a descriptive ValueError naming the key and the missing cycle-slot prefix instead of misleadingly attributing the failure to num_layers.
  • Adds unit tests in tests/unit/raiden_unscan_test.py.

Verification

  • pytest tests/unit/raiden_unscan_test.py (11/11 passed).

Checklist

Before submitting this PR, please make sure (put X in square brackets):

  • I have performed a self-review of my code. For an optional AI review, add the gemini-review label.
  • I have necessary comments in my code, particularly in hard-to-understand areas.
  • I have run end-to-end tests tests and provided workload links above if applicable.
  • I have made or will make corresponding changes to the doc if needed, including adding new documentation pages to the relevant Table of Contents (toctree directive) as explained in our documentation.

@gemini-code-assist gemini-code-assist Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request introduces support for inhomogeneous layer cycles in unscan_layers by adding a cycle_interval parameter, which allows scanning blocks of heterogeneous layers and folding the slot index into the physical layer number. It also adds a comprehensive unit test suite for the unscan_layers functionality. The review feedback suggests optimizing the regular expression matching to avoid redundant calls, adding validation to ensure the parsed slot index is strictly less than the cycle interval to prevent silent weight overwriting, and expanding the unit tests to cover the new cycle_interval logic and its error paths.

Comment thread src/maxtext/integration/tunix/weight_mapping/raiden_unscan.py
Comment thread tests/unit/raiden_unscan_test.py

@igorts-git igorts-git left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

please address review bot comments. I think they are meaningful.

@YixuanWang-99
YixuanWang-99 force-pushed the yixuann-m1-raiden-unscan branch from 683a591 to a5e2f1b Compare September 9, 2026 17:56
Comment thread tests/unit/raiden_unscan_test.py
@codecov

codecov Bot commented Sep 9, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.

📢 Thoughts on this report? Let us know!

@YixuanWang-99
YixuanWang-99 force-pushed the yixuann-m1-raiden-unscan branch from a5e2f1b to 838de91 Compare September 9, 2026 19:05
@khatwanimohit
khatwanimohit force-pushed the yixuann-m1-raiden-unscan branch from 838de91 to fc0f48a Compare September 9, 2026 19:12
@YixuanWang-99
YixuanWang-99 force-pushed the yixuann-m1-raiden-unscan branch from fc0f48a to c49c431 Compare September 9, 2026 19:13
@khatwanimohit
khatwanimohit force-pushed the yixuann-m1-raiden-unscan branch 2 times, most recently from 4cbb2eb to d15d159 Compare September 9, 2026 20:20
@YixuanWang-99
YixuanWang-99 force-pushed the yixuann-m1-raiden-unscan branch 2 times, most recently from 3a654c0 to d2947fd Compare September 9, 2026 20:53
@YixuanWang-99
YixuanWang-99 changed the base branch from yixuann-m7-drift-guard to main September 9, 2026 20:53
@khatwanimohit
khatwanimohit force-pushed the yixuann-m1-raiden-unscan branch from d2947fd to 0172b97 Compare September 9, 2026 21:23
…scan

- Support cycle_interval in unscan_layers
- Raise descriptive ValueError naming the key and missing cycle-slot prefix when slot is None and cycle_interval > 1
- Add unit tests for unscan_layers and inhomogeneous layer cycles
@khatwanimohit
khatwanimohit force-pushed the yixuann-m1-raiden-unscan branch from 0172b97 to 309aade Compare September 9, 2026 21:24
@copybara-service
copybara-service Bot merged commit b1f02c8 into main Sep 9, 2026
60 of 61 checks passed
@copybara-service
copybara-service Bot deleted the yixuann-m1-raiden-unscan branch September 9, 2026 22:49
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants