Skip to content

Codegen restructure: align with trainer sequence - #1394

Merged
github-actions[bot] merged 1 commit into
mindspore-ai:masterfrom
shawny999:codegen_v2
Nov 10, 2026
Merged

github-actions[bot] merged 1 commit into
mindspore-ai:masterfrom
shawny999:codegen_v2

Conversation

@shawny999

@shawny999 shawny999 commented Oct 31, 2026 •

Copy link
Copy Markdown
Contributor

Paired: GitHub #1394 ↔ GitCode !1876

What type of PR is this?

/kind feature
/kind refactor

What does this PR do / why do we need it:

This PR restructures HyperParallel Codegen into an offline, device-free model artifact generation workflow.

The main changes include:

  • Add a single-process Codegen CLI for generating and checking model bundles without starting distributed training or occupying NPUs.
  • Generate model source files, readable diffs, and frozen parallel-plan metadata from the same YAML configuration used by native training.
  • Preserve native module replacements, CP/EP local-compute patches, boundary information, and parameter placement rules in generated artifacts.
  • Reuse the native model construction and sharding-plan pipeline through isolated Codegen overlays, without adding model-specific behavior to the generic Trainer or planner.
  • Move model-specific source compatibility and sharding rules into model-owned adapters.
  • Add rank-0 Trainer sidecar support while keeping Codegen generation failures isolated from native training.
  • Remove legacy runtime, backend, spec, loader, offline-mesh, and generated-training entry paths that are no longer used.
  • Add bundle structure, snapshot, replacement, session, source-composition, native-isolation, and parallel-strategy regression tests.
  • Update the user guide for the new CLI generation and artifact-checking workflow.

The implementation has been validated with fresh bundle generation and preflight checks for the following model families:

  • DeepSeek-V3
  • DeepSeek-V3.2
  • DeepSeek-V4.1
  • Kimi-K2.6
  • Kimi-K3
  • Kimi-Linear
  • Qwen2-MoE
  • Qwen3
  • Qwen3.5
  • Qwen3-MoE
  • Qwen3-VL-MoE

Qwen3-VL-MoE also adds a model-owned visual-tower sharding rule so both native training and Codegen can classify all trainable parameters without weakening the generic planner’s strict coverage checks.

Which issue(s) this PR fixes:

#613

Code review checklist [illustration]:

  • whether to verify the function's return value (It is forbidden to use void to mask the return values of security functions and self-developed functions. C++ STL functions can be masked if there is no problem)
  • Whether to comply with SOLID principle / Demeter's law
  • Whether there is UT test case && the test case is a valid (if there is no test case, please explain the reason)
  • Whether the API change is involved
  • Whether official document modification is involved

Special notes for your reviewers:

  • The Codegen CLI and legacy Codegen integration APIs are changed as part of this refactor.
  • Native training YAML does not require a Codegen-specific switch.
  • Codegen generation is single-process and does not initialize HCCL/NCCL.
  • Generated bundles are validation artifacts and are not included in this PR.
  • Model-specific compatibility and parameter-role rules remain under their corresponding model adapters.
  • The generic Trainer and sharding planner do not contain model-name-specific branches.

@CLAassistant

CLAassistant commented Oct 31, 2026 •

Copy link
Copy Markdown

CLA assistant check
All committers have signed the CLA.

@shawny999
shawny999 force-pushed the codegen_v2 branch 10 times, most recently from e553963 to 86cb291 Compare November 3, 2026 09:45
@shawny999

Copy link
Copy Markdown
Contributor Author

/retest

@mindspore-ai-bot

Copy link
Copy Markdown

Gate trigger failed: please fix the /check-pr findings and obtain the pr-check-pass label before retrying.

@shawny999

Copy link
Copy Markdown
Contributor Author

/check-pr

@shawny999

Copy link
Copy Markdown
Contributor Author

/retest

@shawny999

Copy link
Copy Markdown
Contributor Author

/check-pr

@nepdada

nepdada commented Nov 3, 2026

Copy link
Copy Markdown
Contributor

/retest

@mindspore-ai-bot

Copy link
Copy Markdown

Gate trigger failed: please fix the /check-pr findings and obtain the pr-check-pass label before retrying.

@nepdada

nepdada commented Nov 3, 2026

Copy link
Copy Markdown
Contributor

/check-pr

@nepdada

nepdada commented Nov 3, 2026

Copy link
Copy Markdown
Contributor

/retest

@mindspore-ai-bot

Copy link
Copy Markdown

Gate trigger failed: please fix the /check-pr findings and obtain the pr-check-pass label before retrying.

@nepdada

nepdada commented Nov 3, 2026

Copy link
Copy Markdown
Contributor

/retest

@mindspore-ai-bot

Copy link
Copy Markdown

🔵 The pipeline #1389 is running. Please wait a moment... (Link)

@mindspore-ai-bot

Copy link
Copy Markdown
Project Name Build_Stage Build Result Details
Hyper-parallel_Github_Gate - ❌ FAILURE 1389
Check_Lizard FAILURE 1389
Check_Markdownlint FAILURE 1389
Check_Pylint FAILURE 1389

@mindspore-ai-bot

Copy link
Copy Markdown
Platform Type Module Excution Time Time Limit
- distributed_data 342.589s 450.0s
- activation_memory 469.63s 375.0s
- shard 211.053s 300.0s
- optimizer 306.623s 300.0s
- pipeline_parallel 744.809s 675.0s
- auto_models 135.294s 150.0s
- accuracy 124.988s 120.0s
- tensor_parallel 270.799s 300.0s
- clip_grad 119.646s 120.0s
- distributed_checkpoint 124.515s 120.0s
- swap_optimizer 360.231s 300.0s
- multicore 137.12s 120.0s
- expert_parallel 182.34712s 200.0s
- fully_shard 375.494s 600.0s
- dtensor 88.316s 120.0s
- llama3_examples 99.353s 120.0s
- context_parallel 93.615s 120.0s
- init_weights 93.448s 118.75s
- process_group 88.747s 90.0s

@mindspore-ai-bot

Copy link
Copy Markdown
Project Name Build_Stage Build Result Details
Hyper-parallel_Github_Gate - ✅ SUCCESS 1554

suteng
suteng previously approved these changes Nov 9, 2026
yaoyifanx
yaoyifanx previously approved these changes Nov 9, 2026
@shawny999

Copy link
Copy Markdown
Contributor Author

/check-pr

@shawny999

Copy link
Copy Markdown
Contributor Author

/retest

@mindspore-ai-bot

Copy link
Copy Markdown

🔵 The pipeline #1566 is running. Please wait a moment... (Link)

@mindspore-ai-bot

Copy link
Copy Markdown
Project Name Build_Stage Build Result Details
Hyper-parallel_Github_Gate - ❌ FAILURE 1566

yaoyifanx
yaoyifanx previously approved these changes Nov 9, 2026
@shawny999

Copy link
Copy Markdown
Contributor Author

/check-pr

@shawny999

Copy link
Copy Markdown
Contributor Author

/retest

@mindspore-ai-bot

Copy link
Copy Markdown

🔵 The pipeline #1569 is running. Please wait a moment... (Link)

@mindspore-ai-bot

Copy link
Copy Markdown
Project Name Build_Stage Build Result Details
Hyper-parallel_Github_Gate - ❌ FAILURE 1569

@shawny999

Copy link
Copy Markdown
Contributor Author

/check-pr

suteng
suteng previously approved these changes Nov 9, 2026
@shawny999

Copy link
Copy Markdown
Contributor Author

/retest

@mindspore-ai-bot

Copy link
Copy Markdown

🔵 The pipeline #1576 is running. Please wait a moment... (Link)

@mindspore-ai-bot

Copy link
Copy Markdown
Project Name Build_Stage Build Result Details
Hyper-parallel_Github_Gate - ❌ FAILURE 1576
Check_Markdownlint FAILURE 1576

@shawny999

Copy link
Copy Markdown
Contributor Author

/retest

@mindspore-ai-bot

Copy link
Copy Markdown

🔵 The pipeline #1577 is running. Please wait a moment... (Link)

@shawny999

Copy link
Copy Markdown
Contributor Author

codegen-code-review.html

@mindspore-ai-bot

Copy link
Copy Markdown
Project Name Build_Stage Build Result Details(Full/Incremental) Detail
Coverage Line Coverage ✅ SUCCESS 76%/73% Link

@mindspore-ai-bot

Copy link
Copy Markdown
Platform Type Module Excution Time Time Limit
- distributed_data 225.256s 450.0s
- activation_memory 373.849s 375.0s
- shard 165.701s 300.0s
- auto_models 88.23s 150.0s
- optimizer 231.905s 300.0s
- multicore 111.44188s 120.0s
- pipeline_parallel 528.288s 675.0s
- swap_optimizer 272.835s 300.0s
- tensor_parallel 212.446s 300.0s
- context_parallel 145.617s 120.0s
- accuracy 75.946s 120.0s
- fully_shard 275.846s 600.0s
- clip_grad 66.556s 120.0s
- distributed_checkpoint 69.208s 120.0s
- expert_parallel 151.07987s 200.0s
- init_weights 67.363s 118.75s
- dtensor 69.466s 120.0s
- process_group 61.603s 90.0s
- llama3_examples 57.55s 120.0s

@mindspore-ai-bot

Copy link
Copy Markdown
Project Name Build_Stage Build Result Details
Hyper-parallel_Github_Gate - ✅ SUCCESS 1577

@shawny999

Copy link
Copy Markdown
Contributor Author

/check-pr

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

9 participants