Skip to content

Preserve GLM-5.2 Megatron accuracy-compatible specs - #3

Draft
zrr1999 wants to merge 3 commits into
PFCCLab:mainfrom
zrr1999:glm52-bit-exact-alignment
Draft

Preserve GLM-5.2 Megatron accuracy-compatible specs#3
zrr1999 wants to merge 3 commits into
PFCCLab:mainfrom
zrr1999:glm52-bit-exact-alignment

Conversation

@zrr1999

@zrr1999 zrr1999 commented Jul 27, 2026

Copy link
Copy Markdown

Summary

  • infer MTP layer count from the GLM model configuration
  • prefer the configured routed-expert count during Megatron construction
  • preserve accuracy-compatible DSA/RMSNorm submodule specs when mcore-bridge rewrites the GLM DSA spec

These changes let the existing ms-swift + mcore-bridge path construct the reduced official GLM-5.2 profile without replacing Megatron modules with toy/eager stand-ins.

Validation

  • official-weight GLM-5.2 checkpoint conversion/load completed through the bridge
  • the resulting Megatron-Core model ran CUDA/bfloat16 forward/backward on NVIDIA H800
  • forward loss matches the independent Paddle path at 12.416634559631348

Status

Draft while strict backward, optimizer, and 100-step parity remain under validation.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant