Repository navigation
Conversation
xuxl1008
marked this pull request as ready for review
October 13, 2026 02:43
xuxl1008
requested review from
liuchongming74,
suteng,
yangzhenzhang02 and
yaoyifanx
as code owners
October 13, 2026 02:43
xuxl1008
force-pushed
the
feat/v41-cp-optimization
branch
4 times, most recently
from
October 16, 2026 08:30
d1fb8e3 to
c781723
Compare
Exchange SWA halos and optional document-causal compressed KV prefixes with differentiable all-to-all-v. Reduce async all-gather gradients with SUM reduce-scatter and reuse packed sequence routing plans. Add optional zigzag and greedy Indexer task allocation using low-rank query residuals. Default greedy blocks to 2048 original query rows and reject explicit block sizes outside greedy mode.
xuxl1008
force-pushed
the
feat/v41-cp-optimization
branch
from
October 20, 2026 03:33
c781723 to
54d7172
Compare
This was referenced Oct 20, 2026
26 tasks
5 tasks done
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Paired: GitHub #1143 ↔ GitCode !1762
本 PR 优化 DeepSeek V4.1 的上下文并行通信,支持自然长度的 packed 文档,并增加可选的 Indexer 任务均衡。目标分支为
master。基于已合入 #1068 融合适配的最新 master(
f95e4528),当前分支仅保留一个独立 CP 提交,不再携带旧融合前置提交。保留 master 的 Q-only teacher、辅助 loss 接线、重计算选择缓存及共享 bank 写入保护。主要改动:
all_to_all_v交换同文档左侧 halo,保留all_gather对照。反向将重叠消费者的梯度 SUM 回 owner。main_kv_comm、index_k_comm独立支持all_gather(默认)和all_to_all_v。main KV 分发原 Q owner 所需的文档因果前缀;Index K 分发原 owner 的 KL 与均衡 worker 的 LI 需求并集,同一目的 rank 的同一文档只发送最长所需前缀。Full/Reindex/Reuse 复用对应共享 bank。none、zigzag、greedy、hybrid。hybrid 由显式文档长度阈值和最小 zigzag 片段阈值共同选择长文档,保留完整头尾配对片段;开头余数与其余文档进入同一个 greedy 池,共享每 worker 不超过2*local_Q的容量预算。在已有
SharedCompressedDSAAttentionreplacement 中配置:两项 zigzag 阈值是 hybrid-only 的显式正整数,上例为配置示例,不是通用性能最优值。
indexer_balance_block_size用于 greedy/hybrid,默认 2048 个原始 Q 行,必须为正整数且不大于 local Q;none/zigzag 必须省略。hybrid 的完整 zigzag 片段不再按这个 block size 二次切分。一个共享 Index K source 的所有消费者必须采用相同通信和均衡配置。精度检查覆盖 r1/r2、Full/Reindex/Reuse、多共享源、packed、短文档和空压缩 bank:
1.99e-12,更新后参数最大绝对差3.73e-92.37e-5上述 AG 的“统一 FP32 SUM”只在验证 driver 中设置,产品公共 AG 仍保持原 dtype。原生 BF16 RS 与 A2AV owner 累加顺序不同,跨通信路径多步训练不承诺逐位一致。跨均衡布局时,worker 投影 GEMM 形状、LI 并列项也可能改变临界 TopK;通信精度对照固定相同的任务布局。上述误差是实测记录,不是新增验收容差。
Index K 反向优化的独立 CP8 实测(1M、r2、单次 dispatch 前反向,非整模型):
实际宽度六层 attention 链、64K、CP8 的前反向加参数 SUM 为
1109.175 → 1108.587 ms,峰值约18810 MiB,不构成显著整步加速证据。低秩 qr 交换的收益也需连同接收端投影、pack/unpack、启动和同步成本评估。普通因果 r1/r2 与 TP=1 的均衡路径已覆盖;candidate pool、分页 Index K、新 block 交换搜索和新的 overlap 调度不包含在本次 CP 提交。完整模型 FSDP+CP 的长稳以及 1M/大 CP 整步收益仍需另测。
2026-10-20 基线整理复核:独立 CP patch 在 rebase 前后
git range-diff一致;现有相关 44 项 UT、CPU/Gloo CP8 六层共享链 2 步 AdamW(ragged、owner-empty、hybrid-mix)通过,全量/owner-only 反向的输出、TopK、输入及参数梯度、参数与 optimizer 状态逐位一致;CPU/Gloo CP4 选择缓存 64 个 rank-case 通过。此前 NPU 数据保持原实验范围,本次没有新增 NPU 性能结论。新的 overlap 与 Engram CP 优化在独立工作分支,不包含在本 PR。拆分进展(2026-10-22):本 PR 的代码提交暂时保留为完整对照;新增异步调度已按功能放入以下个人 fork 草稿,尚未写入本 PR 的 head。前置自然文档修复 #1250 合入后,再按 A → B → C 依次迁移到主仓 master。
Engram halo、分页 Index K、融合接收/TopK 恢复、融合 owner reduce 和后置 block 交换继续保留在本地,不包含在上述三个拆分提交。