Skip to content

feat(qy): QY ops backend with FA and longctx MoE/RoPE kernels (2026.11.12) - #1038

Open
Ceng23333 wants to merge 7 commits into
InfiniTensor:masterfrom
Ceng23333:feat/qy-backend-20261112
Open

Ceng23333 wants to merge 7 commits into
InfiniTensor:masterfrom
Ceng23333:feat/qy-backend-20261112

Conversation

@Ceng23333

@Ceng23333 Ceng23333 commented Nov 12, 2026 •

Copy link
Copy Markdown
Contributor

Summary

  • Denglin QY native CUDA ops backend (add/gemm M0 path + MoE/RoPE/FA kernels) rebased onto current master.
  • Torch instantiate.h / stream_guard.h wire WITH_QY C10 specialization.
  • Gate Device::Type::kQy behind WITH_QY so non-QY unit jobs still compile against InfiniRT master (no kQy enum).

Motivation / related

Type of change

  • feat
  • fix (clang-format; kQy CI gate; QY stream_guard include)

Platforms affected

  • QY (Denglin)
  • NVIDIA/Iluvatar/etc.: compile surface only via WITH_QY gate (no functional change when QY off)

Validation

  • Host Denglin QY, DENGLIN_DEVICES=3.
  • Build against RT ci: add clang-format checking GitHub Action #54 head 047340f45efa; pytest tests/test_add.py tests/test_gemm.py --devices qy → 1633 passed, 6 skipped.
  • See PR comments: QY required smoke (PR head) (+ detailed).
  • Logs: nov_milestone/logs/prhead-verify-20261113/ops-pytest-add-gemm.log

Checklist

  • clang-format-21 clean (CI green)
  • Conventional Commits title/branch (feat/qy-backend-20261112)
  • Full non-QY unit matrix: some jobs fail on self-hosted git proxy 127.0.0.1:11312 connection refused (infra, not kQy compile)

Stack PR index (merge order)

  1. InfiniOps: feat(qy): QY ops backend with FA and longctx MoE/RoPE kernels (2026.11.12) #1038 — 71e85162ab96
  2. InfiniRT: feat(qy): enable QY CUDA runtime device backend (2026.11.12) InfiniRT#54 — 047340f45efa
  3. InfiniCCL: feat(qy): enable InfiniCCL NCCL backend for Denglin QY (2026.11.12) InfiniCCL#90 — 691c3092c4c9
  4. InfiniCore: chore(qy): pin InfiniRT/Ops/CCL for QY MoE bf16-warpnk (2026.11.12) InfiniCore#1583 — 0b69ee9002bd
  5. InfiniLM: feat(qy): MiniCPM5 FA/longctx/cudagraph + MoE bf16/warpnk surpass (2026.11.12) InfiniLM#618 — 36beae5376f4 → base refactor/adopt-modern-infini-stack

ITW pin: https://github.com/Ceng23333/InfiniTensorWorktree/tree/v2026.11.12-qy-moe-bf16-warpnk

Ceng23333 and others added 4 commits November 12, 2026 17:26
Enable Denglin/QY native and torch paths, linked flash-attn varlen/kvcache
providers via *_no_sinks, and reshape_and_cache_flash for paged caching.
Promote mul_scalar, silu_and_mul, moe_sum/align/topk_sigmoid, and rope/rope_pack kernels from the longctx adapting lane onto feat/qy-backend.
Co-authored-by: Cursor <cursoragent@cursor.com>
Upstream moved device instantiation to instantiate.h macros; add kQy
alongside the rebased QY backend so torch providers link under WITH_QY.

Co-authored-by: Cursor <cursoragent@cursor.com>
bench and others added 2 commits November 13, 2026 10:56
Format failing clang-format-21 paths and wrap Device::Type::kQy behind
WITH_QY so unit jobs building against InfiniRT master still compile.
WITHOUT this, WITH_QY torch ops fail to instantiate StreamGuard.

Co-authored-by: Cursor <cursoragent@cursor.com>
@Ceng23333

Copy link
Copy Markdown
Contributor Author

QY required smoke (PR head)

  • Head SHA: 71e85162ab96192d7fda620e1c80e21f02fadc3f
  • Host: Denglin QY, DENGLIN_DEVICES=3
  • Command: pytest tests/test_add.py tests/test_gemm.py --devices qy (m0 ops config build against InfiniRT PR head 047340f45efa)
  • Result: PASS (1633 passed, 6 skipped)
  • Log: nov_milestone/logs/prhead-verify-20261113/ops-pytest-add-gemm.log
1633 passed, 6 skipped, 1 warning in 15.64s

@Ceng23333

Copy link
Copy Markdown
Contributor Author

QY required smoke (PR head) — detailed

  • Head SHA: 71e85162ab96192d7fda620e1c80e21f02fadc3f
  • Host: Denglin QY, DENGLIN_DEVICES=3
  • Build: m0 ops config (add,gemm) vs InfiniRT PR head 047340f45efa
  • Command: pytest tests/test_add.py tests/test_gemm.py --devices qy
  • Overall: PASS
  • Log: nov_milestone/logs/prhead-verify-20261113/ops-pytest-add-gemm.log
Suite Device Passed Skipped Failed Wall time
test_add + test_gemm qy 1633 6 0 15.64s

Also on this tip: clang-format-21 CI green; kQy gated behind WITH_QY for non-QY unit compile; stream_guard.h includes QY C10. Remaining unit failures on nvidia/ascend are infra git proxy 127.0.0.1:11312 connection refused (not kQy enum).

Co-authored-by: Cursor <cursoragent@cursor.com>
@Ceng23333

Copy link
Copy Markdown
Contributor Author

CI note (infra re-run follow-up)

Tip: 67a6b3c10fc7 (ci: retrigger unit jobs after runner proxy flake) after prior tip 71e85162ab96.

What we did

  1. Waited out prior CI run 43356761569 / shadow 43356761509 — ci / unit / iluvatar ended cancelled; shadow iluvatar failure; nvidia/ascend already red.
  2. Re-ran failed jobs on both runs via rerun-failed-jobs.
  3. First pass after re-run still died at checkout on runner proxy http://127.0.0.1:11312 (Connection refused, git exit 128) — hosts included Nvidia-Server-005, Ascend-910C-Server-001, shadow Nvidia-Server-002 / Ascend-910C-Server-002.
  4. One empty-commit retrigger → new CI 43376178404 / shadow 43376178438.

Current classification (not a kQy / compile defect)

Check Runner (latest tip) Result
clang-format / ruff / docs — green
ci / unit / metax Metax-Server-006 green (others still running)
ci / unit / nvidia Nvidia-Server-006 red — actions/checkout → proxy :11312 Connection refused (exit 128); never reaches build
ci / unit / ascend Ascend-910C-Server-000 red — same
shadow nvidia / ascend Nvidia-Server-001 / Ascend-910C-Server-002 red — same
ci / unit / iluvatar (+ shadow) Iluvatar-Server-006 / 000 still in_progress at comment time (past checkout)

Required format is green. Remaining nvidia/ascend unit reds are runner proxy checkout infra (127.0.0.1:11312), not a QY/kQy/Device::Type failure. No further Ops code commits planned for this flake; needs org runner/proxy attention.

Comment thread CMakeLists.txt
if(WITH_QY)
add_compile_definitions(WITH_QY=1)

set(QY_ROOT "")

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

CMake 清空用户指定的 QY_ROOT。 -DQY_ROOT=... 被覆盖,非默认 SDK 路径配置失败

@baominghelly

Copy link
Copy Markdown
Collaborator


template <>
__device__ __forceinline__ half mulScalarValue<half>(half value, double alpha) {
return __hmul(value, __float2half(static_cast<float>(alpha)));

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

标量乘法提前把 alpha 转为 FP16。 输入约 0.01、alpha=100000 时,结果应为 1000,当前计算会得到 inf

Comment on lines +28 to +31
for (; d + 4 <= dim; d += 4) {
*reinterpret_cast<uint64_t *>(out + d) =
*reinterpret_cast<const uint64_t *>(row + d);
}

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

代码里的“前进 4 个元素”在 FP32 下等于前进 16 字节,而每次实际只复制了 8 字节。
会导致FP32 pack/unpack 漏拷一半数据。

@baominghelly

Copy link
Copy Markdown
Collaborator

cuda/qy下面不放具体算子的文件,请全部迁移到ops文件夹下面对应的算子文件夹下面。

@baominghelly

Copy link
Copy Markdown
Collaborator

我发现include/下面的launcher都没有被调用,可以等用到的时候再加;如果加的话,也是加在调用算子的文件夹下面

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants