Refactor platform dev v1.2.0 by zhangts20 · Pull Request #3 · zhangts20/LightLLM

zhangts20 · 2026-08-13T11:13:39Z

对比查看

…ak_ref.

…host-register/device helpers.

…nels.

… allreduce.

…iton kernels.

…ion and mm+AR.

…ion.

zhangtaoshan and others added 30 commits August 13, 2026 15:59

Initial commit

54c0182

Initial commit of BackendGraph

731bdc4

Fix BackendRuntime

85d57ed

Initial commit of BackendOps

a339f73

Add dist_backend in BackendRuntime

6dc6eef

Fix some Backend calls

9245a5c

Initial commit of ascend backend

c365fc4

support non-ascend platforms

8441672

optimize graph calls

289e9f0

fix non-ascend platforms

93ef09e

fix about multimodal

7182483

add weight/infer ops

477387f

fix about graph

4de6709

misc

afaf573

add maca backend

11c299e

add register_host_register_worker

b249b2b

fix about multimodal radix cache

ccd637c

refactor ops

05b8d6d

add attn op registration

3749bdb

refactor ops/att registration

730e9b3

add sampling registration

691099f

refactor plugins

2f3be99

misc

af73a47

Simplify platform plugins and backend registration.

5ee099c

Fix Ascend Qwen2-VL mrope/prefill and Gemma3 multimodal accuracy.

c96cb06

Fix Ascend decode cudagraph race on post-replay attn param update.

66b1a3e

Fix Ascend decode hang: always sync ACL attn params before graph replay.

fb23f4d

Fix Gemma3 KV head_dim: use config head_dim instead of hardcoded 256.

57148fa

Optimize Ascend ACL graph decode with pipelined attn param update.

b0cc7e4

Add Ascend default-on opts: fused mm+AR, paged KV, rope hoist, rms/we…

61bd50f

…ak_ref.

zhangtaoshan and others added 19 commits August 13, 2026 19:12

chore: sync with v1.2.0

5758c52

Fix platform rebase leftovers.

8e85125

Fix Ascend Qwen3.6 GDN accuracy/ACL graph decode and harden non-CUDA …

bab7e74

…host-register/device helpers.

Fix maca shared-memory OOM in prefill and linear-attention Triton ker…

7ba6a4e

…nels.

Fix MetaX CUDA Graph decode KV layout, long-context kernels, and MCCL…

7b84b5f

… allreduce.

Reduce FlashInfer prefill KV index allocation to the actual token count.

f138c87

Optimize Ascend Qwen3.6 paged KV memory, ACL graph decode, and GDN Tr…

9fae50f

…iton kernels.

Fix paged KV token allocation and add cache on the page size.

db02ab1

Fix Ascend Qwen3.6 MTP GDN prefill and decode kernels.

0271dd1

[MetaX] Enable w8a8-vllm quantization.

4f2a82a

[Ascend] Add W8A8 dense and MoE inference with fused SwiGLU quantizat…

0d9ec87

…ion and mm+AR.

Use libc.so.6 to support both x86 and ARM.

7d39ed0

[Ascend] Add paged FA3 INT8 KV cache with chunked prefill dequantizat…

da5e715

…ion.

[MetaX] Add paged FA3 attention support for prefill and decode.

08aa1f0

[Ascend] Speed up long prefill and fix NPU kernel errors.

1e31e08

[Ascend] Split GDN gating, chunk prefill, and mrope NPU kernels by SoC.

e64bed3

[MetaX] Add paged FA3 INT8 KV cache for Qwen3.6 prefill and decode.

3a4b31e

[MetaX] Speed up INT8 KV prefix prefill with one-shot dequant and FA.

58fb33f

[Ascend] Speed up GDN decode and fix ACL graph compatibility.

62b1545

Navigation Menu

Refactor platform dev v1.2.0 - #3

Refactor platform dev v1.2.0#3
zhangts20 wants to merge 49 commits into
mainfrom
refactor_platform_dev_v1.2.0

zhangts20 commented Aug 13, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

Conversation

zhangts20 commented Aug 13, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant