Add Qwen4-Exp support to VarlenNGramHashMapping by Copilot · Pull Request #32467 · microsoft/onnxruntime

Copilot · 2026-09-08T02:10:58Z

Description

Extends com.microsoft::VarlenNGramHashMapping from PR #32358 with Qwen4-Exp semantics across CPU, CUDA, and WebGPU.

Add optional per-head offsets after modulo.
Use eos_token_id for missing history and optional EOS-boundary resets.
Support token-major segment_ids resets within packed requests.
Accept multiplier tensors with additional elements.
Preserve request-local history and packed-sequence isolation.
Update operator schemas, generated documentation, and cross-provider tests.

Motivation and Context

Qwen-3.8-Flash-Next uses Qwen4-Exp n-gram hashing, which differs from the DeepSeek V3 implementation through EOS-aware history resets and offsets into a shared embedding table. The existing variable-length operator could not reproduce those hash IDs for packed OgaEngine batches.

…pport-qwen-3-8 Co-authored-by: kunal-vaishnavi <115581922+kunal-vaishnavi@users.noreply.github.com>

Co-authored-by: kunal-vaishnavi <115581922+kunal-vaishnavi@users.noreply.github.com>

azure-pipelines · 2026-09-08T02:11:07Z

Azure Pipelines: There may be pipelines that require an authorized user to comment /azp run to run.

Copilot

🟡 Changes recommended

Segment isolation can fail across chunk boundaries, a CUDA arena assertion is ineffective, aarch64 parallelism can cause OOMs, and most changes are outside the declared scope.

Once you've addressed the issues Copilot identified, you can request another Copilot review.

Pull request overview

Nominally adds Qwen4-Exp semantics to VarlenNGramHashMapping, but the branch also contains extensive unrelated API, CUDA, Web, dependency, CI, and packaging work.

Changes:

Adds EOS, segment-reset, head-offset, and expanded multiplier handling across CPU, CUDA, WebGPU, schemas, docs, and tests.
Introduces broad CUDA workspace, plugin, kernel, WebGPU/WebNN, MLAS, and public API changes.
Updates dependencies, generated assets, packaging, and CI infrastructure.

File summaries

File	Description
`.github/workflows/linux_cuda_no_cudnn.yml`	Updates CUDA workflow path triggers.
`.github/workflows/windows_cuda_no_cudnn.yml`	Updates CUDA workflow path triggers.
`ThirdPartyNotices.txt`	Updates third-party notices.
`cgmanifests/cgmanifest.json`	Updates dependency manifest.
`cmake/CMakeLists.txt`	Updates build configuration.
`cmake/deps.txt`	Changes dependency pins.
`cmake/external/deep_gemm.cmake`	Adds DeepGEMM integration.
`cmake/external/onnxruntime_external_deps.cmake`	Updates external dependencies.
`cmake/external/xnnpack.cmake`	Updates XNNPACK configuration.
`cmake/onnxruntime.cmake`	Updates core build sources.
`cmake/onnxruntime_cuda_cccl.cmake`	Updates CUDA CCCL handling.
`cmake/onnxruntime_cuda_source_filters.cmake`	Updates CUDA source filters.
`cmake/onnxruntime_mlas.cmake`	Adds MLAS sources.
`cmake/onnxruntime_providers_cpu.cmake`	Updates CPU provider build.
`cmake/onnxruntime_providers_cuda.cmake`	Updates CUDA provider build.
`cmake/onnxruntime_providers_cuda_plugin.cmake`	Updates CUDA plugin build.
`cmake/onnxruntime_providers_pch.cmake`	Updates provider precompiled headers.
`cmake/onnxruntime_providers_webgpu.cmake`	Updates WebGPU provider build.
`cmake/onnxruntime_python.cmake`	Updates Python binding build.
`cmake/onnxruntime_training.cmake`	Updates training build.
`cmake/onnxruntime_unittests.cmake`	Registers additional tests.
`cmake/onnxruntime_webassembly.cmake`	Updates WebAssembly build.
`cmake/patches/.gitattributes`	Updates patch attributes.
`cmake/patches/cpuinfo/enable_deinit_refcounting.patch`	Adds CPUInfo lifetime patch.
`cmake/patches/xnnpack/release_cpuinfo_after_hardware_config.patch`	Adds XNNPACK CPUInfo-release patch.
`cmake/vcpkg-ports/cpuinfo/portfile.cmake`	Applies CPUInfo patch.
`cmake/vcpkg-ports/xnnpack/portfile.cmake`	Applies XNNPACK patch.
`docs/ContribOperators.md`	Regenerates contrib operator documentation.
`docs/OperatorKernels.md`	Regenerates kernel documentation.
`docs/annotated_partitioning/attention_workspace_estimation.md`	Documents attention workspace estimation.
`docs/annotated_partitioning/cuda_kernel_workspace_inventory.md`	Documents CUDA workspace usage.
`docs/annotated_partitioning/future_directions_constrained_env.md`	Documents future constrained-environment work.
`docs/contrib_ops/cuda/moe_qmoe.md`	Updates QMoE documentation.
`docs/contrib_ops/cuda/paged_attention.md`	Updates paged-attention documentation.
`include/onnxruntime/core/framework/op_kernel.h`	Extends kernel workspace interfaces.
`include/onnxruntime/core/framework/op_kernel_context.h`	Extends kernel-context output access.
`include/onnxruntime/core/framework/workspace_input_shape.h`	Adds workspace shape utilities.
`include/onnxruntime/core/framework/workspace_requirement.h`	Adds workspace requirement types.
`include/onnxruntime/core/session/onnxruntime_c_api.h`	Adds a public preallocated-output API.
`include/onnxruntime/core/session/onnxruntime_cxx_api.h`	Declares its C++ wrapper.
`include/onnxruntime/core/session/onnxruntime_cxx_inline.h`	Implements its C++ wrapper.
`include/onnxruntime/ep/adapter/op_kernel.h`	Updates EP adapter kernel support.
`java/src/main/native/ai_onnxruntime_providers_OrtCUDAProviderOptions.c`	Updates CUDA JNI option handling.
`java/src/main/native/ai_onnxruntime_providers_OrtTensorRTProviderOptions.c`	Updates TensorRT JNI option handling.
`js/web/docs/webnn-operators.md`	Updates WebNN operator documentation.
`js/web/lib/wasm/jsep/webnn/tensor-manager.ts`	Updates WebNN tensor management.
`js/web/test/e2e/browser-test-wasm-image-tensor-image.js`	Adds image-tensor browser coverage.
`js/web/test/e2e/run-data.js`	Updates browser test data handling.
`js/web/test/e2e/run.js`	Updates browser test runner.
`js/web/test/e2e/simple-http-server.js`	Updates test HTTP server.
`js/web/test/e2e/test-data/generate_test_image.py`	Adds deterministic image generation.
`js/web/test/e2e/test-data/tensor-image.jpg`	Adds generated image fixture.
`onnxruntime/contrib_ops/cpu/bert/varlen_ngram_hash_mapping.cc`	Implements Qwen behavior on CPU.
`onnxruntime/contrib_ops/cpu/bert/varlen_ngram_hash_mapping.h`	Extends the CPU kernel declaration.
`onnxruntime/contrib_ops/cpu/transformers/beam_search_impl_whisper.h`	Updates Whisper beam search.
`onnxruntime/contrib_ops/cpu/transformers/beam_search_parameters.cc`	Updates beam-search parameters.
`onnxruntime/contrib_ops/cpu/transformers/beam_search_parameters.h`	Extends beam-search parameter declarations.
`onnxruntime/contrib_ops/cuda/bert/attention_data.h`	Extends CUDA attention metadata.
`onnxruntime/contrib_ops/cuda/bert/cutlass_fmha/memory_efficient_attention.h`	Updates memory-efficient attention.
`onnxruntime/contrib_ops/cuda/bert/flash_attention/flash_api.cc`	Updates Flash Attention dispatch.
`onnxruntime/contrib_ops/cuda/bert/flash_attention/flash_api.h`	Extends Flash Attention declarations.
`onnxruntime/contrib_ops/cuda/bert/packed_attention.cc`	Adds packed-attention workspace handling.
`onnxruntime/contrib_ops/cuda/bert/packed_attention.h`	Extends packed-attention declarations.
`onnxruntime/contrib_ops/cuda/bert/packed_attention_workspace.cc`	Implements packed-attention workspace logic.
`onnxruntime/contrib_ops/cuda/bert/packed_attention_workspace.h`	Declares packed-attention workspace logic.
`onnxruntime/contrib_ops/cuda/bert/packed_attention_workspace_estimate.cc`	Implements workspace estimation.
`onnxruntime/contrib_ops/cuda/bert/packed_attention_workspace_estimate.h`	Declares workspace estimation.
`onnxruntime/contrib_ops/cuda/bert/packed_multihead_attention.cc`	Updates packed multi-head attention.
`onnxruntime/contrib_ops/cuda/bert/packed_multihead_attention.h`	Extends packed MHA declarations.
`onnxruntime/contrib_ops/cuda/bert/paged_attention.cc`	Updates paged attention.
`onnxruntime/contrib_ops/cuda/bert/paged_attention.h`	Extends paged-attention declarations.
`onnxruntime/contrib_ops/cuda/bert/paged_attention_impl.cu`	Updates paged-attention kernels.
`onnxruntime/contrib_ops/cuda/bert/tensorrt_fused_multihead_attention/mha_runner.cu`	Updates TensorRT MHA runner.
`onnxruntime/contrib_ops/cuda/bert/tensorrt_fused_multihead_attention/mha_runner.h`	Extends MHA runner declarations.
`onnxruntime/contrib_ops/cuda/bert/varlen_ngram_hash_mapping.cc`	Implements Qwen CUDA dispatch.
`onnxruntime/contrib_ops/cuda/bert/varlen_ngram_hash_mapping.h`	Extends the CUDA kernel declaration.
`onnxruntime/contrib_ops/cuda/bert/varlen_ngram_hash_mapping_impl.cu`	Implements Qwen CUDA kernels.
`onnxruntime/contrib_ops/cuda/bert/varlen_ngram_hash_mapping_impl.h`	Extends CUDA implementation interfaces.
`onnxruntime/contrib_ops/cuda/bert/xqa/mha_impl.cuh`	Updates XQA MHA implementation.
`onnxruntime/contrib_ops/cuda/bert/xqa/xqa_paged_impl_gen.cuh`	Updates generated XQA dispatch.
`onnxruntime/contrib_ops/cuda/bert/xqa/xqa_paged_loader.cu`	Updates XQA paged loading.
`onnxruntime/contrib_ops/cuda/bert/xqa/xqa_paged_loader.h`	Extends XQA loader declarations.
`onnxruntime/contrib_ops/cuda/bert/xqa/xqa_paged_loader_impl.cuh`	Updates XQA loader implementation.
`onnxruntime/contrib_ops/cuda/bert/xqa/xqa_paged_spec_dec_bf16_bf16_256.cu`	Adds XQA specialization.
`onnxruntime/contrib_ops/cuda/bert/xqa/xqa_paged_spec_dec_fp16_fp16_256.cu`	Adds XQA specialization.
`onnxruntime/contrib_ops/cuda/bert/xqa/xqa_paged_spec_dec_fp16_fp8_256.cu`	Adds XQA specialization.
`onnxruntime/contrib_ops/cuda/bert/xqa/xqa_paged_spec_dec_fp16_int8_256.cu`	Adds XQA specialization.
`onnxruntime/contrib_ops/cuda/llm/fpA_intB_gemm/fpA_intB_gemm.h`	Extends GEMM workspace support.
`onnxruntime/contrib_ops/cuda/llm/moe_gemm/deep_gemm_sm90.cu`	Adds DeepGEMM SM90 implementation.
`onnxruntime/contrib_ops/cuda/llm/moe_gemm/deep_gemm_sm90.h`	Declares DeepGEMM support.
`onnxruntime/contrib_ops/cuda/llm/moe_gemm/moe_kernels.cu`	Updates MoE kernels.
`onnxruntime/contrib_ops/cuda/llm/moe_gemm/moe_kernels.h`	Extends MoE kernel interfaces.
`onnxruntime/contrib_ops/cuda/math/matmul_block_scaled_fp4.cu`	Updates block-scaled FP4 MatMul.
`onnxruntime/contrib_ops/cuda/math/matmul_block_scaled_fp4_tiling.h`	Updates FP4 tiling logic.
`onnxruntime/contrib_ops/cuda/moe/moe_quantization.cc`	Updates MoE quantization.
`onnxruntime/contrib_ops/cuda/moe/moe_quantization.h`	Extends MoE quantization declarations.
`onnxruntime/contrib_ops/cuda/moe/qmoe_kernels.cu`	Updates QMoE kernels.
`onnxruntime/contrib_ops/cuda/moe/qmoe_kernels.h`	Extends QMoE kernel declarations.
`onnxruntime/contrib_ops/cuda/quantization/matmul_nbits.cc`	Adds MatMulNBits workspace handling.
`onnxruntime/contrib_ops/cuda/quantization/matmul_nbits.h`	Extends MatMulNBits declarations.
`onnxruntime/contrib_ops/cuda/quantization/matmul_nbits_workspace_estimate.h`	Adds graph-free workspace estimation.
`onnxruntime/contrib_ops/cuda/transformers/generation_cuda_impl.cu`	Updates CUDA generation kernels.
`onnxruntime/contrib_ops/cuda/transformers/generation_device_helper.cc`	Updates generation device helpers.
`onnxruntime/contrib_ops/webgpu/bert/varlen_ngram_hash_mapping.cc`	Implements Qwen behavior on WebGPU.
`onnxruntime/contrib_ops/webgpu/bert/varlen_ngram_hash_mapping.h`	Extends the WebGPU kernel declaration.
`onnxruntime/core/framework/node_shape_resolver.h`	Adds shape resolution support.
`onnxruntime/core/framework/op_kernel.cc`	Implements kernel workspace behavior.
`onnxruntime/core/framework/op_kernel_context_internal.h`	Extends internal kernel context.
`onnxruntime/core/framework/session_state.cc`	Integrates workspace declarations.
`onnxruntime/core/graph/contrib_ops/bert_defs.cc`	Extends the operator schema.
`onnxruntime/core/mlas/inc/mlas.h`	Exposes MLAS linear attention.
`onnxruntime/core/mlas/lib/aarch64/linear_attention_sve_asm.S`	Adds SVE assembly implementation.
`onnxruntime/core/mlas/lib/layernorm.cpp`	Updates LayerNorm implementation.
`onnxruntime/core/mlas/lib/layernorm_kernel_avx2.cpp`	Updates AVX2 LayerNorm kernel.
`onnxruntime/core/mlas/lib/linear_attention.h`	Adds linear-attention interfaces.
`onnxruntime/core/mlas/lib/mlasi.h`	Extends internal MLAS declarations.
`onnxruntime/core/mlas/lib/platform.cpp`	Registers platform kernels.
`onnxruntime/core/mlas/lib/sve/gen_sve_asm.py`	Adds SVE assembly generation.
`onnxruntime/core/mlas/lib/sve/linear_attention_asm_sve.h`	Declares SVE assembly routines.
`onnxruntime/core/mlas/lib/sve/linear_attention_compute_sve.h`	Adds SVE compute helpers.
`onnxruntime/core/mlas/lib/sve/linear_attention_kernel_sve.cpp`	Adds SVE kernel entry points.
`onnxruntime/core/mlas/lib/sve/linear_attention_sve.h`	Declares SVE linear attention.
`onnxruntime/core/mlas/lib/sve/linear_attention_sve_impl.cpp`	Implements SVE linear attention.
`onnxruntime/core/optimizer/graph_transformer_utils.cc`	Updates transformer selection.
`onnxruntime/core/optimizer/layer_norm_fusion.cc`	Updates LayerNorm fusion.
`onnxruntime/core/providers/cuda/cu_inc/topk_warp_sort.cuh`	Updates TopK warp sorting.
`onnxruntime/core/providers/cuda/cuda_execution_provider.cc`	Integrates CUDA workspace handling.
`onnxruntime/core/providers/cuda/math/topk_hybrid.cuh`	Adds hybrid TopK logic.
`onnxruntime/core/providers/cuda/math/topk_impl.cuh`	Updates TopK implementation.
`onnxruntime/core/providers/cuda/math/topk_impl_bf16.cu`	Updates BF16 TopK specialization.
`onnxruntime/core/providers/cuda/math/topk_impl_f16.cu`	Updates FP16 TopK specialization.
`onnxruntime/core/providers/cuda/math/topk_impl_f32.cu`	Updates FP32 TopK specialization.
`onnxruntime/core/providers/cuda/math/topk_smallk.cuh`	Adds small-K TopK path.
`onnxruntime/core/providers/cuda/plugin/cuda_arena.cc`	Updates plugin arena behavior.
`onnxruntime/core/providers/cuda/plugin/cuda_arena.h`	Extends plugin arena state.
`onnxruntime/core/providers/cuda/plugin/cuda_ep_factory.cc`	Updates CUDA plugin factory.
`onnxruntime/core/providers/cuda/plugin/cuda_ep_factory.h`	Extends factory declarations.
`onnxruntime/core/providers/cuda/plugin/cuda_kernel_adapter.h`	Updates kernel adapter support.
`onnxruntime/core/providers/cuda/plugin/cuda_stream_plugin.cc`	Updates stream cleanup behavior.
`onnxruntime/core/providers/cuda/plugin/cuda_stream_plugin.h`	Extends stream state.
`onnxruntime/core/providers/cuda/reduction/reduction_functions.cu`	Updates reduction kernels.
`onnxruntime/core/providers/cuda/reduction/reduction_scan_utils.h`	Updates scan utilities.
`onnxruntime/core/providers/cuda/reduction/reduction_utils.cuh`	Updates reduction utilities.
`onnxruntime/core/providers/cuda/tensor/concat.cc`	Updates CUDA Concat.
`onnxruntime/core/providers/cuda/tensor/slice.cc`	Updates CUDA Slice.
`onnxruntime/core/providers/cuda/tensor/slice.h`	Extends CUDA Slice declarations.
`onnxruntime/core/providers/cuda/tensor/split.cc`	Updates CUDA Split.
`onnxruntime/core/providers/tensorrt/tensorrt_execution_provider.cc`	Updates TensorRT provider behavior.
`onnxruntime/core/providers/webgpu/nn/conv.cc`	Updates WebGPU convolution.
`onnxruntime/core/providers/webgpu/nn/fuse_utils.h`	Updates WebGPU fusion utilities.
`onnxruntime/core/providers/webgpu/nn/im2col_matmul.cc`	Updates im2col MatMul dispatch.
`onnxruntime/core/providers/webgpu/nn/im2col_matmul.h`	Extends im2col MatMul parameters.
`onnxruntime/core/providers/webgpu/nn/im2col_matmul.wgsl.template`	Updates im2col shader template.
`onnxruntime/core/providers/webgpu/nn/pool.cc`	Updates WebGPU pooling.
`onnxruntime/core/providers/webgpu/nn/pool.h`	Extends WebGPU pooling declarations.
`onnxruntime/core/providers/webgpu/program_cache_key.cc`	Updates program cache keys.
`onnxruntime/core/providers/webgpu/program_cache_key.h`	Extends cache-key declarations.
`onnxruntime/core/providers/webgpu/shader_helper.cc`	Updates shader helper behavior.
`onnxruntime/core/providers/webgpu/webgpu_context.cc`	Updates WebGPU context handling.
`onnxruntime/core/providers/webgpu/webgpu_provider_factory.cc`	Updates provider factory behavior.
`onnxruntime/core/providers/webnn/builders/impl/normalization_op_builder.cc`	Updates WebNN normalization.
`onnxruntime/core/providers/webnn/builders/map_info.h`	Updates WebNN operator mapping.
`onnxruntime/core/providers/webnn/builders/op_builder_factory.cc`	Updates WebNN builder registration.
`onnxruntime/core/session/custom_ops.cc`	Updates custom-op kernel context.
`onnxruntime/core/session/onnxruntime_c_api.cc`	Implements the new C API.
`onnxruntime/core/session/ort_apis.h`	Declares the C API implementation.
`onnxruntime/python/onnxruntime_inference_collection.py`	Updates Python inference bindings.
`onnxruntime/python/onnxruntime_pybind_iobinding.cc`	Updates Python I/O binding.
`onnxruntime/python/onnxruntime_pybind_ortvalue.cc`	Updates Python OrtValue behavior.
`onnxruntime/python/onnxruntime_pybind_state.cc`	Updates Python module bindings.
`onnxruntime/python/onnxruntime_pybind_state_common.h`	Updates shared Python binding declarations.
`onnxruntime/test/autoep/library/example_plugin_ep/ep.cc`	Updates example plugin EP.
`onnxruntime/test/autoep/library/example_plugin_ep/ep_factory.cc`	Updates example EP factory.
`onnxruntime/test/autoep/library/example_plugin_ep/ep_factory.h`	Extends example factory declarations.
`onnxruntime/test/autoep/library/example_plugin_ep/ep_stream_support.cc`	Adds example stream support.
`onnxruntime/test/autoep/library/example_plugin_ep/ep_test_hooks.cc`	Adds plugin test hooks.
`onnxruntime/test/autoep/library/example_plugin_ep/ep_test_hooks.h`	Declares plugin test hooks.
`onnxruntime/test/autoep/library/example_plugin_ep/example_plugin_ep_library.lds`	Updates plugin symbol exports.
`onnxruntime/test/autoep/test_autoep_utils.cc`	Updates AutoEP test utilities.
`onnxruntime/test/autoep/test_autoep_utils.h`	Extends AutoEP test helpers.
`onnxruntime/test/autoep/test_execution.cc`	Expands AutoEP execution tests.
`onnxruntime/test/autoep/test_webgpu_allocators.cc`	Adds WebGPU allocator tests.
`onnxruntime/test/contrib_ops/beam_search_test.cc`	Updates beam-search tests.
`onnxruntime/test/contrib_ops/cuda_kernels/qmoe_fp4_to_fp8_kernel_test.cc`	Adds QMoE conversion tests.
`onnxruntime/test/contrib_ops/engram_ops_test.cc`	Adds Qwen mapping tests.
`onnxruntime/test/contrib_ops/matmul_block_scaled_fp4_test.cc`	Expands FP4 MatMul tests.
`onnxruntime/test/contrib_ops/paged_attention_op_test.cc`	Expands paged-attention tests.
`onnxruntime/test/framework/max_shape_override_test.cc`	Adds maximum-shape tests.
`onnxruntime/test/framework/session_state_test.cc`	Adds workspace/session tests.
`onnxruntime/test/mlas/bench/bench_linear_attention.cpp`	Adds linear-attention benchmarks.
`onnxruntime/test/mlas/unittest/test_layernorm.cpp`	Expands LayerNorm tests.
`onnxruntime/test/mlas/unittest/test_linear_attention.cpp`	Adds linear-attention tests.
`onnxruntime/test/optimizer/graph_transform_test.cc`	Expands graph-transform tests.
`onnxruntime/test/perftest/ort_test_session.cc`	Updates performance-test session setup.
`onnxruntime/test/providers/cpu/math/topk_op_test.cc`	Expands TopK tests.
`onnxruntime/test/providers/cpu/nn/pool_op_test.cc`	Expands pooling tests.
`onnxruntime/test/providers/cpu/tensor/concat_op_test.cc`	Expands Concat tests.
`onnxruntime/test/providers/cpu/tensor/slice_op.test.cc`	Expands Slice tests.
`onnxruntime/test/providers/cpu/tensor/split_op_test.cc`	Expands Split tests.
`onnxruntime/test/providers/cuda/plugin/cuda_plugin_arena_test.cc`	Adds CUDA plugin arena tests.
`onnxruntime/test/providers/cuda/test_cases/generation_cuda_impl_test.cc`	Adds generation CUDA tests.
`onnxruntime/test/providers/cuda/test_cases/matmul_nbits_e2e_workspace_test.cc`	Adds workspace end-to-end tests.
`onnxruntime/test/providers/cuda/test_cases/matmul_nbits_workspace_test.cc`	Adds MatMulNBits workspace tests.
`onnxruntime/test/providers/cuda/test_cases/packed_attention_workspace_estimate_test.cc`	Adds attention workspace tests.
`onnxruntime/test/providers/cuda/test_cases/reduction_functions_test.cc`	Expands reduction tests.
`onnxruntime/test/python/onnxruntime_test_python.py`	Expands Python API tests.
`onnxruntime/test/python/quantization/test_subgraph.py`	Updates quantization tests.
`onnxruntime/test/python/transformers/test_paged_attention.py`	Expands paged-attention tests.
`onnxruntime/test/shared_lib/cpuinfo_dlopen_test.cc`	Adds CPUInfo dynamic-loading tests.
`onnxruntime/test/shared_lib/cpuinfo_dlopen_test_library.cc`	Adds CPUInfo test library.
`onnxruntime/test/shared_lib/cpuinfo_dlopen_test_library.def`	Exports CPUInfo test symbols.
`onnxruntime/test/shared_lib/cpuinfo_refcount_test.cc`	Adds CPUInfo reference-count tests.
`onnxruntime/test/testdata/transform/fusion/layer_norm_fp16.onnx`	Adds LayerNorm fusion model.
`onnxruntime/test/testdata/transform/fusion/layer_norm_fp16.py`	Adds model-generation script.
`orttraining/orttraining/training_ops/cuda/tensor/slice_grad.cc`	Updates CUDA SliceGrad.
`orttraining/orttraining/training_ops/cuda/tensor/slice_grad.h`	Extends SliceGrad declarations.
`plugin-ep-cuda/paths.txt`	Updates CUDA plugin source paths.
`tools/ci_build/github/azure-pipelines/c-api-noopenmp-packaging-pipelines-cuda13.yml`	Updates CUDA 13 packaging pipeline.
`tools/ci_build/github/azure-pipelines/c-api-noopenmp-packaging-pipelines.yml`	Updates packaging pipeline.
`tools/ci_build/github/azure-pipelines/c-api-noopenmp-test-pipelines.yml`	Updates C API test pipeline.
`tools/ci_build/github/azure-pipelines/custom-nuget-packaging-pipeline.yml`	Updates custom NuGet packaging.
`tools/ci_build/github/azure-pipelines/dml-nuget-packaging.yml`	Updates DML NuGet packaging.
`tools/ci_build/github/azure-pipelines/main-release-pipeline.yml`	Updates release pipeline.
`tools/ci_build/github/azure-pipelines/npm-packaging-pipeline.yml`	Updates npm packaging policy.
`tools/ci_build/github/azure-pipelines/qnn-ep-nuget-packaging-pipeline.yml`	Updates QNN NuGet packaging.
`tools/ci_build/github/azure-pipelines/stages/nodejs-npm-packaging-stage.yml`	Removes deprecated npm suffix handling.
`tools/ci_build/github/azure-pipelines/stages/nuget-combine-cuda-stage.yml`	Restricts prerelease suffixes.
`tools/ci_build/github/azure-pipelines/stages/nuget-win-cuda-packaging-stage.yml`	Restricts prerelease suffixes.
`tools/ci_build/github/azure-pipelines/stages/set_packaging_variables_stage.yml`	Restricts prerelease suffixes.
`tools/ci_build/github/azure-pipelines/templates/android-java-api-aar.yml`	Configures authenticated package feeds.
`tools/ci_build/github/azure-pipelines/templates/c-api-cpu.yml`	Restricts prerelease suffixes.
`tools/ci_build/github/azure-pipelines/templates/c-api-linux-cpu.yml`	Mounts Git feed configuration.
`tools/ci_build/github/azure-pipelines/templates/esrp_nuget.yml`	Updates ESRP signing task.
`tools/ci_build/github/azure-pipelines/templates/jobs/checkout-and-git-redirect.yml`	Reorders checkout and submodule setup.
`tools/ci_build/github/azure-pipelines/templates/linux-wasm-ci.yml`	Updates dependency-install script.
`tools/ci_build/github/azure-pipelines/templates/mac-esrp-dylib.yml`	Updates ESRP signing task.
`tools/ci_build/github/azure-pipelines/templates/make_java_win_binaries.yml`	Restricts prerelease suffixes.
`tools/ci_build/github/azure-pipelines/templates/qnn-ep-win.yml`	Updates QNN Windows pipeline.
`tools/ci_build/github/azure-pipelines/templates/setup-feeds-and-python-steps.yml`	Adds Gradle plugin feed configuration.
`tools/ci_build/github/azure-pipelines/templates/web-ci.yml`	Adds authenticated feed setup.
`tools/ci_build/github/azure-pipelines/templates/win-ci.yml`	Restricts prerelease suffixes.
`tools/ci_build/github/azure-pipelines/templates/win-esrp-dll.yml`	Updates ESRP signing task.
`tools/ci_build/github/linux/build_cuda_plugin_package.sh`	Changes aarch64 build parallelism.
`tools/python/trigger_and_wait_pipelines.py`	Updates pipeline trigger tooling.
`tools/python/wgsl_template/test/in_tree_golden/static-cpp-literal/generated/nn/im2col_matmul.h`	Regenerates literal im2col shader output.
`tools/python/wgsl_template/test/in_tree_golden/static-cpp-literal/index.h`	Updates literal template parameters.
`tools/python/wgsl_template/test/in_tree_golden/static-cpp-literal/index_impl.h`	Updates generated literal hashes.
`tools/python/wgsl_template/test/in_tree_golden/static-cpp/generated/nn/im2col_matmul.h`	Regenerates im2col shader output.
`tools/python/wgsl_template/test/in_tree_golden/static-cpp/generated/tensor/oihw_to_ohwi.h`	Regenerates tensor shader output.
`tools/python/wgsl_template/test/in_tree_golden/static-cpp/generated/tensor/pad.h`	Regenerates padding shader output.
`tools/python/wgsl_template/test/in_tree_golden/static-cpp/index.h`	Updates template parameters.
`tools/python/wgsl_template/test/in_tree_golden/static-cpp/index_impl.h`	Updates generated template hashes.
`tools/python/wgsl_template/test/in_tree_golden/static-cpp/string_table.h`	Regenerates shader string table.

Review details

Suppressed comments (1)

onnxruntime/test/providers/cuda/plugin/cuda_plugin_arena_test.cc:259

This second lookup uses the same nonexistent key, so it also returns zero and EXPECT_GE(0, 0) passes regardless of whether the quarantined allocation was retained. Query the exported TotalAllocated key here as well.

Files reviewed: 203/253 changed files
Comments generated: 5
Review effort level: Balanced

💡 Configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

…t-qwen-3-8 Co-authored-by: kunal-vaishnavi <115581922+kunal-vaishnavi@users.noreply.github.com>

Co-authored-by: kunal-vaishnavi <115581922+kunal-vaishnavi@users.noreply.github.com>

…for-ogaengine' into copilot/add-op-support-qwen-3-8 Co-authored-by: kunal-vaishnavi <115581922+kunal-vaishnavi@users.noreply.github.com>

Co-authored-by: kunal-vaishnavi <115581922+kunal-vaishnavi@users.noreply.github.com>

…for-ogaengine' into copilot/add-op-support-qwen-3-8 # Conflicts: # onnxruntime/core/providers/webgpu/math/subgroup_matrix_matmul.cc # onnxruntime/core/providers/webgpu/math/subgroup_matrix_matmul_8x16x16.wgsl.template # tools/python/wgsl_template/test/in_tree_golden/static-cpp-literal/generated/math/subgroup_matrix_matmul_8x16x16.h # tools/python/wgsl_template/test/in_tree_golden/static-cpp-literal/index_impl.h # tools/python/wgsl_template/test/in_tree_golden/static-cpp/generated/math/subgroup_matrix_matmul_8x16x16.h # tools/python/wgsl_template/test/in_tree_golden/static-cpp/generated/nn/im2col_matmul.h # tools/python/wgsl_template/test/in_tree_golden/static-cpp/generated/tensor/oihw_to_ohwi.h # tools/python/wgsl_template/test/in_tree_golden/static-cpp/generated/tensor/pad.h # tools/python/wgsl_template/test/in_tree_golden/static-cpp/index_impl.h # tools/python/wgsl_template/test/in_tree_golden/static-cpp/string_table.h Co-authored-by: kunal-vaishnavi <115581922+kunal-vaishnavi@users.noreply.github.com>

Co-authored-by: kunal-vaishnavi <115581922+kunal-vaishnavi@users.noreply.github.com>

…for-ogaengine' into copilot/add-op-support-qwen-3-8 # Conflicts: # onnxruntime/contrib_ops/cpu/bert/varlen_ngram_hash_mapping.cc # onnxruntime/contrib_ops/cuda/bert/varlen_ngram_hash_mapping_impl.cu # onnxruntime/contrib_ops/webgpu/bert/varlen_ngram_hash_mapping.cc Co-authored-by: kunal-vaishnavi <115581922+kunal-vaishnavi@users.noreply.github.com>

…for-ogaengine' into copilot/add-op-support-qwen-3-8 # Conflicts: # onnxruntime/contrib_ops/cuda/bert/varlen_ngram_hash_mapping_impl.cu Co-authored-by: kunal-vaishnavi <115581922+kunal-vaishnavi@users.noreply.github.com>

Co-authored-by: kunal-vaishnavi <115581922+kunal-vaishnavi@users.noreply.github.com>

Copilot

Copilot review overview

🟡 Changes recommended

The new segment-state output lacks type inference, and scalar EOS validation is inconsistent across providers.

Get a fresh assessment by requesting another Copilot review.

Review effort: Balanced
Findings: 5

Open (5)

Enforce scalar rank for CUDA NGramHashMapping input · New
Enforce scalar rank for CUDA NGramHashMapping input · New
Enforce scalar rank for WebGPU NGramHashMapping input · New
Move unreachable segment ID type inference to VarlenNGramHashMapping · New
Propagate present_segment_ids element type during schema inference · New

Resolved since last review (5)

segment_ids cannot enforce isolation across calls: the kernels compare only adjacent IDs from the…
This no longer implements the documented eight-job cap: on hosts with 8 or more CPUs it selects…
The allocator exports this statistic as TotalAllocated (cuda_allocator_plugin.h:73), not…
The generator creates a 256×256 image, so the CLI description reports the wrong dimensions.
This public C API/workspace-estimation addition is unrelated to the stated Qwen…

+                      "head_offsets must have shape ((max_ngram_size - 1) * n_head_per_ngram)");
+  }
+  if (eos_token_id != nullptr) {
+    ORT_RETURN_IF_NOT(eos_token_id->Shape().Size() == 1, "eos_token_id must be a scalar");


+                      "head_offsets must have shape ((max_ngram_size - 1) * n_head_per_ngram)");
+  }
+  if (eos_token_id != nullptr) {
+    ORT_RETURN_IF_NOT(eos_token_id->Shape().Size() == 1, "eos_token_id must be a scalar");


+                      "head_offsets must have shape ((max_ngram_size - 1) * n_head_per_ngram)");
+  }
+  if (eos_token_id != nullptr) {
+    ORT_RETURN_IF_NOT(eos_token_id->Shape().Size() == 1, "eos_token_id must be a scalar");


+          if (ctx.getNumOutputs() > 2) {
+            propagateElemTypeFromInputToOutput(ctx, 3, 2);
+          }


+        .Output(2,
+                "present_segment_ids",
+                "Trailing max_ngram_size - 1 segment ids corresponding to present_ids, with shape "
+                "(batch_size, max_ngram_size - 1). Feed this back as past_segment_ids on the next call.",
+                "S",
+                OpSchema::Optional)


Copilot AI and others added 3 commits September 5, 2026 00:53

Merge remote-tracking branch 'origin/pr-32358' into copilot/add-op-su…

501155e

…pport-qwen-3-8 Co-authored-by: kunal-vaishnavi <115581922+kunal-vaishnavi@users.noreply.github.com>

Add Qwen support to VarlenNGramHashMapping

a391c49

Co-authored-by: kunal-vaishnavi <115581922+kunal-vaishnavi@users.noreply.github.com>

Fix VarlenNGramHashMapping test compilation

ea0c865

Co-authored-by: kunal-vaishnavi <115581922+kunal-vaishnavi@users.noreply.github.com>

Copilot AI assigned Copilot and kunal-vaishnavi Sep 8, 2026

Copilot created this pull request from a session on behalf of kunal-vaishnavi September 8, 2026 02:11 View session

kunal-vaishnavi marked this pull request as ready for review September 8, 2026 02:11

Copilot AI balanced review requested due to automatic review settings September 8, 2026 02:11

Copilot started reviewing on behalf of kunal-vaishnavi September 8, 2026 02:12 View session

kunal-vaishnavi changed the base branch from main to copilot/update-ngramhashmapping-for-ogaengine September 8, 2026 02:12

kunal-vaishnavi requested a review from a team as a code owner September 8, 2026 02:12

Copilot AI reviewed Sep 8, 2026

View reviewed changes

Copilot started work on behalf of kunal-vaishnavi September 8, 2026 02:27 View session

Copilot AI and others added 4 commits September 8, 2026 02:29

Merge remote-tracking branch 'origin/main' into copilot/add-op-suppor…

177a7c9

…t-qwen-3-8 Co-authored-by: kunal-vaishnavi <115581922+kunal-vaishnavi@users.noreply.github.com>

Preserve segment boundaries across varlen n-gram calls

d62feec

Co-authored-by: kunal-vaishnavi <115581922+kunal-vaishnavi@users.noreply.github.com>

Narrow stacked PR to VarlenNGramHashMapping

7702f7a

Co-authored-by: kunal-vaishnavi <115581922+kunal-vaishnavi@users.noreply.github.com>

Infer present segment state type

056551f

Co-authored-by: kunal-vaishnavi <115581922+kunal-vaishnavi@users.noreply.github.com>

Copilot stopped work on behalf of kunal-vaishnavi due to an error September 8, 2026 03:26
Copilot had to stop work due to a timeout.

Copilot AI requested a review from kunal-vaishnavi September 8, 2026 03:26

Copilot started work on behalf of kunal-vaishnavi September 8, 2026 03:53 View session

Merge remote-tracking branch 'origin/copilot/update-ngramhashmapping-…

6a5d872

…for-ogaengine' into copilot/add-op-support-qwen-3-8 Co-authored-by: kunal-vaishnavi <115581922+kunal-vaishnavi@users.noreply.github.com>

Copilot finished work on behalf of kunal-vaishnavi September 8, 2026 04:41

Copilot started work on behalf of kunal-vaishnavi September 8, 2026 14:27 View session

Fix WebGPU buffer limit and generated docs

1561435

Co-authored-by: kunal-vaishnavi <115581922+kunal-vaishnavi@users.noreply.github.com>

Copilot finished work on behalf of kunal-vaishnavi September 8, 2026 14:43

Copilot started work on behalf of kunal-vaishnavi September 8, 2026 19:31 View session

Copilot AI and others added 2 commits September 8, 2026 19:34

Align stacked PR tree with target branch

03d4c98

Co-authored-by: kunal-vaishnavi <115581922+kunal-vaishnavi@users.noreply.github.com>

Copilot finished work on behalf of kunal-vaishnavi September 8, 2026 19:36

Copilot started work on behalf of kunal-vaishnavi September 9, 2026 23:07 View session

Copilot finished work on behalf of kunal-vaishnavi September 10, 2026 00:12

Copilot started work on behalf of kunal-vaishnavi September 10, 2026 00:12 View session

Copilot started work on behalf of kunal-vaishnavi September 10, 2026 00:13 View session

Copilot finished work on behalf of kunal-vaishnavi September 10, 2026 00:14

Copilot stopped work on behalf of kunal-vaishnavi due to an error September 10, 2026 00:14
The session was cancelled by the user.

Copilot stopped work on behalf of kunal-vaishnavi due to an error September 10, 2026 00:18
The session was cancelled by the user.

Copilot stopped work on behalf of kunal-vaishnavi due to an error September 10, 2026 00:20
The session was cancelled by the user.

Copilot stopped work on behalf of kunal-vaishnavi due to an error September 10, 2026 00:21
The session was cancelled by the user.

Copilot stopped work on behalf of kunal-vaishnavi due to an error September 10, 2026 00:28
The session was cancelled by the user.

Copilot started work on behalf of kunal-vaishnavi September 10, 2026 23:43 View session

Copilot finished work on behalf of kunal-vaishnavi September 10, 2026 23:55

Copilot started work on behalf of kunal-vaishnavi September 11, 2026 00:08 View session

Copilot finished work on behalf of kunal-vaishnavi September 11, 2026 00:10

Akshay Sonawane (apsonawane) reviewed Sep 14, 2026

View reviewed changes

Comment thread onnxruntime/contrib_ops/cpu/bert/varlen_ngram_hash_mapping.cc Outdated

Akshay Sonawane (apsonawane) reviewed Sep 14, 2026

View reviewed changes

Comment thread onnxruntime/contrib_ops/cuda/bert/varlen_ngram_hash_mapping_impl.cu Outdated

Copilot AI mentioned this pull request Sep 15, 2026

[CUDA][WebGPU] Add packed sparse attention indexer for continuous batching #32618

Open

Copilot started work on behalf of kunal-vaishnavi September 18, 2026 08:41 View session

Copilot AI and others added 2 commits September 18, 2026 08:43

Address VarlenNGramHashMapping review feedback

cac48c2

Co-authored-by: kunal-vaishnavi <115581922+kunal-vaishnavi@users.noreply.github.com>

Copilot finished work on behalf of kunal-vaishnavi September 18, 2026 09:30

Akshay Sonawane (apsonawane) requested a balanced review from Copilot September 18, 2026 23:16

Copilot started reviewing on behalf of Akshay Sonawane (apsonawane) September 18, 2026 23:17 View session

Copilot AI reviewed Sep 18, 2026

View reviewed changes

Navigation Menu

Add Qwen4-Exp support to VarlenNGramHashMapping - #32467

Add Qwen4-Exp support to VarlenNGramHashMapping#32467
kunal-vaishnavi with Copilot wants to merge 15 commits into
copilot/update-ngramhashmapping-for-ogaenginefrom
copilot/add-op-support-qwen-3-8

Copilot AI commented Sep 8, 2026

Uh oh!

azure-pipelines Bot commented Sep 8, 2026

Uh oh!

Copilot AI left a comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Copilot AI left a comment

Uh oh!

This branch has not been deployed

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

Conversation

Copilot AI commented Sep 8, 2026

Description

Motivation and Context

Uh oh!

azure-pipelines Bot commented Sep 8, 2026

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

🟡 Changes recommended

Suppressed comments (1)

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Copilot review overview

🟡 Changes recommended

Uh oh!

This branch has not been deployed

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants