Skip to content

KleidiAI f32 deptwise convolution kernel integrated - #11046

Open
melkap01-Arm wants to merge 14 commits into
google:masterfrom
melkap01-Arm:KleidiAI_f32_dwconv_kernel_integration
Open

KleidiAI f32 deptwise convolution kernel integrated#11046
melkap01-Arm wants to merge 14 commits into
google:masterfrom
melkap01-Arm:KleidiAI_f32_dwconv_kernel_integration

Conversation

@melkap01-Arm

@melkap01-Arm melkap01-Arm commented Aug 21, 2026

Copy link
Copy Markdown
Contributor

Summary
Integrates the KleidiAI F32 SME2 depthwise-convolution kernel into XNNPACK’s NHWC convolution operator path and updates KleidiAI to v1.30.0.
Unlike XNNPACK’s existing indirect DWConv microkernels, the KleidiAI kernel uses a planar/direct interface. It receives input and output base pointers together with their physical strides, dimensions and padding. The dedicated KAI path therefore does not create or consume an XNNPACK indirection buffer.

Implementation

  • Selects the KleidiAI kernel on SME2 hardware for supported dense F32 3×3, stride-1 depthwise convolutions.

  • Introduces the dedicated xnn_microkernel_type_kai_dwconv instead of presenting KAI as an ordinary xnn_dwconv_ukernel_fn.

  • Routes KAI through separate creation, reshape and setup paths with its own execution context.

  • Reuses the common DWConv packed-weight allocation and caching logic while supplying KAI-specific weight packers.

  • Distinguishes the KAI packed-weight layout in the weights-cache fingerprint.

  • Configures the planar input/output strides and padding directly, without creating an indirection buffer or requiring workspace.

  • Falls back to the existing XNNPACK DWConv implementation when the operation or hardware is unsupported.

  • Updates Bazel, CMake and GN integration for the KleidiAI DWConv sources.

  • Adds operator coverage for numerical correctness, the dedicated KAI microkernel type, zero workspace and ordinary-DWConv fallback.

Performance benchmark using Mobilenet_v1_f32.tflite model will be added for MacBook Pro and Android X300

Signed-off-by: melkap01 <melike.kaptan@arm.com>

@qukhan qukhan left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm going to need to bit more context here. The changes in this PR feel out of place considering the existing code.

You're adding architecture specific #if blocks in code that doesn't have any of that (src/operators/convolution-nhwc.c, src/operator-run.c).

From what I understand (I may be wrong), this is because the KleidiAI dwconv doesn't have to setup indirection buffers?

If that's the case, can I suggest creating a new microkernel type (kai_dwconv?, direct_dwconv?) and instead of hijacking existing functions. This seems to be what you started doing since you have a new config object that is specific to this kernel.

Comment thread src/f32-dwconv/f32-dwconv-9pvc-minmax-neonsme2.c
Comment thread src/xnnpack/dwconv.h Outdated
Comment thread test/operators/convolution-operator-tester.h Outdated
Comment thread src/operators/convolution-nhwc.c Outdated
Comment thread src/configs/dwconv-config.c Outdated
-XNNPACK dw conv kernel regisrty is not followed, KAI planar path seperated
-unnecassry conditional compile gurards removed
-copywrites improved

Signed-off-by: melkap01 <melike.kaptan@arm.com>
Signed-off-by: melkap01 <melike.kaptan@arm.com>
…view suggested

Signed-off-by: melkap01 <melike.kaptan@arm.com>
Signed-off-by: melkap01 <melike.kaptan@arm.com>
Comment thread src/configs/dwconv-config.c Outdated
Comment thread src/configs/dwconv-config.c Outdated
Comment thread src/configs/dwconv-config.c Outdated
Comment thread src/configs/dwconv-config.c Outdated
…e files

Signed-off-by: melkap01 <melike.kaptan@arm.com>
@melkap01-Arm

Copy link
Copy Markdown
Contributor Author

I'm going to need to bit more context here. The changes in this PR feel out of place considering the existing code.

You're adding architecture specific #if blocks in code that doesn't have any of that (src/operators/convolution-nhwc.c, src/operator-run.c).

From what I understand (I may be wrong), this is because the KleidiAI dwconv doesn't have to setup indirection buffers?

If that's the case, can I suggest creating a new microkernel type (kai_dwconv?, direct_dwconv?) and instead of hijacking existing functions. This seems to be what you started doing since you have a new config object that is specific to this kernel.

The initial design improved with latest add-ons.
This integrated KleidiAI kernel is a planar kernel and does not set up indirection buffers as stated.
Latest changes added to improve the design are :

  • The planar KAI DWConv path now uses dedicated xnn_kai_dwconv_config and kai_f32_dwconv_context types, with platform-specific gating confined to dwconv-config.c.
  • All KAI-specific compile-time guards from operator-run.c and convolution-nhwc.c are removed.
  • KAI planar DWConv path is separated from XNNPACK’s indirect DWConv path end-to-end—from configuration and kernel selection through operator creation, reshape, setup, and execution—while preserving the existing indirect path as the fallback.

@melkap01-Arm
melkap01-Arm marked this pull request as ready for review August 27, 2026 15:18
@qukhan

qukhan commented Sep 3, 2026

Copy link
Copy Markdown
Collaborator

This is the failing test reported by feedback/copybara: https://github.com/google/XNNPACK/actions/runs/33484906048/job/99782600207

@melkap01-Arm

Copy link
Copy Markdown
Contributor Author

This is the failing test reported by feedback/copybara: https://github.com/google/XNNPACK/actions/runs/33484906048/job/99782600207

Hi, I was aware of this and actually it looked like a timeout issue to me. And I rebased the code twice to trigger the run again. But I see on the Copybara import PR, the tip of the branch is behind my latest merge commits. I am not sure it is something expected or not but I believe a new run would help us here. Do you have an idea how we can trigger the Copybara to checkout the latest commit on the branch ?

copybara-service Bot pushed a commit that referenced this pull request Sep 4, 2026
--
6394001 by melkap01 <melike.kaptan@arm.com>:

KleidiAI f32 deptwise convolution kernel integrated

Signed-off-by: melkap01 <melike.kaptan@arm.com>

--
cbc87ae by melkap01 <melike.kaptan@arm.com>:

review comments addressed:
-XNNPACK dw conv kernel regisrty is not followed, KAI planar path seperated
-unnecassry conditional compile gurards removed
-copywrites improved

Signed-off-by: melkap01 <melike.kaptan@arm.com>

--
685a6b0 by melkap01 <melike.kaptan@arm.com>:

tests improved to use the transient_indirection_buffer getter

Signed-off-by: melkap01 <melike.kaptan@arm.com>

--
962e9b8 by melkap01 <melike.kaptan@arm.com>:

Dedicated KleidiAI path from topdown configuration implementaed as review suggested

Signed-off-by: melkap01 <melike.kaptan@arm.com>

--
6a858e3 by melkap01 <melike.kaptan@arm.com>:

test error fix

Signed-off-by: melkap01 <melike.kaptan@arm.com>

--
894d53a by melkap01 <melike.kaptan@arm.com>:

KleidiAI config defined and compile guards removed from the two source files

Signed-off-by: melkap01 <melike.kaptan@arm.com>

--
640640d by melkap01 <melike.kaptan@arm.com>:

guards removed from dwconv-config.c

Signed-off-by: melkap01 <melike.kaptan@arm.com>
FUTURE_COPYBARA_INTEGRATE_REVIEW=#11046 from melkap01-Arm:KleidiAI_f32_dwconv_kernel_integration f9e015d
PiperOrigin-RevId: 973908152
@qukhan

qukhan commented Sep 4, 2026

Copy link
Copy Markdown
Collaborator

Syncing the code should be enough for that.

There are some errors on the Bazel build. This is for //third_party/XNNPACK/bench/subgraph:unary but there are 395 of these.

XNNPACK/src/f32-dwconv/f32-dwconv-9pvc-minmax-neonsme2.c:12:10: error: module //XNNPACK:neonsme2_prod_microkernels_private does not depend on a module exporting 'KleidiAI/kai/kai_common.h'
   12 | #include "KleidiAI/kai/kai_common.h"
      |          ^
XNNPACK/src/f32-dwconv/f32-dwconv-9pvc-minmax-neonsme2.c:13:10: error: module //XNNPACK:neonsme2_prod_microkernels_private does not depend on a module exporting 'KleidiAI/kai/ukernels/dwconv/dwconv_f32_f32_f32p/kai_dwconv_clamp_f32_f32_f32p1vlx1b_3x3_s1_4xc_sme2_mla.h'
   13 | #include "KleidiAI/kai/ukernels/dwconv/dwconv_f32_f32_f32p/kai_dwconv_clamp_f32_f32_f32p1vlx1b_3x3_s1_4xc_sme2_mla.h"
      |          ^
2 errors generated.

XNNPACK/src/f32-dwconv/f32-dwconv-9pvc-minmax-neonsme2.c:12:10: error: module //XNNPACK:neonsme2_prod_microkernels_private does not depend on a module exporting 'KleidiAI/kai/kai_common.h'
   12 | #include "KleidiAI/kai/kai_common.h"
      |          ^
XNNPACK/src/f32-dwconv/f32-dwconv-9pvc-minmax-neonsme2.c:13:10: error: module //XNNPACK:neonsme2_prod_microkernels_private does not depend on a module exporting 'KleidiAI/kai/ukernels/dwconv/dwconv_f32_f32_f32p/kai_dwconv_clamp_f32_f32_f32p1vlx1b_3x3_s1_4xc_sme2_mla.h'
   13 | #include "KleidiAI/kai/ukernels/dwconv/dwconv_f32_f32_f32p/kai_dwconv_clamp_f32_f32_f32p1vlx1b_3x3_s1_4xc_sme2_mla.h"
      |          ^
2 errors generated.

copybara-service Bot pushed a commit that referenced this pull request Sep 4, 2026
--
6394001 by melkap01 <melike.kaptan@arm.com>:

KleidiAI f32 deptwise convolution kernel integrated

Signed-off-by: melkap01 <melike.kaptan@arm.com>

--
cbc87ae by melkap01 <melike.kaptan@arm.com>:

review comments addressed:
-XNNPACK dw conv kernel regisrty is not followed, KAI planar path seperated
-unnecassry conditional compile gurards removed
-copywrites improved

Signed-off-by: melkap01 <melike.kaptan@arm.com>

--
685a6b0 by melkap01 <melike.kaptan@arm.com>:

tests improved to use the transient_indirection_buffer getter

Signed-off-by: melkap01 <melike.kaptan@arm.com>

--
962e9b8 by melkap01 <melike.kaptan@arm.com>:

Dedicated KleidiAI path from topdown configuration implementaed as review suggested

Signed-off-by: melkap01 <melike.kaptan@arm.com>

--
6a858e3 by melkap01 <melike.kaptan@arm.com>:

test error fix

Signed-off-by: melkap01 <melike.kaptan@arm.com>

--
894d53a by melkap01 <melike.kaptan@arm.com>:

KleidiAI config defined and compile guards removed from the two source files

Signed-off-by: melkap01 <melike.kaptan@arm.com>

--
640640d by melkap01 <melike.kaptan@arm.com>:

guards removed from dwconv-config.c

Signed-off-by: melkap01 <melike.kaptan@arm.com>
FUTURE_COPYBARA_INTEGRATE_REVIEW=#11046 from melkap01-Arm:KleidiAI_f32_dwconv_kernel_integration f9e015d
PiperOrigin-RevId: 973908152
Signed-off-by: melkap01 <melike.kaptan@arm.com>
@melkap01-Arm

melkap01-Arm commented Sep 4, 2026

Copy link
Copy Markdown
Contributor Author

Syncing the code should be enough for that.

There are some errors on the Bazel build. This is for //third_party/XNNPACK/bench/subgraph:unary but there are 395 of these.

XNNPACK/src/f32-dwconv/f32-dwconv-9pvc-minmax-neonsme2.c:12:10: error: module //XNNPACK:neonsme2_prod_microkernels_private does not depend on a module exporting 'KleidiAI/kai/kai_common.h'
   12 | #include "KleidiAI/kai/kai_common.h"
      |          ^
XNNPACK/src/f32-dwconv/f32-dwconv-9pvc-minmax-neonsme2.c:13:10: error: module //XNNPACK:neonsme2_prod_microkernels_private does not depend on a module exporting 'KleidiAI/kai/ukernels/dwconv/dwconv_f32_f32_f32p/kai_dwconv_clamp_f32_f32_f32p1vlx1b_3x3_s1_4xc_sme2_mla.h'
   13 | #include "KleidiAI/kai/ukernels/dwconv/dwconv_f32_f32_f32p/kai_dwconv_clamp_f32_f32_f32p1vlx1b_3x3_s1_4xc_sme2_mla.h"
      |          ^
2 errors generated.

XNNPACK/src/f32-dwconv/f32-dwconv-9pvc-minmax-neonsme2.c:12:10: error: module //XNNPACK:neonsme2_prod_microkernels_private does not depend on a module exporting 'KleidiAI/kai/kai_common.h'
   12 | #include "KleidiAI/kai/kai_common.h"
      |          ^
XNNPACK/src/f32-dwconv/f32-dwconv-9pvc-minmax-neonsme2.c:13:10: error: module //XNNPACK:neonsme2_prod_microkernels_private does not depend on a module exporting 'KleidiAI/kai/ukernels/dwconv/dwconv_f32_f32_f32p/kai_dwconv_clamp_f32_f32_f32p1vlx1b_3x3_s1_4xc_sme2_mla.h'
   13 | #include "KleidiAI/kai/ukernels/dwconv/dwconv_f32_f32_f32p/kai_dwconv_clamp_f32_f32_f32p1vlx1b_3x3_s1_4xc_sme2_mla.h"
      |          ^
2 errors generated.

Thanks. I added a direct @KleidiAI//:common dependency because the SME2 DWConv wrapper directly
includes KleidiAI/kai/kai_common.h. This should address the kai_common.h module-dependency error.
and the next internal check should show whether it correctly exports the relevant kernel header or whether another adjustment is needed.

copybara-service Bot pushed a commit that referenced this pull request Sep 4, 2026
--
6394001 by melkap01 <melike.kaptan@arm.com>:

KleidiAI f32 deptwise convolution kernel integrated

Signed-off-by: melkap01 <melike.kaptan@arm.com>

--
cbc87ae by melkap01 <melike.kaptan@arm.com>:

review comments addressed:
-XNNPACK dw conv kernel regisrty is not followed, KAI planar path seperated
-unnecassry conditional compile gurards removed
-copywrites improved

Signed-off-by: melkap01 <melike.kaptan@arm.com>

--
685a6b0 by melkap01 <melike.kaptan@arm.com>:

tests improved to use the transient_indirection_buffer getter

Signed-off-by: melkap01 <melike.kaptan@arm.com>

--
962e9b8 by melkap01 <melike.kaptan@arm.com>:

Dedicated KleidiAI path from topdown configuration implementaed as review suggested

Signed-off-by: melkap01 <melike.kaptan@arm.com>

--
6a858e3 by melkap01 <melike.kaptan@arm.com>:

test error fix

Signed-off-by: melkap01 <melike.kaptan@arm.com>

--
894d53a by melkap01 <melike.kaptan@arm.com>:

KleidiAI config defined and compile guards removed from the two source files

Signed-off-by: melkap01 <melike.kaptan@arm.com>

--
640640d by melkap01 <melike.kaptan@arm.com>:

guards removed from dwconv-config.c

Signed-off-by: melkap01 <melike.kaptan@arm.com>

--
528fa26 by melkap01 <melike.kaptan@arm.com>:

dependency translation addressed

Signed-off-by: melkap01 <melike.kaptan@arm.com>
FUTURE_COPYBARA_INTEGRATE_REVIEW=#11046 from melkap01-Arm:KleidiAI_f32_dwconv_kernel_integration 528fa26
PiperOrigin-RevId: 973908152
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants