Skip to content

subgraph: validate filter shape dims[1..3] in conv/deconv/depthwise-conv - #11119

Open
tinhien11 wants to merge 2 commits into
google:masterfrom
tinhien11:fix/filter-shape-validation
Open

subgraph: validate filter shape dims[1..3] in conv/deconv/depthwise-conv#11119
tinhien11 wants to merge 2 commits into
google:masterfrom
tinhien11:fix/filter-shape-validation

Conversation

@tinhien11

Copy link
Copy Markdown

subgraph: validate filter shape dims[1..3] in conv/deconv/depthwise-conv

Summary

xnn_define_convolution_2d validated only filter_value->shape.dim[0] (== group_output_channels * groups) but not dim[1], dim[2], dim[3] against kernel_height, kernel_width, and group_input_channels. When a filter tensor is declared with dims[1..3] smaller than the convolution's kernel parameters, weight packing (xnn_pack_f32_conv_goki_w) reads nc * ks * kc elements from the filter buffer — where ks = kernel_height * kernel_width comes from the convolution parameters, not the filter's declared shape — causing a heap out-of-bounds read during xnn_create_runtime_v2.

xnn_define_deconvolution_2d and xnn_define_depthwise_convolution_2d had no filter shape validation at all (not even dim[0]).

This PR adds full 4D filter shape validation to all three operators, rejecting mismatched shapes at define-time with xnn_status_invalid_parameter.

Root Cause

Filter shape layout (NHWC convention):

  • Conv/Deconv: [groups * group_output_channels, kernel_height, kernel_width, group_input_channels]
  • Depthwise: [1, kernel_height, kernel_width, input_channels * depth_multiplier]

Weight packing reads nc * ks * kc floats from the filter buffer, where:

  • nc = group_output_channels (from conv params)
  • ks = kernel_height * kernel_width (from conv params)
  • kc = group_input_channels (from conv params)

The packed-weights destination buffer is sized from convolution parameters (safe — no OOB write), but the source filter buffer is sized from the declared filter shape. When dims[1..3] are smaller than the conv params, the read overflows the filter buffer.

PoC

Filter declared as {32, 3, 3, 3} = 864 floats. Convolution defined with kernel_height=5, kernel_width=5, group_input_channels=3, group_output_channels=32, groups=1. The dim[0] check passed (32 == 32 * 1). Weight packing reads 32 * 25 * 3 = 2400 floats from the 864-float buffer → 1536-float (6144-byte) heap OOB read.

ASan output (before fix, HEAD 6c48650)

==51790==ERROR: AddressSanitizer: heap-buffer-overflow on address 0x61f000000e90
READ of size 4 at 0x61f000000e90 thread T0
    #0 __asan_memmove+0x2a4
    #1 xnn_pack_f32_conv_goki_w packing.cc:3553
    #2 create_igemm convolution-nhwc.c:452
    #3 create_convolution2d_nhwc convolution-nhwc.c:806
    #4 xnn_create_convolution2d_nhwc_pf32 convolution-nhwc.c:2559
    #5 create_convolution_operator convolution-2d.c:261
    #6 xnn_create_runtime_v4 runtime.c:727
    #7 xnn_create_runtime_v2 runtime.c:219

0x61f000000e90 is located 144 bytes after 3456-byte region [0x61f000000080,0x61f000000e00)
SUMMARY: AddressSanitizer: heap-buffer-overflow in xnn_pack_f32_conv_goki_w packing.cc:3553

After fix

[*] calling xnn_define_convolution_2d (kernel 5x5, filter shape {32,3,3,3})...
xnn_define_convolution_2d failed: 2  (xnn_status_invalid_parameter)

Bug blocked at define-time, no crash at runtime.

Testing

  • convolution-2d-test: 11/11 PASSED
  • deconvolution-2d-test: 8/8 PASSED
  • depthwise-convolution-2d-test: 8/8 PASSED

Built with -fsanitize=address -g -O1, all tests pass under ASan.

Fixes #11118

xnn_define_convolution_2d validated only filter_value->shape.dim[0]
(== group_output_channels * groups) but not dims[1..3] against
kernel_height, kernel_width, and group_input_channels. When a filter
tensor is declared with dims[1..3] smaller than the convolution's
kernel parameters, weight packing (xnn_pack_f32_conv_goki_w) reads
nc * ks * kc elements from the filter buffer — where ks comes from
the convolution parameters, not the filter's declared shape — causing
a heap out-of-bounds read during xnn_create_runtime_v2.

xnn_define_deconvolution_2d and xnn_define_depthwise_convolution_2d
had no filter shape validation at all.

Add full 4D filter shape validation to all three operators, rejecting
mismatched shapes at define-time with xnn_status_invalid_parameter.

Fixes google#11118
copybara-service Bot pushed a commit that referenced this pull request Sep 3, 2026
--
041763c by tinhien11 <thanhtinpk092007@gmail.com>:

subgraph: validate filter shape dims[1..3] in conv/deconv/depthwise-conv

xnn_define_convolution_2d validated only filter_value->shape.dim[0]
(== group_output_channels * groups) but not dims[1..3] against
kernel_height, kernel_width, and group_input_channels. When a filter
tensor is declared with dims[1..3] smaller than the convolution's
kernel parameters, weight packing (xnn_pack_f32_conv_goki_w) reads
nc * ks * kc elements from the filter buffer — where ks comes from
the convolution parameters, not the filter's declared shape — causing
a heap out-of-bounds read during xnn_create_runtime_v2.

xnn_define_deconvolution_2d and xnn_define_depthwise_convolution_2d
had no filter shape validation at all.

Add full 4D filter shape validation to all three operators, rejecting
mismatched shapes at define-time with xnn_status_invalid_parameter.

Fixes #11118

FUTURE_COPYBARA_INTEGRATE_REVIEW=#11119 from tinhien11:fix/filter-shape-validation 041763c
PiperOrigin-RevId: 975821639
The new conv/deconv/depthwise filter shape checks reject the dummy
geometry used by three structural tests that never execute the graph:

- SUBGRAPH_FP16.convolution_{weights,bias}_used_by_another_node declared
  a 1x1 filter ({2, 1, 1, 3}) against Kernel{3, 3}; use a 1x1 kernel so
  the shapes are self-consistent. Output extents are unchanged.
- SUBGRAPH_NCHW.bottleneck declared filter google#10 as {4, 1, 1, 4} for a
  convolution with group_input_channels=8; use {4, 1, 1, 8}.
copybara-service Bot pushed a commit that referenced this pull request Sep 4, 2026
--
041763c by tinhien11 <thanhtinpk092007@gmail.com>:

subgraph: validate filter shape dims[1..3] in conv/deconv/depthwise-conv

xnn_define_convolution_2d validated only filter_value->shape.dim[0]
(== group_output_channels * groups) but not dims[1..3] against
kernel_height, kernel_width, and group_input_channels. When a filter
tensor is declared with dims[1..3] smaller than the convolution's
kernel parameters, weight packing (xnn_pack_f32_conv_goki_w) reads
nc * ks * kc elements from the filter buffer — where ks comes from
the convolution parameters, not the filter's declared shape — causing
a heap out-of-bounds read during xnn_create_runtime_v2.

xnn_define_deconvolution_2d and xnn_define_depthwise_convolution_2d
had no filter shape validation at all.

Add full 4D filter shape validation to all three operators, rejecting
mismatched shapes at define-time with xnn_status_invalid_parameter.

Fixes #11118

--
c2ec208 by tinhien11 <thanhtinpk092007@gmail.com>:

subgraph tests: fix filter shapes rejected by define-time validation

The new conv/deconv/depthwise filter shape checks reject the dummy
geometry used by three structural tests that never execute the graph:

- SUBGRAPH_FP16.convolution_{weights,bias}_used_by_another_node declared
  a 1x1 filter ({2, 1, 1, 3}) against Kernel{3, 3}; use a 1x1 kernel so
  the shapes are self-consistent. Output extents are unchanged.
- SUBGRAPH_NCHW.bottleneck declared filter #10 as {4, 1, 1, 4} for a
  convolution with group_input_channels=8; use {4, 1, 1, 8}.

FUTURE_COPYBARA_INTEGRATE_REVIEW=#11119 from tinhien11:fix/filter-shape-validation c2ec208
PiperOrigin-RevId: 975821639
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Heap buffer overflow read in weight packing when filter shape dims[1..3] mismatch convolution kernel params

2 participants