Skip to content

Add struct types to Halide - #9416

Draft
alexreinking wants to merge 3 commits into
mainfrom
alexreinking/struct-types
Draft

alexreinking wants to merge 3 commits into
mainfrom
alexreinking/struct-types

Conversation

@alexreinking

@alexreinking alexreinking commented Sep 1, 2026

Copy link
Copy Markdown
Member

This PR adds struct types to Halide. This allows for inputs and outputs to be defined in terms of popular wire formats, even if they aren't prepared for efficient computation. For example, you can define GGML's q5_0 and q8_0 formats like so:

Type q5_0 = Type::Struct({{"d", Float(16)}, {"qh", UInt(32)}, {"qs", UInt(8), 16}});
Type q8_0 = Type::Struct({{"d", Float(16)}, {"qs", Int(8), 32}});

Then you can write a pipeline that consumes them:

ImageParam x{q4_0_type(), 1, "x"};
ImageParam y{q8_0_type(), 1, "y"};

Var b("b"), k("k"), u("u");
RDom r(0, x.dim(0).extent(), 0, 32, "r");  // block, quant

// Dequantize one 4-bit weight to float: quant k is a nibble of the
// packed byte k % 16 -- its low nibble for k < 16, its high nibble
// otherwise -- biased by -8 to signed [-8, 7], times the block delta.
Expr nib = field(x(b), "qs")[k % 16];  // uint8
Func x_wt("x_wt");
x_wt(k, b) = cast<float>(field(x(b), "d")) * 
             (cast<int32_t>(select(k < 16, nib % 16, nib / 16)) - 8);

// Dequantize one int8 activation to float: quant k is int8 k, times the
// block delta.
Func y_wt("y_wt");
y_wt(k, b) = cast<float>(field(y(b), "d")) * 
             cast<int32_t>(field(y(b), "qs")[k]);

// Dequantize each side and sum every product.
Func qdot{"qdot"};
qdot() = 0.0f;
qdot() += x_wt(r[1], r[0]) * y_wt(r[1], r[0]);

As you can see, struct types support both individual elements, as well as fixed-size array elements. Fields are accessed with a named field(expr, "name") intrinsic. The tests include additional cases for struct-typed fields and output structs.

This adjusts the Type and halide_type_t representations in the following ways:

  1. Type: like Handle, a Struct type has a compile-time known field layout.
  2. halide_type_t: structs get their own kind, and the reserved field records the number of bytes in the struct.

All layouts are expected to be packed. No padding is inserted around or in between field elements.

Breaking changes

None—it's a new feature.

Checklist

  • Tests added or updated (not required for docs, CI config, or typo fixes)
  • Documentation updated (if public API changed)
  • Python bindings updated (if public API changed)
  • Benchmarks are included here if the change is intended to affect performance.
  • Commits include AI attribution where applicable (see Code of Conduct)

@alexreinking
alexreinking force-pushed the alexreinking/struct-types branch from 9f99483 to 069ce04 Compare September 1, 2026 17:16
@alexreinking
alexreinking changed the base branch from alexreinking/hoist-splitting to alexreinking/rfactor-hoisting September 1, 2026 17:32
@alexreinking
alexreinking force-pushed the alexreinking/struct-types branch from 069ce04 to 50683f7 Compare September 1, 2026 18:21
@alexreinking
alexreinking force-pushed the alexreinking/struct-types branch from 50683f7 to e9123a1 Compare September 1, 2026 19:06
@codecov

codecov Bot commented Sep 2, 2026

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 64.13662% with 189 lines in your changes missing coverage. Please review.
✅ Project coverage is 70.03%. Comparing base (d33fd25) to head (1444b20).
⚠️ Report is 1 commits behind head on main.

Files with missing lines Patch % Lines
src/IROperator.cpp 59.23% 34 Missing and 19 partials ⚠️
src/LowerStructTypes.cpp 70.05% 30 Missing and 20 partials ⚠️
src/Type.cpp 73.14% 10 Missing and 19 partials ⚠️
src/IRPrinter.cpp 0.00% 10 Missing and 2 partials ⚠️
src/CodeGen_Vulkan_Dev.cpp 0.00% 10 Missing ⚠️
src/CodeGen_D3D12Compute_Dev.cpp 0.00% 8 Missing ⚠️
src/Type.h 80.00% 2 Missing and 3 partials ⚠️
src/FuseGPUThreadLoops.cpp 0.00% 3 Missing and 1 partial ⚠️
src/Deserialization.cpp 81.25% 1 Missing and 2 partials ⚠️
src/IROperator.h 57.14% 3 Missing ⚠️
... and 8 more
Additional details and impacted files
@@            Coverage Diff             @@
##             main    #9416      +/-   ##
==========================================
- Coverage   70.21%   70.03%   -0.18%     
==========================================
  Files         261      262       +1     
  Lines       79792    80250     +458     
  Branches    19451    19588     +137     
==========================================
+ Hits        56022    56207     +185     
- Misses      17966    18111     +145     
- Partials     5804     5932     +128     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@alexreinking
alexreinking force-pushed the alexreinking/struct-types branch from e9123a1 to 425cb95 Compare September 3, 2026 17:10
@alexreinking
alexreinking force-pushed the alexreinking/struct-types branch from 425cb95 to 2740239 Compare September 4, 2026 09:09
@alexreinking
alexreinking force-pushed the alexreinking/struct-types branch from 1464bc0 to 168ce01 Compare September 4, 2026 20:16
@alexreinking
alexreinking force-pushed the alexreinking/struct-types branch from 168ce01 to e272037 Compare September 11, 2026 09:02
Base automatically changed from alexreinking/rfactor-hoisting to main September 13, 2026 01:03
@alexreinking
alexreinking force-pushed the alexreinking/struct-types branch from e272037 to c5e23fa Compare September 13, 2026 01:03
alexreinking and others added 3 commits September 15, 2026 11:06
Introduce first-class packed struct types (Type::Struct) modeled
faithfully in the type system:

- A dedicated ABI type code halide_type_struct=5; a struct's packed byte
  size rides in the halide_type_t reserved field, so struct-typed buffers
  have correct element size/strides. StructTypeInfo (field layout) is
  interned like handle metadata, keeping Type at 8 bytes. is_uint()/etc.
  are honestly false for structs, so no numeric special-casing is needed.

- field()/pack_struct() intrinsics with byte-addressed lowering
  (LowerStructTypes), plus per-field pack_struct ergonomics: an array
  field is filled by a gather() packet, a gather(extent, gen) generator,
  a single expression with one swept `_` placeholder (index arithmetic
  allowed), or a field() copy of a whole same-typed field.

- Python bindings for the type, field/pack_struct/gather, and the
  per-field forms.

- Tests: correctness (CPU + GPU), error cases, Python, and an ARM codegen
  test implementing ggml's q4_0/q8_0 dot product that verifies the packed
  qs arrays lower to dense 128-bit vector loads.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
- ConstantBounds.cpp/PyType.cpp: drop two unnecessary-copy clang-tidy findings.
- struct_type_dot_product.cpp: back the test buffers with 16-byte-aligned
  storage instead of std::vector, whose allocator doesn't guarantee that on
  32-bit ABIs.
- FuseGPUThreadLoops.cpp: relax the GPU shared/global allocation clustering
  assert from "both types are powers of two bytes" to the actual requirement
  ("widest type is a whole multiple of the cluster's byte-granularity type"),
  so a non-power-of-two struct size (e.g. 12 bytes) can share GPU memory with
  other types.
- LowerStructTypes.cpp: build a packed float field (e.g. a struct's fp16
  delta) via a scalar shift/or chain instead of concat_bits's
  vector-shuffle-then-reinterpret lowering. Some AArch64 backends (LLVM < 23)
  can't legalize a bitcast straight from a vector to a scalar half, and any
  chain of pure bitcasts collapses back to that illegal form during
  optimization -- only avoiding the vector shape in the first place works.
- struct_type.cpp: move constant_bounds_test() (dead code -- declared and
  defined but never called since Halide's old internal-test runner was
  removed) into a proper correctness test.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
- CodeGen_D3D12Compute_Dev: struct-backed groupshared/local arrays are
  stored as raw bytes, but Load/Store still consulted the original
  Type::Struct(...) for cast/promotion bookkeeping, crashing print_cast's
  internal_assert(source_type.is_uint()) since a struct is neither int,
  uint, nor float. Treat the storage element as Int(8) (matching what
  print_type_maybe_storage actually emits) and scale the groupshared
  array's declared element count by the struct's byte size.
- CodeGen_Vulkan_Dev: visit(Allocate) passed Type::Struct(...) directly
  to SpvBuilder::declare_type, which has no notion of struct types
  ("SPIRV: Unsupported type"). Use UInt(8) as the element type for
  declaration and later Load/Store bookkeeping instead.
- Serialization/Deserialization: add a Struct TypeCode and StructField
  table to the flatbuffers schema so struct types (and their field
  layout) round-trip instead of being rejected outright. This is needed
  because CI runs JIT compiles through a serialize/deserialize
  round-trip for regression testing.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
@alexreinking
alexreinking force-pushed the alexreinking/struct-types branch from 166d1d5 to 1444b20 Compare September 15, 2026 15:06
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant