fix(linalg): register missing Int16/Uint16/Bool rows in CPU Outer dispatch (fixes segfault) - #1099
fix(linalg): register missing Int16/Uint16/Bool rows in CPU Outer dispatch (fixes segfault)#1099yingjerkao wants to merge 1 commit into
Conversation
…patch
linalg::Outer segfaulted for Int16, Uint16, or Bool CPU inputs. The
CPU Outer_ii dispatch table in linalg_internal_interface.cpp was never
populated with the Int16/Uint16/Bool *source* rows (every other source
dtype has a full 11-entry row; these three had none). Outer.cpp casts
both operands to the promoted dtype and dispatches through
Outer_ii[out_dtype][out_dtype]; when out_dtype is Int16/Uint16/Bool that
entry was a null function pointer, so the call jumped to address 0 and
crashed. Every sibling table (MM_ii, Sum_ii, Diag_ii, Matmul_ii,
Matvec_ii, and GPU cuOuter_ii) already registers these dtypes, so the
omission was an oversight, not intent.
Fix: register the 33 missing rows. The Outer_internal_{i16,u16,b}t*
implementations already exist and are declared -- this is registration
only, no kernel or algorithm change.
Testing: new tests/linalg_test/Outer_test.cpp (OuterDtypeCoverage:
Int16, Uint16, Bool, Int16xBool) checks independently hand-computed
values. On the pre-fix binary these crash with an ASan SEGV at address
0x0 (null function-pointer call) at the linalg::Outer call site;
post-fix all four pass. Existing DtypePromotion.OuterComplexfloatDouble
and LinalgKronTest tests still pass. clang-format (v14) clean.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
There was a problem hiding this comment.
Code Review
This pull request registers the missing Int16, Uint16, and Bool source rows in the CPU Outer_ii dispatch table to prevent null function pointer dereferences and crashes when using these types as the left operand in linalg::Outer. It also adds comprehensive regression tests in tests/linalg_test/Outer_test.cpp to verify correctness across these data types. The review feedback points out several instances in the new test file where legacy cytnx_uint64 typedefs are used as loop counters, which violates the repository style guide's preference for standard C++ types like size_t or uint64_t in new code.
Important
The consumer version of Gemini Code Assist on GitHub is being sunset. Starting June 18, 2026, new organization installations will be blocked, and all code review activity will officially cease on July 17, 2026.
For more details on the timeline and next steps, please review the Help Documentation.
| for (cytnx_uint64 i = 0; i < 3; i++) | ||
| for (cytnx_uint64 j = 0; j < 2; j++) |
There was a problem hiding this comment.
According to the repository style guide, legacy cytnx_XXXX typedefs (such as cytnx_uint64) should not be propagated as ordinary loop counters in new or touched code. Standard C++ types like size_t or uint64_t should be preferred instead.
| for (cytnx_uint64 i = 0; i < 3; i++) | |
| for (cytnx_uint64 j = 0; j < 2; j++) | |
| for (size_t i = 0; i < 3; i++) | |
| for (size_t j = 0; j < 2; j++) |
References
- For new or touched code, avoid propagating legacy cytnx_XXXX typedefs as ordinary loop counters, sizes, flags, or local arithmetic types. Prefer standard C++ types unless the value is specifically a dtype-backed tensor/storage scalar. (link)
| for (cytnx_uint64 i = 0; i < 2; i++) | ||
| for (cytnx_uint64 j = 0; j < 3; j++) |
There was a problem hiding this comment.
According to the repository style guide, legacy cytnx_XXXX typedefs (such as cytnx_uint64) should not be propagated as ordinary loop counters in new or touched code. Standard C++ types like size_t or uint64_t should be preferred instead.
| for (cytnx_uint64 i = 0; i < 2; i++) | |
| for (cytnx_uint64 j = 0; j < 3; j++) | |
| for (size_t i = 0; i < 2; i++) | |
| for (size_t j = 0; j < 3; j++) |
References
- For new or touched code, avoid propagating legacy cytnx_XXXX typedefs as ordinary loop counters, sizes, flags, or local arithmetic types. Prefer standard C++ types unless the value is specifically a dtype-backed tensor/storage scalar. (link)
| for (cytnx_uint64 i = 0; i < 2; i++) | ||
| for (cytnx_uint64 j = 0; j < 3; j++) |
There was a problem hiding this comment.
According to the repository style guide, legacy cytnx_XXXX typedefs (such as cytnx_uint64) should not be propagated as ordinary loop counters in new or touched code. Standard C++ types like size_t or uint64_t should be preferred instead.
| for (cytnx_uint64 i = 0; i < 2; i++) | |
| for (cytnx_uint64 j = 0; j < 3; j++) | |
| for (size_t i = 0; i < 2; i++) | |
| for (size_t j = 0; j < 3; j++) |
References
- For new or touched code, avoid propagating legacy cytnx_XXXX typedefs as ordinary loop counters, sizes, flags, or local arithmetic types. Prefer standard C++ types unless the value is specifically a dtype-backed tensor/storage scalar. (link)
| for (cytnx_uint64 i = 0; i < 2; i++) | ||
| for (cytnx_uint64 j = 0; j < 2; j++) |
There was a problem hiding this comment.
According to the repository style guide, legacy cytnx_XXXX typedefs (such as cytnx_uint64) should not be propagated as ordinary loop counters in new or touched code. Standard C++ types like size_t or uint64_t should be preferred instead.
| for (cytnx_uint64 i = 0; i < 2; i++) | |
| for (cytnx_uint64 j = 0; j < 2; j++) | |
| for (size_t i = 0; i < 2; i++) | |
| for (size_t j = 0; j < 2; j++) |
References
- For new or touched code, avoid propagating legacy cytnx_XXXX typedefs as ordinary loop counters, sizes, flags, or local arithmetic types. Prefer standard C++ types unless the value is specifically a dtype-backed tensor/storage scalar. (link)
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## master #1099 +/- ##
==========================================
+ Coverage 72.71% 72.77% +0.05%
==========================================
Files 226 226
Lines 28151 28184 +33
Branches 71 71
==========================================
+ Hits 20471 20510 +39
+ Misses 7659 7653 -6
Partials 21 21
Flags with carried forward coverage won't be shown. Click here to find out more.
... and 1 file with indirect coverage changes Continue to review full report in Codecov by Harness.
🚀 New features to boost your workflow:
|
IvanaGyro
left a comment
There was a problem hiding this comment.
The comments from bots are worth to be resolved. Except that LGTM.
|
Outer.cpp casts both operands to I'm wondering why this PR doesn't replace the whole mechanism by a much simpler implementation. src/linalg/Outer.cpp:24 rejects either operand unless Tensor Outer(const Tensor &lhs, const Tensor &rhs) {
cytnx_error_msg(lhs.is_void(), "[ERROR] lhs must be initialized.%s", "\n");
cytnx_error_msg(rhs.is_void(), "[ERROR] rhs must be initialized.%s", "\n");
cytnx_error_msg(lhs.device() != rhs.device(),
"[ERROR] the two tensors must be on the same device.%s", "\n");
cytnx_error_msg(lhs.rank() != 1, "[ERROR] tensor #1 should have rank-1.%s", "\n");
cytnx_error_msg(rhs.rank() != 1, "[ERROR] tensor #2 should have rank-1.%s", "\n");
return Kron(lhs, rhs).reshape(std::vector<cytnx_uint64>{lhs.shape()[0], rhs.shape()[0]});
} |
ianmccul
left a comment
There was a problem hiding this comment.
see top-level comment above
|
Addressing @ianmccul's review comment: rather than register the missing Int16/Uint16/Bool rows, #1105 reimplements |
ianmccul
left a comment
There was a problem hiding this comment.
hmm, not sure how to 'unsubmit' approval but it doesn't matter, I assume this PR is no longer relevant
|
Closing as superseded by #1105. This PR registered the missing As a result, on current
Nothing here is left to salvage, so closing rather than rebasing. Thanks! |
Problem: PR #1100 added more thorough GPU Kron/Outer tests but was stacked on the (now-closed) #1099 branch and functionally superseded by #1105/#1106, so it could not be merged as-is. Fix: fold #1100's stronger tests into master's existing tests/gpu/linalg_test/{Kron,Outer}_test.cpp, keeping master's unique regression cases: - 2-D hand-computed Kron/Outer with fractional and negative values (independent expected values, not a same-path comparison); - complex x complex hand-computed cases; - the ComplexFloat x Double -> ComplexDouble promotion discriminator (#984/#999); - a broad GPU-vs-CPU differential over every real/complex dtype and several shapes (GpuMatchesCpuAllDtypes), with the looser ComplexFloat tolerance at ~1e6-magnitude products (matches Mul_test); - kept master's Int16 hand-computed Kron and the #1099 Int16-diagonal Outer segfault regression guard. Testing: built gpu_test_main (CUDA 13, RTX 4070 Ti SUPER, USE_CUTENSOR=OFF) and ran GpuKron.*/GpuOuter.* -> 11/11 passed. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Problem: PR #1100 added more thorough GPU Kron/Outer tests but was stacked on the (now-closed) #1099 branch and functionally superseded by #1105/#1106, so it could not be merged as-is. Fix: fold #1100's stronger tests into master's existing tests/gpu/linalg_test/{Kron,Outer}_test.cpp, keeping master's unique regression cases: - 2-D hand-computed Kron/Outer with fractional and negative values (independent expected values, not a same-path comparison); - complex x complex hand-computed cases; - the ComplexFloat x Double -> ComplexDouble promotion discriminator (#984/#999); - a broad GPU-vs-CPU differential over every real/complex dtype and several shapes (GpuMatchesCpuAllDtypes), with the looser ComplexFloat tolerance at ~1e6-magnitude products (matches Mul_test); - kept master's Int16 hand-computed Kron and the #1099 Int16-diagonal Outer segfault regression guard. Testing: built gpu_test_main (CUDA 13, RTX 4070 Ti SUPER, USE_CUTENSOR=OFF) and ran GpuKron.*/GpuOuter.* -> 11/11 passed. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
…1003) Addresses the Codex review on #1108. Naming and types (CLAUDE.md touched-code contract): loop indices and dimension locals move from cytnx_uint64 to std::size_t, and bR/bC become b_rows/b_cols. These are ordinary test counters, not dtype-backed tensor scalars. The mixed-precision discriminator did not discriminate. Its operands (2.0, -0.5) are all exactly representable in float32, so an implementation that reported ComplexDouble but multiplied through ComplexFloat produced bit-identical results and the test passed. The Double operands are now 0.1 and 3.3, which float32 cannot represent: a narrowed multiply lands 5.9e-9 to 1.9e-7 away from the double product, against a tolerance tightened from 1e-6 to 1e-12. Expected values are evaluated in host double arithmetic rather than written as decimals, because 1.5 * 3.3 is 4.949999999999999 and the exact product is the point. Same change to the Outer counterpart. Verified this is now a real discriminator: substituting the value a float32 path would produce (4.949999809265137 for 1.5 * 3.3) fails the test, and it passes again on the double product. Under the previous operands that substitution was a no-op. Bool is no longer skipped in either sweep. The claim that "Kron has no Bool kernel" was wrong -- cuKron_general is templated over bool and src/linalg/Outer.cpp documents Bool as covered on both CPU and GPU after the missing dispatch row was fixed (#1099). Both sweeps now run it and pass, so the comment was the only thing standing between this path and coverage. Mixed signed/unsigned pairs get hand-computed tests in both files. Both sweeps build their operands from a single dtype, leaving every off-diagonal promotion row untested. type_promote steps an unsigned operand up to the adjacent signed type when the other side is signed and higher precision, so Int32 (x) Uint32 is Int32 and Uint32 (x) Int16 is Int32 -- negative products must survive rather than wrap. Expected dtypes and values are asserted independently. The broad GPU-vs-CPU sweeps are left as consistency checks and now say so. They share an implementation with their oracle and cannot catch a regression in code common to both paths; the hand-computed tests carry the correctness burden, and they now cover a wider dtype surface than before. Kron.Gpu* and Outer.Gpu*: 13/13 pass on an RTX 4070 Ti SUPER (CUDA 13.0). clang-format-14 clean. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Problem: PR #1100 added more thorough GPU Kron/Outer tests but was stacked on the (now-closed) #1099 branch and functionally superseded by #1105/#1106, so it could not be merged as-is. Fix: fold #1100's stronger tests into master's existing tests/gpu/linalg_test/{Kron,Outer}_test.cpp, keeping master's unique regression cases: - 2-D hand-computed Kron/Outer with fractional and negative values (independent expected values, not a same-path comparison); - complex x complex hand-computed cases; - the ComplexFloat x Double -> ComplexDouble promotion discriminator (#984/#999); - a broad GPU-vs-CPU differential over every real/complex dtype and several shapes (GpuMatchesCpuAllDtypes), with the looser ComplexFloat tolerance at ~1e6-magnitude products (matches Mul_test); - kept master's Int16 hand-computed Kron and the #1099 Int16-diagonal Outer segfault regression guard. Testing: built gpu_test_main (CUDA 13, RTX 4070 Ti SUPER, USE_CUTENSOR=OFF) and ran GpuKron.*/GpuOuter.* -> 11/11 passed. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
…1003) Addresses the Codex review on #1108. Naming and types (CLAUDE.md touched-code contract): loop indices and dimension locals move from cytnx_uint64 to std::size_t, and bR/bC become b_rows/b_cols. These are ordinary test counters, not dtype-backed tensor scalars. The mixed-precision discriminator did not discriminate. Its operands (2.0, -0.5) are all exactly representable in float32, so an implementation that reported ComplexDouble but multiplied through ComplexFloat produced bit-identical results and the test passed. The Double operands are now 0.1 and 3.3, which float32 cannot represent: a narrowed multiply lands 5.9e-9 to 1.9e-7 away from the double product, against a tolerance tightened from 1e-6 to 1e-12. Expected values are evaluated in host double arithmetic rather than written as decimals, because 1.5 * 3.3 is 4.949999999999999 and the exact product is the point. Same change to the Outer counterpart. Verified this is now a real discriminator: substituting the value a float32 path would produce (4.949999809265137 for 1.5 * 3.3) fails the test, and it passes again on the double product. Under the previous operands that substitution was a no-op. Bool is no longer skipped in either sweep. The claim that "Kron has no Bool kernel" was wrong -- cuKron_general is templated over bool and src/linalg/Outer.cpp documents Bool as covered on both CPU and GPU after the missing dispatch row was fixed (#1099). Both sweeps now run it and pass, so the comment was the only thing standing between this path and coverage. Mixed signed/unsigned pairs get hand-computed tests in both files. Both sweeps build their operands from a single dtype, leaving every off-diagonal promotion row untested. type_promote steps an unsigned operand up to the adjacent signed type when the other side is signed and higher precision, so Int32 (x) Uint32 is Int32 and Uint32 (x) Int16 is Int32 -- negative products must survive rather than wrap. Expected dtypes and values are asserted independently. The broad GPU-vs-CPU sweeps are left as consistency checks and now say so. They share an implementation with their oracle and cannot catch a regression in code common to both paths; the hand-computed tests carry the correctness burden, and they now cover a wider dtype surface than before. Kron.Gpu* and Outer.Gpu*: 13/13 pass on an RTX 4070 Ti SUPER (CUDA 13.0). clang-format-14 clean. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Salvaged from the closed #1100. Extends the GPU Kron/Outer coverage from 6 cases to 14, with every expected value derived independently -- literals or a per-element recompute from the raw inputs -- rather than by comparing against another Cytnx path that could share the same bug. Coverage added: - Fractional and negative Double values, recomputed per element from the inputs. - Complex x complex, hand-computed. - The #984/#999 promotion discriminator: ComplexFloat (x) Double must produce ComplexDouble, computed and stored through that output type. - Int16, including negative products. - Mixed signed/unsigned promotion: Int32 (x) Uint32 -> Int32 and Uint32 (x) Int16 -> Int32. Both sweeps build their operands from a single dtype, so every off-diagonal promotion row was previously untested. Both cases carry negative products, so a result computed or stored through an unsigned type wraps and fails loudly. - Kron's rank-0 scalar path. `if (lhs.is_scalar() || rhs.is_scalar()) return lhs * rhs` (Kron.cpp:26) is a broadcast multiply, not the Kronecker kernel, and promotes through Mul rather than Kron's own output-type selection. Shape {1, 1} does not reach it -- that is still rank 2 -- so it had no coverage. Both operand orders plus a mixed-dtype pair. Two corrections to earlier claims in this file: The mixed-precision discriminator did not discriminate. Its operands (2.0, -0.5, 1.5, -2.5, 4.0) are all exactly representable in float32, so an implementation reporting ComplexDouble while multiplying through ComplexFloat produced a bit-identical result and passed. The Double operands are now 0.1 and 3.3, which float32 cannot represent, with the tolerance tightened from 1e-6 to 1e-12: a narrowed multiply lands 5.9e-9 to 1.9e-7 from the double product. Verified it now discriminates -- substituting the value a float32 path produces (4.949999809265137 for 1.5 * 3.3) fails the test, and it passes on the double product. Under the old operands that substitution was a no-op. "Kron has no Bool kernel" was wrong. cuKron_general is templated over bool and src/linalg/Outer.cpp documents Bool as covered on CPU and GPU after the missing dispatch row was fixed (#1099). The skip is removed from both sweeps and Bool passes, so that comment was the only thing keeping a historically broken path out of coverage. Uint16 sweep operands are bounded to [0, 1000]. GetRandRange gives the narrow types their full numeric_limits range, so Uint16 drew from [0, 65535]; two such values promote to int before multiplying and 65535 * 65535 exceeds INT_MAX, which is undefined behaviour in the CPU oracle and GPU kernel alike and made the comparison compiler-dependent. The underlying kernel hazard is filed as #1160 -- bounding the inputs stops this test depending on it, it does not fix it. Test names follow the suite convention (`TEST(Kron, Gpu...)`), which this PR had been converting six existing cases *off*; across tests/gpu the split is 406 such names against 4. The complex hand-computed cases assert through AreNearlyEqTensor, which also checks device, dtype, shape and contiguity. GPU suite: 100% tests passed, 0 failed out of 831. Refs #1003. Refs #1099, #1160. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Problem
linalg::Outersegfaults forInt16,Uint16, orBoolinputs on the CPU.Outer.cpppromotes both operands to a common dtype and dispatches throughOuter_ii[out_dtype][out_dtype]. The CPUOuter_iidispatch table inlinalg_internal_interface.cppwas never populated with the Int16, Uint16,or Bool source rows — every other source dtype has a full 11-entry row, but
these three had none. So when the promoted dtype is
Int16/Uint16/Bool,Outer_ii[out_dtype][out_dtype]is a null function pointer, and the call jumpsto address
0:Every sibling dispatch table in the same constructor —
MM_ii,Sum_ii,Diag_ii,Matmul_ii,Matvec_ii, and even the GPUcuOuter_ii— alreadyregisters these dtypes, so the omission is an oversight, not intent. (The GPU
path works; this is CPU-only.)
Reproduces on
master(86cb96c). Minimal repro:Fix
Register the 33 missing rows (
Int16,Uint16,Boolsources × 11 targetdtypes). The
Outer_internal_{i16,u16,b}t*implementations already exist andare declared — this is registration only, no kernel or algorithm change.
Testing
tests/linalg_test/Outer_test.cpp(OuterDtypeCoverage:Int16,Uint16,Bool,Int16xBool) checks independently hand-computed values(dtype and value, with negative and boolean cases).
SEGV above at the
linalg::Outercall site; post-fix all four pass.DtypePromotion.OuterComplexfloatDoubleandLinalgKronTesttestsstill pass.
clang-format(v14) clean.