test(linalg): GPU correctness coverage for Kron and Outer (#1003, confirms #999) by yingjerkao · Pull Request #1100 · Cytnx-dev/Cytnx

yingjerkao · 2026-07-20T04:45:16Z

Problem

Kron and Outer had no GPU tests — their CUDA paths (cuKron_internal.cuh, cuOuter_internal.cu) were entirely uncovered. This is the coverage the #1003 Outer/Kron typed-dispatch consolidation needs as a safety net, and it also confirms #999 is resolved.

Fix

Add tests/gpu/linalg_test/Kron_test.cpp and Outer_test.cpp. Expected values are computed independently from the operation definition (literals or per-element recompute), not by comparing against another Cytnx path that could share the same bug:

Double / ComplexDouble — hand-written literal results (Kron also covers fractional + negative values).
Mixed ComplexFloat × Double → ComplexDouble — the promotion discriminator behind GPU build broken: cuKron fails to compile (CUDA 13) after the #982 type_promote change #999/fix(linalg): replace lower-enum-index dtype selection with Type.type_promote #984: the product must be computed and stored through the promoted output type. Checks output dtype and value.
Broad GPU-vs-CPU cross-check over every real/complex dtype as secondary coverage. ComplexFloat uses a looser tolerance because ~1e6-magnitude float32 complex products diverge from the CPU at the ULP; a single real multiply is bit-identical.

Inputs are built on the CPU and moved with .to(Device.cuda) (GPU arange ignores a non-unit step for real dtypes, #1070).

Testing

All 9 tests pass on an RTX 4070 Ti SUPER (CUDA 13.0, sm_89):

[  PASSED  ] 9 tests.   (GpuKron: 5, GpuOuter: 4)

Verified against a build with the parent commit's Outer_ii fix applied (the Outer cross-check's CPU reference needs the Int16/Uint16 dispatch rows). The cuKron/cuOuter/Kron.cpp/Outer.cpp sources under test are identical to origin/master.

Notes

Stacked on fix(linalg): register missing Int16/Uint16/Bool rows in CPU Outer dispatch (fixes segfault) #1099 (the CPU Outer Int16/Uint16/Bool dispatch fix). The Outer GPU cross-check's CPU reference crashes without it. Will retarget to master once fix(linalg): register missing Int16/Uint16/Bool rows in CPU Outer dispatch (fixes segfault) #1099 merges.
No GPU CI runner exists, so these compile/run only under a local USE_CUDA build (as done here).

Part of #1003. Confirms #999.

Kron and Outer had no GPU tests. Add tests/gpu/linalg_test/Kron_test.cpp and Outer_test.cpp exercising the GPU paths with independently hand-computed expected values (not GPU-vs-CPU as the primary oracle): - Double and ComplexDouble with literal and per-element-recomputed results (Kron also covers fractional/negative values); - the mixed ComplexFloat x Double -> ComplexDouble promotion discriminator (output dtype AND value), the case that motivated #999 / #984 -- the product must be computed and stored through the promoted output type; - a broad GPU-vs-CPU cross-check over every real/complex dtype as secondary coverage (ComplexFloat gets a looser tolerance because ~1e6-magnitude float32 complex products diverge from the CPU one at the ULP; a single real multiply is bit-identical). Inputs are built on the CPU and moved with .to(Device.cuda) since GPU arange ignores a non-unit step for real dtypes (#1070). The Outer cross-check's CPU reference relies on the Int16/Uint16 Outer dispatch fix in the parent commit; without it those dtypes crash. Testing: all 9 tests pass on an RTX 4070 Ti SUPER (CUDA 13.0, sm_89). This also confirms #999 (the GPU library builds clean and Kron is numerically correct). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

gemini-code-assist

Code Review

This pull request adds GPU test suites for the Kronecker product (Kron) and outer product (Outer) linear algebra operations, verifying correctness across various data types, fractional/negative values, complex numbers, and type promotions. The reviewer feedback focuses on adhering to the repository's style guide by replacing legacy cytnx_XXXX typedefs (such as cytnx_uint64 and cytnx_double) with standard C++ types (like uint64_t and double) for loop counters, local variables, and lambda parameters in the newly added test files.

Important

The consumer version of Gemini Code Assist on GitHub is being sunset. Starting June 18, 2026, new organization installations will be blocked, and all code review activity will officially cease on July 17, 2026.
For more details on the timeline and next steps, please review the Help Documentation.

gemini-code-assist · 2026-07-20T04:46:32Z

+        for (cytnx_uint64 i = 0; i < 4; i++)
+          for (cytnx_uint64 j = 0; j < 4; j++)


Avoid using legacy typedefs like cytnx_uint64 for ordinary loop counters in new code. Use standard C++ types such as uint64_t or size_t instead.

Suggested change

for (cytnx_uint64 i = 0; i < 4; i++)

for (cytnx_uint64 j = 0; j < 4; j++)

for (uint64_t i = 0; i < 4; i++)

for (uint64_t j = 0; j < 4; j++)

References

Avoid propagating legacy cytnx_XXXX typedefs as ordinary loop counters, sizes, flags, or local arithmetic types in new or touched code. Prefer standard C++ types. ^(link)

gemini-code-assist · 2026-07-20T04:46:33Z

+        const cytnx_uint64 bR = 2, bC = 3;
+        for (cytnx_uint64 i = 0; i < 6; i++)
+          for (cytnx_uint64 j = 0; j < 6; j++) {


Avoid using legacy typedefs like cytnx_uint64 for local sizes and loop counters in new code. Standard C++ types like uint64_t or size_t are preferred.

Suggested change

const cytnx_uint64 bR = 2, bC = 3;

for (cytnx_uint64 i = 0; i < 6; i++)

for (cytnx_uint64 j = 0; j < 6; j++) {

const uint64_t bR = 2, bC = 3;

for (uint64_t i = 0; i < 6; i++)

for (uint64_t j = 0; j < 6; j++) {

References

Avoid propagating legacy cytnx_XXXX typedefs as ordinary loop counters, sizes, flags, or local arithmetic types in new or touched code. Prefer standard C++ types. ^(link)

gemini-code-assist · 2026-07-20T04:46:33Z

+        // out[i,j] = a[0, j] * b[i, 0]
+        // (1+1i)*(0+1i) = -1+1i ; (2-1i)*(0+1i) = 1+2i
+        // (1+1i)*(3+0i) =  3+3i ; (2-1i)*(3+0i) = 6-3i
+        auto near = [&](cytnx_uint64 i, cytnx_uint64 j, double re, double im) {


Avoid using legacy typedefs like cytnx_uint64 for lambda parameters representing indices in new code. Use standard C++ types like uint64_t instead.

Suggested change

auto near = [&](cytnx_uint64 i, cytnx_uint64 j, double re, double im) {

auto near = [&](uint64_t i, uint64_t j, double re, double im) {

References

Avoid propagating legacy cytnx_XXXX typedefs as ordinary loop counters, sizes, flags, or local arithmetic types in new or touched code. Prefer standard C++ types. ^(link)

gemini-code-assist · 2026-07-20T04:46:33Z

+        ASSERT_EQ(out.dtype(), Type.ComplexDouble);
+        ASSERT_EQ(out.shape(), (std::vector<cytnx_uint64>{2, 2}));
+        // out[i,j] = a[i,0] * b[0,j]
+        auto near = [&](cytnx_uint64 i, cytnx_uint64 j, double re, double im) {


Avoid using legacy typedefs like cytnx_uint64 for lambda parameters representing indices in new code. Use standard C++ types like uint64_t instead.

Suggested change

auto near = [&](cytnx_uint64 i, cytnx_uint64 j, double re, double im) {

auto near = [&](uint64_t i, uint64_t j, double re, double im) {

References

Avoid propagating legacy cytnx_XXXX typedefs as ordinary loop counters, sizes, flags, or local arithmetic types in new or touched code. Prefer standard C++ types. ^(link)

gemini-code-assist · 2026-07-20T04:46:33Z

+          if (dtype == Type.Bool) continue;  // Kron has no Bool kernel
+          // See Outer_test's note: ~1e6-magnitude complex-float products diverge
+          // from the CPU at the float32 ULP, so ComplexFloat gets a looser tol.
+          const cytnx_double tol = (dtype == Type.ComplexFloat) ? 0.1 : 1e-6;


Avoid using legacy typedefs like cytnx_double for local arithmetic types such as tolerance variables in new code. Use standard C++ types like double instead.

Suggested change

const cytnx_double tol = (dtype == Type.ComplexFloat) ? 0.1 : 1e-6;

const double tol = (dtype == Type.ComplexFloat) ? 0.1 : 1e-6;

References

Avoid propagating legacy cytnx_XXXX typedefs as ordinary loop counters, sizes, flags, or local arithmetic types in new or touched code. Prefer standard C++ types. ^(link)

gemini-code-assist · 2026-07-20T04:46:33Z

+        for (cytnx_uint64 i = 0; i < 3; i++)
+          for (cytnx_uint64 j = 0; j < 2; j++)


Avoid using legacy typedefs like cytnx_uint64 for ordinary loop counters in new code. Use standard C++ types such as uint64_t or size_t instead.

Suggested change

for (cytnx_uint64 i = 0; i < 3; i++)

for (cytnx_uint64 j = 0; j < 2; j++)

for (uint64_t i = 0; i < 3; i++)

for (uint64_t j = 0; j < 2; j++)

References

Avoid propagating legacy cytnx_XXXX typedefs as ordinary loop counters, sizes, flags, or local arithmetic types in new or touched code. Prefer standard C++ types. ^(link)

gemini-code-assist · 2026-07-20T04:46:33Z

+        ASSERT_EQ(out.shape(), (std::vector<cytnx_uint64>{2, 2}));
+        // (1+2i)*(0-1i) = 2-1i ; (1+2i)*(2+2i) = -2+6i
+        // (-3+1i)*(0-1i)= 1+3i ; (-3+1i)*(2+2i)= -8-4i
+        auto near = [&](cytnx_uint64 i, cytnx_uint64 j, double re, double im) {


Avoid using legacy typedefs like cytnx_uint64 for lambda parameters representing indices in new code. Use standard C++ types like uint64_t instead.

Suggested change

auto near = [&](cytnx_uint64 i, cytnx_uint64 j, double re, double im) {

auto near = [&](uint64_t i, uint64_t j, double re, double im) {

References

Avoid propagating legacy cytnx_XXXX typedefs as ordinary loop counters, sizes, flags, or local arithmetic types in new or touched code. Prefer standard C++ types. ^(link)

gemini-code-assist · 2026-07-20T04:46:33Z

+
+        ASSERT_EQ(out.dtype(), Type.ComplexDouble);
+        ASSERT_EQ(out.shape(), (std::vector<cytnx_uint64>{2, 2}));
+        auto near = [&](cytnx_uint64 i, cytnx_uint64 j, double re, double im) {


Avoid using legacy typedefs like cytnx_uint64 for lambda parameters representing indices in new code. Use standard C++ types like uint64_t instead.

Suggested change

auto near = [&](cytnx_uint64 i, cytnx_uint64 j, double re, double im) {

auto near = [&](uint64_t i, uint64_t j, double re, double im) {

References

Avoid propagating legacy cytnx_XXXX typedefs as ordinary loop counters, sizes, flags, or local arithmetic types in new or touched code. Prefer standard C++ types. ^(link)

gemini-code-assist · 2026-07-20T04:46:33Z

+        for (auto dtype : dtype_list) {
+          if (dtype == Type.Bool) continue;  // Outer has no Bool kernel
+          SCOPED_TRACE("dtype " + std::to_string(dtype));
+          const cytnx_double tol = (dtype == Type.ComplexFloat) ? 0.1 : 1e-6;


Avoid using legacy typedefs like cytnx_double for local arithmetic types such as tolerance variables in new code. Use standard C++ types like double instead.

Suggested change

const cytnx_double tol = (dtype == Type.ComplexFloat) ? 0.1 : 1e-6;

const double tol = (dtype == Type.ComplexFloat) ? 0.1 : 1e-6;

References

Avoid propagating legacy cytnx_XXXX typedefs as ordinary loop counters, sizes, flags, or local arithmetic types in new or touched code. Prefer standard C++ types. ^(link)

chatgpt-codex-connector

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 19fc8688bf

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

Open a pull request for review
Mark a draft as ready
Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

chatgpt-codex-connector · 2026-07-20T04:48:44Z

+        b.at<cytnx_double>({0, 0}) = 2.0;
+        b.at<cytnx_double>({0, 1}) = -0.5;


Use precision-sensitive doubles in this discriminator

In this mixed-promotion regression test, the Double operands are exactly representable as float, and the expected tolerance is loose enough that an implementation which still narrows the multiply to complex64 before producing a ComplexDouble result would pass with these values. That leaves the stated “full double precision” behavior untested for the ComplexFloat × Double path; use a non-float-representable double or a magnitude/cancellation case whose result changes after float narrowing.

Useful? React with 👍 / 👎.

chatgpt-codex-connector · 2026-07-20T04:48:44Z

+        b.at<cytnx_double>({0}) = 3;
+        b.at<cytnx_double>({1}) = 0.5;


Use precision-sensitive doubles in this discriminator

This mixed-promotion test also uses Double values that are exactly representable as float, so it only verifies the dtype and not the promised full-double computation path. If the GPU path accidentally multiplies in complex64 and then writes/casts to ComplexDouble, these expected values still pass within 1e-6; choose a non-float-representable double value or another case whose product differs after float narrowing.

Useful? React with 👍 / 👎.

yingjerkao · 2026-07-22T02:16:28Z

Closing — its GPU test coverage has been salvaged into #1108, and the rest is superseded by #1105/#1106.

Background: this PR was stacked on the (now-closed) #1099 branch, and the Outer→Kron rewrite + GPU test files it depended on already landed on master via #1105/#1106 — so it can't merge as-is (it would try to re-apply changes already present and conflict).

But its tests were more thorough than what landed, so rather than lose them, #1108 folds this PR's stronger cases into master's existing tests/gpu/linalg_test/{Kron,Outer}_test.cpp:

2-D hand-computed Kron/Outer (fractional + negative values),
complex × complex hand-computed cases,
the ComplexFloat × Double → ComplexDouble promotion discriminator,
the broad GPU-vs-CPU all-dtype differential (GpuMatchesCpuAllDtypes),

while keeping master's Int16 hand-computed Kron and the #1099 Int16-diagonal Outer segfault guard. Verified on GPU (CUDA 13, RTX 4070 Ti): 11/11 pass. Thanks!

Problem: PR #1100 added more thorough GPU Kron/Outer tests but was stacked on the (now-closed) #1099 branch and functionally superseded by #1105/#1106, so it could not be merged as-is. Fix: fold #1100's stronger tests into master's existing tests/gpu/linalg_test/{Kron,Outer}_test.cpp, keeping master's unique regression cases: - 2-D hand-computed Kron/Outer with fractional and negative values (independent expected values, not a same-path comparison); - complex x complex hand-computed cases; - the ComplexFloat x Double -> ComplexDouble promotion discriminator (#984/#999); - a broad GPU-vs-CPU differential over every real/complex dtype and several shapes (GpuMatchesCpuAllDtypes), with the looser ComplexFloat tolerance at ~1e6-magnitude products (matches Mul_test); - kept master's Int16 hand-computed Kron and the #1099 Int16-diagonal Outer segfault regression guard. Testing: built gpu_test_main (CUDA 13, RTX 4070 Ti SUPER, USE_CUTENSOR=OFF) and ran GpuKron.*/GpuOuter.* -> 11/11 passed. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

Salvaged from the closed #1100. Extends the GPU Kron/Outer coverage from 6 cases to 14, with every expected value derived independently -- literals or a per-element recompute from the raw inputs -- rather than by comparing against another Cytnx path that could share the same bug. Coverage added: - Fractional and negative Double values, recomputed per element from the inputs. - Complex x complex, hand-computed. - The #984/#999 promotion discriminator: ComplexFloat (x) Double must produce ComplexDouble, computed and stored through that output type. - Int16, including negative products. - Mixed signed/unsigned promotion: Int32 (x) Uint32 -> Int32 and Uint32 (x) Int16 -> Int32. Both sweeps build their operands from a single dtype, so every off-diagonal promotion row was previously untested. Both cases carry negative products, so a result computed or stored through an unsigned type wraps and fails loudly. - Kron's rank-0 scalar path. `if (lhs.is_scalar() || rhs.is_scalar()) return lhs * rhs` (Kron.cpp:26) is a broadcast multiply, not the Kronecker kernel, and promotes through Mul rather than Kron's own output-type selection. Shape {1, 1} does not reach it -- that is still rank 2 -- so it had no coverage. Both operand orders plus a mixed-dtype pair. Two corrections to earlier claims in this file: The mixed-precision discriminator did not discriminate. Its operands (2.0, -0.5, 1.5, -2.5, 4.0) are all exactly representable in float32, so an implementation reporting ComplexDouble while multiplying through ComplexFloat produced a bit-identical result and passed. The Double operands are now 0.1 and 3.3, which float32 cannot represent, with the tolerance tightened from 1e-6 to 1e-12: a narrowed multiply lands 5.9e-9 to 1.9e-7 from the double product. Verified it now discriminates -- substituting the value a float32 path produces (4.949999809265137 for 1.5 * 3.3) fails the test, and it passes on the double product. Under the old operands that substitution was a no-op. "Kron has no Bool kernel" was wrong. cuKron_general is templated over bool and src/linalg/Outer.cpp documents Bool as covered on CPU and GPU after the missing dispatch row was fixed (#1099). The skip is removed from both sweeps and Bool passes, so that comment was the only thing keeping a historically broken path out of coverage. Uint16 sweep operands are bounded to [0, 1000]. GetRandRange gives the narrow types their full numeric_limits range, so Uint16 drew from [0, 65535]; two such values promote to int before multiplying and 65535 * 65535 exceeds INT_MAX, which is undefined behaviour in the CPU oracle and GPU kernel alike and made the comparison compiler-dependent. The underlying kernel hazard is filed as #1160 -- bounding the inputs stops this test depending on it, it does not fix it. Test names follow the suite convention (`TEST(Kron, Gpu...)`), which this PR had been converting six existing cases *off*; across tests/gpu the split is 406 such names against 4. The complex hand-computed cases assert through AreNearlyEqTensor, which also checks device, dtype, shape and contiguity. GPU suite: 100% tests passed, 0 failed out of 831. Refs #1003. Refs #1099, #1160. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

gemini-code-assist Bot reviewed Jul 20, 2026

View reviewed changes

chatgpt-codex-connector Bot reviewed Jul 20, 2026

View reviewed changes

yingjerkao mentioned this pull request Jul 20, 2026

refactor(linalg): typed GPU dispatch for Outer + retire Kron GPU promotion trait (#1003) #1101

Merged

yingjerkao requested review from IvanaGyro and ianmccul July 20, 2026 08:02

yingjerkao mentioned this pull request Jul 22, 2026

test(linalg): strengthen GPU Kron/Outer coverage (salvaged from #1100) #1108

Merged

yingjerkao closed this Jul 22, 2026

		for (cytnx_uint64 i = 0; i < 4; i++)
		for (cytnx_uint64 j = 0; j < 4; j++)

	auto near = [&](cytnx_uint64 i, cytnx_uint64 j, double re, double im) {
	auto near = [&](uint64_t i, uint64_t j, double re, double im) {

	const cytnx_double tol = (dtype == Type.ComplexFloat) ? 0.1 : 1e-6;
	const double tol = (dtype == Type.ComplexFloat) ? 0.1 : 1e-6;

		for (cytnx_uint64 i = 0; i < 3; i++)
		for (cytnx_uint64 j = 0; j < 2; j++)

		b.at<cytnx_double>({0, 0}) = 2.0;
		b.at<cytnx_double>({0, 1}) = -0.5;

Uh oh!

Conversation

yingjerkao commented Jul 20, 2026

Problem

Fix

Testing

Notes

Uh oh!

gemini-code-assist Bot left a comment

Choose a reason for hiding this comment

Code Review

Uh oh!

gemini-code-assist Bot Jul 20, 2026

Choose a reason for hiding this comment

Uh oh!

gemini-code-assist Bot Jul 20, 2026

Choose a reason for hiding this comment

Uh oh!

gemini-code-assist Bot Jul 20, 2026

Choose a reason for hiding this comment

Uh oh!

gemini-code-assist Bot Jul 20, 2026

Choose a reason for hiding this comment

Uh oh!

gemini-code-assist Bot Jul 20, 2026

Choose a reason for hiding this comment

Uh oh!

gemini-code-assist Bot Jul 20, 2026

Choose a reason for hiding this comment

Uh oh!

gemini-code-assist Bot Jul 20, 2026

Choose a reason for hiding this comment

Uh oh!

gemini-code-assist Bot Jul 20, 2026

Choose a reason for hiding this comment

Uh oh!

gemini-code-assist Bot Jul 20, 2026

Choose a reason for hiding this comment

Uh oh!

chatgpt-codex-connector Bot left a comment

Choose a reason for hiding this comment

💡 Codex Review

Uh oh!

chatgpt-codex-connector Bot Jul 20, 2026

Choose a reason for hiding this comment

Uh oh!

chatgpt-codex-connector Bot Jul 20, 2026

Choose a reason for hiding this comment

Uh oh!

yingjerkao commented Jul 22, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant