fix(pybind): collapse operator overloads per Python type; add __ne__/__bool__; fix numpy dtype loss

[yingjerkao]

Problem

Addresses #928/#916 and finishes the core of #692. Every Tensor/UniTensor arithmetic operator was bound ~24× (one per C++ scalar type + numpy scalars). Beyond the stub blowup this caused live dtype bugs: pybind11's arithmetic caster accepts any __index__-bearing object even in the no-convert pass, so plain fixed-width int overloads registered ahead of the numpy-scalar ones silently ate np.int32/uint64/… and produced Int64 results; t[0] = np.float32(x) raised a hard RuntimeError (Tensor __setitem__ had no numpy coverage and complex128 registered first). Separately, t1 != t2 returned a single always-False bool for non-empty tensors (Python's default __ne__ negated the elementwise __eq__ Tensor via __len__), and bool(t) fell through to __len__.

Fix

• Every binary/reverse/in-place operator on Tensor and UniTensor collapsed to a per-Python-distinguishable-type keep-set (24→14 / 23→13): Tensor operand, numpy scalars (32-bit + integer dtypes preserved), arbitrary-precision py::int_ with int64/uint64 runtime dispatch, double, complex128, Scalar — ordering is load-bearing and documented once in the new shared pybind/pyint_dispatch.hpp ("KEEP-SET ORDERING" note, cross-referencing PR Bind ExpH/ExpM with per-dtype overloads + Python-layer fixes (toward clean stubtest) #915 whose ExpH/ExpM work pioneered the pattern).
• __setitem__/set_elem numpy coverage + ordering fixed on both classes; __pow__/__ipow__ accept ints and numpy scalars.
• New elementwise __ne__ (composed from existing kernels, Bool result, mirrors __eq__ including the Scalar operand) and numpy-semantics __bool__ (ValueError for multi-element or Void; truthiness of the single element otherwise). Behavior change: if tensor: previously meant len(tensor) > 0.

Testing

pytests/binding_dtype_test.py (20 tests) with per-test baseline red/green documented — 10 genuinely red pre-fix (dtype loss, setitem error, __ne__/__bool__ traps). Full pytest 55 passed. Known pre-existing gap (numpy scalar on the LEFT of Tensor hits __iter__) documented, out of scope.

Stacking

Base branch is fix/pybind-inplace-return-self (PR #986). Merge #986 first, then retarget to master.

🤖 Generated with Claude Code

[gemini-code-assist]

Code Review

This pull request collapses redundant per-dtype operator overloads for both Tensor and UniTensor down to a canonical "keep-set" of Python-distinguishable types, resolving several numpy interop and type-preservation bugs. It also introduces the missing __ne__ and __bool__ operators for Tensor, and adds a helper dispatch_pyint to safely handle Python arbitrary-precision integers. The feedback suggests optimizing the dispatch_pyint function to immediately catch negative overflow and check for conversion exceptions earlier. Additionally, it recommends optimizing the __ne__ implementation to use self.Cpr(rhs).Cpr(0) instead of (1 - self.Cpr(rhs)).astype(cytnx::Type.Bool) to avoid unnecessary type promotion and memory allocations.

Important

The consumer version of Gemini Code Assist on GitHub is being sunset. Starting June 18, 2026, new organization installations will be blocked, and all code review activity will officially cease on July 17, 2026.
For more details on the timeline and next steps, please review the Help Documentation.

[gemini-code-assist]

In dispatch_pyint, if overflow < 0 (negative overflow), calling PyLong_AsUnsignedLongLong is guaranteed to fail and set a Python exception because negative values cannot be converted to unsigned integers. We can immediately check if (overflow < 0) and throw the error without calling PyLong_AsUnsignedLongLong.

Additionally, we should check PyErr_Occurred() right after PyLong_AsLongLongAndOverflow to handle any conversion exceptions (e.g., if __index__ raises an exception) before proceeding.

    const long long ia = PyLong_AsLongLongAndOverflow(a.ptr(), &overflow);
    if (PyErr_Occurred()) {
      PyErr_Clear();
      cytnx_error_msg(true, "[ERROR] integer scalar conversion failed.%s", "\n");
    }
    if (overflow == 0) return fn(static_cast<cytnx::cytnx_int64>(ia));
    if (overflow < 0) {
      cytnx_error_msg(true, "[ERROR] integer scalar out of the supported int64/uint64 range.%s",
                      "\n");
    }
    const unsigned long long ua = PyLong_AsUnsignedLongLong(a.ptr());
    if (PyErr_Occurred()) {
      PyErr_Clear();
      cytnx_error_msg(true, "[ERROR] integer scalar out of the supported int64/uint64 range.%s",
                      "\n");
    }
    return fn(static_cast<cytnx::cytnx_uint64>(ua));

[gemini-code-assist]

The current implementation of __ne__ uses (1 - self.Cpr(rhs)).astype(cytnx::Type.Bool). This triggers type promotion of the Bool tensor to Int64 (or Double) during the subtraction, allocates a temporary tensor, and then casts it back to Bool via .astype().

A much more efficient and elegant approach is to use self.Cpr(rhs).Cpr(0). Since self.Cpr(rhs) returns a Bool tensor, comparing it elementwise with 0 (which represents false) directly yields the logical NOT of the equality comparison as a Bool tensor, avoiding any type promotion, extra allocations, or casting. This optimization can be applied to all other __ne__ overloads in this file.

Suggested change

	[](cytnx::Tensor &self, const cytnx::Tensor &rhs) {
	return (1 - self.Cpr(rhs)).astype(cytnx::Type.Bool);
	})
	[](cytnx::Tensor &self, const cytnx::Tensor &rhs) {
	return self.Cpr(rhs).Cpr(0);
	})

[pcchen]

Code Review

Collapses every Tensor/UniTensor arithmetic operator from ~24 per-C++-dtype overloads to a ~14-entry per-Python-distinguishable-type "keep-set" with a load-bearing registration order (centralized in the new pybind/pyint_dispatch.hpp), fixes real numpy dtype-loss bugs, and adds elementwise __ne__ + numpy-semantics __bool__ on Tensor.

Correctness — verified

• dispatch_pyint is sound: PyLong_AsLongLongAndOverflow -> int64 when it fits (covers all negatives), else uint64, else a clean out-of-range error. Cpr(rhs) is literally *this == rhs, so __eq__ and __ne__ share one kernel. ✅
• The keep-set ordering rationale is accurate. Both documented pybind11 traps are real: the __index__ no-convert trap (numpy ints greedily eaten by a plain integral overload) and the __float__-fallback trap (complex128 capturing np.float32). The __add__/__eq__ orderings match the documented sequence, and dropping numpy_scalar<double>/<complex128> is correct — np.float64/np.complex128 are float/complex subclasses absorbed by the double/complex128 casters in the no-convert pass. ✅
• __ne__ = (1 - self.Cpr(rhs)).astype(Bool), composed from existing kernels, mirrors __eq__ (incl. the Scalar operand); fixes the old always-False trap. ✅
• __bool__ raises ValueError on Void/multi-element (numpy semantics), else the single element's truthiness (with a proper complex-zero check). ✅
• UniTensor gets the same collapse but intentionally no __eq__/__ne__/__bool__ — verified it has zero __eq__ bindings, so !=/bool() fall back to safe Python identity defaults. Sound scoping (unlike Tensor, where an elementwise __eq__ made the default != actively wrong). ✅
• Tests (20) are excellent: dtype preservation for every numpy scalar type, large-int -> uint64, __ne__ (not-silently-wrong / elementwise / Scalar), __bool__ (raises + single-element), UniTensor coverage — plus test_imatmul_preserves_identity, which also closes the @= identity gap noted on refactor(pybind): bind in-place methods directly, drop the c__* shadow API #986. ✅
• Compilation is green on all wheel platforms (macOS x2, Ubuntu x2), which also confirms the re-enabled operator==(Tensor, Scalar) path builds. ✅

Findings (all low / nits — none blocking)

1. Documented behavior change — bool(t). if tensor: no longer means len(tensor) > 0; multi-element/Void now raise ValueError. Well flagged in the PR/code/tests, but it's a real API break — worth a changelog/release-note entry since downstream if tensor: idioms will start raising.
2. Undocumented minor change — Python bool operands. Dropping the C++ cytnx_bool overload routes Python True/False through py::int_ (int64 path) instead of Bool. numpy-consistent and harmless for float tensors, but for a small-int-dtype tensor it can shift the result dtype (int16 + True -> Int64 vs Int16). Untested edge; worth a one-line note.
3. dispatch_pyint return-type coupling (latent). Both branches must deduce the same type from the generic fn — true for all current call sites, but a footgun for future callers; a short comment or static_assert would harden it.
4. Nit: the pyint_dispatch.hpp error message still uses the pre-fix(error): bounded formatting, namespaced impl, single-report errors #983 "...range.%s", "\n" idiom (zero-vararg calls are legal now).
5. Nit (already acknowledged in-code): dispatch_pyint duplicates Bind ExpH/ExpM with per-dtype overloads + Python-layer fixes (toward clean stubtest) #915's two-arg helper; the comment already plans to fold them once Bind ExpH/ExpM with per-dtype overloads + Python-layer fixes (toward clean stubtest) #915 merges.
6. CI: wheels compile green, but confirm a fresh BuildAndTest/pytest run on the current master-merged state before merge (the visible test run predates the retarget).

The left-of-Tensor numpy-scalar gap (np.float32(1.0) + t hits __iter__) is pre-existing, explicitly documented, and out of scope — reasonable.

Verdict

Strong, unusually well-documented PR that fixes several genuine dtype bugs and centralizes a subtle, easy-to-regress ordering invariant with excellent tests. Findings are all notes, not blockers. Approve-with-nits, pending a fresh full-test CI run.

Posted by Claude Code on behalf of @pcchen

[pcchen]

Approving. This fixes several genuine numpy dtype-loss bugs (silent Int64/complex coercion, the t[0] = np.float32(x) hard error), replaces the always-False != with a correct elementwise __ne__, and adds numpy-semantics __bool__ — while collapsing the ~24x per-dtype overloads behind a well-documented, centralized keep-set ordering. The ordering rationale (the __index__ no-convert and __float__-fallback traps) is accurate, __ne__/__eq__ share one kernel, UniTensor's omission of __eq__/__ne__/__bool__ is a sound identity-fallback choice, and the 20-test suite is thorough (including the @= identity test that also covers the #986 gap). Compilation is green on all wheel platforms.

Non-blocking follow-ups (fine here or in a fast-follow):

• Add a changelog/release-note line for the bool(t) behavior change (if tensor: now raises ValueError for multi-element/Void instead of meaning len > 0).
• Optional: note the Python-bool-operand dtype shift (now routed through the int64 py::int_ path), and add a short comment/static_assert on dispatch_pyint's same-return-type requirement.

Please confirm a fresh full BuildAndTest/pytest run is green on the current master-merged state before merging (the visible test run predates the retarget).

Posted by Claude Code on behalf of @pcchen