Skip to content

fix(Tensor & UniTensor): support rank-zero tensors and zero extents - #1026

Merged
ianmccul merged 27 commits into
masterfrom
refactor/tensor-rank-zero
Jul 11, 2026
Merged

fix(Tensor & UniTensor): support rank-zero tensors and zero extents#1026
ianmccul merged 27 commits into
masterfrom
refactor/tensor-rank-zero

Conversation

@ianmccul

@ianmccul ianmccul commented Jul 9, 2026

Copy link
Copy Markdown
Collaborator

Summary

This PR adds proper rank-0 Tensor and UniTensor. Resolves #1016. It also adds proper support for tensors that have extent zero (such as an empty vector), following NumPy and PyTorch, and basically every other tensor library out there. Resolves #1040.

Historically Cytnx represented several scalar results as rank-1 tensors with shape {1}. That makes shape metadata ambiguous: a one-leg length-1 tensor is not the same object as a scalar. This PR starts separating those cases by treating an empty shape () as the scalar tensor shape, while preserving ordinary shape {1} as a rank-1 tensor.

This adds more code than I was expecting, but of the net lines added, +942 are tests/pytests, and the rest is hardening, replacing the old rank-1-size-1 scalar hacks with explicit rank-0 behavior, and about about 93 added cytnx_error_msg checks, 55 added tests, and hardened many existing tests.

The biggest user-visible change in Python is that contractions over all legs of a tensor (including inner products etc) now return a proper rank-0 tensor, so e.g. the example DMRG code needs

-        return psivec, energy[0].item()
+        return psivec, energy.item()

I think that's an improvement: previously the energy was a rank-1 tensor.

Also added Tensor and UniTensor helpers .is_void(), .is_scalar(), and .is_empty(). The old model was not able to distinguish properly genuinely empty tensors from rank-0 tensors where shape is empty, and was not able to distinguish a rank-1 size-1 tensor from a genuine scalar.

UniTensor had no canonical way to store its symmetry data. Functions that needed the symmetry data would typically get it from _bonds[0].syms(), which of course doesn't exist for a rank-0 UniTensor. The symmetry data is now held by UniTensor itself. To avoid too much code churn, existing behavior of setting the symmetry data from _bonds[0].syms() is still supported as a fallback, but retrieving the symmetry data should be done via this->syms().

This should be regarded as a breaking change for Tensor file I/O. Previous Cytnx cannot distinguish a genuine rank-1 size-1 tensor from a scalar-like tensor that should have been rank 0. Loading old Tensor files remains supported, but old shape {1} is preserved as rank-1 rather than auto-converted to rank 0. Tensors saved in the new version are not loadable by old Cytnx because old Cytnx would choke on a genuine rank-0 Tensor. The Tensor file magic number has changed so new Tensor files loaded by old Cytnx will fail early.

The SVD-style return values now follow a consistent rank-zero convention.

  • Svd / Gesvd on Tensor return Tensor objects, and Svd / Gesvd on UniTensor return UniTensor objects: the singular values S remain a diagonal object in the same API family as U and vT, with block/symmetric SVD preserving the singular-value bond structure. Truncated SVD routines keep the existing homogeneous vector return type, so the optional return_err object is also a Tensor for Tensor APIs and a UniTensor for UniTensor APIs.
  • return_err == 0 appends nothing.
  • return_err == 1 appends a scalar rank-0 object containing the largest actually discarded singular value, or scalar zero if nothing was discarded. (I think this is a little inconsistent in the sense that it does not distinguish the case of discarding nothing at all vs discarding actual zero singular values. I'd favor returning some sentinel like -1 in the case where truely nothing was discarded.)
  • return_err > 1 appends a rank-1 vector containing all actually discarded singular values in descending order; if nothing was discarded, this is a tensor with shape {0}. For block/symmetric truncation, “discarded” means after the real truncation decision, including exact-degeneracy enlargement and min_blockdim, so the error return reports values that were actually removed.

The core API now has a clean distinction: is_void() means uninitialized, is_scalar() means rank zero with one value, and is_empty() means initialized with logical size() == 0 (i.e. it has rank > 0 but at least one extent is zero).

Changes

  • Allow Tensor to be initialized with an empty shape, allocating one storage element.
  • Add Tensor::is_scalar(), true for initialized rank-0 tensors only.
  • Add Tensor::is_void() and expose it to Python, replacing uses of shape.empty() as an initialized/non-void proxy.
  • Support rank-0 scalar access through item(), at({}), get({}), and set({}, ...).
  • Keep .item() as a size-one element extractor, NumPy-style: shape (), {1}, and {1,1} all work if storage size is one.
  • Make fully indexed Tensor::get(...) return a rank-0 Tensor rather than shape {1}.
  • Expose Tensor.is_scalar() and Tensor.is_void() to Python, also the UniTensor versions.
  • Make Python len(tensor) raise for rank-0 tensors, matching NumPy rank-0 array behavior.
  • Add rank-0 Python indexing support for tensor[()] and dense unitensor[()].
  • Update Tensor save/load to use a versioned Tensor header, keep loading legacy Tensor files, and avoid empty-vector data access for rank-0 metadata.
  • Preserve singleton broadcasting for ordinary Tensor: shape {}, {1}, {1,1}, and other one-element shapes can broadcast in binary arithmetic and assignment. Tensor::is_scalar() remains a structural query that is true only for rank 0. UniTensor broadcasting remains restricted to true rank-0 scalars because singleton bonds can carry symmetry information.
  • Reject void tensor arithmetic before dtype promotion/dispatch, so void tensors do not fall into scalar-broadcast setup.
  • Apply the same initialized-Tensor precondition to every raw scalar and Scalar overload of Add, Sub, Mul, Div, and Mod, in both operand orders, so Void inputs fail with a controlled operation-level error.
  • Add a narrow helper for out-of-place binary scalar-broadcast output initialization. This can go away in a later arithmetic-kernel refactor.
  • Make dense no-bond UniTensor use a rank-0 backing block.
  • Normalize loaded dense no-bond UniTensor blocks to rank 0 when the stored block has one element.
  • Make scalar-valued reductions such as Norm() and Vectordot() return rank-0 tensors.
  • Make Block/Fermionic Norm() return rank-0 Tensor results as well.
  • Preserve Block/Fermionic rank-0 full-trace results instead of converting out-of-place Trace() to DenseUniTensor.
  • Store Block/Fermionic symmetry metadata separately, so rank-0 Block/Fermionic UniTensors do not need a fake public bond just to answer syms().
  • Add a constructor path for initializing the symmetry data of a Block/Fermionic UniTensor. If symmetry data is not supplied, it is inferred from the first added Bond.
  • Fix existing full-trace behavior for BlockUniTensor::Trace_ and BlockFermionicUniTensor::Trace_: after tracing the last two bonds, the old code could leave _bonds empty while syms() still indexed _bonds[0].
  • Make Sort return rank-0 scalar tensors unchanged before reading the last shape dimension.
  • Make Matmul_dg reject rank-0 operands before reading matrix dimensions from shape().back() / shape()[0].
  • Update Tensor::append to distinguish void tensors from rank-0 tensors, reject rank-0 lhs append, and allow rank-0 rhs append as a scalar element.
  • Breaking change: DirectSum now throws for rank-0 tensors. The direct sum of scalars is not well defined here; previous behavior treated scalars as rank-1 size-1 tensors and returned a rank-1 size-2 tensor.
  • Drive-by bug fix in Kron the lhs_pad_left branch prepends ones to rhs’s shape and assigns that to the lhs padded shape. That can make the lhs padded product larger than lhs storage.
  • Drive-by bug fix in diagonal permute() resized the block vector and then left every block Void, so it manufactured bad blocks. permute_nosignflip() in Fermionic has the same diagonal branch bug.
  • Fixed lots of bugs exposed by tests and other code assuming that a scalar tensor could be accessed using [0] and similar. This ought to produce an exception on accessing an axis beyond the rank of the tensor, but in many cases (e.g. Tensordot) would silently read invalid memory if you supply it invalid axes to contract over.
  • Bug fix in _from_numpy that does not handle rank-0 numpy arrays.
  • Drive-by fixes to add UInt16 and Int16 to _from_numpy and numpy functions.
  • Some minor code cleanups in the touched code.
  • Fixed cuQuantum terr init now uses S.dtype() (real) instead of in.dtype() (possibly complex) (a real dtype bug for complex-input truncated SVD) and skips allocation when return_err == 0.
  • Drive-by (unintended) bug fix in the free linalg::Norm(UniTensor) function, that would silently return Double on input Float tensors (differently to the member function UniTensor::Norm and differently to Tensor). This was via initializing the variable to accumulate the block norms vla zeros(1), which defaults to Double. Fixed by removing the free function and forwarding to the member function.
  • Update the Tensor and UniTensor guides to document rank-0 construction, zero extents, Void/empty/scalar queries, item() on size-one tensors, and the distinction between Tensor singleton broadcasting and symmetry-carrying UniTensors.

Notes

I removed the int overloads of the C++ generators zeros() and ones(), and changed the array versions to take std::initializer_list. The problem with the int overload is that zeros({}), to create a rank-0 tensor of zeros, would instead call the int overload and end up as zeros(0). Now, to make a rank-1 tensor you need to write zeros({N}). This is not a problem in Python, which continues to have zeros(N) and zeros([]) versions, since Python can distinguish [] and 0. This affects a lot of tests that construct rank-1 tensors. There are quite a few of these that look suspicious, eg zeros(3 * 4 * 5).reshape({3,4,5}).astype(Types.Double).to(Device.cuda) in a GPU test, and was presumably intended to be the equivalent zeros({3,4,5}, Types.Double, Device.cuda), which has always worked. I fixed all of the ones I saw, and added some more tests of the generators.

There is no version bump to the UniTensor version number introduced in #1022 - I am assuming that there is no tensor data saved using master branch in the few hours between that PR landing and this one.

AI review notes that there are a few round-trip GPU<->CPU data transfers in the scalar broadcast CUDA kernels. They are all pre-existing, improved in some cases by explicit transfer rather than relying on unified memory, but largely outside the scope of this PR. Those will be addressed by #1003.

Tests

Ran full CPU + GPU tests + pytests

@gemini-code-assist gemini-code-assist Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request introduces comprehensive support for rank-0 (scalar) Tensors and UniTensors across Cytnx, updating initialization, reshaping, indexing, serialization, and element-wise binary arithmetic operators to handle empty shapes. The review feedback highlights a critical safety issue in Cpr.cpp where bypassing the contiguity check for scalar broadcasting could lead to silent numerical corruption on non-contiguous tensors. Additionally, an improvement was suggested in Tensor.hpp to replace raw memcpy with iterator-based vector initialization for safer and cleaner code.

Important

The consumer version of Gemini Code Assist on GitHub is being sunset. Starting June 18, 2026, new organization installations will be blocked, and all code review activity will officially cease on July 17, 2026.
For more details on the timeline and next steps, please review the Help Documentation.

Comment thread src/linalg/Cpr.cpp
Comment thread include/Tensor.hpp Outdated

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 1781bb972f

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread src/linalg/Arithmetic_shape.hpp Outdated
Comment thread src/backend/Tensor_impl.cpp Outdated
Comment thread pybind/tensor_py.cpp
Comment thread src/DenseUniTensor.cpp Outdated
@codecov

codecov Bot commented Jul 9, 2026

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 80.91248% with 205 lines in your changes missing coverage. Please review.
✅ Project coverage is 57.29%. Comparing base (aa62024) to head (6375602).
✅ All tests successful. No failed tests found.

Files with missing lines Patch % Lines
src/Tensor.cpp 52.94% 32 Missing ⚠️
src/backend/StorageImplementation.cpp 48.97% 25 Missing ⚠️
src/linalg/Div.cpp 34.21% 25 Missing ⚠️
src/linalg/Sub.cpp 36.84% 24 Missing ⚠️
include/UniTensor.hpp 78.68% 13 Missing ⚠️
src/linalg/Mul.cpp 50.00% 13 Missing ⚠️
include/backend/Tensor_impl.hpp 52.38% 10 Missing ⚠️
src/linalg/Mod.cpp 79.48% 8 Missing ⚠️
src/DenseUniTensor.cpp 90.14% 7 Missing ⚠️
src/backend/Tensor_impl.cpp 83.33% 7 Missing ⚠️
... and 14 more
Additional details and impacted files
@@            Coverage Diff             @@
##           master    #1026      +/-   ##
==========================================
+ Coverage   55.63%   57.29%   +1.65%     
==========================================
  Files         228      229       +1     
  Lines       32896    33337     +441     
  Branches       71       71              
==========================================
+ Hits        18303    19100     +797     
+ Misses      14573    14216     -357     
- Partials       20       21       +1     
Flag Coverage Δ
cpp 57.22% <80.91%> (+1.67%) ⬆️
python 63.35% <ø> (+0.26%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

Components Coverage Δ
C++ backend 54.98% <79.41%> (+1.68%) ⬆️
Python bindings 71.99% <97.72%> (+1.63%) ⬆️
Python package 63.35% <ø> (+0.26%) ⬆️
Files with missing lines Coverage Δ
include/Bond.hpp 93.87% <ø> (ø)
include/backend/Storage.hpp 96.64% <100.00%> (+1.38%) ⬆️
pybind/generator_py.cpp 95.52% <100.00%> (+24.09%) ⬆️
src/Accessor.cpp 56.20% <100.00%> (+2.35%) ⬆️
src/Bond.cpp 66.46% <ø> (-0.20%) ⬇️
src/Generator.cpp 91.66% <100.00%> (-0.50%) ⬇️
src/UniTensor.cpp 88.65% <100.00%> (ø)
src/algo/Concatenate.cpp 100.00% <100.00%> (ø)
src/algo/Sort.cpp 88.23% <100.00%> (+1.56%) ⬆️
src/backend/Storage.cpp 67.35% <100.00%> (+0.16%) ⬆️
... and 44 more

... and 11 files with indirect coverage changes


Continue to review full report in Codecov by Harness.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update aa62024...6375602. Read the comment docs.

🚀 New features to boost your workflow:
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 5b35c5cb10

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread src/DenseUniTensor.cpp Outdated
Comment thread src/DenseUniTensor.cpp Outdated
Comment thread include/Tensor.hpp Outdated
Comment thread src/backend/Tensor_impl.cpp

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 321f6b7d2c

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread src/DenseUniTensor.cpp
Comment thread src/linalg/Arithmetic_shape.hpp Outdated
Comment thread src/linalg/Vectordot.cpp
@ianmccul

Copy link
Copy Markdown
Collaborator Author

/gemini review

@gemini-code-assist gemini-code-assist Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request introduces comprehensive support for rank-0 (scalar) Tensors and UniTensors in Cytnx, enabling proper scalar indexing, broadcast arithmetic, reductions, and trace operations, alongside an updated versioned serialization format. The review feedback identifies important safety and robustness improvements, specifically recommending checks for uninitialized tensors in element access methods to prevent undefined behavior, and adding an explicit header inclusion in tests to avoid potential compilation failures.

Important

The consumer version of Gemini Code Assist on GitHub is being sunset. Starting June 18, 2026, new organization installations will be blocked, and all code review activity will officially cease on July 17, 2026.
For more details on the timeline and next steps, please review the Help Documentation.

Comment thread include/backend/Tensor_impl.hpp
Comment thread include/backend/Tensor_impl.hpp
Comment thread include/backend/Tensor_impl.hpp
Comment thread tests/DenseUniTensor_test.cpp

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 210bfdc818

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread src/linalg/Directsum.cpp Outdated
Comment thread src/DenseUniTensor.cpp Outdated
Comment thread src/linalg/iAdd.cpp Outdated

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: d7cb323c3a

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread src/DenseUniTensor.cpp Outdated
Comment thread src/DenseUniTensor.cpp Outdated
Comment thread src/linalg/Cpr.cpp Outdated

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 93e9f1c45a

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread src/linalg/Arithmetic_shape.hpp Outdated
Comment thread src/linalg/Norm.cpp Outdated
Comment thread src/linalg/Sum.cpp

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 9f07c50d0b

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread src/linalg/Arithmetic_shape.hpp Outdated
Comment on lines +20 to +22
inline Tensor HostScalarForGpuBroadcast(const Tensor &tensor, const int op_device) {
if (op_device != Device.cpu && tensor.is_scalar() && tensor.device() != Device.cpu) {
return tensor.to(Device.cpu);

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Apply GPU scalar host copies to typed scalar overloads

Fresh evidence beyond the earlier Tensor/Tensor CUDA scalar fix is that this helper is only used by the Tensor/Tensor arithmetic paths; typed scalar overloads such as scalar_cuda + 1.0, 1.0 + scalar_cuda, and the analogous Sub/Mul/Div/Mod/Cpr overloads still pass the rank-0 CUDA Tensor storage directly to the CUDA size-1 branches. Those branches host-read *_in[0] before launching, so a newly supported rank-0 CUDA Tensor combined with a C++/Python numeric scalar can still read the device-resident managed value without the host-copy/synchronizing path this helper provides.

Useful? React with 👍 / 👎.

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is long-standing behavior. Size-1 operands are passed by value to kernels using _Lin[0] / _Rin[0] on the host side before launch. If that storage is actually on GPU, this relies on managed-memory host access and synchronization behavior. It is terrible thing to do, but beyond the scope of this PR. It should get fixed by #1003 .

Comment thread src/backend/Tensor_impl.cpp
Comment thread src/backend/Tensor_impl.cpp
Comment thread src/backend/Tensor_impl.cpp
@ianmccul
ianmccul force-pushed the refactor/tensor-rank-zero branch from 9f07c50 to f6f6f5c Compare July 10, 2026 06:05

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 092b7d70b6

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread src/BlockUniTensor.cpp Outdated
Comment thread src/BlockUniTensor.cpp Outdated
Comment thread src/BlockUniTensor.cpp Outdated
@ianmccul

Copy link
Copy Markdown
Collaborator Author

@codex review

@ianmccul

Copy link
Copy Markdown
Collaborator Author

/gemini review

@gemini-code-assist gemini-code-assist Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request introduces comprehensive support for rank-0 scalar Tensors and UniTensors across the codebase, updating backend implementations, linear algebra operations, and Python bindings to correctly handle scalar structures instead of treating them as 1-element rank-1 tensors. The review feedback highlights several violations of the repository style guide, specifically pointing out that newly introduced helper functions and utilities should be renamed from CamelCase to snake_case.

Important

The consumer version of Gemini Code Assist on GitHub is being sunset. Starting June 18, 2026, new organization installations will be blocked, and all code review activity will officially cease on July 17, 2026.
For more details on the timeline and next steps, please review the Help Documentation.

Comment thread src/BlockFermionicUniTensor.cpp Outdated
Comment thread src/BlockUniTensor.cpp Outdated
Comment thread src/DenseUniTensor.cpp Outdated
Comment thread src/linalg/Arithmetic_shape.hpp Outdated
Comment thread src/linalg/Arithmetic_shape.hpp Outdated
Comment thread src/linalg/Arithmetic_shape.hpp Outdated
Comment thread pybind/tensor_py.cpp Outdated
Comment thread pybind/unitensor_py.cpp Outdated
@ianmccul ianmccul changed the title fix(Tensor): support dense rank-zero tensors fix(Tensor): support rank-zero tensors Jul 10, 2026
@chatgpt-codex-connector

Copy link
Copy Markdown

Codex Review: Didn't find any major issues. Bravo.

Reviewed commit: 88ac4c1064

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

@ianmccul

Copy link
Copy Markdown
Collaborator Author

According to the repository style guide (line 48), new identifiers should use snake_case instead of CamelCase. All cases identified by Gemini have been resolved.

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: db35534582

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread src/BlockFermionicUniTensor.cpp Outdated
Comment on lines +1330 to +1333
if (!Lidx.empty()) {
memcpy(Lidx.data(), tmp->_inner_to_outer_idx[b].data(),
sizeof(cytnx_uint64) * this->_bonds.size());
}

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Use located sign indices for scalar fermionic contracts

When a rank-0 BlockFermionicUniTensor is used as the left factor in an outer-product contraction with a non-scalar fermionic tensor, this new empty-Lidx path lets the loop run once per output/RHS block, but the sign update below still indexes this->_signflip[b]. The scalar left operand has only one sign entry, so any RHS with more than one block reads past the scalar's sign vector instead of reusing IDL[0]/IDR[0], making valid scalar-factor contractions crash or corrupt the result.

Useful? React with 👍 / 👎.

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is pre-existing, will open a separate issue

@ianmccul

Copy link
Copy Markdown
Collaborator Author

@codex review
/gemini review

@gemini-code-assist gemini-code-assist Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request introduces comprehensive support for rank-0 (scalar) Tensors and UniTensors across the Cytnx library, updating C++ implementations, Python bindings, and serialization formats. Key mathematical operations, reductions, and element-wise arithmetic have been updated to return or broadcast rank-0 tensors correctly instead of 1-element rank-1 tensors. Additionally, Block and BlockFermionic UniTensors now properly preserve symmetry metadata for rank-0 blocks. The review feedback highlights a performance optimization opportunity in the contraction loops of both BlockFermionicUniTensor and BlockUniTensor, where pre-allocating the index vectors outside the loop can prevent repeated dynamic memory allocations in a hot path.

Important

The consumer version of Gemini Code Assist on GitHub is being sunset. Starting June 18, 2026, new organization installations will be blocked, and all code review activity will officially cease on July 17, 2026.
For more details on the timeline and next steps, please review the Help Documentation.

Comment thread src/BlockFermionicUniTensor.cpp Outdated
Comment thread src/BlockUniTensor.cpp Outdated
@ianmccul

Copy link
Copy Markdown
Collaborator Author

I think we're close to final version now.
@codex review
/gemini review

@gemini-code-assist gemini-code-assist Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request introduces comprehensive support for rank-0 (scalar) Tensors and UniTensors across Dense, Block, and BlockFermionic implementations, updating various linear algebra operations and adding extensive unit tests. The review feedback highlights critical out-of-bounds read risks in Tensor_impl.cpp due to empty locator initialization, which should be corrected to {{0}}. Additionally, the feedback suggests performing is_void() checks before is_scalar() in Kron.cpp, using the is_void() and rank() helper methods for better consistency, accumulating loop results locally in Norm.cpp to avoid proxy overhead, replacing C-style casts with reinterpret_cast to adhere to the style guide, and resolving narrowing conversion warnings in iota_view usages.

Important

The consumer version of Gemini Code Assist on GitHub is being sunset. Starting June 18, 2026, new organization installations will be blocked, and all code review activity will officially cease on July 17, 2026.
For more details on the timeline and next steps, please review the Help Documentation.

Comment thread src/backend/Tensor_impl.cpp
Comment thread src/backend/Tensor_impl.cpp
Comment thread src/linalg/Kron.cpp Outdated
Comment thread src/linalg/Directsum.cpp Outdated
Comment thread src/linalg/Diag.cpp Outdated
Comment thread src/BlockFermionicUniTensor.cpp
Comment thread src/BlockUniTensor.cpp
Comment thread src/BlockFermionicUniTensor.cpp
Comment thread src/BlockUniTensor.cpp
Comment thread src/DenseUniTensor.cpp

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 48f686493e

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread src/linalg/Mod.cpp Outdated
Comment thread src/linalg/Cpr.cpp Outdated
Comment thread src/linalg/Arithmetic_shape.hpp Outdated
@ianmccul

Copy link
Copy Markdown
Collaborator Author

@codex review
/gemini review

@gemini-code-assist gemini-code-assist Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request implements comprehensive support for rank-0 scalar Tensors and UniTensors across the Cytnx library. It introduces is_scalar() and is_void() predicates, allows empty shapes {} for rank-0 representations, and updates core linear algebra operations (such as Sum, Min, Max, Norm, Det, Vectordot, and Trace) to return rank-0 scalar tensors instead of rank-1 tensors of size 1. Additionally, it updates binary arithmetic operations, in-place operators, and element access/assignment to support rank-0 scalar broadcasting and empty-tuple () indexing in both C++ and Python bindings. File serialization has also been updated with a new versioned magic number to correctly preserve rank-0 structures. There are no review comments provided, so we have no further feedback to offer.

Important

The consumer version of Gemini Code Assist on GitHub is being sunset. Starting June 18, 2026, new organization installations will be blocked, and all code review activity will officially cease on July 17, 2026.
For more details on the timeline and next steps, please review the Help Documentation.

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 1b16d4fae7

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread pybind/unitensor_py.cpp
ianmccul and others added 11 commits July 11, 2026 15:46
Remove the C++ scalar-size zeros/ones overloads so {} consistently denotes a rank-zero scalar shape and {N} denotes a rank-one tensor. Keep Python integer convenience in the bindings and clean up tests to use dtype/device-aware generators directly.

Co-Authored-By: Codex <codex@openai.com>
Co-Authored-By: Codex <codex@openai.com>
Co-Authored-By: Codex <codex@openai.com>
Finish the review-driven rank-zero hardening, zero-extent support, singleton Tensor broadcasting, documentation, stubs, and regression coverage.

Co-Authored-By: Codex <noreply@openai.com>
@ianmccul
ianmccul force-pushed the refactor/tensor-rank-zero branch from 84c21cd to 6375602 Compare July 11, 2026 08:03
@ianmccul

Copy link
Copy Markdown
Collaborator Author

Latest version addressess all review items

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 6375602e39

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread src/linalg/Max.cpp
Comment thread src/backend/Tensor_impl.cpp
Comment thread src/linalg/Cpr.cpp

@yingjerkao yingjerkao left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good to me.

@ianmccul

Copy link
Copy Markdown
Collaborator Author

currently surveying the codebase for more zero-extent edge cases.

@ianmccul

Copy link
Copy Markdown
Collaborator Author

will fix the extent-0 edge cases in a separate PR. this needs to get merged.

@ianmccul
ianmccul merged commit 2270685 into master Jul 11, 2026
17 checks passed
@ianmccul
ianmccul deleted the refactor/tensor-rank-zero branch July 11, 2026 08:34
yingjerkao added a commit that referenced this pull request Jul 11, 2026
BlockUniTensorTest.gpu_Trace read the rank-0 scalar from UT_diag.Trace(0, 1)
via at({0}), which #1026 (rank-0 UniTensor semantics) now rejects with
"rank-0 scalar expects no locator indices". The CPU Trace test was updated to
at({}) in #1026; the GPU test was missed because GPU tests don't run in CI.
Mirror the CPU fix. Surfaced by this PR's new GPU CI on its first run.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
yingjerkao added a commit that referenced this pull request Jul 13, 2026
After rebasing onto current master (Axpy API removal #787f9d8c, rank-zero
support #1026, Exp/Expf consolidation #1047), the committed stubs no longer
matched what pybind11-stubgen 2.5.5 produces from the current bindings. The
textual rebase auto-merged the .pyi files but generated files must be
regenerated, not merged.

Regenerated all stubs from a fresh openblas-cpu build (pybind11 3.0.4,
pybind11-stubgen 2.5.5). Net effect over the rebased tree is Device.pyi and
__init__.pyi only (the drift from #1026/#1047); linalg.pyi already matched.
Regeneration is idempotent, and `mypy.stubtest cytnx.cytnx.linalg` reports no
ExpH/ExpM overload-cannot-match. linalg_exp_test.py: 26 passed.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
yingjerkao added a commit that referenced this pull request Jul 13, 2026
Brings in #1039, #1047 (39 commits). Clean textual merge, but #1026's new
test `UniTensor_baseTest.rank_detects_label_bond_mismatch` desyncs bonds/labels
via `bonds().pop_back()` -- which no longer compiles now that this PR makes the
mutable `bonds()` accessor const (immutable Bond, #846). That desync is
unreachable through the public API by design, so the test now pokes the
internal `_bonds` vector (const_cast) to still exercise rank()'s defensive
size-mismatch guard.

Verified: openblas-cpu + RUN_TESTS build clean (CI's editable-install compile
gate reproduced locally); UniTensor_base/BlockFermionic/Bond gtests 38 passed.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
yingjerkao added a commit that referenced this pull request Jul 15, 2026
…ue-division (#941)

Reimplements the deferred half of #1015 on current master, which had since
refactored the same six arithmetic files for rank-zero support (#1026),
zero-extent (#1043), and type_promote (#984).

- Convert CPU out-of-place AND in-place Add/Sub/Mul/Div/Cpr/Mod to typed
  std::visit over as_storage_variant() with storage_cast<TO> and typed
  *InternalImpl<TO,TL,TR> kernels; the dead legacy dispatch tables are removed.
- True division (#941): `/` and `/=` yield a floating result (int/int -> Double)
  via make_floating_point_dtype; the rank-zero / broadcast output still routes
  through #1026's init_broadcast_binary_output (seeded with the floating dtype).
- Mixed-dtype in-place TENSOR ops promote the LHS via storage_as_type_or_replace
  (no more truncation into the narrower LHS storage -- #941's core bug).
- Weak-scalar (#980, per #1015's binding ruling): `tensor op= python-scalar`
  (a rank-0 RHS via scalar_as_rank0_tensor) PRESERVES the LHS dtype for
  Add/Sub/Mul and follows true-division for Div (Int64 /= 2.0 -> Double,
  Float /= 3.0 -> Float), keyed on Rt.rank()==0 in DispatchInplaceArithmeticCPU;
  complex-into-real still throws.
- floordiv conforms to #1049 (scalar `ut // 2.0` kept, `ut1 // ut2` raises);
  #1015's original unbind-all is dropped.
- normalize_() now promotes an integer block to Double under true division; its
  stale "stays Int32" comment is updated. Mod keeps an eager is_complex guard so
  a zero-extent complex Mod still throws (ZeroExtentLinalgTest).

CPU only; GPU stays on the legacy dtype tables behind the unchanged public API
(#1013 tracks the GPU conversion).

Testing: full openblas-cpu (non-ASan) test_main 1784 passed / 11 skipped /
0 failed -- MixedDtypeArithmetic regressions, weak-scalar Tensor.ScalarInplace*,
ZeroExtentLinalgTest, and DenseUniTensorTest true-division/normalize all green;
clang-format v14 clean.

Reimplements ff30d00, deferred from #1015 (#941).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
yingjerkao added a commit that referenced this pull request Jul 15, 2026
…ue-division (#941)

Reimplements the deferred half of #1015 on current master, which had since
refactored the same six arithmetic files for rank-zero support (#1026),
zero-extent (#1043), and type_promote (#984).

- Convert CPU out-of-place AND in-place Add/Sub/Mul/Div/Cpr/Mod to typed
  std::visit over as_storage_variant() with storage_cast<TO> and typed
  *InternalImpl<TO,TL,TR> kernels; the dead legacy dispatch tables are removed.
- True division (#941): `/` and `/=` yield a floating result (int/int -> Double)
  via make_floating_point_dtype; the rank-zero / broadcast output still routes
  through #1026's init_broadcast_binary_output (seeded with the floating dtype).
- Mixed-dtype in-place TENSOR ops promote the LHS via storage_as_type_or_replace
  (no more truncation into the narrower LHS storage -- #941's core bug).
- Weak-scalar (#980, per #1015's binding ruling): `tensor op= python-scalar`
  (a rank-0 RHS via scalar_as_rank0_tensor) PRESERVES the LHS dtype for
  Add/Sub/Mul and follows true-division for Div (Int64 /= 2.0 -> Double,
  Float /= 3.0 -> Float), keyed on Rt.rank()==0 in DispatchInplaceArithmeticCPU;
  complex-into-real still throws.
- floordiv conforms to #1049 (scalar `ut // 2.0` kept, `ut1 // ut2` raises);
  #1015's original unbind-all is dropped.
- normalize_() now promotes an integer block to Double under true division; its
  stale "stays Int32" comment is updated. Mod keeps an eager is_complex guard so
  a zero-extent complex Mod still throws (ZeroExtentLinalgTest).

CPU only; GPU stays on the legacy dtype tables behind the unchanged public API
(#1013 tracks the GPU conversion).

Testing: full openblas-cpu (non-ASan) test_main 1784 passed / 11 skipped /
0 failed -- MixedDtypeArithmetic regressions, weak-scalar Tensor.ScalarInplace*,
ZeroExtentLinalgTest, and DenseUniTensorTest true-division/normalize all green;
clang-format v14 clean.

Reimplements ff30d00, deferred from #1015 (#941).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
yingjerkao added a commit that referenced this pull request Jul 26, 2026
BlockUniTensorTest.gpu_Trace read the rank-0 scalar from UT_diag.Trace(0, 1)
via at({0}), which #1026 (rank-0 UniTensor semantics) now rejects with
"rank-0 scalar expects no locator indices". The CPU Trace test was updated to
at({}) in #1026; the GPU test was missed because GPU tests don't run in CI.
Mirror the CPU fix. Surfaced by this PR's new GPU CI on its first run.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
yingjerkao added a commit that referenced this pull request Jul 26, 2026
BlockUniTensorTest.gpu_Trace read the rank-0 scalar from UT_diag.Trace(0, 1)
via at({0}), which #1026 (rank-0 UniTensor semantics) now rejects with
"rank-0 scalar expects no locator indices". The CPU Trace test was updated to
at({}) in #1026; the GPU test was missed because GPU tests don't run in CI.
Mirror the CPU fix. Surfaced by this PR's new GPU CI on its first run.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Tensor should support zero extents Design question: should scalars be rank-0 (shape-{}) tensors?

3 participants