Skip to content

Fix ARM64 ASIMD dot kernel operand constraints under LTO - #6009

Merged
martin-frbg merged 2 commits into
OpenMathLib:developfrom
mpwaser:mpwaser/update-openblas-latest
Sep 3, 2026
Merged

Fix ARM64 ASIMD dot kernel operand constraints under LTO#6009
martin-frbg merged 2 commits into
OpenMathLib:developfrom
mpwaser:mpwaser/update-openblas-latest

Conversation

@mpwaser

@mpwaser mpwaser commented Sep 3, 2026

Copy link
Copy Markdown
Contributor

Summary

Fix incorrect operand constraints in the ARM64 ASIMD dot kernel that can produce wrong results with GCC full LTO. A small cblas_strmv regression test is included.

Cause and fix

An inline-assembly block has the instructions themselves and an operand list describing their effects to the compiler. GCC does not derive those effects from the instructions. Here the operand list says that the array pointers (x, y), their strides (inc_x, inc_y), and the loop counter (j) are only read, although the instructions overwrite all five.

This becomes a real failure under full LTO. GCC sees that both strides are 1 and, based on the incorrect operand list, may keep them in the same register. The kernel converts each stride from elements to bytes by multiplying it by four because one float occupies four bytes:

expected: inc_x 1 -> 4, inc_y 1 -> 4
actual:   shared register 1 -> 4 -> 16

The shared register is therefore multiplied twice and becomes 16. The pointers advance 16 bytes (four floats) instead of 4 bytes (one float), so the dot product reads the wrong elements.

Declaring the five values as +&r fixes the contract: + tells GCC that the value is both read and overwritten, while & prevents its register from being reused for another still-live input (r selects a general-purpose register). This keeps the two strides separate. n, which is not modified, remains input-only. The assembly instructions and numerical operations are unchanged.

This is separate from #5917 and #5918, which fixed accumulator initialization and vector-register clobbers in the same kernel.

Validation

Tested on Linux AArch64 with static, single-threaded VORTEX builds and full LTO:

  • Unmodified develop with the new test: fails with expected 85, got 52.
  • This branch with GCC 15.2.0: all 125 unit tests pass.
  • This branch with Clang 22.1.3: all 125 unit tests pass.
  • GNU Make and CMake test builds pass.

The reproducer multiplies a 3x3 lower-triangular matrix by [11, 13, 17]; the correct result is [22, 85, 252].

CI coverage

Permanently covering the original failure would require an additional static VORTEX full-LTO ARM64 job. That is possible, but the ongoing runner cost and maintenance seem disproportionate for this narrow case, so this PR only adds the regression test. I can add targeted CI coverage if maintainers prefer.

The inline assembly modifies its pointers, strides, and loop counter. Declare them as early-clobber read/write operands so LTO cannot assign them to registers that still contain live inputs.
@martin-frbg

Copy link
Copy Markdown
Collaborator

Ugh, that haunted ThunderX2 kernel strikes again... Thanks for the fix.

@martin-frbg martin-frbg added this to the 0.3.35 milestone Sep 3, 2026
@martin-frbg
martin-frbg merged commit ee0ba0c into OpenMathLib:develop Sep 3, 2026
100 of 106 checks passed
@martin-frbg

Copy link
Copy Markdown
Collaborator

As an aside, having a full-LTO build in CI doesn't sound bad at all, as it would also serve to verify that LTO is possible with the codebase (I assume this would have to exclude the linktest done on shared library builds, as that one ignores actual function signatures)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants