[kaleb-himes]

Description

sha512: SHA-512/224 FinalRaw and Transform no longer modify state or input

Sha512FinalRaw: for SHA-512/224 (28-byte digest, not a multiple of 8) the little-endian path byte-reversed sha512->digest in place before copying it out and never reversed it back, so wc_Sha512_224FinalRaw() corrupted the running hash. Any Update after FinalRaw then produced the wrong digest. It now reverses into a local copy, copies digestSz bytes out and zeroes the copy, so sha512->digest is never modified. 5172ff7 (v5.8.0) removed an earlier local copy, citing a gcc SEGV on AVX-512 -march=native builds. That SEGV occurs only under AddressSanitizer with detect_stack_use_after_return=1 (a gcc code-generation issue, GCC Bugzilla 110027), never in a non-sanitizer build. ee2e634 later fixed the multiple-of-8 digest sizes but left SHA-512/224. The v5.2.1 and v6.0.0 FIPS modules predate it and are not affected.

wc_Sha512Transform: the input block is const, but on little-endian builds
without the Intel AVX path the function byte-reversed it in place. Each path
now leaves the caller's block and the hash state alone:

• generic C: copies into sha->buffer and reverses there, as Sha512Update
  does, then restores sha->buffer.
• ARM, RISC-V and PPC asm: pass the block unchanged on both endiannesses,
  as Sha512Update's bulk path does. On ppc64le this also fixes a wrong
  digest: the block was reversed before PPC64 asm that already loads it
  with ldbrx, so it was reversed twice. On big-endian the old code reversed
  the block once, so Transform disagreed with Update.
• ARMv5 (WOLFSSL_ARM_ARCH < 6) and RISC-V vector crypto
  (WOLFSSL_RISCV_VECTOR_CRYPTO_ASM): copy into the local aligned buffer
  first, because the ARMv5 base asm loads words with LDR and the RISC-V
  vector asm loads with vl8re64.v, which RVV 1.0 section 7.9 allows to trap
  on a misaligned address.
• AArch64: saves and restores sha->buffer around the transform, because the
  C CPU step (Transform_Sha512_C) writes it. Before, Transform overwrote
  input pending from an earlier Update.
  No path reads or writes the caller's block through a word64 pointer.

Testing

• test_wc_Sha512Transform: checks the caller's block is unchanged after
  Transform. Skipped for FIPS modules before v7.0.0: the frozen v5 and v6
  sha512.c still reverse the caller's block when the Intel AVX path is not
  used (seen on ARMv8 with --enable-fips=v5).
• test_wc_Sha512_224FinalRaw: hashes "ab", reads FinalRaw, hashes "c", and
  checks the result is SHA-512/224("abc") from the FIPS 180-4 examples.

Verified (--enable-opensslextra, master sha512.c as the control):

• x86_64 non-FIPS: master fails test_sha512.c:279 and :360; this passes;
  make check passes.
• aarch64 macOS (Apple M1 Max, --enable-armasm): master fails :279 and :360;
  this passes.
• aarch64 macOS, --enable-armasm with WOLFSSL_ARMASM_NO_NEON (C CPU step):
  Update("ab"), Transform, Final now matches the x86 and NEON builds;
  master gives a different digest.
• aarch64 big-endian (qemu, Arm GNU Toolchain 14.2 aarch64_be): Transform
  and Update of the same block now give the same state with both the NEON
  asm and the C CPU step; before they differed.
• riscv64 (BeagleV, --enable-riscv-asm): master fails :279 and :360; the
  Transform changes pass.
• riscv64 vector crypto (qemu -cpu max, --enable-riscv-asm=zv,zvkb,zvkned)
  and base (--enable-riscv-asm): SHA-512 tests pass, and an aligned and a
  misaligned Transform of the same block give the same digest.
• ppc64le (qemu, --enable-ppc64-asm and =inline): before, the SHA-512,
  SHA-512/224 and SHA-512/256 Transform tests got the wrong digest; this
  passes.
• ppc64 big-endian (qemu, --enable-ppc64-asm): passes, unchanged.
• ARMv5TE (qemu, --enable-armasm, -march=armv5te): passes.
• --enable-fips=v5 (v5.2.1 module): Transform test passes with the version
  gate and fails at test_sha512.c:278 without it.
• test_wc_Sha512Transform also compares Update("ab"), Transform,
  Update("c"), Final against Transform, Update("abc"), Final, so Transform
  must keep input pending from an earlier Update.
  • AVX-512 (AWS Xeon Platinum 8488C and EPYC 9R14): gcc 11.5, 12.4, 13.3 and 14.2 and clang 15, at -march=native -O2, -O3 and -O3 -mprefer-vector-width=512. Every SHA-512 family FinalRaw at output offsets 0 to 63 and context offsets 0 to 56 shows no fault and no state change, with each SHA-512 lane (AVX2+RORX, AVX2, AVX1+RORX, AVX1, C) forced and confirmed. The same holds in the Linux 6.12 kernel module self-test (C and AVX2). Under the original ASan sanitizer recipe the new FinalRaw also shows no fault on gcc 13.3, 14.2 and 15.2. Controls: an aligned zmm store to a misaligned address faults on that hardware, and master fails the SHA-512/224 state check.

Fixes Fenrir:

• F-17180: wc_Sha512Transform byte-swaps the caller's const input block in
  place on little-endian builds.
• F-3343: wc_Sha512Transform performs unaligned word64 byte-reversal on the
  caller's data pointer.
• F-17178: ppc64le wc_Sha512Transform reverses the block twice.

NOTE: Not changed here:

• --enable-ppc64-asm=crypto on ppc64le loads the message with lxvd2x and no
  little-endian byte swap, so SHA-256 and SHA-512 KATs fail. The asm is
  generated from wolfSSL/scripts, so the fix is in a separate pair: branch
  PQ-FS-2026-Part4-LabPrep-ppc64-sha2-crypto-le (scripts a879f1dd, wolfssl
  fb82943).
• Big-endian aarch64 --enable-armasm fails the SHA-256 and SHA-512 KATs on
  master; that is not addressed here.

Related PR's:
#11999
https://github.com/wolfSSL/scripts/pull/706

Checklist

• added tests
• updated/added doxygen
• updated appropriate READMEs
• Updated manual and documentation

[github-actions]

MemBrowse Memory Report · Explore what changed →

gcc-arm-cortex-m4-pkcs7

• FLASH: .text +64 B (+0.0%, 222,804 B / 262,144 B, total: 85% used)

zephyr-frdm_rw612

• FLASH: log_const_area -8 B, rodata -8 B, text +1,592 B (+0.6%, 284,040 B / 67,108,864 B, total: 0% used)

No memory changes detected for:

• gcc-arm-cortex-m0plus
• gcc-arm-cortex-m3
• gcc-arm-cortex-m4
• gcc-arm-cortex-m4-baremetal
• gcc-arm-cortex-m4-crypto-only
• gcc-arm-cortex-m4-dtls13
• gcc-arm-cortex-m4-min-ecc
• gcc-arm-cortex-m4-openssl-compat
• gcc-arm-cortex-m4-pq
• gcc-arm-cortex-m4-rsa-only
• gcc-arm-cortex-m4-sp-math
• gcc-arm-cortex-m4-tls12
• gcc-arm-cortex-m4-tls13
• gcc-arm-cortex-m7
• gcc-arm-cortex-m7-pq
• gcc-arm-cortex-m7-tls13
• stm32-sim-stm32h753

---

_{Catch memory regressions in CI with MemBrowse. Free for open source.}

[kaleb-himes]

@claude review please

[kaleb-himes]

@claude review again please.

[kaleb-himes]

Rebased on master to resolve CI issues that had landed in master.

[wolfSSL-Fenrir-bot]

Fenrir Automated Review — PR #11983

Scan targets checked: none
Unchanged since last review (not re-run): wolfcrypt-src, wolfcrypt-bugs, wolfssl-bugs

Fenrir result: Approved ✅

No new issues found in the changed files.

Advisory only — this automated result does not count as a GitHub approval.

Review tier: Lite

[Copilot]

🟢 Approval recommended

No active review issues remain for the SHA-512 state-preservation and input-handling changes.

No open issues 🎉

_{Review effort: Balanced}

[SparkiDev]

Would rather copy and reverse rather than have the digest in a reversed state.

[kaleb-himes]

Absolutely! Yes this is the first solution I reached for (a temporary buffer to hold the temporary state so I didn't have to touch the live state but other feedback led me to see if it could be solved without using additional stack or heap. The stack implementation requires an extra ForceZero but I benchmarked both implementations and both were less than a fraction of a nanosecond different. I'll gladly go with the initial implementation. Thanks @SparkiDev. (Will include AVX512 test results in the PR summary once I finish fully vetting the stack approach too.)

[Copilot]

🟢 Approval recommended

No active review issues are recorded for the SHA-512 state and input preservation changes.

No open issues 🎉

1 resolved since last review

• Avoid reintroducing the unaligned scratch-array workaround

_{Review effort: Balanced}

[wolfSSL-Fenrir-bot]

Fenrir Automated Review — PR #11983

Scan targets checked: wolfcrypt-src, wolfcrypt-bugs, wolfssl-bugs
Coverage: 2 of 2 in-scope changed file(s) opened by the reviewer

Fenrir result: Approved ✅

No new issues found in the changed files.

Advisory only — this automated result does not count as a GitHub approval.

Review tier: Max