[Frauschi]

Cuts the temporary heap that WOLFSSL_MLKEM_MAKEKEY_SMALL_MEM and WOLFSSL_MLKEM_ENCAPSULATE_SMALL_MEM need, and lets WOLFSSL_MLKEM_CACHE_A be combined with them, which previously stopped the build with Can't cache A with small memory code. Cipher texts and shared secrets are unchanged, and so is the number of Keccak permutations per operation when nothing is cached.

Heap reduction

The small memory code held whole vectors that it only ever used a polynomial at a time.

• Key generation multiplies each polynomial of matrix A into the result as soon as it is sampled and reuses that one polynomial for the next (mlkem_gen_matrix_i_acc() replaces mlkem_gen_matrix_i()). It now needs a single polynomial of scratch for every parameter set.
• Encapsulation compresses each polynomial of u straight into the cipher text as soon as it is complete, so v reuses u's polynomial, the message polynomial reuses y and e_2 reuses the matrix scratch. Scratch drops from 3k to k + 2 polynomials.
• Decapsulation no longer keeps a second copy of the whole re-encapsulated cipher text. mlkem_encapsulate_seeds() takes the received cipher text and compares each encoded block against it, in constant time, as it is produced.

Peak heap per operation, measured with a counting allocator installed through wolfSSL_SetAllocators() around each call (--enable-cryptonly --enable-mlkem with both small memory options, portable C implementation):

Parameter set	Operation	master	this PR	Change
ML-KEM-512	Key generation	1,024	512	-50%
ML-KEM-512	Encapsulation	3,072	2,048	-33%
ML-KEM-512	Decapsulation	3,840	2,400	-38%
ML-KEM-768	Key generation	1,536	512	-67%
ML-KEM-768	Encapsulation	4,608	2,560	-44%
ML-KEM-768	Decapsulation	5,696	2,912	-49%
ML-KEM-1024	Key generation	2,048	512	-75%
ML-KEM-1024	Encapsulation	6,144	3,072	-50%
ML-KEM-1024	Decapsulation	7,712	3,424	-56%

All figures are bytes. They are the same whether the key was generated locally or decoded. For comparison, the default (non small memory) build peaks at 7,424 / 11,840 / 17,440 bytes for ML-KEM-512 / 768 / 1024 decapsulation; this PR leaves that build's heap use unchanged, and speed is unchanged in both builds.

Matrix A cache with the small memory options

Key generation now writes each polynomial of A into the cache as it generates it, and encapsulation reads the transpose of the cached matrix (A_trans[i,j] is A[j,i]) instead of regenerating it. Neither needs extra scratch: the generator takes the stride between polynomials, which is zero when the matrix is not kept, so the heap figures above hold with WOLFSSL_MLKEM_CACHE_A defined too.

In the small memory encapsulation, a key built by wc_MlKemKey_DecodePublicKey() or wc_MlKemKey_DecodePrivateKey() also keeps the matrix it generates on first use. This is the usual situation for a peer's key, which previously regenerated all k*k polynomials on every call. With WOLFSSL_MLKEM_DYNAMIC_KEYS encapsulation never allocates: a key keeps the matrix only when key generation already allocated it, and a key that was only decoded streams the matrix as before.

The cost is the matrix inside the key object: sizeof(MlKemKey) grows from 5,048 to 13,240 bytes, the same as WOLFSSL_MLKEM_CACHE_A already costs without the small memory options. With WOLFSSL_MLKEM_DYNAMIC_KEYS it is instead a kk512 byte allocation per generated key.

Throughput for ML-KEM-768, small memory build, operations per second (median of three 1 s runs, Apple M5 Pro):

Operation	Without cache	With WOLFSSL_MLKEM_CACHE_A
Encapsulation, generated key	101,926	211,587
Encapsulation, decoded key	101,894	211,332
Decapsulation, generated key	86,567	154,909
Decapsulation, decoded key	86,608	155,279

Fixes

These were found while doing the above. The first also affects master.

• --enable-mlkem=original, =all and =ml-kem selected no parameter set and no operation, so configure disabled all of them and the build stopped at No Kyber key size chosen. Each group now defaults on unless one of its members is named.
• Small memory key generation overwrote a matrix expansion failure with the result of the following hash and returned a corrupt key as success. The error is now propagated.
• Since the small memory encapsulation writes the cipher text block by block, a failure partway through zeroes the caller's buffer instead of leaving part of a cipher text in it.
• mlkemkey_encapsulate() only honours a comparison request in the small memory build. The default build now returns BAD_FUNC_ARG instead of skipping the comparison, which would have read as a match.
• In the heapless build, the cipher text comparison block is zeroized on exit, and is only reserved and zeroized where it can be used (decapsulation).

Testing

• wolfcrypt/test/test.c: new mlkem_cache_a_decode_test() for every ML-KEM and Kyber parameter set. It reuses one key object across MakeKey / DecodePublicKey / DecodePrivateKey, checks that both sides agree on the shared secret, and runs two rounds per decoded key so that the second round reads the matrix kept by the first.
• tests/api/test_mlkem.c: test_wc_mlkem_decap_fo_reject also tampers with a later block of u and with the trailing c_2. The small memory decapsulation compares block by block, and the existing offsets only reached the first block.
• tests/unit-mcdc/test_mlkem_poly_hash_fault_whitebox.c: drives mlkem_gen_matrix_i_acc() with every stride and orientation its callers pass (streaming, row cache, transposed cache) under hash fault injection.
• New pq-all rows: mlkem-cache-a-small-mem, mlkem-cache-a-makekey-small-mem, mlkem-cache-a-encapsulate-small-mem, mlkem-cache-a-dynamic-keys-small-mem, mlkem-small-mem-no-malloc, mlkem-small-mem-api-tests (the only small memory row that links unit.test), mlkem-cache-a-small-stack, mlkem-cache-a-kyber-only and mlkem-original-mode.
• New multi-arch row thumb2-mlkem-small-mem: the small memory options with WOLFSSL_MLKEM_CACHE_A on the Thumb-2 assembly, run under qemu-arm.
• New wolfCrypt-Wconversion row for WOLFSSL_MLKEM_CACHE_A with both small memory options.

[github-actions]

MemBrowse Memory Report · Explore what changed →

zephyr-frdm_rw612

• FLASH: log_const_area -8 B, rodata -8 B, text +1,592 B (+0.6%, 284,040 B / 67,108,864 B, total: 0% used)

No memory changes detected for:

• gcc-arm-cortex-m0plus
• gcc-arm-cortex-m3
• gcc-arm-cortex-m4
• gcc-arm-cortex-m4-baremetal
• gcc-arm-cortex-m4-crypto-only
• gcc-arm-cortex-m4-dtls13
• gcc-arm-cortex-m4-min-ecc
• gcc-arm-cortex-m4-openssl-compat
• gcc-arm-cortex-m4-pkcs7
• gcc-arm-cortex-m4-pq
• gcc-arm-cortex-m4-rsa-only
• gcc-arm-cortex-m4-sp-math
• gcc-arm-cortex-m4-tls12
• gcc-arm-cortex-m4-tls13
• gcc-arm-cortex-m7
• gcc-arm-cortex-m7-pq
• gcc-arm-cortex-m7-tls13
• stm32-sim-stm32h753

---

_{Catch memory regressions in CI with MemBrowse. Free for open source.}

[Frauschi]

@wolfSSL-Fenrir-bot review balanced

[wolfSSL-Fenrir-bot]

Fenrir Automated Review — PR #11565

Scan targets checked: wolfcrypt-src, wolfcrypt-bugs, wolfssl-src, wolfssl-bugs
Coverage: 6 of 6 in-scope changed file(s) opened by the reviewer

Fenrir result: Approved ✅

No new issues found in the changed files.

Advisory only — this automated result does not count as a GitHub approval.

Review tier: Balanced

[Copilot]

Copilot review overview

🔵 Needs a closer look

The changes affect constant-time cryptographic processing, memory aliasing, and many compile-time configurations, warranting final expert review.

Review effort: Balanced
Findings: None

What changed in this PR

Reduces ML-KEM small-memory scratch usage, enables matrix-A caching, and strengthens error handling and regression coverage.

Changes:

• Streams matrix generation and ciphertext compression to reduce temporary memory.
• Supports cached matrix A across small-memory and dynamic-key configurations.
• Adds fault, decode-cache, rejection, configuration, and compiler-warning coverage.

File	Description
wolfssl/​wolfcrypt/​wc_mlkem.h	Updates internal ML-KEM interfaces and scratch sizing.
wolfcrypt/​src/​wc_mlkem.c	Integrates reduced-memory operations and cache invalidation.
wolfcrypt/​src/​wc_mlkem_poly.c	Implements streamed matrix multiplication and ciphertext encoding.
wolfcrypt/​test/​test.c	Tests cache invalidation after key decoding.
tests/​api/​test_mlkem.c	Extends ciphertext rejection tests across blocks.
tests/​unit-mcdc/​test_mlkem_poly_hash_fault_whitebox.c	Covers streamed matrix generation fault paths.
configure.ac	Corrects ML-KEM mode-only option defaults.
.github/​workflows/​wolfCrypt-Wconversion.yml	Adds warning-clean small-memory cache coverage.
.github/​configs/​pq-all.json	Adds ML-KEM configuration test variants.
ChangeLog.md	Documents the memory and cache improvements.

---

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

[Frauschi]

jenkins retest this please

[Frauschi]

Jenkins retest this please

[Frauschi]

Jenkins retest this please

[dgarske]

Blocker: Thumb2 ARMASM no longer compiles with ML-KEM-1024

mlkem_vec_compress_11() / mlkem_vec_compress_11_c() gained a kp parameter, but under WOLFSSL_ARMASM && WOLFSSL_ARMASM_THUMB2 the _c symbol is #defined to the assembly routine mlkem_thumb2_vec_compress_11(byte* r, sword16* v) (wc_mlkem_poly.c:6657, wc_mlkem.h:892, thumb2-mlkem-asm_c.c:3992), which still takes two arguments and compresses a fixed four polynomials. Reproduced with arm-none-eabi-gcc -mcpu=cortex-m4 -mthumb compiling wc_mlkem_poly.c with WOLFSSL_ARMASM, WOLFSSL_ARMASM_THUMB2, WOLFSSL_HAVE_MLKEM:

wc_mlkem_poly.c:6657:37: error: too many arguments to function 'mlkem_thumb2_vec_compress_11'

Master compiles the same file. CI is green because no row builds Thumb2 ARMASM with ML-KEM: mldsa-mlkem-arm32-armasm is AArch32 NEON (arm-linux-gnueabihf), thumb2-gcm-table is AES only, and the cortex-m4-pq cross rows are plain C. Two consequences:

1. The full-memory call mlkem_vec_compress_11(c1, u, k) breaks every Cortex-M ML-KEM-1024 build with the assembly enabled, independent of the small-memory options.
2. Thumb2 is deliberately not in the small-memory #error (only USE_INTEL_SPEEDUP and AArch64 are), so WOLFSSL_MLKEM_ENCAPSULATE_SMALL_MEM + Thumb2 asm is a supported combination. Once the signature is fixed, the new per-polynomial call mlkem_vec_compress_11(cb, u, 1) would hand a one-polynomial buffer to an assembly routine that reads and writes four, so dropping kp is not a fix either. The Thumb2 routine needs a polynomial count (the generator is scripts/kyber/thumb2/kyber.rb, Sean's), or wc_mlkem_poly.c needs a C fallback for kp != 4 on that port. mlkem_thumb2_vec_compress_10 already takes the polynomial count (wc_mlkem.h:890), so the 10-bit path is fine; compress_11 just never needed one until now.

Suggest a CI row that cross-compiles --enable-armasm for cortex-m4 with ML-KEM so this class of break is caught; the existing cortex-m4-pq row is the natural place.

[dgarske]

Other items @Frauschi

1. Encapsulation now writes into the key object (wc_mlkem.c, mlkemkey_encapsulate, both the WOLFSSL_MLKEM_CACHE_A full-memory branch and the small-memory fillA path). Before this PR key->a and MLKEM_FLAG_A_SET were only written by MakeKey, so a decoded public key could be shared read-only between threads encapsulating concurrently. Now the first wc_MlKemKey_Encapsulate() on a decoded key fills key->a and sets the flag, and a second thread racing it reads a half-written matrix. Suggest either documenting that an MlKemKey is not shareable across concurrent encapsulations in a CACHE_A build (the header comment on WOLFSSL_MLKEM_CACHE_A is the place), or gating the fill-on-encapsulate behind its own define so CACHE_A alone keeps the old read-only behaviour. The server-side TLS use (one key share per connection) is unaffected either way.

2. PR body "Fixes" section is stale. "Stale cached matrix after decoding a key" and "Unchecked squeeze in the rejection sampling loops" are already on master (121521b9e01 / 6f594eeed11, kaleb-himes, Oct 4 and Oct 14, and the mlkem_xof_squeezeblocks() return is checked at wc_mlkem_poly.c:4265/4532 on master). This branch carries no change for either; the body should drop them or say they landed separately, so the ChangeLog entry does not claim them twice.

Minor (take or leave)

• wc_mlkem.c mlkemkey_encapsulate: (void)compVecSz; under the small-memory branch and (void)fail; under the full-memory one are the only uses in those branches; consider scoping the variables into the branch that uses them instead.
• mlkem_encapsulate_seeds() now takes 14 parameters. Fine for a WOLFSSL_LOCAL, but a one-line comment at the call site in wc_mlkem.c saying which of c/cmp is live would help the next reader.
• tests/api/test_mlkem.c: the two new tamper offsets (ctLen - 200, ctLen - 1) are good; ctLen - 200 lands in a later u block for all three parameter sets (ML-KEM-512 u is 640 bytes), so the comment is accurate.

[dgarske]

Thumb2 ARMASM no longer compiles with ML-KEM-1024 after the mlkem_vec_compress_11() signature change. On that port mlkem_vec_compress_11_c is #defined to the assembly routine mlkem_thumb2_vec_compress_11(r, v) (wc_mlkem_poly.c:6657, wc_mlkem.h:892), which still takes two arguments and always compresses four polynomials, so wc_mlkem_poly.c:6771 fails:

wc_mlkem_poly.c:6657:37: error: too many arguments to function 'mlkem_thumb2_vec_compress_11'

(arm-none-eabi-gcc -mcpu=cortex-m4 -mthumb with WOLFSSL_ARMASM, WOLFSSL_ARMASM_THUMB2, WOLFSSL_HAVE_MLKEM; master compiles the same file.) Thumb2 is deliberately not in the small-memory #error, so once the signature matched, the new per-polynomial mlkem_vec_compress_11(cb, u, 1) would hand one polynomial to a routine that reads and writes four. No CI row builds Thumb2 assembly with ML-KEM (mldsa-mlkem-arm32-armasm is AArch32 NEON), which is why this is green.

The generator fix is in wolfSSL/scripts: write_vec_compress_11 takes the polynomial count and ends the vector at v + k * 512, the same as write_vec_compress_10. Attached is the regenerated thumb2-mlkem-asm.S / thumb2-mlkem-asm_c.c plus the wc_mlkem.h prototype, on top of 6c2b4c2752. Cross-built --host=armv7m-linux-gnueabihf --enable-armasm --enable-mlkem and ran testwolfcrypt under qemu-arm in the default build, with WOLFSSL_MLKEM_MAKEKEY_SMALL_MEM + WOLFSSL_MLKEM_ENCAPSULATE_SMALL_MEM, and with those plus WOLFSSL_MLKEM_CACHE_A: all pass. Could you fold it into this branch? A cortex-m4 --enable-armasm ML-KEM cross-compile row would catch this class of break.

Patch

diff --git a/wolfcrypt/src/port/arm/thumb2-mlkem-asm.S b/wolfcrypt/src/port/arm/thumb2-mlkem-asm.S
index a32e66c9bd..bef67429ac 100644
--- a/wolfcrypt/src/port/arm/thumb2-mlkem-asm.S
+++ b/wolfcrypt/src/port/arm/thumb2-mlkem-asm.S
@@ -3706,7 +3706,7 @@ L_mlkem_thumb2_vec_compress_10_loop:
 	.type	mlkem_thumb2_vec_compress_11, %function
 mlkem_thumb2_vec_compress_11:
 	PUSH	{r4, r5, r6, r7, r8, r9, r10, r11, lr}
-	ADD	r2, r1, #0x800
+	ADD	r2, r1, r2, LSL #9
 #ifndef WOLFSSL_ARM_ARCH_7M
 	MOV	r8, #0xdbb6
 	MOVT	r8, #0x9d7
@@ -4362,7 +4362,7 @@ L_mlkem_thumb2_vec_compress_11_loop:
 #endif
 #endif /* !WOLFSSL_ARM_ARCH_7M */
 	POP	{r4, r5, r6, r7, r8, r9, r10, r11, pc}
-	/* Cycle Count = 807 */
+	/* Cycle Count = 808 */
 	.size	mlkem_thumb2_vec_compress_11,.-mlkem_thumb2_vec_compress_11
 	.text
 	.align	4
diff --git a/wolfcrypt/src/port/arm/thumb2-mlkem-asm_c.c b/wolfcrypt/src/port/arm/thumb2-mlkem-asm_c.c
index b3342b9b9f..b8bdb02624 100644
--- a/wolfcrypt/src/port/arm/thumb2-mlkem-asm_c.c
+++ b/wolfcrypt/src/port/arm/thumb2-mlkem-asm_c.c
@@ -3989,16 +3989,19 @@ WC_OMIT_FRAME_POINTER void mlkem_thumb2_vec_compress_10(byte* r, sword16* v,
 }
 
 #ifndef WOLFSSL_NO_VAR_ASSIGN_REG
-WC_OMIT_FRAME_POINTER void mlkem_thumb2_vec_compress_11(byte* r_p, sword16* v_p)
+WC_OMIT_FRAME_POINTER void mlkem_thumb2_vec_compress_11(byte* r_p, sword16* v_p,
+    unsigned int k_p)
 #else
-WC_OMIT_FRAME_POINTER void mlkem_thumb2_vec_compress_11(byte* r, sword16* v)
+WC_OMIT_FRAME_POINTER void mlkem_thumb2_vec_compress_11(byte* r, sword16* v,
+    unsigned int k)
 #endif /* !WOLFSSL_NO_VAR_ASSIGN_REG */
 {
 #ifndef WOLFSSL_NO_VAR_ASSIGN_REG
     register byte* r __asm__ ("r0") = (byte*)r_p;
     register sword16* v __asm__ ("r1") = (sword16*)v_p;
+    register unsigned int k __asm__ ("r2") = (unsigned int)k_p;
 #else
-    void* L_asm_args[2] = {(void*)(size_t)r, (void*)(size_t)v
+    void* L_asm_args[3] = {(void*)(size_t)r, (void*)(size_t)v, (void*)(size_t)k
     };
     void** L_asm_args_p = L_asm_args;
 #endif /* !WOLFSSL_NO_VAR_ASSIGN_REG */
@@ -4006,9 +4009,9 @@ WC_OMIT_FRAME_POINTER void mlkem_thumb2_vec_compress_11(byte* r, sword16* v)
     __asm__ __volatile__ (
 #ifdef WOLFSSL_NO_VAR_ASSIGN_REG
         "PUSH	{%[L_asm_args]}\n\t"
-        "LDM	%[L_asm_args], {r0, r1}\n\t"
+        "LDM	%[L_asm_args], {r0, r1, r2}\n\t"
 #endif /* WOLFSSL_NO_VAR_ASSIGN_REG */
-        "ADD	r2, r1, #0x800\n\t"
+        "ADD	r2, r1, r2, LSL #9\n\t"
 #ifndef WOLFSSL_ARM_ARCH_7M
         "MOV	r8, #0xdbb6\n\t"
         "MOVT	r8, #0x9d7\n\t"
@@ -4679,12 +4682,12 @@ WC_OMIT_FRAME_POINTER void mlkem_thumb2_vec_compress_11(byte* r, sword16* v)
 #endif /* !WOLFSSL_ARM_ARCH_7M */
 #ifdef WOLFSSL_NO_VAR_ASSIGN_REG
         "POP	{%[L_asm_args]}\n\t"
-        "STM	%[L_asm_args], {r0, r1}\n\t"
+        "STM	%[L_asm_args], {r0, r1, r2}\n\t"
 #endif /* WOLFSSL_NO_VAR_ASSIGN_REG */
 #ifndef WOLFSSL_NO_VAR_ASSIGN_REG
-        : [r] "+r" (r), [v] "+r" (v)
+        : [r] "+r" (r), [v] "+r" (v), [k] "+r" (k)
         :
-        : "memory", "cc", "r2", "r3", "r4", "r5", "r6", "r7", "r8", "r9", "r10",
+        : "memory", "cc", "r3", "r4", "r5", "r6", "r7", "r8", "r9", "r10",
             "r11"
 #else
         : [L_asm_args] "+r" (L_asm_args_p)
@@ -4696,6 +4699,7 @@ WC_OMIT_FRAME_POINTER void mlkem_thumb2_vec_compress_11(byte* r, sword16* v)
 #ifdef WOLFSSL_NO_VAR_ASSIGN_REG
     r = (byte*)(size_t)L_asm_args[0];
     v = (sword16*)(size_t)L_asm_args[1];
+    k = (unsigned int)(size_t)L_asm_args[2];
 #endif /* WOLFSSL_NO_VAR_ASSIGN_REG */
 }
 
diff --git a/wolfssl/wolfcrypt/wc_mlkem.h b/wolfssl/wolfcrypt/wc_mlkem.h
index 766ea35613..68b57cde5c 100644
--- a/wolfssl/wolfcrypt/wc_mlkem.h
+++ b/wolfssl/wolfcrypt/wc_mlkem.h
@@ -889,7 +889,8 @@ WOLFSSL_LOCAL void mlkem_thumb2_basemul_mont_add(sword16* r, const sword16* a,
 WOLFSSL_LOCAL void mlkem_thumb2_csubq(sword16* p);
 WOLFSSL_LOCAL void mlkem_thumb2_vec_compress_10(byte* r, sword16* v,
     unsigned int k);
-WOLFSSL_LOCAL void mlkem_thumb2_vec_compress_11(byte* r, sword16* v);
+WOLFSSL_LOCAL void mlkem_thumb2_vec_compress_11(byte* r, sword16* v,
+    unsigned int k);
 WOLFSSL_LOCAL unsigned int mlkem_thumb2_rej_uniform(sword16* p,
     unsigned int len, const byte* r, unsigned int rLen);
 #elif defined(WOLFSSL_ARMASM)

[Frauschi]

@dgarske

• Thumb2 compress_11: fixed in c8e499d, pushed before your review and identical to your patch; the generator side is wolfSSL/scripts#708. thumb2-gcm-table builds ML-KEM through --enable-all and caught it, and the new thumb2-mlkem-small-mem row covers the one-polynomial path.
• Encapsulation writing the key: master already resets and squeezes key->prf on every encapsulation, so one MlKemKey was never safe for concurrent encapsulation. The cache fill doesn't change that.
• Fixes section: dropped both from the description; they landed in 121521b and 6f594ee.
• compVecSz / fail: fail is a parameter and compVecSz is set in the shared key-type switch, so scoping either needs an #ifdef per case; left as is.
• mlkem_encapsulate_seeds call site: the comment where cb is chosen already says the cipher text is compared a block at a time when cmp is set.