Repository navigation
Commit bce51f1
committed
arm64: fix the SME2 sbgemm kernel's ZA handoff and BFMOPA predicates
Two fixes from Michael Tesch's M4 Pro testing:
- s2b_blk had its own __arm_new("za"), so the call from the equally-
annotated s2b_drive set up a lazy ZA save that __arm_tpidr2_save
(OpenBLAS's aborting weak stub) committed. It now shares ZA from the
drive function through S2_S S2_ZA like the fp32/fp64 block functions.
- BFMOPA governs per 16-bit element, but the predicates were
svptrue_b32(), which enables only the even bf16 lanes and drops the
k+1 half of every pair (every k >= 2 wrong, k = 1 right). Now
svptrue_b16().
Also apply the suggested thread cap to SBGEMM: sme_level3_threads()
capped only real precision (!BFLOAT16), so default threads contended
for the SME units - 839 GFLOPS at n = 4096 against about 1190 on one
thread and 2347 with one thread per unit on the M4 Pro. The condition
is now (!BFLOAT16 || !BGEMM): bgemm keeps the NEON kernels and its
threading.
With the fixes his run reports the sbgemm/BLAS/CBLAS/utest suites
passing and 3000 randomized trials clean; 1193 GFLOPS at 2048^2 single
thread against 0.48 on develop.1 parent c151d85 commit bce51f1
2 files changed
Lines changed: 4 additions & 5 deletions
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
157 | 157 | | |
158 | 158 | | |
159 | 159 | | |
160 | | - | |
| 160 | + | |
161 | 161 | | |
162 | 162 | | |
163 | 163 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
92 | 92 | | |
93 | 93 | | |
94 | 94 | | |
95 | | - | |
| 95 | + | |
96 | 96 | | |
97 | 97 | | |
98 | 98 | | |
| |||
116 | 116 | | |
117 | 117 | | |
118 | 118 | | |
119 | | - | |
120 | | - | |
121 | | - | |
| 119 | + | |
| 120 | + | |
122 | 121 | | |
123 | 122 | | |
124 | 123 | | |
| |||
0 commit comments