Skip to content

Avoid false sharing: accumulate iteration counts locally, store once at the end - #95

Closed
quarckster wants to merge 1 commit into
openssl:mainfrom
quarckster:counter-false-sharing
Closed

Avoid false sharing: accumulate iteration counts locally, store once at the end#95
quarckster wants to merge 1 commit into
openssl:mainfrom
quarckster:counter-false-sharing

Conversation

@quarckster

@quarckster quarckster commented Aug 3, 2026

Copy link
Copy Markdown
Member

Every multi-threaded benchmark counts iterations with counts[num]++ on a plain size_t array. The counters are logically private, but 8 threads × 8 bytes = one 64-byte cache line, so every thread invalidates the line for all the others on every iteration. For short-op benchmarks (evp_cipher shared mode ≈ 100 ns/op) this line ping-pong dominates the measured result.

Worse, the damage depends on where malloc (16-byte aligned) happens to place the array relative to a line boundary:

all 8 in one line:   |c0 c1 c2 c3 c4 c5 c6 c7|          8 threads contend  → 0.30 µs
split 6+2:        |.. c0 c1 c2 c3 c4 c5|c6 c7 ..|       mixed              → 0.20 µs
split 4+4:        |.... c0 c1 c2 c3|c4 c5 c6 c7 ....|   two groups of 4    → 0.15 µs

On Windows the heap randomizes placement per process start, so the same binary hops between these levels from run to run. On Linux placement is deterministic, so results are flat — until an unrelated change shifts the startup allocation sequence and re-rolls the layout: we watched a master commit "regress" evp_cipher evp_shared by exactly 2.0x on every Linux worker overnight, with no real performance change.

The fix: workers accumulate their count in a local variable and store it to the shared array once, when the thread finishes — the pattern pkeyread and the evp_kdf/evp_pkey/evp_rand benchmarks already use — applied to the remaining 14 benchmarks:

before:  every iteration:  counts[num]++         all threads write one shared line
after:   every iteration:  count++               thread-local, stays in a register
         at thread end:    counts[num] = count   one write per thread

Also fixes evp_setpeer.c's counter allocation, which used sizeof(OSSL_TIME) for a size_t array.

A/B evidence — same host, same hour, interleaved runs, evp_cipher -t -o evp_shared -a AES-256-CBC 8, pinned to 8 CPUs:

pr-win11-ab pr-debian-ab

The effect grows with thread count: at ≥16 threads the counters span several lines, but each line still carries up to 8 writers and the lines now bounce between L3/CCD domains. µs/op, 5 runs each on the same hosts:

threads Debian unpatched Debian patched Win11 unpatched Win11 patched
8 0.295 0.088 0.15–0.30 (lottery) 0.130
16 0.294 0.087 0.32–0.42 0.13–0.15
32 0.325–0.370 0.089
64 0.417 0.130

The unpatched binary was measuring the counter cache line, not the cipher: patched, the per-op cost is flat from 8 to 32 threads (0.090 µs) — the apparent degradation to 0.42 µs and the run-to-run scatter were entirely the artifact. Note for consumers tracking results over time: short-op benchmark levels step down when this lands.

@esyr

esyr commented Aug 3, 2026

Copy link
Copy Markdown
Member

I would rather increment a local counter and then update the global array at the end of the thread's run (as it is done in pkeyread.c and some evp_*.c already), there's no need to hammer global data by each thread, and no need to excessively allocate global data.

@quarckster
quarckster force-pushed the counter-false-sharing branch from edcf3ef to eb518a0 Compare August 3, 2026 14:04
@quarckster

Copy link
Copy Markdown
Member Author

I would rather increment a local counter and then update the global array at the end of the thread's run (as it is done in pkeyread.c and some evp_*.c already), there's no need to hammer global data by each thread, and no need to excessively allocate global data.

Done. Reworked to local accumulation with a single store at thread end, as suggested (matching pkeyread/evp_kdf). Re-ran the A/B on both platforms with the new implementation: identical results (win11 30-run max/min 1.99→1.07; Linux 3.3x). Description and figures updated.

@esyr esyr left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM.

Comment thread source/evp_setpeer.c
}

counts = OPENSSL_malloc(sizeof(OSSL_TIME) * threadcount);
counts = OPENSSL_malloc(sizeof(size_t) * threadcount);

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This should be a separate commit, ideally.

@github-project-automation github-project-automation Bot moved this from Waiting Review to Waiting Merge in Development Board Aug 3, 2026
@esyr esyr changed the title Space per-thread counters one cache line apart to avoid false sharing Avoid false sharing: accumulate iteration counts locally, store once at the end Aug 3, 2026
@quarckster
quarckster force-pushed the counter-false-sharing branch from eb518a0 to 8db6401 Compare August 4, 2026 11:38
…at thread end

Assisted-by: Claude:claude-opus-5
@quarckster
quarckster force-pushed the counter-false-sharing branch from 8db6401 to 26b4268 Compare August 4, 2026 11:42
esyr pushed a commit that referenced this pull request Aug 4, 2026
…at thread end

Assisted-by: Claude:claude-opus-5

Reviewed-by: Eugene Syromiatnikov <esyr@openssl.org>
Reviewed-by: Nikola Pajkovsky <nikolap@openssl.org>
MergeDate: Tue Aug  4 11:43:11 2026
(Merged from #95)
@esyr

esyr commented Aug 4, 2026

Copy link
Copy Markdown
Member

Applied to main, thank you for your contribution.

@esyr esyr closed this Aug 4, 2026
@github-project-automation github-project-automation Bot moved this from Waiting Merge to Done in Development Board Aug 4, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

Status: Done

Development

Successfully merging this pull request may close these issues.

4 participants