From 232b9cd6b628d75123506a6ee2b06b5229adeb24 Mon Sep 17 00:00:00 2001 From: Kang Jaehong Date: Sun, 11 Oct 2026 03:41:14 +0900 Subject: [PATCH 1/3] Measure through a benchmark package of its own The performance suites were XCTest cases inside the test targets, timed by XCTest's meter on Apple platforms and by a clock of the harness's own elsewhere, read as a mean of ten runs, and compared by running each side fifteen times over. They move to `Benchmarks`, a package of their own on the package-benchmark harness, so that nothing a client resolves includes the harness: the library's manifest still has no dependencies. Every case comes across, each comparison with it: the standard library's `Mutex`, `DispatchSemaphore`, `pthread_rwlock_t`, a concurrent `DispatchQueue`, and an `actor`. What is measured is the library as it ships, with no `-enable-testing`; a sample is a million turns, so the scaled output reads per turn; and the harness reports percentiles of the clock and of instructions retired, keeps baselines, and compares a run against one. Allocation and retain counts are there for the asking, and not by default, since the hooks that count them slow the paths they count. Workers live on a pool of threads kept for the whole run: the allocation counter sets up a per-thread record on a thread's first free, which for a thread that frees nothing before it exits is from inside the exit, where the allocator traps. A contended sample runs in stretches of at most a tenth of a second, every worker parked between them. --- Benchmarks/.gitignore | 1 + .../AsyncMutexBenchmarks.swift | 198 +++++ .../AsyncRWLockBenchmarks.swift | 93 +++ .../AsyncSemaphoreBenchmarks.swift | 277 +++++++ .../Benchmarks.swift | 91 +++ .../Harness.swift | 682 ++++++++++++++++++ .../MutexBenchmarks.swift | 89 +++ .../RWLockBenchmarks.swift | 305 ++++++++ .../SemaphoreBenchmarks.swift | 78 ++ Benchmarks/Package.resolved | 78 ++ Benchmarks/Package.swift | 48 ++ 11 files changed, 1940 insertions(+) create mode 100644 Benchmarks/.gitignore create mode 100644 Benchmarks/Benchmarks/SynchronizationKitBenchmarks/AsyncMutexBenchmarks.swift create mode 100644 Benchmarks/Benchmarks/SynchronizationKitBenchmarks/AsyncRWLockBenchmarks.swift create mode 100644 Benchmarks/Benchmarks/SynchronizationKitBenchmarks/AsyncSemaphoreBenchmarks.swift create mode 100644 Benchmarks/Benchmarks/SynchronizationKitBenchmarks/Benchmarks.swift create mode 100644 Benchmarks/Benchmarks/SynchronizationKitBenchmarks/Harness.swift create mode 100644 Benchmarks/Benchmarks/SynchronizationKitBenchmarks/MutexBenchmarks.swift create mode 100644 Benchmarks/Benchmarks/SynchronizationKitBenchmarks/RWLockBenchmarks.swift create mode 100644 Benchmarks/Benchmarks/SynchronizationKitBenchmarks/SemaphoreBenchmarks.swift create mode 100644 Benchmarks/Package.resolved create mode 100644 Benchmarks/Package.swift diff --git a/Benchmarks/.gitignore b/Benchmarks/.gitignore new file mode 100644 index 0000000..3e3c971 --- /dev/null +++ b/Benchmarks/.gitignore @@ -0,0 +1 @@ +.benchmarkBaselines/ diff --git a/Benchmarks/Benchmarks/SynchronizationKitBenchmarks/AsyncMutexBenchmarks.swift b/Benchmarks/Benchmarks/SynchronizationKitBenchmarks/AsyncMutexBenchmarks.swift new file mode 100644 index 0000000..810ab45 --- /dev/null +++ b/Benchmarks/Benchmarks/SynchronizationKitBenchmarks/AsyncMutexBenchmarks.swift @@ -0,0 +1,198 @@ +// +// AsyncMutexBenchmarks.swift +// SynchronizationKitBenchmarks +// +// Copyright (c) 2026 Kang Jaehong +// SPDX-License-Identifier: Apache-2.0 +// + +import Benchmark +import Dispatch +import SynchronizationKit + +// What `AsyncMutex` costs: the uncontended take, and the handoff through +// the wait queue as the queue gets longer — and what an `actor` costs on +// the same turns, since the README says to prefer one wherever it fits. +// Each queue case runs once per implementation, so the two land in one +// report. +// +// The closure suspends once inside the lock, so every take under contention +// is a real handoff rather than a spin, and what grows with the task count +// is the queue a departing holder chooses the next holder from. +// +// An actor cannot hold anything across an `await`, so the suspension the +// mutex's closure makes inside the lock is made in the actor cases after +// the actor's method has returned: each turn is still one step and one +// suspension, and what the two differ in is the handoff — through the +// mutex's wait queue, or through the actor's mailbox — which is what the +// numbers say the choice between them costs. + +/// A reference to hold the lock by; `MutexBox` says why. +private final class AsyncMutexBox: @unchecked Sendable { + let lock = AsyncMutex(ChasePayload()) +} + +/// The same payload behind an actor. +private actor ActorBox { + private var payload = ChasePayload() + + /// One write and one step of the chase from `index`. + func step(from index: Int) -> Int { + payload.writes &+= 1 + return payload.cycle[index] + } + + var writes: Int { + payload.writes + } +} + +/// The queue lengths the handoff is measured through, each with the turns +/// per task that make a sample up to the scaling factor. Past a few hundred +/// waiters, what shows is the queue's own bookkeeping: a handoff there +/// costs what one in a long queue does, and a regression that scales with +/// the queue costs several times it. +let queueLengths = [ + (scenario: "uncontended", tasks: 1, iterations: 1_000_000), + (scenario: "short queue", tasks: 8, iterations: 125_000), + (scenario: "long queue", tasks: 64, iterations: 15_625), + (scenario: "very long queue", tasks: 500, iterations: 2_000), +] + +private func registerHandoff(_ scenario: String, tasks: Int, iterations: Int) { + Benchmark("AsyncMutex \(scenario)", configuration: .handoff) { benchmark in + precondition(tasks * iterations == benchmark.turnsPerSample) + benchmark.measureTaskContention( + tasks: tasks, + iterations: iterations, + makeFixture: AsyncMutexBox.init + ) { box, task in + var index = task + for _ in 0 ..< iterations { + try await box.lock.withLock { payload in + payload.writes &+= 1 + index = payload.cycle[index] + await Task.yield() + } + } + return index + } check: { box in + // Nobody holds the lock any more, so the try cannot fail; and it + // cannot throw, which keeps the task's result from needing a + // reader. + let expected = benchmark.turnsPerSample + let finished = DispatchSemaphore(value: 0) + Task.detached { + let writes = await box.lock.withLockIfAvailable { $0.writes } + precondition(writes == expected, "the workload did not run") + finished.signal() + } + finished.wait() + } + } +} + +private func registerActorHandoff(_ scenario: String, tasks: Int, iterations: Int) { + Benchmark("AsyncMutex \(scenario), actor", configuration: .handoff) { benchmark in + precondition(tasks * iterations == benchmark.turnsPerSample) + benchmark.measureTaskContention( + tasks: tasks, + iterations: iterations, + makeFixture: ActorBox.init + ) { box, task in + var index = task + for _ in 0 ..< iterations { + index = await box.step(from: index) + await Task.yield() + } + return index + } check: { box in + let expected = benchmark.turnsPerSample + let finished = DispatchSemaphore(value: 0) + Task.detached { + let writes = await box.writes + precondition(writes == expected, "the workload did not run") + finished.signal() + } + finished.wait() + } + } +} + +/// Two tasks at the harness's priority hand the lock back and forth through +/// a queue of `lows` waiters at low priority; the semaphore cases' +/// `registerPriorityHandoff` says why the low waiters are detached tasks +/// the workers spawn and do not await, why each takes one turn and spawns +/// its successor, and how the sample ends only once they have all left. The +/// one thing particular to this primitive is what makes the successor +/// necessary: a filler that held the lock while a high task queued was +/// escalated to that task's priority, and stays there. +private func registerPriorityHandoff(lows: Int, iterations: Int) { + final class Fixture: Sendable { + let lock = AsyncMutex(0) + let finished = Mutex(0) + let fillers: Mutex + let drained = Gate() + + init(lows: Int) { + fillers = Mutex(lows) + } + } + @Sendable func fill(_ fixture: Fixture) { + // The task is deliberately neither awaited nor kept: awaiting it + // would raise it to the awaiter's priority, and the fixture's + // flags are how the fillers end. + _ = Task.detached(priority: .low) { + try await fixture.lock.withLock { _ in + await Task.yield() + } + if fixture.finished.withLock({ $0 < 2 }) { + fill(fixture) + } else if fixture.fillers.withLock({ $0 -= 1; return $0 == 0 }) { + fixture.drained.open() + } + } + } + Benchmark("AsyncMutex high priority among low waiters", configuration: .handoff) { benchmark in + precondition(2 * iterations == benchmark.turnsPerSample) + benchmark.measureTaskContention( + tasks: lows + 2, + iterations: iterations, + makeFixture: { Fixture(lows: lows) }, + steps: { $0 < 2 ? nil : 0 } + ) { fixture, task in + var index = task + if task < 2 { + for _ in 0 ..< iterations { + try await fixture.lock.withLock { holds in + holds += 1 + index = Chase.cycle[index] + await Task.yield() + } + } + fixture.finished.withLock { $0 += 1 } + await fixture.drained.wait() + } else { + fill(fixture) + } + return index + } check: { fixture in + let expected = benchmark.turnsPerSample + let finished = DispatchSemaphore(value: 0) + Task.detached { + let holds = await fixture.lock.withLockIfAvailable { $0 } + precondition(holds == expected, "the workload did not run") + finished.signal() + } + finished.wait() + } + } +} + +func registerAsyncMutexBenchmarks() { + for (scenario, tasks, iterations) in queueLengths { + registerHandoff(scenario, tasks: tasks, iterations: iterations) + registerActorHandoff(scenario, tasks: tasks, iterations: iterations) + } + registerPriorityHandoff(lows: 512, iterations: 500_000) +} diff --git a/Benchmarks/Benchmarks/SynchronizationKitBenchmarks/AsyncRWLockBenchmarks.swift b/Benchmarks/Benchmarks/SynchronizationKitBenchmarks/AsyncRWLockBenchmarks.swift new file mode 100644 index 0000000..625651d --- /dev/null +++ b/Benchmarks/Benchmarks/SynchronizationKitBenchmarks/AsyncRWLockBenchmarks.swift @@ -0,0 +1,93 @@ +// +// AsyncRWLockBenchmarks.swift +// SynchronizationKitBenchmarks +// +// Copyright (c) 2026 Kang Jaehong +// SPDX-License-Identifier: Apache-2.0 +// + +import Benchmark +import Dispatch +import SynchronizationKit + +// What `AsyncRWLock` costs: the uncontended take of either side, the handoff +// between writers as the queue gets longer, and a read-mostly mix in which +// readers are admitted together and a writer waits them out. +// +// The writer cases are `AsyncMutex`'s handoff on the other primitive — the +// closure suspends once inside the lock, so every take under contention is +// a real handoff — and the two share their counts, so the numbers sit side +// by side. What the read side adds to the same wait queue is in the mixed +// cases, where a writer's arrival ends a batch of readers and its release +// admits the next. + +/// A reference to hold the lock by; `MutexBox` says why. +private final class AsyncRWLockBox: @unchecked Sendable { + let lock = AsyncRWLock(ChasePayload()) +} + +/// `tasks` tasks each take the lock `iterations` times, one turn in +/// `writeEvery` for writing and the rest for reading, suspending once inside +/// it either way. A `writeEvery` of one is all writers. +private func registerHandoff(_ scenario: String, tasks: Int, iterations: Int, writeEvery: Int = 1) { + Benchmark("AsyncRWLock \(scenario)", configuration: .handoff) { benchmark in + precondition(tasks * iterations == benchmark.turnsPerSample) + benchmark.measureTaskContention( + tasks: tasks, + iterations: iterations, + makeFixture: AsyncRWLockBox.init + ) { box, task in + var index = task + for turn in 0 ..< iterations { + if turn % writeEvery == 0 { + try await box.lock.withWriteLock { payload in + payload.writes &+= 1 + index = payload.cycle[index] + await Task.yield() + } + } else { + try await box.lock.withReadLock { payload in + index = payload.cycle[index] + await Task.yield() + } + } + } + return index + } check: { box in + // Nobody holds the lock any more, so the try cannot fail; and it + // cannot throw, which keeps the task's result from needing a + // reader. + let finished = DispatchSemaphore(value: 0) + Task.detached { + let writes = await box.lock.withReadLockIfAvailable { $0.writes } + let expected = tasks * ((iterations + writeEvery - 1) / writeEvery) + precondition(writes == expected, "the workload did not run") + finished.signal() + } + finished.wait() + } + } +} + +func registerAsyncRWLockBenchmarks() { + // One task alone, reading: the take and release of the read side with + // nobody to wait for or to wake. `writeEvery` past the last turn, so + // the only write is the first. + registerHandoff("uncontended reads", tasks: 1, iterations: 1_000_000, writeEvery: 1_000_000) + registerHandoff("uncontended writes", tasks: 1, iterations: 1_000_000) + + for (scenario, tasks, iterations) in queueLengths.dropFirst() { + registerHandoff("writer \(scenario)", tasks: tasks, iterations: iterations) + } + + // One turn in eight a write: enough writers that the readers rarely run + // long unopposed, few enough that most admissions are of readers. + for (scenario, tasks, iterations) in queueLengths[1 ... 2] { + registerHandoff( + "read-mostly \(scenario)", + tasks: tasks, + iterations: iterations, + writeEvery: 8 + ) + } +} diff --git a/Benchmarks/Benchmarks/SynchronizationKitBenchmarks/AsyncSemaphoreBenchmarks.swift b/Benchmarks/Benchmarks/SynchronizationKitBenchmarks/AsyncSemaphoreBenchmarks.swift new file mode 100644 index 0000000..55cf442 --- /dev/null +++ b/Benchmarks/Benchmarks/SynchronizationKitBenchmarks/AsyncSemaphoreBenchmarks.swift @@ -0,0 +1,277 @@ +// +// AsyncSemaphoreBenchmarks.swift +// SynchronizationKitBenchmarks +// +// Copyright (c) 2026 Kang Jaehong +// SPDX-License-Identifier: Apache-2.0 +// + +import Benchmark +import Dispatch +import SynchronizationKit + +// What `AsyncSemaphore` costs: the uncontended take, and the handoff through +// the wait queue as the queue gets longer. +// +// The semaphore is used as a lock — a count of one, taken around the chase +// and a suspension — so the numbers sit beside `AsyncMutex`'s, which differs +// from this in holding a value and in escalating its holder. + +/// The semaphore and what it guards, held by reference; `MutexBox` says why. +/// The payload is guarded by the semaphore, which is what makes the +/// unchecked conformance right. +private final class AsyncSemaphoreBox: MeasuredLock, @unchecked Sendable { + let semaphore = AsyncSemaphore(value: 1) + private var payload = ChasePayload() + + /// The blocking `wait()`, for the thread case. + func step(from index: Int) -> Int { + semaphore.wait() + payload.writes &+= 1 + let end = payload.cycle[index] + semaphore.signal() + return end + } + + /// One write and one step of the chase from `index`, with a suspension + /// inside the semaphore. + func stepAndYield(from index: Int) async throws -> Int { + try await semaphore.wait() + payload.writes &+= 1 + let end = payload.cycle[index] + await Task.yield() + semaphore.signal() + return end + } + + var writes: Int { + semaphore.wait() + let writes = payload.writes + semaphore.signal() + return writes + } +} + +private func registerHandoff(_ scenario: String, tasks: Int, iterations: Int) { + Benchmark("AsyncSemaphore \(scenario)", configuration: .handoff) { benchmark in + precondition(tasks * iterations == benchmark.turnsPerSample) + benchmark.measureTaskContention( + tasks: tasks, + iterations: iterations, + makeFixture: AsyncSemaphoreBox.init + ) { box, task in + var index = task + for _ in 0 ..< iterations { + index = try await box.stepAndYield(from: index) + } + return index + } check: { box in + precondition(box.writes == benchmark.turnsPerSample, "the workload did not run") + } + } +} + +/// Every task but the first waits on a count that is never raised, so +/// nothing it waits for can arrive; the first cancels them once they are +/// all queued, in a random order so that each departure is from somewhere +/// in the middle, and the round repeats. Every `wait()` must throw, and a +/// waiter moves its chase only when its child's did, so a child that +/// returned instead fails the chase check. +/// +/// A waiter announces its child just before the child waits, and no +/// cancellation goes out until every waiter has announced. The queue itself +/// is not visible from outside the package, so a child announced but not +/// yet queued when its cancellation lands — a window of a few instructions +/// on another thread — is cancelled on arrival rather than from the queue; +/// its `wait()` throws all the same, and the chase check holds. +/// +/// The handles a round cancels are collected under a lock and taken out of +/// it before the cancellations go out. A waiter registers its next child +/// only after the previous one has been cancelled and awaited, so the +/// registry fills to the waiter count exactly once per round. +private func registerCancellation(waiters: Int, rounds: Int) { + final class Fixture: Sendable { + let semaphore = AsyncSemaphore(value: 0) + let handles = Mutex<[Task]>([]) + let announced = Mutex(0) + } + // A turn is one cancellation, so the sample is `waiters × rounds` of + // them: a thousand, over a queue long enough to be a long queue. + var configuration = Benchmark.Configuration.handoff + configuration.scalingFactor = .kilo + Benchmark( + "AsyncSemaphore cancellation in long queue", + configuration: configuration + ) { benchmark in + precondition(waiters * rounds == benchmark.turnsPerSample) + benchmark.measureTaskContention( + tasks: waiters + 1, + iterations: rounds, + makeFixture: Fixture.init + ) { fixture, task in + var index = task + if task == 0 { + var generator = SplitMix64(seed: 0x5EED) + for _ in 0 ..< rounds { + var handles: [Task] = [] + while true { + if fixture.announced.withLock({ $0 == waiters }) { + handles = fixture.handles.withLock { $0.count == waiters ? $0 : [] } + } + if !handles.isEmpty { + break + } + await Task.yield() + } + fixture.handles.withLock { $0.removeAll(keepingCapacity: true) } + fixture.announced.withLock { $0 = 0 } + handles.shuffle(using: &generator) + for handle in handles { + handle.cancel() + } + index = Chase.cycle[index] + } + } else { + for _ in 0 ..< rounds { + let child = Task { @Sendable in + fixture.announced.withLock { $0 += 1 } + try await fixture.semaphore.wait() + } + fixture.handles.withLock { $0.append(child) } + do { + try await child.value + } catch is CancellationError { + index = Chase.cycle[index] + } + } + } + return index + } + } +} + +/// Two tasks at the harness's priority hand the count back and forth +/// through a queue of `lows` waiters at low priority, which are served only +/// at a signal that finds neither of the two queued. Each handoff to one of +/// the two is what the queue does to place a high-priority arrival among +/// the low ones and to let it go, which is what would grow with `lows` if +/// the queue walked to find either. +/// +/// The low waiters are detached tasks of their own, since the harness runs +/// every worker at one priority. Each remaining worker spawns one and +/// returns without awaiting it — awaiting a task raises it to the awaiter's +/// priority, the runtime's doing — and a filler takes one turn and spawns +/// its successor rather than looping: a semaphore escalates nobody, but the +/// mutex's does, and the two measure alike. A filler holds nothing it could +/// chase over, so its worker takes no steps. +/// +/// Once the two are done, each filler's next turn is its last, and the last +/// of them opens a gate the two wait at, so no filler outlives the sample +/// into the next. The drain — one turn per filler — is timed with the rest, +/// the same amount every sample. +private func registerPriorityHandoff(lows: Int, iterations: Int) { + final class Fixture: Sendable { + let semaphore = AsyncSemaphore(value: 1) + let finished = Mutex(0) + let fillers: Mutex + let drained = Gate() + let holds = Atomic(0) + + init(lows: Int) { + fillers = Mutex(lows) + } + } + /// One turn at low priority, then a fresh filler in its place, or one + /// fewer once the two high tasks are done. A filler is never reused: + /// where the OS escalates a holder, one that held while a high task + /// queued was raised to that task's priority and stays there, so its + /// next turn would be a high-priority arrival. + @Sendable func fill(_ fixture: Fixture) { + // The task is deliberately neither awaited nor kept: awaiting it + // would raise it to the awaiter's priority, and the fixture's + // flags are how the fillers end. + _ = Task.detached(priority: .low) { + try await fixture.semaphore.wait() + await Task.yield() + fixture.semaphore.signal() + if fixture.finished.withLock({ $0 < 2 }) { + fill(fixture) + } else if fixture.fillers.withLock({ $0 -= 1; return $0 == 0 }) { + fixture.drained.open() + } + } + } + Benchmark( + "AsyncSemaphore high priority among low waiters", + configuration: .handoff + ) { benchmark in + precondition(2 * iterations == benchmark.turnsPerSample) + benchmark.measureTaskContention( + tasks: lows + 2, + iterations: iterations, + makeFixture: { Fixture(lows: lows) }, + steps: { $0 < 2 ? nil : 0 } + ) { fixture, task in + var index = task + if task < 2 { + for _ in 0 ..< iterations { + try await fixture.semaphore.wait() + index = Chase.cycle[index] + fixture.holds.wrappingAdd(1, ordering: .relaxed) + await Task.yield() + fixture.semaphore.signal() + } + fixture.finished.withLock { $0 += 1 } + await fixture.drained.wait() + } else { + fill(fixture) + } + return index + } check: { fixture in + precondition( + fixture.holds.load(ordering: .relaxed) == benchmark.turnsPerSample, + "the workload did not run" + ) + } + } +} + +func registerAsyncSemaphoreBenchmarks() { + for (scenario, tasks, iterations) in queueLengths { + registerHandoff(scenario, tasks: tasks, iterations: iterations) + } + + // The other thing a queue of this length has to do in constant time. + registerCancellation(waiters: 500, rounds: 2) + + // And the third: a high-priority arrival placed, and served, ahead of a + // long queue at low priority. + registerPriorityHandoff(lows: 512, iterations: 500_000) + + // The blocking `wait()`, contended by threads the way `Semaphore` is in + // its own cases. The two numbers are not the same measurement, and the + // gap between them — several times over — is not overhead in the queue. + // Measured, it is a context switch per handoff: this semaphore hands the + // count to the waiter at the head of the queue, so every signal moves + // the work to another thread, where `Semaphore` only raises the count, + // and the thread that just signalled takes it back before the one it + // woke has run. That barging is what its number is made of — a few + // thousand switches across a million handoffs, against one or two for + // each of them here — and giving it up is what the no-overtaking and + // priority guarantees cost. Folding the slow path's two critical + // sections into one was tried and moved nothing. + Benchmark("AsyncSemaphore contended threads", configuration: .contended) { benchmark in + benchmark.measureContention( + workers: contendedWorkers, + makeFixture: AsyncSemaphoreBox.init + ) { box, worker, share in + var index = worker + share.eachTurn { + index = box.step(from: index) + } + return index + } check: { box in + precondition(box.writes == benchmark.turnsPerSample, "the workload did not run") + } + } +} diff --git a/Benchmarks/Benchmarks/SynchronizationKitBenchmarks/Benchmarks.swift b/Benchmarks/Benchmarks/SynchronizationKitBenchmarks/Benchmarks.swift new file mode 100644 index 0000000..2183299 --- /dev/null +++ b/Benchmarks/Benchmarks/SynchronizationKitBenchmarks/Benchmarks.swift @@ -0,0 +1,91 @@ +// +// Benchmarks.swift +// SynchronizationKitBenchmarks +// +// Copyright (c) 2026 Kang Jaehong +// SPDX-License-Identifier: Apache-2.0 +// + +import Benchmark + +/// What the harness runs: every benchmark in this target, one register call +/// per primitive, each measured beside what a client would otherwise write. +/// +/// Every sample takes as many turns as its scaling factor says and the +/// output is scaled, so what prints is the cost of one turn: one take and +/// release of the lock around one step of the chase that `Harness.swift` +/// describes. Read instructions for anything uncontended — it varies by a +/// fraction of a percent between runs where the clock varies by tens — and +/// the clock under contention, where the scheduler dominates both; on Linux +/// the instruction count needs performance counters the kernel may not let +/// a process read, and the column is then absent rather than wrong. +/// +/// What a primitive allocates and retains on the way — for a lock's fast +/// path it should be nothing — is not measured by default, because +/// counting it costs what it counts: the harness hooks every allocation and +/// every retain and release to count them, which added an eighth to the +/// clock of an asynchronous handoff and instructions of its own to every +/// path that retains. The counts are a pass of their own: +/// +/// swift package benchmark --metric mallocCountTotal \ +/// --metric retainCount --metric releaseCount +/// +/// Nothing here fails on a regression, and no threshold is checked in: the +/// numbers are to be read, or compared against a baseline taken on the same +/// machine before a change. +let benchmarks: @Sendable () -> Void = { + // The harness keeps its defaults in a `nonisolated(unsafe)` static, + // written once here before any benchmark reads it. + unsafe Benchmark.defaultConfiguration = .init( + metrics: [ + .wallClock, + .instructions, + ], + timeUnits: .nanoseconds, + scalingFactor: .mega + ) + + registerMutexBenchmarks() + registerSemaphoreBenchmarks() + registerRWLockBenchmarks() + registerAsyncMutexBenchmarks() + registerAsyncRWLockBenchmarks() + registerAsyncSemaphoreBenchmarks() +} + +extension Benchmark.Configuration { + /// An uncontended case: a million turns per sample, as many samples as + /// fit in a few seconds. Computed rather than stored so that it reads + /// the defaults as they are when a benchmark is registered. + static var uncontended: Self { + .init(maxDuration: .seconds(3)) + } + + /// A case contended by threads: the same million turns shared between + /// the workers, a handful of samples after one to warm up, since one + /// sample is a whole contended run — or as many as fit in ten seconds, + /// on a machine where a handoff costs tens of microseconds. Context + /// switches on top of the defaults, since under contention they are + /// what a handoff costs; skipped on a machine too small for the result + /// to say anything about contention. + static var contended: Self { + .init( + metrics: unsafe Benchmark.defaultConfiguration.metrics + [.cpuTotal, .contextSwitches], + warmupIterations: 1, + maxDuration: .seconds(10), + maxIterations: 5, + skip: !hasRoomToContend + ) + } + + /// A case contended by tasks: as `contended`, but a task needs no core + /// of its own, so nothing is skipped. + static var handoff: Self { + .init( + metrics: unsafe Benchmark.defaultConfiguration.metrics + [.cpuTotal, .contextSwitches], + warmupIterations: 1, + maxDuration: .seconds(10), + maxIterations: 5 + ) + } +} diff --git a/Benchmarks/Benchmarks/SynchronizationKitBenchmarks/Harness.swift b/Benchmarks/Benchmarks/SynchronizationKitBenchmarks/Harness.swift new file mode 100644 index 0000000..90b3cb4 --- /dev/null +++ b/Benchmarks/Benchmarks/SynchronizationKitBenchmarks/Harness.swift @@ -0,0 +1,682 @@ +// +// Harness.swift +// SynchronizationKitBenchmarks +// +// Copyright (c) 2026 Kang Jaehong +// SPDX-License-Identifier: Apache-2.0 +// + +// The harness every benchmark here measures through, kept in one place so +// that a Mutex measurement and an RWLock measurement differ only in the lock. +// +// Two ways to measure a lock badly, both of which this repository has already +// been caught by, and what is done about each: +// +// - Nothing forces one critical section to finish before the next begins, so +// the processor overlaps them and the work inside stops costing anything. +// Every section measured through this follows a pointer chase — a single +// cycle through a protected array, each load depending on the one before — +// which the processor cannot run ahead of. +// - Threads left to the scheduler land on efficiency cores and the spread +// swallows the effect. Every thread here is pinned to `.userInteractive`. +// +// A benchmark whose work gets optimized away reports excellent numbers, so +// each measurement checks that the chase actually happened before it accepts +// a result, and traps if it did not: no number is better than one that means +// nothing. +// +// Every sample takes exactly as many turns as the benchmark's scaling factor +// says — a million, for the most part — so that what the harness prints, +// scaled, is the cost of one turn. The contended measurements share those +// turns out between their workers from a budget; the uncontended ones loop +// over `scaledIterations`. + +import Benchmark +import Dispatch +import Foundation +import SynchronizationKit + +// MARK: - The chase + +/// What a measured lock protects: a permutation the readers chase, and a +/// counter the writers move. +struct ChasePayload: Sendable { + var cycle: [Int] + var writes = 0 + + init() { + cycle = Chase.cycle + } + + /// `steps` steps of the chase from `index`, and where it ended. + func chase(from index: Int, steps: Int) -> Int { + var index = index + for _ in 0 ..< steps { + index = cycle[index] + } + return index + } +} + +/// The pointer chase every measurement runs, and where it ends. +enum Chase { + static let slots = 1024 + + /// A single cycle visiting every slot, so a chase never short-circuits + /// into a small loop that would sit in cache. + static let cycle: [Int] = { + var order = Array(1 ..< slots) + var state = UInt64(0x2545_F491_4F6C_DD1D) + for position in stride(from: order.count - 1, to: 0, by: -1) { + state ^= state << 13 + state ^= state >> 7 + state ^= state << 17 + order.swapAt(position, Int(state % UInt64(position + 1))) + } + + var cycle = [Int](repeating: 0, count: slots) + var previous = 0 + for next in order { + cycle[previous] = next + previous = next + } + cycle[previous] = 0 + return cycle + }() + + /// Where following the cycle `steps` times from `start` ends up. + /// + /// The cycle visits every slot once, so it repeats with period `slots` + /// and the answer is reachable in at most that many steps however long + /// the run was. Comparing against this is what makes the checks below + /// say something: a chase that never moved lands on `start`, not here. + static func end(from start: Int, steps: Int) -> Int { + var index = start + for _ in 0 ..< (steps % slots) { + index = cycle[index] + } + return index + } +} + +// MARK: - The machine + +/// How many workers a contended case runs: every core but two, so the odd +/// one out — a writer, a signaller — and the harness each have one. +let contendedWorkers = max(2, ProcessInfo.processInfo.activeProcessorCount - 2) + +/// Twice the cores, so most of the time the lock is handed to a thread that +/// has to be woken for it. +let oversubscribedWorkers = ProcessInfo.processInfo.activeProcessorCount * 2 + +/// Whether the machine is big enough for a contended result to say anything +/// about contention. A case that needs it is skipped on a smaller one. +let hasRoomToContend = ProcessInfo.processInfo.activeProcessorCount >= 4 + +// MARK: - Sharing the work out + +/// A counter the measured blocks can add to from several threads. +/// +/// Lock-free on purpose. A lock here would be a second lock inside the +/// measured window, contended by every worker at the moment they all finish — +/// noise on the same scale as the handoff being measured. A class because +/// `Atomic` is noncopyable and so cannot be captured by a measured block. +final class Tally: @unchecked Sendable { + private let storage = Atomic(0) + + var value: Int { + storage.load(ordering: .acquiring) + } + + func add(_ operand: Int) { + storage.wrappingAdd(operand, ordering: .acquiringAndReleasing) + } +} + +/// Where each worker of a contended case leaves its result: one slot per +/// worker, written by that worker alone and read once every worker has +/// finished, so that nothing is checked or counted inside the measured +/// window. +/// +/// `@unchecked Sendable` on that discipline: no slot is written by two +/// workers, and the semaphore the harness joins the workers on orders every +/// write before the reads. +/// +/// `@safe`, with the buffer marked `@unsafe`: every access to it is spelled +/// out below, and nothing hands the pointer out. +@safe +private final class WorkerResults: @unchecked Sendable { + @unsafe private let slots: UnsafeMutablePointer<(end: Int, steps: Int, turns: Int)> + private let count: Int + + init(count: Int) { + self.count = count + unsafe slots = UnsafeMutablePointer<(end: Int, steps: Int, turns: Int)> + .allocate(capacity: count) + unsafe slots.initialize(repeating: (end: 0, steps: 0, turns: 0), count: count) + } + + deinit { + unsafe slots.deinitialize(count: count) + unsafe slots.deallocate() + } + + func record(end: Int, steps: Int, turns: Int, for worker: Int) { + unsafe slots[worker] = (end: end, steps: steps, turns: turns) + } + + func result(of worker: Int) -> (end: Int, steps: Int, turns: Int) { + unsafe slots[worker] + } +} + +/// The turns a group of workers has left between them. +/// +/// One atomic counter, drawn down a batch at a time. Lock-free for the +/// reason `Tally` is: every worker touches it, and a lock here would be a +/// second contended primitive inside the measured window. The batch keeps +/// the counter itself from becoming that — a worker comes back to it once +/// every `WorkShare.batch` turns, not every turn. +final class WorkBudget: @unchecked Sendable { + private let remaining: Atomic + + init(total: Int) { + remaining = Atomic(total) + } + + /// Claims up to `count` turns, and returns how many were left to claim: + /// `count`, fewer at the end, or zero once the budget is spent. + func claim(upTo count: Int) -> Int { + var left = remaining.load(ordering: .relaxed) + while left > 0 { + let claiming = min(count, left) + let (exchanged, observed) = remaining.compareExchange( + expected: left, + desired: left - claiming, + ordering: .relaxed + ) + if exchanged { + return claiming + } + left = observed + } + return 0 + } +} + +/// One worker's share of a group's budget: `eachTurn` runs the worker's +/// body once per turn until the budget is spent. +/// +/// A method taking the body rather than a sequence to iterate, so that the +/// loop runs on locals: a turn of the cheapest primitive here is a few +/// nanoseconds, and an iterator's call and the exclusivity checks on its +/// stored properties cost as much again — measured, they tripled the +/// contended `Mutex` number. A class, so that the harness can read how many +/// turns the worker took once its body has returned; it is created on the +/// worker's thread and used only there. +final class WorkShare { + /// How many turns are claimed from the budget at once: often enough + /// that a slow worker does not hold much back, seldom enough that the + /// budget's counter is not contended. Claiming every turn was measured + /// at ten times the cost of the turns themselves. + static let batch = 64 + + /// How many batches pass between looks at the clock: a look costs tens + /// of nanoseconds, and this spreads one over a thousand turns. + private static let batchesBetweenLooks = 16 + + private let budget: WorkBudget + private let deadline: ContinuousClock.Instant + + /// How many turns the worker has taken, and how many chase steps they + /// came to. + private(set) var turns = 0 + private(set) var steps = 0 + + init(of budget: WorkBudget, until deadline: ContinuousClock.Instant) { + self.budget = budget + self.deadline = deadline + } + + /// Runs `turn` once per turn claimed from the budget, until it is spent + /// or the deadline has passed, counting `steps` chase steps for each: + /// one, unless the body chases further and says so. + func eachTurn(steps: Int = 1, _ turn: () -> Void) { + var taken = 0 + var batches = 0 + while true { + let claimed = budget.claim(upTo: Self.batch) + if claimed == 0 { + break + } + for _ in 0 ..< claimed { + turn() + } + taken += claimed + batches += 1 + if batches == Self.batchesBetweenLooks { + batches = 0 + if ContinuousClock.now >= deadline { + break + } + } + } + turns += taken + self.steps += taken * steps + } +} + +// MARK: - The threads + +/// The threads the contended cases run their workers on, started once and +/// kept for the whole run. +/// +/// Kept rather than made afresh for every sample because of what happens +/// when one ends. The harness counts allocations through an interposer that +/// keeps a record per thread, set up the first time the thread frees; a +/// thread exiting frees its thread-specific data from inside the threading +/// library's own teardown, and a thread whose first free that is sets the +/// record up there, where the allocator traps. A pool never exits a thread. +/// It also keeps thread creation out of a sample altogether, parked or not. +final class WorkerPool: @unchecked Sendable { + /// One thread and the job it is waiting for. The job is written before + /// the semaphore is signalled and read after it is waited on, which is + /// what makes the unchecked conformance right. The thread takes the job + /// out of the slot before running it, not after: the harness hands the + /// next job over as soon as the running one has signalled that it is + /// done, which is before it has returned, and a slot cleared after the + /// return would lose it. + private final class Worker: @unchecked Sendable { + private let wake = DispatchSemaphore(value: 0) + private var job: (@Sendable () -> Void)? + + init() { + let thread = Thread { [self] in + while true { + wake.wait() + let job = self.job + self.job = nil + job?() + } + } + thread.qualityOfService = .userInteractive + thread.start() + } + + func run(_ job: @escaping @Sendable () -> Void) { + self.job = job + wake.signal() + } + } + + /// Enough for the widest case: the oversubscribed readers with a + /// writer, or the contended workers with four. + static let shared = WorkerPool(size: oversubscribedWorkers + 4) + + private let workers: [Worker] + + private init(size: Int) { + workers = (0 ..< size).map { _ in Worker() } + } + + /// Runs `job` on the thread numbered `worker`. + func run(on worker: Int, _ job: @escaping @Sendable () -> Void) { + precondition(worker < workers.count, "more workers than the pool has threads") + workers[worker].run(job) + } +} + +// MARK: - Measuring + +extension Benchmark { + /// How many turns one sample takes, which is the scaling factor so that + /// the scaled output reads per turn. + var turnsPerSample: Int { + configuration.scalingFactor.rawValue + } + + /// Runs one thread over a fresh fixture, timing only the work. + /// + /// `work` is handed the fixture and the turns to take; it chases the + /// cycle once per turn from index zero and returns where it ended, which + /// the harness checks against where that many steps should have ended. + /// `check` sees the fixture afterwards for whatever else the body has to + /// have done. + func measureUncontended( + makeFixture: () -> Fixture, + work: (Fixture, _ turns: Range) -> Int, + check: (Fixture) -> Void = { _ in } + ) { + let fixture = makeFixture() + startMeasurement() + let end = work(fixture, scaledIterations) + stopMeasurement() + precondition(end == Chase.end(from: 0, steps: turnsPerSample), "the chase did not advance") + check(fixture) + } + + /// Runs threads over a fresh fixture, timing only the part where they are + /// actually contending. + /// + /// The workers are parked on the pool's threads first, then released + /// together, so nothing but the work is inside the measured window. + /// + /// The sample's turns are shared out between the groups in `groups` — + /// readers, say, and writers — in proportion to their worker counts, and + /// the work is not divided up front within a group: its workers draw + /// from the group's budget as they go, a batch at a time, so a worker + /// that happens to land on a slower core takes fewer turns and the group + /// finishes together rather than waiting on its slowest member. On a + /// chip with more than one kind of core that is the difference between + /// measuring the primitive and measuring which cores the scheduler + /// picked. Workers are numbered across the groups in order, so the first + /// group's workers come first. + /// + /// `work` is handed the fixture, its worker's number, and its share of + /// the group's budget, whose `eachTurn` runs its turns; it chases the + /// cycle once per turn, or the `steps` it tells `eachTurn`, from an + /// index that starts at its number and returns where it ended. The + /// harness checks each against where that many steps should have + /// ended, and that every budget was spent, so a body whose work was + /// optimized away traps rather than measuring nothing. `check` sees the + /// fixture afterwards, with each group's budget, for whatever else the + /// body has to have done. + /// + /// The sample is run in stretches of at most `stretch`, every worker + /// parked between one and the next, and `work` is called once per + /// stretch. The budgets carry over, the measurement runs through, and + /// a sample that finishes inside one stretch — most of them — never + /// pauses at all. + func measureContention( + groups: [Int], + makeFixture: () -> Fixture, + work: @escaping @Sendable (Fixture, _ worker: Int, _ share: WorkShare) -> Int, + check: (Fixture, _ budgets: [Int]) -> Void = { _, _ in } + ) { + let fixture = makeFixture() + let parked = DispatchSemaphore(value: 0) + let start = DispatchSemaphore(value: 0) + let finished = DispatchSemaphore(value: 0) + let workers = groups.reduce(0, +) + let totals = budgetTotals(for: groups) + let budgets = totals.map(WorkBudget.init) + var stretches: [WorkerResults] = [] + var turnsTaken = 0 + + startMeasurement() + repeat { + let results = WorkerResults(count: workers) + let deadline = ContinuousClock.now + Self.stretch + + var worker = 0 + for (count, budget) in zip(groups, budgets) { + for _ in 0 ..< count { + let number = worker + WorkerPool.shared.run(on: number) { + let share = WorkShare(of: budget, until: deadline) + parked.signal() + start.wait() + let end = work(fixture, number, share) + results.record( + end: end, + steps: share.steps, + turns: share.turns, + for: number + ) + finished.signal() + } + worker += 1 + } + } + + for _ in 0 ..< workers { + parked.wait() + } + for _ in 0 ..< workers { + start.signal() + } + for _ in 0 ..< workers { + finished.wait() + } + + for worker in 0 ..< workers { + turnsTaken += results.result(of: worker).turns + } + stretches.append(results) + } while turnsTaken < turnsPerSample + stopMeasurement() + + for results in stretches { + for worker in 0 ..< workers { + let (end, steps, _) = results.result(of: worker) + precondition( + end == Chase.end(from: worker, steps: steps), + "worker \(worker)'s chase did not end where its steps say" + ) + } + } + precondition(turnsTaken == turnsPerSample, "the budgets were not spent") + check(fixture, totals) + } + + /// The longest the workers run between pauses. + private static let stretch: Duration = .milliseconds(100) + + /// `measureContention(groups:)` for the common case of one group. + func measureContention( + workers: Int, + makeFixture: () -> Fixture, + work: @escaping @Sendable (Fixture, _ worker: Int, _ share: WorkShare) -> Int, + check: (Fixture) -> Void = { _ in } + ) { + measureContention(groups: [workers], makeFixture: makeFixture, work: work) { fixture, _ in + check(fixture) + } + } + + /// The sample's turns shared out between `groups` in proportion to their + /// worker counts, the remainder going to the last. + private func budgetTotals(for groups: [Int]) -> [Int] { + let workers = groups.reduce(0, +) + var budgets = groups.map { turnsPerSample * $0 / workers } + budgets[budgets.count - 1] += turnsPerSample - budgets.reduce(0, +) + return budgets + } + + /// Runs `tasks` tasks over a fresh fixture, for the asynchronous + /// primitives. + /// + /// Task creation is measured with the rest: there is no cheap way to + /// park a crowd of tasks and release them together, and a task costs + /// microseconds against the thousands of handoffs each one then makes. + /// Detached at `.userInitiated` for the same reason the threads are + /// pinned. + /// + /// `work` chases the cycle once per iteration from its task's number + /// and returns where it ended, and the harness checks each task against + /// where `iterations` steps should have ended — or where `steps` says, + /// for a case in which some tasks chase less. `check` sees the fixture + /// afterwards. + /// + /// Read the single-task numbers with the executor in mind. A turn here + /// is a take, a yield, and a release, and on Linux the yield costs + /// according to how much the task did before it: a few hundred + /// nanoseconds after almost nothing, several microseconds after half a + /// microsecond of work, as the pool hands the resumed task to another + /// thread. Measured directly, with nothing but an atomic spin before + /// the yield, so it is the executor's and not a lock's; on macOS the + /// cost stays flat until the work runs to microseconds. A primitive + /// whose take and release together cross that line therefore reads as + /// several times another's there while differing by a fraction, so + /// compare Linux numbers between primitives only at like per-turn + /// work, and against macOS not at all. + func measureTaskContention( + tasks: Int, + iterations: Int, + makeFixture: () -> Fixture, + steps: (_ task: Int) -> Int? = { _ in nil }, + work: @escaping @Sendable (Fixture, _ task: Int) async throws -> Int, + check: (Fixture) -> Void = { _ in } + ) { + let fixture = makeFixture() + let finished = DispatchSemaphore(value: 0) + let results = WorkerResults(count: tasks) + let failed = Tally() + + startMeasurement() + for task in 0 ..< tasks { + Task.detached(priority: .userInitiated) { + do { + let end = try await work(fixture, task) + results.record(end: end, steps: 0, turns: 0, for: task) + } catch { + failed.add(1) + } + finished.signal() + } + } + for _ in 0 ..< tasks { + finished.wait() + } + stopMeasurement() + + precondition(failed.value == 0, "a task threw") + for task in 0 ..< tasks { + let expected = Chase.end(from: task, steps: steps(task) ?? iterations) + precondition( + results.result(of: task).end == expected, + "task \(task)'s chase did not advance" + ) + } + check(fixture) + } +} + +// MARK: - A lock taken around one step + +/// A lock the harness can measure as a plain lock: one write and one step of +/// the chase under it. One protocol so that one set of cases serves every +/// implementation — this package's `Mutex` and the standard library's, a +/// semaphore with a count of one, `DispatchSemaphore` — and every adopter is +/// final, so the cases are specialized for each and no measured turn is +/// dispatched through it. +protocol MeasuredLock: Sendable { + init() + + /// One write and one step of the chase from `index`, under the lock. + func step(from index: Int) -> Int + + /// How many writes have happened, read under the lock. + var writes: Int { get } +} + +/// Registers the uncontended case for `Lock`, and the contended ones if +/// `oversubscribed` lists any, under `name` — "Mutex contended", say — with +/// `implementation` appended where the lock is a comparison rather than the +/// primitive itself. +func registerLockCases( + _: Lock.Type, + named name: String, + implementation: String? = nil, + contended: [(scenario: String, workers: Int)] +) { + let suffix = implementation.map { ", \($0)" } ?? "" + + Benchmark("\(name) uncontended\(suffix)", configuration: .uncontended) { benchmark in + benchmark.measureUncontended(makeFixture: Lock.init) { lock, turns in + var index = 0 + for _ in turns { + index = lock.step(from: index) + } + return index + } check: { lock in + precondition(lock.writes == benchmark.turnsPerSample, "the workload did not run") + } + } + + for (scenario, workers) in contended { + Benchmark("\(name) \(scenario)\(suffix)", configuration: .contended) { benchmark in + benchmark.measureContention( + workers: workers, + makeFixture: Lock.init + ) { lock, worker, share in + var index = worker + share.eachTurn { + index = lock.step(from: index) + } + return index + } check: { lock in + precondition(lock.writes == benchmark.turnsPerSample, "the workload did not run") + } + } + } +} + +// MARK: - Helpers the asynchronous cases share + +/// A one-shot signal: `wait()` suspends until `open()` has been called, and +/// every task waiting at that moment resumes. +/// +/// Built on the package's own `Mutex` and a list of continuations rather +/// than an `AsyncStream`, which is single-consumer by contract and shares +/// only by accident on older runtimes. +final class Gate: Sendable { + private struct State { + var isOpen = false + var waiters: [CheckedContinuation] = [] + } + + private let state = Mutex(State()) + + init() {} + + /// Opens the gate and resumes every waiter. Idempotent: a second call + /// finds nobody waiting and changes nothing. + func open() { + let waiters = state.withLock { state in + state.isOpen = true + let waiters = state.waiters + state.waiters.removeAll() + return waiters + } + for waiter in waiters { + waiter.resume() + } + } + + /// Suspends until the gate is open. Returns at once if it already is. + func wait() async { + await withCheckedContinuation { continuation in + let isOpen = state.withLock { state in + if !state.isOpen { + state.waiters.append(continuation) + } + return state.isOpen + } + if isOpen { + continuation.resume() + } + } + } +} + +/// A seeded generator, so that a sample shuffles the same way every time. +/// +/// SplitMix64: a 64-bit state, one addition, three xor-shift-multiply +/// rounds. Not a quality the benchmarks depend on, only a sequence they can +/// repeat. +struct SplitMix64: RandomNumberGenerator { + private var state: UInt64 + + init(seed: UInt64) { + state = seed + } + + mutating func next() -> UInt64 { + state &+= 0x9E37_79B9_7F4A_7C15 + var z = state + z = (z ^ (z >> 30)) &* 0xBF58_476D_1CE4_E5B9 + z = (z ^ (z >> 27)) &* 0x94D0_49BB_1331_11EB + return z ^ (z >> 31) + } +} diff --git a/Benchmarks/Benchmarks/SynchronizationKitBenchmarks/MutexBenchmarks.swift b/Benchmarks/Benchmarks/SynchronizationKitBenchmarks/MutexBenchmarks.swift new file mode 100644 index 0000000..5b27016 --- /dev/null +++ b/Benchmarks/Benchmarks/SynchronizationKitBenchmarks/MutexBenchmarks.swift @@ -0,0 +1,89 @@ +// +// MutexBenchmarks.swift +// SynchronizationKitBenchmarks +// +// Copyright (c) 2026 Kang Jaehong +// SPDX-License-Identifier: Apache-2.0 +// + +import Benchmark +import SynchronizationKit + +#if canImport(Darwin) +import Synchronization +#endif + +// What `Mutex` costs — and on Apple platforms, where it is this package's +// own, what it costs against the standard library's, which is what a client +// on a new enough OS would otherwise use. +// +// On Apple platforms each case runs twice, once per implementation, so the +// two land in one report: the package's is an `os_unfair_lock` reached +// through a raw-layout struct, the standard library's the same lock reached +// through its own, and the numbers say what the reaching costs. Elsewhere +// the package's `Mutex` is the standard library's, re-exported, so the +// comparison would be of a thing with itself and only the first half runs — +// as the cost of the lock every asynchronous primitive here keeps its state +// under. + +/// A reference to hold the lock by. +/// +/// Capturing a noncopyable value in a measured closure currently fails to +/// compile — "copy of noncopyable typed value", reported by the compiler as +/// its own bug — so the closure captures this instead and reaches the lock +/// through it. +final class MutexBox: MeasuredLock, @unchecked Sendable { + let lock = SynchronizationKit.Mutex(ChasePayload()) + + func step(from index: Int) -> Int { + lock.withLock { + $0.writes &+= 1 + return $0.cycle[index] + } + } + + var writes: Int { + lock.withLock { $0.writes } + } +} + +#if canImport(Darwin) +@available(macOS 15, *) +final class StandardMutexBox: MeasuredLock, @unchecked Sendable { + let lock = Synchronization.Mutex(ChasePayload()) + + func step(from index: Int) -> Int { + lock.withLock { + $0.writes &+= 1 + return $0.cycle[index] + } + } + + var writes: Int { + lock.withLock { $0.writes } + } +} +#endif + +/// Every core but two, so the handoff is between threads that each have +/// somewhere to run; and twice the cores, so most of the time the lock is +/// handed to a thread that has to be woken for it. +private let contendedScenarios = [ + (scenario: "contended", workers: contendedWorkers), + (scenario: "oversubscribed", workers: oversubscribedWorkers), +] + +func registerMutexBenchmarks() { + registerLockCases(MutexBox.self, named: "Mutex", contended: contendedScenarios) + + #if canImport(Darwin) + if #available(macOS 15, *) { + registerLockCases( + StandardMutexBox.self, + named: "Mutex", + implementation: "standard library", + contended: contendedScenarios + ) + } + #endif +} diff --git a/Benchmarks/Benchmarks/SynchronizationKitBenchmarks/RWLockBenchmarks.swift b/Benchmarks/Benchmarks/SynchronizationKitBenchmarks/RWLockBenchmarks.swift new file mode 100644 index 0000000..2bc68a4 --- /dev/null +++ b/Benchmarks/Benchmarks/SynchronizationKitBenchmarks/RWLockBenchmarks.swift @@ -0,0 +1,305 @@ +// +// RWLockBenchmarks.swift +// SynchronizationKitBenchmarks +// +// Copyright (c) 2026 Kang Jaehong +// SPDX-License-Identifier: Apache-2.0 +// + +import Benchmark +import Dispatch +import SynchronizationKit + +#if canImport(Darwin) +import Darwin +#elseif canImport(Glibc) +import Glibc +#elseif canImport(Musl) +import Musl +#endif + +// What `RWLock` costs, on whichever backend the running OS provides — and +// what the three things a client would otherwise write cost on the same +// turns: the platform's `pthread_rwlock_t`; a concurrent `DispatchQueue`, +// reads in `sync` and writes behind a `.barrier`; and this package's +// `Mutex`, which the README says to prefer until reads are frequent, writes +// rare, and the read section long enough for parallel reading to pay. Each +// case runs once per implementation, so the four land in one report. +// +// A critical section is one step of the chase, except in the `section` +// cases, which run the read-mostly mix again with every section 64, 256, +// and 1024 steps long — from about a dictionary lookup to about a +// microsecond of work. That is where a reader-writer lock is meant to pull +// ahead of a mutex, and the series says at what length this one does. +// +// On glibc the package's `RWLock` is itself a `pthread_rwlock_t`, +// configured writer-preferring, so the `pthread_rwlock_t` cases there +// measure the default configuration against it rather than a different +// lock. + +/// A lock the cases below can measure as a reader-writer lock: a chase +/// under the read lock, and a chase and a write under the write lock, each +/// returning where the chase ended. One protocol so that one set of cases +/// serves every implementation; every adopter is final, so the cases are +/// specialized for each and no measured turn is dispatched through it. +private protocol MeasuredReadWriteLock: Sendable { + init() + + /// `steps` steps of the chase from `index` under the read lock. + func read(from index: Int, steps: Int) -> Int + + /// One write and `steps` steps of the chase from `index` under the write + /// lock. + func write(from index: Int, steps: Int) -> Int + + /// How many writes have happened, read under the lock. + var writes: Int { get } +} + +/// A reference to hold the lock by; `MutexBox` says why. +private final class RWLockBox: MeasuredReadWriteLock, @unchecked Sendable { + let lock = RWLock(ChasePayload()) + + func read(from index: Int, steps: Int) -> Int { + lock.withReadLock { $0.chase(from: index, steps: steps) } + } + + func write(from index: Int, steps: Int) -> Int { + lock.withWriteLock { + $0.writes &+= 1 + return $0.chase(from: index, steps: steps) + } + } + + var writes: Int { + lock.withReadLock { $0.writes } + } +} + +/// The platform's reader-writer lock, in its default configuration, and +/// what it guards. Allocated rather than stored inline: a +/// `pthread_rwlock_t` must not move once initialized, and a pointer is the +/// plain way to promise that. `@safe`, with the pointer marked `@unsafe`: +/// every access to it is spelled out below, and it is what the lock guards, +/// so nothing else reaches it. +@safe +private final class PthreadLockBox: MeasuredReadWriteLock, @unchecked Sendable { + @unsafe private let lock: UnsafeMutablePointer + private var payload = ChasePayload() + + init() { + unsafe lock = UnsafeMutablePointer.allocate(capacity: 1) + unsafe lock.initialize(to: pthread_rwlock_t()) + let result = unsafe pthread_rwlock_init(lock, nil) + precondition(result == 0, "pthread_rwlock_init failed") + } + + deinit { + unsafe pthread_rwlock_destroy(lock) + unsafe lock.deinitialize(count: 1) + unsafe lock.deallocate() + } + + func read(from index: Int, steps: Int) -> Int { + var result = unsafe pthread_rwlock_rdlock(lock) + precondition(result == 0, "pthread_rwlock_rdlock failed") + let end = payload.chase(from: index, steps: steps) + result = unsafe pthread_rwlock_unlock(lock) + precondition(result == 0, "pthread_rwlock_unlock failed") + return end + } + + func write(from index: Int, steps: Int) -> Int { + var result = unsafe pthread_rwlock_wrlock(lock) + precondition(result == 0, "pthread_rwlock_wrlock failed") + payload.writes &+= 1 + let end = payload.chase(from: index, steps: steps) + result = unsafe pthread_rwlock_unlock(lock) + precondition(result == 0, "pthread_rwlock_unlock failed") + return end + } + + var writes: Int { + var result = unsafe pthread_rwlock_rdlock(lock) + precondition(result == 0, "pthread_rwlock_rdlock failed") + let writes = payload.writes + result = unsafe pthread_rwlock_unlock(lock) + precondition(result == 0, "pthread_rwlock_unlock failed") + return writes + } +} + +/// The reader-writer pattern Dispatch offers: a concurrent queue, reads +/// submitted with `sync`, writes with `sync` and the `.barrier` flag. +private final class QueueLockBox: MeasuredReadWriteLock, @unchecked Sendable { + private let queue = DispatchQueue( + label: "SynchronizationKitBenchmarks.QueueLockBox", + attributes: .concurrent + ) + private var payload = ChasePayload() + + func read(from index: Int, steps: Int) -> Int { + queue.sync { payload.chase(from: index, steps: steps) } + } + + func write(from index: Int, steps: Int) -> Int { + queue.sync(flags: .barrier) { + payload.writes &+= 1 + return payload.chase(from: index, steps: steps) + } + } + + var writes: Int { + queue.sync { payload.writes } + } +} + +/// This package's `Mutex`, taken for reads and writes alike. +private final class MutexLockBox: MeasuredReadWriteLock, @unchecked Sendable { + let lock = Mutex(ChasePayload()) + + func read(from index: Int, steps: Int) -> Int { + lock.withLock { $0.chase(from: index, steps: steps) } + } + + func write(from index: Int, steps: Int) -> Int { + lock.withLock { + $0.writes &+= 1 + return $0.chase(from: index, steps: steps) + } + } + + var writes: Int { + lock.withLock { $0.writes } + } +} + +/// How many turns of a read-mostly worker are reads for each write. +private let readsPerWrite = 99 + +/// A lock and the count of writes its workers made, which is what the count +/// of writes it saw has to come to. +private final class ReadMostlyFixture: Sendable { + let lock = Lock() + let expectedWrites = Atomic(0) +} + +/// Registers every case for `Lock` under "RWLock", with `implementation` +/// appended where the lock is a comparison rather than the primitive +/// itself. +private func registerReadWriteLockCases( + _: Lock.Type, + implementation: String? = nil +) { + let suffix = implementation.map { ", \($0)" } ?? "" + + /// Runs one reader and writer mix over a fresh lock, every section + /// `steps` long. Writers come first in the worker numbering. + func registerContention(_ scenario: String, readers: Int, writers: Int, steps: Int = 1) { + Benchmark("RWLock \(scenario)\(suffix)", configuration: .contended) { benchmark in + benchmark.measureContention( + groups: [writers, readers], + makeFixture: Lock.init + ) { lock, worker, share in + var index = worker + if worker < writers { + share.eachTurn(steps: steps) { + index = lock.write(from: index, steps: steps) + } + } else { + share.eachTurn(steps: steps) { + index = lock.read(from: index, steps: steps) + } + } + return index + } check: { lock, budgets in + precondition(lock.writes == budgets[0], "the write side did not run") + } + } + } + + func registerUncontended(_ scenario: String, writing: Bool) { + Benchmark("RWLock \(scenario)\(suffix)", configuration: .uncontended) { benchmark in + benchmark.measureUncontended(makeFixture: Lock.init) { lock, turns in + var index = 0 + if writing { + for _ in turns { + index = lock.write(from: index, steps: 1) + } + } else { + for _ in turns { + index = lock.read(from: index, steps: 1) + } + } + return index + } check: { lock in + precondition( + lock.writes == (writing ? benchmark.turnsPerSample : 0), + "the workload did not run" + ) + } + } + } + + /// Runs `contendedWorkers` workers over a fresh lock, each writing once + /// in every `readsPerWrite + 1` turns and reading otherwise, every + /// section `steps` long. The writes are spread across the workers and + /// through the run rather than given to one worker to make flat out, so + /// no writer is forever queued behind the readers. + func registerReadMostly(_ scenario: String, steps: Int = 1) { + Benchmark("RWLock \(scenario)\(suffix)", configuration: .contended) { benchmark in + benchmark.measureContention( + workers: contendedWorkers, + makeFixture: ReadMostlyFixture.init + ) { fixture, worker, share in + let lock = fixture.lock + var index = worker + var turn = 0 + var writes = 0 + share.eachTurn(steps: steps) { + turn += 1 + if turn % (readsPerWrite + 1) == 0 { + index = lock.write(from: index, steps: steps) + writes += 1 + } else { + index = lock.read(from: index, steps: steps) + } + } + fixture.expectedWrites.wrappingAdd(writes, ordering: .relaxed) + return index + } check: { fixture in + precondition( + fixture.lock.writes == fixture.expectedWrites.load(ordering: .relaxed), + "the write side did not run" + ) + } + } + } + + registerUncontended("uncontended reads", writing: false) + registerUncontended("uncontended writes", writing: true) + + // Every worker reads and nobody writes: what a read costs when the lock + // is doing the one thing a reader-writer lock is for. The section is one + // step, so the number is the lock and nothing else. + registerContention("concurrent reads", readers: contendedWorkers, writers: 0) + + registerReadMostly("read-mostly") + for steps in [64, 256, 1024] { + registerReadMostly("read-mostly, section \(steps)", steps: steps) + } + + // Deliberately more readers than cores, so a departing writer has a + // queue to hand the lock to. This is where releasing every one of them + // at once tells against one system call each. + registerContention("writer handoff", readers: oversubscribedWorkers, writers: 1) + + registerContention("mixed contention", readers: contendedWorkers, writers: 4) +} + +func registerRWLockBenchmarks() { + registerReadWriteLockCases(RWLockBox.self) + registerReadWriteLockCases(PthreadLockBox.self, implementation: "pthread_rwlock_t") + registerReadWriteLockCases(QueueLockBox.self, implementation: "DispatchQueue") + registerReadWriteLockCases(MutexLockBox.self, implementation: "Mutex") +} diff --git a/Benchmarks/Benchmarks/SynchronizationKitBenchmarks/SemaphoreBenchmarks.swift b/Benchmarks/Benchmarks/SynchronizationKitBenchmarks/SemaphoreBenchmarks.swift new file mode 100644 index 0000000..31842c5 --- /dev/null +++ b/Benchmarks/Benchmarks/SynchronizationKitBenchmarks/SemaphoreBenchmarks.swift @@ -0,0 +1,78 @@ +// +// SemaphoreBenchmarks.swift +// SynchronizationKitBenchmarks +// +// Copyright (c) 2026 Kang Jaehong +// SPDX-License-Identifier: Apache-2.0 +// + +import Benchmark +import Dispatch +import SynchronizationKit + +// What `Semaphore` costs against `DispatchSemaphore`, which is what a client +// would otherwise reach for, on whichever backend the running OS provides. +// +// The semaphore is used as a lock — a count of one, taken around the chase — +// so that the same cases apply and the numbers sit beside the locks'. Each +// case runs twice, once per implementation. +// +// A semaphore has no fast path past the kernel once anybody is waiting, so +// the contended case is the handoff itself: every release wakes somebody. + +/// The semaphore and what it guards, held by reference; `MutexBox` says why. +/// The payload is guarded by the semaphore, which is what makes the +/// unchecked conformance right. +final class SemaphoreBox: MeasuredLock, @unchecked Sendable { + let semaphore = Semaphore(value: 1) + private var payload = ChasePayload() + + func step(from index: Int) -> Int { + semaphore.wait() + payload.writes &+= 1 + let end = payload.cycle[index] + semaphore.signal() + return end + } + + var writes: Int { + semaphore.wait() + let writes = payload.writes + semaphore.signal() + return writes + } +} + +final class DispatchSemaphoreBox: MeasuredLock, @unchecked Sendable { + let semaphore = DispatchSemaphore(value: 1) + private var payload = ChasePayload() + + func step(from index: Int) -> Int { + semaphore.wait() + payload.writes &+= 1 + let end = payload.cycle[index] + semaphore.signal() + return end + } + + var writes: Int { + semaphore.wait() + let writes = payload.writes + semaphore.signal() + return writes + } +} + +private let contendedScenarios = [ + (scenario: "contended", workers: contendedWorkers), +] + +func registerSemaphoreBenchmarks() { + registerLockCases(SemaphoreBox.self, named: "Semaphore", contended: contendedScenarios) + registerLockCases( + DispatchSemaphoreBox.self, + named: "Semaphore", + implementation: "DispatchSemaphore", + contended: contendedScenarios + ) +} diff --git a/Benchmarks/Package.resolved b/Benchmarks/Package.resolved new file mode 100644 index 0000000..b4f5839 --- /dev/null +++ b/Benchmarks/Package.resolved @@ -0,0 +1,78 @@ +{ + "originHash" : "e233765cfa91948110910a65b3a72a7923fadda5bead5c9a4ccc431eb307db97", + "pins" : [ + { + "identity" : "benchmark", + "kind" : "remoteSourceControl", + "location" : "https://github.com/ordo-one/benchmark.git", + "state" : { + "revision" : "1d1b7a56ade88f69c0cef1fa423b320059315f1a", + "version" : "1.37.0" + } + }, + { + "identity" : "hdrhistogram-swift", + "kind" : "remoteSourceControl", + "location" : "https://github.com/HdrHistogram/hdrhistogram-swift.git", + "state" : { + "revision" : "c2e1210df04b4fff47e53f2f9dad9cc45ae15d63", + "version" : "0.2.0" + } + }, + { + "identity" : "malloc-interposer", + "kind" : "remoteSourceControl", + "location" : "https://github.com/ordo-one/malloc-interposer.git", + "state" : { + "revision" : "e4385a5c5c344c5de84bf84a78f2a359801fa585", + "version" : "1.5.0" + } + }, + { + "identity" : "swift-argument-parser", + "kind" : "remoteSourceControl", + "location" : "https://github.com/apple/swift-argument-parser.git", + "state" : { + "revision" : "6a52f3251125d74daf04fcbd5e6f08a75d074382", + "version" : "1.8.2" + } + }, + { + "identity" : "swift-atomics", + "kind" : "remoteSourceControl", + "location" : "https://github.com/apple/swift-atomics.git", + "state" : { + "revision" : "0442cb5a3f98ab802acb777929fdb446bda11a34", + "version" : "1.3.1" + } + }, + { + "identity" : "swift-numerics", + "kind" : "remoteSourceControl", + "location" : "https://github.com/apple/swift-numerics", + "state" : { + "revision" : "0c0290ff6b24942dadb83a929ffaaa1481df04a2", + "version" : "1.1.1" + } + }, + { + "identity" : "swift-system", + "kind" : "remoteSourceControl", + "location" : "https://github.com/apple/swift-system.git", + "state" : { + "revision" : "869129b7bf4ecc57b97d0193ad29690ca2134750", + "version" : "1.8.1" + } + }, + { + "identity" : "texttable", + "kind" : "remoteSourceControl", + "location" : "https://github.com/ordo-one/TextTable.git", + "state" : { + "revision" : "a27a07300cf4ae322e0079ca0a475c5583dd575f", + "version" : "0.0.2" + } + } + ], + "version" : 3 +} diff --git a/Benchmarks/Package.swift b/Benchmarks/Package.swift new file mode 100644 index 0000000..9c4375a --- /dev/null +++ b/Benchmarks/Package.swift @@ -0,0 +1,48 @@ +// swift-tools-version: 6.3 +// +// Package.swift +// SynchronizationKitBenchmarks +// +// Copyright (c) 2026 Kang Jaehong +// SPDX-License-Identifier: Apache-2.0 +// + +import PackageDescription + +// A package of its own rather than a target of the one above it, so that +// nothing a client resolves ever includes the benchmark harness: the manifest +// they read has no dependencies, and this one is only ever opened from inside +// this directory. +let package = Package( + name: "Benchmarks", + // The harness's floor, not the library's. The library deploys further + // back than this and is not held to it by being measured here. + platforms: [ + .macOS(.v13), + ], + dependencies: [ + // The library with its default traits, which is every primitive: the + // benchmarks measure them all. + .package(path: "../"), + .package(url: "https://github.com/ordo-one/benchmark.git", from: "1.36.4"), + ], + targets: [ + .executableTarget( + name: "SynchronizationKitBenchmarks", + dependencies: [ + .product(name: "Benchmark", package: "benchmark"), + .product(name: "SynchronizationKit", package: "swift-synchronization-kit"), + ], + path: "Benchmarks/SynchronizationKitBenchmarks", + swiftSettings: [ + .enableUpcomingFeature("ApproachableConcurrency"), + .enableUpcomingFeature("InternalImportsByDefault"), + .enableUpcomingFeature("MemberImportVisibility"), + .strictMemorySafety(), + ], + plugins: [ + .plugin(name: "BenchmarkPlugin", package: "benchmark"), + ] + ), + ] +) From 1e8e6ff06c0928f6609c714d2b4dbbf080a47bbb Mon Sep 17 00:00:00 2001 From: Kang Jaehong Date: Sat, 10 Oct 2026 06:02:55 +0900 Subject: [PATCH 2/3] Take the measurements out of the test suites The XCTest performance suites, and the harness they measured through, go: the benchmark package measures the same cases now. What they leave behind in the test utilities goes with them, the Atomic dependency the harness's counter needed and the AddressSanitizer flag it skipped on. No XCTest case is left in the package. CI follows. The release run is the stress suites alone, on both workflows, and the tests build and run without XCTest everywhere, which the WebAssembly row alone did before: the corelibs runner traps on an empty list, and Android's runner loop takes Swift Testing only. The benchmarks run on one macOS and one Linux row per Swift minor, the platforms the harness supports, and put their table in the job's summary without failing on a number; their package is in the path filter, and the job's timeout makes room for them. Two rows stay marked experimental for failures that showed in the measurements: the arm64 Windows actor increment (#6) and the arm64 Linux dispatch crash (#11). Neither row runs the measurements any more, so each is kept marked until a run without them shows it clear. --- .github/workflows/apple-platforms.yml | 12 +- .github/workflows/swift.yml | 153 +++-- Package.swift | 8 +- .../Measurement.swift | 570 ----------------- .../TestEnvironment.swift | 3 - .../AsyncMutexPerformanceTests.swift | 209 ------- .../AsyncRWLockPerformanceTests.swift | 112 ---- .../AsyncSemaphorePerformanceTests.swift | 246 -------- .../MutexPerformanceTests.swift | 174 ------ .../RWLockPerformanceTests.swift | 576 ------------------ .../SemaphorePerformanceTests.swift | 117 ---- 11 files changed, 97 insertions(+), 2083 deletions(-) delete mode 100644 Sources/SynchronizationKitTestUtils/Measurement.swift delete mode 100644 Tests/SynchronizationKitAsyncMutexTests/AsyncMutexPerformanceTests.swift delete mode 100644 Tests/SynchronizationKitAsyncRWLockTests/AsyncRWLockPerformanceTests.swift delete mode 100644 Tests/SynchronizationKitAsyncSemaphoreTests/AsyncSemaphorePerformanceTests.swift delete mode 100644 Tests/SynchronizationKitMutexTests/MutexPerformanceTests.swift delete mode 100644 Tests/SynchronizationKitRWLockTests/RWLockPerformanceTests.swift delete mode 100644 Tests/SynchronizationKitSemaphoreTests/SemaphorePerformanceTests.swift diff --git a/.github/workflows/apple-platforms.yml b/.github/workflows/apple-platforms.yml index ee90941..53252f8 100644 --- a/.github/workflows/apple-platforms.yml +++ b/.github/workflows/apple-platforms.yml @@ -49,7 +49,7 @@ jobs: # names of the suites' types; the steps below say how it is used here. env: DEVELOPER_DIR: ${{ matrix.xcode.developer_dir }} - SUITES: PerformanceTests|StressTests + SUITES: StressTests strategy: fail-fast: false # Two axes, crossed. The Xcode axis is the one swift.yml already runs: the @@ -267,11 +267,11 @@ jobs: # # Both configurations in turn, as swift.yml does and for its reasons: # release is the library as a client gets it, where the optimizer's bugs - # and the measurements are, and it is built without testability so that - # what is stressed and measured is what ships; debug is where `@testable - # import` works unasked, and where the unit tests that need it are - # compiled at all. Release matters most on the watchOS device row, whose - # 32-bit pointer branch is compiled nowhere else. Both name the same + # are, and it is built without testability so that what is stressed is + # what ships; debug is where `@testable import` works unasked, and where + # the unit tests that need it are compiled at all. Release matters most + # on the watchOS device row, whose 32-bit pointer branch is compiled + # nowhere else. Both name the same # derived data, where Xcode keeps each configuration's products apart, # and the run that follows reuses what was built here instead of # compiling the package again. diff --git a/.github/workflows/swift.yml b/.github/workflows/swift.yml index 7a45049..8808954 100644 --- a/.github/workflows/swift.yml +++ b/.github/workflows/swift.yml @@ -10,15 +10,16 @@ name: Swift # would only run the same commit twice. # # And only for a change that can alter what is built here: the sources, the -# tests, the manifest and what it resolved to, and this file. A commit to the -# README or the changelog runs nothing. A branch filter and a path filter must -# both be satisfied. +# tests, the benchmarks, the manifest and what it resolved to, and this file. +# A commit to the README or the changelog runs nothing. A branch filter and a +# path filter must both be satisfied. on: push: branches: [main, develop, "release/**", "hotfix/**"] paths: &paths - Sources/** - Tests/** + - Benchmarks/** - Package.swift - Package.resolved - .github/workflows/swift.yml @@ -60,8 +61,9 @@ jobs: # A dropped wake in a lock does not fail a test, it stops one, and without # this the only backstop is GitHub's 360-minute default — six hours this # repository has already paid once. The joins inside the suite are bounded - # by hand for the same reason; this covers whatever they cannot. - timeout-minutes: 30 + # by hand for the same reason; this covers whatever they cannot, with room + # for the benchmarks on the rows that run them. + timeout-minutes: 45 # All are inert where the matrix leaves them unset. DEVELOPER_DIR selects # the toolchain on macOS by pointing at a pinned Xcode: the runner keeps a # minor-level symlink at the current patch (Xcode_26.6.app -> @@ -86,7 +88,7 @@ jobs: TEST_OPTIONS: ${{ matrix.test_options }} SHORT_TESTS: ${{ matrix.short_tests }} ANDROID: ${{ matrix.android }} - SUITES: PerformanceTests|StressTests + SUITES: StressTests strategy: fail-fast: false # Two axes, crossed: every target under every Swift minor, less the pairs @@ -150,9 +152,12 @@ jobs: macos: macOS 27 developer_dir: /Applications/Xcode_27.0.app/Contents/Developer - # macOS has no entry of its own: its runner and its Xcode are its - # minor's. The Intel image is the one place the suite runs on x86_64 - # Darwin, a slice that is otherwise compiled and never run. + # macOS has an entry only to say it runs the benchmarks: its runner + # and its Xcode are its minor's. The Intel image is the one place + # the suite runs on x86_64 Darwin, a slice that is otherwise + # compiled and never run. + - target: macOS + benchmarks: true - target: macOS Intel os: macOS 26 variant: Intel @@ -168,6 +173,7 @@ jobs: - target: Linux os: Ubuntu 24.04 runner: ubuntu-24.04 + benchmarks: true - target: Linux arm64 os: Ubuntu 24.04 variant: arm64 @@ -217,14 +223,12 @@ jobs: # and the single-threaded parts of the rest. The threaded flavor is # in development snapshots only, and is not built here. # - # Two departures from the other targets. XCTest is off: with the - # measurement suites compiled out there is no XCTest case left, and - # the corelibs runner traps on an empty list rather than reporting - # none. And the stress suites run at their short scale: the Wasm - # SDK emits no tail calls, so an `await` that never actually - # suspends nests a frame instead of unwinding one, and the long - # scale's thousands of them in a row exhaust the interpreter's call - # stack, which `swift test` has no way to raise. + # One departure from the other targets: the stress suites run at + # their short scale. The Wasm SDK emits no tail calls, so an + # `await` that never actually suspends nests a frame instead of + # unwinding one, and the long scale's thousands of them in a row + # exhaust the interpreter's call stack, which `swift test` has no + # way to raise. - target: WebAssembly wasip1 os: WebAssembly variant: wasip1 @@ -232,7 +236,6 @@ jobs: pinned: true swift_sdk: swift-{0}-RELEASE_wasm sdk_url: https://download.swift.org/swift-{0}-release/wasm-sdk/swift-{0}-RELEASE/swift-{0}-RELEASE_wasm.artifactbundle.tar.gz - test_options: --disable-xctest short_tests: true # Cross-compiles to Android through the Swift SDK for it, and runs the # tests on an emulator. This is the only target where the bionic side @@ -370,7 +373,9 @@ jobs: # Windows on arm64 loses an increment inside an `actor`, the # measurements' comparison and none of this package's code: 127999 # of 128000, always short by one. Under 6.3 every time so far, under - # 6.4 one time in four. (#6) + # 6.4 one time in four. The measurements have since moved to the + # benchmark package, which does not run on Windows, so the pair is + # kept marked until a run without them shows it clear. (#6) - { swift: "6.3", target: Windows arm64, experimental: true } - { swift: "6.4", target: Windows arm64, experimental: true } # The 6.4 toolchain's WasmKit implements `poll_oneoff`, so a sleep @@ -387,8 +392,10 @@ jobs: # asynchronous measurements: about one run in two. A program of an # actor, detached tasks and `Task.yield`, with none of this # package in it, does the same on this runner once in 130 runs and - # on x86_64 never. The sanitized arm64 rows skip the measurements - # and are not marked. (#11) + # on x86_64 never. The measurements have since moved to the + # benchmark package, which this row does not run, so the pair is + # kept marked until a run without them shows it clear; the + # sanitized arm64 rows never ran them and are not marked. (#11) - { swift: "6.3", target: Linux arm64, experimental: true } - { swift: "6.4", target: Linux arm64, experimental: true } @@ -525,11 +532,10 @@ jobs: # Release is the library as a client gets it, and a lock is a type whose # bugs the optimizer is entitled to create: reordering, dead stores, # accesses folded into a register. A debug run cannot see any of them, so - # the stress suites run in release, and the measurements with them, which - # say nothing unoptimized. Neither needs more than the public interface, - # and so nothing in release is compiled with `-enable-testing`, which - # publishes internal symbols and costs the optimizer some of what it may - # assume: what is stressed and measured is what ships. + # the stress suites run in release. They need no more than the public + # interface, and so nothing in release is compiled with + # `-enable-testing`, which publishes internal symbols and costs the + # optimizer some of what it may assume: what is stressed is what ships. # # Debug is where `@testable import` works without being asked for, so the # unit tests run there. The files that reach internal declarations are @@ -567,21 +573,24 @@ jobs: # can only run where a full toolchain exists. # # The stress suites run at length on every push, on every row but the - # ones that say `short_tests`: the flag turns their - # repetition up fiftyfold, a tenth of that under the sanitizer, and - # what that costs — some ten seconds on a runner — is less than the - # measurements take, while what it finds, a wake lost in one - # interleaving out of many, is what a change to a wait or wake path - # most needs found before it merges. `stressScale` in the test utils - # says how the flag is read, and why the factor is not higher. Only the - # release build takes it, being the only one that runs them. + # ones that say `short_tests`: the flag turns their repetition up + # fiftyfold, a tenth of that under the sanitizer, and what that costs — + # some ten seconds on a runner — is small beside what it finds, a wake + # lost in one interleaving out of many, which is what a change to a wait + # or wake path most needs found before it merges. `stressScale` in the + # test utils says how the flag is read, and why the factor is not + # higher. Only the release build takes it, being the only one that runs + # them. + # + # XCTest is left out of both builds: no XCTest case is left in the + # package, and the step that runs the tests says why that matters. - name: swift build --build-tests if: ${{ !matrix.skip_tests }} shell: bash run: | for configuration in debug release; do echo "::group::$configuration" - test_flags="-c $configuration${SWIFT_SDK:+ --swift-sdk $SWIFT_SDK}${SANITIZE:+ --sanitize $SANITIZE} $TEST_OPTIONS" + test_flags="-c $configuration --disable-xctest${SWIFT_SDK:+ --swift-sdk $SWIFT_SDK}${SANITIZE:+ --sanitize $SANITIZE} $TEST_OPTIONS" if [ "$configuration" = release ] && [ -z "$SHORT_TESTS" ]; then test_flags="$test_flags -Xswiftc -DSYNCHRONIZATIONKIT_LONG_TESTS" fi @@ -596,11 +605,10 @@ jobs: # is spelled out because, left to their defaults on the runner, # avdmanager writes the AVD where the emulator does not look. `-cores` # likewise: an AVD's default is fewer than the runner has, few enough - # that the measurements skip themselves for want of four, and that the - # long stress scale starves the tests that poll against a budget. The boot - # wait is the only wait, and bounded: an emulator that died on launch - # never becomes a device, and `adb wait-for-device` would sit out the - # job's whole timeout on it. + # that the long stress scale starves the tests that poll against a + # budget. The boot wait is the only wait, and bounded: an emulator that + # died on launch never becomes a device, and `adb wait-for-device` + # would sit out the job's whole timeout on it. # # The emulator outlives this step — the runner reaps stray processes # when the job ends, not before — and adb goes on PATH for the step that @@ -653,28 +661,30 @@ jobs: # --skip-build` would, by hand: stage the bundle with every shared # library it loads — the Swift runtime from the SDK, libc++ from the NDK # — push the lot, and run it over adb, which hands back the exit status. - # Once for each testing library, because a runner runs one per - # invocation, and once per runner: Swift 6.3 builds a single bundle for - # the package, where 6.4's build system builds a runner for each test - # target. Named one at a time: a glob on one line runs the first runner - # with the rest as its arguments, which ran one target's tests, none of - # its XCTest cases, and succeeded. XCTest is left out of the debug run: - # its runner takes names and no pattern, and every XCTest case here is a - # measurement, which release runs whole. A runner for one target may hold - # no suite the selection names, and Swift Testing says so by exiting - # `EX_UNAVAILABLE`, 69, which is not a failure; `swift test` makes the - # same allowance on the other rows. A function called on the left of `||` - # runs without `-e`, so the steps a later one depends on say for - # themselves that the function ends with them. + # Once per runner: Swift 6.3 builds a single bundle for the package, + # where 6.4's build system builds a runner for each test target. Named + # one at a time: a glob on one line runs the first runner with the rest + # as its arguments, which ran one target's tests and succeeded. A + # runner for one target may hold no suite the selection names, and + # Swift Testing says so by exiting `EX_UNAVAILABLE`, 69, which is not a + # failure; `swift test` makes the same allowance on the other rows. A + # function called on the left of `||` runs without `-e`, so the steps + # a later one depends on say for themselves that the function ends + # with them. + # + # Swift Testing alone, everywhere, which the build flags above say with + # `--disable-xctest`: no XCTest case is left in the package — the + # measurements that were XCTest's live in the benchmark package now — + # and the corelibs runner traps on an empty list rather than reporting + # none. - name: Run tests if: ${{ !matrix.skip_tests }} shell: bash run: | run_tests() { - local configuration="$1" test_flags="$2" select="$3" testable="" libraries=(swift-testing) + local configuration="$1" test_flags="$2" select="$3" testable="" if [ "$configuration" = release ]; then testable="--disable-testable-imports" - libraries=(xctest swift-testing) fi if [ -z "$ANDROID" ]; then @@ -691,19 +701,15 @@ jobs: cp -aL "$ANDROID_SWIFT_SDK/ndk-sysroot/usr/lib/$arch-linux-android/libc++_shared.so" "$staging" || return 1 adb push "$staging" "$device" || return 1 - local runners=("$staging"/*-test-runner) runner library selection code status=0 + local runners=("$staging"/*-test-runner) runner code status=0 if [ ! -e "${runners[0]}" ]; then runners=("$staging"/*.xctest); fi if [ ! -e "${runners[0]}" ]; then echo "No test runner was built." >&2; return 1; fi for runner in "${runners[@]}"; do - for library in "${libraries[@]}"; do - echo "$(basename "$runner") · $library" - selection="" - if [ "$library" = swift-testing ]; then selection="$select '$SUITES'"; fi - adb shell "cd $device && ./$(basename "$runner") --testing-library $library $selection" || { - code=$? - if [ "$library" != swift-testing ] || [ $code -ne 69 ]; then status=1; fi - } - done + echo "$(basename "$runner")" + adb shell "cd $device && ./$(basename "$runner") --testing-library swift-testing $select '$SUITES'" || { + code=$? + if [ $code -ne 69 ]; then status=1; fi + } done return $status } @@ -716,3 +722,20 @@ jobs: run_tests release "$release_test_flags" --filter || status=1 echo "::endgroup::" exit $status + + # The benchmarks, on the rows that say so: one macOS and one Linux per + # minor, which is where the harness runs. They measure the library as + # it ships, built by the benchmark package's own manifest against the + # checkout above it, and put the table in the job's summary. Nothing + # fails on a number: a shared runner is noisy, and what the numbers are + # for is reading, here and against a baseline taken on one machine, + # which the README says how to do. `pipefail`, so that a benchmark that + # crashes fails the step through the tee. + - name: Run benchmarks + if: ${{ matrix.benchmarks }} + working-directory: Benchmarks + shell: bash + run: | + set -o pipefail + swift package benchmark --format markdown --no-progress --grouping metric | tee "$RUNNER_TEMP/benchmarks.md" + sed -n '/^## Baseline/,$p' "$RUNNER_TEMP/benchmarks.md" >> "$GITHUB_STEP_SUMMARY" diff --git a/Package.swift b/Package.swift index 3824734..5c70cae 100644 --- a/Package.swift +++ b/Package.swift @@ -288,17 +288,15 @@ let package = Package( // not. // // The asynchronous dependencies are for the helpers that reach into the - // wait queue; Atomic is for the measurement harness's counter; Mutex - // guards the gate's waiters. The synchronous suites pay for the first - // in build time and nothing else: what they import from here is a pair - // of globals, the stress dial, and the harness. + // wait queue; Mutex guards the gate's waiters. The synchronous suites + // pay for the first in build time and nothing else: what they import + // from here is a pair of globals and the stress dial. .target( name: "SynchronizationKitTestUtils", dependencies: [ "SynchronizationKitAsyncCore", "SynchronizationKitAsyncMutex", "SynchronizationKitAsyncRWLock", - "SynchronizationKitAtomic", "SynchronizationKitMutex", ], swiftSettings: commonSwiftSettings, diff --git a/Sources/SynchronizationKitTestUtils/Measurement.swift b/Sources/SynchronizationKitTestUtils/Measurement.swift deleted file mode 100644 index 307b89f..0000000 --- a/Sources/SynchronizationKitTestUtils/Measurement.swift +++ /dev/null @@ -1,570 +0,0 @@ -// -// Measurement.swift -// SynchronizationKit -// -// Copyright (c) 2026 Kang Jaehong -// SPDX-License-Identifier: Apache-2.0 -// - -// The harness the performance suites measure through, kept in one place so -// that a Mutex measurement and an RWLock measurement differ only in the lock. -// -// XCTest rather than the testing library: measurement has no equivalent -// there, and the two coexist in one target. The measurements print under -// plain `swift test`, so none of this needs Xcode — only baselines do, and -// those live in a scheme or test plan that this package has no place to -// keep. Read the numbers; nothing here fails on a regression. -// -// Apple's XCTest measures wall clock and CPU counters, and reports. corelibs -// XCTest, on Linux and Windows, measures wall clock alone — and passes a -// verdict: a spread of more than ten percent across the runs, once it is -// more than a tenth of a second, fails the test, with no baseline involved -// and no way to turn it off. A contended measurement on a shared runner -// exceeds that whenever the runner's other tenants do, which is what turned -// a reading into a red build. So off Apple platforms the harness keeps its -// own clock, ten runs like corelibs', and prints what corelibs would have -// printed, minus the verdict. The two are told apart once, in -// `_measureLock`, and the harness reads the same everywhere else. Nothing -// here on a platform without XCTest at all — the static Linux SDK — since -// nothing runs tests there. -// -// Two ways to measure a lock badly, both of which this repository has already -// been caught by, and what is done about each: -// -// - Nothing forces one critical section to finish before the next begins, so -// the processor overlaps them and the work inside stops costing anything. -// Every section measured through this follows a pointer chase — a single -// cycle through a protected array, each load depending on the one before — -// which the processor cannot run ahead of. -// - Threads left to the scheduler land on efficiency cores and the spread -// swallows the effect. Every thread here is pinned to `.userInteractive`. -// -// A benchmark whose work gets optimized away reports excellent numbers, so -// each measurement asserts that the chase actually happened before it accepts -// a result. -// Dispatch as well as XCTest: WASI has the latter and not the former, and the -// threads every measurement here drives are joined through Dispatch. -#if canImport(XCTest) && canImport(Dispatch) -import Dispatch -import Foundation -import SynchronizationKitAtomic -package import XCTest - -/// What a measured lock protects: a permutation the readers chase, and a -/// counter the writers move. -package struct ChasePayload: Sendable { - package var cycle: [Int] - package var writes = 0 - - package init() { - cycle = Chase.cycle - } -} - -/// The pointer chase every measurement runs, and where it ends. -package enum Chase { - package static let slots = 1024 - - /// A single cycle visiting every slot, so a chase never short-circuits - /// into a small loop that would sit in cache. - package static let cycle: [Int] = { - var order = Array(1 ..< slots) - var state = UInt64(0x2545_F491_4F6C_DD1D) - for position in stride(from: order.count - 1, to: 0, by: -1) { - state ^= state << 13 - state ^= state >> 7 - state ^= state << 17 - order.swapAt(position, Int(state % UInt64(position + 1))) - } - - var cycle = [Int](repeating: 0, count: slots) - var previous = 0 - for next in order { - cycle[previous] = next - previous = next - } - cycle[previous] = 0 - return cycle - }() - - /// Where following the cycle `steps` times from `start` ends up. - /// - /// The cycle visits every slot once, so it repeats with period `slots` - /// and the answer is reachable in at most that many steps however long - /// the run was. Comparing against this is what makes the checks below - /// say something: a chase that never moved lands on `start`, not here. - package static func end(from start: Int, steps: Int) -> Int { - var index = start - for _ in 0 ..< (steps % slots) { - index = cycle[index] - } - return index - } -} - -/// A counter the measured blocks can add to from several threads. -/// -/// Lock-free on purpose. An `NSLock` here would put a second lock inside the -/// measured window, contended by every worker at the moment they all finish — -/// noise on the same scale as the handoff being measured. A class because -/// `Atomic` is noncopyable and so cannot be captured by a measured block. -private final class Tally: @unchecked Sendable { - private let storage = Atomic(0) - - var value: Int { - storage.load(ordering: .acquiring) - } - - func add(_ operand: Int) { - storage.wrappingAdd(operand, ordering: .acquiringAndReleasing) - } -} - -/// Where each worker of a contended case leaves its result: one slot per -/// worker, written by that worker's thread alone and read once every thread -/// has finished, so that nothing is checked or counted inside the measured -/// window. -/// -/// `@unchecked Sendable` on that discipline: no slot is written by two -/// threads, and the semaphore the harness joins the threads on orders every -/// write before the reads. -/// -/// `@safe`, with the buffer marked `@unsafe`: every access to it is spelled -/// out below, and nothing hands the pointer out. -@safe -private final class _WorkerResults: @unchecked Sendable { - @unsafe private let slots: UnsafeMutablePointer<(end: Int, steps: Int, turns: Int)> - private let count: Int - - init(count: Int) { - self.count = count - unsafe slots = UnsafeMutablePointer<(end: Int, steps: Int, turns: Int)> - .allocate(capacity: count) - unsafe slots.initialize(repeating: (end: 0, steps: 0, turns: 0), count: count) - } - - deinit { - unsafe slots.deinitialize(count: count) - unsafe slots.deallocate() - } - - func record(end: Int, steps: Int, turns: Int, for worker: Int) { - unsafe slots[worker] = (end: end, steps: steps, turns: turns) - } - - func result(of worker: Int) -> (end: Int, steps: Int, turns: Int) { - unsafe slots[worker] - } -} - -/// The turns a group of workers has left between them. -/// -/// One atomic counter, drawn down a batch at a time. Lock-free for the -/// reason `Tally` is: every worker touches it, and a lock here would be a -/// second contended primitive inside the measured window. The batch keeps -/// the counter itself from becoming that — a worker comes back to it once -/// every `WorkShare.batch` turns, not every turn. -package final class _WorkBudget: @unchecked Sendable { - private let remaining: Atomic - - package init(total: Int) { - remaining = Atomic(total) - } - - /// Claims up to `count` turns, and returns how many were left to claim: - /// `count`, fewer at the end, or zero once the budget is spent. - package func claim(upTo count: Int) -> Int { - var left = remaining.load(ordering: .relaxed) - while left > 0 { - let claiming = min(count, left) - let (exchanged, observed) = remaining.compareExchange( - expected: left, - desired: left - claiming, - ordering: .relaxed - ) - if exchanged { - return claiming - } - left = observed - } - return 0 - } -} - -/// One worker's share of a group's budget: `eachTurn` runs the worker's -/// body once per turn until the budget is spent. -/// -/// A method taking the body rather than a sequence to iterate, and -/// inlinable, so that the loop runs on locals in the test's own module: a -/// turn of the cheapest primitive here is a few nanoseconds, and an -/// iterator's call and the exclusivity checks on its stored properties -/// cost as much again — measured, they tripled the contended `Mutex` -/// number. A class, so that the harness can read how many turns the worker -/// took once its body has returned; it is created on the worker's thread -/// and used only there. -package final class WorkShare { - /// How many turns are claimed from the budget at once: often enough - /// that a slow worker does not hold much back, seldom enough that the - /// budget's counter is not contended. Claiming every turn was measured - /// at ten times the cost of the turns themselves. - @usableFromInline - package static let batch = 64 - - @usableFromInline - internal let budget: _WorkBudget - - /// How many turns the worker has taken, and how many chase steps they - /// came to. - package private(set) var turns = 0 - package private(set) var steps = 0 - - package init(of budget: _WorkBudget) { - self.budget = budget - } - - /// Runs `turn` once per turn claimed from the budget, until it is spent, - /// counting `steps` chase steps for each: one, unless the body chases - /// further and says so. - @inlinable - package func eachTurn(steps: Int = 1, _ turn: () -> Void) { - var taken = 0 - while true { - let claimed = budget.claim(upTo: Self.batch) - if claimed == 0 { - break - } - for _ in 0 ..< claimed { - turn() - } - taken += claimed - } - _record(turns: taken, steps: taken * steps) - } - - @usableFromInline - internal func _record(turns: Int, steps: Int) { - self.turns += turns - self.steps += steps - } -} - -/// What a measured block starts and stops: XCTest's meter on Apple -/// platforms, the harness's own clock elsewhere. -private struct MeasurementClock { - fileprivate let _start: () -> Void - fileprivate let _stop: () -> Void - - package func start() { - _start() - } - - package func stop() { - _stop() - } -} - -#if !canImport(Darwin) -/// Ten wall-clock samples and the line corelibs XCTest would print for them. -private final class _WallClock { - private var began: DispatchTime? - private(set) var samples: [Double] = [] - - func start() { - began = .now() - } - - func stop() { - guard let began else { - return - } - samples.append(Double(DispatchTime.now().uptimeNanoseconds - began.uptimeNanoseconds) / 1e9) - self.began = nil - } - - var report: String { - let average = samples.reduce(0, +) / Double(samples.count) - let variance = samples.reduce(0) { $0 + ($1 - average) * ($1 - average) } / Double(samples.count) - let deviation = variance.squareRoot() - let relative = average > 0 ? deviation / average * 100 : 0 - let values = samples.map { String(format: "%.6f", $0) }.joined(separator: ", ") - return String( - format: "measured [Time, seconds] average: %.3f, relative standard deviation: %.3f%%, values: [%@]", - average, relative, values - ) - } -} -#endif - -extension XCTestCase { - /// Measures `block` with the metrics the platform has, starting and - /// stopping on the block's say-so through the clock it is handed if - /// `manually` is set, and around the whole block otherwise. - /// - /// On Apple platforms: wall clock for the contention story, and the CPU - /// counters because instructions retired barely varies where elapsed - /// time does. Read instructions retired, not elapsed time, for anything - /// uncontended: it varies by a fraction of a percent between runs where - /// the clock varies by tens. Under contention the scheduler dominates - /// both. The metrics are built fresh per call: `XCTMetric` is not - /// `Sendable`, so one shared array could not be a static in the first - /// place, and a metric is free to carry state from the run it just took - /// part in. - /// - /// Elsewhere: the harness's own wall clock, for the reason the file - /// header gives; ten runs, printed in corelibs' own shape so a log reads - /// the same either way. - private func _measureLock(manually: Bool, _ block: (MeasurementClock) -> Void) { - #if canImport(Darwin) - let options = XCTMeasureOptions() - if manually { - options.invocationOptions = [.manuallyStart, .manuallyStop] - } - let clock = MeasurementClock(_start: { self.startMeasuring() }, _stop: { self.stopMeasuring() }) - measure(metrics: [XCTClockMetric(), XCTCPUMetric()], options: options) { - block(clock) - } - #else - let wallClock = _WallClock() - let clock = MeasurementClock(_start: { wallClock.start() }, _stop: { wallClock.stop() }) - for _ in 0 ..< 10 { - if manually { - block(clock) - } else { - wallClock.start() - block(clock) - wallClock.stop() - } - } - print("Test Case '\(name)' \(wallClock.report)") - #endif - } - - /// Skips the measurement in a debug build, where an unoptimized one says - /// nothing about anything, and under a sanitizer, where the instrumented - /// build being timed is not the one that ships. - /// - /// ThreadSanitizer cannot be measured at all: Apple's XCTest measurement - /// worker crashes in `objc_release` partway through — the correctness - /// suites pass on the same run, and no race is reported before it. Since - /// the number would say nothing anyway, there is nothing there worth - /// chasing that crash for. - package func skipUnlessMeasurable() throws { - #if DEBUG - throw XCTSkip("Measurements only mean something optimized; build for release.") - #else - try XCTSkipIf( - threadSanitizerIsLoaded || addressSanitizerIsLoaded, - "A sanitized build is not the one that ships; timing it would say nothing." - ) - #endif - } - - /// Skips a contended case on a machine too small for the result to say - /// anything about contention. - package func skipUnlessRoomToContend() throws { - try XCTSkipUnless( - ProcessInfo.processInfo.activeProcessorCount >= 4, - "Too few cores for a contention measurement to mean anything." - ) - } - - /// Runs threads over a fresh fixture, timing only the part where they are - /// actually contending. - /// - /// Threads are started and parked first, then released together, so - /// thread creation stays outside the measured window. - /// - /// The work is not divided up front. Each group in `groups` — readers, - /// say, or writers — has a budget of `workers × iterations` turns that - /// its workers draw from as they go, a batch at a time, so a worker that - /// happens to land on a slower core takes fewer turns and the group - /// finishes together rather than waiting on its slowest member. On a - /// chip with more than one kind of core that is the difference between - /// measuring the primitive and measuring which cores the scheduler - /// picked. Workers are numbered across the groups in order, so the first - /// group's workers come first. - /// - /// `work` is handed the fixture, its worker's number, and its share of - /// the group's budget, whose `eachTurn` runs its turns; it chases the - /// cycle once per turn, or the `steps` it tells `eachTurn`, from an - /// index that starts at its number and returns where it ended. The - /// harness checks each against where that many steps should have - /// ended, and that every budget was spent, so a body whose work was - /// optimized away fails rather than measuring nothing. `check` sees the - /// fixture afterwards for whatever else the body has to have done. - package func measureContention( - groups: [(workers: Int, iterations: Int)], - makeFixture: () -> Fixture, - work: @escaping @Sendable (Fixture, _ worker: Int, _ share: WorkShare) -> Int, - check: (Fixture) -> Void = { _ in } - ) { - _measureLock(manually: true) { clock in - let fixture = makeFixture() - let parked = DispatchSemaphore(value: 0) - let start = DispatchSemaphore(value: 0) - let finished = DispatchSemaphore(value: 0) - let budgets = groups.map { _WorkBudget(total: $0.workers * $0.iterations) } - let workers = groups.reduce(0) { $0 + $1.workers } - - let results = _WorkerResults(count: workers) - - var worker = 0 - for (group, budget) in zip(groups, budgets) { - for _ in 0 ..< group.workers { - let number = worker - let thread = Thread { - let share = WorkShare(of: budget) - parked.signal() - start.wait() - let end = work(fixture, number, share) - results.record( - end: end, - steps: share.steps, - turns: share.turns, - for: number - ) - finished.signal() - } - thread.qualityOfService = .userInteractive - thread.start() - worker += 1 - } - } - - for _ in 0 ..< workers { - parked.wait() - } - - clock.start() - for _ in 0 ..< workers { - start.signal() - } - for _ in 0 ..< workers { - finished.wait() - } - clock.stop() - - var turnsTaken = 0 - for worker in 0 ..< workers { - let (end, steps, turns) = results.result(of: worker) - XCTAssertEqual( - end, - Chase.end(from: worker, steps: steps), - "worker \(worker)'s chase did not end where its steps say" - ) - turnsTaken += turns - } - XCTAssertEqual( - turnsTaken, - groups.reduce(0) { $0 + $1.workers * $1.iterations }, - "the budgets were not spent" - ) - check(fixture) - } - } - - /// `measureContention(groups:)` for the common case of one group. - package func measureContention( - workers: Int, - iterations: Int, - makeFixture: () -> Fixture, - work: @escaping @Sendable (Fixture, _ worker: Int, _ share: WorkShare) -> Int, - check: (Fixture) -> Void = { _ in } - ) { - measureContention( - groups: [(workers: workers, iterations: iterations)], - makeFixture: makeFixture, - work: work, - check: check - ) - } - - /// Runs `tasks` tasks over a fresh fixture, for the asynchronous - /// primitives. - /// - /// The whole block is measured, task creation included: there is no - /// cheap way to park a crowd of tasks and release them together, and a - /// task costs microseconds against the tens of thousands of handoffs - /// each one then makes. Detached at `.userInitiated` for the same reason - /// the threads are pinned. - /// - /// `work` is checked as for `measureContention`: it chases the cycle - /// once per iteration from its task's number, and the sum of where the - /// chases ended has to be what the cycle says. - /// - /// Read the single-task numbers with the executor in mind. A turn here - /// is a take, a yield, and a release, and on Linux the yield costs - /// according to how much the task did before it: a few hundred - /// nanoseconds after almost nothing, several microseconds after half a - /// microsecond of work, as the pool hands the resumed task to another - /// thread. Measured directly, with nothing but an atomic spin before - /// the yield, so it is the executor's and not a lock's; on macOS the - /// cost stays flat until the work runs to microseconds. A primitive - /// whose take and release together cross that line therefore reads as - /// several times another's there while differing by a fraction, so - /// compare Linux numbers between primitives only at like per-turn - /// work, and against macOS not at all. - package func measureTaskContention( - tasks: Int, - iterations: Int, - makeFixture: () -> Fixture, - work: @escaping @Sendable (Fixture, _ task: Int) async throws -> Int, - check: (Fixture) -> Void = { _ in } - ) { - _measureLock(manually: false) { _ in - let fixture = makeFixture() - let finished = DispatchSemaphore(value: 0) - let chased = Tally() - let failed = Tally() - - for task in 0 ..< tasks { - Task.detached(priority: .userInitiated) { - do { - chased.add(try await work(fixture, task)) - } catch { - failed.add(1) - } - finished.signal() - } - } - for _ in 0 ..< tasks { - finished.wait() - } - - let expectedChase = (0 ..< tasks) - .reduce(0) { $0 + Chase.end(from: $1, steps: iterations) } - XCTAssertEqual(failed.value, 0, "a task threw") - XCTAssertEqual(chased.value, expectedChase, "the chase did not advance") - check(fixture) - } - } - - /// Runs one thread over a fresh fixture. Nothing to park, so the whole - /// block is measured. - /// - /// `work` chases the cycle once per iteration from index zero and - /// returns where it ended, as for `measureContention`. - package func measureUncontended( - iterations: Int, - makeFixture: () -> Fixture, - work: (Fixture) -> Int, - check: (Fixture) -> Void = { _ in } - ) { - _measureLock(manually: false) { _ in - let fixture = makeFixture() - XCTAssertEqual( - work(fixture), - Chase.end(from: 0, steps: iterations), - "the chase did not advance" - ) - check(fixture) - } - } - - /// How many workers a contended case runs: every core but two, so the - /// odd one out — a writer, a signaller — and the harness each have one. - package var contendedWorkers: Int { - max(2, ProcessInfo.processInfo.activeProcessorCount - 2) - } -} -#endif diff --git a/Sources/SynchronizationKitTestUtils/TestEnvironment.swift b/Sources/SynchronizationKitTestUtils/TestEnvironment.swift index da8a597..9ca6c36 100644 --- a/Sources/SynchronizationKitTestUtils/TestEnvironment.swift +++ b/Sources/SynchronizationKitTestUtils/TestEnvironment.swift @@ -51,9 +51,6 @@ private func sanitizerIsLoaded(entryPoint: String) -> Bool { /// Whether the ThreadSanitizer runtime is loaded into this process. public let threadSanitizerIsLoaded = sanitizerIsLoaded(entryPoint: "__tsan_init") -/// Whether the AddressSanitizer runtime is loaded into this process. -public let addressSanitizerIsLoaded = sanitizerIsLoaded(entryPoint: "__asan_init") #else public let threadSanitizerIsLoaded = false -public let addressSanitizerIsLoaded = false #endif diff --git a/Tests/SynchronizationKitAsyncMutexTests/AsyncMutexPerformanceTests.swift b/Tests/SynchronizationKitAsyncMutexTests/AsyncMutexPerformanceTests.swift deleted file mode 100644 index 21bb8f1..0000000 --- a/Tests/SynchronizationKitAsyncMutexTests/AsyncMutexPerformanceTests.swift +++ /dev/null @@ -1,209 +0,0 @@ -// -// AsyncMutexPerformanceTests.swift -// SynchronizationKit -// -// Copyright (c) 2026 Kang Jaehong -// SPDX-License-Identifier: Apache-2.0 -// - -// Gated on Dispatch: the threads these tests drive are started and joined -// through it, and WASI has none. What runs there is the suite next door -// that needs no thread of its own. -#if canImport(Dispatch) -import Dispatch -import SynchronizationKitAsyncMutex -import SynchronizationKitMutex -import SynchronizationKitTestUtils -import XCTest - -/// What `AsyncMutex` costs: the uncontended take, and the handoff through -/// the wait queue as the queue gets longer — and what an `actor` costs on -/// the same turns, since the README says to prefer one wherever it fits. -/// Each queue case runs once per implementation, so the two land in one -/// report. -/// -/// The harness, and why it measures the way it does, is in -/// `Measurement.swift`; `RWLockPerformanceTests` says how to run these. The -/// closure suspends once inside the lock, so every take under contention is -/// a real handoff rather than a spin, and what grows with the task count is -/// the queue a departing holder chooses the next holder from. -/// -/// An actor cannot hold anything across an `await`, so the suspension the -/// mutex's closure makes inside the lock is made in the actor cases after -/// the actor's method has returned: each turn is still one step and one -/// suspension, and what the two differ in is the handoff — through the -/// mutex's wait queue, or through the actor's mailbox — which is what the -/// numbers say the choice between them costs. -final class AsyncMutexPerformanceTests: XCTestCase { - /// A reference to hold the lock by; `RWLockPerformanceTests.LockBox` - /// says why. - final class LockBox: @unchecked Sendable { - let lock = AsyncMutex(ChasePayload()) - } - - /// The same payload behind an actor. - actor ActorBox { - private var payload = ChasePayload() - - /// One write and one step of the chase from `index`. - func step(from index: Int) -> Int { - payload.writes &+= 1 - return payload.cycle[index] - } - - var writes: Int { - payload.writes - } - } - - override func setUpWithError() throws { - try skipUnlessMeasurable() - } - - private func measureHandoff(tasks: Int, iterations: Int) { - measureTaskContention(tasks: tasks, iterations: iterations, makeFixture: LockBox.init) { box, task in - var index = task - for _ in 0 ..< iterations { - try await box.lock.withLock { payload in - payload.writes &+= 1 - index = payload.cycle[index] - await Task.yield() - } - } - return index - } check: { box in - // Nobody holds the lock any more, so the try cannot fail; and it - // cannot throw, which keeps the task's result from needing a - // reader. - let finished = DispatchSemaphore(value: 0) - Task.detached { - let writes = await box.lock.withLockIfAvailable { $0.writes } - XCTAssertEqual(writes, tasks * iterations, "the workload did not run") - finished.signal() - } - finished.wait() - } - } - - private func measureActorHandoff(tasks: Int, iterations: Int) { - measureTaskContention( - tasks: tasks, - iterations: iterations, - makeFixture: ActorBox.init - ) { box, task in - var index = task - for _ in 0 ..< iterations { - index = await box.step(from: index) - await Task.yield() - } - return index - } check: { box in - let finished = DispatchSemaphore(value: 0) - Task.detached { - let writes = await box.writes - XCTAssertEqual(writes, tasks * iterations, "the workload did not run") - finished.signal() - } - finished.wait() - } - } - - /// Two tasks at the harness's priority hand the lock back and forth - /// through a queue of `lows` waiters at low priority; the semaphore - /// suite's `measurePriorityHandoff` says why the low waiters are - /// detached tasks the workers spawn and do not await, why each takes - /// one turn and spawns its successor, and how the sample ends only - /// once they have all left. The one thing particular to this suite is - /// what makes the successor necessary: a filler that held the lock - /// while a high task queued was escalated to that task's priority, and - /// stays there. - private func measurePriorityHandoff(lows: Int, iterations: Int) { - final class Fixture: Sendable { - let lock = AsyncMutex(0) - let finished = Mutex(0) - let fillers: Mutex - let drained = Gate() - - init(lows: Int) { - fillers = Mutex(lows) - } - } - @Sendable func fill(_ fixture: Fixture) { - // The task is deliberately neither awaited nor kept: awaiting it - // would raise it to the awaiter's priority, and the fixture's - // flags are how the fillers end. - _ = Task.detached(priority: .low) { - try await fixture.lock.withLock { _ in - await Task.yield() - } - if fixture.finished.withLock({ $0 < 2 }) { - fill(fixture) - } else if fixture.fillers.withLock({ $0 -= 1; return $0 == 0 }) { - fixture.drained.open() - } - } - } - measureTaskContention(tasks: lows + 2, iterations: iterations, makeFixture: { Fixture(lows: lows) }) { fixture, task in - var index = task - if task < 2 { - for _ in 0 ..< iterations { - try await fixture.lock.withLock { holds in - holds += 1 - index = Chase.cycle[index] - await Task.yield() - } - } - fixture.finished.withLock { $0 += 1 } - await fixture.drained.wait() - } else { - for _ in 0 ..< iterations { - index = Chase.cycle[index] - } - fill(fixture) - } - return index - } - } - - func testUncontended() { - measureHandoff(tasks: 1, iterations: 100_000) - } - - func testUncontendedActor() { - measureActorHandoff(tasks: 1, iterations: 100_000) - } - - func testShortQueue() { - measureHandoff(tasks: 8, iterations: 10_000) - } - - func testShortQueueActor() { - measureActorHandoff(tasks: 8, iterations: 10_000) - } - - func testLongQueue() { - measureHandoff(tasks: 64, iterations: 2_000) - } - - func testLongQueueActor() { - measureActorHandoff(tasks: 64, iterations: 2_000) - } - - /// Past a few hundred waiters, what shows is the queue's own - /// bookkeeping: a handoff here costs what one in `testLongQueue` does, - /// and a regression that scales with the queue costs several times it. - func testVeryLongQueue() { - measureHandoff(tasks: 512, iterations: 250) - } - - func testVeryLongQueueActor() { - measureActorHandoff(tasks: 512, iterations: 250) - } - - /// A high-priority arrival placed, and served, ahead of a long queue at - /// low priority. - func testHighPriorityAmongLowWaiters() { - measurePriorityHandoff(lows: 512, iterations: 10_000) - } -} -#endif diff --git a/Tests/SynchronizationKitAsyncRWLockTests/AsyncRWLockPerformanceTests.swift b/Tests/SynchronizationKitAsyncRWLockTests/AsyncRWLockPerformanceTests.swift deleted file mode 100644 index eedb7a4..0000000 --- a/Tests/SynchronizationKitAsyncRWLockTests/AsyncRWLockPerformanceTests.swift +++ /dev/null @@ -1,112 +0,0 @@ -// -// AsyncRWLockPerformanceTests.swift -// SynchronizationKit -// -// Copyright (c) 2026 Kang Jaehong -// SPDX-License-Identifier: Apache-2.0 -// - -// Gated on Dispatch: the threads these tests drive are started and joined -// through it, and WASI has none. What runs there is the suite next door -// that needs no thread of its own. -#if canImport(Dispatch) -import Dispatch -import SynchronizationKitAsyncRWLock -import SynchronizationKitTestUtils -import XCTest - -/// What `AsyncRWLock` costs: the uncontended take of either side, the -/// handoff between writers as the queue gets longer, and a read-mostly mix -/// in which readers are admitted together and a writer waits them out. -/// -/// The harness, and why it measures the way it does, is in -/// `Measurement.swift`; `RWLockPerformanceTests` says how to run these. The -/// writer cases are `AsyncMutexPerformanceTests`' handoff on the other -/// primitive — the closure suspends once inside the lock, so every take -/// under contention is a real handoff — and the two share their counts, so -/// the numbers sit side by side. What the read side adds to the same wait -/// queue is in the mixed cases, where a writer's arrival ends a batch of -/// readers and its release admits the next. -final class AsyncRWLockPerformanceTests: XCTestCase { - /// A reference to hold the lock by; `RWLockPerformanceTests.LockBox` - /// says why. - final class LockBox: @unchecked Sendable { - let lock = AsyncRWLock(ChasePayload()) - } - - override func setUpWithError() throws { - try skipUnlessMeasurable() - } - - /// `tasks` tasks each take the lock `iterations` times, one turn in - /// `writeEvery` for writing and the rest for reading, suspending once - /// inside it either way. A `writeEvery` of one is all writers. - private func measureHandoff(tasks: Int, iterations: Int, writeEvery: Int = 1) { - measureTaskContention(tasks: tasks, iterations: iterations, makeFixture: LockBox.init) { box, task in - var index = task - for turn in 0 ..< iterations { - if turn % writeEvery == 0 { - try await box.lock.withWriteLock { payload in - payload.writes &+= 1 - index = payload.cycle[index] - await Task.yield() - } - } else { - try await box.lock.withReadLock { payload in - index = payload.cycle[index] - await Task.yield() - } - } - } - return index - } check: { box in - // Nobody holds the lock any more, so the try cannot fail; and it - // cannot throw, which keeps the task's result from needing a - // reader. - let finished = DispatchSemaphore(value: 0) - Task.detached { - let writes = await box.lock.withReadLockIfAvailable { $0.writes } - let expected = tasks * ((iterations + writeEvery - 1) / writeEvery) - XCTAssertEqual(writes, expected, "the workload did not run") - finished.signal() - } - finished.wait() - } - } - - /// One task alone, reading: the take and release of the read side with - /// nobody to wait for or to wake. `writeEvery` past the last turn, so - /// the only write is the first. - func testUncontendedReads() { - measureHandoff(tasks: 1, iterations: 100_000, writeEvery: 100_000) - } - - func testUncontendedWrites() { - measureHandoff(tasks: 1, iterations: 100_000) - } - - func testWriterShortQueue() { - measureHandoff(tasks: 8, iterations: 10_000) - } - - func testWriterLongQueue() { - measureHandoff(tasks: 64, iterations: 2_000) - } - - /// Past a few hundred waiters, what shows is the queue's own - /// bookkeeping, as in `AsyncMutexPerformanceTests.testVeryLongQueue`. - func testWriterVeryLongQueue() { - measureHandoff(tasks: 512, iterations: 250) - } - - /// One turn in eight a write: enough writers that the readers rarely run - /// long unopposed, few enough that most admissions are of readers. - func testReadMostlyShortQueue() { - measureHandoff(tasks: 8, iterations: 10_000, writeEvery: 8) - } - - func testReadMostlyLongQueue() { - measureHandoff(tasks: 64, iterations: 2_000, writeEvery: 8) - } -} -#endif diff --git a/Tests/SynchronizationKitAsyncSemaphoreTests/AsyncSemaphorePerformanceTests.swift b/Tests/SynchronizationKitAsyncSemaphoreTests/AsyncSemaphorePerformanceTests.swift deleted file mode 100644 index 688bbb0..0000000 --- a/Tests/SynchronizationKitAsyncSemaphoreTests/AsyncSemaphorePerformanceTests.swift +++ /dev/null @@ -1,246 +0,0 @@ -// -// AsyncSemaphorePerformanceTests.swift -// SynchronizationKit -// -// Copyright (c) 2026 Kang Jaehong -// SPDX-License-Identifier: Apache-2.0 -// - -// Gated on Dispatch: the threads these tests drive are started and joined -// through it, and WASI has none. What runs there is the suite next door -// that needs no thread of its own. -#if canImport(Dispatch) -import SynchronizationKitAsyncSemaphore -import SynchronizationKitMutex -import SynchronizationKitTestUtils -import XCTest - -/// What `AsyncSemaphore` costs: the uncontended take, and the handoff -/// through the wait queue as the queue gets longer. -/// -/// The harness, and why it measures the way it does, is in -/// `Measurement.swift`; `RWLockPerformanceTests` says how to run these. The -/// semaphore is used as a lock — a count of one, taken around the chase and -/// a suspension — so the numbers sit beside `AsyncMutex`'s, which differs -/// from this in holding a value and in escalating its holder. -final class AsyncSemaphorePerformanceTests: XCTestCase { - /// The semaphore and what it guards, held by reference; - /// `RWLockPerformanceTests.LockBox` says why. The payload is guarded by - /// the semaphore, which is what makes the unchecked conformance right. - final class LockBox: @unchecked Sendable { - let semaphore = AsyncSemaphore(value: 1) - var payload = ChasePayload() - } - - override func setUpWithError() throws { - try skipUnlessMeasurable() - } - - private func measureHandoff(tasks: Int, iterations: Int) { - measureTaskContention(tasks: tasks, iterations: iterations, makeFixture: LockBox.init) { box, task in - var index = task - for _ in 0 ..< iterations { - try await box.semaphore.wait() - box.payload.writes &+= 1 - index = box.payload.cycle[index] - await Task.yield() - box.semaphore.signal() - } - return index - } check: { box in - XCTAssertEqual(box.payload.writes, tasks * iterations, "the workload did not run") - } - } - - /// Every task but the first waits on a count that is never raised, so - /// nothing it waits for can arrive; the first cancels them once they are - /// all queued, in a random order so that each departure is from somewhere - /// in the middle, and the round repeats. No cancellation is issued until - /// the harness's count confirms every waiter has joined — each has one - /// child at a time and none can be served, so the count says they all - /// have — which is what makes each one a departure from the queue rather - /// than a task that never reached it. Every `wait()` must throw, and a - /// waiter moves its chase only when its child's did, so a child that - /// returned instead fails the chase check. - /// - /// The handles a round cancels are collected under a lock and taken out - /// of it before the cancellations go out. A waiter registers its next - /// child only after the previous one has been cancelled and awaited, so - /// the registry fills to the waiter count exactly once per round. - private func measureCancellation(waiters: Int, rounds: Int) { - final class Fixture: Sendable { - let semaphore = AsyncSemaphore(value: 0) - let handles = Mutex<[Task]>([]) - } - measureTaskContention(tasks: waiters + 1, iterations: rounds, makeFixture: Fixture.init) { fixture, task in - var index = task - if task == 0 { - var generator = SplitMix64(seed: 0x5EED) - for _ in 0 ..< rounds { - var handles: [Task] = [] - while true { - await fixture.semaphore.waitForWaiters(waiters) - handles = fixture.handles.withLock { $0.count == waiters ? $0 : [] } - if !handles.isEmpty { - break - } - await Task.yield() - } - fixture.handles.withLock { $0.removeAll(keepingCapacity: true) } - handles.shuffle(using: &generator) - for handle in handles { - handle.cancel() - } - index = Chase.cycle[index] - } - } else { - for _ in 0 ..< rounds { - let child = Task { @Sendable in - try await fixture.semaphore.wait() - } - fixture.handles.withLock { $0.append(child) } - do { - try await child.value - } catch is CancellationError { - index = Chase.cycle[index] - } - } - } - return index - } - } - - /// Two tasks at the harness's priority hand the count back and forth - /// through a queue of `lows` waiters at low priority, which are served - /// only at a signal that finds neither of the two queued. Each handoff - /// to one of the two is what the queue does to place a high-priority - /// arrival among the low ones and to let it go, which is what would grow - /// with `lows` if the queue walked to find either. - /// - /// The low waiters are detached tasks of their own, since the harness - /// runs every worker at one priority. Each remaining worker spawns one - /// and returns without awaiting it — awaiting a task raises it to the - /// awaiter's priority, the runtime's doing — and a filler takes one turn - /// and spawns its successor rather than looping: a semaphore escalates - /// nobody, but the mutex suite's does, and the two measure alike. A - /// filler holds nothing it could chase over, so its worker chases before - /// it spawns. - /// - /// Once the two are done, each filler's next turn is its last, and the - /// last of them opens a gate the two wait at, so no filler outlives the - /// sample into the next. The drain — one turn per filler — is timed with - /// the rest, the same amount every sample. - private func measurePriorityHandoff(lows: Int, iterations: Int) { - final class Fixture: Sendable { - let semaphore = AsyncSemaphore(value: 1) - let finished = Mutex(0) - let fillers: Mutex - let drained = Gate() - - init(lows: Int) { - fillers = Mutex(lows) - } - } - /// One turn at low priority, then a fresh filler in its place, or one - /// fewer once the two high tasks are done. A filler is never reused: - /// where the OS escalates a holder, one that held while a high task - /// queued was raised to that task's priority and stays there, so its - /// next turn would be a high-priority arrival. - @Sendable func fill(_ fixture: Fixture) { - // The task is deliberately neither awaited nor kept: awaiting it - // would raise it to the awaiter's priority, and the fixture's - // flags are how the fillers end. - _ = Task.detached(priority: .low) { - try await fixture.semaphore.wait() - await Task.yield() - fixture.semaphore.signal() - if fixture.finished.withLock({ $0 < 2 }) { - fill(fixture) - } else if fixture.fillers.withLock({ $0 -= 1; return $0 == 0 }) { - fixture.drained.open() - } - } - } - measureTaskContention(tasks: lows + 2, iterations: iterations, makeFixture: { Fixture(lows: lows) }) { fixture, task in - var index = task - if task < 2 { - for _ in 0 ..< iterations { - try await fixture.semaphore.wait() - index = Chase.cycle[index] - await Task.yield() - fixture.semaphore.signal() - } - fixture.finished.withLock { $0 += 1 } - await fixture.drained.wait() - } else { - for _ in 0 ..< iterations { - index = Chase.cycle[index] - } - fill(fixture) - } - return index - } - } - - func testUncontended() { - measureHandoff(tasks: 1, iterations: 100_000) - } - - func testShortQueue() { - measureHandoff(tasks: 8, iterations: 10_000) - } - - func testLongQueue() { - measureHandoff(tasks: 64, iterations: 2_000) - } - - /// Past a few hundred waiters, what shows is the queue's own - /// bookkeeping: a handoff here costs what one in `testLongQueue` does, - /// and a regression that scales with the queue costs several times it. - func testVeryLongQueue() { - measureHandoff(tasks: 512, iterations: 250) - } - - /// The other thing a queue of this length has to do in constant time. - func testCancellationInLongQueue() { - measureCancellation(waiters: 512, rounds: 250) - } - - /// And the third: a high-priority arrival placed, and served, ahead of - /// a long queue at low priority. - func testHighPriorityAmongLowWaiters() { - measurePriorityHandoff(lows: 512, iterations: 10_000) - } - - // MARK: - Threads - - // The blocking `wait()`, contended by threads the way `Semaphore` is in - // its own suite. The two numbers are not the same measurement, and the - // gap between them — several times over — is not overhead in the queue. - // Measured, it is a context switch per handoff: this semaphore hands the - // count to the waiter at the head of the queue, so every signal moves - // the work to another thread, where `Semaphore` only raises the count, - // and the thread that just signalled takes it back before the one it - // woke has run. That barging is what its number is made of — a few - // thousand switches across a million handoffs, against one or two for - // each of them here — and giving it up is what the no-overtaking and - // priority guarantees cost. Folding the slow path's two critical - // sections into one was tried and moved nothing. - - func testContendedThreads() throws { - try skipUnlessRoomToContend() - measureContention(workers: contendedWorkers, iterations: 20_000, makeFixture: LockBox.init) { box, worker, share in - var index = worker - share.eachTurn { - box.semaphore.wait() - box.payload.writes &+= 1 - index = box.payload.cycle[index] - box.semaphore.signal() - } - return index - } check: { box in - XCTAssertEqual(box.payload.writes, self.contendedWorkers * 20_000, "the workload did not run") - } - } -} -#endif diff --git a/Tests/SynchronizationKitMutexTests/MutexPerformanceTests.swift b/Tests/SynchronizationKitMutexTests/MutexPerformanceTests.swift deleted file mode 100644 index c5c77da..0000000 --- a/Tests/SynchronizationKitMutexTests/MutexPerformanceTests.swift +++ /dev/null @@ -1,174 +0,0 @@ -// -// MutexPerformanceTests.swift -// SynchronizationKit -// -// Copyright (c) 2026 Kang Jaehong -// SPDX-License-Identifier: Apache-2.0 -// - -// Gated on Dispatch: the threads these tests drive are started and joined -// through it, and WASI has none. What runs there is the suite next door -// that needs no thread of its own. -#if canImport(Dispatch) -import Foundation -import SynchronizationKitTestUtils -import XCTest - -import SynchronizationKitMutex - -#if canImport(Darwin) -import Synchronization -#endif - -/// What `Mutex` costs — and on Apple platforms, where it is this package's -/// own, what it costs against the standard library's, which is what a client -/// on a new enough OS would otherwise use. -/// -/// The harness, and why it measures the way it does, is in -/// `Measurement.swift`; `RWLockPerformanceTests` says how to run these. On -/// Apple platforms each case runs twice, once per implementation, so the two -/// land in one report: the package's is an `os_unfair_lock` reached through a -/// raw-layout struct, the standard library's the same lock reached through -/// its own, and the numbers say what the reaching costs. Elsewhere the -/// package's `Mutex` is the standard library's, re-exported, so the -/// comparison would be of a thing with itself and only the first half runs — -/// as the cost of the lock every asynchronous primitive here keeps its -/// state under. -final class MutexPerformanceTests: XCTestCase { - /// A reference to hold the lock by; `RWLockPerformanceTests.LockBox` - /// says why. - final class LockBox: @unchecked Sendable { - let lock = SynchronizationKitMutex.Mutex(ChasePayload()) - } - - #if canImport(Darwin) - @available(macOS 15, iOS 18, tvOS 18, watchOS 11, visionOS 2, *) - final class StandardLockBox: @unchecked Sendable { - let lock = Synchronization.Mutex(ChasePayload()) - } - #endif - - override func setUpWithError() throws { - try skipUnlessMeasurable() - } - - #if canImport(Darwin) - static let standardLibraryNeedsNewerOS = "The standard library's Mutex needs a newer OS." - #endif - - // MARK: - Uncontended - - func testUncontended() { - measureUncontended(iterations: 500_000, makeFixture: LockBox.init) { box in - var index = 0 - for _ in 0 ..< 500_000 { - box.lock.withLock { - $0.writes &+= 1 - index = $0.cycle[index] - } - } - return index - } check: { box in - XCTAssertEqual(box.lock.withLock { $0.writes }, 500_000, "the workload did not run") - } - } - - #if canImport(Darwin) - func testUncontendedStandardLibrary() throws { - guard #available(macOS 15, iOS 18, tvOS 18, watchOS 11, visionOS 2, *) else { - throw XCTSkip(Self.standardLibraryNeedsNewerOS) - } - - measureUncontended(iterations: 500_000, makeFixture: StandardLockBox.init) { box in - var index = 0 - for _ in 0 ..< 500_000 { - box.lock.withLock { - $0.writes &+= 1 - index = $0.cycle[index] - } - } - return index - } check: { box in - XCTAssertEqual(box.lock.withLock { $0.writes }, 500_000, "the workload did not run") - } - } - #endif - - // MARK: - Contended - - // Every core but two, so the handoff is between threads that each have - // somewhere to run; and twice the cores, so most of the time the lock is - // handed to a thread that has to be woken for it. - - func testContended() throws { - try skipUnlessRoomToContend() - measureContended(workers: contendedWorkers, iterations: 50_000) - } - - #if canImport(Darwin) - func testContendedStandardLibrary() throws { - try skipUnlessRoomToContend() - guard #available(macOS 15, iOS 18, tvOS 18, watchOS 11, visionOS 2, *) else { - throw XCTSkip(Self.standardLibraryNeedsNewerOS) - } - measureContendedStandardLibrary(workers: contendedWorkers, iterations: 50_000) - } - - #endif - - func testOversubscribed() throws { - try skipUnlessRoomToContend() - measureContended(workers: ProcessInfo.processInfo.activeProcessorCount * 2, iterations: 10_000) - } - - #if canImport(Darwin) - func testOversubscribedStandardLibrary() throws { - try skipUnlessRoomToContend() - guard #available(macOS 15, iOS 18, tvOS 18, watchOS 11, visionOS 2, *) else { - throw XCTSkip(Self.standardLibraryNeedsNewerOS) - } - measureContendedStandardLibrary( - workers: ProcessInfo.processInfo.activeProcessorCount * 2, - iterations: 10_000 - ) - } - #endif - - private func measureContended(workers: Int, iterations: Int) { - measureContention(workers: workers, iterations: iterations, makeFixture: LockBox.init) { box, worker, share in - var index = worker - share.eachTurn { - box.lock.withLock { - $0.writes &+= 1 - index = $0.cycle[index] - } - } - return index - } check: { box in - XCTAssertEqual(box.lock.withLock { $0.writes }, workers * iterations, "the workload did not run") - } - } - - #if canImport(Darwin) - @available(macOS 15, iOS 18, tvOS 18, watchOS 11, visionOS 2, *) - private func measureContendedStandardLibrary(workers: Int, iterations: Int) { - measureContention( - workers: workers, - iterations: iterations, - makeFixture: StandardLockBox.init - ) { box, worker, share in - var index = worker - share.eachTurn { - box.lock.withLock { - $0.writes &+= 1 - index = $0.cycle[index] - } - } - return index - } check: { box in - XCTAssertEqual(box.lock.withLock { $0.writes }, workers * iterations, "the workload did not run") - } - } - #endif -} -#endif diff --git a/Tests/SynchronizationKitRWLockTests/RWLockPerformanceTests.swift b/Tests/SynchronizationKitRWLockTests/RWLockPerformanceTests.swift deleted file mode 100644 index 0eb3592..0000000 --- a/Tests/SynchronizationKitRWLockTests/RWLockPerformanceTests.swift +++ /dev/null @@ -1,576 +0,0 @@ -// -// RWLockPerformanceTests.swift -// SynchronizationKit -// -// Copyright (c) 2026 Kang Jaehong -// SPDX-License-Identifier: Apache-2.0 -// - -// Gated on Dispatch: the threads these tests drive are started and joined -// through it, and WASI has none. What runs there is the suite next door -// that needs no thread of its own. -#if canImport(Dispatch) -import Dispatch -import Foundation -import SynchronizationKitAtomic -import SynchronizationKitMutex -import SynchronizationKitTestUtils -import XCTest - -import SynchronizationKitRWLock - -#if canImport(Darwin) -import Darwin -#elseif canImport(Glibc) -import Glibc -#elseif canImport(Musl) -import Musl -#elseif canImport(Android) -import Android -#endif - -/// What `RWLock` costs, on whichever backend the running OS provides — and -/// what the three things a client would otherwise write cost on the same -/// turns: the platform's `pthread_rwlock_t`; a concurrent `DispatchQueue`, -/// reads in `sync` and writes behind a `.barrier`; and this package's -/// `Mutex`, which the README says to prefer until reads are frequent, writes -/// rare, and the read section long enough for parallel reading to pay. Each -/// case runs once per implementation, so the four land in one report. -/// -/// The harness, and why it measures the way it does, is in -/// `Measurement.swift`. Every case is skipped in a debug build, so an -/// ordinary `swift test` is untouched and no environment variable has to be -/// remembered. Measure in release: -/// -/// swift test -c release -Xswiftc -enable-testing \ -/// --filter RWLockPerformanceTests -/// -/// `-enable-testing` is what the rest of the target's `@testable` imports need -/// in release; it is not what ships. It makes internal symbols externally -/// visible, which costs the optimizer some of what it may assume, so read these -/// as a self-consistent series rather than as the cost of a release build. -/// -/// A critical section is one step of the chase, except in the `Section` -/// cases, which run the read-mostly mix again with every section 64, 256, -/// and 1024 steps long — from about a dictionary lookup to about a -/// microsecond of work. That is where a reader-writer lock is meant to pull -/// ahead of a mutex, and the series says at what length this one does. -/// -/// On glibc and bionic the package's `RWLock` is itself a `pthread_rwlock_t`, -/// configured writer-preferring, so the `Pthread` cases there measure the -/// default configuration against it rather than a different lock. Where no -/// platform module offers one — Windows — those cases are absent and the -/// other three comparisons run. -final class RWLockPerformanceTests: XCTestCase { - /// A reference to hold the lock by. - /// - /// Capturing a noncopyable value in a measured block currently fails to - /// compile — "copy of noncopyable typed value", reported by the compiler as - /// its own bug — so the block captures this instead and reaches the lock - /// through it. - final class LockBox: MeasuredReadWriteLock, @unchecked Sendable { - let lock = RWLock(ChasePayload()) - - func read(from index: Int, steps: Int) -> Int { - lock.withReadLock { $0.chase(from: index, steps: steps) } - } - - func write(from index: Int, steps: Int) -> Int { - lock.withWriteLock { - $0.writes &+= 1 - return $0.chase(from: index, steps: steps) - } - } - - var writes: Int { - lock.withReadLock { $0.writes } - } - } - -#if canImport(Darwin) || canImport(Glibc) || canImport(Musl) || canImport(Android) - /// The platform's reader-writer lock, in its default configuration, and - /// what it guards. Allocated rather than stored inline: a - /// `pthread_rwlock_t` must not move once initialized, and a pointer is - /// the plain way to promise that. `@safe`, with the pointer marked - /// `@unsafe`: every access to it is spelled out below, and it is what the - /// lock guards, so nothing else reaches it. - @safe - final class PthreadLockBox: MeasuredReadWriteLock, @unchecked Sendable { - @unsafe private let lock: UnsafeMutablePointer - private var payload = ChasePayload() - - init() { - unsafe lock = UnsafeMutablePointer.allocate(capacity: 1) - unsafe lock.initialize(to: pthread_rwlock_t()) - let result = unsafe pthread_rwlock_init(lock, nil) - precondition(result == 0, "pthread_rwlock_init failed") - } - - deinit { - unsafe pthread_rwlock_destroy(lock) - unsafe lock.deinitialize(count: 1) - unsafe lock.deallocate() - } - - func read(from index: Int, steps: Int) -> Int { - var result = unsafe pthread_rwlock_rdlock(lock) - precondition(result == 0, "pthread_rwlock_rdlock failed") - let end = payload.chase(from: index, steps: steps) - result = unsafe pthread_rwlock_unlock(lock) - precondition(result == 0, "pthread_rwlock_unlock failed") - return end - } - - func write(from index: Int, steps: Int) -> Int { - var result = unsafe pthread_rwlock_wrlock(lock) - precondition(result == 0, "pthread_rwlock_wrlock failed") - payload.writes &+= 1 - let end = payload.chase(from: index, steps: steps) - result = unsafe pthread_rwlock_unlock(lock) - precondition(result == 0, "pthread_rwlock_unlock failed") - return end - } - - var writes: Int { - var result = unsafe pthread_rwlock_rdlock(lock) - precondition(result == 0, "pthread_rwlock_rdlock failed") - let writes = payload.writes - result = unsafe pthread_rwlock_unlock(lock) - precondition(result == 0, "pthread_rwlock_unlock failed") - return writes - } - } -#endif - - /// The reader-writer pattern Dispatch offers: a concurrent queue, reads - /// submitted with `sync`, writes with `sync` and the `.barrier` flag. - final class QueueLockBox: MeasuredReadWriteLock, @unchecked Sendable { - private let queue = DispatchQueue( - label: "SynchronizationKitRWLockTests.QueueLockBox", - attributes: .concurrent - ) - private var payload = ChasePayload() - - func read(from index: Int, steps: Int) -> Int { - queue.sync { payload.chase(from: index, steps: steps) } - } - - func write(from index: Int, steps: Int) -> Int { - queue.sync(flags: .barrier) { - payload.writes &+= 1 - return payload.chase(from: index, steps: steps) - } - } - - var writes: Int { - queue.sync { payload.writes } - } - } - - /// This package's `Mutex`, taken for reads and writes alike. - final class MutexLockBox: MeasuredReadWriteLock, @unchecked Sendable { - let lock = Mutex(ChasePayload()) - - func read(from index: Int, steps: Int) -> Int { - lock.withLock { $0.chase(from: index, steps: steps) } - } - - func write(from index: Int, steps: Int) -> Int { - lock.withLock { - $0.writes &+= 1 - return $0.chase(from: index, steps: steps) - } - } - - var writes: Int { - lock.withLock { $0.writes } - } - } - - override func setUpWithError() throws { - try skipUnlessMeasurable() - } - - // MARK: - Harness - - /// Runs one reader and writer mix over a fresh lock, every section - /// `steps` long. Writers come first in the worker numbering. - private func measureContention( - _: Lock.Type, - readers: Int, - writers: Int, - iterations: Int, - steps: Int = 1 - ) { - measureContention( - groups: [ - (workers: writers, iterations: iterations), - (workers: readers, iterations: iterations), - ], - makeFixture: Lock.init - ) { lock, worker, share in - var index = worker - if worker < writers { - share.eachTurn(steps: steps) { - index = lock.write(from: index, steps: steps) - } - } else { - share.eachTurn(steps: steps) { - index = lock.read(from: index, steps: steps) - } - } - return index - } check: { lock in - XCTAssertEqual(lock.writes, writers * iterations, "the write side did not run") - } - } - - private func measureUncontended( - _: Lock.Type, - writing: Bool, - iterations: Int - ) { - measureUncontended(iterations: iterations, makeFixture: Lock.init) { lock in - var index = 0 - if writing { - for _ in 0 ..< iterations { - index = lock.write(from: index, steps: 1) - } - } else { - for _ in 0 ..< iterations { - index = lock.read(from: index, steps: 1) - } - } - return index - } check: { lock in - XCTAssertEqual(lock.writes, writing ? iterations : 0, "the workload did not run") - } - } - - // MARK: - Uncontended - - func testUncontendedReads() { - measureUncontended(LockBox.self, writing: false, iterations: 500_000) - } - -#if canImport(Darwin) || canImport(Glibc) || canImport(Musl) || canImport(Android) - func testUncontendedReadsPthread() { - measureUncontended(PthreadLockBox.self, writing: false, iterations: 500_000) - } -#endif - - func testUncontendedReadsDispatchQueue() { - measureUncontended(QueueLockBox.self, writing: false, iterations: 500_000) - } - - func testUncontendedReadsMutex() { - measureUncontended(MutexLockBox.self, writing: false, iterations: 500_000) - } - - func testUncontendedWrites() { - measureUncontended(LockBox.self, writing: true, iterations: 500_000) - } - -#if canImport(Darwin) || canImport(Glibc) || canImport(Musl) || canImport(Android) - func testUncontendedWritesPthread() { - measureUncontended(PthreadLockBox.self, writing: true, iterations: 500_000) - } -#endif - - func testUncontendedWritesDispatchQueue() { - measureUncontended(QueueLockBox.self, writing: true, iterations: 500_000) - } - - func testUncontendedWritesMutex() { - measureUncontended(MutexLockBox.self, writing: true, iterations: 500_000) - } - - // MARK: - Readers alone - - // Every worker reads and nobody writes: what a read costs when the lock - // is doing the one thing a reader-writer lock is for. The section is one - // step, so the number is the lock and nothing else. - - func testConcurrentReads() throws { - try skipUnlessRoomToContend() - measureContention(LockBox.self, readers: contendedWorkers, writers: 0, iterations: 50_000) - } - -#if canImport(Darwin) || canImport(Glibc) || canImport(Musl) || canImport(Android) - func testConcurrentReadsPthread() throws { - try skipUnlessRoomToContend() - measureContention( - PthreadLockBox.self, - readers: contendedWorkers, - writers: 0, - iterations: 50_000 - ) - } -#endif - - func testConcurrentReadsDispatchQueue() throws { - try skipUnlessRoomToContend() - measureContention( - QueueLockBox.self, - readers: contendedWorkers, - writers: 0, - iterations: 50_000 - ) - } - - func testConcurrentReadsMutex() throws { - try skipUnlessRoomToContend() - measureContention( - MutexLockBox.self, - readers: contendedWorkers, - writers: 0, - iterations: 50_000 - ) - } - - // MARK: - Read-mostly - - /// How many turns of a read-mostly worker are reads for each write. - static let readsPerWrite = 99 - - /// A lock and the count of writes its workers made, which is what the - /// count of writes it saw has to come to. Made afresh with the lock: the - /// harness runs a measured block several times over. - private final class ReadMostlyFixture: Sendable { - let lock = Lock() - let expectedWrites = Atomic(0) - } - - /// Runs `contendedWorkers` workers over a fresh lock, each writing once - /// in every `readsPerWrite + 1` turns and reading otherwise, every - /// section `steps` long. The writes are spread across the workers and - /// through the run rather than given to one worker to make flat out, so - /// no writer is forever queued behind the readers. - private func measureReadMostly( - _: Lock.Type, - steps: Int = 1, - iterations: Int = 50_000 - ) throws { - try skipUnlessRoomToContend() - measureContention( - workers: contendedWorkers, - iterations: iterations, - makeFixture: ReadMostlyFixture.init - ) { fixture, worker, share in - let lock = fixture.lock - var index = worker - var turn = 0 - var writes = 0 - share.eachTurn(steps: steps) { - turn += 1 - if turn % (Self.readsPerWrite + 1) == 0 { - index = lock.write(from: index, steps: steps) - writes += 1 - } else { - index = lock.read(from: index, steps: steps) - } - } - fixture.expectedWrites.wrappingAdd(writes, ordering: .relaxed) - return index - } check: { fixture in - XCTAssertEqual( - fixture.lock.writes, - fixture.expectedWrites.load(ordering: .relaxed), - "the write side did not run" - ) - } - } - - func testReadMostly() throws { - try measureReadMostly(LockBox.self) - } - -#if canImport(Darwin) || canImport(Glibc) || canImport(Musl) || canImport(Android) - func testReadMostlyPthread() throws { - try measureReadMostly(PthreadLockBox.self) - } -#endif - - func testReadMostlyDispatchQueue() throws { - try measureReadMostly(QueueLockBox.self) - } - - func testReadMostlyMutex() throws { - try measureReadMostly(MutexLockBox.self) - } - - // MARK: - The same, with sections long enough for reading in parallel to pay - - func testReadMostlySection64() throws { - try measureReadMostly(LockBox.self, steps: 64, iterations: 20_000) - } - -#if canImport(Darwin) || canImport(Glibc) || canImport(Musl) || canImport(Android) - func testReadMostlySection64Pthread() throws { - try measureReadMostly(PthreadLockBox.self, steps: 64, iterations: 20_000) - } -#endif - - func testReadMostlySection64DispatchQueue() throws { - try measureReadMostly(QueueLockBox.self, steps: 64, iterations: 20_000) - } - - func testReadMostlySection64Mutex() throws { - try measureReadMostly(MutexLockBox.self, steps: 64, iterations: 20_000) - } - - func testReadMostlySection256() throws { - try measureReadMostly(LockBox.self, steps: 256, iterations: 10_000) - } - -#if canImport(Darwin) || canImport(Glibc) || canImport(Musl) || canImport(Android) - func testReadMostlySection256Pthread() throws { - try measureReadMostly(PthreadLockBox.self, steps: 256, iterations: 10_000) - } -#endif - - func testReadMostlySection256DispatchQueue() throws { - try measureReadMostly(QueueLockBox.self, steps: 256, iterations: 10_000) - } - - func testReadMostlySection256Mutex() throws { - try measureReadMostly(MutexLockBox.self, steps: 256, iterations: 10_000) - } - - func testReadMostlySection1024() throws { - try measureReadMostly(LockBox.self, steps: 1024, iterations: 4_000) - } - -#if canImport(Darwin) || canImport(Glibc) || canImport(Musl) || canImport(Android) - func testReadMostlySection1024Pthread() throws { - try measureReadMostly(PthreadLockBox.self, steps: 1024, iterations: 4_000) - } -#endif - - func testReadMostlySection1024DispatchQueue() throws { - try measureReadMostly(QueueLockBox.self, steps: 1024, iterations: 4_000) - } - - func testReadMostlySection1024Mutex() throws { - try measureReadMostly(MutexLockBox.self, steps: 1024, iterations: 4_000) - } - - // MARK: - A writer releasing a crowd - - // Deliberately more readers than cores, so a departing writer has a queue to - // hand the lock to. This is where releasing every one of them at once tells - // against one system call each. - - func testWriterHandoff() throws { - try skipUnlessRoomToContend() - measureContention( - LockBox.self, - readers: ProcessInfo.processInfo.activeProcessorCount * 2, - writers: 1, - iterations: 4_000 - ) - } - -#if canImport(Darwin) || canImport(Glibc) || canImport(Musl) || canImport(Android) - func testWriterHandoffPthread() throws { - try skipUnlessRoomToContend() - measureContention( - PthreadLockBox.self, - readers: ProcessInfo.processInfo.activeProcessorCount * 2, - writers: 1, - iterations: 4_000 - ) - } -#endif - - func testWriterHandoffDispatchQueue() throws { - try skipUnlessRoomToContend() - measureContention( - QueueLockBox.self, - readers: ProcessInfo.processInfo.activeProcessorCount * 2, - writers: 1, - iterations: 4_000 - ) - } - - func testWriterHandoffMutex() throws { - try skipUnlessRoomToContend() - measureContention( - MutexLockBox.self, - readers: ProcessInfo.processInfo.activeProcessorCount * 2, - writers: 1, - iterations: 4_000 - ) - } - - // MARK: - Mixed - - func testMixedContention() throws { - try skipUnlessRoomToContend() - measureContention(LockBox.self, readers: contendedWorkers, writers: 4, iterations: 20_000) - } - -#if canImport(Darwin) || canImport(Glibc) || canImport(Musl) || canImport(Android) - func testMixedContentionPthread() throws { - try skipUnlessRoomToContend() - measureContention( - PthreadLockBox.self, - readers: contendedWorkers, - writers: 4, - iterations: 20_000 - ) - } -#endif - - func testMixedContentionDispatchQueue() throws { - try skipUnlessRoomToContend() - measureContention( - QueueLockBox.self, - readers: contendedWorkers, - writers: 4, - iterations: 20_000 - ) - } - - func testMixedContentionMutex() throws { - try skipUnlessRoomToContend() - measureContention( - MutexLockBox.self, - readers: contendedWorkers, - writers: 4, - iterations: 20_000 - ) - } -} - -/// A lock the harness above can measure as a reader-writer lock: a chase -/// under the read lock, and a chase and a write under the write lock, each -/// returning where the chase ended. One protocol so that one harness serves -/// every implementation; it is private to this file and every adopter is -/// final, so the harness is specialized for each and no measured turn is -/// dispatched through it. -private protocol MeasuredReadWriteLock: Sendable { - init() - - /// `steps` steps of the chase from `index` under the read lock. - func read(from index: Int, steps: Int) -> Int - - /// One write and `steps` steps of the chase from `index` under the write - /// lock. - func write(from index: Int, steps: Int) -> Int - - /// How many writes have happened, read under the lock. - var writes: Int { get } -} - -extension ChasePayload { - /// `steps` steps of the chase from `index`, and where it ended. - fileprivate func chase(from index: Int, steps: Int) -> Int { - var index = index - for _ in 0 ..< steps { - index = cycle[index] - } - return index - } -} -#endif diff --git a/Tests/SynchronizationKitSemaphoreTests/SemaphorePerformanceTests.swift b/Tests/SynchronizationKitSemaphoreTests/SemaphorePerformanceTests.swift deleted file mode 100644 index 4aa6091..0000000 --- a/Tests/SynchronizationKitSemaphoreTests/SemaphorePerformanceTests.swift +++ /dev/null @@ -1,117 +0,0 @@ -// -// SemaphorePerformanceTests.swift -// SynchronizationKit -// -// Copyright (c) 2026 Kang Jaehong -// SPDX-License-Identifier: Apache-2.0 -// - -// Gated on Dispatch: the threads these tests drive are started and joined -// through it, and WASI has none. What runs there is the suite next door -// that needs no thread of its own. -#if canImport(Dispatch) -import Dispatch -import Foundation -import SynchronizationKitSemaphore -import SynchronizationKitTestUtils -import XCTest - -/// What `Semaphore` costs against `DispatchSemaphore`, which is what a client -/// would otherwise reach for, on whichever backend the running OS provides. -/// -/// The harness, and why it measures the way it does, is in -/// `Measurement.swift`; `RWLockPerformanceTests` says how to run these. The -/// semaphore is used as a lock — a count of one, taken around the chase — so -/// that the same harness applies and the numbers sit beside the locks'. Each -/// case runs twice, once per implementation. -final class SemaphorePerformanceTests: XCTestCase { - /// The semaphore and what it guards, held by reference; - /// `RWLockPerformanceTests.LockBox` says why. - final class LockBox: @unchecked Sendable { - let semaphore = Semaphore(value: 1) - var payload = ChasePayload() - } - - final class DispatchLockBox: @unchecked Sendable { - let semaphore = DispatchSemaphore(value: 1) - var payload = ChasePayload() - } - - override func setUpWithError() throws { - try skipUnlessMeasurable() - } - - // MARK: - Uncontended - - func testUncontended() { - measureUncontended(iterations: 500_000, makeFixture: LockBox.init) { box in - var index = 0 - for _ in 0 ..< 500_000 { - box.semaphore.wait() - box.payload.writes &+= 1 - index = box.payload.cycle[index] - box.semaphore.signal() - } - return index - } check: { box in - XCTAssertEqual(box.payload.writes, 500_000, "the workload did not run") - } - } - - func testUncontendedDispatch() { - measureUncontended(iterations: 500_000, makeFixture: DispatchLockBox.init) { box in - var index = 0 - for _ in 0 ..< 500_000 { - box.semaphore.wait() - box.payload.writes &+= 1 - index = box.payload.cycle[index] - box.semaphore.signal() - } - return index - } check: { box in - XCTAssertEqual(box.payload.writes, 500_000, "the workload did not run") - } - } - - // MARK: - Contended - - // A semaphore has no fast path past the kernel once anybody is waiting, - // so this is the handoff itself: every release wakes somebody. - - func testContended() throws { - try skipUnlessRoomToContend() - measureContention(workers: contendedWorkers, iterations: 20_000, makeFixture: LockBox.init) { box, worker, share in - var index = worker - share.eachTurn { - box.semaphore.wait() - box.payload.writes &+= 1 - index = box.payload.cycle[index] - box.semaphore.signal() - } - return index - } check: { box in - XCTAssertEqual(box.payload.writes, self.contendedWorkers * 20_000, "the workload did not run") - } - } - - func testContendedDispatch() throws { - try skipUnlessRoomToContend() - measureContention( - workers: contendedWorkers, - iterations: 20_000, - makeFixture: DispatchLockBox.init - ) { box, worker, share in - var index = worker - share.eachTurn { - box.semaphore.wait() - box.payload.writes &+= 1 - index = box.payload.cycle[index] - box.semaphore.signal() - } - return index - } check: { box in - XCTAssertEqual(box.payload.writes, self.contendedWorkers * 20_000, "the workload did not run") - } - } -} -#endif From 29031754a71896ec386c21c96449783e1923106c Mon Sep 17 00:00:00 2001 From: Kang Jaehong Date: Sat, 10 Oct 2026 06:41:13 +0900 Subject: [PATCH 3/3] Point the README at the benchmark package and its numbers Running the tests no longer measures anything, so the README's account of how to measure moves to a section of its own: how to run the benchmark package, how to keep a baseline and compare against it, and why the allocation and retain counts are a pass of their own. The performance tables are measured again by the benchmark package, as the median of seven runs of the whole suite, each run's figure the median of its samples, where the old ones were XCTest's means over seven runs. The prose beside them is checked against the new figures. Two sentences move with them: a turn among one task in eight writing costs what it now measures, and readers at once cost no more than a `Mutex`, where they measured as costing less under the old harness, whose contended `Mutex` ran half again slower than it does with its threads kept warm. --- README.md | 94 ++++++++++++++++++++++++++++++++++++------------------- 1 file changed, 61 insertions(+), 33 deletions(-) diff --git a/README.md b/README.md index 50815e1..141ae45 100644 --- a/README.md +++ b/README.md @@ -119,28 +119,29 @@ are documented on the types themselves. ## Performance -Measured on an Apple M4 Pro, macOS 26.6.2, Swift 6.4, at v1.1.2, by the -performance suites described under [Running the tests](#running-the-tests), +Measured on an Apple M4 Pro, macOS 26.7.1, Swift 6.4, at v1.1.2, by the +benchmarks described under [Running the benchmarks](#running-the-benchmarks), built as the library ships; contended cases run twelve threads. Each figure -is the mean of seven runs; read them as a comparison on one machine. +is the median of seven runs, each run's figure the median of its samples; +read them as a comparison on one machine. | | This package (ns/op) | Alternative (ns/op) | | --- | --- | --- | -| `Mutex`, uncontended / contended | 1.7 / 8.0 | Standard library `Mutex`: 1.7 / 7.8 | -| `Semaphore`, uncontended / contended handoff | 3.4 / 702 | `DispatchSemaphore`: 3.4 / 1,720 | -| `RWLock`, uncontended read / write | 3.0 / 5.4 | `pthread_rwlock_t`: 5.5 / 5.5 · concurrent `DispatchQueue`: 162 / 162 | -| `RWLock`, read across twelve threads | 4.4 | `pthread_rwlock_t`: 458 · concurrent `DispatchQueue`: 1,420 · `Mutex`: 8.1 | -| `AsyncMutex`, uncontended / handoff | 603 / 1,920 | `actor`: 369 / 461 | -| `AsyncRWLock`, uncontended read / write / writer handoff | 708 / 598 / 2,070 | | -| `AsyncSemaphore`, uncontended / handoff | 381 / 1,380 | | +| `Mutex`, uncontended / contended | 1.7 / 5.1 | Standard library `Mutex`: 1.7 / 5.1 | +| `Semaphore`, uncontended / contended handoff | 3.5 / 709 | `DispatchSemaphore`: 3.5 / 1,750 | +| `RWLock`, uncontended read / write | 3.2 / 5.5 | `pthread_rwlock_t`: 5.7 / 6.0 · concurrent `DispatchQueue`: 207 / 204 | +| `RWLock`, read across twelve threads | 4.8 | `pthread_rwlock_t`: 317 · concurrent `DispatchQueue`: 1,210 · `Mutex`: 5.0 | +| `AsyncMutex`, uncontended / handoff | 616 / 1,990 | `actor`: 394 / 479 | +| `AsyncRWLock`, uncontended read / write / writer handoff | 734 / 626 / 2,090 | | +| `AsyncSemaphore`, uncontended / handoff | 406 / 1,490 | | A turn of the asynchronous primitives is a take, a `Task.yield()`, and a release; a handoff resumes the next waiter across threads of the cooperative -pool, and costs the same at 8, 64, and 512 waiters. The actor is cheaper on +pool, and costs the same at 8, 64, and 500 waiters. The actor is cheaper on every count because it cannot hold across the yield. What `AsyncMutex` buys is holding across an `await`, and this is its price. `AsyncRWLock`'s writers hand off as `AsyncMutex` does; with one turn in eight a write and the rest reads, a -turn costs 2,180 ns among 8 tasks and 3,820 among 64. +turn costs 2,340 ns among 8 tasks and 4,050 among 64. `RWLock` against the alternatives on a read-mostly mix — twelve threads, each writing once in a hundred turns — as the critical section grows, in @@ -148,13 +149,13 @@ nanoseconds per turn: | Critical section | `RWLock` | `Mutex` | `pthread_rwlock_t` | `DispatchQueue` + barrier | | --- | --- | --- | --- | --- | -| ~1 ns | 37 | 8.9 | 418 | 1,610 | -| ~70 ns | 82 | 125 | 569 | 1,660 | -| ~300 ns | 145 | 426 | 588 | 1,730 | -| ~1.1 µs | 272 | 1,380 | 516 | 1,750 | +| ~1 ns | 30 | 5.7 | 360 | 1,620 | +| ~70 ns | 85 | 122 | 507 | 1,640 | +| ~300 ns | 159 | 432 | 592 | 1,650 | +| ~1.1 µs | 279 | 1,400 | 514 | 1,720 | Readers touch nothing in common while no writer is about, so twelve threads -reading at once cost each of them less than a `Mutex` would; what they pay +reading at once cost each of them no more than a `Mutex` would; what they pay here is the write in every hundred turns, which turns that off for a spell and is what a `Mutex` still wins at the shortest section. By a few dozen nanoseconds of reading `RWLock` is ahead, at a few hundred it takes a third @@ -216,19 +217,8 @@ the Mach semaphore path is exercised only on a simulator runtime older than the versions in the table above, which the Apple Platforms workflow pins one of for that reason. -A debug build skips the measurements, since an unoptimized one says nothing. -There is one performance suite per primitive, each measured beside what a -client would otherwise write: `Mutex` beside the standard library's, -`Semaphore` beside `DispatchSemaphore`, `RWLock` beside `pthread_rwlock_t`, a -concurrent `DispatchQueue`, and `Mutex`, and `AsyncMutex` beside an `actor`. -Read the numbers; nothing there fails on a regression. - -```sh -swift test -c release --disable-testable-imports --filter PerformanceTests -``` - -Every primitive also has a stress suite. A plain run takes it at a size that -does not slow a local run; the flag below turns the repetition up fiftyfold, +Every primitive has a stress suite. A plain run takes it at a size that does +not slow a local run; the flag below turns the repetition up fiftyfold, which is how CI runs it on every push. ```sh @@ -237,9 +227,47 @@ swift test -c release --disable-testable-imports \ ``` CI runs the unit tests in debug, where `@testable import` reaches the -internals they check, and the stress suites and the measurements in release, -built as the library ships, since a lock's bugs are the ones the optimizer -creates. +internals they check, and the stress suites in release, built as the library +ships, since a lock's bugs are the ones the optimizer creates. + +### Running the benchmarks + +The benchmarks are a package of their own under `Benchmarks`, so that +nothing a client resolves includes the harness. There is one set of cases +per primitive, each measured beside what a client would otherwise write: +`Mutex` beside the standard library's, `Semaphore` beside +`DispatchSemaphore`, `RWLock` beside `pthread_rwlock_t`, a concurrent +`DispatchQueue`, and `Mutex`, and `AsyncMutex` beside an `actor`. The +harness runs on macOS and Linux, builds the library as it ships, and reports +percentiles of the wall clock and instructions per turn, with CPU time and +context switches under contention. Read the numbers; nothing there fails on +a regression. + +```sh +cd Benchmarks +swift package benchmark +``` + +What a primitive allocates and retains per turn is a pass of its own, +since the harness counts them by hooking every allocation, retain, and +release, which slows the paths that make them: + +```sh +swift package benchmark --metric mallocCountTotal \ + --metric retainCount --metric releaseCount +``` + +To see what a change did, keep a baseline before it and compare after: + +```sh +swift package --allow-writing-to-directory .benchmarkBaselines/ \ + benchmark baseline update before +swift package benchmark baseline compare before +``` + +`--filter` takes a regular expression over the case names, which +`swift package benchmark list` prints. CI runs the lot on one macOS and one +Linux row per Swift minor and puts the table in the job's summary. ## Contributing