UTS: align AWAIT_STATE, negative-assertion draining and features-spec revision guidance

uts/README.md

@@ -141,8 +141,9 @@ Pseudocode maps to language idioms rather than prescribing exact syntax:
   unit tier before asserting, especially for negative assertions ("nothing happened"). Involves
   no real delay and is never rendered as a timed sleep; implementations define a shared helper
   using the platform's zero-delay yield (e.g. `flushAsync()` awaiting a `setImmediate` in
-  ably-js, or `runCurrent()` on a kotlinx-coroutines test scheduler) — see the timer guidance
-  in [docs/writing-derived-tests.md](docs/writing-derived-tests.md).
+  ably-js, or `runCurrent()` on a kotlinx-coroutines test scheduler), repeated or replaced by an
+  explicit queue drain where the SDK chains work across queues, threads or loop iterations — see
+  the timer guidance in [docs/writing-derived-tests.md](docs/writing-derived-tests.md).
 - **Language-inapplicable inputs**: a test input that cannot be constructed in a given language
   (e.g. a non-string map key in JavaScript, where object keys are always coerced to strings; or a
   `null` argument where the SDK's signature makes null indistinguishable from "omitted") makes

uts/docs/writing-derived-tests.md

@@ -77,13 +77,13 @@ A test failure has exactly three possible causes. Work through them in order:
 
 #### 2a. Is the UTS spec wrong?
 
-Compare the UTS spec's claim against the relevant Ably **features spec** — the ultimate authority. Every spec lives under the repo's [`specifications/`](https://github.com/ably/specification/tree/main/specifications) directory; the raw, fetchable base is `https://raw.githubusercontent.com/ably/specification/refs/heads/main/specifications/`. Fetch the one for the module under test:
+Compare the UTS spec's claim against the relevant Ably **features spec** — the ultimate authority. Every spec lives under the repo's [`specifications/`](https://github.com/ably/specification/tree/main/specifications) directory. **Read it from the same revision as the UTS spec you are translating**: from the same local clone, or from GitHub at that commit (`https://raw.githubusercontent.com/ably/specification/<sha>/specifications/`). If the UTS spec came from `main`, resolve `main` to its SHA once and read both from that SHA. A UTS spec and the features spec it is judged against must come from one revision, because features-spec and UTS changes often land in the same commit; comparing against a different revision can misclassify a spec error. The features spec for each module (links point at `main` for browsing; substitute the SHA when reading):
 
 - Core / Realtime / REST — [`features.md`](https://raw.githubusercontent.com/ably/specification/refs/heads/main/specifications/features.md)
 - LiveObjects (`objects`) — [`objects-features.md`](https://raw.githubusercontent.com/ably/specification/refs/heads/main/specifications/objects-features.md)
 - Chat — [`chat-features.md`](https://raw.githubusercontent.com/ably/specification/refs/heads/main/specifications/chat-features.md)
 
-Related authorities in the **same** `specifications/` directory back specific areas — e.g. [`protocol.md`](https://raw.githubusercontent.com/ably/specification/refs/heads/main/specifications/protocol.md) (wire protocol) and [`encryption.md`](https://raw.githubusercontent.com/ably/specification/refs/heads/main/specifications/encryption.md) — fetch them too when a test touches that area.
+Related authorities in the **same** `specifications/` directory back specific areas — e.g. [`protocol.md`](https://raw.githubusercontent.com/ably/specification/refs/heads/main/specifications/protocol.md) (wire protocol) and [`encryption.md`](https://raw.githubusercontent.com/ably/specification/refs/heads/main/specifications/encryption.md) — read them too, from the same revision, when a test touches that area.
 
 A UTS spec can be wrong two ways: it **contradicts the features spec**, or it is **internally inconsistent** (e.g. a replayed serial that doesn't match the value its own harness produces, or a test title that names a state its body never injects). Either way the **spec is the source of truth and must be fixed there** — do **not** silently rewrite the test to match the features spec and move on. Quietly "fixing the test" hides the spec bug, leaves the spec wrong for every other SDK to re-hit, and defeats the point of a single source of truth. Instead:
 
@@ -297,7 +297,7 @@ Unit tests must not use real timers (`setTimeout`, `setInterval`, `sleep`, `dela
 
 - **For time-dependent SDK behaviour** (timeouts, retries, heartbeats): use fake timers that replace the SDK's timer API and can be advanced deterministically.
 - **For waiting on async event delivery** (mock message propagation, promise settlement): yield to the event loop with a zero-delay mechanism like `setImmediate`, `process.nextTick`, or equivalent. Define a `flushAsync()` helper and use it everywhere instead of `setTimeout(resolve, N)`. This is the rendering of the spec's `process_pending_events()` convention (see `uts/README.md`).
-- **For "prove a negative" assertions** (confirming something did NOT happen): a single event-loop yield is sufficient — if the event hasn't fired after one pass through the macrotask queue, it won't fire from the current stimulus.
+- **For "prove a negative" assertions** (confirming something did NOT happen): call `process_pending_events()` to drain all already-queued work, then assert. In a single-queue runtime (JavaScript) one event-loop yield usually does this, provided the SDK does not re-queue work from within a callback — if the event hasn't fired after one pass through the macrotask queue, it won't fire from the current stimulus. Where the SDK chains work across queues, threads or loop iterations, one yield is not enough and gives false passes: drain until the SDK's internal work is quiescent (e.g. loop until its queue is empty), or use a control-signal quiescence check as in [`objects/helpers/standard_test_pool.md`](../objects/helpers/standard_test_pool.md#negative-assertion-quiescence) "Negative-assertion quiescence".
 
 The only acceptable use of a real timer is a **safety timeout on test execution** — a long deadline (e.g. 5 seconds) that fails the test if an expected event never arrives, preventing the test from hanging indefinitely. This is a test-level safeguard, not a delay mechanism.
 

uts/docs/writing-test-specs.md

@@ -650,9 +650,11 @@ AWAIT_STATE client.connection.state == ConnectionState.connecting
 ```
 
 This means implementations should:
-- Check if condition is already true -> proceed
-- Otherwise wait for state change events with timeout
-- Fail if timeout expires
+- Subscribe to state change events first, then check whether the condition is already true,
+  and proceed on whichever happens first, exactly once (checking before subscribing leaves a
+  window in which a transition is missed)
+- Otherwise wait for a state change event that satisfies the condition, with a timeout
+- Fail if the timeout expires, and remove the listener on success, timeout or cancellation
 
 In **integration and proxy** specs these timeouts are **wall-clock time** (the test waits on a
 real server — see the *Timeout Strategy* section in `docs/integration-testing.md`); in unit specs they may be
@@ -841,7 +843,8 @@ wall-clock* in `writing-derived-tests.md` for the traps on both sides).
 
 For a **negative assertion** at unit tier — proving something did *not* happen — there is
 nothing to poll for: use `process_pending_events()` (see the pseudocode conventions in
-`uts/README.md`) to let already-queued events settle, then assert. Never a fixed `WAIT`.
+`uts/README.md`) to drain all already-queued work, then assert (see *No real timers in unit
+tests* in `writing-derived-tests.md` for SDKs that chain work across queues). Never a fixed `WAIT`.
 
 ```pseudo
 # Good - negative assertion: settle pending events, then assert nothing happened

uts/docs/writing-uts-spec-translator-skills.md

@@ -190,7 +190,7 @@ Where the SDK already has a test-options type, add the hooks to it, and implemen
 Most flakiness in both existing SDKs came from concurrency, not from translation. Before writing the skill, know:
 
 - Which thread or queue SDK callbacks fire on, and whether listeners can run concurrently with the test body.
-- How to drain the SDK's internal work queue (this becomes `process_pending_events()`). One zero-delay yield (e.g. one `await asyncio.sleep(0)`) may not drain chained callbacks; loop until the queue is empty if you can observe it. *Divergence from `writing-derived-tests.md`* ("No real timers in unit tests"), which says one event-loop yield suffices for a negative assertion: that holds for a single JavaScript macrotask queue, but not where the SDK chains work across queues, threads or loop iterations (both existing SDKs needed explicit queue drains). Use one yield only where you have shown it drains everything.
+- How to drain the SDK's internal work queue (this becomes `process_pending_events()`). One zero-delay yield (e.g. one `await asyncio.sleep(0)`) may not drain chained callbacks; loop until the queue is empty if you can observe it (`writing-derived-tests.md` "No real timers in unit tests"). One yield is usually enough for a single JavaScript macrotask queue (unless the SDK re-queues work from within a callback), but not where the SDK chains work across queues, threads or loop iterations (both existing SDKs needed explicit queue drains). Use one yield only where you have shown it drains everything.
 - Whether the test runner virtualises time (kotlinx `runTest` does; this breaks real-network timeouts), and, for async runners, the plugin mode and event-loop scope that suite-level fixtures need (e.g. pytest-asyncio).
 - **Whether the runner runs tests in parallel** (xUnit does by default; pytest-xdist and NUnit `[Parallelizable]` when enabled; Swift Testing unless `.serialized`). If any injected mock, fake clock or log sink is process-global, serialise the UTS suites or make the hooks per-client, and say which in the skill. ably-cocoa marks its core realtime/rest and integration suites `@Suite(.serialized)`; its objects unit suites, which inject nothing process-global, are plain parallel `struct`s.
 - Whether the language enforces concurrency safety at compile time (Swift 6 `Sendable` drove ably-cocoa's capture types).
@@ -561,7 +561,7 @@ Every rule below is a MUST unless marked otherwise.
 
 ### 5.7 Waits must catch events
 
-- **`AWAIT_STATE` must be event-latched and race-free**: subscribe first, then check the current state, and complete exactly once (whichever of the listener and the immediate check wins). Checking before subscribing leaves a window in which the transition is missed. ably-java's `awaitState` does this, with an atomic single-winner resume. *Divergence from `writing-test-specs.md`* ("State Transitions" lists check, then wait): the observable semantics are the same, but a check-then-subscribe implementation misses a transition that happens between the two calls, so subscribe first. The helper also removes its listener on success, timeout and cancellation; leaked listeners keep firing into later tests.
+- **`AWAIT_STATE` must be event-latched and race-free**: subscribe first, then check the current state, and complete exactly once (whichever of the listener and the immediate check wins), as `writing-test-specs.md` "State Transitions" requires. Checking before subscribing leaves a window in which the transition is missed. ably-java's `awaitState` does this, with an atomic single-winner resume. The helper also removes its listener on success, timeout and cancellation; leaked listeners keep firing into later tests.
 - **A state wait fails fast on FAILED.** If the connection enters FAILED while waiting for another state, fail immediately with the error reason instead of running out the timeout (ably-cocoa's integration `awaitState`). Don't treat a state that is already FAILED at the start of the wait as a failure when the test has just requested a transition out of it (`connect()`, or `authorize()` from FAILED per RTC8c); latch the next transition instead.
 - **Transient states use record-and-verify** (`writing-test-specs.md` "Verifying Transient States"): record all state changes, let the cycle finish, then assert `CONTAINS_IN_ORDER`. Even a correct latch misses a state entered and left before the call. Don't insert an intermediate `AWAIT_STATE disconnected`.
 - **Polls return the settled value.** For a value-assigned `x = poll_until(…)`, the helper returns the value and later assertions use it. Don't re-read afterwards: an eventually-consistent store can return fewer items on the second read.
@@ -603,7 +603,7 @@ Fill in the last column for your language; the skill MUST map every construct it
 |---|---|---|---|---|
 | `AWAIT expr` | Await an async call | `try await op`, or a continuation-bridged helper for callback APIs | `.await()` on a future, or `suspendCancellableCoroutine` around a callback | |
 | `f = op()` … later `AWAIT f` (incl. `AWAIT f FAILS WITH error`) | Deferred future: start an operation, act, then await it | `let t = Task { try await op() }` … `try await t.value` (objects module notes) | Hold the `CompletableFuture` (``val getFuture = channel.`object`.get()``) … `getFuture.await()` (generated tests, e.g. `RealtimeObjectTest`) | |
-| `AWAIT_STATE x.state == ConnectionState.connected` (also `ChannelState.attached`, bare `CONNECTED`) | Proceed if already true, otherwise wait for the state event; fail on timeout (`writing-test-specs.md` "State Transitions") | `awaitConnectionState(client, .connected)` (unit); `await awaitState(client, .connected)` (integration) | `awaitState(client, ConnectionState.connected)` | |
+| `AWAIT_STATE x.state == ConnectionState.connected` (also `ChannelState.attached`, bare `CONNECTED`) | Subscribe, then proceed if already true, otherwise wait for the state event; fail on timeout (`writing-test-specs.md` "State Transitions"; [5.7](#57-waits-must-catch-events)) | `awaitConnectionState(client, .connected)` (unit); `await awaitState(client, .connected)` (integration) | `awaitState(client, ConnectionState.connected)` | |
 | `… WITH timeout: N seconds` (`Ns`, `N second`) | Bound on the wait; wall-clock at integration tier | `timeout:` parameter on helpers; no per-call timeout on native `await` (note it in a comment) | `awaitState(client, state, 10.seconds)`; `withRealTimeout` for integration futures | |
 | `AWAIT UNTIL <cond>`, `WAIT_FOR <cond>` *(undocumented)* | Wait until the condition holds | — | — | |
 | `AWAIT_ERROR future` *(undocumented)* | Await a future expected to fail; bind its error | — | — | |
@@ -897,11 +897,11 @@ Examples of the pattern:
 Pick **one** source for docs and specs, and pin it. This guide recommends **reading everything from the local spec clone** the module directory came from:
 
 - the UTS docs (`<module-dir>/../docs/writing-derived-tests.md`, `proxy.md`);
-- in evaluate mode, the features specs the decision tree needs (`<module-dir>/../../specifications/features.md`, `objects-features.md`, `protocol.md`), not the GitHub URLs in `writing-derived-tests.md`.
+- in evaluate mode, the features specs the decision tree needs (`<module-dir>/../../specifications/features.md`, `objects-features.md`, `protocol.md`), rather than fetching them from GitHub.
 
 Record the clone's HEAD SHA, whether it is dirty and which files are; a translated spec that is locally modified is handled as in 9.1. You MAY warn when the clone is behind `origin/main`.
 
-- *Divergence from `writing-derived-tests.md`* (Phase 2, "2a. Is the UTS spec wrong?", which links the features specs on GitHub `main`): read them from the same clone as the UTS specs instead, because the UTS spec and the features spec it is judged against must come from one revision, and the clone's SHA is recorded in every header and report (9.1). Fetching `main` gives a revision that nothing records.
+- Read the features specs from the same clone as the UTS specs: the UTS spec and the features spec it is judged against must come from one revision (`writing-derived-tests.md` Phase 2, "2a. Is the UTS spec wrong?"), and the clone's SHA is recorded in every header and report (9.1). Fetching `main` gives a revision that nothing records.
 - *Divergence (existing skills):* both existing skills fetch `writing-derived-tests.md` from GitHub `main`, and tell the model to "fetch" the features spec with no pinned source, while reading specs from the local clone, so the two can skew.
 
 ---