Skip to content

Add Ping/Pong to the websocket protocol, with network stats in the TypeScript, Rust and C# SDKs - #6087

Open
krisajenkins wants to merge 8 commits into
masterfrom
websocket-ping-pong-rtt
Open

krisajenkins wants to merge 8 commits into
masterfrom
websocket-ping-pong-rtt

Conversation

@krisajenkins

@krisajenkins krisajenkins commented Oct 6, 2026 •

Copy link
Copy Markdown
Contributor

Description of Changes

Adds Ping/Pong messages to the v2 websocket protocol, and round-trip time, jitter and server-clock estimates to the TypeScript, Rust and C# SDKs.

Part of #5926. The Unreal SDK will follow in a separate PR, because it builds on #6318 (Unreal automatic reconnect).

Clients have no way to measure how far away the server is, or what time it is there. Games want the round-trip time to tune input prediction and show a latency readout, and they want the server's clock to schedule things against reducer timestamps. The server already sends websocket Ping frames, but browsers hide control frames from JavaScript (#5525) and none of our SDKs surface them, so this needs to be a protocol message. It can't be an ordinary data message either: the server handles a client's messages one at a time, so a ping queued behind an expensive Subscribe would report the query's evaluation time as network latency.

Protocol and server. ClientMessage::Ping { request_id, client_send_time } is answered with ServerMessage::Pong { request_id, client_send_time, server_receive_time, server_hold_duration }. A new stage in the websocket actor, ws_answer_pings, answers every Ping as soon as it is read off the socket, ahead of earlier messages still being processed, through the unordered channel. That covers Pings batched with other messages in a v3 payload as well as lone ones, so the hold duration always covers the Ping's whole time in the server. A payload holding only Pings is dropped once they're answered. Any other payload passes through to the handler, which now ignores Pings. A Ping is a latency probe, not a barrier: its Pong can overtake replies to messages sent before it.

SDKs. Each SDK pings on InitialConnection and every second after (configurable, zero disables), and timestamps Pongs where they come off the socket, so app-side queueing isn't counted as network time. The maths is shared: RFC 6298 smoothed RTT and jitter, the minimum RTT over the last 10 samples, a server-clock estimate taken from the lowest-RTT sample that slews at most 5% and never goes backwards, and pongWait, the age of the oldest unanswered Ping. The stats are absent until the first Pong, rather than falling back to the local clock. A one-off ping() resolves with its own timings.

Stats One-off ping Builder
TypeScript conn.networkStats conn.ping() withPingInterval, withPingTimeout
Rust conn.network_stats() conn.ping() with_ping_interval, with_ping_timeout (browser only)
C# conn.stats.Network conn.Ping() WithPingInterval, WithPingTimeout (WebGL only)

Dead-connection detection. In a browser, a network that fails silently leaves the websocket open with nothing arriving, and automatic reconnect waits for a close that may not come for minutes. In the TypeScript SDK, the Rust browser build and C# under WebGL, the connection is now treated as lost once at least two Pings are unanswered and the oldest is older than max(ping timeout, 4 × RTO). That goes through each SDK's existing connection-loss path. Native Rust keeps its existing 30-second idle check. Native C# gets no check, and has none today: .NET's keepalive only sends unsolicited Pongs. That gap is left as it is here, and needs a decision.

Docs. The connection page gets a "Connection Metrics" section with tabbed examples, and the SDK reference pages get the new methods.

API and ABI breaking changes

None breaking, but there is a compatibility edge. The protocol change is additive: new ClientMessage::Ping and ServerMessage::Pong variants. Existing clients are unaffected, since the server only sends a Pong in reply to a Ping. A client built on these SDKs and talking to an older server is disconnected by its first Ping. #5926 accepts this rather than bumping the protocol version.

The SDK APIs are additive. In Rust, network_stats() and ping() are generated inherent methods on DbConnection, so older generated bindings still compile.

Rollback safety impact

The server change is safe to roll back on its own. Once SDK releases that send Pings are in use, though, rolling the server back to a version without Ping would disconnect those clients on their first Ping. The SDK releases should follow the server deployment, not precede it.

Expected complexity level and risk

  1. The protocol change is small. The riskiest part is ws_answer_pings in crates/client-api/src/routes/subscribe.rs, which sits in the websocket actor's receive path and decodes v3 payloads ahead of the message handler. A mistake there could drop or reorder ordinary messages. Each SDK's change touches its connection's receive path and close/reconnect handling. The stats maths is self-contained and unit-tested in each SDK.

Testing

Added:

  • client-api unit tests that ws_answer_pings answers lone Pings and Pings batched in a v3 payload, drops payloads holding only Pings, passes other payloads through unchanged, leaves undecodable payloads for the handler, and passes v1 through untouched.
  • Smoketests that open a raw v3 websocket to a published module and check that a lone Ping, two batched Pings, and two Pings batched behind a OneOffQuery are each answered with a Pong that echoes its send time and reports a plausible receive time and hold. The query is still answered too. With ws_answer_pings removed from the pipeline, these fail.
  • Unit tests of the stats maths in each SDK. The Rust and C# tests are ports of the TypeScript ones: SRTT/RTTVAR ordering, the min-RTT window, offset slewing, the never-backwards clamp, Pongs answering earlier Pings, and pongWait.
  • TypeScript connection tests that Pings start after InitialConnection and repeat, that ping() resolves from its own Pong and rejects before connecting or on close, and that the dead-connection check fires and goes through reconnect.
  • C# reconnect tests that Ping() completes without FrameTick, Pongs stay out of the request trackers, and a reconnect or Disconnect fails pending Pings, resets the stats and ignores a late Pong from the old socket.
  • The Rust connect_disconnect SDK test client now awaits ping() against a real server for all four module languages, and checks that network_stats() is populated.
  • Codegen tests and snapshots for the new Rust DbConnection methods.

Manual:

  • A .NET console client on the C# SDK, connected to a local server: Ping() measured 0.54 ms RTT, the stats filled in, ServerNow stayed within 0.18 ms of the wall clock, and both were cleared after Disconnect.

Gaps:

  • The browser-side dead-connection checks for Rust (wasm) and C# (WebGL) are compile-checked only, not run in a browser.

  • No test checks that a Pong overtakes slow work queued ahead of its Ping. The handler only enqueues a Subscribe and moves on, so an in-order Pong also wins that race, and the test couldn't tell the two apart.

  • Reviewer: sanity-check the ordering semantics of ws_answer_pings. A Pong can now overtake replies to messages sent before its Ping.

@krisajenkins
krisajenkins force-pushed the websocket-ping-pong-rtt branch 3 times, most recently from 01d9f65 to ef3ed0f Compare October 7, 2026 12:13
@krisajenkins
krisajenkins requested a review from aasoni October 7, 2026 15:22
@krisajenkins
krisajenkins marked this pull request as ready for review October 7, 2026 22:32
@krisajenkins
krisajenkins force-pushed the websocket-ping-pong-rtt branch from ef3ed0f to a6139a6 Compare October 8, 2026 09:25

@aasoni aasoni left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Overall looks quite good, only small comments

@krisajenkins

Copy link
Copy Markdown
Contributor Author

Thanks @aasoni! I'll address those comments today.

What's the best next step? Should I merge what we have, then look at supporting the other languages? Or would you prefer one large PR at the end?

@krisajenkins
krisajenkins force-pushed the websocket-ping-pong-rtt branch from a6139a6 to fa56d9b Compare October 8, 2026 13:50
@aasoni

aasoni commented Oct 8, 2026

Copy link
Copy Markdown
Contributor

Thanks @aasoni! I'll address those comments today.

What's the best next step? Should I merge what we have, then look at supporting the other languages? Or would you prefer one large PR at the end?

I prefer that we merge this and do the other languages in another PR as long as we don't end up forgetting to do the other languages 😅

@krisajenkins
krisajenkins enabled auto-merge October 8, 2026 14:09
@krisajenkins
krisajenkins force-pushed the websocket-ping-pong-rtt branch 3 times, most recently from 8aa1560 to 93797a5 Compare October 9, 2026 07:06
@krisajenkins
krisajenkins force-pushed the websocket-ping-pong-rtt branch 3 times, most recently from e73babb to 13ee3ba Compare November 16, 2026 14:37
@krisajenkins krisajenkins changed the title Add Ping/Pong websocket messages and TypeScript SDK network stats Add Ping/Pong to the websocket protocol, with network stats in the TypeScript, Rust and C# SDKs Nov 16, 2026
@krisajenkins

Copy link
Copy Markdown
Contributor Author

@aasoni has reviewed TypeScript and Rust.
@lisandroct perhaps you can review the C# please?

And @JasonAtClockwork - the Unreal code is blocked on @aasoni's reconnect handler work being merged. I'll open a separate PR for it.

Game developers want to know their connection's round-trip time, how much it
jitters, and what time the server thinks it is. Those numbers feed lag
compensation, interpolation and the "ping" display that nearly every
multiplayer game has.

You'd think the protocol already provides this. The server sends a WebSocket
Ping frame every 15 seconds, and the client answers each one with a Pong. But
browsers never expose control frames to JavaScript (#5525), and none of our
SDKs surface them, so that round trip is invisible to the application.

So people approximate it. BitCraft syncs its clock at login with 20 sequential
requests to `GET /v1/database/:name/unstable/timestamp`, shows an ICMP ping to
the region host, and reads reducer latency from the C# SDK's `conn.stats`.
Each measures something slightly different from the real connection: the
timestamp route uses a separate HTTP connection, ICMP bypasses the WebSocket
entirely, and reducer timings include queueing and execution on the server.
Pogly runs its own ping/pong system on top of ours (#5357), and others time
reducer calls, which runs a transaction for every sample.

A ping sent as an ordinary data message would mislead too. The server handles
a client's messages one at a time, and runs `Subscribe` and `OneOffQuery` to
completion before moving on:

    while let Some((data, timer)) = recv_handler.next().await {
        let result = message_handler(data, timer).await;

A ping sent just after an expensive subscription would wait for that query to
finish, and the client would report the query's evaluation time as network
latency. Nothing errors; the figure is just quietly wrong whenever the server
is busy with that client, which is exactly when a game cares.

This adds `ClientMessage::Ping` and `ServerMessage::Pong` to the v2 schema,
and so to v3, which shares it. A `Ping` carries an opaque
`client_send_time: u64`, which the `Pong` echoes along with the server's
`server_receive_time` and `server_hold_duration`. Only clients that send a
Ping ever receive a Pong, so existing clients are unaffected. A new client
talking to an older server is disconnected by its first Ping, which we accept
rather than bumping the protocol version.

The websocket actor answers pings before they reach the incoming queue. A new
stream stage, `ws_answer_pings`, sits between the socket and `ws_recv_queue`.
It decodes any payload small enough to be a lone Ping and replies through the
unordered channel, which the send loop services ahead of ordered messages.
The hold duration is measured when the encoder builds the Pong, as late as
possible before serialisation. Pings batched with other messages in one v3
payload fall through to the normal handler, which answers them in order.

- `decode_lone_ping` only decodes payloads of up to 32 bytes, so only small
  messages are ever decoded twice.
- The Rust SDK gets an `unreachable!` arm for `Pong`, as it never sends pings.
- The TypeScript, C# and Unreal SDKs are unchanged. Deriving the RTT, jitter
  and server-clock metrics there is follow-up work on #5926.
…tion to the TypeScript SDK.

An app built on the TypeScript SDK has no way to tell how far away the server
is or what time it is there. Games want the round-trip time to tune input
prediction and show a latency readout, and they want the server's clock to
schedule things against reducer timestamps. Today the only option is to
guess from the local wall clock, which can be minutes off and gives no
latency information at all. The server now answers ClientMessage::Ping with
a Pong carrying its receive time and how long it held the Ping, which is
everything needed to measure both, but the SDK's generated protocol types
predate those messages, so it can neither send a Ping nor decode a Pong.

There is a second, related gap. Picture a client with automatic reconnect
whose network goes away without the socket being told: Wi-Fi drops, a NAT
forgets the mapping, or (to reproduce it) Chrome DevTools switches the tab to
"Offline". Surely automatic reconnect kicks in? It doesn't. The websocket
stays OPEN, and reconnect waits for an onclose that won't arrive until the OS
gives up on TCP, possibly minutes later. Nothing errors and nothing closes, so
it presents as "the connection looks fine but nothing is happening". Once the
SDK is sending a Ping every second, those Pings are a liveness probe as well
as a measurement, and a run of unanswered ones is exactly the signal that is
missing.

Regenerated src/sdk/client_api with pnpm generate:client-api to add the Ping
and Pong types and their ClientMessage/ServerMessage variants. On top of
that, the connection now pings once on InitialConnection and every second
after that, and exposes the results as conn.networkStats. It is undefined
until the first Pong, then a fresh frozen snapshot on each read: rtt and
jitter (RFC 6298 SRTT and RTTVAR), rttLatest, rttMin over the last 10
samples, serverNow, a Timestamp on the same clock as ctx.timestamp,
serverTimeLatest, the most recent Pong's unadjusted server_receive_time, so
a debug display can show what was measured alongside what was inferred, and
pongWait, how long the oldest unanswered Ping has been waiting, which stays
below rtt on a healthy link and keeps rising on a dead one. The clock offset
comes from the lowest-RTT sample in the window, is applied immediately the
first time, then slews at most 5% of elapsed time and never goes backwards.
There is deliberately no wall-clock fallback: until the server has answered,
the SDK doesn't know its time.

On each ping tick, before sending, the connection is treated as lost once the
oldest unanswered Ping is older than max(ping timeout, 4 x RTO) and at least
two Pings are unanswered. That goes through the existing #handleConnectionLoss
path, so with automatic reconnect it reconnects, and without it onDisconnect
fires with the error; either way pending ping() calls reject.

For a debug overlay or a "test my connection" button, conn.ping() sends a
Ping on demand and returns a promise for a frozen PingResult built from its
own Pong: rtt (round trip minus server hold), roundTrip, serverHold,
serverReceiveTime, and the sentAt/receivedAt performance.now() readings. The
sample feeds networkStats like any other, and ping() works with periodic
pinging disabled.

- The maths lives in a pure NetworkStatsTracker (network_stats.ts) that takes
  every time as an argument, so it is unit-tested deterministically.
- Pings bypass the outbound queue and v3 coalescing, one per websocket
  message, so they never wait behind queued frames.
- Arrival is timestamped as the first statement of the raw socket onmessage
  in WebsocketDecompressAdapter, before decompression or waiting on earlier
  frames, and carried through to the Pong handler as receivedAt.
- Each Ping takes a requestId from the same counter as reducer and
  procedure calls, and Pongs are matched to pending pings by it.
- A Pong answers every Ping sent up to its clientSendTime, not just its own,
  because the server's fast-path Pongs can overtake one another.
- Requiring two unanswered Pings stops a stale send time from before a sleep
  or page freeze tripping the timeout on its own.
- The RTO multiple gives slow links room (a steady 3 s round trip gets 36 s).
  RTO is RFC 6298's SRTT + max(G, 4*RTTVAR) from measured samples only:
  RFC 6298's 1 s floor would make 4 s the shortest possible timeout, silently
  overriding a shorter withPingTimeout.
- withPingInterval(ms) on the builder sets the interval; 0 disables pinging
  and the liveness timeout, leaving networkStats undefined until an explicit
  ping(). withPingTimeout(ms) sets the timeout, defaulting to 10 s. Pinging
  stops, the stats reset and pending pings reject when the socket closes;
  ping() rejects before InitialConnection.
- NetworkStats and PingResult are exported as types.
- The esm (gzip) size-limit budget rises from 50 kB to 55 kB. The new code
  adds about 1.9 kB gzipped to dist/index.mjs and its doc comments, which
  this unminified bundle keeps, about 1.7 kB more, taking it to 52.6 kB.
  The minified budgets still pass with room to spare.
- The shared design notes in TODO.md gain serverTimeLatest for every SDK, and
  the TypeScript items are ticked off.
- Unrelated generator drift in client_api/index.ts: the CLI version header
  moves from 2.0.0 to 2.11.0, and the current template adds a procedures
  accessor export.
- The codegen did not run cleanly: generate-client-api assumes the cargo target
  dir is ./target, and spacetimedb-cli generate --module-def still demands a
  module source directory. Both were worked around locally without changes.
A Rust client has no way to tell how far away the server is or what time it is there. Games want the round-trip time to tune input prediction and show a latency readout, and they want the server's clock to schedule things against reducer timestamps. The server answers ClientMessage::Ping with a Pong carrying everything needed to measure both, and the TypeScript SDK already uses it, but the Rust SDK never sends a Ping. Worse, parse_loop meets a Pong with unreachable!("The Rust SDK does not send pings"), so the SDK's protocol handling had a panic waiting for the day one arrived.

Where the work lives is decided by the SDK's queues. A server message passes through the WebSocket task, then the parse loop, before it reaches code the application drives with frame_tick or advance_one_message. Timestamp a Pong anywhere after the socket and the app's own queueing is counted as network latency. Resolve a ping() there and it hangs until the app next advances. So the WebSocket loop owns all Ping timing. It reads the monotonic clock as each binary frame arrives, before decompressing it, consumes Pongs on the spot instead of queueing them, and stamps each Ping just before writing it to the socket in its own WebSocket message. It starts only once it has seen InitialConnection itself. Connection code reads the stats and asks for on-demand Pings through a small shared handle.

The maths is a direct port of the TypeScript NetworkStatsTracker, with its tests: RFC 6298 SRTT and RTTVAR, rtt_min over the last 10 samples, an offset estimate from the lowest-RTT sample that is applied immediately the first time, then slews at most 5% and never goes backwards, and pong_wait, the age of the oldest unanswered Ping.

Dead-connection detection differs by build. Native builds already have a 30 s idle check on WebSocket frames, so they keep it and add nothing. The Pongs now arriving every second don't weaken that check, because on a dead link they stop arriving too. The browser build has no transport keepalive, because the browser's WebSocket hides protocol Ping frames from the page. So, on each tick, it closes the socket once two or more Pings are unanswered past max(ping timeout, 4 x RTO), and reports Error::ConnectionUnresponsive through the existing error-disconnect path. The Rust SDK has no auto-reconnect, so this disconnects, with on_disconnect receiving the error. That exposed a hazard: run_threaded and run_background_task panic on any error that isn't a normal disconnect, which would have turned every network drop in the browser into a wasm trap. They now treat ConnectionUnresponsive as an expected end, since on_disconnect has already seen it. run_async still returns it.

- conn.network_stats() -> Option<NetworkStats> is None until the first Pong and resets when the connection closes. NetworkStats has rtt, rtt_latest, rtt_min, jitter, server_now, server_time_latest and pong_wait.
- conn.ping() returns a future of Result<PingResult>. It sends eagerly, as the TypeScript Promise does, takes its request_id from the shared counter, feeds the stats, works with periodic pinging disabled, and fails with NotYetConnected before InitialConnection or Disconnected when the connection closes.
- The builder gains with_ping_interval(Duration), 1 s by default, where zero disables periodic Pings. Under the browser feature only, it also gains with_ping_timeout(Duration), 10 s by default.
- network_stats() and ping() need a codegen change, because the generated DbConnection keeps its imp private. They are generated as inherent methods, so older generated bindings still compile without them. The codegen snapshot and the checked-in test-client bindings are updated to match.
- Running the SDK test regenerated connect_disconnect_client's bindings, which moves its CLI version header from 2.10.1 to 2.12.0.
- The browser build uses web_time::Instant, because std::time::Instant panics on wasm32-unknown-unknown, and gloo-timers intervals. Both are new optional dependencies under the browser feature.
- The connect_disconnect test client now awaits ping() after connecting, and checks rtt <= round_trip and that network_stats() is Some. That test passes natively for all four module languages. The browser loop is compile-checked natively and for wasm32, but its end-to-end test was not run.
The server now speaks ClientMessage::Ping and ServerMessage::Pong, but the C#
SDK's generated protocol types predate them. Before the SDK can measure round
trips, it has to be able to say Ping and to decode Pong at all. Worse, both
message switches in SpacetimeDBClient.cs end in
'default: throw new InvalidOperationException()', so once the bindings know
about Pong, the first one to arrive would kill the parse thread unless it has
an arm of its own.

The regeneration adds Ping.g.cs and Pong.g.cs, each with a Unity .meta file
like its neighbours, and appends Ping as ClientMessage tag 6 and Pong as
ServerMessage tag 9, matching the protocol. Nothing else drifted. Both
switches in SpacetimeDBClient.cs get a Pong arm that does nothing for now;
the next change gives it real work. TrackerMetadataForMessage already has a
fallback, and the SDK never switches over ClientMessage, so neither needed
changing.

tools~/gen-client-api.sh doesn't run as-is, so its steps were run by hand,
with no change to the tools:

- 'spacetime generate --module-def' still runs the module-source-dir check
  (crates/cli/src/subcommands/generate.rs), which ignores --module-def, and
  --module-path is in the same clap group so can't be combined with it. The
  CLI was run from a scratch directory containing an empty spacetimedb/
  subdirectory, via --manifest-path, with the schema from
  'cargo run -p spacetimedb-client-api-messages --example get_ws_schema_v2'
  on stdin. The generated Types/*.g.cs were copied over as the script's mv
  does.
- The script's 'cargo build' of crates/standalone isn't needed for generation
  and was skipped. Everything runs through cargo run, so a non-default
  CARGO_TARGET_DIR doesn't matter here.

The C# SDK tests pass (110/110, -warnaserror), dotnet format reports no
changes, and the Godot project builds.
… detection to the C# SDK.

A Unity or .NET client has no way to tell how far away the server is or what
time it is there. Games want the round-trip time to tune input prediction and
show a latency readout, and they want the server's clock to schedule things
against reducer timestamps. The server answers a Ping with a Pong carrying
everything needed to measure both, and the TypeScript and Rust SDKs already
use it, but the C# SDK never sends one.

Under Unity WebGL there's a second problem. The socket there is the
browser's WebSocket, which can't see a network that fails silently. The
socket stays open with nothing arriving: no close, no error, and an app that
thinks it is still connected. Reconnect.cs can't help, because nothing ever
tells it the connection is gone.

This ports the shared design from TODO.md, using the TypeScript SDK's
network_stats.ts as the reference for the maths:

- conn.stats.Network is a NetworkStats? snapshot with Rtt, RttLatest, RttMin,
  Jitter and PongWait as TimeSpans, and ServerNow and ServerTimeLatest as
  Timestamps. It is null until the first Pong, and again after the
  connection closes or is lost. Each read takes a fresh snapshot under a
  lock, so it is safe from any thread.
- conn.Ping() returns a Task<PingResult> with Rtt, RoundTrip, ServerHold,
  ServerReceiveTime, and SentAt/ReceivedAt as Stopwatch timestamps. Like a
  one-off query, it completes on the parse thread without waiting for
  FrameTick, with continuations run asynchronously. It fails at once before
  InitialConnection or after close, and every pending Ping() fails when the
  connection closes.
- The builder gains WithPingInterval(TimeSpan), 1 s by default, where zero
  disables periodic Pings. The timer is a Stopwatch deadline checked in
  FrameTick, so it works the same under Unity and plain .NET. The first Ping
  goes out on InitialConnection, before the user's connect callback and the
  subscription replay.

Pongs are timestamped where they come off the socket: on the native receive
thread as ReceiveAsync returns, and in HandleWebGLMessage before the message
waits in the dispatch queue for the next FrameTick. The stamp travels with
the message, so neither the parse queue nor frame time counts as network
time. ParseMessage pulls Pongs out right after decoding and drops their
parse-queue tracker entry, so Pings never show up in the existing request
trackers or reach the apply queue. Every C# send is already its own
WebSocket message, so Pings go out alone without any change to the send path.

Dead-connection detection (rule 8) is compiled only under
UNITY_WEBGL && !UNITY_EDITOR. Before each periodic Ping, if at least two are
unanswered and the oldest is older than max(ping timeout, 4 x RTO), the
connection is failed through HandleSocketFailure, so it reconnects if
automatic reconnect is on and disconnects with an error otherwise.
WithPingTimeout(TimeSpan), 10 s by default, is declared under UNITY_WEBGL
alone, so user code calling it still compiles in the Unity Editor with the
WebGL target selected; there the value is stored but unused.

Native builds don't get this check, as the design says, but the premise
behind that doesn't hold for C#. The SDK never sets
ClientWebSocket.Options.KeepAliveInterval, so .NET's 30 s default applies, and
that keepalive sends unsolicited Pong frames and expects no reply. It keeps
middleboxes from timing out, but it detects nothing. KeepAliveTimeout, which
would detect a dead connection, needs .NET 9 and isn't available on
netstandard2.1. So native C# builds currently have no dead-connection
detection at all. That's left unchanged here and needs a decision.

Other details:

- Every close path bumps socketGeneration and then resets the stats under
  the tracker's lock. HandlePong checks the generation under the same lock,
  so a late Pong from a replaced socket can't leak into its successor's
  stats.
- Pings take request ids from a counter of their own, shared by periodic
  Pings and Ping(). The design asks for the counter that reducer and
  procedure calls use, but the C# SDK has none: each takes its ids from its
  own NetworkRequestTracker.
- On native builds the send queue is first in, first out, and ClientWebSocket
  allows one send at a time, so a Ping can wait behind outbound messages
  already queued, and that wait counts towards its RTT. Left as is.
- No codegen change. The generated DbConnection inherits Ping(), and its
  body uses only qualified names, so the inherited Ping() can't collide with
  a user type called Ping.
- NetworkStatsTests.cs ports all 20 tests from network_stats.test.ts. Four
  more in ReconnectTests check the following:
  - nothing is sent before InitialConnection, and Pings then repeat;
  - Ping() resolves from its own Pong without FrameTick, and Pongs stay out
    of the trackers;
  - a reconnect fails pending Pings, resets the stats and ignores a late Pong
    from the old socket;
  - Disconnect does the same without reconnect.
  FakeSocket keeps Pings in a queue of their own, so the existing assertions
  are unchanged.

Tests pass (134, up from 110, with -warnaserror), dotnet format reports no
changes, and the Godot build has only its existing warnings. The WebGL code
was compile-checked by building the SDK sources with UNITY_WEBGL defined
against stand-ins for the few Unity APIs it uses, with no new warnings; it
wasn't run in a browser. End to end, a net8.0 console client on this SDK
talked to a local standalone server running templates/basic-rs's module.
Ping() measured an rtt of 0.54 ms (a 0.61 ms round trip, 0.07 ms server
hold), stats.Network filled in, PongWait stayed under 0.05 ms, ServerNow
stayed within 0.18 ms of the wall clock and never went backwards, reducers
and subscriptions still worked, and after Disconnect, Network was null and
Ping() failed.
A client measures round-trip time by sending a Ping and reading the Pong. The
server stamps the Pong with how long it held the Ping, and the client subtracts
that from the RTT, so that time spent inside the server doesn't count as
network latency. That only works if the hold duration covers everything the
server did with the Ping.

The fast path in `ws_answer_pings` answers a Ping the moment it's read off the
socket, but only when the Ping is alone in its websocket message. Suppose a
v3 client batches two Pings in one payload, or a Ping with other messages:
the TypeScript SDK micro-batches its outbound messages, and a third-party
client might send its Pings through that same queue. The payload skips the
fast path and goes to the message handler, which answers the Pings in order:

    ws_v2::ClientMessage::Ping(ping) => {
        let held = timer.elapsed();
        ...
        client.send_message(None, ws_v2::ServerMessage::Pong(pong))

`timer` is stamped when the handler dequeues the payload, not when the socket
read it, and the Pong then goes out through the client's ordered outbound
channel, where `held` misses its wait too. In a local test, a lone Ping
measured 0.5 ms RTT with 50 µs of hold. Two batched Pings measured 2.5 ms RTT
but reported only 3–6 µs of hold. In a smoketest run against a server
without this fix, the second of two batched Pongs arrived 4.4 ms after the
first, yet both reported the same 172 µs of hold. Either way, the client
computes an RTT several milliseconds too high.

Nothing errors and the numbers look plausible, so a client would just report
a worse connection than it has. Our SDKs send each Ping alone, so they never
hit this, and the unit tests only covered the lone case.

Now `ws_answer_pings` answers every Ping it finds in a v3 payload, through
`PendingPong` and `unordered_tx` exactly as in the lone case, and the message
handler ignores Pings.

- A payload holding only Pings is dropped once they are answered. Any other
  payload passes through unchanged, so the handler still sees, and ignores,
  the Pings in it.
- A payload that fails to decode passes through with none of its Pings
  answered, so the handler reports the error exactly as before.
- v2 is one message per payload, so it is only decoded when its first byte is
  the Ping tag. Like the handler, it decodes the first message and ignores any
  trailing bytes.
- A Pong can now overtake the replies to messages batched ahead of its Ping.
  The `Ping` doc comment now says a Ping is a latency probe, not a barrier.
- v3 messages aren't length-prefixed, so finding the Pings means decoding the
  whole payload, and the handler decodes it again. Measured in a release
  build, the extra decode costs the same as the handler's own: about 14 µs per
  MB of reducer arguments, essentially one memcpy. That isn't worth threading
  decoded messages through to the handler.
The server answers every protocol-level Ping in `ws_answer_pings`, a stage
in `ws_recv_task` that sits in front of the message handler, and the
handler itself now ignores Pings. That stage has unit tests, but they
exercise it on its own. If it were ever dropped from the pipeline, every
existing test would still pass, and every Ping would go unanswered: the
SDKs' network stats would never appear, and in browser builds the
connection would be closed as dead every ten seconds.

These tests open a raw v3 websocket to a published module and check that
Pings come back as Pongs:

- a Ping sent alone in its own websocket message;
- two Pings batched in one payload, which the server answers and then drops;
- two Pings batched behind a OneOffQuery, where the payload must also pass
  through so that the query is still answered.

Each Pong must echo its Ping's `client_send_time`, and report a
`server_receive_time` close to the local clock and a plausible hold
duration. With `ws_answer_pings` removed from the pipeline, both tests fail
for lack of a reply.

They don't check that a Pong overtakes slow work queued ahead of its Ping.
I tried, with a Subscribe to a view that takes about 380 ms to evaluate,
but the handler only enqueues a Subscribe (and a OneOffQuery or reducer
call) on the module instance and moves on, so even a Ping answered in order
beats the SubscribeApplied. The same test passed against a server without
the batched-Ping fix, so it couldn't tell the two apart.
…t SDKs.

The TypeScript, Rust and C# SDKs now ping the server every second and expose
the results: round-trip time, jitter, an estimate of the server's clock, and
how long the oldest ping has gone unanswered. They also offer a one-off
ping(), and new builder settings for the ping interval and, in browser
builds, the ping timeout. None of this is in the docs, so the only way to
find it is to read the SDK source, and some of it isn't obvious from the
API alone. The stats are deliberately absent until the first reply rather
than falling back to the local clock, `serverNow` is on the same clock as
`ctx.timestamp`, and whether a silently failed network is ever noticed
depends on which SDK and which build you use.

This adds a "Connection Metrics" section to the connection page, with tabbed
examples like the rest of the page, covering what each metric means, the
server clock and how to choose a fallback explicitly, the ping interval,
one-off pings, and dead-connection detection per SDK. The reference pages
get entries for the new builder methods, the stats accessor and ping(), each
linking back to the connection page.

- The Unreal tabs say metrics are coming soon, matching how the page
  already handles automatic reconnect there. The Unreal reference page is
  unchanged.
- The C# tab says plainly that non-WebGL builds don't detect a silently
  failed network. .NET's ClientWebSocket keepalive only sends unsolicited
  Pongs, and the SDK has no check of its own there, so `PongWait` is the
  only signal.
@krisajenkins
krisajenkins force-pushed the websocket-ping-pong-rtt branch from 13ee3ba to 2c4b954 Compare November 17, 2026 08:13

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants