Skip to content

Deploy v0.10.1 to Staging - #113

Merged
Ashex merged 8 commits into
devfrom
main
Nov 12, 2026
Merged

Ashex merged 8 commits into
devfrom
main

Conversation

@Ashex

@Ashex Ashex commented Nov 12, 2026

Copy link
Copy Markdown
Collaborator

Deploys release v0.10.1 (b6b64228) to staging. Merging this into dev starts Publish Railway release candidate, which builds the relay, jetstream and administration images once as sha-<merge commit> and hands their digests to the private infrastructure receiver for staging.

Changes since v0.10.0

  • Jetstream live ingestion keeps running across Relay restarts: transient 408/425/429/5xx dial rejections are retried from the same cursor, and an unexpected stream end exits the process instead of stalling (fix(jetstream): keep live ingest running across relay restarts #111, TECH-883).
  • Verification uses 128 workers (per-account queue 256), and upstream reads are paced at 500 frames/s by default. Configure with JETSTREAM_LIVE_VERIFY_PARALLELISM, JETSTREAM_LIVE_UPSTREAM_READ_RATE and JETSTREAM_LIVE_UPSTREAM_READ_BURST.
  • Repeated verification failure logs are summarised per account.
  • The Release workflow pushes the validated release tag (Push the validated release tag from the Release workflow #109).

Staging note

Staging Jetstream is currently stalled: its cursor has been stuck at 196,202 since a Relay 502 at 11:18 UTC today. This deploy restarts it, and it should resume from that cursor and replay the backlog at the paced rate.

Checks after deploy

  • All three Railway deployments reach SUCCESS.
  • Jetstream logs show upstream connected with the resume cursor, and the /status upstream cursor advances to the Relay's head.
  • verify queue overflow dropped event is rare or absent during catch-up, and verification failures appear only as summarised lines.
  • Relay /xrpc/_health and Administration /health respond.

🤖 Generated with Claude Code

Ashex and others added 8 commits November 12, 2026 12:00
The Changesets v2 migration set push-git-tags to false, and
scripts/release.mjs only tags the runner's checkout, so the v0.10.0
release run created and validated its tag without publishing it. Push
the tag after validation, refusing a tag that does not resolve to the
checked-out release commit. A plain push never moves an existing tag.

This work was assisted by a machine
A single HTTP 502/503 from the relay during reconnect made atmos end the
upstream iterator, and the live consumer returned nil, leaving Jetstream
running without ingestion for hours. On restart the stale cursor replayed a
large backlog that overflowed the per-DID verify queue (TECH-883).

- Retry transient dial rejections (408, 425, 429, 5xx) with backoff and log
  each successful upstream connection.
- Return ErrUpstreamStreamEnded when the iterator ends while the consumer is
  still running, so the process exits instead of stalling silently.
- Raise verify parallelism to 128 (per-DID queue 256) and pace upstream reads
  at 500 frames/s by default; both are configurable.
- Summarise repeated verification failure log lines per DID and class.

This work was assisted by a machine
- Mark the changeset minor: it adds three configuration options.
- Assert that redials after transient rejections resume from the persisted
  cursor.
- Add hypercerts markers to the remaining upstream-file hunks and drop a
  pass-through classifier wrapper.

This work was assisted by a machine
…-live-ingest-silently-stops-after-a

fix(jetstream): keep live ingest running across relay restarts
…d-tag

Push the validated release tag from the Release workflow
@coderabbitai

coderabbitai Bot commented Nov 12, 2026

Copy link
Copy Markdown
Contributor

Important

Review skipped

Auto reviews are disabled on base/target branches other than the default branch.

Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

Run configuration

  • Source mode: Auto
  • Repository: Unavailable
  • Branch: Unavailable
  • File: No configuration file from this repository applied
  • Commit: Unavailable
  • Layers: defaults
  • Review profile: CHILL
  • Plan: Advanced
  • Run ID: c8df6879-997f-4fa4-93df-65d40cdf2463

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

  • Trigger review
  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

Share

@sonarqubecloud

Copy link
Copy Markdown

@Ashex
Ashex merged commit edf3d85 into dev Nov 12, 2026
20 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant