Skip to content

Amortize hosted relay authorization and defer peer formation - #2162

Draft
santoshkumarradha wants to merge 4 commits into
devfrom
fix/hosted-relay-fastpath
Draft

santoshkumarradha wants to merge 4 commits into
devfrom
fix/hosted-relay-fastpath

Conversation

@santoshkumarradha

Copy link
Copy Markdown
Member

Summary

Store requests currently authorize twice and synchronously commit advisory presence. Directory reads also incur repeated quorum work. This keeps mutations as consensus commits while amortizing read authority and moving presence off the request path.

mutations → Raft commit → durable apply → revoke watches and held polls
reads → confirmed applied authority → local view
blob I/O → one authorization → applied-index recheck
new empty identity → singleton commit → background learners → joint consensus

Read authority is invalidated before member work, transport delivery, configuration changes and activation replacement. Lost authority fails closed. Existing identities retain their voting configuration. A rejected upload remains tracked until its underlying storage operation finishes.

Evidence

  • Before: store requests perform repeated authorization/presence commits; large cold writes exposed a snapshot excluding a newly added learner.
    After: regression tests cover proposal-free warm reads, authorization once, revoke during I/O, immediate watch/poll closure, snapshot catch-up and crash-safe singleton formation.
  • Validation: 248 unit tests, 166 relay proof tests, the separate actor proof, and the affected Go state/consensus packages pass. make test-quick and make pr-ready BASE=20eb6952de6482ea8f8709784ff447b9d9ed3fc3 pass.
  • Draft acceptance: public internet transfers have verified all 2,000 files / 50 MiB in both directions, including interruption/resume. Five alternating before/after runs, live UI evidence, the complete local e2e run, and CI results are being completed. No comparative internet speed claim is made yet.

Tests asserting quorum refusal, revocation, replay and fence safety retain their assertions. Fixture changes establish the intended configuration before faults; controlled activation replacement invalidates known authority while an unannounced replacement still fails closed. Capacity fixtures account for one-voter births without increasing the memory budget. Legacy migration fault points follow the same durable boundaries. API takeover tests now supply the observed holder tuple and explicitly reject blind takeover, matching the existing consensus security contract.

Merge Danger

Door: one-way

New singleton births persist immutable provenance. Older relay code refuses that configuration; rollback must preserve availability through compatible code rather than restoring a local directory writer.

Blast Radius: hosted

Changes authority, new-identity bootstrap, presence and committed-change notifications. Fail-closed and revocation tests are required. Production deployment and merging remain gated; this PR is a draft pending performance acceptance.

Copy link
Copy Markdown
Member Author

Agent review round 1 — architecture

Reviewed the design against the quorum/revocation ruling. Rejected a direct local directory writer: concurrent identity writers still require consensus. The implementation keeps mutation commits and uses an invalidated, short-lived confirmation of clean applied state for reads. Fresh singleton provenance is immutable; an established voting configuration cannot be reduced during recovery. No remaining architecture blocker was found within this review.

Copy link
Copy Markdown
Member Author

Agent review round 2 — correctness

Review found that a compacted snapshot could exclude a newly admitted learner, causing repeated full snapshot transfers. Learner admission now refreshes the snapshot before catch-up and retains the resulting durable effects. The archived large-body cases pass, including restart and staging cuts. Additional checks cover every formation turn and refusal to recreate a lost published member. Reviewed production revision: a8005c4.

Copy link
Copy Markdown
Member Author

Agent review round 3 — security and cost

Review found two gaps: a replicated revoke could bypass public-handler watch closure, and an immediately rejected upload could release its in-flight slot before storage finished. Committed-apply notifications now close watches and held polls; upload tracking lasts until actual handler completion. Both findings have red-to-green regressions. Existing quorum refusal, replay and revocation assertions remain. Validation at a8005c4: 166 relay proofs, the separate actor proof, 248 unit tests, and fresh affected Go-package checks pass. Performance acceptance and remaining end-to-end fixture work are still in progress; these agent reviews do not authorize merging.

Copy link
Copy Markdown
Member Author

Agent follow-up review of ca15a2543a → b2a0a59d9e: Standards 0 findings; Spec 0 findings. The transport change removes recovery reports only for anchors absent from the actual durable configuration. Configured voters, learners, outgoing voters, and next learners retain failure reporting; unavailable evidence refuses recovery rather than guessing membership. Authority checks and required consensus barriers remain intact. Unit suite: 248/248 passed. Fresh public-internet before/after transfer measurements for this revision are in progress; prior revision measurements are not treated as evidence for the new revision. This review is not merge authorization.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants