Repository navigation
feat: spec for one hostname per tenant with optional cloud storage - #45
Merged
Merged
Conversation
A tenant is one hostname with one shared list. Local storage on the origin machine stays the default; R2 is optional per tenant and per link. Covers the Worker pass-through, Share Bar row badges, and both migrations.
Bucket rows become display data, the alias fold gets a safe partial rollback, members cannot remove machine links, healthz keeps members working while the origin is off, and the tasks split with a stop before each production migration. Warnings go to the implementation notes.
The r2 setup refuses a tenant host, the storage task splits in two, and bucket reads on the origin get their own reader. Remaining warnings go to the implementation notes.
Cloud verbs on a profile with a tunnel use their own bucket reader and never touch the index. Round 3 warnings are recorded as build rules.
A route Worker passes misses and its healthz fetch to the tunnel origin without re-entry. Access gates before it. POST and WebSocket pass through. The header cf-cache-status tells an origin answer apart from an edge error.
A custom domain rebinds in place only with override_existing_origin. An Access app PUT keeps the AUD when it sends the full read-back. The Toolkit token holds every admin scope on both accounts. Both accounts run Workers Paid. The spec now names the rebind flag, the full-body PUT, the route fail-open flag, the 503 rule, and the token sources.
A tunnel profile with bucket= now takes --cloud and --local per add, with storage_default as the tenant default. A local add reserves its id with a v:2 machine pointer before any byte is served or any Access app exists, and fails closed when R2 refuses it. rand_id checks the local index and the bucket. rm deletes the origin's own pointer after the row. api-token runs the publisher refusals on bucket=. Suite: 1112 ok, 0 FAIL.
Green suite run, the Worker tests, and two negative controls (pointer after publish, pointer failure as a warning), each red and then green after restore.
On an origin with R2 on, rm, refresh, and hits of an id with no local row take the r2 path through the origin's own bucket reader, which never assigns or writes index.tsv. A member's rm and refresh of a machine link die before any write. The origin's interactive ls and prune expire cloud rows, sweep orphan uploads, and reconcile pointers: orphan pointers over 10 minutes old go, a row with no pointer gets one, a shadowed id is named. The orphan sweep reads a machine pointer as readable. The access sweep decides per id. api-token warns 14 days before the publisher token expires. Suite: 1151 ok, 0 FAIL.
TASK-2b ticked; deltas logged (member hits keeps SPEC-007 row 13, the add lock, the reader). Negative controls for rows 3 and 26 ran red then green on tree 25dd9e9.
With PASS set the Worker runs as a route on the tenant host: a v:1 cloud record serves from R2 as before, a v:2 machine pointer, a miss, a bucket error or a 2 s stall pass through to the origin, and any other record answers 404. The pass-through rewraps the origin's answer, returns a 101 untouched, and answers edge errors (502, 520 to 527, 530, a 503 without cf-cache-status, a thrown fetch) with a 503 offline page. An ALIASES host answers 301 to the same path on HOST. /healthz probes the origin and sets X-Share-Tunnel. r2_healthz reads a 503 with a sound pair and X-Share-Tunnel: 0 as Worker up. WORKER_VERSION 3, WORKER_RECORD_V 2. Suite: 1211 ok, 0 FAIL; worker.mjs PASS.
TASK-3 ticked; deltas logged (the method rule with PASS empty, the record shapes, the healthz header file). Negative controls for rows 7, 9, and 29 ran red then green on tree a6ff281.
setup <host> --r2 --bucket <b> runs on the origin with the admin token: every read and refusal first (the role, share's Worker and a private bucket, no route of another script, no local id holding a cloud record), then the marker on a new bucket, the Worker with PASS and ALIASES, workers.dev off, a pointer per local row under the index lock, the four config keys, a fail-open route, and both healthz legs. A rerun converges. --no-r2 deletes the route and drops the keys. SPEC-007's admin setup and a member purge refuse a tenant host. Dry seam only; nothing was created on Cloudflare. Suite: 1243 ok, 0 FAIL.
TASK-4 ticked; deltas logged (step order on a new bucket, the route glob, the kept config keys, the v0.8.0 marker finding). Negative controls for rows 11 and 32 ran red then green on tree a2cd324.
A member reads no bucket in hits, so a 0 on an id it did not add may be a machine link the Worker never counts. The count now comes with a line naming the origin's hits. Recorded as DEC-012.
Steps 4, 7, 10, 12, 14, and 16: the alias's Worker and bucket are checked, each gated cloud link's app gains the tenant destinations under the gate probe, the marker and the alias domain move to the tenant Worker, and the alias destinations go last. A die after the config write prints the alias rollback list; --no-r2 refuses while an alias is set. Members keep a 301-confirmed alias so their rm deletes a folded app.
ls and state read the local index plus the bucket through a separate reader whose rows never reach rows(), the index, or a Caddyfile. Each row carries storage, type, and by; ls tags each link; state gains r2, storage_default, cloud_error, and cloud_more, reads the bucket only with a stored token, and fetches the newest 25 records. A member lists the origin's links from their pointers, checked field by field.
The hidden-row count includes orphan pointers and unreadable records, so the menu line hedges with about. The storage picker names this Mac when the tenant lists several hosts.
Covers the tenant setup--r2 flow, member join, an outage, encoded paths, rollback, and the Dwarves alias-fold rehearsal with its printed rollback list, modeled on e2e-r2.sh and e2e.sh. One live run against a throwaway Cloudflare zone found and fixed two scripting bugs (a missing setup hostname argument, and a gated add on an origin that still had the admin token in scope); results and the unresolved tunnel-token stub issue are in the TASK-9a notes.
import takes one share's tar on stdin: it validates the id, dates, by, name, source, and opts, pre-scans the listing (plain files and directories only, no absolute, .., or dotfile path), extracts with no owner, xattr, or flag bits, rechecks links, and publishes the row with its original dates. setup --token-stdin reads the token from one line of stdin. The tar flag measurement is in the implementation notes.
Green runs per commit, six negative controls with their red rows, the tar measurement pointer, and the rollback note.
…/one-host-per-tenant # Conflicts: # docs/implementation-notes/one-host-per-tenant.md
…ost-per-tenant # Conflicts: # docs/implementation-notes/one-host-per-tenant.md
cmd_migrate called the target's setup with no --tunnel-name, so a source on the default tunnel name handed the target the identical tunnel id. migrate_retire then deleted it, taking the new origin down right after a reported success, and reused a tunnel DEC-010 forbids reusing. Always pass the target a deterministic name distinct from the source's own (the current name plus -m), and make migrate_retire refuse the tunnel and DNS deletes outright if the target's tunnel id ever still matches this machine's.
tests/share.sh printed the same "PASS" or "N FAILED" summary whether the run finished or crashed partway, since nothing marked an actual finish. The EXIT trap now checks a reached_end flag set only at the true end of the script: missing, it prints ABORTED and exits 2. The summary also prints the total check count. Adds a self-test that truncates the script at a fixed anchor and confirms the truncated run reports ABORTED and exits 2.
The old_host guard in cmd_setup only fired when old_host was set, so import into a not_setup profile followed by setup <other-host> served its gated rows with no gate: there was no old hostname to compare the new one against. Refuse when old_host is empty and the profile already holds an access= row. migrate's own switch (--token-stdin) is exempt: it never renames the hostname, so a row's existing Access app still covers it, and import-then-setup for the same tenant must keep working.
pass() treated any 502 as the whole machine offline, same as Cloudflare's own 520-527 edge codes. A 502 never comes from the edge on this path; it is Caddy's reverse_proxy answer when a live-port share's upstream port is dead, while the machine and tunnel are fine. Only 530, 520-527, and a 503 with no cf-cache-status still mean offline; a 502 goes back unchanged like any other origin answer. Bumps WORKER_VERSION and WORKER_SHA for the source change.
The implementation notes claimed the migrate preflight reads tunnel_name from the API by tunnel_id when the config key is absent, but cmd_migrate never did that: a config missing tunnel_name (an older share, or a hand edit) printed its rollback line with an empty --tunnel-name. cmd_migrate now reads it back from the API when the local key is missing, and dies naming the manual fix instead if neither tunnel_name nor a readable tunnel_id exists. Corrects the implementation note to describe where this actually runs.
alias_dests drop answered HTTP 400 from Cloudflare in three of three live rehearsals, right after the same PUT shape succeeded for add two calls earlier. The read-back app carries a legacy self_hosted_domains mirror of its original single destination; a later PUT is not kept in sync with it automatically, so once drop's own PUT removes that exact destination, the stale self_hosted_domains entry no longer appears among the new destinations and Cloudflare refuses the request. The PUT body now drops self_hosted_domains and domain so the API recomputes them from destinations, matching what a growing PUT (add) already tolerated by coincidence. cf_dry's PUT handler now rejects the same inconsistency a real PUT does, and the alias-fold fixture carries a self_hosted_domains field, so the existing row 14/30 dry-seam coverage catches this without live Cloudflare access.
Applies the run1-to-run3 script fixes: wait for a new publisher token to reach the R2 S3 endpoint before using it cold, read a POST's full body across TCP segments in the echo origin, tolerate a missing gate.seen file, and correct two Access-app lookups that assumed a folded app gets renamed to the tenant host (SPEC-008: it keeps its name on the alias). Also fixes L8: etag_before read the pointer through A's own profile right after L7's --no-r2 removed bucket= from it, so the call had no bucket to run against. Reads the pointer directly by bucket name instead, the same sigv4 S3 pattern leftovers() uses.
Batch 5: red/green per finding, the two live-rehearsal fixes, and the final full-suite, worker, Swift, and shellcheck runs.
The pointer pipeline in alias_rollback ends in a grep. With no local rows and no machine records the grep matches nothing, and under set -e the assignment killed the run before the list or the die message printed. A tenant that folded an alias and then expired every local row would get a silent exit 1 from --no-r2 and from every tenant_die. Found by the live R1 rehearsal's rollback leg.
…bin/share README, how-it-works, and setup gain the tenant model: R2 on the origin, per-link storage, the alias fold, pointer records, the Worker's checks, and migrate. ADR-0008 records the decision. SPEC-008 gains DEC-013 and the 502 rows now match the Worker the fix pass shipped (version 4). tests/share.sh checks that every verb, flag, config key, and state key the tenant docs name is in bin/share, with three negative controls.
The WebSocket client resolves through DoH like every other leg (the local resolver answered No route to host for a name created seconds earlier). The R1 rollback removes the aliases= line from the profile config, as the printed alias rollback list says, before --no-r2. The R1 cleanup deletes the alias bucket's objects with the admin seam, since that profile holds no stored publisher token.
The marker ETag is parsed from r2-call's one-line code=<n> etag=<etag> output, so the marker host restore carries a real If-Match. The alias liveness check reads /healthz: a standalone r2 Worker answers 404 at the site root by design.
… TASK-10 Batch 6 of the verification record: five live e2e runs ending at 73/73 with the cleanup proven from outside, the alias_rollback fix with its red and green run, and five more negative controls (one spec control is not reachable in its spec form; a substitute is recorded). Adds the passing run log, the setup usage lines to the bin/share header, and marks TASK-8, TASK-9a, and TASK-10 in SPEC-008.
… reports CI lints bin/share and tests/share.sh with a shellcheck that flags SC2015 twice in bin/share (a cd && find | grep pipeline, a test-and-grep line) and SC2002 once in tests/share.sh. Same behavior, restructured so both the old and the new shellcheck stay quiet. Lint has failed on every push of this branch since the tenant work began.
… gated links Three safety changes from the lead's review of the live rehearsals. migrate_retire refuses the DNS and tunnel delete when the new origin's tunnel id could not be read back, instead of skipping its own collision guard. setup retries a Cloudflare 5xx on the custom-domain add (the standalone r2 setup and the alias bind), three tries with doubling backoff, and fails loud after. add --access polls the published link until it answers the Access challenge (302 or 403), 30 s by default, before it prints the link. A link that still answers anything else exits 1 with a warning that names the status and the rm command.
…n run them The migrate rows run setup on the target, which dies with "missing: cloudflared" on a runner without one, so rows 19, 20, 21, and 31 failed on both CI test jobs. Stand-ins sit last on PATH, so a real binary wins. A failing row 19 now prints migrate's own output, and the lint finding in that line is fixed. Records the batch, the CI causes, and the passing live e2e log.
The remote setup builds its own PATH from --remote-bin plus the system directories, so stand-ins appended to the caller's PATH were never found on CI. They now sit in the remote bin directories the migrate rows pass.
read -s turns terminal echo off after it writes the prompt, and expect typed the token the instant the prompt matched, so on the macOS runner the token sometimes echoed. A 0.3 s pause before the send removes the race. The failure showed on macOS CI in three of five runs of this branch.
…runner SHARE_R2_WAIT counts whole seconds, so a 1 s window closes mid-loop when three dry healthz calls straddle a second boundary. The passing tunnel-down case now waits up to 8 s and still returns on the third 200; the failing case stays at 1 s. It failed on a local run under load and on the macOS runner.
… name A bare-file snapshot keeps its generated index.html beside the file at the id root. migrate archived only the name, so the receiver counted one file where the sender's manifest counted two, and import refused the share as a truncated copy. Every single-file link would have failed the personal move. The archive is now the whole id directory and import checks that the name is in it. The suite fixture gains the generated index.html it had been missing. Found by the live rehearsal of the move.
After a failed switch the target profile is already serving this hostname, so its ports are in use by that very profile. The preflight died with "port for this profile is already in use" on the rerun, which made the printed rollback a dead end. A target already serving the same hostname now passes. Found by the live rehearsal's failure-then-rerun leg.
tests/e2e-migrate.sh runs migrate on throwaway names: a failed switch and its printed rollback, the rerun that moves the links, and a second name moved from a real second machine to this one over ssh. Records the passing 38/38 run, the two product bugs it found, and marks the migrate rehearsal done in SPEC-008.
tests/prod-snapshot.sh records a live tenant's DNS, Worker domains and routes, Access apps, tunnels, buckets, launchd jobs, share profiles, and link probes into one file, so a production step can be rolled back from exact state. It never writes. Modes: dwarves (s.d.foundation and f.d.foundation) and personal (s.han.ws).
tieubao
marked this pull request as ready for review
October 6, 2026 13:34
This was referenced Oct 6, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Spec only:
docs/specs/SPEC-008-one-host-per-tenant.md, plus build notes indocs/implementation-notes/one-host-per-tenant.md. There is no code in this PR. The title isfeat:so that the merge of the eventual build counts as a minor release.What the spec decides
setup <host> --r2 --bucket <b>on the origin, admin token). Once on, each link chooses--cloudor--local, with an admin-setstorage_default.<host>/*over the existing CNAME. The Worker checks R2 first; anything that is not a cloud record goes through to the tunnel withfetch(request). There is no internal hostname and no service token. The rollback is deleting one route.ls,stateandprofiles --jsontag each row withstorage,typeandby. Machine links getv:2pointer records, so members see them and ids never collide.filesfolds intodfoundation.f.d.foundationanswers 301 to the same path ons.d.foundation. The bucket and the guide's Access app (same AUD) are adopted in place.share migrate --to <mini>copies each share over ssh into the Mini'sshare import, keeping its id. The Mini gets a new tunnel, then the Air's tunnel and rows are retired. Nothing on the Air is deleted.Validation
/kit:spec-validateran three rounds with fresh-context reviewers (7, then 5, then 3 lenses):The remaining warnings are recorded in the Decision Log (round 3) and in the implementation notes as build rules.
Before anything runs in production
TASK-1a and TASK-1b are spikes on the edge behavior: route pass-through, Access running before a route Worker, custom-domain rebind, and AUD retention. Both production migrations (TASK-11a, TASK-11b) begin with an explicit stop for Han's go, after the throwaway-hostname rehearsals R1 and R2.