diff --git a/.azure/deployment-plan.md b/.azure/deployment-plan.md index b7c6cfe..e6d4a5a 100644 --- a/.azure/deployment-plan.md +++ b/.azure/deployment-plan.md @@ -3,7 +3,8 @@ **Direction: subscription-free attendee lab; existing-resource presenter capstone.** No new Azure deployment is required for the selected workshop. The organizer has an existing Azure DocumentDB instance available for a presenter-only demonstration. -Availability is not a claim that the application round trip has been rehearsed. +A previous local Mac mini → managed Azure round trip passed; this does not +prove the delivery revision on the conference laptop/network. ## Mandatory attendee path @@ -62,14 +63,22 @@ Automated gates in this revision: - Complete-reference recovery and expected 216-entry catalog. - No cloud endpoint used by attendee startup or database integration tests. -Still requiring environment-specific acceptance (not replaced by local/CI checks): +Executed on preceding main source (see [dated acceptance](../docs/acceptance.md)): -- [ ] Fresh revised attendee/organizer Codespaces builds and cold startup timing. -- [ ] Measured two-site resource needs on representative accounts/machines. -- [ ] Presenter actual laptop → approved managed Azure seed, copy-out, retry, copy-back. -- [ ] Approved real-network access and private credential delivery to the presenter. -- [ ] Verified backup recording and calendar-application import. -- [ ] Full 105-minute timed rehearsal, pairing fallback, export and safe stopping. +- [x] Fresh two-core attendee Codespace, 4m47s cold, actual Site A/B 34/34 plus integration. +- [x] Actual Google Calendar UI import, 3/3 events. +- [x] Local Mac mini → managed Azure round trip; temporary resource cleaned up. + +Still requiring environment-specific acceptance (not replaced by those receipts): + +- [x] Reconnect/onboarding revision: restarted attendee Codespace, 47/47 root tests, + 2/2 live integration, same-app cold recovery and original-wait-process recovery; + interruption cleanup 2/2. See the dated acceptance record for exact source scope. +- [ ] New cold Codespace creation on this revision (the polish retest reused the existing one). +- [ ] Fresh Organizer profile and representative account/machine capacity checks. +- [ ] Presenter existing-resource rehearsal on the conference laptop/network. +- [ ] Verified backup recording; Apple Calendar UI and repeat-import handling. +- [ ] Full 105-minute human rehearsal, pairing fallback, export and safe stopping. No unattended cleanup worker, cloud IAM grants, Kubernetes dependency, background sync, Azure app hosting or new cloud resources are introduced by this plan. diff --git a/.github/workflows/verify.yml b/.github/workflows/verify.yml index 52d03ee..78705a3 100644 --- a/.github/workflows/verify.yml +++ b/.github/workflows/verify.yml @@ -19,6 +19,7 @@ jobs: cache: npm - run: npm ci --ignore-scripts --no-audit --no-fund - run: npm test + - run: npm run test:docs - name: Start and seed only the two self-hosted sites run: npm run workspace:start - name: Exercise two independent database servers @@ -31,3 +32,16 @@ jobs: - name: Stop this ephemeral CI project only if: always() run: docker compose -f compose.yaml down --volumes + + reconnect-regression: + runs-on: ubuntu-latest + timeout-minutes: 20 + steps: + - uses: actions/checkout@v4 + - uses: actions/setup-node@v4 + with: + node-version: '24' + cache: npm + - run: npm ci --ignore-scripts --no-audit --no-fund + - name: First-use failure, original wait process, concurrent recovery, healthy outage + run: npm run test:reconnect diff --git a/README.md b/README.md index d92bf34..84c48af 100644 --- a/README.md +++ b/README.md @@ -19,10 +19,15 @@ usage; it is not a promise of free or offline hosting. The app includes the [official ATO schedule](https://2026.allthingsopen.org/schedule) snapshot captured October 12, 2026: **216 distinct scheduled entries, October 18–20**. -The earlier single-development-database app/API was rehearsed in Codespaces. -The revised two-site cold Codespaces startup, timed classroom delivery, calendar-app -import, and presenter Azure round trip still require their own acceptance rehearsal. -Automated local/CI checks are not proof of those environments. +On the preceding main source, a fresh **two-core attendee Codespace** reached both +sites in **4m47s**, actual Site A/B checks passed **34/34 plus integration**, Google +Calendar imported **3/3 events**, and a real **local Mac mini → managed Azure** +round trip passed (temporary cloud resource cleaned up). See the +[acceptance record](docs/acceptance.md) for scope. **This reconnect/onboarding +revision also passed a restarted-Codespace attendee run: 47/47 tests, actual +browser exercises, and cold first-connection recovery in the same app process.** +Timed human classroom delivery and conference-laptop networking are +not verified; automated checks are not substitutes. The web schedule is primary; October 9 Monday/Tuesday PDFs were cross-checked, not silently merged. See [source notes](docs/instructor.md#schedule-data) for @@ -36,9 +41,13 @@ cross-listings, unresolved disagreements, and the excluded invalid break time. private development credentials, starts both databases and the app, waits for each database, and seeds the same catalog in both. It does not copy favorites. 3. When the terminal says **Site A and Site B are ready and both catalogs are - seeded**, open port **3000** in the **Ports** tab. Keep visibility **Private**. -4. Follow the [attendee guide](docs/lab.md). `TODO_LIST_SESSIONS` is intentional - until you finish the first exercise; it is not a database connection failure. + seeded**, select **Ports** beside Terminal in the editor's bottom panel. Find + row **3000** and use **Open in Browser** (globe). Keep visibility **Private**. +4. Follow the [attendee guide](docs/lab.md), also linked persistently in the planner. + The blue **Exercise 1** checkpoint is intentional, not a broken database. +5. Edit in Codespaces: **Explorer → exercises → starter → repository.js**, or + **Ctrl/Cmd+P**, then type `exercises/starter/repository.js`. A GitHub file link + is a read-only source view, not the editable file in your running Codespace. Every workshop terminal command runs **inside Codespaces**, unless explicitly following the optional [laptop-local appendix](docs/local.md). @@ -163,7 +172,9 @@ The [captured source](data/schedule-source.json) and [audit report](data/schedule-import-report.json) preserve provenance. A replacement snapshot requires reviewed migration or fresh databases. -Calendar export is a snapshot, not a subscription. Published blocks may contain +Calendar export is a snapshot, not a subscription. Follow the +[Google Calendar / Apple Calendar import steps](docs/lab.md#9-take-your-selections-with-you). +Published blocks may contain sequential keynotes/lightning talks; no per-talk timing is invented. Overlap warnings need interpretation. Favorites neither reserve seats nor grant entry to separately registered events. @@ -205,6 +216,7 @@ With Node.js 24.13 or newer (already available inside Codespaces): ```sh npm ci --ignore-scripts npm test +npm run test:docs ``` With the two-site Compose stack running: @@ -217,8 +229,11 @@ Integration checks use uniquely named temporary databases on **two different local server services**, never the configured cloud endpoint. They remove only their own temporary databases. Checks cover independent favorites, additive round trips, timestamp/duplicate preservation, partial results, indexes, -migrations, and HTTP target switching. CI runs these checks on Linux; this is -not a live Azure or fresh Codespaces claim. +migrations with real before/after inspection, the optional read-only aggregation, +and HTTP target switching. For first-use failure/concurrent/wait-process recovery +on a **new isolated stack**, run `npm run test:reconnect` from the Docker host; +see [scope and cleanup](docs/acceptance.md#how-to-verify-the-revision). CI runs +these checks on Linux; this is not a fresh Codespaces or live Azure claim. See the [lab](docs/lab.md), [instructor guide](docs/instructor.md), [presenter runbook](docs/presenter.md), and [delivery plan](.azure/deployment-plan.md). diff --git a/docs/acceptance.md b/docs/acceptance.md new file mode 100644 index 0000000..a4ae1ce --- /dev/null +++ b/docs/acceptance.md @@ -0,0 +1,187 @@ +# Acceptance record and remaining gates + +## Executed evidence before the reconnect / attendee-polish revision + +Recorded October 17, 2026 against the preceding main source +`e6eae569f0879ea59656c847db6c13a1dee0b7d1` (or tree-identical rehearsal source): + +- **Fresh two-core attendee Codespace:** cold creation to both Site A/B catalogs + ready took **4 minutes 47 seconds** in that run. The actual Site A/B acceptance + checklist passed **34/34**, plus the live database integration test. This is + one measured environment, not a promised startup time for every account/network. +- **Attendee revisit:** the same Codespace was stopped/restarted and the two + starter exercises were completed again with new notebook database names. + Browsing, favorites, restart persistence, connection-only switching, selected + copying in both directions, safe repeat, migration and calendar download worked. + Editor focus required terminal assistance for the code replacements: not a + novice usability study or an entirely mouse-and-keyboard editing rehearsal. +- **Google Calendar web:** an actual downloaded calendar was imported through the + UI, **3/3 events**, with titles, timezones, UIDs, locations and Unicode checked. + No claim of Apple Calendar UI acceptance or universal repeat-import behavior. +- **Managed Azure data plane:** actual local **Mac mini workstation → managed + Azure DocumentDB** seed/switch/copy/retry/copy-back passed. The temporary cloud + resource used for that authorized test was cleaned up. This is real local-to- + managed evidence, not a conference-laptop/network rehearsal or an instruction + to provision attendee resources. Normal delivery still uses an approved + existing presenter resource and narrow access. + +No account emails, private Codespace identifiers, subscription IDs, connection +strings or credentials are needed in this public record. + +## Defect found during acceptance, and this revision's scope + +The preceding source had a confirmed first-connection recovery defect with the +pinned MongoDB Node.js driver 7.1.0: if a cached client's first lazy connection +failed, later lazy commands could see a closed topology even after the database +was healthy. A previously connected client recovered from an ordinary outage. +The prior success record must **not** be read as proof of first-failure recovery; +restarting the app was then a workaround. + +This revision explicitly awaits the driver's public `connect()` on every async +connection acquisition. The readiness CLI reacquires **inside** its retry loop. +It does not replay writes, change TLS policy, silently fall back to another site, +or replace credentials/volumes. Shutdown rejects new/racing acquisitions and +closes pools after pending connects settle. + +New offline tests cover connection lifecycles, concurrent acquisition, bounded +wait retries, exercise guidance, bounded document inspection and the optional +query's snapshot-derived expected result. The live regression command below +checks first-use failure, the original waiting CLI process, concurrent clients, +a healthy-client outage control, unchanged favorite digests, and the **same app +PID** across recovery. A test added to source is not an executed live result. + +## Executed reconnect / attendee-polish acceptance + +Recorded October 17, 2026. The same attendee Codespace was restarted and rebuilt +from `8993fac38709a5d6cdece1f8170ffe9392111d14`, then fast-forwarded to +`a3f9c5aeb16f8350b3ece7ca74887987ddc94824` for the harness-only cleanup update. +That update changed no application/UI code; the final record update is docs-only. +Earlier notebooks, credentials, volumes and solved exercises were preserved with +private backups and separate fresh notebook database names. + +- Final repository-root tests in Codespaces: **47/47 passed**, no skips; + **46 local documentation links** checked. +- Actual two-server database integration: **2/2 passed**, including the optional + official-catalog aggregation. +- Browser flow: blue Exercise 1, edit the first TODO, 216 entries, blue Exercise 2, + save overlapping favorites, persistence, initially empty Site B, two records + copied, repeat Already present, and a third favorite copied back. Complete- + reference switching preserved both solved files and notebook timestamps. +- **Main-app cold-connection regression:** Site B was stopped before that app + process ever acquired it. The browser switch failed with a genuine 503 and + stayed on Site A. After starting only Site B, the same browser retry succeeded + with the **same server PID, start time, container and request-token hash**. + No app restart or code edit was needed to recover. +- The isolated live harness independently proved recovery of the **original + waiting CLI process**, concurrent first acquisitions and a previously healthy + client. All scoped cleanup receipts passed. +- Interruption regressions: **2/2 passed** in Codespaces, including repeated + SIGINT/SIGTERM during cleanup and independent empty resource inventories. +- Actual document inspection showed the same session change from version 1 with + no attendanceMode to version 2/in-person. Migration changed **216 then 0**; + the favorite timestamp was unchanged. The optional solved database aggregation + returned **17 groups / 74 Monday entries**, matching the reference. +- Actual browser calendar download contained **three events**, validated against + the real state. This revision did not repeat the earlier Google Calendar import + or managed Azure test; calendar serialization and transfer semantics are unchanged. +- Resume to first SSH was about **39 seconds**; the explicit revised-code + build/start/seed took **27 seconds**, measured separately. Neither is a fresh + cold-creation measurement or a promised attendee timing. +- The exact Codespace was stopped and its **Shutdown** state verified; notebooks + and solved work remain retained. No new Azure resources were used in this retest. + +Application actions and dialogs used a real browser; code edits used the +Codespaces CLI after locating the editable files. This is expert-assisted +acceptance, not a novice or timed classroom study. During premerge testing, +canonical main guide links still served the previous guide, so the trusted +branch guide was used; the docs and app are published together. + +## How to verify the revision + +Offline, from the repository root with Node 24.13+: + +```sh +npm ci --ignore-scripts +npm test +npm run test:docs +``` + +On an already authorized attendee stack rebuilt from this source: + +```sh +docker compose -f compose.yaml exec -T app npm run test:database +``` + +The integration test creates/drops only uniquely named owned temporary databases +on the two self-hosted servers; it does not select cloud. Rebuild the app image +when scripts/tests/package metadata change; source bind mounts alone do not +update those copied image files. + +For the destructive-outage regression, **do not stop an attendee's databases**. +Run this from the repository root on a Docker host (including a disposable +Codespace terminal), not inside the app container: + +```sh +npm run test:reconnect +``` + +Requires Docker Compose **2.24.4+** for its port override. It creates a fresh random +`ato-reconnect-*` project, private temporary `.env`, independent volumes and a +random **loopback-only** app port. It ignores ambient workshop credentials and +never adopts an existing project. It stops/starts only its Site B, captures +sanitized pass/fail receipts, and removes **only its own ephemeral project and +volumes** through a single idempotent cleanup path. **SIGINT / SIGTERM** abort +in-flight HTTP requests and probe waits, cancel child commands (with a five-second +force-termination grace), and enter that same cleanup path with a nonzero exit. +Repeated signals do not cancel cleanup. Its Compose deadline is 120 seconds; +each resource-inventory check has a 15-second deadline (plus termination grace). +The private directory is removed only after successful `down` and independent +empty inventories for the project's **containers, volumes and networks**. + +Normal notebooks and `.env` are untouched. If cleanup or verification fails, +the process exits nonzero and retains its private configuration, printing only +the generated project name/private directory for manual review; it does not run +a global prune or claim success. SIGKILL, machine loss, or an unreachable Docker +daemon cannot guarantee cleanup. CI runs the normal regression as a separate +ephemeral job. The application runs directly without file watching in this +harness so checking its PID really establishes no app restart. + +Opt-in interruption regression (Docker host, separate from the offline suite): + +```sh +node --test test/reconnect-interruption.integration.js +``` + +This creates two fresh owned projects sequentially, sends SIGTERM and SIGINT +respectively after startup, repeats each signal during cleanup, and independently +checks nonzero handled exits, absent containers/volumes/networks, and removal of +only each run's private temporary directory. It does not stop an existing stack. + +**Harness-only proof, October 17:** after adding handled-signal cleanup, the +normal live reconnect regression passed all five recovery/cleanup checks. The +real interruption test passed both SIGTERM (exit 143) and SIGINT (exit 130), +including repeated signals during cleanup. Independent filtered inventories +confirmed zero owned containers, volumes and networks and absent private +configuration directories for all three runs. These harness results do not +constitute another Codespace, browser, calendar or Azure acceptance run. + +Then reopen/restart the actual attendee Codespace and follow [the lab](lab.md): +verify the friendly first screen, edit TODO 1, save TODO 2, inspect before/after, +optional query, Site A/B copying and calendar helper links in a real browser. +Offline UI mocks are not a rendered-browser acceptance test. + +## Still unverified delivery gates + +- A brand-new cold Codespace on this revision (the executed polish test restarted + the existing Codespace; the earlier 4m47s cold result remains scoped above). +- Fresh Organizer-profile acceptance (not required for attendees). +- Representative attendee accounts, quotas, resource needs, and conference + laptop/Wi-Fi networking; one two-core Codespace measurement is not capacity proof. +- Full **105-minute human classroom rehearsal**, including pairing fallback, + editing fluency, export and safe stopping. +- Approved existing-resource presenter rehearsal on the actual conference laptop + and network, and a verified backup recording. +- Apple Calendar GUI import and client-specific repeated-import handling. + +The default attendee path remains Site A/B only. It never requires Azure, +Kubernetes, a laptop Docker installation, or automatic calendar API access. diff --git a/docs/instructor.md b/docs/instructor.md index 85cd558..2b49597 100644 --- a/docs/instructor.md +++ b/docs/instructor.md @@ -60,13 +60,20 @@ Optional GitHub CLI/SSH diagnostics are helper tools, not attendee prerequisites ## Observable checkpoints -- Both sites ready and seeded; the intentional TODO explains the empty planner. +- Both sites ready and seeded; a blue Exercise 1 checkpoint explains the empty + planner, points to the editable file, and links to progressively revealed hints. + The persistent implementation label describes the chosen file, not completion. - TODO 1: 216 entries. TODO 2: save two favorites and retain them across app restart. - Switch A → B: same catalog, no automatic favorite movement; A remains intact. - Copy A → B; switch back to A before repeating that same direction; Already present. - New favorite at B; copy B → A; original A choices survive. - Removal is local to its site, not propagated to copies. -- Export from the intended site, preserve code, stop without deleting volumes. +- Inspect an actual session before/after migration and an unchanged favorite + timestamp; explain zero migrated on rerun as idempotency. Index definition is + visible, not a performance claim. Optional advanced aggregation: 17 Monday + track groups / 74 published program entries, not attendee counts. +- Export from the intended site, follow calendar-specific import instructions, + preserve code, stop without deleting volumes. ## Schedule data @@ -151,8 +158,10 @@ Use an approved dedicated demo database; no generic deletion of shared resources Use actual laptop-local DocumentDB for the live hybrid story. A Codespaces source must instead be labeled self-hosted-to-managed. Rehearse the same catalog, local favorites, empty managed favorites, explicit copy/retry, and return trip before -class. The repo's earlier managed round trip remains unverified until that proof -is recorded. Have a checked recording; if live connectivity fails, switch to it +class. A local Mac mini → managed Azure round trip has now passed on the preceding +source, with its temporary resource cleaned up; see the [acceptance record](acceptance.md). +That is not proof of this revision on the conference laptop/network. Have a +checked recording; if live connectivity fails, switch to it rather than spend attendee time debugging access. Validated cloud TLS stays enabled. Observe current source IP/DNS and use approved @@ -190,11 +199,17 @@ retain Site A's existing volume. ## Rehearsal checklist +See the [dated acceptance record](acceptance.md) for what actually ran on the +preceding source and what still needs proof on this revision. + - Fresh attendee Codespace with both instances and measured resource/startup cost. - Setup upgrade from a legacy `.env` without changing old credentials/data. - Starter/reference switching and restart persistence at both sites. - Same catalog, independent favorites before copying, safe repeat/return trip. - Second-site outage: clear error, no fallback; recovery retains both notebooks. + Include B unavailable on its **first** use, then restored with the **same app + PID**. `npm run test:reconnect` performs that only in its own disposable project; + never run manual outage drills on attendees' existing notebooks. - Usable calendar import, honest simulation labels, private forwarded port. - Actual presenter laptop/Azure round trip, narrow access, validated TLS, recording. - Safe stopping, code/data retention explanation, existing-resource cleanup owner. diff --git a/docs/lab.md b/docs/lab.md index 2150adb..a21adf1 100644 --- a/docs/lab.md +++ b/docs/lab.md @@ -23,20 +23,37 @@ before seeding. Never delete volumes to get past a catalog mismatch. Follow [Start in Codespaces](../README.md#start-in-codespaces), choosing **ATO 2026 - Attendee**. Wait for **both** sites to be ready and seeded, then open -private port **3000**. Run the commands below in the Codespaces terminal. +private port **3000**. In the bottom panel of the Codespaces editor, choose +**Ports** (beside Terminal), find row **3000**, then use its **Open in Browser** +(globe) action. If Ports is hidden, use **View → Terminal**, then select **Ports** +in that panel. Keep **Port Visibility → Private** in the row's context menu. +Run the commands below in the **Terminal** tab of this same Codespace. + +The planner's top-right **Attendee guide (opens new tab)** link stays visible +throughout the lab. The exercise checkpoint also links directly to its step and +hints. Keep the guide and your private planner open in separate tabs. If account policy or quota blocks startup, tell a helper early for pairing or follow-along. Do not share logins, expose public ports, or install Docker during the timed lab. Retry a diagnosed setup failure with `npm run workspace:start`. Check the active label: **Site A — self-hosted**. The implementation label should -say **TODO starter**. `TODO_LIST_SESSIONS` is intentional; `DATABASE_UNAVAILABLE` +say **Exercise implementation (your editable code)**. This labels the selected +file, not whether you have finished it. The blue **Exercise 1 — Read the catalog** +checkpoint is expected; `DATABASE_UNAVAILABLE` is not. The internal CLI name `local` means Site A; `site-b` means Site B. Neither means managed Azure. Both sites have separate credentials and volumes. ## 2. Read the catalog -Open [exercises/starter/repository.js](../exercises/starter/repository.js). +In the **Codespaces editor**, use the left **Explorer** panel: +**exercises → starter → repository.js**. Or press **Ctrl+P** (Windows/Linux) / +**Cmd+P** (Mac), type `exercises/starter/repository.js`, and press Enter. + +The [GitHub source link](../exercises/starter/repository.js) opens a **read-only +file view**, not your running Codespace's editable file. Edit in Codespaces, not +in that GitHub tab. Leave the imports and `saveFavorite` function in place. + Complete `listSessions(db, catalogVersion)`: - Use `sessions`. @@ -45,7 +62,44 @@ Complete `listSessions(db, catalogVersion)`: - Return the cursor's array. Do not change connection code. Save and refresh the browser: expect **216 entries**. -The same repository function will run against both sites. +The same repository function will run against both sites. If a genuine connection +or catalog error appears instead of the blue checkpoint, diagnose that error; +completing a TODO will not repair a database outage. + +
+Hint 1 — What does find return? + +`db.collection("sessions")` selects a collection. `find({ catalogVersion })` +selects only this seeded catalog's documents and returns a **cursor**, not an +array. A cursor describes work for the server; `toArray()` executes/collects its +results for this small, bounded catalog. Return that promise (or await it). + +
+ +
+Hint 2 — Stable ordering + +Chain `.sort({ startsAt: 1, _id: 1 })` between `find` and `toArray`. +`1` means ascending; `_id` breaks ties between parallel entries with the same +start time. `_id` is the stable session identifier, not the array position. + +
+ +
+Solution / recovery reference + +```js +return db.collection("sessions") + .find({ catalogVersion }) + .sort({ startsAt: 1, _id: 1 }) + .toArray(); +``` + +Compare with [the complete repository](../examples/complete/repository.js), or +use the non-destructive reference mode below. Do not replace the whole starter +file if you only want to complete one exercise. + +
Need recovery without replacing your exercise? @@ -69,6 +123,47 @@ Complete `saveFavorite(db, sessionId)`: - Set `savedAt` with `$setOnInsert` so retries preserve the initial timestamp. - Return the driver's update result. +Save the file and refresh the planner. Trying **Save favorite** before this +function is implemented shows the blue **Exercise 2** checkpoint, not a database +failure. After implementation, try the save again. + +
+Hint 1 — Give a favorite a stable identity + +Use `db.collection("favorites").updateOne(...)`. Its first argument is +`{ _id: sessionId }`. The database's unique `_id` identifies one favorite per +session, so repeating a save selects the same record rather than another row. +An `upsert` updates a match, or inserts a document when nothing matches. + +
+ +
+Hint 2 — Preserve when it was first saved + +The second argument is an update document. `$set` would overwrite a timestamp +on every save; `$setOnInsert` applies only when the upsert inserts a new record. +Use `{ $setOnInsert: { savedAt: new Date().toISOString() } }` and the third +argument `{ upsert: true }`. Return the driver's result so the UI can distinguish +`upsertedCount === 1` (new) from an already-saved favorite. + +
+ +
+Solution / recovery reference + +```js +return db.collection("favorites").updateOne( + { _id: sessionId }, + { $setOnInsert: { savedAt: new Date().toISOString() } }, + { upsert: true }, +); +``` + +The [complete repository](../examples/complete/repository.js) uses the same +standard MongoDB driver, not a different deployment-specific SDK. + +
+ Save two genuinely parallel Monday sessions, for example: - “Local to Cloud and Back: Building Hybrid Applications on Open Source” @@ -103,7 +198,10 @@ Switch back to **Site A** and confirm the originals remain. No repository-method rewrite was required. Setup seeded catalogs, not favorites. If switching fails, the API does not silently fall back. Check the active label, -sanitized error, and helper guidance. Do not delete data to repair connectivity. +sanitized error, and helper guidance. Once that site is healthy, retry the switch +in the same app: acquisition explicitly reconnects, including after a failed +first connection. This does **not** replay a failed save/copy automatically. +Do not delete data or restart the app as a routine connection repair. ## 5. Copy selected favorites and retry @@ -152,28 +250,141 @@ that as self-hosted-to-managed, not on-prem-to-cloud. ## 8. Make changes repeatable -If time permits, follow the instructor against Site A: +If time permits, follow the instructor against **Site A (`local`)**. First inspect +one actual session and one actual favorite, then add the index and migrate: ```sh +docker compose -f compose.yaml exec app npm run inspect -- --target local docker compose -f compose.yaml exec app npm run indexes -- --target local docker compose -f compose.yaml exec app npm run migrate -- --target local +docker compose -f compose.yaml exec app npm run inspect -- --target local +docker compose -f compose.yaml exec app npm run migrate -- --target local docker compose -f compose.yaml exec app npm run check -- --target local ``` -Run the migration again: expect **zero newly migrated records**. It adds -`attendanceMode` to version-1 documents. Re-seeding does not undo it. Use -`--target site-b` for the same operations on Site B. +Compare the **same session `_id`** in the two `inspect` outputs. On an unmigrated +notebook, the first actual document has `schemaVersion: 1` and **no** +`attendanceMode` field. After migration it has `schemaVersion: 2` and +`attendanceMode: "in-person"`. The title/time are unchanged. These are projected +fields from a real query, not canned example JSON; only one session and one +favorite are printed. No credentials or system documents are printed. + +The favorite output shows its actual `_id` and `savedAt` timestamp. Neither +index creation nor this session migration changes that timestamp. If `favorite` +is `null`, save a session on **this target** and inspect again. Repeated saves +and copies preserve each destination's first-save timestamp (Exercise 2). + +The second migration should report **zero newly migrated records**: the filter +only matches version-1 records without `attendanceMode`, so already migrated +records are untouched. This is **idempotency**, not a failed migration. If your +notebook was migrated earlier, the first inspection will already show version 2 +and both runs can report zero. Do not reset it just to recreate the before state. +Re-seeding does not undo migration. To try on Site B, substitute `--target site-b` +in **every** command; the browser's active target does not select a CLI target. + +`inspect` also displays the actual `schedule_order` index definition after +creation; `check` lists index names and document versions. This makes the index +repeatable and visible, **not** proof that the engine chooses it or that a tiny +catalog gets faster. Skip this extension before sacrificing copy-back or export. + +### Optional developer challenge — Monday program entries by track + +This is an **optional read-only aggregation**, not a third required planner TODO. +It queries the real database, not an in-memory replacement. It counts **published +program entries**, including breaks/registration and shared blocks—not attendees, +seat capacity, or unique speakers. Finish the two required exercises first. + +1. In Codespaces, open `exercises/optional/program-counts.js` with Explorer or + Ctrl/Cmd+P. Replace its deliberate optional exception with an aggregation + pipeline returned by `programCountsPipeline(catalogVersion)`. +2. Match this catalog and Monday **October 19, 2026 in America/New_York**. + Group by `track`, count each entry as one, then sort by count descending and + track ascending to break ties. Return rows shaped `{ _id: , entries: }`. +3. Run on explicit **Site A**, without changing the planner: + + ```sh + docker compose -f compose.yaml exec app npm run challenge -- --target local + ``` + + Or use `--target site-b` to compare that site's seeded catalog. Azure is not + accepted by this runner. It permits only read-only stages, limits output to + 30 groups, and uses a five-second server query budget. + +
+Hint 1 — Date type and conference timezone + +`startsAt` is a normalized **ISO UTC string**, not a BSON Date. String comparisons +work for these consistently formatted timestamps. Monday's bounds are inclusive +`2026-10-19T04:00:00.000Z` and exclusive `2026-10-20T04:00:00.000Z` +(EDT = UTC−4). Use `$match` with `catalogVersion` and `$gte` / `$lt` on `startsAt`. +Do not use today's date or midnight UTC for a conference-local day. + +
+ +
+Hint 2 — Group and count, then order + +Use `{ $group: { _id: "$track", entries: { $sum: 1 } } }`. +In an aggregation expression `"$track"` reads that field; `"track"` would be a +literal string and collapse all entries into one group. Follow with +`{ $sort: { entries: -1, _id: 1 } }`. + +
+ +
+Solution and expected result from the checked-in snapshot + +See [the complete pipeline](../examples/complete/program-counts.js), or execute +it without replacing your optional exercise: + +```sh +docker compose -f compose.yaml exec app npm run challenge -- --target local --reference +``` + +The October 12 snapshot produces **17 track groups totaling 74 Monday entries**. +The first three are `Inclusion & Diversity in Open Source: 10`, `Break: 9`, and +`Open Source Accessibility Summit: 8`. The offline test derives these from the +216-entry snapshot; the live integration test compares every aggregate row with +an independent conference-timezone calculation. Different snapshots must not +silently reuse these expected counts. -The index exercise makes the definition repeatable; it does not guarantee an -engine uses it for sorting or that a tiny dataset visibly speeds up. Compare -index names and document versions. Skip this extension before sacrificing the -return trip, calendar export, or cleanup. +
## 9. Take your selections with you On the site containing your final choices, choose **Download calendar (.ics)**. -Import the snapshot manually into your calendar. Conflicting selections are -exported too; warnings inform you rather than silently choosing for you. +Save the `.ics` file when prompted; the helper link below that button returns to +these import steps. Conflicting selections are exported too; warnings inform you +rather than silently choosing for you. + +**Google Calendar (web):** + +1. Open Google Calendar in your own account. If experimenting, first create a + separate calendar (Settings → Add calendar → Create new calendar). +2. Open **Settings → Import & export → Import**. +3. Choose **Select file from your computer**, select the downloaded `.ics`, then + choose the intended **Add to calendar** destination. Check it before importing. +4. Click **Import**, then return to the calendar and inspect the imported events. + +**Apple Calendar (macOS):** + +1. Open Calendar. For experiments, create a separate calendar with **File → New + Calendar** and select the account/location you intend to use. +2. Choose **File → Import**, select the downloaded `.ics`, and choose **Import**. +3. In the destination dialog, select that calendar (or create a new one) and + confirm. Inspect the events in Calendar afterward. + +Check the displayed timezone: the workshop starts **October 19, 2026 at 9:00 a.m. +Eastern = 6:00 a.m. Pacific**. The file encodes UTC instants; your calendar may +show them in your own timezone. A shifted display is not automatically a bad +export. + +This is a **snapshot, not a live subscription**. A later favorite change or +schedule update does not update an imported calendar. Although event IDs are +stable, calendar clients differ in how they handle repeated imports; do not +promise universal deduplication or update-in-place. Use your own separate test +calendar if experimenting, review it before reimporting, and never import into +a shared/team calendar unintentionally. The workshop makes no Google API writes. The export contains real titles/dates from the captured schedule. Check the [official schedule](https://2026.allthingsopen.org/schedule) for changes and diff --git a/docs/local.md b/docs/local.md index a517aab..7f08238 100644 --- a/docs/local.md +++ b/docs/local.md @@ -20,7 +20,7 @@ docker compose -f compose.yaml exec app npm run seed -- --target site-b Stop at any failing command, diagnose it, then retry; do not skip readiness or pretend the second site is available. These commands access only local sites. -Open **http://localhost:3000**. The starter reports `TODO_LIST_SESSIONS` until +Open **http://localhost:3000**. The starter shows a blue **Exercise 1** checkpoint until the first exercise is complete. Follow the [lab](lab.md); here both independent sites run on your physical laptop rather than inside Codespaces. It is still a two-site simulation, not two physical sites or cloud providers. diff --git a/docs/organizer.md b/docs/organizer.md index 2ca60d3..e1e52ea 100644 --- a/docs/organizer.md +++ b/docs/organizer.md @@ -10,7 +10,8 @@ The presenter uses one existing, authorized Azure instance for an optional capst 1. Confirm GitHub/Codespaces account access, available usage, machine capacity, conference connectivity, storage retention and the pairing/follow-along plan. 2. Measure a fresh **ATO 2026 - Attendee** Codespace with both database services. - A previous warm single-database rehearsal is not a two-site cold-start result. + The preceding main source measured 4m47s on a fresh two-core attendee Codespace; + repeat on the delivery revision and representative accounts/network. 3. Rehearse TODOs/reference recovery, app restart persistence, both copy directions, duplicate-safe retry, second-site failure/recovery and a real calendar-app import. 4. Follow the [presenter runbook](presenter.md) on the actual presenter laptop, @@ -63,6 +64,10 @@ merely to deliver the current attendee lab or existing-instance presenter demo. Automated tests cover configuration/scenario separation, legacy `.env` preservation, catalog ownership, bounded two-site startup, and real two-server data flows in CI. -They do not replace fresh Codespaces, conference network, live Azure, calendar-app, -or timed-classroom rehearsal. Record those results only after executing them. -See the [delivery plan](../.azure/deployment-plan.md). +The preceding source has fresh two-core Codespaces (4m47s; 34/34 plus integration), +Google Calendar (3/3), and actual local Mac mini → managed Azure data-plane proof; +the temporary Azure resource was cleaned up. See the [acceptance record](acceptance.md) +for the restarted-Codespace polish retest and verified same-process reconnect fix. +Human timing, new cold creation on this revision, and conference-laptop/network +proof remain pending. See also the +[delivery plan](../.azure/deployment-plan.md). diff --git a/examples/complete/program-counts.js b/examples/complete/program-counts.js new file mode 100644 index 0000000..ea50aca --- /dev/null +++ b/examples/complete/program-counts.js @@ -0,0 +1,12 @@ +export function programCountsPipeline(catalogVersion) { + return [ + { $match: { + catalogVersion, + // Monday Oct 19 in America/New_York (EDT = UTC-4), not the current date. + // startsAt is a normalized ISO UTC string, not a BSON Date in this catalog. + startsAt: { $gte: "2026-10-19T04:00:00.000Z", $lt: "2026-10-20T04:00:00.000Z" }, + } }, + { $group: { _id: "$track", entries: { $sum: 1 } } }, + { $sort: { entries: -1, _id: 1 } }, + ]; +} diff --git a/exercises/optional/program-counts.js b/exercises/optional/program-counts.js new file mode 100644 index 0000000..b898f0c --- /dev/null +++ b/exercises/optional/program-counts.js @@ -0,0 +1,8 @@ +import { WorkshopError } from "../../src/errors.js"; + +// OPTIONAL: never imported by the planner or either required exercise. +// Build a pipeline counting Monday's published program entries by track. +// The guide has a starting point, progressive hints, and a reference solution. +export function programCountsPipeline(catalogVersion) { + throw new WorkshopError(501, "OPTIONAL_QUERY", "Build programCountsPipeline in exercises/optional/program-counts.js, or add --reference to run the read-only solution. See the optional challenge in docs/lab.md."); +} diff --git a/package.json b/package.json index a328f0c..f0af064 100644 --- a/package.json +++ b/package.json @@ -19,8 +19,12 @@ "indexes": "node scripts/database.mjs indexes", "migrate": "node scripts/database.mjs migrate", "check": "node scripts/database.mjs check", + "inspect": "node scripts/database.mjs inspect", + "challenge": "node scripts/program-counts.mjs", + "test:reconnect": "node scripts/test-reconnect.mjs", "import:schedule": "node scripts/import-schedule.mjs", "test": "node --test test/*.test.js", + "test:docs": "node scripts/check-doc-links.mjs", "test:database": "node --test test/database.integration.js", "wait:database": "node scripts/database.mjs wait" }, diff --git a/scripts/check-doc-links.mjs b/scripts/check-doc-links.mjs new file mode 100644 index 0000000..a2be140 --- /dev/null +++ b/scripts/check-doc-links.mjs @@ -0,0 +1,32 @@ +// Repository-root documentation check, intentionally separate from runtime +// npm test: the app image doesn't contain README/docs/.azure authoring files. +import { readFile, readdir, stat } from "node:fs/promises"; +import { dirname, join, resolve } from "node:path"; +import { fileURLToPath } from "node:url"; + +const root = fileURLToPath(new URL("../", import.meta.url)); +async function markdownFiles(directory) { + const entries = await readdir(directory, { withFileTypes: true }); + return (await Promise.all(entries.map((entry) => entry.isDirectory() + ? markdownFiles(join(directory, entry.name)) + : entry.name.endsWith(".md") ? [join(directory, entry.name)] : []))).flat(); +} +const files = [join(root, "README.md"), ...await markdownFiles(join(root, "docs")), ...await markdownFiles(join(root, ".azure"))]; +let checked = 0; +for (const file of files) { + const source = await readFile(file, "utf8"); + for (const match of source.matchAll(/\[[^\]]*\]\(([^\s)]+)(?:\s+"[^"]*")?\)/g)) { + if (/^[a-z]+:/i.test(match[1])) continue; + const [path, anchor] = match[1].split("#"); + const target = path ? resolve(dirname(file), decodeURIComponent(path)) : file; + await stat(target); // Fail on missing files; never fetch external URLs. + if (anchor && target.endsWith(".md")) { + const content = await readFile(target, "utf8"); + const anchors = [...content.matchAll(/^#{1,6}\s+(.+)$/gm)].map(([, title]) => title + .toLowerCase().replace(/[^\p{L}\p{N}_\-\s]/gu, "").replace(/\s/g, "-")); + if (!anchors.includes(decodeURIComponent(anchor))) throw new Error(`Missing heading link: ${match[1]} in ${file}`); + } + checked++; + } +} +console.log(`Checked ${checked} local documentation links; no network requests.`); diff --git a/scripts/database.mjs b/scripts/database.mjs index 31707f5..58316db 100644 --- a/scripts/database.mjs +++ b/scripts/database.mjs @@ -1,33 +1,54 @@ +import { pathToFileURL } from "node:url"; +import { inspectDatabase } from "../src/inspection.js"; import { setTimeout as delay } from "node:timers/promises"; import { readConfig } from "../src/config.js"; import { createConnections } from "../src/connections.js"; import { readCatalog, requireCatalog, seedCatalog } from "../src/catalog.js"; import { WorkshopError, reportError } from "../src/errors.js"; +// Retry only readiness, not writes. The deadline may be exceeded by at most +// one bounded driver acquisition/ping already in flight. +export async function waitForDatabase(connections, target, { + timeoutMS = 240000, now = Date.now, sleep = delay, +} = {}) { + const deadline = now() + timeoutMS; + for (;;) { + try { + // Acquisition MUST be inside this try on every attempt: the first + // connection itself may fail, before there is any Db handle to ping. + const db = await connections.db(target); + await db.command({ ping: 1 }); + return; + } catch (error) { + const remaining = deadline - now(); + if (remaining <= 0) throw error; + await sleep(Math.min(2000, remaining)); + if (now() >= deadline) throw error; + } + } +} + async function run() { const [operation, flag, target, ...extra] = process.argv.slice(2); - if (!["wait", "seed", "indexes", "migrate", "check"].includes(operation) || + if (!["wait", "seed", "indexes", "migrate", "check", "inspect"].includes(operation) || flag !== "--target" || !["local", "site-b", "cloud"].includes(target) || extra.length) { - throw new WorkshopError(400, "USAGE", "Use: npm run -- --target . The target is always explicit."); + throw new WorkshopError(400, "USAGE", "Use: npm run -- --target . The target is always explicit."); } const connections = createConnections(readConfig()); try { - const db = connections.db(target); + if (operation === "inspect" && target === "cloud") { + throw new WorkshopError(400, "LOCAL_TARGET_REQUIRED", "The attendee inspection exercise accepts only local (Site A) or site-b."); + } if (operation === "wait") { - const deadline = Date.now() + 240000; - for (;;) { - try { - await db.command({ ping: 1 }); - console.log(JSON.stringify({ target, ready: true })); - return; - } catch (error) { - if (Date.now() >= deadline) throw error; - await delay(2000); - } - } + await waitForDatabase(connections, target); + console.log(JSON.stringify({ target, ready: true })); + return; } + const db = await connections.db(target); await db.command({ ping: 1 }); - if (operation === "seed") { + if (operation === "inspect") { + console.log(JSON.stringify({ target, ...await inspectDatabase(db) }, null, 2)); + } else if (operation === "seed") { const result = await seedCatalog(db, await readCatalog()); console.log(JSON.stringify({ target, ...result, favoritesChanged: false })); } else if (operation === "indexes") { @@ -40,7 +61,11 @@ async function run() { { catalogVersion: catalog.fingerprint, schemaVersion: 1, attendanceMode: { $exists: false } }, { $set: { attendanceMode: "in-person", schemaVersion: 2 } }, ); - console.log(JSON.stringify({ target, migrated: result.modifiedCount })); + console.log(JSON.stringify({ target, migrated: result.modifiedCount, + message: result.modifiedCount === 0 + ? "No eligible version-1 records remain. Re-running is idempotent; existing values were preserved." + : "Added attendanceMode and advanced eligible sessions to schemaVersion 2. Run inspect on this same target to see a real document.", + })); } else { const catalog = await requireCatalog(db); const [sessions, favorites, indexes, schemaVersions] = await Promise.all([ @@ -56,7 +81,7 @@ async function run() { } } -run().catch((error) => { +if (process.argv[1] && import.meta.url === pathToFileURL(process.argv[1]).href) run().catch((error) => { reportError(error); if (error instanceof WorkshopError) console.error(error.message); process.exitCode = 1; diff --git a/scripts/program-counts.mjs b/scripts/program-counts.mjs new file mode 100644 index 0000000..6beee30 --- /dev/null +++ b/scripts/program-counts.mjs @@ -0,0 +1,47 @@ +import { pathToFileURL } from "node:url"; +import { readConfig } from "../src/config.js"; +import { createConnections } from "../src/connections.js"; +import { requireCatalog } from "../src/catalog.js"; +import { WorkshopError, reportError } from "../src/errors.js"; + +// A deliberately small read-only stage vocabulary. $out/$merge and nested +// write pipelines are not accepted by this exercise runner. +export function validatePipeline(pipeline) { + if (!Array.isArray(pipeline) || pipeline.length < 1 || pipeline.length > 8 || pipeline.some((stage) => + !stage || Object.keys(stage).length !== 1 || !["$match", "$group", "$sort", "$limit", "$project"].includes(Object.keys(stage)[0]))) { + throw new WorkshopError(400, "READ_ONLY_PIPELINE", "Use 1–8 stages: $match, $group, $sort, $limit, or $project. No write stages are allowed."); + } + return pipeline; +} + +export async function programCounts(db, buildPipeline) { + const catalog = await requireCatalog(db); + const pipeline = validatePipeline(buildPipeline(catalog.fingerprint)); + // Force the actual active catalog boundary even if an exercise omits it. + const rows = await db.collection("sessions").aggregate([ + { $match: { catalogVersion: catalog.fingerprint } }, ...pipeline, + { $project: { _id: 1, entries: 1 } }, { $limit: 30 }, + ], { maxTimeMS: 5000 }).toArray(); + return { catalog: catalog.version, rows, outputLimit: 30 }; +} + +async function run() { + const [flag, target, mode, ...extra] = process.argv.slice(2); + if (flag !== "--target" || !["local", "site-b"].includes(target) || (mode && mode !== "--reference") || extra.length) { + throw new WorkshopError(400, "USAGE", "Use npm run challenge -- --target [--reference]. This optional query is read-only and excludes Azure."); + } + const implementation = mode === "--reference" + ? await import("../examples/complete/program-counts.js") + : await import("../exercises/optional/program-counts.js"); + const connections = createConnections(readConfig()); + try { + const result = await programCounts(await connections.db(target), implementation.programCountsPipeline); + console.log(JSON.stringify({ target, ...result }, null, 2)); + } finally { await connections.close(); } +} + +if (process.argv[1] && import.meta.url === pathToFileURL(process.argv[1]).href) run().catch((error) => { + reportError(error); + if (error instanceof WorkshopError) console.error(error.message); + process.exitCode = 1; +}); diff --git a/scripts/test-reconnect.mjs b/scripts/test-reconnect.mjs new file mode 100644 index 0000000..7bba454 --- /dev/null +++ b/scripts/test-reconnect.mjs @@ -0,0 +1,249 @@ +// Host-side, opt-in live regression. NEVER points at an existing workshop stack. +// Requires Docker Compose >=2.24.4 (!override). Generates its own private .env, +// project name, random loopback port and volumes; cleanup selects that project. +import assert from "node:assert/strict"; +import { spawn } from "node:child_process"; +import { randomUUID } from "node:crypto"; +import { mkdtemp, writeFile, rm } from "node:fs/promises"; +import { tmpdir } from "node:os"; +import { join } from "node:path"; +import { fileURLToPath } from "node:url"; +import { createInterface } from "node:readline"; +import { setup } from "./setup.mjs"; + +const root = fileURLToPath(new URL("../", import.meta.url)); +const project = `ato-reconnect-${randomUUID().replaceAll("-", "")}`; +// Install handlers before the first asynchronous work, including private setup. +const interrupted = new AbortController(); +let receivedSignal; +const children = new Set(); +const onSignal = (signal) => { + receivedSignal ||= signal; + interrupted.abort(new Error("Harness interrupted")); + // Cleanup commands are deliberately immune to repeated signals. Their own + // deadlines still apply; killing cleanup would recreate the original leak. + for (const running of children) if (!running.cleanup) running.cancel(); +}; +process.on("SIGINT", onSignal); +process.on("SIGTERM", onSignal); +const directory = await mkdtemp(join(tmpdir(), "ato-reconnect-")); +const override = join(directory, "compose.reconnect.yaml"); +// Preserve only ordinary runtime environment; explicitly override interpolation +// inputs that could otherwise choose a host port, mode or private forwarding host. +const env = { ...process.env }; +for (const key of Object.keys(env)) { + if (/^(COMPOSE_|WORKSHOP_|DOCUMENTDB_|SITE_B_|LOCAL_|CLOUD_)/.test(key) || key === "ACTIVE_TARGET") delete env[key]; +} +Object.assign(env, { COMPOSE_PROJECT_NAME: project, WORKSHOP_PORT: "3000", + WORKSHOP_MODE: "complete", WORKSHOP_SCENARIO: "attendee", WORKSHOP_PUBLIC_HOST: "", + LOCAL_DATABASE_NAME: "reconnect_a", SITE_B_DATABASE_NAME: "reconnect_b" }); +const base = ["compose", "--project-name", project, "--project-directory", root, + "--env-file", join(directory, ".env"), "-f", join(root, "compose.yaml"), + "-f", join(root, "compose.complete.yaml"), "-f", override]; +function command(args, { timeout = 300000, input = "ignore", cleanup = false } = {}) { + if (!cleanup) interrupted.signal.throwIfAborted(); + const child = spawn("docker", args, { cwd: root, env, stdio: [input, "pipe", "pipe"] }); + let stdout = ""; + let forceTimer; + let cancelled = false; + const cancel = () => { + if (cancelled) return; + cancelled = true; + child.kill("SIGTERM"); + forceTimer = setTimeout(() => child.kill("SIGKILL"), 5000); + }; + const running = { child, cleanup, cancel }; + children.add(running); + // Capture no stderr: compose/build/driver failures can contain private values. + child.stdout.on("data", (chunk) => { stdout += chunk; if (stdout.length > 2000000) cancel(); }); + child.stderr.resume(); + // Closing a cancelled interactive pipe can race a probe's next write. + child.stdin?.on("error", () => cancel()); + const timer = setTimeout(cancel, timeout); + running.result = new Promise((resolve) => { + // A spawn error is followed by close; wait for close before cleanup so a + // cancelled Compose process cannot still create resources after down. + child.once("error", () => { cancelled = true; }); + child.once("close", (code) => { + clearTimeout(timer); clearTimeout(forceTimer); children.delete(running); + resolve({ ok: code === 0 && !cancelled, stdout }); + }); + }); + return running; +} +const request = (url, options = {}) => fetch(url, { + ...options, signal: AbortSignal.any([interrupted.signal, AbortSignal.timeout(20000)]), +}); +async function run(args, options) { + const result = await command([...base, ...args], options).result; + interrupted.signal.throwIfAborted(); + assert.ok(result.ok, "Scoped Compose command failed (raw output suppressed)"); + return result.stdout.trim(); +} +const exec = (...args) => run(["exec", "-T", "app", ...args]); +const receipt = (check, details = {}) => console.log(JSON.stringify({ check, passed: true, ...details })); +let owned = false; +let cleanupPromise; +let probe; +let failure = false; +let phase = "ownership-and-startup"; +try { + // An existing project, however unlikely, must never be adopted or cleaned up. + for (const list of [["ps", "-aq"], ["volume", "ls", "-q"], ["network", "ls", "-q"]]) { + const inventory = await command([...list, "--filter", `label=com.docker.compose.project=${project}`]).result; + assert.ok(inventory.ok && !inventory.stdout.trim(), "Project ownership preflight failed"); + } + // setup ignores ambient credentials and generates independent private values. + await setup(directory, {}, () => {}); + await writeFile(override, `services:\n app:\n init: false\n command: ["node", "src/server.js"]\n ports: !override\n - "127.0.0.1::3000"\n`, { mode: 0o600 }); + owned = true; + await run(["up", "--build", "-d", "--wait", "--wait-timeout", "240"], { timeout: 600000 }); + receipt("owned-ephemeral-project-ready", { project, privateDirectory: directory }); + for (const target of ["local", "site-b"]) { + await exec("npm", "run", "wait:database", "--", "--target", target); + await exec("npm", "run", "seed", "--", "--target", target); + } + const address = await run(["port", "app", "3000"]); + assert.match(address, /^127\.0\.0\.1:\d+$/); + const origin = `http://${address}`; + const get = async (path) => { + const response = await request(`${origin}${path}`); + assert.equal(response.status, 200); + return response.json(); + }; + const initial = await get("/api/config"); + const appId = await run(["ps", "-q", "app"]); + const appIdentity = async () => { + const result = await command(["inspect", "--format", "{{.State.Pid}} {{.State.StartedAt}} {{.RestartCount}}", appId]).result; + assert.ok(result.ok); + return result.stdout.trim(); + }; + const identity = await appIdentity(); // init:false + direct node => actual app PID + const post = (path, body) => request(`${origin}/api/${path}`, { + method: "POST", + headers: { "Content-Type": "application/json", "X-Workshop-Token": initial.csrfToken }, + body: JSON.stringify(body), + }); + const state = await get("/api/state"); + assert.equal(state.sessions.length, 216); + assert.equal((await post("favorites", { target: "local", sessionId: state.sessions[0].id })).status, 200); + const snapshot = async () => JSON.parse(await exec("node", "test/reconnect.probe.mjs", "snapshot")); + const before = await snapshot(); + assert.equal(before.local.favorites, 1); + assert.equal(before["site-b"].favorites, 0); + + probe = command([...base, "exec", "-T", "app", "node", "test/reconnect.probe.mjs"], { input: "pipe", timeout: 360000 }); + const lines = createInterface({ input: probe.child.stdout })[Symbol.asyncIterator](); + const nextLine = async () => { + let timer; + let abort; + try { + interrupted.signal.throwIfAborted(); + const value = await Promise.race([ + lines.next(), new Promise((_, reject) => { + abort = () => reject(interrupted.signal.reason); + interrupted.signal.addEventListener("abort", abort, { once: true }); + timer = setTimeout(() => reject(new Error("Probe timeout")), 25000); + }), + ]); + assert.equal(value.done, false); + return JSON.parse(value.value); + } finally { + clearTimeout(timer); + if (abort) interrupted.signal.removeEventListener("abort", abort); + } + }; + const probeReady = await nextLine(); + assert.equal(probeReady.ready, true); + const acquire = async (expected) => { + interrupted.signal.throwIfAborted(); + probe.child.stdin.write(JSON.stringify({ expected }) + "\n"); + const result = await nextLine(); + assert.equal(result.pid, probeReady.pid); + assert.equal(result.expected, expected); + assert.equal(result.passed, true); + }; + + phase = "first-connection-outage"; + await run(["stop", "documentdb-site-b"]); + // Original wait process must span the outage, including its first failure. + const waiter = command([...base, "exec", "-T", "app", "npm", "run", "wait:database", "--", "--target", "site-b"], { timeout: 270000 }); + let waiterFinished = false; + waiter.result.then(() => { waiterFinished = true; }); + assert.equal((await post("target", { target: "local", nextTarget: "site-b" })).status, 503); + assert.equal((await get("/api/config")).target, "local"); + assert.equal((await get("/api/state")).favorites.length, 1); + await acquire("unavailable"); // three first acquisitions, same persistent manager + assert.equal(waiterFinished, false); + await run(["start", "documentdb-site-b"]); + // Separate fresh client proves actual readiness before retrying the old ones. + await exec("npm", "run", "wait:database", "--", "--target", "site-b"); + const waited = await waiter.result; + assert.ok(waited.ok && /"target":"site-b","ready":true/.test(waited.stdout)); + receipt("original-wait-process-recovers-after-first-connection-failure"); + await acquire("ready"); + assert.equal((await post("target", { target: "local", nextTarget: "site-b" })).status, 200); + assert.equal((await get("/api/state")).sessions.length, 216); + assert.equal((await get("/api/state")).favorites.length, 0); + assert.equal(await appIdentity(), identity); + assert.equal((await get("/api/config")).csrfToken, initial.csrfToken); + receipt("first-unavailable-site-b-recovers-same-app-pid-and-concurrent-client"); + + phase = "healthy-client-outage"; + // Control: an already healthy client also recovers without target fallback. + await run(["stop", "documentdb-site-b"]); + assert.equal((await request(`${origin}/api/health`)).status, 503); + assert.equal((await get("/api/config")).target, "site-b"); + await acquire("unavailable"); + await run(["start", "documentdb-site-b"]); + await exec("npm", "run", "wait:database", "--", "--target", "site-b"); + await acquire("ready"); + await get("/api/health"); + assert.equal(await appIdentity(), identity); + assert.equal((await get("/api/config")).csrfToken, initial.csrfToken); + assert.deepEqual(await snapshot(), before); + receipt("healthy-client-outage-recovers-same-pid-no-fallback-notebooks-unchanged"); + probe.child.stdin.end(); + assert.ok((await probe.result).ok); + receipt("concurrent-acquisitions-close-cleanly"); +} catch { + failure = true; + console.error(JSON.stringify({ error: receivedSignal ? "RECONNECT_REGRESSION_INTERRUPTED" : "RECONNECT_REGRESSION_FAILED", + signal: receivedSignal, phase, message: "A bounded regression check stopped; raw command output was suppressed to protect private settings." })); +} finally { + await cleanupOwned(); + process.removeListener("SIGINT", onSignal); + process.removeListener("SIGTERM", onSignal); +} +if (failure || receivedSignal) process.exitCode = receivedSignal === "SIGINT" ? 130 : receivedSignal === "SIGTERM" ? 143 : 1; + +// One path for success, failed assertions, SIGINT and SIGTERM. Keep private +// configuration for manual recovery if down OR ANY inventory verification fails. +function cleanupOwned() { + return cleanupPromise ||= (async () => { + try { + console.log(JSON.stringify({ event: "owned-cleanup-started", project })); + const pending = [...children]; + for (const running of pending) running.cancel(); + await Promise.all(pending.map(({ result }) => result)); + if (owned) { + const cleanup = await command([...base, "down", "--volumes", "--remove-orphans"], { timeout: 120000, cleanup: true }).result; + let empty = true; + for (const list of [["ps", "-aq"], ["volume", "ls", "-q"], ["network", "ls", "-q"]]) { + const remaining = await command([...list, "--filter", `label=com.docker.compose.project=${project}`], { timeout: 15000, cleanup: true }).result; + if (!remaining.ok || remaining.stdout.trim()) empty = false; + } + assert.ok(cleanup.ok && empty, "Owned cleanup could not be verified"); + } + await rm(directory, { recursive: true }); + if (owned) { + receipt("only-owned-ephemeral-project-cleaned", { project, containers: 0, volumes: 0, networks: 0, privateDirectoryRemoved: true }); + } else { + console.log(JSON.stringify({ event: "private-setup-removed", project, privateDirectoryRemoved: true })); + } + } catch { + failure = true; + console.error(JSON.stringify({ error: "OWN_PROJECT_CLEANUP_FAILED", project, privateDirectory: directory })); + } + })(); +} diff --git a/src/connections.js b/src/connections.js index ed475e1..14a6741 100644 --- a/src/connections.js +++ b/src/connections.js @@ -1,16 +1,23 @@ import { MongoClient } from "mongodb"; import { WorkshopError } from "./errors.js"; -export function createConnections(config) { +export function createConnections(config, { Client = MongoClient } = {}) { const clients = new Map(); + const pending = new Set(); + let closing = false; + let closed; + const assertOpen = () => { + if (closing) throw new WorkshopError(503, "CONNECTIONS_CLOSED", "The app is shutting down. No new database operation was started."); + }; return { - db(target) { + async db(target) { + assertOpen(); if (!Object.hasOwn(config.targets, target)) { throw new WorkshopError(409, "TARGET_NOT_CONFIGURED", "This target is not configured. Select a target configured for this workshop scenario."); } if (!clients.has(target)) { const settings = config.targets[target]; - clients.set(target, new MongoClient(settings.uri, { + clients.set(target, new Client(settings.uri, { appName: "ato-2026-workshop", tls: true, tlsAllowInvalidCertificates: settings.developmentCertificate, @@ -22,10 +29,31 @@ export function createConnections(config) { timeoutMS: 10000, })); } - return clients.get(target).db(config.targets[target].database); + const client = clients.get(target); + // Explicit connect is required on EVERY acquisition. With driver 7.1, a + // failed first lazy connection can leave a closed topology behind. The + // public connect() repairs it and serializes concurrent connection attempts. + // This reconnects before an operation; it never replays a failed write. + const connecting = client.connect(); + pending.add(connecting); + try { + await connecting; + assertOpen(); + return client.db(config.targets[target].database); + } finally { + pending.delete(connecting); + } }, - async close() { - await Promise.all([...clients.values()].map((client) => client.close())); + close() { + // Terminal, idempotent shutdown. Drain in-flight acquisitions before closing + // pools; callers racing with shutdown must not receive a newly opened Db. + // App shutdown first drains HTTP requests (createApp.close). + closing = true; + closed ||= (async () => { + await Promise.allSettled([...pending]); + await Promise.all([...clients.values()].map((client) => client.close())); + })(); + return closed; }, }; } diff --git a/src/inspection.js b/src/inspection.js new file mode 100644 index 0000000..a816b39 --- /dev/null +++ b/src/inspection.js @@ -0,0 +1,22 @@ +import { requireCatalog } from "./catalog.js"; + +// Deliberately bounded and read-only: never enumerate system collections or +// print connection settings. Projection also excludes unrelated custom fields. +export async function inspectDatabase(db) { + const catalog = await requireCatalog(db); + const [sessions, favorites, indexes] = await Promise.all([ + db.collection("sessions").find({ catalogVersion: catalog.fingerprint }, { + projection: { _id: 1, title: 1, startsAt: 1, schemaVersion: 1, attendanceMode: 1 }, + }).sort({ startsAt: 1, _id: 1 }).limit(1).toArray(), + db.collection("favorites").find({}, { + projection: { _id: 1, savedAt: 1 }, + }).sort({ _id: 1 }).limit(1).toArray(), + db.collection("sessions").listIndexes().toArray(), + ]); + return { + catalog: catalog.version, + session: sessions[0] || null, + favorite: favorites[0] || null, + indexes: indexes.map(({ name, key }) => ({ name, key })), + }; +} diff --git a/src/public/index.html b/src/public/index.html index c8fa9a6..89dd5b3 100644 --- a/src/public/index.html +++ b/src/public/index.html @@ -93,6 +93,7 @@ .stack { display: grid; gap: 12px; } .status { font-size: 0.875rem; color: var(--cp-text-muted); } .notice { border-left: 4px solid var(--cp-warning); padding: 12px 16px; background: var(--cp-surface); border-radius: 0.625rem; margin: 16px 0; } + .notice.info { border-left-color: var(--cp-link); } .notice.error { border-left-color: var(--cp-danger); } .notice.success { border-left-color: var(--cp-success); } .metrics { display: flex; gap: 32px; margin: 24px 0; } @@ -141,7 +142,10 @@

All Things Open 2026 / Hands-on workshop

- +

Build your conference day.

@@ -166,6 +170,12 @@

Connecting...

+ @@ -195,7 +205,9 @@

Your favorites

    Save a session to start your day.

    - Your downloaded calendar works without this app. It is a snapshot, not a live subscription. + Your downloaded calendar works without this app. It is a snapshot, not a live subscription. + Google Calendar / Apple Calendar import steps +

    Copy, don't synchronize

    Select favorites above to copy to the other database. Existing choices are kept; removals are not propagated.

    @@ -269,7 +281,11 @@

    Confirm selected favorites

    }, }); const data = await response.json(); - if (!response.ok) throw new Error(`${data.error.code}: ${data.error.message}`); + if (!response.ok) { + const error = new Error(`${data.error.code}: ${data.error.message}`); + error.code = data.error.code; + throw error; + } return data; } @@ -279,6 +295,36 @@

    Confirm selected favorites

    if (id === "message") $(id).className = `notice ${error ? "error" : "success"}`; } + function showFailure(error) { + const step = config?.mode === "starter" + ? (error.code === "TODO_LIST_SESSIONS" ? 1 : error.code === "TODO_SAVE_FAVORITE" ? 2 : undefined) + : undefined; + if (step) { + $("exercise").hidden = false; + $("exercise-title").textContent = step === 1 + ? "Exercise 1 — Read the catalog" : "Exercise 2 — Save your first favorite"; + $("exercise-help").textContent = step === 1 + ? "Your database is reachable. Implement listSessions to make its catalog appear; this is an expected exercise checkpoint, not a system failure." + : "The catalog is working. Implement saveFavorite, then try Save favorite again. Your existing favorites have not been reset."; + $("exercise-guide").href = `https://github.com/documentdb/ato-2026-workshop/blob/main/docs/lab.md#${step === 1 ? "2-read-the-catalog" : "3-save-favorites-and-prove-persistence"}`; + } else { + $("exercise").hidden = true; + notice("error", error.message); + } + if (!state) { + $("catalog-empty").hidden = false; + $("catalog-empty").textContent = step === 1 + ? "Next: complete Exercise 1 in exercises/starter/repository.js, save, and refresh." + : "The catalog could not be loaded. Resolve the error above, then refresh."; + $("catalog-source").textContent = step === 1 + ? "Catalog source and snapshot details will appear after Exercise 1. The attendee guide explains the included schedule." + : "Catalog source details are unavailable until this database can be read."; + $("status").textContent = step === 1 + ? "Database reachable. Ready for Exercise 1." + : "This request did not complete. Check the error above; no fallback target was used."; + } + } + async function perform(action) { if (busy) return; busy = true; @@ -287,7 +333,7 @@

    Confirm selected favorites

    try { await action(); } catch (error) { - notice("error", error.message); + showFailure(error); } finally { busy = false; controls(); @@ -306,7 +352,7 @@

    Confirm selected favorites

    function renderConfiguration() { $("target-name").textContent = label(config.target); - $("mode").textContent = `${config.mode === "starter" ? "TODO starter" : "Complete reference"} / Same application data layer`; + $("mode").textContent = `${config.mode === "starter" ? "Exercise implementation (your editable code)" : "Complete reference"} / Same application data layer`; $("target-select").replaceChildren(); for (const { id: target } of config.targets) { const option = node("option", label(target)); @@ -362,6 +408,7 @@

    Confirm selected favorites

    body: JSON.stringify({ target: config.target, sessionId: session.id }), }); await loadState(); + $("exercise").hidden = true; notice("message", saved ? "Removed from the active database only." : "Favorite saved in the active database."); }); }); @@ -422,6 +469,9 @@

    Confirm selected favorites

    async function refresh() { state = undefined; selected.clear(); + $("exercise").hidden = true; + $("catalog-source").textContent = "Checking this catalog's source and snapshot date..."; + $("favorites-empty").hidden = false; $("sessions").replaceChildren(); $("favorites").replaceChildren(); $("copy-results").replaceChildren(); @@ -466,7 +516,7 @@

    Confirm selected favorites

    link.download = `${state.catalog.kind === "practice" ? "practice-" : ""}conference-favorites.ics`; link.click(); setTimeout(() => URL.revokeObjectURL(url), 1000); - notice("message", "Calendar export requested. Save the .ics file if your browser prompts, then import it manually; this is not a live subscription."); + notice("message", "Calendar export requested. Save the .ics file if prompted, then follow the Google Calendar / Apple Calendar import steps below the download button. This is a snapshot, not a live subscription."); })); $("copy").addEventListener("click", () => { $("copy-route").textContent = `${label(config.target)} -> ${label(destination())}`; diff --git a/src/server.js b/src/server.js index 83610f3..99f9d14 100644 --- a/src/server.js +++ b/src/server.js @@ -88,18 +88,18 @@ export async function createApp({ config, connections, repository, logger = cons } const target = activeTarget; if (request.method === "GET" && url.pathname === "/api/health") { - await connections.db(target).command({ ping: 1 }); + await (await connections.db(target)).command({ ping: 1 }); json(200, { ready: true, target, mode: config.mode, location: config.location }); return; } if (request.method === "GET" && url.pathname === "/api/state") { - const state = await readPlanner(connections.db(target), repository); + const state = await readPlanner(await connections.db(target), repository); json(200, { target, ...state }); return; } if (request.method === "GET" && url.pathname === "/api/calendar") { assertTarget(url.searchParams.get("target")); - const state = await readPlanner(connections.db(target), repository); + const state = await readPlanner(await connections.db(target), repository); if (state.orphanedFavorites.length) { throw new WorkshopError(409, "CATALOG_DRIFT", "Some favorites have no matching session. Resolve the catalog drift before exporting."); } @@ -120,14 +120,14 @@ export async function createApp({ config, connections, repository, logger = cons if (!["local", "site-b", "cloud"].includes(data.nextTarget)) { throw new WorkshopError(400, "INVALID_TARGET", "Choose one of the configured database targets."); } - await connections.db(data.nextTarget).command({ ping: 1 }); + await (await connections.db(data.nextTarget)).command({ ping: 1 }); assertTarget(data.target); activeTarget = data.nextTarget; json(200, details()); return; } if (url.pathname === "/api/favorites") { - const db = connections.db(target); + const db = await connections.db(target); await requireSession(db, data.sessionId); if (request.method === "POST") { const result = await repository.saveFavorite(db, data.sessionId); @@ -142,7 +142,7 @@ export async function createApp({ config, connections, repository, logger = cons if (data.source !== target || !["local", "site-b", "cloud"].includes(data.destination) || data.source === data.destination) { throw new WorkshopError(400, "INVALID_TRANSFER", "Copy from the active target to the other configured target."); } - const result = await copyFavorites(connections.db(data.source), connections.db(data.destination), data.sessionIds); + const result = await copyFavorites(await connections.db(data.source), await connections.db(data.destination), data.sessionIds); if (!result.complete) { logger.error(JSON.stringify({ event: "copy_incomplete", statuses: result.results.map((item) => item.status) })); } diff --git a/test/connections.test.js b/test/connections.test.js new file mode 100644 index 0000000..b9f420f --- /dev/null +++ b/test/connections.test.js @@ -0,0 +1,122 @@ +import test from "node:test"; +import assert from "node:assert/strict"; +import { createConnections } from "../src/connections.js"; +import { waitForDatabase } from "../scripts/database.mjs"; + +const config = { targets: { + local: { uri: "mongodb://site-a.invalid", database: "test_a", developmentCertificate: true }, + "site-b": { uri: "mongodb://site-b.invalid", database: "test_b", developmentCertificate: true }, + cloud: { uri: "mongodb://managed.invalid", database: "test_cloud", developmentCertificate: false }, +} }; +function mockDriver() { + const clients = []; + const failure = () => Object.assign(new Error("synthetic failure"), { name: "MongoServerSelectionError" }); + class Client { + constructor(uri, options) { + Object.assign(this, { uri, options, connects: 0, closes: 0, topology: null }); + clients.push(this); + } + async connect() { + this.connects++; + if (Client.gate) await Client.gate; + // Model the pinned driver's failed-lazy-connect state. Only explicit + // connect may repair it; db()/commands never repair a closed topology. + this.topology = Client.available ? "connected" : "closed"; + if (!Client.available) throw failure(); + return this; + } + db(name) { + return { name, command: async () => { + if (this.topology !== "connected" || !Client.available) throw failure(); + return { ok: 1 }; + } }; + } + async close() { this.closes++; this.topology = "closed"; } + } + Client.available = false; + return { Client, clients }; +} + +test("first failed acquisition retries explicitly on the same cached client, without replaying commands", async () => { + const { Client, clients } = mockDriver(); + const connections = createConnections(config, { Client }); + await assert.rejects(connections.db("site-b"), { name: "MongoServerSelectionError" }); + assert.equal(clients[0].topology, "closed"); + Client.available = true; + assert.deepEqual(await (await connections.db("site-b")).command({ ping: 1 }), { ok: 1 }); + assert.equal(clients.length, 1); + assert.equal(clients[0].connects, 2); + Client.available = false; + await assert.rejects(clients[0].db("test_b").command({ ping: 1 })); + assert.equal(clients[0].connects, 2, "a failed operation isn't automatically retried"); + Client.available = true; + await (await connections.db("site-b")).command({ ping: 1 }); + assert.equal(clients[0].connects, 3); + await connections.close(); +}); + +test("concurrent failed and healthy acquisitions settle, reuse one client, and keep TLS/timeouts", async () => { + const { Client, clients } = mockDriver(); + const connections = createConnections(config, { Client }); + const failed = await Promise.allSettled(Array.from({ length: 3 }, () => connections.db("site-b"))); + assert.ok(failed.every(({ status }) => status === "rejected")); + Client.available = true; + assert.equal((await Promise.all(Array.from({ length: 3 }, () => connections.db("site-b")))).length, 3); + assert.equal(clients.length, 1); + assert.equal(clients[0].connects, 6); + const { options } = clients[0]; + assert.equal(options.tls, true); + assert.equal(options.tlsAllowInvalidCertificates, true); + assert.equal(options.tlsAllowInvalidHostnames, false); + assert.equal(options.retryWrites, false); + assert.equal(options.serverSelectionTimeoutMS, 5000); + assert.equal(options.connectTimeoutMS, 5000); + assert.equal(options.timeoutMS, 10000); + await connections.db("cloud"); + assert.equal(clients[1].options.tlsAllowInvalidCertificates, false); + await assert.rejects(connections.db("unknown"), { code: "TARGET_NOT_CONFIGURED" }); + await connections.close(); + assert.ok(clients.every(({ closes }) => closes === 1)); +}); + +test("close is terminal and idempotent and waits for acquisitions without handing out a closing Db", async () => { + const { Client, clients } = mockDriver(); + Client.available = true; + let release; + Client.gate = new Promise((resolve) => { release = resolve; }); + const connections = createConnections(config, { Client }); + const acquisition = assert.rejects(connections.db("local"), { code: "CONNECTIONS_CLOSED" }); + const closing = connections.close(); + assert.equal(connections.close(), closing); + assert.equal(clients[0].closes, 0); + await assert.rejects(connections.db("site-b"), { code: "CONNECTIONS_CLOSED" }); + release(); + await Promise.all([acquisition, closing]); + assert.equal(clients.length, 1); + assert.equal(clients[0].closes, 1); + await assert.rejects(connections.db("local"), { code: "CONNECTIONS_CLOSED" }); +}); + +test("wait retries acquisition INSIDE its loop, then the original invocation succeeds", async () => { + let attempts = 0; + let clock = 0; + const connections = { async db(target) { + assert.equal(target, "site-b"); + if (++attempts < 3) throw new Error("not ready yet"); + return { async command(command) { assert.deepEqual(command, { ping: 1 }); } }; + } }; + await waitForDatabase(connections, "site-b", { timeoutMS: 6000, now: () => clock, sleep: async (ms) => { clock += ms; } }); + assert.equal(attempts, 3); + assert.equal(clock, 4000); +}); + +test("wait deadline is bounded even when every acquisition rejects", async () => { + let attempts = 0; + let clock = 0; + const failure = new Error("unavailable"); + await assert.rejects(waitForDatabase({ async db() { attempts++; throw failure; } }, "site-b", { + timeoutMS: 4500, now: () => clock, sleep: async (ms) => { clock += ms; }, + }), (error) => error === failure); + assert.equal(attempts, 3); + assert.equal(clock, 4500); +}); diff --git a/test/database.integration.js b/test/database.integration.js index d8700a4..697c754 100644 --- a/test/database.integration.js +++ b/test/database.integration.js @@ -1,3 +1,6 @@ +import { inspectDatabase } from "../src/inspection.js"; +import { programCounts } from "../scripts/program-counts.mjs"; +import { programCountsPipeline } from "../examples/complete/program-counts.js"; import test from "node:test"; import assert from "node:assert/strict"; import { randomUUID } from "node:crypto"; @@ -24,21 +27,11 @@ test("two real independent DocumentDB servers: independent data, retry-safe roun const config = readConfig({ ...localEnv, LOCAL_DATABASE_NAME: `verify_source_${runId}` }); const sourceConnections = createConnections(config); const destinationConnections = createConnections(readConfig({ ...localEnv, SITE_B_DATABASE_NAME: `verify_destination_${runId}` })); - const source = sourceConnections.db("local"); - const destination = destinationConnections.db("site-b"); - assert.notEqual(config.targets.local.uri.split("@")[1], config.targets["site-b"].uri.split("@")[1], "integration requires two distinct server endpoints"); - // Site A's credentials must not authenticate to Site B (or vice versa). - for (const [from, to] of [["local", "site-b"], ["site-b", "local"]]) { - const foreign = createConnections({ targets: { local: { - ...config.targets[from], - uri: `${config.targets[from].uri.split("@")[0]}@${config.targets[to].uri.split("@")[1]}`, - } } }); - try { - await assert.rejects(foreign.db("local").command({ ping: 1 }), { code: 18 }); - } finally { await foreign.close(); } - } + let source; + let destination; let sourceOwned = false; let destinationOwned = false; + // Register ownership-aware cleanup before the first asynchronous acquisition. t.after(async () => { try { if (sourceOwned) await source.dropDatabase(); @@ -50,6 +43,19 @@ test("two real independent DocumentDB servers: independent data, retry-safe roun } } }); + source = await sourceConnections.db("local"); + destination = await destinationConnections.db("site-b"); + assert.notEqual(config.targets.local.uri.split("@")[1], config.targets["site-b"].uri.split("@")[1], "integration requires two distinct server endpoints"); + // Site A's credentials must not authenticate to Site B (or vice versa). + for (const [from, to] of [["local", "site-b"], ["site-b", "local"]]) { + const foreign = createConnections({ targets: { local: { + ...config.targets[from], + uri: `${config.targets[from].uri.split("@")[0]}@${config.targets[to].uri.split("@")[1]}`, + } } }); + try { + await assert.rejects(async () => (await foreign.db("local")).command({ ping: 1 }), { code: 18 }); + } finally { await foreign.close(); } + } await source.createCollection("verification_ownership"); sourceOwned = true; await destination.createCollection("verification_ownership"); @@ -105,6 +111,10 @@ test("two real independent DocumentDB servers: independent data, retry-safe roun assert.match(drift.results[0].message, /details differ/); await destination.collection("sessions").updateOne({ _id: ids[0] }, { $set: { room: originalSession.room } }); + const beforeMigration = await inspectDatabase(source); + assert.equal(beforeMigration.session.schemaVersion, 1); + assert.equal(Object.hasOwn(beforeMigration.session, "attendanceMode"), false); + assert.ok(beforeMigration.favorite.savedAt); await source.collection("sessions").createIndex({ startsAt: 1, _id: 1 }, { name: "schedule_order" }); const migrate = () => source.collection("sessions").updateMany( { catalogVersion: catalog.fingerprint, schemaVersion: 1, attendanceMode: { $exists: false } }, @@ -112,6 +122,12 @@ test("two real independent DocumentDB servers: independent data, retry-safe roun ); assert.equal((await migrate()).modifiedCount, catalog.sessions.length); assert.equal((await migrate()).modifiedCount, 0); + const afterMigration = await inspectDatabase(source); + assert.equal(afterMigration.session._id, beforeMigration.session._id); + assert.equal(afterMigration.session.schemaVersion, 2); + assert.equal(afterMigration.session.attendanceMode, "in-person"); + assert.equal(afterMigration.favorite.savedAt, beforeMigration.favorite.savedAt); + assert.ok(afterMigration.indexes.some(({ name }) => name === "schedule_order")); await seedCatalog(source, catalog); assert.equal(await source.collection("sessions").countDocuments({ schemaVersion: 2 }), catalog.sessions.length); const state = await readPlanner(source, repository); @@ -120,7 +136,7 @@ test("two real independent DocumentDB servers: independent data, retry-safe roun const app = await createApp({ config: { ...config, mode: "complete" }, - connections: { db: (target) => { + connections: { db: async (target) => { if (target === "local") return source; if (target === "site-b") return destination; throw new WorkshopError(409, "TARGET_NOT_CONFIGURED", "No cloud access in this test."); @@ -167,3 +183,34 @@ test("two real independent DocumentDB servers: independent data, retry-safe roun await app.close(); } }); + +test("optional aggregation queries the actual official snapshot on an owned local test database", async (t) => { + const connections = createConnections(readConfig({ + ...process.env, ACTIVE_TARGET: "local", WORKSHOP_SCENARIO: "attendee", + LOCAL_DATABASE_NAME: `verify_query_${randomUUID().replaceAll("-", "")}`, + })); + let db; + let owned = false; + t.after(async () => { + try { if (owned) await db.dropDatabase(); } + finally { await connections.close(); } + }); + db = await connections.db("local"); + await db.createCollection("verification_ownership"); + owned = true; + const catalog = await readCatalog(); + await seedCatalog(db, catalog); + const result = await programCounts(db, programCountsPipeline); + // Independent calculation: conference timezone rather than pipeline bounds. + const expected = new Map(); + for (const session of catalog.sessions) { + const day = new Intl.DateTimeFormat("en-CA", { timeZone: catalog.timeZone }).format(new Date(session.startsAt)); + if (day === "2026-10-19") expected.set(session.track, (expected.get(session.track) || 0) + 1); + } + assert.deepEqual(result.rows, [...expected].map(([_id, entries]) => ({ _id, entries })) + .sort((a, b) => b.entries - a.entries || a._id.localeCompare(b._id))); + assert.equal(result.rows.length, 17); + assert.equal(result.rows.reduce((sum, row) => sum + row.entries, 0), 74); + assert.equal(await db.collection("sessions").countDocuments(), 216); + assert.equal(await db.collection("favorites").countDocuments(), 0); +}); diff --git a/test/experience.test.js b/test/experience.test.js new file mode 100644 index 0000000..3bd63ea --- /dev/null +++ b/test/experience.test.js @@ -0,0 +1,117 @@ +import test from "node:test"; +import assert from "node:assert/strict"; +import { readFile } from "node:fs/promises"; +import vm from "node:vm"; +import { readCatalog } from "../src/catalog.js"; +import { inspectDatabase } from "../src/inspection.js"; +import { programCounts, validatePipeline } from "../scripts/program-counts.mjs"; +import { programCountsPipeline } from "../examples/complete/program-counts.js"; + +async function browser() { + const html = await readFile(new URL("../src/public/index.html", import.meta.url), "utf8"); + const script = [...html.matchAll(/]*>([\s\S]*?)<\/script>/g)].at(-1)[1]; + const elements = new Map(); + const element = () => ({ hidden: true, textContent: "", value: "", children: [], + addEventListener() {}, setAttribute() {}, append(...nodes) { this.children.push(...nodes); }, + replaceChildren(...nodes) { this.children = nodes; }, + }); + const context = vm.createContext({ URLSearchParams, Intl, Date, Set, Map, Error, console, + document: { + getElementById(id) { if (!elements.has(id)) elements.set(id, element()); return elements.get(id); }, + createElement: element, querySelectorAll: () => [], + }, + }); + // Exercise the actual page functions without auto-boot/network. Separate HTTP + // tests verify the endpoint contracts. This is not a rendered-browser proof. + vm.runInContext(script.replace(/ perform\(refresh\);\s*$/, ""), context); + vm.runInContext('config = {mode:"starter", target:"local", targets:[{id:"local",label:"Site A"}], scenario:"attendee", location:"codespaces"}', context); + return { elements, context, run: (source) => vm.runInContext(source, context) }; +} + +test("expected TODOs show a guided checkpoint, not red error or indefinite loading", async () => { + const { elements, run } = await browser(); + run('showFailure({code:"TODO_LIST_SESSIONS",message:"TODO"})'); + assert.equal(elements.get("exercise").hidden, false); + assert.match(elements.get("exercise-title").textContent, /Exercise 1/); + assert.match(elements.get("exercise-help").textContent, /listSessions/); + assert.match(elements.get("exercise-guide").href, /#2-read-the-catalog$/); + assert.doesNotMatch(elements.get("catalog-empty").textContent, /Loading/); + assert.doesNotMatch(elements.get("catalog-source").textContent, /Loading/); + assert.equal(elements.get("error")?.hidden ?? true, true); + run('showFailure({code:"TODO_SAVE_FAVORITE",message:"TODO"})'); + assert.match(elements.get("exercise-title").textContent, /Exercise 2/); + assert.match(elements.get("exercise-help").textContent, /saveFavorite/); + assert.match(elements.get("exercise-guide").href, /#3-save-favorites-and-prove-persistence$/); + run('renderConfiguration()'); + assert.match(elements.get("mode").textContent, /Exercise implementation/); + assert.doesNotMatch(elements.get("mode").textContent, /TODO|unfinished/); +}); + +test("real errors remain errors; completed exercise state is not inferred from mode", async () => { + const { elements, run } = await browser(); + run('showFailure({code:"DATABASE_UNAVAILABLE",message:"Database unavailable"})'); + assert.equal(elements.get("exercise").hidden, true); + assert.equal(elements.get("error").hidden, false); + assert.match(elements.get("catalog-empty").textContent, /could not be loaded/); + run('config.mode="complete"; showFailure({code:"TODO_LIST_SESSIONS",message:"Unexpected TODO"})'); + assert.equal(elements.get("exercise").hidden, true); + assert.equal(elements.get("error").textContent, "Unexpected TODO"); +}); + +test("inspect returns actual bounded projected sessions/favorites and index definitions", async () => { + const calls = []; + const session = { _id: "sample", title: "Published entry", startsAt: "2026-10-19T13:00:00.000Z", schemaVersion: 1 }; + const favorite = { _id: "sample", savedAt: "2026-10-17T13:00:00.000Z" }; + const db = { collection(name) { return { + async findOne() { return { _id: "catalog", version: "test", fingerprint: "fp" }; }, + find(filter, options) { + calls.push({ name, filter, options }); + return { sort(order) { calls.at(-1).sort = order; return this; }, limit(count) { calls.at(-1).limit = count; return this; }, + async toArray() { return [structuredClone(name === "sessions" ? session : favorite)]; } }; + }, + listIndexes() { return { async toArray() { return [{ name: "schedule_order", key: { startsAt: 1, _id: 1 }, ignored: "not emitted" }]; } }; }, + }; } }; + const before = await inspectDatabase(db); + assert.deepEqual(before.session, session); + assert.equal(Object.hasOwn(before.session, "attendanceMode"), false); + assert.deepEqual(before.favorite, favorite); + assert.ok(calls.every(({ limit }) => limit === 1)); + assert.deepEqual(calls[0].filter, { catalogVersion: "fp" }); + assert.deepEqual(calls[1].options.projection, { _id: 1, savedAt: 1 }); + assert.deepEqual(before.indexes, [{ name: "schedule_order", key: { startsAt: 1, _id: 1 } }]); + Object.assign(session, { schemaVersion: 2, attendanceMode: "in-person" }); + const after = await inspectDatabase(db); + assert.equal(after.session.schemaVersion, 2); + assert.equal(after.session.attendanceMode, "in-person"); + assert.equal(after.favorite.savedAt, before.favorite.savedAt); +}); + +test("optional reference counts match the real fixed snapshot, not attendees or the current day", async () => { + const catalog = await readCatalog(); + const pipeline = programCountsPipeline(catalog.fingerprint); + assert.deepEqual(validatePipeline(pipeline), pipeline); + assert.equal(pipeline[0].$match.catalogVersion, catalog.fingerprint); + const range = pipeline[0].$match.startsAt; + const selected = catalog.sessions.filter(({ startsAt }) => startsAt >= range.$gte && startsAt < range.$lt); + assert.ok(selected.every(({ startsAt }) => new Intl.DateTimeFormat("en-CA", { timeZone: catalog.timeZone }).format(new Date(startsAt)) === "2026-10-19")); + const counts = new Map(); + for (const row of selected) counts.set(row.track, (counts.get(row.track) || 0) + 1); + const expected = [...counts].map(([_id, entries]) => ({ _id, entries })).sort((a, b) => b.entries - a.entries || a._id.localeCompare(b._id)); + assert.deepEqual(pipeline[1], { $group: { _id: "$track", entries: { $sum: 1 } } }); + assert.deepEqual(pipeline[2], { $sort: { entries: -1, _id: 1 } }); + // These values are independently computed from data/catalog.json; changing + // the snapshot requires reviewing both the challenge's expected result and test. + assert.equal(selected.length, 74); + assert.equal(expected.length, 17); + assert.deepEqual(expected[0], { _id: "Inclusion & Diversity in Open Source", entries: 10 }); + let actualPipeline; + const db = { collection(name) { return { + async findOne() { assert.equal(name, "workshop_meta"); return { _id: "catalog", fingerprint: catalog.fingerprint, version: catalog.version }; }, + aggregate(stages, options) { actualPipeline = stages; assert.equal(options.maxTimeMS, 5000); return { async toArray() { return expected; } }; }, + }; } }; + assert.deepEqual((await programCounts(db, programCountsPipeline)).rows, expected); + assert.deepEqual(actualPipeline.at(-1), { $limit: 30 }); + for (const stage of ["$out", "$merge", "$lookup", "$unionWith"]) { + assert.throws(() => validatePipeline([{ [stage]: "forbidden" }]), { code: "READ_ONLY_PIPELINE" }); + } +}); diff --git a/test/http.test.js b/test/http.test.js index 2395839..b99b573 100644 --- a/test/http.test.js +++ b/test/http.test.js @@ -8,7 +8,7 @@ test("HTTP target changes are explicit; errors are sanitized and writes require const calls = []; const config = { activeTarget: "local", mode: "complete", location: "laptop", targets: { local: {} } }; const connections = { - db(target) { + async db(target) { calls.push(target); if (target !== "local") throw new WorkshopError(409, "TARGET_NOT_CONFIGURED", "No cloud target."); return { command: async () => ({ ok: 1 }) }; @@ -55,7 +55,7 @@ test("unavailable Site B does not switch the active target or fall back after a const app = await createApp({ config, repository: {}, logger: { error() {} }, connections: { - db(target) { + async db(target) { calls.push(target); return { async command() { if (target === "site-b" && unavailable) { diff --git a/test/reconnect-interruption.integration.js b/test/reconnect-interruption.integration.js new file mode 100644 index 0000000..14a8070 --- /dev/null +++ b/test/reconnect-interruption.integration.js @@ -0,0 +1,75 @@ +// Opt-in Docker-host test; NOT part of offline npm test or the app container's +// database suite. Every subprocess creates its own random project and .env. +import test from "node:test"; +import assert from "node:assert/strict"; +import { spawn, execFile } from "node:child_process"; +import { access } from "node:fs/promises"; +import { promisify } from "node:util"; +import { fileURLToPath } from "node:url"; +import { createInterface } from "node:readline"; + +const root = fileURLToPath(new URL("../", import.meta.url)); +const execute = promisify(execFile); +for (const signal of ["SIGTERM", "SIGINT"]) { + test(`${signal} routes through owned cleanup; a repeated signal cannot interrupt cleanup`, { timeout: 780000 }, async () => { + const child = spawn(process.execPath, ["scripts/test-reconnect.mjs"], { + cwd: root, stdio: ["ignore", "pipe", "pipe"], + }); + const events = []; + let ownership; + let repeated = false; + let timedOut = false; + let force; + const timer = setTimeout(() => { + timedOut = true; + child.kill("SIGTERM"); + force = setTimeout(() => child.kill("SIGKILL"), 180000); + }, 600000); + const closed = new Promise((resolve, reject) => { + child.once("error", reject); + child.once("close", (code, exitSignal) => resolve({ code, exitSignal })); + }); + // Only parsed harness receipts, never raw child/driver/Compose output. + const read = async (stream) => { + for await (const line of createInterface({ input: stream })) { + let event; + try { event = JSON.parse(line); } catch { continue; } + events.push(event); + if (event.check === "owned-ephemeral-project-ready") { + ownership = event; + child.kill(signal); + } + if (event.event === "owned-cleanup-started" && ownership) { + repeated = child.kill(signal); + } + } + }; + try { + const readers = Promise.all([read(child.stdout), read(child.stderr)]); + const result = await closed; + await readers; + assert.equal(timedOut, false, "Harness did not reach its owned-project checkpoint within the startup budget"); + assert.ok(ownership, "No owned-project receipt"); + assert.match(ownership.project, /^ato-reconnect-[a-f0-9]{32}$/); + assert.equal(repeated, true, "Did not send a second signal during cleanup"); + assert.equal(result.exitSignal, null, "Harness exited via default signal termination instead of cleanup"); + assert.equal(result.code, signal === "SIGINT" ? 130 : 143); + assert.ok(!events.some(({ error }) => error === "OWN_PROJECT_CLEANUP_FAILED"), "Harness reported cleanup failure"); + const cleaned = events.find(({ check }) => check === "only-owned-ephemeral-project-cleaned"); + assert.equal(cleaned?.project, ownership.project); + assert.equal(cleaned?.privateDirectoryRemoved, true); + for (const list of [["ps", "-aq"], ["volume", "ls", "-q"], ["network", "ls", "-q"]]) { + // Independently verify every resource family, narrowly by the generated + // project label. This test does not adopt/delete any pre-existing stack. + const { stdout } = await execute("docker", [...list, "--filter", `label=com.docker.compose.project=${ownership.project}`], { timeout: 15000 }); + assert.equal(stdout.trim(), "", "Owned resources remain after interruption"); + } + await assert.rejects(access(ownership.privateDirectory), { code: "ENOENT" }); + console.log(JSON.stringify({ check: "signal-cleanup-independently-verified", signal, project: ownership.project, + containers: 0, volumes: 0, networks: 0, privateDirectoryRemoved: true, exitCode: result.code })); + } finally { + clearTimeout(timer); clearTimeout(force); + if (child.exitCode === null && child.signalCode === null) child.kill("SIGTERM"); + } + }); +} diff --git a/test/reconnect.probe.mjs b/test/reconnect.probe.mjs new file mode 100644 index 0000000..98b7f5b --- /dev/null +++ b/test/reconnect.probe.mjs @@ -0,0 +1,43 @@ +// Internal live harness helper, run inside its OWN disposable Compose project. +// JSON lines keep one connection manager alive across host-controlled outages. +import assert from "node:assert/strict"; +import { createHash } from "node:crypto"; +import { createInterface } from "node:readline"; +import { readConfig } from "../src/config.js"; +import { createConnections } from "../src/connections.js"; + +const config = readConfig(); +assert.equal(config.scenario, "attendee"); +const connections = createConnections(config); +try { + if (process.argv[2] === "snapshot") { + const result = {}; + for (const target of ["local", "site-b"]) { + const db = await connections.db(target); + const favorites = await db.collection("favorites").find({}).sort({ _id: 1 }).toArray(); + result[target] = { + sessions: await db.collection("sessions").countDocuments(), + favorites: favorites.length, + favoriteDigest: createHash("sha256").update(JSON.stringify(favorites)).digest("hex"), + }; + } + console.log(JSON.stringify(result)); + } else { + console.log(JSON.stringify({ ready: true, pid: process.pid })); + for await (const line of createInterface({ input: process.stdin })) { + const { expected } = JSON.parse(line); + assert.ok(["ready", "unavailable"].includes(expected)); + const outcomes = await Promise.allSettled(Array.from({ length: 3 }, async () => { + const db = await connections.db("site-b"); + await db.command({ ping: 1 }); + })); + assert.ok(outcomes.every(({ status }) => status === (expected === "ready" ? "fulfilled" : "rejected"))); + if (expected === "unavailable") assert.ok(outcomes.every(({ reason }) => reason.name?.startsWith("Mongo"))); + console.log(JSON.stringify({ expected, concurrent: 3, passed: true, pid: process.pid })); + } + } +} catch { + // Never print a driver's raw message, URI, stack, or supplied input. + console.error(JSON.stringify({ error: "RECONNECT_PROBE_FAILED" })); + process.exitCode = 1; +} finally { await connections.close(); } diff --git a/test/regression-safety.test.js b/test/regression-safety.test.js new file mode 100644 index 0000000..3d9bdd3 --- /dev/null +++ b/test/regression-safety.test.js @@ -0,0 +1,61 @@ +import test from "node:test"; +import assert from "node:assert/strict"; +import { readFile } from "node:fs/promises"; + +// These are packaging/scope guards, not substitutes for the live outage test. +test("live reconnect harness owns a fresh project and never targets an attendee stack", async () => { + const source = await readFile(new URL("../scripts/test-reconnect.mjs", import.meta.url), "utf8"); + assert.match(source, /const project = `ato-reconnect-\$\{randomUUID\(\)/); + assert.match(source, /"--project-name", project/); + assert.match(source, /"--env-file", join\(directory, "\.env"\)/); + assert.match(source, /await setup\(directory, \{\},/); + assert.match(source, /delete env\[key\]/); + assert.match(source, /127\.0\.0\.1::3000/); + assert.match(source, /if \(owned\)/); + assert.match(source, /\[\.\.\.base, "down", "--volumes"/); + assert.doesNotMatch(source, /system.*prune|volume.*prune|docker.*kill.*\$\(/); + assert.match(source, /await appIdentity\(\), identity/); + assert.match(source, /waiter\.result/); + assert.match(source, /assert\.deepEqual\(await snapshot\(\), before\)/); +}); + +test("interruptions share cancellable work and verified, signal-resistant owned cleanup", async () => { + const source = await readFile(new URL("../scripts/test-reconnect.mjs", import.meta.url), "utf8"); + assert.match(source, /process\.on\("SIGINT", onSignal\)/); + assert.match(source, /process\.on\("SIGTERM", onSignal\)/); + assert.match(source, /AbortSignal\.any\(\[interrupted\.signal, AbortSignal\.timeout\(20000\)\]\)/); + assert.match(source, /return cleanupPromise \|\|=/); + assert.match(source, /if \(!running\.cleanup\) running\.cancel\(\)/); + assert.match(source, /child\.kill\("SIGKILL"\), 5000/); + assert.match(source, /await Promise\.all\(pending\.map/); + assert.match(source, /assert\.ok\(cleanup\.ok && empty/); + const cleanup = source.slice(source.indexOf("function cleanupOwned()")); + for (const command of ['["ps", "-aq"]', '["volume", "ls", "-q"]', '["network", "ls", "-q"]']) { + assert.ok(cleanup.includes(command)); + } + assert.ok(cleanup.indexOf("cleanup.ok && empty") < cleanup.indexOf("await rm(directory")); + assert.match(cleanup, /OWN_PROJECT_CLEANUP_FAILED/); +}); + +test("runtime image includes all live harness fixtures and optional exercises", async () => { + const dockerfile = await readFile(new URL("../Dockerfile", import.meta.url), "utf8"); + for (const directory of ["src", "scripts", "test", "examples", "exercises", "data"]) { + assert.ok(dockerfile.includes(`COPY --chown=node:node ${directory} ./${directory}`)); + } + // Root-only docs checks must stay separate: Docker intentionally omits docs. + const pkg = JSON.parse(await readFile(new URL("../package.json", import.meta.url), "utf8")); + assert.equal(pkg.scripts.test, "node --test test/*.test.js"); + assert.equal(pkg.scripts["test:docs"], "node scripts/check-doc-links.mjs"); +}); + +test("persistent guide and calendar links keep accessible labels and safe new-tab navigation", async () => { + const html = await readFile(new URL("../src/public/index.html", import.meta.url), "utf8"); + assert.match(html, /Attendee guide \(opens new tab\)/); + assert.match(html, /Google Calendar \/ Apple Calendar import steps/); + assert.match(html, /id="exercise" role="status" aria-labelledby="exercise-title"/); + assert.match(html, /id="error" role="alert"/); + assert.match(html, /id="status" role="status"/); + for (const [, anchor] of html.matchAll(/(]+)>/g)) { + if (anchor.includes('target="_blank"')) assert.ok(anchor.includes('rel="noopener noreferrer"')); + } +});