Add weft_shop, a streamed storefront example, and fix the Weft bugs it found

.changeset/vite-plugin-weft-testing-cookies.md

@@ -0,0 +1,8 @@
+---
+'@e-cline/weft_vite_plugin': patch
+---
+
+The test harness keeps cookies, as a browser does for the app's origin: what the server's
+`Set-Cookie` headers set is sent with every request after, until it expires (`cookies`,
+`clearCookies`). Opening a page closes the one before it (`closePage`): its root is unmounted, so
+its intervals, subscriptions and history listener stop, and can't act in the next test.

.changeset/vite-plugin-weft-testing-redirects.md

@@ -0,0 +1,9 @@
+---
+'@e-cline/weft_vite_plugin': patch
+---
+
+The test harness's `fetch` follows redirects on the app's origin as a browser's does (a 303, or a
+POST's 301 or 302, with a `GET`; at most 20; `redirect: 'manual'` and `'error'` respected), and its
+responses say where they ended up (`url`, `redirected`); a redirect to another origin fails. A page
+opened at a URL that redirects is at the URL it led to (`WeftTesting.url`), and payloads report it
+to `Navigation`, as in a browser.

.changeset/weft-core-dehydrated-retry.md

@@ -0,0 +1,8 @@
+---
+'@e-cline/weft': patch
+---
+
+Fixes a pending `Suspense` boundary rendered twice: when its content arrived after hydration had
+started, and suspended as it hydrated (an entrypoint's chunk still loading, say), the boundary
+rendered its content on the client after the server's, which stayed, unhydrated. It now stays as the
+server sent it until what it waits on settles, then hydrates.

.changeset/weft-dom-hydration-and-props.md

@@ -0,0 +1,11 @@
+---
+'@e-cline/weft_dom': patch
+---
+
+- `autoFocus` focuses an element rendered after the page has loaded (a dialog's field, say), once
+  the commit has put it in the page: the attribute alone only works for elements there as the page
+  loads.
+- Hydration compares `style` values as the browser reads them back, so the same style written
+  another way (`hsl(...)`, read back as `rgb(...)`; `-0%` as `0%`) isn't reported as a difference.
+- New props: `ariaExpanded`, `ariaSelected`, `ariaControls`, `ariaCurrent`, `ariaHidden`,
+  `ariaLive`, `ariaModal`, `ariaBusy`, `required`, `autoComplete`, `inputMode` and `maxLength`.

.changeset/weft-server-navigation-commit.md

@@ -0,0 +1,9 @@
+---
+'@e-cline/weft_server': patch
+---
+
+- `Navigation.navigate`, `load` and `refresh` resolve once the new page is on screen (its transition
+  has committed), as documented, rather than as soon as it starts rendering: code awaiting them (to
+  clear a pending state, say) no longer runs while the old page is still showing.
+- `Bootstrap.start` takes a `~signal`: aborting it stops the page (its root unmounted, the history
+  no longer followed), for tests that open one page after another in a document.

.changeset/weft-server-redirects.md

@@ -0,0 +1,11 @@
+---
+'@e-cline/weft_server': patch
+---
+
+A navigation the server redirects lands where the redirect led. `Decoder.fetch` now resolves with
+`Decoder.fetched` (the payload's `stream`, and the `url` a redirect led to, if one did) instead of
+the bare stream; `webFetch` reports where the browser's `fetch` ended up. `Navigation.url` is that
+URL, `navigate` calls `onNavigate` with it (so the browser's history gets it, not the URL asked
+for), and a `load` or `refresh` that was redirected calls the new `~onRedirect` with it (the
+browser's replaces the history's entry). `Decoder.Latest.fetchElement` resolves with the element and
+its URL (`Decoder.Latest.fetched`).

.changeset/weft-signals-server.md

@@ -0,0 +1,7 @@
+---
+'@e-cline/weft_signals': patch
+---
+
+The hooks work in an entrypoint rendered on the server (to HTML): they give the signal's value
+there, where they failed before (`useSyncExternalStore` without a server snapshot, error 15), and
+the page fell back to rendering on the client.

.github/workflows/ci.yml

@@ -53,8 +53,9 @@ jobs:
       - run: vp run test
 
   e2e:
-    # The browser tests (examples/weft_notes/e2e): a few, against the built app in Chromium, on one
-    # Node version. Everything a browser isn't needed for is tested in the job above.
+    # The browser tests (examples/weft_notes/e2e and examples/weft_shop/e2e), against the built apps
+    # in Chromium, on one Node version. Everything a browser isn't needed for is tested in the job
+    # above.
     name: Browser tests
     runs-on: ubuntu-latest
     steps:
@@ -80,3 +81,4 @@ jobs:
           pnpm --filter @e-cline/weft_notes_example exec playwright install
           --with-deps chromium
       - run: vp run --filter @e-cline/weft_notes_example e2e
+      - run: vp run --filter @e-cline/weft_shop_example e2e

CONTRIBUTING.md

@@ -70,8 +70,9 @@ pnpm run build
 pnpm run test
 ```
 
-Run the browser tests (a few, in [`examples/weft_notes/e2e`](examples/weft_notes/e2e), against the
-built app), which need Playwright's Chromium installed once:
+Run the browser tests (in [`examples/weft_notes/e2e`](examples/weft_notes/e2e) and
+[`examples/weft_shop/e2e`](examples/weft_shop/e2e), against the built apps), which need Playwright's
+Chromium installed once:
 
 ```bash
 pnpm --filter @e-cline/weft_notes_example exec playwright install chromium

README.md

@@ -8,8 +8,8 @@ WARNING: None of this is intended for real production use.
 
 Weft is the UI framework in here: `weft` (components, hooks and a fiber reconciler), `weft_dom` (the
 DOM renderer) and `weft_signals` (hooks for reading `signals`), plus the `weft_app` example. Server
-components for it are under way: `weft_server` and `weft_vite_plugin`, with the `weft_notes` and
-`weft_split` examples.
+components for it are under way: `weft_server` and `weft_vite_plugin`, with the `weft_notes`,
+`weft_split` and `weft_shop` examples.
 
 ## Packages
 

examples/weft_shop/README.md

@@ -0,0 +1,130 @@
+# weft_shop
+
+A storefront, "Loom & Co.", built with [Weft's server components](../../packages/weft_server): the
+largest of the examples, and a test bed for how Weft's features work together, with the latency of
+real services in between. Its home and category pages are server-driven: what they're made of is
+data the server decides on for each request, and each part of them streams in by itself.
+
+- **Server-driven pages** ([`src/server/Cms.res`](src/server/Cms.res),
+  [`src/server/Blocks.res`](src/server/Blocks.res)): the home page and each category's banner are a
+  tree of blocks (heroes, rails of products, promotions, tabs, columns, live deals, an experiment)
+  kept as data and changed at runtime, and personalised: signed-in shoppers are welcomed, get
+  recommendations and a rail to buy again, and each half of the experiment sees its own promotion.
+  Every block is a server component waiting on its own service, in a `Suspense` boundary of its own,
+  inside a `BlockFrame` entrypoint: an error boundary (a block that fails says so, with Retry) that
+  fetches the block again from its own route (`useRefetchable`), on Retry or, for live deals, every
+  few seconds. Containers render their blocks the same way, so a rail in a tab streams, fails and
+  refetches by itself too, and blocks keep their state when a refresh brings a layout that moved
+  them.
+- **Streaming everywhere**: the shell first, then the layout, then each block, then each card's
+  stock, each under its own boundary; a product's offer, reviews (a promise, read on the client with
+  `Hooks.use`), rails and questions; an order's delivery estimate.
+- **The catalog** ([`src/server/Pages.res`](src/server/Pages.res)): filters the server decides on
+  per category (`FacetPanel`), each change a navigation to the page's new URL (showing what was
+  picked until it's rendered), with the results in a grid (`ProductGrid`) keyed by the filters, a
+  quick filter over the cards already there (`useDeferredValue`), and "Load more" from the
+  `catalog.more` RPC, whose answer is the cards themselves, rendered on the server, their stock
+  still streaming.
+- **Search**: suggestions as you type (`search.suggest`, through `Rpc.call` with an `AbortSignal`,
+  so each query cancels the one before it), results on Enter, and the box in the header keeping what
+  was typed as the shopper navigates.
+- **A product's page**: variants whose offer (price, stock, the add to cart button) is fetched again
+  by itself (`ProductOffer`), keeping the quantity chosen; a gallery; reviews sorted in a
+  transition.
+- **The cart, client and server** ([`src/client/Store.res`](src/client/Store.res),
+  [`src/client/ShopContext.res`](src/client/ShopContext.res)): the cart an RPC answers with goes to
+  a `signals` atom, which `ShopShell` (the entrypoint around every page, providing the page's
+  context) reads and hands down, newest of its and the server's, so the header's count, the drawer
+  (a portal) and the cart page follow at once; the cart page's quantities change before the server
+  answers (a reducer), and its totals, rendered on the server, are fetched again whenever the cart's
+  version goes up.
+- **Context and currency**: every price is a `Price` entrypoint reading the currency from
+  `ShopContext`, so picking one re-renders every price on the page with nothing fetched, and the
+  `prefs.setCurrency` RPC makes it the session's, for the next pages the server renders.
+- **Sessions and auth** ([`src/server/Session.res`](src/server/Session.res)): an `sid` cookie that
+  `@e-cline/servy` middleware sets on the first response without one, naming what the server keeps
+  for the shopper; signing in (a name, no password: a demo) from a dialog rendered into `<body>` (a
+  portal) that takes the focus (`autoFocus`), or from the sign-in page the account page redirects a
+  signed-out shopper to (a navigation lands at the URL a redirect leads to); RPC handlers finding
+  the session from the HTTP request, even on a visitor's first request.
+- **Checkout**: a three-step form (a reducer, labels linked by `useId`; an empty cart is redirected
+  back to the cart), the address checked as the RPC will check it before moving on, and the order
+  refused, saying which item, if someone else bought the last of it first. The order's page
+  refreshes itself until it's delivered.
+- **RPCs** ([`src/shared/Api.res`](src/shared/Api.res), handled in
+  [`src/server/ApiServer.res`](src/server/ApiServer.res)): eleven, with object, variant, option and
+  custom codecs, `prepare` steps that send nothing for an empty promo code or name, `check`s on both
+  sides, and errors the shopper is told about.
+- **The rest**: an offline banner (`useSyncExternalStore` with a server snapshot), a countdown that
+  renders the server's time first so hydration matches, toasts, and a stylesheet in each page's
+  head.
+
+The "database" ([`src/server/Db.res`](src/server/Db.res)) is in memory and the same every time.
+Every slow call waits on [`src/server/Latency.res`](src/server/Latency.res): in `vp dev` and
+`vp run start`, a random time from a seeded generator, with the recommendations and questions
+services failing now and then; in the tests, nothing opens until the test says. `WEFT_SHOP_LATENCY`
+(`random`, `fixed`, `off` or `manual`), `WEFT_SHOP_SEED` and `WEFT_SHOP_FAIL_RATE` set it.
+
+Run it:
+
+```bash
+pnpm --filter @e-cline/weft_shop_example exec vp run dev
+```
+
+or build it and serve the build:
+
+```bash
+pnpm --filter @e-cline/weft_shop_example exec vp run start
+```
+
+## Tests
+
+[`tests/`](tests) runs the app's scenarios through
+[the plugin's harness](../../tools/weft_vite_plugin#testing-an-app): the server in a worker thread
+(with its controls, [`tests/ShopControl.res`](tests/ShopControl.res), called there), each page in
+jsdom, cookies kept as a browser keeps them. Each test starts with every latency gate closed, and
+opens them in the order it's about:
+
+- [`Home_test.res`](tests/Home_test.res): the home page streaming block by block in any order, and
+  in chunks of any size; a boundary whose content arrives while an entrypoint's chunk is still
+  loading hydrated once it has, with nothing rendered twice; navigating away from a page still
+  streaming, which stops its services; blocks working once hydrated; a refresh bringing a layout
+  with blocks moved, added and removed, each keeping its state; the experiment; a block that fails,
+  and Retry; live deals refetching themselves; personalisation after signing in, and recommendations
+  failing; the currency changing every price at once; a page closed when the next opens.
+- [`Catalog_test.res`](tests/Catalog_test.res): a category page's blocks, filters and results
+  streaming by themselves; "Load more" (server-rendered cards from the RPC, their stock streaming
+  after); the quick filter, with nothing fetched; filters and sort as navigations, the grid starting
+  afresh, the results staying until the new ones are ready, the range reset when the filters are
+  cleared; search suggestions, a newer query cancelling an older one's request; search results.
+- [`Product_test.res`](tests/Product_test.res): the offer, reviews, rail and questions streaming by
+  themselves; hydration in chunks of any size; another variant's offer fetched by itself, keeping
+  the quantity (clamped to the stock); adding to the cart, refused past the stock; the drawer; a
+  sold out variant; reviews sorted in a transition; the questions failing, and Retry; the wishlist,
+  signed out and in, and put back when the server says no.
+- [`Cart_test.res`](tests/Cart_test.res): the cart's lines and server-rendered totals; quantities
+  changed at once, the totals following by themselves; promo codes (an empty one never sent, unknown
+  and expired ones refused); checking out, the address checked on the client, the order placed, and
+  its page refreshing itself until it's delivered; an item that ran out first; the session's cookie,
+  even for an RPC that's a visitor's first request; the account page redirecting to sign in, and
+  back after, and out again; an empty cart's checkout redirecting to the cart; names refused on both
+  sides.
+
+[`e2e/`](e2e) checks, in Chromium against the build, what only a browser can, served twice:
+
+- [`shop.e2e.ts`](e2e/shop.e2e.ts), with the latency gated (`WEFT_SHOP_TEST_ROUTES=1` adds routes
+  that open the gates): the home page streaming; blocks arriving while a chunk is held back,
+  hydrated once it's loaded, not rendered twice; a whole purchase, filter to order; the sign-in
+  dialog's focus and Escape; links to pages that redirect, landing at the redirect's URL; the
+  currency kept across a reload; the back and forward buttons through filtered pages; a failed block
+  retried; and the pages readable with JavaScript off.
+- [`random.e2e.ts`](e2e/random.e2e.ts), with seeded random latency and failures, as `vp run start`
+  serves it: every page streaming in completely, once, and hydrating; navigating quickly through
+  pages still streaming; and a purchase, retrying whatever fails.
+
+Each with nothing in the console. Run them (they need Chromium:
+`pnpm --filter @e-cline/weft_shop_example exec playwright install chromium`):
+
+```bash
+pnpm --filter @e-cline/weft_shop_example exec vp run e2e
+```

examples/weft_shop/e2e/playwright.config.ts

@@ -0,0 +1,59 @@
+import {fileURLToPath} from 'node:url';
+
+import {defineConfig, devices} from '@playwright/test';
+
+// The browser tests, against the build (`vp run e2e` builds it first), served twice by `serve.mjs`:
+// once with its latency gated (`manual`, opened by the tests through the test-only routes), for
+// tests that choose the order things arrive in; and once with seeded random latency and failures,
+// as `vp run start` serves it, for tests that only say what must happen eventually.
+const gated = Number(process.env.WEFT_SHOP_E2E_PORT ?? 4417);
+const random = gated + 1;
+const cwd = fileURLToPath(new URL('..', import.meta.url));
+
+export default defineConfig({
+  testDir: '.',
+  fullyParallel: false,
+  // One server (and one database) each: the tests take turns.
+  workers: 1,
+  reporter: process.env.CI ? 'github' : 'list',
+  projects: [
+    {
+      name: 'gated',
+      testMatch: 'shop.e2e.ts',
+      use: {...devices['Desktop Chrome'], baseURL: `http://127.0.0.1:${gated}`},
+    },
+    {
+      name: 'random latency',
+      testMatch: 'random.e2e.ts',
+      use: {
+        ...devices['Desktop Chrome'],
+        baseURL: `http://127.0.0.1:${random}`,
+      },
+    },
+  ],
+  webServer: [
+    {
+      command: 'node serve.mjs',
+      cwd,
+      url: `http://127.0.0.1:${gated}/test/ready`,
+      env: {
+        PORT: String(gated),
+        WEFT_SHOP_TEST_ROUTES: '1',
+        WEFT_SHOP_LATENCY: 'manual',
+      },
+      reuseExistingServer: false,
+    },
+    {
+      command: 'node serve.mjs',
+      cwd,
+      url: `http://127.0.0.1:${random}/c/shoes`,
+      env: {
+        PORT: String(random),
+        WEFT_SHOP_LATENCY: 'random',
+        WEFT_SHOP_SEED: '7',
+        WEFT_SHOP_FAIL_RATE: '0.3',
+      },
+      reuseExistingServer: false,
+    },
+  ],
+});