-
Notifications
You must be signed in to change notification settings - Fork 0
feat: add the streaming server helper (BON-5) #39
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Merged
Merged
Changes from all commits
Commits
Show all changes
7 commits
Select commit
Hold shift + click to select a range
1f24781
feat: add the streaming swap bootstrap
hunterbecton 84d6346
feat: add @camp.dev/bones/server streaming helper
hunterbecton aa694f7
docs: document the streaming wire protocol
hunterbecton 3d111d6
docs: load the element module async in streaming shells
hunterbecton 5af7e74
docs: state streaming ordering, CSP, and invalid-id caveats
hunterbecton e39bc14
docs: correct id-validation, pre-upgrade, and CSP wording
hunterbecton 675f669
docs: qualify view-transition support in the streaming reveal
hunterbecton File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,5 @@ | ||
| --- | ||
| "@camp.dev/bones": minor | ||
| --- | ||
|
|
||
| Add `@camp.dev/bones/server`: `streamBones` streams an HTML shell with busy `<bones-boundary>` regions and flushes out-of-order `<template>` + swap-script chunks as promises settle, plus the primitives (`BOOTSTRAP_SCRIPT`, `renderBoundary`, `renderChunk`, `renderErrorChunk`) that emit each wire-protocol piece for servers in any stack. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -7,6 +7,7 @@ | |
| "---Reference---", | ||
| "...api", | ||
| "---Guides---", | ||
| "streaming", | ||
| "styling", | ||
| "examples" | ||
| ] | ||
|
|
||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,131 @@ | ||
| --- | ||
| title: Streaming | ||
| description: Stream the shell immediately, then fill each boundary as its data resolves. | ||
| --- | ||
|
|
||
| `@camp.dev/bones/server` streams an HTML page in two stages. The shell goes out immediately, with each slow region rendered as a busy `<bones-boundary>`, so the first paint shows skeletons. As each region's data resolves — in any order — the server flushes a chunk that swaps the real content in. The client needs no library: a small inline script sent with the shell performs the swaps. When `@camp.dev/bones/element` is on the page, each reveal also gets the element's `min-duration` timing and view-transition crossfade. | ||
|
|
||
| ```ts | ||
| import { renderBoundary, streamBones } from "@camp.dev/bones/server"; | ||
|
|
||
| const shell = `<!doctype html> | ||
| <html lang="en"> | ||
| <head> | ||
| <link rel="stylesheet" href="https://unpkg.com/@camp.dev/bones/src/css/auto.css" /> | ||
| <script type="module" async src="https://unpkg.com/@camp.dev/bones/dist/element/index.mjs"></script> | ||
| </head> | ||
| <body> | ||
| <h1>Dashboard</h1> | ||
| ${renderBoundary("stats", "<p>Three lines of stats load here.</p>")}`; | ||
|
|
||
| export default { | ||
| fetch(): Response { | ||
| return new Response(streamBones(shell, { stats: loadStats().then(renderStats) }), { | ||
| headers: { "content-type": "text/html; charset=utf-8" }, | ||
| }); | ||
| }, | ||
| }; | ||
| ``` | ||
|
|
||
| The shell never closes `body` or `html` — both closing tags are optional in HTML, and the end of the response ends the document. Later chunks parse inside the still-open body. | ||
|
|
||
| ## streamBones | ||
|
|
||
| `streamBones(shell, slots, options?)` returns a `ReadableStream<Uint8Array>`. | ||
|
|
||
| - `shell` is the shell HTML, ending inside an open `<body>`. It is sent as-is, never parsed or validated. `streamBones` appends the bootstrap script and streams the concatenation as its first chunk. | ||
| - `slots` maps slot ids to promises of HTML. Each settlement flushes one chunk immediately, in settlement order. The stream closes when all promises have settled. | ||
| - `options.onError(id, error)` renders the error HTML for a rejected slot. Returning `undefined` (and throwing) falls back to the bare error chunk, which keeps the boundary's fallback children. Without the callback, every rejection gets the bare chunk. | ||
| - Ids must match `[A-Za-z0-9_-]+` and be unique in the document. An invalid id throws before the stream starts — synchronously, before any slot promise gets a handler, so pair it with valid ids rather than relying on it at runtime: a rejecting slot promise in the same call would surface as an unhandled rejection. | ||
| - Cancelling the stream — the client disconnected — stops all further flushes. The slot promises are not aborted; their late settlements are discarded. | ||
|
|
||
| On Workers, Bun, and Deno, pass the stream to `Response` directly. On Node's `http` server, convert it with `Readable.fromWeb(stream).pipe(res)`. | ||
|
|
||
| Send `Content-Type: text/html; charset=utf-8`, and defeat proxy buffering where it applies — `X-Accel-Buffering: no` for nginx. Compression that buffers the whole response before sending also defeats streaming. | ||
|
|
||
| ## Primitives | ||
|
|
||
| `streamBones` is built from four exports that each emit one protocol piece. A server that owns its response stream can call them directly and skip `streamBones`. | ||
|
|
||
| | Export | Emits | | ||
| | ------------------------------------------ | ------------------------------------------------------------------------------------------------------------------------------- | | ||
| | `BOOTSTRAP_SCRIPT` | The inline swap runtime in one `<script>` tag. Send once, after the shell, before any chunk. | | ||
| | `renderBoundary(id, fallbackHtml, attrs?)` | A pending boundary. `attrs` is a raw attribute string appended to the tag, such as `'precision="measured" min-duration="600"'`. | | ||
| | `renderChunk(id, html)` | A content chunk: the template and the swap call. | | ||
| | `renderErrorChunk(id, html?)` | An error chunk. Without `html` it is a bare swap call, and the boundary keeps its fallback children. | | ||
|
|
||
| The three render functions validate their id and throw on a mismatch. | ||
|
|
||
| ## Wire protocol v1 | ||
|
|
||
| This section is the protocol's specification. A server in any language can implement it; the primitives above are the reference implementation. | ||
|
|
||
| **The shell.** The document head and as much of the body as renders immediately. Each loading region is a pending boundary: | ||
|
|
||
| ```html | ||
| <bones-boundary busy aria-busy="true" inert data-bones-slot="stats"> | ||
| <p>Placeholder shaped like the real content.</p> | ||
| </bones-boundary> | ||
| ``` | ||
|
|
||
| The ids are unique per document and match `[A-Za-z0-9_-]+`. The fallback children give the stylesheets something to skeletonize; an empty boundary shows nothing while it waits. The shell ends inside an open `<body>`, and no `</body>` or `</html>` appears anywhere in the stream. | ||
|
|
||
| Load the element module with an async script (`<script type="module" async src="…">`): without `async`, module scripts wait for the document to finish parsing, which on a streamed page is the end of the stream — the element would upgrade only after the last chunk. | ||
|
|
||
| **The bootstrap.** Sent once, directly after the shell, before any chunk. This is the readable form of `BOOTSTRAP_SCRIPT`: | ||
|
|
||
| {/* prettier-ignore */} | ||
| ```html | ||
| <script> | ||
| function __bonesSwap(id, err) { | ||
| var t = document.querySelector('template[data-bones-chunk="' + id + '"]'); | ||
| var b = document.querySelector('[data-bones-slot="' + id + '"]'); | ||
| if (!b) { if (t) t.remove(); return; } | ||
| if (t) { b.replaceChildren(t.content); t.remove(); } | ||
| if (err) b.setAttribute("data-bones-error", ""); | ||
| if (typeof b.busy === "boolean") b.busy = false; | ||
| else { | ||
| b.removeAttribute("busy"); | ||
| b.removeAttribute("aria-busy"); | ||
| b.removeAttribute("inert"); | ||
| } | ||
| } | ||
| </script> | ||
| ``` | ||
|
|
||
| Content swaps in before `busy` clears, so once the element has upgraded, the reveal honors `min-duration` and — where the browser supports the View Transitions API and the user has not asked for reduced motion — runs inside a view transition; otherwise the swap applies immediately. Anywhere the element is not upgraded yet — the page loads no element module, or a chunk beats the async module on first load — the attributes come off directly and the content is simply there: a boundary whose chunk wins that race skips the timing and the crossfade by design, because holding finished content hostage to a script download would be worse than an unanimated reveal. | ||
|
|
||
| **A content chunk.** Flushed together, template first: | ||
|
|
||
| {/* prettier-ignore */} | ||
| ```html | ||
| <template data-bones-chunk="stats"><section>…real content…</section></template> | ||
| <script>__bonesSwap("stats")</script> | ||
| ``` | ||
|
|
||
| The script executes when its closing tag parses, so the template is always complete first, even if the network splits the chunk mid-tag. | ||
|
|
||
| **An error chunk.** With author-rendered content, or bare: | ||
|
|
||
| {/* prettier-ignore */} | ||
| ```html | ||
| <script>__bonesSwap("stats",1)</script> | ||
| ``` | ||
|
|
||
| A bare error chunk keeps the boundary's fallback children. Either form sets `data-bones-error=""` on the boundary; style it with `bones-boundary[data-bones-error]`. | ||
|
|
||
| **End of stream.** After the last chunk, close the connection. Nothing else is sent. | ||
|
|
||
| **Constraints.** Chunk content must be balanced HTML: a nested, balanced `<template>` inside content is fine, but a stray `</template>` terminates the chunk's template early. Nothing in the protocol escapes or sanitizes content — the server owns that. Streamed content can itself contain a pending boundary whose chunk arrives later, because templates and swap calls resolve against the whole document — but the order matters: an inner boundary's chunk must flush after the chunk that introduces it. A swap call whose boundary is not in the document yet discards its template, and the boundary it later arrives inside stays busy forever. With `streamBones`, derive the inner promise from the outer's: `inner: outer.then(() => loadInner())`. | ||
|
|
||
| The protocol runs on inline scripts, and neither `streamBones` nor the primitives accept a nonce yet, so it works only under a `script-src` policy that allows `unsafe-inline`. A server that implements the protocol itself can stamp its own nonce on every `<script>` tag it emits, bootstrap included. | ||
|
|
||
| ## Implementing the protocol in another stack | ||
|
|
||
| A server in any language streams Bones by emitting five things: | ||
|
|
||
| 1. The shell, with each pending region as `<bones-boundary busy aria-busy="true" inert data-bones-slot="ID">fallback</bones-boundary>`, and no `</body>` or `</html>` anywhere. | ||
| 2. The bootstrap `<script>` above, once, directly after the shell. | ||
| 3. For each resolved region: `<template data-bones-chunk="ID">content</template><script>__bonesSwap("ID")</script>`, flushed together. | ||
| 4. For each failed region: the same shape with `__bonesSwap("ID",1)`, or the bare `<script>__bonesSwap("ID",1)</script>` to keep the fallback. | ||
| 5. Nothing else. Close the connection after the last chunk. | ||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,31 @@ | ||
| // --------------------------------------------------------------------------- | ||
| // The inline swap runtime, sent once per streamed document. | ||
| // | ||
| // Readable form (the tests execute the shipped string against both branches): | ||
| // | ||
| // function __bonesSwap(id, err) { | ||
| // var t = document.querySelector('template[data-bones-chunk="' + id + '"]'); | ||
| // var b = document.querySelector('[data-bones-slot="' + id + '"]'); | ||
| // if (!b) { if (t) t.remove(); return; } | ||
| // if (t) { b.replaceChildren(t.content); t.remove(); } | ||
| // if (err) b.setAttribute("data-bones-error", ""); | ||
| // if (typeof b.busy === "boolean") b.busy = false; | ||
| // else { | ||
| // b.removeAttribute("busy"); | ||
| // b.removeAttribute("aria-busy"); | ||
| // b.removeAttribute("inert"); | ||
| // } | ||
| // } | ||
| // | ||
| // The typeof check picks the path. An upgraded <bones-boundary> gets | ||
| // `busy = false` and runs its own draining and hide machinery. Anything else | ||
| // — the element module absent or still loading, or a plain [data-bones-slot] | ||
| // target — gets the three attributes removed directly. Clearing aria-busy or | ||
| // inert from outside a showing element would fight its attribute defense, so | ||
| // the bootstrap never touches them on an upgraded element. | ||
| // --------------------------------------------------------------------------- | ||
|
|
||
| export const BOOTSTRAP_JS = | ||
| 'function __bonesSwap(e,r){var t=document.querySelector(\'template[data-bones-chunk="\'+e+\'"]\'),n=document.querySelector(\'[data-bones-slot="\'+e+\'"]\');n?(t&&(n.replaceChildren(t.content),t.remove()),r&&n.setAttribute("data-bones-error",""),"boolean"==typeof n.busy?n.busy=!1:(n.removeAttribute("busy"),n.removeAttribute("aria-busy"),n.removeAttribute("inert"))):t&&t.remove()}'; | ||
|
|
||
| export const BOOTSTRAP_SCRIPT = `<script>${BOOTSTRAP_JS}</script>`; |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,93 @@ | ||
| // --------------------------------------------------------------------------- | ||
| // @camp.dev/bones/server — the wire protocol, as functions | ||
| // | ||
| // Runs anywhere with Web Streams: no DOM, no Node APIs, no imports from the | ||
| // element or React entries. streamBones is built from the exported | ||
| // primitives, so the high-level path and the documented protocol cannot | ||
| // drift. Ids are validated instead of escaped: [A-Za-z0-9_-]+ needs no | ||
| // escaping in an attribute value or a JS string literal. | ||
| // --------------------------------------------------------------------------- | ||
|
|
||
| import { BOOTSTRAP_SCRIPT } from "./bootstrap.ts"; | ||
|
|
||
| export { BOOTSTRAP_SCRIPT }; | ||
|
|
||
| const ID_PATTERN = /^[A-Za-z0-9_-]+$/; | ||
|
|
||
| function assertId(id: string): void { | ||
| if (!ID_PATTERN.test(id)) { | ||
| throw new Error(`bones slot id ${JSON.stringify(id)} must match [A-Za-z0-9_-]+`); | ||
| } | ||
| } | ||
|
|
||
| export function renderBoundary(id: string, fallbackHtml: string, attrs?: string): string { | ||
| assertId(id); | ||
| const extra = attrs === undefined || attrs === "" ? "" : ` ${attrs}`; | ||
| return `<bones-boundary busy aria-busy="true" inert data-bones-slot="${id}"${extra}>${fallbackHtml}</bones-boundary>`; | ||
| } | ||
|
|
||
| export function renderChunk(id: string, html: string): string { | ||
| assertId(id); | ||
| return `<template data-bones-chunk="${id}">${html}</template><script>__bonesSwap("${id}")</script>`; | ||
| } | ||
|
|
||
| export function renderErrorChunk(id: string, html?: string): string { | ||
| assertId(id); | ||
| if (html === undefined) return `<script>__bonesSwap("${id}",1)</script>`; | ||
| return `<template data-bones-chunk="${id}">${html}</template><script>__bonesSwap("${id}",1)</script>`; | ||
| } | ||
|
|
||
| export interface StreamBonesOptions { | ||
| /** | ||
| * Renders the error HTML for a rejected slot. Returning undefined (and | ||
| * throwing) falls back to the bare error chunk, which keeps the boundary's | ||
| * fallback children. | ||
| */ | ||
| onError?: (id: string, error: unknown) => string | undefined; | ||
| } | ||
|
|
||
| export function streamBones( | ||
| shell: string, | ||
| slots: Record<string, Promise<string>>, | ||
| options: StreamBonesOptions = {}, | ||
| ): ReadableStream<Uint8Array> { | ||
| const ids = Object.keys(slots); | ||
| for (const id of ids) assertId(id); | ||
| const encoder = new TextEncoder(); | ||
| let cancelled = false; | ||
| return new ReadableStream<Uint8Array>({ | ||
| start(controller) { | ||
| const send = (html: string): void => { | ||
| if (!cancelled) controller.enqueue(encoder.encode(html)); | ||
| }; | ||
| send(shell + BOOTSTRAP_SCRIPT); | ||
| if (ids.length === 0) { | ||
| controller.close(); | ||
| return; | ||
| } | ||
| let pending = ids.length; | ||
| const settle = (chunk: string): void => { | ||
| send(chunk); | ||
| pending -= 1; | ||
| if (pending === 0 && !cancelled) controller.close(); | ||
| }; | ||
| for (const id of ids) { | ||
| slots[id].then( | ||
| (html) => settle(renderChunk(id, html)), | ||
| (error) => { | ||
| let html: string | undefined; | ||
| try { | ||
| html = options.onError?.(id, error); | ||
| } catch { | ||
| html = undefined; | ||
| } | ||
| settle(renderErrorChunk(id, html)); | ||
| }, | ||
| ); | ||
| } | ||
| }, | ||
| cancel() { | ||
| cancelled = true; | ||
| }, | ||
| }); | ||
| } |
Oops, something went wrong.
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.