Repository navigation
chore: merge main into rewrite/main - #1035
Conversation
…geset (#823) * feat(website): add Astro marketing site with landing film, docs, changelog and community pages * fix(website): address PR review and husky install script Require hero film metadata to match copied video, improve ScreenBar and reveal a11y, fix deploy docs paths, and make husky install exit cleanly when the package is unavailable.
* feat(nip77): add negentropy wire messages and settings Signed-off-by: Priyanshubhartistm <bhartipriyanshustm@gmail.com> * feat(nip77): serve negentropy sessions over the websocket Signed-off-by: Priyanshubhartistm <bhartipriyanshustm@gmail.com> * test(nip77): cover negentropy wire protocol and sessions Signed-off-by: Priyanshubhartistm <bhartipriyanshustm@gmail.com> --------- Signed-off-by: Priyanshubhartistm <bhartipriyanshustm@gmail.com> Co-authored-by: Ricardo Cabral <me@ricardocabral.io>
…e Workers (#828) * ci(website): version the site through changesets website/site joins the pnpm workspace as nostream-website, so the marketing site is versioned by the same changesets flow as the relay. Its standalone lockfile folds into the root one. Verified: changeset version bumps nostream-website only (nostream stays at its own version), site builds from the merged lockfile. * ci(website): deploy the static site to Cloudflare Workers wrangler.jsonc binds the built dist/ as a Workers static-assets site (html_handling auto-trailing-slash, SPA fallback for unknown paths — the site has no 404 page and no server code; everything is prerendered). wrangler joins website/site as a devDependency. Deploy Website runs after a release PR merges to main (paths cover the site, its lockfile and every repo file the build reads at build time: README, docs, deploy runbooks, CHANGELOG, package.json), plus workflow_dispatch. It builds with SITE_URL from the website-production environment and runs wrangler deploy with CLOUDFLARE_API_TOKEN from that environment. ema.tnsor.network rides on a Workers Custom Domain — set it on the worker at first deploy. Dry-run verified: wrangler reads the 96 built files, 0.37 KiB worker. * ci: teach knip the website workspace The workspace merge made knip scan website/site, where it cannot trace Astro's build-time imports (astro, sharp, the CSS-imported @fontsource packages) or wrangler (invoked by CI, not imported), so all five read as unused dependencies and check:deps failed. Each gets an explicit ignoreDependencies row in the website/site workspace block, scoped to that workspace only. * ci(website): point the deploy at the website environment Cameri provisioned the environment under the name 'website' with CLOUDFLARE_API_TOKEN (secret) and CLOUDFLARE_ACCOUNT_ID (var), so the job references that environment directly. SITE_URL is pinned to https://ema.tnsor.network — the site is built for one production domain, no need for a configurable var. * ci: apply Greptile review fixes to the website deploy - workflow_dispatch on deploy-website.yml ignores the push-branch filter; the job now refuses to run off main - website releases (nostream-website@ tags) no longer build the relay container image: release.yml gates on the relay's vX.Y.Z tag shape - pnpm.onlyBuiltDependencies moves from website/site/package.json to pnpm-workspace.yaml, where pnpm 10 reads it; esbuild postinstalls run again instead of being silently skipped
A release PR (changeset-release/main) only touches manifests and changelogs — the code it carries is identical to what just finished green on main, and the squash-merge runs the full suite on main anyway (checks.yml push trigger). Keeping lint, build check and commitlint on the release PR preserves the branch-protection signal while dropping the two long suites (unit + integration/docker). Same gate shape as changeset-check already uses (github.head_ref != 'changeset-release/main').
…e admin console (#830) * feat(nip77): advertise NIP-77 in NIP-11 supported_nips when active Signed-off-by: Priyanshubhartistm <bhartipriyanshustm@gmail.com> * feat(nip77): expose nip77 settings in the admin console Signed-off-by: Priyanshubhartistm <bhartipriyanshustm@gmail.com> --------- Signed-off-by: Priyanshubhartistm <bhartipriyanshustm@gmail.com> Co-authored-by: Ricardo Cabral <me@ricardocabral.io>
… config (#832) * fix(website): bind ema.tnsor.network as the worker's custom domain in config The deploy never attaches the domain: nothing in wrangler.jsonc asked for it, and the dashboard attachment went to a different worker (a staging OnlyGPUs app holds ema.tnsor.network and serves its homepage while every other path 404s). Declaring the route with custom_domain makes every deploy idempotently attach the domain to nostream-website — last writer wins, so CI redeploys self-heal the binding instead of relying on a one-time dashboard click. Requires the deploy token to carry Workers Domains:Edit (attach may fail with an auth error if not; the run log will name it). * chore: add changeset for the website custom-domain fix --------- Co-authored-by: Ricardo Cabral <me@ricardocabral.io>
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
) * ci(website): deploy on nostream-website release publish, not on main pushes The site now deploys when the changesets workflow tags and publishes nostream-website@X.Y.Z (the release event this workflow filters on). Merging relay work to main no longer touches the site deploy; a site deploy happens exactly when a release PR that bumps nostream-website merges. Checkout on a release event is the tagged commit, so the deployed assets are the released build. Relay image releases keep their v* gate in release.yml — the tag namespaces do not overlap. workflow_dispatch remains for manual deploys of main. * fix(website): warn instead of failing while the hero film is stale The film guard hard-failed every site build since #830 added NIP-77 (supportedNips 24 -> 25): re-rendering the film needs the GPU pipeline and no GPU is available to the sandbox or the runners. Per operator order the mismatch branch now warns (loudly, with the re-render commands) and exits 0, unblocking releases; the comment in the script names the day, the order, and where to restore exit 1.
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
* feat(website): user-benefit copy with (i) detail tooltips Homepage copy led with protocol internals — NIP numbers, sliding-window rate limits, SSE, Redis fan-out — where visitors want the job it does. Visible copy now states the benefit; the technical detail moved under a new InfoTip component (hover, keyboard focus, or tap) on every pillar, audience bullet and console caption. 'Limits that hold' becomes 'Keeps abusive traffic out — throttles users with efficient methods to prevent abuse', per Cameri's example. * feat(website): per-PR previews at /preview/<n> PRs touching the site build under the /preview/<n>/ base and deploy their own assets-only worker (nostream-web-pr-<n>, workers.dev URL); the PR gets a comment with the preview link, and a closed-PR run deletes the worker. The production worker becomes a router (worker/router.ts + wrangler.router.jsonc): it serves the production assets itself via an ASSETS binding and proxies /preview/<n>/* to the PR worker, keeping the preview URL on ema.tnsor.network. The custom-domain cutover happens on the next production deploy; rollback is deploying the old config again. Preview workflow (deploy-website-preview.yml) skips with a notice when the website-preview GitHub environment or its CLOUDFLARE_API_TOKEN is absent, so PRs are not blocked before the one-time setup. Base-path handling: astro.config reads SITE_BASE; hand-written root paths (nav, media, icons) are prefixed by scripts/apply-base.mjs at build time (Astro's base covers managed URLs only). Media refs now go through import.meta.env.BASE_URL. * fix(website): scope preview job to the website-preview environment Env-scoped secrets only resolve when the job declares the environment; without this the preview deploy would run with an empty token. * fix(website): guard previews on env existence only Listing environment secrets needs permissions the runner's github.token does not carry, so the secrets probe always failed and routed to the skip branch even after the secrets were set — every preview run skipped with 'lacks CLOUDFLARE_API_TOKEN'. Existence is the only check the runner token can make; with the environment now in place previews run directly from PRs. * fix(website): point the router at the real preview hostname Workers are served from <name>.<account-subdomain>.workers.dev, not <name>.workers.dev — nostream-web-pr-842.workers.dev does not resolve. Verified the real URL from the deploy log and against the live worker. * fix(website): address review findings on InfoTip + LIM copy Greptile P2s on InfoTip.astro: - Screen readers heard only 'Technical detail: label' with no access to the detail. The bubble now carries a unique id referenced by aria-controls (aria-describedby is awkward on a toggle; details/summary exposes state natively) and aria-expanded tracks open state. - Touch: a tap does not reliably set :hover or :focus-visible on a button. The toggle is now a native <details><summary> — tap opens, outside click and Escape close (small inline script), so touch and keyboard both get a reliable open/close with no dependencies. - Cards cut the bubble: Pillars .cell had overflow: hidden. Removed there (the .count watermark is now self-clipped instead, max-width + overflow on itself), so bubbles escape the card. Console's overflow:hidden is on the screenshot frame only, which is correct. - Pointer path: the bubble sat across an 8px gap with pointer-events: none, so moving onto it closed it. The open state now accepts pointer events and a transparent bridge spans the (now 4px) gap; closed state still ignores pointers. Ferryx's copy note: the LIM body rephrased to target spam/abusive traffic rather than reading as throttling regular clients — 'Keeps spam and abusive traffic from slowing the relay for everyone else, so normal clients stay fast.' * fix(website): sync aria-expanded via the details toggle event The toggle event dispatches on the <details> element, not the summary, so the summary-attached listener missed programmatic closes (outside click, Escape) and aria-expanded could stay true. Listen on the details element itself, which covers every close path including closeAll(). * fix(website): one in-place preview comment per PR The pipeline commented anew on every push, piling up identical preview links. The comment is now upserted: a hidden marker identifies it, the step flips it to a 'preview is rendering' state before the deploy and back to the link afterwards, and the close path updates the same comment instead of adding a cleanup note. One comment per PR, always. Existing duplicates on #842 deleted; the survivor carries the marker. * fix(website): lead the preview comment with the working URL The ema.tnsor.network/preview/<n> form only works once the router ships with this PR's merge, but the direct nostream-web-pr-<n>.cameri.workers.dev URL serves the build as soon as the deploy lands — the comment now leads with the working link and notes when the domain form activates. The rendering state points at the direct URL too.
* fix(website): skip the dependents probe when deleting previews wrangler delete probes the account's KV namespaces to warn about dependents — a call the deploy token (Workers Scripts scope only) cannot make, so it errored with auth 10000 after the delete had already succeeded. --force skips that probe; preview workers are assets-only with no bindings, so the check is pure overhead. A missing worker still errors and the || echo catches it. * chore: add changeset for preview delete --force
… polish from review (#845)
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
* fix(website): deploy main, not the tag ref Two fixes to the release-triggered deploy observed in run 38835604913: 1. The website environment's custom branch policy does not admit tag refs, and the release event's ref IS a tag — the deploy can never pass environment protection. Checkout refs/heads/main instead: the release commit is main's tip by construction (changesets opens the release PR from main), so the deployed build is the released content. 2. Restore job-level concurrency (lost when the workflow landed on main without it): a relay release must fail the job's tag filter before it enters the deploy queue, not after, or it can replace a queued website deploy. * chore(website): add the changeset for the deploy fix * fix(website): check out the release tag, not main The environment's branch policy admits nostream-website@* (every release deploy since 1.1.1 has run on the tag ref), so the refs/heads/main pin was never needed for admission — and it was wrong for correctness: main can advance between the release publish and the job starting, and the pin would build and deploy unreleased content. Keep the default tag checkout so the deployed build is exactly the released commit. Also restore the router-worker deploy line the branch predates. * docs(website): correct the changeset's checkout description The changeset still claimed releases deploy main because the environment rejects tag refs; the fix actually keeps the triggering tag (which the policy admits) and the real change is the job-scoped deploy queue. Rewritten before it becomes release notes. --------- Co-authored-by: Abhay <pandeyabhay967@gmail.com>
* docs: link the project website from the README intro * docs: add the changeset the repo requires * docs: modernize the README Add a table of contents, fix the NIP checklist to match package.json's supportedNips (adds NIP-42/43/50/77, marks NIP-26 as removed upstream, drops the duplicate NIP-44), reorder the major sections into a start-first flow (quick starts before the payments guide, standalone next to Docker, troubleshooting near the end), fix the stale ./scripts reference in the NWC step (the wrapper scripts are gone), and normalize the indented code fences to plain ones. No factual content changed: URLs, environment variable names and command invocations are byte-identical to the previous version. * docs: drop the changeset — a README change does not release the website * docs: add an empty changeset — docs-only change, no release
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
* docs: add NIP parity audit for the rewrite Adds docs/rewrite/nip-parity.md and .changeset/wise-berries-nail.md. * docs(rewrite): correct three NIP-audit claims found in review - NIP-09: DeleteEventStrategy collects only e tags; a-tag deletion is absent (row 09, new delta section, new Phase 2 item 12) - NIP-28: kind 41 (channel metadata) is replaceable and routed to ReplaceableEventStrategy; 40 and 42-49 use DefaultEventStrategy - NIP-11: max_filters, max_filter_values and search_supported do have tests; the untested set is min_prefix, supported_nip_extensions, supported_mips, max_event_tags Addresses Greptile's three P2 threads on #872. Oracle passes.
* feat(admin): add bounded NIP-66 probe history and timeline Closes #800 * fix(admin): address Greptile review on probe history Register nip66.historyRetentionHours in defaults, trim history on read, use rank-based Redis cleanup, isolate history failures from NIP-66 publish, and preserve timeline selection during dashboard polling. * fix(website): register Cloudflare worker entry for Knip
* feat(admin): align console with nostream.dev theme Restyle the admin console and fix Grafana solo-panel embeds for in-dashboard charts. * fix(admin): address Greptile review on console theme PR Load Grafana panels only after sign-in, expose full probe details via title, and fix dashboard.css edit instructions.
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
[High impact] The merge changes relay access, admin operations, and deployment workflows. The PR still needs the previously reported fix that allows Findings
|
| if (!enqueued) { | ||
| logger.error('probe failed notification was not enqueued') | ||
| } |
There was a problem hiding this comment.
PROBE_TRANSITION_SCRIPT records the new status before the alert is saved to the outbox. If the database is temporarily unavailable, this branch only logs the failure. Later probes see the same status and produce no alert, so operators miss the outage or recovery notice. Keep the transition pending until its alert has been saved, with a safe retry path.
There was a problem hiding this comment.
Same as the moderation lock: this file came from main in this merge and the fix is a behaviour change, which does not belong in a sync. It is worth an issue — a probe that fails to save its alert loses the alert permanently — and I will file it if you want it tracked now.
There was a problem hiding this comment.
Yes—please file an issue for it. Since this behavior came from main, it should be tracked as a follow-up rather than changed in this sync PR. The issue can capture that a failed enqueue after the status transition is persisted suppresses a retry on subsequent probe runs.
The merge brought main's default document wholesale, and `defaults_roundtrip` refuses it: serializing the typed settings has to reproduce the document exactly, so every key the typed schema does not model fails the round trip. Five surfaces did, all of them added on main: - `nip77` (negentropy sync) - deferred on this branch, and the config tests say so by name (`deferred_nip77_surface_is_not_modelled`) - `peering` - `payments.subscriptionPlans` - `nip66.historyRetentionRuns` / `historyRetentionHours` - `admin.notifications.events.probe.failed` / `probe.recovered` / `ssl.expiring` A toggle nothing emits is a setting that silently does nothing, so each came out of the document with a comment naming where it went, rather than staying in a document the daemon cannot round-trip. The legacy TypeScript tree that also merged reads all of them through optional chaining, so an absent key is the feature being off, not a crash. `mirroring.static` went the other way: main deleted the worker and the key together, but this branch still parses it (`NostreamSettings::mirroring`), so the block is restored rather than left as a parser with nothing to read. Verified: `cargo test -p nostream-config` green including the round trip, `cargo test -p nostream-cli --test config_cli` green, `config validate` reports "settings valid: 0 issues" on the edited document.
My earlier call was wrong and the integration suite said so: the validator's key list is the defaults document itself, in both runtimes, so deleting a section from `resources/default-settings.yaml` makes every configuration that mentions it invalid - `Settings validation accepts a complete peering configuration` failed exactly that way. main's document is correct; this branch's schema was incomplete. So the shapes are modelled, and the document is main's: - `nip77` (Nip77Settings), `peering` (PeeringSettings with scopes, terms and peers), `payments.subscriptionPlans` (SubscriptionPlan), the two `nip66.historyRetention*` fields, and the three probe/SSL notification toggles. Modelled as data, so the document round-trips and validates; the Rust relay does not answer NIP-77 yet and the guided editor still has no category for it, which is what `deferred_nip77_surface_is_not_modelled` actually asserts. Two validation rules arrived with main and are mirrored here, because the parity fixture is a capture of the TypeScript validator and it now grades them: - `info.relay_url` must parse, carry a host, and use ws, wss, http or https — same order, same messages. - a rate limit's `period` is now graded as not-a-number, not-whole, not-at-least- one rather than a single positivity check. The fixture is regenerated against the merge commit (d286d80) rather than 242a300, from a capture tree built out of the merged sources: the TS validator in this tree is main's, so it is the oracle again, and `deferred` is emptied because nothing is deferred any more. Verified: `cargo test --workspace` green, `config validate` reports "settings valid: 0 issues", and the round trip reproduces the document.
|
On the Greptile threads, since two of them point at a revision that no longer exists:
|
Modelling `nip77`, `peering` and subscription plans made the document round-trip and validate for the wrong reason: the rules that reject a bad value were never carried over, so `config validate` blessed documents the TypeScript app refuses. One file, two verdicts, and the Rust one said yes. Ported: the `nip66.historyRetentionHours` 0-or-at-least-1 rule, the three `nip77` caps (positive integers when enabled), subscription plans (an object; a required, unreserved, unique id; a positive and unique amount; positive interval and grace), and peering (`minRequestPowDifficulty` an integer 0..=256, a ws/wss `endpoint`, a URI-shaped `contact`, and scopes plus contact-or-endpoint required once enabled). `min_request_pow_difficulty` was `Option<u8>`, which cannot hold the 256 the TypeScript validator accepts, so a valid operator document failed to load as typed and Rust replaced the whole configuration with defaults. It is `u16`. Five new parity cases measure this against the capture — `nip77_zero_cap`, `plan_negative_amount`, `plan_duplicate_amount`, `peering_pow_out_of_range`, `peering_enabled_without_scopes` — all green, and `cargo test --workspace` is clean. Still not ported, and tracked rather than implied: the peering scopes, terms and peers graph rules, which reject a malformed feed, price or peer entry. A configuration that trips one of those is accepted here and refused by the TypeScript app until #1036 lands the behaviour.
| if !to_number_is_positive(key(plan, "graceSeconds")) { | ||
| issues.push(Issue::new( | ||
| format!("{prefix}.graceSeconds"), | ||
| "graceSeconds must be a positive integer", | ||
| )); |
There was a problem hiding this comment.
The new check rejects graceSeconds: 0, which the TypeScript validator accepts and the payment code uses to end access without a grace period. A previously valid shared settings file now fails Rust validation, and load() replaces the operator's entire configuration with shipped defaults.
Allow non-negative integers here and add a zero-grace case to the parity fixture.
Aikido's scan of the branch-sync PR reports one new CRITICAL here, and it is a real advisory rather than a scan artefact: astro before 7.2.8 runs code from an optimized AVIF image (GHSA for the AVIF optimization path), and the site pinned ^5.14.0, which resolves to 5.18.2 today. Three advisories close with this bump: astro remote code execution via AVIF image optimization fixed in >= 7.2.8 astro reflected XSS via an unescaped slot name fixed in >= 6.3.3 astro host-header SSRF in the prerendered error fetch fixed in >= 6.4.6 sharp goes to ^0.35.5 for the libvips, libheif and librsvg advisories reachable from the same tree. Astro 7 changes the default Markdown processor, so `markdown.rehypePlugins` in astro.config.mjs needs @astrojs/markdown-remark installed explicitly; without it the build refuses with a message naming the package. Nothing else in the config or in src/ needed to change. Verified: `pnpm run build` completes (15 pages, the brand and NIP prebuild checks pass), and `pnpm audit` no longer reports a critical or an astro advisory. The remaining highs are transitive under wrangler and the astro toolchain (tmp, browserslist, follow-redirects, undici, zod, body-parser), which a major bump of the toolchain would change, not this one.
knip reads static imports, and @astrojs/markdown-remark is loaded by astro itself for the rehype plugin in astro.config.mjs, so the dependency check called a needed package unused. The changeset is empty on purpose: website/site is private and is not in the release set.
Brings #1038 and the release commit that followed it onto the sync branch, so the guard the site change closes is closed here too rather than carried as a branch difference. No conflicts.
Why
mainis 142 commits ahead ofrewrite/main, which is 30 ahead of it. This PR mergesmainintorewrite/mainso the two lines stop drifting: everything that landed onmainsince the split — the knex 3.2.9 security bump, the deploy retry script, the workflow path filters, the website preview workflows — arrives on the rewrite branch in one reviewed merge instead of a dozen cherry-picks.It supersedes #1025, which carried the knex fix onto this branch alone; that fix is in
mainvia #1024 and arrives here with the merge.Conflicts, and how each was settled
Three files, each resolved toward the side that matches the code this branch runs.
.github/workflows/checks.yml— union. The rewrite's filters forresources/andcrates/stay;main's.github/workflows/**and.github/scripts/**globs are added to thesrcfilter, because a unit test reads those workflows and ci(release): publish legal image tags and keep latest off prereleases #965 skipped the very suite arelease.ymlchange broke.CONFIGURATION.md— the rewrite's rows stay.mainremovedmirroring.statictogether with its TypeScript worker. This branch still parses the setting (nostream-config:settings.rs,guided.rs,validate.rs,tests/mirror_filters.rs), somain's "removed" row describes a worker that never existed here.resources/default-settings.yaml— both kept. The rewrite'sconnection.maxConnectionsPerIpblock stays, andmain's comment onperiodunits stays above therateLimitslist it describes.Two more things the merge did not report as conflicts
resources/default-settings.yamlmerged cleanly and was still wrong. main's document carries five configuration surfaces this branch's typed schema did not model, anddefaults_roundtriprefuses a document the typed settings cannot reproduce exactly. The first attempt removed them from the document; the integration suite rejected that, correctly — the validator's key list is the defaults document in both runtimes, so a section removed from it makes every configuration that mentions it invalid (Settings validation accepts a complete peering configurationis that test). The document is main's, and the schema grew:nip77,peering,payments.subscriptionPlans,nip66.historyRetention*and the probe/SSL notification toggles are modelled as data. Behaviour is not claimed for them — the Rust relay does not answer NIP-77 yet, and the guided editor still has no category for it, which is what the branch's own deferral test actually asserts.Two validator rules changed on main and had to be mirrored, because the parity fixture is a capture of the TypeScript validator:
info.relay_urlmust parse, carry a host and use ws/wss/http/https (same order, same messages), and a rate limit'speriodis now graded as not-a-number, not-whole, not-at-least-one instead of one positivity check. The fixture is regenerated against this merge commit rather than the old capture commit, from a capture tree built out of the merged sources.mirroring.staticwent the other way: main deleted the worker and the key together, while this branch still parses it, so the block is restored.Verification
pnpm run build:check— clean.cargo test --workspace— green, including the defaults round trip and the parity fixture.cargo test -p nostream-cli --test config_cliandconfig validate— green,settings valid: 0 issues.pnpm run lint— 597 files, 0 errors (3 warnings, all pre-existing inmain's code).grepfor conflict markers across the tree — none.knexreads3.2.9in the merged tree, which is what fix(deps): bump knex to 3.2.9 for the search_path SQL injection (rewrite/main) #1025 wanted.The merge commit is
8940ddc;rewrite/mainis untouched until this is merged.