Skip to content

Run Hosted on PostgreSQL 18.6 and install pgstencil from npm - #795

Merged
nedtwigg merged 10 commits into
mainfrom
postgres-18
Sep 25, 2026
Merged

nedtwigg merged 10 commits into
mainfrom
postgres-18

Conversation

@nedtwigg

@nedtwigg nedtwigg commented Sep 25, 2026 •

Copy link
Copy Markdown
Member

What changed

  • Move Hosted's database tooling and packaged pgstencil runtime to PostgreSQL 18.6, matching Neon's production and preview projects. The backup image is postgres:18.6-alpine, so pg_dump can back up that server version.
  • Install audited pgstencil and @pgstencil/auth 0.2.1 from npm. Remove the vendored archives, root overrides, and sync script. Exempt only pgstencil from the pnpm and Renovate dependency cooldowns; group future updates.
  • Read installed package provenance during production preflight, require matching clean pgstencil commits, and check registry integrity in the lockfile. The Hosted security spec and audit prompt verify each package's npm attestation, Fulcio workflow identity and commit, and agreement with the installed provenance; they also require the commit on pgstencil main to have a passing security-audit.
  • Preserve the audit rule that cancelled pgstencil check runs are ignored while at least one success and no failure are required.

The installed 0.2.1 packages have npm provenance attestations. Their dist/provenance.json names pgstencil merge commit 1bba14e9490fc1eb35c7e7a7e2090ec36a165d00, whose security-audit passed.

Verification

  • pnpm install --frozen-lockfile
  • pnpm --filter dormouse-hosted typecheck
  • pnpm --filter dormouse-hosted test (16 deploy tests, 18 Vitest tests)
  • pnpm --filter dormouse-hosted build
  • verifyPackages() from the installed packages
  • pnpm lint:specs
  • node --test scripts/security-audit.test.mjs (49 tests)
  • node website/scripts/generate-deps.js (disclosure unchanged)

The backup script has not run against a real 18.6 server. That occurs after production provisioning and before first deployment. This PR is ready for review. Dormouse-bot approved the latest commit; CI is in progress.

nedtwigg and others added 2 commits September 24, 2026 20:46
Neon runs the production and preview databases on 18.6, and pg_dump refuses a
server newer than itself, so the backup and restore-test image moves from
postgres:17.11 to postgres:18.6. The audit prompt also learns that a stopped
manual dispatch leaves a cancelled security-audit run on a pgstencil commit
beside the real one, and that a cancelled run is not a verdict.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
pgstencil main now ships compose.yaml on postgres:18.6-alpine, so Hosted's
integration tests run on the version production runs. The 0.2.0 release prep
renames the archives; the pins, the production preflight, and the artifact
test follow. Its security-audit run on e79cc4d succeeded.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
@cloudflare-workers-and-pages

cloudflare-workers-and-pages Bot commented Sep 25, 2026 •

Copy link
Copy Markdown

Deploying mouseterm with  Cloudflare Pages  Cloudflare Pages

Latest commit: 3a2826a
Status: ✅  Deploy successful!
Preview URL: https://122cedab.mouseterm.pages.dev
Branch Preview URL: https://postgres-18.mouseterm.pages.dev

View logs

@dormouse-bot dormouse-bot left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is feedback on a draft, not a merge verdict. Mark the PR ready when you want the full review.

The audit prompt now lets a cancelled security-audit run through, but the FAIL IF rule it enforces wasn't updated. docs/specs/security-hosted.md still fails the audit when the commit's security-audit check run "is missing or not success". For a commit whose runs are [cancelled, success], like pgstencil 03de7dd, the prompt says pass and the spec it cites says fail, so the audit agent gets two rules that disagree. AGENTS.md makes the security spec the owner of FAIL IF rules, with the prompt only claiming them. The spec's rule needs the same change, for example: "…or none of that commit's security-audit check runs is success, or any has a conclusion other than success or cancelled."

"None failed" in the prompt also leaves timed_out, startup_failure, and action_required open to interpretation, and none of them is a passing verdict. The suggestion below lists the one conclusion to ignore instead of the ones to reject.

Comment thread .github/audit/hosted.md Outdated
nedtwigg and others added 2 commits September 24, 2026 20:50
The security-hosted FAIL IF still failed a commit with a cancelled duplicate
run. The prompt now ignores cancelled and requires at least one success and
no other conclusion, and the spec says the same.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
@nedtwigg nedtwigg changed the title Run Hosted's database tooling on PostgreSQL 18.6 Run Hosted on PostgreSQL 18.6 and install pgstencil from npm Sep 25, 2026

@dormouse-bot dormouse-bot left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is feedback on a draft, not a merge verdict. Mark the PR ready when you want the full review.

The pgstencil cooldown exemption contradicts a published guarantee. docs/specs/security.md lists "No newly published dependency is adopted for 24 hours, security fixes included" as a public guarantee, and it links to docs/specs/security-supply-chain.md -> "Cooldown and alerts". This PR adds minimumReleaseAgeExclude for pgstencil / @pgstencil/* in pnpm-workspace.yaml and a Renovate rule with minimumReleaseAge: null for the same packages, but neither spec was updated. The guarantee on /docs/security is now false for a runtime dependency of Hosted, and the audit's FAIL IF lines, which check only that minimumReleaseAge: 1440 and the per-manager rules exist, won't notice. If the exemption is intended, the owning spec should state it as a rule, with the 2FA-gated, audit-gated release as the rationale. The security.md row should also carry the exception, and the FAIL IF should bound the exclusion list so that no other package can be added to it without the audit failing.

The lockfile test hard-codes 0.2.1 (inline below). The Renovate group is set up to open a PR as soon as a release is approved, but every such PR will fail this test until someone edits it by hand. hosted/README.md -> "Update pgstencil" doesn't mention that step.

Comment thread hosted/server/tests/artifacts.test.ts Outdated
@nedtwigg
nedtwigg marked this pull request as ready for review September 25, 2026 07:23

@dormouse-bot dormouse-bot left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The lockfile test still fails the next in-range pgstencil release, this time through the specifier check (inline). .github/renovate.json sets rangeStrategy: "update-lockfile" for every npm dependency, and the new pgstencil rule doesn't override it. A 0.2.2 release satisfies ^0.2.1, so Renovate updates only pnpm-lock.yaml and leaves hosted/package.json at ^0.2.1. The installed version then becomes 0.2.2 and toBe("^0.2.2") fails.

Comment thread hosted/server/tests/artifacts.test.ts Outdated

@dormouse-bot dormouse-bot left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The link from the installed bytes to the audited pgstencil commit is dist/provenance.json, and whoever publishes the package writes that file. The cooldown exemption depends on "audited on their main commit". So a publish that bypassed pgstencil's release workflow could put any passing main commit in that file, and neither verifyPackages nor the audit would notice. npm already has a signed record of the same fact. Both 0.2.1 packages carry a SLSA provenance attestation from diffplug/pgstencil .github/workflows/release.yml on refs/heads/main, and its resolvedDependencies names 1bba14e9490fc1eb35c7e7a7e2090ec36a165d00, the commit dist/provenance.json claims (https://registry.npmjs.org/-/npm/v1/attestations/pgstencil@0.2.1). One option is for the audit prompt's provenance step to also require that attestation, check its workflow identity, and require the same commit. That would make the exemption rest on a signature, not on a claim inside the package.

The rewritten FAIL IF dropped the old check that no runtime import depends on a sibling source checkout (inline). docs/specs/hosted.md still states that rule, but the registry-lockfile check doesn't cover a relative import into ../pgstencil.

Comment thread docs/specs/security-hosted.md Outdated

@dormouse-bot dormouse-bot left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The new attestation check reads the builder identity from the DSSE payload, but the signer writes that payload. npm's own verification (pacote verifyAttestations, which npm audit signatures runs) checks a provenance bundle against the Fulcio/Rekor trust root and the subject digest, with no identity policy. So a bundle from any GitHub workflow verifies, and a statement in it can name diffplug/pgstencil release.yml in externalParameters.workflow and any commit in resolvedDependencies. GitHub's OIDC sets the workflow and commit in the Fulcio certificate: the SAN is the job_workflow_ref, and extension 1.3.6.1.4.1.57264.1.13 is the source-repository digest. The prompt and the FAIL IF should read identity and commit from there, with the payload required to agree.

Separately, npm audit signatures --json puts a package with no attestation in neither invalid nor missing (missing covers registry signatures only), so the prompt should also require each package to appear in verified.

Comment thread .github/audit/hosted.md Outdated
Comment thread docs/specs/security-hosted.md Outdated
@nedtwigg
nedtwigg merged commit 0b79ede into main Sep 25, 2026
8 checks passed
@nedtwigg
nedtwigg deleted the postgres-18 branch September 25, 2026 13:56
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants