diff --git a/.github/production-repositories.jsonc b/.github/production-repositories.jsonc index b2db555..925bab0 100644 --- a/.github/production-repositories.jsonc +++ b/.github/production-repositories.jsonc @@ -1,10 +1,15 @@ // Public GitHub repositories served by the production remote cache. // Each key is a cache namespace. Its endpoint is // https://voidzero-remote-cache..workers.dev/projects/ -// Each production deployment enables every listed namespace and disables every other namespace, -// overriding `pnpm operator policy`. Data in a removed namespace expires with its retention. +// A value is "owner/repo", or { "repository": "owner/repo", "pullRequests": true } to also accept +// uploads from same-repository pull_request runs that target the default branch. This interim +// opt-in lets unreviewed PR code, including Renovate branches, publish entries that default-branch +// builds restore. Read docs/e2e-plan.md#production-deployment before enabling it. +// Each production deployment enables every listed namespace, disables every other namespace, and +// sets each namespace's pull request opt-in from this list, overriding `pnpm operator policy`. +// Data in a removed namespace expires with its retention. { "rolldown": "rolldown/rolldown", "vite-plus": "voidzero-dev/vite-plus", - "void": "voidzero-dev/void", + "void": { "repository": "voidzero-dev/void", "pullRequests": true }, } diff --git a/README.md b/README.md index be4d1c8..901ba4c 100644 --- a/README.md +++ b/README.md @@ -1,6 +1,6 @@ # Vite+ remote cache for Cloudflare -This repository is Vite+'s Cloudflare implementation of a remote cache server for `vp run`, designed in the [remote cache RFC](rfcs/0001-remote-cache.md) ([PR #716](https://github.com/voidzero-dev/vite-task/pull/716)): a TypeScript Worker, primary D1 metadata, and a private R2 Standard bucket. Anyone can read an enabled namespace. Only a signed GitHub Actions token for the registered public repository's main-branch `push` job can publish. +This repository is Vite+'s Cloudflare implementation of a remote cache server for `vp run`, designed in the [remote cache RFC](rfcs/0001-remote-cache.md) ([PR #716](https://github.com/voidzero-dev/vite-task/pull/716)): a TypeScript Worker, primary D1 metadata, and a private R2 Standard bucket. Anyone can read an enabled namespace. Only a signed GitHub Actions token for the registered public repository's main-branch `push` job can publish, unless an operator opts the namespace in to [pull request uploads](#pull-request-uploads). The package contains no `vp run` client adapter. Cache keys, values, and blobs remain opaque. A successful lookup does not prove that a result is reusable; a client must validate its inputs and output archive. @@ -76,7 +76,15 @@ For a deployment on Workers Paid, explicitly select `--profile paid` when you ne pnpm operator setup --name my-public-cache --namespace docs --repo owner/repository --origin https://cache.example.com --profile paid --retention-days 30 --byte-limit 30000000000 ``` -Set GitHub Actions `id-token: write` only on the trusted publishing job. The token audience must equal the printed endpoint. No cache secret is needed in GitHub. A token with a customized `sub` can work because policy uses signed IDs, branch, visibility, and event claims. Tokens for fork repositories, tags, PRs, `pull_request_target`, or `workflow_run` are denied. +Set GitHub Actions `id-token: write` only on the trusted publishing job. The token audience must equal the printed endpoint. No cache secret is needed in GitHub. A token with a customized `sub` can work because policy uses signed IDs, branch, visibility, and event claims. Tokens for fork repositories, tags, `pull_request_target`, or `workflow_run` are denied. PR tokens are denied too, unless the namespace opts in to pull request uploads. + +### Pull request uploads + +`pull_request` tokens are denied by default. As an interim measure, an operator can opt a namespace in with `pnpm operator policy --namespace docs --pull-requests on`, or pass `--pull-requests on` to the setup that creates the namespace. `--pull-requests off` withdraws the opt-in. An opted-in namespace accepts a `pull_request` token that passes the same audience, repository/owner ID, and visibility checks as a push and also has `ref` `refs/pull//merge`, `ref_type` `branch`, and a `base_ref` equal to the registered default branch. PRs that target another branch, `pull_request_target`, `workflow_run`, tags, and workflows in fork repositories remain denied. + +The opt-in carries a risk: PR entries share the namespace with default-branch entries. Code from any same-repository PR, including Renovate dependency branches, runs before review and can publish an entry that a later default-branch build restores. That is why it is off by default. The long-term fix is to [scope pull request uploads to the pull request](https://github.com/voidzero-dev/vite-plus-remote-cache-cloudflare/issues/16). + +Fork PR workflows run in the base repository, so their tokens carry its repository and owner IDs; the server cannot tell them from same-repository PRs. The policy depends on GitHub not granting `id-token: write` to fork PR workflows. GitHub grants write permissions to them only when the repository enables **Send write tokens to workflows from pull requests**, a setting [GitHub offers for private repositories only](https://docs.github.com/en/repositories/managing-your-repositorys-settings-and-features/enabling-features-for-your-repository/managing-github-actions-settings-for-a-repository#enabling-workflows-for-forks-of-private-repositories). GitHub treats Dependabot PRs like fork PRs. ## Limits and authorization @@ -99,7 +107,7 @@ Set GitHub Actions `id-token: write` only on the trusted publishing job. The tok The `LIMITS` Wrangler variable is a JSON object with optional keys `key`, `value`, `metadata`, `fetch`, `blob`, `store`, `headers`, and `deadlineMs`. It can lower the tested ceilings. Envelope limits must accommodate their field limits and framing. Raising ceilings requires code changes and new memory, CPU, and account-limit measurements. An HTTP request without `Content-Length` reserves the full store limit. Actual bytes replace that reservation only at publication; pending, retired, and deleting generations remain charged. -GitHub authorization uses `jose`, `RS256`, the fixed issuer `https://token.actions.githubusercontent.com`, and its fixed HTTPS JWKS endpoint. The server checks exact string types for audience, repository/owner IDs, visibility, branch, ref type, and event. It requires integer `exp`, `nbf`, and `iat`, permits 30 seconds of skew for not-before/issued-at checks, accepts at most a 15-minute token lifetime, and never publishes after expiry. Token headers cannot supply signing-key URLs or embedded keys. +GitHub authorization uses `jose`, `RS256`, the fixed issuer `https://token.actions.githubusercontent.com`, and its fixed HTTPS JWKS endpoint. The server checks exact string types for audience, repository/owner IDs, visibility, branch (`ref`, or `base_ref` for an opted-in pull request), ref type, and event. It requires integer `exp`, `nbf`, and `iat`, permits 30 seconds of skew for not-before/issued-at checks, accepts at most a 15-minute token lifetime, and never publishes after expiry. Token headers cannot supply signing-key URLs or embedded keys. Tokens are limited to 16 KiB. JWKS bodies are limited to 64 KiB, requests to five seconds, cached keys to ten minutes, and refresh attempts to one per 30 seconds per isolate, including failures. Unknown key IDs cannot cause unlimited refreshes. Missing usable keys fail closed. A maintainer's policy change increments its version; publication checks that version, current scope/deployment state, token expiry, lease, bytes, and identity quotas inside the D1 transaction. @@ -112,6 +120,7 @@ pnpm operator status pnpm operator bind --namespace another-project --repo owner/another-repository pnpm operator bind --namespace docs --repo owner/repository pnpm operator policy --namespace docs --writes off +pnpm operator policy --namespace docs --pull-requests on pnpm operator policy --namespace docs --enabled off pnpm operator policy --namespace docs --enabled on --writes on pnpm operator policy --namespace docs --retention-days 30 --byte-limit 30000000000 @@ -125,7 +134,7 @@ pnpm operator upgrade `status` reports policy, charged bytes, entry/association counts, generation states, cleanup eligibility, and actual D1 storage. It warns at 400 MB. Set provider alerts at 80% of request, CPU, R2 operation/storage, and D1 read/write/storage allowances. Monitor cleanup delay and pending bytes, and pause writes before a backlog reaches the storage ceiling. Public requests and other services in the same account can exhaust allowances even when this cache's byte budget is respected. -Sampled structured logs include request ID, namespace, operation, exact/fallback/miss/error outcome, HTTP status, request/response sizes, duration, D1 rows/query latency/storage, and R2 operation attempts. Verified writes add signed repository ID, workflow ref, run ID/attempt, and commit SHA. They exclude tokens, keys, values, blobs, and database errors. `LOG_SAMPLE_RATE` defaults to `0.1` and applies to failures too. Provider transfer metrics account for disconnected downloads; logged response size describes the selected response. A server exact hit is distinct from a client-validated cache hit. +Sampled structured logs include request ID, namespace, operation, exact/fallback/miss/error outcome, HTTP status, request/response sizes, duration, D1 rows/query latency/storage, and R2 operation attempts. Verified writes add signed repository ID, workflow ref, run ID/attempt, commit SHA, event name, and a pull request's head branch. They exclude tokens, keys, values, blobs, and database errors. `LOG_SAMPLE_RATE` defaults to `0.1` and applies to failures too. Provider transfer metrics account for disconnected downloads; logged response size describes the selected response. A server exact hit is distinct from a client-validated cache hit. ### Cleanup and withdrawal diff --git a/docs/e2e-plan.md b/docs/e2e-plan.md index 648558e..f2aa664 100644 --- a/docs/e2e-plan.md +++ b/docs/e2e-plan.md @@ -72,7 +72,7 @@ GitHub can suppress `pull_request` workflows when a PR has merge conflicts. Reso ## Authorization and test levels -The production write policy requires a real GitHub token with `event_name=push`, the registered repository and owner IDs, its configured main branch, and the exact namespace audience. A PR token or a `workflow_dispatch` token cannot pass this policy. GitHub signs the event claim; the test runner cannot change it. +The production write policy requires a real GitHub token with `event_name=push`, the registered repository and owner IDs, its configured main branch, and the exact namespace audience. A `workflow_dispatch` token cannot pass this policy, and neither can a PR token unless its namespace opts in to [pull request uploads](../README.md#pull-request-uploads). Staging keeps that opt-in off, so PR runs keep checking that PR tokens are rejected. GitHub signs the event claim; the test runner cannot change it. | Level | Trigger | What it proves | | ---------------------------- | ------------------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------- | @@ -177,6 +177,8 @@ The workflow repeats `pnpm check` and `pnpm smoke`, then runs `pnpm ci:deploy pr To bind another repository, add `"namespace": "owner/repository"` to `.github/production-repositories.jsonc`, merge the change, and run the workflow. Only public repositories can publish. The list decides which namespaces serve reads and accept uploads: each deployment enables every listed namespace and disables every other namespace, so a `pnpm operator policy` change lasts only until the next deployment. To withdraw a repository, remove its entry and run the workflow. For an immediate stop, also run `pnpm operator policy --namespace --enabled off`. Data in a disabled namespace expires with its retention; `pnpm operator purge` deletes it sooner. Deployments never change the cache-wide `pnpm operator deployment` switches or move a namespace to another repository. To roll back, re-run an earlier successful production run. A re-run deploys its original commit. +An entry can also be an object. `{ "repository": "owner/repository", "pullRequests": true }` also accepts uploads from `pull_request` runs in that repository whose PR targets its default branch; `"owner/repository"` is short for `"pullRequests": false`. Each deployment also sets every namespace's pull request opt-in from the list, overriding `pnpm operator policy --pull-requests`. This opt-in is interim, and only `void` has it. Code from any same-repository PR, including Renovate dependency branches, runs before review and can publish an entry that a later default-branch build restores. Enable it only for a repository whose maintainers accept that risk. The long-term fix is to [scope pull request uploads to the pull request](https://github.com/voidzero-dev/vite-plus-remote-cache-cloudflare/issues/16). Fork PRs stay out only because GitHub does not grant their workflows `id-token: write`; see [Pull request uploads](../README.md#pull-request-uploads). + Production uses the same `CLOUDFLARE_ACCOUNT_ID` and `CLOUDFLARE_API_TOKEN` secrets as staging. Secrets in the `production` environment take precedence, so a production-only token can replace them without workflow changes. Internal PR staging runs receive the repository secrets; while both environments share one token, anyone who can push a branch here can change production resources. Add required reviewers to the `production` environment to approve each deployment. Operator commands read `wrangler.operator.json`, which the workflow does not keep. To recreate it, check out the deployed commit and repeat `pnpm operator setup` with the Worker name from `scripts/ci/production.ts` and an entry from `.github/production-repositories.jsonc`. Setup redeploys that checkout. @@ -191,7 +193,7 @@ Before a production release, record the commit, workflow URL, Cloudflare plan, r | Maximum payloads and real Cron | Run two concurrent authorized stores with 4 MiB values and 64 MiB blobs; verify their digests. Expire a fixture and observe scheduled D1/R2 deletion and accounting release | | Upload interruption | Disconnect a real authorized multipart request after its first R2 part. Verify unchanged mappings, retained reservation, upload abortion, and eventual byte release | | Lease/token expiry and policy race | Delay a staging upload across expiry or change policy before publication. Neither mapping changes; old data remains readable | -| Real hostile workflow identities | Use controlled fork, tag, PR, `pull_request_target`, and `workflow_run` jobs. All writes fail; public reads still work | +| Real hostile workflow identities | Use controlled fork, tag, PR, `pull_request_target`, and `workflow_run` jobs against a namespace without the pull request opt-in. All writes fail; public reads still work | | Rate limits | Send a bounded burst to the staging namespace. Check `429`, `Retry-After`, recovery, and a finite identity set for unknown routes | | Retention and lifecycle | Observe expiry, replacement grace, unfinished multipart abortion, and lifecycle deletion over actual retention windows | | Migration and rollback | Upgrade a populated staging database, verify old entries, and restore a compatible previous Worker version. Never roll code back across an incompatible schema change | diff --git a/docs/self-hosting.md b/docs/self-hosting.md index 699dc35..57c3076 100644 --- a/docs/self-hosting.md +++ b/docs/self-hosting.md @@ -1,6 +1,6 @@ # Deploy a remote cache for your GitHub repository -This guide deploys the service in your Cloudflare account and gives your repository its own cache endpoint. You can bind any **public repository on GitHub.com**. Setup reads its default branch and permits uploads from GitHub Actions `push` jobs on that branch. All cache reads are public. Private repositories and other Git providers are not supported. +This guide deploys the service in your Cloudflare account and gives your repository its own cache endpoint. You can bind any **public repository on GitHub.com**. Setup reads its default branch and permits uploads from GitHub Actions `push` jobs on that branch. You can also [allow pull request uploads](#allow-pull-request-uploads), with the risk described there. All cache reads are public. Private repositories and other Git providers are not supported. **Current status:** the server and operator CLI work in this source tree. The `vp run` remote-cache client adapter is not implemented here. You can deploy and verify the HTTP service now. Automatic task uploads and reuse require a compatible client; the proposed Vite+ settings are described separately below. @@ -153,7 +153,7 @@ Inspect the deployed repository binding and storage state: pnpm operator status ``` -In `scopes`, confirm that `scope_id` is `web-app`, `repository` is your repository, `branch` matches its default branch, and `endpoint` matches the printed URL. `enabled` and `writes_enabled` should both be `1` for a new namespace. The deployment-level switches must also be enabled. +In `scopes`, confirm that `scope_id` is `web-app`, `repository` is your repository, `branch` matches its default branch, and `endpoint` matches the printed URL. `enabled` and `writes_enabled` should both be `1` for a new namespace, and `pull_requests` should be `0` unless you [allowed pull request uploads](#allow-pull-request-uploads). The deployment-level switches must also be enabled. Check that an anonymous upload is rejected: @@ -169,7 +169,7 @@ An empty cache also returns `404` for a valid lookup until an authorized client A compatible client needs two settings: the complete namespace endpoint and whether it can upload. Developer machines and PR jobs can read without credentials. For uploads, the job must run in the bound public repository on a `push` to its registered default branch. Grant that job `id-token: write`; GitHub documents this [OIDC permission and custom audiences](https://docs.github.com/en/actions/reference/security/oidc). -The client requests a GitHub OIDC token whose audience is the endpoint, then sends the token as `Authorization: Bearer ` to `/store`. `GITHUB_TOKEN`, personal access tokens, and Cloudflare API tokens are not accepted as cache upload credentials. PR, tag, and `workflow_dispatch` tokens cannot upload. A default branch named `master` works when that is the branch saved by setup. +The client requests a GitHub OIDC token whose audience is the endpoint, then sends the token as `Authorization: Bearer ` to `/store`. `GITHUB_TOKEN`, personal access tokens, and Cloudflare API tokens are not accepted as cache upload credentials. Tag and `workflow_dispatch` tokens cannot upload. PR tokens cannot upload unless the namespace [allows pull request uploads](#allow-pull-request-uploads). A default branch named `master` works when that is the branch saved by setup. ### Proposed Vite+ configuration — client implementation required @@ -198,7 +198,7 @@ env: VP_REMOTE_CACHE: read-write ``` -Keep the job's checkout, dependency installation, and `vp run` build steps. For PR builds, use `VP_REMOTE_CACHE: read` and omit `id-token: write`. The RFC also proposes `VP_REMOTE_CACHE: off` to disable remote caching. The URL is public and can be stored as a GitHub repository variable. No Cloudflare secrets or GitHub environment are required for application cache access. +Keep the job's checkout, dependency installation, and `vp run` build steps. For PR builds, use `VP_REMOTE_CACHE: read` and omit `id-token: write`, unless the namespace [allows pull request uploads](#allow-pull-request-uploads). The RFC also proposes `VP_REMOTE_CACHE: off` to disable remote caching. The URL is public and can be stored as a GitHub repository variable. No Cloudflare secrets or GitHub environment are required for application cache access. ## Add or update repository bindings @@ -227,6 +227,20 @@ pnpm operator policy --namespace web-app --enabled off Making the GitHub repository private does not withdraw previously published cache data. Use the operator to disable or purge the namespace. The [operations reference](../README.md#operations) covers retention changes, upgrades, backups, purge, and teardown. +## Allow pull request uploads + +By default, only `push` jobs on the default branch can upload. As an interim measure, you can also accept uploads from `pull_request` runs in the bound repository whose PR targets that branch: + +```sh +pnpm operator policy --namespace web-app --pull-requests on +``` + +For a new namespace, you can instead add `--pull-requests on` to its `setup` command. `--pull-requests off` withdraws the opt-in, and uploads already in progress then fail to publish. Same-repository PR jobs can then use the upload settings above, including `id-token: write`. PRs that target other branches, `pull_request_target`, `workflow_run`, tags, and workflows in fork repositories stay denied. The [authorization reference](../README.md#pull-request-uploads) lists the signed claims the server checks. + +Understand the risk before enabling it. PR entries share the namespace with default-branch entries. Code from any PR branch in your repository, including Renovate dependency branches, runs before review and can publish an entry that a later default-branch build restores. That is why the opt-in is off by default. The long-term fix, [scoping pull request uploads to the pull request](https://github.com/voidzero-dev/vite-plus-remote-cache-cloudflare/issues/16), is not implemented yet. + +Fork PR runs carry your repository's IDs, so the server cannot tell them from same-repository PR runs. It relies on GitHub not granting `id-token: write` to fork PR workflows. GitHub grants fork PR workflows write permissions only when **Send write tokens to workflows from pull requests** is enabled, a setting [it offers for private repositories only](https://docs.github.com/en/repositories/managing-your-repositorys-settings-and-features/enabling-features-for-your-repository/managing-github-actions-settings-for-a-repository#enabling-workflows-for-forks-of-private-repositories). + ## Troubleshooting | Symptom | Check | @@ -235,7 +249,7 @@ Making the GitHub repository private does not withdraw previously published cach | Cloudflare API returns `403` | Token permissions, account scope, and R2 activation | | Namespace requests return `404` | Exact endpoint, namespace, and scope/deployment switches; fetch misses are also `404` | | `/store` returns `401` | Missing, invalid, or expired GitHub OIDC token | -| `/store` returns `403` | Token audience, repository/owner IDs, registered branch, `push` event, and write switch | +| `/store` returns `403` | Token audience, repository/owner IDs, registered branch, `push` event or PR opt-in, and write switch | | `/store` returns `503` | Quotas, concurrent uploads, storage availability, and policy changes; inspect `pnpm operator status` | | `vp run` does not contact the service | This source tree has no remote-cache client adapter; the RFC settings alone cannot add it | diff --git a/migrations/0002_pull_requests.sql b/migrations/0002_pull_requests.sql new file mode 100644 index 0000000..86418a2 --- /dev/null +++ b/migrations/0002_pull_requests.sql @@ -0,0 +1,12 @@ +-- Per-namespace opt-in for uploads from pull_request runs whose base is the registered branch. +-- Off by default: such uploads run unreviewed PR code, and default-branch builds restore their entries. +ALTER TABLE scopes ADD COLUMN pull_requests INTEGER NOT NULL DEFAULT 0 CHECK (pull_requests IN (0, 1)); + +-- Turning the opt-in off must also invalidate uploads that captured the previous policy. +DROP TRIGGER changed_policy; +CREATE TRIGGER changed_policy AFTER UPDATE OF endpoint, repository_id, repository_owner_id, branch, + enabled, writes_enabled, pull_requests, retention_seconds, byte_limit, entry_limit, association_limit ON scopes +WHEN NEW.policy_version = OLD.policy_version +BEGIN + UPDATE scopes SET policy_version = OLD.policy_version + 1 WHERE scope_id = NEW.scope_id; +END; diff --git a/rfcs/0001-remote-cache.md b/rfcs/0001-remote-cache.md index 4165fca..6fba25a 100644 --- a/rfcs/0001-remote-cache.md +++ b/rfcs/0001-remote-cache.md @@ -25,7 +25,7 @@ PR #713 defines the HTTP contract. This RFC defines the Cloudflare storage, auth | Data | CBOR envelope; opaque binary keys and values; optional opaque blob | | Lookup | Exact key first, then one secondary-key association | | Store | Replace the entry and secondary association together after object storage succeeds | -| Access control | Anonymous reads; GitHub OIDC writes restricted to the registered repository and main-branch push events | +| Access control | Anonymous reads; GitHub OIDC writes from the registered repository's main-branch pushes and, by interim opt-in, its PRs to main | | Transfer | One multipart HTTP store request; internal R2 multipart upload for larger blobs | | Client mode | `--remote-cache` or `VP_REMOTE_CACHE` selects `off`, `read`, or `read-write`; defaults to `read` with an endpoint and `off` without one | | Defaults | Seven-day retention, 8 GB total R2 budget, 64 MiB maximum blob | @@ -180,7 +180,7 @@ Use `VP_REMOTE_CACHE_URL` to override the endpoint on a host. Without an endpoin Task-level `remoteCache: false` excludes remote reads and uploads but retains local caching. `cache: false`, `--no-cache`, and tool-requested cache disabling also prevent uploads. The remote cache mode does not override these exclusions. -Version 1 uploads require GitHub Actions OIDC authorization for a main-branch push job. Enable uploads in that job with `VP_REMOTE_CACHE=read-write` or `--remote-cache=read-write`. +Version 1 uploads require GitHub Actions OIDC authorization for a main-branch push job, or, with the [interim pull request opt-in](#interim-pull-request-opt-in), a same-repository PR job that targets the main branch. Enable uploads in that job with `VP_REMOTE_CACHE=read-write` or `--remote-cache=read-write`. ### Upload lifecycle @@ -248,7 +248,7 @@ These conditions use [GitHub's documented claims](https://docs.github.com/en/act GitHub supports different subject formats. Repository IDs and branch/event claims avoid dependence on one text format for `sub`. A write grant for an organization or repository does not replace the complete set of checks. -A `pull_request_target` job can run in the base repository's default-branch context. Thus, `ref=refs/heads/main` alone cannot authorize writes. Version 1 denies `pull_request`, `pull_request_target`, `workflow_run`, tags, branches other than main, and all other event types. +A `pull_request_target` job can run in the base repository's default-branch context. Thus, `ref=refs/heads/main` alone cannot authorize writes. Version 1 denies `pull_request` (except under the interim opt-in below), `pull_request_target`, `workflow_run`, tags, branches other than main, and all other event types. Fork repositories have different IDs. They cannot write to the upstream namespace, even from their own `main` branch. Reusable workflows must have matching repository, branch, and event claims for the caller. The called workflow's identity alone grants no permission. See GitHub's [workflow event behavior](https://docs.github.com/en/actions/reference/workflows-and-actions/events-that-trigger-workflows#pull_request_target). @@ -256,6 +256,7 @@ Fork repositories have different IDs. They cannot write to the upstream namespac | --------------------------------------------------------------------------------- | ----------------------------- | ----------------------------- | ------------- | | Anonymous developer or fork contributor | Allow | Allow | `401` | | Registered repository, public, main-branch push, valid audience/token | Allow | Allow | Allow | +| Same-repository PR that targets main, namespace opted in, valid audience/token | Allow | Allow | Allow | | Valid GitHub token with wrong repo, owner, visibility, branch, event, or audience | Allow | Allow | `403` | | Invalid, expired, or forged token | Allow without using the token | Allow without using the token | `401` | @@ -263,6 +264,12 @@ A local command cannot obtain write permission through environment variables alo This separation prevents results from different projects from mixing. All enabled version 1 namespaces remain public. Keep the R2 bucket private so reads pass through Worker routing, retention checks, and budgets. Apply the same store verifier to every exposed route and alias. +#### Interim pull request opt-in + +An operator can opt a namespace in to `pull_request` tokens. The opt-in is off by default and stored with the namespace policy; changing it invalidates pending uploads like any other policy change. For an opted-in namespace, a token with `event_name` `pull_request` passes when the issuer, audience, repository, owner, visibility, and time checks above pass, `ref` is `refs/pull//merge`, `ref_type` is `branch`, and `base_ref` is the configured main branch's short name, such as `main`. GitHub signs these values for a PR's merge-branch run; see its [OIDC claims](https://docs.github.com/en/actions/reference/security/oidc) and [`GITHUB_REF` for `pull_request`](https://docs.github.com/en/actions/reference/workflows-and-actions/events-that-trigger-workflows#pull_request). PRs that target other branches, `pull_request_target`, `workflow_run`, tags, and workflows in fork repositories remain denied. + +This exception weakens the publication trust model below. PR code, including dependency-update branches such as Renovate's, runs before review and can publish into the namespace that main-branch builds restore from. Fork PR runs carry the base repository's IDs, so only GitHub's refusal to grant `id-token: write` to fork PR workflows keeps them out. The long-term design gives pull request uploads their own scope ([issue #16](https://github.com/voidzero-dev/vite-plus-remote-cache-cloudflare/issues/16)). + ### Publication trust and revocation A GitHub token proves the job's identity. It does not prove that uploaded bytes match the commit or contain no secrets. The trusted publishing workflow builds the commit that triggered the main-branch job. It selects only results intended for public release. @@ -547,7 +554,7 @@ jobs: DOCS_SITE_ORIGIN: ${{ vars.DOCS_SITE_ORIGIN }} ``` -The Worker checks signed repository, branch, and event claims. PR workflows use the default `read` mode with the same command and endpoint, and omit `id-token: write`. +The Worker checks signed repository, branch, and event claims. PR workflows use the default `read` mode with the same command and endpoint, and omit `id-token: write`, unless the namespace has the [interim pull request opt-in](#interim-pull-request-opt-in). A workflow that handles both main-branch pushes and PRs can select the mode once for all run commands: diff --git a/scripts/ci.ts b/scripts/ci.ts index 0a845cc..0db1a31 100644 --- a/scripts/ci.ts +++ b/scripts/ci.ts @@ -199,11 +199,12 @@ async function deploy(settings: Settings): Promise { [scope, `${settings.origin}/projects/${scope}`], ); // Recover policy changes left by an interrupted e2e run. These resources belong to CI only. + // PR runs check that pull request tokens cannot upload, so keep that opt-in off. await query(io, config, 'UPDATE deployment SET enabled = 1, writes_enabled = 1'); await query( io, config, - `UPDATE scopes SET enabled = 1, writes_enabled = 1, + `UPDATE scopes SET enabled = 1, writes_enabled = 1, pull_requests = 0, byte_limit = 2000000000, entry_limit = 1000, association_limit = 2000`, ); }, diff --git a/scripts/ci/production.ts b/scripts/ci/production.ts index 5b041b7..ddaa314 100644 --- a/scripts/ci/production.ts +++ b/scripts/ci/production.ts @@ -21,22 +21,41 @@ export const production: ProductionConfig = { // Add or remove production repositories in this file. const repositoriesFile = new URL('../../.github/production-repositories.jsonc', import.meta.url); -export function parseRepositories(text: string): Record { +export interface ProductionRepository { + repository: string; + // Also accept uploads from same-repository `pull_request` runs that target the default branch. + pullRequests: boolean; +} + +export function parseRepositories(text: string): Record { const errors: ParseError[] = []; const value: unknown = parse(text, errors, { allowTrailingComma: true }); if (errors.length || !value || typeof value !== 'object' || Array.isArray(value)) - throw new Error('Production repositories must be a JSONC object of namespace to owner/repo'); + throw new Error('Production repositories must be a JSONC object keyed by namespace'); const entries = Object.entries(value); if (!entries.length || entries.length > 100) throw new Error('List between 1 and 100 production repositories'); - for (const [namespace, repository] of entries) + const repositories: Record = {}; + for (const [namespace, entry] of entries) { + // "owner/repo" is shorthand for { "repository": "owner/repo", "pullRequests": false }. + const fields: Record = + typeof entry === 'string' + ? { repository: entry } + : entry && typeof entry === 'object' && !Array.isArray(entry) + ? entry + : {}; + const { repository, pullRequests = false, ...unknown } = fields; if ( !/^[a-z0-9][a-z0-9-]{0,62}$/.test(namespace) || typeof repository !== 'string' || - !/^[A-Za-z0-9_.-]+\/[A-Za-z0-9_.-]+$/.test(repository) + !/^[A-Za-z0-9_.-]+\/[A-Za-z0-9_.-]+$/.test(repository) || + typeof pullRequests !== 'boolean' || + Object.keys(unknown).length ) throw new Error(`Invalid production repository entry: ${namespace}`); - return Object.fromEntries(entries) as Record; + repositories[namespace] = { repository, pullRequests }; + } + return repositories; } export function productionTarget( @@ -52,15 +71,20 @@ export function productionTarget( if (context.ref !== config.ref) throw new Error(`Production deploys only from ${config.ref}`); if (/-(ci|staging)$/.test(config.name)) throw new Error('Production cannot use CI staging resources'); - const bindings = parseRepositories(repositories); - const listed = JSON.stringify(Object.keys(bindings)); + const entries = Object.entries(parseRepositories(repositories)); + const listed = JSON.stringify(entries.map(([namespace]) => namespace)); + const pullRequests = JSON.stringify( + entries.filter(([, entry]) => entry.pullRequests).map(([namespace]) => namespace), + ); return { name: config.name, profile: config.profile, - bindings, - // The list decides which namespaces serve reads and accept uploads, overriding - // `pnpm operator policy`. Only changed rows are updated: the changed_policy trigger bumps - // policy_version on any switch update, which stops in-flight uploads from publishing. + bindings: Object.fromEntries( + entries.map(([namespace, entry]) => [namespace, entry.repository]), + ), + // The list decides which namespaces serve reads, accept uploads, and accept pull request + // uploads, overriding `pnpm operator policy`. Only changed rows are updated: the changed_policy + // trigger bumps policy_version on any switch update, which stops in-flight uploads from publishing. async prepare(deployed, io) { await query( io, @@ -81,6 +105,18 @@ export function productionTarget( io.print( `Disabled namespace ${String(scope['scope_id'])}: it is not in the repository list`, ); + const switched = await query( + io, + deployed, + `UPDATE scopes SET pull_requests = (scope_id IN (SELECT value FROM json_each(?))) + WHERE pull_requests != (scope_id IN (SELECT value FROM json_each(?))) + RETURNING scope_id, pull_requests`, + [pullRequests, pullRequests], + ); + for (const scope of switched) + io.print( + `${scope['pull_requests'] === 1 ? 'Enabled' : 'Disabled'} pull request uploads for namespace ${String(scope['scope_id'])}`, + ); }, }; } diff --git a/scripts/operator.ts b/scripts/operator.ts index b14aec2..f9be6c0 100644 --- a/scripts/operator.ts +++ b/scripts/operator.ts @@ -228,6 +228,7 @@ export async function runOperator( 'association-limit': { type: 'string' }, enabled: { type: 'string' }, writes: { type: 'string' }, + 'pull-requests': { type: 'string' }, confirm: { type: 'string' }, help: { type: 'boolean' }, }, @@ -257,6 +258,7 @@ export async function runOperator( throw new Error('Retention cannot exceed 365 days'); toggle(values.enabled); toggle(values.writes); + toggle(values['pull-requests']); if (command === 'setup') { const name = identifier(values.name); if (name.length < 3 || name.endsWith('-')) @@ -328,8 +330,8 @@ export async function runOperator( await query( io, config, - `INSERT INTO scopes (scope_id, endpoint, repository, repository_id, repository_owner_id, branch, retention_seconds, byte_limit, entry_limit, association_limit) - VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?) ON CONFLICT(scope_id) DO NOTHING`, + `INSERT INTO scopes (scope_id, endpoint, repository, repository_id, repository_owner_id, branch, pull_requests, retention_seconds, byte_limit, entry_limit, association_limit) + VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?) ON CONFLICT(scope_id) DO NOTHING`, [ namespace, `${origin.origin}/projects/${namespace}`, @@ -337,6 +339,7 @@ export async function runOperator( repo.id, repo.owner, repo.branch, + toggle(values['pull-requests']) ?? 0, positive(values['retention-days'], 7) * 86400, positive(values['byte-limit'], 8000000000), positive(values['entry-limit'], 20000), @@ -516,11 +519,12 @@ export async function runOperator( io, config, `UPDATE scopes SET enabled = coalesce(?, enabled), writes_enabled = coalesce(?, writes_enabled), - retention_seconds = coalesce(?, retention_seconds), byte_limit = coalesce(?, byte_limit), entry_limit = coalesce(?, entry_limit), + pull_requests = coalesce(?, pull_requests), retention_seconds = coalesce(?, retention_seconds), byte_limit = coalesce(?, byte_limit), entry_limit = coalesce(?, entry_limit), association_limit = coalesce(?, association_limit), policy_version = policy_version + 1 WHERE scope_id = ?`, [ toggle(values.enabled), toggle(values.writes), + toggle(values['pull-requests']), values['retention-days'] ? positive(values['retention-days'], 7) * 86400 : null, optionalPositive(values['byte-limit']), optionalPositive(values['entry-limit']), diff --git a/src/auth.ts b/src/auth.ts index b4dd1fb..b87e07a 100644 --- a/src/auth.ts +++ b/src/auth.ts @@ -50,6 +50,8 @@ export interface WriteIdentity { run_id?: string; run_attempt?: string; sha?: string; + event_name?: string; + head_ref?: string; } export async function authorize( @@ -121,22 +123,43 @@ export async function authorize( ) { throw new HttpError(401, 'invalid_token'); } + // A `push` must update the registered branch. A namespace that opts in also accepts + // `pull_request` runs whose PR targets that branch. For those, GitHub signs `ref` + // `refs/pull//merge`, `ref_type` `branch`, and the target branch's short name (`main`) + // in `base_ref`. The `closed` run of a merged PR has the base branch in `ref` and is denied. + // Fork PR runs carry the base repository's IDs too; only GitHub withholding `id-token: write` + // from them keeps them out. + const event = + payload.event_name === 'push' + ? payload.ref === scope.branch + : payload.event_name === 'pull_request' && + scope.pull_requests === 1 && + typeof payload.ref === 'string' && + /^refs\/pull\/[1-9][0-9]*\/merge$/.test(payload.ref) && + typeof payload.base_ref === 'string' && + `refs/heads/${payload.base_ref}` === scope.branch; if ( !scope.writes_enabled || payload.aud !== scope.endpoint || payload.repository_id !== scope.repository_id || payload.repository_owner_id !== scope.repository_owner_id || payload.repository_visibility !== 'public' || - payload.ref !== scope.branch || payload.ref_type !== 'branch' || - payload.event_name !== 'push' + !event ) { throw new HttpError(403, 'write_policy'); } const identity: WriteIdentity = { exp, repository_id: scope.repository_id }; - for (const name of ['workflow_ref', 'run_id', 'run_attempt', 'sha'] as const) { + for (const name of [ + 'workflow_ref', + 'run_id', + 'run_attempt', + 'sha', + 'event_name', + 'head_ref', + ] as const) { const value = payload[name]; - if (typeof value === 'string' && value.length <= 512) identity[name] = value; + if (typeof value === 'string' && value && value.length <= 512) identity[name] = value; } return identity; } diff --git a/src/database.ts b/src/database.ts index 104ce01..e224e10 100644 --- a/src/database.ts +++ b/src/database.ts @@ -10,6 +10,8 @@ export interface Scope { branch: string; policy_version: number; writes_enabled: number; + // 1 when `pull_request` runs whose PR targets `branch` may also publish. + pull_requests: number; } export interface Generation { generation_id: string; diff --git a/src/index.ts b/src/index.ts index ffcd1c4..5b7c0cb 100644 --- a/src/index.ts +++ b/src/index.ts @@ -116,6 +116,8 @@ export default { run_id: identity.run_id, run_attempt: identity.run_attempt, sha: identity.sha, + event_name: identity.event_name, + head_ref: identity.head_ref, } : {}), }), diff --git a/test/ci.test.ts b/test/ci.test.ts index 50c2e86..fd6832b 100644 --- a/test/ci.test.ts +++ b/test/ci.test.ts @@ -188,8 +188,18 @@ test('production deploys only manual default-branch runs from this repository', test('production repository lists map namespaces to public owner/repo names', () => { assert.deepEqual( - parseRepositories('// comment\n{ "one": "acme/one", "two-x": "Acme/two.js", }'), - { one: 'acme/one', 'two-x': 'Acme/two.js' }, + parseRepositories( + '// comment\n{ "one": "acme/one", "two-x": "Acme/two.js", ' + + '"three": { "repository": "acme/three", "pullRequests": true }, ' + + '"four": { "repository": "acme/four", "pullRequests": false }, "five": { "repository": "acme/five" }, }', + ), + { + one: { repository: 'acme/one', pullRequests: false }, + 'two-x': { repository: 'Acme/two.js', pullRequests: false }, + three: { repository: 'acme/three', pullRequests: true }, + four: { repository: 'acme/four', pullRequests: false }, + five: { repository: 'acme/five', pullRequests: false }, + }, ); for (const text of [ '', @@ -199,12 +209,19 @@ test('production repository lists map namespaces to public owner/repo names', () '{ "-one": "acme/one" }', '{ "one": "https://github.com/acme/one" }', '{ "one": 1 }', + '{ "one": null }', + '{ "one": ["acme/one"] }', + '{ "one": {} }', + '{ "one": { "repository": "https://github.com/acme/one" } }', + '{ "one": { "repository": "acme/one", "pullRequests": "true" } }', + '{ "one": { "repository": "acme/one", "pullRequest": true } }', '{ "one": "acme/one" ', ]) assert.throws(() => parseRepositories(text), /production repositor/i); }); -test('production deployments enable listed namespaces and disable all others', async () => { +// The production prepare step against the harness database, with every other operator call refused. +async function productionDatabase() { const h = await harness(); const config = await readTemplate(); config.d1_databases[0]!.database_id = database; @@ -230,6 +247,13 @@ test('production deployments enable listed namespaces and disable all others', a lifecycle: unexpected, print: (message) => printed.push(message), }; + const prepare = (repositories: string) => + productionTarget(dispatch, production, repositories).prepare!(config, io); + return { h, printed, prepare }; +} + +test('production deployments enable listed namespaces and disable all others', async () => { + const { h, printed, prepare } = await productionDatabase(); const policies = async () => ( await h.db @@ -238,7 +262,6 @@ test('production deployments enable listed namespaces and disable all others', a ) .all() ).results; - const target = productionTarget(dispatch, production, '{ "test": "owner/repo" }'); const expected = [ { scope_id: 'other', enabled: 0, writes_enabled: 0, policy_version: 2 }, { scope_id: 'test', enabled: 1, writes_enabled: 1, policy_version: 3 }, @@ -247,15 +270,58 @@ test('production deployments enable listed namespaces and disable all others', a // A manual upload pause on a listed namespace lasts only until the next deployment. // The pause and its reversal each bump policy_version. await h.db.prepare("UPDATE scopes SET writes_enabled = 0 WHERE scope_id = 'test'").run(); - await target.prepare!(config, io); + await prepare('{ "test": "owner/repo" }'); assert.deepEqual(await policies(), expected); assert.deepEqual(printed, ['Disabled namespace other: it is not in the repository list']); // A deployment that changes no switch leaves in-flight uploads alone. printed.length = 0; - await target.prepare!(config, io); + await prepare('{ "test": "owner/repo" }'); + assert.deepEqual(await policies(), expected); + assert.deepEqual(printed, []); + } finally { + await h.close(); + } +}); + +test('production deployments accept pull request uploads only for opted-in namespaces', async () => { + const { h, printed, prepare } = await productionDatabase(); + const policies = async () => + ( + await h.db + .prepare('SELECT scope_id, pull_requests, policy_version FROM scopes ORDER BY scope_id') + .all() + ).results; + const list = + '{ "test": { "repository": "owner/repo", "pullRequests": true }, "other": "owner/repo" }'; + const expected = [ + { scope_id: 'other', pull_requests: 0, policy_version: 3 }, + { scope_id: 'test', pull_requests: 1, policy_version: 2 }, + ]; + try { + // A manual opt-in that the list doesn't make lasts only until the next deployment. + // Each switch bumps policy_version, which stops uploads admitted under the old policy. + await h.db.prepare("UPDATE scopes SET pull_requests = 1 WHERE scope_id = 'other'").run(); + await prepare(list); + assert.deepEqual(await policies(), expected); + assert.deepEqual(printed.sort(), [ + 'Disabled pull request uploads for namespace other', + 'Enabled pull request uploads for namespace test', + ]); + + printed.length = 0; + await prepare(list); assert.deepEqual(await policies(), expected); assert.deepEqual(printed, []); + + // Switching back to the string form withdraws the opt-in. + printed.length = 0; + await prepare('{ "test": "owner/repo", "other": "owner/repo" }'); + assert.deepEqual(await policies(), [ + { scope_id: 'other', pull_requests: 0, policy_version: 3 }, + { scope_id: 'test', pull_requests: 0, policy_version: 3 }, + ]); + assert.deepEqual(printed, ['Disabled pull request uploads for namespace test']); } finally { await h.close(); } diff --git a/test/helpers.ts b/test/helpers.ts index 537c7cd..a4ae97b 100644 --- a/test/helpers.ts +++ b/test/helpers.ts @@ -1,4 +1,4 @@ -import { readFile } from 'node:fs/promises'; +import { readdir, readFile } from 'node:fs/promises'; import { fileURLToPath, URL } from 'node:url'; import { build } from 'esbuild'; import { Miniflare, convertV4MiniflareOptions } from 'miniflare'; @@ -77,14 +77,16 @@ export async function harness( try { const db = await mf.getD1Database('INDEX'); const bucket = await mf.getR2Bucket('ARTIFACTS'); - const migration = await readFile( - new URL('../migrations/0001_cache.sql', import.meta.url), - 'utf8', - ); - // Preserve complete trigger bodies. Each prepared statement is a real D1 migration statement. - const statements = migration.match( - /CREATE TRIGGER[\s\S]*?\nEND;|(?:CREATE TABLE|CREATE (?:UNIQUE )?INDEX|INSERT INTO)[\s\S]*?;/g, - )!; + // Apply every migration in order, like `wrangler d1 migrations apply`. Preserve complete + // trigger bodies. Each prepared statement is a real D1 migration statement. + const migrations = new URL('../migrations/', import.meta.url); + const statements: string[] = []; + for (const name of (await readdir(migrations)).filter((file) => file.endsWith('.sql')).sort()) + statements.push( + ...(await readFile(new URL(name, migrations), 'utf8')).match( + /CREATE TRIGGER[\s\S]*?\nEND;|(?:CREATE TABLE|CREATE (?:UNIQUE )?INDEX|INSERT INTO|ALTER TABLE|DROP TRIGGER)[\s\S]*?;/g, + )!, + ); const migrate = () => db.batch(statements.map((sql) => db.prepare(sql))); if (options.initializeDatabase !== false) { await migrate(); diff --git a/test/operator.test.ts b/test/operator.test.ts index 201c1f4..05d0425 100644 --- a/test/operator.test.ts +++ b/test/operator.test.ts @@ -110,7 +110,16 @@ test('operator setup validates before writes and preserves withdrawal on retries const row = await h.db.prepare("SELECT * FROM scopes WHERE scope_id = 'new'").first(); assert.equal(row!.enabled, 0); assert.equal(row!.writes_enabled, 0); + assert.equal(row!.pull_requests, 0); assert.equal(row!.policy_version, 2); + await assert.rejects( + runOperator(['policy', '--namespace', 'new', '--pull-requests', 'yes'], io), + /on or off/, + ); + await runOperator(['policy', '--namespace', 'new', '--pull-requests', 'on'], io); + const opted = await h.db.prepare("SELECT * FROM scopes WHERE scope_id = 'new'").first(); + assert.equal(opted!.pull_requests, 1); + assert.equal(opted!.policy_version, 3); assert.equal(config.workers_dev, false); assert.equal(config.preview_urls, false); assert.deepEqual(config.routes, [{ pattern: 'cache.example.com', custom_domain: true }]); @@ -258,14 +267,19 @@ test('operator setup creates resources and initializes an empty database', async 'owner/repo', '--origin', 'https://cache.example.com', + '--pull-requests', + 'on', ], io, ); assert.ok(database && bucket && deployed); assert.equal(config.vars['GC_BATCH_SIZE'], '16'); assert.deepEqual(JSON.parse(config.vars['NAMESPACES']!), ['test']); - const scope = await h.db.prepare("SELECT endpoint FROM scopes WHERE scope_id = 'test'").first(); + const scope = await h.db + .prepare("SELECT endpoint, pull_requests FROM scopes WHERE scope_id = 'test'") + .first(); assert.equal(scope!.endpoint, 'https://cache.example.com/projects/test'); + assert.equal(scope!.pull_requests, 1); } finally { await h.close(); } diff --git a/test/service.test.ts b/test/service.test.ts index 22d8775..3b4daee 100644 --- a/test/service.test.ts +++ b/test/service.test.ts @@ -146,6 +146,67 @@ test('signed GitHub policy denies forks, wrong owners, private repos, non-push e } }); +test('pull request tokens publish only to opted-in namespaces from PRs that target the default branch', async () => { + const h = await harness(); + // Claims GitHub signs for a same-repository `pull_request` run. + const pr = { + event_name: 'pull_request', + ref: 'refs/pull/27/merge', + ref_type: 'branch', + base_ref: 'main', + head_ref: 'renovate/vite-plus', + }; + const status = async (claims: Record, scope = 'test') => + ( + await h.store(bytes('A'), bytes('S'), bytes('V'), undefined, { + scope, + token: await h.token({ aud: `https://cache.example.com/projects/${scope}`, ...claims }), + }) + ).status; + try { + assert.equal(await status(pr), 403); + await h.db.prepare("UPDATE scopes SET pull_requests = 1 WHERE scope_id = 'test'").run(); + assert.equal(await status(pr), 200); + // Namespaces without the opt-in keep the default-branch push policy. + assert.equal(await status(pr, 'other'), 403); + assert.equal(await status({}, 'other'), 200); + for (const claims of [ + { base_ref: 'feature' }, + { base_ref: 'refs/heads/main' }, + { base_ref: '' }, + { base_ref: undefined }, + // The `closed` run of a merged PR carries the base branch in `ref`. + { ref: 'refs/heads/main' }, + { ref: 'refs/pull/27/head' }, + { ref: 'refs/pull/27/merge/x' }, + { ref_type: 'tag' }, + { event_name: 'push' }, + { event_name: 'pull_request_target', ref: 'refs/heads/main' }, + { event_name: 'workflow_run', ref: 'refs/heads/main' }, + { repository_id: '999' }, + { repository_owner_id: '999' }, + { repository_visibility: 'private' }, + { aud: 'https://attacker.example' }, + ]) + assert.equal(await status({ ...pr, ...claims }), 403, JSON.stringify(claims)); + // Push tokens are unaffected by the opt-in. + assert.equal(await status({}), 200); + assert.equal(await status({ ref: 'refs/heads/feature' }), 403); + // Turning the opt-in off also stops PR uploads that were already admitted. + const admitted = await reserve( + h.db, + await getScope(h.db, 'test'), + Math.floor(Date.now() / 1000) + 300, + 100, + ); + await h.db.prepare("UPDATE scopes SET pull_requests = 0 WHERE scope_id = 'test'").run(); + await assert.rejects(publish(h.db, admitted, bytes('B'), bytes('T')), { status: 503 }); + assert.equal(await status(pr), 403); + } finally { + await h.close(); + } +}); + test('JWKS outages fail closed and have a refresh cooldown', async () => { const h = await harness({ jwksStatus: 503 }); try {