diff --git a/.gitignore b/.gitignore index ed29b187..8b3bab56 100644 --- a/.gitignore +++ b/.gitignore @@ -98,3 +98,7 @@ deno.lock # Python bytecode — from running the .github/scripts checks locally. __pycache__/ *.pyc + +# Local review artifacts +pr-review-*.md +pr-review-*.html diff --git a/docs/AGENTS.md b/docs/AGENTS.md index 45504ec8..e15a8484 100644 --- a/docs/AGENTS.md +++ b/docs/AGENTS.md @@ -81,6 +81,7 @@ Read these when working on the relevant area: - **[Making API calls](api-patterns.md)** - HTTP clients, Zod snake_case-to-camelCase transforms, `ApiError.fromHttpError()` - **[Working with resources](resources.md)** - `Resource` interface, adding new resources, site module, unified deploy - **[Deployments](deployments.md)** - Deploys addressed by commit, wrangler config, asset manifest hashing, direct asset uploads (Workers) and presigned uploads (static) +- **[Versions](versions.md)** - Recording a build as an immutable version and serving it, staged uploads with server-verified digests, raw resource payloads, the `BASE44_VERSIONS_API` gate - **[Plugins](plugins.md)** - Plugin config, namespaces, entity extension rules, function namespacing, pull/deploy behavior - **[Error handling](error-handling.md)** - Error hierarchy, throwing patterns, error codes, `CLIExitError`, `process.exit` ban - **[Writing tests](testing.md)** - Testkit, Given/When/Then pattern, API mocks, fixtures, test overrides diff --git a/docs/versions.md b/docs/versions.md new file mode 100644 index 00000000..b63eb711 --- /dev/null +++ b/docs/versions.md @@ -0,0 +1,133 @@ +# Versions + +**Keywords:** versions, publish, deployment, rollback, artifact set, sha256, digest, staged upload, presigned, x-amz-checksum-sha256, entities, agents, raw payloads, provenance, commit, idempotency key, step, BASE44_VERSIONS_API, env gate, build sandbox + +A **version** is one immutable thing a build produced — every frontend file, plus the entity and agent payloads the app declares. A **deployment** is a version made live at an environment. Recording one and serving one are separate acts, which is what makes a rollback a deploy of an older version rather than a second code path. + +This lives in `src/core/version/`: `artifacts.ts` (the build-output walk and the raw resource reads), `api.ts` (the three HTTP calls), `publish.ts` (orchestration, and the step names behind the shared tagger in `core/errors.ts`), `gate.ts` (the env gate), `schema.ts` (wire types). + +Where to build and what to collect is **not** here — it is `core/project/target.ts`, because nothing it resolves is about a version and `base44 build` needs the same answer without importing this lane. + +It is **not** `src/core/site/` — see [Deployments](deployments.md). That lane ships a build to the legacy hosting API and its `deploymentId` names a Cloudflare script; this one records a version on the platform's version plane and its `deploymentId` names a deployment there. A caller that could not tell the two apart would publish by accident, so they are separate commands with separate envelope field names. + +## Two hashes, never conflated + +`hashAsset` in `core/site/manifest.ts` is the first 32 hex characters of `sha256(utf8(app_id) ‖ bytes)` — a **provider upload identifier**, salted so a tenant can only collide with its own files. + +`ArtifactFile.digest` is a **full sha256 over the stored bytes**, streamed so a large file is never read whole. It is durable artifact identity, and the platform signs it into the upload URL. Different purpose, different moment, different value: conflating them produces an asset that uploads fine and never dedupes, or a digest check that fails on a correct file. + +## The flow + +`createVersion(artifacts, options)` in `api.ts` — three calls, in this order and no other, so nothing is recorded until the bytes are in place: + +1. **Declare.** `POST versions` with `assets` (path, size, digest per file), optionally a `site_worker` beside it, plus the raw `entities` and `agents` payloads and `source_commit`. The response carries a `session_id` and one presigned PUT per declared file, in declared order. +2. **Upload.** `putPresigned` per file — the same PUT the static deployments lane uses, same ky retry policy. Each PUT sends the server's `Content-Type` **and** its `x-amz-checksum-sha256` verbatim; deriving either locally would 403 on any mapping difference. Uploads are paired with declared files **by position**, not by path: the frontend and the Worker's modules are separate namespaces, so the same name can appear in both and mean two different files. +3. **Finalize.** `POST versions/{session_id}/finalize`, no body. The set was fixed at declare, so there is nothing left for the caller to change. + +The response carries `version_id` and `manifest_hash`, and no flag for "this content already existed" — the hash **is** the identity, so a caller asking whether a rebuild changed anything compares it against the last one. An existing version is not necessarily the one being served, so such a flag would be misleading anyway. + +Each of the three responses is parsed through its Zod schema and a mismatch raises `SchemaValidationError` — the house pattern from [Making API calls](api-patterns.md), and the only thing keeping this client aligned with the server. There is no generated type and no shared contract fixture here, the same as every other CLI↔platform surface: a server that renames or retypes a response field fails the publish with an error naming the field, rather than propagating `undefined`. In the other direction the server's request models forbid unknown fields, so a field this client sends that the server no longer accepts is a 422. + +What that does **not** catch is a change of meaning behind an unchanged shape. Nothing here does; the lane is small enough that both sides are reviewed together. + +`setEnvironmentVersion(environment, versionId, options)` is one `PATCH /environments/{name}` carrying the version id and an idempotency key. The key is what makes a repeat the SAME call rather than a second publish, so the request is retried — bounded, and **only when a key is sent**, including on our own timeout, which says nothing about whether the server committed. Without a key a repeat is a deliberate redeploy and is never retried. **An environment serves one version, so making a version live is editing that pointer — there is no deployment to create.** The `Deployment` record the switch leaves behind is how the plane remembers what it prepared, returned so a caller can correlate a log line. + +Nothing else is the caller's to say: the app comes from the credential, and so do the acting principal, the runtime environment variables, every artifact key, the manifest hash and the publication revision. The request models on the server forbid unknown fields, so sending one is an error rather than a silent drop. + +## An app with a server of its own + +`collectSiteWorker(projectRoot)` reads through `resolveFullStackBuild` — the same call `site deploy` makes — which looks for `.wrangler/deploy/config.json`, the redirect file a `@cloudflare/vite-plugin` build leaves behind. No second reader, because a second reader is a second opinion about what the framework built. `publish` and `versions create` both collect through one `collectArtifacts`, for the same reason. + +A commit is a static app **or** a full-stack one, never both, and backend functions ride on either. So the two are mutually exclusive on the wire and declaring both is refused: + +- **Static** — no `site_worker`, so the platform serves the assets from S3. They must contain `index.html`, because any unmatched path is answered with that one file. +- **Full-stack** — a `site_worker` is named, so the Worker serves them, and they are taken from its own `assets.directory` rather than the project's build output. No entry file is required: its own `not_found_handling` decides. A Worker that answers every path itself declares no assets at all, which is a complete app. + +One set, two readings, and the Worker's presence is the whole of the difference — so there is no way to claim both, because there is no second field to claim it in. The **manifest** still has a `static_bundle` field and it is not a description: the platform reads it as *serve this from S3* and hands the prefix straight to the dist service. Which field names the set is the server's to decide from the declaration, not the producer's to state. + +`main` is sent as the module set names it, not as the config wrote it. The platform matches the entry against the names it was sent, so a surviving `./` would name a module nothing in the set provides. + +`compatibility_date`, `compatibility_flags`, each module's `type` and the whole `serving_config` ride along. They are part of the Worker's **identity** on the platform, not metadata: the same modules under a different compatibility date are a different Worker, and so are the same bytes read as `text` instead of `data`, or served with `run_worker_first` flipped. A version that dropped any of them would record two different Workers as one and could never tell them apart afterwards. + +`serving_config` decides what happens to a request BEFORE the Worker runs — whether it runs at all, which asset a path resolves to, what a miss gets. It is named for that rather than for the assets, which are the set of files at the top of the declaration. Its FIELDS are wrangler's own `assets` block, name for name — `html_handling`, `not_found_handling`, `run_worker_first`, `headers`, `redirects` — because that is what the producer read. A block stating none of them is sent as `null`, the same as no block at all: a bare `assets: { directory }` describes the identical Worker, and recording them apart would split one version in two. + +**Nothing deploys this yet.** The platform records the Worker on the version and stores its modules, and refuses a full-stack app at admission — so today this proves the transport, not a publish. + +## Why the digest is signed into the URL + +The platform pins content type, content length **and** sha256 into each presigned PUT, so S3 itself rejects a body that does not hash to the declared digest. The URL is permission to write exactly one payload, once — which is what lets the server commit those bytes with a server-side copy instead of reading them back to re-hash. A frontend of any size is recorded without its bytes passing through a worker. + +The practical consequence for this CLI: **send the checksum the server gave you, unchanged.** `PresignedAssetUpload.checksumSha256` is optional because the legacy static lane's URLs pin only type and length. + +## Resources go up raw + +`collectResources` reads `entities/` and `agents/` with `readJsonFile` and sends the parsed payloads untouched, keyed by the file's path with the schema extension stripped — `entities/Todo.jsonc` → `Todo`, `agents/support/triage.jsonc` → `support/triage`. That is the same name the platform derives from the same file. + +Deliberately **not** `entityResource.readAll` / `agentResource.readAll`. The platform's own extractor and validation are authoritative, and this CLI's stricter entity schema refuses real Builder apps — which is exactly why `site deploy` reads no resources at all. Re-introducing the strict parse here would reproduce that block. + +## One commit, and it is provenance rather than identity + +`resolveProvenanceCommit(projectRoot, explicit?)` in `core/site/git-hash.ts` returns `undefined` rather than failing when there is no checkout. A version is identified by its **content**; the commit is recorded beside it and never hashed, so a build outside a git checkout is still a complete version. + +That is the whole difference from `resolveGitHash`, whose caller addresses a deployment *by* the hash and therefore cannot go without one. + +One commit, not two: an app's frontend and backend are the same app at the same source, so a version records a single `source_commit`. + +## A Builder repo carries no CLI config + +`resolveBuildTarget(projectRoot, overrides)` in `core/project/target.ts` fills in `npm run build` and `dist` **only when a repo has no config at all**, and **writes nothing**. The python driver it replaces used to overwrite `base44/config.jsonc` with a minimal config before building, destroying any checked-in configuration — and, for a full-stack app, its build command. + +A config that is present wins, field by field, and one that omits a field still gets today's error: omitting `site.buildCommand` is a deliberate statement, and answering it with a guessed `npm run build` would change what `base44 build` does for every project that relies on that error. `requireOutputDir(target)` raises at the point of collection rather than at resolution, so a project missing both is told about its build command first — the one it hits first. + +## Which step failed + +A publish is three steps and they fail differently: a user's build failing, an artifact set the platform refused, and a lost publication race are three incidents with three responses. `tagStep(step, run)` attaches the step to the error through a non-enumerable symbol — the original error type, message, status code and request id all survive — and `Base44Command` writes it into the `--json` error envelope as `step`. + +A callback that throws synchronously is tagged too; `run().catch(...)` would let that one escape untagged. + +## Commands + +**`base44 publish [--no-build] [--output-dir ] [--target ] [--git-hash ] [--concurrency ]`** — build, record a version, serve it. Under `--json`, stdout is a single `{environment, versionId, manifestHash, deploymentId}` document. + +**`base44 versions create`** — record built output without serving it. A version can sit unpublished for as long as it likes. + +**`base44 versions deploy [--target ]`** — point an environment at a recorded version: no checkout, no build, no upload. Passing an older id is how a rollback is done. + +`base44 build` is not part of this group and is not gated, but the lane depends on two things about it: it needs **no credential** (the publish sandbox builds before minting a key that can deploy), and it resolves its config through `resolveBuildTarget` (a Builder repo has none). What it builds and what it prints are unchanged. + +The group is plural to match `agents`, `entities`, `functions`, `secrets` and `workflows` — and because `base44 version` shadowed `base44 --version` two lines above it in `--help`. + +## Upload concurrency + +`DEFAULT_VERSION_UPLOAD_CONCURRENCY` is 8, `MAX_VERSION_UPLOAD_CONCURRENCY` is 16. Measured on the build sandbox's pipe: at 3, a 25 500-asset app moved 27 assets/s (~109 ms per PUT, 23 KiB mean — latency-bound) and needed ~930 s of the ~450 s a build leaves, so it was SIGKILLed mid-upload every time. 8 is the rate the python driver it replaced already sustained to the same bucket; 16 failed a degraded pipe on 2026-07-02. + +## What one declaration may cost + +`MAX_FILE_COUNT` is 50 000, matching the server. It is a cost bound, not a guess +about app size: the platform signs one presigned URL per declared file — measured +at ~108 µs of blocking crypto each — and holds the whole declared set in Redis +until the version finalizes. At the ceiling that is ~5.4 s and ~7 MB for a single +request, which is why declaring is rate-limited far more tightly than finalizing +or deploying. + +The largest frontend ever measured through the build sandbox is 25 500 assets, so +the ceiling is roughly 2x a real worst case. Raising it on one side alone only +earns a rejection after the walk. + +## The env gate + +The whole lane is one env var. With `BASE44_VERSIONS_API=1` (or `true`; internal gate, not user-facing yet) `base44 publish` and the `versions` group are registered; without it they are **not registered at all**, so they are absent from `--help` and typing one is an unknown command. `versionsApiEnabled()` in `core/version/gate.ts` is read in exactly one place: the registration in `program.ts`. + +Deliberately **not** `BASE44_DEPLOYMENTS_API`. That one selects the legacy deployments transport for `site deploy`, and the build sandbox already sets it for that arm; one var switching both lanes would make them impossible to roll out apart. + +## The automated consumer + +The platform's build sandbox runs the lane in two execs and builds once: + +``` +base44 build --json # no credential in the environment +base44 publish --no-build --json \ # the key exists only for this one + --git-hash --concurrency 8 +``` + +Two execs because the key now carries publish authority and the build is code the repo controls, so the key is minted only after the build finishes and revoked on the way out. `--no-build` is what keeps it one build rather than two. diff --git a/packages/cli/src/cli/commands/project/build.ts b/packages/cli/src/cli/commands/project/build.ts index 747329ce..6e733923 100644 --- a/packages/cli/src/cli/commands/project/build.ts +++ b/packages/cli/src/cli/commands/project/build.ts @@ -1,22 +1,18 @@ import type { Command } from "commander"; import { runSiteBuild } from "@/cli/commands/project/site-build.js"; import type { CLIContext, RunCommandResult } from "@/cli/types.js"; -import { Base44Command, theme } from "@/cli/utils/index.js"; -import { ConfigInvalidError } from "@/core/errors.js"; -import { readProjectConfig } from "@/core/project/index.js"; +import { Base44Command, requireApp, theme } from "@/cli/utils/index.js"; +import { resolveBuildTarget } from "@/core/project/index.js"; async function buildAction(ctx: CLIContext): Promise { - const { app } = ctx; - if (!app?.projectRoot) { - throw new ConfigInvalidError( - "base44 build requires a linked local project. Run it from a project with base44/.app.jsonc.", - ); - } + const app = requireApp(ctx); + // Not readProjectConfig: a Builder repo carries no CLI config, and this is the + // step a publish sandbox runs inside one. A present config still wins. + const target = await resolveBuildTarget(app.projectRoot); - const { project } = await readProjectConfig(app.projectRoot); await runSiteBuild(ctx, { - root: project.root, - buildCommand: project.site?.buildCommand, + root: target.root, + buildCommand: target.buildCommand, appId: app.id, }); @@ -26,7 +22,9 @@ async function buildAction(ctx: CLIContext): Promise { } export function getBuildCommand(): Command { - return new Base44Command("build") + // No credential: it calls no API, and a publish sandbox runs it before minting + // a key that can deploy. The app id is still required. + return new Base44Command("build", { requireAuth: false }) .description("Build the site with the Base44 app id injected") .action(buildAction); } diff --git a/packages/cli/src/cli/commands/publish.ts b/packages/cli/src/cli/commands/publish.ts new file mode 100644 index 00000000..716ed366 --- /dev/null +++ b/packages/cli/src/cli/commands/publish.ts @@ -0,0 +1,101 @@ +import type { Command } from "commander"; +import { runSiteBuild } from "@/cli/commands/project/site-build.js"; +import { + concurrencyOption, + gitHashOption, + outputDirOption, + targetOption, +} from "@/cli/commands/versions/options.js"; +import type { CLIContext, RunCommandResult } from "@/cli/types.js"; +import { Base44Command, requireApp, theme } from "@/cli/utils/index.js"; +import { resolveBuildTarget } from "@/core/project/index.js"; +import { resolveProvenanceCommit } from "@/core/site/index.js"; +import { + collectArtifacts, + publishVersion, + tagStep, +} from "@/core/version/index.js"; + +interface PublishOptions { + build?: boolean; + outputDir?: string; + gitHash?: string; + target?: string; + concurrency?: number; +} + +/** + * Build, record a version, and serve it. Not `site deploy`, whose `deploymentId` + * means a Cloudflare script rather than a deployment on this plane — separate + * envelope names so the two cannot be confused. + */ +async function publishAction( + ctx: CLIContext, + options: PublishOptions, +): Promise { + const { runTask, log, jsonMode } = ctx; + const app = requireApp(ctx); + // Tagged: a config this command cannot read is a version it cannot produce, + // the same reason `collectArtifacts` is tagged below. + const target = await tagStep("create_version", () => + resolveBuildTarget(app.projectRoot, { outputDir: options.outputDir }), + ); + + if (options.build !== false) { + await tagStep("build", () => + runSiteBuild(ctx, { + root: target.root, + buildCommand: target.buildCommand, + appId: app.id, + }), + ); + } + + const gitHash = await resolveProvenanceCommit(target.root, options.gitHash); + const result = await runTask( + "Publishing...", + async (updateMessage) => { + // Inside the tag, so a missing output directory reports as a + // create_version failure rather than an envelope with no step. + const artifacts = await tagStep("create_version", () => + collectArtifacts(target), + ); + return await publishVersion(artifacts, { + sourceCommit: gitHash, + target: options.target, + concurrency: options.concurrency, + progress: { + onDeclared: ({ fileCount }) => + updateMessage(`Uploading ${fileCount} files`), + onUpload: ({ uploadedFiles, totalFiles }) => + updateMessage(`Uploaded ${uploadedFiles} of ${totalFiles} files`), + }, + }); + }, + { successMessage: "Published", errorMessage: "Publish failed" }, + ); + + if (!jsonMode) { + log.message(theme.styles.dim(`version ${result.versionId}`)); + } + return { + outroMessage: `${result.environment} now serves ${result.versionId}`, + stdout: jsonMode ? `${JSON.stringify(result, null, 2)}\n` : undefined, + }; +} + +export function getPublishCommand(): Command { + return new Base44Command("publish") + .description( + "Build the app, record it as a version, and serve that version", + ) + .option( + "--no-build", + "Publish the existing build output without rebuilding", + ) + .addOption(outputDirOption()) + .addOption(targetOption()) + .addOption(gitHashOption()) + .addOption(concurrencyOption()) + .action(publishAction); +} diff --git a/packages/cli/src/cli/commands/versions/create.ts b/packages/cli/src/cli/commands/versions/create.ts new file mode 100644 index 00000000..070b23c7 --- /dev/null +++ b/packages/cli/src/cli/commands/versions/create.ts @@ -0,0 +1,62 @@ +import type { Command } from "commander"; +import { + concurrencyOption, + gitHashOption, + outputDirOption, +} from "@/cli/commands/versions/options.js"; +import type { CLIContext, RunCommandResult } from "@/cli/types.js"; +import { Base44Command, requireApp } from "@/cli/utils/index.js"; +import { resolveBuildTarget } from "@/core/project/index.js"; +import { resolveProvenanceCommit } from "@/core/site/index.js"; +import { collectArtifacts, createVersion } from "@/core/version/index.js"; + +interface CreateOptions { + outputDir?: string; + gitHash?: string; + concurrency?: number; +} + +/** Record a build that already exists. No build of its own, and no deploy. */ +async function createAction( + ctx: CLIContext, + options: CreateOptions, +): Promise { + const { runTask, jsonMode } = ctx; + const target = await resolveBuildTarget(requireApp(ctx).projectRoot, { + outputDir: options.outputDir, + }); + const gitHash = await resolveProvenanceCommit(target.root, options.gitHash); + + const version = await runTask( + "Creating version...", + async (updateMessage) => + await createVersion(await collectArtifacts(target), { + sourceCommit: gitHash, + concurrency: options.concurrency, + progress: { + onDeclared: ({ fileCount }) => + updateMessage(`Uploading ${fileCount} files`), + onUpload: ({ uploadedFiles, totalFiles }) => + updateMessage(`Uploaded ${uploadedFiles} of ${totalFiles} files`), + }, + }), + { + successMessage: "Version created", + errorMessage: "Create version failed", + }, + ); + + return { + outroMessage: `Version ${version.versionId} (${version.manifestHash})`, + stdout: jsonMode ? `${JSON.stringify(version, null, 2)}\n` : undefined, + }; +} + +export function getVersionCreateCommand(): Command { + return new Base44Command("create") + .description("Record the built output as a version, without deploying it") + .addOption(outputDirOption()) + .addOption(gitHashOption()) + .addOption(concurrencyOption()) + .action(createAction); +} diff --git a/packages/cli/src/cli/commands/versions/deploy.ts b/packages/cli/src/cli/commands/versions/deploy.ts new file mode 100644 index 00000000..a1db89c7 --- /dev/null +++ b/packages/cli/src/cli/commands/versions/deploy.ts @@ -0,0 +1,45 @@ +import { randomUUID } from "node:crypto"; +import type { Command } from "commander"; +import { targetOption } from "@/cli/commands/versions/options.js"; +import type { CLIContext, RunCommandResult } from "@/cli/types.js"; +import { Base44Command } from "@/cli/utils/index.js"; +import { + DEFAULT_ENVIRONMENT, + setEnvironmentVersion, +} from "@/core/version/index.js"; + +/** Point an environment at an existing version: no checkout, no build, no + * upload. Pointing it at an older one is the rollback. */ +async function deployAction( + { runTask, jsonMode }: CLIContext, + versionId: string, + options: { target?: string }, +): Promise { + const environment = options.target ?? DEFAULT_ENVIRONMENT; + const result = await runTask( + `Pointing ${environment} at ${versionId}...`, + async () => + await setEnvironmentVersion(environment, versionId, { + idempotencyKey: randomUUID(), + }), + { + successMessage: "Environment updated", + errorMessage: "Could not update the environment", + }, + ); + + return { + outroMessage: `${result.name} now serves ${result.versionId}`, + stdout: jsonMode ? `${JSON.stringify(result, null, 2)}\n` : undefined, + }; +} + +export function getVersionDeployCommand(): Command { + return new Base44Command("deploy") + .description( + "Point an environment at an already-recorded version (also the rollback)", + ) + .argument("", "The version to serve") + .addOption(targetOption()) + .action(deployAction); +} diff --git a/packages/cli/src/cli/commands/versions/index.ts b/packages/cli/src/cli/commands/versions/index.ts new file mode 100644 index 00000000..8114d655 --- /dev/null +++ b/packages/cli/src/cli/commands/versions/index.ts @@ -0,0 +1,10 @@ +import { Command } from "commander"; +import { getVersionCreateCommand } from "./create.js"; +import { getVersionDeployCommand } from "./deploy.js"; + +export function getVersionsCommand(): Command { + return new Command("versions") + .description("Record app versions and serve them") + .addCommand(getVersionCreateCommand()) + .addCommand(getVersionDeployCommand()); +} diff --git a/packages/cli/src/cli/commands/versions/options.ts b/packages/cli/src/cli/commands/versions/options.ts new file mode 100644 index 00000000..0efdebef --- /dev/null +++ b/packages/cli/src/cli/commands/versions/options.ts @@ -0,0 +1,54 @@ +import { InvalidArgumentError, Option } from "commander"; +import { isGitCommitHash } from "@/core/utils/git.js"; +import { + DEFAULT_VERSION_UPLOAD_CONCURRENCY, + MAX_VERSION_UPLOAD_CONCURRENCY, +} from "@/core/version/index.js"; + +/** + * The options the versions lane's commands share, built here so two `--help` + * screens cannot describe the same flag differently. + */ + +export function outputDirOption(): Option { + return new Option( + "--output-dir ", + "Build output directory (defaults to the project's, else dist)", + ); +} + +export function targetOption(): Option { + return new Option("--target ", "Environment to serve the version at"); +} + +export function gitHashOption(): Option { + return new Option( + "--git-hash ", + "Commit the build came from (defaults to the checkout's HEAD)", + ).argParser((value) => { + if (!isGitCommitHash(value)) { + throw new InvalidArgumentError( + "Expected a git commit hash (7-64 hex chars).", + ); + } + return value; + }); +} + +export function concurrencyOption(): Option { + return new Option("--concurrency ", "Parallel file uploads") + .default(DEFAULT_VERSION_UPLOAD_CONCURRENCY) + .argParser((value) => { + const parsed = Number(value); + if ( + !Number.isInteger(parsed) || + parsed < 1 || + parsed > MAX_VERSION_UPLOAD_CONCURRENCY + ) { + throw new InvalidArgumentError( + `Expected a whole number between 1 and ${MAX_VERSION_UPLOAD_CONCURRENCY}.`, + ); + } + return parsed; + }); +} diff --git a/packages/cli/src/cli/program.ts b/packages/cli/src/cli/program.ts index 070dfc74..06d896cd 100644 --- a/packages/cli/src/cli/program.ts +++ b/packages/cli/src/cli/program.ts @@ -18,14 +18,17 @@ import { getLinkCommand } from "@/cli/commands/project/link.js"; import { getLogsCommand } from "@/cli/commands/project/logs.js"; import { getScaffoldCommand } from "@/cli/commands/project/scaffold.js"; import { getVisibilityCommand } from "@/cli/commands/project/visibility.js"; +import { getPublishCommand } from "@/cli/commands/publish.js"; import { getSandboxCommand } from "@/cli/commands/sandbox/index.js"; import { getSecretsCommand } from "@/cli/commands/secrets/index.js"; import { getSiteCommand } from "@/cli/commands/site/index.js"; import { getTypesCommand } from "@/cli/commands/types/index.js"; +import { getVersionsCommand } from "@/cli/commands/versions/index.js"; import { getWorkflowsCommand } from "@/cli/commands/workflows/index.js"; import { getWorkspaceCommand } from "@/cli/commands/workspace/index.js"; import { Base44Command } from "@/cli/utils/index.js"; import { BASE44_APP_ID_ENV_VAR } from "@/core/consts.js"; +import { versionsApiEnabled } from "@/core/version/gate.js"; import packageJson from "../../package.json"; import { getDevCommand } from "./commands/dev.js"; import { getExecCommand } from "./commands/exec.js"; @@ -117,6 +120,13 @@ export function createProgram(context: CLIContext): Command { // Register site commands program.addCommand(getSiteCommand()); + // Registered on the enabled lane only: with the gate off they are absent from + // --help and rejected as unknown commands, rather than exposing a lane that is + // still being integrated against the platform. + if (versionsApiEnabled()) { + program.addCommand(getPublishCommand()); + program.addCommand(getVersionsCommand()); + } // Register types command program.addCommand(getTypesCommand()); diff --git a/packages/cli/src/cli/utils/command/Base44Command.ts b/packages/cli/src/cli/utils/command/Base44Command.ts index 5c99e93b..cbbe25e0 100644 --- a/packages/cli/src/cli/utils/command/Base44Command.ts +++ b/packages/cli/src/cli/utils/command/Base44Command.ts @@ -15,7 +15,12 @@ import { formatPlainUpgradeMessage, startUpgradeCheck, } from "@/cli/utils/upgradeNotification.js"; -import { ApiError, InvalidInputError, isCLIError } from "@/core/errors.js"; +import { + ApiError, + InvalidInputError, + isCLIError, + stepOf, +} from "@/core/errors.js"; import { resolveBranchName } from "@/core/resources/branch/api.js"; /** @@ -44,6 +49,12 @@ function writeJsonError(error: unknown): void { const envelope: Record = { error: error instanceof Error ? error.message : String(error), }; + // Which step of a multi-step command broke. One exit code for all of them is + // how a sandbox log stops being diagnostic. + const step = stepOf(error); + if (step !== undefined) { + envelope.step = step; + } if (isCLIError(error)) { envelope.code = error.code; if (error.details.length > 0) { diff --git a/packages/cli/src/cli/utils/command/index.ts b/packages/cli/src/cli/utils/command/index.ts index 70788e23..af21f5ba 100644 --- a/packages/cli/src/cli/utils/command/index.ts +++ b/packages/cli/src/cli/utils/command/index.ts @@ -1 +1,2 @@ export * from "./Base44Command.js"; +export * from "./middleware.js"; diff --git a/packages/cli/src/cli/utils/command/middleware.ts b/packages/cli/src/cli/utils/command/middleware.ts index 742287f0..3fca8f47 100644 --- a/packages/cli/src/cli/utils/command/middleware.ts +++ b/packages/cli/src/cli/utils/command/middleware.ts @@ -6,6 +6,8 @@ import { readAuth, seedAuthFromEnv, } from "@/core/auth/index.js"; +import { InternalError } from "@/core/errors.js"; +import type { AppContext } from "@/core/project/index.js"; import { initAppContext } from "@/core/project/index.js"; /** @@ -52,3 +54,21 @@ export async function ensureAppContext( ctx.app = appContext; ctx.errorReporter.setContext({ appId: appContext.id }); } + +/** + * The app this command resolved. Optional on `CLIContext` only for the few + * commands declaring `requireAppContext: false`; everywhere else + * {@link ensureAppContext} has already returned one or thrown. + * + * Narrow here rather than defaulting at the call site: an app id defaulted to + * `""` is inlined by Vite, so the build and the publish both succeed and the + * served app addresses no app. + */ +export function requireApp(ctx: Pick): AppContext { + if (!ctx.app) { + throw new InternalError( + "This command read an app context it never resolved — it is declared with requireAppContext: false.", + ); + } + return ctx.app; +} diff --git a/packages/cli/src/core/errors.ts b/packages/cli/src/core/errors.ts index b57ab370..a6492a79 100644 --- a/packages/cli/src/core/errors.ts +++ b/packages/cli/src/core/errors.ts @@ -616,3 +616,47 @@ export function isUserError(error: unknown): error is UserError { export function isSystemError(error: unknown): error is SystemError { return error instanceof SystemError; } + +/** + * Which step of a multi-step command a failure came from. + * + * Here rather than beside any one command: the envelope that reads the tag is + * the generic command framework, and a framework importing a feature module to + * read its own output is the dependency backwards. The vocabulary stays with + * the command that owns it. + */ +const STEP = Symbol.for("base44.commandStep"); + +/** Tag an error with its step without wrapping it, so the original type, + * status and request id still reach the envelope. */ +export async function tagStep( + step: string, + run: () => Promise, +): Promise { + try { + // Awaited inside the try so a callback that throws SYNCHRONOUSLY is tagged + // too — `run().catch(...)` would let that one escape untagged. + return await run(); + } catch (error) { + // The innermost tag wins: an outer step re-tagging would report where the + // failure surfaced rather than where it happened. `isExtensible` too: a + // library that freezes its errors would turn the tag into a TypeError and + // lose the original entirely. + if ( + error !== null && + typeof error === "object" && + !(STEP in error) && + Object.isExtensible(error) + ) { + Object.defineProperty(error, STEP, { value: step, enumerable: false }); + } + throw error; + } +} + +/** The step a tagged error came from, or `undefined` for an untagged one. */ +export function stepOf(error: unknown): string | undefined { + return error !== null && typeof error === "object" && STEP in error + ? (error as Record)[STEP] + : undefined; +} diff --git a/packages/cli/src/core/project/index.ts b/packages/cli/src/core/project/index.ts index 8fc230d3..af853afa 100644 --- a/packages/cli/src/core/project/index.ts +++ b/packages/cli/src/core/project/index.ts @@ -5,4 +5,5 @@ export * from "./create.js"; export * from "./deploy.js"; export * from "./find-root.js"; export * from "./schema.js"; +export * from "./target.js"; export * from "./template.js"; diff --git a/packages/cli/src/core/project/target.ts b/packages/cli/src/core/project/target.ts new file mode 100644 index 00000000..67946988 --- /dev/null +++ b/packages/cli/src/core/project/target.ts @@ -0,0 +1,101 @@ +import { dirname, join, resolve } from "node:path"; +import { PROJECT_SUBDIR } from "@/core/consts.js"; +import { ConfigNotFoundError } from "@/core/errors.js"; +import { readProjectSettings } from "@/core/project/index.js"; +import type { ProjectWithPaths } from "@/core/project/types.js"; + +/** From the app template every Builder app is seeded from. */ +const DEFAULT_BUILD_COMMAND = "npm run build"; +const DEFAULT_OUTPUT_DIRECTORY = "dist"; + +/** A project's resolved layout: where its build runs, and what it leaves behind. */ +export interface BuildTarget { + root: string; + /** Where `entitiesDir` and `agentsDir` are resolved from. */ + configDir: string; + /** `undefined` when a config is present and declares none; `runSiteBuild` + * reports that, as it always has. */ + buildCommand?: string; + /** `null` for the same reason. {@link requireOutputDir} raises at collection, + * so a project missing both hears about its build command first. */ + outputDir: string | null; + entitiesDir: string; + agentsDir: string; +} + +/** + * Where to build, and what to collect afterwards — filling in what a Builder + * repo does not carry, without writing a file. The sandbox used to overwrite + * `base44/config.jsonc` with a minimal one, destroying checked-in configuration. + * + * Here and not under `version/`: nothing it resolves is about a version. `build` + * needs the identical answer and is not a publish command, so a module that made + * it import the versions lane to ask would have the dependency backwards. + * + * Defaults apply only when there is no config at all: one that omits a field + * said so deliberately, and still gets today's error. + */ +export async function resolveBuildTarget( + projectRoot: string | undefined, + overrides: { outputDir?: string } = {}, +): Promise { + const project = await readSettingsIfPresent(projectRoot); + const root = project?.root ?? projectRoot ?? process.cwd(); + + return { + root, + configDir: project + ? dirname(project.configPath) + : join(root, PROJECT_SUBDIR), + buildCommand: project ? project.site?.buildCommand : DEFAULT_BUILD_COMMAND, + outputDir: outputDirectory(project, root, overrides.outputDir), + entitiesDir: project?.entitiesDir ?? "entities", + agentsDir: project?.agentsDir ?? "agents", + }; +} + +function outputDirectory( + project: ProjectWithPaths | null, + root: string, + override: string | undefined, +): string | null { + const configured = + override ?? + (project ? project.site?.outputDirectory : DEFAULT_OUTPUT_DIRECTORY); + return configured ? resolve(root, configured) : null; +} + +/** The directory to collect a build from, or the error saying the project never + * named one. */ +export function requireOutputDir(target: BuildTarget): string { + if (target.outputDir === null) { + throw new ConfigNotFoundError("No site configuration found.", { + hints: [ + { + message: + 'Add \'site.outputDirectory\' to your config.jsonc (e.g., "site": { "outputDirectory": "dist" })', + }, + { message: `Or pass --output-dir , relative to ${target.root}` }, + ], + }); + } + return target.outputDir; +} + +/** + * The project's settings, or `null` when it has none. A config that is present + * and invalid still throws — publishing past a broken one is how a typo becomes + * a version built the wrong way. + */ +async function readSettingsIfPresent( + projectRoot?: string, +): Promise { + try { + return await readProjectSettings(projectRoot); + } catch (error) { + if (error instanceof ConfigNotFoundError) { + return null; + } + throw error; + } +} diff --git a/packages/cli/src/core/site/deployment.ts b/packages/cli/src/core/site/deployment.ts index d7f35461..10af687e 100644 --- a/packages/cli/src/core/site/deployment.ts +++ b/packages/cli/src/core/site/deployment.ts @@ -2,11 +2,10 @@ import { readFile } from "node:fs/promises"; import { join } from "node:path"; import { InvalidInputError } from "@/core/errors.js"; import { getAppContext } from "@/core/project/app-config.js"; -import { pathExists } from "@/core/utils/fs.js"; import type { FinalizePayload } from "./api.js"; import { createDeployment, finalizeDeployment } from "./api.js"; +import { resolveFullStackBuild } from "./full-stack.js"; import { buildAssetManifest } from "./manifest.js"; -import { collectModules } from "./modules.js"; import type { AssetManifestResult, CreateDeploymentRequest, @@ -15,10 +14,6 @@ import type { } from "./schema.js"; import { uploadDeploymentAssets } from "./upload.js"; import type { ResolvedWranglerConfig } from "./wrangler-config.js"; -import { - detectFullStackArtifact, - resolveWranglerConfig, -} from "./wrangler-config.js"; type WorkerConfig = NonNullable; @@ -117,18 +112,12 @@ async function resolveWorkerBuild( projectRoot: string, progress?: DeploymentProgress, ): Promise { - const redirectPath = await detectFullStackArtifact(projectRoot); - if (!redirectPath) { + const built = await resolveFullStackBuild(projectRoot); + if (!built) { return null; } - const config = await resolveWranglerConfig(redirectPath); - - const assetsDir = - config.assetsDirectory && (await pathExists(config.assetsDirectory)) - ? config.assetsDirectory - : null; - + const { config, modules, assetsDir } = built; return { config: { main: config.main, @@ -136,7 +125,7 @@ async function resolveWorkerBuild( compatibility_flags: config.compatibilityFlags, assets: buildAssetsConfig(config.assetsConfig, progress), }, - modules: await collectModules(config), + modules, assetsDir, }; } diff --git a/packages/cli/src/core/site/full-stack.ts b/packages/cli/src/core/site/full-stack.ts new file mode 100644 index 00000000..9f0dbea8 --- /dev/null +++ b/packages/cli/src/core/site/full-stack.ts @@ -0,0 +1,42 @@ +import { pathExists } from "@/core/utils/fs.js"; +import { collectModules } from "./modules.js"; +import type { WorkerModule } from "./schema.js"; +import type { ResolvedWranglerConfig } from "./wrangler-config.js"; +import { + detectFullStackArtifact, + resolveWranglerConfig, +} from "./wrangler-config.js"; + +export interface FullStackBuild { + config: ResolvedWranglerConfig; + modules: WorkerModule[]; + /** + * Where the Worker serves assets from, confirmed to exist. `null` is a Worker + * that answers every path itself — a complete app, not a broken build. + */ + assetsDir: string | null; +} + +/** + * The full-stack artifact this build left behind, or `null` for a plain static + * site. ONE reader for both lanes — two would eventually disagree about the + * same directory. + */ +export async function resolveFullStackBuild( + projectRoot: string, +): Promise { + const redirectPath = await detectFullStackArtifact(projectRoot); + if (!redirectPath) { + return null; + } + + const config = await resolveWranglerConfig(redirectPath); + return { + config, + modules: await collectModules(config), + assetsDir: + config.assetsDirectory && (await pathExists(config.assetsDirectory)) + ? config.assetsDirectory + : null, + }; +} diff --git a/packages/cli/src/core/site/git-hash.ts b/packages/cli/src/core/site/git-hash.ts index 0e42e4de..d4fe4d05 100644 --- a/packages/cli/src/core/site/git-hash.ts +++ b/packages/cli/src/core/site/git-hash.ts @@ -25,6 +25,25 @@ export async function resolveGitHash( return hash; } +/** + * The commit this build came from, or `undefined` when there is none. + * + * For a version the commit is PROVENANCE — recorded, never hashed, and not part + * of what the version is — so a build outside a checkout is still a complete + * version. That is the whole difference from {@link resolveGitHash}, whose + * caller addresses a deployment BY the hash and so cannot go without one. + */ +export async function resolveProvenanceCommit( + projectRoot: string, + explicit?: string, +): Promise { + if (explicit) { + return await resolveGitHash(projectRoot, explicit); + } + const hash = await gitHead(projectRoot); + return hash && isGitCommitHash(hash) ? hash : undefined; +} + async function gitHead(projectRoot: string): Promise { try { const { stdout } = await execa("git", ["rev-parse", "HEAD"], { diff --git a/packages/cli/src/core/site/index.ts b/packages/cli/src/core/site/index.ts index f36431f2..d95877b0 100644 --- a/packages/cli/src/core/site/index.ts +++ b/packages/cli/src/core/site/index.ts @@ -2,6 +2,7 @@ export * from "./api.js"; export * from "./config.js"; export * from "./deploy.js"; export * from "./deployment.js"; +export * from "./full-stack.js"; export * from "./git-hash.js"; export * from "./manifest.js"; export * from "./modules.js"; diff --git a/packages/cli/src/core/site/manifest.ts b/packages/cli/src/core/site/manifest.ts index ef50fba0..608663cf 100644 --- a/packages/cli/src/core/site/manifest.ts +++ b/packages/cli/src/core/site/manifest.ts @@ -1,8 +1,10 @@ +import type { Hash } from "node:crypto"; import { createHash } from "node:crypto"; import { createReadStream } from "node:fs"; import { stat } from "node:fs/promises"; import { basename, extname, join } from "node:path"; import { globby } from "globby"; +import pMap from "p-map"; import { InvalidInputError } from "@/core/errors.js"; import type { AssetFile, @@ -59,6 +61,64 @@ function getAssetContentType(filePath: string): string { ); } +/** + * Every file a build emitted, sorted. One rule for both lanes: `.assetsignore` + * with full gitignore semantics, plus the names no build ever ships. + */ +async function walkBuildOutput(outputDir: string): Promise { + // globby returns forward-slash paths on every platform. Never pass `ignore` + // alongside `ignoreFiles`: globby globs for ignore files using that option, so + // it would find none and silently apply no patterns — hence the filter below. + const found = await globby("**/*", { + cwd: outputDir, + dot: true, + onlyFiles: true, + followSymbolicLinks: false, + ignoreFiles: [ASSETS_IGNORE_FILE], + }); + return found.filter((path) => !ALWAYS_IGNORED.has(basename(path))).sort(); +} + +/** One file a build emitted, located and sized. What names it is the caller's. */ +interface BuildFile { + /** Build-relative, forward slashes, no leading "/". */ + path: string; + absolutePath: string; + size: number; +} + +/** Bounded so a large build cannot flood the libuv thread pool. */ +const STAT_CONCURRENCY = 32; + +/** + * Every file {@link walkBuildOutput} found, located and sized. No hash: the two + * lanes' hashes are different values and must never become one. + */ +export async function describeBuildOutput( + outputDir: string, +): Promise { + const relativePaths = await walkBuildOutput(outputDir); + return await pMap( + relativePaths, + async (path) => { + const absolutePath = join(outputDir, ...path.split("/")); + return { path, absolutePath, size: (await stat(absolutePath)).size }; + }, + { concurrency: STAT_CONCURRENCY }, + ); +} + +/** Stream a file through a hash, so a large one never lands in memory whole. */ +export async function hashFileInto( + hash: Hash, + absolutePath: string, +): Promise { + for await (const chunk of createReadStream(absolutePath)) { + hash.update(chunk); + } + return hash; +} + /** * First 32 hex chars of sha256(utf8(app_id) || raw file bytes). The app-id salt * means a tenant can only collide with their own files, so a malicious upload @@ -80,10 +140,10 @@ async function hashAssetFile( appId: string, absolutePath: string, ): Promise { - const hash = createHash("sha256").update(Buffer.from(appId, "utf8")); - for await (const chunk of createReadStream(absolutePath)) { - hash.update(chunk); - } + const hash = await hashFileInto( + createHash("sha256").update(Buffer.from(appId, "utf8")), + absolutePath, + ); return hash.digest("hex").slice(0, 32); } @@ -100,30 +160,15 @@ export async function buildAssetManifest( const manifest: Record = {}; const filesByHash = new Map(); - // globby returns forward-slash paths on every platform, which is how the - // manifest keys them. Never pass `ignore` alongside `ignoreFiles`: globby - // globs for ignore files using that option, so it would find none and - // silently apply no patterns — hence the filter below. - const found = await globby("**/*", { - cwd: assetsDir, - dot: true, - onlyFiles: true, - followSymbolicLinks: false, - ignoreFiles: [ASSETS_IGNORE_FILE], - }); - const relativeFilePaths = found.filter( - (path) => !ALWAYS_IGNORED.has(basename(path)), - ); + const files = await describeBuildOutput(assetsDir); - if (relativeFilePaths.length > MAX_ASSET_COUNT) { + if (files.length > MAX_ASSET_COUNT) { throw new InvalidInputError( - `Too many static assets: found ${relativeFilePaths.length}, the limit is ${MAX_ASSET_COUNT} files.`, + `Too many static assets: found ${files.length}, the limit is ${MAX_ASSET_COUNT} files.`, ); } - for (const relativePath of relativeFilePaths.sort()) { - const absolutePath = join(assetsDir, ...relativePath.split("/")); - const { size } = await stat(absolutePath); + for (const { path: relativePath, absolutePath, size } of files) { const hash = await hashAssetFile(appId, absolutePath); manifest[`/${relativePath}`] = { hash, size }; diff --git a/packages/cli/src/core/site/schema.ts b/packages/cli/src/core/site/schema.ts index f0883f75..0c631286 100644 --- a/packages/cli/src/core/site/schema.ts +++ b/packages/cli/src/core/site/schema.ts @@ -84,6 +84,13 @@ export interface PresignedAssetUpload { contentLength: number; /** Presigned S3 URL — the URL itself is the credential. */ url: string; + /** + * Base64 sha256 the server signed in, when it signed one. Sent as + * `x-amz-checksum-sha256`, which is what makes S3 itself reject a body that + * is not the declared one. Absent on the legacy static lane, whose URLs pin + * only type and length. + */ + checksumSha256?: string; } /** diff --git a/packages/cli/src/core/site/upload.ts b/packages/cli/src/core/site/upload.ts index cdc266a3..8a9152be 100644 --- a/packages/cli/src/core/site/upload.ts +++ b/packages/cli/src/core/site/upload.ts @@ -218,14 +218,33 @@ async function uploadPresignedAsset( `Server requested upload of unknown asset path: ${upload.path}`, ); } - const content = await readFile(file.absolutePath); + await putPresigned(upload, file.absolutePath); +} + +/** + * PUT one file to the URL the server signed for it. Which file that is, is the + * caller's to decide: the site lane resolves it by path, the version lane by + * the order it declared, because a version can declare two sets whose paths + * overlap. + */ +export async function putPresigned( + upload: PresignedAssetUpload, + absolutePath: string, +): Promise { + const content = await readFile(absolutePath); try { await ky.put(upload.url, { body: new Uint8Array(content), - // The server signed this exact Content-Type into the URL — deriving - // our own value would 403 on any mapping difference. - headers: { "Content-Type": upload.contentType }, + headers: { + // The server signed these exact values into the URL — deriving our own + // would 403 on any mapping difference, and the checksum is what makes + // S3 refuse a body other than the one that was declared. + "Content-Type": upload.contentType, + ...(upload.checksumSha256 + ? { "x-amz-checksum-sha256": upload.checksumSha256 } + : {}), + }, timeout: 120_000, retry: UPLOAD_RETRY, }); diff --git a/packages/cli/src/core/version/api.ts b/packages/cli/src/core/version/api.ts new file mode 100644 index 00000000..02f48c0b --- /dev/null +++ b/packages/cli/src/core/version/api.ts @@ -0,0 +1,241 @@ +import type { KyResponse, RetryOptions } from "ky"; +import pMap from "p-map"; +import type { ZodType } from "zod"; +import { getAppClient } from "@/core/clients/index.js"; +import { + ApiError, + InternalError, + SchemaValidationError, +} from "@/core/errors.js"; +import { putPresigned } from "@/core/site/upload.js"; +import type { ResolvedAssetsConfig } from "@/core/site/wrangler-config.js"; +import type { + ArtifactFile, + ArtifactSet, + CreateVersionProgress, + CreateVersionResponse, + EnvironmentResponse, + WorkerModuleArtifact, +} from "@/core/version/schema.js"; +import { + CreateVersionResponseSchema, + DeclareVersionResponseSchema, + EnvironmentResponseSchema, +} from "@/core/version/schema.js"; + +/** + * Measured on the sandbox's pipe: 3 needed ~930s of the ~450s a build leaves for + * a 25.5k-asset app; 16 failed a degraded pipe on 2026-07-02. + */ +export const DEFAULT_VERSION_UPLOAD_CONCURRENCY = 8; +export const MAX_VERSION_UPLOAD_CONCURRENCY = 16; + +function declaredFile({ path, size, digest }: ArtifactFile) { + return { path, size, digest }; +} + +function declaredModule(module: WorkerModuleArtifact) { + return { ...declaredFile(module), type: module.type }; +} + +/** + * Wrangler's own `assets` block, back in its own snake_case. The FIELDS are + * theirs; what contains them is ours, named for what it decides. + * + * A config that states no setting collapses to `null`, the same as no config at + * all: a bare `assets: { directory }` and an absent block describe the identical + * Worker, and recording them apart would split one version into two. + */ +function declaredServingConfig(config: ResolvedAssetsConfig | null) { + const stated = { + html_handling: config?.htmlHandling ?? null, + not_found_handling: config?.notFoundHandling ?? null, + run_worker_first: config?.runWorkerFirst ?? null, + headers: config?.headers ?? null, + redirects: config?.redirects ?? null, + }; + return Object.values(stated).every((v) => v === null) ? null : stated; +} + +async function post( + path: string, + json: unknown, + doing: string, +): Promise { + try { + return await getAppClient().post(path, { json, timeout: 180_000 }); + } catch (error) { + throw await ApiError.fromHttpError(error, doing); + } +} + +/** + * Ky retries neither PATCH nor a POST by default, and rightly: a repeat is a + * second request unless something makes it the same one. An idempotency key is + * exactly that — the server answers a repeated key with the publication it + * already made — so retrying is only safe WITH one, and this is only ever + * passed then. + */ +const KEYED_RETRY: RetryOptions = { + limit: 3, + methods: ["patch"], + statusCodes: [408, 500, 502, 503, 504], + // The case the key exists for, and the one ky skips by default: our own + // deadline expiring says nothing about whether the server committed. HTTP 408 + // above is the server reporting a timeout; this is the client giving up on a + // request that may well have landed. + retryOnTimeout: true, +}; + +async function patch( + path: string, + json: unknown, + doing: string, + retry?: RetryOptions, +): Promise { + try { + return await getAppClient().patch(path, { + json, + timeout: 180_000, + ...(retry ? { retry } : {}), + }); + } catch (error) { + throw await ApiError.fromHttpError(error, doing); + } +} + +function parse(schema: ZodType, body: unknown, what: string): T { + const result = schema.safeParse(body); + if (!result.success) { + throw new SchemaValidationError( + `Invalid ${what} response from server`, + result.error, + ); + } + return result.data; +} + +/** + * Declare, upload, then commit — in that order, so an interrupted run leaves + * staged objects that expire rather than a version naming files that are gone. + */ +export async function createVersion( + artifacts: ArtifactSet, + options: { + sourceCommit?: string; + concurrency?: number; + progress?: CreateVersionProgress; + } = {}, +): Promise { + const declared = parse( + DeclareVersionResponseSchema, + await ( + await post( + "versions", + { + assets: artifacts.assets.map(declaredFile), + ...(artifacts.siteWorker + ? { + site_worker: { + main: artifacts.siteWorker.main, + modules: artifacts.siteWorker.modules.map(declaredModule), + compatibility_date: artifacts.siteWorker.compatibilityDate, + compatibility_flags: artifacts.siteWorker.compatibilityFlags, + serving_config: declaredServingConfig( + artifacts.siteWorker.servingConfig, + ), + }, + } + : {}), + entities: artifacts.entities, + agents: artifacts.agents, + source_commit: options.sourceCommit, + }, + "declaring a version", + ) + ).json(), + "declare", + ); + + // Paired by POSITION, in the order the server signed them: assets and modules + // are separate namespaces, so the two may share a path. + const declaredFiles = [ + ...artifacts.assets, + ...(artifacts.siteWorker?.modules ?? []), + ]; + + options.progress?.onDeclared?.({ fileCount: declaredFiles.length }); + + if (declared.uploads.length !== declaredFiles.length) { + throw new InternalError( + `Declared ${declaredFiles.length} files but the server signed ${declared.uploads.length} upload URLs.`, + ); + } + // Necessary, not sufficient — but it catches an order drift here rather than + // as an S3 checksum rejection part way through the uploads. + const drifted = declared.uploads.findIndex( + (upload, index) => upload.path !== declaredFiles[index].path, + ); + if (drifted !== -1) { + throw new InternalError( + `Upload ${drifted} is signed for ${declared.uploads[drifted].path}, but that slot declared ${declaredFiles[drifted].path}.`, + ); + } + + let uploadedFiles = 0; + await pMap( + declared.uploads, + async (upload, index) => { + await putPresigned(upload, declaredFiles[index].absolutePath); + uploadedFiles++; + options.progress?.onUpload?.({ + uploadedFiles, + totalFiles: declared.uploads.length, + }); + }, + { concurrency: options.concurrency ?? DEFAULT_VERSION_UPLOAD_CONCURRENCY }, + ); + + return parse( + CreateVersionResponseSchema, + await ( + await post( + `versions/${encodeURIComponent(declared.sessionId)}/finalize`, + {}, + "creating a version", + ) + ).json(), + "create version", + ); +} + +/** + * Point an environment at a recorded version. An environment serves one version, + * so this is a pointer edit, not a deployment — and an older version is the + * rollback. + */ +export async function setEnvironmentVersion( + environment: string, + versionId: string, + options: { idempotencyKey?: string } = {}, +): Promise { + return parse( + EnvironmentResponseSchema, + await ( + await patch( + `environments/${encodeURIComponent(environment)}`, + { + version_id: versionId, + ...(options.idempotencyKey + ? { idempotency_key: options.idempotencyKey } + : {}), + }, + "setting the environment's version", + // Same body, same key, so a lost response is replayed into the answer + // the server already committed rather than reported as a failure. + options.idempotencyKey ? KEYED_RETRY : undefined, + ) + ).json(), + "environment", + ); +} diff --git a/packages/cli/src/core/version/artifacts.ts b/packages/cli/src/core/version/artifacts.ts new file mode 100644 index 00000000..06ee18b8 --- /dev/null +++ b/packages/cli/src/core/version/artifacts.ts @@ -0,0 +1,193 @@ +import { createHash } from "node:crypto"; +import { join, resolve } from "node:path"; +import { globby } from "globby"; +import pMap from "p-map"; +import { CONFIG_FILE_EXTENSION_GLOB } from "@/core/consts.js"; +import { InvalidInputError } from "@/core/errors.js"; +import type { BuildTarget } from "@/core/project/target.js"; +import { requireOutputDir } from "@/core/project/target.js"; +import type { FullStackBuild } from "@/core/site/full-stack.js"; +import { resolveFullStackBuild } from "@/core/site/full-stack.js"; +import { describeBuildOutput, hashFileInto } from "@/core/site/manifest.js"; +import { pathExists, readJsonFile } from "@/core/utils/fs.js"; +import type { + ArtifactFile, + ArtifactSet, + SiteWorkerArtifact, +} from "@/core/version/schema.js"; + +/** Must match the server's ceiling: declaring more only earns a late rejection. */ +const MAX_FILE_COUNT = 50_000; + +/** One open descriptor per file in flight; an unbounded fan-out hits EMFILE. */ +const HASH_CONCURRENCY = 32; + +/** Served for any unmatched path — but only when the platform is what serves. */ +const ENTRY = "index.html"; + +/** Deliberately not `hashAsset` — see {@link ArtifactFile.digest}. */ +async function digestFile(absolutePath: string): Promise { + const hash = await hashFileInto(createHash("sha256"), absolutePath); + return `sha256:${hash.digest("hex")}`; +} + +/** + * Every file a build emitted, addressed and hashed. An empty directory is an + * empty set, not an error: a Worker that answers every path itself is a complete + * app whose assets directory exists and holds nothing. + * + * `entryFile` is the caller saying this set must be ENTERABLE, which is one + * rule, not two — a set nothing can enter is as broken empty as it is without + * its entry, and the platform is what serves it. + */ +export async function collectBuildOutput( + outputDir: string, + options: { entryFile?: string } = {}, +): Promise { + const found = await describeBuildOutput(outputDir); + + if (found.length > MAX_FILE_COUNT) { + throw new InvalidInputError( + `Too many files: found ${found.length}, the limit is ${MAX_FILE_COUNT}.`, + ); + } + if (options.entryFile) { + if (found.length === 0) { + throw new InvalidInputError( + `No files found in ${outputDir}. Build the site before creating a version.`, + { + hints: [ + { message: "Run 'base44 build' first", command: "base44 build" }, + ], + }, + ); + } + if (!found.some((f) => f.path === options.entryFile)) { + throw new InvalidInputError( + `${outputDir} has no ${options.entryFile}, so nothing could enter the site.`, + ); + } + } + + // Bounded: an unbounded Promise.all hits EMFILE at ~1.5k open descriptors. + return await pMap( + found, + async (file) => ({ ...file, digest: await digestFile(file.absolutePath) }), + { concurrency: HASH_CONCURRENCY }, + ); +} + +/** + * The app's own server, when the framework built one — otherwise `null`. Reads + * through {@link resolveFullStackBuild}, the same call the deploy lane makes. + */ +export async function collectSiteWorker( + projectRoot: string, +): Promise { + const built = await resolveFullStackBuild(projectRoot); + return built ? await describeSiteWorker(built) : null; +} + +/** + * The same description from a build already resolved — so `collectArtifacts`, + * which needs the assets directory too, reads the wrangler config ONCE. A second + * read is a second opinion about what the framework built. + */ +async function describeSiteWorker( + built: FullStackBuild, +): Promise { + const { config, modules } = built; + // By identity, not by position: the platform matches the entry against the + // module NAMES it was sent, and `main` in the config may still carry a "./". + const entry = resolve(config.configDir, config.main); + const main = modules.find((m) => m.absolutePath === entry)?.name; + if (!main) { + throw new InvalidInputError( + `The Worker's entry module ${config.main} is not among the ${modules.length} modules collected from ${config.configDir}.`, + ); + } + + return { + main, + modules: await pMap( + modules, + async ({ name, absolutePath, size, type }) => ({ + path: name, + absolutePath, + size, + digest: await digestFile(absolutePath), + // Carried, not re-derived: the rules that decided it are the wrangler + // config's, and a second guess from the extension would disagree. + type, + }), + { concurrency: HASH_CONCURRENCY }, + ), + compatibilityDate: config.compatibilityDate, + compatibilityFlags: config.compatibilityFlags, + // Read from wrangler's `assets` block, recorded under what it decides. + servingConfig: config.assetsConfig, + }; +} + +/** + * The app's declared entities and agents, raw — deliberately not the validated + * resource readers, whose stricter entity schema refuses real Builder apps. + * + * Keyed the way the platform names the same file: `agents/support/triage.jsonc` + * is `support/triage`. + */ +async function readRawResources(dir: string): Promise> { + if (!(await pathExists(dir))) { + return {}; + } + const files = await globby(`**/*.${CONFIG_FILE_EXTENSION_GLOB}`, { + cwd: dir, + onlyFiles: true, + followSymbolicLinks: false, + }); + const payloads: Record = {}; + for (const relativePath of files.sort()) { + const name = relativePath.replace(/\.jsonc?$/, ""); + payloads[name] = await readJsonFile(join(dir, ...relativePath.split("/"))); + } + return payloads; +} + +export async function collectResources( + configDir: string, + dirs: { entitiesDir: string; agentsDir: string }, +): Promise> { + const [entities, agents] = await Promise.all([ + readRawResources(join(configDir, dirs.entitiesDir)), + readRawResources(join(configDir, dirs.agentsDir)), + ]); + return { entities, agents }; +} + +/** + * Everything one build produced, ready to declare. + * + * One reader, so `publish` and `versions create` cannot disagree about what a + * build left behind — a second would eventually declare a Worker's own files as + * a static bundle, which is the one thing the platform reads as "serve from S3". + */ +export async function collectArtifacts( + target: BuildTarget, +): Promise { + const built = await resolveFullStackBuild(target.root); + const siteWorker = built ? await describeSiteWorker(built) : null; + return { + // One set, from wherever this build put it. The entry rule applies only + // when the PLATFORM serves it: a Worker's own asset settings answer an + // unmatched path, and a Worker that serves nothing is a complete app. + assets: built + ? built.assetsDir + ? await collectBuildOutput(built.assetsDir) + : [] + : await collectBuildOutput(requireOutputDir(target), { + entryFile: ENTRY, + }), + ...(siteWorker ? { siteWorker } : {}), + ...(await collectResources(target.configDir, target)), + }; +} diff --git a/packages/cli/src/core/version/gate.ts b/packages/cli/src/core/version/gate.ts new file mode 100644 index 00000000..0edf3793 --- /dev/null +++ b/packages/cli/src/core/version/gate.ts @@ -0,0 +1,17 @@ +const VERSIONS_API_ENV = "BASE44_VERSIONS_API"; + +/** + * Internal gate for the versions lane, not user-facing yet. With it off + * `publish` and `versions` are not registered at all, so they are absent from + * `--help` rather than half-integrated commands a user can stumble into. + * + * Not `BASE44_DEPLOYMENTS_API`: that selects the legacy transport for + * `site deploy` and the sandbox already sets it, so one var would tie the two + * lanes together. + */ +export function versionsApiEnabled( + env: NodeJS.ProcessEnv = process.env, +): boolean { + const value = env[VERSIONS_API_ENV]; + return value === "1" || value === "true"; +} diff --git a/packages/cli/src/core/version/index.ts b/packages/cli/src/core/version/index.ts new file mode 100644 index 00000000..f56ef0a4 --- /dev/null +++ b/packages/cli/src/core/version/index.ts @@ -0,0 +1,5 @@ +export * from "./api.js"; +export * from "./artifacts.js"; +export * from "./gate.js"; +export * from "./publish.js"; +export * from "./schema.js"; diff --git a/packages/cli/src/core/version/publish.ts b/packages/cli/src/core/version/publish.ts new file mode 100644 index 00000000..de0f7563 --- /dev/null +++ b/packages/cli/src/core/version/publish.ts @@ -0,0 +1,72 @@ +import { randomUUID } from "node:crypto"; +import { tagStep as tagError } from "@/core/errors.js"; +import { createVersion, setEnvironmentVersion } from "@/core/version/api.js"; +import type { + ArtifactSet, + CreateVersionProgress, +} from "@/core/version/schema.js"; + +/** + * Which of the three steps a publish broke in. A failed build, a rejected + * artifact set and a lost publication race need three different responses, so + * the step travels with the failure and out through the `--json` envelope. + */ +type PublishStep = "build" | "create_version" | "deploy"; + +/** The environment a publish points at unless told otherwise. */ +export const DEFAULT_ENVIRONMENT = "production"; + +/** + * {@link tagError}, narrowed to this command's vocabulary — the mechanism is + * shared error infrastructure, the three step names are publish's own. + */ +export async function tagStep( + step: PublishStep, + run: () => Promise, +): Promise { + return await tagError(step, run); +} + +interface PublishResult { + environment: string; + versionId: string; + manifestHash: string; + deploymentId: string; +} + +/** + * Record the artifact set as a version and serve it. The deploy key is generated + * once here, so a lost response reads back the deployment this call already made. + */ +export async function publishVersion( + artifacts: ArtifactSet, + options: { + sourceCommit?: string; + target?: string; + concurrency?: number; + progress?: CreateVersionProgress; + } = {}, +): Promise { + const version = await tagStep("create_version", () => + createVersion(artifacts, { + sourceCommit: options.sourceCommit, + concurrency: options.concurrency, + progress: options.progress, + }), + ); + const environment = await tagStep("deploy", () => + setEnvironmentVersion( + options.target ?? DEFAULT_ENVIRONMENT, + version.versionId, + { + idempotencyKey: randomUUID(), + }, + ), + ); + return { + environment: environment.name, + versionId: environment.versionId, + manifestHash: environment.manifestHash, + deploymentId: environment.deploymentId, + }; +} diff --git a/packages/cli/src/core/version/schema.ts b/packages/cli/src/core/version/schema.ts new file mode 100644 index 00000000..a22820e4 --- /dev/null +++ b/packages/cli/src/core/version/schema.ts @@ -0,0 +1,117 @@ +import { z } from "zod"; +import type { ModuleType } from "@/core/site/schema.js"; +import type { ResolvedAssetsConfig } from "@/core/site/wrangler-config.js"; + +/** + * A file the build produced. `digest` is a full sha256, signed into the upload + * URL so S3 refuses any other body — NOT `hashAsset`, which truncates + * sha256(app id ‖ bytes) to key a provider cache. Conflating them gives a file + * that uploads fine and never dedupes. + */ +export interface ArtifactFile { + /** Build-relative, forward slashes, no leading "/". */ + path: string; + absolutePath: string; + size: number; + digest: string; +} + +/** One of the Worker's own modules. Their own namespace: `index.js` as a module + * is not `index.js` as an asset. */ +export interface WorkerModuleArtifact extends ArtifactFile { + /** + * How the runtime hands this module to its importer. The SAME bytes are a + * string under `text` and an ArrayBuffer under `data`, so this is a setting + * rather than a fact about the file — which is why it travels beside the + * digest instead of being re-derived from the path. + */ + type: ModuleType; +} + +/** + * The app's own server. Its presence is what says the Worker SERVES the set's + * assets, rather than the platform serving them from S3; the settings below are + * part of the Worker's identity, not metadata. + */ +export interface SiteWorkerArtifact { + main: string; + modules: WorkerModuleArtifact[]; + compatibilityDate: string | null; + compatibilityFlags: string[]; + /** + * What happens to a request before this Worker runs — whether it runs at all, + * which asset a path resolves to, what a miss gets. Not the assets' config and + * not the Worker's: it decides how the two are routed between. Two builds + * differing only here are different Workers and must not record as one. + */ + servingConfig: ResolvedAssetsConfig | null; +} + +/** Everything one build produced, as the create-version call describes it. */ +export interface ArtifactSet { + /** The app's files. Who serves them is what `siteWorker` says. */ + assets: ArtifactFile[]; + /** Absent for an app with no server of its own — almost every app. */ + siteWorker?: SiteWorkerArtifact; + /** Raw payloads by name. The server normalizes and hashes them. */ + entities: Record; + agents: Record; +} + +export interface CreateVersionProgress { + onDeclared?: (info: { fileCount: number }) => void; + onUpload?: (progress: { uploadedFiles: number; totalFiles: number }) => void; +} + +export const DeclareVersionResponseSchema = z + .object({ + session_id: z.string(), + uploads: z.array( + z.object({ + path: z.string(), + url: z.string(), + content_type: z.string(), + content_length: z.number(), + checksum_sha256: z.string(), + }), + ), + }) + .transform((data) => ({ + sessionId: data.session_id, + uploads: data.uploads.map((upload) => ({ + path: upload.path, + url: upload.url, + contentType: upload.content_type, + contentLength: upload.content_length, + checksumSha256: upload.checksum_sha256, + })), + })); + +export const CreateVersionResponseSchema = z + .object({ + version_id: z.string(), + /** The identity: unchanged across two builds means unchanged content. */ + manifest_hash: z.string(), + }) + .transform((data) => ({ + versionId: data.version_id, + manifestHash: data.manifest_hash, + })); + +export type CreateVersionResponse = z.infer; + +export const EnvironmentResponseSchema = z + .object({ + name: z.string(), + version_id: z.string(), + manifest_hash: z.string(), + deployment_id: z.string(), + }) + .transform((data) => ({ + name: data.name, + versionId: data.version_id, + manifestHash: data.manifest_hash, + deploymentId: data.deployment_id, + })); + +export type EnvironmentResponse = z.infer; diff --git a/packages/cli/tests/cli/build.spec.ts b/packages/cli/tests/cli/build.spec.ts index bb05dc6c..d0f53f68 100644 --- a/packages/cli/tests/cli/build.spec.ts +++ b/packages/cli/tests/cli/build.spec.ts @@ -35,6 +35,17 @@ describe("build command", () => { t.expectResult(result).toContain("Build failed"); }); + it("needs no credential, so a publish sandbox can build before it holds one", async () => { + await t.givenProject(fixture("with-buildable-site")); + + const result = await t.run("build"); + + t.expectResult(result).toSucceed(); + expect(await t.readProjectFile("build-env.txt")).toBe( + `BUILD_APP=${t.api.appId}`, + ); + }); + it("fails when not in a project directory", async () => { await t.givenLoggedIn({ email: "test@example.com", name: "Test User" }); diff --git a/packages/cli/tests/cli/publish.spec.ts b/packages/cli/tests/cli/publish.spec.ts new file mode 100644 index 00000000..c82e10f9 --- /dev/null +++ b/packages/cli/tests/cli/publish.spec.ts @@ -0,0 +1,319 @@ +import { createHash } from "node:crypto"; +import { describe, expect, it } from "vitest"; +import { fixture, setupCLITests } from "./testkit/index.js"; + +const SESSION = "sess-1"; +const INDEX = "
"; +const APP_JS = "console.log(1)"; + +function sha256(content: string): string { + return `sha256:${createHash("sha256").update(content).digest("hex")}`; +} + +describe("publish command", () => { + const t = setupCLITests(); + + const mockPublishApi = () => { + t.api + .mockVersionDeclare(SESSION) + .mockPresignedUpload("/index.html") + .mockPresignedUpload("/assets/app.js") + .mockVersionFinalize({ version_id: "ver-1", manifest_hash: "sha256:abc" }) + .mockEnvironmentSet({ + name: "production", + version_id: "ver-1", + manifest_hash: "sha256:abc", + deployment_id: "dep-1", + }); + }; + + it("declares every built file by a full sha256 over its bytes", async () => { + t.givenEnv({ BASE44_VERSIONS_API: "1" }); + await t.givenLoggedInWithProject(fixture("publishable")); + mockPublishApi(); + + const result = await t.run("publish", "--no-build"); + + t.expectResult(result).toSucceed(); + expect(t.api.versionDeclareRequests[0]).toMatchObject({ + assets: [ + { path: "assets/app.js", size: APP_JS.length, digest: sha256(APP_JS) }, + { path: "index.html", size: INDEX.length, digest: sha256(INDEX) }, + ], + }); + }); + + it("sends the app's resources raw, keyed the way the platform names them", async () => { + t.givenEnv({ BASE44_VERSIONS_API: "1" }); + await t.givenLoggedInWithProject(fixture("publishable")); + mockPublishApi(); + + const result = await t.run("publish", "--no-build"); + + t.expectResult(result).toSucceed(); + expect(t.api.versionDeclareRequests[0]).toMatchObject({ + entities: { + Todo: { + name: "Todo", + type: "object", + properties: { title: { type: "string" } }, + // Passed through untouched: the platform's validation is + // authoritative, and the CLI's own entity schema would refuse this. + unknown_builder_field: true, + }, + }, + agents: { helper: { name: "helper", instructions: "help" } }, + }); + }); + + it("uploads every declared file with the checksum the server signed in", async () => { + t.givenEnv({ BASE44_VERSIONS_API: "1" }); + await t.givenLoggedInWithProject(fixture("publishable")); + mockPublishApi(); + + const result = await t.run("publish", "--no-build"); + + t.expectResult(result).toSucceed(); + const uploaded = t.api.presignedUploadRequests; + expect(uploaded.map((u) => u.path).sort()).toEqual([ + "/assets/app.js", + "/index.html", + ]); + expect( + uploaded.find((u) => u.path === "/index.html")?.data.toString(), + ).toBe(INDEX); + }); + + it("names the environment in the path and the version in the body", async () => { + // Everything else — the app, the principal, env vars — is the platform's to + // resolve, and there is deliberately no field for any of them. + t.givenEnv({ BASE44_VERSIONS_API: "1" }); + await t.givenLoggedInWithProject(fixture("publishable")); + mockPublishApi(); + + const result = await t.run("publish", "--no-build"); + + t.expectResult(result).toSucceed(); + expect(t.api.environmentNames).toEqual(["production"]); + expect( + Object.keys(t.api.versionDeployRequests[0] as object).sort(), + ).toEqual(["idempotency_key", "version_id"]); + }); + + it("emits both references in the --json envelope", async () => { + // Its own field names: `site deploy`'s `deploymentId` means a Cloudflare + // script on the legacy lane, and a caller that could not tell the two apart + // would publish by accident. + t.givenEnv({ BASE44_VERSIONS_API: "1" }); + await t.givenLoggedInWithProject(fixture("publishable")); + mockPublishApi(); + + const result = await t.run("publish", "--no-build", "--json"); + + t.expectResult(result).toSucceed(); + expect(JSON.parse(result.stdout)).toEqual({ + environment: "production", + versionId: "ver-1", + manifestHash: "sha256:abc", + deploymentId: "dep-1", + }); + }); + + it("names the step that failed", async () => { + // A user's build failing, a rejected artifact set and a lost publication + // race are three incidents with three responses. + t.givenEnv({ BASE44_VERSIONS_API: "1" }); + await t.givenLoggedInWithProject(fixture("publishable")); + t.api.mockVersionDeclareError({ + status: 409, + body: { message: "this app declares 3 backend functions" }, + }); + + const result = await t.run("publish", "--no-build", "--json"); + + t.expectResult(result).toFail(); + expect(JSON.parse(result.stdout)).toMatchObject({ + step: "create_version", + statusCode: 409, + }); + }); + + it("names create_version when the output directory is missing", async () => { + // Local validation is part of producing the version. Before, this emitted an + // envelope with no `step`, so a sandbox could not tell a rejected build + // output from a transport failure. + t.givenEnv({ BASE44_VERSIONS_API: "1" }); + await t.givenLoggedInWithProject(fixture("publishable")); + + const result = await t.run( + "publish", + "--no-build", + "--output-dir", + "does-not-exist", + "--json", + ); + + t.expectResult(result).toFail(); + expect(JSON.parse(result.stdout).step).toBe("create_version"); + }); + + it("builds first unless told not to", async () => { + t.givenEnv({ BASE44_VERSIONS_API: "1" }); + await t.givenLoggedInWithProject(fixture("publishable")); + mockPublishApi(); + + const result = await t.run("publish"); + + t.expectResult(result).toSucceed(); + expect(await t.readProjectFile("build-env.txt")).toBe( + `BUILD_APP=${t.api.appId}`, + ); + }); +}); + +describe("versions deploy points an environment", () => { + const t = setupCLITests(); + + it("serves an existing version with no build and no upload", async () => { + // Which is also what a rollback is: the same call with an older version id. + t.givenEnv({ BASE44_VERSIONS_API: "1" }); + await t.givenLoggedInWithProject(fixture("publishable")); + t.api.mockEnvironmentSet({ + name: "production", + version_id: "ver-old", + manifest_hash: "sha256:old", + deployment_id: "dep-9", + }); + + const result = await t.run("versions", "deploy", "ver-old", "--json"); + + t.expectResult(result).toSucceed(); + expect(t.api.environmentNames).toEqual(["production"]); + expect(t.api.presignedUploadRequests).toEqual([]); + expect(JSON.parse(result.stdout)).toEqual({ + name: "production", + versionId: "ver-old", + manifestHash: "sha256:old", + deploymentId: "dep-9", + }); + }); +}); + +describe("the versions lane is gated", () => { + const t = setupCLITests(); + + it("does not exist with the gate off", async () => { + // Not hidden — absent. A command that runs but is unlisted is discoverable + // by anyone who reads the source, and cannot be un-shipped once someone + // scripts against it. + await t.givenLoggedInWithProject(fixture("publishable")); + + for (const argv of [["publish"], ["versions", "deploy", "ver-1"]]) { + const result = await t.run(...argv); + + t.expectResult(result).toFail(); + t.expectResult(result).toContain("unknown command"); + } + }); + + it("is absent from --help with the gate off", async () => { + await t.givenLoggedInWithProject(fixture("publishable")); + + const result = await t.run("--help"); + + expect(result.stdout).not.toContain("publish"); + expect(result.stdout).not.toContain("versions"); + }); + + it("appears once the gate is on", async () => { + t.givenEnv({ BASE44_VERSIONS_API: "1" }); + await t.givenLoggedInWithProject(fixture("publishable")); + + const result = await t.run("--help"); + + expect(result.stdout).toContain("publish"); + expect(result.stdout).toContain("versions"); + }); +}); + +describe("publish command, for an app with a server of its own", () => { + const t = setupCLITests(); + + // The Worker's own build directory, and the assets directory it serves from. + const SERVER_INDEX = + 'import handler from "./assets/chunk-abc.js";\nexport default { fetch: handler };\n'; + const CLIENT_INDEX = "

Hello

\n"; + + const mockFullStackApi = () => + t.api + .mockVersionDeclare(SESSION) + .mockPresignedUpload("/index.html") + .mockPresignedUpload("/assets/app-123.js") + .mockPresignedUpload("/index.js") + .mockPresignedUpload("/index.js.map") + .mockPresignedUpload("/assets/chunk-abc.js") + .mockVersionFinalize({ version_id: "ver-1", manifest_hash: "sha256:abc" }) + .mockEnvironmentSet({ + name: "production", + version_id: "ver-1", + manifest_hash: "sha256:abc", + deployment_id: "dep-1", + }); + + async function publish() { + t.givenEnv({ BASE44_VERSIONS_API: "1" }); + await t.givenLoggedInWithProject(fixture("fullstack-project")); + mockFullStackApi(); + return await t.run("publish", "--no-build"); + } + + it("declares the Worker alongside the frontend, in one version", async () => { + // One build, one source commit, one version — the app's assets and the + // app's server are the same app. + const result = await publish(); + + t.expectResult(result).toSucceed(); + expect(t.api.versionDeclareRequests[0]).toMatchObject({ + site_worker: { + main: "index.js", + compatibility_date: "2025-04-01", + compatibility_flags: ["nodejs_compat"], + }, + }); + }); + + it("takes the assets from the Worker's own directory", async () => { + // Collecting the project's build output instead would ask the platform to + // serve the Worker's files from S3, past every route it owns. + const result = await publish(); + + t.expectResult(result).toSucceed(); + const declared = t.api.versionDeclareRequests[0] as { + assets: Array<{ path: string }>; + site_worker: { modules: Array<{ path: string }> }; + }; + expect(declared.assets.map((f) => f.path)).toEqual([ + "assets/app-123.js", + "index.html", + ]); + expect(declared.site_worker.modules.map((m) => m.path).sort()).toEqual([ + "assets/chunk-abc.js", + "index.js", + "index.js.map", + ]); + }); + + it("puts each declared file's own bytes at the URL signed for it", async () => { + // Paired by position, not by path: `index.js` names a module here and an + // asset in other builds, and swapping the two would upload each under the + // other's checksum. + const result = await publish(); + + t.expectResult(result).toSucceed(); + const uploaded = new Map( + t.api.presignedUploadRequests.map((u) => [u.path, u.data.toString()]), + ); + expect(uploaded.get("/index.js")).toBe(SERVER_INDEX); + expect(uploaded.get("/index.html")).toBe(CLIENT_INDEX); + }); +}); diff --git a/packages/cli/tests/cli/testkit/TestAPIServer.ts b/packages/cli/tests/cli/testkit/TestAPIServer.ts index 83123a30..4fd549c1 100644 --- a/packages/cli/tests/cli/testkit/TestAPIServer.ts +++ b/packages/cli/tests/cli/testkit/TestAPIServer.ts @@ -358,7 +358,7 @@ interface ErrorResponse { // ─── ROUTE HANDLER TYPES ───────────────────────────────────── -type Method = "GET" | "POST" | "PUT" | "DELETE"; +type Method = "GET" | "POST" | "PUT" | "PATCH" | "DELETE"; interface RouteEntry { method: Method; @@ -436,6 +436,7 @@ export class TestAPIServer { | "get" | "post" | "put" + | "patch" | "delete"; this.app[method](entry.path, entry.handler); } @@ -851,6 +852,91 @@ export class TestAPIServer { return this; } + // ─── VERSION ENDPOINTS ──────────────────────────────────── + + /** Captured JSON bodies of POST versions (declare) requests. */ + readonly versionDeclareRequests: unknown[] = []; + /** Captured JSON bodies of PATCH environments/{name} requests. */ + readonly versionDeployRequests: unknown[] = []; + /** Captured environment names the PATCH addressed. */ + readonly environmentNames: string[] = []; + + /** + * Mock POST /api/apps/{appId}/versions. `uploads` is built from the declared + * files, pointed at this server's own presigned-style PUT targets, so a test + * exercises the real declare -> upload -> finalize order. + */ + mockVersionDeclare(sessionId: string): this { + this.pendingRoutes.push({ + method: "POST", + path: `/api/apps/${this.appId}/versions`, + handler: (req, res) => { + const body = req.body as { + assets: Array<{ path: string; size: number; digest: string }>; + site_worker?: { + modules: Array<{ path: string; size: number; digest: string }>; + assets: Array<{ path: string; size: number; digest: string }>; + }; + }; + this.versionDeclareRequests.push(body); + // The app's assets, then the Worker's modules — the slot order the + // server signs them in, which is what the client pairs uploads against. + const declared = [...body.assets, ...(body.site_worker?.modules ?? [])]; + res.status(200).json({ + session_id: sessionId, + uploads: declared.map((file) => ({ + path: file.path, + url: `${this.baseUrl}/presigned/${file.path}`, + content_type: "application/octet-stream", + content_length: file.size, + checksum_sha256: Buffer.from( + file.digest.replace("sha256:", ""), + "hex", + ).toString("base64"), + })), + }); + }, + }); + return this; + } + + mockVersionFinalize(response: { + version_id: string; + manifest_hash: string; + }): this { + return this.addRoute( + "POST", + `/api/apps/${this.appId}/versions/:sessionId/finalize`, + response, + ); + } + + mockEnvironmentSet(response: { + name: string; + version_id: string; + manifest_hash: string; + deployment_id: string; + }): this { + this.pendingRoutes.push({ + method: "PATCH", + path: `/api/apps/${this.appId}/environments/:name`, + handler: (req, res) => { + this.versionDeployRequests.push(req.body); + this.environmentNames.push(String(req.params.name)); + res.status(200).json(response); + }, + }); + return this; + } + + mockVersionDeclareError(error: ErrorResponse): this { + return this.addErrorRoute( + "POST", + `/api/apps/${this.appId}/versions`, + error, + ); + } + /** Mock the Cloudflare assets endpoint to always fail with the given error. */ mockAssetUploadError(error: ErrorResponse): this { return this.addErrorRoute("POST", "/cf-assets/upload", error); diff --git a/packages/cli/tests/core/project-target.spec.ts b/packages/cli/tests/core/project-target.spec.ts new file mode 100644 index 00000000..29b1b82b --- /dev/null +++ b/packages/cli/tests/core/project-target.spec.ts @@ -0,0 +1,118 @@ +import { + mkdir, + mkdtemp, + readdir, + readFile, + rm, + writeFile, +} from "node:fs/promises"; +import { tmpdir } from "node:os"; +import { join, resolve } from "node:path"; +import { afterEach, beforeEach, describe, expect, it } from "vitest"; +import { SchemaValidationError } from "@/core/errors.js"; +import { requireOutputDir, resolveBuildTarget } from "@/core/project/target.js"; + +describe("resolveBuildTarget", () => { + let root: string; + + beforeEach(async () => { + root = await mkdtemp(join(tmpdir(), "b44-project-")); + }); + + afterEach(async () => { + await rm(root, { recursive: true, force: true }); + }); + + async function writeConfig(config: unknown): Promise { + await mkdir(join(root, "base44"), { recursive: true }); + await writeFile( + join(root, "base44", "config.jsonc"), + JSON.stringify(config), + ); + } + + it("supplies a Builder repo's missing defaults", async () => { + // Builder repos carry no CLI config at all, and `base44 build` used to throw + // ConfigNotFoundError on one. + const target = await resolveBuildTarget(root); + + expect(target).toEqual({ + root, + configDir: join(root, "base44"), + buildCommand: "npm run build", + outputDir: resolve(root, "dist"), + entitiesDir: "entities", + agentsDir: "agents", + }); + }); + + it("does not guess for a config that is present and omits a field", async () => { + // Omitting one is a deliberate statement, and every project that relies on + // the existing error still gets it. Only a wholly absent config is defaulted. + await writeConfig({ name: "my-app" }); + + const target = await resolveBuildTarget(root, { outputDir: "dist" }); + + expect(target.buildCommand).toBeUndefined(); + }); + + it("names no output directory when a present config names none", async () => { + // Reported at the point of collection, not here: a project missing both is + // told about its build command first, which is the one it hits first. + await writeConfig({ + name: "my-app", + site: { buildCommand: "npm run build" }, + }); + + const target = await resolveBuildTarget(root); + + expect(target.outputDir).toBeNull(); + expect(() => requireOutputDir(target)).toThrow( + /No site configuration found/, + ); + }); + + it("writes nothing", async () => { + // The sandbox used to overwrite base44/config.jsonc with a minimal one, + // destroying any checked-in configuration — and, for a full-stack app, its + // build command. + await resolveBuildTarget(root); + + expect(await readdir(root)).toEqual([]); + }); + + it("leaves a checked-in config exactly as it was", async () => { + const config = { + name: "my-app", + site: { buildCommand: "pnpm build", outputDirectory: "build" }, + }; + await writeConfig(config); + const before = await readFile(join(root, "base44", "config.jsonc"), "utf8"); + + const target = await resolveBuildTarget(root); + + expect(await readFile(join(root, "base44", "config.jsonc"), "utf8")).toBe( + before, + ); + expect(target.buildCommand).toBe("pnpm build"); + expect(target.outputDir).toBe(resolve(root, "build")); + }); + + it("honors an explicit output directory over both", async () => { + await writeConfig({ name: "my-app", site: { outputDirectory: "out" } }); + + const target = await resolveBuildTarget(root, { outputDir: "elsewhere" }); + + expect(target.outputDir).toBe(resolve(root, "elsewhere")); + }); + + it("still fails on a config that is present and invalid", async () => { + // Only a MISSING config is defaulted. Publishing past a broken one is how a + // typo becomes a version built the wrong way. + await writeConfig({ site: { buildCommand: 42 } }); + + await expect(resolveBuildTarget(root)).rejects.toBeInstanceOf( + SchemaValidationError, + ); + }); +}); diff --git a/packages/cli/tests/core/require-app.spec.ts b/packages/cli/tests/core/require-app.spec.ts new file mode 100644 index 00000000..ef74ffd5 --- /dev/null +++ b/packages/cli/tests/core/require-app.spec.ts @@ -0,0 +1,16 @@ +import { describe, expect, it } from "vitest"; +import { requireApp } from "@/cli/utils/command/middleware.js"; +import { InternalError } from "@/core/errors.js"; + +describe("requireApp", () => { + it("hands back the app the lifecycle resolved", () => { + expect(requireApp({ app: { id: "app-1" } }).id).toBe("app-1"); + }); + + it("raises rather than letting a command run without one", () => { + // Unreachable for a command that did not opt out of app context — which is + // the point: the alternative was defaulting the id to "", and Vite inlines + // that into a dist whose SDK addresses no app, with nothing failing. + expect(() => requireApp({ app: undefined })).toThrow(InternalError); + }); +}); diff --git a/packages/cli/tests/core/site-full-stack.spec.ts b/packages/cli/tests/core/site-full-stack.spec.ts new file mode 100644 index 00000000..ae4f82f8 --- /dev/null +++ b/packages/cli/tests/core/site-full-stack.spec.ts @@ -0,0 +1,68 @@ +import { mkdir, mkdtemp, rm, writeFile } from "node:fs/promises"; +import { tmpdir } from "node:os"; +import { join } from "node:path"; +import { afterEach, beforeEach, describe, expect, it } from "vitest"; +import { resolveFullStackBuild } from "@/core/site/full-stack.js"; + +describe("resolveFullStackBuild", () => { + let projectRoot: string; + let distDir: string; + + async function writeBuild(config: Record = {}) { + await mkdir(join(distDir, "client"), { recursive: true }); + await writeFile(join(distDir, "client", "index.html"), "

Hi

\n"); + await writeFile(join(distDir, "index.js"), "export default {};"); + await writeFile( + join(distDir, "wrangler.json"), + JSON.stringify({ + main: "index.js", + no_bundle: true, + rules: [{ type: "ESModule", globs: ["**/*.js"] }], + assets: { directory: "./client" }, + ...config, + }), + ); + await mkdir(join(projectRoot, ".wrangler", "deploy"), { recursive: true }); + await writeFile( + join(projectRoot, ".wrangler", "deploy", "config.json"), + JSON.stringify({ configPath: "../../dist/wrangler.json" }), + ); + } + + beforeEach(async () => { + projectRoot = await mkdtemp(join(tmpdir(), "b44-fs-")); + distDir = join(projectRoot, "dist"); + }); + + afterEach(async () => { + await rm(projectRoot, { recursive: true, force: true }); + }); + + it("reports nothing for a project that built a plain static site", async () => { + expect(await resolveFullStackBuild(projectRoot)).toBeNull(); + }); + + it("answers with the config, the modules and the assets directory", async () => { + await writeBuild(); + + const built = await resolveFullStackBuild(projectRoot); + + expect(built?.config.main).toBe("index.js"); + expect(built?.modules.map((m) => m.name)).toEqual(["index.js"]); + expect(built?.assetsDir).toBe(join(distDir, "client")); + }); + + it("reports no assets directory when the config declares none", async () => { + await writeBuild({ assets: undefined }); + + expect((await resolveFullStackBuild(projectRoot))?.assetsDir).toBeNull(); + }); + + it("reports no assets directory when the build produced none", async () => { + // Declared but absent: the Worker answers every path itself, which is a + // complete app — not a build to refuse. + await writeBuild({ assets: { directory: "./nothing-here" } }); + + expect((await resolveFullStackBuild(projectRoot))?.assetsDir).toBeNull(); + }); +}); diff --git a/packages/cli/tests/core/version-api.spec.ts b/packages/cli/tests/core/version-api.spec.ts new file mode 100644 index 00000000..c2901650 --- /dev/null +++ b/packages/cli/tests/core/version-api.spec.ts @@ -0,0 +1,134 @@ +import ky from "ky"; +import { beforeEach, describe, expect, it, vi } from "vitest"; +import { setEnvironmentVersion } from "../../src/core/version/api.js"; + +/** + * A real ky client over a fake network, so these exercise ky's own retry loop + * rather than asserting the options we handed it. Configuration that reads + * correctly and retries nothing is exactly the bug under test. + */ +const fakeFetch = vi.fn(); +vi.mock("../../src/core/clients/index.js", async (importOriginal) => { + const actual = + await importOriginal(); + return { + ...actual, + getAppClient: () => + ky.create({ + prefixUrl: "https://api.test/api/apps/app-1/", + fetch: (...args: unknown[]) => fakeFetch(...args), + }), + }; +}); + +const DEPLOYED = { + name: "production", + version_id: "ver-1", + manifest_hash: "sha256:abc", + deployment_id: "dep-1", +}; + +function committed(): Response { + return new Response(JSON.stringify(DEPLOYED), { + status: 200, + headers: { "content-type": "application/json" }, + }); +} + +function unavailable(): Response { + return new Response("", { status: 503 }); +} + +/** What ky raises when OUR deadline expires — which says nothing about the server. */ +function clientTimeout(): Error { + const error = new Error("Request timed out"); + error.name = "TimeoutError"; + return error; +} + +/** Read as each call is made: by assertion time the request body is consumed. */ +const bodiesSent: unknown[] = []; +let plan: Array<() => Response | Error> = []; + +/** What the network answers, in order. The last entry answers every call after. */ +function answers(...entries: Array<() => Response | Error>): void { + plan = entries; +} + +describe("setEnvironmentVersion", () => { + beforeEach(() => { + fakeFetch.mockReset(); + bodiesSent.length = 0; + plan = [committed]; + fakeFetch.mockImplementation(async (request: Request) => { + bodiesSent.push(await request.clone().json()); + const answer = (plan.length > 1 ? plan.shift() : plan[0]) as () => + | Response + | Error; + const result = answer(); + if (result instanceof Error) { + throw result; + } + return result; + }); + }); + + it("replays a keyed deploy whose response never arrived", async () => { + // The case the key exists for: the server may well have committed, so the + // second attempt is answered from the publication it already made rather + // than making a second one. + answers(clientTimeout, committed); + + const answer = await setEnvironmentVersion("production", "ver-1", { + idempotencyKey: "key-1", + }); + + expect(answer.deploymentId).toBe("dep-1"); + expect(fakeFetch).toHaveBeenCalledTimes(2); + }); + + it("replays it with the same key, or the replay is a second publish", async () => { + answers(clientTimeout, committed); + + await setEnvironmentVersion("production", "ver-1", { + idempotencyKey: "key-1", + }); + + expect(bodiesSent[0]).toEqual({ + version_id: "ver-1", + idempotency_key: "key-1", + }); + expect(bodiesSent[1]).toEqual(bodiesSent[0]); + }); + + it("replays a keyed deploy the server failed to answer", async () => { + answers(unavailable, committed); + + const answer = await setEnvironmentVersion("production", "ver-1", { + idempotencyKey: "key-1", + }); + + expect(answer.deploymentId).toBe("dep-1"); + expect(fakeFetch).toHaveBeenCalledTimes(2); + }); + + it("does not replay without a key, because a repeat is a second publish", async () => { + // Which is exactly what a deliberate redeploy is — so a dropped packet must + // not quietly become two publications. + answers(clientTimeout); + + await expect( + setEnvironmentVersion("production", "ver-1"), + ).rejects.toThrow(); + expect(fakeFetch).toHaveBeenCalledTimes(1); + }); + + it("gives up rather than replaying forever", async () => { + answers(clientTimeout); + + await expect( + setEnvironmentVersion("production", "ver-1", { idempotencyKey: "key-1" }), + ).rejects.toThrow(); + expect(fakeFetch).toHaveBeenCalledTimes(4); + }); +}); diff --git a/packages/cli/tests/core/version-artifacts.spec.ts b/packages/cli/tests/core/version-artifacts.spec.ts new file mode 100644 index 00000000..730013e8 --- /dev/null +++ b/packages/cli/tests/core/version-artifacts.spec.ts @@ -0,0 +1,429 @@ +import { createHash } from "node:crypto"; +import { mkdir, mkdtemp, rm, writeFile } from "node:fs/promises"; +import { tmpdir } from "node:os"; +import { join } from "node:path"; +import { afterEach, beforeEach, describe, expect, it } from "vitest"; +import { InvalidInputError } from "@/core/errors.js"; +import { hashAsset } from "@/core/site/manifest.js"; +import { + collectArtifacts, + collectBuildOutput, + collectResources, + collectSiteWorker, +} from "@/core/version/artifacts.js"; + +/** What `collectArtifacts` requires of the set the PLATFORM serves. */ +const ENTRY = "index.html"; + +function sha256(content: string): string { + return `sha256:${createHash("sha256").update(content).digest("hex")}`; +} + +describe("collectBuildOutput", () => { + let outputDir: string; + + beforeEach(async () => { + outputDir = await mkdtemp(join(tmpdir(), "b44-build-")); + await writeFile(join(outputDir, "index.html"), "

Hello

\n"); + }); + + afterEach(async () => { + await rm(outputDir, { recursive: true, force: true }); + }); + + it("names each file by a full sha256 over its bytes", async () => { + const files = await collectBuildOutput(outputDir); + + expect(files).toEqual([ + { + path: "index.html", + absolutePath: join(outputDir, "index.html"), + size: 15, + digest: sha256("

Hello

\n"), + }, + ]); + }); + + it("is not the provider asset hash, which is salted and truncated", async () => { + // Conflating the two produces a file that uploads fine and never dedupes, + // or a digest check that fails on a correct file. + const [file] = await collectBuildOutput(outputDir); + + expect(file.digest).not.toContain( + hashAsset("app-1", Buffer.from("

Hello

\n")), + ); + expect(file.digest.replace("sha256:", "")).toHaveLength(64); + }); + + it("keys nested files by forward-slash paths with no leading slash", async () => { + // The platform turns each path into a key under the frontend's prefix, and + // refuses a leading slash outright. + await mkdir(join(outputDir, "assets")); + await writeFile(join(outputDir, "assets", "app.js"), "console.log(1);"); + + const paths = (await collectBuildOutput(outputDir)).map((f) => f.path); + + expect(paths).toEqual(["assets/app.js", "index.html"]); + }); + + it("honors .assetsignore", async () => { + await writeFile(join(outputDir, ".assetsignore"), "*.map\n"); + await writeFile(join(outputDir, "app.js.map"), "{}"); + + const paths = (await collectBuildOutput(outputDir)).map((f) => f.path); + + expect(paths).toEqual(["index.html"]); + }); + + it("hashes a large set without exhausting file descriptors", async () => { + // An unbounded Promise.all opens one descriptor per file and dies with + // EMFILE around 1.5k on a default limit — well under the 100k this + // advertises. 1200 is enough to fail the unbounded version reliably. + // Written sequentially: the point under test is the COLLECTOR's fan-out, + // so the fixture must not be what runs out of descriptors. + await mkdir(join(outputDir, "many")); + for (let i = 0; i < 1200; i++) { + await writeFile( + join(outputDir, "many", `f${i}.js`), + `export const x = ${i};`, + ); + } + + const files = await collectBuildOutput(outputDir); + + expect(files).toHaveLength(1201); + expect(new Set(files.map((f) => f.digest)).size).toBe(1201); + }, 30_000); + + it("refuses a set with no entry point when one is required", async () => { + await rm(join(outputDir, "index.html")); + await writeFile(join(outputDir, "app.js"), "console.log(1);"); + + await expect( + collectBuildOutput(outputDir, { entryFile: ENTRY }), + ).rejects.toThrow(/no index\.html/); + }); + + it("refuses an empty output directory when an entry is required", async () => { + // Only for the set the PLATFORM serves. A Worker's assets may be empty: + // the Worker answers every path itself. + await rm(join(outputDir, "index.html")); + + await expect( + collectBuildOutput(outputDir, { entryFile: ENTRY }), + ).rejects.toBeInstanceOf(InvalidInputError); + }); +}); + +describe("collectResources", () => { + let configDir: string; + + beforeEach(async () => { + configDir = await mkdtemp(join(tmpdir(), "b44-config-")); + }); + + afterEach(async () => { + await rm(configDir, { recursive: true, force: true }); + }); + + const dirs = { entitiesDir: "entities", agentsDir: "agents" }; + + it("keys a payload by its path with the schema extension stripped", async () => { + // The same name the platform derives from the same file, so a version made + // here and one made in-process describe the same entity. + await mkdir(join(configDir, "entities")); + await writeFile( + join(configDir, "entities", "Todo.jsonc"), + '{ "name": "Todo", "type": "object" }', + ); + await mkdir(join(configDir, "agents", "support"), { recursive: true }); + await writeFile(join(configDir, "agents", "support", "triage.json"), "{}"); + + const { entities, agents } = await collectResources(configDir, dirs); + + expect(entities).toEqual({ Todo: { name: "Todo", type: "object" } }); + expect(agents).toEqual({ "support/triage": {} }); + }); + + it("sends the payload raw, without the CLI's stricter validation", async () => { + // The platform's extractor and validation are authoritative. Applying the + // CLI's entity schema here is exactly what blocks real Builder apps, which + // is why `site deploy` reads no resources at all. + await mkdir(join(configDir, "entities")); + await writeFile( + join(configDir, "entities", "Odd.jsonc"), + '{ "no_name_field": true, "properties": { "x": { "type": "whatever" } } }', + ); + + const { entities } = await collectResources(configDir, dirs); + + expect(entities).toEqual({ + Odd: { no_name_field: true, properties: { x: { type: "whatever" } } }, + }); + }); + + it("reads an app that declares none as declaring none", async () => { + expect(await collectResources(configDir, dirs)).toEqual({ + entities: {}, + agents: {}, + }); + }); +}); + +describe("collectSiteWorker", () => { + let projectRoot: string; + let distDir: string; + + async function writeFullStackBuild( + config: Record = {}, + ): Promise { + await mkdir(join(distDir, "client"), { recursive: true }); + await writeFile(join(distDir, "client", "index.html"), "

Hi

\n"); + await writeFile(join(distDir, "index.js"), "export default {};"); + await writeFile( + join(distDir, "wrangler.json"), + JSON.stringify({ + main: "index.js", + no_bundle: true, + rules: [{ type: "ESModule", globs: ["**/*.js"] }], + assets: { directory: "./client" }, + ...config, + }), + ); + await mkdir(join(projectRoot, ".wrangler", "deploy"), { recursive: true }); + await writeFile( + join(projectRoot, ".wrangler", "deploy", "config.json"), + JSON.stringify({ configPath: "../../dist/wrangler.json" }), + ); + } + + beforeEach(async () => { + projectRoot = await mkdtemp(join(tmpdir(), "b44-fullstack-")); + distDir = join(projectRoot, "dist"); + }); + + afterEach(async () => { + await rm(projectRoot, { recursive: true, force: true }); + }); + + it("describes a worker for a build that declared no assets", async () => { + // A Worker that answers every path itself is a complete app, not a build + // to refuse. Whether it has assets is the SET's business, not the worker's. + await writeFullStackBuild({ assets: undefined }); + + expect(await collectSiteWorker(projectRoot)).not.toBeNull(); + }); + + it("reports no worker for an app that has no server of its own", async () => { + // Almost every app: there is no redirect file, so there is nothing to read. + expect(await collectSiteWorker(projectRoot)).toBeNull(); + }); + + it("describes each module the same way it describes a frontend file", async () => { + await writeFullStackBuild(); + + const worker = await collectSiteWorker(projectRoot); + + expect(worker?.modules).toEqual([ + { + path: "index.js", + absolutePath: join(distDir, "index.js"), + size: 18, + digest: sha256("export default {};"), + // Not a fact about the bytes: the same file is a string under `text` + // and an ArrayBuffer under `data`, so it rides beside the digest. + type: "esm", + }, + ]); + }); + + it("names the entry as the module set names it, not as the config wrote it", async () => { + // The platform matches `main` against the module names it was sent, so a + // "./" that survived would name a module nothing in the set provides. + await writeFullStackBuild({ main: "./index.js" }); + + const worker = await collectSiteWorker(projectRoot); + + expect(worker?.main).toBe("index.js"); + }); + + it("carries the settings the modules were built for", async () => { + await writeFullStackBuild({ + compatibility_date: "2026-01-01", + compatibility_flags: ["nodejs_compat"], + }); + + const worker = await collectSiteWorker(projectRoot); + + expect(worker?.compatibilityDate).toBe("2026-01-01"); + expect(worker?.compatibilityFlags).toEqual(["nodejs_compat"]); + }); + + it("carries how Cloudflare is told to serve the files the worker owns", async () => { + // These decide whether the worker even runs for a request, so a version + // that dropped them would describe two different workers identically. + await writeFullStackBuild({ + assets: { + directory: "./client", + run_worker_first: true, + not_found_handling: "single-page-application", + html_handling: "force-trailing-slash", + }, + }); + + const worker = await collectSiteWorker(projectRoot); + + expect(worker?.servingConfig).toMatchObject({ + runWorkerFirst: true, + notFoundHandling: "single-page-application", + htmlHandling: "force-trailing-slash", + }); + }); + + it("states no setting when the config states none", async () => { + // The defaults apply, which is not the same claim as any particular value — + // so nothing is filled in with one. + await writeFullStackBuild(); + + const worker = await collectSiteWorker(projectRoot); + + expect(worker?.servingConfig).toEqual({ + htmlHandling: undefined, + notFoundHandling: undefined, + runWorkerFirst: undefined, + headers: undefined, + redirects: undefined, + }); + }); + + it("leaves the assets out of the module set", async () => { + // They are declared as the frontend. Declared twice, they would upload + // twice and land in two different prefixes. + await writeFullStackBuild(); + await writeFile(join(distDir, "client", "app.js"), "console.log(1);"); + + const worker = await collectSiteWorker(projectRoot); + + expect(worker?.modules.map((m) => m.path)).toEqual(["index.js"]); + }); +}); + +describe("who serves the frontend decides what it must contain", () => { + let outputDir: string; + + beforeEach(async () => { + outputDir = await mkdtemp(join(tmpdir(), "b44-serves-")); + await writeFile(join(outputDir, "app.js"), "console.log(1);"); + }); + + afterEach(async () => { + await rm(outputDir, { recursive: true, force: true }); + }); + + it("refuses a set with no entry when the platform is what serves", async () => { + // Any unmatched path is answered with that one file, so without it there is + // nothing to enter. + await expect( + collectBuildOutput(outputDir, { entryFile: ENTRY }), + ).rejects.toThrow(InvalidInputError); + }); + + it("accepts a set with no entry when a Worker serves", async () => { + // A server-rendered app renders its own HTML, and the Worker's asset + // settings decide what an unmatched path gets. + const files = await collectBuildOutput(outputDir); + + expect(files.map((f) => f.path)).toEqual(["app.js"]); + }); + + it("reads an existing but empty directory as an empty set", async () => { + // A Worker that answers every path itself is a complete app. Refusing here + // made it succeed with NO assets directory and fail with an empty one. + const empty = join(outputDir, "nothing"); + await mkdir(empty, { recursive: true }); + + await expect(collectBuildOutput(empty)).resolves.toEqual([]); + }); +}); + +describe("collectArtifacts", () => { + let projectRoot: string; + + async function fullStackProject(): Promise { + const dist = join(projectRoot, "dist"); + await mkdir(join(dist, "client"), { recursive: true }); + await writeFile(join(dist, "client", "index.html"), "

Hi

\n"); + await writeFile(join(dist, "index.js"), "export default {};"); + await writeFile( + join(dist, "wrangler.json"), + JSON.stringify({ + main: "index.js", + no_bundle: true, + rules: [{ type: "ESModule", globs: ["**/*.js"] }], + assets: { directory: "./client" }, + }), + ); + await mkdir(join(projectRoot, ".wrangler", "deploy"), { recursive: true }); + await writeFile( + join(projectRoot, ".wrangler", "deploy", "config.json"), + JSON.stringify({ configPath: "../../dist/wrangler.json" }), + ); + } + + function target() { + return { + root: projectRoot, + configDir: join(projectRoot, "base44"), + outputDir: join(projectRoot, "dist", "client"), + entitiesDir: "entities", + agentsDir: "agents", + }; + } + + beforeEach(async () => { + projectRoot = await mkdtemp(join(tmpdir(), "b44-collect-")); + }); + + afterEach(async () => { + await rm(projectRoot, { recursive: true, force: true }); + }); + + it("takes the assets from the Worker's own directory when one built", async () => { + // Not the project's `site.outputDirectory`. Naming the wrong directory here + // is how a Worker's files get served from S3, past every route it owns — + // which is why every command collects through this one reader. + await fullStackProject(); + + const artifacts = await collectArtifacts(target()); + + expect(artifacts.assets.map((f) => f.path)).toEqual(["index.html"]); + expect(artifacts.assets[0].absolutePath).toBe( + join(projectRoot, "dist", "client", "index.html"), + ); + expect(artifacts.siteWorker).not.toBeUndefined(); + }); + + it("takes them from the build output when no Worker built", async () => { + await mkdir(join(projectRoot, "dist", "client"), { recursive: true }); + await writeFile( + join(projectRoot, "dist", "client", "index.html"), + "

Hi

\n", + ); + + const artifacts = await collectArtifacts(target()); + + expect(artifacts.assets.map((f) => f.path)).toEqual(["index.html"]); + expect(artifacts.siteWorker).toBeUndefined(); + }); + + it("requires an entry only when the platform is what serves", async () => { + // A Worker answers an unmatched path itself; S3 needs an index.html to + // enter. One asset set, two rules, and the Worker's presence picks. + await fullStackProject(); + await rm(join(projectRoot, "dist", "client", "index.html")); + + const artifacts = await collectArtifacts(target()); + + expect(artifacts.assets).toEqual([]); + }); +}); diff --git a/packages/cli/tests/core/version-gate.spec.ts b/packages/cli/tests/core/version-gate.spec.ts new file mode 100644 index 00000000..38eb48f2 --- /dev/null +++ b/packages/cli/tests/core/version-gate.spec.ts @@ -0,0 +1,22 @@ +import { describe, expect, it } from "vitest"; +import { versionsApiEnabled } from "@/core/version/gate.js"; + +describe("versionsApiEnabled", () => { + it("is off unless the env var says otherwise", () => { + expect(versionsApiEnabled({})).toBe(false); + expect(versionsApiEnabled({ BASE44_VERSIONS_API: "" })).toBe(false); + expect(versionsApiEnabled({ BASE44_VERSIONS_API: "0" })).toBe(false); + expect(versionsApiEnabled({ BASE44_VERSIONS_API: "yes" })).toBe(false); + }); + + it("takes the same two values the deployments gate takes", () => { + expect(versionsApiEnabled({ BASE44_VERSIONS_API: "1" })).toBe(true); + expect(versionsApiEnabled({ BASE44_VERSIONS_API: "true" })).toBe(true); + }); + + it("is a separate switch from the deployments lane", () => { + // One var switching both would make them impossible to roll out apart, and + // the build sandbox already sets BASE44_DEPLOYMENTS_API for the legacy arm. + expect(versionsApiEnabled({ BASE44_DEPLOYMENTS_API: "1" })).toBe(false); + }); +}); diff --git a/packages/cli/tests/core/version-publish.spec.ts b/packages/cli/tests/core/version-publish.spec.ts new file mode 100644 index 00000000..ade179cc --- /dev/null +++ b/packages/cli/tests/core/version-publish.spec.ts @@ -0,0 +1,65 @@ +import { describe, expect, it } from "vitest"; +import { ApiError, stepOf } from "@/core/errors.js"; +import { tagStep } from "@/core/version/publish.js"; + +describe("which step a publish broke in", () => { + it("carries the step out with the failure", async () => { + // A user's build failing, an artifact set the platform refused and a lost + // publication race are three incidents with three responses. One exit code + // for all of them is how a sandbox log stops being diagnostic. + await tagStep("create_version", () => + Promise.reject(new ApiError("rejected", { statusCode: 400 })), + ).catch((error) => { + expect(error.message).toBe("rejected"); + expect(stepOf(error)).toBe("create_version"); + }); + }); + + it("tags a callback that throws synchronously too", async () => { + await tagStep("build", () => { + throw new Error("the build command exited 1"); + }).catch((error) => { + expect(stepOf(error)).toBe("build"); + }); + }); + + it("leaves the error otherwise untouched", async () => { + // Wrapping it would cost the envelope the status and request id a caller + // needs to look the failure up server-side. + const original = new ApiError("upstream said no", { + statusCode: 502, + requestId: "req-1", + }); + + await tagStep("deploy", () => Promise.reject(original)).catch((error) => { + expect(error).toBe(original); + expect(error.statusCode).toBe(502); + expect(error.requestId).toBe("req-1"); + }); + }); + + it("keeps the innermost step when steps nest", async () => { + await tagStep("deploy", () => + tagStep("create_version", () => Promise.reject(new Error("inner"))), + ).catch((error) => { + expect(stepOf(error)).toBe("create_version"); + }); + }); + + it("reports no step for a failure that never passed through one", () => { + expect(stepOf(new Error("unrelated"))).toBeUndefined(); + }); +}); + +describe("an error the step cannot be attached to", () => { + it("survives a frozen error rather than replacing it", async () => { + // A library that freezes its errors would otherwise turn the tag into a + // TypeError, losing the message, status and request id entirely. + const frozen = Object.freeze(new ApiError("frozen", { statusCode: 418 })); + + await expect(tagStep("deploy", () => Promise.reject(frozen))).rejects.toBe( + frozen, + ); + expect(stepOf(frozen)).toBeUndefined(); + }); +}); diff --git a/packages/cli/tests/fixtures/publishable/base44/agents/helper.jsonc b/packages/cli/tests/fixtures/publishable/base44/agents/helper.jsonc new file mode 100644 index 00000000..8e24cab1 --- /dev/null +++ b/packages/cli/tests/fixtures/publishable/base44/agents/helper.jsonc @@ -0,0 +1 @@ +{ "name": "helper", "instructions": "help" } diff --git a/packages/cli/tests/fixtures/publishable/base44/config.jsonc b/packages/cli/tests/fixtures/publishable/base44/config.jsonc new file mode 100644 index 00000000..cf8a03d0 --- /dev/null +++ b/packages/cli/tests/fixtures/publishable/base44/config.jsonc @@ -0,0 +1,7 @@ +{ + "name": "Publishable Project", + "site": { + "buildCommand": "node -e \"require('fs').writeFileSync('build-env.txt', 'BUILD_APP=' + process.env.VITE_BASE44_APP_ID)\"", + "outputDirectory": "site-output" + } +} diff --git a/packages/cli/tests/fixtures/publishable/base44/entities/Todo.jsonc b/packages/cli/tests/fixtures/publishable/base44/entities/Todo.jsonc new file mode 100644 index 00000000..d2dad3c8 --- /dev/null +++ b/packages/cli/tests/fixtures/publishable/base44/entities/Todo.jsonc @@ -0,0 +1,8 @@ +{ + // Deliberately not what the CLI's own entity schema accepts: the platform's + // extractor is authoritative, and a strict parse here blocks real Builder apps. + "name": "Todo", + "type": "object", + "properties": { "title": { "type": "string" } }, + "unknown_builder_field": true +} diff --git a/packages/cli/tests/fixtures/publishable/site-output/assets/app.js b/packages/cli/tests/fixtures/publishable/site-output/assets/app.js new file mode 100644 index 00000000..29348283 --- /dev/null +++ b/packages/cli/tests/fixtures/publishable/site-output/assets/app.js @@ -0,0 +1 @@ +console.log(1) \ No newline at end of file diff --git a/packages/cli/tests/fixtures/publishable/site-output/index.html b/packages/cli/tests/fixtures/publishable/site-output/index.html new file mode 100644 index 00000000..06a67d94 --- /dev/null +++ b/packages/cli/tests/fixtures/publishable/site-output/index.html @@ -0,0 +1 @@ +
\ No newline at end of file