diff --git a/.env.example b/.env.example index 16ef3c1..bbd032c 100644 --- a/.env.example +++ b/.env.example @@ -1,12 +1,8 @@ # Live SDK verification env file. # Copy this to `.env` or `.env.local` and fill only the values needed by the live targets you run. -# Optional Infisical coordinates for `pnpm secrets:setup` or `infisical run`. -# Keep INFISICAL_TOKEN in the runner environment rather than this file. -PUTIO_SDK_TYPESCRIPT_INFISICAL_PROJECT_ID="" -PUTIO_SDK_TYPESCRIPT_INFISICAL_PATH="" -PUTIO_SDK_TYPESCRIPT_INFISICAL_ENV="dev" -PUTIO_INFISICAL_DOMAIN="https://eu.infisical.com/api" +# Optional SOPS ciphertext input for `pnpm secrets:setup`. +# PUTIO_SDK_TYPESCRIPT_SOPS_FILE=/path/to/typescript.sops.env # Bootstrap creds — used by `bootstrap:tokens` and credential live tests to mint fresh tokens. PUTIO_TEST_USERNAME="" diff --git a/AGENTS.md b/AGENTS.md index 9fa6ead..64420bc 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -29,7 +29,8 @@ `.worktreeinclude` carries `.env` and `.repos` into managed worktrees; Claude symlinks `.repos`. Run `vp install`, `vp config`, then `vp run verify`. Use -`pnpm secrets:setup` if live-test env is missing or stale. +`pnpm secrets:setup` with `PUTIO_SDK_TYPESCRIPT_SOPS_FILE` if live-test env is +missing or stale. ## Repo-Specific Guidance @@ -47,7 +48,7 @@ symlinks `.repos`. Run `vp install`, `vp config`, then `vp run verify`. Use - Default tests exclude `test/live/**`. - Use `vp run test:live` or the single-target live commands in [Testing](./docs/TESTING.md) when verifying against the real API. -- Live tests expect maintainer-supplied `PUTIO_SDK_TYPESCRIPT_INFISICAL_*`; `pnpm secrets:setup` writes ignored `.env.local`, and `pnpm secrets:clean` removes it. +- Live tests accept maintainer-supplied `PUTIO_SDK_TYPESCRIPT_SOPS_FILE`; `pnpm secrets:setup` validates and writes ignored `.env.local`, and `pnpm secrets:clean` removes it. - Keep package-surface verification healthy; `lint:package` is the publication safety net for tarball metadata, public types, and ESM entrypoints. ## Skills diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index 7782bd5..56b9491 100644 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -57,15 +57,13 @@ The package-surface checks do not require live credentials. Use live tests when Bootstrap runtime tokens from the rendered live credentials: ```bash -pnpm secrets:setup # materializes .env.local from the repo-owned Infisical path +pnpm secrets:setup # validates SOPS ciphertext and materializes .env.local pnpm bootstrap:tokens # mints fresh first/third-party tokens against the live API ``` -`secrets:setup` requires the Infisical CLI and access to the put.io frontend -Development environment. Set the onboarding-provided -`PUTIO_SDK_TYPESCRIPT_INFISICAL_*` variables in this repo or worktree shell -before running the command. Run `pnpm secrets:clean` before tearing down the -worktree. +`secrets:setup` requires SOPS 3.10 or newer, an authorized age identity, and +`PUTIO_SDK_TYPESCRIPT_SOPS_FILE` pointing to the maintainer-provided ciphertext. +Run `pnpm secrets:clean` before tearing down the worktree. For single-target commands, safety rules, and fixture expectations, see [Testing](./docs/TESTING.md). diff --git a/docs/TESTING.md b/docs/TESTING.md index c83c7fb..047f894 100644 --- a/docs/TESTING.md +++ b/docs/TESTING.md @@ -168,9 +168,9 @@ safe owned MP4 fixture for media flag, URL, HLS, watch status, and start-from coverage. The shared-friend clone fixture is seeded from the configured secondary account. -Use `pnpm secrets:setup` to render the shared live variables from Infisical into -`.env.local`. The live harness also accepts legacy local aliases when they are -already exported in the shell. +Use `pnpm secrets:setup` to validate the maintainer-provided SOPS ciphertext and +render shared live variables into `.env.local`. The live harness also accepts +legacy local aliases when they are already exported in the shell. Keep token values out of command output, docs, comments, and commits. @@ -199,22 +199,19 @@ runs only the named test files, and revokes the fresh first-party session before exiting, including when a test fails. It never writes the runtime tokens to an env file. -An unattended runner can inject a scoped `INFISICAL_TOKEN` machine-identity -access token and run the same command without materializing secrets: +An unattended runner with a scoped age identity can run a command without +materializing secrets: ```bash -infisical run --silent \ - --domain "$PUTIO_INFISICAL_DOMAIN" \ - --projectId "$PUTIO_SDK_TYPESCRIPT_INFISICAL_PROJECT_ID" \ - --env "$PUTIO_SDK_TYPESCRIPT_INFISICAL_ENV" \ - --path "$PUTIO_SDK_TYPESCRIPT_INFISICAL_PATH" \ - -- pnpm test:live:fresh -- test/live/account.test.ts test/live/tunnel.test.ts +sops exec-env --same-process "$PUTIO_SDK_TYPESCRIPT_SOPS_FILE" \ + 'pnpm test:live:fresh -- test/live/account.test.ts test/live/tunnel.test.ts' ``` -Run `pnpm secrets:setup` once per worktree to materialize `.env.local` from the -Infisical `/sdk-typescript` path. The materialized file is `0600` and -gitignored. Live commands auto-load `.env.local` first and then `.env`; -already-exported environment variables keep highest priority. +Run `pnpm secrets:setup` once per worktree with +`PUTIO_SDK_TYPESCRIPT_SOPS_FILE` pointing to the supplied ciphertext. The +materialized file is `0600` and gitignored. Live commands auto-load +`.env.local` first and then `.env`; already-exported environment variables keep +highest priority. ```bash pnpm secrets:setup # one-time per worktree @@ -224,9 +221,9 @@ pnpm test:live # runs the broader live suite against pre-existing tok pnpm secrets:clean # before `git worktree remove` ``` -`secrets:setup` requires the Infisical CLI and access to the put.io frontend -Development environment. You can copy `.env.example` manually when using your -own live credentials, and unit tests do not require live credentials. +`secrets:setup` requires SOPS 3.10 or newer and an authorized age identity. You can +copy `.env.example` manually when using your own live credentials, and unit +tests do not require live credentials. `bootstrap:live-fixtures` validates and seeds the live fixtures that are safe to prepare through the public SDK. It establishes the secondary friendship/shared diff --git a/package.json b/package.json index d42de77..e354c64 100644 --- a/package.json +++ b/package.json @@ -50,6 +50,7 @@ "prepack": "vp pack", "secrets:setup": "bash ./scripts/secrets-setup.sh", "secrets:clean": "rm -f .env.local .env.local.* .env.local.swp", + "test:secrets": "bash ./scripts/secrets-setup.test.sh", "test": "vp test --passWithNoTests", "test:compat": "node ./scripts/test-compat-node.ts && node ./scripts/test-compat-browser.ts && node ./scripts/test-compat-bun.ts", "test:compat:browser": "node ./scripts/test-compat-browser.ts", @@ -60,7 +61,7 @@ "test:live:fresh": "vp pack && node ./scripts/test-live-fresh.ts", "validate:routes": "vp pack && vp run validate:routes:packed", "validate:routes:packed": "node ./scripts/validate-route-matrix.ts", - "verify": "vp check . && vp pack && knip && knip --production --no-gitignore && vp run validate:routes:packed && vp test run --coverage --passWithNoTests" + "verify": "vp check . && vp pack && knip && knip --production --no-gitignore && vp run validate:routes:packed && vp test run --coverage --passWithNoTests && vp run test:secrets" }, "dependencies": { "effect": "4.0.0-beta.101" diff --git a/scripts/secrets-render.mjs b/scripts/secrets-render.mjs new file mode 100644 index 0000000..dde0d18 --- /dev/null +++ b/scripts/secrets-render.mjs @@ -0,0 +1,69 @@ +import { readFileSync, writeFileSync } from "node:fs"; + +const [payloadPath, outputPath] = process.argv.slice(2); + +if (!payloadPath || !outputPath) { + throw new Error("expected payload and output paths"); +} + +const expectedKeys = [ + "PUTIO_CLIENT_ID", + "PUTIO_CLIENT_ID_FIRST_PARTY", + "PUTIO_CLIENT_ID_THIRD_PARTY", + "PUTIO_CLIENT_SECRET_FIRST_PARTY", + "PUTIO_TEST_PASSWORD", + "PUTIO_TEST_TOTP_REFERENCE", + "PUTIO_TEST_USERNAME", + "PUTIO_TOKEN_FIRST_PARTY", + "PUTIO_TOKEN_THIRD_PARTY", +]; + +const payload = JSON.parse(readFileSync(payloadPath, "utf8")); + +if (!payload || typeof payload !== "object" || Array.isArray(payload)) { + throw new Error("decrypted payload must be a JSON object"); +} + +const actualKeys = Object.keys(payload).sort(); +if (JSON.stringify(actualKeys) !== JSON.stringify(expectedKeys)) { + throw new Error("decrypted payload key inventory does not match the SDK contract"); +} + +for (const value of Object.values(payload)) { + if (typeof value !== "string" || value.length === 0) { + throw new Error("decrypted payload contains an empty or non-string value"); + } + + if ( + (value.startsWith('"') && value.endsWith('"')) || + (value.startsWith("'") && value.endsWith("'")) + ) { + throw new Error("decrypted payload contains a quote-wrapped value"); + } + + if (value.includes("\0") || value.includes("\n") || value.includes("\r")) { + throw new Error("decrypted payload contains an unsafe control character"); + } +} + +for (const key of [ + "PUTIO_CLIENT_ID", + "PUTIO_CLIENT_ID_FIRST_PARTY", + "PUTIO_CLIENT_ID_THIRD_PARTY", +]) { + if (!/^[0-9]+$/.test(payload[key])) { + throw new Error("decrypted payload contains an invalid numeric identifier"); + } +} + +const delimiters = ['"', "'", "`"]; +const render = (value) => { + const delimiter = delimiters.find((candidate) => !value.includes(candidate)); + if (!delimiter) { + throw new Error("decrypted payload contains a value that cannot be rendered safely"); + } + return `${delimiter}${value}${delimiter}`; +}; + +const dotenv = actualKeys.map((key) => `${key}=${render(payload[key])}`).join("\n"); +writeFileSync(outputPath, `${dotenv}\n`, { mode: 0o600 }); diff --git a/scripts/secrets-setup.sh b/scripts/secrets-setup.sh index e194398..5c3f241 100644 --- a/scripts/secrets-setup.sh +++ b/scripts/secrets-setup.sh @@ -3,29 +3,57 @@ set -euo pipefail umask 077 +fail() { + printf 'FAILED: %s\n' "$1" >&2 + exit 1 +} + +repo_root="$(cd "$(dirname "${BASH_SOURCE[0]}")/.." && pwd)" +cd "$repo_root" + +ciphertext="${PUTIO_SDK_TYPESCRIPT_SOPS_FILE:?Set PUTIO_SDK_TYPESCRIPT_SOPS_FILE to the SDK ciphertext file}" output="${SECRETS_OUTPUT:-.env.local}" -infisical_domain="${PUTIO_INFISICAL_DOMAIN:-https://eu.infisical.com/api}" -infisical_project_id="${PUTIO_SDK_TYPESCRIPT_INFISICAL_PROJECT_ID:?Set PUTIO_SDK_TYPESCRIPT_INFISICAL_PROJECT_ID for this repo}" -infisical_env="${PUTIO_SDK_TYPESCRIPT_INFISICAL_ENV:-dev}" -infisical_path="${PUTIO_SDK_TYPESCRIPT_INFISICAL_PATH:?Set PUTIO_SDK_TYPESCRIPT_INFISICAL_PATH for this repo}" -if ! command -v infisical >/dev/null 2>&1; then - echo "Infisical CLI is required. Install it with: brew install infisical" >&2 - exit 1 -fi +command -v sops >/dev/null 2>&1 || fail "sops is required" + +[ -f "$ciphertext" ] || fail "ciphertext input must be one regular file" +[ ! -L "$ciphertext" ] || fail "ciphertext input must not be a symlink" +case "$output" in + /*|..|../*|*/../*) fail "SECRETS_OUTPUT must be a repository-relative ignored path" ;; +esac +git check-ignore -q -- "$output" || fail "output path is not gitignored: $output" +[ ! -L "$output" ] || fail "output path must not be a symlink: $output" +[ ! -e "$output" ] || [ -f "$output" ] || fail "output path must be a regular file: $output" -tmp_env="$(mktemp)" +status="$(sops filestatus --input-type dotenv "$ciphertext" 2>/dev/null)" \ + || fail "SOPS 3.10 or newer could not inspect the dotenv ciphertext input" +printf '%s\n' "$status" | node -e ' + let input = "" + process.stdin.setEncoding("utf8") + process.stdin.on("data", (chunk) => { input += chunk }) + process.stdin.on("end", () => { + try { + if (JSON.parse(input).encrypted !== true) process.exitCode = 1 + } catch { + process.exitCode = 1 + } + }) +' \ + || fail "ciphertext input is not encrypted" + +tmp_dir="$(mktemp -d "${TMPDIR:-/tmp}/putio-sdk-secrets.XXXXXX")" cleanup() { - rm -f "$tmp_env" + rm -rf "$tmp_dir" } trap cleanup EXIT -infisical export \ - --silent \ - --domain "$infisical_domain" \ - --projectId "$infisical_project_id" \ - --env "$infisical_env" \ - --path "$infisical_path" \ - --format dotenv \ - --output-file "$tmp_env" -install -m 600 "$tmp_env" "$output" +payload_json="$tmp_dir/payload.json" +rendered_env="$tmp_dir/rendered.env" +sops decrypt --output-type json --output "$payload_json" "$ciphertext" \ + || fail "could not decrypt ciphertext input" +chmod 600 "$payload_json" + +node ./scripts/secrets-render.mjs "$payload_json" "$rendered_env" \ + || fail "decrypted payload failed validation or safe dotenv rendering" +install -m 600 "$rendered_env" "$output" +printf 'ok wrote %s\n' "$output" diff --git a/scripts/secrets-setup.test.sh b/scripts/secrets-setup.test.sh new file mode 100644 index 0000000..97b5997 --- /dev/null +++ b/scripts/secrets-setup.test.sh @@ -0,0 +1,223 @@ +#!/usr/bin/env bash + +set -euo pipefail +umask 077 + +repo_root="$(cd "$(dirname "${BASH_SOURCE[0]}")/.." && pwd)" +cd "$repo_root" + +tmp_dir="$(mktemp -d "${TMPDIR:-/tmp}/putio-sdk-secrets-test.XXXXXX")" +output=".env.local.sops-test.$$" +cleanup() { + rm -rf "$tmp_dir" + rm -f "$output" +} +trap cleanup EXIT + +mkdir -p "$tmp_dir/bin" +fake_sops="$tmp_dir/bin/sops" +cat >"$fake_sops" <<'SH' +#!/usr/bin/env bash +set -euo pipefail + +case "${1:-}" in + filestatus) + [ "${2:-}" = "--input-type" ] + [ "${3:-}" = "dotenv" ] + printf '{"encrypted":%s}\n' "${FAKE_SOPS_ENCRYPTED:-true}" + ;; + decrypt) + output="" + while [ "$#" -gt 0 ]; do + case "$1" in + --output) + shift + output="$1" + ;; + esac + shift + done + [ -n "$output" ] + install -m 600 "$FAKE_SOPS_PAYLOAD" "$output" + ;; + *) + exit 2 + ;; +esac +SH +chmod 700 "$fake_sops" + +ciphertext="$tmp_dir/payload.sops.env" +payload="$tmp_dir/payload.json" +printf 'ciphertext fixture\n' >"$ciphertext" + +write_valid_payload() { + cat >"$payload" <<'JSON' +{ + "PUTIO_CLIENT_ID": "1", + "PUTIO_CLIENT_ID_FIRST_PARTY": "2", + "PUTIO_CLIENT_ID_THIRD_PARTY": "3", + "PUTIO_CLIENT_SECRET_FIRST_PARTY": "secret with spaces and = signs", + "PUTIO_TEST_PASSWORD": "test-password", + "PUTIO_TEST_TOTP_REFERENCE": "op://example/item/field", + "PUTIO_TEST_USERNAME": "test@example.com", + "PUTIO_TOKEN_FIRST_PARTY": "first-party-token", + "PUTIO_TOKEN_THIRD_PARTY": "third-party-token" +} +JSON +} + +run_setup() { + PATH="$tmp_dir/bin:$PATH" \ + FAKE_SOPS_PAYLOAD="$payload" \ + PUTIO_SDK_TYPESCRIPT_SOPS_FILE="$ciphertext" \ + SECRETS_OUTPUT="$output" \ + bash ./scripts/secrets-setup.sh +} + +expect_failure() { + if "$@" >/dev/null 2>&1; then + printf 'FAILED: command unexpectedly succeeded\n' >&2 + exit 1 + fi +} + +write_valid_payload +run_setup >/dev/null +if output_mode="$(stat -c '%a' "$output" 2>/dev/null)"; then + : +else + output_mode="$(stat -f '%Lp' "$output")" +fi +[ "$output_mode" = 600 ] +node - "$output" <<'NODE' +const path = process.argv[2] +process.loadEnvFile(path) +const expected = { + PUTIO_CLIENT_ID: "1", + PUTIO_CLIENT_ID_FIRST_PARTY: "2", + PUTIO_CLIENT_ID_THIRD_PARTY: "3", + PUTIO_CLIENT_SECRET_FIRST_PARTY: "secret with spaces and = signs", + PUTIO_TEST_PASSWORD: "test-password", + PUTIO_TEST_TOTP_REFERENCE: "op://example/item/field", + PUTIO_TEST_USERNAME: "test@example.com", + PUTIO_TOKEN_FIRST_PARTY: "first-party-token", + PUTIO_TOKEN_THIRD_PARTY: "third-party-token", +} +for (const [key, value] of Object.entries(expected)) { + if (process.env[key] !== value) process.exit(1) +} +NODE +rm -f "$output" + +mutate_payload() { + node --input-type=module - "$payload" "$1" <<'NODE' +import { readFileSync, writeFileSync } from "node:fs" +const [path, mutation] = process.argv.slice(2) +const payload = JSON.parse(readFileSync(path, "utf8")) +switch (mutation) { + case "missing": delete payload.PUTIO_CLIENT_ID; break + case "extra": payload.EXTRA = "unexpected"; break + case "empty": payload.PUTIO_TEST_PASSWORD = ""; break + case "numeric": payload.PUTIO_CLIENT_ID = "not-numeric"; break + case "double-wrapper": payload.PUTIO_TEST_PASSWORD = '\"quoted\"'; break + case "single-wrapper": payload.PUTIO_TEST_PASSWORD = "'quoted'"; break + case "special": payload.PUTIO_CLIENT_SECRET_FIRST_PARTY = "single'quote\\backslash"; payload.PUTIO_TEST_PASSWORD = 'double"quote\\backslash'; payload.PUTIO_TOKEN_FIRST_PARTY = "single'double\"back\\slash"; break + case "unrenderable": payload.PUTIO_TEST_PASSWORD = "single'double\"back`tick"; break + case "multiline": payload.PUTIO_TEST_PASSWORD = "line one\nline two"; break + case "nul": payload.PUTIO_TEST_PASSWORD = "before\0after"; break + default: process.exit(2) +} +writeFileSync(path, JSON.stringify(payload)) +NODE +} + +write_valid_payload +mutate_payload special +run_setup >/dev/null +node - "$output" <<'NODE' +const path = process.argv[2] +process.loadEnvFile(path) +const expected = { + PUTIO_CLIENT_SECRET_FIRST_PARTY: "single'quote\\backslash", + PUTIO_TEST_PASSWORD: 'double"quote\\backslash', + PUTIO_TOKEN_FIRST_PARTY: "single'double\"back\\slash", +} +for (const [key, value] of Object.entries(expected)) { + if (process.env[key] !== value) process.exit(1) +} +NODE +rm -f "$output" + +write_valid_payload +mutate_payload missing +expect_failure run_setup +[ ! -e "$output" ] + +write_valid_payload +mutate_payload extra +expect_failure run_setup +[ ! -e "$output" ] + +write_valid_payload +mutate_payload empty +expect_failure run_setup +[ ! -e "$output" ] + +write_valid_payload +mutate_payload numeric +expect_failure run_setup +[ ! -e "$output" ] + +write_valid_payload +mutate_payload double-wrapper +expect_failure run_setup +[ ! -e "$output" ] + +write_valid_payload +mutate_payload single-wrapper +expect_failure run_setup +[ ! -e "$output" ] + +write_valid_payload +mutate_payload unrenderable +expect_failure run_setup +[ ! -e "$output" ] + +write_valid_payload +mutate_payload multiline +expect_failure run_setup +[ ! -e "$output" ] + +write_valid_payload +mutate_payload nul +expect_failure run_setup +[ ! -e "$output" ] + +write_valid_payload +expect_failure env \ + PATH="$tmp_dir/bin:$PATH" \ + FAKE_SOPS_ENCRYPTED=false \ + FAKE_SOPS_PAYLOAD="$payload" \ + PUTIO_SDK_TYPESCRIPT_SOPS_FILE="$ciphertext" \ + SECRETS_OUTPUT="$output" \ + bash ./scripts/secrets-setup.sh +[ ! -e "$output" ] + +expect_failure env \ + PATH="$tmp_dir/bin:$PATH" \ + FAKE_SOPS_PAYLOAD="$payload" \ + PUTIO_SDK_TYPESCRIPT_SOPS_FILE="$ciphertext" \ + SECRETS_OUTPUT=README.md \ + bash ./scripts/secrets-setup.sh + +symlinked_ciphertext="$tmp_dir/symlinked.sops.env" +ln -s "$ciphertext" "$symlinked_ciphertext" +expect_failure env \ + PATH="$tmp_dir/bin:$PATH" \ + FAKE_SOPS_PAYLOAD="$payload" \ + PUTIO_SDK_TYPESCRIPT_SOPS_FILE="$symlinked_ciphertext" \ + SECRETS_OUTPUT="$output" \ + bash ./scripts/secrets-setup.sh + +printf 'ok SOPS setup renders validated ignored output and fails closed\n'