diff --git a/docs/analysis/2026-09-20-ci17-runner-inventory.md b/docs/analysis/2026-09-20-ci17-runner-inventory.md new file mode 100644 index 000000000..5c59c70f5 --- /dev/null +++ b/docs/analysis/2026-09-20-ci17-runner-inventory.md @@ -0,0 +1,106 @@ +# CI-17: conservative runner and reusable-workflow inventory + +This is an implementation prerequisite for [#3170](https://github.com/Chris0Jeky/Taskdeck/issues/3170), +not the private-cutover rehearsal mechanism. It does not change a workflow, activate a mode, +suppress a job, configure a runner, authorize a visibility change, or close an acceptance box. +The existing [hard-issue map PR](https://github.com/Chris0Jeky/Taskdeck/pull/3281) identifies this +workflow-graph proof as the smallest useful independent slice of that issue. + +## What this proves + +The dependency-free [inventory module](../../scripts/ci/smart-ci/workflow-runner-inventory.mjs) +discovers runner-bearing jobs and job-level reusable-workflow calls across the supplied source +set. The existing Smart CI test glob runs its regression suite and compares the Windows/opaque +runner sites and all their ancestor call jobs with a checked-in review snapshot. A new site, +changed selector, matrix, condition, input, trigger or route changes that surface and requires +explicit review. Ordinary step bodies are not copied into the report. + +The source boundary is deliberately narrow: top-level, `jobs`, and individual jobs must be +unquoted block mappings at indentation 0/2/4. Duplicate keys, merge keys, aliased/inline job +mappings, unexpected structural indentation and multiple-document syntax make discovery +incomplete. This is not a general YAML parser or an Actions schema validator; actionlint +remains separate. Runner arrays, groups, aliases, custom labels and expressions are retained +as opaque candidates, not discarded. Nested matrix and caller input text is preserved, +including blank and hash-prefixed lines inside block scalars. + +Only the finite reviewed scalar spellings `ubuntu-latest`, `ubuntu-22.04` and `ubuntu-24.04` +are classified as Linux literals. That is a statement about source spelling, not verified +runner identity. Every other selector remains on the conservative review surface. Conditions +and matrix exclusions never remove candidates: the module does not evaluate expressions, +`if: false`, include/exclude precedence, environment or repository variables, or caller inputs. +For example, even the API caller that supplies `platform: linux` remains visible. +To keep large projections from expanding the JSON report without bound, the inventory fails +closed with `projection-limit` when the projected trigger text exceeds 8 MiB, and the reviewed +surface returns `graphComplete: false` with no candidates when candidate-plus-ancestor caller +projections exceed its 8 MiB bound. These are discovery failures, not qualification results. + +Local calls are resolved within the supplied workflow set. Missing callees, external or +unresolved references, cycles and duplicate/non-canonical files make `graphComplete` false. +All disconnected runner sites are included, not just jobs reachable from `ci-required.yml`. +Ancestor traversal retains separate caller job IDs, including parallel calls from one file, +without enumerating exponentially many paths. Source bounds are 256 files, 2 MiB per file, +16 MiB aggregate and 4,096 jobs. The loader rejects non-regular workflow entries, including +file symlinks; this is not a filesystem sandbox or a concurrent-file-mutation guarantee. + +GitHub supports multiple [runner-selector shapes](https://docs.github.com/en/actions/reference/workflows-and-actions/workflow-syntax#jobsjob_idruns-on), +[matrices](https://docs.github.com/en/actions/reference/workflows-and-actions/workflow-syntax#jobsjob_idstrategymatrix) +and [reusable calls](https://docs.github.com/en/actions/reference/workflows-and-actions/workflow-syntax#jobsjob_iduses). +The conservative treatment is intentional: resolving some expression strings would create a +second, weaker scheduler rather than evidence that no Windows job can run. + +## Measured source inventory + +At main `307c3b8b50bec1cb0bfaea3e570a942bcb1d4451`, there are **36 workflow files, 50 runner jobs +and 41 job-level reusable calls**. Five runner sites need Windows/dynamic review: + +| Workflow and job | Selector | Known caller jobs | +| --- | --- | --- | +| `release-desktop.yml / build-backend` | `matrix.os`, with Windows in `include` | Direct release workflow entry | +| `reusable-api-integration.yml / api-integration` | `matrix.os`, supplied by an input-dependent expression | `ci-required / api-integration` and `api-integration-windows` | +| `reusable-backend-unit.yml / backend-unit` | `matrix.os`, Linux and Windows matrix | `ci-required / backend-unit` | +| `reusable-docs-governance.yml / worktree-helper-windows` | Literal `windows-latest` | `ci-required / docs-governance` | +| `reusable-frontend-unit.yml / frontend-unit` | `matrix.os`, Linux and Windows matrix | `ci-required / frontend-unit` | + +These are source sites, not executed job counts or proof of a particular private-mode run. +The four matrix selectors remain opaque even though their current text mentions Windows. + +## Running and updating the inventory + +```sh +node scripts/ci/smart-ci/workflow-runner-inventory.mjs +node --test scripts/ci/smart-ci/workflow-runner-inventory.test.mjs +node --test scripts/ci/smart-ci/*.test.mjs +``` + +The CLI prints JSON and exits 2 for incomplete or unavailable source. Exit 0 means discovery +completed, **not that a Linux-only rehearsal is safe**. Neither the CLI nor the report emits +qualification, bounded-execution or guard-authority flags. Never use `graphComplete`, exit 0, +this snapshot test, or an empty candidate array to authorize suppressing CI work. The source +set itself has no authenticated commit or protected-base provenance supplied by this module. + +Review the changed runner site and its caller controls before updating +`test-support/workflow-runner-inventory.snapshot.json`. A snapshot is a drift alarm, not a +trust boundary: a PR can edit both source and snapshot. Trusted-base execution and approval +of a real rehearsal guard still have to be implemented and separately reviewed. + +## Verification and residuals + +The supplied ZIP identifies `6818072c413609fd2b9a9e37c778e866998d5b1e`. GitHub comparison against +the above main found 45 subsequent commits, with no changes to the workflows, Smart CI scripts +or policy used here. Publication must use real GitHub ancestry, not the synthetic local ZIP +commit. The unrelated live backend and frontend changes are not replaced. + +Local Windows / Node 22.16.0: **54 focused tests and all 608 Smart CI tests pass**, with zero +failures or skips. A separate local PyYAML 6.0.3 structural cross-check found exactly the same +50 runner-job and 41 reusable-call identities; PyYAML is not a shipped dependency. Mutation +checks dropping opaque candidates, dropping ancestor calls and ignoring duplicate keys were +rejected by 11, 4 and 3 focused tests respectively. Every mutation was removed before the final +suite. The initial literal-only negative control was deliberately insufficient, not existing +production code; it is not committed. + +This slice does not test guard coverage, authenticated mode propagation, actual scheduling, +non-vacuous Linux/security evidence, private-mode runs or trusted receipts. It does not prove +Windows runtime behavior from a Linux junction test. Exact-head hosted results and fresh +review belong on the PR; local results are additive, not R4 qualification. #3170 and #2337 +remain open. R4 maintainer/fresh-context review and the human decisions in OUTSTANDING_TASKS.md +J.3(b)/J.4 remain unchanged; canonical J.2 registration is a coordinator follow-up. diff --git a/scripts/ci/smart-ci/test-support/workflow-runner-inventory.snapshot.json b/scripts/ci/smart-ci/test-support/workflow-runner-inventory.snapshot.json new file mode 100644 index 000000000..e756affb0 --- /dev/null +++ b/scripts/ci/smart-ci/test-support/workflow-runner-inventory.snapshot.json @@ -0,0 +1,130 @@ +{ + "schemaVersion": 1, + "graphComplete": true, + "candidates": [ + { + "id": ".github/workflows/release-desktop.yml#build-backend", + "file": ".github/workflows/release-desktop.yml", + "job": "build-backend", + "events": "workflow_dispatch:\n inputs:\n tag:\n description: \"Existing version tag to build and PUBLISH (e.g. v0.1.1). Leave BLANK when dispatching from a BRANCH for a rehearsal: builds and smoke-tests the Windows x64 archive, uploads artifacts, and publishes nothing. Dispatching from a TAG ref publishes that tag even with this left blank.\"\n required: false\n type: string\n preview_tag:\n description: \"REHEARSAL ONLY (e.g. v0.3.0). Renders the release page as if this tag existed, so a no-publish dispatch previews the real stable page instead of the v0.0.0-dryrun placeholder. It NEVER names, creates, touches or publishes a Release, and it never reaches the built archive name or the version stamped into the binaries. A dispatch that also PUBLISHES (a tag input, or a tag ref) is REFUSED before any build runs — use exactly one of the two.\"\n required: false\n type: string\npush:\n tags:\n - \"v*\"", + "condition": null, + "needs": "[resolve-source, build-frontend]", + "strategy": "fail-fast: false\nmatrix:\n include:\n - rid: win-x64\n os: windows-latest\n archive_ext: zip", + "selector": "${{ matrix.os }}", + "classification": "opaque", + "callers": [] + }, + { + "id": ".github/workflows/reusable-api-integration.yml#api-integration", + "file": ".github/workflows/reusable-api-integration.yml", + "job": "api-integration", + "events": "workflow_call:\n inputs:\n platform:\n description: linux or windows for independent scheduling; all other values retain both platforms\n required: false\n default: all\n type: string\n dotnet-version:\n description: .NET SDK version used for API integration tests\n required: false\n default: \"8.0.x\"\n type: string", + "condition": null, + "needs": null, + "strategy": "fail-fast: false\nmatrix:\n os: ${{ fromJSON(inputs.platform == 'linux' && '[\"ubuntu-latest\"]' || inputs.platform == 'windows' && '[\"windows-latest\"]' || '[\"ubuntu-latest\", \"windows-latest\"]') }}", + "selector": "${{ matrix.os }}", + "classification": "opaque", + "callers": [ + { + "id": ".github/workflows/ci-required.yml#api-integration", + "file": ".github/workflows/ci-required.yml", + "job": "api-integration", + "events": "push:\n branches:\n - main\n - master\npull_request:\nmerge_group:", + "condition": null, + "needs": "- backend-architecture\n- release-workflow-contract", + "strategy": null, + "reference": "./.github/workflows/reusable-api-integration.yml", + "callee": ".github/workflows/reusable-api-integration.yml", + "inputs": "platform: linux\ndotnet-version: 8.0.x\n\n # Windows still qualifies the same complete suite, but does not serialize Linux E2E." + }, + { + "id": ".github/workflows/ci-required.yml#api-integration-windows", + "file": ".github/workflows/ci-required.yml", + "job": "api-integration-windows", + "events": "push:\n branches:\n - main\n - master\npull_request:\nmerge_group:", + "condition": null, + "needs": "- backend-architecture\n- release-workflow-contract", + "strategy": null, + "reference": "./.github/workflows/reusable-api-integration.yml", + "callee": ".github/workflows/reusable-api-integration.yml", + "inputs": "platform: windows\ndotnet-version: 8.0.x" + } + ] + }, + { + "id": ".github/workflows/reusable-backend-unit.yml#backend-unit", + "file": ".github/workflows/reusable-backend-unit.yml", + "job": "backend-unit", + "events": "workflow_call:\n inputs:\n dotnet-version:\n description: .NET SDK version used for backend unit and contract tests\n required: false\n default: \"8.0.x\"\n type: string", + "condition": null, + "needs": null, + "strategy": "fail-fast: false\nmatrix:\n os:\n - ubuntu-latest\n - windows-latest", + "selector": "${{ matrix.os }}", + "classification": "opaque", + "callers": [ + { + "id": ".github/workflows/ci-required.yml#backend-unit", + "file": ".github/workflows/ci-required.yml", + "job": "backend-unit", + "events": "push:\n branches:\n - main\n - master\npull_request:\nmerge_group:", + "condition": null, + "needs": "- backend-architecture\n- release-workflow-contract", + "strategy": null, + "reference": "./.github/workflows/reusable-backend-unit.yml", + "callee": ".github/workflows/reusable-backend-unit.yml", + "inputs": "dotnet-version: 8.0.x" + } + ] + }, + { + "id": ".github/workflows/reusable-docs-governance.yml#worktree-helper-windows", + "file": ".github/workflows/reusable-docs-governance.yml", + "job": "worktree-helper-windows", + "events": "workflow_call:\n inputs:\n node-version:\n description: Node.js version for docs governance checks\n required: false\n default: \"24.13.1\"\n type: string", + "condition": null, + "needs": null, + "strategy": null, + "selector": "windows-latest", + "classification": "windows-literal", + "callers": [ + { + "id": ".github/workflows/ci-required.yml#docs-governance", + "file": ".github/workflows/ci-required.yml", + "job": "docs-governance", + "events": "push:\n branches:\n - main\n - master\npull_request:\nmerge_group:", + "condition": null, + "needs": null, + "strategy": null, + "reference": "./.github/workflows/reusable-docs-governance.yml", + "callee": ".github/workflows/reusable-docs-governance.yml", + "inputs": "node-version: 24.13.1\n\n # A live release dispatch cannot be rehearsed on a PR, so release hardening is\n # held by executable contracts instead: tag/version grammar runs for real and\n # workflow provenance, fail-fast, credential and cleanup invariants are asserted\n # structurally.\n # Inline (not a reusable call) — it needs nothing beyond the top-level\n # `contents: read`, and a reusable callee may not declare more than its caller\n # grants (GitHub fails such a call at plan time with startup_failure)." + } + ] + }, + { + "id": ".github/workflows/reusable-frontend-unit.yml#frontend-unit", + "file": ".github/workflows/reusable-frontend-unit.yml", + "job": "frontend-unit", + "events": "workflow_call:\n inputs:\n node-version:\n description: Node.js version used for frontend quality gates\n required: false\n default: \"24.13.1\"\n type: string", + "condition": null, + "needs": null, + "strategy": "fail-fast: false\nmatrix:\n os:\n - ubuntu-latest\n - windows-latest", + "selector": "${{ matrix.os }}", + "classification": "opaque", + "callers": [ + { + "id": ".github/workflows/ci-required.yml#frontend-unit", + "file": ".github/workflows/ci-required.yml", + "job": "frontend-unit", + "events": "push:\n branches:\n - main\n - master\npull_request:\nmerge_group:", + "condition": null, + "needs": "- release-workflow-contract\n- paper-color-audit", + "strategy": null, + "reference": "./.github/workflows/reusable-frontend-unit.yml", + "callee": ".github/workflows/reusable-frontend-unit.yml", + "inputs": "node-version: 24.13.1" + } + ] + } + ] +} diff --git a/scripts/ci/smart-ci/workflow-runner-inventory.mjs b/scripts/ci/smart-ci/workflow-runner-inventory.mjs new file mode 100644 index 000000000..6f65096a2 --- /dev/null +++ b/scripts/ci/smart-ci/workflow-runner-inventory.mjs @@ -0,0 +1,249 @@ +#!/usr/bin/env node +// CI-17 discovery prerequisite. This is NOT an Actions expression evaluator, +// general YAML parser, rehearsal guard, or authorization to skip any CI job. +import { readdirSync, readFileSync, realpathSync, lstatSync } from 'node:fs'; +import { join } from 'node:path'; +import { fileURLToPath } from 'node:url'; + +const MAX_FILES = 256; +const MAX_FILE_BYTES = 2 * 1024 * 1024; +const MAX_TOTAL_BYTES = 16 * 1024 * 1024; +const MAX_JOBS = 4096; +const MAX_DIAGNOSTICS = 4096; +const MAX_PROJECTED_EVENT_BYTES = 8 * 1024 * 1024; +const MAX_PROJECTED_SURFACE_BYTES = 8 * 1024 * 1024; +const PATH = /^\.github\/workflows\/[A-Za-z0-9_.-]+\.ya?ml$/; +const compare = (a, b) => a < b ? -1 : a > b ? 1 : 0; +const indentation = (line) => line.length - line.trimStart().length; +const significant = (line) => line.trim() && !line.trimStart().startsWith('#'); +const blockMapping = (field) => field && /^(?:#.*)?$/.test(field.value.trim()); + +// Read only the block mapping boundaries needed for workflow -> jobs -> job. +// Reject unfamiliar structural syntax rather than silently losing an entire job. +// Deeper values (steps, matrices, expressions) are deliberately NOT interpreted. +function fields(lines, depth, path, addDiagnostic) { + const result = new Map(); + let current = null; + for (const source of lines) { + if (!significant(source.text)) { + if (current) current.children.push(source); + continue; + } + const indent = indentation(source.text); + if (/^\s*\t/.test(source.text) || indent < depth || (indent > depth && !current)) { + addDiagnostic({ file: path, line: source.line, code: 'unsupported-indentation' }); + continue; + } + if (indent > depth) { + current.children.push(source); + continue; + } + const match = source.text.slice(depth).match(/^([A-Za-z_][A-Za-z0-9_-]*):(?:\s+(.*))?$/); + if (!match) { + addDiagnostic({ file: path, line: source.line, code: 'unsupported-mapping' }); + current = null; + continue; + } + current = { key: match[1], value: match[2] ?? '', line: source.line, depth, children: [] }; + if (result.has(current.key)) addDiagnostic({ file: path, line: source.line, code: 'duplicate-key' }); + // Keep the first occurrence; duplicate diagnostics already make discovery incomplete. + else result.set(current.key, current); + } + return result; +} + +function controlText(field) { + if (!field) return null; + // Preserve blank/hash-prefixed lines inside YAML block scalars. Removing them + // can erase a meaningful change to a caller input or a dynamic selector. + const prefix = ' '.repeat(field.depth + 2); + const children = field.children.map(({ text }) => text.startsWith(prefix) ? text.slice(prefix.length) : text); + const keepsTrailingLines = /[|>][^\s]*\+/.test(field.value) || + children.some((line) => /:\s+.*[|>][^\s]*\+(?:[ \t]+#.*)?\s*$/.test(line)); + if (!keepsTrailingLines) { + while (children.length && !children.at(-1).trim()) children.pop(); + } + return [field.value.trim(), ...children].filter((line, index) => index !== 0 || line).join('\n'); +} + +function literal(text) { + if (typeof text !== 'string') return null; + const match = text.match(/^(?:([A-Za-z0-9_./@-]+)|'([A-Za-z0-9_./@-]+)'|"([A-Za-z0-9_./@-]+)")(?:[ \t]+#.*)?$/); + return match ? (match[1] ?? match[2] ?? match[3]) : null; +} + +function classification(selector) { + const value = literal(selector); + // This finite list describes reviewed selector spelling, NOT actual runner identity. + if (['ubuntu-latest', 'ubuntu-22.04', 'ubuntu-24.04'].includes(value)) return 'linux-literal'; + if (value && /^windows-(?:latest|\d[\w.-]*)$/.test(value)) return 'windows-literal'; + return 'opaque'; +} + +function report(runners, calls, diagnostics) { + return { + schemaVersion: 1, + kind: 'smart-ci-workflow-runner-inventory', + graphComplete: diagnostics.length === 0, + runners: runners.sort((a, b) => compare(a.id, b.id)), + calls: calls.sort((a, b) => compare(a.id, b.id)), + diagnostics: diagnostics.sort((a, b) => compare(a.file ?? '', b.file ?? '') || (a.line ?? 0) - (b.line ?? 0) || compare(a.code, b.code)), + }; +} + +/** Inspect a complete supplied workflow set; never evaluate conditions or runner expressions. */ +export function inventoryWorkflowRunners(files) { + const diagnostics = []; + let diagnosticLimitReached = false; + const addDiagnostic = (entry) => { + if (diagnostics.length < MAX_DIAGNOSTICS) { + diagnostics.push(entry); + } else if (!diagnosticLimitReached) { + diagnosticLimitReached = true; + diagnostics.push({ code: 'diagnostic-limit' }); + } + }; + const runners = []; + const calls = []; + const sources = new Map(); + if (!Array.isArray(files)) return report([], [], [{ code: 'invalid-source' }]); + if (files.length === 0) return report([], [], [{ code: 'empty-source' }]); + if (files.length > MAX_FILES) return report([], [], [{ code: 'source-limit' }]); + let bytes = 0; + for (const source of files) { + if (!source || typeof source.path !== 'string' || !PATH.test(source.path) || typeof source.text !== 'string') { + addDiagnostic({ code: 'invalid-source' }); + continue; + } + const size = Buffer.byteLength(source.text, 'utf8'); + bytes += size; + if (size > MAX_FILE_BYTES || bytes > MAX_TOTAL_BYTES) return report([], [], [{ code: 'source-limit' }]); + if (sources.has(source.path)) addDiagnostic({ file: source.path, code: 'duplicate-file' }); + else sources.set(source.path, source.text); + } + + let jobCount = 0; + let projectedEventBytes = 0; + for (const [path, text] of [...sources].sort(([a], [b]) => compare(a, b))) { + const lines = text.split(/\r?\n/).map((line, index) => ({ text: line, line: index + 1 })); + const top = fields(lines, 0, path, addDiagnostic); + const jobsField = top.get('jobs'); + if (!jobsField) { addDiagnostic({ file: path, code: 'jobs-required' }); continue; } + if (!blockMapping(jobsField)) { addDiagnostic({ file: path, line: jobsField.line, code: 'unsupported-jobs-mapping' }); continue; } + const jobs = fields(jobsField.children, 2, path, addDiagnostic); + if (jobs.size === 0) addDiagnostic({ file: path, code: 'jobs-required' }); + const events = controlText(top.get('on')); + projectedEventBytes += Buffer.byteLength(events ?? '', 'utf8') * jobs.size; + if (projectedEventBytes > MAX_PROJECTED_EVENT_BYTES) { + addDiagnostic({ file: path, line: top.get('on')?.line, code: 'projection-limit' }); + return report([], [], diagnostics); + } + for (const [job, node] of jobs) { + jobCount += 1; + if (jobCount > MAX_JOBS) return report([], [], [{ code: 'source-limit' }]); + if (!blockMapping(node)) { addDiagnostic({ file: path, line: node.line, code: 'unsupported-job-mapping' }); continue; } + const properties = fields(node.children, 4, path, addDiagnostic); + const shared = { + id: `${path}#${job}`, file: path, job, line: node.line, + events, + condition: controlText(properties.get('if')), + needs: controlText(properties.get('needs')), + strategy: controlText(properties.get('strategy')), + }; + const selector = properties.get('runs-on'); + const uses = properties.get('uses'); + if (selector && uses) addDiagnostic({ file: path, line: node.line, code: 'ambiguous-job' }); + if (!selector && !uses) addDiagnostic({ file: path, line: node.line, code: 'runner-or-call-required' }); + if (selector) { + const value = controlText(selector); + const scalar = selector.children.some((source) => significant(source.text)) ? value : selector.value.trim(); + runners.push({ ...shared, selector: value, classification: classification(scalar) }); + } + if (uses) { + const reference = controlText(uses); + const target = literal(reference); + const callee = target?.startsWith('./') && PATH.test(target.slice(2)) ? target.slice(2) : null; + calls.push({ ...shared, reference, callee, inputs: controlText(properties.get('with')) }); + if (!callee) addDiagnostic({ file: path, line: uses.line, code: 'unresolved-workflow' }); + else if (!sources.has(callee)) addDiagnostic({ file: path, line: uses.line, code: 'missing-workflow' }); + } + } + } + + const outgoing = new Map([...sources.keys()].map((path) => [path, []])); + for (const call of calls) if (call.callee && sources.has(call.callee)) outgoing.get(call.file).push(call.callee); + const visiting = new Set(); + const visited = new Set(); + function visit(path) { + if (visiting.has(path)) { addDiagnostic({ file: path, code: 'workflow-cycle' }); return; } + if (visited.has(path)) return; + visiting.add(path); + for (const callee of outgoing.get(path)) visit(callee); + visiting.delete(path); + visited.add(path); + } + for (const path of [...sources.keys()].sort(compare)) visit(path); + return report(runners, calls, diagnostics); +} + +/** Stable review surface: all non-reviewed-Linux selector spellings plus every ancestor call. */ +export function reviewedRunnerSurface(inventory) { + const withoutLine = ({ line: _line, ...entry }) => entry; + const candidates = []; + let projectedBytes = 0; + for (const entry of inventory.runners.filter((item) => item.classification !== 'linux-literal')) { + const workflows = new Set([entry.file]); + const callers = new Map(); + const pending = [entry.file]; + while (pending.length) { + const target = pending.pop(); + for (const call of inventory.calls.filter((edge) => edge.callee === target)) { + callers.set(call.id, withoutLine(call)); + if (!workflows.has(call.file)) { workflows.add(call.file); pending.push(call.file); } + } + } + const base = withoutLine(entry); + const orderedCallers = [...callers.values()].sort((a, b) => compare(a.id, b.id)); + const candidate = { ...base, callers: orderedCallers }; + projectedBytes += 2 * Buffer.byteLength(JSON.stringify(candidate), 'utf8') + 256; + if (projectedBytes > MAX_PROJECTED_SURFACE_BYTES) { + return { schemaVersion: 1, graphComplete: false, candidates: [] }; + } + candidates.push(candidate); + } + return { schemaVersion: 1, graphComplete: inventory.graphComplete, candidates }; +} + +/** Load regular workflow files only; reject symlink entries. No remote calls or workflow execution. */ +export function loadWorkflowSources(directory) { + const names = readdirSync(directory).filter((name) => /\.ya?ml$/i.test(name)).sort(compare); + if (names.length > MAX_FILES) throw new Error('Workflow source limit exceeded'); + let total = 0; + return names.map((name) => { + const location = join(directory, name); + const stat = lstatSync(location); + total += stat.size; + if (!stat.isFile() || stat.size > MAX_FILE_BYTES || total > MAX_TOTAL_BYTES) throw new Error('Unsupported workflow source'); + return { path: `.github/workflows/${name}`, text: readFileSync(location, 'utf8') }; + }); +} + +function main() { + let result; + try { + const args = process.argv.slice(2); + if (args.length !== 0 && (args.length !== 2 || args[0] !== '--workflows' || !args[1] || args[1].startsWith('--'))) throw new Error('Invalid arguments'); + result = inventoryWorkflowRunners(loadWorkflowSources(args[1] ?? '.github/workflows')); + } catch { + result = report([], [], [{ code: 'source-unavailable' }]); + } + process.stdout.write(`${JSON.stringify(result, null, 2)}\n`); + // Exit zero means discovery succeeded, never that rehearsal is safe or authorized. + if (!result.graphComplete) process.exitCode = 2; +} + +function isMain() { + try { return Boolean(process.argv[1]) && realpathSync(process.argv[1]) === realpathSync(fileURLToPath(import.meta.url)); } + catch { return false; } +} +if (isMain()) main(); diff --git a/scripts/ci/smart-ci/workflow-runner-inventory.test.mjs b/scripts/ci/smart-ci/workflow-runner-inventory.test.mjs new file mode 100644 index 000000000..b797032f4 --- /dev/null +++ b/scripts/ci/smart-ci/workflow-runner-inventory.test.mjs @@ -0,0 +1,340 @@ +import assert from 'node:assert/strict'; +import { spawnSync } from 'node:child_process'; +import { mkdirSync, mkdtempSync, readFileSync, rmSync, symlinkSync, writeFileSync } from 'node:fs'; +import { tmpdir } from 'node:os'; +import { join } from 'node:path'; +import { test } from 'node:test'; +import { fileURLToPath } from 'node:url'; +import { inventoryWorkflowRunners, loadWorkflowSources, reviewedRunnerSurface } from './workflow-runner-inventory.mjs'; + +const path = (name) => `.github/workflows/${name}.yml`; +const file = (name, text) => ({ path: path(name), text }); +const job = (name, body) => ` ${name}:\n${body}`; +const workflow = (name, body) => file(name, `on: [push, workflow_call]\njobs:\n${body}`); +const runner = (name, selector, extra = '') => workflow(name, job('build', ` runs-on: ${selector}\n${extra} steps:\n - run: echo ok\n`)); +const caller = (name, target, extra = '') => workflow(name, job('call', ` uses: ./${path(target)}\n${extra}`)); +const key = (name, id = 'build') => `${path(name)}#${id}`; +const inventory = (...files) => inventoryWorkflowRunners(files); + +function assertIncomplete(files, code) { + const result = inventoryWorkflowRunners(files); + assert.equal(result.graphComplete, false); + assert.ok(result.diagnostics.some((entry) => entry.code === code), JSON.stringify(result.diagnostics)); +} + +test('literal and opaque runner sites are discovered, never inferred from a step script', () => { + const result = inventory( + runner('linux', 'ubuntu-latest', ' env:\n NOT_A_SELECTOR: windows-latest\n'), + runner('windows', "'windows-latest' # a comment"), + runner('matrix', '${{ matrix.os }}', ' strategy:\n matrix:\n os: [ubuntu-latest, windows-latest]\n'), + ); + assert.equal(result.graphComplete, true); + assert.equal(result.runners.length, 3); + assert.deepEqual(result.runners.map((entry) => [entry.id, entry.classification]), [ + [key('linux'), 'linux-literal'], [key('matrix'), 'opaque'], [key('windows'), 'windows-literal'], + ]); + assert.equal(result.runners[1].strategy.includes('windows-latest'), true); +}); + +for (const [shape, selector] of [ + ['expression', '${{ inputs.runner }}'], + ['flow labels', '[self-hosted, Windows, X64]'], + ['block labels', '\n - ubuntu-latest\n - custom-label'], + ['group', '\n group: windows-runners\n labels: X64'], + ['alias', '*sharedRunner'], + ['folded scalar', '>\n windows-latest'], + ['custom label', 'my-linux-looking-runner'], + ['unreviewed platform', 'macos-latest'], +]) { + test(`${shape} selector stays in the conservative candidate set`, () => { + const result = inventory(runner('fixture', selector)); + assert.equal(result.graphComplete, true); + const surface = reviewedRunnerSurface(result); + assert.equal(surface.candidates.length, 1); + assert.equal(surface.candidates[0].classification, 'opaque'); + }); +} + +test('conditions and matrix exclusions cannot erase a Windows candidate', () => { + const result = inventory(runner('fixture', '${{ matrix.os }}', + ' if: false\n strategy:\n matrix:\n os: [ubuntu-latest, windows-latest]\n exclude:\n - os: windows-latest\n include:\n - os: windows-latest\n')); + const [entry] = reviewedRunnerSurface(result).candidates; + assert.equal(entry.condition, 'false'); + assert.match(entry.strategy, /exclude:/); + assert.match(entry.strategy, /include:/); +}); + +test('transitive reusable callers preserve distinct job routes, conditions and inputs', () => { + const result = inventory( + runner('leaf', '${{ matrix.os }}'), + caller('middle', 'leaf', ' if: false\n with:\n platform: linux\n'), + workflow('root', job('first', ` uses: ./${path('middle')}\n`) + job('second', ` uses: ./${path('middle')}\n needs: first\n`)), + ); + assert.equal(result.graphComplete, true); + const [entry] = reviewedRunnerSurface(result).candidates; + assert.deepEqual(entry.callers.map((edge) => edge.id), [key('middle', 'call'), key('root', 'first'), key('root', 'second')]); + assert.equal(entry.callers[0].condition, 'false'); + assert.match(entry.callers[0].inputs, /platform: linux/); + assert.equal(entry.callers[2].needs, 'first'); + assert.equal(entry.callers[0].events, '[push, workflow_call]'); +}); + +test('identical job IDs in different workflows remain separate', () => { + const result = inventory(runner('a', 'windows-latest'), runner('b', 'windows-latest')); + assert.deepEqual(result.runners.map((entry) => entry.id), [key('a'), key('b')]); +}); + +test('input order and LF versus CRLF do not change the reviewed surface', () => { + const files = [runner('leaf', 'windows-latest'), caller('root', 'leaf')]; + const first = reviewedRunnerSurface(inventoryWorkflowRunners(files)); + const second = reviewedRunnerSurface(inventoryWorkflowRunners(files.reverse().map((entry) => ({ ...entry, text: entry.text.replace(/\n/g, '\r\n') })))); + assert.deepEqual(first, second); +}); + +test('private step contents do not leak into the control projection', () => { + const source = runner('fixture', 'windows-latest'); + source.text += ' - run: |\n jobs:\n hidden:\n runs-on: PRIVATE_STEP_SENTINEL\n'; + const result = inventory(source); + assert.equal(result.graphComplete, true); + assert.equal(result.runners.length, 1); + assert.doesNotMatch(JSON.stringify(result), /PRIVATE_STEP_SENTINEL/); +}); + +for (const [name, text, code] of [ + ['inline jobs', 'jobs: { build: { runs-on: windows-latest } }\n', 'unsupported-jobs-mapping'], + ['aliased jobs', 'jobs: *shared\n', 'unsupported-jobs-mapping'], + ['aliased job', 'jobs:\n build: *shared\n', 'unsupported-job-mapping'], + ['flow job', 'jobs:\n build: { runs-on: windows-latest }\n', 'unsupported-job-mapping'], + ['quoted jobs', '"jobs":\n build:\n runs-on: windows-latest\n', 'unsupported-mapping'], + ['quoted job', 'jobs:\n "build":\n runs-on: windows-latest\n', 'unsupported-mapping'], + ['quoted selector key', 'jobs:\n build:\n "runs-on": windows-latest\n', 'unsupported-mapping'], + ['job merge', 'jobs:\n build:\n <<: *defaults\n runs-on: ubuntu-latest\n', 'unsupported-mapping'], + ['root merge', '<<: *defaults\njobs:\n build:\n runs-on: ubuntu-latest\n', 'unsupported-mapping'], + ['duplicate root', 'jobs:\n build:\n runs-on: ubuntu-latest\njobs:\n hidden:\n runs-on: windows-latest\n', 'duplicate-key'], + ['duplicate job', 'jobs:\n build:\n runs-on: ubuntu-latest\n build:\n runs-on: windows-latest\n', 'duplicate-key'], + ['duplicate selector', 'jobs:\n build:\n runs-on: ubuntu-latest\n runs-on: windows-latest\n', 'duplicate-key'], + ['tab indentation', 'jobs:\n\tbuild:\n\t runs-on: windows-latest\n', 'unsupported-indentation'], + ['alternate indentation', 'jobs:\n build:\n runs-on: windows-latest\n', 'unsupported-indentation'], + ['document boundary', '---\njobs:\n build:\n runs-on: ubuntu-latest\n', 'unsupported-mapping'], + ['missing runner', 'jobs:\n build:\n steps: []\n', 'runner-or-call-required'], + ['runner and call', `jobs:\n build:\n runs-on: ubuntu-latest\n uses: ./${path('leaf')}\n`, 'ambiguous-job'], +]) { + test(`${name} cannot silently become a complete graph`, () => assertIncomplete([file('fixture', text)], code)); +} + +test('empty or incomplete workflow sets never look qualified', () => { + assertIncomplete([], 'empty-source'); + assertIncomplete([file('fixture', 'name: Empty\n')], 'jobs-required'); + assertIncomplete([file('fixture', 'jobs:\n')], 'jobs-required'); + assertIncomplete([caller('root', 'missing')], 'missing-workflow'); +}); + +test('external reusable workflows are recorded and incomplete, not silently omitted', () => { + const result = inventory(workflow('root', job('remote', ' uses: other/repo/.github/workflows/build.yml@abc\n'))); + assert.equal(result.graphComplete, false); + assert.equal(result.calls.length, 1); + assert.equal(result.diagnostics[0].code, 'unresolved-workflow'); +}); + +test('a step-level action is not a reusable workflow call', () => { + const source = runner('linux', 'ubuntu-latest'); + source.text += ' - uses: owner/repo/.github/workflows/not-a-job.yml@abc\n'; + assert.equal(inventory(source).calls.length, 0); +}); + +test('cyclic local workflows cannot produce a complete graph', () => { + assertIncomplete([caller('a', 'b'), caller('b', 'a')], 'workflow-cycle'); + assertIncomplete([caller('self', 'self')], 'workflow-cycle'); +}); + +test('duplicate and non-canonical source paths fail closed', () => { + assertIncomplete([runner('a', 'ubuntu-latest'), runner('a', 'windows-latest')], 'duplicate-file'); + assertIncomplete([{ path: '../hidden.yml', text: runner('a', 'windows-latest').text }], 'invalid-source'); +}); + +test('source limits fail closed before discovery', () => { + assertIncomplete(Array.from({ length: 257 }, (_, index) => runner(`file-${index}`, 'ubuntu-latest')), 'source-limit'); + assertIncomplete([file('large', ' '.repeat(2 * 1024 * 1024 + 1))], 'source-limit'); +}); + +test('large repeated trigger projections fail closed before report expansion', () => { + const jobs = Array.from({ length: 512 }, (_, index) => + ` job-${index}:\n runs-on: windows-latest\n`, + ).join(''); + const result = inventoryWorkflowRunners([ + file('large-events', `on:\n description: ${'x'.repeat(20_000)}\njobs:\n${jobs}`), + ]); + assert.equal(result.graphComplete, false); + assert.equal(result.runners.length, 0); + assert.ok(result.diagnostics.some((entry) => entry.code === 'projection-limit')); +}); + +test('large caller projections fail closed before reviewed-surface expansion', () => { + const jobs = Array.from({ length: 400 }, (_, index) => + ` job-${index}:\n runs-on: windows-latest\n`, + ).join(''); + const result = inventoryWorkflowRunners([ + file('leaf', `on: [push, workflow_call]\njobs:\n${jobs}`), + caller('root', 'leaf', ` with:\n description: ${'x'.repeat(20_000)}\n`), + ]); + assert.equal(result.graphComplete, true); + const surface = reviewedRunnerSurface(result); + assert.equal(surface.graphComplete, false); + assert.deepEqual(surface.candidates, []); +}); + +test('large caller-free projections fail closed before reviewed-surface expansion', () => { + const inventory = { + graphComplete: true, + runners: Array.from({ length: 1100 }, (_, index) => ({ + id: `.github/workflows/large-${index}.yml#build`, + file: `.github/workflows/large-${index}.yml`, + job: 'build', + line: 1, + events: 'x'.repeat(4096), + condition: null, + needs: null, + strategy: null, + selector: 'windows-latest', + classification: 'windows-literal', + })), + calls: [], + }; + + const surface = reviewedRunnerSurface(inventory); + assert.equal(surface.graphComplete, false); + assert.deepEqual(surface.candidates, []); +}); + +test('caps parser diagnostics before appending per-line errors', () => { + const malformed = Array.from({ length: 10_000 }, (_, index) => `? invalid-${index}`).join('\n'); + const result = inventory(file('diagnostic-flood', malformed)); + + assert.equal(result.graphComplete, false); + assert.ok(result.diagnostics.some((entry) => entry.code === 'diagnostic-limit')); + assert.ok(result.diagnostics.length <= 4097); +}); + +test('the reviewed runner surface detects a new Windows job and a new caller', () => { + const leaf = runner('leaf', 'windows-latest'); + const baseline = reviewedRunnerSurface(inventory(leaf)); + assert.notDeepEqual(reviewedRunnerSurface(inventory(leaf, runner('new', 'windows-latest'))), baseline); + assert.notDeepEqual(reviewedRunnerSurface(inventory(leaf, caller('new', 'leaf'))), baseline); + const changed = { ...leaf, text: leaf.text.replace('windows-latest', '${{ vars.runner }}') }; + assert.notDeepEqual(reviewedRunnerSurface(inventory(changed)), baseline); +}); + +test('the current repository Windows/opaque runner sites and routes match the reviewed inventory', () => { + const files = loadWorkflowSources(fileURLToPath(new URL('../../../.github/workflows/', import.meta.url))); + const result = inventoryWorkflowRunners(files); + assert.equal(result.graphComplete, true, JSON.stringify(result.diagnostics)); + const surface = reviewedRunnerSurface(result); + assert.equal(surface.candidates.length, 5, 'expected five baseline Windows or opaque runner sites'); + const expected = JSON.parse(readFileSync(new URL('./test-support/workflow-runner-inventory.snapshot.json', import.meta.url), 'utf8')); + assert.deepEqual(surface, expected, 'runner topology changed: review the new site/route and update the inventory deliberately'); +}); + +test('CLI reports incomplete input with a nonzero exit and no qualification flag', (t) => { + const root = mkdtempSync(join(tmpdir(), 'td-workflow-inventory-')); + t.after(() => rmSync(root, { recursive: true, force: true })); + writeFileSync(join(root, 'bad.yml'), 'jobs: *unknown\n'); + const script = fileURLToPath(new URL('./workflow-runner-inventory.mjs', import.meta.url)); + const result = spawnSync(process.execPath, [script, '--workflows', root], { encoding: 'utf8', timeout: 10_000 }); + assert.equal(result.error, undefined); + assert.equal(result.status, 2); + const report = JSON.parse(result.stdout); + assert.equal(report.graphComplete, false); + assert.equal(Object.hasOwn(report, 'qualification'), false); +}); + + +test('block control values preserve internal blank and hash-prefixed scalar lines', () => { + const extra = ' with:\n description: |\n before\n\n # literal value, not a YAML comment\n after\n'; + const result = inventory(caller('root', 'leaf', extra), runner('leaf', 'windows-latest')); + const [entry] = reviewedRunnerSurface(result).candidates; + assert.match(entry.callers[0].inputs, /before\n\n # literal value, not a YAML comment\n after/); +}); + +test('keep-chomp block scalars preserve trailing blank lines', () => { + const extra = [ + ' with:', + ' description: |+', + ' before', + '', + '', + ].join('\n'); + const result = inventory(caller('root', 'leaf', extra), runner('leaf', 'windows-latest')); + const [entry] = reviewedRunnerSurface(result).candidates; + assert.match(entry.callers[0].inputs, /description: \|\+\n before\n\n$/); +}); + +test('keep-chomp indicators after YAML properties preserve trailing blank lines', () => { + for (const value of ['&saved |+', '!!str |+', '&saved !!str |+']) { + const extra = [ + ' with:', + ` description: ${value}`, + ' before', + '', + '', + ].join('\n'); + const result = inventory(caller('root', 'leaf', extra), runner('leaf', 'windows-latest')); + const [entry] = reviewedRunnerSurface(result).candidates; + assert.match(entry.callers[0].inputs, new RegExp(`description: ${value.replace(/[|+]/g, '\\$&')}\\n before\\n\\n$`)); + } +}); + +test('input alias and folded reusable references stay explicitly unresolved', () => { + for (const value of ['*reference', '>\n ./.github/workflows/leaf.yml']) { + assertIncomplete([workflow('root', job('call', ` uses: ${value}\n`)), runner('leaf', 'windows-latest')], 'unresolved-workflow'); + } +}); + +test('unknown source value types fail closed', () => { + assert.equal(inventoryWorkflowRunners(null).graphComplete, false); + assertIncomplete([null], 'invalid-source'); + assertIncomplete([{ path: path('file'), text: {} }], 'invalid-source'); +}); + +test('source loader includes both YAML extensions and refuses directory or symlink entries', (t) => { + const root = mkdtempSync(join(tmpdir(), 'td-inventory-loader-')); + t.after(() => rmSync(root, { recursive: true, force: true })); + writeFileSync(join(root, 'a.yml'), runner('a', 'ubuntu-latest').text); + writeFileSync(join(root, 'b.yaml'), runner('b', 'windows-latest').text); + writeFileSync(join(root, 'not-a-workflow.txt'), 'not workflow source'); + assert.deepEqual(loadWorkflowSources(root).map((entry) => entry.path), [path('a'), '.github/workflows/b.yaml']); + mkdirSync(join(root, 'directory.yml')); + assert.throws(() => loadWorkflowSources(root), /Unsupported workflow source/); + rmSync(join(root, 'directory.yml'), { recursive: true }); + mkdirSync(join(root, 'target')); + symlinkSync(join(root, 'target'), join(root, 'linked.yml'), 'junction'); + assert.throws(() => loadWorkflowSources(root), /Unsupported workflow source/); +}); + +test('a differently-cased YAML extension is not silently omitted', (t) => { + const root = mkdtempSync(join(tmpdir(), 'td-inventory-extension-')); + t.after(() => rmSync(root, { recursive: true, force: true })); + writeFileSync(join(root, 'upper.YML'), runner('upper', 'windows-latest').text); + assertIncomplete(loadWorkflowSources(root), 'invalid-source'); +}); + +test('importing the inventory has no filesystem or CLI side effects', () => { + const url = new URL('./workflow-runner-inventory.mjs', import.meta.url).href; + const result = spawnSync(process.execPath, ['--input-type=module', '-e', `await import(${JSON.stringify(url)})`], { encoding: 'utf8', timeout: 10_000 }); + assert.equal(result.error, undefined); + assert.equal(result.status, 0); + assert.equal(result.stdout, ''); + assert.equal(result.stderr, ''); +}); + +test('a comment after a plain Linux selector does not create an opaque runner', () => { + const result = inventory(runner('fixture', 'ubuntu-latest', ' # Docker is installed on this runner\n')); + assert.equal(result.graphComplete, true); + assert.equal(result.runners[0].classification, 'linux-literal'); +}); + +test('an attached hash stays part of the runner scalar', () => { + const result = inventory(runner('fixture', 'ubuntu-latest#custom')); + assert.equal(result.graphComplete, true); + assert.equal(result.runners[0].selector, 'ubuntu-latest#custom'); + assert.equal(result.runners[0].classification, 'opaque'); +});