Skip to content
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
48 changes: 48 additions & 0 deletions .github/workflows/quality-gate-main.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,48 @@
name: quality-gate-baseline
on:
push:
branches: [main]

permissions:
contents: write

concurrency:
# github.workflow keeps per-workspace workflow copies (distinct names) isolated.
group: ${{ github.workflow }}
cancel-in-progress: false

jobs:
update-baseline:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
with: { fetch-depth: 0 }
- uses: actions/setup-node@v4
with: { node-version: "22" }

- name: Install quality-gate engine
# shell: bash makes the runner use `bash -o pipefail`, so a failed curl
# (the default `bash -e` has no pipefail) doesn't pass as a green step.
shell: bash
run: curl -fsSL https://raw.githubusercontent.com/alkg-cloud/quality-gate/main/install.sh | bash
env:
# Pin a tag or commit SHA for reproducible CI; defaults to main.
QG_REF: main

- name: Read adapter command from config
id: cfg
run: |
adapter_cmd=$(node -e "console.log(JSON.parse(require('fs').readFileSync('./quality-gate.config.json','utf-8')).adapter.command)")
echo "adapter_cmd=$adapter_cmd" >> "$GITHUB_OUTPUT"

- name: Run adapter
run: ${{ steps.cfg.outputs.adapter_cmd }}
env:
QG_OUTPUT_DIR: ${{ runner.temp }}/qg
QG_MODE: main
QG_CONFIG: ./quality-gate.config.json

- name: Update baseline on orphan branch
run: qg-core update-baseline --config ./quality-gate.config.json --output-dir ${{ runner.temp }}/qg
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
60 changes: 60 additions & 0 deletions .github/workflows/quality-gate-pr.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,60 @@
name: quality-gate
on:
pull_request:
branches: [main]

permissions:
contents: read
pull-requests: write

concurrency:
# github.workflow keeps per-workspace workflow copies (distinct names) isolated.
group: ${{ github.workflow }}-${{ github.ref }}
cancel-in-progress: true

jobs:
quality-gate:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
with: { fetch-depth: 0 }
- uses: actions/setup-node@v4
with: { node-version: "22" }

- name: Install quality-gate engine
# shell: bash makes the runner use `bash -o pipefail`, so a failed curl
# (the default `bash -e` has no pipefail) doesn't pass as a green step.
shell: bash
run: curl -fsSL https://raw.githubusercontent.com/alkg-cloud/quality-gate/main/install.sh | bash
env:
# Pin a tag or commit SHA for reproducible CI; defaults to main.
QG_REF: main

- name: Read adapter command from config
id: cfg
run: |
adapter_cmd=$(node -e "console.log(JSON.parse(require('fs').readFileSync('./quality-gate.config.json','utf-8')).adapter.command)")
echo "adapter_cmd=$adapter_cmd" >> "$GITHUB_OUTPUT"

- name: Run adapter
run: ${{ steps.cfg.outputs.adapter_cmd }}
env:
QG_OUTPUT_DIR: ${{ runner.temp }}/qg
QG_MODE: pr
QG_CONFIG: ./quality-gate.config.json

- name: Run quality gate
run: qg-core pr --config ./quality-gate.config.json --output-dir ${{ runner.temp }}/qg

- name: Post sticky PR comment
uses: marocchino/sticky-pull-request-comment@v2
with:
header: quality-gate-marker-v1
path: ${{ runner.temp }}/qg/pr-comment.md

- name: Upload artifacts
if: always()
uses: actions/upload-artifact@v4
with:
name: quality-gate-report
path: ${{ runner.temp }}/qg
118 changes: 118 additions & 0 deletions .quality-gate/adapter.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,118 @@
#!/usr/bin/env bash
# Quality Gate adapter for Markup (Next.js / pnpm / Vitest / Biome).
# Tools: vitest+v8 (coverage), biome (lint), jscpd (duplication), pnpm audit (security).
# Reads $QG_OUTPUT_DIR, $QG_CONFIG. Writes 6 canonical JSON files into $QG_OUTPUT_DIR.
#
# The CI workflows check out the repo and provide Node only — no `pnpm install`,
# no DB, no fixtures. This adapter therefore bootstraps its own environment
# (mirroring .github/workflows/test.yml) so coverage reflects the real suite.
#
# Ordering matters: lint runs before coverage so Biome never sees the generated
# coverage/ directory; coverage is produced last.
set -euo pipefail
: "${QG_OUTPUT_DIR:?must be set}"
: "${QG_CONFIG:?must be set}"

mkdir -p "$QG_OUTPUT_DIR"
MAX_FILE_LINES=$(jq -r '.thresholds.MAX_FILE_LINES' "$QG_CONFIG")
ROOT="$PWD"

# Scratch dir for tool reports we don't want to leave in the working tree.
SCRATCH="$(mktemp -d)"
trap 'rm -rf "$SCRATCH"' EXIT

# Clear stale coverage output so the lint scan below sees only the committed tree.
rm -rf coverage

# --- Environment bootstrap (no-op locally where deps already exist) ----------
if ! command -v pnpm >/dev/null 2>&1; then
corepack enable >/dev/null 2>&1 || true
corepack prepare pnpm@10.33.0 --activate >/dev/null 2>&1 || true
fi
if [ ! -d node_modules ]; then
pnpm install --frozen-lockfile
fi

# 1. Lint — Biome (json reporter), mirroring `pnpm lint` (biome check .).
# location.path is already a repo-relative string. Count errors + warnings;
# infos are advisory and excluded.
pnpm exec biome check . --reporter=json > "$SCRATCH/biome.json" 2>/dev/null || true
if [ -s "$SCRATCH/biome.json" ] && jq -e '.summary' "$SCRATCH/biome.json" >/dev/null 2>&1; then
jq '
[ .diagnostics[]? | select(.severity == "error" or .severity == "warning") ] as $d
| {
total: ($d | length),
by_file: (
$d
| map(select((.location.path | type) == "string" and (.location.path | length) > 0))
| group_by(.location.path)
| map({ path: .[0].location.path, count: length })
| sort_by(.path)
)
}
' "$SCRATCH/biome.json" > "$QG_OUTPUT_DIR/lint.json"
else
echo '{"_skipped":"biome failed to produce a report"}' > "$QG_OUTPUT_DIR/lint.json"
fi

# 2. Duplication — jscpd over src/ (fetched on demand; no project dependency).
pnpm dlx jscpd src --reporters json --output "$SCRATCH/jscpd" \
--ignore "**/node_modules/**,**/.next/**,**/coverage/**" --silent >/dev/null 2>&1 || true
if [ -f "$SCRATCH/jscpd/jscpd-report.json" ]; then
jq '{ pct: (.statistics.total.percentage // 0), clones: (.statistics.total.clones // 0) }' \
"$SCRATCH/jscpd/jscpd-report.json" > "$QG_OUTPUT_DIR/duplication.json"
else
echo '{"_skipped":"jscpd failed"}' > "$QG_OUTPUT_DIR/duplication.json"
fi

# 3. File size — tracked source files only (git ls-files keeps this deterministic
# across local runs and the fresh CI checkout, ignoring build/output dirs).
git ls-files -z -- '*.js' '*.jsx' '*.ts' '*.tsx' '*.mjs' '*.cjs' \
| while IFS= read -r -d '' f; do
lines=$(wc -l < "$f")
if [ "$lines" -ge "$MAX_FILE_LINES" ]; then
printf '{"path":"%s","lines":%d}\n' "$f" "$lines"
fi
done | jq -s --argjson max "$MAX_FILE_LINES" \
'{ max_lines: $max, violations: (sort_by(.path)) }' > "$QG_OUTPUT_DIR/file_size.json"

# 4. Security — pnpm audit (npm-compatible metadata.vulnerabilities shape).
pnpm audit --json > "$SCRATCH/audit.json" 2>/dev/null || true
if [ -s "$SCRATCH/audit.json" ] && jq -e '.metadata.vulnerabilities' "$SCRATCH/audit.json" >/dev/null 2>&1; then
jq '.metadata.vulnerabilities
| { critical: (.critical // 0), high: (.high // 0), moderate: (.moderate // 0), low: (.low // 0) }' \
"$SCRATCH/audit.json" > "$QG_OUTPUT_DIR/security.json"
else
echo '{"_skipped":"pnpm audit failed or produced no metadata"}' > "$QG_OUTPUT_DIR/security.json"
fi

# 5. Coverage — Vitest (v8 provider) with the json-summary reporter. Runs last so
# the generated coverage/ dir is absent during the lint scan above. Integration
# tests need a generated Prisma client and the fixture zips; the test DB is
# bootstrapped by tests/setup.ts. Build fixtures only if absent so local runs
# don't dirty the committed (non-deterministic) zips.
pnpm exec prisma generate >/dev/null 2>&1 || true
if [ ! -f tests/fixtures/mockups/valid-simple.zip ]; then
pnpm exec tsx tests/fixtures/build-fixtures.ts >/dev/null 2>&1 || true
fi
pnpm exec vitest run --coverage --coverage.provider=v8 --coverage.reporter=json-summary >/dev/null 2>&1 || true
if [ -f coverage/coverage-summary.json ]; then
jq --arg root "$ROOT/" '{
lines_pct: (.total.lines.pct // 0),
files: [
to_entries[] | select(.key != "total")
| { path: (.key | ltrimstr($root)), lines_pct: (.value.lines.pct // 0) }
] | sort_by(.path)
}' coverage/coverage-summary.json > "$QG_OUTPUT_DIR/coverage.json"
else
echo '{"_skipped":"vitest produced no coverage-summary.json"}' > "$QG_OUTPUT_DIR/coverage.json"
fi

# 6. _meta
cat > "$QG_OUTPUT_DIR/_meta.json" <<'JSON'
{
"adapter": "node",
"adapter_version": "0.1.0",
"tools": ["vitest", "biome", "jscpd", "pnpm-audit"]
}
JSON
20 changes: 20 additions & 0 deletions docs/ci.md
Original file line number Diff line number Diff line change
Expand Up @@ -13,6 +13,26 @@ The single CI workflow runs `lint + typecheck + test + build` on every push. Rea

`pnpm test` runs sequentially (`fileParallelism: false, maxWorkers: 1`) because integration tests share `prisma/test.db`. See [Testing](testing.md) for why and what would replace it.

## Quality gate (ratchet)

Alongside the pass/fail pipeline above, a ratchet-based quality gate (the [`alkg-cloud/quality-gate`](https://github.com/alkg-cloud/quality-gate) engine) runs on every PR and fails it on regression. It is **additive** — it does not replace lint/typecheck/test/build. Per-PR metrics are compared against a baseline stored on the orphan `quality-metrics` branch; the baseline is refreshed on every push to `main`.

| Piece | Location | Notes |
|---|---|---|
| Config | `quality-gate.config.json` | Strict ratchet (`epsilon 0`), `MAX_FILE_LINES` 300, `MIN_NEW_FILE_COVERAGE` 60, blocks on `critical` security advisories and warns on `high`. |
| Adapter | `.quality-gate/adapter.sh` | Emits six canonical metric files (`coverage`, `duplication`, `lint`, `file_size`, `security`, `_meta`). A metric whose tool is unavailable emits `{"_skipped":"…"}` rather than a fabricated value. |
| Workflows | `.github/workflows/quality-gate-pr.yml`, `quality-gate-main.yml` | PR gate (posts a sticky comment) and baseline refresh. Both pin Node 22 to match `engines.node` and install the engine from source via the published `install.sh`. |

The adapter bootstraps its own environment (pnpm via corepack, `pnpm install`, `prisma generate`, fixtures) because the gate workflows check out the repo with Node only, then collects metrics with the project's own tools:

- **Coverage** — Vitest with the v8 provider (`coverage/coverage-summary.json`, lines %), running the full unit + integration suite. It is produced last so the generated `coverage/` directory is absent during the lint scan.
- **Lint** — `biome check .` (the same command as `pnpm lint`), counting errors + warnings per file. Adding a new Biome warning (e.g. an unjustified `any`) is a ratchet regression.
- **Duplication** — `jscpd` over `src/`, fetched on demand.
- **File size** — tracked source files (`git ls-files`) at or above `MAX_FILE_LINES`.
- **Security** — `pnpm audit` severity counts.

The required status check is `quality-gate / quality-gate`; merging the bootstrap PR creates the orphan branch and the first baseline.

## Pre-push checklist

```bash
Expand Down
22 changes: 22 additions & 0 deletions quality-gate.config.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,22 @@
{
"schema_version": 1,
"default_branch": "main",
"branch": "quality-metrics",
"thresholds": {
"MAX_FILE_LINES": 300,
"MIN_NEW_FILE_COVERAGE": 60
},
"ratchet": { "strict": true, "epsilon": 0.0 },
"metrics": {
"coverage": { "enabled": true },
"duplication": { "enabled": true },
"lint": { "enabled": true },
"file_size": { "enabled": true },
"security": { "enabled": true, "block_severities": ["critical"], "warn_severities": ["high"] }
},
"adapter": {
"command": "./.quality-gate/adapter.sh",
"name": "node",
"version": "0.1.0"
}
}