Skip to content

ci: port the workflows to .forgejo/, and correct the context-format finding #527

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
ProtocolWarden merged 1 commit into from
Aug 19, 2026
Merged

ci: port the workflows to .forgejo/, and correct the context-format finding #527

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
23 changes: 23 additions & 0 deletions .console/log.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,3 +1,26 @@
## 2026-08-19 — correcting myself: the status context is the JOB name

#526 said `run-name:` pins Forgejo's status context. Wrong, and I generalised
it from a failure case. A two-job workflow with `run-name: probe` produces
`multi / alpha` and `multi / beta` — `probe` appears in neither. A job with
`name: Pretty Job Name` produces `naming / Pretty Job Name`. The format is
`<workflow name> / <job name, or id> (<event>)`, **stable by default**.

The commit-message form I saw first (`audit.yml / ci: probe the... (push)`)
only happens when a run fails *before any job starts*: with no job to name,
Forgejo falls back to the file name and run title. I read a fallback as the
rule.

Also found while porting: Forgejo executes `.github/workflows/` as well as
`.forgejo/`, so pushing OC to Forgejo handed one local runner the entire GitHub
CI suite — 27 tasks, most failing. Operator chose: port everything to
`.forgejo/` and delete `.github/`.

Ordering constraint that falls out: **the deletion cannot merge on GitHub.** It
removes the `audit` status branch protection requires, with enforce_admins on,
so that PR could never go green. The port must land additively first, the fleet
cuts over, and the deletion merges on Forgejo afterwards.

## 2026-08-19 — B4 resolved: the audit context cannot match GitHub's

Forgejo Actions is running: runner 6.3.1 registered against the live instance,
Expand Down
332 changes: 332 additions & 0 deletions .forgejo/workflows/ci.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,332 @@
# Ported from .github/workflows/ci.yml for the Forgejo cutover.
#
# Two deliberate differences from the GitHub original:
#
# * `on:` is pull_request only. On Forgejo a `push` trigger produces a SECOND,
# separate status context on the same head ("... (push)") alongside the
# pull_request one — duplicating every job on a single self-hosted runner and
# leaving an extra check outstanding for any gate that requires nothing
# incomplete.
# * Nothing else. The steps are byte-identical, because a gate that runs a
# different command is not the same gate.
#
# Status context format is `<workflow name:> / <job name:> (<event>)`, so the
# contexts these produce are listed in docs/specs/forgejo-pr-adapter.md and are
# what Forgejo branch protection must require.

name: CI

on:
pull_request:
branches: [main]

jobs:
lint:
name: Lint (ruff)
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- uses: actions/setup-python@v5
with:
python-version: "3.11"
- name: Install ruff
# Install the repo's own pinned toolchain rather than `ruff>=0.5`. A lint
# gate has to run the version the config was written against: `[tool.ruff]`
# here selects a deliberate rule set (BLE001 and S110 are explicitly
# DROPPED as too noisy, per the comment in pyproject), and a newer ruff
# re-enables rules that config never opted into. `ruff>=0.5` floated up to
# 0.16.1 and this job went from clean to 1996 errors — every one of them
# phantom — red-failing CI on main daily from ~2026-07-29. Taking the pin
# from [project.optional-dependencies].dev keeps one source of truth; do
# not reintroduce a version literal here.
run: pip install -e ".[dev]"
- name: Run ruff
run: ruff check .

typecheck:
name: Type check (ty)
runs-on: ubuntu-latest
# Was advisory (continue-on-error) while 21 pre-existing diagnostics
# were unresolved. Cleared on 2026-05-07 — now blocking.
steps:
- uses: actions/checkout@v4
- uses: actions/setup-python@v5
with:
python-version: "3.11"
- name: Install dependencies
run: pip install -e .[dev]
- name: Run ty
run: ty check src/

custodian:
name: Custodian doctor
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- uses: actions/setup-python@v5
with:
python-version: "3.11"
- name: Install dependencies
run: pip install -e .[dev]
- name: Verify .custodian.yaml is well-formed
run: custodian-doctor --strict --repo .

license-check:
name: License headers
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- name: Check SPDX headers
run: |
missing=$(git ls-files "*.py" | xargs grep -L "SPDX-License-Identifier" 2>/dev/null || true)
if [ -n "$missing" ]; then
echo "Missing SPDX-License-Identifier header in:"
echo "$missing"
exit 1
fi

test:
name: Test (pytest)
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- uses: actions/setup-python@v5
with:
python-version: "3.11"
- name: Install dependencies
run: pip install -e .[dev]
- name: Run unit tests excluding slow (PR validation)
if: github.event_name == 'pull_request'
# Unit suite only — integration tests under tests/integration/ need
# live external services (SwitchBoard, Plane, Archon) and run on
# demand, not in CI. PRs exclude slow tests for quick validation.
# Coverage threshold of 85% is the design target from Stage 0.
# CI will fail until coverage improves to meet this target.
# -p no:flaky-detection — the pytest11 entry point imports the whole
# observer package at pytest startup, BEFORE coverage instrumentation,
# marking every module-level line uncovered. The plugin is opt-in by
# design; coverage jobs don't need it loaded.
run: pytest -q tests/unit -m "not slow" -p no:flaky-detection --cov=src --cov-report=html --cov-report=xml --cov-report=term-missing --cov-fail-under=85
- name: Run full unit test suite including slow (main/merge)
if: github.event_name == 'push'
# Unit suite only — integration tests under tests/integration/ need
# live external services (SwitchBoard, Plane, Archon) and run on
# demand, not in CI. Pushes run full suite including slow tests.
# Coverage threshold of 85% is the design target from Stage 0.
# CI will fail until coverage improves to meet this target.
# -p no:flaky-detection — see PR-validation step above.
run: pytest -q tests/unit -p no:flaky-detection --cov=src --cov-report=html --cov-report=xml --cov-report=term-missing --cov-fail-under=85
- name: Upload coverage to Codecov
if: always()
uses: codecov/codecov-action@v4
with:
file: ./coverage.xml
flags: unittests
name: codecov-umbrella
fail_ci_if_error: false
token: ${{ secrets.CODECOV_TOKEN }}
- name: Upload coverage reports as artifacts
if: always()
uses: actions/upload-artifact@v4
with:
name: coverage-reports-${{ github.event_name }}
path: |
coverage_html_report/
coverage.xml
retention-days: 30

reviewer:
name: Reviewer state-machine tests
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- uses: actions/setup-python@v5
with:
python-version: "3.11"
- name: Install dependencies
run: pip install -e .[dev]
- name: Run pr_review_watcher tests
# The reviewer self-review / verdict-gate / Self-Heal Ladder state machine
# lives at tests/test_pr_review_watcher.py (repo ROOT, not tests/unit), so
# the main "Test (pytest)" job — which runs `pytest tests/unit` — never
# exercised it. This dedicated job gates the governance + self-heal code
# (the #313 regression class: never merge over a CONCERNS verdict; resolve
# concerns instead of conceding) on every PR. Standalone, no live services.
# Isolated from tests/unit so it cannot perturb the doc-accuracy suite's
# collection-count assertions.
run: pytest -q tests/test_pr_review_watcher.py -p no:flaky-detection

reviewer-integration:
name: Reviewer integration tests
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- uses: actions/setup-python@v5
with:
python-version: "3.11"
- name: Install dependencies
run: pip install -e .[dev]
- name: Run reviewer integration tests
# tests/integration/reviewer exercises the verdict-consolidation +
# merge-decision state machine (happy path, CI-green gate, verdict
# override, timeout recovery, safety paths, instrumentation) end-to-end
# through pr_review_watcher._phase1. Despite living under tests/integration/,
# these are HERMETIC: the GitHub and Plane clients are fully mocked
# (tests/verdicts/conftest.py), so there are NO live services and NO
# network — they run in <1s. The blanket "integration/ needs live services,
# run on demand" policy that excludes the rest of tests/integration/ from CI
# let this suite DRIFT against the evolving self-merge gates (e.g. the
# require_branch_protection fail-closed gate added in #388). This dedicated
# job runs them on every PR so they cannot rot again.
# -p no:flaky-detection — see the "Test (pytest)" job (plugin is opt-in;
# its pytest11 entry point imports the observer package before coverage
# instrumentation). Isolated from tests/unit so it cannot perturb the
# doc-accuracy suite's collection-count assertions.
run: pytest -q tests/integration/reviewer -p no:flaky-detection

test-rest:
name: Test (suites outside tests/unit)
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- uses: actions/setup-python@v5
with:
python-version: "3.11"
- name: Install dependencies
run: pip install -e .[dev]
- name: Run every suite the other jobs do not
# The complement of `tests/unit`. Two dedicated jobs above already gate
# single files that had drifted (tests/test_pr_review_watcher.py,
# tests/integration/reviewer) — each added after that specific suite rotted
# unnoticed. This job generalises the fix instead of waiting for the next
# one: ~1,830 tests under tests/maintenance/, tests/observer/,
# tests/verdicts/, the rest of tests/integration/, and the top-level
# tests/test_*.py had NO job at all.
#
# What that cost, concretely:
# #509 shipped a regression that broke tests/maintenance/ — found a week
# later by hand (#513).
# #521 merged a BRAND NEW failing test in tests/test_dependency_check.py.
# CI was green, the reviewer verdict was SUCCESS, and main stayed red
# until #522.
# tests/test_proposer_entrypoint.py had been failing since the board-seam
# migration weeks earlier; nothing noticed.
#
# --ignore=tests/unit keeps this isolated from the unit job, whose
# doc-accuracy suite asserts collection counts and is perturbed by running
# alongside other trees. -p no:flaky-detection for the same reason as the
# jobs above: the plugin's pytest11 entry point imports the observer package
# before coverage instrumentation.
run: pytest -q tests/ --ignore=tests/unit -p no:flaky-detection

performance:
name: Performance regression tests
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- uses: actions/setup-python@v5
with:
python-version: "3.11"
- name: Install dependencies
run: pip install -e .[dev]
- name: Run performance regression tests
# Dedicated job surfaces timing-bound failures as a labelled CI check.
# All bounds are <50ms per scenario; actual measurements are ~0.1–0.2ms.
# A failure here means collection time regressed by >250x vs baseline.
run: pytest -q tests/unit -m "perf" --no-header -rN

snapshot:
name: Snapshot validation
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- uses: actions/setup-python@v5
with:
python-version: "3.11"
- name: Install dependencies
run: pip install -e .[dev]
- name: Run snapshot validation tests (PR — quick, excluding slow)
if: github.event_name == 'pull_request'
# Snapshot validation uses 5-layer architecture:
# Layer 1: Schema validation (JSON ↔ Pydantic model roundtrip)
# Layer 2: Completeness validation (required signals present, min 3 non-unavailable)
# Layer 3: Consistency validation (cross-signal semantic checks)
# Layer 4: Real-world accuracy validation (snapshot vs. live tools — marked slow)
# Layer 5: Regression detection (baseline comparison — marked slow)
#
# PR runs exclude slow tests for quick feedback.
# All validation checks must pass; structural failures trigger job failure.
# Transient failures (network, timeouts) are retried up to 3 times.
run: pytest -q tests/integration/observer -m "integration and not slow" --no-header -rN
- name: Run snapshot validation tests (push — full, including slow)
if: github.event_name == 'push'
# Full validation including real-world accuracy (Layer 4) and regression detection (Layer 5).
# These layers test against actual repository state and may take longer.
run: pytest -q tests/integration/observer -m "integration" --no-header -rN
- name: Run snapshot validation tests (scheduled — full, including slow)
if: github.event_name == 'schedule'
# Scheduled runs execute comprehensive validation without time constraints.
# Full validation detects regressions in repository state snapshots,
# even when no new code changes occur. Failures trigger immediate alerts.
run: pytest -q tests/integration/observer -m "integration" --no-header -rN
- name: Upload snapshot validation reports
if: always()
uses: actions/upload-artifact@v4
with:
name: snapshot-validation-reports-${{ github.event_name }}
path: |
tests/integration/observer/validation_reports/
retention-days: 30
strategy:
fail-fast: true

flaky-test-detection:
name: Flaky test detection
runs-on: ubuntu-latest
if: github.event_name == 'push' # Run on merges (not PRs) to detect trends
steps:
- uses: actions/checkout@v4
- uses: actions/setup-python@v5
with:
python-version: "3.11"
- name: Install dependencies
run: pip install -e .[dev]
- name: Run unit tests with flaky detection
# Runs all unit tests with flaky detection plugin enabled.
# Captures test outcomes, analyzes patterns, and saves metrics.
# This provides baseline data for historical trend detection.
run: pytest -q tests/unit --flaky-detection --flaky-storage=.flaky-tests -v --tb=short
- name: Aggregate flakiness history
if: always()
# Aggregates session reports from past 7 days into daily summaries.
# Computes failure rates, trends, and generates recommendations.
# Output: .flaky-tests/aggregations/YYYY-MM-DD-aggregation.json
run: |
python -c "
from pathlib import Path
from operations_center.observer import FlakyTestStorageManager, FlakyTestAggregator, FlakyTestAlertManager
storage = FlakyTestStorageManager.create_local('.flaky-tests')
agg = FlakyTestAggregator(storage)
report = agg.aggregate(days=7)
storage.save_aggregation(report)
alerts = FlakyTestAlertManager.check_alerts(report)
print('Aggregation complete: ' + str(report.flaky_test_count) + ' flaky tests')
for alert in alerts:
print(' [' + alert.severity.value.upper() + '] ' + alert.alert_type + ': ' + alert.description)
"
- name: Upload flaky test metrics
if: always()
uses: actions/upload-artifact@v4
with:
name: flaky-test-metrics-${{ github.run_id }}
path: |
.flaky-tests/runs/
.flaky-tests/aggregations/
retention-days: 90
- name: Report aggregation status
if: always()
run: |
echo "Flaky test aggregation completed"
ls -la .flaky-tests/aggregations/ 2>/dev/null || echo "No aggregation files created"
strategy:
fail-fast: false
Loading
Loading