Skip to content
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
53 changes: 51 additions & 2 deletions CI.md
Original file line number Diff line number Diff line change
@@ -1,11 +1,60 @@
# CI

This document describes how OpenShell's continuous integration works for pull requests, with a focus on what contributors need to do to get their PR tested.
This document describes the current OpenShell continuous integration system,
with a focus on what contributors need to do to get a pull request tested. It
documents implemented workflow behavior rather than the complete desired test
architecture.

For local test commands see [TESTING.md](TESTING.md). For PR conventions see [CONTRIBUTING.md](CONTRIBUTING.md).
For the target testing model and canonical local entry points, see
[TESTING.md](TESTING.md). For pull request conventions, see
[CONTRIBUTING.md](CONTRIBUTING.md).

## Overview

CI implements the testing layers defined in `TESTING.md` through separate
workflow families:

| Testing responsibility | Current CI implementation |
|---|---|
| Static, build, unit, component, and SDK checks | `Branch Checks` across Linux x86_64, Linux ARM64, and macOS ARM64, with additional language-specific jobs |
| Runtime and cross-component validation | Label-selected `Branch E2E Checks` for Docker, Podman, Kubernetes, VM, GPU, MCP, Python, and managed or standalone drivers |
| Nix and `tmachine` validation | Release Dev builds exact-revision artifacts and test archives, then runs CLI conformance in disposable Ubuntu/Docker and Fedora/Podman guests |
| Windows compatibility | Opt-in Windows MSVC x64 and ARM64 jobs; main and manual runs also build release binaries |
| Integrated revision validation | Required statuses on merge-group SHAs before `main` advances |
| Published artifact validation | Release workflows and post-publish release canaries |
| Security analysis | Required dependency and deployment gates plus informational reports described below |

This is not yet the full target state. The project is migrating integration and
Linux installation testing from `mise` tasks and workflow-local shell harnesses
to Nix-built artifacts and `tmachine` wherever the runner can represent the
environment. Today that path is limited to release conformance. `TESTING.md`
defines the desired gates, matrices, migration phases, and deferred decisions.
This document should describe a migration step as complete only after the
workflow enforces it.

### Current tmachine implementation

`.github/workflows/conformance.yml` is the first implementation of this model.
It downloads exact-revision binaries and OCI image artifacts, uses
`nix run .#build-artifacts-test-archives` to package the test suite, and fans out
`nix run .#tmachine -- test <scenario> <testsuite>` on KVM-enabled runners. The
workflow caches prepared disks, while `tmachine` gives each test a fresh writable
overlay. Release Dev currently calls the reusable workflow for these pairs:

| Scenario | Testsuite |
|---|---|
| `ubuntu-docker-rootful` | `conformance` |
| `fedora-podman-rootful` | `conformance` |
| `fedora-podman-rootless` | `conformance` |

This workflow currently runs from Release Dev. It is not a required pull-request
or merge-queue gate. Branch E2E, GPU, Kubernetes, VM, Windows, and release-canary
workflows continue to provide the coverage that has not migrated to tmachine.

The desired merge matrix, release-validation model, and migration sequence are
defined in [TESTING.md](TESTING.md). They describe target behavior and do not
become current CI behavior until the corresponding workflows implement them.

PR CI that runs on NVIDIA self-hosted runners uses NVIDIA's copy-pr-bot. The bot mirrors trusted PR commits to internal `pull-request/<N>` branches in this repository. The gated workflows trigger on pushes to those branches, not on the original PR.

`Branch Checks` run automatically after copy-pr-bot mirrors the PR. `Required CI Gates` posts PR-head statuses that verify the mirror exists, is current, and ran the expected push-based workflows. E2E suites are opt-in because they are more expensive and publish temporary images.
Expand Down
Loading
Loading