Skip to content

ci: Run the test suite once, in the required check - #849

Merged
vvoytovych merged 1 commit into
mainfrom
ci/run-the-suite-once
Nov 3, 2026
Merged

vvoytovych merged 1 commit into
mainfrom
ci/run-the-suite-once

Conversation

@vvoytovych

@vvoytovych vvoytovych commented Nov 3, 2026 •

Copy link
Copy Markdown
Contributor

The required check, ci / Build and test, and the Tests workflow both run the whole unit and envtest suite on every pull request, so each pull request runs it twice (#837). CI now also runs on pushes to release/**, as Tests does, and takes the concurrency group that the other workflows share, so Tests goes. Every other step of Tests already runs in another check, or checks nothing. make test stays for local use.

Where each step of Tests runs now

step where
make manifests generate Verify Generated Files, which fails on any diff
go vet, go fmt Lint: golangci-lint's govet, gofmt and goimports
the build CI: ci-basic's Go step runs go build ./...
the unit and envtest suite CI
go mod tidy, go fmt as a step nowhere: both rewrite files and never fail
-coverprofile cover.out nowhere: nothing reads it

Corner cases

case what happens proof
A push to a release branch CI runs the suite, as Tests did the same branch filter as Lint and Verify Generated Files, which ran on release/v0.34's last push
A newer push to a pull request The older CI run is cancelled the group the other workflows use
A push to main or a release branch Its run is never cancelled the group is the run's own id there
The required check Keeps its name, ci / Build and test the job is unchanged

Test plan

  • actionlint passes on every workflow, and nothing else in the repository names test.yml or the Tests workflow.
  • This pull request's checks: CI runs, Tests does not.
  • After the merge, the next push to a release branch runs CI.

Refs #837

CI (`ci / Build and test`, the required check) and Tests both ran the
whole unit and envtest suite on every pull request. CI now also runs on
pushes to release branches, as Tests did, and takes the concurrency
group every other workflow has, so Tests goes. Its other steps already
run elsewhere: Verify Generated Files fails on a generation diff, Lint
runs go vet, gofmt and goimports, and CI builds the code. go mod tidy
and go fmt rewrote files and never failed.

Refs #837
@vvoytovych vvoytovych self-assigned this Nov 3, 2026
@vvoytovych
vvoytovych marked this pull request as ready for review November 3, 2026 19:26
@vvoytovych
vvoytovych requested a review from a team as a code owner November 3, 2026 19:26
@vvoytovych
vvoytovych requested a review from ecv November 3, 2026 19:26

@ecv ecv left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Approve.

Deleting test.yml removes a second run of the same suite: ci.yaml's ci / Build and test already runs the unit and envtest packages (go test $(go list ./... | grep -v /e2e) -timeout 25m under setup-envtest), and it is the only check the required-ci ruleset requires, so no required check disappears. The 26-minute duplicate goes, and both checks passed at the head.

  • ci.yaml gains release/** pushes and the same cancel-in-progress concurrency test.yml had, so release branches keep their test run.
  • make test also ran manifests generate fmt vet. Generated files are still covered by Verify Generated Files, and vet and fmt by the lint workflow, so nothing that gated before stops gating.

Follow-ups:

  • ci.yaml pins envtest to Kubernetes 1.31.0 while make test uses ENVTEST_K8S_VERSION; keep the two in step, or have CI call the Makefile target.

@vvoytovych

Copy link
Copy Markdown
Contributor Author

Thanks, ecv. Agreed, and the two copies have already drifted: the Makefile installs setup-envtest from release-0.19, and ci.yaml from a release-0.25 commit. After this PR and #848 merge, I'll open a PR in which CI calls one Makefile target, so the Kubernetes and setup-envtest versions live in one place.

@vvoytovych
vvoytovych merged commit 9d39ccd into main Nov 3, 2026
10 checks passed
@vvoytovych
vvoytovych deleted the ci/run-the-suite-once branch November 3, 2026 19:32
vvoytovych added a commit to milo-os/milo that referenced this pull request Nov 3, 2026
Related to #978.

The required check, `ci / Build and test`, runs `go test ./...` with
envtest 1.37.0 on every pull request and on `main`. "IAM API testing"
ran the same `go test ./...` on every push that changed Go files. It is
not required, so it gated nothing, and each pull request ran the suite
twice. It goes, as
[datum-cloud/network-services-operator#849](datum-cloud/network-services-operator#849)
did for NSO, and CI takes the concurrency group the other workflows
share.

## Where each step of IAM API testing runs now

| step | where |
|---|---|
| `go mod download` | CI: `go test` downloads what it needs |
| envtest 1.37.0 | CI: `setup-envtest use 1.37.0`, pinned by commit |
| `go test ./...` | CI, with a 10 min timeout instead of 5 |

## Corner cases

| case | what happens |
|---|---|
| A push to a branch with no pull request | No suite run until its pull
request opens. Milo has no release branches; it releases by tag from
`main`. |
| A newer push to a pull request | The older CI run is cancelled. |
| A push to `main` | Its run is never cancelled: `cancel-in-progress` is
false on the default branch. |

## Test plan

- [x] `ci / Build and test` ran the suite on #978's pull request and on
this one: the quota controller package's envtests took [51.7
s](https://github.com/milo-os/milo/actions/runs/41592340528) and [52.3
s](https://github.com/milo-os/milo/actions/runs/41706707445/job/126163821686).
- [x] `actionlint` passes on `ci.yaml`. The concurrency group is the one
in `fix-newlines.yaml`, `test-environment-validation.yaml` and
`test-prometheus-rules.yaml`, word for word.
- [x] Nothing in the repository names `testing.yaml` or "IAM API
testing".
- [ ] After the merge: the push to `main` runs CI's suite, and "IAM API
testing" no longer runs.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants