Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
59 changes: 41 additions & 18 deletions .github/workflows/devel-vctrs.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -17,11 +17,11 @@
# still needs the dispatch.
#
# What a run tests depends on its trigger. A push or pull request runs only
# the three vctrs compatibility test files (`test-vctrs-compat.R`,
# `test-dplyr-compat.R`, `test-nested-results-set-compat.R`; the `filter` in
# the last step). The weekly schedule runs the whole suite, because a
# development-vctrs break can reach files outside the three: other test files
# call vctrs functions too (`test-nested-tune-bayes-oracles.R` calls
# the vctrs compatibility test files that carry the marker line the last step
# names; a file joins or leaves the watch by that line. The weekly schedule
# runs the whole suite, because a development-vctrs break can reach files
# outside the watched ones: other test files call vctrs functions too
# (`test-nested-tune-bayes-oracles.R` calls
# `vec_cbind_frame_ptype()`), and dplyr and tibble build on vctrs. A dispatch
# runs the whole suite unless its `full` input is set to false.
#
Expand Down Expand Up @@ -63,7 +63,7 @@ on:
workflow_dispatch:
inputs:
full:
description: 'Run the whole suite (false runs the three vctrs test files)'
description: 'Run the whole suite (false runs the watched vctrs test files)'
type: boolean
default: true

Expand All @@ -81,11 +81,16 @@ permissions:

# Extends the block in R-CMD-check.yaml. Keyed on the workflow as well as
# the ref, so cancelling a superseded run of this leg never touches a check run.
# Also keyed on the test step's `FULL_SUITE` switch (the same expression), so a
# three-file run never cancels or replaces a pending or running whole-suite
# run: GitHub keeps one pending run per group and cancels the older one.
# Also keyed on the event name and the `full` input, the two values the test
# step decides its whole-suite switch from, so two runs in one group always make
# the same choice between the whole suite and the watched files (each commit's
# markers decide which files are watched), and a subset run never cancels or
# replaces a pending or
# running whole-suite run: GitHub keeps one pending run per group and cancels
# the older one. A scheduled run and a full dispatch therefore sit in
# different groups and can both run.
concurrency:
group: ${{ github.workflow }}-${{ github.ref }}-${{ github.event_name == 'schedule' || (github.event_name == 'workflow_dispatch' && inputs.full) }}
group: ${{ github.workflow }}-${{ github.ref }}-${{ github.event_name }}-${{ inputs.full }}
cancel-in-progress: ${{ github.ref_name != github.event.repository.default_branch }}

jobs:
Expand Down Expand Up @@ -147,18 +152,36 @@ jobs:
}
shell: Rscript {0}

# testthat matches `filter` against each file name with `test-` and `.R`
# removed. A NULL filter runs every file.
# The whole-suite switch is decided here and nowhere else, from the two
# values the concurrency group above is keyed on. `inputs.full` renders
# as "" outside a dispatch. A subset run takes the test files that carry
# the marker line below as a whole line, so a file renamed within the
# `test-*.R` pattern stays in and a run that finds none stops red. A file
# whose marker line is removed or reworded leaves the watch with no
# signal. testthat matches `filter` against
# each file name with `test-` and `.R` removed. A NULL filter runs every
# file.
- name: Run the tests against it
env:
FULL_SUITE: ${{ github.event_name == 'schedule' || (github.event_name == 'workflow_dispatch' && inputs.full) }}
FULL_INPUT: ${{ inputs.full }}
run: |
full <- identical(Sys.getenv("FULL_SUITE"), "true")
event <- Sys.getenv("GITHUB_EVENT_NAME")
full <- event == "schedule" ||
(event == "workflow_dispatch" && Sys.getenv("FULL_INPUT") == "true")
cat("Whole suite: ", full, "\n", sep = "")
filter <- if (full) {
NULL
} else {
"^(vctrs-compat|dplyr-compat|nested-results-set-compat)$"
filter <- NULL
if (!full) {
marker <- "# devel-vctrs.yaml runs this file on every push and pull request."
files <- list.files("tests/testthat", "^test-.*\\.R$", full.names = TRUE)
watched <- files[vapply(files, function(f) marker %in% readLines(f, warn = FALSE), logical(1))]
if (length(watched) == 0) {
stop("No test file in tests/testthat carries the line '", marker, "'.")
}
cat("Watched files:", basename(watched), sep = "\n ")
cat("\n")
stems <- sub("^test-(.*)\\.R$", "\\1", basename(watched))
stems <- gsub("([][{}()+*^$|\\\\?.])", "\\\\\\1", stems)
filter <- paste0("^(", paste(stems, collapse = "|"), ")$")
}
pkgload::load_all(".", quiet = TRUE)
testthat::test_local(filter = filter, stop_on_failure = TRUE)
Expand Down
6 changes: 6 additions & 0 deletions cairn/DECISIONS.md
Original file line number Diff line number Diff line change
Expand Up @@ -2135,6 +2135,12 @@ upstream, either of which would let the file return to a shared blob.
**Decision:** they do not. A table that only reshapes, labels or joins numbers that another group's oracles check is not a numeric result of its own, and a table that holds no number is not one either. Each such group in `cairn/ORACLES.md` carries one line naming the group it rides on, or saying it holds no number. The user chose this at M164's plan gate over adding a second type to all nine groups.
**Consequences:** a table that computes a new number, such as an average or a rescoring, is a result of its own and owes two types. The three groups keep their one oracle, which checks the reshaping. Falsified by a reshaping table that changes a number (drops, duplicates or misplaces a value) with no test failing.

### D-122 (2026-10-31): `devel-vctrs.yaml` picks its push-time test files by a marker line and keys its concurrency group on the event name and the `full` input. Narrows D-119's filter and concurrency clauses

**Context:** D-119's filter named three test files by hand, so a renamed file dropped out with no signal, and its concurrency group repeated the test step's switch expression with nothing keeping the two copies equal (M153 review). M165's plan gate chose to simplify the workflow over adding a drift test, and kept push and pull-request runs off the whole suite.
**Decision:** on a push or pull request the test step runs the `tests/testthat/test-*.R` files that carry the line `# devel-vctrs.yaml runs this file on every push and pull request.` as a whole line, and stops with an error when none does. The step decides the whole-suite switch from `GITHUB_EVENT_NAME` and `inputs.full` alone, and the concurrency group is keyed on those two raw values with the workflow and ref, so two runs in one group make the same whole-suite choice. On a subset run each commit's markers decide the files. Rejected at the plan gate: a `-compat` file-name convention, which still drops a file renamed out of it.
**Consequences:** a test file joins the push-time watch by taking the marker line, replacing D-119's "adds itself to the filter". A scheduled run and a full dispatch no longer share a group, nor do a `full` false dispatch on the default branch and a push to it, so each pair can run side by side. Falsified by a watched file losing its marker in an edit that no run flags, or by a subset run cancelling or replacing a whole-suite run. (Narrows D-119.)

<!-- Template:

### D-00N (YYYY-MM-DD): Title
Expand Down
4 changes: 2 additions & 2 deletions cairn/PROFILE.md
Original file line number Diff line number Diff line change
Expand Up @@ -41,7 +41,7 @@ rules in tracking-rules:
merge clause below), `test-coverage` runs `covr` to Codecov, annotating a PR but never gating it; `.github/` is
`.Rbuildignore`d.
- Divergences from that stock shape (M11 ×2, M12 rev. M31, M14, M33, M52, M78, M80, M88). **A `concurrency` block** cancels a
superseded run on every ref but the default branch, a distribution channel that keeps a completed check instead. `devel-vctrs.yaml` also keys its group on its full/subset switch, so a three-file run never cancels a whole-suite run (M153). **A
superseded run on every ref but the default branch, a distribution channel that keeps a completed check instead. `devel-vctrs.yaml` also keys its group on the event name and its `full` input, so a subset run never cancels a whole-suite run (M153, M165). **A
`paths-ignore` filter** on both triggers of `R-CMD-check-cran.yaml`, `R-CMD-check-hard.yaml`, `R-CMD-check.yaml`, `devel-vctrs.yaml`,
`pkgdown.yaml`, `prose-sweep.yaml` and `test-coverage.yaml` skips `cairn/**`, `CLAUDE.md`, `.claude/**`, which cannot change what `R CMD
check` sees — that is the test a fourth path must meet; it bites on `push` only, GitHub evaluating it on a
Expand All @@ -62,7 +62,7 @@ rules in tracking-rules:
hunts the hang on demand under a job cap far above the rest, invisible to `ci-usage.py` for carrying neither trigger.
**A devel-vctrs leg** (`devel-vctrs.yaml`, M80) tests against `r-lib/vctrs@main` for the experimental `vec_cbind_frame_ptype()`
method in `R/nested-results.R`, non-gating and not a required check, so upstream cannot freeze a merge. Triggers: `push` and
`pull_request` (the three vctrs compatibility test files), `schedule` (Mondays 06:17 UTC, the whole suite) and `workflow_dispatch`
`pull_request` (the test files carrying its marker line, M165), `schedule` (Mondays 06:17 UTC, the whole suite) and `workflow_dispatch`
(M102, the whole suite unless `full` is false, M153). `cache-version: devel-vctrs` keys its post-devel-install cache apart from the other workflows'. **Three organization
workflows** ride unedited at tidymodels' shared blobs (`lock.yaml`, `pr-commands.yaml`, `format-suggest.yaml`, M33):
no `push`/`pull_request` trigger, so neither the filter nor `ci-usage.py` sees them; `format-suggest.yaml` runs `air format .`.
Expand Down
2 changes: 1 addition & 1 deletion cairn/ROADMAP.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@ _Last hygiene check: 2026-10-31 M164 review. Graduated Settle oracle types for n
## Milestones
| ID | Title | Status | Depends on | Priority | File/Archive |
|---|---|---|---|---|---|
| M165 | Keep the devel-vctrs test choice from drifting | planned | — | normal | milestones/M165-devel-vctrs-drift.md |
| M165 | Keep the devel-vctrs test choice from drifting | review | — | normal | milestones/M165-devel-vctrs-drift.md |
| M164 | Give six one-type results a second oracle | done | — | normal | milestones/archive/M164-second-oracle-types.md |
| M163 | Keep an oracle registry outside tests | done | — | normal | milestones/archive/M163-oracle-registry.md |
| M162 | The shared test pool keeps both daemons | dropped | — | normal | milestones/archive/M162-shared-pool-daemon-loss.md |
Expand Down
Loading
Loading