Repository navigation
feat: make matrix tests the default (issue 74) - #4
Open
bacluc-agent wants to merge 2 commits into
Open
bacluc-agent wants to merge 2 commits into
bacluc-agent wants to merge 2 commits into
Conversation
added 2 commits
October 20, 2026 12:44
Matrix (multi-instance) tests are now generated unconditionally. bacluc-agent/agent-vshn-todo#74
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Removes the
add_matrixcookiecutter option so matrix (multi-instance) tests are always generated. Renders are byte-identical to the oldadd_matrix=youtput, so existing consumers are unaffected.Implements bacluc-agent/agent-vshn-todo#74 (closes via issue link below).
PR target
The issue text names
https://github.com/bacluc/commodore-component-template/as the PR target, but that repo is itself a fork of the canonicalbacluc-agent/commodore-component-template(verified:gh api repos/bacluc/commodore-component-template --jq '{fork,parent:.parent.full_name}'→{"fork": true, "parent": "bacluc-agent/commodore-component-template"}), and this agent has no push access there (push: false). The canonical repo isbacluc-agent/commodore-component-template, so this PR targets itsmain.Changes
cookiecutter.json— deleted the"add_matrix": "y"option.hooks/post_gen_project.py— removed theadd_matrixflag; the renovatepostUpgradeTaskscommand is now unconditionallymake gen-golden-all(renovate.json unchanged vs. oldadd_matrix=youtput).{{ cookiecutter.slug }}/Makefile— removed the inner{%- if cookiecutter.add_matrix == "y" %}conditional;golden-diff-all,gen-golden-all,lint_kubent_alland$(test_instances)now sit directly inside theadd_golden == "y"block.{{ cookiecutter.slug }}/Makefile.vars.mk—test_instancesis now guarded only byadd_golden == "y".{{ cookiecutter.slug }}/.github/workflows/test.yaml—testandgoldenjobs always emitstrategy.matrix.instanceand runmake test -e instance=${{ matrix.instance }}/make golden-diff -e instance=${{ matrix.instance }}(4 conditionals removed)..github/workflows/test.yaml— removed theadd_matrixmatrix dimension (16 combos instead of 32) and the,"add_matrix":"${{matrix.add_matrix}}"key fromcruft_extra_content.test-rendered-test-casesuntouched.Post-edit gate:
grep -rn add_matrix --exclude-dir=.git→ zero hits.Byte-stability evidence (baseline =
de8e8df)Combos:
{};{"add_golden":"n"};{"add_lib":"y","add_pp":"y","add_go_unit":"y","test_cases":"defaults foo bar"};{"add_lib":"y","add_pp":"y","add_go_unit":"y","test_cases":"defaults foo bar","add_golden":"n"}Result: 4/4 combos identical outside
.cruft.json(diff silent for every combo).Test evidence
add_golden=y,test_cases="defaults foo"):grep -A6 '^ test:' test-component/.github/workflows/test.yaml→ test job hasstrategy.matrix.instancewithdefaults,foogrep 'make test -e instance=\${{ matrix.instance }}' ...→ 1 hitgrep 'make golden-diff -e instance=\${{ matrix.instance }}' ...→ 1 hitgrep 'test_instances = ' test-component/Makefile.vars.mk→test_instances = tests/defaults.yml tests/foo.ymladd_golden=nedge case): test job still hasstrategy.matrix.instance+make test -e instance=${{ matrix.instance }}; notest_instancesinMakefile.vars.mk; nogolden-diff-alltarget; nogoldenjob.! grep -rn add_matrix cookiecutter.json hooks/ '{{ cookiecutter.slug }}' .github/→ zero hits.make -C test-component lint→ exit 0make -C test-component -j1 golden-diff-all -e git_volume=→ exit 0make -C test-component test -e instance=defaults→ fails withinvalid volume specification: '../.git:../.git:ro'— pre-existing, not caused by this PR: the identical failure occurs on thede8e8dfbaseline render (rendered output is byte-identical per B1; it's an artifact of running the compile from a directory nested inside another git repo, wheregit rev-parse --git-common-dirreturns a relative path). With the same-e git_volume=override CI uses for golden-diff, the command exits 0 (Compiled 1 targets).actionlint→ clean on the roottest.yamland on rendered component workflows (default,test_cases="defaults foo",add_golden=n).pull_request, head SHAd8320242e9ddc56fee95a5e6c94bc21b5abdc0de, run conclusion success. Job-level: 22/22 jobsconclusion: success— 16 ×test-rendered (add_lib, add_pp, add_golden, add_go_unit)+ 6 ×test-rendered-test-cases. Both job types havecontinue-on-error: true, so the job-level conclusions are the meaningful evidence.Step-8 note
projectsyn/component-icap-virusscanalready hasadd_matrix: "y"in its.cruft.jsonand already usesmatrix.instancein its workflow → unchanged by this PR; no second-repo PR needed.Issue
bacluc-agent/agent-vshn-todo#74: https://github.com/bacluc-agent/agent-vshn-todo/issues/74
Appendix — step-9 deviation, environment, release note (issue 74)
Step 9 deviation:
gh workflow run "Template tests"is impossibleTemplate tests(.github/workflows/test.yaml) has noworkflow_dispatchtrigger on this branch, onmain, or onprojectsyn/commodore-component-template@main(onlySync/sync.yamlis dispatchable). Addingworkflow_dispatchto this workflow is issue-71 scope and is out of scope here. Attempted exactly once:exit=1
gh run list -R bacluc-agent/commodore-component-template --branch issue-74-default-matrix-testsafter the attempt shows no dispatch-triggered run (onlypull_requestevents).Substitute evidence: the workflow runs on this PR (
pull_requestevent):pull_request, head SHAd8320242e9ddc56fee95a5e6c94bc21b5abdc0de, 22/22 jobs success (16test-rendered+ 6test-rendered-test-cases; both job types arecontinue-on-error: true, so job-level conclusions are the evidence).gh run list -R bacluc-agent/commodore-component-template --branch issue-74-default-matrix-testsSync (dry-run)failure is environmental and pre-existingSync (dry-run)fails on every PR in this fork: the fork has zero Actions secrets (gh api repos/bacluc-agent/commodore-component-template/actions/secrets --jq .total_count→0), sowebfactory/ssh-agentaborts at step "Setup SSH" ("The ssh-private-key argument is empty…") before any sync code runs.pr.yaml,sync.yaml,sync_base.yaml,components.yaml,.sync.ymlare all unchanged (diff = 6 files + this hook cleanup, all cookiecutter/hooks/template-workflow).Not fixable inside this PR (would require configuring repository secrets).
Follow-up cleanup commit
2d891efecb79aea061bedd54a3f15980b7aea268Removes the dead
cmdindirection and stalegen-golden(-all)comment inhooks/post_gen_project.py(cosmetic, issue step 2). Hooks are not part of cookiecutter output → rendered output and byte-stability unchanged; verified via a fresh render assertingpostUpgradeTasks.commands == ["make gen-golden-all"].Release note for consumers
Components previously generated with
add_matrix=n: aftercruft updateto this template, runmake gen-golden-allonce — golden files created under the old single-instance layout may be empty/stale for the non-default instances that the matrix now includes.Closes bacluc-agent/agent-vshn-todo#74
Tester evidence (head 2d891ef)
2d891efecb79aea061bedd54a3f15980b7aea268, conclusionsuccess, 22/22 jobs success. Sample job: https://github.com/bacluc-agent/commodore-component-template/actions/runs/39470464570/job/118792853557failureatSetup SSHstep only: "The ssh-private-key argument is empty" (https://github.com/bacluc-agent/commodore-component-template/actions/runs/39470464772/job/118792854582#step:7). Environmental, expected (fork has zero Actions secrets); identical failure on unrelated branches (see appendix above).2d891ef— behavior-neutral inline of the deadcmdvariable + comment fix inhooks/post_gen_project.py;git grep add_matrix→ zero hits; fresh cruft renders re-confirmed (renovatepostUpgradeTaskscorrect foradd_goldeny/n; matrix unconditional in rendered workflow;make test -e instance=defaultsexit 0).Follow-up run update
Run: https://github.com/bacluc-agent/agent-vshn-runner/actions/runs/39494239328 — model: opencode/big-pickle
bacluc-agentis the parent ofbaclucin the GitHub fork network (gh api repos/bacluc/commodore-component-template/compare/main...bacluc-agent:issue-74-default-matrix-tests→{status: ahead, ahead_by: 2, behind_by: 0}). The upstream PR was opened against the issue's required target: feat: make matrix tests the default (issue 74) BacLuc/commodore-component-template#1projectsyn/component-icap-virusscan, no push):cruft==2.16.0update against this branch succeeds ("Good work!"), recorded template commit2d891efecb79aea061bedd54a3f15980b7aea268,add_matrixdropped from the cookiecutter context (False). Rendered-file diff contains only the 4 known pre-existing drift areas (actions/checkout v5→v6 ×4, mikepenz/release-changelog-builder-action v5→v6, jsonnetfmt--in-placemoved fromJSONNETFMT_ARGSto the recipe) and zero issue-74-related changes.make test -e instance=defaults,make test -e instance=with_squid_proxy,make test -e instance=monitoringandmake -j1 golden-diff-allall exit 0.gh workflow run "Template tests" --ref issue-74-default-matrix-testsstill fails verbatim withcould not create workflow dispatch event: HTTP 422: Workflow does not have 'workflow_dispatch' trigger (https://api.github.com/repos/bacluc-agent/commodore-component-template/actions/workflows/385599719/dispatches)/ exit=1; no dispatch run was created (branch run list shows onlypull_request-event runs). Substitute evidence: Template tests runs https://github.com/bacluc-agent/commodore-component-template/actions/runs/39425583492 (shad832024) and https://github.com/bacluc-agent/commodore-component-template/actions/runs/39470464570 (sha2d891ef), both 22/22 jobs success (example job: https://github.com/bacluc-agent/commodore-component-template/actions/runs/39470464570/job/118792853557).Verification (re-audit run)
issue-74-default-matrix-tests@2d891efecb79aea061bedd54a3f15980b7aea268main@de8e8dfd381493f69cfbf3b773efa45e2617895d(ahead 2, behind 0 → no merge needed)git grep -rn add_matrix→ zero (verified in fresh clone/tmp/commodore-component-template-auditat HEAD2d891efecb79aea061bedd54a3f15980b7aea268, working tree clean)test_cases: "defaults extra"→ test+golden matrices includedefaults/extra,make testdefaults exit 0,make test instance=extraexit 0,make golden-diff-allexit 0test_cases: "alpha beta gamma",add_golden=n→ test matrixalpha/beta/gamma, golden job/targets absent, each instancemake testexit 0,make golden-diff-allexit 2 "No rule to make target" (EXPECTED)projectsyn/component-icap-virusscan@ 5b575977 via cruft update to branch: records2d891ef, diff only pre-existing drift (action v5→v6, jsonnetfmt flag move,.cruft.jsonmetadata/add_matrix key removal), zero issue-74 drift;make testfor defaults, with_squid_proxy, monitoring all exit 0;make golden-diff-allexit 0gh workflow run "Template tests" --ref issue-74-default-matrix-tests→ HTTP 422 "Workflow does not have 'workflow_dispatch' trigger" (documented deviation; no fresh trigger possible without a no-op commit or closing/reopening the PR, both disallowed); existing run 39470464570 for exact head, success, 24/24 jobs, sample job 118792854379#step:8:1