Skip to content

codecov: grade the unit and symmetry test suites together - #147

Merged
stevevanhooser merged 1 commit into
mainfrom
claude/clever-bardeen-hf5jgx
Oct 21, 2026
Merged

stevevanhooser merged 1 commit into
mainfrom
claude/clever-bardeen-hf5jgx

Conversation

@stevevanhooser

Copy link
Copy Markdown
Contributor

Since #146 took the symmetry tests out of the general test job, Codecov only receives the general job's coverage. So 262 lines of +ndr reader/format code that only the symmetry tests reach count as uncovered: 54.94% → 52.21%. This PR uploads the symmetry workflow's coverage too. Codecov merges every upload for a commit, so both suites count in one grade.

Changes

  • .github/workflows/test-symmetry.yml:
    • Runs the make and read suites with MATLAB's CodeCoveragePlugin and writes a Cobertura report. It measures the same +ndr file list matbox.tasks.testToolbox uses, so the two reports line up.
    • Uploads the report with codecov/codecov-action@v4, flags: symmetry, using the same CODECOV_TOKEN secret the general job uses.
    • Make and read now run in one runner.run call: make cases first, then read, since a suite runs in order. A coverage plugin writes its report at the end of each run, so the old two-call version would have kept only the read stage's coverage. Results are split by index, so the per-stage summaries and the Incomplete check are unchanged.
  • .github/workflows/run_tests.yml: the general job's upload is now tagged flags: unittests, so Codecov shows the two sources separately.
  • codecov.yml: codecov.notify.after_n_builds: 2 and comment.after_n_builds: 2. Codecov waits for both uploads before posting its checks and comment, rather than grading whichever lands first, which could flicker red and green. The 3% project threshold is unchanged.

Expected effect

  • Project coverage on this PR should come back to about the pre-tests: exclude the symmetry tests from the general test job #146 figure (~54.9%). Unit tests and symmetry tests are counted together, and Codecov shows them separately under the unittests and symmetry flags.
  • Caveat: a commit where only one of the two workflows runs gets no Codecov status. run_tests.yml skips changes that touch only *.md or only .github/workflows/**, so those commits would have one upload instead of two. That's no worse than today: those commits got no coverage upload at all, because run_tests.yml was the only uploader.

Verification

I wrote this without access to MATLAB, so CI is the first run. The coverage setup copies MatBox's own testToolbox.m (same plugin, same file list). After CI, I'll check that the symmetry job wrote and uploaded symmetry-coverage.xml, and what merged figure Codecov reports.


Generated by Claude Code

Since the symmetry tests left the general test job (#146), Codecov only
saw the general job's coverage, so 262 lines of +ndr reader/format code
that only the symmetry tests reach counted as uncovered.

- test-symmetry.yml: run the make and read suites with CodeCoveragePlugin
  over the same +ndr files matbox.tasks.testToolbox measures, producing a
  Cobertura report, and upload it to Codecov with flag "symmetry". The
  two suites now run in ONE runner.run call (make cases first, then read;
  a suite runs in order), because a coverage plugin writes its report at
  the end of each run and two runs would keep only the second's. Results
  are split by index for the per-stage summaries.
- run_tests.yml: tag the general job's upload with flag "unittests".
- codecov.yml: codecov.notify.after_n_builds and comment.after_n_builds
  = 2, so the status checks and PR comment wait for both uploads instead
  of grading whichever lands first. The 3% project threshold is unchanged.

Written without access to MATLAB; CI runs it.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01QjkvVXsdCVw6NNe5SnGNyX
@github-actions

Copy link
Copy Markdown
Contributor

Test Results

395 tests  ±0   392 ✅ ±0   38s ⏱️ -1s
 41 suites ±0     3 💤 ±0 
  1 files   ±0     0 ❌ ±0 

Results for commit 8d824c4. ± Comparison against base commit d8f86e3.

@codecov

codecov Bot commented Oct 21, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 54.94%. Comparing base (d8f86e3) to head (8d824c4).

Additional details and impacted files
@@            Coverage Diff             @@
##             main     #147      +/-   ##
==========================================
+ Coverage   52.21%   54.94%   +2.72%     
==========================================
  Files         164      164              
  Lines        9598     9598              
==========================================
+ Hits         5012     5274     +262     
+ Misses       4586     4324     -262     
Flag Coverage Δ
symmetry 19.00% <ø> (?)
unittests 52.21% <ø> (?)

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@stevevanhooser
stevevanhooser merged commit b1e8648 into main Oct 21, 2026
8 checks passed
@stevevanhooser
stevevanhooser deleted the claude/clever-bardeen-hf5jgx branch October 21, 2026 15:59
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants