Repository navigation
symmetry: a comparison that could not run fails CI instead of passing - #35
Merged
Merged
Conversation
…#34) test-symmetry.yml reported green on #33 with every video comparison unrun: its NDR-matlab branch was pushed after the run had fallen back to main, Python's read tests skipped (hidden among the 16 skips every run carried by design), and MATLAB's read tests disp+returned, which counts as passed. Fix all three paths, following the bridge tests' rule: - tests/symmetry/conftest.py: under NDR_SYMMETRY_CHECK_STRICT, any skip in a symmetry test fails, with the reason and what to check. A backstop, not a list, like tests/conftest.py's for the bridge. - tests/test_symmetry_strict.py: positive controls in the normal matrix. A real read test against an empty artifact base skips without the flag and fails with it (and fails the control when the backstop is removed). - test-symmetry.yml: - sets NDR_SYMMETRY_CHECK_STRICT for the job; - writes the resolved NDR-matlab ref and why to the job summary; - Stage 2 deselects the pythonArtifacts cases (made in Stage 3) rather than skipping them, so it now reports 0 skipped; - both MATLAB stages fail on any Incomplete result; - Stage 4 deselects by name the one known gap, readOmeZarr's pythonArtifacts case (NDR-python has no OME-Zarr symmetry test yet), and logs it. - AGENTS.md: the fourth entry in "A skip is a silent pass". - read_artifacts/INSTRUCTIONS.md: skips fail in CI; how to run as CI does. Paired with NDR-matlab on the same branch name, where the read tests now end in assumptions (Incomplete) instead of disp+return (Passed). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01QjkvVXsdCVw6NNe5SnGNyX
Stage 1 of the symmetry job died before any test ran: matbox's
installRequirements resolves NDR-matlab's MATLAB dependencies through the
GitHub API unauthenticated, and hit the shared 60 req/hr limit ("GitHub
API rate limit exceeded. Consider using a GITHUB_TOKEN environment
variable."). NDR-matlab's own test-symmetry.yml already passes
GITHUB_TOKEN: ${{ github.token }} in the job env for exactly this; port
the same line here (built-in token, no secret to configure; 5000 req/hr).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01QjkvVXsdCVw6NNe5SnGNyX
This was referenced Oct 21, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes #34. Paired with VH-Lab/NDR-matlab#145 on the same branch name,
claude/clever-bardeen-hf5jgx. That branch was pushed first this time, so this PR's symmetry run pairs with it. Merge VH-Lab/NDR-matlab#145 first.The problem
test-symmetry.ymlreported green on #33 with every video comparison unrun, in three separate ways:main, and nothing said so.disp+return, whichmatlab.unittestreports as Passed.The fix
The bridge tests' rule now applies to symmetry: in CI, a comparison that could not run fails, and anything that can't run by design is deselected openly.
tests/symmetry/conftest.py: a strict-mode backstop. UnderNDR_SYMMETRY_CHECK_STRICT, any skip in a test undertests/symmetry/fails, and the message says what to check. Liketests/conftest.py's bridge backstop, it is derived rather than a list.tests/test_symmetry_strict.py: positive controls that run in the normal test matrix. A real read test against an empty artifact base must skip without the flag and fail with it. Removing the backstop makes the second control fail; I checked that..github/workflows/test-symmetry.yml:NDR_SYMMETRY_CHECK_STRICT=1for the job;main… push the NDR-matlab branch and re-run");pythonArtifactscases (-k "not pythonArtifacts"; Python makes those in Stage 3, and MATLAB reads them in Stage 4) instead of skipping them, so it now reports 0 skipped;INCOMPLETE: <name>;readOmeZarr'spythonArtifactscase, since NDR-python has no OME-Zarr symmetry test yet.AGENTS.md: the fourth entry in "A skip is a silent pass", plus a line saying the rule now covers symmetry.tests/symmetry/read_artifacts/INSTRUCTIONS.md: skips fail in CI, how to run the tests as CI does, and why a new read test needs its MATLAB half to exist first.Verification
pytest: 725 passed (+2 new controls), 15 skipped (same as before), 24 xfailed.blackandruff checkare clean.test_read_video.pywith no artifacts present:The MATLAB stage scripts can only run in CI. This PR's symmetry run is their test, against the paired NDR-matlab branch.
Not in this PR
readOmeZarris MATLAB-only. Python neither reads MATLAB's OME-Zarr artifacts nor makes its own, so the Stage 4 deselection is a declared known gap, not a fix. Porting that symmetry test is a separate piece of work.🤖 Generated with Claude Code
https://claude.ai/code/session_01QjkvVXsdCVw6NNe5SnGNyX