Skip to content

COMP: Make the 'Build and Test' analysis workflow run - #57

Closed
gdevenyi wants to merge 13 commits into
InsightSoftwareConsortium:fix/buildyml-jobsfrom
gdevenyi:pr/ci-dashboard-rewrite
Closed

gdevenyi wants to merge 13 commits into
InsightSoftwareConsortium:fix/buildyml-jobsfrom
gdevenyi:pr/ci-dashboard-rewrite

Conversation

@gdevenyi

@gdevenyi gdevenyi commented Aug 15, 2026 •

Copy link
Copy Markdown
Collaborator

The first commit fixes the workflow's trigger: it fires on branch
main, but this repository's default branch is master, so not one
of its jobs -- coverage, valgrind memcheck, scan-build, ASan+UBSan --
has ever executed. It also replaces the retired macos-11 runner and
the deprecated actions/checkout@v3.

The trigger alone is not enough to make 'Build and Test' work. Every job invokes cmake/travis_dashboard.cmake, which begins

set_from_env(CTEST_SITE "TRAVIS_APP_HOST" REQUIRED)

and aborts with a FATAL_ERROR because GitHub Actions does not set that
variable. The script is a half-migrated hybrid: it wants
TRAVIS_APP_HOST from Travis and AGENT_BUILDDIRECTORY from Azure
Pipelines, and the workflow supplies only the latter. All five jobs
fail in about twenty seconds having compiled nothing.

Each job now configures, builds and tests directly:

coverage --coverage build, tests, an lcov summary
valgrind ctest -T memcheck, then a step that fails on any non-zero
ERROR SUMMARY or any definite or indirect leak
use_prefix builds with NIFTI_PACKAGE_PREFIX set
sanitize ASan and UBSan with -fno-sanitize-recover, then
scan-build --status-bugs
macos Release on macos-14

Three details are recorded in comments so they do not have to be
rediscovered:

  • valgrind will not start without libc6-dbg -- it reports that it
    cannot redirect memcmp and exits;

  • its memcheck needs --trace-children=yes, because most of these tests
    are shell scripts that exec nifti_tool. Without it valgrind
    inspects the shell, finds nothing and reports a clean run;

  • ctest -T memcheck exits 0 even when valgrind reports defects, so the
    following step greps the logs. That grep matches the error count
    itself rather than the absence of a clean summary: with
    --trace-children the concurrent processes share one log and their
    lines interleave, and a real run produced

    ==3336== ERROR SUMMARY: ==3300==
    

    which an absence test would have accepted.

This also drops the submission to my.cdash.org. Posting results to the
project's public dashboard from every pull request, including ones from
forks, is not obviously wanted, and nothing else in the workflow needs
it. cmake/travis_dashboard.cmake is left in place; nothing else refers
to it.

Stated plainly: the workflow itself cannot be run locally. The YAML
parses, and every cmake, ctest, valgrind and scan-build invocation in it
was run by hand against this tree, but the jobs are unverified until
they run on a runner.


Interface impact: none. On the union of all these changes, configured with USE_FSL_CODE=ON and USE_CIFTI_CODE=ON: all 448 exported symbols across libniftiio, libnifti2, libznz, libfslio, libnifticdf and libcifti are identical to master under nm -D --defined-only, and all ten installed headers are identical under gcc -E -P. Under gcc -dM -E one macro definition differs, intentionally and only in text: #61 makes FSL_RADIOLOGICAL read (-1) so it is safe inside an expression. Its value is still -1, checked by compiling against each installed fslio.h and printing it.

Verification. This branch: builds with gcc 16.1.1, ctest unchanged from master (2 of 345 fail on master itself in this environment; #31 and #29 each fix one). The union of all the PRs: 0 errors under both gcc 16.1.1 and clang 22.1.8, ctest 345/345 under each, and the whole suite under valgrind memcheck with --trace-children=yes gives 484 traced processes with no invalid access, no uninitialised value and no leak in any nifti binary.

Coordination. Every line of every branch was compared, whitespace-normalised, against the diffs of the open PRs (#11, #21, #22, #23, #24). Where one of those already changes a line, the line was left alone, and the few deliberate overlaps are named in the text above. What survives is 17 compiler warnings, all of them on those lines: 9 -Wsign-conversion (5 in fslio.c for #22, 2 in nifti2_io.c and 2 in nifti_tester001.c for #24) and 8 -Wcalloc-transposed-args in nifti_findhdrname and nifti_findimgname, which #11 rewrites. No formatting changes appear anywhere, to stay clear of #10 and #12.

One of a set of independent, single-purpose PRs. Each bases on master and can be merged on its own, in any order.

The full set of PRs (35)

The union of all of them is on the fork as all-changes, if you want to build and test the lot at once.

CI and build

Configuration and documentation

Defects

Warning and check classes

This was referenced Aug 15, 2026
@hjmjohnson

Copy link
Copy Markdown
Member

Rebased onto current master (b4876bf) and pushed as c8d48e7, now a single commit. Partially superseded — this needs a maintainer decision before it is worth reviewing in detail.

What master has already fixed, and this PR therefore no longer contributes:

  • The main vs master trigger. master's build.yml already triggers on master, so the first commit of this PR (8dcb6e5, "Trigger the analysis workflow on the branch that exists") is fully redundant and was dropped by the rebase.
  • The Travis script. cmake/travis_dashboard.cmake is gone; master uses cmake/github_dashboard.cmake, which takes CTEST_SITE from RUNNER_OS with a default rather than requiring TRAVIS_APP_HOST. The original "all five jobs fail in twenty seconds having compiled nothing" premise no longer holds — the jobs run.
  • macos-11 is gone; master uses macos-latest.

What is still genuinely new here, and is the only reason to keep the PR open:

  • ctest -T memcheck exits 0 even when valgrind reports defects. master's valgrind job therefore cannot fail on a memory error. This PR adds a log-scanning step that does.
  • --trace-children=yes, without which valgrind inspects the shell wrapper and reports a clean run having never looked at the library, and libc6-dbg, without which valgrind will not start at all.
  • A use_prefix job exercising NIFTI_PACKAGE_PREFIX, and -fno-sanitize-recover=all on the sanitizer job.
  • actions/checkout@v3 -> @v4.
  • Failures land in the Actions log instead of only on CDash, and pull requests from forks stop posting to my.cdash.org.

No overlap with cmake-multi-platform.yml, which covers the plain build matrix, the minimal configuration, the oldest supported CMake, the exported-symbol baseline, and the missing-declarations check — none of coverage, valgrind, sanitizers, scan-build, or the package prefix.

The decision is whether to keep the CDash dashboard path or move these five jobs to direct cmake/ctest invocations. If you want to keep CDash, the valgrind gating and the two valgrind flags should be extracted from this PR and applied to the dashboard script instead; the rest can be dropped. I have not closed or retitled anything.

What changed in this update
  • Rebased onto master. The trigger commit dropped as redundant. The rewrite commit conflicted with master's Travis-to-GitHub migration in build.yml; resolved by taking this PR's version of the jobs: block, since the commit's whole purpose is to replace it, after confirming that master's version of that block contributes nothing the new one lacks (the trigger and the dashboard-script name are the only differences, and the new jobs use neither).
  • Commit message: dropped the Co-Authored-By: naming an AI tool and the Claude-Session: URL; removed the reference to "the previous commit", which no longer exists; and rewrote the rationale, which described travis_dashboard.cmake and a FATAL_ERROR that master has since removed. The message now describes the situation as it is.
  • No change to the workflow content itself beyond the rebase.
Why no test

BUILD class. A GitHub Actions workflow has no runtime seam a ctest can observe; it is only exercised by a runner. Per the commit message, the individual cmake, ctest, valgrind and scan-build invocations were run by hand, but the jobs themselves remain unverified until CI runs them.

@hjmjohnson
hjmjohnson force-pushed the pr/ci-dashboard-rewrite branch 2 times, most recently from 03b4b67 to b3c827a Compare September 23, 2026 00:41
gdevenyi and others added 13 commits September 24, 2026 06:54
The test hard-codes the path to the downstream example project:

    cmake ... ../../nifti_clib/real_easy/minimal_example_of_downstream_usage

That only resolves when the build directory happens to be a sibling of a
source tree named exactly "nifti_clib".  It fails for an in-tree build,
for a build directory named anything else, and on CI, where the checkout
lives at <workspace>/nifti_clib and the path resolves to
<workspace>/nifti_clib/nifti_clib/real_easy/...

CMake now passes CMAKE_SOURCE_DIR to the script, the way the other test
scripts in that directory already receive their arguments.

(cherry picked from commit 6458714)
cmake-multi-platform.yml ran ctest without --output-on-failure, so a red
build reported which test failed and nothing about why.

(cherry picked from commit 919dcea)
CMakeLists.txt sets policy CMP0169 unconditionally, to keep using the
deprecated FetchContent_Populate.  CMP0169 was introduced in CMake 3.30,
and cmake_policy(SET) on an unknown policy is a hard error, so any CMake
older than that fails to configure at all:

    CMake Error at CMakeLists.txt:175 (cmake_policy):
      Policy "CMP0169" is not known to this version of CMake.

That includes the CMake 3.28 shipped by Ubuntu 24.04, which is what the
CI runners and the project's own Dockerfile use, and it sits well above
the cmake_minimum_required(VERSION 3.10.2) the project advertises.

Wrapping it in if(POLICY CMP0169) is the standard idiom.  Verified
configuring with both CMake 3.28.3 and 4.4.2.

(cherry picked from commit 3aefe1b)
cmake-multi-platform.yml is the workflow that runs on every pull
request, and it passes no options at all.  With the defaults that means
cifti/ and fsliolib/ are never compiled, and neither are the blocks
behind FSLSTYLE, PIGZ and REJECT_COMPLEX.

  USE_CIFTI_CODE          OFF
  USE_FSL_CODE            OFF
  FSLSTYLE_NAME_CONFLICTS OFF   -> -DFSLSTYLE
  FSLSTYLE_PIGZ_SUPPORT   OFF   -> -DPIGZ
  FSLSTYLE_REJECT_COMPLEX OFF   -> -DREJECT_COMPLEX

So a PR that changes any of those files collects four green checks that
never built it.  Several open PRs are in exactly that position.

The matrix gains an 'options' axis: 'default' is what a consumer gets
with no arguments, 'all' turns the optional libraries and the FSL parity
defines on.  Both are kept, because the default build is what ships and
a change can break it while the fuller one still compiles.  Four jobs
become eight, each about half a minute.

expat is installed for the Linux 'all' jobs; cifti needs it and the
default jobs do not.

Verified on master before writing this, with the test suite:

  default                          build OK, tests pass
  +cifti +fsl                      build OK, tests pass
  +cifti +fsl +FSLSTYLE            build OK, tests pass, doPigz/doPigz2
                                   present in nifti2_io.c.o

COMPILE_NIFTIUNUSED_CODE is deliberately left out.  It guards code the
project itself labels unused, so compiling it in CI would commit to
keeping it working; whether that code should exist at all is a separate
question.

(cherry picked from commit b6531f0)
Run the Build and Test workflow on this repository's branch. Its
trigger named "main" while the branch is "master", so it had never
executed on master and did not execute on pull requests; valgrind,
AddressSanitizer, UndefinedBehaviorSanitizer, scan-build, gcov
coverage, and the only shared-library configuration were all
unexercised. Also move its macOS entry off the retired macos-11 image.

Restructure the per-PR matrix around the axes that change what is
compiled:

  shared     BUILD_SHARED_LIBS gates TEST_INSTALL, so the ON legs are
             the first to run install_linking and cover the install
             and export path.
  fslstyle   -DFSLSTYLE is not additive.  It rewrites behaviour in
             niftilib and nifti2 through global -DFSLSTYLE, -DPIGZ and
             -DREJECT_COMPLEX, and one site has an #else that changes
             the pixdim[0] value written to disk.  Neither value
             subsumes the other.
  build_type One static Debug leg, which is what makes assert() live.

Drop both macos gcc legs: /usr/bin/gcc there is a clang shim reporting
AppleClang, so they duplicated the macos clang legs exactly. The
optional libraries are built everywhere rather than forming an axis,
since they are additive.

Add three jobs. The minimal configuration builds znzlib and niftilib
alone, which is what a downstream project vendoring the core reader
selects and which nothing else configures. The oldest supported CMake
job guards cmake_minimum_required, invisible to runners that all carry
a recent CMake. The exported symbol baseline diffs the dynamic symbol
set against a committed file, so a change to the ABI has to be updated
in the same commit rather than landing unnoticed.

Verified on Ubuntu 24.04 with GCC 13.3.0, the compiler the Linux
runners use:

  shared, Release, FSLSTYLE=OFF   345/345, install_linking passed
  static, Debug,   FSLSTYLE=ON    344/344, no assertion aborts
  minimal                          configures and builds, no tests
  CMake 3.28.3                     344/344

The baseline holds 448 symbols across six libraries, which is the
figure the linkage changes have been asserting without a way to check
it.

The workflow steps were exercised by running them directly in the
runner container image rather than through act, whose image omits
cmake.

(cherry picked from commit 77b7418)
The dashboard script restates the CTestConfig.cmake settings because
CTest's delayed initialization does not pick them up, and the restated
copy had drifted from the original.

Submit over https rather than http. my.cdash.org serves plain http
without redirecting, so submissions were going unencrypted.

Use the 00:00:00 EST nightly start time that CTestConfig.cmake
declares, rather than 01:00:00 UTC. The two differ by four hours, so
nightly builds were filed under the wrong day on the dashboard.

(cherry picked from commit 063e838)
The 2025 conversion to GitHub Actions replaced .travis.yml with two
workflows but left the dashboard script untouched, so it still
required a Travis environment. The workflow that drives it named the
wrong branch and never ran, which is why nothing surfaced the
mismatch.

Rename travis_dashboard.cmake to github_dashboard.cmake and take its
inputs from the runner that actually exists. CTEST_SITE now comes from
RUNNER_OS rather than the required TRAVIS_APP_HOST, whose absence
aborted the script before it configured anything, and the build name
carries RUNNER_OS in place of TRAVIS_OS_NAME.

Compare the branch name against its value rather than against the
literal string "ENV{BUILD_SOURCEBRANCHNAME}", so the Continuous and
Nightly models can be selected at all, and give the workflow a branch
name on push events as well as on pull requests.

Apply the same corrections to local_dashboard.cmake.

(cherry picked from commit f578e03)
FslGetHdrImgNames and FslSetIntensityScaling are defined here but
declared nowhere, while FslGetIntensityScaling and FslInit are already
published.  axml_recur_find_xml sits beside axml_recur in afni_xml.h
the same way.

The upstream fslio was deleted in 2015 in favour of a C++ replacement,
so this copy is the surviving one and its header is ours to correct.

Additive; no symbol changes.

(cherry picked from commit 4673c40)
Nine functions in fslio.c have external linkage and no declaration in
any header.  Nothing in the tree calls them across a translation unit,
AFNI's vendored copy never calls them, and no public source outside a
vendored copy of this file references them, so a caller would have had
to declare them itself.

Collect their prototypes in one block so the boundary between internal
and published is visible in one place.

FslSetVoxUnits and FslGetVoxUnits have no caller at all and are left
under #if 0 rather than deleted; FslSetTimeUnits and FslGetTimeUnits
are published, so the asymmetry is worth keeping visible.

Removes nine symbols from libfslio, which the baseline records.

(cherry picked from commit 8cdcf1a)
nifti1_io.c and nifti2_io.c each define nifti_fileexists with external
linkage, so libniftiio and libnifti2 export the same name and ELF link
order decides which one a caller linking both resolves to.

Declaring it locally silences the warning without choosing between
publishing it and making it static; that choice needs the duplication
settled first, and it covers a hundred names, not this one.

(cherry picked from commit 54e74e1)
Keeps FslGetAuxFile and FslSetAuxFile deriving the same length so the
pair cannot drift if the field width changes.  sizeof is 24, so the
copy still writes at most 23 bytes.

(cherry picked from commit 0c0e66c)
Two of the five Build and Test jobs have never run to completion.

sanitize-clang-linux invokes scan-build, which lives in clang-tools and
was not installed, so the job exits 127 before configuring.

rel-clang-macos asks brew for "sed", which is not a formula; brew fails
the step and the job exits 1 before configuring. The GNU sed the
dashboard scripts expect is gnu-sed.

Both predate the workflow's first successful run, so neither has
regressed; the trigger named the wrong branch until recently and the
jobs never executed.

(cherry picked from commit 3147438)
The 'Build and Test' jobs drive everything through
cmake/github_dashboard.cmake, so a failure is reported on CDash rather
than in the job log, and the memcheck job does not fail at all: ctest
-T memcheck exits 0 even when valgrind reports defects.

Each job now configures, builds and tests directly: coverage with an
lcov summary, valgrind memcheck followed by a step that fails on any
non-zero ERROR SUMMARY or definite or indirect leak, a build with
NIFTI_PACKAGE_PREFIX set, ASan and UBSan plus scan-build, and a macOS
Release build.  The submission to my.cdash.org is dropped;
cmake/github_dashboard.cmake is left in place.

The workflow cannot be run locally.  The YAML parses and every cmake,
ctest, valgrind and scan-build invocation in it was run by hand against
this tree, but the jobs are unverified until a runner runs them.

(cherry picked from commit b3c827a)
@hjmjohnson
hjmjohnson force-pushed the pr/ci-dashboard-rewrite branch from b3c827a to 075715d Compare September 24, 2026 12:01
@hjmjohnson
hjmjohnson changed the base branch from master to fix/buildyml-jobs September 24, 2026 12:01
@hjmjohnson
hjmjohnson marked this pull request as draft October 6, 2026 15:02
@hjmjohnson
hjmjohnson deleted the branch InsightSoftwareConsortium:fix/buildyml-jobs October 6, 2026 19:08
@hjmjohnson hjmjohnson closed this Oct 6, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants