Repository navigation
Conversation
Closed PR commentHeads up that #26 will conflict with this one, and on a couple of details that may be worth folding in whenever this comes off WIP. The conflicts: #26 rewrites the Two things #26 found in code this PR touches:
On Not asking you to rebase across #26 — if it lands first I am happy to do that rebase myself. |
Closed PR CommentFollowed up on this properly rather than leaving it at "conflicts with #26". I audited every Two are genuine defects, and #26 now fixes both.
strcpy(command, "pigz");
strcat(command, " -n -f > \"");
strcat(command, nim->fname); /* unbounded */
strcat(command, "\"");
The other 145 calls are correct, which is why #26 does not rewrite them:
Rewriting 145 correct calls is 145 chances to get a length expression wrong, in code whose tests compare output byte for byte. I hit exactly that failure mode earlier in this work: a regex that looked right turned So #26 takes a different route to the same goal: This complements what you are doing rather than replacing it. Fortification catches a bad size at run time; Two things about the current implementation that would need sorting either way, repeating them here so they are not lost:
Happy to rebase #26 under this if you would rather land yours first — the two touch different lines apart from the |
|
@gdevenyi your LLM seems to be unaware that with Instead of making it redo everything I worked hard to do, could you instead use it to review the correctness of this PR? It seems to think 145 calls were correct, does it think they are still correct after the changes here? |
There was a problem hiding this comment.
Review by Claude Fable with effort high
Thanks for tackling this — the call-site conversions all look correct. I checked each fixed buffer against its declaration: extcopy[8] holds the 7-char extensions, extnii[8] + ".gz" fills exactly 8 with the NUL, magic[4] holds "n+1\0", and the strlen(...)+8 allocations cover extension + ".gz" + NUL. The QSTR macro change correctly adds memset to preserve the zero-padding strncpy used to provide.
The blockers are all in the new string_helper fallback (see line comments):
- The platform guard only compiles the fallback on glibc < 2.38, so MSVC/MinGW (no
__APPLE__, no__GLIBC__, nostrlcpy) get no definition at all and every call fails to link. Suggestcheck_symbol_exists(strlcpy string.h HAVE_STRLCPY)in CMake instead of enumerating platforms. string_helper.htests__GLIBC__before including any libc header, so the prototypes vanish if the header is included first.- The
strlcatfallback is unsafe underNDEBUGin a case where BSD's realstrlcatis safe.
Also noting (you already flagged the CMake wiring as unfinished): none of the modified .c files #include "string_helper.h" yet, so on pre-2.38 glibc the calls will be implicit declarations even once the file is compiled. Minor: the new files use tabs, // comments, and camelCase parameter names, which don't match the surrounding codebase.
| #ifndef __APPLE__ | ||
|
|
||
| // libc 2.38 and above provides strlcpy and strlcat, so don't redefine them. | ||
| #if defined (__GLIBC__) && ((__GLIBC__ < 2) || ((__GLIBC__ == 2) && (__GLIBC_MINOR__ < 38))) |
There was a problem hiding this comment.
This guard only provides the fallback when __GLIBC__ is defined and < 2.38. On MSVC/MinGW neither __APPLE__ nor __GLIBC__ is defined, so the whole block preprocesses away — and MSVC has no strlcpy, so every call site fails to link. This repo supports MSVC (there are _MSC_VER branches in doPigz).
Rather than enumerating platforms, detect the symbol in CMake:
include(CheckSymbolExists)
check_symbol_exists(strlcpy string.h HAVE_STRLCPY)and guard on #ifndef HAVE_STRLCPY. That also handles musl, the BSDs, and future libcs without further edits.
| #ifndef __APPLE__ | ||
|
|
||
| // libc 2.38 and above provides strlcpy and strlcat, so don't redefine them. | ||
| #if defined (__GLIBC__) && ((__GLIBC__ < 2) || ((__GLIBC__ == 2) && (__GLIBC_MINOR__ < 38))) |
There was a problem hiding this comment.
__GLIBC__ is only defined after a libc header has been included. If a .c file includes string_helper.h before <string.h>, this check sees __GLIBC__ undefined and the prototypes vanish even on old glibc. Add #include <string.h> at the top of this header, before the version check (or better, switch to a CMake HAVE_STRLCPY guard as suggested in the .c file).
|
|
||
| #include <stddef.h> | ||
|
|
||
| size_t strlcpy(char* restrict ioDestination, const char* restrict inSource, size_t inDestinationSize); |
There was a problem hiding this comment.
restrict is not a keyword in C++, and there's no extern "C" guard, so any C++ consumer (e.g. ITK) that includes this header will fail to compile. Wrap the declarations in extern "C" and hide restrict behind #ifdef __cplusplus.
There was a problem hiding this comment.
I just removed restrict
| assert(destinationLength < inDestinationSize); | ||
|
|
||
| ioDestination += destinationLength; | ||
| inDestinationSize -= destinationLength; |
There was a problem hiding this comment.
If the destination is not NUL-terminated within inDestinationSize, the assert on line 41 is compiled out under NDEBUG, and this subtraction underflows to a huge value. The truncation branch below is then skipped and the memcpy writes past the buffer. OpenBSD's strlcat handles this case safely by returning size + strlen(src) without writing:
if (destinationLength >= inDestinationSize)
return inDestinationSize + sourceLength;All call sites in this PR are fine today, but a safety shim that can overflow where the real strlcat cannot defeats the purpose.
| strcat(command, "\""); | ||
| strlcpy(command, "pigz", sizeof(command)); | ||
| strlcat(command, " -n -f > \"", sizeof(command)); | ||
| strlcat(command, nim->fname, sizeof(command)); |
There was a problem hiding this comment.
Behavior change worth handling: the old strcat overflowed on a very long fname (~750+ chars); strlcat now silently truncates instead, which drops the closing quote and hands pigz a mangled command. Since strlcat returns the total length it tried to create, check it against sizeof(command) and fail (return -1) on truncation. Applies to all four copies (doPigz/doPigz2 in both nifti1_io.c and nifti2_io.c).
| strcat(elist[0], extgz); strcat(elist[1], extgz); strcat(elist[2], extgz); | ||
| strlcat(elist[0], extgz, 8); | ||
| strlcat(elist[1], extgz, 8); | ||
| strlcat(elist[2], extgz, 8); |
There was a problem hiding this comment.
The hardcoded 8 is correct today (extnii[8] + ".gz" fills it exactly), but sizeof(extnii) / sizeof(exthdr) / sizeof(extimg) would stay correct if the buffers ever change. Same in the nifti2_io.c copy.
8772037 to
b5ed6d4
Compare
b5ed6d4 to
460ba38
Compare
|
Rebased onto current Blocking: 1. The conflict: this branch reintroduced a use-after-free that #54 fixed
Resolved by combining both sides — this branch's 2. string_helper is dead code, and its guards exclude the platforms that need itThe build confirms it: a full It currently builds green everywhere CI runs only because every one of those platforms already has The guards are also inverted relative to the commit message. Both files wrap their contents in: #ifndef __APPLE__
#if defined (__GLIBC__) && ((__GLIBC__ < 2) || ((__GLIBC__ == 2) && (__GLIBC_MINOR__ < 38)))The inner test requires Result on those targets is an implicit declaration, which is an error under C99 and later with current compilers, or a link failure. 3. Defining strlcpy/strlcat at global scope in a vendored libraryWorth a decision before this lands rather than after. This library is vendored into ITK, AFNI, FSL, dcm2niix, and Slicer. Defining A file-local name such as
Verification
Confirms only that the conflict resolution is sound on a platform with native |
460ba38 to
4f6573d
Compare
|
Re-rebased onto current Still blocking, unchanged since the previous review: Rebase
Commit messagesBoth subjects were bare sentences; they now carry Blocker 1 — string_helper is still dead code (re-verified today)A full Blocker 2 — the guards exclude exactly the platforms that need the fallbackBoth files wrap their contents in: #ifndef __APPLE__
#if defined (__GLIBC__) && ((__GLIBC__ < 2) || ((__GLIBC__ == 2) && (__GLIBC_MINOR__ < 38)))The inner test requires The result on those targets is an implicit declaration — an error with current compilers at C99 and later — or a link failure.
Blocker 3 — defining strlcpy/strlcat at global scope in a vendored libraryUnresolved, and it wants a decision rather than a patch from me. This library is vendored into ITK, AFNI, FSL, dcm2niix, and Slicer. Defining A file-local name such as I have deliberately not redesigned this — the choice of spelling is the author's and the maintainer's. Test coverage — and why I added noneThe conversion itself is The Build and testmacOS/arm64, Ninja, Release, |
b046eb1 to
129a637
Compare
gdevenyi
left a comment
There was a problem hiding this comment.
Review by Google Carbon
Verdict: Request Changes (WIP Draft)
Reviewed the branch history and the bounds-safety / warning-cleanup changes in detail:
- Branch topology (75 orphaned commits): Only the top 2 commits (
7569506and129a637) belong to this PR; the 75 commits beneath them come from pre-resetmasterhistory overlapping#90,#92,#101,#106,#108,#111,#127, etc. Please cherry-pick only the 2strlcpy/strlcatcommits onto currentmaster. - Critical buffer overread &
SIZE_MAXheap/stack overflow instring_helper/string_helper.c(strlcat):strlen(ioDestination)reads pastinDestinationSizeifioDestinationis not NUL-terminated within the bound (standardstrlcatusesstrnlen(ioDestination, inDestinationSize)).- Worse, in
Releasebuilds (-DNDEBUG),assert(destinationLength < inDestinationSize)compiles out: whendestinationLength >= inDestinationSize(including validstrlcat(dst, src, 0)size queries!),inDestinationSize - 1underflows unsignedsize_ttoSIZE_MAXand executesmemcpy(ioDestination, inSource, SIZE_MAX)!
- Broken platform/C11 feature guards, missing CMake wiring, and C11 reserved-identifier / symbol collisions:
#if defined(__GLIBC__)excludes Windows/MSVC (causing theLNK2019CI failures), MinGW, musl, Solaris, etc. Furthermore, undermaster's-std=c11(CMAKE_C_EXTENSIONS OFF) andnifti1_io.c's#define _POSIX_C_SOURCE 200809L,<string.h>hidesstrlcpy/strlcaton both glibc >= 2.38 and macOS (_DARWIN_C_SOURCEis suppressed by_POSIX_C_SOURCE)!string_helper.cis not added to anyCMakeLists.txt,"string_helper.h"is never#included, and exporting globalstrlcpy/strlcatviolates C11 §7.31.13 (identifiers starting withstr+ lowercase letter are reserved for<string.h>) and causes duplicate-symbol errors when statically linkinglibniftiio+libnifti2+libnifticdf.- Recommended design: Replace
string_helper.cwith a private header-onlystatic inline size_t nifti_strlcpy(...)/nifti_strlcat(...)helper — zero symbol collisions, zero CMake target plumbing, and identical behavior on every platform/libc.
- Call-site bugs: Silent command truncation before
popen()indoPigz, uncheckednifti_makebasename/callocbeforestrlen/strlcpy, signedcharUB intoupper(*upt), and dropped zero-padding infill_field(fp->name).
| inDestinationSize -= destinationLength; | ||
| if (sourceLength >= inDestinationSize) | ||
| { | ||
| sourceLength = inDestinationSize - 1; |
There was a problem hiding this comment.
[Google Carbon] Critical SIZE_MAX buffer overflow in Release (-DNDEBUG) builds:
- Line 37 calls
strlen(ioDestination)instead ofstrnlen(ioDestination, inDestinationSize), overreading un-terminated buffers. - When
-DNDEBUGdisables line 41'sassert(destinationLength < inDestinationSize)— e.g. onstrlcat(dst, src, 0)or an already-full buffer wheredestinationLength == inDestinationSize—inDestinationSizebecomes0at line 44, soinDestinationSize - 1wraps toSIZE_MAXand line 49 runsmemcpy(ioDestination, inSource, SIZE_MAX)!
If destinationLength == inDestinationSize (using strnlen), strlcat must immediately return inDestinationSize + sourceLength; without writing a NUL byte.
| #ifndef __APPLE__ | ||
|
|
||
| // libc 2.38 and above provides strlcpy and strlcat, so don't redefine them. | ||
| #if defined (__GLIBC__) && ((__GLIBC__ < 2) || ((__GLIBC__ == 2) && (__GLIBC_MINOR__ < 38))) |
There was a problem hiding this comment.
[Google Carbon] Platform & standard-library visibility issues:
#if defined(__GLIBC__)leaves Windows MSVC (LNK2019), MinGW, and musl Linux without declarations or definitions.- Under
master's strict C11 mode (-std=c11,CMAKE_C_EXTENSIONS=OFF) plus#define _POSIX_C_SOURCE 200809Linnifti1_io.c/nifti2_io.c, system<string.h>hidesstrlcpy/strlcaton both glibc >= 2.38 (_DEFAULT_SOURCErequired) and Apple (_POSIX_C_SOURCEdisables_DARWIN_C_SOURCE). - Defining external
strlcpy/strlcatviolates C11 §7.31.13 reservedstr[a-z]*identifiers and will collide at link time acrosslibniftiio,libnifti2, andlibnifticdf.
Defining static inline size_t nifti_strlcpy(...) and static inline size_t nifti_strlcat(...) in an internal header avoids all libc feature-test macro pitfalls and symbol collisions.
| basename = nifti_makebasename(fname); | ||
| imgname = (char *)calloc(sizeof(char),strlen(basename)+8); | ||
| basename = nifti_makebasename(fname); | ||
| size_t imgnamelength = strlen(basename)+8; |
There was a problem hiding this comment.
[Google Carbon] nifti_makebasename(fname) returns NULL on allocation failure, so strlen(basename) dereferences NULL; likewise imgname = calloc(...) on line 2909 is unchecked before strlcpy(imgname, ...) (assert(ioDestination) / memcpy NULL deref). Also note calloc(sizeof(char), imgnamelength) has transposed arguments (-Wcalloc-transposed-args on GCC >= 14; should be calloc(imgnamelength, sizeof(char))).
| strlcpy(command, "pigz", sizeof(command)); | ||
| strlcat(command, " -n -f > \"", sizeof(command)); | ||
| strlcat(command, nim->fname, sizeof(command)); | ||
| strlcat(command, "\"", sizeof(command)); |
There was a problem hiding this comment.
[Google Carbon] Replacing strcpy/strcat with unchecked strlcat on a shell command passed to popen(command, "w") turns long nim->fname (>= 754 bytes) into silent path truncation with a missing closing quote ", invoking /bin/sh -c on a broken or unintended filename! Use int n = snprintf(command, sizeof(command), "pigz -n -f > \"%s\"", nim->fname); if (n < 0 || (size_t)n >= sizeof(command)) return -1; instead (and same in nifti2/nifti2_io.c).
| tmpstr = nifti_makebasename(argv[iarg]); | ||
| nim->fname = (char *)calloc(1,ll+8) ; strcpy(nim->fname,tmpstr) ; | ||
| nim->iname = (char *)calloc(1,ll+8) ; strcpy(nim->iname,tmpstr) ; | ||
| nim->fname = (char *)calloc(1,ll) ; strlcpy(nim->fname, tmpstr, ll) ; |
There was a problem hiding this comment.
[Google Carbon] Neither tmpstr (from nifti_makebasename) nor nim->fname / nim->iname (from calloc) is checked for NULL before strlcpy.
| return -1 ; | ||
| strlcpy(unam,name,size); | ||
| for( char *upt=unam ; *upt != '\0' ; upt++ ) | ||
| *upt = (char)toupper(*upt) ; |
There was a problem hiding this comment.
[Google Carbon] Passing a plain char (*upt) directly to toupper() is undefined behavior in C11 (§7.4) when char is signed and *upt < 0 (non-ASCII bytes). Cast to unsigned char first: *upt = (char)toupper((unsigned char)*upt);.
|
|
||
| strncpy(fp->name, name, sizeof(fp->name)); | ||
| fp->name[sizeof(fp->name) - 1] = 0; | ||
| strlcpy(fp->name, name, sizeof(fp->name)); |
There was a problem hiding this comment.
[Google Carbon] Note subtle behavioral change: strncpy(fp->name, name, NT_FIELD_NAME_LEN - 1) previously zero-padded the entire fp->name array, whereas strlcpy only writes a single NUL terminator, leaving trailing bytes of fp->name uninitialized if an external caller passes an unzeroed field_s struct (and nifti2/nifti_tool.c's fill_field should stay synchronized).
On macOS at least, this makes strlcpy() unavailable since it's not part of POSIX.
129a637 to
77bdffd
Compare
The old functions can easily overflow buffers but also they are not allowed under -fbounds-safety at all. Most cases were trivial. strncpy() notably zeros the whole buffer, while strlcpy() does not. I examined each case carefully and in most cases calloc() was used previously meaning the buffer was already zeroed. In one case, added a memset(0) to preserve the behaviour, those I'm not sure it really is necessary. In a few cases, too the opportunity to better move variable declaration to first usage or otherwise refactor a little bit.
strlcpy and strlcat came from OpenBSD and have since been added to other BSDs, macOS (and variants), and glibc (though only since 2.38). Add implementations for other platforms.
77bdffd to
7ee4cae
Compare
@hjmjohnson this one needs careful checking.
Also it's not done yet, I have to do some cmake stuff to compile the new file.