Skip to content

Check a later Gzip member's CRC before accepting a short trailer [patch] - #173

Merged
matt-edmondson merged 1 commit into
mainfrom
fix/171-gzip-member-crc
Nov 4, 2026
Merged

matt-edmondson merged 1 commit into
mainfrom
fix/171-gzip-member-crc

Conversation

@matt-edmondson

Copy link
Copy Markdown
Contributor

Fixes #171

What was wrong

To allow concatenated members, CompressedStreamEnd.Gzip.EndsWith accepted any input whose last 4 bytes, read as ISIZE, were smaller than the output so far (|| trailerLength < length), and it never checked a CRC for that case. In truncated input those bytes are payload. Deflate copies stored blocks verbatim, so a cut just after a zero field or a small little-endian integer was reported as success with partial output on the span, stream and async paths. The repro in the issue reproduces on main.

Change

A trailer whose size is less than the whole output is now accepted only if its CRC-32 matches the output it claims to cover:

  • Span path (IsComplete): the whole output is available, so the CRC of output[^trailerLength..] is computed and compared directly. This goes through a new EndsWith(inputTail, output) overload, which ZLib leaves at its default.
  • Stream and async paths: Append records where each read's output began, along with the CRC-32 of the output before it. GZipStream returns from a read at the end of each member, so a later member starts at one of these points. The suffix CRC is derived as crc(all) ^ shift(crc(before), suffixLength), using zlib's crc32_combine arithmetic (Multiply / XToThe), so no output has to be kept.

A last member that is empty (trailer size 0) is no longer accepted. Accepting a zero size is what made eight zero bytes at a cut look like a valid trailer, and an empty trailing member is not something the providers produce. ZLib and Brotli are unchanged.

Tests (TruncatedCompressionTests)

  • Gzip_Rejects_A_Stored_Block_Payload_Cut_After_A_Zeroed_Field: the issue's repro, checked on all four entry points
  • Gzip_Rejects_A_Stored_Block_Payload_Cut_At_Every_Offset: random bytes with small fields followed by zero runs, cut at every offset, on all four entry points
  • Gzip_Rejects_Concatenated_Members_Whose_Last_Member_Is_Cut: every cut inside the last member
  • Gzip_Accepts_Concatenated_Members_On_Every_Path: three members, the first larger than one 80 KiB read, so the CRC combine runs on the stream paths

The three rejection tests fail on main and pass with the fix. The existing Gzip_Accepts_Concatenated_Members still passes. Full suite: 1349 passed, 1 skipped. Essentials.CompressionProviders.Gzip builds for all its targets with -warnaserror.


Generated by Claude Code

To allow concatenated members, the Gzip end check accepted any input whose
last four bytes read as a size below the output so far. In truncated input
those bytes are payload, and stored blocks copy payload verbatim, so a cut
just after a zero or small little-endian field "succeeded" with partial
output on the span, stream and async paths.

Accept a shorter trailer only when its CRC-32 matches the output it claims
to cover: on the span path by checking that suffix directly, and on the
stream paths by deriving the CRC of the output since each read began,
because GZipStream returns from a read at every member's end.

Fixes #171

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_017vfJW671w9qNTjehHKvwnt
@sonarqubecloud

sonarqubecloud Bot commented Nov 3, 2026

Copy link
Copy Markdown

@matt-edmondson
matt-edmondson merged commit 7626969 into main Nov 4, 2026
15 checks passed
@matt-edmondson
matt-edmondson deleted the fix/171-gzip-member-crc branch November 4, 2026 00:54
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Gzip TryDecompress/Decompress still return truncated data as success when the cut input's last 4 bytes read as a number below the output length

2 participants