Skip to content

Tolerate malformed part headers in multipart parsing - #11

Merged
thunderkatz merged 6 commits into
masterfrom
mpasquale-malformed-part-headers
Aug 28, 2026
Merged

thunderkatz merged 6 commits into
masterfrom
mpasquale-malformed-part-headers

Conversation

@mpasquale-3

@mpasquale-3 mpasquale-3 commented Aug 21, 2026

Copy link
Copy Markdown

Summary

My Summary:

When there is an empty line in the header section, which is a malformed line, Apple Mail still renders the HTML perfectly. We need to parse equally permissively to effectively detect phishes. When there are malformed lines in the headers, skip them and see if you can still form a valid header with subsequent lines. Return the recovered parts along with the error so the caller can decide how to proceed.

🤖 Generated with Claude Code

@mpasquale-3
mpasquale-3 force-pushed the mpasquale-malformed-part-headers branch from 26bf54b to ef07c32 Compare August 21, 2026 22:32
@mpasquale-3
mpasquale-3 requested a review from a team August 21, 2026 22:48
@mpasquale-3 mpasquale-3 changed the title Tolerate malformed part headers in multipart parsing textproto: tolerate malformed part headers in multipart parsing Aug 22, 2026
@mpasquale-3 mpasquale-3 changed the title textproto: tolerate malformed part headers in multipart parsing Tolerate malformed part headers in multipart parsing Aug 22, 2026
Base automatically changed from mpasquale-lenient-duplicate-param-recovery to master August 24, 2026 20:06
When a MIME part's header block contains a line with no colon (e.g.
"X-Notice this line is not a valid header field"), ReadHeader fails and
previously newPart() aborted the entire multipart parse.

Now newPart() retries populateHeaders() from the reader's new position
after the bad line. If the remaining lines form a valid header block the
part is returned alongside a MalformedPartHeaderError, allowing callers
to extract the part body (e.g. a phishing HTML payload). If retry also
fails the part is discarded cleanly to the next boundary so subsequent
parts are still reachable.

NextPart() propagates (part, error) when recovery succeeds. Walk() and
the message-level multipartReader.NextPart() treat IsMalformedPartHeader
the same as IsUnknownEncoding / IsUnknownCharset.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
@mpasquale-3
mpasquale-3 force-pushed the mpasquale-malformed-part-headers branch from ef07c32 to 35f6b77 Compare August 24, 2026 20:08
mpasquale-3 and others added 4 commits August 24, 2026 13:12
…ability signal

Callers check part != nil to determine whether the recovered part is usable,
regardless of whether err is non-nil. The invariant mirrors the MalformedHeaderError
removal: non-nil return value means the values are safe to use.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
The `err != nil` guard is redundant in the else-if chain:
NextPart only returns a non-nil part during header recovery,
which always produces a non-nil error, so `part != nil`
is sufficient.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
…ader

Recovery from a malformed part header line was landing as an untyped,
un-greppable error, distinguishable only by checking whether NextPart's
returned part was non-nil. That's inconsistent with how the rest of the
package signals this kind of lenient-recovery condition
(IsUnknownEncoding, IsUnknownCharset), and forces every error-classification
cascade in a caller to be manually audited rather than mechanically checked
for the new case.

Bring back textproto.MalformedPartHeaderError/IsMalformedPartHeader, and
add a message-level IsMalformedPartHeader that forwards to it so callers
of the higher-level Entity/MultipartReader API don't need to import
textproto directly. Entity.Walk uses the typed check instead of a bare
part != nil.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
@thunderkatz

Copy link
Copy Markdown

Since the original author is no longer with the team, picked this back up to close a gap before pulling it into go-mantis.

The removal of MalformedPartHeaderError in favor of a bare part != nil check breaks an implicit invariant every downstream caller relies on: today, a non-nil part alongside a non-nil error only ever means IsUnknownEncoding or IsUnknownCharset. Any caller with an error-classification cascade over those two (which is a common pattern) will silently mis-file the new malformed-header case into an "unexpected error" bucket instead of treating it as recoverable — verified this breaks two call sites in go-mantis (services/mdm and entity_utils/walk.go), both of which drop trailing sibling parts and misreport a benign quirk as an internal error.

Restored textproto.MalformedPartHeaderError / IsMalformedPartHeader, consistent with the existing IsUnknownEncoding/IsUnknownCharset pattern, and added a message.IsMalformedPartHeader forwarder so callers of the higher-level Entity/MultipartReader API don't need to import textproto directly. Entity.Walk now uses the typed check instead of part != nil. Added a TestWalk_malformedPartHeader test at the message package level to cover the forwarded case end-to-end.

thunderkatz
thunderkatz previously approved these changes Aug 28, 2026
@thunderkatz
thunderkatz merged commit 261dfb4 into master Aug 28, 2026
3 checks passed
@thunderkatz
thunderkatz deleted the mpasquale-malformed-part-headers branch August 28, 2026 14:48
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants