Repository navigation
Perfect-SMTP 6.0.1: MTA-STS fetcher close_notify, deadline/cancellation; LocalMTA EINTR - #33
Merged
Merged
Conversation
… on close Closing a NIOSSL channel starts a clean TLS shutdown: it sends close_notify and keeps reading (and discarding) until the peer's close_notify or NIOSSL's shutdownTimeout, 5 s by default. A policy host that streams an endless body and ignores the client's close_notify kept every capped, timed-out or refused fetch's connection reading at full speed for 5 s: on macOS, 50 capped fetches took 15-21 s and the server pushed 40-56 GB. The fetcher now turns autoRead off before closing (on completion, cap, deadline and a refused certificate) and sets the TLS shutdownTimeout to 1 ms; a client that has its response, or has given up, needs no clean TLS shutdown. Acceptance is unchanged: a response is complete only by Content-Length, the chunked terminator, or, for a close-delimited body, the server's close_notify. New tests pin that down (short Content-Length and unterminated chunked bodies, ended by close_notify or by a bare TCP close; a close-delimited body without close_notify), along with the let-go behaviour on the cap, deadline and refused-certificate paths, which fail on 6.0.0 at 5.0 s each. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Review of the close_notify fix found three older problems:
- The deadline only failed the response promise. A server that accepted
and never finished the TLS handshake held the fetch (and its fd) until
the server closed; a slow connect, too, ran past the deadline and then
sent the GET anyway.
- Cancelling the calling task cancelled the deadline with it, and the fetch
then waited for the server indefinitely.
- A resolver error left the response promise uncompleted, which traps
("leaking promise") in debug builds: any NXDOMAIN or SERVFAIL for
mta-sts.<domain>.
As in Perfect-Authentication's NIOTransport, one loop-confined state and an
abort (from a loop-scheduled deadline or a task-cancellation handler) now
fail every pending stage (resolution, each connect attempt, the handshake,
the response), close the channel, and close any channel that connects
after the abort. Every promise is completed on every path. A cancelled
fetch throws CancellationError.
Hardening: the response collector fails an incomplete response on any
close that passes it, so a close that ends through NIOSSL's (now 1 ms)
shutdown timer, which leaves no uncleanShutdown behind, can't end a
close-delimited body as if the server had.
Tests: a stalled handshake against a silent server, cancellation during the
handshake and during the response, a resolver error (all hang or trap on
6.0.0). The refused-certificate test now runs over TLS 1.2: under 1.3 the
client's close_notify could arrive with its Finished and be answered before
the server went deaf, so on 6.0.0 it passed 3 of 8 runs; now it fails 6 of
6 there.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The request target came from `url.path`, which Foundation percent-decodes
and strips of a trailing slash: `/api/` went out as `/api`, `/%3F/?` as
`/??`, and a path with `%20` or non-ASCII never went out at all ("invalid
character in header"). It is now URLComponents' `percentEncodedPath`
(empty becomes `/`) plus `percentEncodedQuery`. Not reachable through
MTASTSPolicyManager, whose path is always `/.well-known/mta-sts.txt`; only
direct callers of the public fetcher were affected. Found in
Perfect-Authentication's NIOTransport, which was copied from this fetcher.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…rget Re-review follow-ups: - The channel was recorded only once its connect succeeded, so a deadline or cancellation during a slow connect returned at once but left the socket open until the connect finished or `connectTimeout` (the whole timeout) passed. The channel is now recorded in the channel initializer, before the connect starts, so an abort closes it (failing the connect); an initializer that runs after an abort refuses the channel. Checked by hand against a blackholed address: cancelled at 0.3 s, the fetch returns in 1 ms and the socket is gone within 0.1 s (before: about 5 s more). - The request target comes from a new `requestTarget(_:)`: a URL without components is refused (`unsupportedURL`) instead of fetching "/", and so is a target holding a space, a control character or a non-ASCII byte (defence in depth: NIO doesn't check the request URI, and older Foundations differ on encoding such characters). - Tests: `;` params, an empty query with a fragment, userinfo and fragment never reaching the target; the stalled-handshake test measures the close from when the fetch returned. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The sendmail stdout/stderr drains used `FileHandle.availableData`, which
corelibs Foundation turns into a fatalError on any read error, EINTR
included ("Interrupted system call", FileHandle.swift:231), so a signal
interrupting the read crashed the process. 6.0.1's full suite hit it in 3
of 3 runs on emulated linux/amd64 (6.2 and 6.4); 6.0.0's passed 2 of 2
there, so the new fetcher tests' load exposes it rather than causing it.
It is not emulation-only: the new test reproduces the trap on native
linux/arm64 against the old code. The drains now read(2) in a loop that
retries EINTR and throws any other error. The stdin write path is
unaffected: corelibs retries EINTR on write.
Test: a reader blocked on a pipe is interrupted five times with SIGUSR1
(handler with SA_RESTART cleared via sigaction, since `siginterrupt` is
deprecated in glibc; SIGUSR1 explicitly unblocked in the reader, since
the test runner's threads start with it blocked) and still returns the
full contents. Against the old code it traps on swift:6.4-noble arm64.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Patch release for
NIOMTASTSHTTPFetcher, the default MTA-STS policy fetcher, plus a Linux crash fix inLocalMTATransport. No API changes.Fixes
A capped, timed-out or refused fetch stops reading at once. Closing a NIOSSL channel starts a clean TLS shutdown, which keeps reading and discarding until the peer's close_notify arrives or
shutdownTimeoutexpires (5 s by default). A policy host that streams an endless body and ignores close_notify therefore kept every capped fetch's connection busy for 5 s. The fetcher now turnsautoReadoff before closing and uses a 1 ms TLSshutdownTimeout. This is the same fix as Perfect-Authentication's NIOTransport.A partial body is still never accepted. A response counts as complete only by its Content-Length, by the chunked terminator, or, for a body that ends at the close, by the server's TLS close_notify. New tests cover a short Content-Length body and an unterminated chunked body (ended by close_notify or by a bare TCP close), plus a close-delimited body without close_notify. The response collector also fails an incomplete response on any close that passes through it.
The deadline and cancellation end every stage of a fetch. These bugs were already in 6.0.0.
The deadline now runs on the event loop and cancellation has a handler. Either one aborts resolution, every connect attempt, the handshake and the response, and closes the channel, including a connect still in flight. A cancelled fetch throws
CancellationError.The request target is the URL's percent-encoded path and query. It used to come from
url.path, which Foundation percent-decodes, so/api/went out as/api,/%3F/?as/??, and%20or non-ASCII paths failed. A target holding a space, a control character or a non-ASCII byte is now refused. This doesn't affectMTASTSPolicyManager, whose path is fixed; only code calling the fetcher directly was affected.LocalMTATransportno longer crashes on an interrupted pipe read (Linux). CorelibsFileHandle.availableDataturns EINTR into afatalError. The drains now use aread(2)loop that retries EINTR. The new test reproduces the crash against the old code on native linux/arm64.Measurements
50 back-to-back capped HTTPS fetches against a local endless-body server that ignores close_notify. Linux runs used the
containerCLI with--memory 6g, release builds.Times cover all 50 fetches; bytes are what the server wrote to loopback. On 6.0.0, every connection stayed open 5.0 s after its fetch returned; on 6.0.1, each closes within 0.01 s. Peak RSS stayed under 60 MB and fds returned to baseline in every run.
Verification
-warnings-as-errors. Full suite passes (498 tests).swift:6.2-nobleandswift:6.4-noble, arm64 and amd64,-warnings-as-errors. Full suite passes. On emulated 6.2 amd64,DirectMXRetryQueueTeststimed out once in 7 runs: a 3 s wait starved by emulation, an existing timing margin.Known issues and follow-ups
shutdownTimeout.DNSResolver's own query timeouts.🤖 Generated with Claude Code