Skip to content

Perfect-SMTP 6.0.1: MTA-STS fetcher close_notify, deadline/cancellation; LocalMTA EINTR - #33

Merged
taplin merged 5 commits into
mainfrom
release-6.0.1
Oct 26, 2026
Merged

taplin merged 5 commits into
mainfrom
release-6.0.1

Conversation

@taplin

@taplin taplin commented Oct 26, 2026

Copy link
Copy Markdown
Contributor

Patch release for NIOMTASTSHTTPFetcher, the default MTA-STS policy fetcher, plus a Linux crash fix in LocalMTATransport. No API changes.

Fixes

  • A capped, timed-out or refused fetch stops reading at once. Closing a NIOSSL channel starts a clean TLS shutdown, which keeps reading and discarding until the peer's close_notify arrives or shutdownTimeout expires (5 s by default). A policy host that streams an endless body and ignores close_notify therefore kept every capped fetch's connection busy for 5 s. The fetcher now turns autoRead off before closing and uses a 1 ms TLS shutdownTimeout. This is the same fix as Perfect-Authentication's NIOTransport.

  • A partial body is still never accepted. A response counts as complete only by its Content-Length, by the chunked terminator, or, for a body that ends at the close, by the server's TLS close_notify. New tests cover a short Content-Length body and an unterminated chunked body (ended by close_notify or by a bare TCP close), plus a close-delimited body without close_notify. The response collector also fails an incomplete response on any close that passes through it.

  • The deadline and cancellation end every stage of a fetch. These bugs were already in 6.0.0.

    • A server that accepted the connection and never finished the TLS handshake held the fetch until the server hung up.
    • Cancelling the caller made the fetch hang.
    • A resolver error left a promise uncompleted, which crashes debug builds with "leaking promise".

    The deadline now runs on the event loop and cancellation has a handler. Either one aborts resolution, every connect attempt, the handshake and the response, and closes the channel, including a connect still in flight. A cancelled fetch throws CancellationError.

  • The request target is the URL's percent-encoded path and query. It used to come from url.path, which Foundation percent-decodes, so /api/ went out as /api, /%3F/? as /??, and %20 or non-ASCII paths failed. A target holding a space, a control character or a non-ASCII byte is now refused. This doesn't affect MTASTSPolicyManager, whose path is fixed; only code calling the fetcher directly was affected.

  • LocalMTATransport no longer crashes on an interrupted pipe read (Linux). Corelibs FileHandle.availableData turns EINTR into a fatalError. The drains now use a read(2) loop that retries EINTR. The new test reproduces the crash against the old code on native linux/arm64.

Measurements

50 back-to-back capped HTTPS fetches against a local endless-body server that ignores close_notify. Linux runs used the container CLI with --memory 6g, release builds.

Platform 6.0.0 6.0.1
macOS, Swift 6.4 15.7–21.4 s, 40–56 GB 0.06–0.08 s, 33–46 MB
Linux 6.2 arm64 28.5–30.5 s, 68–77 GB 0.06–0.07 s, 193–201 MB
Linux 6.4 arm64 33.8–40.1 s, 81–104 GB 0.07–0.09 s, 181–201 MB
Linux 6.2 amd64 (emulated) 68.2–88.3 s, 144–189 GB 0.16–0.17 s, 145–182 MB
Linux 6.4 amd64 (emulated) 87.2–91.7 s, 148–178 GB 0.15–0.16 s, 168–187 MB

Times cover all 50 fetches; bytes are what the server wrote to loopback. On 6.0.0, every connection stayed open 5.0 s after its fetch returned; on 6.0.1, each closes within 0.01 s. Peak RSS stayed under 60 MB and fds returned to baseline in every run.

Verification

  • macOS: Xcode 27.0 (Swift 6.4) and Xcode 26.3 (Swift 6.2.4), debug and release, -warnings-as-errors. Full suite passes (498 tests).
  • Linux: swift:6.2-noble and swift:6.4-noble, arm64 and amd64, -warnings-as-errors. Full suite passes. On emulated 6.2 amd64, DirectMXRetryQueueTests timed out once in 7 runs: a 3 s wait starved by emulation, an existing timing margin.
  • On 6.0.0, every new test fails, hangs or traps.
  • Review: four adversarial review rounds; the final verdict was push.

Known issues and follow-ups

  • SMTP connections themselves still close TLS with NIOSSL's default 5 s shutdownTimeout.
  • An aborted fetch doesn't cancel its DNS lookup, which runs until DNSResolver's own query timeouts.

🤖 Generated with Claude Code

taplin and others added 5 commits October 25, 2026 23:53
… on close

Closing a NIOSSL channel starts a clean TLS shutdown: it sends close_notify
and keeps reading (and discarding) until the peer's close_notify or NIOSSL's
shutdownTimeout, 5 s by default. A policy host that streams an endless body
and ignores the client's close_notify kept every capped, timed-out or
refused fetch's connection reading at full speed for 5 s: on macOS, 50
capped fetches took 15-21 s and the server pushed 40-56 GB. The fetcher now
turns autoRead off before closing (on completion, cap, deadline and a
refused certificate) and sets the TLS shutdownTimeout to 1 ms; a client
that has its response, or has given up, needs no clean TLS shutdown.

Acceptance is unchanged: a response is complete only by Content-Length,
the chunked terminator, or, for a close-delimited body, the server's
close_notify. New tests pin that down (short Content-Length and
unterminated chunked bodies, ended by close_notify or by a bare TCP close;
a close-delimited body without close_notify), along with the let-go
behaviour on the cap, deadline and refused-certificate paths, which fail
on 6.0.0 at 5.0 s each.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Review of the close_notify fix found three older problems:

- The deadline only failed the response promise. A server that accepted
  and never finished the TLS handshake held the fetch (and its fd) until
  the server closed; a slow connect, too, ran past the deadline and then
  sent the GET anyway.
- Cancelling the calling task cancelled the deadline with it, and the fetch
  then waited for the server indefinitely.
- A resolver error left the response promise uncompleted, which traps
  ("leaking promise") in debug builds: any NXDOMAIN or SERVFAIL for
  mta-sts.<domain>.

As in Perfect-Authentication's NIOTransport, one loop-confined state and an
abort (from a loop-scheduled deadline or a task-cancellation handler) now
fail every pending stage (resolution, each connect attempt, the handshake,
the response), close the channel, and close any channel that connects
after the abort. Every promise is completed on every path. A cancelled
fetch throws CancellationError.

Hardening: the response collector fails an incomplete response on any
close that passes it, so a close that ends through NIOSSL's (now 1 ms)
shutdown timer, which leaves no uncleanShutdown behind, can't end a
close-delimited body as if the server had.

Tests: a stalled handshake against a silent server, cancellation during the
handshake and during the response, a resolver error (all hang or trap on
6.0.0). The refused-certificate test now runs over TLS 1.2: under 1.3 the
client's close_notify could arrive with its Finished and be answered before
the server went deaf, so on 6.0.0 it passed 3 of 8 runs; now it fails 6 of
6 there.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The request target came from `url.path`, which Foundation percent-decodes
and strips of a trailing slash: `/api/` went out as `/api`, `/%3F/?` as
`/??`, and a path with `%20` or non-ASCII never went out at all ("invalid
character in header"). It is now URLComponents' `percentEncodedPath`
(empty becomes `/`) plus `percentEncodedQuery`. Not reachable through
MTASTSPolicyManager, whose path is always `/.well-known/mta-sts.txt`; only
direct callers of the public fetcher were affected. Found in
Perfect-Authentication's NIOTransport, which was copied from this fetcher.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…rget

Re-review follow-ups:

- The channel was recorded only once its connect succeeded, so a deadline
  or cancellation during a slow connect returned at once but left the
  socket open until the connect finished or `connectTimeout` (the whole
  timeout) passed. The channel is now recorded in the channel initializer,
  before the connect starts, so an abort closes it (failing the connect);
  an initializer that runs after an abort refuses the channel. Checked by
  hand against a blackholed address: cancelled at 0.3 s, the fetch returns
  in 1 ms and the socket is gone within 0.1 s (before: about 5 s more).
- The request target comes from a new `requestTarget(_:)`: a URL without
  components is refused (`unsupportedURL`) instead of fetching "/", and so
  is a target holding a space, a control character or a non-ASCII byte
  (defence in depth: NIO doesn't check the request URI, and older
  Foundations differ on encoding such characters).
- Tests: `;` params, an empty query with a fragment, userinfo and fragment
  never reaching the target; the stalled-handshake test measures the close
  from when the fetch returned.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The sendmail stdout/stderr drains used `FileHandle.availableData`, which
corelibs Foundation turns into a fatalError on any read error, EINTR
included ("Interrupted system call", FileHandle.swift:231), so a signal
interrupting the read crashed the process. 6.0.1's full suite hit it in 3
of 3 runs on emulated linux/amd64 (6.2 and 6.4); 6.0.0's passed 2 of 2
there, so the new fetcher tests' load exposes it rather than causing it.
It is not emulation-only: the new test reproduces the trap on native
linux/arm64 against the old code. The drains now read(2) in a loop that
retries EINTR and throws any other error. The stdin write path is
unaffected: corelibs retries EINTR on write.

Test: a reader blocked on a pipe is interrupted five times with SIGUSR1
(handler with SA_RESTART cleared via sigaction, since `siginterrupt` is
deprecated in glibc; SIGUSR1 explicitly unblocked in the reader, since
the test runner's threads start with it blocked) and still returns the
full contents. Against the old code it traps on swift:6.4-noble arm64.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@taplin
taplin merged commit 36f1d5e into main Oct 26, 2026
18 checks passed
@taplin
taplin deleted the release-6.0.1 branch October 26, 2026 11:32
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant