Conversation
cboulay
force-pushed
the
fix/272-shm-grow-retained-views
branch
from
October 1, 2026 06:47
8db92a0 to
d5aa26a
Compare
cboulay
changed the base branch from
cboulay/json-schema-settings-stack
to
cboulay/wire-elision
October 1, 2026 06:47
cboulay
force-pushed
the
fix/272-shm-grow-retained-views
branch
2 times, most recently
from
October 1, 2026 06:51
d5aa26a to
7e3cec0
Compare
cboulay
added a commit
that referenced
this pull request
Oct 1, 2026
Split from #273: the parts that stand on their own, without the ChannelFailed policy that is still under discussion. - SHMContext: if closing the mapping raises BufferError because views into it are still alive (a unit kept a zero-copy array), drop the handle and let the mmap be unmapped when the last view is collected, instead of killing the channel when the publisher grows its segment. - SHMContext.wait_closed() also closes the segment: a monitor task cancelled before it first ran never executed its finally, leaking the mapping (the "Exception ignored in SharedMemory.__del__" noise). - Channel: SHM-delivered messages are read-only, matching the TCP path, so a subscriber cannot write into memory shared with the publisher and every other subscriber. - Tests: closing with a live view, and a cross-process grow with a receiver that retains every message.
cboulay
force-pushed
the
cboulay/wire-elision
branch
from
October 1, 2026 16:22
62ffa8e to
c16bff6
Compare
cboulay
force-pushed
the
fix/272-shm-grow-retained-views
branch
from
October 1, 2026 16:22
7e3cec0 to
3871eea
Compare
cboulay
force-pushed
the
fix/272-shm-grow-retained-views
branch
from
October 1, 2026 16:33
3871eea to
2bd2882
Compare
…272) What remains of the #272 work after the uncontroversial parts moved to - Channel: a crashed publisher connection records its exception and notifies its clients; Subscriber.recv_zero_copy raises ChannelFailed (chained from the cause) instead of waiting forever. handle_subscriber logs it at ERROR and keeps serving the stream's other publishers. - GraphServer: a segment whose last lease has ended is marked unlinked and forgotten, so a late SHM_ATTACH is refused (ValueError, which channels already treat as a stale generation) instead of handing back a name the client cannot open. Unlinked entries are also pruned on SHM_CREATE, and a segment is never unlinked twice. - Channel: FileNotFoundError on attach is treated as stale too, for older GraphServers.
cboulay
force-pushed
the
fix/272-shm-grow-retained-views
branch
from
October 1, 2026 17:20
2bd2882 to
b37b82a
Compare
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Part of #272. Draft, pending review (Griff). Stacked on #276.
The uncontroversial parts of the original #273 moved to #275 (tolerant SHM close,
wait_closedleak, read-only SHM messages, and the grow-race fix). What's left here implies policy choices, so it's held for review.Changes
_publisher_connectionrecords the exception and notifies its clients.Subscriber.recv_zero_copy()raises a newChannelFailed, chained from the cause, instead of waiting forever. In a unit,handle_subscriberlogs it at ERROR and keeps serving the stream's other publishers.SHMInfomarks itselfunlinkedwhen its last lease ends (and never unlinks twice). The GraphServer forgets such entries, both on a lateSHM_ATTACHand as a sweep onSHM_CREATE. A late attach is then refused (ValueError, which the channel already treats as a stale generation) instead of succeeding on the server and failing on the client withFileNotFoundError.FileNotFoundErroron attach as stale too, for older GraphServers.With #275's deferred close, a publisher no longer retires a segment a channel still needs, so the last two are defensive.
Open questions
ChannelFailedshould do in a unit: currently log and keep serving other publishers. Alternatives: end the subscriber when none remain, mark the shutdown unclean, or reconnect the channel.Tests
test_subclient.py::test_recv_zero_copy_raises_channel_failedtest_shm.py::test_attaching_an_unlinked_segment_is_refused(fails without the GraphServer change)Full suite: only the 10
test_settings_json_schema/test_inspect_commandfailures that also fail on the base branch.test_shm_resize_race_repro_completesand the grow tests pass 6/6. The earlier hang came from losing the grow race, which #275 now fixes.