[TeoSlayer]

Fixes three dataexchange problems from the Pilot reliability sweep: the default inbox byte cap deletes the whole inbox, replies can't be matched to their requests, and duplicate deliveries are stored twice. The branch has 4 commits over c1f3d84 and merges cleanly with main at 00f5945.

Problem (sweep evidence)

1. With the default config, the inbox byte cap deletes the entire inbox (sweep item 10, HYG-01)

• service.go:803-809 checks the cap with effectiveInboxMaxBytes(). With the default InboxMaxBytes == 0 that is 256 MiB, so the check fires and calls evictInboxOverflowByBytes.
• evictInboxOverflowByBytes (service.go:943-982, lines :968 and :974) compares against the raw s.cfg.InboxMaxBytes, which is 0. It removes every file.
• The periodic pass (:899-901) had the same bug in reverse: with the default config it never applied the byte cap at all.
• web4 cmd/daemon/main.go:375-377 builds ServiceConfig with only IncludeBase64 set, so every default install takes this path.
• The bug came in with 1cd6f58 (Harden dataexchange against resource-exhaustion DoS #25). It is in v0.2.1 (shipped in web4 v1.13.9), v0.2.2 and main.
• Scratch repro: 257 × 1 MiB old files, 5 recent replies, then one incoming message. The log showed inbox eviction (bytes) evicted=262 total_bytes_after=0 max_bytes=0, and 0 of 5 recent replies survived.
• DefaultMaxFrameSize is 64 MiB, so any peer allowed to send can trigger the wipe with about 4 large messages.
• Every incoming message also ran inboxTotalBytes, a full ReadDir plus a stat of every file (service.go:804).

2. send-message --wait returns another query's reply, with exit 0 (sweep item 2, ONB-01 / HYG-02 / NET-02)

• web4 cmd/pilotctl/main.go:4725 sets the cutoff one second before the send.
• waitForInboxReply (main.go:6616-6657) returns the first file in directory order from the right sender. Nothing in the frame ties a reply to its request.
• On the live node, with unique keywords to rule out other agents, 4 of 5 back-to-back list-agents queries got the wrong answer. In a test harness, two concurrent waiters got the same file.
• pilot-mcp query.js and search.js use the same path.
• This PR adds the missing field to the wire format. web4 matches on it in a later PR (see Deploy notes).

3. list-agents answers every request twice (sweep item 11, ONB-05)

• Each request leaves 2 inbox files with the same md5, arriving 83 ns to 743 ms apart. This happened 3 of 3 times on a fresh node and 5 of 5 on the live node.
• The daemon logs inbox message saved twice (service.go:877).
• The late copy is often the wrong answer in item 2. The responder fix belongs in the list-agents service. This PR adds receiver-side dedupe as a safeguard.

What changed

• 7e88ecd fix(inbox): stop the default byte cap from deleting the whole inbox
  • Both eviction paths use the effective cap.
  • Eviction removes the oldest files until the inbox plus the incoming message is at or below 90% of the cap.
  • A message larger than the cap, or larger than the cap minus writes already in flight, is rejected without evicting anything.
  • The cap is checked against a running byte total. One scan seeds it and the periodic pass re-seeds it. The directory is re-read before anything is evicted, because other processes delete inbox files.
  • Regression test TestInboxByteCap_DefaultConfigEvictsOldestNotEverything: on main, 0 of 5 recent replies survive. With the fix, all 5 survive and about 27 old files are evicted.
• e746029 feat: optional request/reply IDs on frames and receiver-side dedupe
  • Frame.MessageID and Frame.ReplyTo. A frame with either one travels as the new TypeTagged (10): [2-byte header_len][JSON header][4-byte inner_type][inner payload].
  • IDs are 1-128 characters from [A-Za-z0-9._:-]. Unknown header keys are ignored.
  • WriteFrame wraps the frame and ReadFrame unwraps it. Helpers: NewMessageID and ValidMessageID.
  • The receiver records message_id / reply_to in the inbox JSON and in message.received / file.received events, only when they are present. Governed and trace frames carry them through. They are correlation metadata only and are not covered by the governed signature.
  • Client.Send waits for the ACK. If the receiver predates TypeTagged, Send re-sends the frame untagged on the same connection and returns SendResult.Tagged == false.
  • Dedupe: if a stored frame carried a MessageID, an identical re-delivery gets the original ACK plus (duplicate) and is not stored again. The window is ServiceConfig.DedupeWindow (default 10 min) and the cache is bounded. Concurrent identical deliveries wait for the first one. If the first attempt fails, it is forgotten so a retry is processed normally.
  • DedupeContentWindow (off by default) extends dedupe to frames without a MessageID.
  • The README explains what a service must do: echo the request's message_id as ReplyTo.
• 0b8e551 fix(inbox): never evict a message another connection is still writing (review finding DX-REV-2)
  • Each open reservation records its destination path. Rescans leave those files out of both the eviction candidates and the on-disk total, so their bytes are counted once.
  • Before: in the reviewer's repro (two ~20 MiB messages saved concurrently into a 64 MiB inbox holding 30 MiB), one message was lost in 3 of 60 and 5 of 60 trials.
  • After: 0 of 180 lost.
• e457424 fix(tagged): fall back for every pre-TypeTagged receiver; keep tagged frames under MaxFrameSize
  • DX-REV-1: v0.2.1 and older (every stable daemon, web4 v1.12.0 through v1.13.9) answer a type-10 frame with ACK UNKNOWN(10) <n> bytes and store nothing. Before this commit, Send reported that as delivered with Tagged == true. Send now treats both ERR UNKNOWN(10) (v0.2.2 and later) and ACK UNKNOWN(10) as "receiver predates TypeTagged" and re-sends untagged. A current receiver unwraps the frame before it ACKs and names the inner type, so it never produces either prefix.
  • DX-REV-3: a payload within 293 bytes of the cap was valid untagged but too large once tagged. The receiver dropped the connection and there was no fallback. WriteFrame now returns ErrTaggedFrameTooLarge before writing anything, and Send handles it by sending the frame untagged. The worst-case header size is exported as MaxTaggedOverhead (293).

Review outcome

An independent review found 3 problems in the first two commits. All 3 are fixed on this branch:

• DX-REV-1 (silent loss to v0.2.1 and older receivers): fixed in e457424
• DX-REV-2 (eviction of an in-flight write): fixed in 0b8e551
• DX-REV-3 (tagged frame over the max frame size): fixed in e457424

Compatibility

• Wire: untagged frames are byte-for-byte unchanged. A frame is tagged only when the sender sets MessageID or ReplyTo. A tagged frame is refused only when it would exceed MaxFrameSize.

• New sender to an old receiver: Client.Send falls back to an untagged frame (Tagged == false). A raw WriteFrame of a tagged frame gets no fallback.

• Old sender to a new receiver: unchanged.

• Go API: additive only. New exported names:

  • Frame.MessageID, Frame.ReplyTo, TypeTagged
  • Client.Send and SendResult{Ack, Tagged, Duplicate}
  • NewMessageID, ValidMessageID
  • ServiceConfig.DedupeWindow, ServiceConfig.DedupeContentWindow
  • ErrTaggedFrameTooLarge, MaxTaggedOverhead

  The new unexported helpers reserveInbox(dir, path, size, max) and inboxWriteDone(path, size, written) are internal.

• Behavior:

  • When the byte cap is full, the oldest messages are now evicted to make room, as InboxMaxBytes was documented to do. Before, the new message was rejected. The two tests that pinned the old behavior now check that the cap holds and that new messages land.
  • Frames without a MessageID are still all stored unless the operator sets DedupeContentWindow.

Cross-version run over TCP

The sender was built from this branch. It ran the same send-and-ACK path as Client.Send, once with MessageID=req-123 and once without an ID. Receivers were real Service instances (harness in scratch/dx-rev-xver). Rerun on 2026-09-24:

Receiver	Send with MessageID	Stored	Send without an ID
v0.2.1 (rebuilt from git archive)	ACK TEXT 12 bytes, Tagged=false	once, no message_id	ACK TEXT 11 bytes, stored once
v0.2.1 (reviewer's binary)	ACK TEXT 12 bytes, Tagged=false	once, no message_id	ACK TEXT 11 bytes, stored once
v0.2.2 (base)	ACK TEXT 12 bytes, Tagged=false	once, no message_id	ACK TEXT 11 bytes, stored once
v0.2.2 (base), governed	ERR TEXT save failed: unsigned legacy frame rejected by governed receiver, Tagged=false	nothing	same rejection
this branch	ACK TEXT 12 bytes, Tagged=true	once, "message_id":"req-123"	ACK TEXT 11 bytes, stored once
this branch, governed	same unsigned-frame rejection, Tagged=true	nothing	same rejection

• Before e457424, v0.2.1 answered the send with an ID as ACK UNKNOWN(10) ... and stored nothing, but it was reported as delivered with Tagged=true.
• The harness sender does not sign frames, so both governed receivers reject both sends with the same error. A tagged frame is rejected the same way as an untagged one.

Tests

All commands ran with GOWORK=off on HEAD e457424:

• gofmt -l .: clean
• go vet ./... and go vet -tags no_dataexchange ./...: clean
• go test -race -covermode=atomic -coverprofile ./... (the CI command): ok, 77.0% coverage
• go test -tags no_dataexchange ./...: ok
• go test -race -count=5 ./: ok
• 20-second fuzz runs of FuzzTaggedFrameRoundTrip and FuzzTaggedFrameRead: pass
• staticcheck ./...: no new findings compared with the base. The same 3 findings exist before and after: ST1019 in examples/main.go and two U1000 in older test files.
• 0b8e551 on its own, exported with git archive: vet and race tests pass

The merge result with main 00f5945 (common v0.5.14) also passes go vet, the CI race and coverage command (77.0%), go test -tags no_dataexchange ./... and gofmt.

Each new test below was red-checked: its fix was disabled in a scratch copy and the test failed.

• zz_inbox_budget_test.go:
  • TestInboxByteCap_InflightWriteNotEvicted: a deterministic interleaving through the afterInboxWrite test hook
  • TestInboxByteCap_RescanDoesNotDoubleCountInflight
  • TestEvictInboxOverflow_SkipsInflightWrite
• zz_correlation_test.go:
  • The old-receiver emulator models three receiver generations: genDropUnknown (v0.2.1 and older), genRejectUnknown and genRejectUnknownGoverned.
  • TestSend_FallsBackForOldReceiver checks that each generation stores the message exactly once.
  • TestSend_UntaggedFrameNeverRetried runs once per generation.
  • New: TestReceiverPredatesTagged (a table of ACKs) and TestSend_CurrentReceiverKeepsTagsForEveryType.
  • New: TestSend_TaggedFrameSizeEndToEndSmallCap re-runs itself in a child process with PILOT_DATAEXCHANGE_MAX_FRAME=65536 against a real Service. It checks that the at-cap frame is stored untagged, that the largest taggable frame (cap − 293) is stored tagged, and that a raw WriteFrame fails without writing.
• zz_tagged_test.go:
  • TestMaxTaggedOverhead (= 293)
  • TestWriteFrame_TaggedFrameOverMaxFrameSize: at the cap and one byte over, it returns an error with 0 bytes written. An exact fit writes 8 + MaxFrameSize bytes.
  • TestSend_TaggedFrameTooLargeSentUntagged: a 64 MiB payload to a receiver that checks the length and discards the payload

Deploy notes

1. No server deploys are needed for this repo alone.
2. After merge, tag a new dataexchange release, then bump it in web4.
3. Order matters for web4's planned --wait request-ID matching (sweep batch 3b). It must depend on a dataexchange release that includes e457424. Otherwise, requests sent with a MessageID to any stable peer (web4 v1.12.0 through v1.13.9, dataexchange v0.2.1 or older) are dropped while reported as delivered.
4. web4 code that uses request IDs must send with Client.Send and treat SendResult.Tagged == false as "match the reply on the sender alone". That covers old receivers and frames within 293 bytes of the max frame size. A raw WriteFrame of a tagged frame gets no fallback.

🤖 Generated with Claude Code

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.

📢 Thoughts on this report? Let us know!