Skip to content

Pin the MaxCapacity comment's cost claim as an assertion - #921

Open
jeremy wants to merge 3 commits into
mainfrom
sdk-alloc-claim-pin
Open

jeremy wants to merge 3 commits into
mainfrom
sdk-alloc-claim-pin

Conversation

@jeremy

@jeremy jeremy commented Sep 16, 2026

Copy link
Copy Markdown
Member

Three comments in this fleet were corrected this week for justifying something with a mechanism that does not exist, and all three were caught by a reader, not by a test. This repo's was the MaxCapacity comment (#919, merged in 4523eac): it said both configurable capacities were "pre-allocated the moment the connector is constructed." New allocates neither. The eager sized allocation is make(map[int64]*list.Element, capacity) in newDedupe, reached from newLoop when an Events iteration starts; the live buffer starts nil and grows in add.

The comment has been fixed. What was missing is the pin.

The rule this establishes: a comment that justifies a constraint by asserting a cost at a time is making a measurable claim, and should be written as a test whose name is the claim.

TestNewAllocationSizeDoesNotVaryWithCapacity sits beside the corrected comment in connector_test.go and measures New at capacity 1 and at eventfeed.MaxCapacity.

Bytes, not an allocation count — the mutation check moved this

The brief asked for allocation count, on a measurement of 8 allocations at capacity 1 and 8 at the ceiling. That measurement reproduces exactly (8.00 at capacities 1, 100, 10,000 and 1,000,000; identical under -race). But the count-based test could not fail. Mutating New to allocate by capacity —

c.mutationProbe = make([]byte, cfg.dedupeCapacity)

— is one allocation whether the capacity is 1 or a million. The count stayed 8 at both ends and the test passed, with New allocating a megabyte at construction. The count is identical in precisely the case that has to fail, so it is the wrong instrument for this claim; the claim is about cost, and cost here is bytes.

The test measures runtime.MemStats.TotalAlloc across the call instead. Re-run against both mutations:

New under test capacity 1 capacity 1,000,000 verdict
unmutated 1,568 B 1,568 B pass
make([]byte, capacity) 1,584 B 1,009,184 B fail
make(map[int64]struct{}, capacity) (the newDedupe shape) 1,616 B 37,834,320 B fail
make([]byte, dedupeCapacity) only 1,584 B 1,009,184 B fail
make([]byte, liveBufferCapacity) only 1,584 B 1,009,184 B fail
if liveBuffer > dedupe { make([]byte, liveBuffer) } 1,568 B 1,009,184 B fail
if dedupe > liveBuffer { make([]byte, dedupe) } 1,568 B 1,009,184 B fail

(The last four report the row that isolates the larger capacity.)

The assertion is the invariance, not the size

Four configurations, each compared against (1, 1), never against a literal: both capacities at the ceiling, and each one at the ceiling alone while the other stays at 1. An unrelated allocation added to New moves them all together and stays green; a New that starts sizing anything by a capacity does not.

The two options move independently because moving them together samples only the diagonal (raised by Codex, #discussion_r4031265435): an allocation taken from the capacities' relationship — fired only when the live buffer capacity exceeds the dedupe capacity, say — reads identically at (1, 1) and (max, max) and leaves a diagonal-only assertion green. Both orderings of that mutation were green against the first version of this test and fail against this one.

No absolute bound: it would add nothing here — any per-capacity sizing already shows as a difference between configurations — and would make this a tripwire for every unrelated refactor, which is how a pin becomes noise.

One guard that is not the size: the test fails if New measures zero bytes. That is not a bound on the magnitude, it is a check that the measurement is still observing something — a construction optimized away would otherwise pass this test vacuously forever, which is the failure mode the PR exists to argue against. newSink keeps each connector reachable for the same reason.

Stability

testing.AllocsPerRun and TotalAlloc both read process-wide counters, so a goroutine left running by an earlier test inflates a measurement. This package runs connectors on their own goroutines throughout its suite, and CI runs it under -race (go test -race -v ./...). That noise can only add, never subtract, so the helper takes the lowest of five single measurements and compares the two floors.

Evidence it is stable rather than merely passing once:

  • a naive single measurement, instrumented inside the full package suite under -race, read exactly the same value in 64/64 windows;
  • go test -count=5 ./pkg/basecamp/eventfeed/... — green, before and after the widening;
  • go test -race -count=3 ./pkg/basecamp/eventfeed/... — green, before and after the widening;
  • byte measurement instrumented and repeated under -race: 18/18 windows at 1,568 both ends.

The min-of-five is kept anyway: the noise source is real and named, it costs four lines, and the alternative to a cheap floor here is a flaky test on the merge gate.

go build ./..., go vet ./... and make go-lint (0 issues) all clean.

The comment now names its own proof

One line added to the MaxCapacity comment in connector.go, naming the test. The reason is the direction this test cannot cover: it catches the code drifting from the comment, not the comment being rewritten wrongly against unchanged code — which is what happened in #919. Without a pointer, the person editing that comment has no signal the claim is checked at all, so the change would defend the direction that did not fail and stay silent on the one that did. Nothing short of review catches a wrongly-rewritten comment, and the pointer does not pretend otherwise; it just puts the proof where that person is looking.

Scope

One claim, one test. Deliberately not sweeping the repo for other cost claims. Two obvious siblings, noted but not touched: the second half of the same MaxCapacity comment ("the live buffer grows to its capacity lazily and pays only for the events it admits" — partly covered by TestSustainedOverflowRetainsTheBacking), and DefaultLiveBufferCapacity's "deliberately decoupled from the dedupe capacity — only event-bearing frames are buffered."

The comment on MaxCapacity justifies the ceiling by saying where a
configured capacity is actually spent: newDedupe sizes its index map by
it eagerly, when the first Events iteration starts. An earlier wording
said instead that both capacities were allocated the moment the
connector was constructed. New allocates neither. Nothing in the suite
disagreed with either version, and the wrong one stood until a reader
checked it against the code.

A comment that justifies a constraint by asserting a cost at a time is
making a measurable claim, so measure it: New's allocation size is
compared at the two ends of the permitted capacity range and must be
the same at both.

Bytes rather than an allocation count. The count cannot carry this
claim -- make([]byte, capacity) in New is one allocation at capacity 1
and one at a million, so the count is identical in exactly the case
that has to fail. Both that shape and the sized map newDedupe actually
uses were introduced into New to confirm the assertion fires; at the
ceiling they read 1,009,184 and 37,834,320 bytes against 1,568
unmutated.
Copilot AI balanced review requested due to automatic review settings September 16, 2026 22:06
@github-actions github-actions Bot added the go label Sep 16, 2026
@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Sep 16, 2026

Copy link
Copy Markdown

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review Completed 2026-09-16T22:43:20.895464Z fb5cae3 Manual request
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

Copilot AI previously approved these changes Sep 16, 2026

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟢 Approved

No unresolved blocking issues were identified.

Pull request overview

Adds a regression test ensuring eventfeed.New allocation cost is independent of configured capacity.

Changes:

  • Compares TotalAlloc at minimum and maximum capacities.
  • Uses repeated measurements and a reachability sink for stability.
File summaries
File Description
go/pkg/basecamp/eventfeed/connector_test.go Adds the allocation-invariance regression test.
Review details
  • Files reviewed: 1/1 changed files
  • Comments generated: 0
  • Review effort level: Lite (auto)

Note

Copilot is running an experiment and ran this review at Lite.


💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 0d9440e7a6

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread go/pkg/basecamp/eventfeed/connector_test.go Outdated
Moving both options together samples only (1, 1) and (max, max), so an
allocation taken from the capacities' relationship rather than from
either alone -- one that fires only when the live buffer capacity
exceeds the dedupe capacity, say -- reads the same at both samples and
leaves the assertion green. Hold each option at 1 while the other goes
to the ceiling, in both directions, and compare all three against the
smallest configuration.

Both relationship-shaped mutations now fail, in the row that isolates
the larger capacity; the two single-option mutations fail in two rows
each.
@jeremy
jeremy requested a balanced review from Copilot September 16, 2026 22:17
Copilot AI dismissed their stale review, a newer Copilot review was requested September 16, 2026 22:18
Copilot AI previously approved these changes Sep 16, 2026

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟢 Approved

The focused test covers both capacities independently and consistently reflects the constructor’s allocation contract.

Review details
  • Files reviewed: 1/1 changed files
  • Comments generated: 0 new
  • Review effort level: Balanced

@jeremy

jeremy commented Sep 16, 2026

Copy link
Copy Markdown
Member Author

Review threads: 1 resolved (1 fixed, 0 declined).

  • #discussion_r4031265435 — Codex, P2: vary the two capacities independently. Fixed in 080ab26d. Correct finding: moving both options together samples only the diagonal, so an allocation taken from the capacities' relationship reads the same at (1, 1) and (max, max). Each option now goes to the ceiling alone, in both directions, and all three configurations are compared against (1, 1). Verified by mutation, not by inspection — both relationship-shaped mutations were green against the first version and fail against this one.

Two things worth flagging beyond the threads, because each changed the shape of the PR.

The allocation-count instrument was falsified, not just refined. The original plan was to assert an allocation count, on a measurement of 8 allocations at capacity 1 and 8 at the ceiling. That measurement reproduces exactly. The mutation check is what rejected it — make([]byte, capacity) in New is one allocation whether the capacity is 1 or a million, so the count-based test stayed green with New allocating a megabyte at construction. The count is identical in precisely the case that has to fail. The test measures TotalAlloc bytes instead; the full mutation table is in the description.

The comment now names the test. fb5cae31 adds one line to the MaxCapacity comment pointing at TestNewAllocationSizeDoesNotVaryWithCapacity. The test catches the code drifting from the comment; it cannot catch the comment being rewritten wrongly against unchanged code, which is the direction that actually failed in #919. Without the pointer, the person editing that comment has no signal the claim is checked at all. Nothing short of review catches a wrongly-rewritten comment and the pointer does not pretend otherwise — it puts the proof where that person is looking.

Both reviewers reported on fb5cae31: Copilot approved (0 comments, no suppressed findings), Codex "Didn't find any major issues." CI green, MERGEABLE, CLEAN, no unresolved threads. Not arming auto-merge — this one is yours to land.

The test catches the code drifting from the comment. It cannot catch
the comment being rewritten wrongly against unchanged code, which is
the direction that actually failed in #919 -- and a person editing that
comment has no signal the claim is checked at all. Naming the test is
the only thing in reach that puts the proof in front of them.

Nothing short of review catches a wrongly-rewritten comment; the
pointer is not an attempt to.
@jeremy
jeremy requested a balanced review from Copilot September 16, 2026 22:40
@jeremy

jeremy commented Sep 16, 2026

Copy link
Copy Markdown
Member Author

@codex review

Copilot AI dismissed their stale review, a newer Copilot review was requested September 16, 2026 22:40
@chatgpt-codex-connector

Copy link
Copy Markdown

Codex Review: Didn't find any major issues. Breezy!

Reviewed commit: fb5cae3191

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟢 Approved

The focused changes are fully reviewed with no unresolved issues.

Review details
  • Files reviewed: 2/2 changed files
  • Comments generated: 0 new
  • Review effort level: Lite (auto)

Note

Copilot is running an experiment and ran this review at Lite.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants