Skip to content

Docs: correct statements in the 1.0.4 benchmark page that the figures contradict - #59

Merged
bompus merged 3 commits into
mainfrom
bench/doc-corrections
Nov 1, 2026
Merged

bompus merged 3 commits into
mainfrom
bench/doc-corrections

Conversation

@bompus

@bompus bompus commented Nov 1, 2026

Copy link
Copy Markdown
Owner

The benchmark page and README merged in #58 say five rounds for every memory figure, though s3-lite-client's round 4 ran out of attempts and 42 of its cells have four. They also say the four added clients have no speed, memory, import or size measurements, though the same page now reports them. A cross-vendor read (GPT-6.1 Sol) of the merged head found these and a few smaller ones; each was checked against the saved results before changing it.

  • Rounds. The caption's round count now covers every displayed arm ("4 to 5"), and the text names the exception.
  • Multipliers. The sentence that every multiplier clears twice its band now says a multiplier followed by "no clear change" is a ratio of medians only. The memory report defines ncc the same way.
  • Four clients. The paragraph saying they have no measurements now points to the Speed, Memory and Size sections and says only the other-service timings leave them out.
  • Earlier statements. The old "imports in about 1.0 ms" clause is removed (that figure came from the previous import-only timer), and the CPU warm-up explanation is labelled as earlier experiments, not repeated on Node 26.13.0.
  • Reproduction text. A unit that used its four attempts is a recorded gap with exit 9, not exit 7; other-service timings point at the release 1.0.4 protocol.
  • README. The Bun lower endpoint is 0.65 µs (it was rounded twice); the chart's alt text says nine other libraries.

No measurement, summary or chart value changed; bun run check passes.

…(measured clients, rounds, multipliers, earlier experiments, exit codes, README endpoints)
@bompus bompus added the author:anthropic Pull request written by a anthropic model label Nov 1, 2026

@bompus-ai-review bompus-ai-review Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Bompus AI Review (GPT-6.1 Terra, max reasoning): 1 finding(s).

Comment thread docs/benchmarks.md Resolved

@bompus-ai-review bompus-ai-review Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Bompus AI Review (GPT-6.1 Terra, max reasoning): 3 finding(s).

Comment thread docs/benchmarks.md Resolved
Comment thread bench/make-memory-assets.mjs Resolved
Comment thread bench/analyze-memory.mjs Resolved

@bompus-ai-review bompus-ai-review Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Bompus AI Review (GPT-6.1 Terra, max reasoning): 0 finding(s).

Review 3: only high findings and above are posted from here on.

@bompus
bompus merged commit de70772 into main Nov 1, 2026
7 checks passed
@bompus
bompus deleted the bench/doc-corrections branch November 1, 2026 07:48
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

author:anthropic Pull request written by a anthropic model

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant