Skip to content

THRIFT-6428: Reduce allocations in Ruby memory buffer hex diagnostics - #4016

Merged
kpumuk merged 1 commit into
apache:masterfrom
kpumuk:rb-memory-buffer-inspect
Oct 16, 2026
Merged

kpumuk merged 1 commit into
apache:masterfrom
kpumuk:rb-memory-buffer-inspect

Conversation

@kpumuk

@kpumuk kpumuk commented Oct 16, 2026

Copy link
Copy Markdown
Member

MemoryBufferTransport#inspect_buffer creates two temporary Strings per byte and retains an Array of hexadecimal tokens before joining them. Large diagnostic dumps consequently allocate substantially more memory than the returned text requires.

Reuse frozen byte spellings and append directly to the result String. Preserve the complete lowercase, unpadded dump, spacing, cursor positions, output encoding, and mutability without consuming the transport's contents.

Benchmarks

Compared master 5248fb28acda184580bd490cea8a1bea6933535a with this change using Ruby 4.0.7 on aarch64 Linux. The stdlib harness preserves master's original method for legacy and calls the proposed public method for installed.

Benchmark command:

ruby /thrift/src/rb-mem-014-evidence/run-benchmarks.rb legacy installed

The harness checks 32 B, 4 KiB, and 1 MiB buffers in native and forced-pure modes, with three fresh-process ordinary-GC trials and three separate GC-disabled trials per configuration. Every timed iteration performs identical SHA-256 validation work.

For a 1 MiB buffer cycling through all 256 byte values, with the cursor at zero:

Mode Master time Proposed time Change Master peak RSS Proposed peak RSS Change
Native 343.084 ms 116.535 ms −66.03% 89.766 MiB 31.910 MiB −64.45%
Pure 345.028 ms 117.213 ms −66.03% 88.586 MiB 31.812 MiB −64.09%

These are medians of three fresh processes. In both modes, separate GC-disabled trials reduced median allocations from 2,097,165 objects to 11 (>99.999%) and the measured object footprint from 94.041 MiB to 4.006 MiB (−95.74%). Complete output matched at 3,080,193 bytes with identical SHA-256 digests.

The lookup table retains a fixed 12,328 bytes at load time on this Ruby. The full returned dump still grows with input size. Peak RSS includes setup and runtime; shared-machine scheduling limits timing precision. Other Ruby versions and JRuby were not measured.

  • Did you create an Apache Jira ticket? (Request account here, not required for trivial changes)
  • If a ticket exists: Does your pull request title follow the pattern "THRIFT-NNNN: describe my issue"?
  • Did you squash your changes to a single commit? (not required, but preferred)
  • Did you do your best to avoid breaking changes? If one was needed, did you label the Jira ticket with "Breaking-Change"?
  • If your change does not involve any code, include [skip ci] anywhere in the commit message to free up build resources.

Client: rb

Co-Authored-By: OpenAI Codex (gpt-6.1-sol) <codex@openai.com>
@mergeable mergeable Bot added the ruby Pull requests that update Ruby code label Oct 16, 2026
@kpumuk
kpumuk merged commit 9f63b19 into apache:master Oct 16, 2026
109 checks passed
@kpumuk
kpumuk deleted the rb-memory-buffer-inspect branch October 16, 2026 18:58
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

ruby Pull requests that update Ruby code

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant