THRIFT-6431: Amortize Ruby MemoryBuffer compaction and release exhausted storage

[kpumuk]

MemoryBufferTransport replaces its unread backlog after every 4 KiB consumed, making small rolling reads unnecessarily expensive. Exhausting a small shared suffix can also keep a much larger String allocation attached to an empty transport.

The native and pure Ruby implementations now compact only after consuming at least 4 KiB and at least as many bytes as remain, copy the unread suffix into independent storage, and replace exhausted storage. Pure Ruby caches the next compaction boundary to keep ordinary byte reads inexpensive. Returned data, caller-owned Strings and partial-read error behavior are preserved. Exhausted buffers now produce empty diagnostic dumps, reflecting the released storage.

Benchmarks

Compared master 7941f5426deb5eca8d8be0cdb0aed4312914d71a with this change on CRuby 4.0.7, Linux aarch64, using the pure Ruby implementation. Process order alternated between variants.

Memory measurements are medians of three fresh processes per variant and mode. The rolling workload maintains a 2 MiB backlog across 32 reads/writes of 4 KiB. The exhaustion workload consumes 16 MiB followed by its final 1 KiB and keeps only the transport alive.

Memory metric	Master	Proposed	Change
Rolling String footprint growth, GC disabled	128.259 MiB	4.256 MiB	-96.68%
Rolling peak process RSS, ordinary GC	55.133 MiB	29.305 MiB	-46.85%
Rolling objects allocated, ordinary GC	207	114	-44.93%
Rolling GC cycles, ordinary GC	4	0	-4 cycles
String memory retained after exhaustion and GC	16,779,001 bytes	720 bytes	>99.99% lower

String footprint uses ObjectSpace.count_objects_size[:T_STRING]; it is not total allocated memory. The rolling delta excludes initial input setup and includes output verification. Post-GC retention includes small probe overhead. RSS is the whole process's Linux VmHWM, not a delta; releasing String references does not guarantee that the allocator returns memory to the OS immediately.

Elapsed-time measurements use three fresh processes per variant, each with five ordinary-GC samples. The table shows medians of process medians:

Workload	Master	Proposed	Change
2 MiB rolling backlog, 32 × 4 KiB read/write	30.958 ms	1.726 ms	-94.42%
12 × 64 KiB byte drain	103.062 ms	102.592 ms	-0.46%
200,000 Compact i32 reads	87.576 ms	84.516 ms	-3.49%

These are bounded synthetic workloads on one Ruby version and architecture. Small timing differences should not be treated as general throughput guarantees. No benchmark overlapped this change's builds or tests; unrelated host contention was not controlled.

Benchmark command and source

Save the following as benchmark_memory_buffer.rb. From lib/rb, extract the master implementation and run each command three times, alternating their order:

git show 7941f5426deb5eca8d8be0cdb0aed4312914d71a:lib/rb/lib/thrift/transport/memory_buffer_transport.rb > /tmp/master_memory_buffer.rb
MASTER_SOURCE=/tmp/master_memory_buffer.rb bundle exec ruby -Ilib benchmark_memory_buffer.rb
bundle exec ruby -Ilib benchmark_memory_buffer.rb

$LOADED_FEATURES << "thrift_native.rb"
require "thrift"
require "benchmark"
require "json"
load ENV.fetch("MASTER_SOURCE") if ENV["MASTER_SOURCE"]
workloads = {
  rolling: -> {
    buffer = Thrift::MemoryBufferTransport.new("a" * (2 * 1024 * 1024))
    chunk = "b" * 4096
    32.times { buffer.write(chunk); buffer.read_all(4096) }
    raise unless buffer.available == 2 * 1024 * 1024
  },
  byte_reads: -> {
    12.times do
      buffer = Thrift::MemoryBufferTransport.new("a" * 65_536)
      sum = 0
      65_536.times { sum += buffer.read_byte }
      raise unless sum == 97 * 65_536 && buffer.available == 0
    end
  },
  compact_i32: -> {
    buffer = Thrift::MemoryBufferTransport.new("\x54".b * 200_000)
    protocol = Thrift::CompactProtocol.new(buffer)
    sum = 0
    200_000.times { sum += protocol.read_i32 }
    raise unless sum == 42 * 200_000 && buffer.available == 0
  }
}
results = workloads.transform_values do |work|
  samples = 5.times.map { GC.start; Benchmark.realtime { work.call } * 1000 }
  { samples_ms: samples, median_ms: samples.sort[2] }
end
puts JSON.generate(results)

Memory benchmark command and source

Save the following as benchmark_memory_buffer_memory.rb. From lib/rb, use the master source extracted above and run these commands three times, reversing variant order on the second trial:

for mode in runtime alloc retained; do
  MASTER_SOURCE=/tmp/master_memory_buffer.rb bundle exec ruby -Ilib benchmark_memory_buffer_memory.rb "$mode"
  bundle exec ruby -Ilib benchmark_memory_buffer_memory.rb "$mode"
done

runtime measures with ordinary GC; alloc disables GC during the rolling loop; retained measures exhaustion after GC.

$LOADED_FEATURES << "thrift_native.rb"
require "thrift"
require "objspace"
require "json"
load ENV.fetch("MASTER_SOURCE") if ENV["MASTER_SOURCE"]
mode = ARGV.fetch(0)
GC.start
if mode == "retained"
  before = ObjectSpace.count_objects_size[:T_STRING]
  buffer = Thread.new do
    transport = Thrift::MemoryBufferTransport.new("x" * (16 * 1024 * 1024 + 1024))
    transport.read(16 * 1024 * 1024)
    raise unless transport.read_all(1024) == "x" * 1024
    transport
  end.value
  GC.start
  puts JSON.generate(retained_string_bytes: ObjectSpace.count_objects_size[:T_STRING] - before,
    available: buffer.available)
else
  buffer = Thrift::MemoryBufferTransport.new("a" * (2 * 1024 * 1024))
  chunk = "b" * 4096
  GC.start
  before = ObjectSpace.count_objects_size[:T_STRING]
  stats = GC.stat
  GC.disable if mode == "alloc"
  start = Process.clock_gettime(Process::CLOCK_MONOTONIC)
  32.times do
    buffer.write(chunk)
    raise unless buffer.read_all(4096) == "a" * 4096
  end
  elapsed = Process.clock_gettime(Process::CLOCK_MONOTONIC) - start
  after_stats = GC.stat
  after = ObjectSpace.count_objects_size[:T_STRING]
  GC.enable
  raise unless buffer.available == 2 * 1024 * 1024
  puts JSON.generate(elapsed_ms: elapsed * 1000, string_mib: (after - before) / 1048576.0,
    peak_rss_mib: File.read("/proc/self/status")[/^VmHWM:\s+(\d+)/, 1].to_i / 1024.0,
    allocations: after_stats[:total_allocated_objects] - stats[:total_allocated_objects],
    gc_cycles: after_stats[:count] - stats[:count])
end

• Did you create an Apache Jira ticket? (Request account here, not required for trivial changes)
• If a ticket exists: Does your pull request title follow the pattern "THRIFT-NNNN: describe my issue"?
• Did you squash your changes to a single commit? (not required, but preferred)
• Did you do your best to avoid breaking changes? If one was needed, did you label the Jira ticket with "Breaking-Change"?
• If your change does not involve any code, include [skip ci] anywhere in the commit message to free up build resources.

[Copilot]

🟡 Changes recommended

The rolling-backlog tests need assertions that detect regression to eager compaction.

1 open issue

• Test deferred compaction and the equality boundary · New

What changed in this PR

Improves Ruby MemoryBufferTransport memory usage by amortizing compaction and releasing exhausted storage.

Changes:

• Defers compaction until consumed bytes reach both 4 KiB and the unread backlog size.
• Copies unread suffixes into independent storage in native and pure Ruby implementations.
• Adds regression tests and updates exhausted-buffer diagnostic expectations.

File	Description
lib/​rb/​spec/​base_transport_spec.rb	Tests exhaustion, rolling reads, and data preservation.
lib/​rb/​lib/​thrift/​transport/​memory_buffer_transport.rb	Adds deferred compaction with a cached boundary.
lib/​rb/​ext/​memory_buffer.c	Implements matching native compaction and storage release.

_{Review effort: Balanced}

[Copilot]

🟢 Approval recommended

Static review found no blocking issues in either implementation or the regression coverage; tests were not executed.

No open issues 🎉

1 resolved since last review

• Test deferred compaction and the equality boundary

_{Review effort: Balanced}