Skip to content

perf: optimize Qoder body encoding for large requests - #23

Merged
simonsmh merged 1 commit into
simonsmh:mainfrom
acniray:perf/optimize-qoder-body-encoding
Sep 6, 2026
Merged

simonsmh merged 1 commit into
simonsmh:mainfrom
acniray:perf/optimize-qoder-body-encoding

Conversation

@acniray

@acniray acniray commented Sep 3, 2026

Copy link
Copy Markdown
Contributor

Summary

Optimize qoderEncodeBody() to reduce CPU and GC overhead when encoding large request bodies.

The previous implementation built an intermediate rearranged string and then appended the encoded output one character at a time with out += ....

With large Pi tool-call payloads, this caused significant V8 string flattening and garbage collection overhead.

Changes

  • Precompute a byte translation table for the custom Qoder Base64 alphabet
  • Preallocate the final output buffer
  • Perform rearrangement directly into the output buffer
  • Avoid the intermediate rearranged string
  • Avoid repeated single-character string concatenation

The encoded wire format remains unchanged.

Observed issue

In real Pi sessions using Qoder providers, large requests could cause the Node main thread to stay at high CPU usage for a long time.

perf showed hotspots around:

  • String::SlowFlatten
  • String::WriteToFlat
  • Runtime_StringCharCodeAt
  • memmove
  • V8 garbage collection

After replacing the string-concatenation path with the buffer-based implementation, the CPU spike no longer reproduced in the same workflow.

Validation

  • Existing encoding tests pass
  • Type check passes
  • Lint/format checks pass
  • Build passes

@simonsmh
simonsmh merged commit 0149936 into simonsmh:main Sep 6, 2026
1 check passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants