perf(volo-grpc): batch streaming messages and reduce Status overhead

[dirname]

Motivation

• The streaming encoder handles ready messages one at a time, adding buffer and frame-processing overhead for each message.
• Status stores metadata inline, increasing its size. Downstream code returning Result<T, Status> can fail stricter Clippy checks because of clippy::result_large_err. Successful status checks and initial error-response parsing also copy metadata unnecessarily.
• Identity compression and decompression consume the input without producing output. The encoder also marks Some(Identity) messages as compressed.

Solution

Streaming encoder

Add a private EncodeStream that batches messages while the source is ready. It follows tonic's batching strategy, adapted to LinkedBytes nodes, with the same 32 KiB soft threshold.

The threshold controls batching, not the size of individual messages. Completed messages are flushed when the source returns Pending, ends, or yields an error. Compression is still per-message.

The raw encoder stream continues after source errors but stops after encoding errors. An encoding error flushes completed messages and discards the partially encoded message.

The uncompressed path transfers nodes directly and moves the entire trailing buffer with mem::take. Message prefixes are backfilled using node positions and offsets.

The compression path uses LinkedBytes::into_bytes_mut(), so the minimum linkedbytes version is raised to 0.1.16. In 0.1.15, the method is named into_bytesmut.

Status metadata

Store metadata on demand as Option<Box<MetadataMap>> rather than inline. The public API is unchanged. Clones keep independent metadata. When a MetadataMap is passed to a constructor, its reserved capacity is retained even if the map is empty.

Successful status checks no longer copy metadata. Initial error responses transfer their already-owned headers using the same filtering rules. Status::from_header_map still retains metadata when parsing borrowed headers.

Remove the related result_large_err suppressions.

Identity

Encode Some(Identity) as uncompressed data with the compressed flag set to zero. The Identity branches in compress and decompress now append the input bytes to the output and consume the input.

Give each algorithm fresh input and buffers in the consistency test, so consumed input or output from an earlier iteration cannot mask a failure.

Tests

Area	Repository tests	Checks
Streaming and multi-node prefixes	codec::encode::tests::test_encode_stream	Messages with small, 8 KiB, and 64 KiB string payloads round-trip in order with matching contents, including with each enabled compression algorithm
Flushing on Pending	codec::encode::tests::test_encode_pending	A completed message is immediately readable when the source returns Pending
Errors	codec::encode::tests::test_encode_error	Completed messages are delivered and partial messages are discarded; source errors allow continuation, while encoding errors end the stream
Status metadata and parsing	status::tests::{metadata, from_header_map, infer_grpc_status}	Constructors retain reserved capacity for empty metadata; clones are independent; both parsing paths retain duplicate and binary values; the owned path is also checked for sensitivity flags; invalid UTF-8 status messages return errors
Identity and compression	codec::encode::tests::test_encode, codec::compression::tests::test_consistency_for_compression	Identity writes an uncompressed flag and the correct payload; each codec round-trips data and consumes its input independently

Ready-message streaming measurements

The benchmark compared the upstream and PR versions of the Volo server using tonic and Volo clients over loopback on the same Linux ECS instance: AMD EPYC 7T83, KVM, 8 vCPUs, Rust 1.98.1, release build with LTO.

The server used one worker with CPU affinity [0]; the client used two workers with affinity [2, 4]. These were guest CPU masks, set before process startup and inherited by worker threads.

Each stream provided 16 continuously ready messages with 256 B payloads. Concurrency was 32 or 128, with four connections and no TLS or compression. For each client and concurrency level, each version ran six times, with a 2-second warmup and an 8-second measurement window per run. The before/after run order was balanced.

The tonic-client runs used the same client binary across both server versions, built with tonic 0.14.6, tonic-prost 0.14.6, and prost 0.14.4. In the Volo-client runs, both client and server switched to the matching upstream or PR version.

The table reports the median of the per-run throughputs, in completed streams per second. Each run uses its actual elapsed time, including the time to finish in-flight streams.

Concurrency	tonic client: before	tonic client: after	Volo client: before	Volo client: after
32	14,603	53,185	14,623	49,892
128	15,634	59,585	15,722	56,356

These results are specific to the continuously ready, uncompressed streaming workload and the client/server settings above.

[CLAassistant]

All committers have signed the CLA.