Skip to content

feat(client): transport sampler probability heads for score centering - #180

Merged
hallerite merged 4 commits into
mainfrom
feat/score-centering
Oct 19, 2026
Merged

hallerite merged 4 commits into
mainfrom
feat/score-centering

Conversation

@nxphi47

@nxphi47 nxphi47 commented Oct 8, 2026 •

Copy link
Copy Markdown
Contributor

Capture action-independent sampler probability heads for prime-rl #3927 and prime-rl #4012, paired with verifiers #2804.

A positive integer sampling_params["logprobs"] requests a probability head. Without replay, request two extra candidates, exclude cutoff ties, and retain the sampler's original full-vocabulary normalization. With a recorded sampling mask, retain its complete support, including boundary ties, and require probability mass one over that support. Accept native list masks and prime-rl's packed int32 CSR masks; reject missing probabilities or malformed support. Boolean and sampled-only requests keep their existing request path.

Compact numeric transport avoids per-token JSON overhead and accepts native JSON fallback. The response parser detects the payload format directly. Native logprobs accept both numeric token_id fields and token_id:N strings. Validate buffers, probability mass, and sampled-token agreement, including conflicting repeated token IDs.

Validation on 2026-10-19: 103 focused client tests passed, including native/compact and packed/list replay-support cases; seven unrelated model/multimodal cases were deselected. The paired prime-rl stack passed 19 vLLM 0.32 response-format tests, including complete replay support and native/compact equivalence. Renderer Ruff and formatting checks passed. These checks did not launch a GPU inference engine.

Updated onto main without rewriting the existing PR history. The replay-support compatibility follow-up is 5ffade9.

Note

Add compact sampler-head transport for top-logprob score centering

  • Adds binary compact-logprob decoding for generation responses in renderers/client.py, including format, offset, candidate-count, and sampled-token validation
  • generate now requests the head size plus 2 candidates, enables compact transport through extra_args by default, and returns completion_top_ids and completion_top_logprobs
  • Adds a shared head selector in _select_sampler_head that ranks candidates deterministically, is independent of the sampled action, and preserves all candidates covered by a recorded sampling support mask
  • _logprob_token_id accepts numeric token_id fields alongside the existing textual token form; _strip_base64_fields now extracts all matching base64 fields so multi-choice responses keep their buffers
  • Behavioral Change: responses without a positive requested head return None for the new top-logprob fields; callers can override compact transport via extra_args

Macroscope summarized 5ffade9.

@macroscopeapp

macroscopeapp Bot commented Oct 19, 2026 •

Copy link
Copy Markdown

Approvability

Verdict: Needs Human Approval

Macroscope's review found this PR requires human approval — This adds a substantial opt-in sampler-head capability with new request semantics, compact binary transport, replay-mask handling, and additional response fields in the production client. Its cross-component protocol behavior and sizable parsing/selection logic merit human review.

Notes:

  • Macroscope's correctness review did not run, so approvability was decided on eligibility alone.

You can add or adjust custom eligibility rules. Learn more.

@hallerite hallerite left a comment •

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm

@hallerite
hallerite merged commit 70c8204 into main Oct 19, 2026
10 checks passed
@hallerite
hallerite deleted the feat/score-centering branch October 19, 2026 19:30
hallerite added a commit to PrimeIntellect-ai/verifiers that referenced this pull request Oct 19, 2026
)

Carry sampler probability heads from renderer responses onto assistant
nodes and training branches for [prime-rl
#4012](PrimeIntellect-ai/prime-rl#4012), paired
with [renderers
#180](PrimeIntellect-ai/renderers#180).

`TopLogprobs` is a flat IDs/logprobs/counts carrier. It shares the
existing raw-array serializer with sampling masks, checks sampled-token
alignment when attributed, and gives context or uncaptured positions
zero counts when flattening a branch. Public trace records exclude the
arrays. Empty completions preserve an empty carrier.

Validation: 179 unit tests passed; a temporary probe covered multi-turn
alignment, mixed capture, empty completions, malformed rows/counts,
MessagePack round trips, and record exclusion. All contributor hooks and
the configured type gate passed using uv 0.13.1. The full suite was
attempted: four Docker Compose integration cases failed because the
local Docker daemon is unavailable; provider/runtime cases were skipped
where prerequisites were absent.

Updated onto current main without rewriting the existing PR history.

Co-authored-by: hallerite <git@hallerite.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants