Repository navigation
feat(v1): optional sampler logprobs on SamplingMask - #2756
faresobeid wants to merge 9 commits into
Conversation
ApprovabilityVerdict: Would Approve Macroscope's review found this PR approvable — This is a localized, backward-compatible addition of optional sampler logprobs, with existing paths preserved and aggregation/serialization covered by tests. An unresolved high-severity comment still flags the wire round-trip area for masks without logprobs, despite the current diff adding absent-field handling and round-trip coverage. Needs human approval because:
Adjust the Minimum Blocking Severity for this repo — including turning it Off — in Settings. You can add or adjust custom eligibility rules. Learn more. |
1a0edde to
f46cf44
Compare
8069b2b to
cef65c3
Compare
f46cf44 to
1e499fb
Compare
Dismissing prior approval to re-evaluate 1e499fb
cef65c3 to
dc361d8
Compare
1e499fb to
71ad510
Compare
# Conflicts: # verifiers/v1/types.py
Dismissing prior approval to re-evaluate cf6ebfe
ec0bbed to
8f1a6ce
Compare
# Conflicts: # verifiers/v1/types.py
…2778) `SamplingMask.from_sampling_mask` now also accepts prime-rl's packed CSR form. prime-rl's `/inference/v1/generate` server now returns sampling masks as `{"ids": {data, shape, dtype}, "counts": {data, shape, dtype}}` (base64 int32 arrays) instead of `list[list[int]]`. The one-list-per-token form was a large part of the vLLM API server's GC load and JSON size under RL traffic (see the prime-rl PR). Here it also skips building one Python list per token when the client parses the response. - dict input: decode `ids` and `counts` straight into the existing int32 arrays (`_decode_packed`, which #2756 reuses for its packed logprobs). - list input (stock vLLM): unchanged. Companions: renderers PrimeIntellect-ai/renderers#174 / prime-rl PrimeIntellect-ai/prime-rl#3900. Land renderers -> verifiers -> prime-rl; the prime-rl submodule pins must move to the merged commits. ## Validation - `tests/v1/test_trace.py tests/v1/test_graph.py`: pass (one new test: packed and list inputs give the same arrays). - `ruff check` / `ruff format` (0.16.0) on the changed files: clean. <!-- CURSOR_SUMMARY --> --- > [!NOTE] > **Low Risk** > Additive parsing path with existing list behavior preserved; main caveat is packed `shape` is not validated during decode. > > **Overview** > Extends **`SamplingMask.from_sampling_mask`** so verifiers can ingest prime-rl’s packed CSR sampling masks from `/generate`, not only vLLM’s per-token `list[list[int]]` form. When the input is a dict with **`ids`** and **`counts`**, each value is decoded via a new **`_decode_packed`** helper (base64 `data` + `dtype` → `numpy` int32 arrays) instead of concatenating Python lists per token. > > List-based input is unchanged. A unit test asserts packed and list representations yield the same **`ids`** and **`counts`** arrays. > > <sup>Reviewed by [Cursor Bugbot](https://cursor.com/bugbot) for commit 8f1a6ce. Bugbot is set up for automated code reviews on this repo. Configure [here](https://www.cursor.com/dashboard/bugbot).</sup> <!-- /CURSOR_SUMMARY --> <!-- Macroscope's pull request summary starts here --> <!-- Macroscope will only edit the content between these invisible markers, and the markers themselves will not be visible in the GitHub rendered markdown. --> <!-- If you delete either of the start / end markers from your PR's description, Macroscope will append its summary at the bottom of the description. --> > [!NOTE] > ### Accept packed CSR sampling masks in `SamplingMask.from_sampling_mask` > - `SamplingMask.from_sampling_mask` in [types.py](https://github.com/PrimeIntellect-ai/verifiers/pull/2778/files#diff-88a4e4ee0c58ac5dc7344161a011374082a3767eda2ad32af97c942dca982112) now accepts either the existing list-of-lists input or a dict with packed `ids` and `counts` arrays, matching prime-rl's `/generate` format > - Adds a `_decode_packed` helper that base64-decodes the packed `data`, interprets it with the packed dtype, and casts to the requested dtype > - Adds a test in [test_trace.py](https://github.com/PrimeIntellect-ai/verifiers/pull/2778/files#diff-ad6d1dda2b40e73baa4de5ad46c7a5a5580402aef93562589170bfc1bf9d1a99) verifying that list and packed inputs produce identical `ids` and `counts` arrays > - Risk: the packed `shape` metadata in `_decode_packed` is ignored; the arrays are reconstructed only from `counts` and `ids` > > <!-- Macroscope's review summary starts here --> > > <sup><a href="https://app.macroscope.com">Macroscope</a> summarized 8f1a6ce.</sup> > <!-- Macroscope's review summary ends here --> > <!-- Macroscope's pull request summary ends here -->
The base branch was changed.
|
|
||
| ids: Any | ||
| counts: Any | ||
| logprobs: Any = None |
There was a problem hiding this comment.
🟠 High v1/types.py:223
Masks created without sampler logprobs now fail to round-trip across the env-server wire. The serializer iterates every dataclass field, so it encodes logprobs=None as an object-dtype array, which _decode_ndarray cannot read; omit None fields in serialize_sampling_arrays.
🚀 Reply "fix it for me" or copy this AI Prompt for your agent:
In file @verifiers/v1/types.py around line 223:
Masks created without sampler logprobs now fail to round-trip across the env-server wire. The serializer iterates every dataclass field, so it encodes `logprobs=None` as an object-dtype array, which `_decode_ndarray` cannot read; omit `None` fields in `serialize_sampling_arrays`.
# Conflicts: # tests/unit/test_trace.py # verifiers/v1/types.py
There was a problem hiding this comment.
Cursor Bugbot has reviewed your changes and found 1 potential issue.
❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.
Reviewed by Cursor Bugbot for commit 4733b58. Configure here.

What
Carries the sampler's logprobs at the sampling-mask ids onto the trace graph.
SamplingMaskgets an optional float32logprobsarray, parallel toids: the sampler's renormalized logprobs of the kept ids.SamplingMask.from_sampling_mask(mask, logprobs=None)decodeslogprobsfrom prime-rl's/generatesampling_mask_logprobs: packed float32, base64{data, shape, dtype}, the same form as the packedids, decoded with_decode_packed.response_from_generatepassesresult.get("sampling_mask_logprobs"). Stock vLLM does not send it, so the per-row mask path has no logprobs.logprobslength differs fromidsis not attributed (same rule as a mask whose counts do not match the node's sampled positions).Branch.sampling_maskconcatenates the logprobs only when every mask on the path has them; otherwiselogprobsisNoneand ids/counts still merge. Nodes without a mask (prompt, replayed prefix) do not count.__nd__serializer/validator handle every field present on the dataclass. Without logprobs, the serialized form and payload are unchanged.Payload when present: 4 B per kept id.
Consumer
prime-rl score centering, PrimeIntellect-ai/prime-rl#3794, pins this head. It sends
sampling_mask_logprobspacked. Without that consumer this field is unused.Companion
PrimeIntellect-ai/renderers#170 passes
sampling_mask_logprobs(andpayload) through. Order: renderers#170 + this PR -> prime-rl#3794.Validation
tests/unit/test_graph.py::test_sampling_mask_logprobs_attributed_aggregated_and_round_tripped: branch concat, wire round trip, mixed turns ->None, mismatched length not attributed.tests/unit/test_trace.py::test_sampling_mask_from_packed_matches_lists: packed logprobs decode.pytest tests/unit/test_graph.py tests/unit/test_trace.py,ruff check,ruff format --check.Note
Add optional sampler logprobs to
SamplingMasklogprobsarray toSamplingMask, parallel to the flattened mask IDs, andSamplingMask.from_sampling_masknow decodes optional packed logprobs (types.py)response_from_generatepasses generate-result logprobs into the mask when constructingTurnTokens(train.py)Branch.sampling_maskconcatenates per-node logprobs in ID order; returnsNonelogprobs if any non-empty mask on the branch lacks them (trace.py)None-valued array fields (graph.py)MessageNode.serialize_sampling_arraysingraph.pychanges wire format for masks without logprobs — those fields are now absent rather than encodedMacroscope summarized 8a90f95.
Note
Low Risk
Additive optional field with stricter attribution validation; behavior unchanged when logprobs are absent (e.g. stock vLLM).
Overview
Adds optional sampler logprobs on
SamplingMask: a float32 array parallel to flattened maskids, populated from prime-rl’s packedsampling_mask_logprobswhen building turn tokens from/generate.Trace graph behavior: masks with
logprobslength mismatched toidsare no longer attributed to assistant nodes. Branch-levelsampling_maskconcatenates logprobs across turns only when every attributed mask on the path includes them; otherwiselogprobsstaysNonewhile ids/counts still merge. Wire serialization now omits unset ndarray fields onSamplingMask(andtop_logprobs), so payloads without logprobs stay unchanged.Unit tests cover packed decode, multi-turn aggregation, mixed turns, round-trip, and rejection of mismatched lengths.
Reviewed by Cursor Bugbot for commit 8a90f95. Bugbot is set up for automated code reviews on this repo. Configure here.