Skip to content

feat(v1): optional sampler logprobs on SamplingMask - #2756

Open
faresobeid wants to merge 9 commits into
mainfrom
feat/top-logprobs-v2
Open

faresobeid wants to merge 9 commits into
mainfrom
feat/top-logprobs-v2

Conversation

@faresobeid

@faresobeid faresobeid commented Oct 3, 2026 •

Copy link
Copy Markdown
Collaborator

What

Carries the sampler's logprobs at the sampling-mask ids onto the trace graph.

  • SamplingMask gets an optional float32 logprobs array, parallel to ids: the sampler's renormalized logprobs of the kept ids.
  • SamplingMask.from_sampling_mask(mask, logprobs=None) decodes logprobs from prime-rl's /generate sampling_mask_logprobs: packed float32, base64 {data, shape, dtype}, the same form as the packed ids, decoded with _decode_packed. response_from_generate passes result.get("sampling_mask_logprobs"). Stock vLLM does not send it, so the per-row mask path has no logprobs.
  • Attribution: a mask whose logprobs length differs from ids is not attributed (same rule as a mask whose counts do not match the node's sampled positions).
  • Aggregation: Branch.sampling_mask concatenates the logprobs only when every mask on the path has them; otherwise logprobs is None and ids/counts still merge. Nodes without a mask (prompt, replayed prefix) do not count.
  • The __nd__ serializer/validator handle every field present on the dataclass. Without logprobs, the serialized form and payload are unchanged.

Payload when present: 4 B per kept id.

Consumer

prime-rl score centering, PrimeIntellect-ai/prime-rl#3794, pins this head. It sends sampling_mask_logprobs packed. Without that consumer this field is unused.

Companion

PrimeIntellect-ai/renderers#170 passes sampling_mask_logprobs (and payload) through. Order: renderers#170 + this PR -> prime-rl#3794.

Validation

  • tests/unit/test_graph.py::test_sampling_mask_logprobs_attributed_aggregated_and_round_tripped: branch concat, wire round trip, mixed turns -> None, mismatched length not attributed.
  • tests/unit/test_trace.py::test_sampling_mask_from_packed_matches_lists: packed logprobs decode.
  • pytest tests/unit/test_graph.py tests/unit/test_trace.py, ruff check, ruff format --check.

Note

Add optional sampler logprobs to SamplingMask

  • Adds an optional float32 logprobs array to SamplingMask, parallel to the flattened mask IDs, and SamplingMask.from_sampling_mask now decodes optional packed logprobs (types.py)
  • response_from_generate passes generate-result logprobs into the mask when constructing TurnTokens (train.py)
  • Branch.sampling_mask concatenates per-node logprobs in ID order; returns None logprobs if any non-empty mask on the branch lacks them (trace.py)
  • Attribution now rejects masks whose logprobs length differs from their IDs length, and serialization omits None-valued array fields (graph.py)
  • Risk: MessageNode.serialize_sampling_arrays in graph.py changes wire format for masks without logprobs — those fields are now absent rather than encoded

Macroscope summarized 8a90f95.


Note

Low Risk
Additive optional field with stricter attribution validation; behavior unchanged when logprobs are absent (e.g. stock vLLM).

Overview
Adds optional sampler logprobs on SamplingMask: a float32 array parallel to flattened mask ids, populated from prime-rl’s packed sampling_mask_logprobs when building turn tokens from /generate.

Trace graph behavior: masks with logprobs length mismatched to ids are no longer attributed to assistant nodes. Branch-level sampling_mask concatenates logprobs across turns only when every attributed mask on the path includes them; otherwise logprobs stays None while ids/counts still merge. Wire serialization now omits unset ndarray fields on SamplingMask (and top_logprobs), so payloads without logprobs stay unchanged.

Unit tests cover packed decode, multi-turn aggregation, mixed turns, round-trip, and rejection of mismatched lengths.

Reviewed by Cursor Bugbot for commit 8a90f95. Bugbot is set up for automated code reviews on this repo. Configure here.

@faresobeid faresobeid changed the title feat(v1): carry sampler top-k logprobs onto the trace graph feat(v1): optional sampler logprobs on SamplingMask Oct 3, 2026
@faresobeid
faresobeid marked this pull request as ready for review October 3, 2026 11:52
@macroscopeapp

macroscopeapp Bot commented Oct 3, 2026 •

Copy link
Copy Markdown
Contributor

Approvability

Verdict: Would Approve

Macroscope's review found this PR approvable — This is a localized, backward-compatible addition of optional sampler logprobs, with existing paths preserved and aggregation/serialization covered by tests. An unresolved high-severity comment still flags the wire round-trip area for masks without logprobs, despite the current diff adding absent-field handling and round-trip coverage.

Needs human approval because:

  • 1 blocking correctness issue found at or above your repo's Minimum Blocking Severity

Adjust the Minimum Blocking Severity for this repo — including turning it Off — in Settings. You can add or adjust custom eligibility rules. Learn more.

macroscopeapp[bot]
macroscopeapp Bot previously approved these changes Oct 3, 2026
@faresobeid
faresobeid force-pushed the feat/top-logprobs-v2 branch 3 times, most recently from 1a0edde to f46cf44 Compare October 6, 2026 17:18
@faresobeid
faresobeid changed the base branch from main to perf/packed-sampling-mask October 6, 2026 17:18
macroscopeapp[bot]
macroscopeapp Bot previously approved these changes Oct 6, 2026
@faresobeid
faresobeid force-pushed the perf/packed-sampling-mask branch from 8069b2b to cef65c3 Compare October 8, 2026 14:35
@faresobeid
faresobeid force-pushed the feat/top-logprobs-v2 branch from f46cf44 to 1e499fb Compare October 8, 2026 14:35
@macroscopeapp
macroscopeapp Bot dismissed their stale review October 8, 2026 14:36

Dismissing prior approval to re-evaluate 1e499fb

macroscopeapp[bot]
macroscopeapp Bot previously approved these changes Oct 8, 2026
@faresobeid
faresobeid force-pushed the perf/packed-sampling-mask branch from cef65c3 to dc361d8 Compare October 10, 2026 15:21
@faresobeid
faresobeid force-pushed the feat/top-logprobs-v2 branch from 1e499fb to 71ad510 Compare October 10, 2026 15:21
macroscopeapp[bot]
macroscopeapp Bot previously approved these changes Oct 12, 2026
@faresobeid
faresobeid force-pushed the perf/packed-sampling-mask branch from ec0bbed to 8f1a6ce Compare October 14, 2026 14:26
# Conflicts:
#	verifiers/v1/types.py
faresobeid added a commit that referenced this pull request Oct 14, 2026
…2778)

`SamplingMask.from_sampling_mask` now also accepts prime-rl's packed CSR
form.

prime-rl's `/inference/v1/generate` server now returns sampling masks as
`{"ids": {data, shape, dtype}, "counts": {data, shape, dtype}}` (base64
int32 arrays) instead of `list[list[int]]`. The one-list-per-token form
was a large part of the vLLM API server's GC load and JSON size under RL
traffic (see the prime-rl PR). Here it also skips building one Python
list per token when the client parses the response.

- dict input: decode `ids` and `counts` straight into the existing int32
arrays (`_decode_packed`, which #2756 reuses for its packed logprobs).
- list input (stock vLLM): unchanged.

Companions: renderers
PrimeIntellect-ai/renderers#174 / prime-rl
PrimeIntellect-ai/prime-rl#3900. Land renderers
-> verifiers -> prime-rl; the prime-rl submodule pins must move to the
merged commits.

## Validation
- `tests/v1/test_trace.py tests/v1/test_graph.py`: pass (one new test:
packed and list inputs give the same arrays).
- `ruff check` / `ruff format` (0.16.0) on the changed files: clean.

<!-- CURSOR_SUMMARY -->
---

> [!NOTE]
> **Low Risk**
> Additive parsing path with existing list behavior preserved; main
caveat is packed `shape` is not validated during decode.
> 
> **Overview**
> Extends **`SamplingMask.from_sampling_mask`** so verifiers can ingest
prime-rl’s packed CSR sampling masks from `/generate`, not only vLLM’s
per-token `list[list[int]]` form. When the input is a dict with
**`ids`** and **`counts`**, each value is decoded via a new
**`_decode_packed`** helper (base64 `data` + `dtype` → `numpy` int32
arrays) instead of concatenating Python lists per token.
> 
> List-based input is unchanged. A unit test asserts packed and list
representations yield the same **`ids`** and **`counts`** arrays.
> 
> <sup>Reviewed by [Cursor Bugbot](https://cursor.com/bugbot) for commit
8f1a6ce. Bugbot is set up for automated
code reviews on this repo. Configure
[here](https://www.cursor.com/dashboard/bugbot).</sup>
<!-- /CURSOR_SUMMARY -->

<!-- Macroscope's pull request summary starts here -->
<!-- Macroscope will only edit the content between these invisible
markers, and the markers themselves will not be visible in the GitHub
rendered markdown. -->
<!-- If you delete either of the start / end markers from your PR's
description, Macroscope will append its summary at the bottom of the
description. -->
> [!NOTE]
> ### Accept packed CSR sampling masks in
`SamplingMask.from_sampling_mask`
> - `SamplingMask.from_sampling_mask` in
[types.py](https://github.com/PrimeIntellect-ai/verifiers/pull/2778/files#diff-88a4e4ee0c58ac5dc7344161a011374082a3767eda2ad32af97c942dca982112)
now accepts either the existing list-of-lists input or a dict with
packed `ids` and `counts` arrays, matching prime-rl's `/generate` format
> - Adds a `_decode_packed` helper that base64-decodes the packed
`data`, interprets it with the packed dtype, and casts to the requested
dtype
> - Adds a test in
[test_trace.py](https://github.com/PrimeIntellect-ai/verifiers/pull/2778/files#diff-ad6d1dda2b40e73baa4de5ad46c7a5a5580402aef93562589170bfc1bf9d1a99)
verifying that list and packed inputs produce identical `ids` and
`counts` arrays
> - Risk: the packed `shape` metadata in `_decode_packed` is ignored;
the arrays are reconstructed only from `counts` and `ids`
>
> <!-- Macroscope's review summary starts here -->
>
> <sup><a href="https://app.macroscope.com">Macroscope</a> summarized
8f1a6ce.</sup>
> <!-- Macroscope's review summary ends here -->
>
<!-- Macroscope's pull request summary ends here -->
Base automatically changed from perf/packed-sampling-mask to main October 14, 2026 15:18
@faresobeid
faresobeid dismissed macroscopeapp[bot]’s stale review October 14, 2026 15:18

The base branch was changed.

macroscopeapp[bot]
macroscopeapp Bot previously approved these changes Oct 14, 2026

@cursor cursor Bot left a comment •

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Stale Bugbot comment from a previous run.

Comment thread verifiers/v1/types.py
Comment thread verifiers/v1/types.py

ids: Any
counts: Any
logprobs: Any = None

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟠 High v1/types.py:223

Masks created without sampler logprobs now fail to round-trip across the env-server wire. The serializer iterates every dataclass field, so it encodes logprobs=None as an object-dtype array, which _decode_ndarray cannot read; omit None fields in serialize_sampling_arrays.

🚀 Reply "fix it for me" or copy this AI Prompt for your agent:
In file @verifiers/v1/types.py around line 223:

Masks created without sampler logprobs now fail to round-trip across the env-server wire. The serializer iterates every dataclass field, so it encodes `logprobs=None` as an object-dtype array, which `_decode_ndarray` cannot read; omit `None` fields in `serialize_sampling_arrays`.

# Conflicts:
#	tests/unit/test_trace.py
#	verifiers/v1/types.py

@cursor cursor Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cursor Bugbot has reviewed your changes and found 1 potential issue.

Fix All in Cursor

❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.

Reviewed by Cursor Bugbot for commit 4733b58. Configure here.

Comment thread verifiers/v1/graph.py

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant