[faresobeid]

What

Adds by-handle payload segments to v1 traces. The inference server can write router-replay ids and sampling masks to a shared file and return payload segments instead of the inline routed_experts / sampling_mask. Verifiers then moves only the handles.

• PayloadSegment(field, file, offset, pos, rows, dtype, shape): rows fixed-width rows of one per-token array at byte offset of file. pos is the first row's token position.
• TurnTokens.payload: the request's segments, request-absolute positions. The train client fills it from the generate result's payload.
• Trace.payload: list[NodeSegment]: append-only. Each turn's commit clips the request segments to the nodes created this turn (node-local positions). The one position before them, the prefix's final token, goes to the prefix's last node when that node was sampled. That replaces the placeholder-row repair: no row is ever rewritten. Rows of other reused prefix positions are dropped, the same as the inline path.
• Branch.payload: the trace's segments on that path, rebased to branch positions, in append order.
• payload is a delta-stream list field. New entries ship once, so no routing_repairs are needed. It is excluded from to_record because the handles are short-lived.

The inline fields are unchanged. A turn carries one form or the other, depending on the server.

Companion PRs: PrimeIntellect-ai/renderers#170 (passes payload through) and PrimeIntellect-ai/prime-rl#3902 (writer, orchestrator clipping/packing, trainer reads, retention).

Validation

• tests/unit/test_graph.py::test_payload_segments_attributed_per_node_and_rebased_per_branch: a bridged turn (the prior sampled node gets request row path_len-1), a fork off a non-sampled prefix node (its re-forwarded rows are dropped), and Branch.payload rebasing on both branches.
• pytest tests/unit/test_graph.py tests/unit/test_trace.py, ruff check, ruff format --check.

Landing order

1. feat(client): pass through sampling-mask logprobs and by-handle payload segments renderers#170 and this PR
2. feat: pass router-replay ids and sampling masks by handle on multi-node runs prime-rl#3902 (writer, orchestrator, trainer reads, retention; re-pin deps/renderers / deps/verifiers to the merged commits first)
3. Disaggregated P/D: feat(pd): stitch by-handle routed_experts payload segments router#60

---

Note

Medium Risk
Touches turn commit attribution and branch construction for training data; payload handles are wire-only and omitted from persisted traces, so consumers must read shared files before they expire.

Overview
Adds by-handle per-token side arrays to v1 traces so inference can return file offsets instead of inlining large tensors (e.g. routed_experts, sampling_mask).

Introduces PayloadSegment / NodeSegment with clipping and rebasing. The train client maps generate payload into transient TurnTokens.payload. Turn commit appends node-attributed segments on Trace.payload (including the bridged “previous turn final token” row without rewriting earlier rows, and dropping re-forwarded prefix rows). Branch.payload exposes branch-aligned handles for training.

payload streams as an append-only delta field but is excluded from disk records (EXCLUDE_FIELDS). Inline routed_experts / sampling_mask paths are unchanged.

A new graph unit test covers bridged attribution, prefix reuse, and per-branch rebasing.

^{Reviewed by Cursor Bugbot for commit bacf577. Bugbot is set up for automated code reviews on this repo. Configure here.}

Note

Add by-handle payload segments to v1 traces

• Adds PayloadSegment and NodeSegment models in types.py, with a clip operation that intersects a segment with a half-open range and adjusts row count, byte offset, and position.
• response_from_generate in train.py parses payload mappings into typed segments carried transiently on TurnTokens.
• _attribute_payload in graph.py splits each turn's payload across the matching nodes: it clips segments to node spans, converts to node-local positions, and appends non-empty entries to Trace.payload.
• Branch materialization in trace.py copies only payload entries for nodes on each leaf path and rebases positions to branch-wide token coordinates. Serve deltas in delta.py treat payload as an append-only field.
• Behavioral Change: payload is added to EXCLUDE_FIELDS in trace.py, so to_record serialization omits it; the field stays available in the in-memory and wire models.

^{Macroscope summarized bacf577.}

[macroscopeapp]

Approvability

Verdict: Needs Human Approval

Macroscope's review found this PR requires human approval — This adds by-handle payload support across generation parsing, graph attribution, branch materialization, and live trace deltas, with handles referencing short-lived shared files and omitted from persisted records. Existing inline paths remain unchanged, but the cross-component training integration and new runtime attribution behavior warrant focused human review.

^{You can add or adjust custom eligibility rules. Learn more.}