Repository navigation
chore: merge upstream PrimeIntellect-ai/prime-rl main - #2
Merged
Merged
Conversation
…3797) Make `trainer.model.freeze_moe_router` default to on for RL. ## What changes - `model.freeze_moe_router` is now `bool | "auto"`, default `"auto"`. It resolves to `True` on the RL `TrainerConfig` and to `False` on `SFTConfig`, following the per-trainer pattern of `moe_router_dtype`. An explicit `true`/`false` still wins. - `freeze_moe_router()` no longer raises on non-MoE models; it does nothing, so dense RL runs keep working with the new default. - `docs/training.md` knobs table lists the field. ## Why In RL, router updates amplify the train/inference routing mismatch: the trainer's expert choices drift away from the ones inference sampled. So RL keeps the pretrained router by default. SFT is unchanged. What gets frozen: the router gate parameters (`mlp.router.gate.weight`, plus `gate.bias` on gpt-oss). The aux-loss-free `selection_bias` buffer is not a parameter, and nothing on main updates it. Shared-expert gates (Qwen3.5) and Nemotron-H latent projections are not touched. Frozen routers get no optimizer state. ## Breaking changes - RL runs on MoE models no longer train the router. All MoE RL examples are affected (glm-5.3, glm-4.5-air, intellect-3.1, qwen3-30b-a3b, minimax-m2.5, nemotron-3-super). Set `trainer.model.freeze_moe_router = false` for the old behavior. - An explicit `freeze_moe_router = true` on a non-MoE model is now a no-op instead of an error. - Resuming: a checkpoint saved before this change (trainable router) loads into a frozen-router run; the extra router optimizer state is ignored. The reverse, resuming a frozen-router checkpoint with `freeze_moe_router = false`, fails on missing DCP keys (noted in the field docstring). - With LoRA, a router listed in `lora.modules_to_save` is frozen again by the new default unless `freeze_moe_router = false` is set. ## GPU check One node (8×H200), FSDP over 8 ranks, `samsja/mini-glm-moe` (23 MoE layers), SFT with fake data, `freeze_moe_router = true`, `moe_router_dtype = "float32"` (so each router is its own fp32 FSDP unit whose params are all frozen), 5 steps with a checkpoint every step: - Log: "Froze 23 MoE router parameters", "Running 23 MoE router gates in fp32"; all 5 steps ran with finite loss and grad norm. - DCP checkpoints step 1 vs step 5: all 23 fp32 router gate weights are bitwise unchanged; expert weights changed; no optimizer state exists for any router parameter.
…eIntellect-ai#3843) ## Summary Add `iter_moe_blocks(model)` in `trainer/model.py`. It yields the `MoE` MLP of each decoder layer. It replaces the five copies of the decoder-layer MoE lookup in `freeze_moe_router`, `apply_fp32_moe_router`, `get_full_offload_dtype_policy`, `apply_force_balanced_routing` and `get_load_balance_stats`. ## What changed - New `iter_moe_blocks` next to the helpers that use it. The helpers stay in `model.py`, so no imports change. - The four `layer.mlp if hasattr(...) else layer.feed_forward ...` lookups and the `getattr(mlp)` + `hasattr(tokens_per_expert)` scan in `get_load_balance_stats` now call `iter_moe_blocks`. - Removed the stale "HF impl" comment in `get_expert_load_stats`. The HF fallback was removed in PrimeIntellect-ai#3773. ## Behavior change None. The `feed_forward` fallback is gone. Every prime-rl decoder layer uses `mlp`. `get_load_balance_stats` already looked only at `mlp`. Only `MoE` registers `tokens_per_expert`, so it selects the same layers as before. ## Checks - CPU equivalence vs `main` (throwaway script, not in the repo). Tiny Qwen3-MoE (`mlp_only_layers=[1]`) and GLM4-MoE (`first_k_dense_replace=1`) models with fixed router-count buffers. Compared `is_tt_moe_model`, `get_load_balance_stats`, `get_global_moe_stats`, `compute_expert_load_stats`, `get_expert_load_stats` and `get_full_offload_dtype_policy`, plus parameter dtype and `requires_grad`, and router `fp32_gate`/`force_balanced` after the three router helpers. All 16 outputs were bit-identical. - `pytest tests/unit/train tests/unit/orchestrator tests/unit/test_configs* -m "not gpu"`: 450 passed, 1 failed. The failure is `test_cpu_adam.py::test_native_cpu_adamw_matches_fused_torch_and_preserves_gradients` (ninja build), which fails the same way on `main`. - `ruff check` and `ruff format --check`. - GPU: reverse-text MoE RL smoke vs `main` on one node, running. <!-- CURSOR_SUMMARY --> --- > [!NOTE] > **Low Risk** > Mechanical refactor of MoE traversal with no intended behavior change; only risk is if a model relied on the removed feed_forward path (none in this codebase). > > **Overview** > Introduces **`iter_moe_blocks`**, a single iterator over decoder layers whose `mlp` is a **`MoE`**, and rewires MoE router/offload/metrics helpers to use it instead of repeated per-layer `mlp` / `feed_forward` lookups. > > **`freeze_moe_router`**, **`apply_fp32_moe_router`**, **`get_full_offload_dtype_policy`**, **`apply_force_balanced_routing`**, and **`get_load_balance_stats`** now loop over that iterator; the unused **`feed_forward`** fallback is removed. **`get_expert_load_stats`** only drops an outdated comment on the empty tensor guard. > > Intended as a no-op for supported models: same router freezing, fp32 gates, FSDP dtype policy, debug balanced routing, and load-balance / expert-load metrics. > > <sup>Reviewed by [Cursor Bugbot](https://cursor.com/bugbot) for commit 2cf0399. Bugbot is set up for automated code reviews on this repo. Configure [here](https://www.cursor.com/dashboard/bugbot).</sup> <!-- /CURSOR_SUMMARY -->
… and activation checkpointing (PrimeIntellect-ai#3925) With `moe_router_dtype="float32"` (the RL default) and activation checkpointing on, every MoE block ran eager under `torch.compile`. Two things combine to cause it: - The fp32 router is its own FSDP2 unit, and FSDP2 hooks are dynamo-disabled, so the router always breaks the graph. - Under `compile(AC(block))` the checkpoint compiles as a higher-order op, which forbids breaks, so Dynamo ran the whole block eagerly. This PR compiles inside the checkpoint instead: for any checkpointed block that holds a nested FSDP unit (today, the fp32 router), an eager checkpoint now wraps the compiled block (`AC(compile(block))`), so the nested unit's break only splits the block into compiled graphs. To see those units, compile now runs after FSDP. Compiling in place keeps the same module object, so the order changes nothing at runtime. The reorder is safe. The old `AC -> Compile -> FSDP` comment was ported from TorchTitan in PrimeIntellect-ai#923, when `apply_compile` still wrapped each block with `torch.compile(block)`. PrimeIntellect-ai#957 switched to in-place `.compile()` two days later without revisiting it. With in-place compile, both orders trace the same frames: graph breaks, loss and step time match on both workloads, and `fullgraph=True` (with a bf16 router) still compiles with zero graph breaks. | Workload (8x H200, EP 8, full AC) | before (s/step) | after (s/step) | MFU | |---|---|---|---| | GLM-4.5-Air, 16 layers, real data | 6.21, 6.17 | 5.22, 5.21 | 35% to 42% | | Qwen3-30B-A3B, fake data | 5.96, 5.96 | 4.51, 4.51 | 32% to 42% | Two runs per arm, each the median `time/step` over steps 5 to 20, no offloading. Peak memory is unchanged, and loss and grad norm track the baseline within run-to-run noise.  **Figure:** one MoE layer's forward on Qwen3-30B-A3B, before and after this PR. The orange eager elementwise kernels collapse into a few pink compiled Triton kernels, so the layer's forward drops from 16.4 to 11.3 ms. <details><summary>Same view on GLM-4.5-Air</summary>  </details> ## Alternative considered Keeping recompute inside the compiler: drop the block-level checkpoint and instead checkpoint two compiled regions that exclude the router, the attention module and the post-router MoE work. It beat the baseline but was slower than this PR (Qwen3-30B 5.50 vs 4.51 s/step, GLM-4.5-Air 5.85 vs 5.22 s/step) and used more memory, because the norms and the router's fp32 activations stay outside any checkpoint. ## North star: fullgraph fp32 router support Fullgraph compile with an fp32 router stays unsupported, and `model.compile.fullgraph=true` still raises for it. The blocker is the router's FSDP unit itself: FSDP2 applies one mixed-precision policy per unit, so the fp32 router cannot share the block's bf16 unit, and its dynamo-disabled hooks are the graph break above. PyTorch [#196626](pytorch/pytorch#196626) adds per-parameter mixed precision, which lets the router stay in the block's unit. That removes the break, enables fullgraph, and makes this PR's compile-inside-AC path unnecessary, so it should be reverted then. #196626 is on PyTorch `main` but not yet in a release. 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
## Summary Raises the `prime-runs[train]` floor from `>=0.1.3` to `>=0.1.4` and updates `uv.lock` to match. ## Why `prime-runs` 0.1.4 lets a run opt out of Prime Traces and upload its samples to the legacy sample tables instead (PrimeIntellect-ai/prime#998): - `PRIME_TRACES_OPT_OUT=true` - `prime config set-traces-opt-out true`, which sets `traces_opt_out` in `~/.prime/config.json` - `prime_runs.init(traces_opt_out=True)` `prime-runs` 0.1.3 doesn't know about any of these. With 0.1.3 installed, which is what `uv sync` gives today, a user who sets the environment variable or the config key gets no error and no warning, and their runs keep uploading to Prime Traces. Requiring 0.1.4 makes the setting take effect. Nothing changes for runs that don't opt out. The Prime monitor calls `prime_runs.init()` without the new argument, so the default (upload to Prime Traces) is unchanged. A training run that opts out needs pyarrow for the legacy sample encoder. prime-rl already depends on `prime-runs[train]`, which includes it. ## Changes - `pyproject.toml`: `prime-runs[train]>=0.1.3` → `prime-runs[train]>=0.1.4`. - `uv.lock`: `prime-runs` 0.1.3 → 0.1.4. No other package moves; `prime-traces` stays at 0.0.7. The `deps/verifiers` submodule is not changed. Its own floor is raised separately in PrimeIntellect-ai/verifiers#2805, and this PR doesn't depend on it. `uv lock --check` passes. I did not run the test suite. 🤖 Generated with [Claude Code](https://claude.com/claude-code) <!-- CURSOR_SUMMARY --> --- > [!NOTE] > **Low Risk** > Dependency floor bump only; no application code changes and default trace upload behavior stays the same. > > **Overview** > Raises the **`prime-runs[train]`** minimum from **`>=0.1.3`** to **`>=0.1.4`** in `pyproject.toml` and pins **`prime-runs` 0.1.4** in `uv.lock`. > > This ensures installs pick up **Prime Traces opt-out** behavior (`PRIME_TRACES_OPT_OUT`, `prime config set-traces-opt-out`, or `prime_runs.init(traces_opt_out=True)`), which 0.1.3 silently ignored so opted-out runs could still upload to Prime Traces. Default runs that do not opt out are unchanged; `prime-rl` still depends on the **`[train]`** extra for legacy sample encoding when opting out. > > <sup>Reviewed by [Cursor Bugbot](https://cursor.com/bugbot) for commit 06caa96. Bugbot is set up for automated code reviews on this repo. Configure [here](https://www.cursor.com/dashboard/bugbot).</sup> <!-- /CURSOR_SUMMARY --> Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
…training (PrimeIntellect-ai#3809) Keep successful empty environment outcomes in evaluation results. A trace-less episode or clean zero-turn trace retains the verdict returned by Verifiers, while training and curriculum logic exclude zero-turn traces. ## Before The dispatcher changed successful trace-less episodes into `EmptyEpisode` failures. Clean, non-timeout zero-turn traces became `EmptyTrajectory` failures, marking both the trace and episode unsuccessful. Evaluation therefore recorded a failure even when the environment had returned success, while training and curriculum filters relied on those injected errors to exclude empty traces. ```mermaid flowchart TD Env["Verifiers returns an empty successful outcome"] --> Dispatch["Prime-RL dispatcher"] Dispatch --> Episode["No traces: inject EmptyEpisode"] Dispatch --> Trace["Clean zero-turn trace: inject EmptyTrajectory"] Episode --> Failed["Rewrite outcome as failed"] Trace --> Failed Failed --> Eval["Evaluation records a failure"] Failed --> Training["Training and curriculum filter injected errors"] ``` ## After ```mermaid flowchart TD Env["Verifiers environment returns an episode"] --> Dispatch["Prime-RL dispatcher preserves empty outcome verdicts"] Dispatch --> Eval["Evaluation records and metrics"] Dispatch --> Filter["Training trace eligibility"] Filter --> Empty["Zero turns: no training sample"] Filter --> Sample["Eligible trace with sampled tokens: training sample"] Dispatch --> Curriculum["Curriculum ignores zero-turn traces and admits only trainable groups"] ``` Remove the synthetic `EmptyEpisode` and `EmptyTrajectory` errors. `iter_trainable_traces()` filters empty traces alongside failed and untrainable traces before advantages are assigned, so the advantage gate needs no separate zero-turn check. Difficulty sampling also excludes zero-turn traces. Empty groups finish without producing a trainer payload, and mixed groups retain their eligible traces. Curriculum admission also requires a trainable trace, so groups without training samples cannot inflate admission metrics. Sampler observations of completed turns are retained even when token metadata is unavailable. Verifiers' no-agent validation and training timeout handling remain in place. [PrimeIntellect-ai#3824](PrimeIntellect-ai#3824) applies the same ownership rule to partially failed episodes: preserve environment results, then decide training eligibility explicitly. <!-- CURSOR_SUMMARY --> --- > [!NOTE] > **Medium Risk** > Changes evaluation outcome recording and which rollouts feed training/curriculum admission, which can shift eval metrics and training batch composition without touching auth or data stores. > > **Overview** > Stops the dispatcher from **injecting** `EmptyEpisode` / `EmptyTrajectory` errors on successful trace-less or zero-turn outcomes, so **evaluation keeps Verifiers’ success verdict** instead of recording synthetic failures. > > Training and curriculum now **exclude zero-turn traces explicitly**: `iter_trainable_traces()` skips `num_turns == 0`, difficulty-pool reward updates require `num_turns > 0`, and curriculum admission requires **both** passing gates **and** at least one trainable trace via `iter_trainable_traces()`. Scored rollouts without token/advantage metadata can still update sampler difficulty; groups with no trainable samples no longer count as admitted. > > Tests cover mixed groups with non-trainable traces, empty-only groups, and reward-only observations for the sampler. > > <sup>Reviewed by [Cursor Bugbot](https://cursor.com/bugbot) for commit 8582fb8. Bugbot is set up for automated code reviews on this repo. Configure [here](https://www.cursor.com/dashboard/bugbot).</sup> <!-- /CURSOR_SUMMARY -->
…meIntellect-ai#3825) Apply a policy-version cache salt only when inference weights can change during a run. Standalone evaluations and frozen-model requests omit the training salt; live RL rollouts and online evaluation retain it so caches distinguish policy versions. ## Before The dispatcher added the policy version as `cache_salt` to every request using the live policy source. Standalone evaluation used that same source path, so fixed-weight evaluations received a training cache salt even though no weight updates occurred. Frozen-model requests already omitted it. ```mermaid flowchart TD Standalone["Standalone evaluation with fixed weights"] --> Live["Live policy source"] Training["RL or online evaluation with changing weights"] --> Live Live --> Salt["Always add policy-version cache_salt"] Salt --> Request["Verifiers provider request"] Frozen["Frozen model source"] --> Plain["Omit training cache_salt"] Plain --> Request ``` ## After ```mermaid flowchart TD Request["Prime-RL dispatcher builds request sampling"] --> Mode{"Live policy source with changing weights?"} Mode -->|"Yes: RL or online evaluation"| Salt["Set cache_salt to policy version"] Mode -->|"No: standalone evaluation or frozen model"| Plain["Omit training cache_salt"] Salt --> Client["Verifiers client builds provider request"] Plain --> Client Client --> Provider["Inference provider"] ``` The dispatcher records whether policy weights can change. The standalone evaluator disables version salting, SFT online evaluation enables it, and RL orchestration keeps it enabled. The request decision also checks whether the source uses the live policy. <!-- CURSOR_SUMMARY --> --- > [!NOTE] > **Low Risk** > Narrow behavior change to cache salting on eval paths; RL and online eval keep prior salting via the default flag. > > **Overview** > Adds a **`policy_weights_change`** flag on `Dispatcher` so inference **`cache_salt`** (policy version) is only attached when weights can actually change during the run. > > **Standalone eval** (`EvalConfig`) sets the flag to **false**, so fixed-weight evaluation no longer invalidates prefix caches on every training policy version. **SFT online eval** and **RL orchestration** (default **true**) still salt live-sourced requests so caches stay correct across weight updates. > > The cache decision is now **`live_sourced and policy_weights_change`** instead of salting all live policy paths. > > <sup>Reviewed by [Cursor Bugbot](https://cursor.com/bugbot) for commit 5a2d7d9. Bugbot is set up for automated code reviews on this repo. Configure [here](https://www.cursor.com/dashboard/bugbot).</sup> <!-- /CURSOR_SUMMARY -->
…ect-ai#3838) ## Summary Fold `GenerationSource` into `TrainEnv`, its only owner. Move `connect_frozen_client` from `algo/base.py` to `clients.py`, so `algo` no longer imports `clients` lazily. Give `TrainEnv` `setup()`/`aclose()` so orchestrator setup and teardown no longer reach into its parts. ## What changed - `orchestrator/generation_source.py` is deleted. `TrainEnv` (`envs.py`) now holds `clients`, `renderer_config`, `connected` (the frozen source clients it connected) and `uses_live_policy`. Dropping `logprobs` for frozen sources happens in `TrainEnv.__init__`. - `TrainEnv.setup()` connects the frozen generation source and runs `algorithm.setup()` concurrently. `TrainEnv.aclose()` closes `connected` and `algorithm.connected`, never the shared policy clients. - `connect_frozen_client` is a plain function in `clients.py`, next to `check_inference_ready`. `algo/base.py` imports it at module level. It is no longer re-exported from `prime_rl.orchestrator.algo`. - `dispatcher.py` reads the `TrainEnv` directly at the four former `generation_source` call sites. `orchestrator.py` calls `env.setup()` and `env.aclose()`. - `docs/algorithms.md` and the `algo/__init__.py` docstrings refer to `TrainEnv`. ## Behavior change None. ## Checks run - `pytest tests/unit/orchestrator tests/unit/test_configs* -m "not gpu"`: 297 passed. - `ruff check`, `ruff format --check`: clean. - Equivalence script, old `GenerationSource` (`origin/main`) vs new `TrainEnv`, on 4 configs: policy, policy with `top_p=0.9`, sft with a frozen `sampling.source`, opd with a frozen teacher. Identical `sampling_args`, `requires_sampling_masks` and `uses_live_policy`. After `setup()`, the same generation clients (policy object, or the frozen endpoint with the same train client config) and the same algorithm clients. Teardown closes the same clients. - GPU smoke on one node (vLLM 0.31): reverse-text RL (15 steps), sft with a frozen `sampling.source` (5 steps), opd with a frozen teacher (5 steps). All three ran to "Training finished" and exited 0. <!-- CURSOR_SUMMARY --> --- > [!NOTE] > **Low Risk** > Internal orchestrator refactor with stated behavioral equivalence and no changes to training algorithms or dispatch semantics beyond indirection. > > **Overview** > **Refactors train rollout sourcing** by deleting `GenerationSource` and moving its responsibilities onto `TrainEnv`: inference `clients`, live-vs-frozen policy detection (`uses_live_policy`), sampling-arg tweaks (e.g. dropping `logprobs` for frozen endpoints), and frozen-pool connection in `setup()`. > > `TrainEnv.setup()` now connects frozen `sampling.source` endpoints and runs `algorithm.setup()` together; `TrainEnv.aclose()` closes only env-owned clients (`connected` + algorithm pools), not the shared policy pool. The orchestrator and dispatcher call these surfaces instead of reaching into a nested `generation_source`. > > `connect_frozen_client` moves from `algo/base.py` to `clients.py` (next to `check_inference_ready`); it is no longer re-exported from `prime_rl.orchestrator.algo`. Docs and package docstrings now describe `TrainEnv` as owning the sampling half of the algorithm config. > > <sup>Reviewed by [Cursor Bugbot](https://cursor.com/bugbot) for commit e89ca1a. Bugbot is set up for automated code reviews on this repo. Configure [here](https://www.cursor.com/dashboard/bugbot).</sup> <!-- /CURSOR_SUMMARY -->
…llect-ai#3877) Processes that share `~/.triton` on a shared filesystem can hang or crash (seen with Nemotron-H, where many ranks compile the Mamba Triton kernels at once; reproduced below with Qwen3.5 and GLM-4.5-Air). This PR adds `TRITON_CACHE_DIR = /tmp/triton-<uid>` to `DEFAULT_COMMON_ENV_VARS`. It is per-user because `/tmp` is shared by all users on a node. Like the other launcher defaults, it reaches every launched component: the local `rl`/`sft`/`inference` launchers (also used by the single-node sbatch scripts) and the trainer, orchestrator, inference (incl. prefill/decode), and online-eval env blocks of the multi-node and inference sbatch scripts. `[env_vars]` (or a component's `env_vars`) overrides it; a value exported in the shell or `.env` does not. It also removes the `TRITON_CACHE_DIR` overrides from six example configs (glm-4.5-air swe/swe-budget/search/terminal, glm-5.3 and qwen3-30b-a3b sft h200 base): they pointed every user at the same `/tmp` dir and are now covered by the default. `docs/scaling.md` says so in one paragraph. vLLM 0.31's own default: when `TRITON_CACHE_DIR` is unset, its Inductor adaptor points it at `$VLLM_CACHE_ROOT/torch_compile_cache/<hash>/rank_<i>_<j>/triton_cache` (`~/.cache/vllm` by default, so usually on the shared home). That only covers Triton kernels compiled after torch.compile initializes, and it is still on the shared filesystem. vLLM keeps a `TRITON_CACHE_DIR` that is already set, so with this PR vLLM uses `/tmp/triton-<uid>` too. ### Measured cost (one H200 node, vLLM 0.31 venv, Triton 3.7.1) Each run starts from a fresh process with the same config. "Ready" is launch → first trainer step logged (Qwen) or launch → `/health` OK (GLM). For GLM, the first generated token came 1.3–2 s after ready in every run that came up. Inductor's cache dir was emptied before each run except the warm node-local one. | Triton cache | Qwen3.5-4B SFT, 8 GPUs, compiled blocks (step 1 time) | GLM-4.5-Air vLLM TP8 | |---|---|---| | shared, cold (today on a new cluster / after a Triton or torch update) | **failed 2/2**: `FileNotFoundError` on a cache file that another rank was writing (`causal_conv1d_fwd_kernel.llir`, `l2norm_fwd_kernel.ptx`) | **failed 2/2**: 4 of 8 ranks hit `FileNotFoundError` in the cache during Inductor autotune, the other 4 hung in a collective until killed (1st run). The 2nd run crashed instead of hanging. | | shared, warm | not measured cleanly (see below) | 344 s | | node-local, cold (this PR, first job on a node) | 233 s (step 1: 91 s), 164 kernels compiled | 375 s with vLLM's compile cache warm (106 kernels); 465 s if vLLM's cache is also cold (87 kernels) | | node-local, warm (this PR, later jobs on the same node) | 138 s (step 1: 8.3 s) | 344 s | - A cold node-local cache costs about **95 s per node for Qwen3.5** (FLA + Inductor kernels) and about **31 s for GLM-4.5-Air inference** when vLLM's own compile cache is warm. Warm node-local and warm shared take the same time. - The rest of the GLM startup does not depend on the Triton cache: about 3.5 min of Python imports and process spawn from the shared venv, 13 s to load weights. - vLLM: `VLLM_CACHE_ROOT` (`~/.cache/vllm`, shared home) holds vLLM's AOT/Inductor artifacts. With this PR it stays shared. On a new node, vLLM loads those artifacts (`torch.compile` 0.6 s instead of 30 s), and the Triton binaries are compiled into `/tmp/triton-<uid>` (106 kernels, +31 s). `TORCHINDUCTOR_CACHE_DIR` (default `/tmp/torchinductor_<user>`, already node-local) stayed empty in these runs. Inductor's Triton kernels go to `TRITON_CACHE_DIR`. - A crash on the shared cache can also break later starts. After the first failed GLM run, the shared vLLM cache only had AOT artifacts from the 4 ranks that got through. On the next start, those 4 ranks loaded the artifacts and the other 4 recompiled, so the ranks diverged and hung (Triton cache warm-ish, 60 kernels). For the same reason the Qwen "shared, warm" run started from a partial cache and failed again. The warm-shared GLM number comes from a shared dir seeded with a complete cache. ### Possible middle ground (not implemented) Because the cold cost is above 1 min per node for FLA-heavy models, a run could seed the node-local cache from a shared read-only copy at job start. For example, on each node before the components start: `[ -n "$TRITON_CACHE_SEED" ] && cp -rn "$TRITON_CACHE_SEED"/. "$TRITON_CACHE_DIR"/` The seed would be a warm `/tmp/triton-<uid>` copied once to the shared FS. Triton keys entries by source, compiler, and GPU, so stale entries are never reused, only skipped. Only reads hit the shared FS, so the concurrent-write race does not occur. I left it out to keep this PR minimal. <!-- CURSOR_SUMMARY --> --- > [!NOTE] > **Low Risk** > Launch-time env default only; behavior is overridable via config and fixes known multi-rank Triton cache races, with a one-time cold-compile cost on each node. > > **Overview** > **Sets a global launcher default** so every trainer, orchestrator, and inference process gets `TRITON_CACHE_DIR=/tmp/triton-<uid>` via `DEFAULT_COMMON_ENV_VARS`, avoiding concurrent Triton kernel writes under `~/.triton` on a shared filesystem (hangs / `FileNotFoundError` under multi-rank compile). > > **Removes redundant `[env_vars]` overrides** from six GLM-4.5-Air RL and GLM-5.3 / Qwen3-30B SFT example configs that previously pinned Triton cache paths under `/tmp`; those runs now inherit the per-user node-local default (still overridable through config `env_vars`). > > <sup>Reviewed by [Cursor Bugbot](https://cursor.com/bugbot) for commit f216420. Bugbot is set up for automated code reviews on this repo. Configure [here](https://www.cursor.com/dashboard/bugbot).</sup> <!-- /CURSOR_SUMMARY --> --------- Signed-off-by: faresobeid <111092724+faresobeid@users.noreply.github.com>
…ool (PrimeIntellect-ai#3858) ## Summary - Delete `trainer/models/layers/rms_norm.py`. It only re-exported `RMSNorm`/`RMSNormConfig` from `layers/norms.py`, and nothing imported it. - Move `quantize_to_fp8_blockwise` from `trainer/models/fp8.py` into `tools/convert_bf16_to_fp8.py`, its only user. `tools/convert_dcp_to_fp8.py` still imports it through `convert_bf16_to_fp8`. While moving it, the 9-line zero-pad block becomes `F.pad(weight, (0, pad_cols, 0, pad_rows)).contiguous()`; the output is bitwise identical to the old function (bf16/fp16/fp32, padded and unpadded shapes, contiguous and transposed inputs). ## Behavior change None. <!-- CURSOR_SUMMARY --> --- > [!NOTE] > **Low Risk** > Dead-code removal and relocating a tool-only quantizer; no trainer inference or training path changes. > > **Overview** > Removes dead trainer surface area and keeps FP8 checkpoint conversion self-contained in the tooling layer. > > **`trainer/models/layers/rms_norm.py`** is deleted (it only re-exported `RMSNorm` / `RMSNormConfig` with no importers). **`trainer/models/fp8.py`** is removed and **`quantize_to_fp8_blockwise`** now lives in **`tools/convert_bf16_to_fp8.py`**, which was its only consumer; **`convert_dcp_to_fp8.py`** continues to pull quantization helpers from that script. Padding inside the moved helper uses **`F.pad`** instead of manual zero-fill, with the same blockwise e4m3 logic otherwise unchanged. > > <sup>Reviewed by [Cursor Bugbot](https://cursor.com/bugbot) for commit c35ca75. Bugbot is set up for automated code reviews on this repo. Configure [here](https://www.cursor.com/dashboard/bugbot).</sup> <!-- /CURSOR_SUMMARY -->
…ill (PrimeIntellect-ai#3841) ## Summary Two bugs in how `rl` sets `weight_broadcast.inference_world_size` on main: 1. **World size computed before the DP fill.** `auto_setup_weight_broadcast` set `trainer`/`orchestrator.weight_broadcast.inference_world_size = dp * tp` before `auto_setup_deployment` fills the single-node inference GPUs with DP ranks. With `num_infer_gpus = 4` and `data_parallel_size` unset, the NCCL group was sized for 1 inference worker while vLLM started 4, so the first weight update cannot form the group. 2. **A single-node SLURM job cannot re-parse its `rl.json`** (since PrimeIntellect-ai#3788). The resolved `rl.json` always contains `weight_broadcast.inference_world_size` (default 1), and PrimeIntellect-ai#3788's guard rejects any config that sets it. So `rl @ rl.json` in the single-node sbatch fails validation for every NCCL config. ## Fix - Single-node: set `trainer`/`orchestrator.weight_broadcast.inference_world_size = dp * tp` right after the DP fill in `auto_setup_deployment` (non-filesystem only). Multi-node already sets it there. - `auto_setup_weight_broadcast` copies `weight_broadcast` to the trainer and orchestrator without touching the world size, and the guard is removed. The top-level value is now only passed through: `rl` overwrites it whenever it knows the inference size. This also fixes what PrimeIntellect-ai#3924 fixes. ## Behavior change Yes (bug fixes). - Resolved configs for every TOML in `configs/`, `examples/`, `k8s/` that parses as `RLConfig` (41 of 78) are identical before and after, except `trainer`/`orchestrator.weight_broadcast.inference_world_size` goes from 1 to 4 in `examples/basic/hendrycks-sanity/rl.toml` and `configs/ci/nightly/multimodal_color_codeword.toml` (4 infer GPUs, DP unset). - Re-parsing the dumped config (as the single-node sbatch does) fails on main for all 18 single-node NCCL configs and passes for all of them here. - With `inference = None` and NCCL against external servers, a user-set `weight_broadcast.inference_world_size` is now passed through instead of rejected. ## Checks - `pytest tests/unit/test_configs.py tests/unit/orchestrator -m "not gpu"`: 298 passed - `ruff check`, `ruff format --check`: clean - New test `test_single_node_nccl_resolved_json_roundtrips`: dumps a single-node NCCL config like `rl`'s `write_config`, re-parses it, and checks it is equal. It fails on main with the guard error. - `rl @ examples/basic/reverse-text/rl.toml --deployment.num-infer-gpus 2 --slurm.partition all --dry-run`, then re-parse the written `rl.json`: fails on main, passes here (world size 2). - GPU (one node): running: `rl @ examples/basic/hendrycks-sanity/rl.toml --max-steps 3` on main and on this branch, plus the rendered single-node sbatch for reverse-text (2 infer GPUs) run end to end. <!-- CURSOR_SUMMARY --> --- > [!NOTE] > **Medium Risk** > Changes when NCCL weight broadcast groups are sized for single-node runs; mis-sizing previously caused deadlocks or init failures during weight updates. > > **Overview** > Fixes incorrect **NCCL/NIXL** `inference_world_size` on single-node RL runs when vLLM **data parallel size** is inferred from `num_infer_gpus` rather than set explicitly. > > **`auto_setup_weight_broadcast`** no longer derives `trainer`/`orchestrator.weight_broadcast.inference_world_size` from `data_parallel_size × tensor_parallel_size` (which ran before DP was filled) and no longer rejects a user-set `inference_world_size`. **`auto_setup_deployment`** (single-node) now sets `inference_world_size` to `dp × tensor_parallel_size` immediately after the DP/`api_server_count` fill, so the in-memory transfer group matches every inference GPU that joins NCCL. > > Unit tests assert the resolved world size (2 infer GPUs → size 2) and that a dumped `rl.json` round-trips unchanged. > > <sup>Reviewed by [Cursor Bugbot](https://cursor.com/bugbot) for commit f5553f5. Bugbot is set up for automated code reviews on this repo. Configure [here](https://www.cursor.com/dashboard/bugbot).</sup> <!-- /CURSOR_SUMMARY -->
…rimeIntellect-ai#3917) This is an alternative to PrimeIntellect-ai#3907. The RL trainer decoded and preprocessed each micro batch's images on the main thread right before its forward pass, leaving the GPU idle. `WorkerPool`, a small ordered thread pool configured by `data.num_workers` like torch's DataLoader, now prepares micro batches ahead of the loop through a single `prepare_micro_batch` hook. - `num_workers` defaults to 0, which prepares on the main thread as before, and one thread gives the full speedup. Here it counts threads, unlike SFT's DataLoader processes. - An earlier version used forked worker processes. They pickle data through `/dev/shm`, risk copy-on-write memory, and were 0.6 s/step slower than a thread. - Pinned memory and `non_blocking` copies are left to a follow-up that first removes the per-micro-step CUDA syncs. Qwen3.5-9B trainer-only on 4 H200s, mock micro batches of 1024 px JPEGs (about 1 s of CPU decoding each), median s/step, lower is better (benchmark code added in [`331fc83fe`](PrimeIntellect-ai@331fc83) and removed in the next commit): | Arm | main thread | prepared ahead | time saved | |---|---|---|---| | this PR, 1 thread | 24.98 | 18.86 | 24.5% | | this PR, 1 thread, optimizer CPU offload | 26.56 | 20.24 | 23.8% | | earlier process version, 2 workers | 24.79 | 19.02 | 23.3% | | PrimeIntellect-ai#3907 (thread with pinned memory) | 24.98 | 18.63 | 25.4% | Every arm matches the main-thread baseline bit for bit in loss and grad norm. Host memory stays flat over 30 steps, 0.3 GiB above the baseline for the prefetched images. 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
…rimeIntellect-ai#3941) Hosted runs with online evals score real rewards but the run dashboard shows no `eval` metric group (ENG-6573). The dashboard's `eval` group is built from the run's own metric keys (`eval/{env}/...`). The orchestrator's in-process evals reach it through the train monitor, but when evals run in a process of their own (SFT online evals) the metrics are dropped: `PrimeEvalMonitor.log_metrics` was a deliberate no-op. - `PrimeEvalMonitor` now attaches to the parent training run via `$RUN_ID` and forwards each epoch's `eval/...` metrics there. - The eval process never reports the parent run's status: the SDK's atexit crash hook is unregistered and `finalize()` only drains the queue. Without that, an eval process exiting while the trainer is still going would mark the whole run failed. - No parent attached without `$RUN_ID`, so a standalone `prime eval` is unchanged. Note the ticket proposes exposing `environmentshub/resolve` on the platform's internal API instead. That's a separate gap: it silences the `EnvironmentResolutionError` warning and would create per-epoch Evaluation records, but it does not produce the `eval` metric group, and it needs four more `/evaluations` routes that the internal surface also lacks. <!-- CURSOR_SUMMARY --> --- > [!NOTE] > **Medium Risk** > Changes how eval processes interact with the parent platform run (metrics + SDK lifecycle hooks); mistakes could misreport run status or drop metrics, but behavior is gated on $RUN_ID and explicitly avoids finishing the parent. > > **Overview** > Fixes hosted runs where **SFT online evals** compute rewards but the run dashboard never shows the **`eval`** metric group (keys like `eval/{env}/...`). > > **`PrimeEvalMonitor`** now attaches a metrics-only handle to the parent training run when **`$RUN_ID`** is set, and **`log_metrics`** forwards sanitized epoch aggregates there instead of being a no-op. Standalone **`prime eval`** (no **`$RUN_ID`**) is unchanged. > > Lifecycle is guarded so the eval subprocess cannot poison the parent run: the SDK **atexit** hook on the attached parent is **unregistered**, and **`finalize()`** only **`flush`**es the upload queue—never finishes or reports status on the trainer’s run. > > <sup>Reviewed by [Cursor Bugbot](https://cursor.com/bugbot) for commit 8920356. Bugbot is set up for automated code reviews on this repo. Configure [here](https://www.cursor.com/dashboard/bugbot).</sup> <!-- /CURSOR_SUMMARY -->
…PrimeIntellect-ai#3947) On torch 2.13.0 with compile and router replay, some MoE layers silently replay layer 0's routing instead of their own from the second training step on. This PR calls `.contiguous()` on each layer's routed-experts slice, which removes the bug. Each decoder layer `l` receives `routed_experts[:, :, l]`, a view into one shared int32 tensor with axes (batch, sequence, layer, expert slot). When `top_k` is not a multiple of 4, that view starts at a byte offset that is not 16-byte aligned for some layers (odd layers on DeepSeek V4, `top_k = 6`). Inductor copies a misaligned input into a fresh aligned buffer, which starts at offset 0. Once automatic dynamic shapes recompile the layer, AOTAutograd mistakenly applies that offset 0 to the original shared tensor when it rebuilds the layer's routed-experts output. Offset 0 of the shared tensor holds layer 0's experts. A contiguous slice is its own aligned tensor, so Inductor never makes the copy. - Applied to the eight models that passed the raw view. `glm4_moe` already called `.contiguous()`. - Fixed upstream in pytorch/pytorch#191002 (torch 2.14.0 and 2.14.1, no 2.13.x release). The copy is cheap, so it can stay after an upgrade. - Models with `top_k = 8` (Qwen3-30B-A3B, Trinity-Mini, GLM-4.5-Air) were unaffected. DeepSeek V4 Flash RL probe (compile on, router replay on, lr 0), mismatch KL mean, lower is better: | trainer | step 1 | steps 2-10 | |---|---|---| | compiled, strided slice (main) | 0.0008 | 0.022-0.11 | | compiled, contiguous slice (this PR) | 0.0009 | 0.0007-0.0025 | With compile off, the same probe spans 0.0008-0.0032 over steps 1-10. ## Verification A standalone torch reproducer was added in 66641b0 and removed again at the end of this branch, so main never carries it. Run it on one CUDA device from that commit with `TORCHINDUCTOR_FORCE_DISABLE_CACHES=1 uv run python tools/repro_misaligned_replay_alias.py`. In earlier runs it printed `BUG REPRODUCED` on torch 2.13.0 and was clean on the 2.14.0.dev20260809 and 2.16.0.dev20261008 nightlies. 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
## Summary - Bump `deps/verifiers` from `1fa2bbc42` to main `da75527ca`. This pulls in the Harbor adapter changes from PrimeIntellect-ai/verifiers#2808: - Honor reward-file precedence and select named rewards explicitly. - Resolve separate verifier definitions by Harbor image precedence. - Align the adapter with Harbor 0.24.0 and reject multi-step tasks. - Move the `harbor` upload cutoff in `[tool.uv.exclude-newer-package]` from 2026-08-11 to 2026-10-08, which matches `deps/verifiers`. verifiers now pins `harbor==0.24.0`, which the old cutoff excluded. - Relock: `harbor` 0.21.0 → 0.24.0, and `semver` 3.1.0 is new. `deps/prime-envs` is already at its main head (`89dcc7ffc`). The current verifiers pin already has the network notice (PrimeIntellect-ai/verifiers#2769), so this bump supersedes PrimeIntellect-ai#3734. ## Verification - `uv sync --all-extras --all-packages --locked` succeeds. - `uv run pytest -m "not gpu" tests/unit`: 663 passed. 🤖 Generated with [Claude Code](https://claude.com/claude-code) <!-- CURSOR_SUMMARY --> --- > [!NOTE] > **Low Risk** > Lockfile and uv policy changes only; risk is mainly behavioral shifts from Harbor 0.24.0 and updated verifiers Harbor adapter usage in eval/training paths. > > **Overview** > Updates dependency resolution so **`harbor` 0.24.0** (via the `verifiers[harbor]` pin) can install under uv’s rolling **exclude-newer** policy by moving the bounded **`harbor` cutoff** from `2026-08-11` to **`2026-10-08`** in `pyproject.toml`, mirrored in `uv.lock`. > > The lockfile refresh bumps **`harbor` 0.21.0 → 0.24.0**, records new transitive deps **`jsonschema`** and **`semver`**, and aligns the verifiers harbor extra specifier to **`==0.24.0`**. This pairs with the verifiers main bump described in the PR (Harbor adapter alignment and reward/verifier precedence behavior), even though those code changes live outside this diff. > > <sup>Reviewed by [Cursor Bugbot](https://cursor.com/bugbot) for commit 69e62c1. Bugbot is set up for automated code reviews on this repo. Configure [here](https://www.cursor.com/dashboard/bugbot).</sup> <!-- /CURSOR_SUMMARY --> Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
…out (PrimeIntellect-ai#3909) ## Why Since **Sep 2** the `4xa6000` runner has been offline. Since then, `benchmark_regression` has queued on a dead label on **every single GPU Tests run**, hit GitHub's 24h self-hosted queue timeout, and failed the workflow. There has not been one green GPU Tests run in over a month, so the per-PR GPU check is permanently red and real regressions hide in the noise. Measured over Sep 29-Oct 6: all 28 completed runs failed at exactly 1440 +/- 1 min, and 20+ phantom `benchmark_regression` jobs were sitting in the queue at any time. ## What changes 1. **uv cache on the GPU jobs** (`enable-cache: true`, keyed on `uv.lock`, mirroring `cpu_tests.yaml`) — each h200 pod currently re-resolves and re-downloads the full dependency set from PyPI on every job. 2. **`benchmark_regression` removed from the PR matrix.** It cannot simply move to `prime-rl-2xh200`: the test unconditionally runs a 1-GPU **and** a 4-GPU benchmark (a 2-GPU pod cannot host the latter), and it asserts against A6000-captured baselines with a 15% tolerance (H200 results would blow through it). It belongs on benchmark hardware (`benchmarks.yaml`) or behind a 4-GPU scale set when one exists — happy to re-add once either exists. 3. **`max-parallel: 5` on the integration matrix** — a concrete per-PR ceiling: - 9 integration jobs + unit-tests = **6 h200 pods peak per PR**, 2 waves - at current job durations (~10-15 min each) a PR finishes in ~25-35 min - with 2-3 PRs in flight: <= ~18 pods steady-state; the ARC scale set `maxRunners` (suggest 16-20) is the complementary global cap 4. `docs/development.md` CI table updated to match (also adds `gsm8k_eval`, which was missing, and the current runner). ## Result GPU Tests runs **only** on the `prime-rl-2xh200` scale set, and every PR gets a real green or red within ~30-60 min instead of a guaranteed 24h failure. ## Not in this PR (follow-ups) - Bake git/uv into the CI image to kill the per-job `apt-get install git` - PVC-backed `~/.cache/uv` mount to avoid per-pod cache archive downloads - Nightly Tests still target a `research-cluster` runner that does not exist — needs the same treatment or a cron disable <!-- CURSOR_SUMMARY --> --- > [!NOTE] > **Low Risk** > CI workflow and documentation only; no runtime or auth changes. Benchmark regression no longer runs on every PR until re-homed to suitable hardware. > > **Overview** > **GPU Tests** is adjusted so PR runs can finish on live hardware instead of timing out on a dead runner label. > > Both GPU jobs now enable **uv dependency caching** keyed on `uv.lock` (same pattern as `cpu_tests.yaml`), cutting repeated PyPI resolves/downloads on each H200 pod. The integration matrix **drops `benchmark_regression`** (it targeted offline `4xa6000` hardware and cannot run meaningfully on 2×H200), and adds **`max-parallel: 5`** so at most five integration jobs run at once per PR. **`docs/development.md`** updates the CI table to list the current integration scenarios (`gsm8k_eval` included), `max-parallel`, and **`prime-rl-2xh200`** as the runner. > > <sup>Reviewed by [Cursor Bugbot](https://cursor.com/bugbot) for commit 8207cc0. Bugbot is set up for automated code reviews on this repo. Configure [here](https://www.cursor.com/dashboard/bugbot).</sup> <!-- /CURSOR_SUMMARY -->
…ct-ai#3950) SFT samples can contain prefilled assistant turns that must remain in context without becoming training targets. Read an optional `message_loss_mask` dataset column and pass it directly to `renderers.build_training_sample`; for a user/assistant/user/assistant conversation, `[0, 0, 0, 1]` trains only the final assistant turn. The renderer owns token attribution and mask validation, including GLM's assistant-generated closing markers. prime-rl performs no owner walking or template-specific mask reconstruction. Selection only removes existing loss targets and preserves input/target token IDs. Missing or null masks preserve normal supervision; fully masked samples follow the existing skip behavior. `data.columns.message_loss_mask` supports renamed columns. For prompt/completion rows, the mask covers their concatenated message list. This API reads a separate column; datasets with inline `messages[].trainable_mask` need conversion before use. Pins `deps/renderers` to upstream main commit `e3ace557545274385abb30435975c134c7c2429d`, which merges PrimeIntellect-ai/renderers#181. The renderer dependency is now merged. Validation: - Signed pin update `838828713` points to the merged renderer commit. Its Git tree is identical to the previously tested renderer head; no source changes accompany the pin update. - The following tests and checks passed on signed implementation commit `7e0800051`: - 73 SFT dataset tests passed, including real GLM-5 and Qwen3 tokenizers, both dataset layouts, custom column names, missing/null/all-enabled masks, excluded initial/final turns, fully masked samples, and invalid values/lengths. - Ruff 0.13.0 lint and formatting checks passed for the changed Python files; `git diff --check` passed. - CPU tests ran in an isolated Python 3.12 environment with Transformers 5.6.2 and PyTorch 2.14.1. No GPU training run performed. - Commit signature verified locally. <!-- CURSOR_SUMMARY --> --- > [!NOTE] > **Medium Risk** > Changes which tokens receive loss during SFT when datasets supply masks; incorrect masks could silently train the wrong turns, though behavior is opt-in and heavily tested against real renderers. > > **Overview** > Adds **optional per-message SFT loss selection** via a dataset `message_loss_mask` column (0/1 or booleans, one entry per resolved message—including prompt+completion rows). The trainer forwards the column to `build_training_sample` so renderers drop supervision on excluded turns **without changing token IDs** (e.g. `[0, 0, 0, 1]` keeps context but trains only the last assistant reply). Missing/null masks behave as today; all-zero masks still skip the sample. > > **Config and docs:** `SFTColumnsConfig.message_loss_mask` defaults to `"message_loss_mask"` and can be remapped (e.g. `"selection"`); dataset init errors if a remap points at a missing column. `docs/training.md` documents the format and semantics. > > **Tests:** Coverage for GLM-5 and Qwen3, both dataset layouts, custom column names, partial masks, fully masked rows, and invalid mask values/lengths. > > <sup>Reviewed by [Cursor Bugbot](https://cursor.com/bugbot) for commit 8388287. Bugbot is set up for automated code reviews on this repo. Configure [here](https://www.cursor.com/dashboard/bugbot).</sup> <!-- /CURSOR_SUMMARY -->
…rimeIntellect-ai#3948) SFT now pads packed rows only to a multiple of cp, as RL does, instead of to `seq_len`. The fp8 indexer drops its autotuning, which re-benchmarked configs on every new row length and made unpadded rows slower than padded ones on DeepSeek-V4 (PrimeIntellect-ai#3726). - The indexer pads keys to a multiple of 16 and runs one fixed config, within 1.05x of the best. - Every rank, including RL on DeepSeek-V4 and GLM-5, runs that config with bitwise-identical top-k. - SFT and RL throughput count trained tokens, not `seq_len` × rows or the first row × count. - The SFT `seq_len % (2 * cp)` check is gone: CP only needs `seq_len % cp`, which configs enforce. Same samples per arm, time summed over steps 5 to 20, one H200 node: | Workload | Padding on main | main | This PR | Speedup | |---|---|---|---|---| | DeepSeek-V4-Flash 6L, scaleswe, 256K rows, cp 8 | 19.8% | 61.2 s | 38.2 s | 1.60x | | GLM-4.5-Air 8L, math, 131K rows, dp 4 cp 2 | 7.3% | 223.8 s | 200.1 s | 1.12x | | GLM-4.5-Air 8L, scaleswe, 131K rows, dp 4 cp 2 | 19.7% | 701.0 s | 623.0 s | 1.13x | On DeepSeek-V4, the 1.60x exceeds the 1.25x that removing 19.8% of tokens predicts because two effects multiply: main's token-0 pads crowd onto the same experts, and the padding itself costs compute. The DeepSeek-V4 ablation section below separates the two. On GLM with scaleswe, removing 19.7% of the tokens would predict about 1.25x, but it measured 1.13x. Its 4 data-parallel ranks now get rows of different lengths, and each step likely waits for the rank with the longest rows. Loss matches main to within 1e-3 per step. This overlaps PrimeIntellect-ai#3932 in `CatDataset._finalize_pack`. Harness: [`benchmarks/scripts/sft_pad_to_cp/`](https://github.com/PrimeIntellect-ai/prime-rl/tree/0ed0e78b33e8533d7fd1e0d603759dc2ab50e5dd/benchmarks/scripts/sft_pad_to_cp) (removed before merge). ## DeepSeek-V4 ablation Each row changes one thing on the DeepSeek-V4 workload above, to separate the cost of the padding length, of token-0 pads crowding onto the same experts, and of indexer re-tuning: | Arm | Time | Speedup vs main | |---|---|---| | main: pads every row to `seq_len` with token 0 | 61.2 s | 1.00x | | main, but pads with the row's own tokens (PrimeIntellect-ai#3932) | 49.6 s | 1.23x | | pads only to a multiple of cp, indexer still autotuned | 73.6 s | 0.83x | | pads only to a multiple of cp, fixed-config indexer (this PR) | 38.2 s | **1.60x** | ## Alternatives considered - Bucketing the autotune key on `next_power_of_2(S_q)` reached 1.47x on DeepSeek-V4. Each bucket kept its first row's config (up to 1.44x slower later), short rows paid extra tunes, and each rank still tuned on its own rows, so ranks could run different kernels. 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Reduce CI time and GPU usage by selecting the relevant test suites for
each PR, moving hosted eval to CPU, and running independent setup steps
in parallel.
- Select GPU unit tests, SFT, RL, and hosted eval from changed paths
before allocating runners. Shared runtime and dependency changes retain
broad coverage; pushes to `main` and manual runs execute the full suite.
- Keep RL/SFT coverage for checkpoint converters, dashboard code, and
the shared dashboard smoke helper.
- Run `gsm8k_eval` on `ubuntu-latest` because it uses hosted inference.
- Run public submodule initialization and `uv` setup concurrently, then
wait for both before installing dependencies. Keep dependency caching
keyed on `uv.lock` and the five-job limit on GPU integration tests.
- Cancel superseded CPU and style runs on the same PR.
- Give the GPU workflow read-only contents access and the test selector
PR read access.
<!-- CURSOR_SUMMARY -->
---
> [!NOTE]
> **Medium Risk**
> Path-based GPU job gating can skip integration coverage on PRs if
filters miss a dependency; full suite still runs on main and manual
dispatch.
>
> **Overview**
> **GPU CI now picks which jobs to run from the PR diff**, instead of
always running the full GPU unit + integration matrix on every non-draft
PR. A new **Select tests** job uses path filters for unit, SFT, RL, and
eval; pushes to `main` and `workflow_dispatch` still run everything.
RL/SFT integration scenarios are built dynamically (still capped at
`max-parallel: 5` on `prime-rl-2xh200`).
>
> **`gsm8k_eval` is split out** into a dedicated job on `ubuntu-latest`
(hosted inference), so it no longer consumes GPU runners. Integration
pytest invocations use `tests/integration/test_${{ matrix.test }}.py`
from the dynamic matrix rather than per-row `pytest_args`.
>
> **Setup and queueing tweaks:** submodule init and `setup-uv` run in
parallel in CPU/GPU jobs; CPU and style workflows cancel in-progress
runs per PR; GPU workflow gets `workflow_dispatch`, read-only
`contents`, and PR-scoped concurrency cancel. `docs/development.md`
documents the new triggers and runner split.
>
> <sup>Reviewed by [Cursor Bugbot](https://cursor.com/bugbot) for commit
20da35f. Bugbot is set up for automated
code reviews on this repo. Configure
[here](https://www.cursor.com/dashboard/bugbot).</sup>
<!-- /CURSOR_SUMMARY -->
…#3953) Warn when SFT constructs a renderer that does not guarantee prefix stability. The warning explains that each conversation is rendered once into one training sample, that N assistant turns are not expanded into N samples, and that earlier reasoning may be omitted by the template. Cached renderers warn only on construction; stable renderers stay quiet. Custom renderers without the capability are conservatively treated as unknown. Pin `deps/renderers` to merged main revision `41f9290`, which includes PrimeIntellect-ai/renderers#184. It provides the resolved capability and the manual, opt-in witness audit. Sample construction is unchanged. Validation: all **73 SFT dataset tests passed**, including stable/unstable renderer selection and cache-warning behavior. Prime-rl unit-test CI passed on the preceding revision; this update changes only the renderer gitlink to the merged revision. Dependency requirements are unchanged apart from renderer Python >=3.11, compatible with prime-rl Python 3.12. The manually invoked renderer audit passes **59 tests** and is excluded from normal CI discovery.
…ldcache (CVE-2026-93318) (PrimeIntellect-ai#3889) ## security(ci): pin BuildKit builder to v0.33.1 and rotate the GHCR registry buildcache (CVE-2026-93318) ### What changed (rebased onto current main) `.github/workflows/build_image.yaml`: 1. **Builder pin** — `docker/setup-buildx-action@v4` now sets `driver-opts: image=moby/buildkit:v0.33.1`. Without it the job starts the rolling `moby/buildkit:buildx-stable-1` tag, and because these are persistent self-hosted runners (`image-builder`, `image-builder-arm-2204`), the builder version is not even guaranteed to follow the tag — a stale local image can keep a pre-patch builder alive indefinitely. The pin makes the builder version deterministic. 2. **Registry cache rotation** — both shared cache refs rotated: - `:dependency-buildcache-<suffix>` -> `:dependency-buildcache-v2-<suffix>` - `:application-buildcache-<suffix>` -> `:application-buildcache-v2-<suffix>` ### Why rotation (evidence) - The registry-backed cache was introduced in d8f3d01 (2026-08-16, PrimeIntellect-ai#3279) and has been written on nearly every run since — including all runs before the patch was published. - BuildKit v0.33.1 (the CVE fix) was released **2026-09-30T17:08Z**; the rolling `moby/buildkit:buildx-stable-1` tag only moved to that digest at **2026-09-30T17:32Z**. Every cache write before that timestamp came from a builder older than v0.33.1, on runners that may also hold stale local images. - Per the CVE-2026-93318 playbook: stop importing a shared cache that pre-fix builders wrote, rebuild from trusted inputs, and pin the patched builder so every future write is attributable to a fixed version. ### Expected cost The first run per platform suffix after merge does a cold build of the dependency layer + application layer (no cache import). The old `*-buildcache-*` tags can be deleted from the `prime-rl` GHCR package as cleanup — they are cache-only refs, never deployed. ### Out of scope / notes - `docker-push.yaml` (old Docker Hub publisher) no longer exists on main — nothing to pin there. - No GHA-scope (`type=gha`) caches exist in this repo's workflows (verified by grep), so no Actions-cache cleanup applies. - Build image runs are frequent and mostly manual dispatch; this PR should merge before the next `main` image build. ### Security review requested - Cache invalidation: confirm the `*-v2-*` refs fully replace the old buildcache lineage. - Published-image provenance: GHCR `prime-rl` images built before 2026-09-30T17:32Z were produced by a pre-patch builder; assess whether a clean rebuild of the current release tags is warranted. - No applicable CODEOWNERS entry exists in this repo; CI/build owner please review. <!-- CURSOR_SUMMARY --> --- > [!NOTE] > **Medium Risk** > CI-only security remediation with intentional cache busting (slower first builds); no application runtime code changes, but GHCR images built before the fix may need a separate provenance/rebuild decision. > > **Overview** > Remediates **CVE-2026-93318** in the GHCR image build workflow by making the BuildKit builder version fixed and breaking reuse of artifacts produced before the patch. > > **BuildKit pin:** `docker/setup-buildx-action` now sets `driver-opts` to `moby/buildkit:v0.33.1` (digest-pinned) so self-hosted `image-builder` runners do not keep serving a stale local `buildx-stable-1` builder. > > **Cache and deps lineage:** Shared registry cache refs move to `dependency-buildcache-v2-*` and `application-buildcache-v2-*` so layers written by pre-v0.33.1 builders are not imported. Dependency base tags gain a `-v2-` epoch (`deps-${key}-v2-${suffix}`) so existing `deps-${key}` images are not reused as the runtime base when only Dockerfile/lock inputs are unchanged. > > The first build per platform after merge is expected to be cold (no old cache import); published runtime tags are unchanged—only CI remediation naming and builder configuration. > > <sup>Reviewed by [Cursor Bugbot](https://cursor.com/bugbot) for commit 0310e6d. Bugbot is set up for automated code reviews on this repo. Configure [here](https://www.cursor.com/dashboard/bugbot).</sup> <!-- /CURSOR_SUMMARY -->
…f 1s (PrimeIntellect-ai#3952) ## Summary - `WeightWatcher.poll_interval` default: 1.0s → 0.1s - `WeightReceiver.wait_published` sleep: 0.2s → 0.1s ## Why The trainer blocks in `WeightSender._wait_for_receiver_ready` until the orchestrator notices `.sender_ready`. With a 1s poll, that adds 0–1s (~0.5s avg) of idle trainer time on every step. NVIDIA flagged this while benchmarking the ModelExpress broadcast (PrimeIntellect-ai#3890). A poll is one `glob("step_*")` on the broadcast dir (≤2 entries thanks to `_clean`) plus a few `exists()` calls, so 10Hz is negligible. Correctness relies on the marker handshake + `update_lock`, not on the poll timing. Caveat: these are sync FS calls on the orchestrator event loop. On a very slow shared FS, that might show up in `EventLoopLagMonitor`. If it does, we can move them to `asyncio.to_thread`. 🤖 Generated with [Claude Code](https://claude.com/claude-code) <!-- CURSOR_SUMMARY --> --- > [!NOTE] > **Low Risk** > Timing-only tuning on poll loops; no change to weight handshake logic, though more frequent sync FS calls could add event-loop lag on very slow filesystems. > > **Overview** > **Faster weight-version discovery** so the trainer spends less time blocked waiting for the orchestrator to see `.sender_ready`. > > The orchestrator `WeightWatcher` default `poll_interval` drops from **1.0s to 0.1s**, cutting typical idle wait on each broadcast handshake from ~0.5s toward ~0.05s. `WeightReceiver.wait_published` uses the same **0.1s** sleep (was 0.2s) while waiting for the sender marker. > > Handshake correctness is unchanged (markers + `update_lock`); cost is slightly more frequent lightweight FS checks on the orchestrator event loop. > > <sup>Reviewed by [Cursor Bugbot](https://cursor.com/bugbot) for commit 5c825d0. Bugbot is set up for automated code reviews on this repo. Configure [here](https://www.cursor.com/dashboard/bugbot).</sup> <!-- /CURSOR_SUMMARY --> Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
…ng (PrimeIntellect-ai#3782) Use Verifiers' shared sampling config and resolution path for evaluation and rollout dispatch. Parameter names, provider options, source defaults, and agent overrides follow the same rules in native Verifiers and Prime-RL. Training retains its strict config for the distribution the trainer replays. ## Sampling flow ```mermaid flowchart TD T["Train source and agent config<br/>TrainSamplingConfig and replay bounds"] --> V["Validate live-policy sampling<br/>Source owns the distribution"] V --> S["Shared Verifiers SamplingConfig<br/>Normalize aliases and extra_body"] V --> A["AgentConfig.sampling<br/>Sparse overrides across config transport"] E["Eval source and agent config"] --> S E --> A S --> R["RunRequest to env server"] A --> M["Verifiers resolve_agent<br/>Shared merge_defaults"] R --> M M --> P["Provider request<br/>Non-null resolved parameters"] M --> C["Trace<br/>Complete resolved sampling"] ``` ## Shared parameters Verifiers normalizes `max_tokens` / `max_completion_tokens` and flattens `extra_body` before merging each override layer. Source values replace group defaults; agent values replace inherited sampling. An explicit null clears an inherited parameter. Within one sampling block, top-level values take precedence over `extra_body`. A typed training `top_k=20`, or the automatic replay bound of `512`, therefore takes precedence over `extra_body.top_k=-1`. With no typed `top_k`, the disabled sentinel remains effective. Prime-RL carries `vf.SamplingConfig` through dispatch and converts `TrainSamplingConfig` once. Source defaults and agent overrides use Verifiers' `merge_defaults`; duplicate eval schemas and sampling-dictionary converters are removed. Request-specific cache salts are copied onto the shared config. Unresolved agent overrides remain sparse across env-server config boundaries. `resolve_agent` materializes the sampling used for generation, and traces record the complete resolved config. Provider dialects and the renderer client omit nulls when constructing requests. ## Training ownership Live-policy sources validate their sampling when the config loads. Unsupported source parameters, including `frequency_penalty`, are rejected before generation. The source owns the distribution replayed by the trainer. Every agent in a live-policy source, including non-trainable roles, may override only `max_tokens`, `reasoning_effort`, and `chat_template_kwargs`. Distribution parameters belong on the source, even when an agent would repeat the same value. Policy logprobs remain enabled. Evaluation and frozen-model sources retain general agent overrides. This removes the post-rollout sampling comparison from `TrainSink`. ## Dependency Pins Verifiers `2cb0a6d`, the merged commit for [Verifiers PrimeIntellect-ai#2762](PrimeIntellect-ai/verifiers#2762), for shared sampling normalization and resolution. This includes merging per-call judge sampling overrides before constructing SDK requests, preserving unrelated configured provider parameters. The lockfile updates `prime-tunnel` to `0.1.14` to meet the merged dependency's minimum. Before publishing packages outside this source checkout, release that Verifiers change and raise the `verifiers` minimum in both `prime-rl` and `prime-rl-configs`. The current `>=0.3.2.dev158` package constraints do not guarantee these semantics. <!-- CURSOR_SUMMARY --> --- > [!NOTE] > **Medium Risk** > Changes how sampling is resolved and validated for all rollouts; mis-merged configs or rejected agent overrides could break existing TOMLs, though unit tests cover the main inheritance and policy rules. > > **Overview** > **Unifies rollout sampling** on Verifiers' shared `SamplingConfig` and merge rules instead of Prime-RL–specific eval schemas and `to_sampling_args()` dict builders. > > Eval sources and group defaults now use `vf.SamplingConfig`; train/eval env wrappers carry a typed `SamplingConfig` through to the env server (including per-request `cache_salt` via `model_copy`). Policy train sources get **load-time validation**: the source owns distribution parameters (`temperature`, `top_p`, etc.), while agents may only override `max_tokens`, `reasoning_effort`, and `chat_template_kwargs`—frozen-model and eval paths stay permissive. `TrainSink` reads temperature from `env.config.sampling` instead of a resolved args dict. > > **Docs** describe alias normalization (`max_tokens` / `max_completion_tokens`), `extra_body` merging, and live-policy override limits. **Tests** cover inherited sampling merges and policy sampling rejection cases. Lockfile bumps `prime-tunnel` to `0.1.14`. > > <sup>Reviewed by [Cursor Bugbot](https://cursor.com/bugbot) for commit 671a5aa. Bugbot is set up for automated code reviews on this repo. Configure [here](https://www.cursor.com/dashboard/bugbot).</sup> <!-- /CURSOR_SUMMARY -->
…rimeIntellect-ai#3968) Sets `dsa_backend = "cudnn_flashmla"` for the trainer in the GLM-5.3 RL example (`examples/advanced/glm-5.3/swe.toml`). The SFT example already uses it (PrimeIntellect-ai#3829); the RL example still ran the TileLang forward and backward. Measured on 1x8 H200 with the trainer settings of `swe.toml` (131k seq, cp 4, ep 8, fp8 + `deepgemm_fp8`, full AC + offload, compile). Random init, 8 layers (3 dense + 5 MoE), fake data, 2 micro-batches per dp rank, balanced routing: | `dsa_backend` | step 2 / 3 | sparse attn fwd per call | sparse attn bwd per call | |---|---|---|---| | `tilelang` | 16.6 s / 15.5 s | 35 ms | 190 ms | | `cudnn_flashmla` | 12.8 s / 12.1 s | 12 ms | 53 ms | - Entropy and mismatch KL are identical to 4 decimals. - With a 4-layer run (5.7 s), one MoE layer costs 1.6 s instead of 2.07 s. Extrapolated to 78 layers at this shape: ~124 s vs ~160 s per step (-22%). - Kernel check at the CP shape (8k local queries vs 32k keys, cp rank 0 and 3, top-k 2048, sentinel slots): out / dq / dkv relative error vs TileLang 5e-4 to 9e-4, LSE 7e-8. - Needs the `flash-mla` extra, which the README's `uv sync --all-extras` installs. <!-- CURSOR_SUMMARY --> --- > [!NOTE] > **Low Risk** > Single example TOML knob with no production code changes; requires the optional flash-mla dependency for the new backend. > > **Overview** > Aligns the GLM-5.3 RL example trainer config with the SFT example by setting **`dsa_backend = "cudnn_flashmla"`** in `examples/advanced/glm-5.3/swe.toml`, so sparse DSA attention uses the cuDNN/FlashMLA path instead of the default TileLang forward/backward. > > This is a **performance-only** example change for long-context RL training (131k seq, CP/EP, fp8 MoE); behavior should match TileLang within measured numerical tolerance when the `flash-mla` extra is installed. > > <sup>Reviewed by [Cursor Bugbot](https://cursor.com/bugbot) for commit e2135c3. Bugbot is set up for automated code reviews on this repo. Configure [here](https://www.cursor.com/dashboard/bugbot).</sup> <!-- /CURSOR_SUMMARY -->
…t-ai#3982) ## What Drop recipe lines that only repeat what the config already resolves to. The resolved configs do not change, except for `skip_optimizer` (see below). - `[weight_broadcast] type = "nccl"`: `rl` already picks NCCL when inference is managed and LoRA is off. Removed from minimax-m2.5 swe, intellect-3.1 rl, and qwen3-30b-a3b swe/math. The block stays in examples/extra/dynamo/rl.toml (no managed inference, so the default there is filesystem) and in glm-5.3 (it also sets port and timeout). - `ac.freq = 1` (the default) is removed from 8 recipes. The `[*.model.ac]` table header stays, matching the GLM-4.5-Air recipes. - `trainer.dist_timeout_seconds = 3600` / `dist_timeout_seconds = 3600` (the default) is removed from 5 RL recipes and vlm/sft-moe. - `inference.vllm.enable_return_routed_experts = true` is removed where `trainer.enable_router_replay = true` already sets it (glm-4.5-air ×4, nemotron-3-super). - `[trainer.ckpt] skip_optimizer = true` is removed from glm-4.5-air swe-budget/search/terminal and nemotron-3-super swe. - It only affects loading: on resume, it drops the AdamW/Muon state. - It came from the PrimeIntellect-ai#2862/PrimeIntellect-ai#2915 ablation configs. None of these recipes warm-starts from another run's checkpoint. They train from the HF model and resume their own checkpoints, which include optimizer state. - configs/ci/integration/reverse-text-sft/full-offload-resume.toml keeps it on purpose: it tests that path. - docs/scaling.md: the `fused_lm_head_token_chunk_size` default is 8192 (since PrimeIntellect-ai#3302), not 1024. ## Not touched - `fused_lm_head_token_chunk_size = 1024` pins (7 recipes) and `attn = "flash_attention_3"` pins (10). Should they stay as memory/hardware choices? - `[ac_offloading] max_inflight_activations` lines (a separate PR removes the field). - The `[weight_broadcast]` blocks of the GLM-4.5-Air and nemotron recipes (handled in the weight_broadcast.timeout PR, to avoid a conflict). ## Test plan - `pytest tests/unit/orchestrator tests/unit/test_configs.py -m "not gpu"`: 334 passed (validates every TOML under configs/ and examples/). - Resolved configs of all 14 edited recipes, main vs this branch (RLConfig/SFTConfig `model_dump`): - identical, except `trainer.ckpt.skip_optimizer` True → False in the 4 recipes above; - the checkout path in `template_path` also differs (main checkout vs this worktree), which is not a recipe change. <!-- CURSOR_SUMMARY --> --- > [!NOTE] > **Low Risk** > Mostly documentation and deduplication; the only resolved-config change is `skip_optimizer` false on four recipes that resume with full optimizer state, which matches their intended training flow. > > **Overview** > Trims RL/SFT example and advanced recipe TOMLs by deleting settings that already match framework defaults, so recipes only spell out non-default choices. > > Removed redundant **`[weight_broadcast] type = "nccl"`**, **`trainer.model.ac` `freq = 1`**, **`dist_timeout_seconds = 3600`**, and **`inference.vllm.enable_return_routed_experts`** where NCCL broadcast, AC frequency, dist timeout, and routed-expert return are implied (e.g. managed inference + **`enable_router_replay`**). Dropped **`[trainer.ckpt] skip_optimizer = true`** from four GLM-4.5-Air and Nemotron SWE/search/terminal recipes so resume loads optimizer state from their own checkpoints instead of discarding it. > > **`docs/scaling.md`** now documents **`fused_lm_head_token_chunk_size`** default **8192** (was incorrectly **1024**). Explicit pins like **`fused_lm_head_token_chunk_size = 1024`** in memory-tight recipes are unchanged. > > <sup>Reviewed by [Cursor Bugbot](https://cursor.com/bugbot) for commit c87c43d. Bugbot is set up for automated code reviews on this repo. Configure [here](https://www.cursor.com/dashboard/bugbot).</sup> <!-- /CURSOR_SUMMARY -->
…ts (PrimeIntellect-ai#3986) Drops seven typed `[inference.vllm]` fields: `seed`, `enforce_eager`, `dtype`, `trust_remote_code`, `enable_dbo`, `max_loras`, `all2all_backend`, and the `All2AllBackend` alias. Each one only repeated vLLM 0.31's own default, and nothing in prime-rl reads it. `VllmConfig` is `extra="allow"`, so all seven keys still pass through to vLLM unchanged. The `VllmConfig` docstring says to type only fields that prime-rl types, defaults or reads back; these were none of those. `all2all_backend` also suggested control it doesn't have for P/D: the sbatch templates set the per-role backend in `ROLE_EXTRA` (`inference.sbatch.j2:261,276`, `multi_node_rl.sbatch.j2:405,419`). That stays as is. Behavior: I resolved the vLLM serve args (`to_namespace` → `make_arg_parser().parse_args`) on main and on this branch for the default config and for `--vllm.enforce-eager`, `--vllm.enforce-eager true`, `--vllm.max-loras 2`, `--vllm.dtype bfloat16`, `--vllm.seed 7`, `--vllm.all2all-backend deepep_low_latency`, `--vllm.enable-dbo true`, `--vllm.trust-remote-code true`, `@ examples/advanced/glm-5.3/infer/pd.toml`. All seven values are identical. The nested rl form `--inference.vllm.enforce-eager --inference.vllm.max-loras 2` also resolves to `True, 2`. `"enable_dbo" = true` in `examples/advanced/glm-5.3/swe.toml` sits under `prefill_vllm_overrides` and is unaffected. Differences: - `--no-vllm.enforce-eager` (the negated bool flag) only exists for typed fields. Use `--vllm.enforce-eager false`. All four removed bools default to False, so this only matters when undoing a toml that set one to true. - `dtype` now accepts every value vLLM accepts (for example `half`) instead of prime-rl's narrower Literal. `skills/configs/SKILL.md` used `enforce_eager = true` as its TOML boolean example. It now uses `enable_prefix_caching = true`, which is still typed, so `--no-flag` applies to it. ## Test plan - `pytest tests/unit/orchestrator tests/unit/test_configs.py tests/unit/test_parsers.py -m "not gpu"`: 489 passed - Parsed all 78 tomls under `configs/` and `examples/` with every config class on main and on this branch: same results - The resolved-args comparison above - `ruff check` / `ruff format --check` on the changed file
…rimeIntellect-ai#3978) Under `rl`, `inference.deployment.gpus_per_node` was never set from `deployment.gpus_per_node`. `InferenceConfig.auto_setup_disaggregated` sizes `data_parallel_size_local`, `data_parallel_size` and `api_server_count` from the nested copy, which defaults to 8. The launcher, sbatch `dp_per_node` and weight-broadcast world size use the top-level copy. So a P/D run on nodes that do not have 8 GPUs got mismatched engine counts. Example: `examples/advanced/glm-5.3/swe.toml` with `deployment.gpus_per_node = 4` resolved to `data_parallel_size_local = 8` and `api_server_count = 8`, while the template launched 4 engines per node. Changes: - `propagate_shared_fields` now fills `inference.deployment.gpus_per_node` from `deployment.gpus_per_node`. As with the other shared fields, setting both to different values raises. - `RLConfig.validate_shared_configs` requires the two to be equal. This catches a nested-only override that leaves the top-level value at its default. ## Test plan - `pytest tests/unit/orchestrator tests/unit/test_configs.py -m "not gpu"`: 335 passed (includes the new `test_disaggregated_inference_inherits_deployment_gpus_per_node`) - Parsed all 78 tomls under `configs/` and `examples/` with every config class on main and on this branch: same results, none fail - glm-5.3 `swe.toml` with `deployment.gpus_per_node = 4` now resolves `inference.deployment.gpus_per_node = 4`, `dp_local = 4`, `api_server_count = 4` - `ruff check` / `ruff format --check` on the changed files <!-- CURSOR_SUMMARY --> --- > [!NOTE] > **Medium Risk** > Mis-sized inference parallelism and launcher/engine counts on multi-node disaggregated runs; the change is narrow config propagation plus validation with low blast radius beyond P/D sizing. > > **Overview** > Fixes a config bug where **`deployment.gpus_per_node`** at the RL top level did not flow into **`inference.deployment.gpus_per_node`**, so disaggregated P/D runs could size **`data_parallel_size_local`**, **`api_server_count`**, and related fields from the nested default (8) while SLURM/launchers used the top-level value (e.g. 4). > > **`propagate_shared_fields`** now copies top-level **`deployment.gpus_per_node`** into **`inference.deployment.gpus_per_node`** (with the same shared-vs-sub-config conflict rules as other shared fields). **`RLConfig.validate_shared_configs`** additionally requires the two values to match when inference is enabled, catching nested-only overrides that disagree with the top-level deployment. > > A unit test asserts inheritance and that mismatched nested/top-level **`gpus_per_node`** fails validation. > > <sup>Reviewed by [Cursor Bugbot](https://cursor.com/bugbot) for commit 863bbf6. Bugbot is set up for automated code reviews on this repo. Configure [here](https://www.cursor.com/dashboard/bugbot).</sup> <!-- /CURSOR_SUMMARY -->
…el config (PrimeIntellect-ai#3965) vLLM 0.31's routed-experts capturer already returns uint8 or uint16, chosen from the model's expert count (`uint8 if num_experts <= 256 else uint16`). `serialize_routed_experts` cast every request back to uint8 whenever its ids happened to be below 256. On a >256-expert model (NemotronH Super/Ultra: 512 experts), a request whose ids all fall below 256 therefore came out uint8 while the others were uint16. Packing two such samples into one micro batch trips `assert routed_experts.dtype == sample.routed_experts.dtype` in `orchestrator/batch.py`. Multi-turn traces hit the same mix: verifiers concatenates the per-turn arrays, and NumPy widens the result. `serialize_routed_experts` now keeps the capturer's dtype and asserts that it is uint8 or uint16. Every request of a model then has the same dtype. Models with 256 or fewer experts are unchanged (uint8). Tests: the existing serialize test now passes a uint16 array with ids below 256 and checks that it stays uint16. `tests/unit/orchestrator`, `tests/unit/inference`, `tests/unit/utils`, `tests/unit/test_configs.py` pass with `-m "not gpu"`; `ruff check` and `ruff format --check` are clean. <!-- CURSOR_SUMMARY --> --- > [!NOTE] > **Low Risk** > Narrow inference serialization fix; ≤256-expert models stay uint8, and behavior only changes where mixed dtypes previously caused orchestrator batch asserts. > > **Overview** > **Fixes inconsistent routed-experts dtypes** that broke micro-batch packing on models with more than 256 experts (e.g. NemotronH 512). > > `serialize_routed_experts` no longer re-encodes expert indices to the smallest integer width based on per-request max values. It keeps the array dtype from vLLM’s capturer (`uint8` or `uint16` from model expert count), with assertions that the input is 3-D and one of those two types. The serialized payload’s `"dtype"` field reflects that array unchanged. > > Unit tests now decode using `encoded["dtype"]` and assert a `uint16` capture with all ids < 256 stays `uint16` through serialize/decode. > > <sup>Reviewed by [Cursor Bugbot](https://cursor.com/bugbot) for commit e4a8e9e. Bugbot is set up for automated code reviews on this repo. Configure [here](https://www.cursor.com/dashboard/bugbot).</sup> <!-- /CURSOR_SUMMARY -->
…t-ai#3984) Removes `model.index_cache` (`IndexCacheConfig.topk_freq` and `.topk_pattern`). The trainer still turns on IndexShare from the model's own `indexer_types` schedule (`src/prime_rl/trainer/model.py`), as before. Why: the override changes the trainer only. No inference-side config matches it, so setting it makes the trainer pick different top-k indices than vLLM unless the user also passes the same vLLM `hf_overrides` by hand. The setting appears in no config, example or test. The HF-level `use_index_cache` / `index_topk_freq` / `index_topk_pattern` attributes on `GlmMoeDsaConfig` stay, so a checkpoint's own `config.json` still drives them. Breaking: `[trainer.model.index_cache]` / `[model.index_cache]` now fails validation. Overlap with open PRs: - PrimeIntellect-ai#3833 rewrites the same `index_cache` lines in `model.py` (`set_model_options(model_config, "index_cache", ...)`). Whichever lands second drops or rebases that block; the conflict is small. - PrimeIntellect-ai#3879 is based on that code too. - PrimeIntellect-ai#3616 already leaves `trainer.model.index_cache` unset for the reason above: a trainer frequency override replaces the checkpoint schedule and requests weights the checkpoint does not have. Test plan - `pytest tests/unit/orchestrator tests/unit/test_configs.py -m "not gpu"`: 334 passed. - `pytest tests/unit/utils/test_weights.py` (includes `GlmMoeDsaForCausalLM`): 7 passed. - `ruff check` and `ruff format --check` on the changed Python files: clean. <!-- CURSOR_SUMMARY --> --- > [!NOTE] > **Medium Risk** > Breaking config validation for anyone using `index_cache`, and it changes sparse-attention index scheduling to follow the checkpoint only—important for trainer/inference logprob alignment on DSA models. > > **Overview** > **Removes the trainer-only `[trainer.model.index_cache]` override** (`topk_freq` / `topk_pattern`). Configs that still set it will fail validation. > > IndexShare for GLM-5.2-style DSA models is unchanged in behavior: when the checkpoint’s `indexer_types` includes `"shared"` layers, the trainer still sets `use_index_cache` on the HF config and logs the full/shared layer counts. It no longer applies manual `index_topk_freq` / `index_topk_pattern` or clears `indexer_types`, which could diverge from vLLM rollouts. > > Docs no longer mention manually overriding the schedule via `[trainer.model.index_cache]`. > > <sup>Reviewed by [Cursor Bugbot](https://cursor.com/bugbot) for commit bf75c92. Bugbot is set up for automated code reviews on this repo. Configure [here](https://www.cursor.com/dashboard/bugbot).</sup> <!-- /CURSOR_SUMMARY -->
…eIntellect-ai#3991) ## What Renames `tasks_per_minute` to `dispatch_per_minute` on `orchestrator`, the `eval` entrypoint and the SFT online `[eval]` block. This uses the same name and wording as PrimeIntellect-ai#3652. The old name was wrong. The dispatcher takes one rate-limiter token per **episode** it starts (`Dispatcher.acquire`, called once per scheduled episode), not one per task. With `group_size = 8`, `tasks_per_minute = 80` started only 10 tasks per minute. The docstrings now say what is counted: at most N episodes start per minute, and a group counts `group_size` times. Breaking: configs that set `tasks_per_minute` now fail validation (`extra="forbid"`). No shipped config, doc, test or skill sets it or mentions it. ## Overlap PrimeIntellect-ai#3652 (draft, orchestrator as hook-wired components) makes the same rename. This PR lands it on its own and does not depend on that refactor. ## Test plan - `pytest tests/unit/orchestrator tests/unit/eval tests/unit/test_configs.py -m "not gpu"`: 357 passed. - `ruff check` and `ruff format --check` on the changed files. <!-- CURSOR_SUMMARY --> --- > [!NOTE] > **Low Risk** > Breaking config field rename only; runtime rate limiting behavior is unchanged aside from clearer documentation. > > **Overview** > **Breaking rename:** `tasks_per_minute` → `dispatch_per_minute` on orchestrator config, eval/SFT online eval config (`ServedEvalConfig`), and everywhere it is passed into `Dispatcher` (orchestrator + eval runner). > > The limit semantics are unchanged in code (still `AsyncLimiter` with one token per scheduled episode in `Dispatcher.acquire`), but the name and docstrings now match behavior: the cap is **episodes started per minute**, and a rollout group with `group_size` N consumes N tokens. Configs that still use `tasks_per_minute` will fail validation under `extra="forbid"`. > > <sup>Reviewed by [Cursor Bugbot](https://cursor.com/bugbot) for commit 0436304. Bugbot is set up for automated code reviews on this repo. Configure [here](https://www.cursor.com/dashboard/bugbot).</sup> <!-- /CURSOR_SUMMARY -->
…Intellect-ai#3979) Multi-node RL jobs that crash currently sit idle for an hour. On any non-zero exit (crash, OOM, SIGTERM, wall-time), `multi_node_rl.sbatch.j2` ran an unconditional `sleep {{ cleanup_grace_period }}` (default 3600 s, PrimeIntellect-ai#2670) before tearing down. The sleep keeps the task alive so `--kill-on-bad-exit` does not reap trainers that are mid-checkpoint. But it waits the full hour even when no checkpoint is being written, so a crashed N-node job idles for 1 h. Now the node waits only while the newest `checkpoints/step_*/trainer` has no DCP `.metadata` yet, i.e. a trainer save is still in flight. It polls every 10 s, with `cleanup_grace_period` as the cap, and tears down as soon as that save finishes or there is none. Runs without `[ckpt]` tear down immediately. The checkpoint dir follows `ckpt.output_dir` when it is set. The default stays 3600 s; it is now only an upper bound. Limitation: if the crash happens right before a save starts (step dir not created yet), we no longer wait for that save. Only that save is lost; the previous checkpoint is intact. **Interaction with PrimeIntellect-ai#3875 (requeue + step watchdog):** before this change, every requeue of a crashed multi-node RL job first waited the full grace period. Now it waits only for an in-flight checkpoint. `.metadata` is the same completeness marker that PrimeIntellect-ai#3906 uses for "latest complete checkpoint". The SFT and inference templates do not use `cleanup_grace_period` and are unchanged. Test plan (all run): - `pytest tests/unit/orchestrator tests/unit/test_configs.py -m "not gpu"`: 334 passed. - `rl @ examples/advanced/glm-4.5-air/swe.toml --dry-run`: the rendered sbatch has the wait loop with the run's `checkpoints/` path and passes `bash -n`. With `--no-ckpt` the loop is not rendered. - Ran the rendered teardown snippet in bash against a temp checkpoint dir: - empty dir: no wait; - newest step complete: no wait; - newest step in flight: waits until the cap; - older incomplete step but newest complete: no wait; - in-flight save that completes after 1 s: waits ~1 s. - `ruff check` and `ruff format --check` on the changed Python files. <!-- CURSOR_SUMMARY --> --- > [!NOTE] > **Medium Risk** > Changes multi-node job teardown timing and checkpoint detection logic; incorrect polling could tear down early or delay allocation release, but scope is limited to SLURM RL launcher scripts. > > **Overview** > Multi-node SLURM RL jobs no longer **sleep the full `cleanup_grace_period` (default 1h) on every failure**. Teardown after a non-zero exit now waits only while an **in-flight trainer checkpoint** is detected. > > The `multi_node_rl.sbatch.j2` teardown block is gated on both `cleanup_grace_period` and a rendered **`ckpt_dir`** (from `get_ckpt_dir` when `[ckpt]` is enabled). Instead of a fixed `sleep`, it polls every 10s: if the newest `step_*/trainer` directory lacks DCP **`.metadata`**, it keeps the task alive (preserving `--kill-on-bad-exit` behavior for peer trainers); otherwise it tears down immediately. **`cleanup_grace_period` is documented as an upper bound**; runs without checkpoints skip the wait entirely. > > `write_slurm_script` in `rl.py` passes `ckpt_dir` into the multi-node template renders. > > <sup>Reviewed by [Cursor Bugbot](https://cursor.com/bugbot) for commit 8168264. Bugbot is set up for automated code reviews on this repo. Configure [here](https://www.cursor.com/dashboard/bugbot).</sup> <!-- /CURSOR_SUMMARY -->
…t-ai#3980) ## What - `weight_broadcast.timeout` default goes from 1200 s to 3600 s (`configs/shared.py`). - It now also bounds the orchestrator's wait for the trainer's startup broadcast. `orchestrator.ckpt.wait_for_weights_timeout` and the hardcoded `STARTUP_WEIGHT_WAIT_TIMEOUT_S = 1200` fallback are removed. Configs that set the old field fail validation (`extra="forbid"`); no in-repo config sets it. - The `[weight_broadcast]` blocks in five recipes only repeated `type = "nccl"` (already the `rl` default when inference is managed and LoRA is off) and `timeout = 3600`. They are removed: glm-4.5-air swe/swe-budget/search/terminal and nemotron-3-super swe. glm-5.3 keeps its `timeout = 12000`. - Orchestrator `ckpt` and `resume` docstrings: `resume` had two docstrings and `ckpt` had none. The stray one now documents `ckpt`. ## Why - Every large multi-node recipe raises the timeout: 3600 s in five recipes, 12000 s in glm-5.3. - Measured startups come close to 1200 s or pass it. GLM-4.5-Air TP8 cold start takes 465-623 s. GLM-5.3 and DeepSeek-V4 inference startup with cudagraph capture took about 16 min. - vLLM's own engine-ready timeout is already 4200 s (`VLLM_ENGINE_READY_TIMEOUT_S`). - The startup wait and the per-step waits had two knobs with the same 1200 s default. The startup one was under `ckpt`, but it applies without `ckpt` too. - Trade-off: a consumer that dies is detected after 60 min instead of 20. ## Overlap - PrimeIntellect-ai#3652 (mikasenghaas, draft) reformats the `wait_timeout = ...` line that this PR removes. Whichever lands second has a one-line conflict. - Rebased over PrimeIntellect-ai#3983, which removed the neighbouring orchestrator `ckpt.keep_last`/`keep_interval` fields. The orchestrator `CheckpointConfig` now has only `interval`. ## Test plan - Rebased onto origin/main 32c48f5. `pytest tests/unit/orchestrator tests/unit/test_configs.py -m "not gpu"`: 335 passed. `test_configs` validates every TOML under `configs/` and `examples/`. - `ruff check` and `ruff format --check` on the changed files: clean. - Resolved `RLConfig` of the 5 edited recipes, main vs this branch: identical (apart from the checkout path in `template_path`). <!-- CURSOR_SUMMARY --> --- > [!NOTE] > **Medium Risk** > Changes orchestrator startup behavior and timeout defaults for trainer–inference weight sync; mis-tuned timeouts could delay failure detection (up to 60 min) or still timeout on very slow large-model startups. > > **Overview** > **Unifies weight-broadcast timing** so one knob (`weight_broadcast.timeout`) covers both the trainer handshake and the orchestrator’s wait for the trainer’s **startup** broadcast. The default rises from **1200 s to 3600 s**, and docs note raising it for large models on slow filesystems. > > **Removes** `orchestrator.ckpt.wait_for_weights_timeout` and the orchestrator’s `STARTUP_WEIGHT_WAIT_TIMEOUT_S` fallback; startup sync now uses `config.weight_broadcast.timeout` in `sync_startup`. **Orchestrator config** docstrings are fixed so `ckpt` is documented on the right field. > > **Five example/advanced recipes** drop redundant `[weight_broadcast]` blocks that only repeated NCCL type and `timeout = 3600` (now defaults). > > <sup>Reviewed by [Cursor Bugbot](https://cursor.com/bugbot) for commit e318f1c. Bugbot is set up for automated code reviews on this repo. Configure [here](https://www.cursor.com/dashboard/bugbot).</sup> <!-- /CURSOR_SUMMARY -->
…Intellect-ai#3988) Replaces the three switches `model.fsdp_cpu_offload`, `model.optim_cpu_offload` and `model.full_offload` with one field: ```toml [model] offload = "optimizer" # default; also "full", "none" ``` This applies to both the RL trainer and SFT. The `fsdp` switch is not migrated: the mode is removed outright (rationale below). The `[model.full_offload] numa_bind` knob is dropped: no recipe, deployment, template or doc sets it. Full offload now always pins each rank's CPUs to its GPU's NUMA node. That was already the default. Ranks whose GPU reports no NUMA node still skip binding with a warning. On hosts without GPU sysfs topology (`/sys/bus/pci/devices/<bus>/numa_node`), binding is no longer optional: full offload fails there instead of being switched off. Why: the three switches could only be used one at a time, enforced by `cpu_offload_mutual_exclusion`. Because `optim_cpu_offload` defaulted to `true`, enabling full offload also meant writing `optim_cpu_offload = false`, otherwise validation failed (see the old `configs/advanced/deepseek-v4-flash/sft.toml` comment). With one value the conflicting combinations cannot be expressed, so that validator is deleted. The default behaviour does not change: optimizer-state offload stays on. Why drop `fsdp`: the mode was the stock torch FSDP2 `CPUOffloadPolicy` with no in-tree config, example, or test user; Muon rejected it and it broke NCCL weight broadcast. It was kept only for long-context runs whose fp32 masters/moments no longer fit on GPU (the reason it was restored in PrimeIntellect-ai#3052); the `full` mode added in PrimeIntellect-ai#3234 keeps FP32 masters, moments, and gradients in host RAM, covering that memory pressure instead. Gone with it: the `CPUOffloadPolicy` selection in `setup_fsdp`, CPU-side model init in `setup_model`/`load_dcp_from_hf`, `_move_buffers_to_cuda`, and the fsdp leg of the Gloo special case. Changes: - `ModelConfig.offload: Literal["none", "optimizer", "full"] = "optimizer"` replaces the three fields. `OptimizerInBackwardOffloadConfig` (and with it `numa_bind`) and its bool-normalizing `BeforeValidator` are gone. - Validators in `TrainerConfig` and `SFTConfig`: - full offload accepts only AdamW or SignSGD and disables clipping, now reading `offload == "full"`; - the `Muon optimizer does not support FSDP CPU offload` validators are deleted with the mode. - Runtime: - `setup_optimizer(..., offload=...)` replaces `cpu_offload` + `full_offload_config`, and its "both enabled" check is gone. - `FullCPUOffloadOptimizer` drops its unused `offload_config` argument. - `setup_full_cpu_optimizer_offload()` takes no arguments and always binds. - `model.py`: FSDP always uses the default `OffloadPolicy()`; CPU-side model init in `setup_model`/`load_dcp_from_hf` and `_move_buffers_to_cuda` are gone. - Tomls: `glm-4.5-air/swe-budget.toml` and `ci/integration/reverse-text-sft/full-offload-resume.toml` → `offload = "full"`; `glm-5.3/sft/h200/16k-12-node.toml` → `offload = "none"`; `glm-5.3/sft/h200/base.toml` → `offload = "optimizer"`. Also updated `docs/scaling.md`, the `deepseek-v4-flash/sft.toml` comment, and `skills/training/start-run/SKILL.md`, which documented the removed fields. Breaking: the old keys now fail with `extra_forbidden`, and `offload = "fsdp"` fails the `Literal`. There is no shim. Interaction with PrimeIntellect-ai#3849 (one trainer-common config base): PrimeIntellect-ai#3849 moves the full-offload and Muon/FSDP validators from `TrainerConfig`/`SFTConfig` onto `BaseTrainerConfig`. This PR edits the full-offload validator's condition and deletes the Muon/FSDP one, so the conflict is textual only: keep PrimeIntellect-ai#3849's single copy of the full-offload validators with `self.model.offload == "full"` and drop its Muon/FSDP copy. Whichever lands second takes a small rebase. Other open PRs that set or read the old keys need a rebase: - tomls: PrimeIntellect-ai#3883, PrimeIntellect-ai#3879, PrimeIntellect-ai#3766, PrimeIntellect-ai#3765, PrimeIntellect-ai#3729, PrimeIntellect-ai#3629, PrimeIntellect-ai#3627, PrimeIntellect-ai#3616, PrimeIntellect-ai#3544 - code: PrimeIntellect-ai#3885 (`optim_cpu_offload`), PrimeIntellect-ai#3739 (`full_offload` validators), PrimeIntellect-ai#3724, PrimeIntellect-ai#3719, PrimeIntellect-ai#3699, PrimeIntellect-ai#3610, PrimeIntellect-ai#3487 (`fsdp_cpu_offload` / `enable_gloo`) Test plan - `pytest tests/unit/orchestrator tests/unit/test_configs.py tests/unit/train/test_cpu_adam.py tests/unit/train/test_state_offload.py -m "not gpu"`: 336 passed, 1 skipped (Qwen3-VL snapshot not cached on this host), 4 deselected (GPU; this host has no GPU). The existing offload config tests are updated to the new field. `test_load_configs` covers the 4 edited tomls. - `cli(TrainerConfig|SFTConfig, ...)`: `--model.offload full|none` and the default (`optimizer`) resolve as expected; `optim_cpu_offload` and `offload = "fsdp"` are rejected. - `import prime_rl.trainer.rl.train, prime_rl.trainer.sft.train` works. - `ruff check` and `ruff format --check` on the changed Python files: clean. - Not run: GPU or multi-node training with `offload = "full"`. <!-- CURSOR_SUMMARY --> --- > [!NOTE] > **Medium Risk** > Breaking TOML/API surface touches distributed training memory paths (FSDP offload, optimizer-in-backward, Gloo); behavior should match prior modes when mapped correctly, but mis-migrated configs will fail at parse time. > > **Overview** > **Breaking config change:** `model.fsdp_cpu_offload`, `model.optim_cpu_offload`, and `model.full_offload` are replaced by a single **`model.offload`** setting (`"optimizer"` default, or `"full"`, `"fsdp"`, `"none"`). RL and SFT share the same field; example TOMLs and scaling docs are updated accordingly. > > The mutual-exclusion validator between the old flags is removed because only one mode can be expressed. **`[model.full_offload]` / `numa_bind` is dropped** — `"full"` always runs NUMA binding and CPU optimizer thread setup via parameterless `setup_full_cpu_optimizer_offload()`. Runtime paths (`setup_optimizer`, FSDP CPU offload in `model.py`, Gloo when offload is `"full"` or `"fsdp"`) key off the new enum; `FullCPUOffloadOptimizer` no longer takes an offload config object. > > Default training behavior is unchanged: optimizer-state CPU offload remains the default (`"optimizer"`). Old TOML keys are not aliased and should fail validation. > > <sup>Reviewed by [Cursor Bugbot](https://cursor.com/bugbot) for commit 35da73a. Bugbot is set up for automated code reviews on this repo. Configure [here](https://www.cursor.com/dashboard/bugbot).</sup> <!-- /CURSOR_SUMMARY --> --------- Co-authored-by: sami jaghouar <sami.jaghouar@gmail.com>
…4011) Qwen3.5 training with context parallelism can fail on the first forward after PrimeIntellect-ai#3925 places compilation inside activation checkpointing. FLA's CP setup reads tensor-valued sequence boundaries and calls `torch.searchsorted`; Inductor fails to specialize the resulting scalar: ``` GuardOnDataDependentSymNode: Could not extract specialized integer from data-dependent expression (u0//4) ``` Wrap `build_cp_context` with `torch.compiler.disable`, alongside the existing eager CP convolution wrapper. CP setup runs eagerly while the surrounding block remains eligible for compilation. CP calculations, model parameters and checkpointing settings are unchanged. Validation ran on an allocated India H200 compute node with PyTorch 2.13.0+cu130 and FLA 0.5.2. A bounded CUDA/FSDP/checkpoint reproducer used two sequence-boundary fixtures and logical CP size 4: | Layout | Result | Maximum absolute output/input-gradient/parameter-gradient difference vs eager | | --- | --- | --- | | Earlier outer compilation | Pass | 0 | | Inner compilation introduced by PrimeIntellect-ai#3925 | Same `(u0//4)` compiler failure | — | | Inner compilation with eager FLA CP metadata | Pass | 1.9073486328125e-6 | This reproducer uses one CUDA/FSDP rank; it does not exercise real four-rank CP collectives or a full Qwen training step. Full distributed training validation remains outstanding. Ruff and `git diff --check` passed for the patch. No test or experiment files are included in this diff. <!-- CURSOR_SUMMARY --> --- > [!NOTE] > **Low Risk** > Narrow trainer workaround for a known torch.compile + FLA CP setup issue; behavior matches eager CP setup with negligible numeric drift in repro. > > **Overview** > Fixes Qwen3.5 **context-parallel (CP)** training failures when the block is compiled inside activation checkpointing (PrimeIntellect-ai#3925). Inductor cannot specialize scalars from FLA’s `build_cp_context` (`torch.searchsorted` on tensor sequence boundaries), which triggers `GuardOnDataDependentSymNode`. > > The change wraps **`build_cp_context`** with **`torch.compiler.disable`** (same pattern as the existing eager CP **`causal_conv1d`** wrapper) and calls **`build_cp_context_eager`** in the forward path when CP is enabled. CP math, parameters, and checkpointing are unchanged; only CP metadata setup runs outside the compiled graph. > > <sup>Reviewed by [Cursor Bugbot](https://cursor.com/bugbot) for commit c192fc9. Bugbot is set up for automated code reviews on this repo. Configure [here](https://www.cursor.com/dashboard/bugbot).</sup> <!-- /CURSOR_SUMMARY -->
# Conflicts: # docs/training.md
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
11 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Merges 37 new commits from
PrimeIntellect-ai/prime-rlmain(through 73140de, PrimeIntellect-ai#4011) into the fork.Notable upstream changes:
trainer.model.freeze_moe_router, feat(trainer): freeze MoE router by default in RL PrimeIntellect-ai/prime-rl#3797)tasks_per_minute→dispatch_per_minute(chore(configs)!: rename tasks_per_minute to dispatch_per_minute PrimeIntellect-ai/prime-rl#3991), a singlemodel.offloadfield for CPU offloading (refactor(configs)!: one model.offload field for CPU offloading PrimeIntellect-ai/prime-rl#3988), andmodel.index_cacheremoved (chore(trainer)!: remove the model.index_cache override PrimeIntellect-ai/prime-rl#3984)GenerationSourceis folded intoTrainEnv(chore(orchestrator): fold GenerationSource into TrainEnv PrimeIntellect-ai/prime-rl#3838)Conflict resolution
docs/training.md: in the[orchestrator.train.algo]row, kept the fork'stemporal_grpoentry and added upstream's newtrainer.model.freeze_moe_routerrow.The
temporal_grpocode merged cleanly.uv run pytest tests/unit/orchestrator/test_temporal_grpo.py tests/unit/orchestrator/test_algorithms.pypasses on the merged tree (37 passed).🤖 Generated with Claude Code