[snimu]

Stacked on #4062 (pretrain-core); merge after it.

Log-likelihood evals ([evals]) for uv run pretrain: HellaSwag, ARC-Easy/Challenge, PIQA, WinoGrande, OpenBookQA, SciQ and LAMBADA in cloze form, formatted as in lm-evaluation-harness and scored inside the trainer (trainer/pretrain/evals/{tasks,loglik}.py). Each task logs acc, acc_norm (length-normalized by the answer without its leading space, as the harness does) and bpb of the gold continuation, a smooth signal while accuracy is still at chance; LAMBADA logs only bpb. evals.limit caps items per task. They run after the optimizer update, like the perplexity evals, and the launcher prefetches the task datasets.

Validation

• Pretrained Llama-3.2-1B, zero-shot, full sets, in-trainer vs lm-evaluation-harness:

Task	metric	ours, 1 GPU	ours, 8 GPUs	lm-eval
hellaswag	acc / acc_norm	0.481 / 0.642	0.481 / 0.642	0.481 / 0.643
arc_easy	acc / acc_norm	0.666 / 0.620	0.662 / 0.620	0.663 / 0.617
arc_challenge	acc / acc_norm	0.317 / 0.368	0.313 / 0.370	0.312 / 0.369
piqa	acc / acc_norm	0.751 / 0.747	0.752 / 0.747	0.754 / 0.748
winogrande	acc	0.608	0.613	0.612
openbookqa	acc / acc_norm	0.276 / 0.374	0.278 / 0.374	0.280 / 0.372
sciq	acc / acc_norm	0.934 / 0.912	0.932 / 0.913	0.933 / 0.912

1-GPU (RTX PRO 6000) and 8-GPU (H200) bpb agree to the fourth decimal; accuracies differ by up to 0.4 points (near-tie choices).

• Unit tests for request packing and scoring (tests/unit/train/pretrain/test_evals.py).

[snimu]

Superseded by #4091: task evals now run in the asynchronous eval worker on model snapshots (fixed eval context, prime-rl model code, logs into the training W&B run). The in-trainer part (held-out loss/bpb through the training loss path) is in #4062. Branch kept for the record.