Issues

• SFTTrainer drops document boundaries when only eval_packing is enabled

  huggingface/trl#7788 · Filter by author mottopanikeiku opened on Oct 18, 2026

• GRPOTrainer: resuming from a checkpoint inside a generation block shifts the prompt schedule (same code in RLOOTrainer)

  huggingface/trl#7787 · Filter by author TarangKhanna opened on Oct 18, 2026

• AsyncGRPO/AsyncDistillation merged-weight sync resumes vLLM before publishing the new rollout version

  huggingface/trl#7782 · Filter by author lindicaphxag-tech opened on Oct 17, 2026
  1

• Async OPD rollout can mislabel the sampling policy version during LoRA weight sync

  huggingface/trl#7781 · Filter by author lindicaphxag-tech opened on Oct 17, 2026
  1

• Normalize async distillation over tokens with usable teacher support

  huggingface/trl#7780 · Filter by author lindicaphxag-tech opened on Oct 17, 2026
  5

• QLoRA with fp16=True crashes: trainers cast the adapters to bf16 unconditionally

  huggingface/trl#7776 · Filter by author qgallouedec opened on Oct 16, 2026
  1

• SFTTrainer: chat_template_path crashes with vision-language models

  huggingface/trl#7775 · Filter by author qgallouedec opened on Oct 16, 2026
  1

• VLM training with vLLM in server mode ignores the images for Gemma3

  huggingface/trl#7772 · Filter by author albertvillanova opened on Oct 16, 2026
  

• LoRA on MoE models with shared experts does not train the dense and shared expert layers with transformers v5 and peft 0.19 to 0.21.2

  huggingface/trl#7767 · Filter by author albertvillanova opened on Oct 16, 2026
  1

  

• AsyncGRPO: TokenBudgetBatcher closes a micro-batch even when another row has free room

  huggingface/trl#7766 · Filter by author clementtisseau opened on Oct 16, 2026
  1

  

• LoRA on MoE models does not train the routed experts with transformers v5 and peft<0.19

  huggingface/trl#7765 · Filter by author albertvillanova opened on Oct 16, 2026
  1

  

• [Docs] Docs for Async GRPO and the Harbor example still use Qwen3.5, which will crash on startup

  huggingface/trl#7761 · Filter by author shawnghu opened on Oct 16, 2026
  1

• Add an AgentEnv integration: trl.experimental.agentenv

  huggingface/trl#7759 · Filter by author earakely-scale opened on Oct 15, 2026

• Proposal: audit zero-advantage group filtering under imperfect rewards

  huggingface/trl#7747 · Filter by author GBX-Max1220 opened on Oct 15, 2026
  1

• Agree on when to add, drop and change the default Python version

  huggingface/trl#7742 · Filter by author albertvillanova opened on Oct 15, 2026
  

• Add an AMD GPU runner to CI

  huggingface/trl#7735 · Filter by author qgallouedec opened on Oct 14, 2026
  1

• DiffusionGemma SFT example evicts sampled-block prefix from sliding-window KV cache

  huggingface/trl#7734 · Filter by author taking-lying-flat opened on Oct 14, 2026
  5

• GRPO/RLOO on Trainium: each new prompt length bucket recompiles generation for ~12 minutes

  huggingface/trl#7733 · Filter by author qgallouedec opened on Oct 14, 2026

• [RFC] Document torchrun as the default launcher

  huggingface/trl#7732 · Filter by author qgallouedec opened on Oct 14, 2026

• Agree on when to move the vLLM supported range

  huggingface/trl#7722 · Filter by author albertvillanova opened on Oct 14, 2026
  

• Saving a checkpoint fails on torch 2.6 when generating the model card: UnicodeDecodeError: 'ascii' codec can't decode byte 0xc3

  huggingface/trl#7720 · Filter by author albertvillanova opened on Oct 14, 2026
  3

  

• import trl computes importlib.metadata.packages_distributions() eagerly (1-3 s with torch >= 2.14 on Python 3.12)

  huggingface/trl#7716 · Filter by author MakazhanAlpamys opened on Oct 14, 2026
  3

• CheckpointError and wrong gradients in later training tests after a training error on the same test worker

  huggingface/trl#7714 · Filter by author albertvillanova opened on Oct 14, 2026
  1

  

• Proposing to remove TPOTrainer

  huggingface/trl#7709 · Filter by author qgallouedec opened on Oct 13, 2026

• Iterable datasets with at least as many shards as processes skip and duplicate prompts in multi-process GRPO/RLOO/Distillation/GOLD training

  huggingface/trl#7690 · Filter by author Strongich opened on Oct 13, 2026
  1