[albertvillanova]

This PR makes the trainers warn when a LoRA config would silently not train the dense and shared expert layers of an MoE model, which happens with transformers v5 and peft 0.19 to 0.21.2.

Closes #7767.

Motivation

With transformers v5 and peft 0.19 to 0.21.2, the latest release, target_modules like ["gate_proj", "up_proj", "down_proj"] or "all-linear" only train the routed experts of MoE models that have a dense first layer or shared experts (GLM-4-MoE, DeepSeek-V3, ...): the dense and shared expert layers get no LoRA, with no warning. It is fixed on peft main, but these versions remain in TRL's supported range. See #7767 for the root cause.

Solution

Before creating the PEFT model, the trainers log a warning when all of these hold:

• the config is a LoraConfig, with transformers>=5.0.0 and peft>=0.19.0,<0.21.3.dev0 (peft main, which has the fix, is 0.21.3.dev0)
• the model has fused routed experts (3D parameters under .experts.)
• the model also has nn.Linear layers named like expert projections (gate_proj, up_proj, down_proj, w1, w2, w3), i.e. the dense or shared expert layers
• target_modules would reach them ("all-linear", or a leaf name in that set)

It is a warning rather than an error: the routed experts are still trained, and no target_modules spelling avoids the issue with these versions, so an error would block training these models on the latest PEFT release.

Tests will be added in a follow-up PR, together with those for the #7769 guard. Verified locally with SFT on transformers 5.19.0:

peft	GLM-4-MoE MLP targets	all-linear	attention only	Qwen3-MoE MLP targets	dense model
0.19.1	warning	(fails in peft)	none	none	none
0.20.0, 0.21.2	warning	warning	none	none	none
main (0.21.3.dev0)	none	none	none	none	none

On transformers 4.56.2 with peft 0.21.2, nothing is logged.

Changes

• Log a warning before get_peft_model in the SFT, DPO, KTO, GRPO, RLOO, Reward and Distillation trainers, with the same block in each

---

Note

Low Risk
Adds conditional logging only; no change to training logic or PEFT wrapping behavior.

Overview
Adds a runtime warning immediately before get_peft_model in SFT, DPO, KTO, GRPO, RLOO, Reward, and Distillation trainers when LoRA would silently skip MoE dense/shared MLP layers under transformers v5 and peft 0.19.0–0.21.2 (fixed on peft main ≥ 0.21.3.dev0).

The check runs only for LoraConfig when the model looks like a fused-expert MoE (3D .experts. params plus separate nn.Linear expert projections) and target_modules is "all-linear" or names those projection layers. Training still proceeds; the log points users to install peft from main and links #7767.

Supporting import tweaks: import peft and LoraConfig where needed for version checks (e.g. distillation, reward, sft).

^{Reviewed by Cursor Bugbot for commit d641808. Bugbot is set up for automated code reviews on this repo. Configure here.}

[bot-ci-comment]

The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

[qgallouedec]

Thanks, LGTM. Checked on tiny GLM-4-MoE with peft 0.21.0: it warns for both the MLP targets and all-linear, and stays silent on Qwen2. @BenjaminBossan any chance of a patch release with huggingface/peft#3715 and huggingface/peft#3946?

[BenjaminBossan]

@qgallouedec How important would that be? We still have those FSDP/TP PRs open, it would be nice to have them merged first, but if these other two PRs are blocking, we can do a patch release beforehand.

[albertvillanova]

From the TRL side it's not blocking: with these PRs, TRL users on peft 0.19 to 0.21.2 get a warning, and the fix is on main. A patch release would let them train the dense and shared expert layers without installing peft from main, so it would be nice to have, but waiting for the FSDP/TP PRs is fine.

[albertvillanova]

Removed the routed experts guard that this branch still carried from #7769 (d641808), so this PR now only adds the warning. The guard stays in #7769.

[BenjaminBossan]

From the TRL side it's not blocking: with these PRs, TRL users on peft 0.19 to 0.21.2 get a warning, and the fix is on main. A patch release would let them train the dense and shared expert layers without installing peft from main, so it would be nice to have, but waiting for the FSDP/TP PRs is fine.

Got it, thanks. Let's see if we get the TP/FSDP PRs in soon. Don't hesitate to ping me if PEFT becomes the blocker on something important.