Remove the experimental Nash-MD trainer

[albertvillanova]

This PR removes trl.experimental.nash_md (NashMDTrainer, NashMDConfig), added in #1853 and moved to trl.experimental in #4477.

Motivation

Reporting the inputs discussed in #7182:

• Usage. 16 non-CPU users over the last two months. One model has ever carried both the nash-md and trl tags, and none was created this year. Nobody has spoken up for it in [RFC] Criteria for removing code from trl.experimental #7182.
• External issues and pull requests. Seven issues and seven pull requests from outside the maintainer team in the last twelve months, none of them describing a training run. Four of the issues were filed on a single day by one account and are two distinct problems worded twice, and the pull requests are drive-by fixes that sweep Online DPO, XPO and Nash-MD together.
• Subsumed. No, and this is the one input pointing the other way: no stable trainer covers Nash learning from human feedback, so this removes a capability rather than consolidating one.
• Cost. 580 lines of trainer and config, plus GeometricMixtureWrapper, which has no other consumer.
• Downstream consumers. None. NashMDTrainer is imported by no major training framework.
• Owner. None has come forward.
• Age. Available since 2024-09, in trl.experimental since 2025-11-13, deprecated import path dropped 2026-02-10. Long enough to have been found.

Solution

trl.experimental carries no stability guarantee, so there is no deprecation cycle. The code stays in the git history.

The paper index entry stays and points at the last release that shipped the implementation, following the discussion in #7182.

Changes

• Remove NashMDTrainer, NashMDConfig and their tests
• Delete the Nash-MD trainer doc page and drop it from the toctree, the trainer index, the dataset format table and the vLLM integration page
• Drop NashMDTrainer from the telemetry allowlist
• Keep the Nash-MD paper index entry, pointing at the last release shipping the trainer

---

Note

Medium Risk
This is a breaking removal of a public experimental API (trl.experimental.nash_md); any code still importing it will fail at import time, though usage was reportedly minimal.

Overview
Removes the experimental Nash-MD trainer (NashMDTrainer, NashMDConfig, and GeometricMixtureWrapper) from trl.experimental.nash_md, along with its test suite (~580 lines of trainer/config code).

Documentation is updated to match: the dedicated trainer page is deleted, and Nash-MD is dropped from the docs toctree, trainer index, dataset-format compatibility table, and vLLM integration examples. The paper index entry for Nash Learning from Human Feedback is kept but now states the implementation shipped only through TRL v1.13 and lives in git history.

NashMDTrainer is also removed from the telemetry trainer allowlist in base_trainer.py. There is no deprecation period because trl.experimental has no stability guarantee.

^{Reviewed by Cursor Bugbot for commit 39a9e10. Bugbot is set up for automated code reviews on this repo. Configure here.}

[bot-ci-comment]

The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

[qgallouedec]

LGTM.

Closing after both this one and #7256:

• #6951
• #6952
• #6805
• #6800
• #6806
• #6809

Shrink to Online DPO:

• #6861
• #6950
• #7045
• #6862

[albertvillanova]

Went through the list against main.

Closed three of the six: #6951, #6952 and #6805. Also closed #6821, whose every file was deleted by #7132. The two issues went as "not planned", since the code was removed rather than the bugs fixed.

I left the other three open, because each still reproduces in a trainer we kept.