Skip to content

Auto-load processing_class in PRMTrainer when omitted - #6389

Closed
DaoyuanLi2816 wants to merge 1 commit into
huggingface:mainfrom
DaoyuanLi2816:fix/prm-trainer-processing-class-validation-gap
Closed

DaoyuanLi2816 wants to merge 1 commit into
huggingface:mainfrom
DaoyuanLi2816:fix/prm-trainer-processing-class-validation-gap

Conversation

@DaoyuanLi2816

@DaoyuanLi2816 DaoyuanLi2816 commented Jul 15, 2026 •

Copy link
Copy Markdown
Contributor

What does this PR do?

Auto-loads processing_class from the model when it is omitted, matching SFT, DPO, and Reward trainers. This applies with either the default or a custom data collator and for both raw and pre-tokenized datasets.

PRMConfig now exposes trust_remote_code for tokenizer loading.

Tests

  • processing-class autoload coverage for all three dataset/collator paths (3 passed)
  • Ruff check and format check on the changed files

AI writing disclosure

  • AI-assisted: some parts were suggested or improved by AI, but the PR was written and reviewed by a human.

@qgallouedec

Copy link
Copy Markdown
Member

thanks. I think to better align with the other trainer, a better path would be to automatically load the processor in all cases. see how it's done in SFT, DPO etc.

@DaoyuanLi2816 DaoyuanLi2816 changed the title Raise ValueError when PRMTrainer needs processing_class to tokenize the dataset Auto-load processing_class in PRMTrainer when omitted Jul 16, 2026
@DaoyuanLi2816
DaoyuanLi2816 force-pushed the fix/prm-trainer-processing-class-validation-gap branch from 2e137bc to 390547c Compare July 24, 2026 06:55

Copy link
Copy Markdown
Contributor Author

Updated this to auto-load the tokenizer whenever processing_class is omitted, including custom collators and pre-tokenized datasets. Added coverage for all three paths.

@DaoyuanLi2816

Copy link
Copy Markdown
Contributor Author

@qgallouedec this now loads the processor automatically in all cases, as you suggested, rather than only for the collator path — it covers custom collators and pre-tokenized datasets too, with coverage for all three. Ready for another look whenever you have time.

@qgallouedec

Copy link
Copy Markdown
Member

Closing in favour of #7133, which removes trl.experimental.prm. The autoload is right and matches what SFT, DPO and Reward do, there is just no trainer left to carry it. thks for the work on it.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants