Repository navigation
Auto-load processing_class in PRMTrainer when omitted - #6389
DaoyuanLi2816 wants to merge 1 commit into
Conversation
|
thanks. I think to better align with the other trainer, a better path would be to automatically load the processor in all cases. see how it's done in SFT, DPO etc. |
processing_class in PRMTrainer when omitted
2e137bc to
390547c
Compare
|
Updated this to auto-load the tokenizer whenever |
|
@qgallouedec this now loads the processor automatically in all cases, as you suggested, rather than only for the collator path — it covers custom collators and pre-tokenized datasets too, with coverage for all three. Ready for another look whenever you have time. |
|
Closing in favour of #7133, which removes |
What does this PR do?
Auto-loads
processing_classfrom the model when it is omitted, matching SFT, DPO, and Reward trainers. This applies with either the default or a custom data collator and for both raw and pre-tokenized datasets.PRMConfignow exposestrust_remote_codefor tokenizer loading.Tests
AI writing disclosure