Method comparison experiments

Overview of PEFT method comparison experiments

Here we collect experiments for the PEFT method comparison benchmarks to reference later and avoid duplicate work. For a dataset of experiment results, consult https://huggingface.co/datasets/peft-internal-testing/benchmark-graveyard. Many experiments were discussed in this thread but there might be others.

MetaMathQA benchmark

https://github.com/huggingface/peft/tree/main/method_comparison/MetaMathQA

Adaption prompt

• https://github.com/huggingface/peft/pull/3301

IA3

• https://github.com/huggingface/peft/pull/3337

P-tuning

• https://github.com/huggingface/peft/pull/3356

PVeRA

• https://github.com/huggingface/peft/pull/3256

Image generation benchmark

https://github.com/huggingface/peft/tree/main/method_comparison/image-gen

LoRA

• LoRA + DoRA: https://github.com/huggingface/peft/pull/3341