🤗 PEFT welcomes new merging methods

Hugging Face Blog · 2 years ago

Hugging Face PEFT released new methods for merging LoRA adapters, including concatenation, linear arithmetic, SVD, TIES, DARE, and magnitude pruning techniques. The library enables on-the-fly merging of multiple adapter checkpoints from the same base model without downloading separate model files. The merged adapters combine capabilities of individual adapters—such as merging Hinglish and mental health adapters to handle queries in both languages with relevant advice.