In plain words: It blends several models fine-tuned from the same starting point: randomly delete most of each model's small tweaks, scale up the rest, and fuse them with no retraining. The combined model can beat every source model, and one version ranked first among 7-billion-parameter models.
Abstract · Language Models are Super Mario: Absorbing Abilities from Homologous Models as a Free Lunch
In this paper, we unveil that Language Models (LMs) can acquire new capabilities by assimilating parameters from homologous models without retraining or GPUs. We first introduce DARE to set most delta parameters (i.e., the disparity between fine-tuned and pre-trained parameters) to zeros without affecting the abilities of Supervised Fine-Tuning (SFT) LMs, which randomly Drops delta parameters with a ratio $p$ And REscales the remaining ones by $1 / (1 - p)$ to approximate the original embeddings. Then, we use DARE as a versatile plug-in to sparsify delta parameters of multiple SFT homologous models for mitigating parameter interference and merge them into a single model by parameter fusing. We experiment with encoder- and decoder-based LMs, showing that: (1) SFT delta parameter value ranges are typically small (within 0.002) with extreme redundancy, and DARE can effortlessly eliminate 90% or even 99% of them; (2) DARE can merge multiple task-specific LMs into one LM with diverse capabilities. Notably, this phenomenon is more pronounced in large-scale LMs, where the merged LM reveals the potential to surpass the performance of any source LM, providing a new discovery. We also utilize DARE to create a merged LM that ranks first among models with 7 billion parameters on the Open LLM Leaderboard.
Le Yu, Bowen Yu, Haiyang Yu, Fei Huang, Yongbin Li
arXiv:2311.03099 · cs.CL, cs.LG · submitted Nov 6, 2023 · updated Jun 13, 2024
abstract · pdf · html · Accepted at ICML 2024
The drop and rescale method outlined in the paper makes the latent space increasingly sparse, which in turn allows weights to merged without much interference or degradation.
My instinct is that while merging models will have some use cases, ultimately these insights will lead to innovations in training and architecture that have the same result but with better computational efficiency.
For example instead of training an 8x7b mixture of experts then merging, just incorporate the sparsity constraint while pre-training a single 7b model (somehow).