In plain words: EvoMerge treats model improvement like evolution: blending two trained models mixes their strengths, while extra fine-tuning makes small tweaks. Unlike plain fine-tuning, which can sharpen one skill while dulling others, it aims to raise reasoning without losing general ability.
Abstract
Extensive fine-tuning on Large Language Models does not always yield better results. Oftentimes, models tend to get better at imitating one form of data without gaining greater reasoning ability and may even end up losing some intelligence. Here I introduce EvoMerge, a systematic approach to large language model training and merging. Leveraging model merging for weight crossover and fine-tuning for weight mutation, EvoMerge establishes an evolutionary process aimed at pushing models beyond the limits of conventional fine-tuning.
Yushu Jiang
arXiv:2402.00070 · cs.NE, cs.AI, cs.CL, cs.LG · submitted Jan 30, 2024
abstract · pdf · html · The current submission is the first draft, published for the sole purpose of sharing an idea and encouraging community effort. A more consolidated version may come later
