about
Blending Is All You Need – Cheaper and Better to Trillion-Param LLM (arxiv.org)
5 points by ashmitgg on Jan 10, 2024 | hide | past | pdf | 1 comment on HN

In plain words: Instead of one giant chat model, this blends three smaller ones and combines their replies into a single answer. In 30 days of real user testing, three models with 6–13 billion learned settings matched or beat a 175-billion-plus model like ChatGPT.

Abstract · Blending Is All You Need: Cheaper, Better Alternative to Trillion-Parameters LLM

In conversational AI research, there's a noticeable trend towards developing models with a larger number of parameters, exemplified by models like ChatGPT. While these expansive models tend to generate increasingly better chat responses, they demand significant computational resources and memory. This study explores a pertinent question: Can a combination of smaller models collaboratively achieve comparable or enhanced performance relative to a singular large model? We introduce an approach termed "blending", a straightforward yet effective method of integrating multiple chat AIs. Our empirical evidence suggests that when specific smaller models are synergistically blended, they can potentially outperform or match the capabilities of much larger counterparts. For instance, integrating just three models of moderate size (6B/13B paramaeters) can rival or even surpass the performance metrics of a substantially larger model like ChatGPT (175B+ paramaters). This hypothesis is rigorously tested using A/B testing methodologies with a large user base on the Chai research platform over a span of thirty days. The findings underscore the potential of the "blending" strategy as a viable approach for enhancing chat AI efficacy without a corresponding surge in computational demands.

Xiaoding Lu, Zongyi Liu, Adian Liusie, Vyas Raina, Vineet Mudupalli, Yuwen Zhang, William Beauchamp
arXiv:2401.02994 · cs.CL, cs.AI · submitted Jan 4, 2024 · updated Jan 23, 2024
abstract · pdf · html

add comment on HN
Also discussed: Jan 2024 (147 points, 45 comments)

yuck

> In this work, we introduce Blended, an innovative and simple approach where we demonstrate that, surprisingly, if responses are selected randomly from a group of base chat AIs, the resulting combined chat AI is highly capable and engaging, and can outperform systems with orders of magnitude more parameters.

> To assess the ’quality’ of a chat AI, we consider two main proxy functions: the industry standard user retention and the main objective function, user engagement.

.... is all you need, is all you need to write a paper i guess