In plain words: A family of five open models, from 22 to 334 million parameters, that turn text into vectors for search, trained with a new data recipe. Each was the most accurate retriever for its size at release, and the largest beat OpenAI's and Cohere's models.
Abstract
This report describes the training dataset creation and recipe behind the family of \texttt{arctic-embed} text embedding models (a set of five models ranging from 22 to 334 million parameters with weights open-sourced under an Apache-2 license). At the time of their release, each model achieved state-of-the-art retrieval accuracy for models of their size on the MTEB Retrieval leaderboard, with the largest model, arctic-embed-l outperforming closed source embedding models such as Cohere's embed-v3 and Open AI's text-embed-3-large. In addition to the details of our training recipe, we have provided several informative ablation studies, which we believe are the cause of our model performance.
Luke Merrick, Danmei Xu, Gaurav Nuti, Daniel Campos
arXiv:2405.05374 · cs.CL, cs.AI, cs.IR · submitted May 8, 2024
abstract · pdf · html · 17 pages, 11 Figures, 9 tables