In plain words: It turns any text into one vector by training on many automatically matched text pairs, pulling related texts closer and pushing unrelated ones apart. With no labeled data it beat the classic keyword-matching search baseline, and after fine-tuning it beat models 40 times larger.
Abstract · Text Embeddings by Weakly-Supervised Contrastive Pre-training
This paper presents E5, a family of state-of-the-art text embeddings that transfer well to a wide range of tasks. The model is trained in a contrastive manner with weak supervision signals from our curated large-scale text pair dataset (called CCPairs). E5 can be readily used as a general-purpose embedding model for any tasks requiring a single-vector representation of texts such as retrieval, clustering, and classification, achieving strong performance in both zero-shot and fine-tuned settings. We conduct extensive evaluations on 56 datasets from the BEIR and MTEB benchmarks. For zero-shot settings, E5 is the first model that outperforms the strong BM25 baseline on the BEIR retrieval benchmark without using any labeled data. When fine-tuned, E5 obtains the best results on the MTEB benchmark, beating existing embedding models with 40x more parameters.
Liang Wang, Nan Yang, Xiaolong Huang, Binxing Jiao, Linjun Yang, Daxin Jiang, Rangan Majumder, Furu Wei
arXiv:2212.03533 · cs.CL, cs.IR · submitted Dec 7, 2022 · updated Feb 22, 2024
abstract · pdf · html · 17 pages, v2 fixes the SummEval numbers