In plain words: A recommendation system that trains continuously instead of in batches, giving every user and item its own slot in a lookup table so entries never clash, and dropping old or rare ones to save memory. It runs in BytePlus's product, learning from clicks fast.
Abstract · Monolith: Real Time Recommendation System With Collisionless Embedding Table
Building a scalable and real-time recommendation system is vital for many businesses driven by time-sensitive customer feedback, such as short-videos ranking or online ads. Despite the ubiquitous adoption of production-scale deep learning frameworks like TensorFlow or PyTorch, these general-purpose frameworks fall short of business demands in recommendation scenarios for various reasons: on one hand, tweaking systems based on static parameters and dense computations for recommendation with dynamic and sparse features is detrimental to model quality; on the other hand, such frameworks are designed with batch-training stage and serving stage completely separated, preventing the model from interacting with customer feedback in real-time. These issues led us to reexamine traditional approaches and explore radically different design choices. In this paper, we present Monolith, a system tailored for online training. Our design has been driven by observations of our application workloads and production environment that reflects a marked departure from other recommendations systems. Our contributions are manifold: first, we crafted a collisionless embedding table with optimizations such as expirable embeddings and frequency filtering to reduce its memory footprint; second, we provide an production-ready online training architecture with high fault-tolerance; finally, we proved that system reliability could be traded-off for real-time learning. Monolith has successfully landed in the BytePlus Recommend product.
Zhuoran Liu, Leqi Zou, Xuan Zou, Caihua Wang, Biao Zhang, Da Tang, Bolin Zhu, Yijie Zhu, Peng Wu, Ke Wang, Youlong Cheng
arXiv:2209.07663 · cs.IR · submitted Sep 16, 2022 · updated Sep 27, 2022
abstract · pdf · html · ORSUM@ACM RecSys 2022
A collisionless embedding table is a type of data structure that is used to store information in a way that minimizes the chances of two pieces of information colliding or conflicting with each other. This is often done by using a hashing algorithm to map data to specific locations in the table, which reduces the likelihood of two pieces of data being stored in the same location.
Embeddings are a way of representing data in a lower-dimensional space. In this case, the embedding is used to represent the data from the user's interactions. This can be used to make predictions about relationships between vectors.
The advantage of storing the dot products of the embeddings would be that it would allow for a more efficient calculation of the similarity between two vectors. Dot products are a measure of how similar two vectors are, so by storing the dot products of the embeddings, it would be possible to quickly calculate the similarity between any two vectors. This would be especially useful in a recommendation system, where it is often necessary to calculate the similarity between a user's vector and a large number of other vectors in order to find the most similar items.