about
Pixie: System for Recommending 3B Items to 200M Users in RealTime (arxiv.org)
2 points by phab on Jul 14, 2020 | hide | past | pdf | discuss on HN

In plain words: Pixie walks randomly across a giant graph of pins and users, starting from the pins someone has saved, to find related pins instantly instead of precomputing lists in batches. It lifted user engagement by up to 50% over the old batch system.

Abstract · Pixie: A System for Recommending 3+ Billion Items to 200+ Million Users in Real-Time

User experience in modern content discovery applications critically depends on high-quality personalized recommendations. However, building systems that provide such recommendations presents a major challenge due to a massive pool of items, a large number of users, and requirements for recommendations to be responsive to user actions and generated on demand in real-time. Here we present Pixie, a scalable graph-based real-time recommender system that we developed and deployed at Pinterest. Given a set of user-specific pins as a query, Pixie selects in real-time from billions of possible pins those that are most related to the query. To generate recommendations, we develop Pixie Random Walk algorithm that utilizes the Pinterest object graph of 3 billion nodes and 17 billion edges. Experiments show that recommendations provided by Pixie lead up to 50% higher user engagement when compared to the previous Hadoop-based production system. Furthermore, we develop a graph pruning strategy at that leads to an additional 58% improvement in recommendations. Last, we discuss system aspects of Pixie, where a single server executes 1,200 recommendation requests per second with 60 millisecond latency. Today, systems backed by Pixie contribute to more than 80% of all user engagement on Pinterest.

Chantat Eksombatchai, Pranav Jindal, Jerry Zitao Liu, Yuchen Liu, Rahul Sharma, Charles Sugnet, Mark Ulrich, Jure Leskovec
arXiv:1711.07601 · cs.IR, cs.LG, cs.PF, cs.SI · submitted Nov 21, 2017
abstract · pdf · html

add comment on HN