about
Limits of predictability in the top-N recommendation problem (arxiv.org)
2 points by the88doctor on Mar 24, 2023 | hide | past | pdf | 1 comment on HN

In plain words: They find how predictable a shopper's N most likely next purchases are by spotting patterns in past buying and applying a math rule that caps how accurate a guess can be. Their estimate lands much closer to the real ceiling than earlier ones.

Abstract · Limits of Predictability in Top-N Recommendation

Top-N recommendation aims to recommend each consumer a small set of N items from a large collection of items, and its accuracy is one of the most common indexes to evaluate the performance of a recommendation system. While a large number of algorithms are proposed to push the Top-N accuracy by learning the user preference from their history purchase data, a predictability question is naturally raised - whether there is an upper limit of such Top-N accuracy. This work investigates such predictability by studying the degree of regularity from a specific set of user behavior data. Quantifying the predictability of Top-N recommendations requires simultaneously quantifying the limits on the accuracy of the N behaviors with the highest probability. This greatly increases the difficulty of the problem. To achieve this, we firstly excavate the associations among N behaviors with the highest probability and describe the user behavior distribution based on the information theory. Then, we adopt the Fano inequality to scale and obtain the Top-N predictability. Extensive experiments are conducted on the real-world data where significant improvements are observed compared to the state-of-the-art methods. We have not only completed the predictability calculation for N targets but also obtained predictability that is much closer to the true value than existing methods. We expect our results to assist these research areas where the quantitative requirement of Top-N predictability is required.

En Xu, Zhiwen Yu, Ying Zhang, Bin Guo, Lina Yao
arXiv:2303.13091 · cs.IR, cs.IT · submitted Mar 23, 2023
abstract · pdf · html

add comment on HN

it's almost weekend, you should just take a rest from unvelling the future, just a bit.