Density Weighting for Multi-Interest Personalized Recommendation

no code implementations • 3 Aug 2023 • Nikhil Mehta, Anima Singh, Xinyang Yi, Sagar Jain, Lichan Hong, Ed H. Chi

When the data distribution is highly skewed, the gains observed by learning multiple representations diminish since the model dominates on head items/interests, leading to poor performance on tail items.

Recommendation Systems

Better Generalization with Semantic IDs: A Case Study in Ranking for Recommendations

no code implementations • 13 Jun 2023 • Anima Singh, Trung Vu, Nikhil Mehta, Raghunandan Keshavan, Maheswaran Sathiamoorthy, Yilin Zheng, Lichan Hong, Lukasz Heldt, Li Wei, Devansh Tandon, Ed H. Chi, Xinyang Yi

To strike a good balance of memorization and generalization, we propose to use Semantic IDs -- a compact discrete item representation learned from frozen content embeddings using RQ-VAE that captures the hierarchy of concepts in items -- as a replacement for random item ids.

Memorization Recommendation Systems

Understanding and Improving Knowledge Distillation

no code implementations • 10 Feb 2020 • Jiaxi Tang, Rakesh Shivanna, Zhe Zhao, Dong Lin, Anima Singh, Ed H. Chi, Sagar Jain

Knowledge Distillation (KD) is a model-agnostic technique to improve model quality while having a fixed capacity budget.

Knowledge Distillation Model Compression