Developments·Paper
Yandex Music's generative recommender Sona showed no plateau with model size or training data in tested ranges
Researchers released a tech report (arXiv 2608.11015) on Sona, a generative music recommender in A/B testing at Yandex Music. Scaling sweeps showed training loss and recall metrics kept improving with both model size (20M–260M parameters) and training-data window (1–8 weeks), with no plateau in tested ranges. A distilled Teacher Ranker trained on a year of logs delivered +4.53% Active Users over the production cascade in a 7-day smart-speaker A/B test.
7 engagement2 sourcesFri, Oct 2, 2026, 01:59 PM
AI Overview
- Yandex Music's Sona generative recommender scaled from 20M to 260M parameters with training loss still dropping at matched exposure
- 130M model grew from 1 to 8 weeks of data, lifting Recall@10 from 0.1798 to 0.2519, still rising at 8 weeks
- History Compression running the deep stack on the latest 2,048 events reached 0.8709 Recall@1000 at about half inference cost
- A distilled Teacher Ranker trained on a year of logs gained +4.53% Active Users over the production cascade in a 7-day smart-speaker A/B test
- Next steps include sparse mixture-of-experts backbones and sparse or linear attention; optimal use of extra inference compute stays open
Related
Developments·Paper
Paper proposes dynamic single-level large semantic codebook for generative recommendation
A new arXiv paper proposes replacing multi-level residual quantization in generative recommendation with a single-level large semantic codebook, aiming to reduce decoding cost and adapt to shifting item distributions. The approach retains a separate collaborative dimension to preserve user-item interaction signals.
A new arXiv paper proposes replacing multi-level residual quantization in generative recommendation with a single-level large semantic codebook, aiming to reduce decoding cost and adapt to shifting item distributions. The approach retains a separate collaborative dimension to preserve user-item interaction signals.
0 engagement1 sourcearxivFri, Aug 21, 2026, 11:58 AM
Developments·Paper
Kuaishou-affiliated researchers propose ReST, a recommendation-native Transformer scaling framework for industrial ranking
An arXiv paper introduces ReST, a framework for scaling sequence Transformers in industrial recommendation ranking. It addresses two obstacles unique to recommendation: noisy, sparsely supervised behavior sequences and tight latency budgets when scoring many candidates against one shared user history. The paper matters to practitioners building large-scale ranking systems where NLP-style Transformer scaling does not transfer directly.
An arXiv paper introduces ReST, a framework for scaling sequence Transformers in industrial recommendation ranking. It addresses two obstacles unique to recommendation: noisy, sparsely supervised behavior sequences and tight latency budgets when scoring many candidates against one shared user history. The paper matters to practitioners building large-scale ranking systems where NLP-style Transformer scaling does not transfer directly.
0 engagement1 sourcearxivTue, Sep 1, 2026, 01:40 PM
Developments·Paper
SequenceO1: End-to-End 100K-Sequence Recommendation Model Deployed at Full Traffic on Douyin
An arxiv paper introduces SequenceO1, an end-to-end approach for modeling ultra-long (100K) user behavior sequences in sequential recommendation. It targets production ranking models that face strict latency, memory, communication, and training-throughput constraints at billion scale.
An arxiv paper introduces SequenceO1, an end-to-end approach for modeling ultra-long (100K) user behavior sequences in sequential recommendation. It targets production ranking models that face strict latency, memory, communication, and training-throughput constraints at billion scale.
0 engagement1 sourcearxivTue, Sep 8, 2026, 08:49 AM
Developments·Paper
Arxiv paper studies optimal model scaling under limited pre-training data
An arxiv paper examines pre-training when data, not compute, is the constraint. Across 10M-100M word budgets, two corpora, and multiple downstream evaluations, it finds parameter count helps only up to an optimal scale before overfitting degrades generalization, and argues standard Transformers scale down poorly because embeddings consume a large share of parameters. The work matters to practitioners training models on proprietary or small datasets.
An arxiv paper examines pre-training when data, not compute, is the constraint. Across 10M-100M word budgets, two corpora, and multiple downstream evaluations, it finds parameter count helps only up to an optimal scale before overfitting degrades generalization, and argues standard Transformers scale down poorly because embeddings consume a large share of parameters. The work matters to practitioners training models on proprietary or small datasets.
0 engagement1 sourcearxivThu, Aug 27, 2026, 11:18 AM
Developments·Paper
Researchers propose selection rule for batch size vs. negatives in recommender training
A new arXiv paper addresses the memory trade-off in training large-scale neural recommenders with sampled softmax, asking whether to prioritize larger batches or more negative items under a fixed memory budget. The authors propose a selection rule to guide this choice for memory-constrained training.
A new arXiv paper addresses the memory trade-off in training large-scale neural recommenders with sampled softmax, asking whether to prioritize larger batches or more negative items under a fixed memory budget. The authors propose a selection rule to guide this choice for memory-constrained training.
0 engagement1 sourcearxivTue, Aug 11, 2026, 03:29 PM