The headline result has a caveat
The authors report average improvements of 9.43% in Recall@20 and 14.80% in NDCG@20 against the strongest baseline on each dataset. The comparison covers six real-world datasets, Amazon, Google, Gowalla, Yelp, ML10M and Netflix, using top-20 recommendation scores as its main headline measures.
That overall result has a clear exception. On Google, Mult-BiW's Recall@20 was 0.2122, below CPR's 0.2205, while its NDCG@20 was 0.1233 versus CPR's 0.1235. The reported relative differences were -3.76% and -0.14%, so the strongest baseline was ahead on both listed top-20 measures.
A three-part approach to popularity bias
Mult-BiW combines three pieces. Mult-IPS handles global preferences through an IPS-weighted multinomial learning objective. Bi-Weighting, or BiW, smooths propensity estimates with a collection model, while Progressive Bi-Weighting, or PBiW, moves gradually toward popularity debiasing during training.
The computable BiW estimate blends an individual user-item propensity estimate with that user's average estimated propensity across all items. PBiW then schedules the smoothing coefficient over training epochs. The paper describes this schedule as a progressive shift from representation learning toward debiasing.
The evidence comes from offline benchmarks
The benchmark used long-tailed training data and balanced validation and test data in a 70%/10%/20% split. ML10M and Netflix were not perfectly balanced. Before testing, the paper applied 20-core preprocessing, retaining users and items with at least 20 interactions, across the six main datasets.
The primary offline endpoints were Recall@10, NDCG@10, Recall@20 and NDCG@20. Matrix factorization was the default model backbone, with additional LightGCN tests for methods using inverse-propensity weighting.
For the main comparisons, the authors used five repeated runs and a two-sample t-test for marked comparisons, with a reported p-value threshold below 0.01. The supplied analysis notes that confidence intervals were not reported.
The gains are not evenly spread
The component tests favored the multinomial likelihood across datasets. Pairwise loss was generally second-best, while pointwise losses performed worse. Both fixed and progressive Bi-Weighting outperformed the compared IPS and CIPS versions, with PBiW performing best in most cases.
The popularity analysis also pointed to a trade-off. The authors report lower popularity-frequency correlation for Mult-BiW than for CPR and UBPR, alongside larger gains for tail and less-popular items. Head-item accuracy was moderately lower, suggesting that the method's balance came with some cost at the most popular end.
Training time was another reported advantage under matrix factorization. Mult-BiW was faster than CPR on four datasets and had comparable training time on ML10M and Netflix.
What the tests cannot answer
These are offline benchmark findings, not a user-level deployment study. Recall, NDCG, popularity-frequency curves and Pearson correlations are indirect outcomes rather than direct measures of user satisfaction or broad fairness. The main evidence also comes from six datasets and a limited set of model backbones.
The theoretical analysis does not remove the need for accurate propensity estimates. It provides an assumption-dependent upper bound on empirical Mult-IPS bias caused by inaccurate propensities, under bounded local loss and a positive lower bound on BiW propensity estimates. That is a formal bound under stated conditions, not an empirically estimated effect size from the benchmark.
The manuscript is labeled arXiv:2609.35041v1 and dated 28 September 2026. It also presents an ACM journal citation with a placeholder DOI. The paper states that its implementation and preprocessed datasets are publicly available on GitHub, although that availability was not independently verified in the supplied review.
Paper data and sources
Original title: Mitigating Popularity Bias in Recommendation with Global Listwise Learning and Progressive Bi-Weighting
Authors: Tianyu Zhu, Jiandong Ding, Yansong Shi et al.
Journal/Repository: arXiv
Status: Preprint, not yet peer-reviewed
First online: 2026-09-28
DOI: Not available
Original paper · Full text