A recommendation-list evaluator called SWIM was the best performer in the paper's offline comparisons on two public datasets, while a Kuaishou main-feed test reported positive relative lifts in app stay time and Retention (LT7). The reported lifts were 0.351% for app stay time and 0.048% for Retention (LT7).
The online figures are differences against a baseline, not absolute levels of minutes watched or users retained. They describe the reported gap, but do not by themselves establish that SWIM caused the change. The 95% confidence intervals were [0.27%, 0.42%] for app stay time and [0.01%, 0.09%] for Retention (LT7).
A model of the session, not just the request
SWIM is built around the idea that a recommendation list is consumed step by step. It treats ongoing consumption as a finite-horizon, session-level survival process and breaks the value of a list into two pieces: the probability that a user reaches each position and the reward associated with the item at that reached position. The approach lets the evaluator account for the fact that later items matter only if the session continues that far.
To represent that history, the evaluator uses a causal-masked Transformer, a model that encodes prefix-aware list representations while restricting each step to information available from the preceding context. The paper's theoretical discussion says a request-only evaluator cannot eliminate prediction error for session-wise targets, whereas SWIM conditions the boundary transition on the preceding session prefix.
The test was deliberately constrained
The offline target list contained six items. RecFlow provided 120 candidates for each request. KuaiRand did not provide the upstream candidate set, so the researchers evaluated it as a 6-to-6 re-ordering task. That makes the KuaiRand result a narrower test of ordering than a full candidate-set re-ranking exercise.
After task construction, RecFlow contained 34,574 users, 965,634 sessions, 3,308,233 requests and 14,181,768 items. KuaiRand contained 27,285 users, 3,339,426 sessions, 54,805,516 requests and 31,922,917 items.
Requests were split in time, with the earliest 90% used for training and the latest 10% for testing. In generator-evaluator comparisons, systems produced 20 candidate lists of six items for scoring. The study's central questions were whether SWIM could outperform representative re-ranking baselines and whether its main pieces were important.
Where SWIM led
The offline metrics were NDCG and AUC, calculated from binary labels showing whether a video was viewed. SWIM's reported RecFlow score was 0.2031 plus or minus 0.0015 for NDCG and 0.7185 plus or minus 0.0017 for AUC. On KuaiRand, the figures were 0.7375 plus or minus 0.0011 and 0.7804 plus or minus 0.0013. The paper does not specify the statistical meaning of the plus-or-minus values.
An ablation test, which removes parts of a model to see what they contribute, pointed in the same direction. Full SWIM scored 0.2031 NDCG and 0.7185 AUC. Removing prefix features lowered those scores to 0.1975 and 0.7093. Removing the boundary component gave 0.2014 and 0.7128, while removing survival gave 0.1952 and 0.7071. All three variants were below full SWIM, although no uncertainty intervals were reported for this comparison.
What the live test adds
The online evaluation lasted seven days and ran on Kuaishou's main-feed re-ranking pipeline. The paper describes a platform with more than 400 million daily active users and used 5% of traffic for the test. For scoring, the generator produced about 70 lists from 60 candidates.
Against baseline, app stay time showed a 0.351% relative lift, with a 95% confidence interval of [0.27%, 0.42%]. Retention (LT7) showed a 0.048% relative lift, with a 95% interval of [0.01%, 0.09%]. Both were positive reported differences, but the paper gives relative lifts rather than absolute outcomes.
That is evidence that the method can be evaluated in a live recommendation pipeline, but it remains a narrow result. The supplied study describes one seven-day Kuaishou main-feed test, not a general result for every recommendation platform or user group.
The document is marked as a preprint, with an arXiv version-1 date of 25 Aug 2026 and an accompanying CIKM '26 reference. It says code is available at the listed GitHub repository. The acknowledgements list support from Kuaishou Technology and the National Natural Science Foundation of China, grant U24A20253.
Paper data and sources
Original title: SWIM: Step-Wise Integrated Measure for Session-supervised List Evaluation in Generative Re-ranking
Authors: Yuanhao Pu, Chenghao Zhang, Chao Feng et al.
Journal/Repository: arXiv
Status: Preprint, not yet peer-reviewed
First online: 2026-08-25
DOI: 10.1145/3799682.3840732
Original paper · Full text