A stronger offline score at a comparable budget
An arXiv preprint reports that TransRetrieval, a Transformer-based retrieval system for recommendation, scored 0.603 on Recall@2000, the paper's measure of retrieval performance among the top 2,000 candidates, compared with 0.576 for a Production Baseline, while using 35% fewer FLOPs, a measure of computation. TransRetrieval-128D5L reached 0.657, 5.4 percentage points above the smaller configuration. In the reported Industrial comparison, it outperformed RankMixer-768D-16T by 12.6 percentage points at 1.91 MFLOPs versus 150.99 MFLOPs.
The design is intended to handle heterogeneous features across several recommendation domains. Weighted-average aggregation is presented as a way to resolve scale misalignment: it decouples embedding norms from feature cardinality and restores the assumption that the model is processing comparable tokens. In the same 64D-5L configuration, weighted average reached 0.618 overall Recall@2000, compared with 0.608 for weighted sum, 0.593 for LayerNorm plus weighted sum and 0.571 for max pooling.
The design spends saved computation carefully
Target features are compressed into one token by a three-layer MLP, or multilayer perceptron, with hidden sizes [8D, 4D, D], Parametric ReLU activations, LayerNorm between layers and a plain linear final layer. In the tested 64D-3L comparison, compressing eight target tokens to one cut MFLOPs by about 85%, from 3.04 to 0.45. Recall@2000 fell from 0.624 to 0.603, a 2.1-point decrease, while a 128D-5L one-token model reached 0.657 at 1.91 MFLOPs.
Domain information enters through element-wise addition, so all Transformer parameters, including every attention head and feed-forward network layer, are shared across domains. Position-style domain embedding delivered 0.603 overall Recall@2000 at exactly the same inference cost as the zero-domain-token baseline; training on a single domain's data alone produced lower recall.
Scaling is visible in two datasets
Offline evaluation used two four-domain datasets. The Alibaba Industrial dataset contained 250 million users, 52 million targets and 40 billion interactions. KuaiRand contained 24,943 users, 536,491 targets and 152 million interactions. Both were split by time, with historical interactions used for training and future interactions used for evaluation.
Across the tested TransRetrieval sizes, Industrial Recall@2000 rose from 0.464 to 0.657 as the backbone grew from 32D-1L to 128D-5L, an increase of 19.3 percentage points. On KuaiRand, it rose from 0.251 to 0.473, up 22.2 points. The reported fit values for the paper's log-linear trend were 0.82 for Industrial and 0.88 for KuaiRand. KuaiFormer was nearly flat at 0.485 to 0.495 when its dimension was doubled.
Baselines were implemented within the same framework and preprocessing pipeline, while retaining their original feature-processing designs. That setup provides a common implementation and preprocessing setting, but the evidence does not establish that the same scaling pattern would hold across more architectures, domains or compute ranges.
The business result needs a cautious reading
In a month-long online A/B test, the authors changed only the retrieval model and exposed it to 5% of production traffic. TransRetrieval-128D5L served at 230 queries per second, with 99th-percentile latency under 40 milliseconds, matching the production baseline's latency. User-side key-value pairs were computed once per request and reused across candidate scoring; an expand operator broadcast the single cache without copying.
The test reported an overall revenue lift of 2.53%, with a 95% confidence interval from 2.22% to 2.70% and p<0.0001. It also reported a 1.28% lift in Revenue Per Mille, or RPM, the paper's revenue-per-thousand measure, with a 95% confidence interval from 1.14% to 1.56% and p<0.0001.
Those figures are reported outcomes, not proof that the retrieval model caused the business gains. The online report does not specify assignment randomization, allocation details, traffic-unit counts, exposure denominators or an analysis plan, so the size of the model's causal contribution cannot be determined from the reported results.
Several boundaries also matter. The evidence covers only the two datasets and one online deployment; the Industrial data were production-sampled and their access details were not reported, while KuaiRand was filtered to its first four domains. The scaling analysis used a limited set of model configurations, and the preprint reports no code release or preregistration. Recall@2000 remains an offline surrogate whose gains may not translate directly into user or business outcomes.
The authors report support from the National Natural Science Foundation of China, Public Computing Cloud, Renmin University of China and its world-class-universities fund, Alibaba Group's Alibaba Innovative Research Program and the Qiushi Academic Project of Renmin University of China. The document identifies itself as arXiv:2608.25528v1, dated 26 August 2026.
Paper data and sources
Original title: TransRetrieval: Scaling Up Transformer-Based Retrieval for Industrial Recommendation
Authors: Zhifei Zheng, Yunfei Liu, Bin Liu et al.
Journal/Repository: arXiv
Status: Preprint, not yet peer-reviewed
First online: 2026-08-26
DOI: 10.1145/3799682.3840118
Original paper · Full text