At one tested setting, a query-only router for document retrieval reported both a higher quality score and far lower average response time than the ML pipeline. RetrievalRouter reached 0.755 nDCG@5 at 0.666 seconds per query when λ = 0.1—2.5% more effective than ML and 12.4 times faster.
The comparison covered 11 retrieval benchmarks spanning financial, scientific and open-domain corpora. Its central question was whether query text alone could guide a router to the retrieval pipeline that best fits each query.
The trade-off behind the test
The paper’s main quality measure, nDCG@5, summarizes the ranking of the top five retrieved documents; latency is the time each query takes. Among static options, ML reached 0.737 nDCG@5 but took 8.283 seconds per query. BM25 was faster at 0.019 seconds but scored 0.510 nDCG, while MR reached 0.733 nDCG at 1.121 seconds. No static pipeline combined high retrieval effectiveness with low latency.
Choosing with a query
The router selects among BM25, TD, TR, MD and MR, while TL and ML remain static baselines. Its model uses Qwen3-0.6B-Base with LoRA adapters, and the base weights stay frozen during training.
Training uses soft targets derived from the full per-query reward vector rather than assigning each query one hard pipeline label. That preserves information about how several choices perform for the same query instead of reducing the decision to a single winner.
Each source dataset was independently divided into 80% training, 10% validation and 10% test data, with the test portion held out during oracle labeling and router training.
A different balance against the adaptive baseline
Against the adaptive strategy-selection baseline attributed to Arabzadeh et al., RetrievalRouter had significantly higher nDCG@5 when λ ranged from 0 to 0.5, although the baseline was faster. At the more latency-oriented setting of λ = 0.7, RetrievalRouter scored 0.630 versus 0.624 and recorded mean latency of 0.148 versus 0.171 seconds. Both measures favored the router numerically, but neither difference was statistically significant; the reported p-values were 0.021 for nDCG and 0.015 for latency.
The paper tested the normality of paired differences with the D'Agostino-Pearson test, then used a paired t-test for normal differences or a two-sided Wilcoxon signed-rank test otherwise. It applied Holm correction for multiple comparisons and reported significance at p < 0.001.
An upper bound remains
A per-query oracle reached 0.90 nDCG@5, leaving roughly 14 additional nDCG points of headroom over learned routing. Because the oracle is an upper-bound comparison rather than a deployable system, the gap signals room for better routing decisions.
The choices respond to visual density
The routing pattern also shifted with visual density. At λ = 0, the share of queries sent to multimodal pipelines rose from 69.7% to 95.2% across the visual-density range. The result indicates that the router increasingly favored multimodal options as visual density increased, even when the objective emphasized accuracy.
Speed has an infrastructure price
The flexibility has a separate infrastructure cost. For the combined corpus, the multimodal-late index was approximately 39 GB, more than 13 times the approximately 3 GB text-dense index.
What is being offered
The authors release code, training scripts and per-query oracle labels covering more than 80,000 queries. The document is identified as arXiv:2608.25625v1 in cs.IR and dated 26 August 2026, making the work a preprint benchmark study rather than a settled production result.
Paper data and sources
Original title: RetrievalRouter: Joint Modality and Architecture Selection for Document Retrieval
Authors: Emre Kuru, Mehmet Onur Keskin, Reza Farahbakhsh, Noel Crespi
Journal/Repository: arXiv
Status: Preprint, not yet peer-reviewed
First online: 2026-08-26
DOI: Not available
Original paper · Full text