A multimodal retrieval preprint reports higher average mAP with around two embeddings per sample, compared with fixed-capacity baselines using 3 to 40 embeddings per sample. Here, an embedding is one vector in a sample's representation. The system, called AdaptiveEmbed, treats representation capacity as a decision made for each sample and selects it using retrieval utility.
The work is an arXiv version 1 preprint dated 26 August 2026. Its evaluation covered eight full-gallery, one-to-one benchmarks across image-text, video-text and audio-text retrieval. Each query had exactly one positive gallery item, and the main measures were bidirectional mAP@ALL, a retrieval score, and mean active-vector count, the average number of vectors used per query.
Reported gains across image, video and audio
On image-text retrieval, AdaptiveEmbed used an average of 2.1 tokens per query. On COCO, it reported mAP values of 63.45 for text-to-image search and 61.14 for image-to-text search. It led on six of the eight image-text directions, including both OpenImages directions against a gallery of over 546,000 items.
The method also reported the strongest multi-vector result on seven of eight video- and audio-text directions. On ActivityNet, its average was 1.9 tokens.
On COCO, AdaptiveEmbed led every reported Recall and nDCG metric against MetaEmbed configurations while activating the fewest tokens on average. In the strict zero-shot ADE20K evaluation, it remained ahead of every fixed configuration on Recall, nDCG and mAP.
A capacity decision for each sample
The method uses two stages. First, it learns structured candidate embeddings with MGCL and SetSim. It then constructs a sample-specific CAES with a Token Selection Transformer trained using UPO and MUA.
Representation and policy training used COCO for image-text, joint ActivityNet and DiDeMo data for video, and joint MACS and Clotho data for audio. Flickr30K, ADE20K and OpenImages were used for strict zero-shot evaluation.
Tests of the allocation strategy
The variants labeled w/o MGCL and w/o bank-feedback reported lower listed mAP values than AdaptiveEmbed. The version without bank feedback used 2.5 average tokens, compared with 2.1 for the full system.
Across the evaluated capacity-threshold and feedback-depth ranges, average mAP varied by no more than 0.43 points on Flickr30K and 0.16 points on Clotho, while average capacity stayed close to two tokens. Replacing the COCO feedback bank with a Flickr30K training bank changed average mAP on Flickr30K by only +0.07 points.
Efficiency still has a limit
The efficiency claim has a specific boundary. The authors describe efficiency as a per-sample matching-time benefit that depends on the data distribution. Each gallery item still stores the full eight-token embedding set, so these results do not show reduced gallery-side storage.
The learned allocation also leaves room for improvement. Oracle AdaptiveEmbed reported 61.0 average mAP at a comparable average capacity of 2.2 tokens, indicating additional headroom beyond the learned allocation.
The supplied document includes an appendix with detailed formulation and allocation construction, but no separate supplement is reported. Funding, conflicts of interest and code availability are not reported in the supplied text.
Paper data and sources
Original title: AdaptiveEmbed: Sample-Adaptive Multi-Vector Representation for Multimodal Retrieval
Authors: Xinze Liu, Lei Yang, Dayan Wu et al.
Journal/Repository: arXiv
Status: Preprint, not yet peer-reviewed
First online: 2026-08-26
DOI: Not available
Original paper · Full text