A computer-vision system designed to find pedestrian images that match detailed descriptions of behavior led the reported results on the original PAB person-anomaly-search test. Called ActPair, it recorded 88.62% Recall@1 and 93.97% mAP, the paper's headline measures of ranking performance.
Recall@1, Recall@5 and Recall@10 track retrieval at the first result, first five results and first 10 results. mAP provides the study's broader ranking measure. In plain terms, the scores describe how well the system orders candidate images for a text query on the benchmark.
Unlike appearance-only retrieval, the task is to distinguish individuals through fine-grained, context-dependent normal or anomalous behaviors. The relevant signal can therefore be the action and context described in the query, rather than appearance alone.
The results come from an arXiv preprint, arXiv:2608.23503v1, dated 24 Aug 2026. The supplied analysis describes a modeling study on benchmark data, so its scores concern retrieval performance in the reported tests rather than deployment-level surveillance outcomes.
A search built around behavior
ActPair combines action-aligned fine-tuning of a vision-language model, parallel retrieval using the original query and a context-grounded rewrite, and late fusion of the two views. It then uses pivot-promote pairwise multimodal reranking to reorder the leading candidates.
At inference, each query view retrieves its top 100 candidates. The fused pool is reduced to the top 10 before reranking, while Qwen inference uses temperature 0 and seed 0 deterministic decoding.
The experiments used PAB with over 1 million image-text pairs spanning 1,600 anomalous behaviors and 1,000 normal behaviors. The transfer benchmark, RSTPReid, contains 20,505 images of 4,101 persons across 15 cameras.
The reported endpoints were Recall@1, Recall@5, Recall@10 and mAP, with progressive ablations used to examine how the pipeline's components changed the ranking results.
A clear margin on PAB
On the original PAB test, ActPair was reported as the best method among compared published baselines. The strongest cited training-free, appearance-oriented baseline reached 44.41% mAP, while X-VLM reached 83.96% mAP; under PAB fine-tuning, ActPair's margins over CMP were 3.69 percentage points in R@1 and 2.31 points in mAP.
Against SSDC under PAB fine-tuning, the reported ActPair margins were 1.41 points in R@1 and 1.10 points in mAP. The paper presents these as benchmark comparisons between methods, indicating relative ranking performance within the tested protocol.
Those figures are not a direct estimate of field performance. The study compares retrieval scores on the listed datasets and does not report deployment-level surveillance outcomes.
What the component tests show
Component tests showed a stepwise pattern on both reported test sets. With the baseline, R@1 was 72.14% on the Challenge Set and 84.98% on the Public Test. Dual-view fusion raised those values to 75.73% and 85.95%; the full framework, after pairwise reranking, reached 84.88% and 88.62%.
On the final Public Test result, 88.62% R@1 was paired with 93.97% mAP. The authors interpret the progression as indicating that the original and rewritten queries offer complementary information and that pairwise comparison improves fine-grained ordering.
An action-versus-pose test held the SigLIP2 setup fixed. Action supervision produced 72.14% R@1, versus 67.95% with pose supervision, a 4.19-point difference. A paired bootstrap 95% confidence interval for that difference ran from 2.38 to 6.07 points.
That result is a comparison within the reported setup, not a test of every possible way to represent behavior. The authors interpret it as support for action-tag supervision over pose-only supervision when the goal is to capture context-conditioned behavior.
Pairwise reranking was tested against pointwise reranking with the same Qwen3.5-9B model and candidate pool. Pairwise produced 84.88% R@1 and 90.14% mAP, compared with 80.13% and 87.70% for pointwise. The reported margins were 4.75 and 2.44 points, while the 95% bootstrap interval for the R@1 difference was 3.08 to 6.42 points.
On efficiency, pairwise reranking averaged 9.09 model calls per query, versus 10.00 for pointwise reranking under the top-10 configuration. That difference applies to the tested setting, which uses a small reranking pool.
Transfer beyond anomaly search
In the direct zero-shot transfer from PAB to RSTPReid, ActPair with pairwise reranking reached 55.25% R@1, 69.05% R@5 and 73.75% R@10. CMP reached 29.15% R@1.
RSTPReid broadens the test to text-based person retrieval, but it is non-anomaly-specific rather than a second anomaly-search benchmark. The authors interpret the transfer scores as evidence that PAB-trained representations retain useful cross-domain information.
That interpretation remains bounded by the two reported datasets. The analysis does not establish generalization to additional person-anomaly-search data, unannotated domains or other context-grounded retrieval tasks.
Where the method still falls short
One constraint is the vocabulary used to describe actions and scenes. The canonical vocabularies require dataset-specific annotations, which limits scalability in domains without those labels. The proposed future direction is an unsupervised MLLM clustering pipeline.
Qualitative examples also show a failure on a long compositional query. The correct image started in first place but was demoted later, and the authors note difficulty when a description combines action phases, multiple people and precise spatial relationships.
The reranker is restricted to a small top-candidate pool. In the reported configuration, each fused list is reduced to 10 candidates before pairwise reranking, leaving its behavior with substantially larger pools untested.
These boundaries matter because strong benchmark scores can coexist with brittle behavior on more complicated language. The open questions include whether the method can handle longer descriptions with multiple people and action phases, whether pairwise reranking can scale beyond the tested pool, and whether the pattern holds on more anomaly-search datasets.
Overall, the paper provides benchmark evidence that an action-focused, dual-view and pairwise pipeline can rank behavior-matching person images strongly in the reported tests. It does not prove causal improvement in real-world surveillance or human safety, and it does not show that the method remains efficient at substantially larger scale.
The preprint reports support from Saigon AI Hub, jointly established by VNG Group JSC and Vietnam National University Ho Chi Minh City, and a VINIF PhD Scholarship for Trong-Thuan Nguyen under code VINIF.2025.TS63.
Paper data and sources
Original title: Action-Aligned Retrieval with Pairwise Multimodal Reranking for Text-Based Person Anomaly Search
Authors: Thanh-Khoi Nguyen, Thanh-Nhan Vo, Trong-Thuan Nguyen, Minh-Triet Tran
Journal/Repository: arXiv
Status: Preprint, not yet peer-reviewed
First online: 2026-08-24
DOI: Not available
Original paper · Full text