GAAT, a multimodal model for paired RGB and infrared drone images, reported its clearest gains in object detection and semantic segmentation. On DroneVehicle, it posted margins of 6.49, 7.72 and 7.66 points over DDQ-DETR across three detection scores, while on KUST4K it recorded the highest listed segmentation scores. The wider set of tests was mixed, with rival systems still leading some classification, tracking and reconstruction measures.
The system's central idea is an alignment-first pipeline. syncPATC learns transformation-consistent reliability priors, which indicate which local correspondences between the two image types are more dependable. MG-Sparse-MMA then uses reliability-ranked queries for local bidirectional fusion, while RA-QCGCL applies those priors to patch- and query-level cross-modal supervision.
UAVMeta contains 2,575 synchronized RGB-IR image pairs spanning daytime and nighttime scenes. The dataset was split into 1,715 training pairs, 572 validation pairs and 288 test pairs. Where applicable, the downstream GAAT variants were initialized from the same UAVMeta-pretrained dual Swin-V2-Base RGB-IR encoder.
On KUST4K semantic segmentation, GAAT(C) reported 82.14 mIoU and 92.41 mAcc, the highest values listed. The reported margins were 1.07 mIoU points over SGFNet and 2.93 mAcc points over DOFA. On UAVMeta, GAAT reported 67.79 mIoU with a standard deviation of 1.18, 78.58 FWIoU with a standard deviation of 1.32, and 87.44 pixel accuracy with a standard deviation of 0.93. Its reported mean differences versus SegFormer-B5 were 4.27, 4.05 and 2.58 points on those measures.
Detection results were strongest on DroneVehicle. GAAT(D) reported 56.59 mAP, 80.12 mAP50 and 67.16 mAP75, with reported differences versus DDQ-DETR of 6.49, 7.72 and 7.66 points. The margin was smaller on UAVMeta, where GAAT(D) reported 44.86 mAP, 62.64 mAP50 and 52.00 mAP75. The corresponding standard deviations were 3.76, 4.40 and 5.13, compared with DDQ-DETR means of 43.33, 61.93 and 50.37, producing reported differences of 1.53, 0.71 and 1.63 points.
The gains were not uniform
Scene classification showed a less decisive pattern. GAAT reported 95.90 and 97.38 accuracy on AID at training ratios of 20% and 50%, and 92.84 and 94.49 on RESISC45 at training ratios of 10% and 20%. On UAVMeta, it reported 63.66 accuracy with a standard deviation of 1.78, 51.97 balanced accuracy with a standard deviation of 0.56, and 48.12 macro-F1 with a standard deviation of 1.23. The paper notes that other models were stronger on several AID and RESISC45 settings.
Change detection produced both a near tie and a table-leading result. On CDD, GAAT(CF) reported 97.85 F1 and 95.79 IoU, within 0.03 and 0.06 points of ScratchFormer. On LEVIR-CD, GAAT(CX) reported the highest listed values, with 95.96 F1 and 92.47 IoU.
Tracking results depended on the sequence. GAAT(B) had the highest pooled HOTA, IDF1 and MOTA on Drone 1, and the highest pooled HOTA and MOTA on Drone 2. CenterTrack nevertheless retained the highest pooled Drone 2 IDF1, at 65.78 versus 60.84 for GAAT(B). In the TSDN novel-view reconstruction setting, GAAT+ThermalGS reported 26.92 PSNR, 0.89 SSIM and 0.11 LPIPS. It led on SSIM and LPIPS, while Thermal3D-GS and ThermalGS tied for the highest PSNR.
On StateBench, GAAT obtained an aggregate MSPA-4D score of 52.69, close to SkySense-S2 at 52.26. It also had the lowest listed FMCS MAE, at 12.66, although other models led individual state-score MAEs.
A broad benchmark record
The authors describe the full GAAT pipeline as having the strongest overall ablation pattern across representative tasks. Individual module removals were used to examine the roles of syncPATC, MG-Sparse-MMA and RA-QCGCL. In supplementary VisDrone-CC crowd counting, GAAT CountReg had the lowest listed MAE, 9.40, and MSE, 14.18.
The findings come from an arXiv version 1 preprint dated 28 Aug 2026. Taken together, the reported scores show a model that led several tests but did not produce a clean sweep: its strongest results sit alongside settings where competing systems remained ahead.
Paper data and sources
Original title: GAAT: Geometry-Aware Alignment Transformer for Multimodal UAV Perception
Authors: Jingpu Yang, Debin Tang, Yilin Sun et al.
Journal/Repository: arXiv
Status: Preprint, not yet peer-reviewed
First online: 2026-08-28
DOI: Not available
Original paper · Full text