A preprint describes a robot-vision system that reported a similar segmentation score to a system adapting on every frame, but triggered far fewer adaptation events. Across the main benchmark, the multi-signal method reported a mean intersection over union, or mIoU, of 44.8 with 3.9 adaptations, compared with 46.2 mIoU and 518 adaptations for per-frame adaptation. The reported gap was 1.4 points, and the multi-signal system had a reported adaptation frequency of 0.8%.
The study asks how to detect significant visual and semantic changes in sequential robot image streams so that training-free adaptation is triggered only when needed.
A detector that waits for confirmation
The proposed detector monitors three complementary signals: visual change, adapter mismatch and semantic drift. In ordinary terms, it checks whether the image looks different, whether the current adapter fits poorly, and whether the scene’s semantic pattern has shifted. It fuses those signals hierarchically before triggering adaptation.
The fusion rule requires confirmation before an adaptation is triggered. When the visual-change signal fires, at least one other signal must confirm it. When the visual signal is quiet, the adapter-mismatch and semantic-drift signals must agree.
The implementation uses a segmentation system called SED with a LoRA module, a small add-on trained for each domain, on every linear layer except the text encoder. The library contains 1,626 adapters of about 10 megabytes each.
Tests across indoor and outdoor scenes
The benchmark combines the indoor Scannet and Scannet++ datasets with the outdoor ACDC and MUSES datasets. The outdoor domains vary in weather and illumination.
The main test formed sequences containing 16 domains drawn from the four datasets and repeated the evaluation over 50 randomly sampled sequences. For each held-out domain, the evaluation discarded that domain’s adapter in a leave-one-out setup.
The multi-signal approach was the strongest non-per-frame method in 13 of the 16 benchmark splits. A periodic-adaptation method led in three named splits, but only by 0.3 to 0.5 mIoU.
The gap stayed small in an out-of-distribution test
In a separate out-of-distribution comparison, zero-shot processing, which used no adaptation, reported 36.6 mIoU. The multi-signal method reached 41.0, while per-frame adaptation reached 41.4. The paper reports a 4.4-point gain over zero-shot and a 0.4-point difference from per-frame adaptation, with adaptation counts of 3.2, 960 and 0 for the multi-signal, per-frame and zero-shot settings, respectively.
The signal ablation produced a different accuracy-to-adaptation balance. The full system reported 44.7 mIoU with 4.1 adaptations, compared with 46.0 mIoU for per-frame adaptation. A combination of the adapter-mismatch and semantic-drift signals reached 45.5 mIoU but triggered 14.4 adaptations, making the full system the reported best trade-off in this test.
Hardware tests measured adaptation overhead
On NVIDIA Jetson AGX Orin hardware, an adaptation took an average of 550 milliseconds at full power and 700 milliseconds at low power. Per-frame adaptation reduced throughput by between 1.6 and 3.3 times, while triggered adaptation with the small model reached 4.2 frames per second at full power, reported as 3.2 times faster than per-frame adaptation.
In one real-world sequence from an Odin robot sensor, the detector processed 2,735 frames. It confirmed 12 adaptations from 42 visual-change activations and rejected 71% of those activations. The confirmed updates added 7 seconds, compared with 25 minutes for per-frame adaptation in the reported sequence.
The evidence remains limited to the tested settings
The benchmark evidence comes from four datasets and 50 randomly sampled sequences, while the real-world evidence comes from one 2,735-frame Odin sequence. The reported deployment result is therefore based on that single sequence.
The document is a preprint. The paper states that its code and benchmark will be released upon publication. It reports partial support from the Wallenberg AI, Autonomous Systems and Software Program, funded by the Knut and Alice Wallenberg Foundation.
Paper data and sources
Original title: When to Adapt: Multi-Signal Domain Shift Detection for Efficient Training-Free Adaptation in Open-Vocabulary Segmentation
Authors: Michele Antonazzi, Alejandra C. Hernandez, José Araujo et al.
Journal/Repository: arXiv
Status: Preprint, not yet peer-reviewed
First online: 2026-09-29
DOI: Not available
Original paper · Full text