A lightweight add-on for computer-vision models improved their performance on fisheye images in tests covering indoor depth, outdoor depth and open-vocabulary segmentation, according to a new arXiv preprint. Called Distortion Extenders, or DEX, the method adapts perspective-trained models without retraining their main backbones.
The reported gains were largest in the indoor depth evaluation. On ScanNet++, DEX produced a 19% lower average RMSE and a 15% higher average δ1 score than LoRA and Calibration Tokens. With UniDepthV2, the authors also report an 8% average improvement over UniK3D across the reported metrics.
Adapting features instead of reshaping the image
The paper starts from a simple hypothesis: the internal feature pattern produced by a fisheye image is shifted away from the pattern associated with perspective imagery. DEX aims to move fisheye features toward that perspective distribution, working directly in latent space rather than explicitly rectifying the input or retraining the backbone.
The backbone stays fixed while small learnable modules are inserted into its blocks. During training, the researchers create synthetic fisheye versions of calibrated perspective images, use the frozen model’s perspective outputs as references, and inverse-align the DEX outputs before measuring the average mismatch. In plain terms, the add-on learns to make the fisheye prediction line up with what the perspective model would have produced for the corresponding scene.
For depth, the method uses spherical range, meaning the distance from the camera to a 3D point rather than only its forward distance. The reported formulation derives that range from the point’s three spatial coordinates.
Improvements across tasks and settings
The outdoor depth results were also positive, though smaller. On KITTI-360, DEX improved average RMSE and δ1 by 11% over LoRA and Calibration Tokens. The same extender set was used for the indoor and outdoor tests.
For open-vocabulary segmentation on WoodScape, DEX improved mean intersection over union, or mIoU, by around 13% over Calibration Tokens. In a reported PanopticDepth evaluation, adding DEX changed depth RMSE and δ1 from 7.273 and 0.267 to 3.583 and 0.354. Segmentation mIoU and weighted IoU changed from 0.160 and 0.300 to 0.282 and 0.833.
DEX also compared favorably with an undistortion pipeline. With UniDepthV2 on ScanNet++, its Chamfer distance and F1 score were 0.117 and 0.841, compared with 0.360 and 0.317 after undistortion. On KITTI-360, the corresponding figures were 0.343 and 0.887 with DEX, versus 2.428 and 0.196 with undistortion.
A useful signal, with uneven precision
Ablation tests, which remove parts of a method to see what changes, pointed to the spherical-range output as especially important on KITTI-360. Full DEX recorded RMSE and δ1 of 1.663 and 0.842 there; removing that extension changed the scores to 7.024 and 0.271. Removing the low-rank decomposition caused smaller changes, to 1.777 and 0.800.
The paper also reports KB-coefficient decoding errors from DEX activations. Mean absolute percentage error was 3.91% for k1, 14.30% for k2, 41.86% for k3 and 78.61% for k4, with an average of 34.67%. The later coefficients were therefore much less precise in these tests, and the result does not by itself establish that DEX can perform practical camera calibration.
The added modules were small in the reported measurements. DEX used 2.8 MB of extra memory and added 0.3 milliseconds of inference time with UniDepthV2, while the corresponding figures with LSeg were 6.1 MB and 2.21 milliseconds.
What remains untested
The experiments used calibrated perspective datasets, including NYUv2, VOID, IRS, Hypersim and Waymo, to create synthetic fisheye training examples. Real fisheye evaluation used ScanNet++, KITTI-360 and WoodScape, and the paper says ground-truth fisheye images were not needed for training.
The paper reports no confidence intervals, standard deviations, significance tests or power analysis for the benchmark differences. Exact evaluation-image counts and per-dataset splits are also not reported, leaving statistical uncertainty around the reported gains unquantified.
The authors identify a basic ceiling: DEX is supervised by the outputs of a frozen backbone, so its performance is upper-bounded by that backbone’s quality. Synthetic distortion training may not represent every real-camera distribution, and the paper reports degradation in a cross-domain analysis when the backbone was trained for a different domain.
The document is an arXiv version 1 preprint dated 28 August 2026. Its evidence is confined to offline computer-vision benchmarks, so it does not show benefits in people, clinical settings or safety-critical deployment. The work reports support from a Google Gift Award, the NSF-2112562 Athena AI Institute and GITCC through KIAT under project P0028922.
The work presents DEX as a low-overhead way to reuse perspective-trained models on fisheye data, with gains reported across several tasks and datasets. The open questions are whether those gains hold across more real cameras and domains, and whether the decoded coefficients can support practical calibration.
Paper data and sources
Original title: From Perspective to Fisheye Depth Estimation and Open-Vocabulary Segmentation
Authors: Rit Gangopadhyay, Alex Wong
Journal/Repository: arXiv
Status: Preprint, not yet peer-reviewed
First online: 2026-08-28
DOI: Not available
Original paper · Full text