Full fine-tuning was the strongest of the adaptation methods tested in a study of models that generate captions for driving video. The authors based that judgment on four automatic caption metrics, which assess the reported text output rather than driving safety or vehicle control.
The test setup
The study compares fine-tuning and prompting as ways to adapt a large language model for autonomous-driving video captioning. Its main dataset, BDDX, is described as containing over 77 hours of driving video across 7,000 videos. BDD100K is described as containing 100,000 videos of about 40 seconds each and was used for an auxiliary task predicting weather, surroundings and time of day. Exact BDDX segment counts and the sizes of its training, validation and test splits are not reported.
The caption target combined the vehicle's action with a reason. The comparison used full fine-tuning and LoRA, a separate model-adaptation method, with the LoRA tests set at ranks 16, 128 and 768; prompt engineering was applied to Video-LLaVA. The outputs were scored with BLEU-4, CIDEr, METEOR and ROUGE-L. CIDEr was reported on a 0 to 1,000 range, while the other three metrics used 0 to 100 ranges.
Video-LLaVA was tested with prompts, but its full fine-tuning and LoRA experiments were not completed because of time and computational restraints. The paper says the BDD100K auxiliary-task results are presented in a supplementary section.
The reported gap
Off-the-shelf models started with low scores. TF-GPT2 recorded 0.02 on BLEU-4, 3.9 on CIDEr, 7.9 on METEOR and 12.4 on ROUGE-L. TF-BERT recorded 0.001, 1.9, 5.4 and 10.3, respectively.
The full fine-tuning configurations recorded 10.0 on BLEU-4, 105.9 on CIDEr, 34.9 on METEOR and 44.3 on ROUGE-L for TF-GPT2. TF-BERT recorded 5.5, 69.6, 32.9 and 41.5 on the same measures. In both cases, the fine-tuned version scored higher than the corresponding off-the-shelf model.
LoRA produced broadly similar scores at the three tested ranks, with no clear rank-dependent separation. For TF-GPT2, ranks 16, 128 and 768 produced BLEU-4/CIDEr/METEOR/ROUGE-L scores of 9.7/83.4/34.4/44.7, 9.0/75.4/33.9/43.6 and 9.3/96.9/33.9/43.2, respectively. The authors describe prompt engineering as weak relative to fine-tuning; the P2 and P3 Video-LLaVA prompts scored above the pretrained/P1 configuration but remained below TF-GPT2 full fine-tuning on the listed metrics.
The paper also includes ADAPT narration and reasoning results as reference comparisons. Narration scored 34.6 on BLEU-4, 247.5 on CIDEr, 30.6 on METEOR and 62.8 on ROUGE-L. The reasoning scores were 11.4, 102.6, 15.2 and 32.0, respectively. The authors' overall interpretation is that full fine-tuning was strongest among the tested adaptation methods and exceeded some ADAPT metrics.
Where the evidence stops
The findings are descriptive rather than a formal estimate of an adaptation effect. No significance tests or uncertainty intervals are reported, and the comparison mixes model architectures and adaptation configurations, so the scores cannot isolate the contribution of adaptation method alone. The outcomes are automatic caption metrics, not evidence that any approach improves driving safety, vehicle control or real-world deployment.
The document is an arXiv preprint, version 1, dated 30 September 2026, and is identified as a technical report based on the first author's master's thesis submitted in 2024. The unfinished Video-LLaVA experiments leave open how that model would perform if full fine-tuning or LoRA were completed.
Paper data and sources
Original title: Comparative study of adapting pre-trained models for driving behavior video captioning
Authors: Sayak Mallick, Philipp Geiger, Augustin Kelava
Journal/Repository: arXiv
Status: Preprint, not yet peer-reviewed
First online: 2026-09-30
DOI: Not available
Original paper · Full text