A computer-vision system designed to catch LipSync forgeries reported high scores across benchmark tests by checking whether lip movements and head pose stay consistent. LipDA’s audio-visual version averaged 96.26% accuracy and 98.91% AUC, a summary measure of detection performance, across the reported benchmark rows. Its visual-only version averaged 94.51% accuracy and 98.69% AUC.
The work is an arXiv version 1 preprint dated 6 October 2026. The evidence comes from computational benchmark tests across the listed datasets and test settings.
Following the link between lips and head
The system’s task has two parts: detect a forgery and estimate the generator family, meaning the broad source system that made it. It looks for inconsistency between lip motion and head pose, then uses audiovisual synchronization and patterns over time to support attribution.
LipDA uses a two-stage design. In the first stage, contrastive lip-pose encoding supports binary detection, or the real-versus-forged decision. In the second, temporal dynamics and audiovisual synchronization are combined for generator-family attribution.
A broad synthetic test set
To train and test it, the authors used LipSync-A, a dataset with approximately 16,000 synthetic videos produced by 15 generation techniques, including approximately 270 commercial-API samples. Its evaluated subset had 3,926 training samples, 841 validation samples and 842 test samples. The training split contained 1,614 authentic videos and 2,312 forged videos from 10 generators spanning seven architectural approaches.
The paper also reports tests on listed LipSync generators not used in the main training setup, as well as a separate identity-swap DeepFake test on Celeb-DF. These remain benchmark evaluations rather than evidence for every possible generator or manipulation type.
Performance varied across generators
In the generator-family test, audio-visual LipDA averaged 97.5% accuracy and 93.9% F1, a summary score for classification quality. On the listed unseen generators, it recorded 87.8% accuracy and 84.3% F1 on Sonic, 84.6% and 84.3% on KDTalker, and 95.5% and 95.4% on OmniSync. The reported figures varied by generator.
LipDA also reached 92.76% accuracy, 91.86% AUC and 98.38% average precision on the Celeb-DF test. The reported false-positive rate was 4.33% and the false-negative rate was 1.54%. That result shows performance on the listed identity-swap benchmark, not on every kind of DeepFake.
In synthetic robustness tests, average AUC was 99.11% under the reported visual perturbations, 13.58% above the next-best baseline. Under challenging audio perturbations, AUC remained above 97.3%.
The model’s ingredients mattered
Component tests, often called ablations, removed or replaced parts of the model and measured the resulting accuracy. On AVLips, accuracy dropped 17.3% without alignment, 44.4% without lip input and 12.0% when CLIP-ViT replaced ResNet-18.
For attribution, the full model scored 97.50% accuracy and 93.90% F1, compared with 74.78% and 73.58% for the temporal-only version, and 92.92% and 93.64% for the audiovisual-synchronization-only version. A finer-grained model-level attribution test reached 82.67% overall accuracy and an 82.04% macro-average F1 on the LipSync-A test split.
A benchmark result, not a universal detector
The authors identify several limits. The approach relies on visible lip dynamics and head motion and can degrade for stationary speakers, persistent downward gaze or prolonged facial occlusion. The attribution module can also degrade under some audio perturbations, and family-level supervised attribution is not a complete open-set answer for entirely new generator paradigms.
The paper reports point estimates only, without confidence intervals or inferential significance tests. The results do not establish performance beyond the listed generators and domains, or show that the method can prevent real-world harms. The authors report support from the National Natural Science Foundation of China and the Fundamental Research Funds for the Central Universities.
Paper data and sources
Original title: Ariadne's Thread of LipSync: Unraveling Forgeries via Inconsistency between Lip Motions and Head Poses
Authors: Tianyi She, Jiawei Liu, Weifeng Liu et al.
Journal/Repository: arXiv
Status: Preprint, not yet peer-reviewed
First online: 2026-10-06
DOI: Not available
Original paper · Full text