An artificial-intelligence model that connects ECG images with summary reports performed best on rhythm abnormalities, while its results were lower for conduction disorders and morphological conditions. Across the 14 conditions tested, mean AUROC was 0.948 in internal testing and 0.921 in external validation for rhythm abnormalities. The corresponding figures were 0.858 and 0.843 for conduction disorders, and 0.741 and 0.750 for morphological conditions. Five of the six rhythm conditions exceeded 0.90 on both datasets.
One model, many condition prompts
The study set out to see whether paired ECG recordings and summary reports could create a reusable representation without training a separate supervised model for each condition. ECG-VLM then treated each condition as a binary prompt classification, comparing a condition prompt with a negative prompt labeled as other; the model itself was not modified between conditions.
For model development and internal evaluation, the researchers used 800,034 paired 12-lead recordings from 161,352 unique patients in MIMIC-IV-ECG. LSAS-ECG supplied another 45,151 12-lead recordings for external validation. MIMIC-IV-ECG was divided at the patient level into training, validation and internal test sets; the internal labels came from reports, while external ground-truth labels were extracted through condition-to-code mappings.
The signals were downsampled to 100 Hz, filtered from 0.5 to 50 Hz, normalized by lead and rendered as 224 by 224 pixel images. Reports were capped at 77 tokens, and 2,648 reports, or 0.33 percent, exceeded the limit. The images were standardized renderings from digital waveforms rather than authentic clinical ECG images.
The model used a ViT-B/32 image encoder and a Transformer text encoder. Pre-trained CLIP weights initialized a fully fine-tuned network trained with a symmetric bidirectional contrastive objective.
The strongest results were not uniform
Performance varied sharply by condition and metric. AUROC, the study's main score for distinguishing the tested conditions, ranged from 0.679 to 0.981 internally and from 0.691 to 0.991 externally. AUPRC ranged from 0.130 to 0.877 internally and from 0.058 to 0.982 externally. The analysis described AUROC as generally stronger than the more variable AUPRC and precision results; internal precision was below 0.20 for AVB, SVT and PQTI.
To turn model scores into condition decisions, the team selected a separate threshold for each condition using labeled validation data. The chosen threshold had to reach sensitivity of at least 0.8 and then maximize the F1 score; if no candidate met the sensitivity requirement, the highest-F1 threshold was used instead. The reported confidence intervals came from 1,000 stratified bootstrap iterations.
A second test of report matching
The model was also tested on ECG-report retrieval, asking whether the report associated with an ECG could be found. Across 100 repeated runs, its top-result retrieval score, Recall@1, fell from 0.413 for rare reports to 0.073 for frequent reports. When reports were matched by content rather than exact wording, frequent-report Recall@1 rose to 0.282.
In comparisons with other systems, ECG-VLM was broadly comparable with ECG-CLIP for rhythm abnormalities. ECG-CLIP generally performed better for conduction disorders, while ECG-VLM had higher AUPRC for several morphological conditions. Against supervised models, ECG-VLM had broadly comparable AUROC but lower internal AUPRC.
Male and female performance was nearly identical, but mean internal AUROC declined across age groups, from 0.891 among people aged 12 to 44 to 0.870 at ages 45 to 74 and 0.831 among those 75 or older. In the external cohort, the pediatric subgroup had the highest AUROC and the lowest AUPRC.
The image test had a clear boundary
The main caution is what the image test did not cover. The ECGs were synthetically rendered from standardized digital waveforms, not taken as printed, scanned or smartphone images, so the results do not establish performance on those formats. The thresholding approach also depended on labeled validation examples for each condition.
Paper data and sources
Original title: Language-supervised electrocardiogram interpretation with vision transformer
Authors: Zhou X, Li T, Nakamura K et al.
Journal/Repository: Health information science and systems
Status: Peer-reviewed
First online: 2026-10-13
DOI: Not available
Original paper · Full text