The proposed image-classification framework reported 85.83% accuracy while sending 1.24 bits per pixel (bpp), the study's measure of transmitted image data. At the same stated accuracy, WebP needed about 1.57 bpp and BPG about 1.82 bpp to match it. That left the proposed system at a lower reported bit rate for the stated accuracy. Its accuracy was 0.98 percentage points below the 86.81% lossless upper-bound accuracy used for the controlled comparison.
A modular route through the network
The idea is to avoid sending token embeddings directly from a lightweight client to a larger server model. The framework coordinates a client-side vision transformer, a learned image-compression model and a large server-side vision transformer. The components are pretrained and used without end-to-end training, giving the system a modular structure aimed at the cost and interoperability problems associated with direct token-embedding transmission.
On the client, the system estimates which image-patch tokens are relevant to classification. Token-aligned LIC uses a one-to-one spatial match between ViT patch tokens and LIC latent vectors to determine which task-relevant latents are transmitted. After reconstruction at the server, a single learnable surrogate token fills unselected positions, while the pretrained backbone stays frozen.
The relevance estimate comes from layer-selective attention rollout, which uses a selected range of attention layers in a single forward pass. For the 12-layer DeiT-Tiny client, that range was layers 7 through 12. In an alternative-client ablation, layer-selective rollout outperformed last-layer attention and ordinary rollout, and was comparable to gradient-weighted attention.
What the test measured
The reported tests used ImageNet classification with 224 by 224 pixel inputs. DeiT-Tiny was used at the client and DeiT-III-Large at the server. Their reported accuracies were 72.2% and 86.8%, respectively. The client stack required 45 MB of memory and 4.47 GMACs, compared with more than 1.2 GB and approximately 64.36 GMACs for the server stack.
Baseline comparisons decoded the transmitted representations back into images and fed them to the same pretrained DeiT-III-Large server classifier. Under that setup, 86.81% was treated as the lossless upper-bound accuracy. The protocol held the final classifier constant across the baseline comparisons.
Where the reported gains appeared
At reported rates below 2.5 bpp, the framework showed higher accuracy at comparable rates than task-agnostic LIC methods and a more favorable trade-off than ELIC. The comparison places the proposed system against learned compression methods that are not tailored to a particular downstream task.
The paper also tested entropy-aware image transmission, or EIT. With EIT, the framework reached 85.89% accuracy at 0.94 bpp; without it, approximately 1.24 bpp was needed for comparable accuracy. The reported EIT operating point used a lower bit rate while keeping the stated accuracy close to the earlier result.
When packets were erased, surrogate-token substitution was consistently the most robust of the three server-side inference strategies across the tested erasure probabilities.
Two component tests focused on what happened around unsent information. The mean-hyperprior model attained higher downstream accuracy than the model without mean imputation; the text links that difference to reconstruction of selected regions affected by adjacent unsent latents. The LIC-aware surrogate token, trained with settings 0.35 and 0.03, consistently outperformed the LIC-agnostic counterpart trained with 0.35 alone.
A result tied to a specific setup
The reported comparisons are tied to a defined computational setup: ImageNet classification with selected DeiT models and LIC configurations, plus the packet-erasure conditions used in the robustness test. They do not by themselves establish performance beyond those settings. The study reports no confidence intervals or formal uncertainty estimates, so the differences should be read as reported comparisons for those configurations.
Paper data and sources
Original title: Token-Oriented Semantic Communication with Pretrained Vision Transformers
Authors: Jiwoong Im, Minwoo Kim, Jaeho Lee et al.
Journal/Repository: arXiv
Status: Preprint, not yet peer-reviewed
First online: 2026-08-26
DOI: Not available
Original paper · Full text