A method for adapting image-recognition systems across simulated federated clients reported higher average accuracy than FedDDA in tests designed to mimic federated learning. The method, called MOSAIC, was ahead on all three image benchmarks under domain shifts and also outperformed FedDDA across three levels of combined domain and label shifts.
The result was a gain in the average across simulated clients, not a clean improvement everywhere. In one two-client-per-domain setting, MOSAIC’s average advantage was 1.15 percentage points on Office31, 0.27 points on OfficeHome and 0.12 points on DomainNet100. At the same time, performance fell by 1.20 points on the DSLR domain and by 2.31 points on DomainNet100’s Quickdraw domain.
Aiming at the mistakes that matter most
Federated vision-language models are trained across clients whose images may come from different visual environments. The study asks whether treating each domain’s effect as a broad, class-agnostic shift misses more specific residual effects, such as a change that harms the model’s image-text margin for one class but not another. That margin is the separation between a correct class description and competing descriptions.
MOSAIC combines several parts: a score intended to identify harmful class-domain effects, a low-rank adapter that represents class and domain residuals with shared structure, private domain factors for each client, image-conditioned gating that chooses among candidate corrections, and reweighting that gives extra attention to harmful pairs. The design keeps some semantic structure shared while allowing domain-specific information to remain private to clients.
The benchmarks were Office31, which has 31 categories, OfficeHome, which has 65, and DomainNet100, built from the first 100 categories of each listed DomainNet domain. The data were divided into simulated federated clients. One protocol used one client per domain, while another used two clients per domain. Separate tests added label shifts controlled by beta values of 0.1, 0.3 and 0.5; domain-only tests used beta equal to zero.
Higher averages across the main tests
Against FedDDA, the comparison method named in the study, MOSAIC’s macro-client top-1 accuracy under domain shift was 96.86% versus 94.90% on Office31, 90.03% versus 88.89% on OfficeHome, and 83.78% versus 83.57% on DomainNet100 in the one-client-per-domain setting. With two clients per domain, the corresponding figures were 95.00% versus 93.85%, 88.49% versus 88.22%, and 82.36% versus 82.24%. Macro-client accuracy is the unweighted average of the test accuracy reported for each client.
The same direction appeared when domain and label shifts were combined. At beta values of 0.1, 0.3 and 0.5, MOSAIC reached 96.46%, 96.02% and 95.57% on Office31; 90.65%, 91.31% and 90.05% on OfficeHome; and 87.36%, 85.99% and 85.28% on DomainNet100. In each case, the reported value was higher than FedDDA’s.
The researchers also compared the complete configuration with component variants. The complete MOSAIC configuration had the highest macro-client average in the reported ablations, while removing factorization, private-domain activation, harmfulness weighting or candidate gating was associated with lower reported performance. The sensitivity analysis also found lower accuracy with aggressive harmful-pair weighting or broad pair selection.
A signal for selective correction
The authors tested whether their harmfulness score tracked the effect they wanted to correct. Across 93 Office31 class-domain pairs, 260 OfficeHome pairs and 600 DomainNet100 pairs, the score’s rank correlation with held-out counterfactual margin gain was 0.724, 0.852 and 0.897, respectively. By comparison, the correlations for simply measuring residual size were negative 0.275, 0.207 and 0.145.
That diagnostic does not make the score a universal predictor of mistakes. Its correlation with held-out error was negative 0.144 on Office31, 0.364 on OfficeHome and 0.619 on DomainNet100. The residuals, harmfulness scores and harmful-set membership were estimated from deterministic training subsets, while held-out test data were reserved for prediction outcomes and counterfactual analyses.
Promising benchmark result, limited reach
The experiments used a frozen CLIP ViT-B/16 backbone and three independent random seeds. Each run reported the highest macro-client test accuracy reached over communication rounds, rather than using a separate validation split. The supplied analysis reports no confidence intervals, so the size and stability of the differences are not fully quantified.
The clients were simulated, and the evaluation was confined to a frozen CLIP ViT-B/16 setup. DomainNet used the DomainNet100 subset, made from the first 100 categories of each listed domain. The reported pattern is a trade-off in macro-client performance, not uniform improvement across domains.
Raw images are not exchanged, but class-feature sums and counts are. The study says those exchanged statistics are not claimed to be privacy-free because secure aggregation was not implemented. The supplied evidence therefore does not establish that the method is privacy-safe.
The document is an arXiv version 1 preprint dated 1 October 2026. Its evidence consists of benchmark results from simulated clients on three image datasets, so the result concerns model performance rather than human or clinical participants. No funding disclosure is reported in the supplied text or metadata.
Paper data and sources
Original title: Beyond Domain-Level Adaptation: Margin-Oriented Semantic-Appearance Interaction Correction for Personalized Federated Vision-Language Models
Authors: Wentao Yue, Qingyu Mao, Tianyou Lai et al.
Journal/Repository: arXiv
Status: Preprint, not yet peer-reviewed
First online: 2026-10-01
DOI: Not available
Original paper · Full text