An arXiv preprint reports that DUMoE, a model that learns user representations from multimodal social-media data, delivered the strongest reported interest-classification results among the listed baselines and the paper's best overall ranking-oriented interaction results in an offline evaluation. The study asks whether learned user representations can support both tasks, but the reported setup does not demonstrate online continual adaptation.
A model built around several time scales
DUMoE encodes multimodal posts and profiles with pretrained UNITE. It models static, short-term and long-term signals in separate branches, then passes them through specialized experts, or submodels, selected by sparse adaptive routing. The design therefore builds a user representation from several time scales while directing different inputs through selected experts.
Training used a class-weighted focal loss for interest classification and binary cross-entropy, or BCE, with randomly sampled negative posts for interaction prediction. It followed three stages: pre-training the backbone, specializing the experts and optimizing the gating mechanism used for routing.
The benchmark and its labels
The benchmark drew on an X dataset covering 15 high-level interest domains. It contained 14,015 users, 7,685,700 posts and 2,890,668 images, with users and their chronological post collections serving as the analytical units.
Interactions were partitioned into chronological collections of 30 posts. GPT-5.1 assigned labels from the 15 predefined domains using up to two preceding collections as context, and a random subset was manually inspected.
Each user's 12 collections were allocated chronologically as eight training collections, two validation collections and two testing collections. Training used a batch size of 128, and the baselines followed the same data splits and evaluation protocol.
The reported scores
On interest classification, DUMoE reported Hit@1 of 0.872, NDCG@3 of 0.940, Recall of 0.856 and KL divergence of 0.323. The paper reports that these values outperformed all baselines across the classification metrics.
The reported Hit@1 gain was 6.2% over MIND and 7.5% over PTUM. DUMoE's Recall gain over MIND was 12.2%, while its KL divergence of 0.323 represented a reported 31.4% reduction from MIND's 0.471.
On interaction prediction, DUMoE reported Accuracy of 0.883, Recall of 0.926, F1 of 0.888 and BCE of 0.276. The paper described Recall and F1 as its best overall ranking-oriented results.
These are point estimates. The paper does not report confidence intervals or significance tests for the headline comparisons, and the ablation results do not include run-to-run variability.
What changed when components were removed
The component tests showed that the long-term branch was the strongest single branch. Hit@1 was 0.774 with Static Only, 0.805 with Short-term Only and 0.862 with Long-term Only, compared with 0.872 for the full classification model.
Removing the adapter or gating also left lower reported scores than the full configuration. Without the adapter, Hit@1 was 0.858 and BCE was 0.329; without gating, the corresponding values were 0.864 and 0.326. Both variants had interaction Accuracy of 0.858, versus 0.883 for the full model.
Training ablations showed a task-specific trade-off. Without stage-wise training, Hit@1 was 0.865 and KL was 0.341. Using only the classification loss produced classification Hit@1/KL of 0.870/0.325 but interaction Accuracy/BCE of 0.799/0.437. Using only the interaction loss reversed the pattern: Accuracy/BCE were 0.869/0.303, while classification Hit@1/KL fell to 0.798/0.473.
Why the result remains a benchmark finding
The authors identify several limits to the result: GPT-generated interest labels may contain noise or domain-specific bias; fixed short-term and long-term windows may not fit users with different activity patterns; and the expert count and top-K routing are fixed. The framework was evaluated in an offline training regime and does not demonstrate online continual adaptation.
The authors state that the code and dataset will be made publicly available under a privacy policy. The central finding therefore remains a report of offline performance on the defined benchmark, while online continual adaptation remains untested.
Paper data and sources
Original title: Drift-Aware Multimodal User Representation Learning via Multi-Scale Temporal Modeling and Sparse Mixture-of-Experts
Authors: Ziqing Qian, Haohang Chen, Shengqi Dang et al.
Journal/Repository: arXiv
Status: Preprint, not yet peer-reviewed
First online: 2026-08-26
DOI: Not available
Original paper · Full text