A new preprint reports that automatic speech-recognition models could stop recognizing a language after training switched to another one yet still show an internal trace of the earlier language. When training returned to that language, the models reached a 70% accuracy target in fewer steps than comparison systems. The experiment involved artificial networks, not people.
Performance changed after the switch
The researchers trained each model on one language and then abruptly switched it to another. After 4,000 updates, recognition accuracy for the first language had fallen to 25.9%, close to 25.4% when the input was shuffled. The later language reached 80% accuracy after 3,000 post-switch updates—30% fewer steps than a model trained only on that language—and final accuracy differed by less than 0.1 percentage points: 93.6% versus 93.7% at the same total training.
To look below performance scores, the researchers compared hidden representations—the internal patterns produced as a model processes speech—using representational similarity analysis, or RSA. The analysis compared responses to 500 held-out samples from the first-trained language, asking whether models organized the same speech input in similar ways. The systems used about 1,000 hours of Common Voice speech in English, French and German, with 2% (20 hours) held out for both validation and test sets.
The trace sat near the model’s input
After first-language accuracy had fallen to chance at about 4,000 steps, models with early exposure to that language remained more similar to the model state before the switch than a model without that exposure. Around 15,000 updates, the similarity measure averaged 8.3% over the final 5,000 steps, compared with 6.3% for the control. The reported 95% confidence ranges—a way of showing uncertainty around the estimates—were 2.9% to 13.7% for the early-exposure models and 2.4% to 10.2% for the control.
The trace was concentrated in the first four encoder layers, before the model’s phoneme-processing stage: relative similarity was 4% to 8% there, while the remaining eight higher layers showed virtually no residual trace. A phoneme probe—a test of whether internal states still carried information about basic speech sounds—also scored higher for models exposed early to the first-trained language than for related-language controls in the pre-phonemic and phonemic layers (p = 0.016 for both), but not in later layers (p = 0.28).
More early training was not always more
The size of the trace changed with the amount of first-language pretraining. Relative similarity rose from about 4% to 8% between 3,400 and 5,000 updates, peaked around 5,000 to 6,700 steps and began to taper by 8,400. That non-linear pattern was consistent across three of four language-switch directions; English-to-French models showed stronger traces with more pretraining.
Relearning was faster
When the models were retrained on the first language, those with earlier exposure reached 70% accuracy in 14.3% fewer steps than models trained only on the later language and 12.8% fewer than related-language controls. The 95% confidence intervals for those differences were 12.3% to 16.2% and 10.9% to 14.8%, respectively. In a further test that combined network layers, the smallest relearning gap appeared when the pre-phonemic layers were used, matching the location of the earlier trace.
A model result, not a human answer
The central limitation is that this was a modeling result, not a test of human learners. The authors say the systems did not develop through biological maturation, so the experiment cannot determine how maturation might interact with the learning dynamics.
Taken together, the results describe a property of this model system: access to the first-trained language fell to chance, while low-level similarity and faster relearning remained detectable. They do not determine how biological maturation would affect human language learning.
Paper data and sources
Original title: Lost but not erased: Finding traces of a forgotten language in neural speech models
Authors: Peter Plantinga, Charlotte Moore, Peter W. Donhauser et al.
Journal/Repository: arXiv
Status: Preprint, not yet peer-reviewed
First online: 2026-08-26
DOI: Not available
Original paper · Full text