An EEG-to-speech model called SENSE had the best value on every listed measure when both the audio and the subject were unseen, and when the subject alone was unseen. In the audio-only condition, it led every listed measure except CLIP-Sim, where it tied FESDE at 0.7601. Those results are benchmark comparisons, not evidence of unrestricted thought decoding. The authors also say reliable word-level reconstruction remains challenging, with WER values near 1.
How the system works
SENSE combines a graph-based EEG encoder defined over electrode geometry with EEG Semantic Conditioning, or ESC. ESC aligns EEG with a pretrained semantic space, a representation of meaning, and uses only congruent trials. In plain language, the model builds the layout of scalp sensors into its EEG representation and trains that representation to line up with meaning-related information.
Training combined EEG self-reconstruction, phoneme supervision and semantic conditioning. At inference, only EEG is required, and the waveform decoder operates without text.
A controlled test of generalization
The tests used a public N400 EEG speech-perception corpus. It originally contained 24 subjects, each with 440 sentences recorded using 128-channel EEG at 512 Hz. Four unreliable subjects were excluded, leaving 20 for analysis. Eighteen supplied training and validation data, while two were held out for subject testing. The training set contained 7,200 EEG-speech pairs from 18 subjects and 400 sentences per subject.
The evaluation separated three situations: 720 pairs with unseen audio, 800 with an unseen subject, and 80 with both unseen. Acoustic metrics used all trials, while semantic metrics were calculated only on congruent trials.
SENSE was trained for 200,000 iterations, while the baselines were trained for 100,000, and results were averaged over five random seeds. Supplementary material provides anonymized code and instructions for setup, data preparation, training and inference. Pretrained checkpoints were not included and are planned for release with a de-anonymized version upon acceptance.
Where the model gained ground
In the main comparison, SENSE had the best value on every listed measure when both audio and subject were unseen, and when the subject alone was unseen. When only the audio was unseen, it was best on every listed measure except CLIP-Sim, where SENSE and FESDE both reported 0.7601.
The cross-subject scaling test found that SENSE surpassed FE-Phoneme's 18-subject WER result using two training subjects. By eight subjects, it matched or exceeded that model on MCD, Mel-Correlation and STOI.
The reported WER was 0.9948 for full SENSE, compared with 1.0323 without ESC, 1.0355 without the graph neural network and 1.0488 without the CTC component. The full model also had better reported metrics than the tested variants in the ablation.
Meaning was easier to match than words
A separate representation test compared matched EEG-text pairs with mismatched pairs. Across 800 held-out trials, matched pairs had a similarity score of 0.37 versus 0.25 for mismatches, a gap of 0.12. Among 400 congruent trials, the scores were 0.40 and 0.25, a gap of 0.15. The result indicates alignment between the tested representations, not reliable word decoding.
The researchers also initialized from N400-trained weights and fine-tuned the model on EEG from naturalistic listened-speech data. Across 19 participants, matched EEG-text similarity was 0.602 versus 0.517 for random mismatches, a gap of 0.085. None of 5,000 random permutations reached the observed gap. This was a representation-level analysis and did not evaluate naturalistic speech reconstruction.
What the results do not show
The main waveform test was confined to the controlled N400 corpus. The authors say WER values near 1 show that reliable word-level reconstruction remains challenging, while the naturalistic experiment measured EEG-text alignment rather than speech synthesis.
An internal attribution analysis highlighted temporal-auditory channels TP7 and TP8, central motor channels C3 and C4, and centro-parietal CPz. The authors treated that pattern as model-internal behavior, not as evidence locating the neural sources that generated the signal.
The study reused a previously released public EEG benchmark collected under prior ethical approval. It reports no new participant recruitment, neural data collection or sensing hardware, and cautions against interpreting the work as unrestricted thought decoding.
The study is an arXiv preprint, version 2 dated 8 October 2026. It lists support from National Research Foundation of Korea grant RS-2025-02217071, Institute of Information & Communications Technology Planning & Evaluation grant RS-2021-II211341, and the AI Seoul Tech Research Support Program of the Seoul Future Foundation.
Paper data and sources
Original title: SENSE: Semantic Neural Speech Synthesis from Brain Dynamics via Spatial Graph Encoding
Authors: Jisoo Park, Seonghak Lee, Hyojin Park, Junseok Kwon
Journal/Repository: arXiv
Status: Preprint, not yet peer-reviewed
First online: 2026-09-29
DOI: Not available
Original paper · Full text