文献类型：期刊文献

英文题名：NEURAL-VOX: NEURal auditory language decoding for voice and text reconstruction

作者：Jin, Zhishuo[1,2];Li, Dongdong[1,2];Zhou, Qin[1,2];Wang, Zhe[1,2]

机构：[1]Minist Educ, Key Lab Smart Mfg Energy Chem Proc, Shanghai 200237, Peoples R China;[2]East China Univ Sci & Technol, Shanghai 200237, Peoples R China

年份：2026

卷号：204

外文期刊名：NEURAL NETWORKS

收录：;EI(收录号：20262420904303);WOS:【SCI-EXPANDED(收录号:WOS:001799704900001)】；

基金：This work was supported in part by the National Natural Science Foundation of China under Grants 62276098.

语种：英文

外文关键词：Brain-computer interface (BCI); Neural decoding; Electroencephalograph (EEG); Cross-modal generation

摘要：Neural decoding of perceived linguistic content from non-invasive brain recordings remains a profound scientific challenge with transformative implications for assistive technologies. Existing approaches often struggle to generate intermediate representations, such as mel spectrograms or phonemes, and seldom integrate multi-modal information to enhance text decoding. This study presents a framework for decoding non-invasive brain activity into text, phoneme sequences, and mel-spectrogram-based acoustic representations, termed NEURAL-VOX. Leveraging a three-stage training strategy, NEURAL-VOX not only improves the accuracy of brain-to-text decoding, but also enables text generation to benefit from joint optimization with speech synthesis. By incorporating multi-scale frequency-domain analysis, our model more effectively captures the hierarchical structure of language processing in neural activity. Experiments across multiple datasets demonstrate that NEURAL-VOX achieves substantial gains over existing methods. The learned phoneme representations encode rich linguistic information and further strengthen text decoding, while model interpretability analysis reveals strong alignment with neurobiological patterns.