Beyond Parallel Tracking: Interactive Multi-Feature Fusion Drives Semantic Reconstruction from Non-invasive Brain Recordings

arXiv cs.CL · 18 hours ago

Researchers developed a multi-feature fusion framework for reconstructing semantic information from non-invasive brain recordings by combining static word embeddings (W2V) with dynamic contextual representations (GPT). The framework evaluated two integration approaches (linear concatenation and non-linear cross-attention), with cross-attention fusion achieving state-of-the-art performance in semantic reconstruction and text generation tasks. The approach addresses the representational mismatch between neural signals and semantic features by simulating how the brain simultaneously integrates word attributes and context during language comprehension.