The goal of this study is to investigate whether high-density electrocorticography (ECoG) signals recorded from the surface of the brain can be used to decode neural representations of Mandarin Chinese speech features, including lexical tone, without requiring overt speech movements. The study focuses on the development and evaluation of decoding algorithms based on neural activity recorded during clinically indicated neurosurgical procedures.
The main questions it aims to answer are:
Can high-density ECoG signals be decoded to reconstruct neural representations of Mandarin Chinese speech features, particularly lexical tone?
Can neural activity recorded during silent auditory speech imagery be decoded to reconstruct tone-specific speech representations without actual articulation?
The study includes two groups of adult patients with neurological conditions who require cortical electrode placement as part of clinically indicated care:
A intraoperative high-density ECoG temporary coverage group, enrolling approximately 50 patients with functional-area glioma or drug-resistant epilepsy who undergo awake neurosurgery with temporary high-density ECoG coverage for clinical functional mapping.
A permanent high-density ECoG implantation group, enrolling approximately 10 patients with severe speech or language impairment caused by neurological conditions such as stroke, brain tumors, amyotrophic lateral sclerosis (ALS), or locked-in syndrome, who receive permanent high-density cortical electrode implantation for long-term monitoring.
Participants will:
Complete preoperative clinical assessments as part of standard medical care, including brain imaging, language function evaluation, and routine neurological assessments
Undergo clinically indicated awake neurosurgical procedures during which high-density ECoG electrodes are placed on the cortical surface for clinical functional localization
Perform language-related tasks, such as listening to speech, imagining speech, and limited spoken responses, while brain electrical activity is recorded for approximately 20-30 minutes during surgery, without altering standard surgical procedures
For participants in the permanent implantation group, participate in long-term follow-up visits approximately every 2 weeks or monthly for up to 12 months after surgery, including evaluation of signal quality and research-related analysis and optimization of decoding algorithms
All surgical procedures involving temporary or permanent electrode placement are performed for clinical indications and have been approved through institutional ethical and scientific review. Participation in this study does not alter standard clinical care for the temporary recording group and does not require additional clinical procedures beyond routine treatment.
This research aims to support the long-term development of silent brain-to-speech communication technologies for individuals with severe speech or motor impairments and to improve understanding of how frontal, parietal, and temporal brain regions represent imagined speech in tonal languages such as Mandarin Chinese.
Age between 20 and 80 years.
Ability and willingness to provide informed consent and comply with study procedures.
No severe preoperative emotional or consciousness disorders that would preclude participation in experimental tasks.
For patients undergoing intraoperative temporary high-density ECoG coverage: Patients with cerebral eloquent-area gliomas or refractory epilepsy undergoing awake craniotomy as part of standard clinical management.
Significant mass effect with severe intracranial hypertension precluding awake craniotomy or electrode implantation.
Severe neurological dysfunction that would prevent participation in study procedures (except for speech or language impairment in the permanent implantation cohort).
Contraindications to MRI scanning or awake craniotomy, including incompatible implanted medical devices, severe claustrophobia, or obstructive sleep apnea syndrome.
Severe psychiatric disorders or cognitive impairment preventing participation in treatment or follow-up assessments (Mini-Mental State Examination score <24).
Single group
Intraoperative high-density ECoG recording
Procedure/SurgeryElectrode implantation is performed via neurosurgical awake craniotomy. Following intraoperative awakening, direct cortical electrical stimulation is applied to localize language functional areas while the subject performs language tasks. Subsequently, high-density electrocorticography is employed to record neural electrical activity. For the temporary coverage group, this enables precise localization of language functional areas; for the permanent implantation group, this facilitates determination of optimal electrode implantation sites.
Permanent high-density ECoG implantation
Procedure/SurgeryIn the permanent implantation group, subjects demonstrating satisfactory intraoperative electrode signals who voluntarily consent to permanent electrode implantation will undergo permanent electrode placement and long-term follow-up. Postoperative follow-up is conducted biweekly or monthly, encompassing: signal quality assessment, analysis of neural encoding mechanisms during speech imagery, and calibration of brain-computer interface decoding algorithms. The trial duration for each subject is approximately 13 months.
Electrode Signal Quality Assessment
Electrode reliability over time is evaluated through electrode impedance, signal-to-noise ratio (SNR), and responsive channel proportion. Low electrode impedance, high SNR, and high responsive channel proportion indicate good signal quality reliability of the device.
Decoding Classification Accuracy
The proportion of correctly classified targets by the decoding model is evaluated for recognition categories (e.g., syllables, words, or sentences).
Encoding Model Analysis
An L2-regularized time-receptive-field (TRF) model (time window: -400 to 400 ms) with parallel five-fold cross-validation is employed to calculate the coefficient of determination (R²) between neural signals and AKT, phonemic features, and Mel-spectrogram, respectively, enabling comprehensive evaluation of the encoding capacity of fronto-parieto-temporal neurons for articulatory and acoustic features.
Adverse Event Incidence Rate
All adverse events are collected through active monitoring (intraoperative observation, postoperative daily ward rounds), adverse event report forms, imaging examinations, laboratory tests (routine blood test, cerebrospinal fluid analysis), and patient complaints. Incidence rates are calculated separately for each adverse event category (intraoperative electrical stimulation-related, device-related, drug-related, pregnancy-related, etc.) using the formula: number of patients experiencing specific adverse events divided by total enrolled patients multiplied by 100%.
Click on the lead sponsor to view all of their trials.
