In this program, the investigators will develop novel multimodal neural-behavioral-physiological monitoring tools (software and hardware), and machine learning models for mental states within social processes and beyond. The tools consist of a multimodal skin-like wearable sensor for physiological and biochemical sensing; a conversational virtual human platform to evoke naturalistic social processes; audiovisual affect recognition software; synchronization tools; and machine learning methods to model the multimodal data. The investigators will demonstrate the tools in healthy subjects without neural recordings and in patients with drug-resistant epilepsy who already have intracranial EEG (iEEG) electrodes implanted based on clinical criteria for standard monitoring to localize seizures, which is unrelated to our study.
Patients who suffer from drug-resistant epilepsy and already have intracranial EEG (iEEG) electrodes implanted based on clinical criteria for their standard seizure localization (unrelated to our study) will be eligible. Most patients are healthy adults outside of their epilepsy.
Subjects >= 18 are only included in this study.
All patients with the above conditions and with already-implanted electrodes who are willing to participate and able to cooperate and follow research instructions will be recruited.
None
Single group
Virtual human social interaction
BehavioralThe participants will perform a social interaction with the virtual human agent platform to assess mental states.
Cold pressor test
BehavioralHealthy participants will immerse one hand in cold water for less than 3 minutes, while physiological data will be collected by the new wearable sensor from the other hand.
Positive and Negative Affect Schedule (PANAS)
Momentary self-reports of the Positive and Negative Affect Schedule (PANAS) will be administered in a tablet-based application, which measures momentary positive and negative affect. These self-reports return scores indicative of momentary mental state. Self-reports are obtained intermittently from the participants.
Visual Analog Scale (VAS)
Momentary self-reports of the Visual Analog Scale (VAS) of depression, anxiety, and energy will be administered in a tablet-based application. These are fast self-reports to indicate the affect level along a line from none to prominent. These self-reports return scores indicative of momentary mental state. Self-reports are obtained intermittently from the participants.
Hamilton Depression Rating Scales (HAMD-6)
Momentary self-reports of Hamilton Depression Rating Scales (HAMD-6) will be administered in a tablet-based application, which is widely used to measure the severity of the depressive state. These self-reports return scores indicative of momentary mental state. Self-reports are obtained intermittently from the participants.
Hamilton Anxiety Rating Scales (HAM-A)
Momentary self-reports of Hamilton Anxiety Rating scales (HAM-A) will be administered in a tablet-based application, which is widely used to measure the severity of the anxiety state. These self-reports return scores indicative of momentary mental state. Self-reports are obtained intermittently from the participants.
Physiological measurements - heart rate variability (HRV)
Heart rate variability will be collected from each participant by using either commercial systems or the new wearable sensor.
Physiological measurements - skin conductance (SC)
Skin conductance will be collected from each participant by using either commercial systems or the new wearable sensor.
Physiological measurements - cortisol level
Cortisol level will be collected from each participant by using either commercial systems or the new wearable sensor.
Neural measurements
In epilepsy patients who already have intracranial EEG recordings for their standard clinical monitoring (unrelated to this study), we will also obtain iEEG activity measurements. EEG power will be the main focus of the analysis, but the raw signal will also be analyzed.
Click on the lead sponsor to view all of their trials.
University of Southern California
Lead sponsor
National Institute of Mental Health (NIMH)
Collaborator
