Introduction

Name: Sarah Badran
Student ID: 1782750
Course: PSYCH 403A1 - Neuroimaging and Neurostimulation
Assignment 6 - Option B: The 'Perfect' Hybrid System

Page 1 visual content
Page 1 visual content

1. Which Modalities and Why?

Electroencephalography (EEG) and functional near-infrared spectroscopy (fNIRS) are two of the most complementary non-invasive techniques available to cognitive neuroscience. EEG allows the recording of electric potentials from the brain that change at a rapid rate due to the summation of postsynaptic currents across cortical pyramidal neurons. EEG has the ability to record changes in electric potential very quickly (milliseconds) and minimal time lag because the changing electric field can propagate through the scalp and skull (Luck, 2014). However, the conducting layers of the head do smear information, and therefore accuracy in specifying the location varies by approximately a centimetre.

Functional near-infrared spectroscopy (fNIRS) measures changes in optical absorption due to the two types of hemoglobin: oxygenated (HbO) and deoxygenated (HbR), by emitting light between 700-900 nanometers. This light wavelength can penetrate a few millimetres into the cortical tissues, allowing for localized hemodynamic responses associated with the metabolism of neurons to be detected. While this methodology has lower temporal resolution and is slower (0.1-1 Hz) compared to EEG (1000 Hz), it provides spatial specificity with the goal of having temporal precision used during EEG. Overall, EEG and fNIRS provide information on neurovascular coupling (Yeung & Chu, 2022) or the cascade of events linking the firing of neurons and the local consumption of oxygen.

The hybrid methodology connects two important timescales: the electrophysiological millisecond range and the vascular second range. For instance, during a visual-attention task, EEG can pinpoint the P100 and N200 components, which reflect early sensory processing and selective-attention processing, while fNIRS maps the corresponding HbO increase in occipital and parietal cortices at the same time. Taken together, both systems allow researchers to subscribe to when a cognitive process is taking place with where in the cortex it is taking place. Because both systems are safe, portable, inexpensive, and tolerable for repeated measures across diverse populations, they are particularly well-suited to ecological experiments in non-laboratory scanners (Pinti et al., 2018).

Page 2 visual content
Page 2 visual content

2. How Would They Be Integrated?

The EEG-fNIRS hybrid proposed is a compact and ergonomically-designed cap evolving distinct dry metallic EEG electrodes coated with compatible optical transmission LED emission + photodiode detection combinations. The electrodes are positioned at the 64 standard 10-20 locations while the fNIRS optodes are interleaved based on a 30 mm spacing from the source to detector to create sampling through overlapping sensitivity volumes (Li et al., 2022). The electrodes and fNIRS optodes are isolated using a conductive mesh shielding to mitigate electromagnetic disturbance and light leakage. The complete configuration is lightweight (530 g) and has flexible padding with thermoplastic elastomers for comfort and passive ventilation to discourage sweating during time-intensive recordings.

The data acquisition occurs simultaneously with synchronized sampling clocks for EEG 1000 Hz and fNIRS 25 Hz. Both data streams are also time-locked via integrated hardware trigger pulses accurate to plus/minus 2 ms. These safeguards provide the close coupling necessary to identify between hemodynamic changes in arterial and venous blood oxygenation to earlier neural events. At the back of the cap, a compact processing unit executes preliminary amplification and artifact rejection before being wirelessly transmitted to a laptop or cloud server. Machine-learning processes on the GPU seamlessly integrate fast electrical bursts with slower oxygenation curves producing an integrated overview of unified cortical activation maps (Li et al., 2023).

User ergonomics are key considerations in the design. The cap utilizes stretch-fit materials, allowing it to accommodate different head sizes. In addition to its comfort, it features replaceable electrode pads and optode mounts specifically designed for hygiene. A rechargeable lithium-ion cell, capable of lasting 8-10 hours, provides power. Total radio-frequency emission remains below 1 mW/cm^2. EEG and fNIRS are both generally accepted as being non-significant risk for research purposes, which allows for repeated measures, in a safe manner, in research and clinical environments (Delpy & Cope, 1997). Therefore, the integration strategy strikes a balance between high-quality multimodal data and participant comfort, safety, and portability.

Page 3 visual content
Page 3 visual content

3. What Does Multimodal Data Look Like?

EEG and fNIRS produce streams of data simultaneously. EEG generates data as voltage traces (in microvolts) sampled at 1 kHz, whereas fNIRS data consists of concentration changes (derived from optical intensity) in oxygenated and deoxygenated hemoglobin (Delta[HbO], Delta[HbR]) ranging from 10 to 50 Hz (Chiarelli et al., 2017). The two modalities are co-registered to a head model post-artifact removal and band-pass filtering, which enables the generation of 4D activity maps (space x time x modality x participant). Subsequently, the EEG event-related potentials are juxtaposed on the oxygenation heatmaps for simultaneous visualizations of timing and activation spatial patterns of the neural substrates. Real-time data fusion on a GPU allows for sub-100 ms latency maps to be posted on the screen.

Page 4 visual content
Page 4 visual content

4. Technical and Practical Challenges

One major challenge is interference, motion artifacts, computational requirements, and privacy. The effects of crosstalk between optical and electrical components could be minimized through shielding and the optimal positioning of the sensors (Mussi et al., 2022). Motion artifacts from participant head movements and optode displacements can also be reduced through adaptive filters with accelerometer data or wavelet-based methods (Hossain et al., 2022). The degree of data synchronization across EEG and fNIRS platforms generates high data rates, requiring not only sufficient computational resources but also access to video graphics unit (GPU) accelerated pipelines for real-time fusibility and visualizations. Finally, safeguarding the security of participant data is of the utmost importance, most notably through end-to-end encryption, and role-based access controls and security (Pinti et al., 2018).

Page 5 visual content
Page 5 visual content

5. Real-World Applications

EEG-fNIRS hybrids are showing more promise in cognitive naturalistic experiments and clinical settings. These systems may provide researchers with the tools to observe natural fluctuations in cognitive workload, engaged attention, working memory, and decision-making in environments such as classrooms and workspaces (Pinti et al., 2018). Clinical efforts include the localization of seizure foci, assessing neurovascular functions, and aiding recovery from post-stroke physical rehabilitation with neurofeedback (Chen et al., 2023). In applications where the user performs intentional movements, such as rehabilitation of the motor domain or the operation of a brain-computer interface, results demonstrate an increase in performance classification when EEG and fNIRS features were fused earlier in the processing stream (Li et al., 2023). Once the portability of the fNIRS + EEG system becomes more adept, the prospect as a platform for monitoring mental states (i.e. cognitions) in real-world settings becomes plausible.

Page 6 visual content
Page 6 visual content

References

  1. Chen, J., Xia, Y., Zhou, X., Vidal Rosas, E., Thomas, A., Loureiro, R., Cooper, R. J., Carlson, T., & Zhao, H. (2023). fNIRS-EEG BCIs for Motor Rehabilitation: A Review. Bioengineering, 10(12), 1393. https://doi.org/10.3390/bioengineering10121393
  2. Chiarelli, A. M., Zappasodi, F., Di Pompeo, F., & Merla, A. (2017). Simultaneous functional near-infrared spectroscopy and electroencephalography for monitoring of human brain activity and oxygenation: a review. Neurophotonics, 4(04), 1. https://doi.org/10.1117/1.nph.4.4.041411
  3. Delpy, D. T., & Cope, M. (1997). Quantification in tissue near–infrared spectroscopy. Philosophical Transactions of the Royal Society of London. Series B: Biological Sciences, 352(1354), 649–659. https://doi.org/10.1098/rstb.1997.0046
  4. Hossain, M. S., Muhammad, Mamun, Ali, Ashrif, A., Serkan Kiranyaz, Amith Khandakar, Alhatou, M., Habib, R., & Hossain, M. M. (2022). Motion Artifacts Correction from Single-Channel EEG and fNIRS Signals Using Novel Wavelet Packet Decomposition in Combination with Canonical Correlation Analysis. Sensors, 22(9), 3169–3169. https://doi.org/10.3390/s22093169
  5. Li, R., Yang, D., Fang, F., Hong, K.-S., Reiss, A. L., & Zhang, Y. (2022). Concurrent fNIRS and EEG for Brain Function Investigation: A Systematic, Methodology-Focused Review. Sensors, 22(15), 5865. https://doi.org/10.3390/s22155865
  6. Li, Y., Zhang, X., & Ming, D. (2023). Early-stage fusion of EEG and fNIRS improves classification of motor imagery. Frontiers in Neuroscience, 16. https://doi.org/10.3389/fnins.2022.1062889
  7. Luck, S. J. (2014). An introduction to the event-related potential technique (2nd ed.). Mit Press.
  8. Mughal, N. E., Khan, M. J., Khalil, K., Javed, K., Sajid, H., Naseer, N., Ghafoor, U., & Hong, K.-S. (2022). EEG-fNIRS-based hybrid image construction and classification using CNN-LSTM. Frontiers in Neurorobotics, 16. https://doi.org/10.3389/fnbot.2022.873239
  9. Mussi, M. G., & Adams, K. D. (2022). EEG hybrid brain-computer interfaces: A scoping review applying an existing hybrid-BCI taxonomy and considerations for pediatric applications. Frontiers in Human Neuroscience, 16. https://doi.org/10.3389/fnhum.2022.1007136
  10. Pinti, P., Aichelburg, C., Gilbert, S., Hamilton, A., Hirsch, J., Burgess, P., & Tachtsidis, I. (2018). A Review on the Use of Wearable Functional Near-Infrared Spectroscopy in Naturalistic Environments. Japanese Psychological Research, 60(4), 347–373. https://doi.org/10.1111/jpr.12206
  11. Yeung, M. K., & Chu, V. W. (2022). Viewing neurovascular coupling through the lens of combined EEG–fNIRS: A systematic review of current methods. Psychophysiology, 59(6). https://doi.org/10.1111/psyp.14054