Combining event-related optical signal (EROS) and time-resolved functional infrared spectroscopy (TR-fNIRS)

EROS and TR-fNIRS report different kinds of data in regards to brain activity. EROS detects changes in the light scattering properties of neural tissue, thought to be related to individual neuronal activity. The latter uses the same light signal and photon arrival time measurement technology (NIRS) to track oxygenated and deoxygenated hemoglobin changes. This report is a brief, cursory proposal of combining EROS and time-resolved fNIRS into a working hybrid system as a literature review. The table below compares the three relevant neuroimaging techniques on a wide array of dimensions, from spatial and temporal resolution to real-world accessibility that will be discussed in the rest of this report.

fNIRS has good spatial resolution, although not as powerful as fMRI, but poorer temporal resolution due to the detection of blood-flow responses being slower (on the scale of seconds). With its strengths in both temporal (on the scale of milliseconds) and spatial recording, EROS is prone to signal contamination and produces a lower signal-to-noise ratio due to measurements being limited to the cerebral cortex (Gratton & Fabiana, 2001). fNIRS detection also has limited depth, causing it to suffer from the same problem. Time-resolved NIRS enhances the depth sensitivity of NIRS by recording the time it took for single photons to arrive at the detector (light that travels farther into brain tissue takes longer, or has a longer “time of flight,” than light travelling closer to the surface) (Abdalmalak et al., 2020). This greater depth penetration reduces scalp and superficial tissue contamination to the signal.

The main benefit of combining these specific techniques is the combination of hemodynamic data with neuronal activation responses. Detection of light scattering in EROS, although limited to only structures up to around 3 cm of the brain, is still better than EEG and EROS also compensates for spatial resolution lacking by ERPs (Tse et al., 2007). Secondly, time-resolved techniques can provide absolute values (in hemoglobin content) as opposed to fNIRS alone. These technologies also make up for the poorer temporal resolution in fMRI.

The devices used in time-resolved techniques are more complex than fNIRS. Furthermore, in the integration of EROS with TR-fNIRS, the main obstacle is that light scattering itself limits depth sensitivity, as diffusion of photons in a medium decreases photons detected in one focused direction (Torricelli et al., 2013). This problem would have to be explored and solved to make this combined system an effective one. EROS and fNIRS can be used to measure responses concurrently, modulating light beams to gain time-resolved responses. However, building in the separate measurement systems for both EROS and TR-fNIRS requires sophistication, for instance in synchronizing temporal measurements. The more millisecond scale of neural activity detected by EROS is to be synchronized with the slower hemodynamic responses detected by (TR-)fNIRS. One solution is using a single clock to time-lock measurements to specific events. Thus, we synchronize data at the occurrence or onset of specific events, such as elicited responses to experimental stimulus.

The same optodes can be used to emit and detect light for this multimodal system to be placed on the scalp. Although, again, the measurement systems for detecting EROS and for TR-fNIRS may need more sophistication. To produce anatomical images containing spatial data, we can scan cerebral images using MRI, then, after simultaneously acquiring date of light scattering properties and time-resolved hemodynamic changes from subjects, optode locations can be projected onto the MRI reconstructions (Chen et al., 2017). There are many steps to this procedure that can be time-consuming and computationally costly.

The detection and measurement devices of this multimodal system are ones already employed in their independent uses. To obtain the separate time-resolved data vs. EROS data—oxygenated hemoglobin concentrations and changes in light scattering/absorption properties in neural tissue respectively—two data can be collected: 1) the intensity of the light reaching the detector and 2) the time-of-flight of individual photons reaching the detectors. Mapping the resulting EROS and hemoglobin concentration readings should be displayed in combined activation topographic maps of the target brain areas across multiple snapshots in time for comparison, especially because EROS data is differential. Photon delay or time-of-flight should give spatial information about what area of the brain the signal is coming from. Oxygenated and deoxygenated hemoglobin concentrations at differing layers of the brain provides spatial depth information. Meanwhile, calculations of EROS derived from the light reaching the detector all-in-all is a better tell of temporal information. Activation maps and time course graphs would likely be based on elicited or target events in experimental situations.

The figures above display example sketches of time course graphs for 1) EROS events based on photon delay and 2) oxygenated blood concentration levels time stamps. We can overlap oxygenated hemoglobin concentration and deoxygenated hemoglobin concentration. The timing is specific to a specific event, for example, a stimulus presentation or a simple motor task. By comparing these graphs, information that can be attained from EROS and TR-fNIRS, we can get temporal and spatial information about changes in activity in neural tissue and blood oxygen. In essence, data from this multimodal data will likely be richer when focused on responses to specific events.

It is especially because these individual systems are still emerging technologies that there are still challenges to be faced. Individually, EROS and fNIRS are relatively low cost and both are attractive options because of their portability. On the other hand, time-resolved NIRS technology is bulky, limiting dynamic range and portability, and is costly (Gratton & Fabiani, 2009). This would be an obstacle in widely introducing this multimodal system into labs and clinical settings. It also puts us far from more widespread, commercial uses. For instance, among detection systems in TR-NIRS such as non-linear optic gating, time-correlated single-photon counting (TCSPC), and the use of time-gated intensified charge coupled device (ICCD), the streak camera is one of significant consideration due to its high temporal resolution (1-10 picoseconds) and usability as a multi-channel detector (Torricelli et al., 2013) (more channels improve spatial resolution). Its greatest drawbacks are its costs and complexity, hindering it from being more widely used in clinical settings. Availability and costs for light sources and data acquisition for different techniques also pose a challenge, especially when the choices of devices to use narrow when we select for faster temporal detection (Contini et al., 2006). In terms of light sources, “there is always a trade-off between output power and pulse duration” (Torricelli et al., 2013). As an example, for pulsed diode lasers, to obtain a less than 100 picosecond pulse duration, an output (capacity) power is preferred at less than 1mW. This choice affects its clinical usability. All of these considerations put together explain the lack of commercial devices for TR-NIRS systems. It would be useful for TR-fNIRS to be further tested in clinical and BCI applications to strengthen our understanding of how it could work in combination with EROS.

These obstacles don’t diminish the usefulness of integrating these systems together. There are a number of existing studies exploring research questions and clinical applications where time-resolved techniques combined with EROS measurements would be useful. One study was mentioned above: whether it would be effective to detect hemodynamic shifts earlier using EEG combined with fNIRS (Jawad Khan et al., 2018). The study’s practical purpose in exploring this was in an attempt to reduce the time it took for mental commands to be detected for a quicker BCI response by having the detection devices predict them ahead of time. The strain that bulky time-resolved NIRS technology would put on clients, however, remains a drawback. Still, the idea of using different types of neuroimaging systems to create a smarter, more user-responsive single multimodal system is compelling in the world of BCIs. Similarly, some recent research (e.x., Abdalmalak et al., 2017; Abdalmalak et al., 2020) studied TR-fNIRS as a base for BCIs, particularly medical BCIs geared to patients with disorders of consciousness (DOC), because of the advantage of its depth sensitivity. Is a multimodal EROS and TR-fNIRS approach toward BCIs for patients with disorders of consciousness (DOC) not only a feasible one, but one that enhances current technologies? Can this technique generate BCI actions faster and more accurately from sharper data from brain regions, produced by statistically comparing neuronal changes and hemodynamic data? Another area of interest is measuring and studying cognitive load. fNIRS has been used in studies of cognitive load. For example, a recent study by Karmakar and colleagues (2023) generated topographical brain images in a fNIRS-based experiment exploring a deep learning model convolutional neural network (CNN) to classify different brain responses into different cognitive states. Stepping back to look at the bigger picture, we can ask how the proposed multimodal system in this report, with its greater depth sensitivity, can provide richer data to help us test and develop deep learning models in clinical and neuroscientific studies? Can this system extract clearer data that can further differentiate mental states—a problem addressed in the study discussed above—to improve classification accuracy? How can this help our understanding of cognitive load? As we move towards enhancing the portability and accessibility of TR-fNIRS and consider it being used in tandem with EROS detection, there is a potential for greater insight into these questions and real-world, clinical matters.