Introduction

Modern neuroscience relies on brain imaging techniques that balance competing priorities such as spatial versus temporal resolution, depth penetration versus portability, and precision versus cost (Deffieux et al., 2021). Functional magnetic resonance imaging (fMRI) provides detailed, whole-brain visualization of hemodynamic activity but remains slow, immobile, and expensive. In contrast, functional near-infrared spectroscopy (fNIRS) offers lightweight, wearable monitoring of cortical blood oxygenation, yet its depth penetration is limited to superficial cortical regions (Scarapiccha et al., 2017). To bridge this divide, this paper proposes an optimized hybrid system that combines fMRI and fNIRS. The aim of this paper is to integrate the spatial precision and whole-brain mapping of fMRI with the temporal flexibility and portability of fNIRS. Together, these modalities could enable researchers to map deep-brain networks within the scanner and then monitor corresponding cortical activity continuously in natural environments (see Figure 1 for an overview of the proposed workflow).

Page 1 visual content
Page 1 visual content

Which Modalities and Why

Functional MRI (fMRI) measures brain activity by detecting changes in the magnetic properties of blood, known as the blood-oxygen-level-dependent (BOLD) signal, which reflects local neural metabolism (Buxton, 2013). It provides millimeter-scale spatial resolution across the entire brain, including deep structures such as the thalamus and basal ganglia. However, because the BOLD response depends on slow vascular changes, the temporal resolution of fMRI is limited to approximately one to two seconds. Although fMRI remains indispensable for identifying precise activation patterns during cognitive tasks, its immobility, cost, and sensitivity to motion restrict its use to controlled laboratory environments.

Functional near-infrared spectroscopy (fNIRS), by contrast, emits light in the near-infrared range (700โ€“900 nm) to quantify relative changes in oxy- and deoxy-hemoglobin concentrations in the upper layers of the cortex (Kim et al., 2017). It is completely noninvasive, silent, and portable, allowing researchers to record brain activity in naturalistic settings such as classrooms, clinics, or workplaces. fNIRS offers superior temporal sampling (up to 10โ€“100 Hz) and greater tolerance to movement compared to fMRI, though it cannot image deep brain structures.

Combining these modalities leverages their complementary strengths. fMRI can establish an individualโ€™s baseline neural architecture with sub-millimeter accuracy, while fNIRS can track those same cortical regions repeatedly and flexibly in real-world environments. Together, they offer a new level of multimodal coverage that no single imaging technique can achieve alone (Scarapiccha et al., 2017).

Page 2 visual content
Page 2 visual content

Integration of the Two Modalities

In the proposed hybrid system, participants would first undergo an fMRI session while wearing a fiber-optic, MR-compatible fNIRS cap. Adopting the methodology of a study conducted by Zhang et al. (2006), the cap would be constructed using non-metallic components to prevent magnetic interference and would allow simultaneous recording of BOLD and optical signals. These initial calibration sessions would align each fNIRS channel with its corresponding fMRI voxel, creating a subject-specific map that links optical signals to anatomical coordinates. Once the calibration is complete, the same fNIRS cap could then be used in portable, real-world settings, where new data would be continuously compared to the original fMRI reference (as shown in Figure 1).

The temporal synchronization between modalities presents another challenge. While fMRI records one volume roughly every two seconds, fNIRS can sample between ten and fifty times per second. To merge these signals, temporal interpolation and lag-compensation algorithms would be used to align the slower hemodynamic trends of fMRI with the faster optical measurements (Yuan & Ye, 2013). Shared event markers, such as auditory tones or button presses, would further help synchronize data across both systems. A Kalman-filter model could combine the BOLD signal with the rapid fNIRS fluctuations, producing a continuous and temporally smoothed estimate of neural activity at sub-second precision (Durantin et al., 2016).

Spatial integration would rely on using the individualโ€™s high-resolution structural MRI scan as a three-dimensional anatomical template. Each fNIRS optode position could be co-registered to the scalp landmarks visible in that MRI, ensuring accurate localization of surface signals. Advanced machine-learning models could even use fNIRS surface patterns to predict deeper BOLD activity, effectively extending optical coverage beyond its natural limit (Liu et al., 2015). In this way, fMRI provides the scaffolding, and fNIRS supplies the dynamic, real-time updates.

Page 3 visual content
Page 3 visual content

Nature of the Multimodal Data

The hybrid system would produce data at multiple scales. fMRI generates volumetric voxel matrices representing BOLD intensity over time, whereas fNIRS provides continuous time-series traces of changes in oxy- and deoxy-hemoglobin concentration (Yuan & Ye, 2013). Once co-registered and temporally aligned, the combined output could be visualized as dynamic cortical activation maps, with color-coded oxygenation changes overlaid on the anatomical brain model. An example of this type of fused data visualization is illustrated in Figure 2.

Page 4 visual content
Page 4 visual content

Technical and Practical Challenges

Developing and deploying such a hybrid system involves several engineering and logistical challenges. Magnetic compatibility remains a key issue, as conventional optodes can distort the MRI field. This can be mitigated by constructing optodes from plastic fiber housings and employing optical isolation to prevent interference (Duffy et al., 2015). Differences in sampling rates between fMRI and fNIRS can be addressed through temporal resampling and model-based fusion algorithms.

Motion artifacts are another persistent concern in fNIRS, particularly when subjects move freely during real-world data collection. Incorporating motion sensors and short-channel regression techniques can help correct for this problem (Brigadoi et al., 2014). The enormous volume of multimodal data also requires significant computational power; real-time compression and GPU-based analysis could help maintain manageable data loads (Tran & Cambria, 2018). Cost represents an additional challenge, but this could be offset by using a single fMRI session for calibration and relying on affordable fNIRS hardware for extended field use. Finally, user comfort must be prioritized through the development of lightweight, flexible, and wireless fNIRS caps that participants can wear for hours without discomfort.

Page 5 visual content
Page 5 visual content

Real-World Applications

A hybrid fMRIโ€“fNIRS system could transform both basic neuroscience and clinical practice. In rehabilitation, fMRI could identify the networks impaired by stroke, while portable fNIRS would track cortical reorganization during months of therapy (Golestani et al., 2013; Cao et al., 2015). In mental-health research, baseline fMRI scans could characterize emotion-regulation networks, and ongoing fNIRS monitoring could detect subtle shifts signaling relapse or recovery (Ellard et al., 2018; Huang et al., 2025). The system could also advance neuroergonomics by monitoring cognitive workload in pilots, drivers, and surgeons. fNIRS caps, calibrated to each userโ€™s fMRI profile, could detect fatigue or attentional lapses in real time (Harrivel et al., 2012). In developmental neuroscience and education, researchers could use the hybrid approach to follow brain maturation over time, combining laboratory precision with naturalistic observation in real-world learning environments (Atteveldt et al., 2018).

Page 6 visual content
Page 6 visual content

Limitations and Future Directions

Despite its potential, this hybrid approach faces several inherent limitations. fNIRS remains restricted to superficial cortical layers, making it difficult to directly observe deep-brain networks. Both modalities rely on hemodynamic responses, which are indirect and temporally delayed indicators of neuronal activity. Furthermore, fMRI calibration sessions are expensive and limited to specialized facilities.

Future advances may help overcome these constraints. The emergence of low-field or portable MRI systems could make calibration more accessible (Morris, 2025). Improvements in optode design, such as quantum-dotโ€“based light sources and ultrafast photodiodes, may enhance sensitivity and penetration depth. Finally, AI-driven real-time fusion algorithms could predict whole-brain dynamics from a reduced set of surface measurements, paving the way for truly wearable neuroimaging systems. Together, these innovations would evolve the hybrid model into a continuous, scalable, and ecologically valid method for mapping the human brain.

Page 7 visual content
Page 7 visual content

Conclusion

By combining fMRIโ€™s unparalleled spatial resolution with fNIRSโ€™s accessibility and temporal responsiveness, this hybrid framework represents a powerful bridge between laboratory-based neuroimaging and real-world monitoring. It offers a scientifically rigorous yet flexible approach for studying cognition, emotion, and clinical recovery over time. Integrating these modalities not only enhances the quality and continuity of neural data but also moves neuroscience closer to achieving truly comprehensive, multiscale mapping of the human brain.

๐Ÿ“š References

  1. Deffieux, T., Demenรฉ, C., & Tanter, M. (2021). Functional Ultrasound Imaging: A New Imaging Modality for Neuroscience. Neuroscience, 474, 110-121.
  2. Scarapicchia, V., Brown, C., Mayo, C., & Gawryluk, J. R. (2017). Functional Magnetic Resonance Imaging and Functional Near-Infrared Spectroscopy: Insights from Combined Recording Studies. Frontiers in human neuroscience, 11, 419. https://doi.org/10.3389/fnhum.2017.00419
  3. Buxton, R. B. (2013). The physics of functional magnetic resonance imaging (fMRI). Reports on Progress in Physics, 76(9). https://doi.org/10.1088/0034-4885/76/9/096601
  4. Kim, H. Y., Seo, K., Jeon, H. J., Lee, U., & Lee, H. (2017). Application of Functional Near-Infrared Spectroscopy to the Study of Brain Function in Humans and Animal Models. Molecules and Cells, 40(8), 523โ€“532. https://doi.org/10.14348/molcells.2017.0153
  5. Zhang, X., Toronov, V. Y., & Webb, A. G. (2006). Integrated measurement system for simultaneous functional magnetic resonance imaging and diffuse optical tomography in human brain mapping. Review of Scientific Instruments, 77(11), 114301. https://doi.org/10.1063/1.2364138
  6. Yuan, Z., & Ye, J. (2013). Fusion of fNIRS and fMRI data: identifying when and where hemodynamic signals are changing in human brains. Frontiers in Human Neuroscience, 7. https://doi.org/10.3389/fnhum.2013.00676
  7. Durantin, G., Scannella, S., Gateau, T., Delorme, A., & Dehais, F. (2016). Processing Functional Near Infrared Spectroscopy Signal with a Kalman Filter to Assess Working Memory during Simulated Flight. Frontiers in Human Neuroscience, 9. https://doi.org/10.3389/fnhum.2015.00707
  8. Liu, N., Cui, X., Bryant, D. M., Glover, G. H., & Reiss, A. L. (2015). Inferring deep-brain activity from cortical activity using functional near-infrared spectroscopy. Biomedical Optics Express, 6(3), 1074. https://doi.org/10.1364/boe.6.001074
  9. Yang, L., & Wang, Z. (2025). Applications and advances of combined fMRI-fNIRs techniques in brain functional research. Frontiers in Neurology, 16. https://doi.org/10.3389/fneur.2025.1542075
  10. Boubela, R. N., Kalcher, K., Huf, W., Naลกel, C., & Moser, E. (2016). Big Data Approaches for the Analysis of Large-Scale fMRI Data Using Apache Spark and GPU Processing: A Demonstration on Resting-State fMRI Data from the Human Connectome Project. Frontiers in Neuroscience, 9. https://doi.org/10.3389/fnins.2015.00492
  11. Duffy, B. A., Choy, M., Chuapoco, M. R., Madsen, M., & Lee, J. H. (2015). MRI compatible optrodes for simultaneous LFP and optogenetic fMRI investigation of seizure-like afterdischarges. NeuroImage, 123, 173โ€“184. https://doi.org/10.1016/j.neuroimage.2015.07.038
  12. Brigadoi, S., Ceccherini, L., Cutini, S., Scarpa, F., Scatturin, P., Selb, J., Gagnon, L., Boas, D. A., & Cooper, R. J. (2014). Motion artifacts in functional near-infrared spectroscopy: A comparison of motion correction techniques applied to real cognitive data. NeuroImage, 85, 181โ€“191. https://doi.org/10.1016/j.neuroimage.2013.04.082
  13. Tran, H.N., & Cambria, E. (2018). GPU-based Commonsense Paradigms Reasoning for Real-Time Query Answering and Multimodal Analysis. ArXiv, abs/1807.08804.
  14. Golestani, A.-M., Tymchuk, S., Demchuk, A., & Goodyear, B. G. (2012). Longitudinal Evaluation of Resting-State fMRI After Acute Stroke With Hemiparesis. Neurorehabilitation and Neural Repair, 27(2), 153โ€“163. https://doi.org/10.1177/1545968312457827
  15. Cao, J., Khan, B., Hervey, N., Tian, F., Delgado, M. R., Clegg, N. J., Smith, L., Roberts, H., Tulchin-Francis, K., Shierk, A., Shagman, L., MacFarlane, D., Liu, H., & Alexandrakis, G. (2015). FNIRS-based evaluation of cortical plasticity in children with cerebral palsy undergoing constraint-induced movement therapy. Proceedings of SPIE, the International Society for Optical Engineering/Proceedings of SPIE, 9305, 93050N93050N. https://doi.org/10.1117/12.2076995
  16. Huang, A., Wang, R., Wen, A., Xu, L., Li, N., Gao, Y., Lu, W., Guo, S., Wang, J., & Wang, L. (2025). Clinical value of predicting relapse within 3 months in alcohol-dependent patients using fNIRS in verbal fluency task. Scientific Reports, 15(1), 5283โ€“5283. https://doi.org/10.1038/s41598-025-89775-7
  17. Harrivel, A.R., Hylton, A., & Hearn, T.A. (2012). Best Practices for the Application of Functional Near Infrared Spectroscopy to Operator State Sensing.
  18. van Atteveldt, N. M., van Kesteren, M., Braams, B., & Krabbendam, L. (2018). Neuroimaging of learning and development: improving ecological validity. Frontline Learning Research, 6(3), 186โ€“203. https://doi.org/10.14786/flr.v6i3.366
  19. Morris, J.W. (2025). Slug-Mapper: Magnetic Scanner for Ultra Low-Field MRI Scanners.