Advancing the study of neural networks depends on the ability to map where, when, and how brain activity unfolds in response to both natural cognitive processes and neurological dysfunctions. Yet each neuroimaging modality imposes inherent trade-offs that can limit the completeness and interpretability of the information obtained. These constraints have made multimodal approaches that combine neural stimulation with complementary imaging techniques increasingly necessary. Integrating transcranial direct current stimulation (tDCS), functional magnetic resonance imaging (fMRI), and functional near-infrared spectroscopy (fNIRS) into a single trimodal framework represents a promising strategy for overcoming the limitations of single-modality designs. Such a system offers whole-brain spatial coverage, surface-level temporal sensitivity, and direct causal manipulation of neural activity, enabling a more comprehensive characterization of large-scale brain function.
A Trimodal Approach to Brain Function: Combining tDCS, fMRI, and fNIRS for Mechanistic Insight
Introduction
Combining Modalities
Functional magnetic resonance imaging (fMRI) provides detailed spatial maps but exhibits slow temporal dynamics due to the nature of the blood-oxygenation-level-dependent signal (Cui et al., 2010). Methods with stronger temporal resolution often fail to reach deeper brain structures and are highly susceptible to noise, which complicates interpretation. Additional concerns involve ecological validity and the difficulty of translating laboratory findings into real-world clinical settings. Integrating imaging tools that compensate for one another's weaknesses has therefore become an increasingly important strategy. Transcranial direct current stimulation (tDCS), a noninvasive method that modulates cortical excitability, is frequently paired with neuroimaging to investigate its mechanisms of action (Filmer et al., 2014). With the combination of tDCS, fMRI, and fNIRS, researchers can move beyond correlational interpretations and gain direct insight into how neural networks reorganize during and after stimulation.
Rationale for Trimodal Approach
The rationale for combining tDCS, fMRI, and fNIRS is grounded in the idea that each modality compensates for the limitations of the others. Together, these tools address the core constraints of human neuroscience, including resolution, depth, and causal manipulation. fMRI contributes macroscale spatial resolution across the entire brain, including subcortical structures, and therefore provides the anatomical foundation of the system (Novi et al., 2023). The primary weakness of fMRI is the slow and indirect nature of the BOLD response, which limits temporal interpretation. fNIRS complements fMRI by offering fast measurements of changes in oxyhemoglobin and deoxyhemoglobin at the cortical surface with relatively high temporal precision and strong tolerance to motion artifacts (Strangman et al., 2002). Although restricted to superficial cortex, fNIRS can refine or validate BOLD signals in corresponding cortical areas, improving interpretability (Cui et al., 2010). tDCS introduces a causal perturbation to neural circuits by altering cortical excitability. Because tDCS produces diffuse current flow, concurrent imaging is essential to understand how stimulation affects both targeted and distant brain regions (Saiote et al., 2013).
Implementation Challenges
Implementing tDCS, fMRI, and fNIRS in a single experimental environment introduces significant engineering and methodological challenges. The MRI environment imposes strict safety and artifact-reduction requirements. tDCS electrodes must be fabricated from nonferrous materials, such as carbon fiber, and connected via shielded cables that safely exit the scanner room. These precautions reduce the risk of radiofrequency-induced heating and minimize distortion in the fMRI signal (Saiote et al., 2013). fNIRS hardware must also be fully compatible with MRI. Light sources and detectors require nonmetallic construction, and optical fibers must be routed so that they do not interfere with the MRI coil or produce optical noise (Plichta et al., 2006). Achieving temporal alignment across all systems requires synchronized timing signals to ensure coordination among stimulation onset, fMRI slice acquisition, and fNIRS sampling.
Data Integration and Analysis
Because fMRI typically samples every two or three seconds while fNIRS samples multiple times per second, temporal fusion demands deconvolution or cross-modal regression techniques to match signal time courses (Cui et al., 2010). Spatial integration also requires substantial computational work. Structural MRI provides an anatomical reference for coregistering fNIRS optodes and tDCS electrode locations to the cortical surface. Finite element modeling of the tDCS montage can then be aligned with fMRI data to compare predicted current distribution with measured hemodynamic changes (Bhutta et al., 2016). A unified spatial framework is essential for coherent interpretation.
Applications and Future Directions
Despite its advantages, the trimodal approach faces notable practical and methodological challenges. The computational load is substantial because high-resolution fMRI combined with high-frequency fNIRS results in sizeable datasets that require specialized software and significant processing resources (Patel et al., 2020). Financial constraints also restrict widespread adoption because implementing a full trimodal system requires access to a high-field MRI scanner, MR-compatible fNIRS equipment, and MR-safe tDCS devices. Practical challenges include lengthy setup procedures, demanding coregistration steps, and patient discomfort. Individuals must remain as still as possible in the scanner while experiencing tDCS sensations and wearing multiple devices simultaneously. Continuous monitoring is required to prevent skin irritation, overheating, or hardware displacement. Despite these challenges, the trimodal approach is well-suited for research questions that require both whole-brain imaging and causal inference. Usually, the system can reveal how tDCS alters connectivity within memory or attention networks while fNIRS simultaneously monitors prefrontal hemodynamic responses (Di Rosa et al., 2019). In clinical contexts, such as stroke rehabilitation, fMRI can identify preserved and damaged pathways, while fNIRS verifies immediate cortical responses to stimulation. tDCS parameters may then be individualized in real time to optimize effective connectivity (Qi et al., 2024). For mood disorders, researchers can evaluate whether stimulation enhances connectivity between prefrontal regulatory regions and limbic structures, which may inform targeted neuromodulation strategies.
Conclusion
The integration of tDCS, fMRI, and fNIRS represents meaningful progress toward achieving causal and spatiotemporally resolved neuroimaging. Although considerable engineering, computational, and financial obstacles remain, the benefits of this approach are significant. By combining neural stimulation with multiple complementary imaging techniques, researchers can move beyond descriptive models and obtain mechanistic insight into how the brain reorganizes in response to targeted stimuli or intervention, thereby more accurately reflecting real-life situations. The trimodal framework has strong potential to advance both experimental research and clinical practice by providing comprehensive, causal, and individualized information about human function.
References
- Bhutta, M. R., Woo, N. S., Khan, M. J., & Hong, K. (2016). Effect of anodal tDCS on human prefrontal cortex observed by fNIRS. IEEE Xplore , 43 , 957–961. https://doi.org/10.1109/biorob.2016.7523752
- Cui, X., Bray, S., Bryant, D. M., Glover, G. H., & Reiss, A. L. (2010). A quantitative comparison of NIRS and fMRI across multiple cognitive tasks. NeuroImage , 54 (4), 2808–2821. https://doi.org/10.1016/j.neuroimage.2010.10.069
- Di Rosa, E., Brigadoi, S., Cutini, S., Tarantino, V., Dell’Acqua, R., Mapelli, D., Braver, T. S., & Vallesi, A. (2019). Reward motivation and neurostimulation interact to improve working memory performance in healthy older adults: A simultaneous tDCS-fNIRS study. NeuroImage , 202 , 116062. https://doi.org/10.1016/j.neuroimage.2019.116062
- Filmer, H. L., Dux, P. E., & Mattingley, J. B. (2014). Applications of transcranial direct current stimulation for understanding brain function. Trends in Neurosciences , 37 (12), 742–753. https://doi.org/10.1016/j.tins.2014.08.003
- Lüdemann, L., Förschler, A., Wust, P., & Zimmer, C. (2007). Quantification of fMRI BOLD signal and volume applied to the somatosensory cortex. Zeitschrift Für Medizinische Physik , 17 (2), 108–117. https://doi.org/10.1016/j.zemedi.2006.11.008
- Novi, S. L., Junior, De Castro Carvalho, A., Forti, R. M. M., Cendes, F., Yasuda, C. L., & Mesquita, R. C. (2023). Revealing the spatiotemporal requirements for accurate subject identification with resting-state functional connectivity: a simultaneous fNIRS-fMRI study. Neurophotonics , 10 (01), 013510. https://doi.org/10.1117/1.nph.10.1.013510
- Patel, R., Dawidziuk, A., Darzi, A., Singh, H., & Leff, D. R. (2020). Systematic review of combined functional near-infrared spectroscopy and transcranial direct-current stimulation studies. Neurophotonics , 7 (02), 1. https://doi.org/10.1117/1.nph.7.2.020901
- Saiote, C., Turi, Z., Paulus, W., & Antal, A. (2013). Combining functional magnetic resonance imaging with transcranial electrical stimulation. Frontiers in Human Neuroscience , 7 , 435. https://doi.org/10.3389/fnhum.2013.00435
- Strangman, G., Culver, J. P., Thompson, J. H., & Boas, D. A. (2002). A Quantitative Comparison of Simultaneous BOLD fMRI and NIRS Recordings during Functional Brain Activation. NeuroImage , 17 (2), 719–731. https://doi.org/10.1006/nimg.2002.1227
- Qi, S., Yu, J., Li, L., Dong, C., Ji, Z., Cao, L., Wei, Z., & Liang, Z. (2024). Advances in non-invasive brain stimulation: enhancing sports performance function and insights into exercise science. Frontiers in Human Neuroscience , 18 , 1477111. https://doi.org/10.3389/fnhum.2024.1477111
Other Course Work
📊 Assignment 1: EEG Analysis
📄 Assignment 1 - Angel Nwaka - Angel Nwaka.pdf
View Original PDFAssignment 1
Angel Nwaka
Raw Plot
Filtered Plot
Power Spectra
Alpha Power Spectrum
🎨 Assignment 2: BrainImation
📄 Psych 403 Assignment 02 - Angel Nwaka.pdf
View Original PDF- Name: Angel Nwaka Psych 403A1 I used the neural mandala visualization to animate real-time EEG data recorded during a 15-minute cardio session on a StairMaster using the Muse headband. I then compared this live data (EEG Data 01) to simulated data that I created to mimic a relaxed state, where I manually set attention = 0.3 and meditation = 0.8 (EEG Data 02). During the workout recording, the mandala rotated more slowly and displayed smaller, sparser neural points. In contrast, the simulated relation data produced a faster rotation and larger, more clustered points. For the code, I altered the mandala code template by:
- Changing the colour scheme
- Adding layers to the points to give a soft “bouncy” rotational effect
- Reduced both the size and brightness of the particles/points Through trial and error, I discovered how each EEG variable influenced the animation: meditation controlled the rotation speed, while attention affected particle size and intensity. The code let connectionThreshold = 0.5 + eegData.meditation * 0.5; was used to determine how frequently neural connection lines appeared between mandala layers. The most captivating part of this project, aside from wearing the Muse headband to the gym and noticing people’s reactions, was seeing how real-time feedback from physical activities could be transformed into a visual form, as well as playing around with the animations and code to see what I could make out of it, and understanding the different functions. Initially, when I struggled with connecting the headband to BrainImation, I tried experimenting with other apps like Muse and appreciated how brain data can be used to control and improve factors like cognitive performance, focus, and mental strength. Exercise Data >> Small and sparse particles >> Slow rotation Relaxed Simulated Data Attention = 0.3 , Meditation = 0.8 >>Large and close neural points >> Mandala rotation fast
Code-
// Neural mandala
let angle = 0 ;
function setup() {
colorMode( HSB , 589 , 10 , 100 , 1 );
noStroke();
}
function draw() {
background( 0 , 0 , 0 , 0.1 );
translate(width/ 2 , height/ 2 );
let layers = 75 ;
let connectionThreshold = 0.5 + eegData.meditation * 0.5 ;
// control density of connections
for ( let layer = 0 ; layer < layers; layer++) {
let radius = 0 + layer * 40 ;
let points = 15 + layer * 3 ;
for ( let i = 0 ; i < points; i++) {
let a = angle + ( TWO_PI / points) * i;
let x = cos(a) * radius;
let y = sin(a) * radius;
let hue = (angle * 60 + layer * 30 ) % 360 ;
let brightness = 50 + eegData.attention * 40 ;
let size = 3.75 + eegData.alpha * layer * 1.5 ;
fill(hue, 80 , brightness, 0.7 );
noStroke();
ellipse(x, y, size);
}
}
angle += eegData.meditation * 0.02 + 0.005 ;
}
🎥 EEG Data 02 - Angel Nwaka.mp4
💡 Videos require Google Drive access. Open in new tab if it doesn't load.
🎥 EEG Data 01 - Angel Nwaka.mp4
💡 Videos require Google Drive access. Open in new tab if it doesn't load.
🎯 Midterm Project
📄 Nwaka_403Midterm - Angel Nwaka.pdf
View Original PDFNwaka Angel
Psych 403 Midterm 01
1. This animation builds off my Neural Mandala from Assignment 2, where simulated EEG
signals are represented as an appealing, evolving art form. The EEG variables: alpha, beta,
attention, and meditation, each influence different visual elements, controlling how the mandala
rotates, expands, and glows. What makes this project unique is its dynamic feedback loop, which
allows the system to adjust itself based on the user’s calm threshold, mainly in these lines of my
code (38 - 44) -
if (frameCount % 60 === 0) {
if (alpha > targetAlpha) {
targetAlpha = min(1, targetAlpha + 0.002);
} else {
targetAlpha = max(0.6, targetAlpha - 0.001);}
}
This then compares the current alpha activity with the variable targetAlpha (set to 0.8). The
targetAlpha value acts as a moving threshold that represents the brain’s “calm zone.” When alpha
meets or exceeds this threshold, the code is designed to trigger a short p5.Oscillator tone, an
auditory feedback that signifies reaching the desired mental state of calm focus.
However, I initially could not achieve this because the sound wasn’t working the way I wanted it
to. It started with the sound playing continuously, which didn’t seem correct and was annoying to
listen to. So I modified it in a way that the sound would only play once the target mental state
was reached (mimicking real-time neurofeedback training). Whenever meditation was above 0.8,
there would be a sound.
Through trial, error, and a lot of back-and-forth problem-solving (with help from ChatGPT), I
learned how to construct the layered, petal-like structure of the mandala. I experimented with
radii, sine wave modulation, and rotation speed until it resembled a rotating neural flower,
complete with a pulsing center that responds to alpha strength. Figuring out how to center the
mandala and synchronize its motion was frustrating at first, but quite rewarding when things
started working and the output looked both visually appealing and functionally responsive
(looked like it was actually doing what it was meant to do).
2. For this part, I chose to measure Mismatch Negativity (MMN), an ERP that reflects the brain’s
automatic detection of auditory and visual deviant frequencies. To achieve this, I implemented
two visual stimulus categories to elicit an MMN-like response: a frequent stimulus (a filled black
circle) and a rare stimulus (an annulus appearing after the circle) in this line of code -
currentCategory = random() < 0.8 ? "frequent" : "rare";
It took me a while to understand that these frequent trials occur approximately 80% of the time,
establishing a visual regularity, while the rare trials (20%) act as the deviant condition and how
this probability structure mimcis the oddball paradigms used in EEG data to elicit MMN
responses, especially because I had absolutely no clue what the oddball sequence was and had to
do a bit of reading to make sure I had a sense of what I was doing.
Getting the timing right for the stimuli was one of my biggest struggles. Coordinating the 30-ms
circle, 60-ms gap, and 30-ms annulus using the millis() function required precision. Debugging
these intervals and worrying about whether I was breaking other parts of the code helped me
realize how even small timing errors can completely distort averaged waveforms in real EEG
experiments.
This was absolutely harder and more confusing than Question 1 because it required a lot of
back-and-forth between ChatGPT, short articles on understanding MMN signals, and
trial-and-error experimentation. Overall, seeing the averaged ERP traces update in real time with
simulated data (attention set to 0.9 and meditation set to 0.1 to mimic a state of intense focus and
low relaxation), as I couldn’t connect the Muse to BrainImation on my laptop, gave me a much
deeper insight into how these signals are measured. I definitely have a better appreciation for
ERP studies, beyond the theoretical perspective from my classes, by seeing how and why the
functions operate in practice. This project was definitely an emotional rollercoaster, but it helped
me transform abstract concepts into tangible understanding and gave me a new respect for the
precision and complexity involved in ERP research.
🎥 Question 1 - Nwaka - Angel Nwaka.mp4
💡 Videos require Google Drive access. Open in new tab if it doesn't load.
🎥 Question 2 - Nwaka - Angel Nwaka.mp4
💡 Videos require Google Drive access. Open in new tab if it doesn't load.
📝 Nwaka_midterm_part1 - Angel Nwaka.txt
💡 Code is embedded in this portfolio - opens instantly in the live BrainImation editor (no internet required!)
📝 Nwaka_midterm_part2 - Angel Nwaka.txt
💡 Code is embedded in this portfolio - opens instantly in the live BrainImation editor (no internet required!)