01 · Bridge2020
Attention in Psychology, Neuroscience, and Machine Learning
Grace W. Lindsay · Frontiers in Computational Neuroscience
Cross-disciplinary review
A broad review of how attention is defined, measured, and implemented across psychology, neuroscience, and machine learning.
Brain–network connectionThe best starting point for separating biological attention from the several operations called attention in artificial networks.
Interpretive limit & full citation
A conceptual synthesis; it does not establish that transformer attention is a cortical mechanism.
Grace W. Lindsay (2020). Frontiers in Computational Neuroscience.
overviewgainselection
02 · Bridge2018
How biological attention mechanisms improve task performance in a large-scale visual system model
Grace W. Lindsay & Kenneth D. Miller · eLife
Biologically motivated model
Applies biologically motivated attentional modulation to a deep convolutional visual model and evaluates the resulting changes in task performance.
Brain–network connectionDirectly tests how neural gain changes can improve a functioning visual network, linking cellular modulation to behavioral benefit.
Interpretive limit & full citation
Uses an artificial visual model rather than a complete cortical circuit or transformer.
Grace W. Lindsay; Kenneth D. Miller (2018). eLife.
visiongainfeature selectionCNN
03 · Bridge2026
Modeling attention and binding in the brain through bidirectional recurrent gating
Saeed Salehi et al. · Nature Communications 17, 4072
Biologically motivated model
Combines bottom-up visual processing with top-down and lateral recurrent gating. The trained model reproduces several attentional behaviors and neuron-like response properties.
Brain–network connectionA direct architecture-level attempt to unify spatial selection, visual search, and binding through recurrent modulation.
Interpretive limit & full citation
Qualitative similarities and architectural constraints support a model account; they do not identify the exact implementation in cortex.
Saeed Salehi; Jordan Lei; Ari S. Benjamin; Klaus-Robert Müller; Konrad P. Kording (2026). Nature Communications 17, 4072.
recurrencegatingbindingvision
04 · Bridge2026
The neuroscience of transformers
Peter König & Mario Negrello · Cognitive Neurodynamics 20, 197
Mechanistic hypothesis
Proposes correspondences between transformer operations and laminar cortical organization, generating hypotheses about contextual routing and integration.
Brain–network connectionThe most explicit cortical microcircuit comparison in this collection. Published 8 October 2026; included for direct relevance, rather than established influence.
Interpretive limit & full citation
The authors frame the mapping as a structured hypothesis, not evidence that cortex literally computes transformer equations.
Peter König; Mario Negrello (2026). Cognitive Neurodynamics 20, 197.
transformercortical layersroutingoscillations
05 · Bridge2022
Relating transformers to models and neural representations of the hippocampal formation
James C. R. Whittington et al. · ICLR
Model and mathematical correspondence
Shows that transformers with recurrent positional representations can reproduce spatially tuned representations and connect mathematically to hippocampal models.
Brain–network connectionLinks attention-based retrieval to place/grid representations and hippocampal indexing.
Interpretive limit & full citation
The result depends on architectural modifications and training; it is not a general claim about all transformers or biological attention.
James C. R. Whittington; Joseph Warren; Timothy E. J. Behrens (2022). ICLR.
transformerhippocampusmemoryposition
06 · Bridge2021
Hopfield Networks is All You Need
Hubert Ramsauer et al. · ICLR
Mathematical correspondence
Derives a continuous-state modern Hopfield memory whose update corresponds to transformer attention and demonstrates useful memory layers.
Brain–network connectionProvides a precise mathematical bridge between attention and associative retrieval.
Interpretive limit & full citation
Equivalence to an artificial associative-memory update does not demonstrate a biological synaptic implementation.
Hubert Ramsauer; Bernhard Schäfl; Johannes Lehner; Philipp Seidl; Michael Widrich; Thomas Adler; Lukas Gruber; Markus Holzleitner; Milena Pavlović; Geir Kjetil Sandve; Victor Greiff; David Kreil; Michael Kopp; Günter Klambauer; Johannes Brandstetter; Sepp Hochreiter (2021). ICLR.
associative memorytransformerenergy
07 · Bridge2016
Using Fast Weights to Attend to the Recent Past
Jimmy Ba et al. · NeurIPS
Biologically motivated model
Introduces rapidly changing weights that temporarily store recent activity and enable content-dependent retrieval.
Brain–network connectionOffers a synaptic-memory alternative to retaining and querying explicit copies of past activation vectors.
Interpretive limit & full citation
Biological plausibility is a modeling proposal; exact fast-weight rules and training are not established cortical mechanisms.
Jimmy Ba; Geoffrey Hinton; Volodymyr Mnih; Joel Z. Leibo; Catalin Ionescu (2016). NeurIPS.
fast weightsmemorysynaptic dynamics
08 · Bridge2016
Human Attention in Visual Question Answering: Do Humans and Deep Networks look at the same regions?
Abhishek Das et al. · EMNLP
Human behavioral comparison
Collects human relevance maps for answering image questions and compares them with attention maps from VQA networks.
Brain–network connectionAn empirical test of whether human and machine selection pick the same image regions.
Interpretive limit & full citation
Human maps are collected through an image-sharpening interface; they are not direct neural recordings or ordinary eye-tracking measures.
Abhishek Das; Harsh Agrawal; C. Lawrence Zitnick; Devi Parikh; Dhruv Batra (2016). EMNLP.
human attentionVQAspatial maps
09 · Bridge2020
Understanding Attention: In Minds and Machines
Shriraj P. Sawant & Shruti Singh · NeurIPS ML-RSA workshop / arXiv
Cross-disciplinary review
Reviews neural-network attention alongside biological concepts, emphasizing adaptive allocation of limited resources.
Brain–network connectionUseful for defining the level of comparison: shared computational purpose versus shared implementation.
Interpretive limit & full citation
A workshop synthesis, rather than a physiological validation of a particular attention circuit.
Shriraj P. Sawant; Shruti Singh (2020). NeurIPS ML-RSA workshop / arXiv.
overviewresource limitsdefinitions
10 · Bridge2021
Neural Attention Models in Deep Learning: Survey and Taxonomy
Alana Santana & Esther Colombini · arXiv preprint, 2112.05909
Cross-disciplinary review
Organizes attention models using criteria motivated by psychological and neuroscientific theories.
Brain–network connectionMakes differences among artificial attention architectures explicit instead of treating them as one biological construct.
Interpretive limit & full citation
This entry cites the verified 2021 preprint version; taxonomy and plausibility judgments are not experimental circuit evidence.
Alana Santana; Esther Colombini (2021). arXiv preprint, 2112.05909.
taxonomybiological plausibilitydefinitions
11 · Brain foundations1995
Neural Mechanisms of Selective Visual Attention
Robert Desimone & John Duncan · Annual Review of Neuroscience 18, 193–222
Biological theory and review
Develops the biased-competition account: representations compete and attention biases the outcome toward behaviorally relevant information.
Brain–network connectionA biological reference point for competitive weighting, gating, and task-dependent selection in artificial models.
Interpretive limit & full citation
Predates transformers and makes no direct comparison with modern self-attention.
Robert Desimone; John Duncan (1995). Annual Review of Neuroscience 18, 193–222.
biased competitionvisionselection
12 · Brain foundations2009
The Normalization Model of Attention
John H. Reynolds & David J. Heeger · Neuron 61, 168–185
Computational biological model
Models attention as modulation of stimulus drive within a divisive normalization framework, accounting for different forms of response gain.
Brain–network connectionA concrete biological computation to compare with artificial reweighting and competition.
Interpretive limit & full citation
Divisive normalization and transformer softmax are different operations; any equivalence requires a specific derivation.
John H. Reynolds; David J. Heeger (2009). Neuron 61, 168–185.
normalizationgainvision
13 · Brain foundations2012
Normalization as a canonical neural computation
Matteo Carandini & David J. Heeger · Nature Reviews Neuroscience 13, 51–62
Biological computation review
Reviews normalization as a recurring operation across sensory and cognitive neural systems, including attention.
Brain–network connectionExplains why controlling relative activity may be a reusable architectural principle for biological and artificial networks.
Interpretive limit & full citation
Biological normalization should not be conflated with layer normalization, batch normalization, or attention softmax.
Matteo Carandini; David J. Heeger (2012). Nature Reviews Neuroscience 13, 51–62.
normalizationcompetitiongain
14 · Brain foundations1993
A neurobiological model of visual attention and invariant pattern recognition based on dynamic routing of information
Bruno A. Olshausen et al. · Journal of Neuroscience 13, 4700–4719
Computational biological model
Proposes controlled routing through visual pathways to select information and support invariant recognition.
Brain–network connectionAn early explicit architecture for attention as routing, useful alongside query-dependent information transfer in transformers.
Interpretive limit & full citation
A historical model with different circuitry and learning assumptions from modern attention networks.
Bruno A. Olshausen; Charles H. Anderson; David C. Van Essen (1993). Journal of Neuroscience 13, 4700–4719.
routingvisioninvariance
15 · Brain foundations1995
Modeling visual attention via selective tuning
John K. Tsotsos et al. · Artificial Intelligence 78, 507–545
Computational biological model
Uses selection within a visual hierarchy to restrict processing and resolve competition.
Brain–network connectionProvides an architecture that combines hierarchical processing with top-down selection rather than uniform all-to-all interaction.
Interpretive limit & full citation
A theory of visual attention; the comparison to transformers here is an editorial reading connection.
John K. Tsotsos; Sean M. Culhane; Winky Yan Kei Wai; Yuzhong Lai; Neal Davis; Fernando Nuflo (1995). Artificial Intelligence 78, 507–545.
top-downhierarchyselection
16 · Brain foundations1998
A model of saliency-based visual attention for rapid scene analysis
Laurent Itti et al. · IEEE TPAMI 20, 1254–1259
Computational biological model
Combines multiscale visual features into a saliency map that guides sequential selection.
Brain–network connectionThe classic computational benchmark for stimulus-driven selection, contrasting with learned task-dependent attention.
Interpretive limit & full citation
Saliency maps are not transformer self-attention, and bottom-up conspicuity does not exhaust human attention.
Laurent Itti; Christof Koch; Ernst Niebur (1998). IEEE TPAMI 20, 1254–1259.
saliencybottom-upvision
17 · Brain foundations2001
Computational modelling of visual attention
Laurent Itti & Christof Koch · Nature Reviews Neuroscience 2, 194–203
Computational review
Reviews computational accounts of how attention selects locations and objects in visual scenes.
Brain–network connectionHelps distinguish stimulus-driven saliency, goal-driven selection, and the machinery used to implement them.
Interpretive limit & full citation
A pre-deep-learning foundation, not a direct comparison with contemporary transformers.
Laurent Itti; Christof Koch (2001). Nature Reviews Neuroscience 2, 194–203.
saliencytop-downvision
18 · Brain foundations2005
A mechanism for cognitive dynamics: neuronal communication through neuronal coherence
Pascal Fries · Trends in Cognitive Sciences 9, 474–480
Mechanistic hypothesis and review
Proposes that phase relationships between neuronal groups regulate the effectiveness and selectivity of communication.
Brain–network connectionOffers temporal coordination as a mechanism for changing effective connectivity, beyond static weighted sums.
Interpretive limit & full citation
A physiological hypothesis; no demonstrated equivalence to query-key dot products or softmax.
Pascal Fries (2005). Trends in Cognitive Sciences 9, 474–480.
coherenceoscillationsroutingEEG
19 · Brain foundations2015
Rhythms for Cognition: Communication through Coherence
Pascal Fries · Neuron 88, 220–235
Mechanistic hypothesis and review
Updates communication-through-coherence with evidence and a discussion of frequency-specific interactions.
Brain–network connectionA key reading for comparing instantaneous machine attention with temporally organized biological communication.
Interpretive limit & full citation
Oscillatory coupling supports a different explanatory level from artificial attention weights; the cross-domain comparison is interpretive.
Pascal Fries (2015). Neuron 88, 220–235.
coherenceoscillationsroutingEEG
20 · Brain foundations2010
Shaping Functional Architecture by Oscillatory Alpha Activity: Gating by Inhibition
Ole Jensen & Ali Mazaheri · Frontiers in Human Neuroscience 4, 186
Mechanistic hypothesis and review
Proposes that alpha-band activity suppresses task-irrelevant processing and shapes functional pathways.
Brain–network connectionEspecially relevant to EEG: compares selection by suppressing irrelevant channels with selection by increasing relevant weights.
Interpretive limit & full citation
Alpha gating is not a one-to-one implementation of a transformer mask or head.
Ole Jensen; Ali Mazaheri (2010). Frontiers in Human Neuroscience 4, 186.
alphainhibitiongatingEEG
21 · Machine architectures2017
Attention Is All You Need
Ashish Vaswani et al. · NeurIPS
Engineering architecture
Introduces the transformer, using multi-head attention and feedforward layers for sequence transduction.
Brain–network connectionDefines the artificial operation that most current brain–transformer comparisons try to explain.
Interpretive limit & full citation
An engineering paper, not a neuroscientific account; biological comparisons in this catalogue are explicitly interpretive.
Ashish Vaswani; Noam Shazeer; Niki Parmar; Jakob Uszkoreit; Llion Jones; Aidan N. Gomez; Lukasz Kaiser; Illia Polosukhin (2017). NeurIPS.
transformerself-attentionquery key value
22 · Machine architectures2015
Neural Machine Translation by Jointly Learning to Align and Translate
Dzmitry Bahdanau et al. · ICLR
Engineering architecture
Learns soft alignment over source representations while producing each target token, reducing the fixed-vector bottleneck.
Brain–network connectionA foundational example of context-dependent selection based on the current processing state.
Interpretive limit & full citation
The alignment mechanism was developed for translation and does not identify a biological attentional circuit.
Dzmitry Bahdanau; Kyunghyun Cho; Yoshua Bengio (2015). ICLR.
alignmentcross-attentionlanguage
23 · Machine architectures2014
Recurrent Models of Visual Attention
Volodymyr Mnih et al. · NeurIPS
Biologically inspired engineering
Learns a policy for sequentially sampling high-resolution glimpses and integrating them with a recurrent state.
Brain–network connectionCloser to active visual sampling and limited sensory resources than dense attention over an entire input.
Interpretive limit & full citation
Task success alone does not show that learned glimpse policies match human eye movements or covert attention.
Volodymyr Mnih; Nicolas Heess; Alex Graves; Koray Kavukcuoglu (2014). NeurIPS.
glimpsesrecurrenceactive vision
24 · Machine architectures2015
Show, Attend and Tell: Neural Image Caption Generation with Visual Attention
Kelvin Xu et al. · ICML / PMLR 37, 2048–2057
Engineering architecture
Introduces visual attention for image captioning, with soft and stochastic attention variants.
Brain–network connectionUseful for comparing spatial selection during linguistic output with human task-guided visual attention.
Interpretive limit & full citation
An attractive attention map is not proof of shared human relevance judgments or neural mechanisms.
Kelvin Xu; Jimmy Ba; Ryan Kiros; Kyunghyun Cho; Aaron Courville; Ruslan Salakhutdinov; Rich Zemel; Yoshua Bengio (2015). ICML / PMLR 37, 2048–2057.
visioncaptioningspatial maps
25 · Machine architectures2014
Neural Turing Machines
Alex Graves et al. · arXiv preprint, 1410.5401
Engineering architecture
Combines a neural controller with differentiable external memory accessed through attentional read and write operations.
Brain–network connectionClarifies the connection between attention, content-addressable retrieval, and memory control.
Interpretive limit & full citation
The explicit memory matrix and read/write heads are computational abstractions, not identified anatomical structures.
Alex Graves; Greg Wayne; Ivo Danihelka (2014). arXiv preprint, 1410.5401.
external memoryaddressingretrieval
26 · Machine architectures2015
DRAW: A Recurrent Neural Network For Image Generation
Karol Gregor et al. · ICML / PMLR 37, 1462–1471
Engineering architecture
Generates images through repeated attentional reading and writing within a recurrent generative model.
Brain–network connectionShows that attention can control iterative construction as well as recognition, a useful connection to active inference and imagery.
Interpretive limit & full citation
The link to biological imagery is a proposed reading connection, not a result validated against neural data.
Karol Gregor; Ivo Danihelka; Alex Graves; Danilo Rezende; Daan Wierstra (2015). ICML / PMLR 37, 1462–1471.
glimpsesgenerationrecurrence
27 · Machine architectures2020
Object-Centric Learning with Slot Attention
Francesco Locatello et al. · NeurIPS
Engineering architecture
Iteratively assigns visual features to exchangeable slots through competition, producing object-centered representations.
Brain–network connectionAn explicit artificial mechanism for grouping and binding, suitable for comparison with object-based attention.
Interpretive limit & full citation
Slots should not be identified with cortical object files without behavioral and physiological tests.
Francesco Locatello; Dirk Weissenborn; Thomas Unterthiner; Aravindh Mahendran; Georg Heigold; Jakob Uszkoreit; Alexey Dosovitskiy; Thomas Kipf (2020). NeurIPS.
bindingobjectscompetitionrecurrence
28 · Machine architectures2021
Perceiver: General Perception with Iterative Attention
Andrew Jaegle et al. · ICML / PMLR 139, 4651–4664
Biologically inspired engineering
Uses attention to compress large inputs into a smaller latent array, followed by iterative latent processing.
Brain–network connectionProvides a concrete architecture for limited processing capacity and integration across sensory modalities.
Interpretive limit & full citation
A latent bottleneck is not evidence of a particular biological attention network or global workspace.
Andrew Jaegle; Felix Gimeno; Andy Brock; Oriol Vinyals; Andrew Zisserman; João Carreira (2021). ICML / PMLR 139, 4651–4664.
bottleneckmultimodalcross-attention
29 · Machine architectures2021
An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale
Alexey Dosovitskiy et al. · ICLR
Engineering architecture
Applies a transformer to sequences of image patches and shows strong recognition after large-scale pretraining.
Brain–network connectionDefines a major visual architecture to compare against CNNs and recurrent cortical models.
Interpretive limit & full citation
The original paper establishes recognition performance rather than biological attentional equivalence.
Alexey Dosovitskiy; Lucas Beyer; Alexander Kolesnikov; Dirk Weissenborn; Xiaohua Zhai; Thomas Unterthiner; Mostafa Dehghani; Matthias Minderer; Georg Heigold; Sylvain Gelly; Jakob Uszkoreit; Neil Houlsby (2021). ICLR.
vision transformerpatchesself-attention
30 · Machine architectures2019
Attention is not Explanation
Sarthak Jain & Byron C. Wallace · NAACL-HLT
Interpretability experiment
Shows that attention weights in tested NLP models can diverge from other importance measures and can sometimes vary without materially changing predictions.
Brain–network connectionEssential methodological context when comparing an artificial attention map with eye movements, EEG, or neural relevance.
Interpretive limit & full citation
The findings concern tested architectures and tasks; they do not prove that every attention-based explanation is invalid.
Sarthak Jain; Byron C. Wallace (2019). NAACL-HLT.
causalityinterpretabilityattention maps
No matching papers. Try a broader term or reset filters.