DOE PAGES title logo U.S. Department of Energy
Office of Scientific and Technical Information

Title: Neural correlates of sparse coding and dimensionality reduction

Journal Article · · PLoS Computational Biology (Online)
ORCiD logo [1];  [2];  [3];  [2];  [2]
  1. Univ. of California, Irvine, CA (United States); Univ. of Washington, Seattle, WA (United States); DOE/OSTI
  2. Univ. of California, Irvine, CA (United States)
  3. Univ. of California, Irvine, CA (United States); Sandia National Lab. (SNL-NM), Albuquerque, NM (United States)

Supported by recent computational studies, there is increasing evidence that a wide range of neuronal responses can be understood as an emergent property of nonnegative sparse coding (NSC), an efficient population coding scheme based on dimensionality reduction and sparsity constraints. We review evidence that NSC might be employed by sensory areas to efficiently encode external stimulus spaces, by some associative areas to conjunctively represent multiple behaviorally relevant variables, and possibly by the basal ganglia to coordinate movement. In addition, NSC might provide a useful theoretical framework under which to understand the often complex and nonintuitive response properties of neurons in other brain areas. Although NSC might not apply to all brain areas (for example, motor or executive function areas) the success of NSC-based models, especially in sensory areas, warrants further investigation for neural correlates in other regions.

Research Organization:
Sandia National Lab. (SNL-NM), Albuquerque, NM (United States)
Sponsoring Organization:
Air Force Research Laboratory (AFRL); Defense Advanced Research Projects Agency (DARPA); National Science Foundation (NSF); Sandia National Laboratories; USDOE National Nuclear Security Administration (NNSA); Washington Research Foundation Funds for Innovation in Neuroengineering and Data-Intensive Discovery
Grant/Contract Number:
NA0003525
OSTI ID:
1627259
Journal Information:
PLoS Computational Biology (Online), Journal Name: PLoS Computational Biology (Online) Journal Issue: 6 Vol. 15; ISSN 1553-7358
Publisher:
Public Library of ScienceCopyright Statement
Country of Publication:
United States
Language:
English

References (244)

Computational analysis of the role of the hippocampus in memory journal June 1994
Evidence against a single coordinate system representation in the motor cortex journal June 2006
A network that uses few active neurones to code visual input predicts the diverse shapes of cortical receptive fields journal October 2006
Adaptive neural coding: from biological to behavioral decision-making journal October 2015
The role of primary motor cortex in goal-directed movements: insights from neurophysiological studies on non-human primates journal December 2003
Why neurons mix: high dimensionality for higher cognition journal April 2016
Locomotion Controls Spatial Integration in Mouse Visual Cortex journal May 2013
Effective Connectivity Reveals Largely Independent Parallel Networks of Face and Body Patches journal December 2016
Modeling the motor cortex: Optimality, recurrent neural networks, and spatial dynamics journal March 2016
Spatial Structure of Complex Cell Receptive Fields Measured with Natural Images journal March 2005
Dissociable Codes of Odor Quality and Odorant Structure in Human Piriform Cortex journal February 2006
Representations of Odor in the Piriform Cortex journal October 2009
Modulation of Visual Responses by Behavioral State in Mouse Visual Cortex journal February 2010
Neural Activity in Barrel Cortex Underlying Vibrissa-Based Object Localization in Mice journal September 2010
Synaptic Mechanisms Underlying Sparse Coding of Active Touch journal April 2011
Physiological Properties and Behavioral Correlates of Hippocampal Granule Cells and Mossy Cells journal February 2017
Neural Manifolds for the Control of Movement journal June 2017
Parietal cortex, navigation, and the construction of arbitrary reference frames for spatial information journal February 2009
Information processing, dimensionality reduction and reinforcement learning in the basal ganglia journal December 2003
Questioning the role of sparse coding in the brain journal July 2015
A two-layer sparse coding model learns simple and complex cell receptive fields and topography from natural images journal August 2001
A multi-layer sparse coding network learns contour coding from natural images journal June 2002
Sparse coding with an overcomplete basis set: A strategy employed by V1? journal December 1997
Wiring Optimization in Cortical Circuits journal April 2002
Vision and the statistics of the visual environment journal April 2003
Emergence of simple-cell receptive field properties by learning a sparse code for natural images journal June 1996
Learning the parts of objects by non-negative matrix factorization journal October 1999
Complex objects are represented in macaque inferotemporal cortex by the combination of feature columns journal August 2001
Sparse temporal coding of elementary tactile features during active whisker sensation journal May 2009
A face feature space in the macaque temporal lobe journal August 2009
Connectivity reflects coding: a model of voltage-based STDP with homeostasis journal January 2010
Decorrelation and efficient coding by retinal ganglion cells journal March 2012
Spatiotemporal receptive fields of barrel cortex revealed by reverse correlation of synaptic input journal May 2014
Olfactory bulb coding of odors, mixtures and sniffs is a linear sum of odor time profiles journal January 2015
Retrosplenial cortex maps the conjunction of internal and external spaces journal July 2015
Underlying principles of visual shape selectivity in posterior inferotemporal cortex journal July 2004
Correlating whisker behavior with membrane potential in barrel cortex of awake mice journal April 2006
What does the retrosplenial cortex do? journal October 2009
High-dimensional geometry of population responses in visual cortex journal June 2019
Stimulus- and goal-oriented frameworks for understanding natural vision journal December 2018
Expectation-induced modulation of metastable activity underlies faster coding of sensory stimuli journal April 2019
Independent generation of sequence elements by motor cortex journal February 2021
Causal Inference and Explaining Away in a Spiking Network journal December 2015
Performance-optimized hierarchical models predict neural responses in higher visual cortex journal May 2014
Wiring cost and topological participation of the mouse brain connectome journal July 2015
The “Parts and Wholes” of Face Recognition: A Review of the Literature journal October 2016
Neuronal networks in the brain journal May 1990
Simple Memory: A Theory for Archicortex journal July 1971
Functional Neuroanatomy of the Basal Ganglia journal October 2012
A Theory of How the Brain Might Work journal January 1990
Receptive fields of single neurones in the cat's striate cortex journal October 1959
Receptive fields, binocular interaction and functional architecture in the cat's visual cortex journal January 1962
Receptive field integration and submodality convergence in the hand area of the post-central gyrus of the alert monkey. journal October 1978
Communication in Neuronal Networks journal September 2003
Sparse Coding and Decorrelation in Primary Visual Cortex During Natural Vision journal February 2000
Spontaneous behaviors drive multidimensional, brainwide activity journal April 2019
Compressed Sensing, Sparsity, and Dimensionality in Neuronal Information Processing and Data Analysis journal July 2012
Cortical Control of Arm Movements: A Dynamical Systems Perspective journal July 2013
Reassessing Models of Basal Ganglia Function and Dysfunction journal July 2014
Natural Image Statistics and Neural Representation journal March 2001
Temporal Complexity and Heterogeneity of Single-Neuron Activity in Premotor and Motor Cortex journal June 2007
Responses of somatosensory area 2 neurons to actively and passively generated limb movements journal March 2013
MSTd Neuronal Basis Functions for the Population Encoding of Heading Direction journal August 2003
Revealing unobserved factors underlying cortical activity with a rectified latent variable model applied to neural population recordings journal March 2017
Spatial Structure and Symmetry of Simple-Cell Receptive Fields in Macaque Primary Visual Cortex journal July 2002
Sparse Spectrotemporal Coding of Sounds journal June 2003
Spatial Transformations in the Parietal Cortex Using Basis Functions journal March 1997
Energy Efficient Neural Codes journal April 1996
Stimulus-Driven Population Activity Patterns in Macaque Primary Visual Cortex text January 2016
Sparse Representation of Sounds in the Unanesthetized Auditory Cortex journal January 2008
Using Matrix and Tensor Factorizations for the Single-Trial Analysis of Population Spike Trains journal November 2016
Reinforcement-Driven Dimensionality Reduction - A Model for Information Processing in the Basal Ganglia journal January 2000
Theory for the development of neuron selectivity: orientation specificity and binocular interaction in visual cortex journal January 1982
Cortical Representation of Natural Complex Sounds: Effects of Acoustic Features and Auditory Object Category journal June 2010
Convergence of Vestibular and Visual Self-Motion Signals in an Area of the Posterior Sylvian Fissure journal August 2011
Component placement optimization in the brain journal April 1994
Structure of Receptive Fields in Area 3b of Primary Somatosensory Cortex in the Alert Monkey journal April 1998
Why New Neurons? Possible Functions for Adult Hippocampal Neurogenesis journal February 2002
Precedence of the Eye Region in Neural Processing of Faces journal November 2012
How Attention Enhances Spatial Resolution: Preferential Gain Enhancement of High Spatial Frequency Neurons journal November 2016
Do We Know What the Early Visual System Does? journal November 2005
The Representation of Stimulus Orientation in the Early Stages of Somatosensory Processing journal January 2008
Simple Learned Weighted Sums of Inferior Temporal Neuronal Firing Rates Accurately Predict Human Core Object Recognition Performance journal September 2015
Cues, context, and long-term memory: the role of the retrosplenial cortex in spatial cognition journal August 2014
Dendritic synapse location and neocortical spike-timing-dependent plasticity journal January 2010
Presynaptic Inhibition in the Striatum of the Basal Ganglia Improves Pattern Classification and Thus Promotes Superior Goal Selection journal December 2015
Theta phase precession in hippocampal neuronal populations and the compression of temporal sequences journal January 1996
Spatial selectivity of unit activity in the hippocampal granular layer journal April 1993
Computational analysis of the role of the hippocampus in memory journal June 1994
Simplified neuron model as a principal component analyzer journal November 1982
Forming sparse representations by local anti-Hebbian learning journal December 1990
Forming sparse representations by local anti-Hebbian learning journal December 1990
Evidence against a single coordinate system representation in the motor cortex journal June 2006
A network that uses few active neurones to code visual input predicts the diverse shapes of cortical receptive fields journal October 2006
Comparison of neuronal responses in primate inferior-temporal cortex and feed-forward deep neural network model with regard to information processing of faces journal February 2021
Hierarchical structure in perceptual representation journal October 1977
A two-layer sparse coding model learns simple and complex cell receptive fields and topography from natural images journal August 2001
A multi-layer sparse coding network learns contour coding from natural images journal June 2002
Sparse coding with an overcomplete basis set: A strategy employed by V1? journal December 1997
Wiring Optimization in Cortical Circuits journal April 2002
Modeling receptive fields with non-negative sparse coding journal June 2003
Energy as a constraint on the coding and processing of sensory information journal August 2001
Vision and the statistics of the visual environment journal April 2003
The Code for Facial Identity in the Primate Brain journal June 2017
Adaptive neural coding: from biological to behavioral decision-making journal October 2015
The role of primary motor cortex in goal-directed movements: insights from neurophysiological studies on non-human primates journal December 2003
Why neurons mix: high dimensionality for higher cognition journal April 2016
Effective Connectivity Reveals Largely Independent Parallel Networks of Face and Body Patches journal December 2016
Denoising sound signals in a bioinspired non-negative spectro-temporal domain journal March 2015
Extracting spatial–temporal coherent patterns in large-scale neural recordings using dynamic mode decomposition journal January 2016
Barlow versus Hebb: When is it time to abandon the notion of feature detectors and adopt the cell assembly as the unit of cognition? journal July 2018
Modeling the motor cortex: Optimality, recurrent neural networks, and spatial dynamics journal March 2016
Spatial Structure of Complex Cell Receptive Fields Measured with Natural Images journal March 2005
Dissociable Codes of Odor Quality and Odorant Structure in Human Piriform Cortex journal February 2006
Encoding and Decoding of Overlapping Odor Sequences journal August 2006
Odor Representations in Olfactory Cortex: “Sparse” Coding, Global Inhibition, and Oscillations journal June 2009
Representations of Odor in the Piriform Cortex journal September 2009
Representations of Odor in the Piriform Cortex journal October 2009
Modulation of Visual Responses by Behavioral State in Mouse Visual Cortex journal February 2010
Neural Activity in Barrel Cortex Underlying Vibrissa-Based Object Localization in Mice journal September 2010
Neural Mechanisms of Tactile Motion Integration in Somatosensory Cortex journal February 2011
Synaptic Mechanisms Underlying Sparse Coding of Active Touch journal March 2011
Synaptic Mechanisms Underlying Sparse Coding of Active Touch journal April 2011
Sparse Incomplete Representations: A Potential Role of Olfactory Granule Cells journal October 2011
Anatomical Connections of the Functionally Defined “Face Patches” in the Macaque Monkey journal June 2016
Physiological Properties and Behavioral Correlates of Hippocampal Granule Cells and Mossy Cells journal February 2017
Spatial Representations of Granule Cells and Mossy Cells of the Dentate Gyrus journal February 2017
Neural Manifolds for the Control of Movement journal June 2017
Motor Cortex Embeds Muscle-like Commands in an Untangled Population Response journal February 2018
Parietal cortex, navigation, and the construction of arbitrary reference frames for spatial information journal February 2009
Tracking the flow of hippocampal computation: Pattern separation, pattern completion, and attractor dynamics journal March 2016
Information processing, dimensionality reduction and reinforcement learning in the basal ganglia journal December 2003
Pattern separation in the hippocampus journal October 2011
Experimental evidence for sparse firing in the neocortex journal June 2012
Questioning the role of sparse coding in the brain journal July 2015
Conjunctive coding in an evolved spiking model of retrosplenial cortex. journal October 2018
Some informational aspects of visual perception. journal January 1954
Emergence of simple-cell receptive field properties by learning a sparse code for natural images journal June 1996
Learning the parts of objects by non-negative matrix factorization journal October 1999
Computational approaches to sensorimotor transformations journal November 2000
Complex objects are represented in macaque inferotemporal cortex by the combination of feature columns journal August 2001
Efficient auditory coding journal February 2006
The importance of mixed selectivity in complex cognitive tasks journal May 2013
Context-dependent computation by recurrent dynamics in prefrontal cortex journal November 2013
Sparse temporal coding of elementary tactile features during active whisker sensation journal May 2009
A face feature space in the macaque temporal lobe journal August 2009
Connectivity reflects coding: a model of voltage-based STDP with homeostasis journal January 2010
Decorrelation and efficient coding by retinal ganglion cells journal March 2012
Integration of visual motion and locomotion in mouse visual cortex journal November 2013
Spatiotemporal receptive fields of barrel cortex revealed by reverse correlation of synaptic input journal May 2014
Dimensionality reduction for large-scale neural recordings journal August 2014
Encoding and decoding in parietal cortex during sensorimotor decision-making journal August 2014
Olfactory bulb coding of odors, mixtures and sniffs is a linear sum of odor time profiles journal January 2015
Retrosplenial cortex maps the conjunction of internal and external spaces journal July 2015
Underlying principles of visual shape selectivity in posterior inferotemporal cortex journal July 2004
Correlating whisker behavior with membrane potential in barrel cortex of awake mice journal April 2006
What does the retrosplenial cortex do? journal October 2009
Inference of neuronal functional circuitry with spike-triggered non-negative matrix factorization journal July 2017
Cortical population activity within a preserved neural manifold underlies multiple motor behaviors journal October 2018
Stimulus- and goal-oriented frameworks for understanding natural vision journal December 2018
Causal Inference and Explaining Away in a Spiking Network journal December 2015
Performance-optimized hierarchical models predict neural responses in higher visual cortex journal May 2014
Nonsensory target-dependent organization of piriform cortex journal November 2014
Wiring cost and topological participation of the mouse brain connectome journal July 2015
Stimulus features coded by single neurons of a macaque body category selective patch journal April 2016
Estimating sparse spectro-temporal receptive fields with natural stimuli journal January 2007
Parts and Wholes in Face Recognition journal May 1993
The “Parts and Wholes” of Face Recognition: A Review of the Literature journal October 2016
A simple white noise analysis of neuronal light responses journal February 2001
Natural image statistics and efficient coding journal May 1996
The relative advantages of sparse versus distributed encoding for associative neuronal networks in the brain journal January 1990
Natural image statistics and efficient coding journal January 1996
A simple white noise analysis of neuronal light responses journal February 2001
Simple Memory: A Theory for Archicortex journal July 1971
A Theory of How the Brain Might Work journal January 1990
Functional Neuroanatomy of the Basal Ganglia journal October 2012
Compressed Nonnegative Sparse Coding conference December 2010
Biologically plausible models of homeostasis and STDP: Stability and learning in spiking neural networks
  • Carlson, Kristofor D.; Richert, Micah; Dutt, Nikil
  • 2013 International Joint Conference on Neural Networks (IJCNN 2013 - Dallas), The 2013 International Joint Conference on Neural Networks (IJCNN) https://doi.org/10.1109/IJCNN.2013.6706961
conference August 2013
Efficient coding and the neural representation of value: Louie & Glimcher journal March 2012
Receptive fields of single neurones in the cat's striate cortex journal October 1959
Receptive fields, binocular interaction and functional architecture in the cat's visual cortex journal January 1962
Receptive field integration and submodality convergence in the hand area of the post-central gyrus of the alert monkey. journal October 1978
Inconvenient Truths about neural processing in primary motor cortex: Neural processing in primary motor cortex journal February 2008
Communication in Neuronal Networks journal September 2003
A Cortical Region Consisting Entirely of Face-Selective Cells journal February 2006
Sparse Coding and Decorrelation in Primary Visual Cortex During Natural Vision journal February 2000
Spontaneous behaviors drive multidimensional, brainwide activity journal April 2019
On the Equivalence of Nonnegative Matrix Factorization and Spectral Clustering conference January 2014
Olfactory Maps in the Brain journal July 2011
Compressed Sensing, Sparsity, and Dimensionality in Neuronal Information Processing and Data Analysis journal July 2012
Cortical Control of Arm Movements: A Dynamical Systems Perspective journal July 2013
Reassessing Models of Basal Ganglia Function and Dysfunction journal July 2014
Perceptual Neural Organization: Some Approaches Based on Network Models and Information Theory journal March 1990
Multimodal Representation of Space in the Posterior Parietal Cortex and its use in Planning Movements journal March 1997
Natural Image Statistics and Neural Representation journal March 2001
Temporal Complexity and Heterogeneity of Single-Neuron Activity in Premotor and Motor Cortex journal June 2007
Convergence of Submodality-Specific Input Onto Neurons in Primary Somatosensory Cortex journal September 2009
Quantifying the signals contained in heterogeneous neural responses and determining their relationships with task performance journal September 2014
Responses of somatosensory area 2 neurons to actively and passively generated limb movements journal March 2013
MSTd Neuronal Basis Functions for the Population Encoding of Heading Direction journal August 2003
Revealing unobserved factors underlying cortical activity with a rectified latent variable model applied to neural population recordings journal March 2017
Spike-Timing Precision Underlies the Coding Efficiency of Auditory Receptor Neurons journal April 2006
Spatial Structure and Symmetry of Simple-Cell Receptive Fields in Macaque Primary Visual Cortex journal July 2002
Higher Order Visual Processing in Macaque Extrastriate Cortex journal January 2008
The Possible Contribution of the Amygdala to Memory journal January 1993
Analyzing Neural Responses to Natural Signals: Maximally Informative Dimensions journal February 2004
Spatial Transformations in the Parietal Cortex Using Basis Functions journal March 1997
What Is the Goal of Sensory Coding? journal July 1994
Energy Efficient Neural Codes journal April 1996
Dimensionality reduction in neural models: An information-theoretic generalization of spike-triggered average and covariance analysis journal February 2006
Stimulus-Driven Population Activity Patterns in Macaque Primary Visual Cortex text January 2016
Stimulus-Driven Population Activity Patterns in Macaque Primary Visual Cortex text January 2016
Experimental evidence for sparse firing in the neocortex. text January 2012
Sparse Representation of Sounds in the Unanesthetized Auditory Cortex journal January 2008
Multiplexing Stimulus Information through Rate and Temporal Codes in Primate Somatosensory Cortex journal May 2013
Stimulus-Driven Population Activity Patterns in Macaque Primary Visual Cortex journal December 2016
Using Matrix and Tensor Factorizations for the Single-Trial Analysis of Population Spike Trains journal November 2016
Categorical Dimensions of Human Odor Descriptor Space Revealed by Non-Negative Matrix Factorization journal September 2013
Reinforcement-Driven Dimensionality Reduction - A Model for Information Processing in the Basal Ganglia journal January 2000
A Simple Coding Procedure Enhances a Neuron's Information Capacity journal October 1981
Theory for the development of neuron selectivity: orientation specificity and binocular interaction in visual cortex journal January 1982
Cortical Representation of Natural Complex Sounds: Effects of Acoustic Features and Auditory Object Category journal June 2010
3D Visual Response Properties of MSTd Emerge from an Efficient, Sparse Population Code journal August 2016
Enhanced Spatial Resolution During Locomotion and Heightened Attention in Mouse Primary Visual Cortex journal June 2016
Multimodal Coding of Three-Dimensional Rotation and Translation in Area MSTd: Comparison of Visual and Vestibular Selectivity journal September 2007
Convergence of Vestibular and Visual Self-Motion Signals in an Area of the Posterior Sylvian Fissure journal August 2011
Component placement optimization in the brain journal April 1994
Efficient Coding of Natural Scenes in the Lateral Geniculate Nucleus: Experimental Test of a Computational Theory journal May 1996
Structure of Receptive Fields in Area 3b of Primary Somatosensory Cortex in the Alert Monkey journal April 1998
Spatial and Temporal Structure of Receptive Fields in Primate Somatosensory Area 3b: Effects of Stimulus Scanning Direction and Orientation journal January 2000
Why New Neurons? Possible Functions for Adult Hippocampal Neurogenesis journal February 2002
Precedence of the Eye Region in Neural Processing of Faces journal November 2012
State-Dependent Population Coding in Primary Auditory Cortex journal February 2015
Do We Know What the Early Visual System Does? journal November 2005
The Representation of Stimulus Orientation in the Early Stages of Somatosensory Processing journal January 2008
Simple Learned Weighted Sums of Inferior Temporal Neuronal Firing Rates Accurately Predict Human Core Object Recognition Performance journal September 2015
Cues, context, and long-term memory: the role of the retrosplenial cortex in spatial cognition journal August 2014
An efficient automated parameter tuning framework for spiking neural networks journal January 2014
Presynaptic Inhibition in the Striatum of the Basal Ganglia Improves Pattern Classification and Thus Promotes Superior Goal Selection journal December 2015
Stimuli Reduce the Dimensionality of Cortical Activity journal February 2016
Physiological Properties and Behavioral Correlates of Hippocampal Granule Cells and Mossy Cells dataset January 2021
Extracting spatial-temporal coherent patterns in large-scale neural recordings using dynamic mode decomposition text January 2014
Demixed principal component analysis of neural population data journal April 2016
Using Matrix and Tensor Factorizations for the Single-Trial Analysis of Population Spike Trains text January 2016

Cited By (3)

Structured sequence processing and combinatorial binding: neurobiologically and computationally informed hypotheses journal December 2019
Improving sensory representations using episodic memory journal December 2019
Structured sequence processing and combinatorial binding: neurobiologically and computationally informed hypotheses journal December 2019