Biological underpinnings for lifelong learning machines

Kudithipudi, Dhireesha; Aguilar-Simon, Mario; Babb, Jonathan; Bazhenov, Maxim; Blackiston, Douglas; Bongard, Josh; Brna, Andrew P.; Chakravarthi Raja, Suraj; Cheney, Nick; Clune, Jeff; Madireddy, Sandeep; Yanguas-Gil, Angel

doi:10.1038/s42256-022-00452-0

Biological underpinnings for lifelong learning machines

Journal Article · 05 March 2022 · Nature Machine Intelligence

DOI:https://doi.org/10.1038/s42256-022-00452-0· OSTI ID:1902727

Kudithipudi, Dhireesha; Aguilar-Simon, Mario; Babb, Jonathan; Bazhenov, Maxim; Blackiston, Douglas; Bongard, Josh; Brna, Andrew P.; Chakravarthi Raja, Suraj; Cheney, Nick; Clune, Jeff; Madireddy, Sandeep; Yanguas-Gil, Angel

Biological organisms learn from interactions with their environment throughout their lifetime. For artificial systems to successfully act and adapt in the real world, it is desirable to similarly be able to learn on a continual basis. This challenge is known as lifelong learning, and remains to a large extent unsolved. In this Perspective article, we identify a set of key capabilities that artificial systems will need to achieve lifelong learning. We describe a number of biological mechanisms, both neuronal and non-neuronal, that help explain how organisms solve these challenges, and present examples of biologically inspired models and biologically plausible mechanisms that have been applied to artificial systems in the quest towards development of lifelong learning machines. We discuss opportunities to further our understanding and advance the state of the art in lifelong learning, aiming to bridge the gap between natural and artificial intelligence. It is an outstanding challenge to develop intelligent machines that can learn continually from interactions with their environment, throughout their lifetime. Kudithipudi et al. review neuronal and non-neuronal processes in organisms that address this challenge and discuss pathways to developing biologically inspired approaches for lifelong learning machines.

Research Organization:: Argonne National Laboratory (ANL)

Sponsoring Organization:: U.S. Department of Defense (DOD) - Defense Advanced Research Projects Agency (DARPA)

DOE Contract Number:: AC02-06CH11357

OSTI ID:: 1902727

Journal Information:: Nature Machine Intelligence, Journal Name: Nature Machine Intelligence Journal Issue: 3 Vol. 4; ISSN 2522-5839

Country of Publication:: United States

Language:: English

References (179)

Biorobotics: Using robots to emulate and investigate agile locomotion Ijspeert, A. J. Science, Vol. 346, Issue 6206 https://doi.org/10.1126/science.1254486	journal	October 2014
Multisensory Integration and the Society for Neuroscience: Then and Now Stein, Barry E.; Stanford, Terrence R.; Rowland, Benjamin A. The Journal of Neuroscience, Vol. 40, Issue 1 https://doi.org/10.1523/JNEUROSCI.0737-19.2019	journal	November 2019
Learning to ignore: A modeling study of a decremental cholinergic pathway and its influence on attention and learning Oros, Nicolas; Chiba, Andrea A.; Nitz, Douglas A. Learning & Memory, Vol. 21, Issue 2 https://doi.org/10.1101/lm.032433.113	journal	January 2014
Multisensory Integration: Space, Time and Superadditivity Holmes, Nicholas P.; Spence, Charles Current Biology, Vol. 15, Issue 18 https://doi.org/10.1016/j.cub.2005.08.058	journal	September 2005
Central Cholinergic Neurons Are Rapidly Recruited by Reinforcement Feedback Hangya, Balázs; Ranade, Sachin P.; Lorenc, Maja Cell, Vol. 162, Issue 5 https://doi.org/10.1016/j.cell.2015.07.057	journal	August 2015
Bioelectric signaling as a unique regulator of development and regeneration Harris, Matthew P. Development, Vol. 148, Issue 10 https://doi.org/10.1242/dev.180794	journal	May 2021
Neurogenesis in the dentate gyrus of the adult rat: age-related decrease of neuronal progenitor proliferation Kuhn, Hg; Dickinson-Anson, H.; Gage, Fh The Journal of Neuroscience, Vol. 16, Issue 6 https://doi.org/10.1523/JNEUROSCI.16-06-02027.1996	journal	March 1996
Automated Shapeshifting for Function Recovery in Damaged Robots Kriegman, Sam; Walker, Stephanie; S. Shah, Dylan Robotics: Science and Systems 2019, Robotics: Science and Systems XV https://doi.org/10.15607/RSS.2019.XV.028	conference	June 2019
Computational Influence of Adult Neurogenesis on Memory Encoding Aimone, James B.; Wiles, Janet; Gage, Fred H. Neuron, Vol. 61, Issue 2 https://doi.org/10.1016/j.neuron.2008.11.026	journal	January 2009
The Adult Ventricular–Subventricular Zone (V-SVZ) and Olfactory Bulb (OB) Neurogenesis Lim, Daniel A.; Alvarez-Buylla, Arturo Cold Spring Harbor Perspectives in Biology, Vol. 8, Issue 5 https://doi.org/10.1101/cshperspect.a018820	journal	April 2016
Solving Ill-Conditioned and Singular Linear Systems: A Tutorial on Regularization Neumaier, Arnold SIAM Review, Vol. 40, Issue 3 https://doi.org/10.1137/S0036144597321909	journal	January 1998
Toward an Architecture for Never-Ending Language Learning Carlson, Andrew; Betteridge, Justin; Kisiel, Bryan Proceedings of the AAAI Conference on Artificial Intelligence, Vol. 24, Issue 1 https://doi.org/10.1609/aaai.v24i1.7519	journal	July 2010
Mastering the game of Go with deep neural networks and tree search Silver, David; Huang, Aja; Maddison, Chris J. Nature, Vol. 529, Issue 7587 https://doi.org/10.1038/nature16961	journal	January 2016
Neuromorphic Silicon Neuron Circuits Indiveri, Giacomo; Linares-Barranco, Bernabé; Hamilton, Tara Julia Frontiers in Neuroscience, Vol. 5 https://doi.org/10.3389/fnins.2011.00073	journal	January 2011
The stability of memories during brain remodeling: A perspective Blackiston, Douglas J.; Shomrat, Tal; Levin, Michael Communicative & Integrative Biology, Vol. 8, Issue 5 https://doi.org/10.1080/19420889.2015.1073424	journal	August 2015
On Having No Head: Cognition throughout Biological Systems Baluška, František; Levin, Michael Frontiers in Psychology, Vol. 7 https://doi.org/10.3389/fpsyg.2016.00902	journal	June 2016
Reward probability and timing uncertainty alter the effect of dorsal raphe serotonin neurons on patience Miyazaki, Katsuhiko; Miyazaki, Kayoko W.; Yamanaka, Akihiro Nature Communications, Vol. 9, Issue 1 https://doi.org/10.1038/s41467-018-04496-y	journal	June 2018
Bioinspired multisensory neural network with crossmodal integration and recognition Tan, Hongwei; Zhou, Yifan; Tao, Quanzheng Nature Communications, Vol. 12, Issue 1 https://doi.org/10.1038/s41467-021-21404-z	journal	February 2021
Phasic norepinephrine: A neural interrupt signal for unexpected events Dayan, Peter; Yu, Angela J. Network: Computation in Neural Systems, Vol. 17, Issue 4 https://doi.org/10.1080/09548980601004024	journal	January 2006
Catastrophic Forgetting, Rehearsal and Pseudorehearsal Robins, Anthony Connection Science, Vol. 7, Issue 2 https://doi.org/10.1080/09540099550039318	journal	June 1995
Skills development in infants: a possible role for widespread neurogenesis? Taliaz, Dekel Frontiers in Behavioral Neuroscience, Vol. 7 https://doi.org/10.3389/fnbeh.2013.00178	journal	January 2013
Integration of new information in memory: new insights from a complementary learning systems perspective McClelland, James L.; McNaughton, Bruce L.; Lampinen, Andrew K. Philosophical Transactions of the Royal Society B: Biological Sciences, Vol. 375, Issue 1799 https://doi.org/10.1098/rstb.2019.0637	journal	April 2020
Task representations in neural networks trained to perform many cognitive tasks Yang, Guangyu Robert; Joglekar, Madhura R.; Song, H. Francis Nature Neuroscience, Vol. 22, Issue 2 https://doi.org/10.1038/s41593-018-0310-2	journal	January 2019
The neural basis of monitoring goal progress Benn, Yael; Webb, Thomas L.; Chang, Betty P. I. Frontiers in Human Neuroscience, Vol. 8 https://doi.org/10.3389/fnhum.2014.00688	journal	September 2014
About Sleep's Role in Memory Rasch, Björn; Born, Jan Physiological Reviews, Vol. 93, Issue 2 https://doi.org/10.1152/physrev.00032.2012	journal	April 2013
Identifying and Quantifying Multisensory Integration: A Tutorial Review Stevenson, Ryan A.; Ghose, Dipanwita; Fister, Juliane Krueger Brain Topography, Vol. 27, Issue 6 https://doi.org/10.1007/s10548-014-0365-7	journal	April 2014
Can sleep protect memories from catastrophic forgetting? González, Oscar C.; Sokolov, Yury; Krishnan, Giri P. eLife, Vol. 9 https://doi.org/10.7554/eLife.51005	journal	August 2020
Human-level concept learning through probabilistic program induction Lake, B. M.; Salakhutdinov, R.; Tenenbaum, J. B. Science, Vol. 350, Issue 6266 https://doi.org/10.1126/science.aab3050	journal	December 2015
Context-dependent odor learning requires the anterior olfactory nucleus. Levinson, Max; Kolenda, Jacob P.; Alexandrou, Gabriella J. Behavioral Neuroscience, Vol. 134, Issue 4 https://doi.org/10.1037/bne0000371	journal	August 2020
Molecular Titration Promotes Oscillations and Bistability in Minimal Network Models with Monomeric Regulators Cuba Samaniego, Christian; Giordano, Giulia; Kim, Jongmin ACS Synthetic Biology, Vol. 5, Issue 4 https://doi.org/10.1021/acssynbio.5b00176	journal	February 2016
Maintaining memories by reactivation Rasch, Björn; Born, Jan Current Opinion in Neurobiology, Vol. 17, Issue 6 https://doi.org/10.1016/j.conb.2007.11.007	journal	December 2007
The second wave of synthetic biology: from modules to systems Purnick, Priscilla E. M.; Weiss, Ron Nature Reviews Molecular Cell Biology, Vol. 10, Issue 6 https://doi.org/10.1038/nrm2698	journal	June 2009
Novelty and Dopaminergic Modulation of Memory Persistence: A Tale of Two Systems Duszkiewicz, Adrian J.; McNamara, Colin G.; Takeuchi, Tomonori Trends in Neurosciences, Vol. 42, Issue 2 https://doi.org/10.1016/j.tins.2018.10.002	journal	February 2019
Hippocampal Offline Reactivation Consolidates Recently Formed Cell Assembly Patterns during Sharp Wave-Ripples van de Ven, Gido M.; Trouche, Stéphanie; McNamara, Colin G. Neuron, Vol. 92, Issue 5 https://doi.org/10.1016/j.neuron.2016.10.020	journal	December 2016
Context and Behavioral Processes in Extinction: Table 1. Bouton, Mark E. Learning & Memory, Vol. 11, Issue 5 https://doi.org/10.1101/lm.78804	journal	September 2004
Morphological Coordination: A Common Ancestral Function Unifying Neural and Non-Neural Signaling Fields, Chris; Bischof, Johanna; Levin, Michael Physiology, Vol. 35, Issue 1 https://doi.org/10.1152/physiol.00027.2019	journal	January 2020
Role of centrifugal projections to the olfactory bulb in olfactory processing Kiselycznyk, Carly L.; Zhang, Steven; Linster, Christiane Learning & Memory, Vol. 13, Issue 5 https://doi.org/10.1101/lm.285706	journal	September 2006
Learning without Forgetting Li, Zhizhong; Hoiem, Derek IEEE Transactions on Pattern Analysis and Machine Intelligence, Vol. 40, Issue 12 https://doi.org/10.1109/TPAMI.2017.2773081	journal	December 2018
Continuous Online Sequence Learning with an Unsupervised Neural Network Model Cui, Yuwei; Ahmad, Subutai; Hawkins, Jeff Neural Computation, Vol. 28, Issue 11 https://doi.org/10.1162/NECO_a_00893	journal	November 2016
A mesoscale connectome of the mouse brain Oh, Seung Wook; Harris, Julie A.; Ng, Lydia Nature, Vol. 508, Issue 7495 https://doi.org/10.1038/nature13186	journal	April 2014
Human-level control through deep reinforcement learning Mnih, Volodymyr; Kavukcuoglu, Koray; Silver, David Nature, Vol. 518, Issue 7540 https://doi.org/10.1038/nature14236	journal	February 2015
A Computational Model of Oxytocin Modulation of Olfactory Recognition Memory Linster, Christiane; Kelsch, Wolfgang eneuro, Vol. 6, Issue 4 https://doi.org/10.1523/ENEURO.0201-19.2019	journal	July 2019
Parsing the role of sleep in memory processing Stickgold, Robert Current Opinion in Neurobiology, Vol. 23, Issue 5 https://doi.org/10.1016/j.conb.2013.04.002	journal	October 2013
Task-agnostic self-modeling machines Kwiatkowski, Robert; Lipson, Hod Science Robotics, Vol. 4, Issue 26 https://doi.org/10.1126/scirobotics.aau9354	journal	January 2019
Locomotion in vertebrates: central mechanisms and reflex interaction Grillner, S. Physiological Reviews, Vol. 55, Issue 2 https://doi.org/10.1152/physrev.1975.55.2.247	journal	April 1975
Task-Based Neuromodulation Architecture for Lifelong Learning Daram, Anurag Reddy; Kudithipudi, Dhireesha; Yanguas-Gil, Angel 2019 20th International Symposium on Quality Electronic Design (ISQED) https://doi.org/10.1109/ISQED.2019.8697362	conference	March 2019
Meta-Learning in Neural Networks: A Survey Hospedales, Timothy M.; Antoniou, Antreas; Micaelli, Paul IEEE Transactions on Pattern Analysis and Machine Intelligence https://doi.org/10.1109/TPAMI.2021.3079209	journal	January 2021
TrueNorth: Design and Tool Flow of a 65 mW 1 Million Neuron Programmable Neurosynaptic Chip Akopyan, Filipp; Sawada, Jun; Cassidy, Andrew IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems, Vol. 34, Issue 10 https://doi.org/10.1109/TCAD.2015.2474396	journal	October 2015
Mapping the mind of a fly Pipkin, Jason eLife, Vol. 9 https://doi.org/10.7554/eLife.62451	journal	October 2020
Experience-Induced Neurogenesis in the Senescent Dentate Gyrus Kempermann, Gerd; Kuhn, H. Georg; Gage, Fred H. The Journal of Neuroscience, Vol. 18, Issue 9 https://doi.org/10.1523/JNEUROSCI.18-09-03206.1998	journal	May 1998
Measuring Structural Similarities in Finite MDPs Wang, Hao; Dong, Shaokang; Shao, Ling Twenty-Eighth International Joint Conference on Artificial Intelligence {IJCAI-19}, Proceedings of the Twenty-Eighth International Joint Conference on Artificial Intelligence https://doi.org/10.24963/ijcai.2019/511	conference	August 2019
The Fly Brain Atlas Scheffer, Louis K.; Meinertzhagen, Ian A. Annual Review of Cell and Developmental Biology, Vol. 35, Issue 1 https://doi.org/10.1146/annurev-cellbio-100818-125444	journal	October 2019
NeuroBayesSLAM: Neurobiologically inspired Bayesian integration of multisensory information for robot navigation Zeng, Taiping; Tang, Fengzhen; Ji, Daxiong Neural Networks, Vol. 126 https://doi.org/10.1016/j.neunet.2020.02.023	journal	June 2020
Hierarchical motor control in mammals and machines Merel, Josh; Botvinick, Matthew; Wayne, Greg Nature Communications, Vol. 10, Issue 1 https://doi.org/10.1038/s41467-019-13239-6	journal	December 2019
Neural Modularity Helps Organisms Evolve to Learn New Skills without Forgetting Old Skills Ellefsen, Kai Olav; Mouret, Jean-Baptiste; Clune, Jeff PLOS Computational Biology, Vol. 11, Issue 4 https://doi.org/10.1371/journal.pcbi.1004128	journal	April 2015
Neurogenesis deep learning: Extending deep networks to accommodate new classes Draelos, Timothy J.; Miner, Nadine E.; Lamb, Christopher C. 2017 International Joint Conference on Neural Networks (IJCNN) https://doi.org/10.1109/IJCNN.2017.7965898	conference	May 2017
Class-Incremental Learning with Generative Classifiers van de Ven, Gido M.; Li, Zhe; Tolias, Andreas S. 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) https://doi.org/10.1109/CVPRW53098.2021.00400	conference	June 2021
Forward and reverse hippocampal place-cell sequences during ripples Diba, Kamran; Buzsáki, György Nature Neuroscience, Vol. 10, Issue 10 https://doi.org/10.1038/nn1961	journal	September 2007
The Synaptic Theory of Memory: A Historical Survey and Reconciliation of Recent Opposition Langille, Jesse J.; Brown, Richard E. Frontiers in Systems Neuroscience, Vol. 12 https://doi.org/10.3389/fnsys.2018.00052	journal	October 2018
Transvenous liver biopsy in critically ill patients: adequacy of tissue samples. Lipchik, E. O.; Cohen, E. B.; Mewissen, M. W. Radiology, Vol. 181, Issue 2 https://doi.org/10.1148/radiology.181.2.1924794	journal	November 1991
The Role of Sleep in Learning and Memory Maquet, Pierre Science, Vol. 294, Issue 5544 https://doi.org/10.1126/science.1062856	journal	November 2001
Competitive Learning: From Interactive Activation to Adaptive Resonance Grossberg, Stephen Cognitive Science, Vol. 11, Issue 1 https://doi.org/10.1111/j.1551-6708.1987.tb00862.x	journal	January 1987
Always Be Dreaming: A New Approach for Data-Free Class-Incremental Learning Smith, James; Hsu, Yen-Chang; Balloch, Jonathan 2021 IEEE/CVF International Conference on Computer Vision (ICCV) https://doi.org/10.1109/ICCV48922.2021.00924	conference	October 2021
A neuromorph's prospectus Boahen, Kwabena Computing in Science & Engineering, Vol. 19, Issue 2 https://doi.org/10.1109/MCSE.2017.33	journal	March 2017
Retention of Memory through Metamorphosis: Can a Moth Remember What It Learned As a Caterpillar? Blackiston, Douglas J.; Silva Casey, Elena; Weiss, Martha R. PLoS ONE, Vol. 3, Issue 3 https://doi.org/10.1371/journal.pone.0001736	journal	March 2008
Spinal-Like Regulator Facilitates Control of a Two-Degree-of-Freedom Wrist Raphael, G.; Tsianos, G. A.; Loeb, G. E. Journal of Neuroscience, Vol. 30, Issue 28 https://doi.org/10.1523/JNEUROSCI.5537-09.2010	journal	July 2010
Intracellular Dynamics in Cuneate Nucleus Neurons Support Self-Stabilizing Learning of Generalizable Tactile Representations Rongala, Udaya B.; Spanne, Anton; Mazzoni, Alberto Frontiers in Cellular Neuroscience, Vol. 12 https://doi.org/10.3389/fncel.2018.00210	journal	July 2018
Hebbian spike-driven synaptic plasticity for learning patterns of mean firing rates Fusi, Stefano Biological Cybernetics, Vol. 87, Issue 5-6 https://doi.org/10.1007/s00422-002-0356-8	journal	December 2002
Behavioral evidence of the role of lumbosacral anatomical specializations in pigeons in maintaining balance during terrestrial locomotion Necker, R.; Janßen, A.; Beissenhirtz, T. Journal of Comparative Physiology A: Sensory, Neural, and Behavioral Physiology, Vol. 186, Issue 4 https://doi.org/10.1007/s003590050440	journal	April 2000
Robot End Effector Tracking Using Predictive Multisensory Integration Wijesinghe, Lakshitha P.; Triesch, Jochen; Shi, Bertram E. Frontiers in Neurorobotics, Vol. 12 https://doi.org/10.3389/fnbot.2018.00066	journal	October 2018
The SpiNNaker Project Furber, Steve B.; Galluppi, Francesco; Temple, Steve Proceedings of the IEEE, Vol. 102, Issue 5 https://doi.org/10.1109/JPROC.2014.2304638	journal	May 2014
Regenerative Adaptation to Electrochemical Perturbation in Planaria: A Molecular Analysis of Physiological Plasticity Emmons-Bell, Maya; Durant, Fallon; Tung, Angela iScience, Vol. 22 https://doi.org/10.1016/j.isci.2019.11.014	journal	December 2019
Bioelectric signaling: Reprogrammable circuits underlying embryogenesis, regeneration, and cancer Levin, Michael Cell, Vol. 184, Issue 8 https://doi.org/10.1016/j.cell.2021.02.034	journal	April 2021
Hippocampal Reactivation of Random Trajectories Resembling Brownian Diffusion Stella, Federico; Baracskay, Peter; O’Neill, Joseph Neuron, Vol. 102, Issue 2 https://doi.org/10.1016/j.neuron.2019.01.052	journal	April 2019
The Biological Basis of Learning and Individuality Kandel, Eric R.; Hawkins, Robert D. Scientific American, Vol. 267, Issue 3 https://doi.org/10.1038/scientificamerican0992-78	journal	September 1992
MetaplasticNet: Architecture with Probabilistic Metaplastic Synapses for Continual Learning Zohora, Fatima Tuz; Karia, Vedant; Daram, Anurag Reddy 2021 IEEE International Symposium on Circuits and Systems (ISCAS) https://doi.org/10.1109/ISCAS51556.2021.9401262	conference	May 2021
Lifelong Learning of Spatiotemporal Representations With Dual-Memory Recurrent Self-Organization Parisi, German I.; Tani, Jun; Weber, Cornelius Frontiers in Neurorobotics, Vol. 12 https://doi.org/10.3389/fnbot.2018.00078	journal	November 2018
Learning dexterous in-hand manipulation Andrychowicz, OpenAI: Marcin; Baker, Bowen; Chociej, Maciek The International Journal of Robotics Research, Vol. 39, Issue 1 https://doi.org/10.1177/0278364919887447	journal	November 2019
Cholinergic Mesopontine Signals Govern Locomotion and Reward through Dissociable Midbrain Pathways Xiao, Cheng; Cho, Jounhong Ryan; Zhou, Chunyi Neuron, Vol. 90, Issue 2 https://doi.org/10.1016/j.neuron.2016.03.028	journal	April 2016
Resolving New Memories: A Critical Look at the Dentate Gyrus, Adult Neurogenesis, and Pattern Separation Aimone, James B.; Deng, Wei; Gage, Fred H. Neuron, Vol. 70, Issue 4 https://doi.org/10.1016/j.neuron.2011.05.010	journal	May 2011
A Neural Substrate of Prediction and Reward Schultz, W.; Dayan, P.; Montague, P. R. Science, Vol. 275, Issue 5306 https://doi.org/10.1126/science.275.5306.1593	journal	March 1997
A Physical Model Suggests That Hip-Localized Balance Sense in Birds Improves State Estimation in Perching: Implications for Bipedal Robots Urbina-Meléndez, Darío; Jalaleddini, Kian; Daley, Monica A. Frontiers in Robotics and AI, Vol. 5 https://doi.org/10.3389/frobt.2018.00038	journal	April 2018
A unified approach to evolving plasticity and neural geometry Risi, Sebastian; Stanley, Kenneth O. 2012 International Joint Conference on Neural Networks (IJCNN 2012 - Brisbane), The 2012 International Joint Conference on Neural Networks (IJCNN) https://doi.org/10.1109/IJCNN.2012.6252826	conference	June 2012
Ectopic eyes outside the head in Xenopus tadpoles provide sensory data for light-mediated learning Blackiston, Douglas J.; Levin, Michael Journal of Experimental Biology, Vol. 216, Issue 6 https://doi.org/10.1242/jeb.074963	journal	March 2013
Self-supervised Visual Feature Learning with Deep Neural Networks: A Survey Jing, Longlong; Tian, Yingli IEEE Transactions on Pattern Analysis and Machine Intelligence https://doi.org/10.1109/TPAMI.2020.2992393	journal	January 2020
Uncertainty, Neuromodulation, and Attention Yu, Angela J.; Dayan, Peter Neuron, Vol. 46, Issue 4 https://doi.org/10.1016/j.neuron.2005.04.026	journal	May 2005
Odor- and context-dependent modulation of mitral cell activity in behaving rats Kay, Leslie M.; Laurent, Gilles Nature Neuroscience, Vol. 2, Issue 11 https://doi.org/10.1038/14801	journal	November 1999
Patterned activation of action potential patterns during offline states in the neocortex: replay and non-replay Liu, Tang-Yu; Watson, Brendon O. Philosophical Transactions of the Royal Society B: Biological Sciences, Vol. 375, Issue 1799 https://doi.org/10.1098/rstb.2019.0233	journal	April 2020
A neural model of schemas and memory encoding Hwu, Tiffany; Krichmar, Jeffrey L. Biological Cybernetics, Vol. 114, Issue 2 https://doi.org/10.1007/s00422-019-00808-7	journal	November 2019
How schema and novelty augment memory formation van Kesteren, Marlieke T. R.; Ruiter, Dirk J.; Fernández, Guillén Trends in Neurosciences, Vol. 35, Issue 4 https://doi.org/10.1016/j.tins.2012.02.001	journal	April 2012
Extracting structural and functional features of widely distributed biological circuits with single cell resolution via tissue clearing and delivery vectors Treweek, Jennifer Brooke; Gradinaru, Viviana Current Opinion in Biotechnology, Vol. 40 https://doi.org/10.1016/j.copbio.2016.03.012	journal	August 2016
Why there are complementary learning systems in the hippocampus and neocortex: Insights from the successes and failures of connectionist models of learning and memory. McClelland, James L.; McNaughton, Bruce L.; O'Reilly, Randall C. Psychological Review, Vol. 102, Issue 3 https://doi.org/10.1037/0033-295X.102.3.419	journal	July 1995
Multitask Learning Caruana, Rich Machine Learning, Vol. 28, Issue 1, p. 41-75 https://doi.org/10.1023/A:1007379606734	journal	January 1997
Is the brain overrated? Schwab, I. R. British Journal of Ophthalmology, Vol. 87, Issue 5 https://doi.org/10.1136/bjo.87.5.525	journal	May 2003
iCaRL: Incremental Classifier and Representation Learning Rebuffi, Sylvestre-Alvise; Kolesnikov, Alexander; Sperl, Georg 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) https://doi.org/10.1109/CVPR.2017.587	conference	July 2017
Neuromodulated attention and goal-driven perception in uncertain domains Zou, Xinyun; Kolouri, Soheil; Pilly, Praveen K. Neural Networks, Vol. 125 https://doi.org/10.1016/j.neunet.2020.01.031	journal	May 2020
Intelligence without representation Brooks, Rodney A. Artificial Intelligence, Vol. 47, Issue 1-3 https://doi.org/10.1016/0004-3702(91)90053-M	journal	January 1991
Sensory-Motor Gestalt: Sensation and Action as the Foundations of Identity, Agency, and Self Berry, Jasmine A.; Valero-Cuevas, Francisco J. The 2020 Conference on Artificial Life https://doi.org/10.1162/isal_a_00340	conference	January 2020
A Bridge Over Troubled Water: Reconsolidation as a Link Between Cognitive and Neuroscientific Memory Research Traditions Hardt, Oliver; Einarsson, Einar Örn; Nader, Karim Annual Review of Psychology, Vol. 61, Issue 1 https://doi.org/10.1146/annurev.psych.093008.100455	journal	January 2010
Assembly-Specific Disruption of Hippocampal Replay Leads to Selective Memory Deficit Gridchyn, Igor; Schoenenberger, Philipp; O’Neill, Joseph Neuron, Vol. 106, Issue 2 https://doi.org/10.1016/j.neuron.2020.01.021	journal	April 2020
Exploring Neuromodulation for Dynamic Learning Daram, Anurag; Yanguas-Gil, Angel; Kudithipudi, Dhireesha Frontiers in Neuroscience, Vol. 14 https://doi.org/10.3389/fnins.2020.00928	journal	September 2020
Cascade Models of Synaptically Stored Memories Fusi, Stefano; Drew, Patrick J.; Abbott, L. F. Neuron, Vol. 45, Issue 4 https://doi.org/10.1016/j.neuron.2005.02.001	journal	February 2005
Top-down models in biology: explanation and control of complex living systems above the molecular level Pezzulo, Giovanni; Levin, Michael Journal of The Royal Society Interface, Vol. 13, Issue 124 https://doi.org/10.1098/rsif.2016.0555	journal	November 2016
Functional architecture of reward learning in mushroom body extrinsic neurons of larval Drosophila Saumweber, Timo; Rohwedder, Astrid; Schleyer, Michael Nature Communications, Vol. 9, Issue 1 https://doi.org/10.1038/s41467-018-03130-1	journal	March 2018
Meta-Transfer Learning for Few-Shot Learning Sun, Qianru; Liu, Yaoyao; Chua, Tat-Seng 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) https://doi.org/10.1109/CVPR.2019.00049	conference	June 2019
Layering as Optimization Decomposition: A Mathematical Theory of Network Architectures Chiang, Mung; Low, Steven H.; Calderbank, A. Robert Proceedings of the IEEE, Vol. 95, Issue 1 https://doi.org/10.1109/JPROC.2006.887322	journal	January 2007
A scalable pipeline for designing reconfigurable organisms Kriegman, Sam; Blackiston, Douglas; Levin, Michael Proceedings of the National Academy of Sciences, Vol. 117, Issue 4 https://doi.org/10.1073/pnas.1910837117	journal	January 2020
Selective Memory Generalization by Spatial Patterning of Protein Synthesis O’Donnell, Cian; Sejnowski, Terrence J. Neuron, Vol. 82, Issue 2 https://doi.org/10.1016/j.neuron.2014.02.028	journal	April 2014
Modeling somatic computation with non-neural bioelectric networks Manicka, Santosh; Levin, Michael Scientific Reports, Vol. 9, Issue 1 https://doi.org/10.1038/s41598-019-54859-8	journal	December 2019
Memory of Sequential Experience in the Hippocampus during Slow Wave Sleep Lee, Albert K.; Wilson, Matthew A. Neuron, Vol. 36, Issue 6 https://doi.org/10.1016/S0896-6273(02)01096-6	journal	December 2002
Beta- and gamma-frequency coupling between olfactory and motor brain regions prior to skilled, olfactory-driven reaching Hermer-Vazquez, Raymond; Hermer-Vazquez, Linda; Srinivasan, Sridhar Experimental Brain Research, Vol. 180, Issue 2 https://doi.org/10.1007/s00221-007-0850-2	journal	February 2007
Control of oscillation periods and phase durations in half-center central pattern generators: a comparative mechanistic analysis Daun, Silvia; Rubin, Jonathan E.; Rybak, Ilya A. Journal of Computational Neuroscience, Vol. 27, Issue 1 https://doi.org/10.1007/s10827-008-0124-4	journal	January 2009
Memory Efficient Experience Replay for Streaming Learning Hayes, Tyler L.; Cahill, Nathan D.; Kanan, Christopher 2019 International Conference on Robotics and Automation (ICRA) https://doi.org/10.1109/ICRA.2019.8793982	conference	May 2019
Brain computation by assemblies of neurons Papadimitriou, Christos H.; Vempala, Santosh S.; Mitropolsky, Daniel Proceedings of the National Academy of Sciences, Vol. 117, Issue 25 https://doi.org/10.1073/pnas.2001893117	journal	June 2020
Generalization of Agent Behavior through Explicit Representation of Context Tutum, Cem; AbdulQuddos, Suhaib; Miikkulainen, Risto 2021 IEEE Conference on Games (CoG) https://doi.org/10.1109/CoG52621.2021.9619141	conference	August 2021
A Neurobiological Schema Model for Contextual Awareness in Robotics Hwu, Tiffany; Kashyap, Hirak J.; Krichmar, Jeffrey L. 2020 International Joint Conference on Neural Networks (IJCNN) https://doi.org/10.1109/IJCNN48605.2020.9206858	conference	July 2020
A Dynamical Biomolecular Neural Network Moorman, Andrew; Samaniego, Christian Cuba; Maley, Carlo 2019 IEEE 58th Conference on Decision and Control (CDC) https://doi.org/10.1109/CDC40024.2019.9030122	conference	December 2019
Memristor design rules for dynamic learning and edge processing applications Yanguas-Gil, Angel APL Materials, Vol. 7, Issue 9 https://doi.org/10.1063/1.5109910	journal	September 2019
Computational principles of synaptic memory consolidation Benna, Marcus K.; Fusi, Stefano Nature Neuroscience, Vol. 19, Issue 12 https://doi.org/10.1038/nn.4401	journal	October 2016
Central Pattern Generators for Locomotion, with Special Reference to Vertebrates Grillner, S.; Wallen, P. Annual Review of Neuroscience, Vol. 8, Issue 1 https://doi.org/10.1146/annurev.ne.08.030185.001313	journal	March 1985
Metaplasticity: tuning synapses and networks for plasticity Abraham, Wickliffe C. Nature Reviews Neuroscience, Vol. 9, Issue 5 https://doi.org/10.1038/nrn2356	journal	May 2008
Endogenous Bioelectric Signaling Networks: Exploiting Voltage Gradients for Control of Growth and Form Levin, Michael; Pezzulo, Giovanni; Finkelstein, Joshua M. Annual Review of Biomedical Engineering, Vol. 19, Issue 1 https://doi.org/10.1146/annurev-bioeng-071114-040647	journal	June 2017
Merging the senses into a robust percept Ernst, Marc O.; Bülthoff, Heinrich H. Trends in Cognitive Sciences, Vol. 8, Issue 4 https://doi.org/10.1016/j.tics.2004.02.002	journal	April 2004
The Neuromodulatory System: A Framework for Survival and Adaptive Behavior in a Challenging World Krichmar, Jeffrey L. Adaptive Behavior, Vol. 16, Issue 6 https://doi.org/10.1177/1059712308095775	journal	December 2008
Dopaminergic Modulation of Long-term Synaptic Plasticity in Rat Prefrontal Neurons Otani, S. Cerebral Cortex, Vol. 13, Issue 11 https://doi.org/10.1093/cercor/bhg092	journal	November 2003
Dynamical Movement Primitives: Learning Attractor Models for Motor Behaviors Ijspeert, Auke Jan; Nakanishi, Jun; Hoffmann, Heiko Neural Computation, Vol. 25, Issue 2 https://doi.org/10.1162/NECO_a_00393	journal	February 2013
A predictive focus of gain modulation encodes target trajectories in insect vision Wiederman, Steven D.; Fabian, Joseph M.; Dunbier, James R. eLife, Vol. 6 https://doi.org/10.7554/eLife.26478	journal	July 2017
Bioelectric gene and reaction networks: computational modelling of genetic, biochemical and bioelectrical dynamics in pattern regulation Pietak, Alexis; Levin, Michael Journal of The Royal Society Interface, Vol. 14, Issue 134 https://doi.org/10.1098/rsif.2017.0425	journal	September 2017
A cellular platform for the development of synthetic living machines Blackiston, Douglas; Lederer, Emma; Kriegman, Sam Science Robotics, Vol. 6, Issue 52 https://doi.org/10.1126/scirobotics.abf1571	journal	March 2021
A hierarchical neural-network model for control and learning of voluntary movement Kawato, M.; Furukawa, Kazunori; Suzuki, R. Biological Cybernetics, Vol. 57, Issue 3 https://doi.org/10.1007/BF00364149	journal	October 1987
How we created neuromorphic engineering Mead, Carver Nature Electronics, Vol. 3, Issue 7 https://doi.org/10.1038/s41928-020-0448-2	journal	July 2020
Coordinated memory replay in the visual cortex and hippocampus during sleep Ji, Daoyun; Wilson, Matthew A. Nature Neuroscience, Vol. 10, Issue 1 https://doi.org/10.1038/nn1825	journal	December 2006
Overcoming catastrophic forgetting in neural networks Kirkpatrick, James; Pascanu, Razvan; Rabinowitz, Neil Proceedings of the National Academy of Sciences, Vol. 114, Issue 13 https://doi.org/10.1073/pnas.1611835114	journal	March 2017
Hippocampal Sequences During Exploration: Mechanisms and Functions Drieu, Céline; Zugaro, Michaël Frontiers in Cellular Neuroscience, Vol. 13 https://doi.org/10.3389/fncel.2019.00232	journal	June 2019
Pseudo-recurrent Connectionist Networks: An Approach to the 'Sensitivity-Stability' Dilemma French, Robert M. Connection Science, Vol. 9, Issue 4 https://doi.org/10.1080/095400997116595	journal	December 1997
Decoding motor imagery from the posterior parietal cortex of a tetraplegic human Aflalo, Tyson; Kellis, Spencer; Klaes, Christian Science, Vol. 348, Issue 6237 https://doi.org/10.1126/science.aaa5417	journal	May 2015
Rites of Passage of the Engram Dudai, Yadin; Eisenberg, Mark Neuron, Vol. 44, Issue 1 https://doi.org/10.1016/j.neuron.2004.09.003	journal	September 2004
Normalized shape and location of perturbed craniofacial structures in the Xenopus tadpole reveal an innate ability to achieve correct morphology Vandenberg, Laura N.; Adams, Dany S.; Levin, Michael Developmental Dynamics, Vol. 241, Issue 5 https://doi.org/10.1002/dvdy.23770	journal	March 2012
Schemas and Memory Consolidation Tse, Dorothy; Langston, Rosamund F.; Kakeyama, Masaki Science, Vol. 316, Issue 5821 https://doi.org/10.1126/science.1135935	journal	April 2007
Gene regulatory networks exhibit several kinds of memory: Quantification of memory in biological and random transcriptional networks Biswas, Surama; Manicka, Santosh; Hoel, Erik iScience, Vol. 24, Issue 3 https://doi.org/10.1016/j.isci.2021.102131	journal	March 2021
Long-range neural and gap junction protein-mediated cues control polarity during planarian regeneration Oviedo, Néstor J.; Morokuma, Junji; Walentek, Peter Developmental Biology, Vol. 339, Issue 1 https://doi.org/10.1016/j.ydbio.2009.12.012	journal	March 2010
Designing neural networks through neuroevolution Stanley, Kenneth O.; Clune, Jeff; Lehman, Joel Nature Machine Intelligence, Vol. 1, Issue 1 https://doi.org/10.1038/s42256-018-0006-z	journal	January 2019
Uncertainty-based modulation for lifelong learning Brna, Andrew P.; Brown, Ryan C.; Connolly, Patrick M. Neural Networks, Vol. 120 https://doi.org/10.1016/j.neunet.2019.09.011	journal	December 2019
Is Heterosynaptic modulation essential for stabilizing hebbian plasiticity and memory Bailey, Craig H.; Giustetto, Maurizio; Huang, Yan-You Nature Reviews Neuroscience, Vol. 1, Issue 1 https://doi.org/10.1038/35036191	journal	October 2000
Re-membering the body: applications of computational neuroscience to the top-down control of regeneration of limbs and other complex organs Pezzulo, G.; Levin, M. Integrative Biology, Vol. 7, Issue 12 https://doi.org/10.1039/C5IB00221D	journal	January 2015
Advancing Neuromorphic Computing With Loihi: A Survey of Results and Outlook Davies, Mike; Wild, Andreas; Orchard, Garrick Proceedings of the IEEE, Vol. 109, Issue 5 https://doi.org/10.1109/JPROC.2021.3067593	journal	May 2021
Mechanisms of top-down attention Baluch, Farhan; Itti, Laurent Trends in Neurosciences, Vol. 34, Issue 4 https://doi.org/10.1016/j.tins.2011.02.003	journal	April 2011
Rapid online learning and robust recall in a neuromorphic olfactory circuit Imam, Nabil; Cleland, Thomas A. Nature Machine Intelligence, Vol. 2, Issue 3 https://doi.org/10.1038/s42256-020-0159-4	journal	March 2020
Relational Neurogenesis for Lifelong Learning Agents Pandit, Tej; Kudithipudi, Dhireesha NICE '20: Neuro-inspired Computational Elements Workshop, Proceedings of the Neuro-inspired Computational Elements Workshop https://doi.org/10.1145/3381755.3381766	conference	June 2020
Upgrading the sleeping brain with targeted memory reactivation Oudiette, Delphine; Paller, Ken A. Trends in Cognitive Sciences, Vol. 17, Issue 3 https://doi.org/10.1016/j.tics.2013.01.006	journal	March 2013
Metalearning and neuromodulation Doya, Kenji Neural Networks, Vol. 15, Issue 4-6 https://doi.org/10.1016/S0893-6080(02)00044-8	journal	June 2002
Nanoconnectomic upper bound on the variability of synaptic plasticity Bartol, Thomas M.; Bromer, Cailey; Kinney, Justin eLife, Vol. 4 https://doi.org/10.7554/eLife.10778	journal	November 2015
Schema-Dependent Gene Activation and Memory Encoding in Neocortex Tse, Dorothy; Takeuchi, Tomonori; Kakeyama, Masaki Science, Vol. 333, Issue 6044 https://doi.org/10.1126/science.1205274	journal	August 2011
Hierarchical neural network model for voluntary movement with application to robotics Kawato, M.; Uno, Y.; Isobe, M. IEEE Control Systems Magazine, Vol. 8, Issue 2 https://doi.org/10.1109/37.1867	journal	April 1988
Diffusion-based neuromodulation can eliminate catastrophic forgetting in simple neural networks Velez, Roby; Clune, Jeff PLOS ONE, Vol. 12, Issue 11 https://doi.org/10.1371/journal.pone.0187736	journal	November 2017
Selective suppression of hippocampal ripples impairs spatial memory Girardeau, Gabrielle; Benchenane, Karim; Wiener, Sidney I. Nature Neuroscience, Vol. 12, Issue 10 https://doi.org/10.1038/nn.2384	journal	September 2009
Short-term plasticity as cause–effect hypothesis testing in distal reward learning Soltoggio, Andrea Biological Cybernetics, Vol. 109, Issue 1 https://doi.org/10.1007/s00422-014-0628-0	journal	September 2014
Protein sequestration generates a flexible ultrasensitive response in a genetic network Buchler, Nicolas E.; Cross, Frederick R. Molecular Systems Biology, Vol. 5, Issue 1 https://doi.org/10.1038/msb.2009.30	journal	January 2009
Dorsal Raphe Dopamine Neurons Modulate Arousal and Promote Wakefulness by Salient Stimuli Cho, Jounhong Ryan; Treweek, Jennifer B.; Robinson, J. Elliott Neuron, Vol. 94, Issue 6 https://doi.org/10.1016/j.neuron.2017.05.020	journal	June 2017
Brain-inspired replay for continual learning with artificial neural networks van de Ven, Gido M.; Siegelmann, Hava T.; Tolias, Andreas S. Nature Communications, Vol. 11, Issue 1 https://doi.org/10.1038/s41467-020-17866-2	journal	August 2020
Reactivation of hippocampal ensemble memories during sleep Wilson, M.; McNaughton, B. Science, Vol. 265, Issue 5172 https://doi.org/10.1126/science.8036517	journal	July 1994
Alleviating catastrophic forgetting using context-dependent gating and synaptic stabilization Masse, Nicolas Y.; Grant, Gregory D.; Freedman, David J. Proceedings of the National Academy of Sciences, Vol. 115, Issue 44 https://doi.org/10.1073/pnas.1803839115	journal	October 2018
Dorsal Raphe Dopamine Neurons Represent the Experience of Social Isolation Matthews, Gillian A.; Nieh, Edward H.; Vander Weele, Caitlin M. Cell, Vol. 164, Issue 4 https://doi.org/10.1016/j.cell.2015.12.040	journal	February 2016
A brain-plausible neuromorphic on-the-fly learning system implemented with magnetic domain wall analog memristors Yue, Kun; Liu, Yizhou; Lake, Roger K. Science Advances, Vol. 5, Issue 4 https://doi.org/10.1126/sciadv.aau8170	journal	April 2019
A critique of comparative studies of brain size Healy, Susan D.; Rowe, Candy Proceedings of the Royal Society B: Biological Sciences, Vol. 274, Issue 1609 https://doi.org/10.1098/rspb.2006.3748	journal	December 2006
Neuromorphic meets neuromechanics, part II: the role of fusimotor drive Jalaleddini, Kian; Minos Niu, Chuanxin; Chakravarthi Raja, Suraj Journal of Neural Engineering, Vol. 14, Issue 2 https://doi.org/10.1088/1741-2552/aa59bd	journal	February 2017
A modeling framework for adaptive lifelong learning with transfer and savings through gating in the prefrontal cortex Tsuda, Ben; Tye, Kay M.; Siegelmann, Hava T. Proceedings of the National Academy of Sciences, Vol. 117, Issue 47 https://doi.org/10.1073/pnas.2009591117	journal	November 2020
Complex neural representation of odour information in the olfactory bulb Li, Anan; Rao, Xiaoping; Zhou, Yang Acta Physiologica, Vol. 228, Issue 1 https://doi.org/10.1111/apha.13333	journal	July 2019
Never-ending learning Mitchell, T.; Cohen, W.; Hruschka, E. Communications of the ACM, Vol. 61, Issue 5 https://doi.org/10.1145/3191513	journal	April 2018
Autonomous functional movements in a tendon-driven limb via limited experience Marjaninejad, Ali; Urbina-Meléndez, Darío; Cohn, Brian A. Nature Machine Intelligence, Vol. 1, Issue 3 https://doi.org/10.1038/s42256-019-0029-0	journal	March 2019
Potential role for adult neurogenesis in the encoding of time in new memories Aimone, James B.; Wiles, Janet; Gage, Fred H. Nature Neuroscience, Vol. 9, Issue 6 https://doi.org/10.1038/nn1707	journal	May 2006
Motor primitives in vertebrates and invertebrates Flash, Tamar; Hochner, Binyamin Current Opinion in Neurobiology, Vol. 15, Issue 6 https://doi.org/10.1016/j.conb.2005.10.011	journal	December 2005
The Super-Turing Computational Power of Plastic Recurrent Neural Networks Cabessa, JÉRÉMie; Siegelmann, Hava T. International Journal of Neural Systems, Vol. 24, Issue 08 https://doi.org/10.1142/S0129065714500294	journal	November 2014
The role of metaplasticity mechanisms in regulating memory destabilization and reconsolidation Finnie, Peter S. B.; Nader, Karim Neuroscience & Biobehavioral Reviews, Vol. 36, Issue 7 https://doi.org/10.1016/j.neubiorev.2012.03.008	journal	August 2012
Decay happens: the role of active forgetting in memory Hardt, Oliver; Nader, Karim; Nadel, Lynn Trends in Cognitive Sciences, Vol. 17, Issue 3 https://doi.org/10.1016/j.tics.2013.01.001	journal	March 2013
Born to learn: The inspiration, progress, and future of evolved plastic artificial neural networks Soltoggio, Andrea; Stanley, Kenneth O.; Risi, Sebastian Neural Networks, Vol. 108 https://doi.org/10.1016/j.neunet.2018.07.013	journal	December 2018
Metaplasticity: the plasticity of synaptic plasticity Abraham, Wickliffe C.; Bear, Mark F. Trends in Neurosciences, Vol. 19, Issue 4 https://doi.org/10.1016/S0166-2236(96)80018-X	journal	April 1996
Synaptic metaplasticity in binarized neural networks Laborieux, Axel; Ernoult, Maxence; Hirtzlin, Tifenn Nature Communications, Vol. 12, Issue 1 https://doi.org/10.1038/s41467-021-22768-y	journal	May 2021
Cognitive functions of the basal forebrain Baxter, Mark G.; Chiba, Andrea A. Current Opinion in Neurobiology, Vol. 9, Issue 2 https://doi.org/10.1016/S0959-4388(99)80024-5	journal	April 1999

Similar Records

Design principles for lifelong learning AI accelerators

Journal Article · 2023 · Nature Electronics · OSTI ID:2572151

Task-Based Neuromodulation Architecture for Lifelong Learning

Conference · 2018 · OSTI ID:1542862

Organismic materials for beyond von Neumann machines

Journal Article · 2020 · Applied Physics Reviews · OSTI ID:1607862

Biological underpinnings for lifelong learning machines

Citation Formats

References (179)

Similar Records

Related Subjects