Ergodicity, lack thereof, and the performance of reservoir computing with memristive networks

Baccetti, Valentina; Zhu, Ruomin; Kuncic, Zdenka; Caravelli, Francesco

doi:10.1088/2632-959X/ad2999

Ergodicity, lack thereof, and the performance of reservoir computing with memristive networks

Journal Article · Wed Mar 13 00:00:00 EDT 2024 · Nano Express

DOI:https://doi.org/10.1088/2632-959X/ad2999· OSTI ID:2323572

; ; ;

Abstract

Networks composed of nanoscale memristive components, such as nanowire and nanoparticle networks, have recently received considerable attention because of their potential use as neuromorphic devices. In this study, we explore ergodicity in memristive networks, showing that the performance on machine leaning tasks improves when these networks are tuned to operate at the edge between two global stability points. We find this lack of ergodicity is associated with the emergence of memory in the system. We measure the level of ergodicity using the Thirumalai-Mountain metric, and we show that in the absence of ergodicity, two different memristive network systems show improved performance when utilized as reservoir computers (RC). We highlight that it is also important to let the system synchronize to the input signal in order for the performance of the RC to exhibit improvements over the baseline.

View Journal Article

Research Organization:: Los Alamos National Laboratory (LANL), Los Alamos, NM (United States)

Sponsoring Organization:: USDOE; USDOE Laboratory Directed Research and Development (LDRD) Program; USDOE National Nuclear Security Administration (NNSA)

Grant/Contract Number:: 89233218CNA000001; AC52-06NA25396

OSTI ID:: 2323572

Alternate ID(s):: OSTI ID: 2305545
OSTI ID: 2426784

Report Number(s):: LA-UR--23-30482

Journal Information:: Nano Express, Journal Name: Nano Express Journal Issue: 1 Vol. 5; ISSN 2632-959X

Publisher:: IOP PublishingCopyright Statement

Country of Publication:: United Kingdom

Language:: English

References (61)

Dynamic Electrical Pathway Tuning in Neuromorphic Nanowire Networks Li, Qiao; Diaz‐Alvarez, Adrian; Iguchi, Ryo Advanced Functional Materials, Vol. 30, Issue 43 https://doi.org/10.1002/adfm.202003679	journal	September 2020
Recent Developments and Perspectives for Memristive Devices Based on Metal Oxide Nanowires Milano, Gianluca; Porro, Samuele; Valov, Ilia Advanced Electronic Materials, Vol. 5, Issue 9 https://doi.org/10.1002/aelm.201800909	journal	November 2018
Memristors—From In‐Memory Computing, Deep Learning Acceleration, and Spiking Neural Networks to the Future of Neuromorphic and Bio‐Inspired Computing Mehonic, Adnan; Sebastian, Abu; Rajendran, Bipin Advanced Intelligent Systems, Vol. 2, Issue 11 https://doi.org/10.1002/aisy.202000085	journal	August 2020
Mean Field Theory of Self‐Organizing Memristive Connectomes Caravelli, Francesco; Milano, Gianluca; Ricciardi, Carlo Annalen der Physik, Vol. 535, Issue 8 https://doi.org/10.1002/andp.202300090	journal	June 2023
Computation at the edge of chaos: Phase transitions and emergent computation Langton, Chris G. Physica D: Nonlinear Phenomena, Vol. 42, Issue 1-3 https://doi.org/10.1016/0167-2789(90)90064-V	journal	June 1990
Learning from mistakes Chialvo, D. R.; Bak, P. Neuroscience, Vol. 90, Issue 4 https://doi.org/10.1016/S0306-4522(98)00472-2	journal	June 1999
Near-memory computing: Past, present, and future Singh, Gagandeep; Chelini, Lorenzo; Corda, Stefano Microprocessors and Microsystems, Vol. 71 https://doi.org/10.1016/j.micpro.2019.102868	journal	November 2019
Echo state networks are universal Grigoryeva, Lyudmila; Ortega, Juan-Pablo Neural Networks, Vol. 108 https://doi.org/10.1016/j.neunet.2018.08.025	journal	December 2018
Reservoir computing with 3D nanowire networks Daniels, R. K.; Mallinson, J. B.; Heywood, Z. E. Neural Networks, Vol. 154 https://doi.org/10.1016/j.neunet.2022.07.001	journal	October 2022
Projective embedding of dynamical systems: Uniform mean field equations Caravelli, F.; Traversa, F. L.; Bonnin, M. Physica D: Nonlinear Phenomena, Vol. 450 https://doi.org/10.1016/j.physd.2023.133747	journal	August 2023
Resistance of Single Ag Nanowire Junctions and Their Role in the Conductivity of Nanowire Networks Bellew, Allen T.; Manning, Hugh G.; Gomes da Rocha, Claudia ACS Nano, Vol. 9, Issue 11 https://doi.org/10.1021/acsnano.5b05469	journal	October 2015
Measures of effective ergodic convergence in liquids Mountain, Raymond D.; Thirumalai, D. The Journal of Physical Chemistry, Vol. 93, Issue 19 https://doi.org/10.1021/j100356a019	journal	September 1989
Quantum computing takes flight Oliver, William D. Nature, Vol. 574, Issue 7779 https://doi.org/10.1038/d41586-019-03173-4	journal	October 2019
Neuromorphic computing with nanoscale spintronic oscillators Torrejon, Jacob; Riou, Mathieu; Araujo, Flavio Abreu Nature, Vol. 547, Issue 7664 https://doi.org/10.1038/nature23011	journal	July 2017
Information processing using a single dynamical node as complex system Appeltant, L.; Soriano, M. C.; Van der Sande, G. Nature Communications, Vol. 2, Issue 1 https://doi.org/10.1038/ncomms1476	journal	September 2011
Emergence of winner-takes-all connectivity paths in random nanowire networks Manning, Hugh G.; Niosi, Fabio; da Rocha, Claudia Gomes Nature Communications, Vol. 9, Issue 1 https://doi.org/10.1038/s41467-018-05517-6	journal	August 2018
A poor man’s coherent Ising machine based on opto-electronic feedback systems for solving optimization problems Böhm, Fabian; Verschaffelt, Guy; Van der Sande, Guy Nature Communications, Vol. 10, Issue 1 https://doi.org/10.1038/s41467-019-11484-3	journal	August 2019
Avalanches and edge-of-chaos learning in neuromorphic nanowire networks Hochstetter, Joel; Zhu, Ruomin; Loeffler, Alon Nature Communications, Vol. 12, Issue 1 https://doi.org/10.1038/s41467-021-24260-z	journal	June 2021
Real-space observation of ergodicity transitions in artificial spin ice Saccone, Michael; Caravelli, Francesco; Hofhuis, Kevin Nature Communications, Vol. 14, Issue 1 https://doi.org/10.1038/s41467-023-41235-4	journal	September 2023
In materia reservoir computing with a fully memristive architecture based on self-organizing nanowire networks Milano, Gianluca; Pedretti, Giacomo; Montano, Kevin Nature Materials, Vol. 21, Issue 2 https://doi.org/10.1038/s41563-021-01099-9	journal	October 2021
Memory devices and applications for in-memory computing Sebastian, Abu; Le Gallo, Manuel; Khaddam-Aljameh, Riduan Nature Nanotechnology, Vol. 15, Issue 7 https://doi.org/10.1038/s41565-020-0655-z	journal	March 2020
Direct observation of a dynamical glass transition in a nanomagnetic artificial Hopfield network Saccone, Michael; Caravelli, Francesco; Hofhuis, Kevin Nature Physics, Vol. 18, Issue 5 https://doi.org/10.1038/s41567-022-01538-7	journal	March 2022
Emergent dynamics of neuromorphic nanowire networks Diaz-Alvarez, Adrian; Higuchi, Rintaro; Sanz-Leon, Paula Scientific Reports, Vol. 9, Issue 1 https://doi.org/10.1038/s41598-019-51330-6	journal	October 2019
Information dynamics in neuromorphic nanowire networks Zhu, Ruomin; Hochstetter, Joel; Loeffler, Alon Scientific Reports, Vol. 11, Issue 1 https://doi.org/10.1038/s41598-021-92170-7	journal	June 2021
In-memory computing with resistive switching devices Ielmini, Daniele; Wong, H. -S. Philip Nature Electronics, Vol. 1, Issue 6 https://doi.org/10.1038/s41928-018-0092-2	journal	June 2018
Intrinsic optimization using stochastic nanomagnets Sutton, Brian; Camsari, Kerem Yunus; Behin-Aein, Behtash Scientific Reports, Vol. 7, Issue 1 https://doi.org/10.1038/srep44370	journal	March 2017
Generalized Formula for the Electric Tunnel Effect between Similar Electrodes Separated by a Thin Insulating Film Simmons, John G. Journal of Applied Physics, Vol. 34, Issue 6 https://doi.org/10.1063/1.1702682	journal	June 1963
Perspective: Memcomputing: Leveraging memory and physics to compute efficiently Di Ventra, Massimiliano; Traversa, Fabio L. Journal of Applied Physics, Vol. 123, Issue 18 https://doi.org/10.1063/1.5026506	journal	May 2018
Coupled oscillators for computing: A review and perspective Csaba, Gyorgy; Porod, Wolfgang Applied Physics Reviews, Vol. 7, Issue 1 https://doi.org/10.1063/1.5120412	journal	March 2020
Physics successfully implements Lagrange multiplier optimization Vadlamani, Sri Krishna; Xiao, Tianyao Patrick; Yablonovitch, Eli Proceedings of the National Academy of Sciences, Vol. 117, Issue 43 https://doi.org/10.1073/pnas.2015192117	journal	October 2020
Broken ergodicity Palmer, R. G. Advances in Physics, Vol. 31, Issue 6 https://doi.org/10.1080/00018738200101438	journal	December 1982
The elusive memristor: properties of basic electrical circuits Joglekar, Yogesh N.; Wolf, Stephen J. European Journal of Physics, Vol. 30, Issue 4 https://doi.org/10.1088/0143-0807/30/4/001	journal	May 2009
A theoretical and experimental study of neuromorphic atomic switch networks for reservoir computing Sillin, Henry O.; Aguilera, Renato; Shieh, Hsien-Hang Nanotechnology, Vol. 24, Issue 38 https://doi.org/10.1088/0957-4484/24/38/384004	journal	September 2013
Dynamic computing random access memory Traversa, F. L.; Bonani, F.; Pershin, Y. V. Nanotechnology, Vol. 25, Issue 28 https://doi.org/10.1088/0957-4484/25/28/285201	journal	June 2014
Modularity and multitasking in neuro-memristive reservoir networks Loeffler, Alon; Zhu, Ruomin; Hochstetter, Joel Neuromorphic Computing and Engineering, Vol. 1, Issue 1 https://doi.org/10.1088/2634-4386/ac156f	journal	August 2021
Ergodic behavior in supercooled liquids and in glasses Thirumalai, D.; Mountain, Raymond D.; Kirkpatrick, T. R. Physical Review A, Vol. 39, Issue 7 https://doi.org/10.1103/PhysRevA.39.3563	journal	April 1989
Learning by mistakes in memristor networks Carbajal, Juan Pablo; Martin, Daniel A.; Chialvo, Dante R. Physical Review E, Vol. 105, Issue 5 https://doi.org/10.1103/PhysRevE.105.054306	journal	May 2022
Computational capacity of LRC , memristive, and hybrid reservoirs Sheldon, Forrest C.; Kolchinsky, Artemy; Caravelli, Francesco Physical Review E, Vol. 106, Issue 4 https://doi.org/10.1103/PhysRevE.106.045310	journal	October 2022
Activated dynamics, loss of ergodicity, and transport in supercooled liquids Thirumalai, D.; Mountain, Raymond D. Physical Review E, Vol. 47, Issue 1 https://doi.org/10.1103/PhysRevE.47.479	journal	January 1993
Ergodicity in natural earthquake fault networks Tiampo, K. F.; Rundle, J. B.; Klein, W. Physical Review E, Vol. 75, Issue 6 https://doi.org/10.1103/PhysRevE.75.066107	journal	June 2007
Complex dynamics and scale invariance of one-dimensional memristive networks Pershin, Yuriy V.; Slipko, Valeriy A.; Di Ventra, Massimiliano Physical Review E, Vol. 87, Issue 2 https://doi.org/10.1103/PhysRevE.87.022116	journal	February 2013
Effective ergodicity in single-spin-flip dynamics Süzen, Mehmet Physical Review E, Vol. 90, Issue 3 https://doi.org/10.1103/PhysRevE.90.032141	journal	September 2014
Complex dynamics of memristive circuits: Analytical results and universal slow relaxation Caravelli, F.; Traversa, F. L.; Di Ventra, M. Physical Review E, Vol. 95, Issue 2 https://doi.org/10.1103/PhysRevE.95.022140	journal	February 2017
Locality of interactions for planar memristive circuits Caravelli, F. Physical Review E, Vol. 96, Issue 5 https://doi.org/10.1103/PhysRevE.96.052206	journal	November 2017
Large-Scale Photonic Ising Machine by Spatial Light Modulation Pierangeli, D.; Marcucci, G.; Conti, C. Physical Review Letters, Vol. 122, Issue 21 https://doi.org/10.1103/PhysRevLett.122.213902	journal	May 2019
Ergodic Dynamics in a Natural Threshold System Tiampo, K. F.; Rundle, J. B.; Klein, W. Physical Review Letters, Vol. 91, Issue 23 https://doi.org/10.1103/PhysRevLett.91.238501	journal	December 2003
Memristor-The missing circuit element Chua, L. IEEE Transactions on Circuit Theory, Vol. 18, Issue 5 https://doi.org/10.1109/TCT.1971.1083337	journal	January 1971
A Versatile Memristor Model With Nonlinear Dopant Kinetics Prodromakis, Themistoklis; Peh, Boon Pin; Papavassiliou, Christos IEEE Transactions on Electron Devices, Vol. 58, Issue 9 https://doi.org/10.1109/TED.2011.2158004	journal	September 2011
Universal Memcomputing Machines Traversa, Fabio Lorenzo; Di Ventra, Massimiliano IEEE Transactions on Neural Networks and Learning Systems, Vol. 26, Issue 11 https://doi.org/10.1109/TNNLS.2015.2391182	journal	November 2015
High-performance combinatorial optimization based on classical mechanics Goto, Hayato; Endo, Kotaro; Suzuki, Masaru Science Advances, Vol. 7, Issue 6 https://doi.org/10.1126/sciadv.abe7953	journal	February 2021
Global minimization via classical tunneling assisted by collective force field formation Caravelli, Francesco; Sheldon, Forrest C.; Traversa, Fabio L. Science Advances, Vol. 7, Issue 52 https://doi.org/10.1126/sciadv.abh1542	journal	December 2021
Neuromorphic learning, working memory, and metaplasticity in nanowire networks Loeffler, Alon; Diaz-Alvarez, Adrian; Zhu, Ruomin Science Advances, Vol. 9, Issue 16 https://doi.org/10.1126/sciadv.adg3289	journal	April 2023
Harnessing Nonlinearity: Predicting Chaotic Systems and Saving Energy in Wireless Communication Jaeger, H. Science, Vol. 304, Issue 5667 https://doi.org/10.1126/science.1091277	journal	April 2004
Optimization by Simulated Annealing Kirkpatrick, S.; Gelatt, C. D.; Vecchi, M. P. Science, Vol. 220, Issue 4598 https://doi.org/10.1126/science.220.4598.671	journal	May 1983
Oscillation and Chaos in Physiological Control Systems Mackey, Michael C.; Glass, Leon Science, Vol. 197, Issue 4300 https://doi.org/10.1126/science.267326	journal	July 1977
A new golden age for computer architecture Hennessy, John L.; Patterson, David A. Communications of the ACM, Vol. 62, Issue 2 https://doi.org/10.1145/3282307	journal	January 2019
Weak ergodicity breaking with deterministic dynamics Bel, G.; Barkai, E. Europhysics Letters (EPL), Vol. 74, Issue 1 https://doi.org/10.1209/epl/i2005-10501-8	journal	April 2006
Topological Properties of Neuromorphic Nanowire Networks Loeffler, Alon; Zhu, Ruomin; Hochstetter, Joel Frontiers in Neuroscience, Vol. 14 https://doi.org/10.3389/fnins.2020.00184	journal	March 2020
Criticality in Large-Scale Brain fMRI Dynamics Unveiled by a Novel Point Process Analysis Tagliazucchi, Enzo; Balenzuela, Pablo; Fraiman, Daniel Frontiers in Physiology, Vol. 3 https://doi.org/10.3389/fphys.2012.00015	journal	January 2012
Optimal Input Representation in Neural Systems at the Edge of Chaos Morales, Guillermo B.; Muñoz, Miguel A. Biology, Vol. 10, Issue 8 https://doi.org/10.3390/biology10080702	journal	July 2021
Asymptotic Behavior of Memristive Circuits Caravelli, Francesco Entropy, Vol. 21, Issue 8 https://doi.org/10.3390/e21080789	journal	August 2019

Similar Records

Ergodicity, lack thereof, and the performance of reservoir computing with memristive networks and nanowire

Technical Report · Wed Mar 13 00:00:00 EDT 2024 · OSTI ID:2386906

Related Subjects

97 MATHEMATICS AND COMPUTING
Computer Science
ergodicity
memory
memristors
nanoscale memristive networks
nanowires
reservoir computing

Ergodicity, lack thereof, and the performance of reservoir computing with memristive networks

Citation Formats

References (61)

Similar Records

Related Subjects