Ergodicity, lack thereof, and the performance of reservoir computing with memristive networks

Baccetti, Valentina; Zhu, Ruomin; Kuncic, Zdenka; Caravelli, Francesco

doi:10.1088/2632-959X/ad2999

Title: Ergodicity, lack thereof, and the performance of reservoir computing with memristive networks

Journal Article · Wed Mar 13 00:00:00 EDT 2024 · Nano Express

DOI:https://doi.org/10.1088/2632-959X/ad2999· OSTI ID:2323572

;

Abstract Networks composed of nanoscale memristive components, such as nanowire and nanoparticle net-works, have recently received considerable attention because of their potential use as neuromorphic devices. In this study, we explore ergodicity in memristive networks, showing that the performance on machine leaning tasks improves when these networks are tuned to operate at the edge between two global stability points. We find this lack of ergodicity is associated with the emergence of memory in the system. We measure the level of ergodicity using the Thirumalai-Mountain metric, and we show that in the absence of ergodicity, two different memristive network systems show improved performance when utilized as reservoir computers (RC). We highlight that it is also important to let the system synchronize to the input signal in order for the performance of the RC to exhibit improvements over the baseline.

View Journal Article

Cite

Export

Save

Sponsoring Organization:: USDOE

OSTI ID:: 2323572

Alternate ID(s):: OSTI ID: 2305545

Journal Information:: Nano Express, Journal Name: Nano Express Vol. 5 Journal Issue: 1; ISSN 2632-959X

Publisher:: IOP PublishingCopyright Statement

Country of Publication:: United Kingdom

Language:: English

References (60)

Dynamic computing random access memory Traversa, F. L.; Bonani, F.; Pershin, Y. V. Nanotechnology, Vol. 25, Issue 28 https://doi.org/10.1088/0957-4484/25/28/285201	journal	June 2014
Emergence of winner-takes-all connectivity paths in random nanowire networks Manning, Hugh G.; Niosi, Fabio; da Rocha, Claudia Gomes Nature Communications, Vol. 9, Issue 1 https://doi.org/10.1038/s41467-018-05517-6	journal	August 2018
Large-Scale Photonic Ising Machine by Spatial Light Modulation Pierangeli, D.; Marcucci, G.; Conti, C. Physical Review Letters, Vol. 122, Issue 21 https://doi.org/10.1103/PhysRevLett.122.213902	journal	May 2019
Neuromorphic computing with nanoscale spintronic oscillators Torrejon, Jacob; Riou, Mathieu; Araujo, Flavio Abreu Nature, Vol. 547, Issue 7664 https://doi.org/10.1038/nature23011	journal	July 2017
The elusive memristor: properties of basic electrical circuits Joglekar, Yogesh N.; Wolf, Stephen J. European Journal of Physics, Vol. 30, Issue 4 https://doi.org/10.1088/0143-0807/30/4/001	journal	May 2009
Avalanches and edge-of-chaos learning in neuromorphic nanowire networks Hochstetter, Joel; Zhu, Ruomin; Loeffler, Alon Nature Communications, Vol. 12, Issue 1 https://doi.org/10.1038/s41467-021-24260-z	journal	June 2021
Quantum computing takes flight Oliver, William D. Nature, Vol. 574, Issue 7779 https://doi.org/10.1038/d41586-019-03173-4	journal	October 2019
Coupled oscillators for computing: A review and perspective Csaba, Gyorgy; Porod, Wolfgang Applied Physics Reviews, Vol. 7, Issue 1 https://doi.org/10.1063/1.5120412	journal	March 2020
Information dynamics in neuromorphic nanowire networks Zhu, Ruomin; Hochstetter, Joel; Loeffler, Alon Scientific Reports, Vol. 11, Issue 1 https://doi.org/10.1038/s41598-021-92170-7	journal	June 2021
Memory devices and applications for in-memory computing Sebastian, Abu; Le Gallo, Manuel; Khaddam-Aljameh, Riduan Nature Nanotechnology, Vol. 15, Issue 7 https://doi.org/10.1038/s41565-020-0655-z	journal	March 2020
A new golden age for computer architecture Hennessy, John L.; Patterson, David A. Communications of the ACM, Vol. 62, Issue 2 https://doi.org/10.1145/3282307	journal	January 2019
Memristors—From In‐Memory Computing, Deep Learning Acceleration, and Spiking Neural Networks to the Future of Neuromorphic and Bio‐Inspired Computing Mehonic, Adnan; Sebastian, Abu; Rajendran, Bipin Advanced Intelligent Systems, Vol. 2, Issue 11 https://doi.org/10.1002/aisy.202000085	journal	August 2020
Near-memory computing: Past, present, and future Singh, Gagandeep; Chelini, Lorenzo; Corda, Stefano Microprocessors and Microsystems, Vol. 71 https://doi.org/10.1016/j.micpro.2019.102868	journal	November 2019
A theoretical and experimental study of neuromorphic atomic switch networks for reservoir computing Sillin, Henry O.; Aguilera, Renato; Shieh, Hsien-Hang Nanotechnology, Vol. 24, Issue 38 https://doi.org/10.1088/0957-4484/24/38/384004	journal	September 2013
Perspective: Memcomputing: Leveraging memory and physics to compute efficiently Di Ventra, Massimiliano; Traversa, Fabio L. Journal of Applied Physics, Vol. 123, Issue 18 https://doi.org/10.1063/1.5026506	journal	May 2018
Memristor-The missing circuit element Chua, L. IEEE Transactions on Circuit Theory, Vol. 18, Issue 5 https://doi.org/10.1109/TCT.1971.1083337	journal	January 1971
Recent Developments and Perspectives for Memristive Devices Based on Metal Oxide Nanowires Milano, Gianluca; Porro, Samuele; Valov, Ilia Advanced Electronic Materials, Vol. 5, Issue 9 https://doi.org/10.1002/aelm.201800909	journal	November 2018
Direct observation of a dynamical glass transition in a nanomagnetic artificial Hopfield network Saccone, Michael; Caravelli, Francesco; Hofhuis, Kevin Nature Physics, Vol. 18, Issue 5 https://doi.org/10.1038/s41567-022-01538-7	journal	March 2022
In-memory computing with resistive switching devices Ielmini, Daniele; Wong, H. -S. Philip Nature Electronics, Vol. 1, Issue 6 https://doi.org/10.1038/s41928-018-0092-2	journal	June 2018
Emergent dynamics of neuromorphic nanowire networks Diaz-Alvarez, Adrian; Higuchi, Rintaro; Sanz-Leon, Paula Scientific Reports, Vol. 9, Issue 1 https://doi.org/10.1038/s41598-019-51330-6	journal	October 2019
Topological Properties of Neuromorphic Nanowire Networks Loeffler, Alon; Zhu, Ruomin; Hochstetter, Joel Frontiers in Neuroscience, Vol. 14 https://doi.org/10.3389/fnins.2020.00184	journal	March 2020
Intrinsic optimization using stochastic nanomagnets Sutton, Brian; Camsari, Kerem Yunus; Behin-Aein, Behtash Scientific Reports, Vol. 7, Issue 1 https://doi.org/10.1038/srep44370	journal	March 2017
Measures of effective ergodic convergence in liquids Mountain, Raymond D.; Thirumalai, D. The Journal of Physical Chemistry, Vol. 93, Issue 19 https://doi.org/10.1021/j100356a019	journal	September 1989
Weak ergodicity breaking with deterministic dynamics Bel, G.; Barkai, E. Europhysics Letters (EPL), Vol. 74, Issue 1 https://doi.org/10.1209/epl/i2005-10501-8	journal	April 2006
Echo state networks are universal Grigoryeva, Lyudmila; Ortega, Juan-Pablo Neural Networks, Vol. 108 https://doi.org/10.1016/j.neunet.2018.08.025	journal	December 2018
Ergodicity in natural earthquake fault networks Tiampo, K. F.; Rundle, J. B.; Klein, W. Physical Review E, Vol. 75, Issue 6 https://doi.org/10.1103/PhysRevE.75.066107	journal	June 2007
Oscillation and Chaos in Physiological Control Systems Mackey, Michael C.; Glass, Leon Science, Vol. 197, Issue 4300 https://doi.org/10.1126/science.267326	journal	July 1977
Computational capacity of LRC , memristive, and hybrid reservoirs Sheldon, Forrest C.; Kolchinsky, Artemy; Caravelli, Francesco Physical Review E, Vol. 106, Issue 4 https://doi.org/10.1103/PhysRevE.106.045310	journal	October 2022
Locality of interactions for planar memristive circuits Caravelli, F. Physical Review E, Vol. 96, Issue 5 https://doi.org/10.1103/PhysRevE.96.052206	journal	November 2017
Modularity and multitasking in neuro-memristive reservoir networks Loeffler, Alon; Zhu, Ruomin; Hochstetter, Joel Neuromorphic Computing and Engineering, Vol. 1, Issue 1 https://doi.org/10.1088/2634-4386/ac156f	journal	August 2021
Optimal Input Representation in Neural Systems at the Edge of Chaos Morales, Guillermo B.; Muñoz, Miguel A. Biology, Vol. 10, Issue 8 https://doi.org/10.3390/biology10080702	journal	July 2021
Universal Memcomputing Machines Traversa, Fabio Lorenzo; Di Ventra, Massimiliano IEEE Transactions on Neural Networks and Learning Systems, Vol. 26, Issue 11 https://doi.org/10.1109/TNNLS.2015.2391182	journal	November 2015
Effective ergodicity in single-spin-flip dynamics Süzen, Mehmet Physical Review E, Vol. 90, Issue 3 https://doi.org/10.1103/PhysRevE.90.032141	journal	September 2014
Complex dynamics of memristive circuits: Analytical results and universal slow relaxation Caravelli, F.; Traversa, F. L.; Di Ventra, M. Physical Review E, Vol. 95, Issue 2 https://doi.org/10.1103/PhysRevE.95.022140	journal	February 2017
Real-space observation of ergodicity transitions in artificial spin ice Saccone, Michael; Caravelli, Francesco; Hofhuis, Kevin Nature Communications, Vol. 14, Issue 1 https://doi.org/10.1038/s41467-023-41235-4	journal	September 2023
Criticality in Large-Scale Brain fMRI Dynamics Unveiled by a Novel Point Process Analysis Tagliazucchi, Enzo; Balenzuela, Pablo; Fraiman, Daniel Frontiers in Physiology, Vol. 3 https://doi.org/10.3389/fphys.2012.00015	journal	January 2012
Activated dynamics, loss of ergodicity, and transport in supercooled liquids Thirumalai, D.; Mountain, Raymond D. Physical Review E, Vol. 47, Issue 1 https://doi.org/10.1103/PhysRevE.47.479	journal	January 1993
Broken ergodicity Palmer, R. G. Advances in Physics, Vol. 31, Issue 6 https://doi.org/10.1080/00018738200101438	journal	December 1982
Reservoir computing with 3D nanowire networks Daniels, R. K.; Mallinson, J. B.; Heywood, Z. E. Neural Networks, Vol. 154 https://doi.org/10.1016/j.neunet.2022.07.001	journal	October 2022
Learning by mistakes in memristor networks Carbajal, Juan Pablo; Martin, Daniel A.; Chialvo, Dante R. Physical Review E, Vol. 105, Issue 5 https://doi.org/10.1103/PhysRevE.105.054306	journal	May 2022
Learning from mistakes Chialvo, D. R.; Bak, P. Neuroscience, Vol. 90, Issue 4 https://doi.org/10.1016/S0306-4522(98)00472-2	journal	June 1999
Harnessing Nonlinearity: Predicting Chaotic Systems and Saving Energy in Wireless Communication Jaeger, H. Science, Vol. 304, Issue 5667 https://doi.org/10.1126/science.1091277	journal	April 2004
In materia reservoir computing with a fully memristive architecture based on self-organizing nanowire networks Milano, Gianluca; Pedretti, Giacomo; Montano, Kevin Nature Materials, Vol. 21, Issue 2 https://doi.org/10.1038/s41563-021-01099-9	journal	October 2021
Information processing using a single dynamical node as complex system Appeltant, L.; Soriano, M. C.; Van der Sande, G. Nature Communications, Vol. 2, Issue 1 https://doi.org/10.1038/ncomms1476	journal	September 2011
Ergodic behavior in supercooled liquids and in glasses Thirumalai, D.; Mountain, Raymond D.; Kirkpatrick, T. R. Physical Review A, Vol. 39, Issue 7 https://doi.org/10.1103/PhysRevA.39.3563	journal	April 1989
Ergodic Dynamics in a Natural Threshold System Tiampo, K. F.; Rundle, J. B.; Klein, W. Physical Review Letters, Vol. 91, Issue 23 https://doi.org/10.1103/PhysRevLett.91.238501	journal	December 2003
A Versatile Memristor Model With Nonlinear Dopant Kinetics Prodromakis, Themistoklis; Peh, Boon Pin; Papavassiliou, Christos IEEE Transactions on Electron Devices, Vol. 58, Issue 9 https://doi.org/10.1109/TED.2011.2158004	journal	September 2011
High-performance combinatorial optimization based on classical mechanics Goto, Hayato; Endo, Kotaro; Suzuki, Masaru Science Advances, Vol. 7, Issue 6 https://doi.org/10.1126/sciadv.abe7953	journal	February 2021
Projective embedding of dynamical systems: Uniform mean field equations Caravelli, F.; Traversa, F. L.; Bonnin, M. Physica D: Nonlinear Phenomena, Vol. 450 https://doi.org/10.1016/j.physd.2023.133747	journal	August 2023
Asymptotic Behavior of Memristive Circuits Caravelli, Francesco Entropy, Vol. 21, Issue 8 https://doi.org/10.3390/e21080789	journal	August 2019
Global minimization via classical tunneling assisted by collective force field formation Caravelli, Francesco; Sheldon, Forrest C.; Traversa, Fabio L. Science Advances, Vol. 7, Issue 52 https://doi.org/10.1126/sciadv.abh1542	journal	December 2021
Complex dynamics and scale invariance of one-dimensional memristive networks Pershin, Yuriy V.; Slipko, Valeriy A.; Di Ventra, Massimiliano Physical Review E, Vol. 87, Issue 2 https://doi.org/10.1103/PhysRevE.87.022116	journal	February 2013
Generalized Formula for the Electric Tunnel Effect between Similar Electrodes Separated by a Thin Insulating Film Simmons, John G. Journal of Applied Physics, Vol. 34, Issue 6 https://doi.org/10.1063/1.1702682	journal	June 1963
Dynamic Electrical Pathway Tuning in Neuromorphic Nanowire Networks Li, Qiao; Diaz‐Alvarez, Adrian; Iguchi, Ryo Advanced Functional Materials, Vol. 30, Issue 43 https://doi.org/10.1002/adfm.202003679	journal	September 2020
A poor man’s coherent Ising machine based on opto-electronic feedback systems for solving optimization problems Böhm, Fabian; Verschaffelt, Guy; Van der Sande, Guy Nature Communications, Vol. 10, Issue 1 https://doi.org/10.1038/s41467-019-11484-3	journal	August 2019
Optimization by Simulated Annealing Kirkpatrick, S.; Gelatt, C. D.; Vecchi, M. P. Science, Vol. 220, Issue 4598 https://doi.org/10.1126/science.220.4598.671	journal	May 1983
Physics successfully implements Lagrange multiplier optimization Vadlamani, Sri Krishna; Xiao, Tianyao Patrick; Yablonovitch, Eli Proceedings of the National Academy of Sciences, Vol. 117, Issue 43 https://doi.org/10.1073/pnas.2015192117	journal	October 2020
Resistance of Single Ag Nanowire Junctions and Their Role in the Conductivity of Nanowire Networks Bellew, Allen T.; Manning, Hugh G.; Gomes da Rocha, Claudia ACS Nano, Vol. 9, Issue 11 https://doi.org/10.1021/acsnano.5b05469	journal	October 2015
Neuromorphic learning, working memory, and metaplasticity in nanowire networks Loeffler, Alon; Diaz-Alvarez, Adrian; Zhu, Ruomin Science Advances, Vol. 9, Issue 16 https://doi.org/10.1126/sciadv.adg3289	journal	April 2023
Mean Field Theory of Self‐Organizing Memristive Connectomes Caravelli, Francesco; Milano, Gianluca; Ricciardi, Carlo Annalen der Physik, Vol. 535, Issue 8 https://doi.org/10.1002/andp.202300090	journal	June 2023

Similar Records

Enhanced read resolution in reconfigurable memristive synapses for Spiking Neural Networks

Journal Article · Wed Apr 17 00:00:00 EDT 2024 · Scientific Reports · OSTI ID:2323572

Das, Hritom; Schuman, Catherine; Chakraborty, Nishith N.; +1 more

Mean Field Theory of Self‐Organizing Memristive Connectomes

Journal Article · Tue Jun 06 00:00:00 EDT 2023 · Annalen der Physik (Leipzig) · OSTI ID:2323572

Caravelli, Francesco; Milano, Gianluca; Ricciardi, Carlo; +1 more

Memristive Mixed-Signal Neuromorphic Systems: Energy-Efficient Learning at the Circuit-Level

Journal Article · Thu Nov 23 00:00:00 EST 2017 · IEEE Journal on Emerging and Selected Topics in Circuits and Systems · OSTI ID:2323572

Chakma, Gangotree; Adnan, Md Musabbir; Wyer, Austin R.; +3 more

Title: Ergodicity, lack thereof, and the performance of reservoir computing with memristive networks

Citation Formats

References (60)

Similar Records

Related Subjects