DOE PAGES title logo U.S. Department of Energy
Office of Scientific and Technical Information

Title: Accelerating Scientific Computing in the Post-Moore’s Era

Journal Article · · ACM Transactions on Parallel Computing
DOI: https://doi.org/10.1145/3380940 · OSTI ID:1649624

Novel uses of graphical processing units for accelerated computation revolutionized the field of high-performance scientific computing by providing specialized workflows tailored to algorithmic requirements. As the era of Moore’s law draws to a close, many new non–von Neumann processors are emerging as potential computational accelerators, including those based on the principles of neuromorphic computing, tensor algebra, and quantum information. While development of these new processors is continuing to mature, the potential impact on accelerated computing is anticipated to be profound. We discuss how different processing models can advance computing in key scientific paradigms: machine learning and constraint satisfaction. Significantly, each of these new processor types utilizes a fundamentally different model of computation, and this raises questions about how to best use such processors in the design and implementation of applications. While many processors are being developed with a specific domain target, the ubiquity of spin-glass models and neural networks provides an avenue for multi-functional applications. Furthermore, this also hints at the infrastructure needed to integrate next-generation processing units into future high-performance computing systems.

Research Organization:
Oak Ridge National Laboratory (ORNL), Oak Ridge, TN (United States)
Sponsoring Organization:
USDOE
Grant/Contract Number:
AC05-00OR22725
OSTI ID:
1649624
Journal Information:
ACM Transactions on Parallel Computing, Journal Name: ACM Transactions on Parallel Computing Journal Issue: 1 Vol. 7; ISSN 2329-4949
Publisher:
Association for Computing MachineryCopyright Statement
Country of Publication:
United States
Language:
English

References (173)

Quantum Computing in the NISQ era and beyond journal August 2018
A Study of Complex Deep Learning Networks on High Performance, Neuromorphic, and Quantum Computers conference November 2016
Most Tensor Problems Are NP-Hard journal November 2013
Local resolution-limit-free Potts model for community detection journal April 2010
Optimizing quantum optimization algorithms via faster quantum gradient computation book January 2019
High-Performance Computing with Quantum Processing Units journal May 2017
Optimizing adiabatic quantum program compilation using a graph-theoretic framework journal April 2018
Error Mitigation for Short-Depth Quantum Circuits journal November 2017
Quantum annealing with manufactured spins journal May 2011
Barren plateaus in quantum neural network training landscapes journal November 2018
Learning and inference on generative adversarial quantum circuits journal May 2019
Ising formulations of many NP problems journal January 2014
Convolutional networks for fast, energy-efficient neuromorphic computing journal September 2016
Long Short-Term Memory journal November 1997
What is the Computational Value of Finite-Range Tunneling? journal August 2016
Sublinear-Time Quantum Computation of the Diameter in CONGEST Networks
  • Le Gall, François; Magniez, Frédéric
  • PODC '18: ACM Symposium on Principles of Distributed Computing, Proceedings of the 2018 ACM Symposium on Principles of Distributed Computing https://doi.org/10.1145/3212734.3212744
conference July 2018
Conversion of artificial recurrent neural networks to spiking neural networks for low-power neuromorphic hardware conference October 2016
Demonstration of a Scaling Advantage for a Quantum Annealer over Simulated Annealing journal July 2018
A Learning Algorithm for Boltzmann Machines* journal January 1985
Temperature Scaling Law for Quantum Annealing Optimizers journal September 2017
Simulating a perceptron on a quantum computer journal March 2015
The basins of attraction of a new Hopfield learning rule journal July 1999
Hardware-efficient variational quantum eigensolver for small molecules and quantum magnets journal September 2017
Label propagation through neuronal synchrony conference July 2010
Training a quantum optimizer journal August 2016
Trapped-ion quantum computing: Progress and challenges journal June 2019
Optimization by quantum annealing: Lessons from hard satisfiability problems journal June 2005
Performance Comparison of the Digital Neuromorphic Hardware SpiNNaker and the Neural Network Simulation Software NEST for a Full-Scale Cortical Microcircuit Model journal May 2018
Neural networks and physical systems with emergent collective computational abilities. journal April 1982
Adversarial quantum circuit learning for pure state approximation journal April 2019
Solvable Model of a Spin-Glass journal December 1975
A fully programmable 100-spin coherent Ising machine with all-to-all connections journal October 2016
Adiabatic quantum programming: minor embedding with hard faults journal November 2013
Phase transitions in a programmable quantum spin glass simulator journal July 2018
Quantum Generative Adversarial Learning journal July 2018
Rebooting Computing: The Road Ahead journal January 2017
Efficient Variational Quantum Simulator Incorporating Active Error Minimization journal June 2017
Production of photonic universal quantum gates enhanced by machine learning journal July 2019
Design and Analysis of a Neuromemristive Reservoir Computing Architecture for Biosignal Processing journal February 2016
SpiNNaker: A 1-W 18-Core System-on-Chip for Massively-Parallel Neural Network Simulation journal August 2013
Scalable energy-efficient, low-latency implementations of trained spiking Deep Belief Networks on SpiNNaker conference July 2015
A Fast Learning Algorithm for Deep Belief Nets journal July 2006
Differentiable learning of quantum circuit Born machines journal December 2018
Simulated annealing book January 1987
Circuit-centric quantum classifiers journal March 2020
Cognitive computing building block: A versatile and efficient digital neuron model for neurosynaptic cores
  • Cassidy, Andrew S.; Merolla, Paul; Arthur, John V.
  • 2013 International Joint Conference on Neural Networks (IJCNN 2013 - Dallas), The 2013 International Joint Conference on Neural Networks (IJCNN) https://doi.org/10.1109/IJCNN.2013.6707077
conference August 2013
Quantum Computation and Quantum Information book January 2011
Network of time-multiplexed optical parametric oscillators as a coherent Ising machine journal October 2014
Phase transitions in random Potts systems and the community detection problem: spin-glass type and dynamic perspectives journal February 2012
Experimental comparison of two quantum computing architectures journal March 2017
Silicon chip delivers quantum speeds [News] journal July 2018
Adiabatic quantum optimization for associative memory recall journal December 2014
In-Datacenter Performance Analysis of a Tensor Processing Unit conference January 2017
Motivation for and Evaluation of the First Tensor Processing Unit journal May 2018
Quantum annealing: The fastest route to quantum computation? journal February 2015
The many facets of community detection in complex networks journal February 2017
Evaluating analytic gradients on quantum hardware journal March 2019
Storing Infinite Numbers of Patterns in a Spin-Glass Model of Neural Networks journal September 1985
Error-backpropagation in temporally encoded networks of spiking neurons journal October 2002
Mapping Generative Models onto a Network of Digital Spiking Neurons journal August 2016
Competitive Hebbian learning through spike-timing-dependent synaptic plasticity journal September 2000
Empirical comparison of algorithms for network community detection conference January 2010
Characteristics of sparsely encoded associative memory journal January 1989
Quantum generative adversarial networks journal July 2018
Opportunities and challenges for quantum-assisted machine learning in near-term quantum computers journal June 2018
Keeneland: Bringing Heterogeneous GPU Computing to the Computational Science Community journal September 2011
Pattern recall in networks of chaotic neurons journal February 2007
Experimental investigation of performance differences between coherent Ising machines and a quantum annealer journal May 2019
Demonstration of quantum advantage in machine learning journal April 2017
The complexity of some polynomial network consistency algorithms for constraint satisfaction problems journal January 1985
The SpiNNaker Project journal May 2014
Glassy Chimeras Could Be Blind to Quantum Speedup: Designing Better Benchmarks for Quantum Annealing Machines journal April 2014
Memristive Mixed-Signal Neuromorphic Systems: Energy-Efficient Learning at the Circuit-Level journal March 2018
Quantum optimization using variational algorithms on near-term quantum devices journal June 2018
A quantum engineer's guide to superconducting qubits journal June 2019
Quantum machine learning journal September 2017
An integrated programming and development environment for adiabatic quantum optimization journal January 2014
A fast quantum mechanical algorithm for database search conference January 1996
Probing many-body dynamics on a 51-atom quantum simulator journal November 2017
A Quantum Adiabatic Evolution Algorithm Applied to Random Instances of an NP-Complete Problem journal April 2001
Sparse Coding via Thresholding and Local Competition in Neural Circuits journal October 2008
Sparse Hardware Embedding of Spiking Neuron Systems for Community Detection
  • Hamilton, Kathleen E.; Imam, Neena; Humble, Travis S.
  • ACM Journal on Emerging Technologies in Computing Systems, Vol. 14, Issue 4 https://doi.org/10.1145/3223048
journal December 2018
Detecting Fuzzy Community Structures in Complex Networks with a Potts Model journal November 2004
Adiabatic quantum computation journal January 2018
An evolutionary optimization framework for neural networks and neuromorphic architectures conference July 2016
Ising formulation of associative memory models and quantum annealing recall journal December 2017
What would you do with 1000 qubits? journal June 2018
Machine learning & artificial intelligence in the quantum domain: a review of recent progress journal June 2018
Cognitive computing systems: Algorithms and applications for networks of neurosynaptic cores
  • Esser, Steve K.; Andreopoulos, Alexander; Appuswamy, Rathinakumar
  • 2013 International Joint Conference on Neural Networks (IJCNN 2013 - Dallas), The 2013 International Joint Conference on Neural Networks (IJCNN) https://doi.org/10.1109/IJCNN.2013.6706746
conference August 2013
Community detection in graphs journal February 2010
Willshaw model: Associative memory with sparse coding and low firing rates journal February 1990
Error mitigation extends the computational reach of a noisy quantum processor journal March 2019
Observation of topological phenomena in a programmable lattice of 1,800 qubits journal August 2018
A coherent Ising machine for 2000-node optimization problems journal October 2016
Locally Competitive Algorithms for Sparse Approximation conference September 2007
Memristive Ion Channel-Doped Biomembranes as Synaptic Mimics journal March 2018
Quantum algorithms: an overview journal January 2016
Quantum-assisted Helmholtz machines: A quantum–classical deep learning framework for industrial datasets in near-term devices journal May 2018
Movement prediction from real-world images using a liquid state machine journal November 2006
Programming Spiking Neural Networks on Intel’s Loihi journal March 2018
Sparse Neural Networks With Large Learning Diversity journal July 2011
Triangle Finding and Listing in CONGEST Networks
  • Izumi, Taisuke; Le Gall, François
  • PODC '17: ACM Symposium on Principles of Distributed Computing, Proceedings of the ACM Symposium on Principles of Distributed Computing https://doi.org/10.1145/3087801.3087811
conference July 2017
Loihi: A Neuromorphic Manycore Processor with On-Chip Learning journal January 2018
Training and operation of an integrated neuromorphic network based on metal-oxide memristors journal May 2015
Thermally assisted quantum annealing of a 16-qubit problem journal May 2013
Quantum Computing Circuits and Devices journal June 2019
Quantum speedup of the traveling-salesman problem for bounded-degree graphs journal March 2017
The theory of variational hybrid quantum-classical algorithms journal February 2016
Quantum Optimization of Fully Connected Spin Glasses journal September 2015
A language and hardware independent approach to quantum–classical computing journal January 2018
Quantum Annealing for Constrained Optimization journal March 2016
Polynomial-Time Algorithms for Prime Factorization and Discrete Logarithms on a Quantum Computer journal January 1999
Quantum Support Vector Machine for Big Data Classification journal September 2014
Neurons with graded response have collective computational properties like those of two-state neurons. journal May 1984
Statistical mechanics of community detection journal July 2006
Dynamic evolving spiking neural networks for on-line spatio- and spectro-temporal pattern recognition journal May 2013
Rapid, parallel path planning by propagating wavefronts of spiking neural activity journal January 2013
Hierarchical quantum classifiers journal December 2018
Coherent Ising machine based on degenerate optical parametric oscillators journal December 2013
Training deep neural networks for binary communication with the Whetstone method journal January 2019
Physics-Inspired Optimization for Quadratic Unconstrained Problems Using a Digital Annealer journal April 2019
Experimental demonstration of associative memory with memristive neural networks journal September 2010
Fast clique minor generation in Chimera qubit connectivity graphs journal October 2015
Gaussian optical Ising machines journal October 2017
Graph Partitioning using Quantum Annealing on the D-Wave System
  • Ushijima-Mwesigwa, Hayato; Negre, Christian F. A.; Mniszewski, Susan M.
  • Proceedings of the Second International Workshop on Post Moores Era Supercomputing - PMES'17 https://doi.org/10.1145/3149526.3149531
conference January 2017
Gradient-based learning applied to document recognition journal January 1998
A variational eigenvalue solver on a photonic quantum processor journal July 2014
Max 2-SAT with up to 108 qubits journal April 2014
Bidirectional associative memories journal January 1988
Quantum sensing journal July 2017
Gibbs sampling with low-power spiking digital neurons conference May 2015
Building block of a programmable neuromorphic substrate: A digital neurosynaptic core
  • Arthur, John V.; Merolla, Paul A.; Akopyan, Filipp
  • 2012 International Joint Conference on Neural Networks (IJCNN 2012 - Brisbane), The 2012 International Joint Conference on Neural Networks (IJCNN) https://doi.org/10.1109/IJCNN.2012.6252637
conference June 2012
Training Products of Experts by Minimizing Contrastive Divergence journal August 2002
Event-Driven Random Back-Propagation: Enabling Neuromorphic Deep Learning Machines journal June 2017
Storing Sparse Messages in Networks of Neural Cliques journal May 2014
Superconducting Optoelectronic Circuits for Neuromorphic Computing journal March 2017
Readiness of Quantum Optimization Machines for Industrial Applications journal July 2019
Solving Constraint Satisfaction Problems with Networks of Spiking Neurons journal March 2016
Generative model benchmarks for superconducting qubits journal June 2019
Quantum circuit learning journal September 2018
Efficient associative memory using small-world architecture journal June 2001
Demonstration of a small programmable quantum computer with atomic qubits journal August 2016
GPU clusters for high-performance computing conference August 2009
A case study in programming a quantum annealer for hard operational planning problems journal December 2014
Quantum reservoir processing journal April 2019
An introduction to quantum machine learning journal October 2014
Enabling and scaling matrix computations on heterogeneous multi-core and multi-GPU systems conference January 2012
The capacity of the Hopfield associative memory journal July 1987
An initialization strategy for addressing barren plateaus in parametrized quantum circuits journal December 2019
Superparamagnetic Clustering of Data journal April 1996
Computational Principle and Performance Evaluation of Coherent Ising Machine Based on Degenerate Optical Parametric Oscillator Network journal April 2016
Minor-embedding in adiabatic quantum computation: II. Minor-universal graph design journal October 2010
A million spiking-neuron integrated circuit with a scalable communication network and interface journal August 2014
Associative memory in networks of spiking neurons journal July 2001
Performance Models for Split-Execution Computing Systems conference May 2016
Identifying the minor set cover of dense connected bipartite graphs via random matching edge sets journal February 2017
Quantum correlation in degenerate optical parametric oscillators with mutual injections journal October 2015
Systematic and deterministic graph minor embedding for Cartesian products of graphs journal April 2017
Scalable Quantum Simulation of Molecular Energies journal July 2016
Approximation by superpositions of a sigmoidal function journal December 1989
Using Stochastic Spiking Neural Networks on SpiNNaker to Solve Constraint Satisfaction Problems journal December 2017
The Super-Turing Computational Power of Plastic Recurrent Neural Networks journal November 2014
Quantum model for coherent Ising machines: Stochastic differential equations with replicator dynamics journal November 2017
Supervised learning with quantum-enhanced feature spaces journal March 2019
Harnessing Disordered-Ensemble Quantum Dynamics for Machine Learning journal August 2017
Minor-embedding in adiabatic quantum computation: I. The parameter setting problem journal September 2008
Delving Deep into Rectifiers: Surpassing Human-Level Performance on ImageNet Classification conference December 2015
Deep learning in neural networks: An overview journal January 2015
Associative Dynamics in a Chaotic Neural Network journal January 1997
Fast Quantum Algorithm for Numerical Gradient Estimation journal July 2005
The use of Multiple Measurements in Taxonomic Problems journal September 1936
A generative modeling approach for benchmarking and training shallow quantum circuits journal May 2019
Practical Quantum Error Mitigation for Near-Future Applications journal July 2018