A Study of Complex Deep Learning Networks on High Performance, Neuromorphic, and Quantum Computers

Conference · Fri Jan 01 04:00:00 EST 2016

OSTI ID:1335350

Potok, Thomas E ^[1]; Schuman, Catherine D ^[1]; Young, Steven R ^[1]; Patton, Robert M ^[1]; Spedalieri, Federico ^[2]; Liu, Jeremy ^[2]; Yao, Ke-Thia ^[2]; Rose, Garrett ^[3]; Chakma, Gangotree ^[3]

---

1. ORNL
2. University of Southern California, Information Sciences Institute
3. University of Tennessee (UT)

Current Deep Learning models use highly optimized convolutional neural networks (CNN) trained on large graphical processing units (GPU)-based computers with a fairly simple layered network topology, i.e., highly connected layers, without intra-layer connections. Complex topologies have been proposed, but are intractable to train on current systems. Building the topologies of the deep learning network requires hand tuning, and implementing the network in hardware is expensive in both cost and power. In this paper, we evaluate deep learning models using three different computing architectures to address these problems: quantum computing to train complex topologies, high performance computing (HPC) to automatically determine network topology, and neuromorphic computing for a low-power hardware implementation. Due to input size limitations of current quantum computers we use the MNIST dataset for our evaluation. The results show the possibility of using the three architectures in tandem to explore complex deep learning networks that are untrainable using a von Neumann architecture. We show that a quantum computer can find high quality values of intra-layer connections and weights, while yielding a tractable time result as the complexity of the network increases; a high performance computer can find optimal layer-based topologies; and a neuromorphic computer can represent the complex topology and weights derived from the other architectures in low power memristive hardware. This represents a new capability that is not feasible with current von Neumann architecture. It potentially enables the ability to solve very complicated problems unsolvable with current computing technologies.

• Research Organization: Oak Ridge National Laboratory (ORNL), Oak Ridge, TN (United States). Oak Ridge Leadership Computing Facility (OLCF)
• Sponsoring Organization: ORNL Program Development; USDOE Office of Science (SC)
• DOE Contract Number: AC05-00OR22725
• OSTI ID: 1335350
• Country of Publication: United States
• Language: English