Efficiently modeling neural networks on massively parallel computers
Neural networks are a very useful tool for analyzing and modeling complex real world systems. Applying neural network simulations to real world problems generally involves large amounts of data and massive amounts of computation. To efficiently handle the computational requirements of large problems, we have implemented at Los Alamos a highly efficient neural network compiler for serial computers, vector computers, vector parallel computers, and fine grain SIMD computers such as the CM-2 connection machine. This paper will describe the mapping used by the compiler to implement feed-forward backpropagation neural networks for a SIMD architecture parallel computer. Thinking Machines Corporation has benchmarked our code at 1.3 billion interconnects per second (approximately 3 gigaflops) on a 64,000 processor CM-2 connection machine (Singer 1990). This mapping is applicable to other SMM computers and can be implemented on computers such as the CM-5 connection machine. Our mapping has virtually no communications overhead with the exception of the communications required for a global summation across the processors. We can efficiently model very large neural networks which have many neurons and interconnects and our mapping can be extend to arbitrarily large networks by merging the memory space of separate processors with fast adjacent processor inter-processor communications. This paper will consider the simulation of only feed forward neural network although this method is extendible to recurrent networks.
- Research Organization:
- Los Alamos National Lab., NM (United States)
- Sponsoring Organization:
- USDOE; DOHHS; USDOE, Washington, DC (United States); Department of Health and Human Services, Washington, DC (United States)
- DOE Contract Number:
- W-7405-ENG-36
- OSTI ID:
- 6964551
- Report Number(s):
- LA-UR-92-3568; CONF-9206318-1; ON: DE93003787
- Resource Relation:
- Conference: NASA conference, Houston, TX (United States), 1-3 Jun 1992
- Country of Publication:
- United States
- Language:
- English
Similar Records
Benchmarking the Connection Machine. Final report
SPECT reconstruction using a backpropagation neural network implemented on a massively parallel SIMD computer