Skip to main content
OSTI.GOVU.S. Department of Energy Office of Scientific and Technical Information
U.S. Department of Energy
Office of Scientific and Technical Information
 

Runtime extension for neural network training with heterogeneous memory

Patent ·
OSTI ID:2293673
; ; ; ;

Systems, apparatuses, and methods for managing buffers in a neural network implementation with heterogeneous memory are disclosed. A system includes a neural network coupled to a first memory and a second memory. The first memory is a relatively low-capacity, high-bandwidth memory while the second memory is a relatively high-capacity, low-bandwidth memory. During a forward propagation pass of the neural network, a run-time manager monitors the usage of the buffers for the various layers of the neural network. During a backward propagation pass of the neural network, the run-time manager determines how to move the buffers between the first and second memories based on the monitored buffer usage during the forward propagation pass. As a result, the run-time manager is able to reduce memory access latency for the layers of the neural network during the backward propagation pass.

Research Organization:
Lawrence Livermore National Laboratory (LLNL), Livermore, CA (United States); Advanced Micro Devices, Inc., Santa Clara, CA (United States)
Sponsoring Organization:
USDOE
DOE Contract Number:
AC52-07NA27344
Assignee:
Advanced Micro Devices, Inc. (Santa Clara, CA)
Patent Number(s):
11,775,799
Application Number:
16/194,958
OSTI ID:
2293673
Country of Publication:
United States
Language:
English

References (9)

Low Overhead Message Passing for High Performance Many-Core Processors conference December 2013
Profile-guided proactive garbage collection for locality optimization journal June 2006
EIE: Efficient Inference Engine on Compressed Deep Neural Network conference June 2016
Efficient FPGA Acceleration of Convolutional Neural Networks Using Logical-3D Compute Array conference January 2016
F-C3D: FPGA-based 3-dimensional convolutional neural network conference September 2017
Optimal Tiling Strategy for Memory Bandwidth Reduction for CNNs book January 2017
Compressing DMA Engine: Leveraging Activation Sparsity for Training Deep Neural Networks conference February 2018
vDNN: Virtualized deep neural networks for scalable, memory-efficient neural network design conference October 2016
moDNN: Memory optimal DNN training on GPUs conference March 2018

Similar Records

Hybrid memory module bridge network and buffers
Patent · Tue Oct 09 00:00:00 EDT 2018 · OSTI ID:1485288
Mechanism for reducing page migration overhead in memory systems
Patent · Tue Jul 02 00:00:00 EDT 2019 · OSTI ID:1568554
Sparse Matrix-Matrix Multiplication on Multilevel Memory Architectures: Algorithms and Experiments
Technical Report · Mon Apr 02 00:00:00 EDT 2018 · OSTI ID:1435688

Related Subjects