skip to main content

SciTech ConnectSciTech Connect

Title: SSD-Optimized Workload Placement with Adaptive Learning and Classification in HPC Environments

In recent years, non-volatile memory devices such as SSD drives have emerged as a viable storage solution due to their increasing capacity and decreasing cost. Due to the unique capability and capacity requirements in large scale HPC (High Performance Computing) storage environment, a hybrid config- uration (SSD and HDD) may represent one of the most available and balanced solutions considering the cost and performance. Under this setting, effective data placement as well as movement with controlled overhead become a pressing challenge. In this paper, we propose an integrated object placement and movement framework and adaptive learning algorithms to address these issues. Specifically, we present a method that shuffle data objects across storage tiers to optimize the data access performance. The method also integrates an adaptive learning algorithm where real- time classification is employed to predict the popularity of data object accesses, so that they can be placed on, or migrate between SSD or HDD drives in the most efficient manner. We discuss preliminary results based on this approach using a simulator we developed to show that the proposed methods can dynamically adapt storage placements and access pattern as workloads evolve to achieve the best system level performance such as throughput.
 [1] ;  [2] ;  [3] ;  [1] ;  [1] ;  [1]
  1. ORNL
  2. The University of Tennessee
  3. University of Tennessee, Knoxville (UTK)
Publication Date:
OSTI Identifier:
DOE Contract Number:
Resource Type:
Resource Relation:
Conference: 30th International Conference on Massive Storage Systems and Technology (MSST 2014), Santa Clara, California, USA, CA, USA, 20140602, 20140606
Research Org:
Oak Ridge National Laboratory (ORNL); Oak Ridge Leadership Computing Facility (OLCF)
Sponsoring Org:
ORNL LDRD Director's R&D
Country of Publication:
United States