skip to main content
OSTI.GOV title logo U.S. Department of Energy
Office of Scientific and Technical Information

Title: Proactive Process-Level Live Migration and Back Migration in HPC Environments

Abstract

As the number of nodes in high-performance computing environments keeps increasing, faults are becoming common place. Reactive fault tolerance (FT) often does not scale due to massive I/O requirements and relies on manual job resubmission. This work complements reactive with proactive FT at the process level. Through health monitoring, a subset of node failures can be anticipated when one's health deteriorates. A novel process-level live migration mechanism supports continued execution of applications during much of process migration. This scheme is integrated into an MPI execution environment to transparently sustain health-inflicted node failures, which eradicates the need to restart and requeue MPI jobs. Experiments indicate that 1-6.5 s of prior warning are required to successfully trigger live process migration while similar operating system virtualization mechanisms require 13-24 s. This self-healing approach complements reactive FT by nearly cutting the number of checkpoints in half when 70% of the faults are handled proactively. The work also provides a novel back migration approach to eliminate load imbalance or bottlenecks caused by migrated tasks. Experiments indicate the larger the amount of outstanding execution, the higher the benefit due to back migration.

Authors:
 [1];  [2];  [1];  [1]
  1. ORNL
  2. North Carolina State University
Publication Date:
Research Org.:
Oak Ridge National Lab. (ORNL), Oak Ridge, TN (United States)
Sponsoring Org.:
USDOE Office of Science (SC)
OSTI Identifier:
1037151
DOE Contract Number:  
DE-AC05-00OR22725
Resource Type:
Journal Article
Journal Name:
Journal of Parallel and Distributed Computing
Additional Journal Information:
Journal Volume: 72; Journal Issue: 2; Journal ID: ISSN 0743-7315
Country of Publication:
United States
Language:
English
Subject:
99 GENERAL AND MISCELLANEOUS//MATHEMATICS, COMPUTING, AND INFORMATION SCIENCE; SUPERCOMPUTERS; FAULT TOLERANT COMPUTERS; INPUT-OUTPUT ANALYSIS

Citation Formats

Wang, Chao, Mueller, Frank, Engelmann, Christian, and Scott, Stephen L. Proactive Process-Level Live Migration and Back Migration in HPC Environments. United States: N. p., 2012. Web. doi:10.1016/j.jpdc.2011.10.009.
Wang, Chao, Mueller, Frank, Engelmann, Christian, & Scott, Stephen L. Proactive Process-Level Live Migration and Back Migration in HPC Environments. United States. doi:10.1016/j.jpdc.2011.10.009.
Wang, Chao, Mueller, Frank, Engelmann, Christian, and Scott, Stephen L. Sun . "Proactive Process-Level Live Migration and Back Migration in HPC Environments". United States. doi:10.1016/j.jpdc.2011.10.009.
@article{osti_1037151,
title = {Proactive Process-Level Live Migration and Back Migration in HPC Environments},
author = {Wang, Chao and Mueller, Frank and Engelmann, Christian and Scott, Stephen L},
abstractNote = {As the number of nodes in high-performance computing environments keeps increasing, faults are becoming common place. Reactive fault tolerance (FT) often does not scale due to massive I/O requirements and relies on manual job resubmission. This work complements reactive with proactive FT at the process level. Through health monitoring, a subset of node failures can be anticipated when one's health deteriorates. A novel process-level live migration mechanism supports continued execution of applications during much of process migration. This scheme is integrated into an MPI execution environment to transparently sustain health-inflicted node failures, which eradicates the need to restart and requeue MPI jobs. Experiments indicate that 1-6.5 s of prior warning are required to successfully trigger live process migration while similar operating system virtualization mechanisms require 13-24 s. This self-healing approach complements reactive FT by nearly cutting the number of checkpoints in half when 70% of the faults are handled proactively. The work also provides a novel back migration approach to eliminate load imbalance or bottlenecks caused by migrated tasks. Experiments indicate the larger the amount of outstanding execution, the higher the benefit due to back migration.},
doi = {10.1016/j.jpdc.2011.10.009},
journal = {Journal of Parallel and Distributed Computing},
issn = {0743-7315},
number = 2,
volume = 72,
place = {United States},
year = {2012},
month = {1}
}