skip to main content


This content will become publicly available on October 26, 2018

Title: Scalable Failure Masking for Stencil Computations using Ghost Region Expansion and Cell to Rank Remapping

In order to achieve exascale systems, application resilience needs to be addressed. Some programming models, such as task-DAG (directed acyclic graphs) architectures, currently embed resilience features whereas traditional SPMD (single program, multiple data) and message-passing models do not. Since a large part of the community's code base follows the latter models, it is still required to take advantage of application characteristics to minimize the overheads of fault tolerance. To that end, this paper explores how recovering from hard process/node failures in a local manner is a natural approach for certain applications to obtain resilience at lower costs in faulty environments. In particular, this paper targets enabling online, semitransparent local recovery for stencil computations on current leadership-class systems as well as presents programming support and scalable runtime mechanisms. Also described and demonstrated in this paper is the effect of failure masking, which allows the effective reduction of impact on total time to solution due to multiple failures. Furthermore, we discuss, implement, and evaluate ghost region expansion and cell-to-rank remapping to increase the probability of failure masking. To conclude, this paper shows the integration of all aforementioned mechanisms with the S3D combustion simulation through an experimental demonstration (using the Titan system) ofmore » the ability to tolerate high failure rates (i.e., node failures every five seconds) with low overhead while sustaining performance at large scales. In addition, this demonstration also displays the failure masking probability increase resulting from the combination of both ghost region expansion and cell-to-rank remapping.« less
 [1] ;  [2] ;  [2] ;  [2] ;  [3] ;  [2] ;  [1]
  1. Rutgers Univ., Piscataway, NJ (United States). Rutgers Discovery Informatics Inst.
  2. Sandia National Lab. (SNL-CA), Livermore, CA (United States)
  3. Sandia National Lab. (SNL-NM), Albuquerque, NM (United States)
Publication Date:
Report Number(s):
Journal ID: ISSN 1064-8275; 651103; TRN: US1800238
Grant/Contract Number:
Accepted Manuscript
Journal Name:
SIAM Journal on Scientific Computing
Additional Journal Information:
Journal Volume: 39; Journal Issue: 5; Journal ID: ISSN 1064-8275
Research Org:
Sandia National Lab. (SNL-NM), Albuquerque, NM (United States)
Sponsoring Org:
USDOE National Nuclear Security Administration (NNSA)
Country of Publication:
United States
OSTI Identifier: