skip to main content
OSTI.GOV title logo U.S. Department of Energy
Office of Scientific and Technical Information

Title: Coping with silent and fail-stop errors at scale by combining replication and checkpointing

Abstract

This paper provides a model and an analytical study of replication as a technique to cope with silent errors, as well as a mixture of both silent and fail-stop errors on large-scale platforms. Compared with fail-stop errors that are immediately detected when they occur, silent errors require a detection mechanism. To detect silent errors, many application-specific techniques are available, either based on algorithms (e.g., ABFT), invariant preservation or data analytics, but replication remains the most transparent and least intrusive technique. We explore the right level (duplication, triplication or more) of replication for two frameworks: (i) when the platform is subject to only silent errors, and (ii) when the platform is subject to both silent and fail-stop errors. A higher level of replication is more expensive in terms of resource usage but enables to tolerate more errors and to even correct some errors, hence there is a trade-off to be found. Replication is combined with checkpointing and comes with two flavors: process replication and group replication. Process replication applies to message-passing applications with communicating processes. Each process is replicated, and the platform is composed of process pairs, or triplets. Group replication applies to black-box applications, whose parallel execution is replicated severalmore » times. The platform is partitioned into two halves (or three thirds). In both scenarios, results are compared before each checkpoint, which is taken only when both results (duplication) or two out of three results (triplication) coincide. Otherwise, one or more silent errors have been detected, and the application rolls back to the last checkpoint, as well as when fail-stop errors have struck. We provide a detailed analytical study for all of these scenarios, with formulas to decide, for each scenario, the optimal parameters as a function of the error rate, checkpoint cost, and platform size. We also report a set of extensive simulation results that nicely corroborates the analytical model.« less

Authors:
; ; ; ; ;
Publication Date:
Research Org.:
Argonne National Lab. (ANL), Argonne, IL (United States)
Sponsoring Org.:
National Science Foundation (NSF)
OSTI Identifier:
1475194
DOE Contract Number:  
AC02-06CH11357
Resource Type:
Journal Article
Resource Relation:
Journal Name: Journal of Parallel and Distributed Computing; Journal Volume: 122; Journal Issue: C
Country of Publication:
United States
Language:
English
Subject:
Checkpointing; Fail-stop errors; Fault tolerance; High-performance computing; Replication; Silent errors

Citation Formats

Benoit, Anne, Cavelan, Aurélien, Cappello, Franck, Raghavan, Padma, Robert, Yves, and Sun, Hongyang. Coping with silent and fail-stop errors at scale by combining replication and checkpointing. United States: N. p., 2018. Web. doi:10.1016/j.jpdc.2018.08.002.
Benoit, Anne, Cavelan, Aurélien, Cappello, Franck, Raghavan, Padma, Robert, Yves, & Sun, Hongyang. Coping with silent and fail-stop errors at scale by combining replication and checkpointing. United States. doi:10.1016/j.jpdc.2018.08.002.
Benoit, Anne, Cavelan, Aurélien, Cappello, Franck, Raghavan, Padma, Robert, Yves, and Sun, Hongyang. Sat . "Coping with silent and fail-stop errors at scale by combining replication and checkpointing". United States. doi:10.1016/j.jpdc.2018.08.002.
@article{osti_1475194,
title = {Coping with silent and fail-stop errors at scale by combining replication and checkpointing},
author = {Benoit, Anne and Cavelan, Aurélien and Cappello, Franck and Raghavan, Padma and Robert, Yves and Sun, Hongyang},
abstractNote = {This paper provides a model and an analytical study of replication as a technique to cope with silent errors, as well as a mixture of both silent and fail-stop errors on large-scale platforms. Compared with fail-stop errors that are immediately detected when they occur, silent errors require a detection mechanism. To detect silent errors, many application-specific techniques are available, either based on algorithms (e.g., ABFT), invariant preservation or data analytics, but replication remains the most transparent and least intrusive technique. We explore the right level (duplication, triplication or more) of replication for two frameworks: (i) when the platform is subject to only silent errors, and (ii) when the platform is subject to both silent and fail-stop errors. A higher level of replication is more expensive in terms of resource usage but enables to tolerate more errors and to even correct some errors, hence there is a trade-off to be found. Replication is combined with checkpointing and comes with two flavors: process replication and group replication. Process replication applies to message-passing applications with communicating processes. Each process is replicated, and the platform is composed of process pairs, or triplets. Group replication applies to black-box applications, whose parallel execution is replicated several times. The platform is partitioned into two halves (or three thirds). In both scenarios, results are compared before each checkpoint, which is taken only when both results (duplication) or two out of three results (triplication) coincide. Otherwise, one or more silent errors have been detected, and the application rolls back to the last checkpoint, as well as when fail-stop errors have struck. We provide a detailed analytical study for all of these scenarios, with formulas to decide, for each scenario, the optimal parameters as a function of the error rate, checkpoint cost, and platform size. We also report a set of extensive simulation results that nicely corroborates the analytical model.},
doi = {10.1016/j.jpdc.2018.08.002},
journal = {Journal of Parallel and Distributed Computing},
number = C,
volume = 122,
place = {United States},
year = {Sat Dec 01 00:00:00 EST 2018},
month = {Sat Dec 01 00:00:00 EST 2018}
}