VerifyIO: Ensuring Correctness of Consistency Semantics in Parallel I/O

Conference · Wed Oct 09 04:00:00 EDT 2024

OSTI ID:2583382

Wang, C. ^[1]; Zhu, Z. ^[2]; Mohror, K. ^[1]; Neuwirth, S. ^[2]; Snir, M. ^[3]

---

1. Lawrence Livermore National Laboratory (LLNL), Livermore, CA (United States)
2. Johannes Gutenberg Univ., Mainz (Germany)
3. Univ. of Illinois at Urbana-Champaign, IL (United States). Beckman Inst. for Advanced Science and Technology

Abstract—High-performance computing (HPC) applications generate and consume substantial amounts of data, typically managed by parallel file systems. These applications access file systems either through the POSIX interface or by using highlevel I/O libraries. While the POSIX consistency model remains dominant in HPC, emerging file systems and popular I/O libraries increasingly adopt alternative consistency models that relax semantics in various ways, creating significant challenges for correctness and portability. This paper addresses these challenges by proposing a trace-driven I/O consistency verification workflow, implemented in our open-source tool, VerifyIO, which collects execution traces, detects data conflicts, and verifies proper synchronization against specified consistency models. Our extensive evaluation of 91 test case executions across three widely used I/O libraries with four I/O consistency models reveals critical consistency issues at both application and implementation levels.

• Research Organization: Lawrence Livermore National Laboratory (LLNL), Livermore, CA (United States)
• Sponsoring Organization: USDOE National Nuclear Security Administration (NNSA)
• DOE Contract Number: AC52-07NA27344
• OSTI ID: 2583382
• Report Number(s): LLNL-CONF-870387
• Country of Publication: United States
• Language: English

Related Subjects

Consistency Semantics
I/O Consistency
MPIIO
POSIX Semantics
Parallel File System