Geomancy: Automated Performance Enhancement through Data Layout Optimization

Conference · Thu Oct 29 04:00:00 EDT 2020

OSTI ID:1657070

Bel, Oceane M. ^[1]; Chang, Kenneth ^[2]; Tallent, Nathan R. ^[1]; Duellmann, Dirk ^[3]; Miller, Ethan L. ^[2]; Faisal Nawab, Faisal Nawab ^[2]; Long, Darrell ^[2]

---

1. BATTELLE (PACIFIC NW LAB)
2. University of California, Santa Cruz
3. CERN

Large distributed storage systems such as high- performance computing (HPC) systems used by national or international laboratories require sufficient performance and scale for demanding scientific workloads and must handle shifting workloads with ease. Ideally, data is placed in locations to optimize performance, but the size and complexity of large storage systems inhibit rapid effective restructuring of data layouts to maintain performance as workloads shift. To address these issues, we have developed Geomancy, a tool that models the placement of data within a distributed storage system and reacts to drops in performance. Using a combination of machine learning techniques suitable for temporal modeling, Geomancy determines when and where a bottleneck may happen due to changing workloads and suggests changes in the layout that mitigate or prevent them. Our approach to optimizing throughput offers benefits for storage systems such as avoiding potential bottlenecks and increasing overall I/O throughput from 11% to 30%.

• Research Organization: Pacific Northwest National Laboratory (PNNL), Richland, WA (United States)
• Sponsoring Organization: USDOE
• DOE Contract Number: AC05-76RL01830
• OSTI ID: 1657070
• Report Number(s): PNNL-SA-155741
• Country of Publication: United States
• Language: English