skip to main content

SciTech ConnectSciTech Connect

Title: An Approach to Enhance pnetCDF Performance in Environmental Modeling Applications

I/O has been considered as a bottleneck in parallel applications. The software package, pnetCDF which works with parallel file systems, was developed to address this issue and provide parallel I/O capability. This study examines the performance of a novel approach which performs data aggregation along either row or column dimension of the spatial domain, and then applies the pnetCDF parallel I/O paradigm. The test was done with three different domain sizes which represents small, moderately large and large data domains, using a small scale Community Multi-scale Air Quality model (CMAQ) mocked up code. The examination includes comparing I/O performance with traditional serial I/O technique, straight application of pnetCDF, and the data aggregation along row and column dimension before applying pnetCDF. After the comparison, “optimal” I/O configurations for this new novel approach were quantified. Data aggregation along the row dimension (pnetCDFcr) works better than along the column dimension (pnetCDFcc) although it may perform slightly worse than straight the pnetCDF method with a small number of processors. When the number of processors becomes larger, pnetCDFcr out performs pnetCDF significantly. If the number of processors keeps increasing, pnetCDF reaches a point that the performance is even worse than the serial I/O technique. Thismore » new approach has also been tested on a real application where it performs two times better than the straight pnetCDF paradigm.« less
; ; ; ;
Publication Date:
OSTI Identifier:
Report Number(s):
DOE Contract Number:
Resource Type:
Journal Article
Resource Relation:
Journal Name: Geoscientific Model Development, 8(4):1033-1046
Research Org:
Pacific Northwest National Laboratory (PNNL), Richland, WA (US)
Sponsoring Org:
Country of Publication:
United States
pnetCDF; scientific applications; I/O