skip to main content
OSTI.GOV title logo U.S. Department of Energy
Office of Scientific and Technical Information

Title: Scientific User Behavior and Data-Sharing Trends in a Petascale File System

Abstract

The Oak Ridge Leadership Computing Facility (OLCF) runs the No. 4 supercomputer in the world, supported by a petascale file system, to facilitate scientific discovery. In this paper, using the daily file system metadata snapshots collected over 500 days, we have studied the behavioral trends of 1, 362 active users and 380 projects across 35 science domains. In particular, we have analyzed both individual and collective behavior of users and projects, highlighting needs from individual communities and the overall requirements to operate the file system. We have analyzed the metadata across three dimensions, namely (i) the projects' file generation and usage trends, using quantitative file system-centric metrics, (ii) scientific user behavior on the file system, and (iii) the data sharing trends of users and projects. To the best of our knowledge, our work is the first of its kind to provide comprehensive insights on user behavior from multiple science domains through metadata analysis of a large-scale shared file system. We envision that this OLCF case study will provide valuable insights for the design, operation, and management of storage systems at scale, and also encourage other HPC centers to undertake similar such efforts.

Authors:
ORCiD logo [1]; ORCiD logo [1]; ORCiD logo [1]; ORCiD logo [1]
  1. ORNL
Publication Date:
Research Org.:
Oak Ridge National Laboratory (ORNL), Oak Ridge, TN (United States)
Sponsoring Org.:
USDOE
OSTI Identifier:
1427663
DOE Contract Number:  
AC05-00OR22725
Resource Type:
Conference
Resource Relation:
Conference: SC17: International Conference for High Performance Computing, Networking, Storage and Analysis - Denver, Colorado, United States of America - 11/12/2017 5:00:00 AM-
Country of Publication:
United States
Language:
English

Citation Formats

Lim, Seung-Hwan, Sim, Hyogi, Vazhkudai, Sudharshan, and Gunasekaran, Raghul. Scientific User Behavior and Data-Sharing Trends in a Petascale File System. United States: N. p., 2017. Web. doi:10.1145/3126908.3126924.
Lim, Seung-Hwan, Sim, Hyogi, Vazhkudai, Sudharshan, & Gunasekaran, Raghul. Scientific User Behavior and Data-Sharing Trends in a Petascale File System. United States. https://doi.org/10.1145/3126908.3126924
Lim, Seung-Hwan, Sim, Hyogi, Vazhkudai, Sudharshan, and Gunasekaran, Raghul. 2017. "Scientific User Behavior and Data-Sharing Trends in a Petascale File System". United States. https://doi.org/10.1145/3126908.3126924. https://www.osti.gov/servlets/purl/1427663.
@article{osti_1427663,
title = {Scientific User Behavior and Data-Sharing Trends in a Petascale File System},
author = {Lim, Seung-Hwan and Sim, Hyogi and Vazhkudai, Sudharshan and Gunasekaran, Raghul},
abstractNote = {The Oak Ridge Leadership Computing Facility (OLCF) runs the No. 4 supercomputer in the world, supported by a petascale file system, to facilitate scientific discovery. In this paper, using the daily file system metadata snapshots collected over 500 days, we have studied the behavioral trends of 1, 362 active users and 380 projects across 35 science domains. In particular, we have analyzed both individual and collective behavior of users and projects, highlighting needs from individual communities and the overall requirements to operate the file system. We have analyzed the metadata across three dimensions, namely (i) the projects' file generation and usage trends, using quantitative file system-centric metrics, (ii) scientific user behavior on the file system, and (iii) the data sharing trends of users and projects. To the best of our knowledge, our work is the first of its kind to provide comprehensive insights on user behavior from multiple science domains through metadata analysis of a large-scale shared file system. We envision that this OLCF case study will provide valuable insights for the design, operation, and management of storage systems at scale, and also encourage other HPC centers to undertake similar such efforts.},
doi = {10.1145/3126908.3126924},
url = {https://www.osti.gov/biblio/1427663}, journal = {},
number = ,
volume = ,
place = {United States},
year = {Wed Nov 01 00:00:00 EDT 2017},
month = {Wed Nov 01 00:00:00 EDT 2017}
}

Conference:
Other availability
Please see Document Availability for additional information on obtaining the full-text document. Library patrons may search WorldCat to identify libraries that hold this conference proceeding.

Save / Share: