skip to main content

DOE PAGESDOE PAGES

Title: The MPO system for automatic workflow documentation

Data from large-scale experiments and extreme-scale computing is expensive to produce and may be used for critical applications. However, it is not the mere existence of data that is important, but our ability to make use of it. Experience has shown that when metadata is better organized and more complete, the underlying data becomes more useful. Traditionally, capturing the steps of scientific workflows and metadata was the role of the lab notebook, but the digital era has resulted instead in the fragmentation of data, processing, and annotation. Here, this article presents the Metadata, Provenance, and Ontology (MPO) System, the software that can automate the documentation of scientific workflows and associated information. Based on recorded metadata, it provides explicit information about the relationships among the elements of workflows in notebook form augmented with directed acyclic graphs. A set of web-based graphical navigation tools and Application Programming Interface (API) have been created for searching and browsing, as well as programmatically accessing the workflows and data. We describe the MPO concepts and its software architecture. We also report the current status of the software as well as the initial deployment experience.
Authors:
 [1] ;  [1] ;  [1] ;  [2] ;  [1] ;  [3] ;  [1] ;  [3] ;  [2] ;  [2] ;  [3]
  1. General Atomics, San Diego, CA (United States)
  2. Massachusetts Inst. of Technology (MIT), Cambridge, MA (United States)
  3. Lawrence Berkeley National Lab. (LBNL), Berkeley, CA (United States)
Publication Date:
Grant/Contract Number:
FC02-04ER54698; SC0008697; AC02-05CH11231; SC0008736
Type:
Accepted Manuscript
Journal Name:
Fusion Engineering and Design
Additional Journal Information:
Journal Volume: 112; Journal Issue: C; Journal ID: ISSN 0920-3796
Publisher:
Elsevier
Research Org:
General Atomics, San Diego, CA (United States)
Sponsoring Org:
USDOE Office of Science (SC), Advanced Scientific Computing Research (ASCR) (SC-21); USDOE Office of Science (SC), Fusion Energy Sciences (FES) (SC-24)
Country of Publication:
United States
Language:
English
Subject:
70 PLASMA PHYSICS AND FUSION TECHNOLOGY; Workflow; Provenance; Metadata; Ontology; DIII-D; EFIT
OSTI Identifier:
1371571
Alternate Identifier(s):
OSTI ID: 1399140

Abla, G., Coviello, E. N., Flanagan, S. M., Greenwald, M., Lee, X., Romosan, A., Schissel, D. P., Shoshani, A., Stillerman, J., Wright, J., and Wu, K. J.. The MPO system for automatic workflow documentation. United States: N. p., Web. doi:10.1016/j.fusengdes.2016.04.023.
Abla, G., Coviello, E. N., Flanagan, S. M., Greenwald, M., Lee, X., Romosan, A., Schissel, D. P., Shoshani, A., Stillerman, J., Wright, J., & Wu, K. J.. The MPO system for automatic workflow documentation. United States. doi:10.1016/j.fusengdes.2016.04.023.
Abla, G., Coviello, E. N., Flanagan, S. M., Greenwald, M., Lee, X., Romosan, A., Schissel, D. P., Shoshani, A., Stillerman, J., Wright, J., and Wu, K. J.. 2016. "The MPO system for automatic workflow documentation". United States. doi:10.1016/j.fusengdes.2016.04.023. https://www.osti.gov/servlets/purl/1371571.
@article{osti_1371571,
title = {The MPO system for automatic workflow documentation},
author = {Abla, G. and Coviello, E. N. and Flanagan, S. M. and Greenwald, M. and Lee, X. and Romosan, A. and Schissel, D. P. and Shoshani, A. and Stillerman, J. and Wright, J. and Wu, K. J.},
abstractNote = {Data from large-scale experiments and extreme-scale computing is expensive to produce and may be used for critical applications. However, it is not the mere existence of data that is important, but our ability to make use of it. Experience has shown that when metadata is better organized and more complete, the underlying data becomes more useful. Traditionally, capturing the steps of scientific workflows and metadata was the role of the lab notebook, but the digital era has resulted instead in the fragmentation of data, processing, and annotation. Here, this article presents the Metadata, Provenance, and Ontology (MPO) System, the software that can automate the documentation of scientific workflows and associated information. Based on recorded metadata, it provides explicit information about the relationships among the elements of workflows in notebook form augmented with directed acyclic graphs. A set of web-based graphical navigation tools and Application Programming Interface (API) have been created for searching and browsing, as well as programmatically accessing the workflows and data. We describe the MPO concepts and its software architecture. We also report the current status of the software as well as the initial deployment experience.},
doi = {10.1016/j.fusengdes.2016.04.023},
journal = {Fusion Engineering and Design},
number = C,
volume = 112,
place = {United States},
year = {2016},
month = {4}
}