skip to main content
OSTI.GOV title logo U.S. Department of Energy
Office of Scientific and Technical Information

Title: Integrating prediction, provenance, and optimization into high energy workflows

Abstract

We propose a novel approach for efficient execution of workflows on distributed resources. The key components of this framework include: performance modeling to quantitatively predict workflow component behavior; optimization-based scheduling such as choosing an optimal subset of resources to meet demand and assignment of tasks to resources; distributed I/O optimizations such as prefetching; and provenance methods for collecting performance data. In preliminary results, these techniques improve throughput on a small Belle II workflow by 20%.

Authors:
; ; ; ; ; ; ; ;
Publication Date:
Research Org.:
Pacific Northwest National Lab. (PNNL), Richland, WA (United States)
Sponsoring Org.:
USDOE
OSTI Identifier:
1434869
Report Number(s):
PNNL-SA-129007
Journal ID: ISSN 1742-6588; KJ0404000
DOE Contract Number:
AC05-76RL01830
Resource Type:
Journal Article
Resource Relation:
Journal Name: Journal of Physics. Conference Series; Journal Volume: 898
Country of Publication:
United States
Language:
English

Citation Formats

Schram, M., Bansal, V., Friese, R. D., Tallent, N. R., Yin, J., Barker, K. J., Stephan, E., Halappanavar, M., and Kerbyson, D. J. Integrating prediction, provenance, and optimization into high energy workflows. United States: N. p., 2017. Web. doi:10.1088/1742-6596/898/6/062052.
Schram, M., Bansal, V., Friese, R. D., Tallent, N. R., Yin, J., Barker, K. J., Stephan, E., Halappanavar, M., & Kerbyson, D. J. Integrating prediction, provenance, and optimization into high energy workflows. United States. doi:10.1088/1742-6596/898/6/062052.
Schram, M., Bansal, V., Friese, R. D., Tallent, N. R., Yin, J., Barker, K. J., Stephan, E., Halappanavar, M., and Kerbyson, D. J. Sun . "Integrating prediction, provenance, and optimization into high energy workflows". United States. doi:10.1088/1742-6596/898/6/062052.
@article{osti_1434869,
title = {Integrating prediction, provenance, and optimization into high energy workflows},
author = {Schram, M. and Bansal, V. and Friese, R. D. and Tallent, N. R. and Yin, J. and Barker, K. J. and Stephan, E. and Halappanavar, M. and Kerbyson, D. J.},
abstractNote = {We propose a novel approach for efficient execution of workflows on distributed resources. The key components of this framework include: performance modeling to quantitatively predict workflow component behavior; optimization-based scheduling such as choosing an optimal subset of resources to meet demand and assignment of tasks to resources; distributed I/O optimizations such as prefetching; and provenance methods for collecting performance data. In preliminary results, these techniques improve throughput on a small Belle II workflow by 20%.},
doi = {10.1088/1742-6596/898/6/062052},
journal = {Journal of Physics. Conference Series},
number = ,
volume = 898,
place = {United States},
year = {Sun Oct 01 00:00:00 EDT 2017},
month = {Sun Oct 01 00:00:00 EDT 2017}
}