skip to main content
OSTI.GOV title logo U.S. Department of Energy
Office of Scientific and Technical Information

Title: Job Management and Task Bundling

Abstract

High Performance Computing is often performed on scarce and shared computing resources. To ensure computers are used to their full capacity, administrators often incentivize large workloads that are not possible on smaller systems. Measurements in Lattice QCD frequently do not scale to machine-size workloads. By bundling tasks together we can create large jobs suitable for gigantic partitions. We discuss METAQ and mpi_jm, software developed to dynamically group computational tasks together, that can intelligently backfill to consume idle time without substantial changes to users’ current workflows or executables.

Authors:
 [1]; ORCiD logo [2];  [3];  [4]
  1. Forschungszentrum Julich GmbH (Julich Research Centre), Germany
  2. ORNL
  3. University of California, Berkeley
  4. LBNL/Univ. California, Berkeley/LLNL
Publication Date:
Research Org.:
Oak Ridge National Lab. (ORNL), Oak Ridge, TN (United States)
Sponsoring Org.:
USDOE
OSTI Identifier:
1459309
DOE Contract Number:  
AC05-00OR22725
Resource Type:
Conference
Resource Relation:
Journal Volume: 175; Conference: International Symposium on Lattice Field Theory (Lattice 2017) - Granada, , Spain - 6/18/2017 8:00:00 AM-6/24/2017 8:00:00 AM
Country of Publication:
United States
Language:
English

Citation Formats

Berkowitz, Evan, Jansen, Gustav R., McElvain, Kenneth, and Walker-Loud, Andre. Job Management and Task Bundling. United States: N. p., 2018. Web. doi:10.1051/epjconf/201817509007.
Berkowitz, Evan, Jansen, Gustav R., McElvain, Kenneth, & Walker-Loud, Andre. Job Management and Task Bundling. United States. doi:10.1051/epjconf/201817509007.
Berkowitz, Evan, Jansen, Gustav R., McElvain, Kenneth, and Walker-Loud, Andre. Thu . "Job Management and Task Bundling". United States. doi:10.1051/epjconf/201817509007. https://www.osti.gov/servlets/purl/1459309.
@article{osti_1459309,
title = {Job Management and Task Bundling},
author = {Berkowitz, Evan and Jansen, Gustav R. and McElvain, Kenneth and Walker-Loud, Andre},
abstractNote = {High Performance Computing is often performed on scarce and shared computing resources. To ensure computers are used to their full capacity, administrators often incentivize large workloads that are not possible on smaller systems. Measurements in Lattice QCD frequently do not scale to machine-size workloads. By bundling tasks together we can create large jobs suitable for gigantic partitions. We discuss METAQ and mpi_jm, software developed to dynamically group computational tasks together, that can intelligently backfill to consume idle time without substantial changes to users’ current workflows or executables.},
doi = {10.1051/epjconf/201817509007},
journal = {},
number = ,
volume = 175,
place = {United States},
year = {Thu Mar 01 00:00:00 EST 2018},
month = {Thu Mar 01 00:00:00 EST 2018}
}

Conference:
Other availability
Please see Document Availability for additional information on obtaining the full-text document. Library patrons may search WorldCat to identify libraries that hold this conference proceeding.

Save / Share: