skip to main content
OSTI.GOV title logo U.S. Department of Energy
Office of Scientific and Technical Information

Title: Application-level regression testing framework using Jenkins

Abstract

Monitoring and testing for regression of large-scale systems such as the NCSA's Blue Waters supercomputer are challenging tasks. In this paper, we describe the solution we came up with to perform those tasks. The goal was to find an automated solution for running user-level regression tests to evaluate system usability and performance. Jenkins, an automation server software, was chosen for its versatility, large user base, and multitude of plugins including collecting data and plotting test results over time. We also describe our Jenkins deployment to launch and monitor jobs on remote HPC system, perform authentication with one-time password, and integrate with our LDAP server for its authorization. We show some use cases and describe our best practices for successfully using Jenkins as a user-level system-wide regression testing and monitoring framework for large supercomputer systems.

Authors:
ORCiD logo [1];  [2];  [2]
  1. Oak Ridge National Lab. (ORNL), Oak Ridge, TN (United States). National Center for Computational Sciences
  2. Univ. of Illinois, Urbana-Champaign, IL (United States). National Center for Supercomputing Applications
Publication Date:
Research Org.:
Oak Ridge National Lab. (ORNL), Oak Ridge, TN (United States)
Sponsoring Org.:
USDOE Office of Science (SC), Advanced Scientific Computing Research (ASCR) (SC-21); National Science Foundation (NSF)
OSTI Identifier:
1399387
Grant/Contract Number:
AC05-00OR22725; OCI-0725070; ACI-1238993
Resource Type:
Journal Article: Accepted Manuscript
Journal Name:
Concurrency and Computation. Practice and Experience
Additional Journal Information:
Journal Volume: 30; Journal Issue: 1; Journal ID: ISSN 1532-0626
Publisher:
Wiley
Country of Publication:
United States
Language:
English
Subject:
97 MATHEMATICS AND COMPUTING; system-monitoring; regression-testing; applications; performance; benchmarking

Citation Formats

Budiardja, Reuben, Bouvet, Timothy, and Arnold, Galen. Application-level regression testing framework using Jenkins. United States: N. p., 2017. Web. doi:10.1002/cpe.4339.
Budiardja, Reuben, Bouvet, Timothy, & Arnold, Galen. Application-level regression testing framework using Jenkins. United States. doi:10.1002/cpe.4339.
Budiardja, Reuben, Bouvet, Timothy, and Arnold, Galen. Tue . "Application-level regression testing framework using Jenkins". United States. doi:10.1002/cpe.4339.
@article{osti_1399387,
title = {Application-level regression testing framework using Jenkins},
author = {Budiardja, Reuben and Bouvet, Timothy and Arnold, Galen},
abstractNote = {Monitoring and testing for regression of large-scale systems such as the NCSA's Blue Waters supercomputer are challenging tasks. In this paper, we describe the solution we came up with to perform those tasks. The goal was to find an automated solution for running user-level regression tests to evaluate system usability and performance. Jenkins, an automation server software, was chosen for its versatility, large user base, and multitude of plugins including collecting data and plotting test results over time. We also describe our Jenkins deployment to launch and monitor jobs on remote HPC system, perform authentication with one-time password, and integrate with our LDAP server for its authorization. We show some use cases and describe our best practices for successfully using Jenkins as a user-level system-wide regression testing and monitoring framework for large supercomputer systems.},
doi = {10.1002/cpe.4339},
journal = {Concurrency and Computation. Practice and Experience},
number = 1,
volume = 30,
place = {United States},
year = {Tue Sep 26 00:00:00 EDT 2017},
month = {Tue Sep 26 00:00:00 EDT 2017}
}

Journal Article:
Free Publicly Available Full Text
This content will become publicly available on September 26, 2018
Publisher's Version of Record

Citation Metrics:
Cited by: 1 work
Citation information provided by
Web of Science

Save / Share: