TAU Gateway: Performance Engineering for Science Gateways
In order to increase access to High Performance Computing (HPC) systems, Science Gateways have been developed to provide web-based interfaces to scientific applications. Science Gateways, which have been in use at Department of Energy Leadership Computing Facilities, throughout NSF XSEDE, and at academic and research institutions around the world, provide a simple interface which hides the technical details of compilation, data transfer, and job management on HPC systems. However, the resultant influx of users without HPC experience increases the burden on support staff and application developers. This project sought to address this problem by making improvements and developing new interfaces to the TAU Performance, a performance tool developed in part with Department of Energy SciDAC funding, to improve usability when applied to software running in a Science Gateway environment. The key innovation is to provide performance data collection and analysis tools through the same interface as the domain-specific interface provided by Science Gateways. A user will collect performance with a single button press in the web interface and will share that data with Science Gateway support staff and application developers, who can in turn analyze and visualize the collected performance data through the Science Gateway interface. Usage of Science Gateways is rapidly increasing. An analysis of usage patterns on the NSF XSEDE (eXtreme Science and Engineering Discovery Environment) systems found a five-fold increase in the number of active Science Gateway users over the period 2011-2017, and the number of active Science Gateway users now exceeds the number of traditional HPC users on XSEDE. However, the development of tools to support Science Gateway use, including performance tools, has not kept up with growth in usage. If this project is successful and is carried over into Phase II and III, performance improvements facilitated by TAU Gateway will provide benefits to support staff and application developers by aiding them in making efficient use of computational resources, will provide benefits to domain scientists by reducing time to solution, thereby accelerating the pace of scientific discovery. In Phase I, we developed a proof-of-concept software package for performance analysis of Science Gateway applications. We developed web interfaces for controlling instrumentation; developed technology for injecting the monitoring environment into pre-existing container images without requiring that their build scripts be modified; and developed a collaborative environment enabling performance results to be easily shared between end users, support staff, and developers; The software was integrated with a production Science Gateway for testing. Should a Phase II award be made, the software developed in Phase I will be extended to support a broad range of visualizations, to which an easy-to-use graphical interface will be provided for developing custom visualizations. We will develop a guided performance engineering system to help users choose what performance experiments to run and how to run them. We will integrate with multiple Science Gateway installations.
- Research Organization:
- ParaTools, Inc.
- Sponsoring Organization:
- USDOE Office of Science (SC), Engineering & Technology. Office of Small Business Innovation Research (SBIR) and Small Business Technology Transfer (STTR) Programs
- DOE Contract Number:
- SC0020465
- OSTI ID:
- 1768338
- Type / Phase:
- SBIR (Phase I)
- Report Number(s):
- DOE-PARATOOLS-1040
- Country of Publication:
- United States
- Language:
- English
Similar Records
The Neutron Science TeraGrid Gateway, a TeraGrid Science Gateway to Support the Spallation Neutron Source
Institute for Sustained Performance, Energy, and Resilience (SuPER)