PyMPI Dynamic Benchmark

Lee, G. L.; Ahn, D.; Gyllenhaal, J.; de Supinski, B. R.

Title: PyMPI Dynamic Benchmark

Abstract

Pynamic is a benchmark designed to test a system's ability to handle the Dynamic Linking and Loading (DLL) requirements of Python-based scientific applications. This benchmark is developed to add a workload to our testing environment, a workload that represents a newly emerging class of DLL behaviors. Pynamic buildins on pyMPI, and MPI extension to Python C-extension dummy codes and a glue layer that facilitates linking and loading of the generated dynamic modules into the resulting pyMPI. Pynamic is configurable, enabling modeling the static properties of a specific code as described in section 5. It does not, however, model any significant computationss of the target and hence, it is not subjected to the same level of control as the target code. In fact, HPC computer vendors and tool developers will be encouraged to add it to their tesitn suite once the code release is completed. an ability to produce and run this benchmark is an effective test for valifating the capability of a compiler and linker/loader as well as an OS kernel and other runtime system of HPC computer vendors. In addition, the benchmark is designed as a test case for stressing code development tools. Though Python has recently gained popularitymore »« less

Authors:: Lee, G. L.; Ahn, D.; Gyllenhaal, J.; de Supinski, B. R.

Publication Date:: Fri Feb 16 00:00:00 EST 2007

Research Org.:: Lawrence Livermore National Laboratory (LLNL), Livermore, CA (United States)

Sponsoring Org.:: USDOE

OSTI Identifier:: 1230967

Report Number(s):: PYNAMIC; 002058WKSTN00

DOE Contract Number:: W-7405-Eng-48

Resource Type:: Software

Software Revision:: 00

Software Package Number:: 002058

Software Package Contents:: Media Directory; Software Abstract; Media includes Source Code;/1 CD ROM

Software CPU:: WKSTN

Open Source:: No

Source Code Available:: Yes

Country of Publication:: United States

Citation Formats


                    Lee, G. L., Ahn, D., Gyllenhaal, J., and de Supinski, B. R. PyMPI Dynamic Benchmark.  
        Computer software.   Vers. 00. USDOE. 16 Feb. 2007.  
        Web.

Copy to clipboard


                    Lee, G. L., Ahn, D., Gyllenhaal, J., & de Supinski, B. R. (2007, February 16).  PyMPI Dynamic Benchmark (Version 00) [Computer software].

Copy to clipboard


                    Lee, G. L., Ahn, D., Gyllenhaal, J., and de Supinski, B. R. PyMPI Dynamic Benchmark.  
        Computer software.   Version 00. February 16, 2007.

Copy to clipboard


                    
@misc{osti_1230967,

  title        = {PyMPI Dynamic Benchmark, Version 00},

  author       = {Lee, G. L. and Ahn, D. and Gyllenhaal, J. and de Supinski, B. R.},

  abstractNote = {Pynamic is a benchmark designed to test a system's ability to handle the Dynamic Linking and Loading (DLL) requirements of Python-based scientific applications. This benchmark is developed to add a workload to our testing environment, a workload that represents a newly emerging class of DLL behaviors. Pynamic buildins on pyMPI, and MPI extension to Python C-extension dummy codes and a glue layer that facilitates linking and loading of the generated dynamic modules into the resulting pyMPI. Pynamic is configurable, enabling modeling the static properties of a specific code as described in section 5. It does not, however, model any significant computationss of the target and hence, it is not subjected to the same level of control as the target code. In fact, HPC computer vendors and tool developers will be encouraged to add it to their tesitn suite once the code release is completed. an ability to produce and run this benchmark is an effective test for valifating the capability of a compiler and linker/loader as well as an OS kernel and other runtime system of HPC computer vendors. In addition, the benchmark is designed as a test case for stressing code development tools. Though Python has recently gained popularity in the HPC community, it heavy DLL operations have hindered certain HPC code development tools, notably parallel debuggers, from performing optimally.},

  
  doi          = {},

  url          = {https://www.osti.gov/biblio/1230967},
  year         = {Fri Feb 16 00:00:00 EST 2007},

  month        = {Fri Feb 16 00:00:00 EST 2007},

  note         = 

}

Copy to clipboard

Software:

To order this software, request consultation services, or receive further information, please fill out the following request.

Request Software

Save / Share:

Export Metadata

Save to My Library

Similar records in OSTI.GOV collections:

Pynamic: the Python Dynamic Benchmark

Conference Lee, G; Ahn, D; de Supinksi, B; ...

Python is widely used in scientific computing to facilitate application development and to support features such as computational steering. Making full use of some of Python's popular features, which improve programmer productivity, leads to applications that access extremely high numbers of dynamically linked libraries (DLLs). As a result, some important Python-based applications severely stress a system's dynamic linking and loading capabilities and also cause significant difficulties for most development environment tools, such as debuggers. Furthermore, using the Python paradigm for large scale MPI-based applications can create significant file IO and further stress tools and operating systems. In this paper, wemore »« less
https://doi.org/10.1109/IISWC.2007.4362185

Full Text Available
The ASC Sequoia Programming Model

Technical Report Seager, M

In the late 1980's and early 1990's, Lawrence Livermore National Laboratory was deeply engrossed in determining the next generation programming model for the Integrated Design Codes (IDC) beyond vectorization for the Cray 1s series of computers. The vector model, developed in mid 1970's first for the CDC 7600 and later extended from stack based vector operation to memory to memory operations for the Cray 1s, lasted approximately 20 years (See Slide 5). The Cray vector era was deemed an extremely long lived era as it allowed vector codes to be developed over time (the Cray 1s were faster in scalarmore »« less
https://doi.org/10.2172/945684

Full Text Available
Diagnosis of Performance Faults in LargeScale MPI Applications via Probabilistic Progress-Dependence Inference

Journal Article Laguna, Ignacio; Ahn, Dong; de Supinski, Bronis; ... - IEEE Transactions on Parallel and Distributed Systems

Debugging large-scale parallel applications is challenging. Most existing techniques provide little information about failure root causes. Further, most debuggers significantly slow down program execution, and run sluggishly with massively parallel applications. This paper presents a novel technique that scalably infers the tasks in a parallel program on which a failure occurred, as well as the code in which it originated. Our technique combines scalable runtime analysis with static analysis to determine the least-progressed task(s) and to identify the code lines at which the failure arose. We present a novel algorithm that infers probabilistically progress dependence among MPI tasks using amore »« less
https://doi.org/10.1109/tpds.2014.2314100

Full Text Available
Decaf: Decoupled Dataflows for In Situ High-Performance Workflows

Technical Report Dreher, M.; Peterka, T.

Decaf is a dataflow system for the parallel communication of coupled tasks in an HPC workflow. The dataflow can perform arbitrary data transformations ranging from simply forwarding data to complex data redistribution. Decaf does this by allowing the user to allocate resources and execute custom code in the dataflow. All communication through the dataflow is efficient parallel message passing over MPI. The runtime for calling tasks is entirely message-driven; Decaf executes a task when all messages for the task have been received. Such a messagedriven runtime allows cyclic task dependencies in the workflow graph, for example, to enact computational steeringmore »« less
https://doi.org/10.2172/1372113

Full Text Available
Design and Performance Characterization of RADICAL-Pilot on Leadership-Class Platforms

Journal Article Merzky, Andre; Turilli, Matteo; Titov, Mikhail; ... - IEEE Transactions on Parallel and Distributed Systems

Many extreme scale scientific applications have workloads comprised of a large number of individual highperformance tasks. The Pilot abstraction decouples workload specification, resource management, and task execution via job placeholders and late-binding. As such, suitable implementations of the Pilot abstraction can support the collective execution of large number of tasks on supercomputers. We introduce RADICAL-Pilot (RP) as a portable, modular and extensible Pilot enabled runtime system. We describe RP's design, architecture and implementation. We characterize its performance and show its ability to scalably execute workloads comprised of tens of thousands heterogeneous tasks on DOE and NSF leadership-class HPC platforms. Specifically,more »« less
https://doi.org/10.1109/tpds.2021.3105994

Full Text Available

Similar Records

To receive further information, fill out the request form below. OSTI staff will begin to process an order for scientific and technical software once the signed site license agreement is received. You may also reach us by email at: .

Software Package Details
Title: PyMPI Dynamic Benchmark
Package: 002058
Revision: 00

Software Request

First Name (required)

Last Name (required)

Email Address (required)

Company / Organization

Affiliation (required)

Address 1 (required)

Address 2

Address 3

Address 4

City (required)

State

Zip Code

Country of Residence (required)

Country of Citizenship (required)

Phone Number

Fax Number

Comments / Questions