PANORAMA: An approach to performance modeling and diagnosis of extreme-scale workflows

Deelman, Ewa; Carothers, Christopher; Mandal, Anirban; Tierney, Brian; Vetter, Jeffrey S.; Baldin, Ilya; Castillo, Claris; Juve, Gideon; Krol, Dariusz; Lynch, Vickie; Mayer, Ben; Meredith, Jeremy; Proffen, Thomas; Ruth, Paul; Ferreira da Silva, Rafael

doi:10.1177/1094342015594515

PANORAMA: An approach to performance modeling and diagnosis of extreme-scale workflows

Journal Article · 14 July 2015 · International Journal of High Performance Computing Applications

DOI:https://doi.org/10.1177/1094342015594515· OSTI ID:1265426

Deelman, Ewa ^[1]; Carothers, Christopher ^[2]; Mandal, Anirban ^[3]; Tierney, Brian ^[4]; Vetter, Jeffrey S. ^[5]; Baldin, Ilya ^[3]; Castillo, Claris ^[3]; Juve, Gideon ^[1]; Krol, Dariusz ^[6]; Lynch, Vickie ^[5]; Mayer, Ben ^[5]; Meredith, Jeremy ^[5]; Proffen, Thomas ^[5]; Ruth, Paul ^[3]; Ferreira da Silva, Rafael ^[1]

Univ. of Southern California, Los Angeles, CA (United States)
Rensselaer Polytechnic Inst., Troy, NY (United States)
Univ. of North Carolina, Chapel Hill, NC (United States)
Lawrence Berkeley National Lab. (LBNL), Berkeley, CA (United States)
Oak Ridge National Lab. (ORNL), Oak Ridge, TN (United States)
Univ. of Southern California, Los Angeles, CA (United States); AGH - Univ. of Science and Technology, Krakow (Poland)

Here we report that computational science is well established as the third pillar of scientific discovery and is on par with experimentation and theory. However, as we move closer toward the ability to execute exascale calculations and process the ensuing extreme-scale amounts of data produced by both experiments and computations alike, the complexity of managing the compute and data analysis tasks has grown beyond the capabilities of domain scientists. Therefore, workflow management systems are absolutely necessary to ensure current and future scientific discoveries. A key research question for these workflow management systems concerns the performance optimization of complex calculation and data analysis tasks. The central contribution of this article is a description of the PANORAMA approach for modeling and diagnosing the run-time performance of complex scientific workflows. This approach integrates extreme-scale systems testbed experimentation, structured analytical modeling, and parallel systems simulation into a comprehensive workflow framework called Pegasus for understanding and improving the overall performance of complex scientific workflows.

View Accepted Manuscript (DOE)

Research Organization:: Oak Ridge National Laboratory (ORNL), Oak Ridge, TN (United States). Spallation Neutron Source

Sponsoring Organization:: USDOE Office of Science (SC), Basic Energy Sciences (BES) (SC-22)

Grant/Contract Number:: AC05-00OR22725; SC0012636

OSTI ID:: 1265426

Journal Information:: International Journal of High Performance Computing Applications, Journal Name: International Journal of High Performance Computing Applications; ISSN 1094-3420

Publisher:: SAGECopyright Statement

Country of Publication:: United States

Language:: English

References (58)

Controlling fairness and task granularity in distributed, online, non-clairvoyant workflow executions: CONTROLLING FAIRNESS AND TASK GRANULARITY IN WORKFLOWS Ferreira da Silva, Rafael; Glatard, Tristan; Desprez, Frédéric Concurrency and Computation: Practice and Experience, Vol. 26, Issue 14 https://doi.org/10.1002/cpe.3303	journal	May 2014
Scalable molecular dynamics with NAMD Phillips, James C.; Braun, Rosemary; Wang, Wei Journal of Computational Chemistry, Vol. 26, Issue 16, p. 1781-1802 https://doi.org/10.1002/jcc.20289	journal	January 2005
Workflow Exception Patterns Russell, Nick; van der Aalst, Wil; ter Hofstede, Arthur Advanced Information Systems Engineering https://doi.org/10.1007/11767138_20	book	January 2006
A Trace-Based Investigation Of The Characteristics Of Grid Workflows Ostermann, Simon; Prodan, Radu; Fahringer, Thomas From Grids to Service and Pervasive Computing https://doi.org/10.1007/978-0-387-09455-7_14	book	January 2008
Workflow Monitoring and Analysis Tool for ASKALON Ostermann, Simon; Plankensteiner, Kassian; Prodan, Radu Grid and Services Evolution https://doi.org/10.1007/978-0-387-85966-8_6	book	October 2008
SCALEA-G: A Unified Monitoring and Performance Analysis System for the Grid Truong, Hong-Linh; Fahringer, Thomas Grid Computing https://doi.org/10.1007/978-3-540-28642-4_24	book	January 2004
A Science-Gateway Workload Archive to Study Pilot Jobs, User Activity, Bag of Tasks, Task Sub-steps, and Workflow Executions da Silva, Rafael Ferreira; Glatard, Tristan European Conference on Parallel Processing: Euro-Par 2012: Parallel Processing Workshops https://doi.org/10.1007/978-3-642-36949-0_10	book	January 2013
A Case Study into Using Common Real-Time Workflow Monitoring Infrastructure for Scientific Workflows Vahi, Karan; Harvey, Ian; Samak, Taghrid Journal of Grid Computing, Vol. 11, Issue 3 https://doi.org/10.1007/s10723-013-9265-4	journal	June 2013
Sassena — X-ray and neutron scattering calculated from molecular dynamics trajectories using massively parallel computers Lindner, Benjamin; Smith, Jeremy C. Computer Physics Communications, Vol. 183, Issue 7 https://doi.org/10.1016/j.cpc.2012.02.010	journal	July 2012
The Grid Workloads Archive Iosup, Alexandru; Li, Hui; Jan, Mathieu Future Generation Computer Systems, Vol. 24, Issue 7 https://doi.org/10.1016/j.future.2008.02.003	journal	July 2008
Workflows and e-Science: An overview of workflow system features and capabilities Deelman, Ewa; Gannon, Dennis; Shields, Matthew Future Generation Computer Systems, Vol. 25, Issue 5 https://doi.org/10.1016/j.future.2008.06.012	journal	May 2009
Characterizing and profiling scientific workflows Juve, Gideon; Chervenak, Ann; Deelman, Ewa Future Generation Computer Systems, Vol. 29, Issue 3 https://doi.org/10.1016/j.future.2012.08.015	journal	March 2013
Characterizing workflow-based activity on a production e-infrastructure using provenance data Madougou, Souley; Shahand, Shayan; Santcroos, Mark Future Generation Computer Systems, Vol. 29, Issue 8 https://doi.org/10.1016/j.future.2013.04.019	journal	October 2013
Self-healing of workflow activity incidents on distributed computing infrastructures Ferreira da Silva, Rafael; Glatard, Tristan; Desprez, Frédéric Future Generation Computer Systems, Vol. 29, Issue 8 https://doi.org/10.1016/j.future.2013.06.012	journal	October 2013
Pegasus, a workflow management system for science automation Deelman, Ewa; Vahi, Karan; Juve, Gideon Future Generation Computer Systems, Vol. 46 https://doi.org/10.1016/j.future.2014.10.008	journal	May 2015
Mantid—Data analysis and visualization package for neutron scattering and μ SR experiments Arnold, O.; Bilheux, J. C.; Borreguero, J. M. Nuclear Instruments and Methods in Physics Research Section A: Accelerators, Spectrometers, Detectors and Associated Equipment, Vol. 764 https://doi.org/10.1016/j.nima.2014.07.029	journal	November 2014
The Spallation Neutron Source in Oak Ridge: A powerful tool for materials research Mason, T. E.; Abernathy, D.; Anderson, I. Physica B: Condensed Matter, Vol. 385-386 https://doi.org/10.1016/j.physb.2006.05.281	journal	November 2006
Rethinking data management for big data scientific workflows Vahi, Karan; Rynge, Mats; Juve, Gideon 2013 IEEE International Conference on Big Data https://doi.org/10.1109/BigData.2013.6691724	conference	October 2013
Workload Characterization for Capacity Planning and Performance Management in IaaS Cloud Mahambre, Shruti; Kulkarni, Purushottam; Bellur, Umesh 2012 IEEE International Conference on Cloud Computing in Emerging Markets (CCEM) https://doi.org/10.1109/CCEM.2012.6354624	conference	October 2012
Data Management Challenges of Data-Intensive Scientific Workflows Deelman, Ewa; Chervenak, Ann 2008 8th IEEE International Symposium on Cluster Computing and the Grid (CCGrid), 2008 Eighth IEEE International Symposium on Cluster Computing and the Grid (CCGRID) https://doi.org/10.1109/CCGRID.2008.24	conference	May 2008
A Lightweight Middleware Monitor for Distributed Scientific Workflows Cruz, Sergio Manuel Serra da; Silva, Fabricio Nogueira da; Gadelha Jr., Luiz M. R. 2008 8th IEEE International Symposium on Cluster Computing and the Grid (CCGrid), 2008 Eighth IEEE International Symposium on Cluster Computing and the Grid (CCGRID) https://doi.org/10.1109/CCGRID.2008.89	conference	May 2008
The Failure Trace Archive: Enabling Comparative Analysis of Failures in Diverse Distributed Systems Kondo, Derrick; Javadi, Bahman; Iosup, Alexandru 2010 10th IEEE/ACM International Conference on Cluster, Cloud and Grid Computing https://doi.org/10.1109/CCGRID.2010.71	conference	May 2010
Practical Resource Monitoring for Robust High Throughput Computing Juve, Gideon; Tovar, Benjamin; Silva, Rafael Ferreira da 2015 IEEE International Conference on Cluster Computing (CLUSTER) https://doi.org/10.1109/CLUSTER.2015.115	conference	September 2015
Managing Large-Scale Workflow Execution from Resource Provisioning to Provenance Tracking: The CyberShake Example Deelman, Ewa; Callaghan, Scott; Field, Edward 2006 Second IEEE International Conference on e-Science and Grid Computing (e-Science'06) https://doi.org/10.1109/E-SCIENCE.2006.261098	conference	December 2006
Application-Level Resource Provisioning on the Grid Singh, Gurmeet; Kesselman, Carl; Deelman, Ewa 2006 Second IEEE International Conference on e-Science and Grid Computing (e-Science'06) https://doi.org/10.1109/E-SCIENCE.2006.261167	conference	December 2006
Dynamic Cloud provisioning for scientific Grid workflows Ostermann, Simon; Prodan, Radu; Fahringer, Thomas 2010 11th IEEE/ACM International Conference on Grid Computing (GRID) https://doi.org/10.1109/GRID.2010.5697953	conference	October 2010
Online Fault and Anomaly Detection for Large-Scale Scientific Workflows Samak, Taghrid; Gunter, Dan; Goode, Monte Communication (HPCC), 2011 IEEE International Conference on High Performance Computing and Communications https://doi.org/10.1109/HPCC.2011.55	conference	September 2011
Workload characterization on a production Hadoop cluster: A case study on Taobao Ren, Zujie; Xu, Xianghua; Wan, Jian 2012 IEEE International Symposium on Workload Characterization (IISWC) https://doi.org/10.1109/IISWC.2012.6402895	conference	November 2012
An introductory exascale feasibility study for FFTs and multigrid Gahvari, Hormozd; Gropp, William 2010 IEEE International Symposium on Parallel & Distributed Processing (IPDPS) https://doi.org/10.1109/IPDPS.2010.5470417	conference	April 2010
The vision of autonomic computing Kephart, J. O.; Chess, D. M. Computer, Vol. 36, Issue 1 https://doi.org/10.1109/MC.2003.1160055	journal	January 2003
Enabling persistent queries for cross-aggregate performance monitoring Mandal, Anirban; Baldin, Ilya IEEE Communications Magazine, Vol. 52, Issue 5 https://doi.org/10.1109/MCOM.2014.6815907	journal	May 2014
Grid Computing Workloads Iosup, Alexandru; Epema, Dick IEEE Internet Computing, Vol. 15, Issue 2 https://doi.org/10.1109/MIC.2010.130	journal	March 2011
On the role of burst buffers in leadership-class storage systems Liu, Ning; Cope, Jason; Carns, Philip 2012 IEEE 28th Symposium on Mass Storage Systems and Technologies (MSST) https://doi.org/10.1109/MSST.2012.6232369	conference	April 2012
Scalable Time Warp on Blue Gene Supercomputers Bauer Jr., David W.; Carothers, Christopher D.; Holder, Akintayo 2009 ACM/IEEE/SCS 23rd Workshop on Principles of Advanced and Distributed Simulation (PADS) https://doi.org/10.1109/PADS.2009.21	conference	June 2009
Aspen: A domain specific language for performance modeling Spafford, Kyle L.; Vetter, Jeffrey S. 2012 SC - International Conference for High Performance Computing, Networking, Storage and Analysis, 2012 International Conference for High Performance Computing, Networking, Storage and Analysis https://doi.org/10.1109/SC.2012.20	conference	November 2012
Modeling a Million-Node Dragonfly Network Using Massively Parallel Discrete-Event Simulation Mubarak, Misbah; Carothers, Christopher D.; Ross, Robert 2012 SC Companion: High Performance Computing, Networking, Storage and Analysis (SCC), 2012 SC Companion: High Performance Computing, Networking Storage and Analysis https://doi.org/10.1109/SC.Companion.2012.56	conference	November 2012
Characterization of scientific workflows Bharathi, Shishir; Chervenak, Ann; Deelman, Ewa 2008 Third Workshop on Workflows in Support of Large-Scale Science (WORKS 2008) https://doi.org/10.1109/WORKS.2008.4723958	conference	November 2008
A Cleanup Algorithm for Implementing Storage Constraints in Scientific Workflow Executions Srinivasan, Sudarshan; Juve, Gideon; Silva, Rafael Ferreira da 2014 9th Workshop on Workflows in Support of Large-Scale Science (WORKS) https://doi.org/10.1109/WORKS.2014.8	conference	November 2014
On deciding between conservative and optimistic approaches on massively parallel platforms Carothers, Christoph; Perumalla, Kalyan S. 2010 Winter Simulation Conference - (WSC 2010), Proceedings of the 2010 Winter Simulation Conference https://doi.org/10.1109/WSC.2010.5679119	conference	December 2010
Community Resources for Enabling Research in Distributed Scientific Workflows Silva, Rafael Ferreira da; Chen, Weiwei; Juve, Gideon 2014 IEEE 10th International Conference on e-Science (e-Science) https://doi.org/10.1109/eScience.2014.44	conference	October 2014
LogP: towards a realistic model of parallel computation Culler, David; Karp, Richard; Patterson, David ACM SIGPLAN Notices, Vol. 28, Issue 7 https://doi.org/10.1145/173284.155333	journal	July 1993
Using simulation to design extremescale applications and architectures: programming model exploration Janssen, Curtis L.; Adalsteinsson, Helgi; Kenny, Joseph P. ACM SIGMETRICS Performance Evaluation Review, Vol. 38, Issue 4 https://doi.org/10.1145/1964218.1964220	journal	March 2011
The structural simulation toolkit Rodrigues, A. F.; CooperBalls, E.; Jacob, B. ACM SIGMETRICS Performance Evaluation Review, Vol. 38, Issue 4 https://doi.org/10.1145/1964218.1964225	journal	March 2011
Auto-scaling to minimize cost and meet application deadlines in cloud workflows Mao, Ming; Humphrey, Marty Proceedings of 2011 International Conference for High Performance Computing, Networking, Storage and Analysis on - SC '11 https://doi.org/10.1145/2063384.2063449	conference	January 2011
Failure prediction and localization in large scientific workflows Samak, Taghrid; Gunter, Dan; Goode, Monte Proceedings of the 6th workshop on Workflows in support of large-scale science - WORKS '11 https://doi.org/10.1145/2110497.2110510	conference	January 2011
LogGP: incorporating long messages into the LogP model---one step closer towards a realistic model for parallel computation Alexandrov, Albert; Ionescu, Mihai F.; Schauser, Klaus E. Proceedings of the seventh annual ACM symposium on Parallel algorithms and architectures - SPAA '95 https://doi.org/10.1145/215399.215427	conference	January 1995
On the communication complexity of 3D FFTs and its implications for Exascale Czechowski, Kenneth; Battaglino, Casey; McClanahan, Chris Proceedings of the 26th ACM international conference on Supercomputing - ICS '12 https://doi.org/10.1145/2304576.2304604	conference	January 2012
Warp speed: executing time warp on 1,966,080 cores Barnes, Peter D.; Carothers, Christopher D.; Jefferson, David R. Proceedings of the 2013 ACM SIGSIM conference on Principles of advanced discrete simulation - SIGSIM-PADS '13 https://doi.org/10.1145/2486092.2486134	conference	January 2013
Toward fine-grained online task characteristics estimation in scientific workflows da Silva, Rafael Ferreira; Juve, Gideon; Deelman, Ewa Proceedings of the 8th Workshop on Workflows in Support of Large-Scale Science - WORKS '13 https://doi.org/10.1145/2534248.2534254	conference	January 2013
Evaluating I/O aware network management for scientific workflows on networked clouds Mandal, Anirban; Ruth, Paul; Baldin, Ilya Proceedings of the Third International Workshop on Network-Aware Data Management - NDM '13 https://doi.org/10.1145/2534695.2534698	conference	January 2013
The macroscopic behavior of the TCP congestion avoidance algorithm Mathis, Matthew; Semke, Jeffrey; Mahdavi, Jamshid ACM SIGCOMM Computer Communication Review, Vol. 27, Issue 3 https://doi.org/10.1145/263932.264023	journal	July 1997
COMPASS: A Framework for Automated Performance Modeling and Prediction Lee, Seyong; Meredith, Jeremy S.; Vetter, Jeffrey S. Proceedings of the 29th ACM on International Conference on Supercomputing - ICS '15 https://doi.org/10.1145/2751205.2751220	conference	January 2015
Efficient optimistic parallel simulations using reverse computation Carothers, Christopher D.; Perumalla, Kalyan S.; Fujimoto, Richard M. ACM Transactions on Modeling and Computer Simulation, Vol. 9, Issue 3 https://doi.org/10.1145/347823.347828	journal	July 1999
A bridging model for parallel computation Valiant, Leslie G. Communications of the ACM, Vol. 33, Issue 8 https://doi.org/10.1145/79173.79181	journal	August 1990
Pegasus: A Framework for Mapping Complex Scientific Workflows onto Distributed Systems Deelman, Ewa; Singh, Gurmeet; Su, Mei-Hui Scientific Programming, Vol. 13, Issue 3 https://doi.org/10.1155/2005/128026	journal	January 2005
Measuring TeraGrid: workload characterization for a high-performance computing federation Hart, David L. The International Journal of High Performance Computing Applications, Vol. 25, Issue 4 https://doi.org/10.1177/1094342010394382	journal	February 2011
Modeling synthetic aperture radar computation with Aspen Spafford, Kyle; Vetter, Jeffrey S.; Benson, Thomas The International Journal of High Performance Computing Applications, Vol. 27, Issue 3 https://doi.org/10.1177/1094342013488262	journal	July 2013
Instantiating a Global Network Measurement Framework Tierney, Brian L.; Boote, Jeff; Boyd, Eric https://doi.org/10.2172/946807	report	December 2008

Cited By (2)

The role of machine learning in scientific workflows Deelman, Ewa; Mandal, Anirban; Jiang, Ming The International Journal of High Performance Computing Applications, Vol. 33, Issue 6 https://doi.org/10.1177/1094342019852127	journal	May 2019
Symbolic regression in materials science Wang, Yiqun; Wagner, Nicholas; Rondinelli, James M. MRS Communications, Vol. 9, Issue 3 https://doi.org/10.1557/mrc.2019.85	journal	June 2019

Similar Records

A characterization of workflow management systems for extreme-scale applications

Journal Article · 2017 · Future Generations Computer Systems · OSTI ID:1408072

Extreme-scale workflows: A perspective from the JLESC international community

Journal Article · 2024 · Future Generations Computer Systems · OSTI ID:2440426

Panorama 360 (Final Report)

Technical Report · 2023 · OSTI ID:1846090

Related Subjects

97 MATHEMATICS AND COMPUTING
extreme scale
performance modeling
scientific workflow

PANORAMA: An approach to performance modeling and diagnosis of extreme-scale workflows

Citation Formats

References (58)

Cited By (2)

Similar Records

Related Subjects