PANORAMA: An approach to performance modeling and diagnosis of extreme-scale workflows

Deelman, Ewa; Carothers, Christopher; Mandal, Anirban; Tierney, Brian; Vetter, Jeffrey S.; Baldin, Ilya; Castillo, Claris; Juve, Gideon; Krol, Dariusz; Lynch, Vickie; Mayer, Ben; Meredith, Jeremy; Proffen, Thomas; Ruth, Paul; Ferreira da Silva, Rafael

doi:10.1177/1094342015594515

Title: PANORAMA: An approach to performance modeling and diagnosis of extreme-scale workflows

Journal Article · Tue Jul 14 00:00:00 EDT 2015 · International Journal of High Performance Computing Applications

DOI:https://doi.org/10.1177/1094342015594515· OSTI ID:1265426

Deelman, Ewa ^[1]; Carothers, Christopher ^[2]; Mandal, Anirban ^[3]; Tierney, Brian ^[4]; Vetter, Jeffrey S. ^[5]; Baldin, Ilya ^[3]; Castillo, Claris ^[3]; Juve, Gideon ^[1]; Krol, Dariusz ^[6]; Lynch, Vickie ^[5]; Mayer, Ben ^[5]; Meredith, Jeremy ^[5]; Proffen, Thomas ^[5]; Ruth, Paul ^[3]; Ferreira da Silva, Rafael ^[1]

Univ. of Southern California, Los Angeles, CA (United States)
Rensselaer Polytechnic Inst., Troy, NY (United States)
Univ. of North Carolina, Chapel Hill, NC (United States)
Lawrence Berkeley National Lab. (LBNL), Berkeley, CA (United States)
Oak Ridge National Lab. (ORNL), Oak Ridge, TN (United States)
Univ. of Southern California, Los Angeles, CA (United States); AGH - Univ. of Science and Technology, Krakow (Poland)

Here we report that computational science is well established as the third pillar of scientific discovery and is on par with experimentation and theory. However, as we move closer toward the ability to execute exascale calculations and process the ensuing extreme-scale amounts of data produced by both experiments and computations alike, the complexity of managing the compute and data analysis tasks has grown beyond the capabilities of domain scientists. Therefore, workflow management systems are absolutely necessary to ensure current and future scientific discoveries. A key research question for these workflow management systems concerns the performance optimization of complex calculation and data analysis tasks. The central contribution of this article is a description of the PANORAMA approach for modeling and diagnosing the run-time performance of complex scientific workflows. This approach integrates extreme-scale systems testbed experimentation, structured analytical modeling, and parallel systems simulation into a comprehensive workflow framework called Pegasus for understanding and improving the overall performance of complex scientific workflows.

View Accepted Manuscript (DOE)

Cite

Export

Save

Research Organization:: Oak Ridge National Laboratory (ORNL), Oak Ridge, TN (United States). Spallation Neutron Source (SNS)

Sponsoring Organization:: USDOE Office of Science (SC), Basic Energy Sciences (BES)

Grant/Contract Number:: AC05-00OR22725; SC0012636

OSTI ID:: 1265426

Journal Information:: International Journal of High Performance Computing Applications, Journal Name: International Journal of High Performance Computing Applications; ISSN 1094-3420

Publisher:: SAGECopyright Statement

Country of Publication:: United States

Language:: English

Citation Metrics:

Cited by: 19 works

Citation information provided by
Web of Science

References (57)

LogGP: incorporating long messages into the LogP model---one step closer towards a realistic model for parallel computation Alexandrov, Albert; Ionescu, Mihai F.; Schauser, Klaus E. Proceedings of the seventh annual ACM symposium on Parallel algorithms and architectures - SPAA '95 https://doi.org/10.1145/215399.215427	conference	January 1995
Mantid—Data analysis and visualization package for neutron scattering and $μ$ SR experiments Arnold, O.; Bilheux, J. C.; Borreguero, J. M. Nuclear Instruments and Methods in Physics Research Section A: Accelerators, Spectrometers, Detectors and Associated Equipment, Vol. 764 https://doi.org/10.1016/j.nima.2014.07.029	journal	November 2014
Warp speed: executing time warp on 1,966,080 cores Barnes, Peter D.; Carothers, Christopher D.; Jefferson, David R. Proceedings of the 2013 ACM SIGSIM conference on Principles of advanced discrete simulation - SIGSIM-PADS '13 https://doi.org/10.1145/2486092.2486134	conference	January 2013
Scalable Time Warp on Blue Gene Supercomputers Bauer Jr., David W.; Carothers, Christopher D.; Holder, Akintayo 2009 ACM/IEEE/SCS 23rd Workshop on Principles of Advanced and Distributed Simulation (PADS) https://doi.org/10.1109/PADS.2009.21	conference	June 2009
Characterization of scientific workflows Bharathi, Shishir; Chervenak, Ann; Deelman, Ewa 2008 Third Workshop on Workflows in Support of Large-Scale Science (WORKS 2008) https://doi.org/10.1109/WORKS.2008.4723958	conference	November 2008
On deciding between conservative and optimistic approaches on massively parallel platforms Carothers, Christoph; Perumalla, Kalyan S. 2010 Winter Simulation Conference - (WSC 2010), Proceedings of the 2010 Winter Simulation Conference https://doi.org/10.1109/WSC.2010.5679119	conference	December 2010
Efficient optimistic parallel simulations using reverse computation Carothers, Christopher D.; Perumalla, Kalyan S.; Fujimoto, Richard M. ACM Transactions on Modeling and Computer Simulation, Vol. 9, Issue 3 https://doi.org/10.1145/347823.347828	journal	July 1999
LogP: towards a realistic model of parallel computation Culler, David; Karp, Richard; Patterson, David ACM SIGPLAN Notices, Vol. 28, Issue 7 https://doi.org/10.1145/173284.155333	journal	July 1993
On the communication complexity of 3D FFTs and its implications for Exascale Czechowski, Kenneth; Battaglino, Casey; McClanahan, Chris Proceedings of the 26th ACM international conference on Supercomputing - ICS '12 https://doi.org/10.1145/2304576.2304604	conference	January 2012
A Lightweight Middleware Monitor for Distributed Scientific Workflows Cruz, Sergio Manuel Serra da; Silva, Fabricio Nogueira da; Gadelha Jr., Luiz M. R. 2008 8th IEEE International Symposium on Cluster Computing and the Grid (CCGrid), 2008 Eighth IEEE International Symposium on Cluster Computing and the Grid (CCGRID) https://doi.org/10.1109/CCGRID.2008.89	conference	May 2008
Managing Large-Scale Workflow Execution from Resource Provisioning to Provenance Tracking: The CyberShake Example Deelman, Ewa; Callaghan, Scott; Field, Edward 2006 Second IEEE International Conference on e-Science and Grid Computing (e-Science'06) https://doi.org/10.1109/E-SCIENCE.2006.261098	conference	December 2006
Data Management Challenges of Data-Intensive Scientific Workflows Deelman, Ewa; Chervenak, Ann 2008 8th IEEE International Symposium on Cluster Computing and the Grid (CCGrid), 2008 Eighth IEEE International Symposium on Cluster Computing and the Grid (CCGRID) https://doi.org/10.1109/CCGRID.2008.24	conference	May 2008
Workflows and e-Science: An overview of workflow system features and capabilities Deelman, Ewa; Gannon, Dennis; Shields, Matthew Future Generation Computer Systems, Vol. 25, Issue 5 https://doi.org/10.1016/j.future.2008.06.012	journal	May 2009
Pegasus: A Framework for Mapping Complex Scientific Workflows onto Distributed Systems Deelman, Ewa; Singh, Gurmeet; Su, Mei-Hui Scientific Programming, Vol. 13, Issue 3 https://doi.org/10.1155/2005/128026	journal	January 2005
Pegasus, a workflow management system for science automation Deelman, Ewa; Vahi, Karan; Juve, Gideon Future Generation Computer Systems, Vol. 46 https://doi.org/10.1016/j.future.2014.10.008	journal	May 2015
Community Resources for Enabling Research in Distributed Scientific Workflows Silva, Rafael Ferreira da; Chen, Weiwei; Juve, Gideon 2014 IEEE 10th International Conference on e-Science (e-Science) https://doi.org/10.1109/eScience.2014.44	conference	October 2014
A Science-Gateway Workload Archive to Study Pilot Jobs, User Activity, Bag of Tasks, Task Sub-steps, and Workflow Executions da Silva, Rafael Ferreira; Glatard, Tristan European Conference on Parallel Processing: Euro-Par 2012: Parallel Processing Workshops https://doi.org/10.1007/978-3-642-36949-0_10	book	January 2013
Self-healing of workflow activity incidents on distributed computing infrastructures Ferreira da Silva, Rafael; Glatard, Tristan; Desprez, Frédéric Future Generation Computer Systems, Vol. 29, Issue 8 https://doi.org/10.1016/j.future.2013.06.012	journal	October 2013
Controlling fairness and task granularity in distributed, online, non-clairvoyant workflow executions: CONTROLLING FAIRNESS AND TASK GRANULARITY IN WORKFLOWS Ferreira da Silva, Rafael; Glatard, Tristan; Desprez, Frédéric Concurrency and Computation: Practice and Experience, Vol. 26, Issue 14 https://doi.org/10.1002/cpe.3303	journal	May 2014
Toward fine-grained online task characteristics estimation in scientific workflows da Silva, Rafael Ferreira; Juve, Gideon; Deelman, Ewa Proceedings of the 8th Workshop on Workflows in Support of Large-Scale Science - WORKS '13 https://doi.org/10.1145/2534248.2534254	conference	January 2013
An introductory exascale feasibility study for FFTs and multigrid Gahvari, Hormozd; Gropp, William 2010 IEEE International Symposium on Parallel & Distributed Processing (IPDPS) https://doi.org/10.1109/IPDPS.2010.5470417	conference	April 2010
Measuring TeraGrid: workload characterization for a high-performance computing federation Hart, David L. The International Journal of High Performance Computing Applications, Vol. 25, Issue 4 https://doi.org/10.1177/1094342010394382	journal	February 2011
Grid Computing Workloads Iosup, Alexandru; Epema, Dick IEEE Internet Computing, Vol. 15, Issue 2 https://doi.org/10.1109/MIC.2010.130	journal	March 2011
The Grid Workloads Archive Iosup, Alexandru; Li, Hui; Jan, Mathieu Future Generation Computer Systems, Vol. 24, Issue 7 https://doi.org/10.1016/j.future.2008.02.003	journal	July 2008
Using simulation to design extremescale applications and architectures: programming model exploration Janssen, Curtis L.; Adalsteinsson, Helgi; Kenny, Joseph P. ACM SIGMETRICS Performance Evaluation Review, Vol. 38, Issue 4 https://doi.org/10.1145/1964218.1964220	journal	March 2011
Characterizing and profiling scientific workflows Juve, Gideon; Chervenak, Ann; Deelman, Ewa Future Generation Computer Systems, Vol. 29, Issue 3 https://doi.org/10.1016/j.future.2012.08.015	journal	March 2013
Practical Resource Monitoring for Robust High Throughput Computing Juve, Gideon; Tovar, Benjamin; Silva, Rafael Ferreira da 2015 IEEE International Conference on Cluster Computing (CLUSTER) https://doi.org/10.1109/CLUSTER.2015.115	conference	September 2015
The vision of autonomic computing Kephart, J. O.; Chess, D. M. Computer, Vol. 36, Issue 1 https://doi.org/10.1109/MC.2003.1160055	journal	January 2003
The Failure Trace Archive: Enabling Comparative Analysis of Failures in Diverse Distributed Systems Kondo, Derrick; Javadi, Bahman; Iosup, Alexandru 2010 10th IEEE/ACM International Conference on Cluster, Cloud and Grid Computing https://doi.org/10.1109/CCGRID.2010.71	conference	May 2010
COMPASS: A Framework for Automated Performance Modeling and Prediction Lee, Seyong; Meredith, Jeremy S.; Vetter, Jeffrey S. Proceedings of the 29th ACM on International Conference on Supercomputing - ICS '15 https://doi.org/10.1145/2751205.2751220	conference	January 2015
Sassena — X-ray and neutron scattering calculated from molecular dynamics trajectories using massively parallel computers Lindner, Benjamin; Smith, Jeremy C. Computer Physics Communications, Vol. 183, Issue 7 https://doi.org/10.1016/j.cpc.2012.02.010	journal	July 2012
On the role of burst buffers in leadership-class storage systems Liu, Ning; Cope, Jason; Carns, Philip 2012 IEEE 28th Symposium on Mass Storage Systems and Technologies (MSST) https://doi.org/10.1109/MSST.2012.6232369	conference	April 2012
Characterizing workflow-based activity on a production e-infrastructure using provenance data Madougou, Souley; Shahand, Shayan; Santcroos, Mark Future Generation Computer Systems, Vol. 29, Issue 8 https://doi.org/10.1016/j.future.2013.04.019	journal	October 2013
Workload Characterization for Capacity Planning and Performance Management in IaaS Cloud Mahambre, Shruti; Kulkarni, Purushottam; Bellur, Umesh 2012 IEEE International Conference on Cloud Computing in Emerging Markets (CCEM) https://doi.org/10.1109/CCEM.2012.6354624	conference	October 2012
Enabling persistent queries for cross-aggregate performance monitoring Mandal, Anirban; Baldin, Ilya IEEE Communications Magazine, Vol. 52, Issue 5 https://doi.org/10.1109/MCOM.2014.6815907	journal	May 2014
Evaluating I/O aware network management for scientific workflows on networked clouds Mandal, Anirban; Ruth, Paul; Baldin, Ilya Proceedings of the Third International Workshop on Network-Aware Data Management - NDM '13 https://doi.org/10.1145/2534695.2534698	conference	January 2013
Auto-scaling to minimize cost and meet application deadlines in cloud workflows Mao, Ming; Humphrey, Marty Proceedings of 2011 International Conference for High Performance Computing, Networking, Storage and Analysis on - SC '11 https://doi.org/10.1145/2063384.2063449	conference	January 2011
The Spallation Neutron Source in Oak Ridge: A powerful tool for materials research Mason, T. E.; Abernathy, D.; Anderson, I. Physica B: Condensed Matter, Vol. 385-386 https://doi.org/10.1016/j.physb.2006.05.281	journal	November 2006
The macroscopic behavior of the TCP congestion avoidance algorithm Mathis, Matthew; Semke, Jeffrey; Mahdavi, Jamshid ACM SIGCOMM Computer Communication Review, Vol. 27, Issue 3 https://doi.org/10.1145/263932.264023	journal	July 1997
Modeling a Million-Node Dragonfly Network Using Massively Parallel Discrete-Event Simulation Mubarak, Misbah; Carothers, Christopher D.; Ross, Robert 2012 SC Companion: High Performance Computing, Networking, Storage and Analysis (SCC), 2012 SC Companion: High Performance Computing, Networking Storage and Analysis https://doi.org/10.1109/SC.Companion.2012.56	conference	November 2012
Workflow Monitoring and Analysis Tool for ASKALON Ostermann, Simon; Plankensteiner, Kassian; Prodan, Radu Grid and Services Evolution https://doi.org/10.1007/978-0-387-85966-8_6	book	October 2008
A Trace-Based Investigation Of The Characteristics Of Grid Workflows Ostermann, Simon; Prodan, Radu; Fahringer, Thomas From Grids to Service and Pervasive Computing https://doi.org/10.1007/978-0-387-09455-7_14	book	January 2008
Dynamic Cloud provisioning for scientific Grid workflows Ostermann, Simon; Prodan, Radu; Fahringer, Thomas 2010 11th IEEE/ACM International Conference on Grid Computing (GRID) https://doi.org/10.1109/GRID.2010.5697953	conference	October 2010
Scalable molecular dynamics with NAMD Phillips, James C.; Braun, Rosemary; Wang, Wei Journal of Computational Chemistry, Vol. 26, Issue 16, p. 1781-1802 https://doi.org/10.1002/jcc.20289	journal	January 2005
Workload characterization on a production Hadoop cluster: A case study on Taobao Ren, Zujie; Xu, Xianghua; Wan, Jian 2012 IEEE International Symposium on Workload Characterization (IISWC) https://doi.org/10.1109/IISWC.2012.6402895	conference	November 2012
The structural simulation toolkit Rodrigues, A. F.; CooperBalls, E.; Jacob, B. ACM SIGMETRICS Performance Evaluation Review, Vol. 38, Issue 4 https://doi.org/10.1145/1964218.1964225	journal	March 2011
Workflow Exception Patterns Russell, Nick; van der Aalst, Wil; ter Hofstede, Arthur Advanced Information Systems Engineering https://doi.org/10.1007/11767138_20	book	January 2006
Failure prediction and localization in large scientific workflows Samak, Taghrid; Gunter, Dan; Goode, Monte Proceedings of the 6th workshop on Workflows in support of large-scale science - WORKS '11 https://doi.org/10.1145/2110497.2110510	conference	January 2011
Application-Level Resource Provisioning on the Grid Singh, Gurmeet; Kesselman, Carl; Deelman, Ewa 2006 Second IEEE International Conference on e-Science and Grid Computing (e-Science'06) https://doi.org/10.1109/E-SCIENCE.2006.261167	conference	December 2006
Aspen: A domain specific language for performance modeling Spafford, Kyle L.; Vetter, Jeffrey S. 2012 SC - International Conference for High Performance Computing, Networking, Storage and Analysis, 2012 International Conference for High Performance Computing, Networking, Storage and Analysis https://doi.org/10.1109/SC.2012.20	conference	November 2012
Modeling synthetic aperture radar computation with Aspen Spafford, Kyle; Vetter, Jeffrey S.; Benson, Thomas The International Journal of High Performance Computing Applications, Vol. 27, Issue 3 https://doi.org/10.1177/1094342013488262	journal	July 2013
A Cleanup Algorithm for Implementing Storage Constraints in Scientific Workflow Executions Srinivasan, Sudarshan; Juve, Gideon; Silva, Rafael Ferreira da 2014 9th Workshop on Workflows in Support of Large-Scale Science (WORKS) https://doi.org/10.1109/WORKS.2014.8	conference	November 2014
Instantiating a Global Network Measurement Framework Tierney, Brian L.; Boote, Jeff; Boyd, Eric https://doi.org/10.2172/946807	report	December 2008
SCALEA-G: A Unified Monitoring and Performance Analysis System for the Grid Truong, Hong-Linh; Fahringer, Thomas Grid Computing https://doi.org/10.1007/978-3-540-28642-4_24	book	January 2004
A Case Study into Using Common Real-Time Workflow Monitoring Infrastructure for Scientific Workflows Vahi, Karan; Harvey, Ian; Samak, Taghrid Journal of Grid Computing, Vol. 11, Issue 3 https://doi.org/10.1007/s10723-013-9265-4	journal	June 2013
Rethinking data management for big data scientific workflows Vahi, Karan; Rynge, Mats; Juve, Gideon 2013 IEEE International Conference on Big Data https://doi.org/10.1109/BigData.2013.6691724	conference	October 2013
A bridging model for parallel computation Valiant, Leslie G. Communications of the ACM, Vol. 33, Issue 8 https://doi.org/10.1145/79173.79181	journal	August 1990

Cited By (2)

The role of machine learning in scientific workflows Deelman, Ewa; Mandal, Anirban; Jiang, Ming The International Journal of High Performance Computing Applications, Vol. 33, Issue 6 https://doi.org/10.1177/1094342019852127	journal	May 2019
Symbolic regression in materials science Wang, Yiqun; Wagner, Nicholas; Rondinelli, James M. MRS Communications, Vol. 9, Issue 3 https://doi.org/10.1557/mrc.2019.85	journal	June 2019

Similar Records

A characterization of workflow management systems for extreme-scale applications

Journal Article · Thu Feb 16 00:00:00 EST 2017 · Future Generations Computer Systems · OSTI ID:1265426

Ferreira da Silva, Rafael; Filgueira, Rosa; Pietri, Ilia; +3 more

Integrated End-to-end Performance Prediction and Diagnosis for Extreme Scientific Workflows

Technical Report · Fri May 19 00:00:00 EDT 2023 · OSTI ID:1265426

Tallent, Nathan; Friese, Ryan; Suetterlein, Josh; +17 more

Panorama 360 (Final Report)

Technical Report · Thu Jun 22 00:00:00 EDT 2023 · OSTI ID:1265426

Deelman, Ewa

Related Subjects

97 MATHEMATICS AND COMPUTING
performance modeling
extreme scale
scientific workflow

Title: PANORAMA: An approach to performance modeling and diagnosis of extreme-scale workflows

Citation Formats

References (57)

Cited By (2)

Similar Records

Related Subjects