The Evolution of the Pegasus Workflow Management Software

Journal Article · Wed May 29 00:00:00 EDT 2019 · Computing in Science and Engineering

DOI:https://doi.org/10.1109/MCSE.2019.2919690· OSTI ID:1600092

Deelman, Ewa ^[1]; Vahi, Karan ^[2]; Rynge, Mats ^[2]; Mayani, Rajiv ^[2]; da Silva, Rafael Ferreira ^[2]; Papadimitriou, George ^[2]; Livny, Miron ^[3]

---

1. Univ. of Southern California, Los Angeles, CA (United States); USC Information Sciences Institute
2. Univ. of Southern California, Los Angeles, CA (United States)
3. Univ. of Wisconsin, Madison, WI (United States)

Since 2001 the Pegasus Workflow Management System has evolved into a robust and scalable system that automates the execution of a number of complex applications running on a variety of heterogeneous, distributed high-throughput, and high-performance computing environments. Pegasus was built on the principle of separation between the workflow description and workflow execution, providing the ability to port and adapt the workflow based on the target execution environment. Through its user-driven research and development it has adapted to the needs of a number of scientific communities, utilizing and developing novel algorithms and software engineering solutions. This paper describes the evolution of Pegasus over time and provides motivations behind the design decisions. The paper concludes with selected lessons learned.

• Research Organization: Univ. of Southern California, Los Angeles, CA (United States)
• Sponsoring Organization: USDOE Office of Science (SC), Advanced Scientific Computing Research (ASCR) (SC-21)
• Grant/Contract Number: SC0012636
• OSTI ID: 1600092
• Journal Information: Computing in Science and Engineering, Journal Name: Computing in Science and Engineering Journal Issue: 4 Vol. 21; ISSN 1521-9615
• Publisher: IEEE
• Country of Publication: United States
• Language: English

Related Subjects

97 MATHEMATICS AND COMPUTING