Skip to main content
OSTI.GOVU.S. Department of Energy Office of Scientific and Technical Information
U.S. Department of Energy
Office of Scientific and Technical Information
 

Workflows Community Summit 2022: A Roadmap Revolution

Technical Report ·
DOI:https://doi.org/10.2172/2006942· OSTI ID:2006942
 [1];  [2];  [3];  [4];  [4];  [3];  [5];  [6];  [7];  [8];  [9];  [10];  [4];  [4];  [11];  [12];  [13];  [14];  [15];  [16] more »;  [17];  [18];  [19];  [20];  [21];  [22];  [23];  [2];  [24];  [25];  [26];  [27];  [28];  [29];  [3];  [2];  [30];  [10];  [31];  [4];  [15];  [32];  [16];  [33];  [34];  [4];  [35];  [36];  [37];  [2];  [38];  [39];  [40];  [9];  [4];  [41];  [6];  [4];  [42];  [43];  [44];  [42];  [45];  [46];  [4];  [47];  [48];  [49];  [50];  [47];  [51];  [10];  [8];  [52];  [3];  [11];  [16];  [26];  [53];  [54];  [55];  [56];  [57];  [58];  [26];  [2];  [4];  [59];  [4];  [60];  [61];  [62];  [63];  [8];  [63];  [4];  [64];  [65];  [8];  [66];  [4];  [4];  [4];  [55];  [39] « less
  1. Oak Ridge National Laboratory (ORNL), Oak Ridge, TN (United States)
  2. Barcelona Supercomputing Center
  3. Agnostiq
  4. ORNL
  5. University of Tennessee at Knoxville
  6. University of Chicago
  7. University of Manchester University, UK
  8. Brookhaven National Laboratory (BNL)
  9. University of Illinois at Urbana-Champaign
  10. Lawrence Livermore National Laboratory (LLNL)
  11. Lawrence Berkeley National Laboratory (LBNL)
  12. Argonne National Laboratory
  13. University of California, San Diego
  14. SINTEF
  15. National Energy Research Scientific Computing Center (NERSC), California
  16. University of Innsbruck, Austria
  17. Technical University of Berlin (TU Berlin, Technische Universitat Berlin), Germany
  18. AGH University of Science and Technology, Krakow, Poland
  19. NOAA Geophysical Fluid Dynamics Laboratory (GFDL), Princeton, NJ
  20. University of Sao Paulo, Brazil
  21. Entangled Networks
  22. University of Georgia
  23. University of Hawaii at Manoa, Honolulu
  24. Egypt Japan University of Science and Technology
  25. University of Southern California, Los Angeles
  26. Argonne National Laboratory (ANL)
  27. Universita degli Studi di Torino, Torino, Italy
  28. Ghent University, Belgium
  29. Vrie Universiteit Amsterdam
  30. Seqera Labs
  31. NCI Australia
  32. American Geophysical Union
  33. German Centre for Integrative Biodiversity Research (iDiv), Halle-Jena-Leipzig, Leipzig, Germany
  34. University of St Andrews, UK
  35. Simula Research Laboratory
  36. University of Jena
  37. GE Research
  38. Universidad Politecnica de Madrid, Spain
  39. Friedrich Schiller University Jena, Germany
  40. Sandia National Laboratories (SNL)
  41. University of Melbourne, Australia
  42. Humboldt-Universitat zu Berlin
  43. Hewlett Packard Enterprise
  44. National Oceanic and Atmospheric Administration (NOAA)
  45. Ludwig-Maximilian University Munich
  46. University of Tennessee, Knoxville (UTK)
  47. RIKEN, Japan
  48. University of Turin, Italy
  49. HPE
  50. Sano Centre for Personalized Computational Medicine
  51. University of Science and Technology AGH
  52. ParTec AG
  53. INRIA
  54. Leibniz Supercomputing Centre
  55. Cyfronet AGH
  56. Utrecht University, Netherlands
  57. Ecole Polytechnique Federale de Lausanne (EPFL), Switzerland
  58. Institut fur Informatik, Technical University of Munich
  59. The University of Manchester
  60. Cray, Inc.
  61. George Mason University, Virginia
  62. University of Maryland, College Park
  63. University of Notre Dame, IN
  64. Rutgers University
  65. University of Warsaw, Poland
  66. Universidad de los Andes, Colombia
+ Show Author Affiliations

Scientific workflows have become integral tools in broad scientific computing use cases. Science discovery is increasingly dependent on workflows to orchestrate large and complex scientific experiments that range from the execution of a cloud-based data preprocessing pipeline to multi-facility instrument-to-edge-to-HPC computational workflows. Given the changing landscape of scientific computing (often referred to as a computing continuum) and the evolving needs of emerging scientific applications, it is paramount that the development of novel scientific workflows and system functionalities seek to increase the efficiency, resilience, and pervasiveness of existing systems and applications. Specifically, the proliferation of machine learning/artificial intelligence (ML/AI) workflows, need for processing large-scale datasets produced by instruments at the edge, intensification of near real-time data processing, support for long-term experiment campaigns, and emergence of quantum computing as an adjunct to HPC, have significantly changed the functional and operational requirements of workflow systems. Workflow systems now need to, for example, support data streams from the edge-to-cloud-to-HPC, enable the management of many small-sized files, allow data reduction while ensuring high accuracy, orchestrate distributed services (workflows, instruments, data movement, provenance, publication, etc.) across computing and user facilities, among others. Further, to accelerate science, it is also necessary that these systems implement specifications/standards and APIs for seamless (horizontal and vertical) integration between systems and applications, as well as enable the publication of workflows and their associated products according to the FAIR principles.

Research Organization:
Oak Ridge National Laboratory (ORNL), Oak Ridge, TN (United States)
Sponsoring Organization:
USDOE Office of Science (SC), Advanced Scientific Computing Research (ASCR)
DOE Contract Number:
AC05-00OR22725
OSTI ID:
2006942
Report Number(s):
ORNL/TM--2023/2885
Country of Publication:
United States
Language:
English

Similar Records

Enabling HPC Scientific Workflows for Serverless
Conference · Fri Nov 01 00:00:00 EDT 2024 · OSTI ID:2538241
NREL Stratus - Enabling Workflows to Fuse Data Streams, Modeling, Simulation, and Machine Learning
Conference · Tue Sep 27 00:00:00 EDT 2022 · OSTI ID:1890739
28 NREL Stratus - Enabling Workflows to Fuse Data Streams, Modeling, Simulation, and Machine Learning: Preprint
Conference · Thu Nov 17 23:00:00 EST 2022 · OSTI ID:1899966

Related Subjects

97 MATHEMATICS AND COMPUTING