Machine learning (ML) is being applied in a number of everyday contexts from image recognition, to natural language processing, to autonomous vehicles, to product recommendation. In the science realm, ML is being used for medical diagnosis, new materials development, smart agriculture, DNA classification, and many others. In this article, we describe the opportunities of using ML in the area of scientific workflow management. Scientific workflows are key to today’s computational science, enabling the definition and execution of complex applications in heterogeneous and often distributed environments. We describe the challenges of composing and executing scientific workflows and identify opportunities for applying ML techniques to meet these challenges by enhancing the current workflow management system capabilities. We foresee that as the ML field progresses, the automation provided by workflow management systems will greatly increase and result in significant improvements in scientific productivity.
Deelman, Ewa, et al. "The role of machine learning in scientific workflows." International Journal of High Performance Computing Applications, vol. 33, no. 6, May. 2019. https://doi.org/10.1177/1094342019852127
Deelman, Ewa, Mandal, Anirban, Jiang, Ming, & Sakellariou, Rizos (2019). The role of machine learning in scientific workflows. International Journal of High Performance Computing Applications, 33(6). https://doi.org/10.1177/1094342019852127
Deelman, Ewa, Mandal, Anirban, Jiang, Ming, et al., "The role of machine learning in scientific workflows," International Journal of High Performance Computing Applications 33, no. 6 (2019), https://doi.org/10.1177/1094342019852127
@article{osti_1523666,
author = {Deelman, Ewa and Mandal, Anirban and Jiang, Ming and Sakellariou, Rizos},
title = {The role of machine learning in scientific workflows},
annote = {Machine learning (ML) is being applied in a number of everyday contexts from image recognition, to natural language processing, to autonomous vehicles, to product recommendation. In the science realm, ML is being used for medical diagnosis, new materials development, smart agriculture, DNA classification, and many others. In this article, we describe the opportunities of using ML in the area of scientific workflow management. Scientific workflows are key to today’s computational science, enabling the definition and execution of complex applications in heterogeneous and often distributed environments. We describe the challenges of composing and executing scientific workflows and identify opportunities for applying ML techniques to meet these challenges by enhancing the current workflow management system capabilities. We foresee that as the ML field progresses, the automation provided by workflow management systems will greatly increase and result in significant improvements in scientific productivity.},
doi = {10.1177/1094342019852127},
url = {https://www.osti.gov/biblio/1523666},
journal = {International Journal of High Performance Computing Applications},
issn = {ISSN 1094-3420},
number = {6},
volume = {33},
place = {United States},
publisher = {SAGE},
year = {2019},
month = {05}}
USDOE National Nuclear Security Administration (NNSA); National Science Foundation (NSF)
Grant/Contract Number:
SC0012636
OSTI ID:
1523666
Journal Information:
International Journal of High Performance Computing Applications, Journal Name: International Journal of High Performance Computing Applications Journal Issue: 6 Vol. 33; ISSN 1094-3420
Philosophical Transactions of the Royal Society A: Mathematical, Physical and Engineering Sciences, Vol. 374, Issue 2065https://doi.org/10.1098/rsta.2015.0202
Zhang, Olive Qing; Kirchberg, Markus; Ko, Ryan K. L.
2011 IEEE 3rd International Conference on Cloud Computing Technology and Science (CloudCom), 2011 IEEE Third International Conference on Cloud Computing Technology and Sciencehttps://doi.org/10.1109/CloudCom.2011.66
Durillo, Juan J.; Fard, Hamid Mohammadi; Prodan, Radu
2012 IEEE 4th International Conference on Cloud Computing Technology and Science (CloudCom), 4th IEEE International Conference on Cloud Computing Technology and Science Proceedingshttps://doi.org/10.1109/CloudCom.2012.6427573
Bhaduri, Kanishka; Das, Kamalika; Matthews, Bryan L.
2011 IEEE International Conference on Data Mining Workshops (ICDMW), 2011 IEEE 11th International Conference on Data Mining Workshopshttps://doi.org/10.1109/ICDMW.2011.62
2014 43nd International Conference on Parallel Processing Workshops (ICCPW), 2014 43rd International Conference on Parallel Processing Workshopshttps://doi.org/10.1109/ICPPW.2014.26
Nemirovsky, Daniel; Arkose, Tugberk; Markovic, Nikola
2017 29th International Symposium on Computer Architecture and High-Performance Computing (SBAC-PAD), 2017 29th International Symposium on Computer Architecture and High Performance Computing (SBAC-PAD)https://doi.org/10.1109/SBAC-PAD.2017.23
2012 SC - International Conference for High Performance Computing, Networking, Storage and Analysis, 2012 International Conference for High Performance Computing, Networking, Storage and Analysishttps://doi.org/10.1109/SC.2012.20
Jain, Nikhil; Bhatele, Abhinav; Robson, Michael P.
Proceedings of the International Conference for High Performance Computing, Networking, Storage and Analysis on - SC '13https://doi.org/10.1145/2503210.2503263
Weitzel, Derek; Bockelman, Brian; Brown, Duncan A.
Proceedings of the Practice and Experience in Advanced Research Computing 2017 on Sustainability, Success and Impact - PEARC17https://doi.org/10.1145/3093338.3093363