In January 2019, the US Department of Energy, Office of Science program in Advanced Scientific Computing Research, convened a workshop to identify priority research directions (PRDs) for in situ data management (ISDM). A fundamental finding of this workshop is that the methodologies used to manage data among a variety of tasks in situ can be used to facilitate scientific discovery from many different data sources—simulation, experiment, and sensors, for example—and that being able to do so at numerous computing scales will benefit real-time decision-making, design optimization, and data-driven scientific discovery. This article describes six PRDs identified by the workshop, which highlight the components and capabilities needed for ISDM to be successful for a wide variety of applications—making ISDM capabilities more pervasive, controllable, composable, and transparent, with a focus on greater coordination with the software stack and a diversity of fundamentally new data algorithms.
Peterka, Tom, et al. "Priority research directions for in situ data management: Enabling scientific discovery from diverse data sources." International Journal of High Performance Computing Applications, Mar. 2020. https://doi.org/10.1177/1094342020913628
Peterka, Tom, Bard, Deborah, Bennett, Janine C., Bethel, E. Wes, Oldfield, Ron A., Pouchard, Line, Sweeney, Christine, & Wolf, Matthew (2020). Priority research directions for in situ data management: Enabling scientific discovery from diverse data sources. International Journal of High Performance Computing Applications. https://doi.org/10.1177/1094342020913628
Peterka, Tom, Bard, Deborah, Bennett, Janine C., et al., "Priority research directions for in situ data management: Enabling scientific discovery from diverse data sources," International Journal of High Performance Computing Applications (2020), https://doi.org/10.1177/1094342020913628
@article{osti_1606603,
author = {Peterka, Tom and Bard, Deborah and Bennett, Janine C. and Bethel, E. Wes and Oldfield, Ron A. and Pouchard, Line and Sweeney, Christine and Wolf, Matthew},
title = {Priority research directions for in situ data management: Enabling scientific discovery from diverse data sources},
annote = {In January 2019, the US Department of Energy, Office of Science program in Advanced Scientific Computing Research, convened a workshop to identify priority research directions (PRDs) for in situ data management (ISDM). A fundamental finding of this workshop is that the methodologies used to manage data among a variety of tasks in situ can be used to facilitate scientific discovery from many different data sources—simulation, experiment, and sensors, for example—and that being able to do so at numerous computing scales will benefit real-time decision-making, design optimization, and data-driven scientific discovery. This article describes six PRDs identified by the workshop, which highlight the components and capabilities needed for ISDM to be successful for a wide variety of applications—making ISDM capabilities more pervasive, controllable, composable, and transparent, with a focus on greater coordination with the software stack and a diversity of fundamentally new data algorithms.},
doi = {10.1177/1094342020913628},
url = {https://www.osti.gov/biblio/1606603},
journal = {International Journal of High Performance Computing Applications},
issn = {ISSN 1094-3420},
place = {United States},
publisher = {SAGE},
year = {2020},
month = {03}}
Brookhaven National Laboratory (BNL), Upton, NY (United States); Lawrence Berkeley National Laboratory (LBNL), Berkeley, CA (United States); Los Alamos National Laboratory (LANL), Los Alamos, NM (United States); Oak Ridge National Laboratory (ORNL), Oak Ridge, TN (United States); Sandia National Laboratories (SNL-CA), Livermore, CA (United States); Sandia National Laboratories (SNL-NM), Albuquerque, NM (United States)
Sponsoring Organization:
USDOE National Nuclear Security Administration (NNSA); USDOE Office of Science (SC), Advanced Scientific Computing Research (ASCR)
International Journal of High Performance Computing Applications, Journal Name: International Journal of High Performance Computing Applications; ISSN 1094-3420
2012 IEEE International Symposium on Parallel & Distributed Processing (IPDPS), 2012 IEEE 26th International Parallel and Distributed Processing Symposiumhttps://doi.org/10.1109/IPDPS.2012.52
Proceedings of the 6th international workshop on Challenges of large applications in distributed environments - CLADE '08https://doi.org/10.1145/1383529.1383533
Gamblin, Todd; LeGendre, Matthew; Collette, Michael R.
Proceedings of the International Conference for High Performance Computing, Networking, Storage and Analysis on - SC '15https://doi.org/10.1145/2807591.2807623
Proceedings of the International Conference for High Performance Computing, Networking, Storage and Analysis on - SC '15https://doi.org/10.1145/2807591.2807650
Proceedings of the First Workshop on In Situ Infrastructures for Enabling Extreme-Scale Analysis and Visualization - ISAV2015https://doi.org/10.1145/2828612.2828619
Proceedings of the First Workshop on In Situ Infrastructures for Enabling Extreme-Scale Analysis and Visualization - ISAV2015https://doi.org/10.1145/2828612.2828624
Proceedings of the 2nd Joint International Workshop on Parallel Data Storage & Data Intensive Scalable Computing Systems - PDSW-DISCS '17https://doi.org/10.1145/3149393.3149396
CCS '18: 2018 ACM SIGSAC Conference on Computer and Communications Security, Proceedings of the 2018 ACM SIGSAC Conference on Computer and Communications Securityhttps://doi.org/10.1145/3243734.3243776
Proceedings of the Workshop on In Situ Infrastructures for Enabling Extreme-Scale Analysis and Visualization - ISAV '18https://doi.org/10.1145/3281464.3281467
Proceedings of the Workshop on In Situ Infrastructures for Enabling Extreme-Scale Analysis and Visualization - ISAV '18https://doi.org/10.1145/3281464.3281468