Toward a Data Scalable Solution for Facilitating Discovery of Scientific Data Resources

Conference · Mon Nov 18 00:00:00 EST 2013

DOI:https://doi.org/10.1145/2534645.2534655· OSTI ID:1123247

Chappell, Alan R.; Choudhury, Sutanay; Feo, John T.; Haglin, David J.; Morari, Alessandro; Purohit, Sumit; Schuchardt, Karen L.; Tumeo, Antonino; Weaver, Jesse R.; Villa, Oreste

Science is increasingly motivated by the need to process larger quantities of data. It is facing severe challenges in data collection, management, and processing, so much so that the computational demands of "data scaling" are competing with, and in many fields surpassing, the traditional objective of decreasing processing time. Example domains with large datasets include astronomy, biology, genomic, climate and weather, and material sciences. This paper presents a real-world use case in which we wish to answer queries provided by domain scientists in order to facilitate discovery of relevant science resources. The problem is that the metadata for these science resources is very large and is growing quickly, rapidly increasing the need for a data scaling solution. We propose the use of our SGEM stack -- a system designed for answering graph-based queries over large datasets on cluster architectures -- for answering complex queries over the metadata, and we report early results for our current capability.

• Research Organization: Pacific Northwest National Lab. (PNNL), Richland, WA (United States)
• Sponsoring Organization: USDOE
• DOE Contract Number: AC05-76RL01830
• OSTI ID: 1123247
• Report Number(s): PNNL-SA-98169; 400470000
• Resource Relation: Conference: DISCS-2013: Proceedings of the International Workshop on Data-Intensive Scalable Computing Systems, November 18, 2013, Denver, CO, 55-60
• Country of Publication: United States
• Language: English