AnalyzeThis: an analysis workflow-aware storage system
- Virginia Tech
- Oak Ridge National Laboratory
- Lawrence Berkeley Laboratory
The need for novel data analysis is urgent in the face of a data deluge from modern applications. Traditional approaches to data analysis incur significant data movement costs, moving data back and forth between the storage system and the processor. Emerging Active Flash devices en-able processing on the flash, where the data already resides. An array of such Active Flash devices allows us to revisit how analysis workflows interact with storage systems. By seamlessly blending together the flash storage and data analysis, we create an analysis workflow-aware storage system, AnalyzeThis. Our guiding principle is that analysis-awareness be deeply ingrained in each and every layer of the storage, elevating data analyses as first-class citizens, and transforming AnalyzeThis into a potent analytics-aware appliance. We implement the AnalyzeThis storage system atop an emulation platform of the Active Flash array. Our results indicate that AnalyzeThis is viable, expediting workflow execution and minimizing data movement.
- Research Organization:
- Oak Ridge National Laboratory (ORNL), Oak Ridge, TN (United States). Oak Ridge Leadership Computing Facility (OLCF)
- Sponsoring Organization:
- USDOE Office of Science (SC)
- DOE Contract Number:
- AC05-00OR22725; AC02-05CH11231
- OSTI ID:
- 1567399
- Journal Information:
- Proceedings of SC15: The International Conference for High Performance Computing, Networking, Storage and Analysis, Conference: International Conference for High Performance Computing, Networking, Storage and Analysis, Austin, Texas, November 15-20, 2015
- Country of Publication:
- United States
- Language:
- English
Similar Records
AnalyzeThis: An Analysis Workflow-Aware Storage System
An Integrated Indexing and Search Service for Distributed File Systems