Data Intensive Computing on Amazon Web Services
- Lawrence Livermore National Lab. (LLNL), Livermore, CA (United States)
The Geophysical Monitoring Program (GMP) has spent the past few years building up the capability to perform data intensive computing using what have been referred to as “big data” tools. These big data tools would be used against massive archives of seismic signals (>300 TB) to conduct research not previously possible. Examples of such tools include Hadoop (HDFS, MapReduce), HBase, Hive, Storm, Spark, Solr, and many more by the day. These tools are useful for performing data analytics on datasets that exceed the resources of traditional analytic approaches. To this end, a research big data cluster (“Cluster A”) was set up as a collaboration between GMP and Livermore Computing (LC).
- Research Organization:
- Lawrence Livermore National Lab. (LLNL), Livermore, CA (United States)
- Sponsoring Organization:
- USDOE
- DOE Contract Number:
- AC52-07NA27344
- OSTI ID:
- 1378527
- Report Number(s):
- LLNL-TR-670186
- Country of Publication:
- United States
- Language:
- English
Similar Records
Deactivation and decommissioning web log analysis using big data technology - 15710
Large-scale seismic waveform quality metric calculation using Hadoop
Center for Technology for Advanced Scientific Componet Software (TASCS)
Conference
·
Wed Jul 01 00:00:00 EDT 2015
·
OSTI ID:1378527
Large-scale seismic waveform quality metric calculation using Hadoop
Journal Article
·
Fri May 27 00:00:00 EDT 2016
· Computers and Geosciences
·
OSTI ID:1378527
+2 more
Center for Technology for Advanced Scientific Componet Software (TASCS)
Technical Report
·
Sun Oct 31 00:00:00 EDT 2010
·
OSTI ID:1378527