Feature-based analysis of large-scale spatio-temporal sensor data on hybrid architectures

Saltz, Joel H.; Teodoro, George; Pan, Tony; Cooper, Lee A. D.; Kong, Jun; Klasky, Scott; Kurc, Tahsin M.

doi:10.1177/1094342013488260

Title: Feature-based analysis of large-scale spatio-temporal sensor data on hybrid architectures

Abstract

The analysis of large sensor datasets for structural and functional features has applications in many domains, including weather and climate modeling, characterization of subsurface reservoirs, and biomedicine. The vast amount of data obtained from state-of-the-art sensors and the computational cost of analysis operations create a barrier to such analyses. In this paper, we describe middleware system support to take advantage of large clusters of hybrid CPU–GPU nodes to address the data and compute-intensive requirements of feature-based analyses of large spatio-temporal datasets.

Authors:

Saltz, Joel H. ^[1]; Teodoro, George ^[1]; Pan, Tony ^[1]; Cooper, Lee A. D. ^[1]; Kong, Jun ^[1]; Klasky, Scott ^[2]; Kurc, Tahsin M. ^[3]

Emory Univ., Atlanta, GA (United States). Center for Comprehensive Informatics and Biomedical Informatics Dept.
Oak Ridge National Lab. (ORNL), Oak Ridge, TN (United States). Scientific Data Group
Emory Univ., Atlanta, GA (United States). Center for Comprehensive Informatics and Biomedical Informatics Dept.; Oak Ridge National Lab. (ORNL), Oak Ridge, TN (United States). Scientific Data Group

Publication Date:: Sun Jun 09 00:00:00 EDT 2013

Research Org.:: Oak Ridge National Laboratory (ORNL), Oak Ridge, TN (United States). Oak Ridge Leadership Computing Facility (OLCF)

Sponsoring Org.:: USDOE Office of Science (SC)

OSTI Identifier:: 1565093

Resource Type:: Accepted Manuscript

Journal Name:: International Journal of High Performance Computing Applications

Additional Journal Information:: Journal Volume: 27; Journal Issue: 3; Journal ID: ISSN 1094-3420

Publisher:: SAGE

Country of Publication:: United States

Language:: English

Subject:: 97 MATHEMATICS AND COMPUTING; Computer Science

Citation Formats


                    Saltz, Joel H., Teodoro, George, Pan, Tony, Cooper, Lee A. D., Kong, Jun, Klasky, Scott, and Kurc, Tahsin M. Feature-based analysis of large-scale spatio-temporal sensor data on hybrid architectures.  United States: N. p., 2013. 
Web.  doi:10.1177/1094342013488260.

Copy to clipboard


                    Saltz, Joel H., Teodoro, George, Pan, Tony, Cooper, Lee A. D., Kong, Jun, Klasky, Scott, & Kurc, Tahsin M. Feature-based analysis of large-scale spatio-temporal sensor data on hybrid architectures.  United States.  https://doi.org/10.1177/1094342013488260

Copy to clipboard


                    Saltz, Joel H., Teodoro, George, Pan, Tony, Cooper, Lee A. D., Kong, Jun, Klasky, Scott, and Kurc, Tahsin M. Sun .  
"Feature-based analysis of large-scale spatio-temporal sensor data on hybrid architectures".  United States.  https://doi.org/10.1177/1094342013488260.  https://www.osti.gov/servlets/purl/1565093.

Copy to clipboard


                    
@article{osti_1565093,

  title        = {Feature-based analysis of large-scale spatio-temporal sensor data on hybrid architectures},

  author       = {Saltz, Joel H. and Teodoro, George and Pan, Tony and Cooper, Lee A. D. and Kong, Jun and Klasky, Scott and Kurc, Tahsin M.},

  abstractNote = {The analysis of large sensor datasets for structural and functional features has applications in many domains, including weather and climate modeling, characterization of subsurface reservoirs, and biomedicine. The vast amount of data obtained from state-of-the-art sensors and the computational cost of analysis operations create a barrier to such analyses. In this paper, we describe middleware system support to take advantage of large clusters of hybrid CPU–GPU nodes to address the data and compute-intensive requirements of feature-based analyses of large spatio-temporal datasets.},

  doi          = {10.1177/1094342013488260},

  journal      = {International Journal of High Performance Computing Applications},

  number       = 3,

  volume       = 27,

  place        = {United States},

  year         = {Sun Jun 09 00:00:00 EDT 2013},

  month        = {Sun Jun 09 00:00:00 EDT 2013}

}

Copy to clipboard

Journal Article:

Free Publicly Available Full Text

Accepted Manuscript (DOE)

Publisher's Version of Record

https://doi.org/10.1177/1094342013488260

Other availability

Search WorldCat to find libraries that may hold this journal

Citation Metrics:

Cited by: 1 work

Citation information provided by
Web of Science

Save / Share:

Export Metadata

Save to My Library

Works referenced in this record:

Keeneland: Bringing Heterogeneous GPU Computing to the Computational Science Community
journal, September 2011

Vetter, Jeffrey S.; Glassbrook, Richard; Dongarra, Jack
Computing in Science & Engineering, Vol. 13, Issue 5
DOI: 10.1109/MCSE.2011.83

Morphological signatures and genomic correlates in glioblastoma
conference, March 2011

Cooper, Lee A. D.; Kong, Jun; Wang, Fusheng
2011 8th IEEE International Symposium on Biomedical Imaging (ISBI 2011), 2011 IEEE International Symposium on Biomedical Imaging: From Nano to Macro
DOI: 10.1109/ISBI.2011.5872714

DAGuE: A generic distributed DAG engine for High Performance Computing
journal, January 2012

Bosilca, George; Bouteiller, Aurelien; Danalis, Anthony
Parallel Computing, Vol. 38, Issue 1-2
DOI: 10.1016/j.parco.2011.10.003

Flexible IO and integration for scientific codes through the adaptable IO system (ADIOS)
conference, January 2008

Lofstead, Jay F.; Klasky, Scott; Schwan, Karsten
Proceedings of the 6th international workshop on Challenges of large applications in distributed environments - CLADE '08
DOI: 10.1145/1383529.1383533

High-throughput Analysis of Large Microscopy Image Datasets on CPU-GPU Cluster Platforms
conference, May 2013

Teodoro, George; Pan, Tony; Kurc, Tahsin M.
2013 IEEE International Symposium on Parallel & Distributed Processing (IPDPS), 2013 IEEE 27th International Symposium on Parallel and Distributed Processing
DOI: 10.1109/IPDPS.2013.11

MapReduce: a flexible data processing tool
journal, January 2010

Dean, Jeffrey; Ghemawat, Sanjay
Communications of the ACM, Vol. 53, Issue 1
DOI: 10.1145/1629175.1629198

Enabling Interoperation of High Performance, Scientific Computing Applications: Modeling Scientific Data with the Sets and Fields (SAF) Modeling System
book, January 2001

Miller, Mark C.; Reus, James F.; Matzke, Robb P.
Computational Science - ICCS 2001
DOI: 10.1007/3-540-45718-6_18

StarPU: a unified platform for task scheduling on heterogeneous multicore architectures
journal, November 2010

Augonnet, Cédric; Thibault, Samuel; Namyst, Raymond
Concurrency and Computation: Practice and Experience, Vol. 23, Issue 2
DOI: 10.1002/cpe.1631

Optimizing retrieval and processing of multi-dimensional scientific datasets
conference, January 2000

Chialin Chang, ; Kurc, T.; Sussman, A.
Proceedings 14th International Parallel and Distributed Processing Symposium. IPDPS 2000
DOI: 10.1109/IPDPS.2000.846013

Biomedical image analysis on a cooperative cluster of GPUs and multicores
conference, January 2008

Hartley, Timothy D. R.; Catalyurek, Umit; Ruiz, Antonio
Proceedings of the 22nd annual international conference on Supercomputing - ICS '08
DOI: 10.1145/1375527.1375533

Cluster I/O with River: making the fast case common
conference, January 1999

Arpaci-Dusseau, Remzi H.; Anderson, Eric; Treuhaft, Noah
Proceedings of the sixth workshop on I/O in parallel and distributed systems - IOPADS '99
DOI: 10.1145/301816.301823

An Integrative Approach for In Silico Glioma Research
journal, October 2010

Cooper, Lee A. D.; Gutman, David A.
IEEE Transactions on Biomedical Engineering, Vol. 57, Issue 10
DOI: 10.1109/TBME.2010.2060338

Distributed processing of very large datasets with DataCutter
journal, October 2001

Beynon, Michael D.; Kurc, Tahsin; Catalyurek, Umit
Parallel Computing, Vol. 27, Issue 11
DOI: 10.1016/S0167-8191(01)00099-0

A parallel software infrastructure for structured adaptive mesh methods
conference, January 1995

Kohn, Scott R.; Baden, Scott B.
Proceedings of the 1995 ACM/IEEE conference on Supercomputing (CDROM) - Supercomputing '95
DOI: 10.1145/224170.224283

A K-Means Cluster Analysis Computer Program With Cross-Tabulations and Next-Nearest-Neighbor Analysis
journal, April 1980

Lee, Howard B.; Macqueen, James B.
Educational and Psychological Measurement, Vol. 40, Issue 1
DOI: 10.1177/001316448004000118

Run-time optimizations for replicated dataflows on heterogeneous environments
conference, January 2010

Teodoro, George; Hartley, Timothy D. R.; Catalyurek, Umit
Proceedings of the 19th ACM International Symposium on High Performance Distributed Computing - HPDC '10
DOI: 10.1145/1851476.1851479

Accelerating Large Scale Image Analyses on Parallel, CPU-GPU Equipped Systems
conference, May 2012

Teodoro, George; Kurc, Tahsin M.; Pan, Tony
2012 IEEE International Symposium on Parallel & Distributed Processing (IPDPS), 2012 IEEE 26th International Parallel and Distributed Processing Symposium
DOI: 10.1109/IPDPS.2012.101

DataStager: scalable data staging services for petascale applications
journal, June 2010

Abbasi, Hasan; Wolf, Matthew; Eisenhauer, Greg
Cluster Computing, Vol. 13, Issue 3
DOI: 10.1007/s10586-010-0135-6

DataSpaces: an interaction and coordination framework for coupled simulation workflows
conference, January 2010

Docan, Ciprian; Parashar, Manish; Klasky, Scott
Proceedings of the 19th ACM International Symposium on High Performance Distributed Computing - HPDC '10
DOI: 10.1145/1851476.1851481

A scalable gaussian process analysis algorithm for biomass monitoring
journal, July 2011

Chandola, Varun; Vatsavai, Ranga Raju
Statistical Analysis and Data Mining, Vol. 4, Issue 4
DOI: 10.1002/sam.10129

Similar Records in DOE PAGES and OSTI.GOV collections:

Parallel Multivariate Spatio-Temporal Clustering of Large Ecological Datasets on Hybrid Supercomputers

Conference Sreepathi, Sarat ; Kumar, Jitendra ; Mills, Richard T. ; ...

A proliferation of data from vast networks of remote sensing platforms (satellites, unmanned aircraft systems (UAS), airborne etc.), observational facilities (meteorological, eddy covariance etc.), state-of-the-art sensors, and simulation models offer unprecedented opportunities for scientific discovery. Unsupervised classification is a widely applied data mining approach to derive insights from such data. However, classification of very large data sets is a complex computational problem that requires efficient numerical algorithms and implementations on high performance computing (HPC) platforms. Additionally, increasing power, space, cooling and efficiency requirements has led to the deployment of hybrid supercomputing platforms with complex architectures and memory hierarchies like themore »« less
https://doi.org/10.1109/CLUSTER.2017.88

Full Text Available
Towards Real-Time Detection and Tracking of Spatio-Temporal Features: Blob-Filaments in Fusion Plasma [Spatio-Temporal Features in Large Irregular Data: Blob-Filaments in Fusion Plasma]

Journal Article Wu, Lingfei ; Wu, Kesheng John ; Sim, Alex ; ... - IEEE Transactions on Big Data

This work was originally motivated the need to identify and track blob filaments, a feature often associated with instability in magnetically confined fusion plasma. Understanding and mitigating such instability would improve fusion reactors and make fusion a truly inexhaustible source of clean energy. Similar spatio-temporal features are important in many other applications, for example, ignition kernels in combustion and tumor cells in a medical image. This work presents an approach for extracting these features by dividing the overall task into three steps: local identification of feature cells, grouping feature cells into extended feature, and tracking movement of feature through overlappingmore »« less
https://doi.org/10.1109/TBDATA.2016.2599929

Full Text Available
The Anatomy of Mr. Scan: A Dissection of Performance of an Extreme Scale GPU-Based Clustering Algorithm, In: 2014 5th Workshop on Latest Advances in Scalable Algorithms for Large-Scale Systems

Conference Welton, Benjamin ; Miller, Barton P. - 2014 5TH WORKSHOP ON LATEST ADVANCES IN SCALABLE ALGORITHMS FOR LARGE-SCALE SYSTEMS (SCALA)

The emergence of leadership class systems with GPU-equipped nodes has the potential to vastly increase the performance of existing distributed applications. However, the inclusion of GPU computation into existing extreme scale distributed applications can reveal scalability issues that were absent in the CPU version. The issues exposed in scaling by a GPU can become limiting factors to overall application performance. We developed an extreme scale GPU-based application to perform data clustering on multi-billion point datasets. In this application, called Mr. Scan, we ran into several of these performance limiting issues. Through the use of complete end-to-end benchmarking of Mr. Scanmore »« less
https://doi.org/10.1109/ScalA.2014.10
Revisiting Huffman Coding: Toward Extreme Performance on Modern GPU Architectures

Conference Tian, Jiannan ; Rivera, Cody ; Di, Sheng ; ...

Today's high-performance computing (HPC) applications are producing vast volumes of data, which are challenging to store and transfer efficiently during the execution, such that data compression is becoming a critical technique to mitigate the storage burden and data movement cost. Huffman coding is arguably the most efficient Entropy coding algorithm in information theory, such that it could be found as a fundamental step in many modern compression algorithms such as DEFLATE. On the other hand, today's HPC applications are more and more relying on the accelerators such as GPU on supercomputers, while Huffman encoding suffers from low throughput on GPUs,more »« less
Full Text Available
Revisiting Huffman Coding: Toward Extreme Performance on Modern GPU Architectures

Conference Tian, Jiannan ; Rivera, Cody ; Di, Sheng ; ...

Today's high-performance computing (HPC) applications are producing vast volumes of data, which are challenging to store and transfer efficiently during the execution, such that data compression is becoming a critical technique to mitigate the storage burden and data movement cost. Huffman coding is arguably the most efficient Entropy coding algorithm in information theory, such that it could be found as a fundamental step in many modern compression algorithms such as DEFLATE. On the other hand, today's HPC applications are more and more relying on the accelerators such as GPU on supercomputers, while Huffman encoding suffers from low throughput on GPUs,more »« less
https://doi.org/10.1109/IPDPS49936.2021.00097

Similar Records

Title: Feature-based analysis of large-scale spatio-temporal sensor data on hybrid architectures

Abstract

Citation Formats

Keeneland: Bringing Heterogeneous GPU Computing to the Computational Science Community journal, September 2011

Morphological signatures and genomic correlates in glioblastoma conference, March 2011

DAGuE: A generic distributed DAG engine for High Performance Computing journal, January 2012

Flexible IO and integration for scientific codes through the adaptable IO system (ADIOS) conference, January 2008

High-throughput Analysis of Large Microscopy Image Datasets on CPU-GPU Cluster Platforms conference, May 2013

MapReduce: a flexible data processing tool journal, January 2010

Enabling Interoperation of High Performance, Scientific Computing Applications: Modeling Scientific Data with the Sets and Fields (SAF) Modeling System book, January 2001

StarPU: a unified platform for task scheduling on heterogeneous multicore architectures journal, November 2010

Optimizing retrieval and processing of multi-dimensional scientific datasets conference, January 2000

Biomedical image analysis on a cooperative cluster of GPUs and multicores conference, January 2008

Cluster I/O with River: making the fast case common conference, January 1999

An Integrative Approach for In Silico Glioma Research journal, October 2010

Distributed processing of very large datasets with DataCutter journal, October 2001

A parallel software infrastructure for structured adaptive mesh methods conference, January 1995

A K-Means Cluster Analysis Computer Program With Cross-Tabulations and Next-Nearest-Neighbor Analysis journal, April 1980

Run-time optimizations for replicated dataflows on heterogeneous environments conference, January 2010

Accelerating Large Scale Image Analyses on Parallel, CPU-GPU Equipped Systems conference, May 2012

DataStager: scalable data staging services for petascale applications journal, June 2010

DataSpaces: an interaction and coordination framework for coupled simulation workflows conference, January 2010

A scalable gaussian process analysis algorithm for biomass monitoring journal, July 2011

Keeneland: Bringing Heterogeneous GPU Computing to the Computational Science Community
journal, September 2011

Morphological signatures and genomic correlates in glioblastoma
conference, March 2011

DAGuE: A generic distributed DAG engine for High Performance Computing
journal, January 2012

Flexible IO and integration for scientific codes through the adaptable IO system (ADIOS)
conference, January 2008

High-throughput Analysis of Large Microscopy Image Datasets on CPU-GPU Cluster Platforms
conference, May 2013

MapReduce: a flexible data processing tool
journal, January 2010

Enabling Interoperation of High Performance, Scientific Computing Applications: Modeling Scientific Data with the Sets and Fields (SAF) Modeling System
book, January 2001

StarPU: a unified platform for task scheduling on heterogeneous multicore architectures
journal, November 2010

Optimizing retrieval and processing of multi-dimensional scientific datasets
conference, January 2000

Biomedical image analysis on a cooperative cluster of GPUs and multicores
conference, January 2008

Cluster I/O with River: making the fast case common
conference, January 1999

An Integrative Approach for In Silico Glioma Research
journal, October 2010

Distributed processing of very large datasets with DataCutter
journal, October 2001

A parallel software infrastructure for structured adaptive mesh methods
conference, January 1995

A K-Means Cluster Analysis Computer Program With Cross-Tabulations and Next-Nearest-Neighbor Analysis
journal, April 1980

Run-time optimizations for replicated dataflows on heterogeneous environments
conference, January 2010

Accelerating Large Scale Image Analyses on Parallel, CPU-GPU Equipped Systems
conference, May 2012

DataStager: scalable data staging services for petascale applications
journal, June 2010

DataSpaces: an interaction and coordination framework for coupled simulation workflows
conference, January 2010

A scalable gaussian process analysis algorithm for biomass monitoring
journal, July 2011