OpenMP Parallelization and Optimization of Graph-Based Machine Learning Algorithms

Meng, Zhaoyi; Koniges, Alice; He, Yun Helen; Williams, Samuel; Kurth, Thorsten; Cook, Brandon; Deslippe, Jack; Bertozzi, Andrea L.

doi:10.1007/978-3-319-45550-1_2

Title: OpenMP Parallelization and Optimization of Graph-Based Machine Learning Algorithms

Abstract

In this paper, we investigate the OpenMP parallelization and optimization of two novel data classification algorithms. The new algorithms are based on graph and PDE solution techniques and provide significant accuracy and performance advantages over traditional data classification algorithms in serial mode. The methods leverage the Nystrom extension to calculate eigenvalue/eigenvectors of the graph Laplacian and this is a self-contained module that can be used in conjunction with other graph-Laplacian based methods such as spectral clustering. We use performance tools to collect the hotspots and memory access of the serial codes and use OpenMP as the parallelization language to parallelize the most time-consuming parts. Where possible, we also use library routines. We then optimize the OpenMP implementations and detail the performance on traditional supercomputer nodes (in our case a Cray XC30), and test the optimization steps on emerging testbed systems based on Intel’s Knights Corner and Landing processors. We show both performance improvement and strong scaling behavior. Finally, a large number of optimization techniques and analyses are necessary before the algorithm reaches almost ideal scaling.

Authors:

Meng, Zhaoyi ^[1]; Koniges, Alice ^[2]; He, Yun Helen ^[2]; Williams, Samuel ^[2]; Kurth, Thorsten ^[2]; Cook, Brandon ^[2]; Deslippe, Jack ^[2]; Bertozzi, Andrea L. ^[3]

Univ. of California, Los Angeles, CA (United States); Lawrence Berkeley National Lab. (LBNL), Berkeley, CA (United States)
Lawrence Berkeley National Lab. (LBNL), Berkeley, CA (United States)
Univ. of California, Los Angeles, CA (United States)

Publication Date:: Wed Sep 21 00:00:00 EDT 2016

Research Org.:: Lawrence Berkeley National Laboratory (LBNL), Berkeley, CA (United States)

Sponsoring Org.:: USDOE Office of Science (SC), Advanced Scientific Computing Research (ASCR); National Science Foundation (NSF); US Air Force Office of Scientific Research (AFOSR)

OSTI Identifier:: 1378982

Grant/Contract Number:: AC02-05CH11231; DMS-1417674; DMS-1045536; FA9550-10-1-0569

Resource Type:: Accepted Manuscript

Journal Name:: Lecture Notes in Computer Science

Additional Journal Information:: Journal Volume: 9903; Journal ID: ISSN 0302-9743

Publisher:: Springer

Country of Publication:: United States

Language:: English

Subject:: 97 MATHEMATICS AND COMPUTING; semi-supervised; unsupervised; data; algorithms; OpenMP; optimization

Citation Formats


                    Meng, Zhaoyi, Koniges, Alice, He, Yun Helen, Williams, Samuel, Kurth, Thorsten, Cook, Brandon, Deslippe, Jack, and Bertozzi, Andrea L. OpenMP Parallelization and Optimization of Graph-Based Machine Learning Algorithms.  United States: N. p., 2016. 
Web.  doi:10.1007/978-3-319-45550-1_2.

Copy to clipboard


                    Meng, Zhaoyi, Koniges, Alice, He, Yun Helen, Williams, Samuel, Kurth, Thorsten, Cook, Brandon, Deslippe, Jack, & Bertozzi, Andrea L. OpenMP Parallelization and Optimization of Graph-Based Machine Learning Algorithms.  United States.  https://doi.org/10.1007/978-3-319-45550-1_2

Copy to clipboard


                    Meng, Zhaoyi, Koniges, Alice, He, Yun Helen, Williams, Samuel, Kurth, Thorsten, Cook, Brandon, Deslippe, Jack, and Bertozzi, Andrea L. Wed .  
"OpenMP Parallelization and Optimization of Graph-Based Machine Learning Algorithms".  United States.  https://doi.org/10.1007/978-3-319-45550-1_2.  https://www.osti.gov/servlets/purl/1378982.

Copy to clipboard


                    
@article{osti_1378982,

  title        = {OpenMP Parallelization and Optimization of Graph-Based Machine Learning Algorithms},

  author       = {Meng, Zhaoyi and Koniges, Alice and He, Yun Helen and Williams, Samuel and Kurth, Thorsten and Cook, Brandon and Deslippe, Jack and Bertozzi, Andrea L.},

  abstractNote = {In this paper, we investigate the OpenMP parallelization and optimization of two novel data classification algorithms. The new algorithms are based on graph and PDE solution techniques and provide significant accuracy and performance advantages over traditional data classification algorithms in serial mode. The methods leverage the Nystrom extension to calculate eigenvalue/eigenvectors of the graph Laplacian and this is a self-contained module that can be used in conjunction with other graph-Laplacian based methods such as spectral clustering. We use performance tools to collect the hotspots and memory access of the serial codes and use OpenMP as the parallelization language to parallelize the most time-consuming parts. Where possible, we also use library routines. We then optimize the OpenMP implementations and detail the performance on traditional supercomputer nodes (in our case a Cray XC30), and test the optimization steps on emerging testbed systems based on Intel’s Knights Corner and Landing processors. We show both performance improvement and strong scaling behavior. Finally, a large number of optimization techniques and analyses are necessary before the algorithm reaches almost ideal scaling.},

  doi          = {10.1007/978-3-319-45550-1_2},

  journal      = {Lecture Notes in Computer Science},

  number       = ,

  volume       = 9903,

  place        = {United States},

  year         = {Wed Sep 21 00:00:00 EDT 2016},

  month        = {Wed Sep 21 00:00:00 EDT 2016}

}

Copy to clipboard

Journal Article:

Free Publicly Available Full Text

Accepted Manuscript (DOE)

Publisher's Version of Record

https://doi.org/10.1007/978-3-319-45550-1_2

Other availability

Search WorldCat to find libraries that may hold this journal

Citation Metrics:

Cited by: 5 works

Citation information provided by
Web of Science

Save / Share:

Export Metadata

Save to My Library

Works referenced in this record:

Multi-class Graph Mumford-Shah Model for Plume Detection Using the MBO scheme
book, January 2015

Hu, Huiyi; Sunu, Justin; Bertozzi, Andrea L.
Lecture Notes in Computer Science
DOI: 10.1007/978-3-319-14612-6_16

Diffuse Interface Models on Graphs for Classification of High Dimensional Data
journal, January 2012

Bertozzi, Andrea L.; Flenner, Arjuna
Multiscale Modeling & Simulation, Vol. 10, Issue 3
DOI: 10.1137/11083109X

An MBO Scheme on Graphs for Classification and Image Processing
journal, January 2013

Merkurjev, Ekaterina; Kostić, Tijana; Bertozzi, Andrea L.
SIAM Journal on Imaging Sciences, Vol. 6, Issue 4
DOI: 10.1137/120886935

Applied Numerical Linear Algebra
book, January 1997

Demmel, James W.
Society for Industrial and Applied Mathematics
DOI: 10.1137/1.9781611971446

Diffuse Interface Models on Graphs for Classification of High Dimensional Data
journal, January 2016

Bertozzi, Andrea L.; Flenner, Arjuna
SIAM Review, Vol. 58, Issue 2
DOI: 10.1137/16M1070426

Roofline: an insightful visual performance model for multicore architectures
journal, April 2009

Williams, Samuel; Waterman, Andrew; Patterson, David
Communications of the ACM, Vol. 52, Issue 4
DOI: 10.1145/1498765.1498785

Spectral grouping using the nystrom method
journal, February 2004

Fowlkes, C.; Belongie, S.
IEEE Transactions on Pattern Analysis and Machine Intelligence, Vol. 26, Issue 2
DOI: 10.1109/TPAMI.2004.1262185

A simple min-cut algorithm
journal, July 1997

Stoer, Mechthild; Wagner, Frank
Journal of the ACM, Vol. 44, Issue 4
DOI: 10.1145/263867.263872

A tutorial on spectral clustering
journal, August 2007

von Luxburg, Ulrike
Statistics and Computing, Vol. 17, Issue 4
DOI: 10.1007/s11222-007-9033-z

Similar Records in DOE PAGES and OSTI.GOV collections:

A matrix-algebraic formulation of distributed-memory maximal cardinality matching algorithms in bipartite graphs

Journal Article Azad, Ariful ; Buluç, Aydın - Parallel Computing

We describe parallel algorithms for computing maximal cardinality matching in a bipartite graph on distributed-memory systems. Unlike traditional algorithms that match one vertex at a time, our algorithms process many unmatched vertices simultaneously using a matrix-algebraic formulation of maximal matching. This generic matrix-algebraic framework is used to develop three efficient maximal matching algorithms with minimal changes. The newly developed algorithms have two benefits over existing graph-based algorithms. First, unlike existing parallel algorithms, cardinality of matching obtained by the new algorithms stays constant with increasing processor counts, which is important for predictable and reproducible performance. Second, relying on bulk-synchronous matrix operations,more »« less
Cited by 4
https://doi.org/10.1016/j.parco.2016.05.007

Full Text Available
Performance Analysis of a High-Level Abstractions-Based Hydrocode on Future Computing Systems

Journal Article

In this paper we present research on applying a domain specific high-level abstractions (HLA) development strategy with the aim to “future-proof” a key class of high performance computing (HPC) applications that simulate hydrodynamics computations at AWE plc. We build on an existing high-level abstraction framework, OPS, that is being developed for the solution of multi-block structured mesh-based applications at the University of Oxford. OPS uses an “active library” approach where a single application code written using the OPS API can be transformed into different highly optimized parallel implementations which can then be linked against the appropriate parallel library enabling executionmore »« less
https://doi.org/10.1007/978-3-319-17248-4_5
Direction-optimizing Label Propagation Framework for Structure Detection in Graphs: Design, Implementation, and Experimental Analysis

Journal Article Liu, Xu T. ; Lumsdaine, Andrew ; Halappanavar, Mahantesh ; ... - ACM Journal of Experimental Algorithmics

Label Propagation is not only a well-known machine learning algorithm for classification but also an effective method for discovering communities and connected components in networks. We propose a new Direction-optimizing Label Propagation Algorithm (DOLPA) framework that enhances the performance of the standard Label Propagation Algorithm (LPA), increases its scalability, and extends its versatility and application scope. As a central feature, the DOLPA framework relies on the use of frontiers and alternates between label push and label pull operations to attain high performance. It is formulated in such a way that the same basic algorithm can be used for finding communitiesmore »« less
https://doi.org/10.1145/3564593

Full Text Available
High-performance epistasis detection in quantitative trait GWAS

Journal Article Weeks, Nathan T. ; Luecke, Glenn R. ; Groth, Brandon M. ; ... - International Journal of High Performance Computing Applications

epiSNP is a program for identifying pairwise single nucleotide polymorphism (SNP) interactions (epistasis) in quantitative-trait genome-wide association studies (GWAS). A parallel MPI version (EPISNPmpi) was created in 2008 to address this computationally expensive analysis on large data sets with many quantitative traits and SNP markers. However, the falling cost of genotyping has led to an explosion of large-scale GWAS data sets that challenge EPISNPmpi’s ability to compute results in a reasonable amount of time. Therefore, we optimized epiSNP for modern multi-core and highly parallel many-core processors to efficiently handle these large data sets. This paper describes the serial optimizations, dynamicmore »« less
Cited by 17
https://doi.org/10.1177/1094342016658110

Full Text Available
Hyperspectral Image Classification Using Graph Clustering Methods

Journal Article Meng, Zhaoyi ; Merkurjev, Ekaterina ; Koniges, Alice ; ... - IPOL - Image Processing On Line

Hyperspectral imagery is a challenging modality due to the dimension of the pixels which can range from hundreds to over a thousand frequencies depending on the sensor. Most methods in the literature reduce the dimension of the data using a method such as principal component analysis, however this procedure can lose information. More recently methods have been developed to address classification of large datasets in high dimensions. This paper presents two classes of graph-based classification methods for hyperspectral imagery. Using the full dimensionality of the data, we consider a similarity graph based on pairwise comparisons of pixels. The graph ismore »« less
https://doi.org/10.5201/ipol.2017.204

Full Text Available

Similar Records

Title: OpenMP Parallelization and Optimization of Graph-Based Machine Learning Algorithms

Abstract

Citation Formats

Multi-class Graph Mumford-Shah Model for Plume Detection Using the MBO scheme book, January 2015

Diffuse Interface Models on Graphs for Classification of High Dimensional Data journal, January 2012

An MBO Scheme on Graphs for Classification and Image Processing journal, January 2013

Applied Numerical Linear Algebra book, January 1997

Diffuse Interface Models on Graphs for Classification of High Dimensional Data journal, January 2016

Roofline: an insightful visual performance model for multicore architectures journal, April 2009

Spectral grouping using the nystrom method journal, February 2004

A simple min-cut algorithm journal, July 1997

A tutorial on spectral clustering journal, August 2007

Multi-class Graph Mumford-Shah Model for Plume Detection Using the MBO scheme
book, January 2015

Diffuse Interface Models on Graphs for Classification of High Dimensional Data
journal, January 2012

An MBO Scheme on Graphs for Classification and Image Processing
journal, January 2013

Applied Numerical Linear Algebra
book, January 1997

Diffuse Interface Models on Graphs for Classification of High Dimensional Data
journal, January 2016

Roofline: an insightful visual performance model for multicore architectures
journal, April 2009

Spectral grouping using the nystrom method
journal, February 2004

A simple min-cut algorithm
journal, July 1997

A tutorial on spectral clustering
journal, August 2007