Diverse Power Iteration Embeddings: Theory and Practice

Huang, Hao; Yoo, Shinjae; Yu, Dantong; Qin, Hong

doi:10.1109/TKDE.2015.2499184

Title: Diverse Power Iteration Embeddings: Theory and Practice

Full Record
Other Related Research

Abstract

Manifold learning, especially spectral embedding, is known as one of the most effective learning approaches on high dimensional data, but for real-world applications it raises a serious computational burden in constructing spectral embeddings for large datasets. To overcome this computational complexity, we propose a novel efficient embedding construction, Diverse Power Iteration Embedding (DPIE). DPIE shows almost the same effectiveness of spectral embeddings and yet is three order of magnitude faster than spectral embeddings computed from eigen-decomposition. Our DPIE is unique in that (1) it finds linearly independent embeddings and thus shows diverse aspects of dataset; (2) the proposed regularized DPIE is effective if we need many embeddings; (3) we show how to efficiently orthogonalize DPIE if one needs; and (4) Diverse Power Iteration Value (DPIV) provides the importance of each DPIE like an eigen value. As a result, such various aspects of DPIE and DPIV ensure that our algorithm is easy to apply to various applications, and we also show the effectiveness and efficiency of DPIE on clustering, anomaly detection, and feature selection as our case studies.

Authors:: Huang, Hao; Yoo, Shinjae; Yu, Dantong; Qin, Hong

Publication Date:: Sat Oct 01 00:00:00 EDT 2016

Research Org.:: Brookhaven National Lab. (BNL), Upton, NY (United States)

Sponsoring Org.:: USDOE; USDOE Office of Science (SC), Advanced Scientific Computing Research (SC-21)

OSTI Identifier:: 1327442

Alternate Identifier(s):: OSTI ID: 1327443; OSTI ID: 1345738

Report Number(s):: BNL-113578-2017-JA
Journal ID: ISSN 1041-4347; 7322265

Grant/Contract Number:: PD 15-025; SC0003361; SC00112704

Resource Type:: Published Article

Journal Name:: IEEE Transactions on Knowledge and Data Engineering

Additional Journal Information:: Journal Name: IEEE Transactions on Knowledge and Data Engineering Journal Volume: 28 Journal Issue: 10; Journal ID: ISSN 1041-4347

Publisher:: Institute of Electrical and Electronics Engineers

Country of Publication:: United States

Language:: English

Subject:: 97 MATHEMATICS AND COMPUTING; power iteration; approximated spectral analysis

Citation Formats


                    Huang, Hao, Yoo, Shinjae, Yu, Dantong, and Qin, Hong. Diverse Power Iteration Embeddings: Theory and Practice.  United States: N. p., 2016. 
Web.  doi:10.1109/TKDE.2015.2499184.

Copy to clipboard


                    Huang, Hao, Yoo, Shinjae, Yu, Dantong, & Qin, Hong. Diverse Power Iteration Embeddings: Theory and Practice.  United States.  https://doi.org/10.1109/TKDE.2015.2499184

Copy to clipboard


                    Huang, Hao, Yoo, Shinjae, Yu, Dantong, and Qin, Hong. Sat .  
"Diverse Power Iteration Embeddings: Theory and Practice".  United States.  https://doi.org/10.1109/TKDE.2015.2499184.

Copy to clipboard


                    
@article{osti_1327442,

  title        = {Diverse Power Iteration Embeddings: Theory and Practice},

  author       = {Huang, Hao and Yoo, Shinjae and Yu, Dantong and Qin, Hong},

  abstractNote = {Manifold learning, especially spectral embedding, is known as one of the most effective learning approaches on high dimensional data, but for real-world applications it raises a serious computational burden in constructing spectral embeddings for large datasets. To overcome this computational complexity, we propose a novel efficient embedding construction, Diverse Power Iteration Embedding (DPIE). DPIE shows almost the same effectiveness of spectral embeddings and yet is three order of magnitude faster than spectral embeddings computed from eigen-decomposition. Our DPIE is unique in that (1) it finds linearly independent embeddings and thus shows diverse aspects of dataset; (2) the proposed regularized DPIE is effective if we need many embeddings; (3) we show how to efficiently orthogonalize DPIE if one needs; and (4) Diverse Power Iteration Value (DPIV) provides the importance of each DPIE like an eigen value. As a result, such various aspects of DPIE and DPIV ensure that our algorithm is easy to apply to various applications, and we also show the effectiveness and efficiency of DPIE on clustering, anomaly detection, and feature selection as our case studies.},

  doi          = {10.1109/TKDE.2015.2499184},

  journal      = {IEEE Transactions on Knowledge and Data Engineering},

  number       = 10,

  volume       = 28,

  place        = {United States},

  year         = {Sat Oct 01 00:00:00 EDT 2016},

  month        = {Sat Oct 01 00:00:00 EDT 2016}

}

Copy to clipboard

Journal Article:

Free Publicly Available Full Text

Publisher's Version of Record
https://doi.org/10.1109/TKDE.2015.2499184

Other availability

Search WorldCat to find libraries that may hold this journal

Citation Metrics:

Cited by: 5 works

Citation information provided by
Web of Science

Save / Share:

Export Metadata

Save to My Library

Similar Records in DOE PAGES and OSTI.GOV collections:

Diverse Power Iteration Embeddings and Its Applications

Conference Huang, H. ; Yoo, S. ; Yu, D. ; ...

Abstract—Spectral Embedding is one of the most effective dimension reduction algorithms in data mining. However, its computation complexity has to be mitigated in order to apply it for real-world large scale data analysis. Many researches have been focusing on developing approximate spectral embeddings which are more efficient, but meanwhile far less effective. This paper proposes Diverse Power Iteration Embeddings (DPIE), which not only retains the similar efficiency of power iteration methods but also produces a series of diverse and more effective embedding vectors. We test this novel method by applying it to various data mining applications (e.g. clustering, anomaly detectionmore »« less
Full Text Available
The Twins Embedding of Type Ia Supernovae. I. The Diversity of Spectra at Maximum Light

Journal Article Boone, K. ; Aldering, G. ; Antilogus, P. ; ... - The Astrophysical Journal

In this work, we study the spectral diversity of Type Ia supernovae (SNe Ia) at maximum light using high signal-to-noise spectrophotometry of 173 SNe Ia from the Nearby Supernova Factory. We decompose the diversity of these spectra into different extrinsic and intrinsic components, and we construct a nonlinear parameterization of the intrinsic diversity of SNe Ia that preserves pairings of "twin"SNe Ia. We call this parameterization the "Twins Embedding."Our methodology naturally handles highly nonlinear variability in spectra, such as changes in the photosphere expansion velocity, and uses the full spectrum rather than being limited to specific spectral line strengths, ratios,more »« less
https://doi.org/10.3847/1538-4357/abec3c

Full Text Available
Evaluation of Advanced Signal Processing Techniques to Improve Detection and Identification of Embedded Defects

Technical Report Clayton, Dwight A. ; Santos-Villalobos, Hector J. ; Baba, Justin S.

By the end of 1996, 109 Nuclear Power Plants were operating in the United States, producing 22% of the Nation’s electricity [1]. At present, more than two thirds of these power plants are more than 40 years old. The purpose of the U.S. Department of Energy Office of Nuclear Energy’s Light Water Reactor Sustainability (LWRS) Program is to develop technologies and other solutions that can improve the reliability, sustain the safety, and extend the operating lifetimes of nuclear power plants (NPPs) beyond 60 years [2]. The most important safety structures in an NPP are constructed of concrete. The structures generallymore »« less
https://doi.org/10.2172/1351761

Full Text Available
TeaNet: Universal neural network interatomic potential inspired by iterative electronic relaxations

Journal Article Takamoto, So ; Izumi, Satoshi ; Li, Ju - Computational Materials Science

A universal interatomic potential for an arbitrary set of chemical elements is urgently needed in computational materials science. Graph convolution neural network (GCN) has rich expressive power, but previously was mainly employed to transport scalars and vectors, not rank ≥ tensors. As classic interatomic potentials were inspired by tight-binding electronic relaxation framework, we want to represent this iterative propagation of rank ≥ tensor information by GCN. Here we propose an architecture called the tensor embedded atom network (TeaNet) where angular interaction is translated into graph convolution through the incorporation of Euclidean tensors, vectors and scalars. By applying the residual networkmore »« less
https://doi.org/10.1016/j.commatsci.2022.111280
Deep active learning for classifying cancer pathology reports

Journal Article De Angeli, Kevin ; Gao, Shang ; Alawad, Mohammed ; ... - BMC Bioinformatics

Abstract Background Automated text classification has many important applications in the clinical setting; however, obtaining labelled data for training machine learning and deep learning models is often difficult and expensive. Active learning techniques may mitigate this challenge by reducing the amount of labelled data required to effectively train a model. In this study, we analyze the effectiveness of 11 active learning algorithms on classifying subsite and histology from cancer pathology reports using a Convolutional Neural Network as the text classification model. Results We compare the performance of each active learning strategy using two differently sized datasets and two different classificationmore »« less
https://doi.org/10.1186/s12859-021-04047-1

Similar Records