skip to main content

Title: Diverse power iteration embeddings: Theory and practice

Manifold learning, especially spectral embedding, is known as one of the most effective learning approaches on high dimensional data, but for real-world applications it raises a serious computational burden in constructing spectral embeddings for large datasets. To overcome this computational complexity, we propose a novel efficient embedding construction, Diverse Power Iteration Embedding (DPIE). DPIE shows almost the same effectiveness of spectral embeddings and yet is three order of magnitude faster than spectral embeddings computed from eigen-decomposition. Our DPIE is unique in that (1) it finds linearly independent embeddings and thus shows diverse aspects of dataset; (2) the proposed regularized DPIE is effective if we need many embeddings; (3) we show how to efficiently orthogonalize DPIE if one needs; and (4) Diverse Power Iteration Value (DPIV) provides the importance of each DPIE like an eigen value. As a result, such various aspects of DPIE and DPIV ensure that our algorithm is easy to apply to various applications, and we also show the effectiveness and efficiency of DPIE on clustering, anomaly detection, and feature selection as our case studies.
Authors:
 [1] ;  [2] ;  [2] ;  [3]
  1. GE Global Research, San Ramon, CA (United States)
  2. Brookhaven National Lab. (BNL), Upton, NY (United States)
  3. Stony Brook Univ., Stony Brook, NY (United States)
Publication Date:
OSTI Identifier:
1327442
Report Number(s):
BNL-113578-2017-JA
Journal ID: ISSN 1041-4347
Grant/Contract Number:
SC00112704; PD 15-025; SC0003361
Type:
Published Article
Journal Name:
IEEE Transactions on Knowledge and Data Engineering
Additional Journal Information:
Journal Volume: 28; Journal Issue: 10; Journal ID: ISSN 1041-4347
Publisher:
IEEE
Research Org:
Brookhaven National Laboratory (BNL), Upton, NY (United States)
Sponsoring Org:
USDOE Office of Science (SC), Advanced Scientific Computing Research (SC-21); USDOE
Country of Publication:
United States
Language:
English
Subject:
97 MATHEMATICS AND COMPUTING; power iteration; approximated spectral analysis