Adaptive dimension reduction for clustering high dimensional data

Technical Report · Tue Oct 01 04:00:00 EDT 2002

DOI:https://doi.org/10.2172/807420· OSTI ID:807420

Ding, Chris ^[1]; He, Xiaofeng; Zha, Hongyuan; Simon, Horst

---

1. LBNL Library

It is well-known that for high dimensional data clustering, standard algorithms such as EM and the K-means are often trapped in local minimum. many initialization methods were proposed to tackle this problem, but with only limited success. In this paper they propose a new approach to resolve this problem by repeated dimension reductions such that K-means or EM are performed only in very low dimensions. Cluster membership is utilized as a bridge between the reduced dimensional sub-space and the original space, providing flexibility and ease of implementation. Clustering analysis performed on highly overlapped Gaussians, DNA gene expression profiles and internet newsgroups demonstrate the effectiveness of the proposed algorithm.

• Research Organization: Ernest Orlando Lawrence Berkeley National Laboratory, Berkeley, CA (US)
• Sponsoring Organization: USDOE Director, Office of Science. Office of Advanced Scientific Computing Research. Mathematical, Information, and Computational Sciences Division (US)
• DOE Contract Number: AC03-76SF00098
• OSTI ID: 807420
• Report Number(s): LBNL--51472; B& R KP1201020
• Country of Publication: United States
• Language: English

Related Subjects

59 BASIC BIOLOGICAL SCIENCES
99 GENERAL AND MISCELLANEOUS
ALGORITHMS
DIMENSIONS
DNA
FLEXIBILITY
GENES
IMPLEMENTATION
INTERNET