A Class of Sparse Johnson–Lindenstrauss Transforms and Analysis of their Extreme Singular Values

Journal Article · Tue Feb 18 00:00:00 EST 2025 · SIAM Journal on Matrix Analysis and Applications

DOI:https://doi.org/10.1137/23m1605661· OSTI ID:2563384

Dzahini, K. J. ^[1]; Wild, S. M. ^[2]

---

1. Argonne National Laboratory (ANL), Argonne, IL (United States)
2. Lawrence Berkeley National Laboratory (LBNL), Berkeley, CA (United States)

The Johnson–Lindenstrauss (JL) lemma is a powerful tool for dimensionality reduction in modern algorithm design. The lemma states that any set of high-dimensional points in a Euclidean space can be projected into lower dimensions while approximately preserving pairwise Euclidean distances. Random matrices satisfying this lemma are called JL transforms (JLTs). Inspired by existing $$s$$-hashing JLTs with exactly $$s$$ nonzero elements on each column, the present work introduces an ensemble of sparse matrices encompassing so-called $$s$$-hashing-like matrices whose expected number of nonzero elements on each column is $$s$$. The independence of the sub-Gaussian entries of these matrices and the knowledge of their exact distribution play an important role in their analyses. Using properties of independent sub-Gaussian random variables, these matrices are demonstrated to be JLTs, and their smallest nontrivial singular values and largest singular values are estimated nonasymptotically using a technique from geometric functional analysis. As the dimensions of the matrix grow to infinity, these singular values are proved to converge almost surely to fixed quantities (by using the universal Bai–Yin law) and in distribution to the Gaussian orthogonal ensemble Tracy–Widom law after proper rescalings. Understanding the behaviors of extreme singular values is important in general because they are often used to define a measure of stability of matrix algorithms. For example, JLTs were recently used in derivative-free optimization algorithmic frameworks to select random subspaces in which are constructed random models or poll directions to achieve scalability, and hence estimating their smallest singular value in particular helps determine the dimension of these subspaces.

• Research Organization: Lawrence Berkeley National Laboratory (LBNL), Berkeley, CA (United States)
• Sponsoring Organization: USDOE Office of Science (SC), Advanced Scientific Computing Research (ASCR). Scientific Discovery through Advanced Computing (SciDAC)
• Grant/Contract Number: AC02-05CH11231
• OSTI ID: 2563384
• Journal Information: SIAM Journal on Matrix Analysis and Applications, Journal Name: SIAM Journal on Matrix Analysis and Applications Journal Issue: 1 Vol. 46; ISSN 0895-4798
• Publisher: Society for Industrial and Applied Mathematics (SIAM)
• Country of Publication: United States
• Language: English

Related Subjects

97 MATHEMATICS AND COMPUTING
Johnson Lindenstrauss transforms