Computing rank‐revealing factorizations of matrices stored out‐of‐core

Heavner, N.; Martinsson, P. G.; Quintana‐Ortí, G.

doi:10.1002/cpe.7726

Computing rank‐revealing factorizations of matrices stored out‐of‐core

Journal Article · Mon Apr 17 00:00:00 EDT 2023 · Concurrency and Computation. Practice and Experience

DOI:https://doi.org/10.1002/cpe.7726· OSTI ID:2575873

Heavner, N. ^[1]; ^[2]; ^[3]

Univ. of Colorado, Boulder, CO (United States)
Univ. of Texas, Austin, TX (United States)
Universitat Jaume I, Castellón (Spain)

This paper describes efficient algorithms for computing rank-revealing factorizations of matrices that are too large to fit in main memory (RAM), and must instead be stored on slow external memory devices such as disks (out-of-core or out-of-memory). Traditional algorithms for computing rank-revealing factorizations (such as the column pivoted QR factorization and the singular value decomposition) are very communication intensive as they require many vector-vector and matrix-vector operations, which become prohibitively expensive when data is not in RAM. Randomization allows to reformulate new methods so that large contiguous blocks of the matrix are processed in bulk. The paper describes two distinct methods. The first is a blocked version of column pivoted Householder QR, organized as a “left-looking” method to minimize the number of the expensive write operations. The second method results employs a UTV factorization. It is organized as an algorithm-by-blocks to overlap computations and I/O operations. As it incorporates power iterations, it is much better at revealing the numerical rank. Numerical experiments on several computers demonstrate that the new algorithms are almost as fast when processing data stored on slow memory devices as traditional algorithms are for data stored in RAM.

View Accepted Manuscript (DOE)

Research Organization:: Univ. of Texas, Austin, TX (United States)

Sponsoring Organization:: USDOE Office of Science (SC), Advanced Scientific Computing Research (ASCR); National Science Foundation (NSF); US Department of the Navy, Office of Naval Research (ONR); Spanish Ministry of Science and Innovation and the Research State Agency

Grant/Contract Number:: SC0022251

OSTI ID:: 2575873

Journal Information:: Concurrency and Computation. Practice and Experience, Journal Name: Concurrency and Computation. Practice and Experience Journal Issue: 22 Vol. 35; ISSN 1532-0626; ISSN 1532-0634

Publisher:: WileyCopyright Statement

Country of Publication:: United States

Language:: English

References (34)

The design and implementation of the parallel out-of-core ScaLAPACK LU, QR, and Cholesky factorization routines D'Azevedo, Eduardo; Dongarra, Jack Concurrency: Practice and Experience, Vol. 12, Issue 15 https://doi.org/10.1002/1096-9128(20001225)12:15<1481::AID-CPE540>3.0.CO;2-V	journal	December 2000
Scheduling dense linear algebra operations on multicore processors Kurzak, Jakub; Ltaief, Hatem; Dongarra, Jack Concurrency and Computation: Practice and Experience, Vol. 22, Issue 1 https://doi.org/10.1002/cpe.1467	journal	January 2010
Linear least squares solutions by householder transformations Businger, Peter; Golub, Gene H. Numerische Mathematik, Vol. 7, Issue 3 https://doi.org/10.1007/BF01436084	journal	June 1965
The approximation of one matrix by another of lower rank Eckart, Carl; Young, Gale Psychometrika, Vol. 1, Issue 3 https://doi.org/10.1007/BF02288367	journal	September 1936
Using desktop computers to solve large-scale dense linear algebra problems Marqués, M.; Quintana-Ortí, G.; Quintana-Ortí, E. S. The Journal of Supercomputing, Vol. 58, Issue 2 https://doi.org/10.1007/s11227-010-0394-2	journal	February 2010
Rank revealing QR factorizations Chan, Tony F. Linear Algebra and its Applications, Vol. 88-89 https://doi.org/10.1016/0024-3795(87)90103-0	journal	April 1987
The FLAME approach: From dense linear algebra algorithms to high-performance multi-accelerator implementations Igual, Francisco D.; Chan, Ernie; Quintana-Ortí, Enrique S. Journal of Parallel and Distributed Computing, Vol. 72, Issue 9 https://doi.org/10.1016/j.jpdc.2011.10.014	journal	September 2012
On selecting a maximum volume sub-matrix of a matrix and related problems Çivril, Ali; Magdon-Ismail, Malik Theoretical Computer Science, Vol. 410, Issue 47-49 https://doi.org/10.1016/j.tcs.2009.06.018	journal	November 2009
Randomized numerical linear algebra: Foundations and algorithms Martinsson, Per-Gunnar; Tropp, Joel A. Acta Numerica, Vol. 29 https://doi.org/10.1017/S0962492920000021	journal	May 2020
An updating algorithm for subspace tracking Stewart, G. W. IEEE Transactions on Signal Processing, Vol. 40, Issue 6 https://doi.org/10.1109/78.139256	journal	June 1992
Parallel out-of-core cholesky and QR factorizations with POOCLAPACK Gunter, B. C.; Reiley, W. C.; van de Geijn, R. A. IEEE International Symposium on Parallel and Distributed Processing, Proceedings 15th International Parallel and Distributed Processing Symposium. IPDPS 2001 https://doi.org/10.1109/IPDPS.2001.925180	conference	January 2001
The singular value decomposition: Its computation and some applications Klema, V.; Laub, A. IEEE Transactions on Automatic Control, Vol. 25, Issue 2 https://doi.org/10.1109/TAC.1980.1102314	journal	April 1980
Calculating the Singular Values and Pseudo-Inverse of a Matrix Golub, G.; Kahan, W. Journal of the Society for Industrial and Applied Mathematics Series B Numerical Analysis, Vol. 2, Issue 2 https://doi.org/10.1137/0702016	journal	January 1965
An Analysis of the Total Least Squares Problem Golub, Gene H.; van Loan, Charles F. SIAM Journal on Numerical Analysis, Vol. 17, Issue 6 https://doi.org/10.1137/0717073	journal	December 1980
A Randomized Algorithm for Principal Component Analysis Rokhlin, Vladimir; Szlam, Arthur; Tygert, Mark SIAM Journal on Matrix Analysis and Applications, Vol. 31, Issue 3 https://doi.org/10.1137/080736417	journal	January 2010
Finding Structure with Randomness: Probabilistic Algorithms for Constructing Approximate Matrix Decompositions Halko, N.; Martinsson, P. G.; Tropp, J. A. SIAM Review, Vol. 53, Issue 2 https://doi.org/10.1137/090771806	journal	January 2011
Some Applications of the Rank Revealing QR Factorization Chan, Tony F.; Hansen, Per Christian SIAM Journal on Scientific and Statistical Computing, Vol. 13, Issue 3 https://doi.org/10.1137/0913043	journal	May 1992
Efficient Algorithms for Computing a Strong Rank-Revealing QR Factorization Gu, Ming; Eisenstat, Stanley C. SIAM Journal on Scientific Computing, Vol. 17, Issue 4 https://doi.org/10.1137/0917055	journal	July 1996
ScaLAPACK Users' Guide Blackford, L. S.; Choi, J.; Cleary, A. Society for Industrial and Applied Mathematics https://doi.org/10.1137/1.9780898719642	book	January 1997
Solving Least Squares Problems Lawson, Charles L.; Hanson, Richard J. https://doi.org/10.1137/1.9781611971217	book	January 1995
An Algorithm for the Principal Component Analysis of Large Data Sets Halko, Nathan; Martinsson, Per-Gunnar; Shkolnisky, Yoel SIAM Journal on Scientific Computing, Vol. 33, Issue 5 https://doi.org/10.1137/100804139	journal	January 2011
Randomized QR with Column Pivoting Duersch, Jed A.; Gu, Ming SIAM Journal on Scientific Computing, Vol. 39, Issue 4 https://doi.org/10.1137/15M1044680	journal	January 2017
Householder QR Factorization With Randomization for Column Pivoting (HQRRP) Martinsson, Per-Gunnar; Quintana OrtÍ, Gregorio; Heavner, Nathan SIAM Journal on Scientific Computing, Vol. 39, Issue 2 https://doi.org/10.1137/16M1081270	journal	January 2017
On Rank-Revealing Factorisations Chandrasekaran, Shivkumar; Ipsen, Ilse C. F. SIAM Journal on Matrix Analysis and Applications, Vol. 15, Issue 2 https://doi.org/10.1137/S0895479891223781	journal	April 1994
A BLAS-3 Version of the QR Factorization with Column Pivoting Quintana-Ortí, Gregorio; Sun, Xiaobai; Bischof, Christian H. SIAM Journal on Scientific Computing, Vol. 19, Issue 5 https://doi.org/10.1137/S1064827595296732	journal	September 1998
Representing linear algebra algorithms in code: the FLAME application program interfaces Bientinesi, Paolo; Quintana-Ortí, Enrique S.; Geijn, Robert A. van de ACM Transactions on Mathematical Software, Vol. 31, Issue 1 https://doi.org/10.1145/1055531.1055533	journal	March 2005
Parallel out-of-core computation and updating of the QR factorization Gunter, Brian C.; Van De Geijn, Robert A. ACM Transactions on Mathematical Software, Vol. 31, Issue 1 https://doi.org/10.1145/1055531.1055534	journal	March 2005
On the Failure of Rank-Revealing QR Factorization Software -- A Case Study Drmač, Zlatko; Bujanović, Zvonimir ACM Transactions on Mathematical Software, Vol. 35, Issue 2 https://doi.org/10.1145/1377612.1377616	journal	July 2008
Programming matrix algorithms-by-blocks for thread-level parallelism Quintana-Ortí, Gregorio; Quintana-Ortí, Enrique S.; Geijn, Robert A. Van De ACM Transactions on Mathematical Software, Vol. 36, Issue 3 https://doi.org/10.1145/1527286.1527288	journal	July 2009
A Runtime System for Programming Out-of-Core Matrix Algorithms-by-Tiles on Multithreaded Architectures Quintana-Ortí, Gregorio; Igual, Francisco D.; Marqués, Mercedes ACM Transactions on Mathematical Software, Vol. 38, Issue 4 https://doi.org/10.1145/2331130.2331133	journal	August 2012
The design and implementation of SOLAR, a portable library for scalable out-of-core linear algebra computations Toledo, Sivan; Gustavson, Fred G. Proceedings of the fourth workshop on I/O in parallel and distributed systems part of the federated computing research conference - IOPADS '96 https://doi.org/10.1145/236017.236029	conference	January 1996
randUTV: A Blocked Randomized Algorithm for Computing a Rank-Revealing UTV Factorization Martinsson, P. G.; Quintana-Ortí, G.; Heavner, N. ACM Transactions on Mathematical Software, Vol. 45, Issue 1 https://doi.org/10.1145/3242670	journal	March 2019
Flame Gunnels, John A.; Gustavson, Fred G.; Henry, Greg M. ACM Transactions on Mathematical Software, Vol. 27, Issue 4 https://doi.org/10.1145/504210.504213	journal	December 2001
Numerical Linear Algebra Kahan, W. Canadian Mathematical Bulletin, Vol. 9, Issue 05 https://doi.org/10.4153/CMB-1966-083-2	journal	December 1966

Similar Records

A block QR factorization algorithm for rank-deficient matrices

Conference · Sat Dec 31 23:00:00 EST 1988 · OSTI ID:6938041

A block QR factorization algorithm using restricted pivoting

Conference · Sat Dec 31 23:00:00 EST 1988 · OSTI ID:5587289

Computing rank-revealing QR factorizations of dense matrices.

Journal Article · Mon Jun 01 00:00:00 EDT 1998 · ACM Trans. Math. Software · OSTI ID:937863

Related Subjects

Computer Science
HQRRP factorization
blocked matrix computations
householder QR factorization
out-of-core computation
partial rank-revealing factorization
randUTV factorization
randomized numerical linear algebra
rank-revealing factorization
shared-memory multicore processors
shared-memory multiprocessors

Computing rank‐revealing factorizations of matrices stored out‐of‐core

Citation Formats

References (34)

Similar Records

Related Subjects