Title: A dictionary learning algorithm for compression and reconstruction of streaming data in preset order

Abstract

There has been an emerging interest in developing and applying dictionary learning (DL) to process massive datasets in the last decade. Many of these efforts, however, focus on employing DL to compress and extract a set of important features from data, while considering restoring the original data from this set a secondary goal. On the other hand, although several methods are able to process streaming data by updating the dictionary incrementally as new snapshots pass by, most of those algorithms are designed for the setting where the snapshots are randomly drawn from a probability distribution. In this paper, we present a new DL approach to compress and denoise massive dataset in real time, in which the data are streamed through in a preset order (instances are videos and temporal experimental data), so at any time, we can only observe a biased sample set of the whole data. Here, our approach incrementally builds up the dictionary in a relatively simple manner: if the new snapshot is adequately explained by the current dictionary, we perform a sparse coding to find its sparse representation; otherwise, we add the new snapshot to the dictionary, with a Gram-Schmidt process to maintain the orthogonality. To compressmore » and denoise noisy datasets, we apply the denoising to the snapshot directly before sparse coding, which deviates from traditional dictionary learning approach that achieves denoising via sparse coding. Compared to full-batch matrix decomposition methods, where the whole data is kept in memory, and other mini-batch approaches, where unbiased sampling is often assumed, our approach has minimal requirement in data sampling and storage: i) each snapshot is only seen once then discarded, and ii) the snapshots are drawn in a preset order, so can be highly biased. Through experiments on climate simulations and scanning transmission electron microscopy (STEM) data, we demonstrate that the proposed approach performs competitively to those methods in data reconstruction and denoising.« less

Authors:

Archibald, Richard  ^[1];

• Search DOE PAGES for author "Archibald, Richard"
• Search DOE PAGES for ORCID "0000-0002-4538-9780"

Tran, Hoang  ^[1]

• Search DOE PAGES for author "Tran, Hoang"
• Search DOE PAGES for ORCID "0000-0001-7068-5621"

---

+ Show Author Affiliations

1. Oak Ridge National Lab. (ORNL), Oak Ridge, TN (United States)

Publication Date:

Fri Apr 01 00:00:00 EDT 2022

Research Org.:

Oak Ridge National Lab. (ORNL), Oak Ridge, TN (United States)

Sponsoring Org.:

USDOE Office of Science (SC), Advanced Scientific Computing Research (ASCR). Scientific Discovery through Advanced Computing (SciDAC); USDOE Laboratory Directed Research and Development (LDRD) Program

OSTI Identifier:

1883981

Grant/Contract Number:  

AC05-00OR22725; AC02-05CH11231

Resource Type:

Accepted Manuscript

Journal Name:

Discrete and Continuous Dynamical Systems - Series S

Additional Journal Information:

Journal Volume: 15; Journal Issue: 4; Journal ID: ISSN 1937-1632

Publisher:

American Institute of Mathematical Sciences (AIMS)

Country of Publication:

United States

Language:

English

Subject:

97 MATHEMATICS AND COMPUTING; dictionary learning; matrix factorization; online algorithm