A dictionary learning algorithm for compression and reconstruction of streaming data in preset order

Journal Article · 31 March 2022 · Discrete and Continuous Dynamical Systems - Series S

DOI: https://doi.org/10.3934/dcdss.2021102 · OSTI ID:1883981

Archibald, Richard  ^[1]; Tran, Hoang  ^[1]

---

1. Oak Ridge National Lab. (ORNL), Oak Ridge, TN (United States)

There has been an emerging interest in developing and applying dictionary learning (DL) to process massive datasets in the last decade. Many of these efforts, however, focus on employing DL to compress and extract a set of important features from data, while considering restoring the original data from this set a secondary goal. On the other hand, although several methods are able to process streaming data by updating the dictionary incrementally as new snapshots pass by, most of those algorithms are designed for the setting where the snapshots are randomly drawn from a probability distribution. In this paper, we present a new DL approach to compress and denoise massive dataset in real time, in which the data are streamed through in a preset order (instances are videos and temporal experimental data), so at any time, we can only observe a biased sample set of the whole data. Here, our approach incrementally builds up the dictionary in a relatively simple manner: if the new snapshot is adequately explained by the current dictionary, we perform a sparse coding to find its sparse representation; otherwise, we add the new snapshot to the dictionary, with a Gram-Schmidt process to maintain the orthogonality. To compress and denoise noisy datasets, we apply the denoising to the snapshot directly before sparse coding, which deviates from traditional dictionary learning approach that achieves denoising via sparse coding. Compared to full-batch matrix decomposition methods, where the whole data is kept in memory, and other mini-batch approaches, where unbiased sampling is often assumed, our approach has minimal requirement in data sampling and storage: i) each snapshot is only seen once then discarded, and ii) the snapshots are drawn in a preset order, so can be highly biased. Through experiments on climate simulations and scanning transmission electron microscopy (STEM) data, we demonstrate that the proposed approach performs competitively to those methods in data reconstruction and denoising.

• Research Organization: Oak Ridge National Laboratory (ORNL), Oak Ridge, TN (United States)
• Sponsoring Organization: USDOE Office of Science (SC), Advanced Scientific Computing Research (ASCR). Scientific Discovery through Advanced Computing (SciDAC); USDOE Laboratory Directed Research and Development (LDRD) Program
• Grant/Contract Number: AC05-00OR22725; AC02-05CH11231
• OSTI ID: 1883981
• Journal Information: Discrete and Continuous Dynamical Systems - Series S, Journal Name: Discrete and Continuous Dynamical Systems - Series S Journal Issue: 4 Vol. 15; ISSN 1937-1632
• Publisher: American Institute of Mathematical Sciences (AIMS)Copyright Statement
• Country of Publication: United States
• Language: English