A neural network for determination of latent dimensionality in Nonnegative Matrix Factorization

Nebgen, Benjamin Tyler; Vangara, Raviteja; Hombrados Herrera, Miguel Angel; Kuksova, Svetlana; Alexandrov, Boian

doi:10.1088/2632-2153/aba372

Title: A neural network for determination of latent dimensionality in Nonnegative Matrix Factorization

Abstract

Non-negative Matrix Factorization (NMF) has proven to be a powerful unsupervised learning method for uncovering hidden features in complex and noisy datasets with applications in data mining, text recognition, dimension reduction, face recognition, anomaly detection, blind source separation, and many other fields. An important input for NMF is the latent dimensionality of the data, that is, the number of hidden features, K, present in the explored dataset. Unfortunately, and this quantity is rarely known a priori. The existing methods for determining latent dimensionality, such as Automatic Relevance Determination (ARD), are mostly heuristic and utilize different characteristics to estimate the number of hidden features. However, all of them require human presence to make a final determination of K. Here we utilize a supervised machine learning approach in combination with a recent method for model determination, called NMFk, to determine the number of hidden features automatically. NMFk performs a set of NMF simulations on an ensemble of matrices, obtained by bootstrapping the initial dataset, and estimates which K produces stable groups of latent features that reconstruct the initial dataset well. We then train a Multi-Layter Perceptron (MLP) classifier network to determine the correct number of latent features utilizing the statistics and characteristicsmore »« less

Authors:

^[1];

^[1]; Hombrados Herrera, Miguel Angel ^[1]; Kuksova, Svetlana ^[1];

^[1]

Los Alamos National Lab. (LANL), Los Alamos, NM (United States)

Publication Date:: Tue Jul 07 00:00:00 EDT 2020

Research Org.:: Los Alamos National Laboratory (LANL), Los Alamos, NM (United States)

Sponsoring Org.:: USDOE Laboratory Directed Research and Development (LDRD) Program; USDOE National Nuclear Security Administration (NNSA)

OSTI Identifier:: 1716806

Report Number(s):: LA-UR-20-20994
Journal ID: ISSN 2632-2153

Grant/Contract Number:: 89233218CNA000001

Resource Type:: Accepted Manuscript

Journal Name:: Machine Learning: Science and Technology

Additional Journal Information:: Journal Volume: 2; Journal Issue: 2; Journal ID: ISSN 2632-2153

Publisher:: IOP Publishing

Country of Publication:: United States

Language:: English

Subject:: 97 MATHEMATICS AND COMPUTING

Citation Formats


                    Nebgen, Benjamin Tyler, Vangara, Raviteja, Hombrados Herrera, Miguel Angel, Kuksova, Svetlana, and Alexandrov, Boian. A neural network for determination of latent dimensionality in Nonnegative Matrix Factorization.  United States: N. p., 2020. 
Web.  doi:10.1088/2632-2153/aba372.

Copy to clipboard


                    Nebgen, Benjamin Tyler, Vangara, Raviteja, Hombrados Herrera, Miguel Angel, Kuksova, Svetlana, & Alexandrov, Boian. A neural network for determination of latent dimensionality in Nonnegative Matrix Factorization.  United States.  https://doi.org/10.1088/2632-2153/aba372

Copy to clipboard


                    Nebgen, Benjamin Tyler, Vangara, Raviteja, Hombrados Herrera, Miguel Angel, Kuksova, Svetlana, and Alexandrov, Boian. Tue .  
"A neural network for determination of latent dimensionality in Nonnegative Matrix Factorization".  United States.  https://doi.org/10.1088/2632-2153/aba372.  https://www.osti.gov/servlets/purl/1716806.

Copy to clipboard


                    
@article{osti_1716806,

  title        = {A neural network for determination of latent dimensionality in Nonnegative Matrix Factorization},

  author       = {Nebgen, Benjamin Tyler and Vangara, Raviteja and Hombrados Herrera, Miguel Angel and Kuksova, Svetlana and Alexandrov, Boian},

  abstractNote = {Non-negative Matrix Factorization (NMF) has proven to be a powerful unsupervised learning method for uncovering hidden features in complex and noisy datasets with applications in data mining, text recognition, dimension reduction, face recognition, anomaly detection, blind source separation, and many other fields. An important input for NMF is the latent dimensionality of the data, that is, the number of hidden features, K, present in the explored dataset. Unfortunately, and this quantity is rarely known a priori. The existing methods for determining latent dimensionality, such as Automatic Relevance Determination (ARD), are mostly heuristic and utilize different characteristics to estimate the number of hidden features. However, all of them require human presence to make a final determination of K. Here we utilize a supervised machine learning approach in combination with a recent method for model determination, called NMFk, to determine the number of hidden features automatically. NMFk performs a set of NMF simulations on an ensemble of matrices, obtained by bootstrapping the initial dataset, and estimates which K produces stable groups of latent features that reconstruct the initial dataset well. We then train a Multi-Layter Perceptron (MLP) classifier network to determine the correct number of latent features utilizing the statistics and characteristics of the NMF solution, obtained from NMFk. In order to train the MLP classifier, a training set of 58,660 matrices with predetermined latent features were factorized with NMFk. The MLP classifier in conjunction with NMFk maintains a greater than 95% success rate when applied to a held out test set. Additionally, when applied to two well-known benchmark datasets, the swimmer and MIT face data, NMFk/MLP correctly recovers the established number of hidden features. Finally, we compare the accuracy of our method to the ARD, AIC and Stability-based methods.},

  doi          = {10.1088/2632-2153/aba372},

  journal      = {Machine Learning: Science and Technology},

  number       = 2,

  volume       = 2,

  place        = {United States},

  year         = {Tue Jul 07 00:00:00 EDT 2020},

  month        = {Tue Jul 07 00:00:00 EDT 2020}

}

Copy to clipboard

Journal Article:

Free Publicly Available Full Text

Accepted Manuscript (DOE)

Publisher's Version of Record

https://doi.org/10.1088/2632-2153/aba372

Other availability

Search WorldCat to find libraries that may hold this journal

Save / Share:

Export Metadata

Save to My Library

Works referenced in this record:

Unsupervised learning of auditory filter banks using non-negative matrix factorisation
conference, March 2008

Bertrand, Alexander; Demuynck, Kris; Stouten, Veronique
2008 IEEE International Conference on Acoustics, Speech and Signal Processing
DOI: 10.1109/icassp.2008.4518709

Similar Records in DOE PAGES and OSTI.GOV collections:

Distributed non-negative matrix factorization with determination of the number of latent features

Journal Article Chennupati, Gopinath ; Vangara, Raviteja ; Skau, Erik West ; ... - Journal of Supercomputing

The holistic analysis and understanding of the latent (that is, not directly observable) variables and patterns buried in large datasets is crucial for data-driven science, decision making and emergency response. Such exploratory analyses require devising unsupervised learning methods for data mining and extraction of the latent features, and non-negative matrix factorization (NMF) is one of the prominent such methods. NMF is based on compute-intense non-convex constrained minimization, which, for large datasets requires fast and distributed algorithms. However, current parallel implementations of NMF fail to estimate the number of latent features. In practice, identifying these features is both difficult and significantmore »« less
https://doi.org/10.1007/s11227-020-03181-6

Full Text Available
A comparative study of machine learning models for predicting the state of reactive mixing

Journal Article Ahmmed, Bulbul ; Mudunuru, Maruti K. ; Karra, Satish ; ... - Journal of Computational Physics

Mixing phenomena are important mechanisms controlling flow, species transport, and reaction processes in fluids and porous media. Accurate predictions of reactive mixing are critical for many Earth and environmental science problems such as contaminant fate and remediation, macroalgae growth, and plankton biomass evolution. Here, to investigate the evolution of mixing dynamics under different scenarios (e.g., anisotropy, fluctuating velocity fields), a finite-element-based numerical model was built to solve the fast, irreversible bimolecular reaction-diffusion equations to simulate a range of reactive-mixing scenarios. A total of 2,315 simulations were performed using different sets of model input parameters comprising various spatial scales of vortexmore »« less
https://doi.org/10.1016/j.jcp.2021.110147

Full Text Available
Discovering Hidden Geothermal Signatures using Unsupervised Machine Learning

Technical Report Vesselinov, Velimir Valentinov ; Ahmmed, Bulbul ; Mudunuru, Maruti K. ; ...

Discovering hidden geothermal resources is a very challenging task. It requires the mining of large datasets, including various diverse data attributes representing subsurface hydrogeological and geothermal conditions. The commonly used Play Fairway Analysis (PFA) typically relies on subject-matter expertise to analyze site or regional data to estimate geothermal conditions and prospectivity. Here, we demonstrate an alternative approach based on machine learning (ML) to process a geothermal dataset of Southwest New Mexico (SWNM). The study region includes low- and medium-temperature hydrothermal systems. However, most of these systems are poorly characterized because of insufficient existing data and limited past explorative studies. Thismore »« less
https://doi.org/10.2172/1781347

Full Text Available
Fault diagnosis of electrical power systems using incremental radial basis function nets

Conference Nagabhushana, T N ; Chandrasekharaiah, H S

Most of the proposed Neural Networks for fault diagnosis of power systems are Multilayer Perceptrons [MLP] employing the popular Back Propagation [BP] learning rule. It has been shown that Back Propagation algorithm usually takes a long time for convergence and sometimes get trapped into local minimum. The algorithm requires the architecture to be fixed initially (i.e., the number of hidden units) before learning begins. Final network size is obtained by repeated trials. When the size of the training set is large, especially in the case of fault diagnosis, such a repeated training consumes a large amount of time and sometimesmore »« less
Machine-learning Classifiers for Intermediate Redshift Emission-line Galaxies

Journal Article Zhang, Kai ; Schlegel, David J. ; Andrews, Brett H. ; ... - The Astrophysical Journal (Online)

Classification of intermediate redshift (z = 0.3-0.8) emission line galaxies as star-forming galaxies, composite galaxies, active galactic nuclei (AGNs), or low-ionization nuclear emission regions (LINERs) using optical spectra alone was impossible because the lines used for standard optical diagnostic diagrams: [N ii], Hα, and [S ii] are redshifted out of the observed wavelength range. In this work, we address this problem using four supervised machine-learning classification algorithms: k-nearest neighbors (KNN), support vector classifier (SVC), random forest (RF), and a multilayer perceptron (MLP) neural network. For input features, we use properties that can be measured from optical galaxy spectra out tomore »« less
Cited by 9
https://doi.org/10.3847/1538-4357/ab397e

Full Text Available

Similar Records

Title: A neural network for determination of latent dimensionality in Nonnegative Matrix Factorization

Abstract

Citation Formats

Unsupervised learning of auditory filter banks using non-negative matrix factorisation conference, March 2008

Unsupervised learning of auditory filter banks using non-negative matrix factorisation
conference, March 2008