Search Menu
DOE PAGES title logo U.S. Department of Energy
Office of Scientific and Technical Information
Search Scholarly Publications

Title: Deep data analysis via physically constrained linear unmixing: universal framework, domain examples, and a community-wide platform

Abstract

Abstract Many spectral responses in materials science, physics, and chemistry experiments can be characterized as resulting from the superposition of a number of more basic individual spectra. In this context, unmixing is defined as the problem of determining the individual spectra, given measurements of multiple spectra that are spatially resolved across samples, as well as the determination of the corresponding abundance maps indicating the local weighting of each individual spectrum. Matrix factorization is a popular linear unmixing technique that considers that the mixture model between the individual spectra and the spatial maps is linear. Here, we present a tutorial paper targeted at domain scientists to introduce linear unmixing techniques, to facilitate greater understanding of spectroscopic imaging data. We detail a matrix factorization framework that can incorporate different domain information through various parameters of the matrix factorization method. We demonstrate many domain-specific examples to explain the expressivity of the matrix factorization framework and show how the appropriate use of domain-specific constraints such as non-negativity and sum-to-one abundance result in physically meaningful spectral decompositions that are more readily interpretable. Our aim is not only to explain the off-the-shelf available tools, but to add additional constraints when ready-made algorithms are unavailable for themore » task. All examples use the scalable open source implementation from https://github.com/ramkikannan/nmflibrary that can run from small laptops to supercomputers, creating a user-wide platform for rapid dissemination and adoption across scientific disciplines.« less

Authors:
ORCiD logo; ; ; ; ; ;
Publication Date:
Research Org.:
Oak Ridge National Laboratory (ORNL), Oak Ridge, TN (United States). Oak Ridge Leadership Computing Facility (OLCF)
Sponsoring Org.:
USDOE Office of Science (SC), Basic Energy Sciences (BES); USDOE Office of Science (SC), Basic Energy Sciences (BES) (SC-22). Materials Sciences & Engineering Division
OSTI Identifier:
1435364
Alternate Identifier(s):
OSTI ID: 1474632
Grant/Contract Number:  
AC05-00OR22725
Resource Type:
Published Article
Journal Name:
Advanced Structural and Chemical Imaging
Additional Journal Information:
Journal Name: Advanced Structural and Chemical Imaging Journal Volume: 4 Journal Issue: 1; Journal ID: ISSN 2198-0926
Publisher:
Springer Science + Business Media
Country of Publication:
Germany
Language:
English
Subject:
97 MATHEMATICS AND COMPUTING; 47 OTHER INSTRUMENTATION; Unmixing; Image segmentation; Scanning probe microscopy; Matrix factorization; Big data; High performance

Citation Formats

Kannan, R., Ievlev, A. V., Laanait, N., Ziatdinov, M. A., Vasudevan, R. K., Jesse, S., and Kalinin, S. V. Deep data analysis via physically constrained linear unmixing: universal framework, domain examples, and a community-wide platform. Germany: N. p., 2018. Web. doi:10.1186/s40679-018-0055-8.
Copy to clipboard
Kannan, R., Ievlev, A. V., Laanait, N., Ziatdinov, M. A., Vasudevan, R. K., Jesse, S., & Kalinin, S. V. Deep data analysis via physically constrained linear unmixing: universal framework, domain examples, and a community-wide platform. Germany. https://doi.org/10.1186/s40679-018-0055-8
Copy to clipboard
Kannan, R., Ievlev, A. V., Laanait, N., Ziatdinov, M. A., Vasudevan, R. K., Jesse, S., and Kalinin, S. V. Mon . "Deep data analysis via physically constrained linear unmixing: universal framework, domain examples, and a community-wide platform". Germany. https://doi.org/10.1186/s40679-018-0055-8.
Copy to clipboard
@article{osti_1435364,
title = {Deep data analysis via physically constrained linear unmixing: universal framework, domain examples, and a community-wide platform},
author = {Kannan, R. and Ievlev, A. V. and Laanait, N. and Ziatdinov, M. A. and Vasudevan, R. K. and Jesse, S. and Kalinin, S. V.},
abstractNote = {Abstract Many spectral responses in materials science, physics, and chemistry experiments can be characterized as resulting from the superposition of a number of more basic individual spectra. In this context, unmixing is defined as the problem of determining the individual spectra, given measurements of multiple spectra that are spatially resolved across samples, as well as the determination of the corresponding abundance maps indicating the local weighting of each individual spectrum. Matrix factorization is a popular linear unmixing technique that considers that the mixture model between the individual spectra and the spatial maps is linear. Here, we present a tutorial paper targeted at domain scientists to introduce linear unmixing techniques, to facilitate greater understanding of spectroscopic imaging data. We detail a matrix factorization framework that can incorporate different domain information through various parameters of the matrix factorization method. We demonstrate many domain-specific examples to explain the expressivity of the matrix factorization framework and show how the appropriate use of domain-specific constraints such as non-negativity and sum-to-one abundance result in physically meaningful spectral decompositions that are more readily interpretable. Our aim is not only to explain the off-the-shelf available tools, but to add additional constraints when ready-made algorithms are unavailable for the task. All examples use the scalable open source implementation from https://github.com/ramkikannan/nmflibrary that can run from small laptops to supercomputers, creating a user-wide platform for rapid dissemination and adoption across scientific disciplines.},
doi = {10.1186/s40679-018-0055-8},
journal = {Advanced Structural and Chemical Imaging},
number = 1,
volume = 4,
place = {Germany},
year = {Mon Apr 30 00:00:00 EDT 2018},
month = {Mon Apr 30 00:00:00 EDT 2018}
}
Copy to clipboard
Journal Article:
Free Publicly Available Full Text
Publisher's Version of Record
https://doi.org/10.1186/s40679-018-0055-8
Copyright Statement
Other availability
Search WorldCat Search WorldCat to find libraries that may hold this journal
Save / Share:
Export Metadata
Save to My Library
You must Sign In or Create an Account in order to save documents to your library.

Works referenced in this record:

Dynamic piezoresponse force microscopy: Spatially resolved probing of polarization dynamics in time and voltage domains
journal, September 2012

  • Kumar, A.; Ehara, Y.; Wada, A.
  • Journal of Applied Physics, Vol. 112, Issue 5
  • DOI: 10.1063/1.4746080

Microscopy: Hasten high resolution
journal, November 2014

  • Pennycook, Stephen J.; Kalinin, Sergei V.
  • Nature, Vol. 515, Issue 7528
  • DOI: 10.1038/515487a

STEM-EELS imaging of complex oxides and interfaces
journal, January 2012

  • Varela, Maria; Gazquez, Jaume; Pennycook, Stephen J.
  • MRS Bulletin, Vol. 37, Issue 01, p. 29-35
  • DOI: 10.1557/mrs.2011.330

Principal component and spatial correlation analysis of spectroscopic-imaging data in scanning probe microscopy
journal, February 2009

‘All models are wrong...’: an introduction to model uncertainty
journal, July 2012

Exact matrix completion via convex optimization
journal, June 2012

Quantitative 3D-KPFM imaging with simultaneous electrostatic force and force gradient detection
journal, April 2015

LIII. On lines and planes of closest fit to systems of points in space
journal, November 1901

  • Pearson, Karl
  • The London, Edinburgh, and Dublin Philosophical Magazine and Journal of Science, Vol. 2, Issue 11
  • DOI: 10.1080/14786440109462720

Direct Mapping of Ion Diffusion Times on LiCoO2 Surfaces with Nanometer Resolution
journal, January 2011

  • Guo, S.; Jesse, S.; Kalnaus, S.
  • Journal of The Electrochemical Society, Vol. 158, Issue 8
  • DOI: 10.1149/1.3604759

MPI-FAUN: An MPI-Based Framework for Alternating-Updating Nonnegative Matrix Factorization
journal, March 2018

  • Kannan, Ramakrishnan; Ballard, Grey; Park, Haesun
  • IEEE Transactions on Knowledge and Data Engineering, Vol. 30, Issue 3
  • DOI: 10.1109/TKDE.2017.2767592

Direct evidence of mesoscopic dynamic heterogeneities at the surfaces of ergodic ferroelectric relaxors
journal, February 2010

Matrix Factorization for Automatic Chemical Mapping from Electron Microscopic Spectral Imaging Datasets
journal, January 2016

  • Shiga, Motoki; Muto, Shunsuke; Tatsumi, Kazuyoshi
  • Transactions of the Materials Research Society of Japan, Vol. 41, Issue 4
  • DOI: 10.14723/tmrsj.41.333

Full-field X-ray reflection microscopy of epitaxial thin-films
journal, October 2014

  • Laanait, Nouamane; Zhang, Zhan; Schlepütz, Christian M.
  • Journal of Synchrotron Radiation, Vol. 21, Issue 6
  • DOI: 10.1107/S1600577514016555

Document clustering based on non-negative matrix factorization
conference, January 2003

  • Xu, Wei; Liu, Xin; Gong, Yihong
  • Proceedings of the 26th annual international ACM SIGIR conference on Research and development in informaion retrieval - SIGIR '03
  • DOI: 10.1145/860435.860485

Hyperspectral coherent Raman imaging - principle, theory, instrumentation, and applications to life sciences: Hyperspectral coherent Raman imaging
journal, December 2015

  • Kano, Hideaki; Segawa, Hiroki; Okuno, Masanari
  • Journal of Raman Spectroscopy, Vol. 47, Issue 1
  • DOI: 10.1002/jrs.4853

Hyperspectral surface-enhanced Raman imaging of labeled silver nanoparticles in single cells
journal, June 2005

  • Wabuyele, Musundi B.; Yan, Fei; Griffin, Guy D.
  • Review of Scientific Instruments, Vol. 76, Issue 6
  • DOI: 10.1063/1.1938667

TEM and EELS measurements of interface roughness in epitaxial Fe/MgO/Fe magnetic tunnel junctions
journal, April 2009

Disorder Identification in Hysteresis Data: Recognition Analysis of the Random-Bond–Random-Field Ising Model
journal, October 2009

Chemical State Evolution in Ferroelectric Films during Tip-Induced Polarization and Electroresistive Switching
journal, October 2016

  • Ievlev, Anton V.; Maksymovych, Petro; Trassin, Morgan
  • ACS Applied Materials & Interfaces, Vol. 8, Issue 43
  • DOI: 10.1021/acsami.6b10784

Joint Bayesian Endmember Extraction and Linear Unmixing for Hyperspectral Imagery
journal, November 2009

  • Dobigeon, N.; Moussaoui, S.; Coulon, M.
  • IEEE Transactions on Signal Processing, Vol. 57, Issue 11
  • DOI: 10.1109/TSP.2009.2025797

VisIRR: Visual analytics for information retrieval and recommendation with large-scale document data
conference, October 2014

  • Choo, Jaegul; Lee, Changhyun; Kim, Hannah
  • 2014 IEEE Conference on Visual Analytics Science and Technology (VAST)
  • DOI: 10.1109/VAST.2014.7042511

Spectral unmixing
journal, January 2002

  • Keshava, N.; Mustard, J. F.
  • IEEE Signal Processing Magazine, Vol. 19, Issue 1
  • DOI: 10.1109/79.974727

Imaging nanoscale lattice variations by machine learning of x-ray diffraction microscopy data
journal, August 2016

Electronic Properties of Isosymmetric Phase Boundaries in Highly Strained Ca-Doped BiFeO 3
journal, April 2014

Learning the parts of objects by non-negative matrix factorization
journal, October 1999

  • Lee, Daniel D.; Seung, H. Sebastian
  • Nature, Vol. 401, Issue 6755
  • DOI: 10.1038/44565

Memory Efficient Principal Component Analysis for the Dimensionality Reduction of Large Mass Spectrometry Imaging Data Sets
journal, March 2013

  • Race, Alan M.; Steven, Rory T.; Palmer, Andrew D.
  • Analytical Chemistry, Vol. 85, Issue 6
  • DOI: 10.1021/ac302528v

Spatial distribution of relaxation behavior on the surface of a ferroelectric relaxor in the ergodic phase
journal, October 2009

  • Kalinin, S. V.; Rodriguez, B. J.; Jesse, S.
  • Applied Physics Letters, Vol. 95, Issue 14
  • DOI: 10.1063/1.3242011

Principal Component Analysis of TOF-SIMS Images of Organic Monolayers
journal, November 2002

  • Biesinger, Mark C.; Paepegaey, Pierre-Yves; McIntyre, N. Stewart
  • Analytical Chemistry, Vol. 74, Issue 22
  • DOI: 10.1021/ac020311n

Fast rank-2 nonnegative matrix factorization for hierarchical document clustering
conference, January 2013

  • Kuang, Da; Park, Haesun
  • Proceedings of the 19th ACM SIGKDD international conference on Knowledge discovery and data mining - KDD '13
  • DOI: 10.1145/2487575.2487606

Automated Interpretation and Extraction of Topographic Information from Time of Flight Secondary Ion Mass Spectrometry Data
journal, December 2017

Matrix Factorization Techniques for Recommender Systems
journal, August 2009

  • Koren, Yehuda; Bell, Robert; Volinsky, Chris
  • Computer, Vol. 42, Issue 8
  • DOI: 10.1109/MC.2009.263

Nanoscale Hard X-Ray Microscopy Methods for Materials Studies
journal, July 2013

Algorithms for nonnegative matrix and tensor factorizations: a unified view based on block coordinate descent framework
journal, March 2013

Hyperopt: A Python Library for Optimizing the Hyperparameters of Machine Learning Algorithms
conference, January 2013

Sparse modeling of EELS and EDX spectral imaging data by nonnegative matrix factorization
journal, November 2016

Atomic-resolution spectroscopic imaging: past, present and future
journal, January 2009

  • Pennycook, S. J.; Varela, M.; Lupini, A. R.
  • Journal of Electron Microscopy, Vol. 58, Issue 3
  • DOI: 10.1093/jmicro/dfn030

X-ray micro-beam characterization of lattice rotations and distortions due to an individual dislocation
journal, November 2013

  • Hofmann, Felix; Abbey, Brian; Liu, Wenjun
  • Nature Communications, Vol. 4, Issue 1
  • DOI: 10.1038/ncomms3774

Spatial resolution, information limit, and contrast transfer in piezoresponse force microscopy
journal, June 2006

Near-Field Hyperspectral Optical Imaging
journal, February 2014

  • Bouillard, Jean-Sebastien G.; Dickson, Wayne; Wurtz, Gregory A.
  • ChemPhysChem, Vol. 15, Issue 4
  • DOI: 10.1002/cphc.201300826

Effect of molybdenum 4 d hole substitution in BaFe 2 As 2
journal, January 2012

Phases and Interfaces from Real Space Atomically Resolved Data: Physics-Based Deep Data Image Analysis
journal, August 2016

Data mining graphene: correlative analysis of structure and electronic degrees of freedom in graphenic monolayers with defects
journal, November 2016

Imaging with Mass Spectrometry
journal, October 1999

  • Pacholski, M. L.; Winograd, N.
  • Chemical Reviews, Vol. 99, Issue 10
  • DOI: 10.1021/cr980137w

Imaging mass spectrometry
journal, January 2007

  • McDonnell, Liam A.; Heeren, Ron M. A.
  • Mass Spectrometry Reviews, Vol. 26, Issue 4
  • DOI: 10.1002/mas.20124

Graph Regularized Nonnegative Matrix Factorization for Data Representation
journal, August 2011

  • Deng Cai,
  • IEEE Transactions on Pattern Analysis and Machine Intelligence, Vol. 33, Issue 8
  • DOI: 10.1109/TPAMI.2010.231

Deep data mining in a real space: separation of intertwined electronic responses in a lightly doped BaFe 2 As 2
journal, October 2016

Mapping chemical and bonding information using multivariate analysis of electron energy-loss spectrum images
journal, October 2006

Role of magnetism in superconductivity of BaF e 2 A s 2 : Study of 5 d Au-doped crystals
journal, September 2015

Bounded matrix factorization for recommender system
journal, December 2013

  • Kannan, Ramakrishnan; Ishteva, Mariya; Park, Haesun
  • Knowledge and Information Systems, Vol. 39, Issue 3
  • DOI: 10.1007/s10115-013-0710-2

Hyperspectral Imaging with Stimulated Raman Scattering by Chirped Femtosecond Lasers
journal, November 2012

  • Fu, Dan; Holtom, Gary; Freudiger, Christian
  • The Journal of Physical Chemistry B, Vol. 117, Issue 16
  • DOI: 10.1021/jp308938t

Three-dimensional X-ray structural microscopy with submicrometre resolution
journal, February 2002

  • Larson, B. C.; Yang, Wenge; Ice, G. E.
  • Nature, Vol. 415, p. 887-890
  • DOI: 10.1038/415887a

Direct Mapping of Ionic Transport in a Si Anode on the Nanoscale: Time Domain Electrochemical Strain Spectroscopy Study
journal, November 2011

  • Jesse, Stephen; Balke, Nina; Eliseev, Eugene
  • ACS Nano, Vol. 5, Issue 12
  • DOI: 10.1021/nn203141g

Spatially Inhomogeneous Metal-Insulator Transition in Doped Manganites
journal, September 1999

Full data acquisition in Kelvin Probe Force Microscopy: Mapping dynamic electric phenomena in real space
journal, August 2016

  • Collins, Liam; Belianinov, Alex; Somnath, Suhas
  • Scientific Reports, Vol. 6, Issue 1
  • DOI: 10.1038/srep30557

Misfit strain relaxation by dislocations in SrRuO3/SrTiO3 (001) heteroepitaxy
journal, May 2004

  • Oh, Sang Ho; Park, Chan Gyung
  • Journal of Applied Physics, Vol. 95, Issue 9
  • DOI: 10.1063/1.1690484

Atom-by-atom spectroscopy at graphene edge
journal, December 2010

Reconstruction of surface potential from Kelvin probe force microscopy images
journal, June 2013

The Race to X-ray Microbeam and Nanobeam Science
journal, December 2011

On the Equivalence of Nonnegative Matrix Factorization and Spectral Clustering
conference, January 2014

  • Ding, Chris; He, Xiaofeng; Simon, Horst D.
  • Proceedings of the 2005 SIAM International Conference on Data Mining
  • DOI: 10.1137/1.9781611972757.70

3D Chemical Image using TOF-SIMS Revealing the Biopolymer Component Spatial and Lateral Distributions in Biomass
journal, October 2012

  • Jung, Seokwon; Foston, Marcus; Kalluri, Udaya C.
  • Angewandte Chemie International Edition, Vol. 51, Issue 48
  • DOI: 10.1002/anie.201205243

Hyperspectral optical imaging of human iris in vivo: characteristics of reflectance spectra
journal, January 2011

  • Medina, José M.; Pereira, Luís M.; Correia, Hélder T.
  • Journal of Biomedical Optics, Vol. 16, Issue 7
  • DOI: 10.1117/1.3595710

Li-ion dynamics and reactivity on the nanoscale
journal, November 2011

    Sort by:
    [ × clear filter / sort ]

    Similar Records in DOE PAGES and OSTI.GOV collections: