Complex optimization for big computational and experimental neutron datasets

Bao, Feng; Archibald, Richard; Niedziela, Jennifer; Bansal, Dipanshu; Delaire, Olivier

doi:10.1088/0957-4484/27/48/484002

Title: Complex optimization for big computational and experimental neutron datasets

Abstract

Here, we present a framework to use high performance computing to determine accurate solutions to the inverse optimization problem of big experimental data against computational models. We demonstrate how image processing, mathematical regularization, and hierarchical modeling can be used to solve complex optimization problems on big data. We also demonstrate how both model and data information can be used to further increase solution accuracy of optimization by providing confidence regions for the processing and regularization algorithms. Finally, we use the framework in conjunction with the software package SIMPHONIES to analyze results from neutron scattering experiments on silicon single crystals, and refine first principles calculations to better describe the experimental data.

Authors:

Bao, Feng ^[1]; Oak Ridge National Lab. ^[2]; Archibald, Richard ^[3]; Niedziela, Jennifer ^[4]; Bansal, Dipanshu ^[4]; Delaire, Olivier ^[5]

Univ. of Tennessee, Chattanooga, TN (United States). Dept. of Mathematics
(ORNL), Oak Ridge, TN (United States). Computer Science and Mathematics Division
Oak Ridge National Lab. (ORNL), Oak Ridge, TN (United States). Computer Science and Mathematics Division
Oak Ridge National Lab. (ORNL), Oak Ridge, TN (United States). Materials Science & Technology Division
Duke Univ., Durham, NC (United States). Dept. of Mechanical Engineering and Materials Science

Publication Date:: Mon Nov 07 00:00:00 EST 2016

Research Org.:: Oak Ridge National Lab. (ORNL), Oak Ridge, TN (United States). Spallation Neutron Source (SNS); Energy Frontier Research Centers (EFRC) (United States). Solid-State Solar-Thermal Energy Conversion Center (S3TEC)

Sponsoring Org.:: USDOE Office of Science (SC), Basic Energy Sciences (BES)

OSTI Identifier:: 1354653

Alternate Identifier(s):: OSTI ID: 1331028

Grant/Contract Number:: AC05-00OR22725; SC0001299; FG02-09ER46577; SC0016166

Resource Type:: Accepted Manuscript

Journal Name:: Nanotechnology

Additional Journal Information:: Journal Volume: 27; Journal Issue: 48; Journal ID: ISSN 0957-4484

Publisher:: IOP Publishing

Country of Publication:: United States

Language:: English

Subject:: 97 MATHEMATICS AND COMPUTING; 36 MATERIALS SCIENCE; image processing; inelastic neutron scattering; hierarchical optimization; mathematical regularization

Citation Formats


                    Bao, Feng, Oak Ridge National Lab., Archibald, Richard, Niedziela, Jennifer, Bansal, Dipanshu, and Delaire, Olivier. Complex optimization for big computational and experimental neutron datasets.  United States: N. p., 2016. 
Web.  doi:10.1088/0957-4484/27/48/484002.

Copy to clipboard


                    Bao, Feng, Oak Ridge National Lab., Archibald, Richard, Niedziela, Jennifer, Bansal, Dipanshu, & Delaire, Olivier. Complex optimization for big computational and experimental neutron datasets.  United States.  https://doi.org/10.1088/0957-4484/27/48/484002

Copy to clipboard


                    Bao, Feng, Oak Ridge National Lab., Archibald, Richard, Niedziela, Jennifer, Bansal, Dipanshu, and Delaire, Olivier. Mon .  
"Complex optimization for big computational and experimental neutron datasets".  United States.  https://doi.org/10.1088/0957-4484/27/48/484002.  https://www.osti.gov/servlets/purl/1354653.

Copy to clipboard


                    
@article{osti_1354653,

  title        = {Complex optimization for big computational and experimental neutron datasets},

  author       = {Bao, Feng and Oak Ridge National Lab. and Archibald, Richard and Niedziela, Jennifer and Bansal, Dipanshu and Delaire, Olivier},

  abstractNote = {Here, we present a framework to use high performance computing to determine accurate solutions to the inverse optimization problem of big experimental data against computational models. We demonstrate how image processing, mathematical regularization, and hierarchical modeling can be used to solve complex optimization problems on big data. We also demonstrate how both model and data information can be used to further increase solution accuracy of optimization by providing confidence regions for the processing and regularization algorithms. Finally, we use the framework in conjunction with the software package SIMPHONIES to analyze results from neutron scattering experiments on silicon single crystals, and refine first principles calculations to better describe the experimental data.},

  doi          = {10.1088/0957-4484/27/48/484002},

  journal      = {Nanotechnology},

  number       = 48,

  volume       = 27,

  place        = {United States},

  year         = {Mon Nov 07 00:00:00 EST 2016},

  month        = {Mon Nov 07 00:00:00 EST 2016}

}

Copy to clipboard

Journal Article:

Free Publicly Available Full Text

Accepted Manuscript (DOE)

Publisher's Version of Record

https://doi.org/10.1088/0957-4484/27/48/484002

Other availability

Search WorldCat to find libraries that may hold this journal

Citation Metrics:

Cited by: 3 works

Citation information provided by
Web of Science

Figures / Tables:

Figure 1: An example of image obtained from experimental data for Silicon (in logarithmic scale). H is the reciprocal lattice unit.

All figures and tables (7 total)

Save / Share:

Export Metadata

Save to My Library

Works referenced in this record:

Hierarchical optimization for neutron scattering problems
journal, June 2016

Bao, Feng; Archibald, Rick; Bansal, Dipanshu
Journal of Computational Physics, Vol. 315
DOI: 10.1016/j.jcp.2016.03.017

Signal recovery from random projections
conference, March 2005

Candes, Emmanuel J.; Romberg, Justin K.
Electronic Imaging 2005, SPIE Proceedings
DOI: 10.1117/12.600722

Robust uncertainty principles: exact signal reconstruction from highly incomplete frequency information
journal, February 2006

Candes, E.J.; Romberg, J.; Tao, T.
IEEE Transactions on Information Theory, Vol. 52, Issue 2, p. 489-509
DOI: 10.1109/TIT.2005.862083

Compressed sensing
journal, April 2006

Donoho, D. L.
IEEE Transactions on Information Theory, Vol. 52, Issue 4
DOI: 10.1109/TIT.2006.871582

Design and operation of the wide angular-range chopper spectrometer ARCS at the Spallation Neutron Source
journal, January 2012

Abernathy, D. L.; Stone, M. B.; Loguillo, M. J.
Review of Scientific Instruments, Vol. 83, Issue 1
DOI: 10.1063/1.3680104

Mantid—Data analysis and visualization package for neutron scattering and $μ$ SR experiments
journal, November 2014

Arnold, O.; Bilheux, J. C.; Borreguero, J. M.
Nuclear Instruments and Methods in Physics Research Section A: Accelerators, Spectrometers, Detectors and Associated Equipment, Vol. 764
DOI: 10.1016/j.nima.2014.07.029

Twisting phonons in complex crystals with quasi-one-dimensional substructures
journal, April 2015

Chen, Xi; Weathers, Annie; Carrete, Jesús
Nature Communications, Vol. 6, Issue 1
DOI: 10.1038/ncomms7723

Heavy-impurity resonance, hybridization, and phonon spectral functions in ${Fe}_{1 - x} M_{x} Si (M = Ir, Os)$
journal, March 2015

Delaire, O.; Al-Qasir, I. I.; May, A. F.
Physical Review B, Vol. 91, Issue 9
DOI: 10.1103/PhysRevB.91.094307

Electron-phonon coupling and thermal transport in the thermoelectric compound ${Mo}_{3} {Sb}_{7 - x} {Te}_{x}$
journal, December 2015

Bansal, Dipanshu; Li, Chen W.; Said, Ayman H.
Physical Review B, Vol. 92, Issue 21
DOI: 10.1103/PhysRevB.92.214301

Orbitally driven giant phonon anharmonicity in SnSe
journal, October 2015

Li, C. W.; Hong, J.; May, A. F.
Nature Physics, Vol. 11, Issue 12
DOI: 10.1038/nphys3492

Glass-like phonon scattering from a spontaneous nanostructure in AgSbTe2
journal, June 2013

Ma, J.; Delaire, O.; May, A. F.
Nature Nanotechnology, Vol. 8, Issue 6
DOI: 10.1038/nnano.2013.95

Phonon scattering rates and atomic ordering in ${Ag}_{1 - x} {Sb}_{1 + x} {Te}_{2 + x} (x = 0, 0.1, 0.2)$ investigated with inelastic neutron scattering and synchrotron diffraction
journal, October 2014

Ma, J.; Delaire, O.; Specht, E. D.
Physical Review B, Vol. 90, Issue 13
DOI: 10.1103/PhysRevB.90.134303

A comparison of four direct geometry time-of-flight spectrometers at the Spallation Neutron Source
journal, April 2014

Stone, M. B.; Niedziela, J. L.; Abernathy, D. L.
Review of Scientific Instruments, Vol. 85, Issue 4
DOI: 10.1063/1.4870050

The Split Bregman Method for L1-Regularized Problems
journal, January 2009

Goldstein, Tom; Osher, Stanley
SIAM Journal on Imaging Sciences, Vol. 2, Issue 2
DOI: 10.1137/080725891

Efficiency of ab-initio total energy calculations for metals and semiconductors using a plane-wave basis set
journal, July 1996

Kresse, G.; Furthmüller, J.
Computational Materials Science, Vol. 6, Issue 1, p. 15-50
DOI: 10.1016/0927-0256(96)00008-0

Efficient iterative schemes for ab initio total-energy calculations using a plane-wave basis set
journal, October 1996

Kresse, G.; Furthmüller, J.
Physical Review B, Vol. 54, Issue 16, p. 11169-11186
DOI: 10.1103/PhysRevB.54.11169

Ab initiomolecular dynamics for liquid metals
journal, January 1993

Kresse, G.; Hafner, J.
Physical Review B, Vol. 47, Issue 1, p. 558-561
DOI: 10.1103/PhysRevB.47.558

Horace : Software for the analysis of data from single crystal spectroscopy experiments at time-of-flight neutron instruments
journal, October 2016

Ewings, R. A.; Buts, A.; Le, M. D.
Nuclear Instruments and Methods in Physics Research Section A: Accelerators, Spectrometers, Detectors and Associated Equipment, Vol. 834
DOI: 10.1016/j.nima.2016.07.036

First principles phonon calculations in materials science
journal, November 2015

Togo, Atsushi; Tanaka, Isao
Scripta Materialia, Vol. 108
DOI: 10.1016/j.scriptamat.2015.07.021

Bregman Iterative Algorithms for $\ell_1$-Minimization with Applications to Compressed Sensing
journal, January 2008

Yin, Wotao; Osher, Stanley; Goldfarb, Donald
SIAM Journal on Imaging Sciences, Vol. 1, Issue 1
DOI: 10.1137/070703983

Figures / Tables found in this record:

Figures/Tables have been extracted from DOE-funded journal article accepted manuscripts.

Similar Records in DOE PAGES and OSTI.GOV collections:

Sequential-Optimization-Based Framework for Robust Modeling and Design of Heterogeneous Catalytic Systems

Journal Article Rangarajan, Srinivas ; Maravelias, Christos T. ; Mavrikakis, Manos - Journal of Physical Chemistry. C

Here, we present a general optimization-based framework for (i) ab initio and experimental data driven mechanistic modeling and (ii) optimal catalyst design of heterogeneous catalytic systems. Both cases are formulated as a nonlinear optimization problem that is subject to a mean-field microkinetic model and thermodynamic consistency requirements as constraints, for which we seek sparse solutions through a ridge (L₂ regularization) penalty. The solution procedure involves an iterative sequence of forward simulation of the differential algebraic equations pertaining to the microkinetic model using a numerical tool capable of handling stiff systems, sensitivity calculations using linear algebra, and gradient-based nonlinear optimization. Amore »« less
Cited by 39
https://doi.org/10.1021/acs.jpcc.7b08089
A Real-Time Big Data Control-Theoretical Framework for Cyber-Physical-Human Systems [Book Chapter]

Other Gusrialdi, Azwirman ; Xu, Ying ; Qu, Zhihua ; ...

It is known that cyber-physical-human systems naturally arise from interdependent infrastructure systems and smart connected communities. Such applications require ubiquitous information sensing and processing, intelligent machine-to-machine communication for a seamless coordination, as well as intelligent interactions between humans and machines. This chapter presents a control-theoretical framework to model heterogeneous physical dynamic systems, information and communication, as well as cooperative controls and/or distributed optimization of such interconnected systems. It is shown that efficient analytical and computational algorithms can be modularly designed and hierarchically implemented to operate and optimize cyber-physical-human systems, first to quantify individually the input–output relationship of nonlinear dynamic behaviorsmore »« less
https://doi.org/10.1007/978-3-030-25446-9_7
Parallel Hybrid Metaheuristics with Distributed Intensification and Diversification for Large-scale Optimization in Big Data Statistical Analysis

Conference Cho, Wendy K. Tam ; Liu, Yan

Important insights into many problems that are traditionally analyzed via statistical models can be obtained by re-formulating and evaluating within a large-scale optimization framework. The theoretical underpinnings of the statistical model often shift the goal of the solution space traversal from a traditional search for a single optimal solution to a traversal with the purpose of yielding a set of high quality, independent solutions. We examine statistical frameworks with astronomical solution spaces where the independence requirement constitutes a significant additional challenge for standard optimization methodologies. We design a hybrid metaheuristic with specialized intensification and diversification protocols in the base searchmore »« less
https://doi.org/10.1109/BigData47090.2019.9006045

Full Text Available
Boosting Big National Lab Data

Journal Article Kleese van Dam, Kerstin - Datanami

Introduction: Big data. Love it or hate it, solving the world’s most intractable problems requires the ability to make sense of huge and complex sets of data and do it quickly. Speeding up the process – from hours to minutes or from weeks to days – is key to our success. One major source of such big data are physical experiments. As many will know, these physical experiments are commonly used to solve challenges in fields such as energy security, manufacturing, medicine, pharmacology, environmental protection and national security. Experiments use different instruments and sensor types to research for example themore »« less
Big Data Analysis of Massive PMU Datasets: A Data Platform Perspective

Conference Kumar, Vijay S. ; Wang, Tianyi ; Aggour, Kareem S. ; ... - 2021 IEEE Power & Energy Society Innovative Smart Grid Technologies Conference (ISGT)

The discovery of `event signatures' and useful insights from very large historical Phasor Measurement Unit (PMU) datasets is predicated on offline Big Data analysis approaches that rely on the generation of predictive features on a massive scale. This paper presents lessons learned from a data platform perspective towards reducing barriers to adoption of Big Data analytics against a real dataset of almost half a trillion data points drawn from over 400 PMUs distributed across the North American power grid. We demonstrate software abstractions and targeted performance optimizations that can lead to significant productivity gains for power systems researchers seeking tomore »« less
https://doi.org/10.1109/isgt49243.2021.9372203

Full Text Available

Similar Records

Title: Complex optimization for big computational and experimental neutron datasets

Abstract

Citation Formats

Figures / Tables:

Hierarchical optimization for neutron scattering problems journal, June 2016

Signal recovery from random projections conference, March 2005

Robust uncertainty principles: exact signal reconstruction from highly incomplete frequency information journal, February 2006

Compressed sensing journal, April 2006

Design and operation of the wide angular-range chopper spectrometer ARCS at the Spallation Neutron Source journal, January 2012

Mantid—Data analysis and visualization package for neutron scattering and μ SR experiments journal, November 2014

Twisting phonons in complex crystals with quasi-one-dimensional substructures journal, April 2015

Heavy-impurity resonance, hybridization, and phonon spectral functions in Fe 1 − x M x Si ( M = Ir , Os ) journal, March 2015

Electron-phonon coupling and thermal transport in the thermoelectric compound Mo 3 Sb 7 − x Te x journal, December 2015

Orbitally driven giant phonon anharmonicity in SnSe journal, October 2015

Glass-like phonon scattering from a spontaneous nanostructure in AgSbTe2 journal, June 2013

Phonon scattering rates and atomic ordering in Ag 1 − x Sb 1 + x Te 2 + x ( x = 0 , 0.1 , 0.2 ) investigated with inelastic neutron scattering and synchrotron diffraction journal, October 2014

A comparison of four direct geometry time-of-flight spectrometers at the Spallation Neutron Source journal, April 2014

The Split Bregman Method for L1-Regularized Problems journal, January 2009

Efficiency of ab-initio total energy calculations for metals and semiconductors using a plane-wave basis set journal, July 1996

Efficient iterative schemes for ab initio total-energy calculations using a plane-wave basis set journal, October 1996

Ab initiomolecular dynamics for liquid metals journal, January 1993

Horace : Software for the analysis of data from single crystal spectroscopy experiments at time-of-flight neutron instruments journal, October 2016

First principles phonon calculations in materials science journal, November 2015

Bregman Iterative Algorithms for $\ell_1$-Minimization with Applications to Compressed Sensing journal, January 2008

Hierarchical optimization for neutron scattering problems
journal, June 2016

Signal recovery from random projections
conference, March 2005

Robust uncertainty principles: exact signal reconstruction from highly incomplete frequency information
journal, February 2006

Compressed sensing
journal, April 2006

Design and operation of the wide angular-range chopper spectrometer ARCS at the Spallation Neutron Source
journal, January 2012

Mantid—Data analysis and visualization package for neutron scattering and $μ$ SR experiments
journal, November 2014

Twisting phonons in complex crystals with quasi-one-dimensional substructures
journal, April 2015

Heavy-impurity resonance, hybridization, and phonon spectral functions in ${Fe}_{1 - x} M_{x} Si (M = Ir, Os)$
journal, March 2015

Electron-phonon coupling and thermal transport in the thermoelectric compound ${Mo}_{3} {Sb}_{7 - x} {Te}_{x}$
journal, December 2015

Orbitally driven giant phonon anharmonicity in SnSe
journal, October 2015

Glass-like phonon scattering from a spontaneous nanostructure in AgSbTe2
journal, June 2013

Phonon scattering rates and atomic ordering in ${Ag}_{1 - x} {Sb}_{1 + x} {Te}_{2 + x} (x = 0, 0.1, 0.2)$ investigated with inelastic neutron scattering and synchrotron diffraction
journal, October 2014

A comparison of four direct geometry time-of-flight spectrometers at the Spallation Neutron Source
journal, April 2014

The Split Bregman Method for L1-Regularized Problems
journal, January 2009

Efficiency of ab-initio total energy calculations for metals and semiconductors using a plane-wave basis set
journal, July 1996

Efficient iterative schemes for ab initio total-energy calculations using a plane-wave basis set
journal, October 1996

Ab initiomolecular dynamics for liquid metals
journal, January 1993

Horace : Software for the analysis of data from single crystal spectroscopy experiments at time-of-flight neutron instruments
journal, October 2016

First principles phonon calculations in materials science
journal, November 2015

Bregman Iterative Algorithms for $\ell_1$-Minimization with Applications to Compressed Sensing
journal, January 2008