Transfer learning suppresses simulation bias in predictive models built from sparse, multi-modal data
- Lawrence Livermore National Lab. (LLNL), Livermore, CA (United States)
Many problems in science, engineering, and business require making predictions based on very few observations. To build a robust predictive model, these sparse data may need to be augmented with simulated data, especially when the design space is multidimensional. Simulations, however, often suffer from an inherent bias. Estimation of this bias may be poorly constrained not only because of data sparsity, but also because traditional predictive models fit only one type of observations, such as scalars or images, instead of all available data modalities, which might have been acquired and simulated at great cost. We combine recent developments in deep learning to build more robust predictive models from multimodal data with a recent, novel technique to suppress the bias, and extend it to take into account multiple data modalities. First, an initial, simulation-trained, neural network surrogate model learns important correlations between different data modalities and between simulation inputs and outputs. Then, the model is partially retrained, or transfer learned, to fit the observations. Using fewer than 10 inertial confinement fusion experiments for retraining, we demonstrate that this technique systematically improves simulation predictions while a simple output calibration makes predictions worse. We also offer extensive cross-validation with real and synthetic data to support our findings. The transfer learning method can be applied to other problems that require transferring knowledge from simulations to the domain of real observations. In conclusion, this paper opens up the path to model calibration using multiple data types, which have traditionally been ignored in predictive models.
- Research Organization:
- Lawrence Livermore National Lab. (LLNL), Livermore, CA (United States)
- Sponsoring Organization:
- USDOE National Nuclear Security Administration (NNSA)
- DOE Contract Number:
- AC52-07NA27344
- OSTI ID:
- 1810642
- Report Number(s):
- LLNL-JRNL-821489; 1028883
- Journal Information:
- IEEE Transactions on Neural Networks and Learning Systems, Related Information: Originally submitted to IEEE Transactions on Neural Networks and Learning Systems, but later version published in Machine Learning: Science and Technology
- Country of Publication:
- United States
- Language:
- English
Similar Records
Using machine learning to derive cloud condensation nuclei number concentrations from commonly available measurements
Robust Group Subspace Recovery: A New Approach for Multi-Modality Data Fusion