Unsupervised physics-informed disentanglement of multimodal data
- Sandia National Laboratories (SNL-NM), Albuquerque, NM (United States)
- Univ. of Pennsylvania, Philadelphia, PA (United States)
- Sandia National Laboratories (SNL-NM), Albuquerque, NM (United States); Arizona State Univ., Tempe, AZ (United States)
- Arizona State Univ., Tempe, AZ (United States)
- Northwestern Univ., Evanston, IL (United States)
- Center for Integrated Nanotechnologies
Here, we introduce physics-informed multimodal autoencoders (PIMA) - a variational inference framework for discovering shared information in multimodal datasets. Individual modalities are embedded into a shared latent space and fused through a product-of-experts formulation, enabling a Gaussian mixture prior to identify shared features. Sampling from clusters allows cross-modal generative modeling, with a mixture-of-experts decoder that imposes inductive biases from prior scientific knowledge and thereby imparts structured disentanglement of the latent space. This approach enables cross-modal inference and the discovery of features in high-dimensional heterogeneous datasets. Consequently, this approach provides a means to discover fingerprints in multimodal scientific datasets and to avoid traditional bottlenecks related to high-fidelity measurement and characterization of scientific datasets.
- Research Organization:
- Sandia National Laboratories (SNL-NM), Albuquerque, NM (United States)
- Sponsoring Organization:
- USDOE Office of Science (SC), Advanced Scientific Computing Research (ASCR); USDOE Laboratory Directed Research and Development (LDRD) Program; National Science Foundation (NSF)
- Grant/Contract Number:
- NA0003525
- OSTI ID:
- 2441141
- Report Number(s):
- SAND--2024-12122J
- Journal Information:
- Foundations of Data Science, Journal Name: Foundations of Data Science Journal Issue: 1 Vol. 7; ISSN 2639-8001
- Publisher:
- AIMSCopyright Statement
- Country of Publication:
- United States
- Language:
- English
Similar Records
Disentangling ferroelectric domain wall geometries and pathways in dynamic piezoresponse force microscopy via unsupervised machine learning
Towards interpretable Cryo-EM: disentangling latent spaces of molecular conformations