TopFusion: Using Topological Feature Space for Fusion and Imputation in Multi-Modal Data

Conference · Mon Aug 14 00:00:00 EDT 2023

DOI:https://doi.org/10.1109/CVPRW59228.2023.00067· OSTI ID:2008416

Myers, Audun D. ^[1]; Kvinge, Henry J.  ^[1]; Emerson, Tegan H. ^[1]

---

1. BATTELLE (PACIFIC NW LAB)

We present a novel multi-modal data fusion technique using topological features. The method, TopFusion, leverages the flexibility of topological data analysis tools (namely persistent homology and persistence images) to map multi-modal datasets into a common feature space by forming a new multi-channel persistence image. Each channel in the image is representative of a view of the data from a modality-dependent filtration. We demonstrate that the topological perspective we take allows for more effective data reconstruction, i.e. imputation. In particular, by performing imputation in topological feature space we are able to outperform the same imputation techniques applied to raw data or alternatively derived features. We show that TopFusion representations can be used as input to downstream deep learning-based computer vision models and doing so achieves comparable performance to other fusion methods for classification on two multi-modal datasets.

• Research Organization: Pacific Northwest National Laboratory (PNNL), Richland, WA (United States)
• Sponsoring Organization: USDOE
• DOE Contract Number: AC05-76RL01830
• OSTI ID: 2008416
• Report Number(s): PNNL-SA-183136
• Resource Relation: Conference: IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW 2023), June 17-24, 2023, Vancouver, B.C., Canada
• Country of Publication: United States
• Language: English