A robust estimator of mutual information for deep learning interpretability

Journal Article · 11 April 2023 · Machine Learning: Science and Technology

DOI: https://doi.org/10.1088/2632-2153/acc444 · OSTI ID:1969147

Piras, Davide ; Peiris, Hiranya V.; Pontzen, Andrew; Lucie-Smith, Luisa; Guo, Ningyuan; Nord, Brian

Abstract We develop the use of mutual information (MI), a well-established metric in information theory, to interpret the inner workings of deep learning (DL) models. To accurately estimate MI from a finite number of samples, we present GMM-MI (pronounced ‘Jimmie’), an algorithm based on Gaussian mixture models that can be applied to both discrete and continuous settings. GMM-MI is computationally efficient, robust to the choice of hyperparameters and provides the uncertainty on the MI estimate due to the finite sample size. We extensively validate GMM-MI on toy data for which the ground truth MI is known, comparing its performance against established MI estimators. We then demonstrate the use of our MI estimator in the context of representation learning, working with synthetic data and physical datasets describing highly non-linear processes. We train DL models to encode high-dimensional data within a meaningful compressed (latent) representation, and use GMM-MI to quantify both the level of disentanglement between the latent variables, and their association with relevant physical quantities, thus unlocking the interpretability of the latent representation. We make GMM-MI publicly available in this GitHub repository.

• Research Organization: Fermi National Accelerator Laboratory (FNAL), Batavia, IL (United States)
• Sponsoring Organization: European Research Council (ERC); Göran Gustafsson Foundation for Research in Natural Sciences and Medicine; National Science Foundation (NSF); Simons Foundation; Swiss National Science Foundation (SNSF); UCL Graduate Research Scholarship (GRS); UCL Overseas Research Scholarship (ORS); USDOE; USDOE Office of Science (SC), High Energy Physics (HEP)
• Grant/Contract Number: AC02-07CH11359
• OSTI ID: 1969147
• Report Number(s): FERMILAB-PUB-22-794-SCD; arXiv:2211.00024
• Journal Information: Machine Learning: Science and Technology, Journal Name: Machine Learning: Science and Technology Journal Issue: 2 Vol. 4; ISSN 2632-2153
• Publisher: IOP PublishingCopyright Statement
• Country of Publication: United Kingdom
• Language: English