Adversarial Perturbations Are Not So Weird: Entanglement of Robust and Non-Robust Features in Neural Network Classifiers

Technical Report · Fri Oct 01 04:00:00 EDT 2021

DOI:https://doi.org/10.2172/1823733· OSTI ID:1823733

Springer, Jacob Mitchell ^[1]; Mitchell, Melanie ^[2]; Kenyon, Garrett ^[1]

---

1. Los Alamos National Lab. (LANL), Los Alamos, NM (United States)
2. Santa Fe Institute, NM (United States)

Neural networks trained on visual data are well-known to be vulnerable to often imperceptible adversarial perturbations. The reasons for this vulnerability are still being debated in the literature. Recently Ilyas et al. (2019) showed that this vulnerability arises, in part, because neural network classifiers rely on highly predictive but brittle “non-robust” features. In this paper we extend the work of Ilyas et al. by investigating the nature of the input patterns that give rise to these features. In particular, we hypothesize that in a neural network trained in a standard way, non-robust features respond to small, “non-semantic” patterns that are typically entangled with larger, robust patterns, known to be more human-interpretable, as opposed to solely responding to statistical artifacts in a dataset. Thus, adversarial examples can be formed via minimal perturbations to these small, entangled patterns. In addition, we demonstrate a corollary of our hypothesis: robust classifiers are more effective than standard (non-robust) ones as a source for generating transferable adversarial examples in both the untargeted and targeted settings. The results we present in this paper provide new insight into the nature of the non-robust features responsible for adversarial vulnerability of neural network classifiers.

• Research Organization: Los Alamos National Laboratory (LANL), Los Alamos, NM (United States)
• Sponsoring Organization: USDOE Laboratory Directed Research and Development (LDRD) Program; USDOE National Nuclear Security Administration (NNSA)
• DOE Contract Number: 89233218CNA000001
• OSTI ID: 1823733
• Report Number(s): LA-UR-21-21945
• Country of Publication: United States
• Language: English

Related Subjects

97 MATHEMATICS AND COMPUTING
Computer Science