A Visual Designer of Layer‐wise Relevance Propagation Models

Huang, Xinyi; Jamonnak, Suphanut; Zhao, Ye; Wu, Tsung Heng; Xu, Wei

doi:10.1111/cgf.14302

Title: A Visual Designer of Layer‐wise Relevance Propagation Models

Journal Article · 29 June 2021 · Computer Graphics Forum

DOI: https://doi.org/10.1111/cgf.14302 · OSTI ID:1808611

Huang, Xinyi ^[1]; Jamonnak, Suphanut ^[1]; Zhao, Ye ^[1]; Wu, Tsung Heng ^[1]; Xu, Wei ^[2]

Kent State University
Brookhaven National Laboratory

Abstract Layer‐wise Relevance Propagation (LRP) is an emerging and widely‐used method for interpreting the prediction results of convolutional neural networks (CNN). LRP developers often select and employ different relevance backpropagation rules and parameters, to compute relevance scores on input images. However, there exists no obvious solution to define a “best” LRP model. A satisfied model is highly reliant on pertinent images and designers' goals. We develop a visual model designer, named as VisLRPDesigner, to overcome the challenges in the design and use of LRP models. Various LRP rules are unified into an integrated framework with an intuitive workflow of parameter setup. VisLRPDesigner thus allows users to interactively configure and compare LRP models. It also facilitates relevance‐based visual analysis with two important functions: relevance‐based pixel flipping and neuron ablation. Several use cases illustrate the benefits of VisLRPDesigner. The usability and limitation of the visual designer is evaluated by LRP users.

View Accepted Manuscript (Publisher)

Cite

Export

Save

Sponsoring Organization:: USDOE

OSTI ID:: 1808611

Journal Information:: Computer Graphics Forum, Journal Name: Computer Graphics Forum Journal Issue: 3 Vol. 40; ISSN 0167-7055

Publisher:: Wiley-BlackwellCopyright Statement

Country of Publication:: Netherlands

Language:: English

References (32)

Visualizing and Understanding Convolutional Networks Zeiler, Matthew D.; Fergus, Rob Computer Vision – ECCV 2014 https://doi.org/10.1007/978-3-319-10590-1_53	book	January 2014
Grad-CAM: Visual Explanations from Deep Networks via Gradient-Based Localization Selvaraju, Ramprasaath R.; Cogswell, Michael; Das, Abhishek International Journal of Computer Vision, Vol. 128, Issue 2 https://doi.org/10.1007/s11263‐019‐01228‐7	journal	October 2019
Methods for interpreting and understanding deep neural networks Montavon, Grégoire; Samek, Wojciech; Müller, Klaus-Robert Digital Signal Processing, Vol. 73 https://doi.org/10.1016/j.dsp.2017.10.011	journal	February 2018
Interpretable deep neural networks for single-trial EEG classification Sturm, Irene; Lapuschkin, Sebastian; Samek, Wojciech Journal of Neuroscience Methods, Vol. 274 https://doi.org/10.1016/j.jneumeth.2016.10.008	journal	December 2016
Explaining nonlinear classification decisions with deep Taylor decomposition Montavon, Grégoire; Lapuschkin, Sebastian; Binder, Alexander Pattern Recognition, Vol. 65 https://doi.org/10.1016/j.patcog.2016.11.008	journal	May 2017
Unmasking Clever Hans predictors and assessing what machines really learn Lapuschkin, Sebastian; Wäldchen, Stephan; Binder, Alexander Nature Communications, Vol. 10, Issue 1 https://doi.org/10.1038/s41467‐019‐08987‐4	journal	March 2019
ImageNet: A large-scale hierarchical image database Deng, Jia; Dong, Wei; Socher, Richard 2009 IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops (CVPR Workshops), 2009 IEEE Conference on Computer Vision and Pattern Recognition https://doi.org/10.1109/CVPR.2009.5206848	conference	June 2009
Learning Deep Features for Discriminative Localization Zhou, Bolei; Khosla, Aditya; Lapedriza, Agata 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) https://doi.org/10.1109/CVPR.2016.319	conference	June 2016
Deep Residual Learning for Image Recognition He, Kaiming; Zhang, Xiangyu; Ren, Shaoqing 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) https://doi.org/10.1109/CVPR.2016.90	conference	June 2016
Grad-CAM: Visual Explanations from Deep Networks via Gradient-Based Localization Selvaraju, Ramprasaath R.; Cogswell, Michael; Das, Abhishek 2017 IEEE International Conference on Computer Vision (ICCV) https://doi.org/10.1109/ICCV.2017.74	conference	October 2017
Understanding and Comparing Deep Neural Networks for Age and Gender Classification Samek, Wojciech; Binder, Alexander; Lapuschkin, Sebastian 2017 IEEE International Conference on Computer Vision Workshop (ICCVW), 2017 IEEE International Conference on Computer Vision Workshops (ICCVW) https://doi.org/10.1109/ICCVW.2017.191	conference	October 2017
Explaining Convolutional Neural Networks using Softmax Gradient Layer-wise Relevance Propagation Iwana, Brian Kenji; Kuroki, Ryohei; Uchida, Seiichi 2019 IEEE/CVF International Conference on Computer Vision Workshop (ICCVW) https://doi.org/10.1109/ICCVW.2019.00513	conference	October 2019
Free-Lunch Saliency via Attention in Atari Agents Nikulin, Dmitry; Ianina, Anastasia; Aliev, Vladimir 2019 IEEE/CVF International Conference on Computer Vision Workshop (ICCVW) https://doi.org/10.1109/ICCVW.2019.00522	conference	October 2019
Interpreting Undesirable Pixels for Image Classification on Black-Box Models Kang, Sin-Han; Jung, Hong-Gyu; Lee, Seong-Whan 2019 IEEE/CVF International Conference on Computer Vision Workshop (ICCVW) https://doi.org/10.1109/ICCVW.2019.00523	conference	October 2019
Class Feature Pyramids for Video Explanation Stergiou, Alexandros; Kapidis, Georgios; Kalliatakis, Grigorios 2019 IEEE/CVF International Conference on Computer Vision Workshop (ICCVW) https://doi.org/10.1109/ICCVW.2019.00524	conference	October 2019
Visual Analytics for Explainable Deep Learning Choo, Jaegul; Liu, Shixia IEEE Computer Graphics and Applications, Vol. 38, Issue 4 https://doi.org/10.1109/MCG.2018.042731661	journal	July 2018
DeepCompare: Visual and Interactive Comparison of Deep Learning Model Performance Murugesan, Sugeerth; Malik, Sana; Du, Fan IEEE Computer Graphics and Applications, Vol. 39, Issue 5 https://doi.org/10.1109/MCG.2019.2919033	journal	September 2019
D³ Data-Driven Documents Bostock, M.; Ogievetsky, V.; Heer, J. IEEE Transactions on Visualization and Computer Graphics, Vol. 17, Issue 12 https://doi.org/10.1109/TVCG.2011.185	journal	December 2011
Towards Better Analysis of Deep Convolutional Neural Networks Liu, Mengchen; Shi, Jiaxin; Li, Zhen IEEE Transactions on Visualization and Computer Graphics, Vol. 23, Issue 1 https://doi.org/10.1109/TVCG.2016.2598831	journal	January 2017
Visualizing the Hidden Activity of Artificial Neural Networks Rauber, Paulo E.; Fadel, Samuel G.; Falcao, Alexandre X. IEEE Transactions on Visualization and Computer Graphics, Vol. 23, Issue 1 https://doi.org/10.1109/TVCG.2016.2598838	journal	January 2017
DeepEyes: Progressive Visual Analytics for Designing Deep Neural Networks Pezzotti, Nicola; Hollt, Thomas; Van Gemert, Jan IEEE Transactions on Visualization and Computer Graphics, Vol. 24, Issue 1 https://doi.org/10.1109/TVCG.2017.2744358	journal	January 2018
ActiVis: Visual Exploration of Industry-Scale Deep Neural Network Models Kahng, Minsuk; Andrews, Pierre Y.; Kalro, Aditya IEEE Transactions on Visualization and Computer Graphics, Vol. 24, Issue 1 https://doi.org/10.1109/TVCG.2017.2744718	journal	January 2018
Visualizing Dataflow Graphs of Deep Learning Models in TensorFlow Wongsuphasawat, Kanit; Smilkov, Daniel; Wexler, James IEEE Transactions on Visualization and Computer Graphics, Vol. 24, Issue 1 https://doi.org/10.1109/TVCG.2017.2744878	journal	January 2018
Visual Analytics in Deep Learning: An Interrogative Survey for the Next Frontiers Hohman, Fred; Kahng, Minsuk; Pienta, Robert IEEE Transactions on Visualization and Computer Graphics, Vol. 25, Issue 8 https://doi.org/10.1109/TVCG.2018.2843369	journal	August 2019
Visual Genealogy of Deep Neural Networks Wang, Qianwen; Yuan, Jun; Chen, Shuxin IEEE Transactions on Visualization and Computer Graphics, Vol. 26, Issue 11 https://doi.org/10.1109/TVCG.2019.2921323	journal	November 2020
Ablate, Variate, and Contemplate: Visual Analytics for Discovering Neural Architectures Cashman, Dylan; Perer, Adam; Chang, Remco IEEE Transactions on Visualization and Computer Graphics, Vol. 26, Issue 1 https://doi.org/10.1109/TVCG.2019.2934261	journal	January 2020
Summit: Scaling Deep Learning Interpretability by Visualizing Activation and Attribution Summarizations Hohman, Fred; Park, Haekyu; Robinson, Caleb IEEE Transactions on Visualization and Computer Graphics, Vol. 26, Issue 1 https://doi.org/10.1109/TVCG.2019.2934659	journal	January 2020
VATLD : A V isual A nalytics System to Assess, Understand and Improve T raffic L ight D etection Gou, Liang; Zou, Lincan; Li, Nanxiang IEEE Transactions on Visualization and Computer Graphics, Vol. 27, Issue 2 https://doi.org/10.1109/TVCG.2020.3030350	journal	February 2021
CNN Explainer: Learning Convolutional Neural Networks with Interactive Visualization Wang, Zijie J.; Turko, Robert; Shaikh, Omar IEEE Transactions on Visualization and Computer Graphics, Vol. 27, Issue 2 https://doi.org/10.1109/TVCG.2020.3030418	journal	February 2021
On Pixel-Wise Explanations for Non-Linear Classifier Decisions by Layer-Wise Relevance Propagation Bach, Sebastian; Binder, Alexander; Montavon, Grégoire PLOS ONE, Vol. 10, Issue 7 https://doi.org/10.1371/journal.pone.0130140	journal	July 2015
Relative Attributing Propagation: Interpreting the Comparative Contributions of Individual Units in Deep Neural Networks Nam, Woo-Jeoung; Gur, Shir; Choi, Jaesik Proceedings of the AAAI Conference on Artificial Intelligence, Vol. 34, Issue 03 https://doi.org/10.1609/aaai.v34i03.5632	journal	April 2020
A causal framework for explaining the predictions of black-box sequence-to-sequence models Alvarez-Melis, David; Jaakkola, Tommi Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing https://doi.org/10.18653/v1/D17-1042	conference	January 2017

Similar Records

A Visual Designer of Layer-wise Relevance Propagation Models

Conference · 2021 · OSTI ID:1813333

Huang, Xinyi; Xu, Wei; Jamonnak, Suphanut; +2 more

Evaluating the Trustworthiness of Explainable Artificial Intelligence (XAI) Methods Applied to Regression Predictions of Arctic Sea Ice Motion

Journal Article · 2025 · Artificial Intelligence for the Earth Systems · OSTI ID:2526286

Hoffman, Lauren; Mazloff, Matthew R.; Gille, Sarah T.; +2 more

Classifying T cell activity in autofluorescence intensity images with convolutional neural networks

Journal Article · 2019 · Journal of Biophotonics · OSTI ID:1623563

Wang, Zijie J.; Walsh, Alex J.; Skala, Melissa C.; +1 more

Title: A Visual Designer of Layer‐wise Relevance Propagation Models

Citation Formats

References (32)

Similar Records

Related Subjects