skip to main content
OSTI.GOV title logo U.S. Department of Energy
Office of Scientific and Technical Information

Title: ShapeShop: Towards Understanding Deep Learning Representations via Interactive Experimentation

Abstract

Deep learning is the driving force behind many recent technologies; however, deep neural networks are often viewed as “black-boxes” due to their internal complexity that is hard to understand. Little research focuses on helping people explore and understand the relationship between a user’s data and the learned representations in deep learning models. We present our ongoing work, ShapeShop, an interactive system for visualizing and understanding what semantics a neural network model has learned. Built using standard web technologies, ShapeShop allows users to experiment with and compare deep learning models to help explore the robustness of image classifiers.

Authors:
; ;
Publication Date:
Research Org.:
Pacific Northwest National Lab. (PNNL), Richland, WA (United States)
Sponsoring Org.:
USDOE
OSTI Identifier:
1361010
Report Number(s):
PNNL-SA-126087
DOE Contract Number:
AC05-76RL01830
Resource Type:
Conference
Resource Relation:
Conference: Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems (CHI EA 2017), May 6-11, 2017, Denver, Colorado, 1694-1699
Country of Publication:
United States
Language:
English

Citation Formats

Hohman, Frederick M., Hodas, Nathan O., and Chau, Duen Horng. ShapeShop: Towards Understanding Deep Learning Representations via Interactive Experimentation. United States: N. p., 2017. Web. doi:10.1145/3027063.3053103.
Hohman, Frederick M., Hodas, Nathan O., & Chau, Duen Horng. ShapeShop: Towards Understanding Deep Learning Representations via Interactive Experimentation. United States. doi:10.1145/3027063.3053103.
Hohman, Frederick M., Hodas, Nathan O., and Chau, Duen Horng. 2017. "ShapeShop: Towards Understanding Deep Learning Representations via Interactive Experimentation". United States. doi:10.1145/3027063.3053103.
@article{osti_1361010,
title = {ShapeShop: Towards Understanding Deep Learning Representations via Interactive Experimentation},
author = {Hohman, Frederick M. and Hodas, Nathan O. and Chau, Duen Horng},
abstractNote = {Deep learning is the driving force behind many recent technologies; however, deep neural networks are often viewed as “black-boxes” due to their internal complexity that is hard to understand. Little research focuses on helping people explore and understand the relationship between a user’s data and the learned representations in deep learning models. We present our ongoing work, ShapeShop, an interactive system for visualizing and understanding what semantics a neural network model has learned. Built using standard web technologies, ShapeShop allows users to experiment with and compare deep learning models to help explore the robustness of image classifiers.},
doi = {10.1145/3027063.3053103},
journal = {},
number = ,
volume = ,
place = {United States},
year = 2017,
month = 5
}

Conference:
Other availability
Please see Document Availability for additional information on obtaining the full-text document. Library patrons may search WorldCat to identify libraries that hold this conference proceeding.

Save / Share:
  • Deep learning systems have been growing in prominence as a way to automatically characterize objects, trends, and anomalies. Given the importance of deep learning systems, researchers have been investigating techniques to optimize such systems. An area of particular interest has been using large supercomputing systems to quickly generate effective deep learning networks: a phase often referred to as “training” of the deep learning neural network. As we scale existing deep learning frameworks—such as Caffe—on these large supercomputing systems, we notice that the parallelism can help improve the computation tremendously, leaving data I/O as the major bottleneck limiting the overall systemmore » scalability. In this paper, we first present a detailed analysis of the performance bottlenecks of Caffe on large supercomputing systems. Our analysis shows that the I/O subsystem of Caffe—LMDB—relies on memory-mapped I/O to access its database, which can be highly inefficient on large-scale systems because of its interaction with the process scheduling system and the network-based parallel filesystem. Based on this analysis, we then present LMDBIO, our optimized I/O plugin for Caffe that takes into account the data access pattern of Caffe in order to vastly improve I/O performance. Our experimental results show that LMDBIO can improve the overall execution time of Caffe by nearly 20-fold in some cases.« less
  • Online social networks, which have been expanding at a blistering speed in the recent years, have emerged as a popular communication infrastructure for Internet users. Meanwhile, malware that specifically targets these online social networks are also on the rise. In this work, we aim to investigate the characteristics of malware propagation in online social networks. Our study is based on a dataset collected from a real-world location-based online social network. We analyze the social structure and user activity patterns of this network. We further use extensive trace-driven simulation to study the impact of initial infection, user click probability, social structure,more » and activity patterns on malware propagation in online social networks. The results from this work has greatly deepened our understanding of the nature of online social network malware and also shed light on how to defend against them effectively.« less
  • Expert-provided operator descriptions are expensive, incomplete, and incorrect. Given the assumptions of noise-free information and an completely-observable state, OBSERVER can autonomously learn and refines new operators through observation and practice. WISER, our learning system, relaxes these assumptions and learns operator preconditions through experimentation utilizing imperfect expert-provided knowledge. Our decision-theoretic formula calculates a probably best state S` for experimentation based on the imperfect knowledge. When a robotic action is executed successfully for the first time in a state S, the corresponding operator`s initial preconditions are learned as parameterized S. We empirically show the number of training examples required to learn themore » initial preconditions as a function of the amount of injected error. The learned preconditions contain all the necessary positive literals, but no negative literals.« less
  • Sharkzor is a web application for machine-learning assisted image sort and summary. Deep learning algorithms are leveraged to infer, augment, and automate the user’s mental model. Initially, images uploaded by the user are spread out on a canvas. The user then interacts with the images to impute their mental model into the applications algorithmic underpinnings. Methods of interaction within Sharkzor’s user interface and user experience support three primary user tasks: triage, organize and automate. The user triages the large pile of overlapping images by moving images of interest into proximity. The user then organizes said images into meaningful groups. Aftermore » interacting with the images and groups, deep learning helps to automate the user’s interactions. The loop of interaction, automation, and response by the user allows the system to quickly make sense of large amounts of data.« less