skip to main content
OSTI.GOV title logo U.S. Department of Energy
Office of Scientific and Technical Information

Title: An Evaluation of Visual and Textual Network Analysis Tools

Abstract

User testing is an integral component of user-centered design, but has only rarely been applied to visualization for cyber security applications. This article presents the results of a comparative evaluation between a visualization-based application and a more traditional, table-based application for analyzing computer network packet captures. We conducted this evaluation as part of the user-centered design process. Participants performed both structured, well-defined tasks and exploratory, open-ended tasks with both tools. We measured accuracy and efficiency for the well-defined tasks, number of insights was measured for exploratory tasks and user perceptions were recorded for each tool. The results of this evaluation demonstrated that users performed significantly more accurately in the well-defined tasks, discovered a higher number of insights and demonstrated a clear preference for the visualization tool. The study design presented may be useful for future researchers performing user testing on visualization for cyber security applications.

Authors:
 [1]
  1. ORNL
Publication Date:
Research Org.:
Oak Ridge National Lab. (ORNL), Oak Ridge, TN (United States)
Sponsoring Org.:
USDOE
OSTI Identifier:
1019009
DOE Contract Number:  
DE-AC05-00OR22725
Resource Type:
Journal Article
Journal Name:
Information Visualization
Additional Journal Information:
Journal Volume: 10; Journal Issue: 2
Country of Publication:
United States
Language:
English
Subject:
99 GENERAL AND MISCELLANEOUS//MATHEMATICS, COMPUTING, AND INFORMATION SCIENCE; ACCURACY; COMPARATIVE EVALUATIONS; COMPUTER NETWORKS; DESIGN; EFFICIENCY; EVALUATION; NETWORK ANALYSIS; SECURITY; TESTING; user testing; comparative evaluation; security visualization; user-centered design

Citation Formats

Goodall, John R. An Evaluation of Visual and Textual Network Analysis Tools. United States: N. p., 2011. Web. doi:10.1057/ivs.2011.2.
Goodall, John R. An Evaluation of Visual and Textual Network Analysis Tools. United States. https://doi.org/10.1057/ivs.2011.2
Goodall, John R. 2011. "An Evaluation of Visual and Textual Network Analysis Tools". United States. https://doi.org/10.1057/ivs.2011.2.
@article{osti_1019009,
title = {An Evaluation of Visual and Textual Network Analysis Tools},
author = {Goodall, John R},
abstractNote = {User testing is an integral component of user-centered design, but has only rarely been applied to visualization for cyber security applications. This article presents the results of a comparative evaluation between a visualization-based application and a more traditional, table-based application for analyzing computer network packet captures. We conducted this evaluation as part of the user-centered design process. Participants performed both structured, well-defined tasks and exploratory, open-ended tasks with both tools. We measured accuracy and efficiency for the well-defined tasks, number of insights was measured for exploratory tasks and user perceptions were recorded for each tool. The results of this evaluation demonstrated that users performed significantly more accurately in the well-defined tasks, discovered a higher number of insights and demonstrated a clear preference for the visualization tool. The study design presented may be useful for future researchers performing user testing on visualization for cyber security applications.},
doi = {10.1057/ivs.2011.2},
url = {https://www.osti.gov/biblio/1019009}, journal = {Information Visualization},
number = 2,
volume = 10,
place = {United States},
year = {Sat Jan 01 00:00:00 EST 2011},
month = {Sat Jan 01 00:00:00 EST 2011}
}