An Evaluation of Visual and Textual Network Analysis Tools
Abstract
User testing is an integral component of user-centered design, but has only rarely been applied to visualization for cyber security applications. This article presents the results of a comparative evaluation between a visualization-based application and a more traditional, table-based application for analyzing computer network packet captures. We conducted this evaluation as part of the user-centered design process. Participants performed both structured, well-defined tasks and exploratory, open-ended tasks with both tools. We measured accuracy and efficiency for the well-defined tasks, number of insights was measured for exploratory tasks and user perceptions were recorded for each tool. The results of this evaluation demonstrated that users performed significantly more accurately in the well-defined tasks, discovered a higher number of insights and demonstrated a clear preference for the visualization tool. The study design presented may be useful for future researchers performing user testing on visualization for cyber security applications.
- Authors:
-
- ORNL
- Publication Date:
- Research Org.:
- Oak Ridge National Lab. (ORNL), Oak Ridge, TN (United States)
- Sponsoring Org.:
- USDOE
- OSTI Identifier:
- 1019009
- DOE Contract Number:
- DE-AC05-00OR22725
- Resource Type:
- Journal Article
- Journal Name:
- Information Visualization
- Additional Journal Information:
- Journal Volume: 10; Journal Issue: 2
- Country of Publication:
- United States
- Language:
- English
- Subject:
- 99 GENERAL AND MISCELLANEOUS//MATHEMATICS, COMPUTING, AND INFORMATION SCIENCE; ACCURACY; COMPARATIVE EVALUATIONS; COMPUTER NETWORKS; DESIGN; EFFICIENCY; EVALUATION; NETWORK ANALYSIS; SECURITY; TESTING; user testing; comparative evaluation; security visualization; user-centered design
Citation Formats
Goodall, John R. An Evaluation of Visual and Textual Network Analysis Tools. United States: N. p., 2011.
Web. doi:10.1057/ivs.2011.2.
Goodall, John R. An Evaluation of Visual and Textual Network Analysis Tools. United States. https://doi.org/10.1057/ivs.2011.2
Goodall, John R. 2011.
"An Evaluation of Visual and Textual Network Analysis Tools". United States. https://doi.org/10.1057/ivs.2011.2.
@article{osti_1019009,
title = {An Evaluation of Visual and Textual Network Analysis Tools},
author = {Goodall, John R},
abstractNote = {User testing is an integral component of user-centered design, but has only rarely been applied to visualization for cyber security applications. This article presents the results of a comparative evaluation between a visualization-based application and a more traditional, table-based application for analyzing computer network packet captures. We conducted this evaluation as part of the user-centered design process. Participants performed both structured, well-defined tasks and exploratory, open-ended tasks with both tools. We measured accuracy and efficiency for the well-defined tasks, number of insights was measured for exploratory tasks and user perceptions were recorded for each tool. The results of this evaluation demonstrated that users performed significantly more accurately in the well-defined tasks, discovered a higher number of insights and demonstrated a clear preference for the visualization tool. The study design presented may be useful for future researchers performing user testing on visualization for cyber security applications.},
doi = {10.1057/ivs.2011.2},
url = {https://www.osti.gov/biblio/1019009},
journal = {Information Visualization},
number = 2,
volume = 10,
place = {United States},
year = {Sat Jan 01 00:00:00 EST 2011},
month = {Sat Jan 01 00:00:00 EST 2011}
}