skip to main content
DOE PAGES title logo U.S. Department of Energy
Office of Scientific and Technical Information

Title: A Heuristic Approach to Value-Driven Evaluation of Visualizations

Abstract

Recently, an approach for determining the value of a visualization was proposed, one moving beyond simple measurements of task accuracy and speed. The value equation contains components for the time savings a visualization provides, the insights and insightful questions it spurs, the overall essence of the data it conveys, and the confidence about the data and its domain it inspires. This articulation of value is purely descriptive, however, providing no actionable method of assessing a visualization's value. In this work, we create a heuristic-based evaluation methodology to accompany the value equation for assessing interactive visualizations. We refer to the methodology colloquially as ICE-T, based on an anagram of the four value components. Our approach breaks the four components down into guidelines, each of which is made up of a small set of low-level heuristics. Evaluators who have knowledge of visualization design principles then assess the visualization with respect to the heuristics. We conducted an initial trial of the methodology on three interactive visualizations of the same data set, each evaluated by 15 visualization experts. As a result, we found that the methodology showed promise, obtaining consistent ratings across the three visualizations and mirroring judgments of the utility of the visualizationsmore » by instructors of the course in which they were developed.« less

Authors:
 [1];  [1];  [2];  [2];  [2];  [1];  [1]
  1. Georgia Inst. of Technology, Atlanta, GA (United States)
  2. Sandia National Lab. (SNL-NM), Albuquerque, NM (United States)
Publication Date:
Research Org.:
Sandia National Lab. (SNL-NM), Albuquerque, NM (United States)
Sponsoring Org.:
USDOE National Nuclear Security Administration (NNSA)
OSTI Identifier:
1485468
Report Number(s):
SAND-2018-11898J
Journal ID: ISSN 1077-2626; 668715
Grant/Contract Number:  
AC04-94AL85000
Resource Type:
Accepted Manuscript
Journal Name:
IEEE Transactions on Visualization and Computer Graphics
Additional Journal Information:
Journal Volume: 25; Journal Issue: 1; Journal ID: ISSN 1077-2626
Publisher:
IEEE
Country of Publication:
United States
Language:
English
Subject:
97 MATHEMATICS AND COMPUTING; visualization evaluation; heuristics; value of visualization

Citation Formats

Wall, Emily, Agnihotri, Meeshu, Matzen, Laura, Divis, Kristin, Haass, Michael, Endert, Alex, and Stasko, John. A Heuristic Approach to Value-Driven Evaluation of Visualizations. United States: N. p., 2018. Web. doi:10.1109/TVCG.2018.2865146.
Wall, Emily, Agnihotri, Meeshu, Matzen, Laura, Divis, Kristin, Haass, Michael, Endert, Alex, & Stasko, John. A Heuristic Approach to Value-Driven Evaluation of Visualizations. United States. doi:10.1109/TVCG.2018.2865146.
Wall, Emily, Agnihotri, Meeshu, Matzen, Laura, Divis, Kristin, Haass, Michael, Endert, Alex, and Stasko, John. Mon . "A Heuristic Approach to Value-Driven Evaluation of Visualizations". United States. doi:10.1109/TVCG.2018.2865146. https://www.osti.gov/servlets/purl/1485468.
@article{osti_1485468,
title = {A Heuristic Approach to Value-Driven Evaluation of Visualizations},
author = {Wall, Emily and Agnihotri, Meeshu and Matzen, Laura and Divis, Kristin and Haass, Michael and Endert, Alex and Stasko, John},
abstractNote = {Recently, an approach for determining the value of a visualization was proposed, one moving beyond simple measurements of task accuracy and speed. The value equation contains components for the time savings a visualization provides, the insights and insightful questions it spurs, the overall essence of the data it conveys, and the confidence about the data and its domain it inspires. This articulation of value is purely descriptive, however, providing no actionable method of assessing a visualization's value. In this work, we create a heuristic-based evaluation methodology to accompany the value equation for assessing interactive visualizations. We refer to the methodology colloquially as ICE-T, based on an anagram of the four value components. Our approach breaks the four components down into guidelines, each of which is made up of a small set of low-level heuristics. Evaluators who have knowledge of visualization design principles then assess the visualization with respect to the heuristics. We conducted an initial trial of the methodology on three interactive visualizations of the same data set, each evaluated by 15 visualization experts. As a result, we found that the methodology showed promise, obtaining consistent ratings across the three visualizations and mirroring judgments of the utility of the visualizations by instructors of the course in which they were developed.},
doi = {10.1109/TVCG.2018.2865146},
journal = {IEEE Transactions on Visualization and Computer Graphics},
number = 1,
volume = 25,
place = {United States},
year = {2018},
month = {9}
}

Journal Article:
Free Publicly Available Full Text
Publisher's Version of Record

Save / Share: