skip to main content
OSTI.GOV title logo U.S. Department of Energy
Office of Scientific and Technical Information

Title: Empirical Analysis of the Subjective Impressions and Objective Measures of Domain Scientists’ Visual Analytic Judgments

Abstract

Scientists often use specific data analysis and presentation methods familiar within their domain. But does high familiarity drive better analytical judgment? This question is especially relevant when familiar methods themselves can have shortcomings: many visualizations used conventionally for scientific data analysis and presentation do not follow established best practices. This necessitates new methods that might be unfamiliar yet prove to be more effective. But there is little empirical understanding of the relationships between scientists’ subjective impressions about familiar and unfamiliar visualizations and objective measures of their visual analytic judgments. To address this gap and to study these factors, we focus on visualizations used for comparison of climate model performance. We report on a comprehensive survey-based user study with 47 climate scientists and present an analysis of : i) relationships among scientists’ familiarity, their perceived lev- els of comfort, confidence, accuracy, and objective measures of accuracy, and ii) relationships among domain experience, visualization familiarity, and post-study preference.

Authors:
; ; ;
Publication Date:
Research Org.:
Pacific Northwest National Lab. (PNNL), Richland, WA (United States)
Sponsoring Org.:
USDOE
OSTI Identifier:
1440653
Report Number(s):
PNNL-SA-123318
DOE Contract Number:  
AC05-76RL01830
Resource Type:
Conference
Resource Relation:
Conference: Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems (CHI 2017), May 6-11, 2017, Denver, Colorado, 1193-1204
Country of Publication:
United States
Language:
English

Citation Formats

Dasgupta, Aritra, Burrows, Susannah M., Han, Kyungsik, and Rasch, Philip J. Empirical Analysis of the Subjective Impressions and Objective Measures of Domain Scientists’ Visual Analytic Judgments. United States: N. p., 2017. Web. doi:10.1145/3025453.3025882.
Dasgupta, Aritra, Burrows, Susannah M., Han, Kyungsik, & Rasch, Philip J. Empirical Analysis of the Subjective Impressions and Objective Measures of Domain Scientists’ Visual Analytic Judgments. United States. doi:10.1145/3025453.3025882.
Dasgupta, Aritra, Burrows, Susannah M., Han, Kyungsik, and Rasch, Philip J. Mon . "Empirical Analysis of the Subjective Impressions and Objective Measures of Domain Scientists’ Visual Analytic Judgments". United States. doi:10.1145/3025453.3025882.
@article{osti_1440653,
title = {Empirical Analysis of the Subjective Impressions and Objective Measures of Domain Scientists’ Visual Analytic Judgments},
author = {Dasgupta, Aritra and Burrows, Susannah M. and Han, Kyungsik and Rasch, Philip J.},
abstractNote = {Scientists often use specific data analysis and presentation methods familiar within their domain. But does high familiarity drive better analytical judgment? This question is especially relevant when familiar methods themselves can have shortcomings: many visualizations used conventionally for scientific data analysis and presentation do not follow established best practices. This necessitates new methods that might be unfamiliar yet prove to be more effective. But there is little empirical understanding of the relationships between scientists’ subjective impressions about familiar and unfamiliar visualizations and objective measures of their visual analytic judgments. To address this gap and to study these factors, we focus on visualizations used for comparison of climate model performance. We report on a comprehensive survey-based user study with 47 climate scientists and present an analysis of : i) relationships among scientists’ familiarity, their perceived lev- els of comfort, confidence, accuracy, and objective measures of accuracy, and ii) relationships among domain experience, visualization familiarity, and post-study preference.},
doi = {10.1145/3025453.3025882},
journal = {},
number = ,
volume = ,
place = {United States},
year = {2017},
month = {5}
}

Conference:
Other availability
Please see Document Availability for additional information on obtaining the full-text document. Library patrons may search WorldCat to identify libraries that hold this conference proceeding.

Save / Share:

Works referenced in this record:

The Benefits of Synchronous Collaborative Information Visualization: Evidence from an Experimental Evaluation
journal, November 2009

  • Bresciani, S.; Eppler, M. J.
  • IEEE Transactions on Visualization and Computer Graphics, Vol. 15, Issue 6
  • DOI: 10.1109/TVCG.2009.188

E-commerce: the role of familiarity and trust
journal, December 2000


A Nested Model for Visualization Design and Validation
journal, November 2009

  • Munzner, Tamara
  • IEEE Transactions on Visualization and Computer Graphics, Vol. 15, Issue 6
  • DOI: 10.1109/TVCG.2009.111

The Role of Uncertainty, Awareness, and Trust in Visual Analytics
journal, January 2016

  • Sacha, Dominik; Senaratne, Hansi; Kwon, Bum Chul
  • IEEE Transactions on Visualization and Computer Graphics, Vol. 22, Issue 1
  • DOI: 10.1109/TVCG.2015.2467591

The Community Earth System Model: A Framework for Collaborative Research
journal, September 2013

  • Hurrell, James W.; Holland, M. M.; Gent, P. R.
  • Bulletin of the American Meteorological Society, Vol. 94, Issue 9
  • DOI: 10.1175/BAMS-D-12-00121.1

Pargnostics: Screen-Space Metrics for Parallel Coordinates
journal, November 2010

  • Dasgupta, A.; Kosara, R.
  • IEEE Transactions on Visualization and Computer Graphics, Vol. 16, Issue 6
  • DOI: 10.1109/TVCG.2010.184

Measurement of trust in complex and dynamic systems using a quantitative approach
journal, September 2004

  • Uggirala, Ananth; Gramopadhye, Anand K.; Melloy, Brain J.
  • International Journal of Industrial Ergonomics, Vol. 34, Issue 3
  • DOI: 10.1016/j.ergon.2004.03.005

Matches, Mismatches, and Methods: Multiple-View Workflows for Energy Portfolio Analysis
journal, January 2016

  • Brehmer, Matthew; Ng, Jocelyn; Tate, Kevin
  • IEEE Transactions on Visualization and Computer Graphics, Vol. 22, Issue 1
  • DOI: 10.1109/TVCG.2015.2466971

Familiarity Vs Trust: A Comparative Study of Domain Scientists' Trust in Visual Analytics and Conventional Analysis Methods
journal, January 2017

  • Dasgupta, Aritra; Lee, Joon-Yong; Wilson, Ryan
  • IEEE Transactions on Visualization and Computer Graphics, Vol. 23, Issue 1
  • DOI: 10.1109/TVCG.2016.2598544

Bridging Theory with Practice: An Exploratory Study of Visualization Use and Design for Climate Model Comparison
journal, September 2015

  • Dasgupta, Aritra; Poco, Jorge; Wei, Yaxing
  • IEEE Transactions on Visualization and Computer Graphics, Vol. 21, Issue 9
  • DOI: 10.1109/TVCG.2015.2413774

Visual comparison for information visualization
journal, September 2011

  • Gleicher, Michael; Albers, Danielle; Walker, Rick
  • Information Visualization, Vol. 10, Issue 4
  • DOI: 10.1177/1473871611416549

Empirical Studies in Information Visualization: Seven Scenarios
journal, September 2012

  • Lam, H.; Bertini, E.; Isenberg, P.
  • IEEE Transactions on Visualization and Computer Graphics, Vol. 18, Issue 9
  • DOI: 10.1109/TVCG.2011.279

A sharper Bonferroni procedure for multiple tests of significance
journal, January 1988


ColorBrewer.org: An Online Tool for Selecting Colour Schemes for Maps
journal, June 2003


Affect- and Cognition-Based Trust as Foundations for Interpersonal Cooperation in Organizations.
journal, February 1995

  • McAllister, D. J.
  • Academy of Management Journal, Vol. 38, Issue 1
  • DOI: 10.2307/256727

Summarizing multiple aspects of model performance in a single diagram
journal, April 2001

  • Taylor, Karl E.
  • Journal of Geophysical Research: Atmospheres, Vol. 106, Issue D7
  • DOI: 10.1029/2000JD900719