Evaluation methodology for comparing memory and communication of analytic processes in visual analytics

Conference · Wed Jan 01 00:00:00 EST 2014

OSTI ID:1185573

Ragan, Eric D ^[1]; Goodall, John R ^[1]

---

1. ORNL

Provenance tools can help capture and represent the history of analytic processes. In addition to supporting analytic performance, provenance tools can be used to support memory of the process and communication of the steps to others. Objective evaluation methods are needed to evaluate how well provenance tools support analyst s memory and communication of analytic processes. In this paper, we present several methods for the evaluation of process memory, and we discuss the advantages and limitations of each. We discuss methods for determining a baseline process for comparison, and we describe various methods that can be used to elicit process recall, step ordering, and time estimations. Additionally, we discuss methods for conducting quantitative and qualitative analyses of process memory. By organizing possible memory evaluation methods and providing a meta-analysis of the potential benefits and drawbacks of different approaches, this paper can inform study design and encourage objective evaluation of process memory and communication.

• Research Organization: Oak Ridge National Lab. (ORNL), Oak Ridge, TN (United States)
• Sponsoring Organization: USDOE Laboratory Directed Research and Development (LDRD) Program
• DOE Contract Number: AC05-00OR22725
• OSTI ID: 1185573
• Resource Relation: Conference: BELIV Workshop 2014, Paris, France, 20141110, 20141110
• Country of Publication: United States
• Language: English