Creating realistic, scenario-based synthetic data for test and evaluation of information analytics software
The Threat Stream Generator (TSG) project at Pacific Northwest National Laboratory has been developing synthetic datasets to test and evaluate visual analytics tools for the past four years. Our activities have ranged from supporting the evaluation of major U.S. Government analytical frameworks to creating four datasets for the IEEE Visual Analytics Science and Technology (VAST) contest over the past two years. We have developed a reasonable method and supporting toolset for creating believable synthetic data sets for different uses. A key differentiator for our datasets is that they contain data concerning one or more invented threats, based on a scenario. Embedding a known threat into the data provides ground truth for analytic tools to work against in evaluating their performance, as well as new opportunities for evaluation researchers to explore techniques given ground truth exists. We describe the process of creating the scenarios and threats and the process of transforming them into data elements, and then we describe how this data is embedded in other data to form a TSG dataset.
- Research Organization:
- Pacific Northwest National Lab. (PNNL), Richland, WA (United States)
- Sponsoring Organization:
- USDOE
- DOE Contract Number:
- AC05-76RL01830
- OSTI ID:
- 962854
- Report Number(s):
- PNNL-SA-58345; 400904120; TRN: US200916%%316
- Resource Relation:
- Conference: Proceedings of the 2008 conference on BEyond time and errors: novel evaLuation methods for Information Visualization (BELIV 2008), April 5, 2008, Florence, Italy., Article No. 8
- Country of Publication:
- United States
- Language:
- English
Similar Records
Smart-DS: Synthetic Models for Advanced, Realistic Testing: Distribution Systems and Scenarios
Smart-DS: Synthetic Models for Advanced, Realistic Testing: Distribution Systems and Scenarios