Harnessing expert knowledge: Defining a Bayesian network decision model with limited data-Model structure for the vibration qualification problem

Rizzo, Davinia B.; Blackburn, Mark R.

doi:10.1002/sys.21431

Harnessing expert knowledge: Defining a Bayesian network decision model with limited data-Model structure for the vibration qualification problem

Journal Article · Fri Mar 30 00:00:00 EDT 2018 · Systems Engineering

DOI:https://doi.org/10.1002/sys.21431· OSTI ID:1432479

^[1]; Blackburn, Mark R. ^[2]

Sandia National Lab. (SNL-NM), Albuquerque, NM (United States)
Stevens Inst. of Technology, Hoboken, NJ (United States)

As systems become more complex, systems engineers rely on experts to inform decisions. There are few experts and limited data in many complex new technologies. This challenges systems engineers as they strive to plan activities such as qualification in an environment where technical constraints are coupled with the traditional cost, risk, and schedule constraints. Bayesian network (BN) models provide a framework to aid systems engineers in planning qualification efforts with complex constraints by harnessing expert knowledge and incorporating technical factors. By quantifying causal factors, a BN model can provide data about the risk of implementing a decision supplemented with information on driving factors. This allows a systems engineer to make informed decisions and examine “what-if” scenarios. This paper discusses a novel process developed to define a BN model structure based primarily on expert knowledge supplemented with extremely limited data (25 data sets or less). The model was developed to aid qualification decisions—specifically to predict the suitability of six degrees of freedom (6DOF) vibration testing for qualification. The process defined the model structure with expert knowledge in an unbiased manner. Finally, validation during the process execution and of the model provided evidence the process may be an effective tool in harnessing expert knowledge for a BN model.

View Accepted Manuscript (DOE)

Research Organization:: Sandia National Laboratories (SNL-NM), Albuquerque, NM (United States)

Sponsoring Organization:: USDOE; USDOE National Nuclear Security Administration (NNSA)

Grant/Contract Number:: AC04-94AL85000; NA0003525

OSTI ID:: 1432479

Alternate ID(s):: OSTI ID: 1478216

Report Number(s):: SAND--2017-2685J; SAND--2018-3223J; 661799

Journal Information:: Systems Engineering, Journal Name: Systems Engineering Journal Issue: 4 Vol. 21; ISSN 1098-1241

Publisher:: WileyCopyright Statement

Country of Publication:: United States

Language:: English

References (20)

From complex questionnaire and interviewing data to intelligent Bayesian network models for medical decision support Constantinou, Anthony Costa; Fenton, Norman; Marsh, William Artificial Intelligence in Medicine, Vol. 67 https://doi.org/10.1016/j.artmed.2016.01.002	journal	February 2016
Metrics for evaluating performance and uncertainty of Bayesian network models Marcot, Bruce G. Ecological Modelling, Vol. 230 https://doi.org/10.1016/j.ecolmodel.2012.01.013	journal	April 2012
Good practice in Bayesian network modelling Chen, Serena H.; Pollino, Carmel A. Environmental Modelling & Software, Vol. 37 https://doi.org/10.1016/j.envsoft.2012.03.012	journal	November 2012
A proposed validation framework for expert elicited Bayesian Networks Pitchforth, Jegar; Mengersen, Kerrie Expert Systems with Applications, Vol. 40, Issue 1 https://doi.org/10.1016/j.eswa.2012.07.026	journal	January 2013
Decision support analysis for safety control in complex project environments based on Bayesian Networks Zhang, Limao; Wu, Xianguo; Ding, Lieyun Expert Systems with Applications, Vol. 40, Issue 11 https://doi.org/10.1016/j.eswa.2012.11.022	journal	September 2013
Risk modelling of a hydrogen refuelling station using Bayesian network Haugom, G. P.; Friis-Hansen, P. International Journal of Hydrogen Energy, Vol. 36, Issue 3 https://doi.org/10.1016/j.ijhydene.2010.04.131	journal	February 2011
Application of a Bayesian model for the quantification of the European methodology for qualification of non-destructive testing Gandossi, Luca; Simola, Kaisa; Shepherd, Barrie International Journal of Pressure Vessels and Piping, Vol. 87, Issue 2-3 https://doi.org/10.1016/j.ijpvp.2009.12.002	journal	February 2010
Analyzing the structure of expert knowledge Bradley, John H.; Paul, Ravi; Seeman, Elaine Information & Management, Vol. 43, Issue 1 https://doi.org/10.1016/j.im.2004.11.009	journal	January 2006
Use of Bayesian Networks for Qualification Planning: A Predictive Analysis Framework for a Technically Complex Systems Engineering Problem Rizzo, Davinia B.; Blackburn, Mark R. Procedia Computer Science, Vol. 61 https://doi.org/10.1016/j.procs.2015.09.173	journal	January 2015
Use of Bayesian Networks for Qualification Planning: Early Results of Factor Analysis Rizzo, Davinia B.; Blackburn, Mark R. Procedia Computer Science, Vol. 95 https://doi.org/10.1016/j.procs.2016.09.354	journal	January 2016
Mapping model validation metrics to subject matter expert scores for model adequacy assessment Teferra, Kirubel; Shields, Michael D.; Hapij, Adam Reliability Engineering & System Safety, Vol. 132 https://doi.org/10.1016/j.ress.2014.07.010	journal	December 2014
Application of cognitive, skill-based, and affective theories of learning outcomes to new methods of training evaluation. Kraiger, Kurt; Ford, J. Kevin; Salas, Eduardo Journal of Applied Psychology, Vol. 78, Issue 2 https://doi.org/10.1037//0021-9010.78.2.311	journal	January 1993
Application of cognitive, skill-based, and affective theories of learning outcomes to new methods of training evaluation. Kraiger, Kurt; Ford, J. Kevin; Salas, Eduardo Journal of Applied Psychology, Vol. 78, Issue 2 https://doi.org/10.1037/0021-9010.78.2.311	journal	January 1993
Assessing structural knowledge. Goldsmith, Timothy E.; Johnson, Peder J.; Acton, William H. Journal of Educational Psychology, Vol. 83, Issue 1 https://doi.org/10.1037/0022-0663.83.1.88	journal	March 1991
Interdisciplinary Aspects of Cost-Risk Tradeoffs Marshall, Peter OCEANS 2006 https://doi.org/10.1109/OCEANS.2006.306980	conference	September 2006
A Method for Integrating Expert Knowledge When Learning Bayesian Networks From Data Cano, A.; Masegosa, A. R.; Moral, S. IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics), Vol. 41, Issue 5 https://doi.org/10.1109/TSMCB.2011.2148197	journal	October 2011
Interdisciplinary Aspects of Cost-Risk Tradeoffs Marshall, Peter OCEANS 2006 https://doi.org/10.1109/oceans.2006.306980	conference	September 2006
Assessment of Catastrophic Risk Using Bayesian Network Constructed from Domain Knowledge and Spatial Data: Assessment of Catastrophic Risk Using Bayesian Network Li, Lianfa; Wang, Jinfeng; Leung, Hareton Risk Analysis, Vol. 30, Issue 7 https://doi.org/10.1111/j.1539-6924.2010.01429.x	journal	April 2010
The case for objective Bayesian analysis Berger, James Bayesian Analysis, Vol. 1, Issue 3 https://doi.org/10.1214/06-BA115	journal	September 2006
Drawbacks of complex models in frequentist and Bayesian approaches to natural-resource management Adkison, Milo D. Ecological Applications, Vol. 19, Issue 1 https://doi.org/10.1890/07-1641.1	journal	January 2009

Cited By (1)

Elemental patterns of verification strategies Salado, Alejandro; Kannan, Hanumanthrao Systems Engineering, Vol. 22, Issue 5 https://doi.org/10.1002/sys.21481	journal	March 2019

Similar Records

Test and Evaluation for Enhanced Security: A Quantitative Method to Incorporate Expert Knowledge into Test Planning Decisions.

Journal Article · Tue Feb 28 23:00:00 EST 2017 · The ITEA (International Test and Evaluation Association) Journal · OSTI ID:1429619

Expert system for LAN media selection

Conference · Sat Oct 01 00:00:00 EDT 1988 · OSTI ID:6566414

The development of a generic failure analysis expert system based on case-based reasoning

Conference · Tue Oct 01 00:00:00 EDT 1996 · OSTI ID:367748

Related Subjects

97 MATHEMATICS AND COMPUTING
Bayesian network
decision model
qualification
structural knowledge assessment

Harnessing expert knowledge: Defining a Bayesian network decision model with limited data-Model structure for the vibration qualification problem

Citation Formats

References (20)

Cited By (1)

Similar Records

Related Subjects