skip to main content
OSTI.GOV title logo U.S. Department of Energy
Office of Scientific and Technical Information

Title: Applicability Analysis of Validation Evidence for Biomedical Computational Models

Abstract

Computational modeling has the potential to revolutionize medicine the way it transformed engineering. However, despite decades of work, there has only been limited progress to successfully translate modeling research to patient care. One major difficulty which often occurs with biomedical computational models is an inability to perform validation in a setting that closely resembles how the model will be used. For example, for a biomedical model that makes in vivo clinically relevant predictions, direct validation of predictions may be impossible for ethical, technological, or financial reasons. Unavoidable limitations inherent to the validation process lead to challenges in evaluating the credibility of biomedical model predictions. Therefore, when evaluating biomedical models, it is critical to rigorously assess applicability, that is, the relevance of the computational model, and its validation evidence to the proposed context of use (COU). However, there are no well-established methods for assessing applicability. In this paper, we present a novel framework for performing applicability analysis and demonstrate its use with a medical device computational model. The framework provides a systematic, step-by-step method for breaking down the broad question of applicability into a series of focused questions, which may be addressed using supporting evidence and subject matter expertise. The frameworkmore » can be used for model justification, model assessment, and validation planning. While motivated by biomedical models, it is relevant to a broad range of disciplines and underlying physics. Finally, the proposed applicability framework could help overcome some of the barriers inherent to validation of, and aid clinical implementation of, biomedical models.« less

Authors:
 [1];  [2];  [3];  [2]
  1. Food and Drug Administration (FDA), Silver Spring, MD (United States). Office of Science and Engineering Laboratories (OSEL). Center for Devices and Radiological Health (CDRH)
  2. Food and Drug Administration (FDA), Silver Spring, MD (United States). Office of Science and Engineering Laboratories (OSEL). Center for Devices and Radiological Health (CDRH)
  3. Sandia National Lab. (SNL-NM), Albuquerque, NM (United States)
Publication Date:
Research Org.:
Food and Drug Administration (FDA), Silver Spring, MD (United States); Sandia National Lab. (SNL-NM), Albuquerque, NM (United States)
Sponsoring Org.:
USDOE; Food and Drug Administration (FDA) (United States)
OSTI Identifier:
1399491
Report Number(s):
SAND2017-2092J
Journal ID: ISSN 2377-2158; 651131
Grant/Contract Number:  
AC04-94AL85000
Resource Type:
Journal Article: Accepted Manuscript
Journal Name:
Journal of Verification, Validation and Uncertainty Quantification
Additional Journal Information:
Journal Volume: 2; Journal Issue: 2; Journal ID: ISSN 2377-2158
Publisher:
ASME
Country of Publication:
United States
Language:
English
Subject:
97 MATHEMATICS AND COMPUTING; computer simulation; simulation; stents; biomedicine

Citation Formats

Pathmanathan, Pras, Gray, Richard A., Romero, Vicente J., and Morrison, Tina M. Applicability Analysis of Validation Evidence for Biomedical Computational Models. United States: N. p., 2017. Web. doi:10.1115/1.4037671.
Pathmanathan, Pras, Gray, Richard A., Romero, Vicente J., & Morrison, Tina M. Applicability Analysis of Validation Evidence for Biomedical Computational Models. United States. https://doi.org/10.1115/1.4037671
Pathmanathan, Pras, Gray, Richard A., Romero, Vicente J., and Morrison, Tina M. 2017. "Applicability Analysis of Validation Evidence for Biomedical Computational Models". United States. https://doi.org/10.1115/1.4037671. https://www.osti.gov/servlets/purl/1399491.
@article{osti_1399491,
title = {Applicability Analysis of Validation Evidence for Biomedical Computational Models},
author = {Pathmanathan, Pras and Gray, Richard A. and Romero, Vicente J. and Morrison, Tina M.},
abstractNote = {Computational modeling has the potential to revolutionize medicine the way it transformed engineering. However, despite decades of work, there has only been limited progress to successfully translate modeling research to patient care. One major difficulty which often occurs with biomedical computational models is an inability to perform validation in a setting that closely resembles how the model will be used. For example, for a biomedical model that makes in vivo clinically relevant predictions, direct validation of predictions may be impossible for ethical, technological, or financial reasons. Unavoidable limitations inherent to the validation process lead to challenges in evaluating the credibility of biomedical model predictions. Therefore, when evaluating biomedical models, it is critical to rigorously assess applicability, that is, the relevance of the computational model, and its validation evidence to the proposed context of use (COU). However, there are no well-established methods for assessing applicability. In this paper, we present a novel framework for performing applicability analysis and demonstrate its use with a medical device computational model. The framework provides a systematic, step-by-step method for breaking down the broad question of applicability into a series of focused questions, which may be addressed using supporting evidence and subject matter expertise. The framework can be used for model justification, model assessment, and validation planning. While motivated by biomedical models, it is relevant to a broad range of disciplines and underlying physics. Finally, the proposed applicability framework could help overcome some of the barriers inherent to validation of, and aid clinical implementation of, biomedical models.},
doi = {10.1115/1.4037671},
url = {https://www.osti.gov/biblio/1399491}, journal = {Journal of Verification, Validation and Uncertainty Quantification},
issn = {2377-2158},
number = 2,
volume = 2,
place = {United States},
year = {Thu Sep 07 00:00:00 EDT 2017},
month = {Thu Sep 07 00:00:00 EDT 2017}
}

Works referenced in this record:

In silico clinical trials: how computer simulation will transform the biomedical industry
journal, May 2016


Incorporation of stochastic engineering models as prior information in Bayesian medical device trials
journal, March 2017


Computational Medicine: Translating Models to Clinical Care
journal, October 2012


Computational systems biology
journal, November 2002


Predictive Medicine: Computational Techniques in Therapeutic Decision-Making
journal, January 1999


Ensuring reliability of safety-critical clinical applications of computational cardiac models
journal, January 2013


Defining predictive maturity for validated numerical simulations
journal, April 2010


Risk-based verification, validation, and accreditation process
conference, April 2010


Verification, validation, and predictive capability in computational engineering and physics
journal, September 2004


Sandia Verification and Validation Challenge Problem: A PCMM-Based Approach to Assessing Prediction Credibility
journal, February 2016


Calibration, validation, and sensitivity analysis: What's what
journal, October 2006


Bayesian calibration of computer models
journal, August 2001


Design considerations for studies of the biomechanical environment of the femoropopliteal arteries
journal, September 2013


Effects of Cyclic Loading on the Uniaxial Behavior of Nitinol
journal, December 2010


in silico Clinical Trials: How Computer Simulation will Transform the Biomedical Industry
text, January 2016


Works referencing / citing this record:

Off‐axis loads cause failure of the distal radius at lower magnitudes than axial loads: A side‐to‐side experimental study
journal, January 2020


Patient-Specific Cardiovascular Computational Modeling: Diversity of Personalization and Challenges
journal, March 2018


Study on the Accuracy of Structural and FSI Heart Valves Simulations
journal, August 2018


Validation and Trustworthiness of Multiscale Models of Cardiac Electrophysiology
journal, February 2018


Functionalized Anatomical Models for Computational Life Sciences
journal, November 2018


Comprehensive Uncertainty Quantification and Sensitivity Analysis for Cardiac Action Potential Models
journal, June 2019


Functionalized Anatomical Models for Computational Life Sciences
text, January 2018


Impact of Modeling Assumptions on Stability Predictions in Reverse Total Shoulder Arthroplasty
journal, August 2018


Functionalized Anatomical Models for Computational Life Sciences
journal, November 2018