Verification and Validation of Systems in which AI is a Key Element

Book · Sun Aug 01 04:00:00 EDT 2021

OSTI ID:1818698

Pullum, Laura ^[1]

---

1. ORNL

Many systems are being considered in which artificial intelligence (AI) will be a key element. Failure of an AI element can lead to system failure, hence the need for AI verification and validation (V&V). This article addresses V&V of such a system, focusing on the issues created by characteristics of AI that make V&V challenging. The element(s) containing AI capabilities is treated as a subsystem and V&V is conducted on that subsystem and its interfaces with other elements of the system under study, just as V&V would be conducted on other subsystems. That is, the high-level definitions of verification and of validation do not change for systems containing one or more AI elements. However, AI V&V challenges require approaches and solutions beyond those for conventional or traditional (those without AI elements) systems. This article provides an overview of how machine learning components/subsystems “fit” in the systems engineering framework (Section 1), identifies characteristics of AI subsystems that create challenges in their V&V (Section 2), describes those challenges (Section 3) and provides some potential solutions (Section 4).

• Research Organization: Oak Ridge National Laboratory (ORNL), Oak Ridge, TN (United States)
• Sponsoring Organization: USDOE
• DOE Contract Number: AC05-00OR22725
• OSTI ID: 1818698
• Country of Publication: United States
• Language: English