Test and Evaluation of Systems with Embedded Machine Learning Components
- Sandia National Lab. (SNL-NM), Albuquerque, NM (United States)
As Machine Learning (ML) continues to advance, it is being integrated into more systems. Often, the ML component represents a significant portion of the system that reduces the burden on the end user or significantly improves task performance. However, the ML component represents an unknown complex phenomenon that is learned from collected data without the need to be explicitly programmed. Despite the improvement in task performance, the models are often black boxes. Evaluating the credibility and the vulnerabilities of ML models poses a gap in current test and evaluation practice. For high consequence applications, the lack of testing and evaluation procedures represents a significant source of uncertainty and risk. To help reduce that risk, here we present considerations to evaluate systems embedded with an ML component within a red-teaming inspired methodology. We focus on (1) cyber vulnerabilities to an ML model, (2) evaluating performance gaps, and (3) adversarial ML vulnerabilities.
- Research Organization:
- Sandia National Laboratories (SNL-NM), Albuquerque, NM (United States)
- Sponsoring Organization:
- USDOD
- Grant/Contract Number:
- NA0003525
- OSTI ID:
- 2311444
- Report Number(s):
- SAND--2023-11084J
- Journal Information:
- ITEA Journal of Test and Evaluation, Journal Name: ITEA Journal of Test and Evaluation Journal Issue: 3 Vol. 44; ISSN 1054-0229
- Publisher:
- International Test and Evaluation AssociationCopyright Statement
- Country of Publication:
- United States
- Language:
- English
Similar Records
Hybrid Attack Graph Generation with Graph Convolutional Deep-Q Learning