Automated Adversary Emulation for Cyber-Physical Systems via Reinforcement Learning

Conference · Mon Nov 16 04:00:00 EST 2020

OSTI ID:1760319

Bhattacharya, Arnab ^[1]; Ramachandran, Thiagarajan ^[1]; Banik, Sandeep ^[2]; Dowling, Chase P. ^[1]; Bopardikar, Shaunak ^[2]

---

1. BATTELLE (PACIFIC NW LAB)
2. Michigan State University

Adversary emulation is an offensive exercise that provides a comprehensive assessment of a system’s resilience against cyber attacks. However, adversary emulation is typically a manual process, making it costly and hard to deploy in cyber-physical systems (CPS) with complex dynamics, vulnerabilities, and operational uncertainties. In this paper, we develop an automated, domain-aware approach to adversary emulation for CPS. We formulate a Markov Decision Process (MDP) model to determine an optimal attack sequence over a hybrid attack graph with cyber (discrete) and physical (continuous) components and related physical dynamics. We apply model-based and model-free reinforcement learning (RL) methods to solve the discrete-continuous MDP in a tractable fashion. As a baseline, we also develop a greedy attack algorithm and compare it with the RL procedures. We summarize our findings through a numerical study on sensor deception attacks in buildings to compare the performance and solution quality of the proposed algorithms.

• Research Organization: Pacific Northwest National Laboratory (PNNL), Richland, WA (United States)
• Sponsoring Organization: USDOE
• DOE Contract Number: AC05-76RL01830
• OSTI ID: 1760319
• Report Number(s): PNNL-SA-156040
• Country of Publication: United States
• Language: English