Skip to main content
OSTI.GOVU.S. Department of Energy Office of Scientific and Technical Information
U.S. Department of Energy
Office of Scientific and Technical Information
 

Automated Adversary-in-the-Loop Cyber-Physical Defense Planning

Journal Article · · ACM Transactions on Cyber-Physical Systems
DOI:https://doi.org/10.1145/3596222· OSTI ID:2228580
 [1];  [2];  [2];  [1]
  1. Michigan State Univ., East Lansing, MI (United States)
  2. Pacific Northwest National Laboratory (PNNL), Richland, WA (United States)
+ Show Author Affiliations

Security of cyber-physical systems (CPS) continues to pose new challenges due to the tight integration and operational complexity of the cyber and physical components. To address these challenges, this article presents a domain-aware, optimization-based approach to determine an effective defense strategy for CPS in an automated fashion—by emulating a strategic adversary in the loop that exploits system vulnerabilities, interconnection of the CPS, and the dynamics of the physical components. Our approach builds on an adversarial decision-making model based on a Markov Decision Process (MDP) that determines the optimal cyber (discrete) and physical (continuous) attack actions over a CPS attack graph. The defense planning problem is modeled as a non-zero-sum game between the adversary and defender. We use a model-free reinforcement learning method to solve the adversary’s problem as a function of the defense strategy. We then employ Bayesian optimization (BO) to find an approximate best-response for the defender to harden the network against the resulting adversary policy. This process is iterated multiple times to improve the strategy for both players. We demonstrate the effectiveness of our approach on a ransomware-inspired graph with a smart building system as the physical process. Numerical studies show that our method converges to a Nash equilibrium for various defender-specific costs of network hardening.

Research Organization:
Pacific Northwest National Laboratory (PNNL), Richland, WA (United States)
Sponsoring Organization:
USDOE; National Science Foundation (NSF)
Grant/Contract Number:
AC05-76RL01830
OSTI ID:
2228580
Report Number(s):
PNNL-SA--163768
Journal Information:
ACM Transactions on Cyber-Physical Systems, Journal Name: ACM Transactions on Cyber-Physical Systems Journal Issue: 3 Vol. 7; ISSN 2378-962X
Publisher:
Association for Computing Machinery (ACM)Copyright Statement
Country of Publication:
United States
Language:
English

References (56)

Taguchi methods: Some technical, cultural and pedagogical perspectives journal May 1993
Principles of robust design methodology journal June 2007
Adaptive Game Playing Using Multiplicative Weights journal October 1999
On stochastic games with additive reward and transition structure journal December 1985
TD(?) converges with probability 1 journal March 1994
On pure stationary almost Markov Nash equilibria in nonzero-sum ARAT stochastic games journal January 2015
A survey of deep learning-based network anomaly detection journal September 2017
Robust optimization with simulated annealing journal December 2009
A Bayesian optimization approach to find Nash equilibria journal July 2018
Active authentication with reinforcement learning based on ambient radio signals journal October 2015
A systems and control perspective of CPS security journal January 2019
A review of attack graph and attack tree visual syntax in cyber security journal February 2020
Impact of the controller model complexity on model predictive control performance for buildings journal October 2017
Reinforcement learning algorithms with function approximation: Recent advances and applications journal March 2014
State of the art of cyber-physical systems security: An automatic control perspective journal March 2019
Towards a framework of enforcing resilient operation of cyber‐physical systems with unknown dynamics journal April 2021
News Briefing: Cyber security - Ukraine grid hack is wake-up call for network operators journal February 2016
A Bayesian Optimization Approach to Compute Nash Equilibrium of Potential Games Using Bandit Feedback journal December 2019
A Vulnerability Assessment Method in Industrial Internet of Things Based on Attack Graph and Maximum Flow journal January 2018
Secure Contingency Prediction and Response for Cyber-Physical Systems conference August 2020
DIFT Games: Dynamic Information Flow Tracking Games for Advanced Persistent Threats conference December 2018
Learning the Associations of MITRE ATT & CK Adversarial Techniques conference June 2020
A Survey of Data Mining and Machine Learning Methods for Cyber Security Intrusion Detection journal July 2016
Resilient Machine Learning for Networked Cyber Physical Systems: A Survey for Machine Learning Security to Securing Machine Learning for CPS journal January 2021
An Advanced Persistent Threat (APT)-Style Cyberattack Testbed for Distributed Energy Resources (DER) conference July 2021
Anti-Jamming Communication Game for UAV-Aided VANETs conference December 2017
FlipNet: Modeling Covert and Persistent Attacks on Networked Resources conference June 2017
Generalization of Deep Learning for Cyber-Physical System Security: A Survey conference October 2018
A Cyber Kill Chain Model for Distributed Energy Resources (DER) Aggregation Systems conference February 2021
Automated Adversary Emulation for Cyber-Physical Systems via Reinforcement Learning conference November 2020
Using hybrid attack graphs to model cyber-physical attacks in the Smart Grid conference August 2012
Stuxnet: Dissecting a Cyberwarfare Weapon journal May 2011
A model-based approach to security analysis for cyber-physical systems conference April 2018
Coding Schemes for Securing Cyber-Physical Systems Against Stealthy Data Injection Attacks journal March 2017
Identifying vulnerabilities and hardening attack graphs for networked systems conference May 2016
Q-Learning-Based Vulnerability Analysis of Smart Grid Against Sequential Topology Attacks journal January 2017
Information-Theoretic Regret Bounds for Gaussian Process Optimization in the Bandit Setting journal May 2012
A Multistage Game in Smart Grid Security: A Reinforcement Learning Solution journal September 2019
Deep Reinforcement Learning for Cyber Security journal August 2023
Petri Net Modeling of Cyber-Physical Attacks on Smart Grid journal December 2011
Power System Reliability Assessment Incorporating Cyber Attacks Against Wind Farm Energy Management Systems journal September 2017
Evaluation of Reinforcement Learning-Based False Data Injection Attack to Automatic Voltage Control journal March 2019
PHY-Layer Spoofing Detection With Reinforcement Learning in Wireless Networks journal December 2016
DQN-Based Power Control for IoT Transmission against Jamming conference June 2018
Dynamic Noncooperative Game Theory, 2nd Edition book January 1998
Secure program execution via dynamic information flow tracking journal October 2004
Fast gradient-descent methods for temporal-difference learning with linear function approximation conference June 2009
Toward hybrid attack dependency graphs conference October 2011
Optimal Defense Policies for Partially Observable Spreading Processes on Bayesian Attack Graphs conference October 2015
Probabilistic Attack Sequence Generation and Execution Based on MITRE ATT&CK for ICS Datasets conference August 2021
Attack net penetration testing conference January 2000
Scalable, graph-based network vulnerability analysis conference November 2002
Nonconvex Robust Optimization for Problems with Constraints journal February 2010
Robust Optimization for Unconstrained Simulation-Based Problems journal February 2010
Designing Fast Absorbing Markov Chains journal June 2014
Automatic Hybrid Attack Graph (AHAG) Generation for Complex Engineering Systems journal November 2019

Figures / Tables (13)

Fig. 1(p. 5)figure Fig. 1
Fig. 2(p. 10)figure Fig. 2
Algorithm 1(p. 12)figure Algorithm 1
Fig. 3(p. 14)figure Fig. 3
Fig. 4(p. 14)figure Fig. 4
Fig. 5(p. 15)figure Fig. 5
Table 1(p. 15)table Table 1
Table 2(p. 16)table Table 2
Fig. 6(p. 17)figure Fig. 6
Fig. 7(p. 17)figure Fig. 7
Fig. 8(p. 18)figure Fig. 8
Fig. 10(p. 19)figure Fig. 10
Fig. 9(p. 19)figure Fig. 9

Similar Records

Automated Adversary Emulation for Cyber-Physical Systems via Reinforcement Learning
Conference · Sun Nov 15 23:00:00 EST 2020 · OSTI ID:1760319
Inferring adversarial behaviour in cyber‐physical power systems using a Bayesian attack graph approach
Journal Article · Fri Feb 10 23:00:00 EST 2023 · IET Cyber-Physical Systems: Theory & Applications · OSTI ID:1924131

Related Subjects

97 MATHEMATICS AND COMPUTING
computer systems organization
computing methodologies
embedded systems security
reinforcement learning
security and privacy
sensors and actuators