Dynamic Role-Based Access Control Policy for Smart Grid Applications: An Offline Deep Reinforcement Learning Approach

Fragkos, Georgios; Johnson, Jay; Tsiropoulou, Eirini Eleni

doi:10.1109/thms.2022.3163185

Dynamic Role-Based Access Control Policy for Smart Grid Applications: An Offline Deep Reinforcement Learning Approach

Journal Article · Fri Apr 22 00:00:00 EDT 2022 · IEEE Transactions on Human-Machine Systems

DOI:https://doi.org/10.1109/thms.2022.3163185· OSTI ID:1872014

^[1]; ^[2]; ^[1]

Univ. of New Mexico, Albuquerque, NM (United States)
Sandia National Lab. (SNL-NM), Albuquerque, NM (United States)

Role-based access control (RBAC) is adopted in the information and communication technology domain for authentication purposes. However, due to a very large number of entities within organizational access control (AC) systems, static RBAC management can be inefficient, costly, and can lead to cybersecurity threats. In this paper, a novel hybrid RBAC model is proposed, based on the principles of offline deep reinforcement learning (RL) and Bayesian belief networks. The considered framework utilizes a fully offline RL agent, which models the behavioral history of users as a Bayesian belief-based trust indicator. Thus, the initial static RBAC policy is improved in a dynamic manner through off-policy learning while guaranteeing compliance of the internal users with the security rules of the system. By deploying our implementation within the smart grid domain and specifically within a Distributed Energy Resources (DER) ecosystem, we provide an end-to-end proof of concept of our model. Finally, detailed analysis and evaluation regarding the offline training phase of the RL agent are provided, while the online deployment of the hybrid RL-based RBAC model into the DER ecosystem highlights its key operation features and salient benefits over traditional RBAC models.

View Accepted Manuscript (DOE)

Research Organization:: Sandia National Laboratories (SNL-NM), Albuquerque, NM (United States)

Sponsoring Organization:: USDOE Office of Energy Efficiency and Renewable Energy (EERE), Renewable Power Office. Solar Energy Technologies Office

Grant/Contract Number:: NA0003525

OSTI ID:: 1872014

Report Number(s):: SAND2022-2911J; 706292

Journal Information:: IEEE Transactions on Human-Machine Systems, Journal Name: IEEE Transactions on Human-Machine Systems Journal Issue: 4 Vol. 52; ISSN 2168-2291

Publisher:: IEEECopyright Statement

Country of Publication:: United States

Language:: English

References (24)

Q-learning Watkins, Christopher J. C. H.; Dayan, Peter Machine Learning, Vol. 8, Issue 3-4 https://doi.org/10.1007/BF00992698	journal	May 1992
Automatic fine-grained access control in SCADA by machine learning Zhou, Lu; Su, Chunhua; Li, Zhen Future Generation Computer Systems, Vol. 93 https://doi.org/10.1016/j.future.2018.04.043	journal	April 2019
A convolutional neural-based learning classifier system for detecting database intrusion via insider attack Bu, Seok-Jun; Cho, Sung-Bae Information Sciences, Vol. 512 https://doi.org/10.1016/j.ins.2019.09.055	journal	February 2020
Human-level control through deep reinforcement learning Mnih, Volodymyr; Kavukcuoglu, Koray; Silver, David Nature, Vol. 518, Issue 7540 https://doi.org/10.1038/nature14236	journal	February 2015
Power system effects and mitigation recommendations for DER cyberattacks Johnson, Jay; Quiroz, Jimmy; Concepcion, Ricky IET Cyber-Physical Systems: Theory & Applications, Vol. 4, Issue 3 https://doi.org/10.1049/iet-cps.2018.5014	journal	September 2019
Intelligent Performance-Aware Adaptation of Control Policies for Optimizing Banking Teller Process Using Machine Learning Tay, Bilal; Mourad, Azzam IEEE Access, Vol. 8 https://doi.org/10.1109/ACCESS.2020.3015616	journal	January 2020
Hybrid Intrusion Detection System Design for Distributed Energy Resource Systems Chavez, A.; Lai, C.; Jacobs, N. 2019 IEEE CyberPELS (CyberPELS) https://doi.org/10.1109/CyberPELS.2019.8925064	conference	April 2019
Database Intrusion Detection System Using Octraplet and Machine Learning Jayaprakash, Souparnika; Kandasamy, Kamalanathan 2018 Second International Conference on Inventive Communication and Computational Technologies (ICICCT) https://doi.org/10.1109/ICICCT.2018.8473029	conference	April 2018
Path planning for intelligent robots based on deep Q-learning with experience replay and heuristic knowledge Jiang, Lan; Huang, Hongyun; Ding, Zuohua IEEE/CAA Journal of Automatica Sinica, Vol. 7, Issue 4 https://doi.org/10.1109/JAS.2019.1911732	journal	July 2020
Human-Swarm-Teaming Transparency and Trust Architecture Hepworth, Adam J.; Baxter, Daniel P.; Hussein, Aya IEEE/CAA Journal of Automatica Sinica, Vol. 8, Issue 7 https://doi.org/10.1109/JAS.2020.1003545	journal	July 2021
Physical Safety and Cyber Security Analysis of Multi-Agent Systems: A Survey of Recent Advances Zhang, Dan; Feng, Gang; Shi, Yang IEEE/CAA Journal of Automatica Sinica, Vol. 8, Issue 2 https://doi.org/10.1109/JAS.2021.1003820	journal	February 2021
Hierarchical Reinforcement Learning With Automatic Sub-Goal Identification Liu, Chenghao; Zhu, Fei; Liu, Quan IEEE/CAA Journal of Automatica Sinica, Vol. 8, Issue 10 https://doi.org/10.1109/JAS.2021.1004141	journal	October 2021
Deep Reinforcement Learning: A Brief Survey Arulkumaran, Kai; Deisenroth, Marc Peter; Brundage, Miles IEEE Signal Processing Magazine, Vol. 34, Issue 6 https://doi.org/10.1109/MSP.2017.2743240	journal	November 2017
Security in Mobile Edge Caching with Reinforcement Learning Xiao, Liang; Wan, Xiaoyue; Dai, Canhuang IEEE Wireless Communications, Vol. 25, Issue 3 https://doi.org/10.1109/MWC.2018.1700291	journal	June 2018
International Development of a Distributed Energy Resource Test Platform for Electrical and Interoperability Certification Johnson, Jay; Apablaza-Arancibia, Estefan; Ninad, Nayeem 2018 IEEE 7th World Conference on Photovoltaic Energy Conversion (WCPEC) (A Joint Conference of 45th IEEE PVSC, 28th PVSEC & 34th EU PVSEC) https://doi.org/10.1109/PVSC.2018.8547588	conference	June 2018
Incorporating Behavior in Attribute Based Access Control Model Using Machine Learning Afshar, Majid; Samet, Saeed; Usefi, Hamid 2021 IEEE International Systems Conference (SysCon) https://doi.org/10.1109/SysCon48628.2021.9447115	conference	April 2021
Scheduling Semiconductor Testing Facility by Using Cuckoo Search Algorithm With Reinforcement Learning and Surrogate Modeling Cao, ZhengCai; Lin, ChengRan; Zhou, MengChu IEEE Transactions on Automation Science and Engineering, Vol. 16, Issue 2 https://doi.org/10.1109/TASE.2018.2862380	journal	April 2019
A Double-Blind Anonymous Evaluation-Based Trust Model in Cloud Computing Environments Zhang, Peiyun; Zhou, Mengchu; Kong, Yang IEEE Transactions on Systems, Man, and Cybernetics: Systems, Vol. 51, Issue 3 https://doi.org/10.1109/TSMC.2019.2906310	journal	March 2021
Security analysis in role-based access control Li, Ninghui; Tripunitara, Mahesh V. ACM Transactions on Information and System Security, Vol. 9, Issue 4 https://doi.org/10.1145/1187441.1187442	journal	November 2006
A Survey of Role Mining Mitra, Barsha; Sural, Shamik; Vaidya, Jaideep ACM Computing Surveys, Vol. 48, Issue 4 https://doi.org/10.1145/2871148	journal	May 2016
A Survey on Deep Learning: Algorithms, Techniques, and Applications Pouyanfar, Samira; Sadiq, Saad; Yan, Yilin ACM Computing Surveys, Vol. 51, Issue 5 https://doi.org/10.1145/3234150	journal	September 2019
Emergence-Based Access Control El Kalam, Anas Abou; Outchakoucht, Aissam; Es-Samaali, Hamza Proceedings of the 1st International Conference on Digital Tools & Uses Congress - DTUC '18 https://doi.org/10.1145/3240117.3240136	conference	January 2018
Dynamic Access Control Policy based on Blockchain and Machine Learning for the Internet of Things Outchakoucht, Aissam; Es-Samaali, Hamza; Philippe, Jean International Journal of Advanced Computer Science and Applications, Vol. 8, Issue 7 https://doi.org/10.14569/IJACSA.2017.080757	journal	January 2017
Counterfactual Multi-Agent Policy Gradients Foerster, Jakob; Farquhar, Gregory; Afouras, Triantafyllos Proceedings of the AAAI Conference on Artificial Intelligence, Vol. 32, Issue 1 https://doi.org/10.1609/aaai.v32i1.11794	journal	April 2018

Similar Records

Centralized and Decentralized Distributed Energy Resource Access Control Implementation Considerations.

Journal Article · Thu Sep 01 00:00:00 EDT 2022 · Energies · OSTI ID:1888513

Recommendations for Distributed Energy Resource Access Control

Technical Report · Thu Dec 31 23:00:00 EST 2020 · OSTI ID:1765273

Role-Based Access Control for Distributed Energy Resources v.1.0

Software · Mon Jul 19 20:00:00 EDT 2021 · OSTI ID:code-61160

Related Subjects

32 ENERGY CONSERVATION, CONSUMPTION, AND UTILIZATION
Bayesian belief
deep reinforcement learning (RL)
distributed energy resources (DER)
machine learning (ML)
offline policy improvement
role based access control (RBAC)
smart gird
trust

Dynamic Role-Based Access Control Policy for Smart Grid Applications: An Offline Deep Reinforcement Learning Approach

Citation Formats

References (24)

Similar Records

Related Subjects