A Novel Multi-Agent Deep Reinforcement Learning-enabled Distributed Power Allocation Scheme for mmWave Cellular Networks
Conference
·
OSTI ID:1984598
- University of Utah
- Idaho National Laboratory
We consider the power allocation problem over shared spectrum for millimeter-Wave (mmWave) cellular downlink. Existing approaches usually find sub-optimal solutions by solving a non-convex optimization which leads to scalability issues due to centralized control. Therefore, distributed and adaptive approaches are desirable. Recently, model-free Deep Reinforcement Learning (DRL) has achieved success in such wireless resource management tasks. By modeling the radio environment as a Markov Decision Process (MDP) with the base stations (BSs) being the agents, power allocation can be automated at the agent level with comparable throughput performance to conventional centralized schemes. The multi-agent setting presents new challenges as the radio environment is impacted by the joint actions of the agents and is no longer stationary from any individual agent’s perspective. Existing literature bypasses this non-stationarity violation by ignoring it which may cause performance degradation. To tackle this issue, we propose a distributed continuous power allocation scheme based on a modified version of multi-agent Deep Deterministic Policy Gradient (MADDPG) that is tailored for the distributed multiple-agent setting. The proposed scheme employs a centralized-training distributed-execution framework where Q-functions are trained over subsets of BSs while each BS determines its transmit power based only on its own local observation. It admits constant per-BS communication and computation complexity and is thus scalable to large networks. Numerical evaluation shows that the proposed scheme adapts well to a wide range of interference conditions and can achieve comparable or better performance than several state-of-the-art non-learning approaches.
- Research Organization:
- Idaho National Laboratory (INL), Idaho Falls, ID (United States)
- Sponsoring Organization:
- 58
- DOE Contract Number:
- AC07-05ID14517
- OSTI ID:
- 1984598
- Report Number(s):
- INL/CON-23-71769-Rev000
- Country of Publication:
- United States
- Language:
- English
Similar Records
Distributed Power Allocation for 6-GHz Unlicensed Spectrum Sharing via Multi-agent Deep Reinforcement Learning
A Q-Learning-Based Approach for Distributed Beam Scheduling in mmWave Networks
Federated Deep Reinforcement Learning for Decentralized VVO of BTM DERs
Conference
·
Wed Apr 05 00:00:00 EDT 2023
·
OSTI ID:1975104
A Q-Learning-Based Approach for Distributed Beam Scheduling in mmWave Networks
Conference
·
Wed Dec 14 23:00:00 EST 2022
·
OSTI ID:1885963
Federated Deep Reinforcement Learning for Decentralized VVO of BTM DERs
Conference
·
Tue Oct 01 00:00:00 EDT 2024
·
OSTI ID:2477510