Energy-efficient underwater acoustic communication based on Dyna-Q with an adaptive action space
Document Type
Article
Publication Date
12-2023
Department
Department of Electrical and Computer Engineering
Abstract
The underwater acoustic (UWA) channel exhibits large spatial–temporal dynamics. This work focuses on adapting the transmission power to the channel condition to achieve energy efficiency in UWA communications. To tackle the unknown channel variation, a reinforcement learning (RL) algorithm called Dyna-Q is introduced. Consider the continuous variation of UWA channels. Instead of using a fixed action space in the Dyna-Q, an adaptive action space with an updated Q-value iteration is proposed in this work. The switching of the transmission power level occurs on a block-by-block basis during transmission, aiming to maximize the system's long-term energy efficiency performance. Using the widely-used communication technique, Orthogonal Frequency-Division Multiplexing (OFDM), as an example, both simulations and experimental data processing results demonstrate that the proposed method outperforms two comparative methods, including the original Dyna-Q with fixed action spaces.
Publication Title
Physical Communication
Recommended Citation
Fan, C.,
Wang, Z.,
&
Yang, K.
(2023).
Energy-efficient underwater acoustic communication based on Dyna-Q with an adaptive action space.
Physical Communication,
61.
http://doi.org/10.1016/j.phycom.2023.102218
Retrieved from: https://digitalcommons.mtu.edu/michigantech-p2/169