Date of Award
2023
Document Type
Campus Access Dissertation
Degree Name
Doctor of Philosophy in Electrical Engineering (PhD)
Administrative Home Department
Department of Electrical and Computer Engineering
Advisor 1
Kaichen Yang
Committee Member 1
Zhaohui Wang
Committee Member 2
Hongyu An
Committee Member 3
Xiaoyong Yuan
Abstract
UWA communication networks represent a highly potential means of facilitating wireless information transfer over medium to long distances in aquatic scenarios. However, the challenging and ever-changing underwater environment presents significant obstacles to developing effective UWA communications. This dissertation leverages the advances in RL and signal processing to design intelligent and effective UWA communication algorithms. Three research topics are studied: 1) Energy-efficient Underwater Acoustic Communication Based on Dyna-Q with an Adaptive Action Space; 2) Adaptive Switching for Communication Profiles in Underwater Acoustic Modems Based on RL; 3) Efficient Adaptive Modulation for UWA Communication Based on the MPSK Constellation Mapping transition.
First, an RL-based algorithm is created for dynamic transmission in long-term adaptive UWA point-to-point communication systems. The dynamics of the UWA channel are learned to strike a balance between energy consumption and the QoS. The adaptive transmission problem is formulated as an MDP which is solved by the proposed Dyna-Q algorithm with adaptive action space. Dyna-Q is an RL algorithm that combines elements of both model-based RL and model-free RL. Utilizing the proposed adaptive action space lowers the prior knowledge requirement of discretization and accelerates convergence. Both the simulation results and experimental data processing reveal the output performance of the proposed method compared with the comparative schemes with fixed action space.
Secondly, an adaptation of the communication strategy to channel dynamics is studied. Specifically, a set of communication strategies is considered, including FH-BFSK, SC communication, and multicarrier communication. Based on the channel condition, an RL algorithm, the DDPG method along with a Gumbel-softmax scheme is employed for intelligent and adaptive switching among those communication strategies. The adaptive switching is performed on a transmission block-by-block basis, with the goal of maximizing long-term system performance. The reward function is defined based on the EE and the SE of the communication strategies. Simulation results and experimental data processing results reveal that the proposed method outperforms a random selection method and a direct feedback method in time-varying channels.
Thirdly, the dissertation presents an efficient adaptive modulation algorithm with a novel BER estimation method based on the MPSK constellation mapping transition since the convergence speed of the RL-based algorithms limits the practical implementation of moving targets and the channel with spatial and temporal variability. Both simulations and experimental data show the effectiveness of the BER estimation and the outperformance of the proposed AM method based on Dyna-Q.
Recommended Citation
Fan, Cheng, "ADAPTIVE MODULATION FOR UNDERWATER ACOUSTIC COMMUNICATION BASED ON REINFORCEMENT LEARNING", Campus Access Dissertation, Michigan Technological University, 2023.