INTELLIGENT GEOSYSTEM ENABLED BY REINFORCEMENT LEARNING (RL)

Author

Aynaz Biniyaz, Michigan Technological UniversityFollow

Date of Award

2022

Document Type

Campus Access Dissertation

Degree Name

Doctor of Philosophy in Civil Engineering (PhD)

Administrative Home Department

Department of Civil, Environmental, and Geospatial Engineering

Advisor 1

Zhen Liu

Committee Member 1

Stanley Vitton

Committee Member 2

Veronica Webster

Committee Member 3

Ye Sun

Abstract

The occurrence of landslides has been increasing in recent years due to intense and prolonged rainfall events. Lowering the groundwater in natural and man-made slopes can help mitigate the hazards. Subsurface drainage systems equipped with pumps have traditionally been regarded as a temporary remedy for lowering the groundwater in geosystems, whereas long-term usage of pumping-based techniques is uncommon due to the associated high operational costs in labor and energy. This dissertation investigates the intelligent control of groundwater in slopes enabled by Deep Reinforcement Learning (DRL), a subfield of machine learning for automated decision-making. The purpose is to develop an intelligent geosystem that can minimize operating costs and enhance the system’s safety without introducing human errors and interventions. First, to prove the concept, a seepage analysis model was implemented using a partial differential equation solver, FEniCS, to simulate the geosystem (i.e., a slope equipped with a pump and subjected to rainfall events). Next, a Deep Q-Network (i.e., a DRL learning agent) was integrated with the seepage model and trained to learn the optimal control policy for regulating the pump’s flow rate. The objective is to keep the groundwater close to the target level during rainfall events and consequently help prevent slope failure. A comparison of the results with traditional Proportional-Integral-Derivative controlled and uncontrolled water tables showed that the geosystem integrated with DRL can (1) dynamically adapt its response to diverse weather events by adjusting the pump’s flow rate and (2) improve the adopted control policy by gaining more experience over time. After proving the concept of DRL for the intelligent geosystem, the knowledge gained in numerical implementation was transferred to a physical lab-scale geosystem that served as a real-world environment for the DRL agent. After pre-training and fine-tuning the DRL agent in the lab, the agent became capable of keeping the water level close to the target level. The findings of this dissertation point out a feasible avenue for developing intelligent geosystems.

Recommended Citation

Biniyaz, Aynaz, "INTELLIGENT GEOSYSTEM ENABLED BY REINFORCEMENT LEARNING (RL)", Campus Access Dissertation, Michigan Technological University, 2022.

https://doi.org/10.37099/mtu.dc.etdr/1506