Deep Reinforcement Learning for Controlling the Groundwater in Slopes

Document Type

Conference Proceeding

Publication Date

3-17-2022

Department

Department of Civil, Environmental, and Geospatial Engineering

Abstract

Extreme weather events are a main cause of landslides in recent years. Real-time control of groundwater tables in slopes can help protect earth slopes in areas prone to flooding. Though subsurface drainage wells equipped with a pumping system is an efficient way to lower the groundwater, it has been mostly employed in short-term projects due to the high-operational costs in labor and energy. To reduce these operational costs, this paper investigates the idea of an autonomous pumping system enabled by Deep Reinforcement Learning (DRL), which is a subfield of machine learning for automated decision-making. Such a system can dynamically adapt its response to rainfall events by controlling the pumping flow rate, and more importantly, can improve the pumping policy over time. To prove the idea of the autonomous pumping system, a seepage analysis model was implemented using a partial differential equation solver, FEniCS, to simulate a lab-scale geo-system, that is, a slope equipped with a pump and subjected to rainfall events, which served as the virtual environment for the reinforcement learning. A Deep Q-learning Network (DQN), that is, a DRL agent, was implemented to learn the optimal control policy based on the trial and error process of the system to achieve the desired objective. This agent was trained to learn how to control the pump’s flow rate to keep the groundwater close to the target level during different rainfall events. A reward function was defined to evaluate the state of the groundwater, which could affect the next action taken by the agent. The goal of the DQN is to find a policy that maximizes the received reward. The training was carried out from scratch without human interventions. Aiming at binary control, the agent learns whether to turn on/off the pump based on the rewards constructed with the distance of the water at the time of decision and the target level. The results showed that a DRL can learn how to control a pump to maintain the water level in a binary control mode, which may point out a promising direction for establishing intelligent geo-systems. Such autonomous control of groundwater can help mitigate landslide hazards as a long-term geotechnical solution.

Publication Title

Geo-Congress 2022

Link to Full Text

Share

COinS