Optimizing crop management with reinforcement learning, imitation learning, and crop simulations

Tao, Ran

Permalink

https://hdl.handle.net/2142/117756 Copy

Description

Title

Optimizing crop management with reinforcement learning, imitation learning, and crop simulations

Author(s)

Tao, Ran

Issue Date

2022-11-17

Director of Research (if dissertation) or Advisor (if thesis)

Hovakimyan, Naira

Department of Study

Mechanical Sci & Engineering

Discipline

Mechanical Engineering

Degree Granting Institution

University of Illinois at Urbana-Champaign

Degree Name

M.S.

Degree Level

Thesis

Keyword(s)

• Reinforcement Learning
• Imitation Learning
• Intelligent Crop Management
• Sustainable Agriculture

Abstract

Crop management, including nitrogen (N) fertilization and irrigation management, has a significant impact on crop yield, economic profit, and the environment. Although management guidelines exist, finding the optimal management practices is challenging given a specific planting environment and a crop. This thesis presents an intelligent crop management system that optimizes N fertilization and irrigation simultaneously via reinforcement learning (RL), imitation learning (IL), and crop simulations using the Decision Support System for Agrotechnology Transfer (DSSAT). The thesis formulates the crop management problem as an RL problem, and uses RL algorithms to train management policies that require all state variables from the simulator as observations (denoted as full observation). We then invoke IL to train management policies that only need a limited amount of state variables that can be easily obtained or measured in the real world (denoted as partial observation) by mimicking the actions of the RL-trained policies under full observation. This thesis includes three case studies in total. The first one focuses on optimizing N fertilization under full observation only. The second one optimizes N fertilization and irrigation simultaneously under full observation, and the last one considers N fertilization and irrigation under partial observation. For the case study focusing on N fertilization, experiments are conducted in simulations on the maize crop in Iowa, where farmers usually do not irrigate, and N management policies are trained with two deep RL algorithms, deep Q-network (DQN) and soft actor-critic (SAC). For the case studies on optimizing N fertilization and irrigation simultaneously, we conduct experiments in simulations on maize crop in Florida, where both irrigation and fertilization are critical for the crop growth. Deep Q-network is applied in the RL-based training for finding management policies under full observation, and the RL-trained policies are then used as the expert to train management policies under partial observation with IL. For each case study, we compare the trained policies with baseline methods, which follow a maize production guideline in the corresponding location. The trained policies under both full and partial observations achieve better outcomes than the baseline methods, resulting in a higher profit or yield with less management input or a smaller environmental impact. Moreover, the partial-observation management policies are directly deployable in the real world as they use readily available information, which paves the way for validating the performance of the framework with field tests.

Graduation Semester

2022-12

Type of Resource

Thesis

Copyright and License Information

Copyright 2022 Ran Tao

Owning Collections