Publications / 2020 Proceedings of the 37th ISARC, Kitakyushu, Japan

Simulation-based Reinforcement Learning Approach towards Construction Machine Automation

Keita Matsumoto, Atsushi Yamaguchi, Takahiro Oka, Masahiro Yasumoto, Satoru Hara, Michitaka Iida and Marek Teichmann
Pages 457-464 (2020 Proceedings of the 37th ISARC, Kitakyushu, Japan, ISBN 978-952-94-3634-7, ISSN 2413-5844)

Research towards automation of heavy construction machines for efficient and safe construction processes that are robust to various environments and disturbances has been conducted for many years. In this paper, we show two contributions towards this objective. Firstly, we explore the use of reinforcement learning to automate construction machines. Secondly, we evaluate the effectiveness of three methods to reduce learning time of reinforcement learning: designing reward function, pre-training with BC, and Changing frame-skip rate. Reinforcement learning approach is expected to gain robustness against disturbances through learning. We run experiments on two different realistic tasks. The first task is to reduce sway of a load suspended from a mobile boom crane, behaving as a single pendulum. The second task is to load an excavator bucket with soil with a hydraulic excavator. We demonstrate the effectiveness of algorithms using the reinforcement learning approach on the commercial simulator, Vortex Studio developed by CM Labs Simulations.

Keywords: Reinforcement Learning; Imitation Learning; Automation; Autonomous; Simulation