Realistic 3D Modeling System for Remote Controlled Construction Machines: Multi-Sensor Fusion Based Approach

An important feature of remote controlled machines is the realistic three-dimensional (3D) modeling system for providing information feedback to the remote operator and facilitating remote operation. This paper describes a system for creating realistic 3D models of actual construction environments for the purpose of improving and extending the capabilities of interactive visual feedback between the operator and remote controlled construction machines during earthmoving operations. The proposed approach utilizes the fused data from a flash LADAR and a stereo vision system. The modeling process consists of three main steps: data filtering for noise reduction, data fusion for data enhancement, and realistic 3D model generation for visualization purposes. Field experiments were conducted on a construction site to validate the proposed system. Preliminary research results show that the resulting realistic 3D model can be successfully employed for the development of remote controlled construction machines that require interactive visual feedback.