Fusion of Time-Of-Flight Camera and Stereo Camera Data for Enhanced 3D Work Environment Representation for Construction Equipment Automation

As the need to develop semiautomated and automated construction equipment increases in the construction industry, 3D work environment representation becomes increasingly important. Accurate, realistic, and rapid 3D representation of the work environment can be utilized for obstacle detection and collision avoidance by providing information about the state of the work environment. This paper describes a system to create a 3D graphical representation of the work environment for safe and efficient equipment operation that has desirable properties and satisfies the constraints on the development of semiautomated and automated construction equipment. For this purpose, a unified framework for a multi-sensor data fusion-based 3D work environment representation system is proposed. The proposed framework consists of four main steps: data acquisition, data filtering, data fusion, and 3D graphical model generation. A field experiment was undertaken on a construction site to validate the proposed system. The preliminary research results show that the resulting 3D graphical representation of the work environment can be successfully employed in the development of construction equipment that assists the operator in identifying obstacles or operates independently.