Multi-Sensor Monitoring for Real-Time 3D Visualization of Construction Equipment

Construction and mining worksites pose unique challenges to workers and equipment operators due to their dynamic and unstructured nature. Narrow haul roads, crowded work spaces and presence of workers and equipment in close proximity, combined with large blind spots and overall poor visibility afforded to operators lead to collisions and run-over-type accidents between equipment, workers, and other entities that may be present on the jobsite. Operations that involve machine-infrastructure interaction, particularly in the case of concealed infrastructure such as excavation and drilling lead to unintended strikes between the equipment’s end-effector and the infrastructure. Such accident-prone scenarios can be avoided by providing operators with real-time feedback and warnings using 3D visualization and proximity monitoring of the entities present in the equipment’s vicinity. Thus the jobsite in the real world is abstracted and represented in a 3D virtual world. However, this requires the ability to create a seamless link between real-world sensors present on the equipment and/or the jobsite and the virtual environment. This paper describes a non-restrictive acquisition allocation framework that allows position and orientation sensor streams from construction and mining job sites to be used for updating 3D virtual scenes in real-time for providing vital collision avoidance warnings to equipment operators.