Publications / 2022 Proceedings of the 39th ISARC, Bogotá, Colombia
Construction robots have the potential to increase construction productivity at job sites and can help overcome industry challenges such as labor shortage and safety risks. User-friendly interfaces are critical for advancing human-robot work collaboration and increasing use of construction robots. However, human-robot interfaces in the context of construction industry applications have been investigated to a limited extent only. This paper proposes a novel sensor-based framework which integrates eye tracking and hand gesture recognition for human-robot interaction in construction. Specifically, it begins with visual detection of construction machines in the first-person views. Then, the machine-of-interest is determined based on the detection results and human gaze points. Finally, a real-time hand gesture recognition system is employed for conveying messages to the machine to guide its operations. So far, the proposed framework was tested in a laboratory setting using a robotic dump truck. The results showed that the proposed framework could serve an effective interface to support the interactions between workers and construction machines.