Wearable Sensor-based Hand Gesture Recognition of Construction Workers

Maintaining good communication is important for keeping the construction site safe and the project running smoothly and on schedule. Hand gestures, as one of the common ways to communicate, are widely used on construction sites due to their simple but effective nature. However, the meaning of these hand gestures was not always captured precisely, which would lead to construction errors and even accidents. This paper presented a feasibility study on investigating whether the hand gestures could be captured and interpreted automatically with wearable sensors. A new dataset which is made of the accelerometer and gyroscope data is created. The created dataset contains 8 classes of hand gestures for instructing tower crane operations and is employed to compare two state-of-the-art deep learning networks, namely, Fully Convolutional Neural Network (FCN) and ResNet, and measure their hand gesture recognition performance. The comparison results indicate that a high classification accuracy (96.9%) could be achieved. Further, a pilot study was conducted in a laboratory environment to test whether the methods used in this study could serve as an interface to help workers control and/or interact with construction machines.