Audio Signal Processing for Activity Recognition of Construction Heavy Equipment

Action recognition and tracking of construction heavy equipment is the first step for benchmarking and analyzing the performance of individual machines and evaluating the productivity of a jobsite as a whole. Aside from direct observations, the current approaches for automatically recognizing and tracking various actions of construction heavy equipment includes: 1) using active sensors such as RFID tags, GPS and accelerometers or 2) computer vision-based activity analysis (processing images or videos). In this paper, we present a novel audio-based approach for activity recognition of construction heavy equipment. Construction machines often produce distinct sound patterns while performing certain activities and it is possible to extract useful information by recording and processing those audio files at construction jobsites. The proposed audio-based framework begins with recording generated sound patterns of construction equipment using commercially available audio recorders. The recorded signal is then fed into a signal enhancement algorithm to reduce background noise commonly found at construction jobsites. The modified audio signal is then converted into a time-frequency representation using the Short-Time Fourier Transform (STFT). A Support Vector Machine (SVM) is then trained to differentiate between the acoustic patterns of the various activities of each machine. The processed audio signal is then finally divided and classified into various activities using a window filtering approach and by setting proper thresholds. We implemented the presented audio-based system at several jobsites as case studies and the results illustrate the efficiency of the system in automatically recognizing various actions of construction heavy equipment.