Real-time activity monitoring is becoming one of the most significant technologies on construction sites because it can be applied to a variety of management problems, such as productivity formula and safety monitoring. However, current monitoring technologies are limited to recognizing postures in an ideal environment rather than dealing with the ambient occlusion and people-intensive situations on real construction sites. Therefore, this study develops a low-cost, non-intrusion and portable tool system in order to trace and track construction activities on complex and crowed construction sites. This system is composed of wireless sensors and a smart algorithm. Each sensor consists of an inertial measurement unit, a communication sensor (bluetooth low energy sensor) and several environmental sensors, which broadcasts the identification, acceleration, palstance and environmental measurements at a constant frequency. Since the dimension of the sensor is only 20 x 15 x 2 mm, it can be easily attached or screwed on to the hand tools as well as integrated with power tools. When a laptop or cell phone receives from these sensors, the construction activities are derived by the artificial intelligence algorithm in a timely manner, providing an visual posture monitoring as well as an automatic record of project progress. In the end, practical experiments of a concrete vibrator and a hammer prove the feasibility and effectiveness of the proposed IMU-based tool tracing and tracking system.