IIn the construction industry, there is often a need to identify and localize assets and activities on the jobsite to assess and improve the performance of their associated processes. Traditional methods for monitoring construction activities are costly and time-consuming. Excavators and dump trucks are among the most common assets used in the construction industry. Consequently, accurately monitoring their activities can reduce time and increase the efficiency of progress monitoring. With the presence of cameras on jobsites and the advancement of methods based on artificial intelligence and computer vision, progress monitoring activities can be automated. Furthermore, by using techniques such as deep learning, a wider range of data resources can be processed, and oftentimes more accurate results can be produced for the purpose of object detection. This research proposes a computer-vision approach that utilizes a Mask Region Based Convolutional Neural Network (Mask-RCNN) to detect excavators and dump trucks in a construction site. This research investigates an innovative technique to achieve high accuracy object detection using relatively small datasets. To overcome the problem of overfitting and improve generalization, a pre-trained model based on a Microsoft COCO dataset is used as a network that presumably has already been trained to distinguish basic features. Finally, the model is further fine-tuned to minimize validation loss.