Construction sites are continuously changing environments where construction workers have to adapt to dynamic situations while executing their work tasks safely and efficiently. Simultaneously, they are surrounded by heavy construction machinery and massive crane loads which they have to be aware of at any time. Frequent interruptions may often lead to a loss in productivity and also causes hazardous conditions or even incidents, injuries or fatalities. Tracking workers paths on site can be used to approach these issues. Recorded tracks can be used to identify close calls of inexperienced or distracted workers. To date, pedestrian workers may participate in customized trainings in order to overcome individual deficits. Machine operators can be assisted to mitigate hazardous situations by warning them from construction workers approaching their machines. Since surveillance cameras are already existent on most construction sites, a video-based detection and tracking system can be implemented at low costs. Relying on video streams, the detection of workers becomes similar to pedestrian detection. Some effort has already been made to elaborate those methods to the needs of construction worker detection. However, in contrast to the frontal view supposed in most pedestrian detection approaches, cameras on construction sites commonly provide oblique or birds-eye view perspectives. This complicates the detection task as most body parts of a worker are occluded. Hence, we evaluate the applicability of pedestrian detection approaches in terms of the camera settings at hand. Ensuing, we propose a concept for the detection and tracking of construction workers which allows to improve the productivity and safety on construction sites.