In order to ensure the safety of construction workers during the construction process, supervision of violations of safety regulations and control of hazardous situations not recognized by the workers are continuously required. However, due to the dynamic nature of the construction site, the hazardous situation is always changing and there is a limit to managing all these risk situations in a manual manner by the supervisor. To address these limitations, various computer vision-based researches are being conducted that can automatically identify and manage risk factors. The aim of this study is to propose a system for detecting potential horizontal and vertical risks of moving or static objects around workers by measuring distance between objects using Stereo Vision. Proposed system consists of motion detection, object detection and distance measurement between objects using depth image from stereo camera. It synthesizes images from multiple cameras into one and uses motion-detection and objectdetection to identify the movement of objects in a vertical or horizontal position. Then, based on the depth information of the stereo camera, the distance is calculated to identify whether a worker is in the hazardous zone. The proposed system was implemented in the lab environment to verify the proposed functionality. The lab test results indicate that the proposed system can detect vertical and horizontal hazards. It is expected to contribute to the prevention of possible accidents by detecting and controlling the dangers around the workers in real time.