Vision-Based Articulated Machine Pose Estimation for Excavation Monitoring and Guidance

The pose of an articulated machine includes the position and orientation of not only the machine base (e.g., tracks or wheels), but also each of its major articulated components (e.g., stick and bucket). The ability to automatically estimate this pose is a crucial component of technical innovations aimed at improving both safety and productivity in many construction tasks. A computer vision based solution using a network of cameras and markers is proposed in this research to enable such a capability for articulated machines. Firstly, a planar marker is magnetically mounted on the end-effector of interest. Another marker is fixed on the jobsite whose 3D pose is pre-surveyed in a project coordinate frame. Then a cluster of at least two cameras respectively observing and tracking the two markers simultaneously forms a camera-marker network and transfers the end-effector's pose into the desired project frame, based on a pre-calibration of the relative poses between each pair of cameras. Through extensive sets of uncertainty analyses and field experiments, this approach is shown to be able to achieve centimeter level depth tracking accuracy within up to 15 meters with only two ordinary cameras (1.1 megapixel each) and a few markers, providing a flexible and cost-efficient alternative to other commercial products that use infrastructure dependent sensors like GPS. A working prototype has been tested on several active construction sites with positive feedback from excavator operators confirming the solution's effectiveness.