Accurate matching between BIM-rendered and real-world images

As the digital representation of the built environment,BIM has been used to assist robot localization. Real-worldimages captured by the robot camera can be compared withBIM-rendered images to estimate the pose. However, there isa perception gap between the BIM environment and reality;image styles are typically too different to be matched. Hence,this study investigates an advanced image feature detectiontechnique, D2-Net, to identify key points and descriptors onBIM-rendered and real-world images. These key features arefurther matched via K Nearest Neighbor Search and RANSAC.The ability to bridge the perception gap can be evaluatedby the image matching performance, which is the Euclideandistance between the projected key points and the number ofinliers. SIFT, as the traditional feature detection technique, wascompared in this study. Results show that the average projectionerror of D2-Net is only 16.55 pixels, while the error of SIFT is187.46 pixels. It demonstrates that the advanced D2-Net can beutilized to detect representative features on BIM-rendered andreal-world images. The matched image pairs can be furtherutilized to estimate the robot pose in BIM. Overall, it aims toenhance the BIM-assisted localization and improve the robot'sreliability as a decision-making tool on-site.