Throughout the life cycle of civil assets, construction, operation and maintenance phases require monitoring to assure reasonable decision makings. Current methods always involve specially-assigned personnel conducting on-site inspections, which are work-intensive, time-consuming and error-prone. Computer vision, as a powerful alternative to manual inspection, has been extensively studied during the past decades. On the basis of existing summary papers, this paper reviews a wide range of literatures, including journal articles, conference proceedings and other resources. Current applications of computer vision during construction, operation and maintenance stages of civil structures are concluded, with a special focus on operation and maintenance phase. This review aims to provide a comprehensive insight about the utilization of computer vision in civil engineering and an inspiring guidance for future research.