This paper addresses the problem of automatic inspection by means of Unmanned Aerial Vehicles (UAV) for health monitoring of infrastructure. Our approach comprises two stages, data collection using unmanned aerial vehicles and image processing with histogram analysis. For the data collection, a 3D model of the monitored structure is first created by using vision-based sensors attached on the UAV. Based on the model developed, geometrical properties are extracted to generate way points necessary for navigating the UAV for image capturing of the structure of different materials, for example, concrete. From the images obtained, our next step is to stick them together using the overlapped field of view. We then create histograms of the stuck images and detect peaks based on cosine similarity. We finally identify a potential crack or surface defect as location of the histogram peaks. The whole process is automatically carried out so that the inspection time is significantly improved while minimising any safety hazards that may be encountered in the UAV inspection process. A prototypical system has been developed with obtained results being evaluated and verified to show its validity.