Construction progress monitoring has been recognized as one of the key elements that lead to the success of a construction project. The first requirement for effective progress monitoring is the collection and analysis of construction progress information. Through the use of image retrieval, progress information about structural components can be derived from the construction site image. In this paper, the method of color model-based, concrete image retrieval is proposed for utilization in construction progress monitoring. For effective concrete image retrieval, a comparison of concrete color models in four invariant color spaces, such as normalized rgb, HSI, YCbCr, and CIELUV, is conducted. Then, the best color configuration and color space to model the inherent concrete color and to efficiently discriminate between concrete and other objects (or non-concrete objects) are determined, using Mahalanobis distance and performance measures. Experimental results show that L-U color configuration in CIELUV color space yield the optimal retrieving performance, and subsequently, the highest retrieval rate of concrete color.