Automation in construction is considered as one of solutions to fundamental problems which the construction industry has confronted. The development of real-time 3D local terrain modeling is the core methodology of achievement of IES (Intelligent Excavating System) which is a mega-size research concerning automation in construction. Stereo vision technique was selected as an appropriate method to achieve 3D terrain modeling in real-time. The objective of this study is to suggest a conceptual method by use of the stereo vision and assess feasibility of practical application under various conditions of construction sites. Bumblebee XB-3 is adopted as the stereo vision system in a prototype of a 3D terrain modeling system. This study is to investigate fundamental theories of the stereo vision system and to conduct tests in indoor and outdoor environments for determining appropriate detection range, finding installation location of cameras, doing visual assessment of the technical appropriateness, and checking reliability of 3D visual data. These decomposed tasks present valuable information for establishment of the conceptual method and improvement on the limitations of the use of stereo vision system for 3D terrain modeling in real-time.