The use of robotic systems on construction sites can efficiently reduce construction time and increase safety by replacing construction workers in monotonous or dangerous operations. Robots for on-site construction applications are challenging and difficult to implement because of the evolving and unstructured nature of construction sites, the inherent complexity of construction tasks, the uniqueness of products, and labor-intensive modeling and commanding, which require significant human effort and expertise. With the development of data-driven techniques such as machine learning and computer vision, more advanced frameworks and algorithms can be developed to increase the level of adoption in the automation of construction robots. To better understand existing challenges and figure out the best strategies to implement high-level autonomous robotic systems for on-site construction, this study (1) summarizes technologies and algorithms used in construction robots and robotic applications in other industries, (2) discusses potential best usage and development of computer vision and machine learning techniques used in related areas to implement higher-level autonomous construction robotic systems, and (3) suggests a preliminary framework that integrates different technologies, such as vision-based data sensing to collect information, advanced algorithm to detect objects and reconstruct models of the built environment, and reinforcement learning to train robots to self-generate execution plans. This will allow construction robots to navigate and localize on construction sites, recognize and fetch materials, and assemble structures per a simulated plan. The proposed conceptual framework could help with the definition of future research areas utilizing complex robotic systems.