Real-time Digital Twin of Robotic construction processes in Mixed Reality

The use of robots in construction has increased the safety and productivity in the construction site. Robots execute complex programmed codes to complete construction related tasks. Unfortunately, there are limitations on project management to monitor the execution process of robot due to undeveloped human-robot collaboration interfaces. Digital Twin (DT) is an emerging technology that has been recently explored as a potential solution to these limitations. DT combines the high performance of a robot and the high cognition of humans to enable robotic construction processes. Mixed Reality provides an immersive environment for human-robot interaction for on-site construction processes. Nevertheless, DT in Mixed Reality has not yet been widely explored and developed in the construction industry. Thus, this work explores effective human-robot collaboration for construction processes through monitoring and investing the effectiveness of on-site real-time DT of robotic process in Mixed Reality construction environments. To do so, this work firstly creates a DT of a robotic construction process by establishing a two-way communication between both, the real and virtual model of the robot. The robotic control of the robotic arm is executed and simulated in 3D modelling and visual programming environment. Once the path is simulated, the command is sent to the robot using LAN connection. Live data collection such as the coordinates of the robot is obtained and sent to the visualisation and BIM-based platforms. This work develops a workflow to send the real-time data to the Mixed Reality glasses for direct visual feedback and the possibility of interacting with the robotic set-up as for example a direct interaction with physical construction site with real time human control. To demonstrate and validate the DT in mixed reality, a simple prototype of robotic masonry construction is developed. The goal is to improve adaptive feedback by collecting data and checking tolerances by using the DT process of constructing a prototype of computationally designed brick wall, which requires high precision labour to be constructed. Thus, usage of robotic process is one solution towards precise construction. This work also uses live data from the robotic process to monitor and provide feedback loop in the construction process. Its execution is visualised in parallel with BIM-based systems to compare the fabrication with the BIM model. This prototype was further validated by industry experts. This work aims to serve the foundation of real-time DT of robotic construction processes in Mixed reality. It further extends the knowledge in construction management for digital fabrication and human-robot collaboration. This work provides a potential future research on data access and manipulation for digital processes in construction site, enabling a whole new series of enactive processes that need adaptability in its fabrication processes such as, concrete 3D printing, self-compacting concrete, and fabric formworks.