|
VIRTUAL VISUALIZATION FOR THE MECHANICAL TRADE
Phillip S. Dunston1, Mark Billinghurst2, Yihong Luo2, Ben Hampson3
1Department of Civil and Environmental Engineering
University of Washington,
Seattle, Washington
(pdunston@u.washington.edu)
2Human Interface Technology Laboratory
University of Washington,
Seattle, Washington
3The McKinstry Company, Seattle, Washington
|
Abstract: 3D CAD visualization tools provide a limited sense of the merging between design and real space and are still constrained by flat media. Furthermore, new computer interface metaphors will emerge that the construction industry should prepare to embrace. An outside-the-computer paradigm is the next step in visualization. Virtual reality and positioning technologies are converging to enable development of new visualization tools for both the design and construction phases. An experimental prototype with three visualization modes is being developed to explore virtual reality applications in mechanical detailing. This paper describes the program of research and development.
Keywords: 3D CAD, augmented reality, computer-aided drawing, detailing, mechanical, piping, virtual reality, visualization
|
1 INTRODUCTION
Three-dimensional computer aided drawing (3D CAD) and walk-through constitute the most advanced practices in construction design visualization today. Less abstract than 2D design visualization, this technology for the construction industry has emerged with the commercial accessibility of high-end graphics computing. As the nature of computing continues to evolve due to faster processing and greater miniaturization, the metaphors for human-computer interfaces will likewise evolve. Currently, the most innovative human-computer interface models of interest to the industrial sector are generally linked to virtual environments.
Researchers in the construction industry are recognizing the potential benefits of virtual reality (VR) technology. For example, Fu and East [1] presented a Web-based virtual design review environment constructed with a virtual reality modeling language (VRML) translation of the CAD drawings. In their virtual worlds-type environment, designers and reviewers meet, each represented by an avatar, within the VRML model to conduct the design reviews. Their idea represented a significant step for civil engineering in exploring the design and |
construction planning benefits of virtual design environments. Issa [2] has asserted that the integration that the industry so much needs can be facilitated by a virtual reality platform supported by knowledge-based database systems. This platform functions as the main interface to construction information for all specialties throughout the life cycle of a construction project. Major application prototypes would integrate construction drawings, design and material specifications and virtual collaborative prototyping. VTT Building Technology researchers [3, 4] have been engaged in long-term experimentation with 4D applications (3D product model + time) based on VR technology along with the associated software and modeling infrastructure. They have explored non-immersive VR and augmented reality, with desktop computer interfaces.
All of the above research efforts are focused upon project information content (e.g., underlying and associated data) and forms of 3D representation and interaction via the standard desktop computer monitor. In light of the considerations mentioned above concerning the evolution of computing, the authors believe there is merit in also experimenting with the modes and means of interaction with the |
|
virtual design. This paper presents the concept of a prototype augmented reality computer-aided-drawing (AR CAD) system designed for exploring the benefits of supporting design, and ultimately construction, with various modes of virtual reality beyond the desktop-monitor-bound interface.
2. MIXED REALITY
Milgram and Kishino [5] describe a virtuality-reality continuum extending from the completely computer-generated world to the completely real world. Inside these extremes, lies a rich range of what is termed mixed reality where the balance between virtual and real can vary substantially. Augmented reality (AR) occupies that place in the continuum where virtual objects are inserted to a predominantly real world scene. AR also allows virtual enhancements to physical interface objects. If VR is to be exploited, a valid question is where in the continuum do architecture, engineering, and construction (AEC) practitioners derive the most benefit. The research that is presented here explores fully immersive VR and augmented reality.
Protoype AR systems have already been developed for visualizing architectual features that cannot normally been seen in finished buildings [6], for aiding with the construction process [7] and for visualizing virtual buildings overlaid on a real outdoor environment [3, 4, 8]. The success of these efforts suggest the usefulness of mixed reality interfaces in the construction industry as a whole. In our work we are interested in applying more innovative AR techniques to the design phase of the construction process, an area that has previously received little attention.
3. MECHANICAL DETAILING
For several reasons, the authors have chosen to focus upon the specific application area of mechanical detailing. Mechanical detailing involves a high degree of attention to spatial constraints and requires collaboration between different specialty areas. Many mechanical contractors now do their detailing in 3D CAD because they have the highest stake in the negotiation of limited space shared with other specialty trades. Therefore, they are already creating the 3D content that can be readily used in VR and AR applications. Mechanical contractors may also install multiple systems (e.g., HVAC and plumbing) in the same facility and therefore require in-house coordination. Thus they make a good candidate for exploring the coordination benefits of VR and AR before greatly increasing the number of collaborators. Finally, mechanical contracting firms must essentially do their own design and construction, including prefabrication. These circumstances offer the opportunity for research and development for all steps in the design-construction process.
| Figure 1 shows an illustration of a typical design from a mechanical contractor, a piping assembly called a skid. Skids are usually preassembled and installed by crane into larger piping networks. Such an assembly was chosen as a proper unit around which to experiment with VR applications for design, planning, and construction.
Figure 1. Example of a Skid Design
The industrial partner involved in the initiation of this research has developed an in-house AutoCAD®-based application for producing 3D drawings such as the one shown in Figure 1. The in-house application generates objects based upon catalog specifications for commercially available parts. Examination of a similar leading commercial product confirmed that the approach of developing a prototype that adds AR capabilities to such an existing system was feasible.
4. ARCAD CONCEPT
The AR CAD tool enables the generation of virtual design spaces that may be used not only by the design function, but also to support the development and execution of construction plans. The experimental prototype under development will provide the piping detailer with the ability to explore the CAD design in both the immersive and non-immersive (AR) virtual reality modes.
4.1 General Description
The system under development consists of the following components (Figure 2):
- A modeling computer running AutoCAD® or other modeling software
- An AR computer running the augmented reality viewing software (the AR interface).
- A head mounted display (HMD) with small camera attached
|
|
Figure 2. Components of AR CAD Prototype
The modeling computer runs AutoCAD® with a specially designed AutoCAD® plug-in. The CAD detailer designs the model on this machine and then sends the 3D model information out to the AR computer using the AutoCAD® plug-in. The plug-in software uses standard network communication code to communicate between computers.
The AR computer runs an augmented reality application that allows a user to see virtual 3D models superimposed over the real world. This application is based on top of a software library called ARToolKit [9], a free augmented reality tracking library developed at the Human Interface Technology Laboratory (HIT Lab) at the University of Washington. ARToolKit is a C software library that uses computer vision techniques to precisely overlay virtual models onto the real world. The AR application receives the 3D model information through the network communication and then instantly creates a 3D virtual model of the design.
Also connected to the AR computer is a lightweight HMD with a small video camera attached. The computer performs image processing on the video image from the camera to find specially marked tracking cards. The user¹s head position can be calculated from a tracking card and a virtual model overlaid on the card. The resultant composite image is fed back into the HMD for the user to see. The result is a view of the real world with 3D virtual models overlaid on it (Figure 3). This display technique enables the user to easily view the model from any perspective above the card.
Figure 3. User with HMD and Camera Viewing a Virtual Object
It is envisioned that the detailer will use the system as follows. The user sits at the modeling computer producing a CAD model of some pipe |
layout for a new building. Beside the detailer is a tracking card and a lightweight HMD. Periodically, the detailer may want to inspect a less abstract representation of the model. After clicking an "export-model-to-AR" button in the CAD package, the detailer can look at the board through the HMD and see a 3D version of the CAD model. The detailer can pick up the card and look at it from any orientation above the marker. The detailer can also transition to a fully immersive perspective to inspect the design up close. When satisfied, the detailer can remove the HMD and return to modeling.
The final product is envisioned to function on a single desktop computer. Figure 4 portrays the ideal setup for the final version of the system being operated in the AR mode. The virtual design appears adjacent to the desktop computer registered on the tracking card for positioning and manipulation.
Figure 4. Ideal Setup for Detailer Using AR CAD
4.2 Prototype Implementation
In the previous section we describe our vision of how the final AR CAD system will be used. The system is still in development and the current prototype AR interface has two major software components: the 3D modeling interface, and an AR interface that includes a VRML parser built on top of the ARToolKit software library. In the current implementation the 3D modeling interface is running on a standard Pentium class Windows PC, while the AR interface is running on an SGI O2 Unix workstation connected by Ethernet network. In this section we describe each of these systems in more detail, outlining the work that has been completed.
4.2.1 The 3D Modeling Interface
The 3D modeling interface combines the popular AutoCAD® software with a custom plug-in. We decided to base our work on a commercially available CAD program because of the huge installed user base and the reluctance of users in the construction industry to learn new software packages. Like many modeling packages, AutoCAD® allows developers to write plug-in modules to provide additional functionality. The designer can use AutoCAD® to |
|
design the assembly models, however we have developed a plug-in that sends model information to the separate computer running the AR interface.
The plug-in program has two functions:
- Keep track of all the models in the virtual scene and individual model information such as position, orientation and scale.
- Communicate with the AR computer.
The communication between modeling computer and AR computer is via network code that is based on standard TCP/IP socket communication libraries. These include the functions of initializing the communication, sending data, receiving data and disconnecting the communication. The data sent by the modeling computer takes the form of a set of text strings containing the name and position, scale and orientation of the various model parts. For example, to create a pipe model number p101a and place it at x, y, z position (100,20,45.5), with orientation (1,0,0,1.57) (rotation axis and amount of rotation in radians) and Sx, Sy, Sz scale (10,2.3,4.5) the following string is sent:
1 add p101a 100 20 45.5 1 0 0 1.57 10 2.3 4.5
After each string is received at the AR computer an acknowledgement is sent to ensure no data is lost.
4.2.2 The AR Interface
The AR viewing software is a custom application running on an SGI O2 computer that is based on the ARToolKit library and the LibVRML97 open source VRML parser [10]. The third component of the software is a database containing simplified 3D models of pipe structures in the VRML file format.
ARToolKit. One of the most difficult aspects of augmented reality is accurate viewpoint tracking. In order for a virtual object to appear to be overlaid exactly on the real world, the user¹s viewpoint must be tracked in real time. ARToolKit uses computer vision techniques to accurately track the camera viewpoint based on square tracking markers.
First the live video image is turned into a binary (black or white) image based on a lighting threshold value. This image is then searched for square regions. ARToolKit finds all the squares in the binary image, many of which are not the tracking markers. For each square, the pattern inside the square is captured and matched against some pre-trained pattern templates. If there is a match, then ARToolKit has found one of the AR tracking markers. ARToolKit then uses the known square size and pattern orientation to calculate the position of the real video camera relative to the physical marker. A 3x4 matrix is filled in with the video camera real world coordinates relative to the card. This matrix is then used to set the position of the virtual camera coordinates. Since the virtual and real camera
| coordinates are the same, the computer graphics that are drawn precisely overlay the real marker.
In the ARCAD application the users wears a HMD with a video camera attached and it is these video images that are used by ARToolKit. Thus tracking the camera viewpoint is the same as tracking the person¹s head. So when they look at a tracking card through the HMD they will see a virtual object on the card (see Figure 5). The tracking patterns can be made as large as desired and the tracking range increases with card size
Figure 5. Augmented Reality Pipe Overlay on a Large Tracking Card Using ARToolKit
LibVRML97 Parser. LibVRML97 is a cross-platform C++ library for reading and viewing VRML files. It is open source code freely available from (http://www.openvrml.org/) and uses the OpenGL graphics library for low-level graphics rendering [11]. In this project we ported the code to the SGI IRIX platform so that is was compatible with the ARToolKit code.
The VRML Model Database. VRML is a file format for describing interactive 3D objects and virtual scenes. It has been used in a variety of application areas such as engineering and scientific visualization, multimedia presentations, entertainment and educational titles, web pages, and shared virtual worlds. Many commercial modeling packages export 3D models in VRML format making it very easy to generate VRML content.
In this project we want to be able to view virtual pipe structure models in real time. Pipe models typically contain many curved surfaces and so are often composed of a large number of polygons, making it difficult to view them interactively. We avoid this problem by using a database of simplified pipe models in VRML format. These models are based on high-polygon models that are used as primitives in the skid CAD design process. They were simplified and converted into VRML format using the software 3D Studio Max. So although the AutoCAD® model of pipe p101a may have over 500 polygons, the simple VRML model of the same pipe is less than 50 polygons.
When the AR application initially starts, only a virtual ground plane is drawn on the real tracking |
|
card. As each command is received from the modeling computer the model number is used to find the equivalent simplified VRML model in the model database. VRML files contain position, orientation and scale information as well as object geometry. So when the model is found it can be moved and scaled to match the position, size and orientation of the same part in the original CAD model. In this way users can easily add, delete and modify models in real time. Over time, as more and more pipe and duct parts are added to the scene, a complete VRML scene will be built up that matches the scene being created in the PC modeling program.
4.3 Development Status
The AR prototype is still in an early stage of development and does not yet have enough features to be evaluated by real users. However we are able to load and view virtual skid models in an AR setting and have simple interactions with these models. Figure 6 shows the AR view of a portion of the skid from Figure 1 as it appears through the HMD. This model contains around 10,000 polygons and is rendered at 7-8 frames per second. The user is able to view this model from any perspective simpy by picking up the card and turning it. Being able to manipulate a physical object to change virtual camera viewpoint is one of the most compelling aspects of the AR interface and has proven very intuitive to use, even by computer novices.
Figure 6. AR View of a Portion of the Skid
Code development has continued to progress toward completion of a fully functioning prototype. The most significant hurdle relates to the integration of hardware and modified component software to achieve satisfactory frame rates for real-time perception of at least 15 frames per second. This work has focused on establishing a experimental prototype based upon commercially viable platforms. Thus we are in the process of moving our AR software from the SGI to PC platform after which we will further explore AR alternatives for interacting with the design (virtual model).
5. AR TOOL FOR CONSTRUCTION |
Once a usable prototype has been completed, the next major step will be to examine how AR techniques can be used on-site in the construction process. A precedent has already been set for how the AR CAD platform might be extended to support the actual fabrication of skids. Researchers at Columbia University [6, 7] described research with AR to explore architectural anatomy and a testbed for applying AR in the construction of spaceframes. The testbed system was designed to direct the constructor in the step-by-step process of correctly assembling the tubular spaceframe elements which appear to be identical but have different inner diameters. Through an HMD, the constructor is able to see the virtual spaceframe elements assembled as designed along with step-by-step text instructions. A barcode reader linked to a parts database is used to confirm the identification of elements that are designed for specific loads and thus locations in the spaceframe. An optical radar tracker was used to accurately register position. A similar scheme might be used for guiding workers in assembling spools and pumps to construct skids.
In a future system, key components would be much the same as in the spaceframe example. However, a more novel scheme for part identification would be desirable to speed up construction. Techniques for position and orientation tracking should be selected for accuracy at greater sight distances and the AR interface could run on a lightweight wearable computer. Ideally, the wearable computer would download the relevant design files and specification data via wireless link to a site computer or even remote office computer.
6. CONCLUSION
In this paper we have described AR CAD, a prototype interface for an augmented reality based design tool. We have shown how it is possible to enhance typical 3D design software with virtual reality technologies that allow the designer to perceive and interact with the virtual design objects in new ways. While most VR research for the AEC industry is focused upon management and communication of project information, there is also a need to keep abreast with the evolution of computing that will change the way we interface with digital information. AR CAD will serve as an environment for exploration of new metaphors for design applications for the AEC industry.
Acknowledgments
The authors are grateful to the McKinstry Company of Seattle, Washington (U.S.A.) for their cooperation in this research and for providing the CAD models used.
REFERENCES |
|
[1] Fu, M. C. and East, W.E., "A Proposed Virtual Design Review Environment", Managing Engineered Construction in Expanding Global Markets, Proceedings of Construction Congress V in Minneapolis, Minnesota, Stuart D. Anderson (ed.), ASCE, pp. 502-509, 1997.
[2] Issa, Raja R. A., "Virtual reality: A Solution To Seamless Technology Integration In The AEC Industry", Construction Congress VI: Building Together for a Better Tomorrow in an Increasingly Complex World, Proceedings of the Congress, Kenneth D. Walsh (editor), ASCE, pp. 1007-1013, 2000.
[3] Leinonen, Jarkko and Kähkönen, Kalle, "New Construction Management Practice Based on the Virtual Reality Technology", Construction Congress VI: Building Together for a Better Tomorrow in an Increasingly Complex World, Proceedings of the Congress, Kenneth D. Walsh (editor), ASCE, pp. 1014-1022, 2000.
[4] VTT, the Technical Research Center of Finland, http://www.vtt.fi, 2000.
[5] Milgram, P. and Kishino, F., "A Taxonomy of Mixed Reality Visual Displays", IECE Trans. on Information and Systems (Special Issue on Networked Reality), Vol. E77-D, No. 12, pp. 1321-1329, 1994.
[6] Feiner, S. K., Webster, A. C., Krueger III, T. E., MacIntyre, B., and Keller, E. J., "Architectural Anatomy", Presence, Massachusetts Institute of Technology, Vol. 4, No. 3, pp. 318-326, 1995.
[7] Webster, A., Feiner, S., MacIntyre, B., Massie, W., and Krueger, T., "Augmented Reality Applications in Architectural Construction", In Designing Digital Space: An Architect¹s Guide to Virtual Reality, Daniela Bertol (ed.), John Wiley & Sons, New York, pp. 193-200, 1997.
[8] T. Höllerer, T., Feiner, S., Pavlik, J. "Situated Documentaries: Embedding Multimedia Presentations in the Real World", In: Proc. ISWC '99, San Francisco, CA, October 18-19, 1999, pp. 79-86
[9] ARToolKit information website. http://www.hitl.washington.edu/research/shared_space/
[10] LibVRML97 http://www.openvrml.org/
[11] OpenGL website http://www.opengl.org/ |
|
|
|
|
