The International Association for Automation and Robotics in Construction
Publications / CCC 2025 - Zadar, Croatia
HISTORICAL BUILDING ROOF STRUCTURE EXTRACTION BASED ON UAV HYPERSPECTRAL IMAGERY RECONSTRUCTION
The sustainable preservation of large-scale, actively inhabited historical villages requires detailed and scalable methods for material-aware structure extraction and health monitoring. Traditional Unmanned Aerial Vehicle (UAV) RGB imagery, while accessible, suffers from poor adaptability in complex environments, particularly when attempting to distinguish between diverse roof types, repair materials, and naturally degraded components under occlusion or varying illumination. To address these limitations, we propose a three-stage framework that reconstructs multispectral imagery from UAV-acquired RGB inputs and leverages prompt-driven semantic segmentation for heritage architecture understanding. First, high-resolution RGB images are captured via UAVs and rigorously filtered to ensure geometric and radiometric quality. These images are then processed using a deep multispectral reconstruction network (MST++), generating spectral cubes that enhance material differentiation. In the final stage, we introduce a semantic extraction pipeline grounded in a custom-built knowledge base of architectural materials and typologies. Semantic prompts (e.g., "damaged clay tile roof", "metal-repaired roof") are generated from this knowledge base and used to guide segmentation via the prompt-aware SAM2 model. Experimental results on aerial data from Lai Chi Wo village in Hong Kong demonstrate that the reconstructed multispectral imagery provides superior performance in segmenting complex features such as metal roofs and rooftop vegetation, compared to RGB inputs. The enhanced spectral sensitivity enables improved alignment between semantic prompts and physical features, yielding accurate material-aware maps. This approach supports long-term heritage building monitoring and sets a foundation for proactive, informed conservation strategies for diverse traditional architectural landscapes.