TRANSFORMING VISION-BASED INDOOR BUILT ENVIRONMENT MANAGEMENT WITH FEW-SHOT LEARNING

Gelare Taherian; Ehsan Rezazadeh Azar

Abstract:

Visual analysis of the built environment plays a critical role in the Architectural, Engineering, Construction, and Operation (AECO) sector. This type of analysis enables detection of visible discrepancies between the as-is and as-planned states of an asset to support proper decision making for different managerial and operational applications, such as construction progress monitoring and quality control. However, traditional approaches rely on manual inspections and expert knowledge, which can be time-consuming, error-prone, and labor-intensive. Therefore, automated detection of target objects within the visualizations is paramount. Recent advancements in computer vision and deep learning fields have enhanced automation of visual analysis, yet conventional deep learning approaches require large and well-annotated datasets, which may be challenging to develop. Alternatively, Few-Shot Learning (FSL) offers a promising solution by enabling the identification of regions of interest with minimal labeled data. This study investigates FSL, namely Prototypical Networks (PNs), to address the challenge of data scarcity in the indoor built environments, where the diversity of objects is high and data scarcity is a significant obstacle for effective training of conventional deep learning techniques. The method is evaluated on a custom dataset of AECO-specific indoor objects, achieving an average accuracy of 87.33% across three different tested folds of novel classes. The results indicate that the method enables the identification of unseen objects with rapid adaptation to new environments and tasks, while requiring only a limited number of labeled examples per class, thereby contributing to a significant reduction in time and effort required in real-world industry applications.