Façade condition assessment for buildings is essential to public safety in cities. Currently, twelve major cities across the U.S. ensure the building façade safety with mandatory façade inspection programs. Even in major cities with the façade inspection programs, there have been seventeen falling debris accidents reported in 2019, three of which were fatal. These accidents indicate a need to improve the current façade inspection practice. Shadowing work conducted by the research team with expert inspectors on three buildings, and analysis of façade inspection programs and guidelines show that inspectors check façades based on defect types or on façade components, whereas existing documentation to guide inspectors are based on major material types. This mismatch results in inspectors checking façade components based on their experience, which might not align well with the expectations of agencies. Systematic and detailed assessment guidance is necessary to get a comprehensive and consistent façade inspection. Towards such systematic guidance and to understand the underlying reasons for continuing accidents, this paper provides the details of an approach to identify generic vocabularies and the relationships between major entities that play a role in the inspection domain for systematic inspection processes. To identify these, we developed a data-driven approach that analyzed around 100 façade inspection reports that were filed to the NYC Department of Buildings (DOB) during the past inspection cycle (2014-2019). Among the twelve major cities, New York City (NYC) has the longest history of façade inspection, and most buildings (14,000) enrolled in the façade inspection program. We believe that study about NYC buildings can provide a general understanding of inspection requirements in other cities where similar problems exist. The developed mechanism is based on natural language processing and unsupervised machine learning techniques and is used to extract the vocabularies of façade elements, defect types, associated defect attributes, and mapping between them. The results also provide the mapping relationship of façade components and defect types for a specific façade type (e.g., stone/limestone). This work provides the foundation for an ontology to be used to systematically guide façade inspection for any given building.