Publications / 2024 Proceedings of the 41st ISARC, Lille, France

Comparative Analysis of Cognitive Agreement between Human Analysts and Generative AI in Construction Safety Risk Assessment

Unmesa Ray, Cristian Arteaga, Jee Woong Park
Pages 452-458 (2024 Proceedings of the 41st ISARC, Lille, France, ISBN 978-0-6458322-1-1, ISSN 2413-5844)
Abstract:

The construction industry struggles with safety risk assessment complexities due to evolving work environments, diverse labor forces, time constraints, regulatory intricacies, and inconsistent practices. While previous studies have highlighted the potential of Artificial Intelligence (AI) in automating processes and enhancing safety assessment, a gap exist in convergence between human analyst and language AI models. Therefore, this study seeks to assess the alignment in identification of risk factors by human analysts and a Language Model (LM) in Occupational Safety and Health Administration (OSHA) accident reports. Furthermore, it offers to: 1) categorize error types, 2) establish an acceptance threshold for LM-generated responses, and 3) evaluate inter-rater reliability in construction accident content analysis. Test results reveal significant convergence between human and machine responses and identifies potential hallucination effects in generative AI, thus paving the way for improved safety risk assessments within the construction industry.

Keywords: construction industry, safety risk assessment, Artificial Intelligence (AI), Occupational Safety and Health Administration (OSHA), Language Model (LM), inter-rater reliability, generative AI