Physical interpretation of machine learning-based recognition of defects for the risk management of existing bridge heritage

The challenge of the research work presented in the paper is to combine the growing interest in monitoring the health condition of existing bridge heritage through systematic and periodic visual inspections with automated recognition of typical bridge defects, which can greatly facilitate the assessment of defect evolution over time. The study focused on the automated identification of defects in existing Reinforced Concrete (RC) bridges exploiting different Deep Learning (DL) approaches and techniques to interpret the obtained predictions. Ensuring the safety of infrastructures is typically a technical and economic issue. Still, in the case of the engineering infrastructure heritage, there are existing bridges and viaducts with a high historical, cultural, and symbolic value. For them, accurate knowledge and characterization of possible degradation processes become particularly important in order to define intervention strategies that combine safety and conservation requirements. With the aim to develop systematic and non-invasive investigation protocols for continuous and effective control of defects and their evolution, a database of existing RC bridge defect images was collected, and the most recurrent defect typologies were classified by domain experts. Some existing Convolutional Neural Networks (CNNs) algorithms were applied to the dataset for automatically recognizing all defects, but the specific novel contribution of the research work is the interpretation of the obtained results in a form that is humanly explainable and directly implementable in new tools for bridge inspections. To interpret the results, Class Activation Maps (CAMs) approaches were employed within available eXplainable Artificial Intelligence (XAI) techniques, which allow to observe the activation zones and nearly perfectly highlight the type of specific defect in a given image. The obtained results, besides suggesting which network works better than others and if the specific defect is effectively recognized, have been evaluated through a quasi-quantitative procedure that compared a qualitative assessment of the CNNs models reliability with two novel indexes representing new explaining metrics of the obtained results. In the end, the outcomes of the proposed study were observed also in a real-life case study. The proposed discussion opens new scenarios in the application of these techniques for supporting road management companies and public organizations in the evaluation of the road networks health state.