Railway fasteners are critical equipment since they keep rails fixed. they should be routinely monitored to guarantee the safety of rail lines. The monitoring process performed by trained staff is time-consuming, boring, dangerous, costly, and plagued with human error, especially for long railways. To tackle these issues, several computer vision algorithms have been developed to control railway lines rapidly and automatically. However, they mostly ignore the fact that fasteners might be covered by stones or that their shapes can be significantly different under actual working conditions. Thus, their results suffer from a high false detection rate. Additionally, various types of fasteners with different features are used in the railroad industry which makes their classification challenging. Another issue is the standardization of images captured by cameras installed on trains. As the weather conditions change, the brightness of the obtained images varies substantially, leading to wrong predictions by AI models. Recently, Yang Ou et al. from the mechanical engineering school of the Southwest Jiaotong University in China proposed a new computer vision model based on the Bayesian hierarchical model (FSM) for classification assuming four different conditions for fasteners: normal (N), partially worn (P), missing (M), and covered (C) (Figure 1). They labeled both training and test datasets using a discriminative model called CRF for automatic image segmentation. They employed the support vector machine (SVM) algorithms to analyze obtained fastener images.
The overall process has four steps: First, the CRF model automatically segments dataset images which include 400 fastener images and 400 corresponding hand-labeled fastener structure images (Figure 2). Second, to deal with the missing spatial structures the fastener structure labels are employed. At this stage, the fastener feature images were obtained using the training dataset and the corresponding fastener structure labels were acquired using the trained CRF model parameters. Third, the FSM model is trained using the fastener feature images and labeled datasets, resulting in the topic distribution representation for the fastener images. This step is performed on the test dataset as well. In the last step, a classification model is acquired for four different types of fasteners in various illumination conditions using an SVM classifier (Figure 3). (refer to reference article for more details)
The standard deviation (STD) using the suggested FSM model is more than 10 percent lower compared to that of the LDA model. The authors compared their classification results to other models as shown in Table 1 (the highest values are bold). The suggested FSM model achieves the highest recall and precision rate (except for missing fasteners (M)) compared to other semantic methods.
Table 1. The comparison of the suggested FSM model recall and precision rate to other semantic methods
The FSM model has an F1-score value for each type of fastener (except for missing fasteners (M)) compared to the other semantic methods, and achieves the highest overall classification accuracy with 94.98%, according to Table 2. The authors claim the reason behind their results is that the fasteners are not always presented in the center of the image, and classification accuracy can be highly affected by this parameter. In some models, supervised learning is only applied to the training dataset and not the test dataset, while in the proposed FSM model fastener structure labels are used in both training and test datasets. (refer to reference article for more details)
Table 2. Comparing the suggested FSM model recall and precision to other semantic methods.
Yang Ou et al. also mentioned some shortages of their model. In short, due to some limitations in the CRF model, the versatility of the proposed model is limited. Additionally, poor image quality in some cases, due to factors such as inadequate image exposure, poor weather conditions, or inappropriate Image brightness, affects results as well. In all mentioned cases, inaccuracies in the segmentation step lead to poor classification. The authors suggest manual re-inspection to deal with such issues.
Although computer vision suffers from certain limitations, installing cameras on trains and automatic monitoring of equipment is still the most efficient and cost-effective solution to ensure railway safety.
The main advantages of employing the suggested FSM model are:
(1) The possibility of automatically obtaining the structure labels for different types of fasteners
(2) Avoid the ambiguity of the feature words using fastener structure labels
(3) More robust and efficient compared to the other methods.
You can enter your email address and subscribe to our newsletter and get the latest practical content. You can enter your email address and subscribe to our newsletter.