Aiex is a leading AI-powered no-code industry solutions platform, empowering managers with tools to analyze big data to improve the accuracy and speed of their processes.
Converting conventional quality control (QC) units into intelligent and automated detection systems is a high priority for manufacturing professionals. Intelligent QC systems reduce cost and time in most production lines in various stages of production and significantly improve the quality of the final products. Traditionally, defects are detected by either human operators or computer models trained by time-consuming machine learning (ML) techniques.
Papavasileiou et al. from the department of mechanical engineering and aeronautics at the university of Patras in Greece devised an automated optical system, which can detect and classify aesthetic defects in different sizes, shapes, and positions immediately after the casting process. Employing this approach eliminates the need for training and minimizes the required resources, resulting in a cost-effective solution with high flexibility in terms of introducing new parts/products of the same material in different stages of production(Figure 1). The authors published their results in the “international journal of computer integrated manufacturing”, 2022.
Computer vision (CV)-based solutions in-line with the industry 4.0 paradigm will be immensely helpful for the existing industrial processes and techniques. However, numerous efforts by engineers and researchers Have failed to meet the manufacturing requirements until now, particularly in the field of intelligent visual inspection systems. Defect detection immediately after the casting process involves three main approaches:
Steel industries, for example, have used CV models to detect potential cracks on steel billet surfaces, monitor the surface of hot-rolled flat steel sheets, and inspect the trimmed edges of steel billets. Some image processing techniques such as Top-Hat filter and Hough transform have also been employed for defect detection in steel industries. In other industries, the brightness, contrast, or contours of images have been used to identify defects. Nevertheless, none of these models exhibited the required flexibility as they focus strictly on specific parts or processes and fail to generalize the solution.
Recently, deep learning approaches have been used to detect defects in hot-rolled steel. Surface defects have also been classified using Scale Invariant Feature Transform, and the use of Convolutional Neural Networks (CNN) in detecting welding defects has been investigated. A pre-trained AlexNet architecture has been utilized to classify black lines and wrinkle defects in leather images. Deep CNN algorithms based on TensorFlow have been applied to detect defects in wooden parts, the surface of solar cells, tungsten inert gas welding, steel wire ropes, cylindrical roller bearing components, etc.
Although the results of such studies are promising, some critical shortcomings still exist that limit their usage in real manufacturing settings. These issues include (i) the trained model’s accuracy for identifying and classifying aesthetic defects highly depends on the quality of training data and the labeling procedure. (ii) the amount of required data may differ and, in some cases, training should be performed from scratch. (iii) training a large, deep-learning model, requires high computational power resulting in a large carbon footprint.
The workflow of the proposed optical system consists of six steps:
The selected camera captures appropriate images. This procedure is performed through the following steps:
Image pre-processing involves three steps:
The authors propose an algorithm (Figure 2a) for detecting defects from selected vertical windows (Figure 2b) after the preprocessing step. The projections calculated in the pre-processing step are investigated using a custom sliding window search algorithm.
Signal processing consists of identifying peaks and valleys in the selected ROIs (Figure 3).
The coordinates of potential defects are obtained in the previous step. The acquired 2D images are classified based on the height and width measurements of each identified defect, which depends on the application’s requirements (Figure 4).
The obtained results from the previous steps are arrays of X and Y coordinates for each identified defect. These sets of coordinates are marked by visible circles inside the image, which can be communicated to the human operator using the Web Application UI.
The proposed optical system has been designed as a standalone application. Figure 5 shows the communication routes between the different layers.
The Presentation Layer includes the UI controller and the Web Application UI. The UI controller provides the image with marked defects in a graphical window before communicating it using the Web Application UI.
The Service Layer performs interface services including the communication traffic handler.
The Logic Layer contains four components: (i) Acquisition Module: to activate the camera and capture 2D images, (ii) Pre-processing Module: to filter/smooth the image and calculate the required values, (iii) Detection Algorithm, and (iv) Classification Algorithm: the two latter components include the core detection and classification algorithms for identifying and classifying defects on the product surface.
This layer groups/models the data used by the optical system. Data models are only used for the internal operations of the application. Thus, an internal reverse connection between the Logic Layer and the data models (Data Layers) has been established.
HDevelop13 and Pylon Viewer software applications were employed to calibrate the camera and the threshold values (Table 1). The proposed algorithms were implemented using the C# programming language inside Microsoft Visual Studio IDE. The optical system runs on a combination of C# and MATLAB scripts. The MVTec Halcon Library (“HALCON – Comprehensive Software for Machine Vision with an IDE) has been used to control the selected camera and process various data used for detection.
Table 1. Selected parameters
A user interface based on HTML5 principles has been deployed on top of a web application to display the results in devices like computers, tablets, and smartphones. Figure 6 shows the developed web UI.
To evaluate the proposed approach for a real-life application, detecting aesthetic defects on the surface of metal parts for the automotive industry is considered as a case study. A set of 25 monoblock aluminum brake calipers are placed inside the field of view of a camera sensor one by one. An industrial robot manipulator has been used for automatic transportation (Figure 7). The brake calipers were mounted on the end effector of the robot manipulator, based on the caliper’s body geometry. A Basler 2D area scan camera (Ag 2020) equipped with Kowa lens (“Kowa 1” LM35HC Lens” 2020) is used to support exposure and focus calibration, and to capture images with a resolution of 3000 × 1700 pixels.
The connection between the camera and the PC has been accomplished using a network switch device (“Linksys Business LGS308MP PoE+ Smart 8 Port Gigabit Network Switch (130W)” 2020). Figure 8 depicts an overview of the connectivity between different components.
Table 2 presents and compares the results of implementing the proposed approach on 25 monoblock aluminum brake calipers. Defects are identified with a success rate of 95.9% with an average execution time of 52 seconds. The total error was 8.7% including either failure to detect actual defects (12 out of 22) or points wrongly interpreted as defects (10 out of 22).
Table 2. Experimental results.
Figure 9 demonstrates the identified defects in monoblock aluminum brake calipers using the proposed optical system.
Images are grouped into three categories based on the size of their defects:: (A) OK, (B) MINOR DEFECTS, and (C) NOT OK. The classification algorithm categorizes results with a success rate of 92%. The mean error for the identified defects’ height and width was 1.48mm and 1.53mm, respectively (Figure 10).
Table 3 presents and compares the detection and classification accuracy with other approaches. Hot-rolled steel plates and steel billets have been selected since their morphological state is similar, even though the presented metal caliper parts are proven to have even more difficult surface conditions. The proposed model exhibits a better performance in detecting actual defects for real applications.
Table 3. Comparison of the performance of the proposed model with other works
The proposed approach improves the quality of inspection for finished products using a simple and cost-effective optical system under the scope of Industry 4.0. The proposed identification and classification algorithms have been successfully implemented in the automotive industry. The results prove that the optical system is fast and cost-effective, and exhibits high reliability for precisely identifying and classifying aesthetic defects without training a model. Additionally, the system is highly flexible and can be used to inspect the quality of different products from the same material.
Authors claim that the proposed approach can be applied in various industries and for different materials such as plastic, paper, food, etc. Furthermore, to achieve minimum execution time, the proposed approach should be implemented on a multithreaded basis. Finally, to automatically select suitable threshold values for various applications, more research should be conducted.
If you have any inquiries about how the AIEX platform can be used to train state-of-the-art models for intelligent defect detection in special cases, please do not hesitate to contact us.
You can enter your email address and subscribe to our newsletter and get the latest practical content. You can enter your email address and subscribe to our newsletter.