8 Mins Read

Intelligent Defect Detection in Metal Parts using Optical System for Industry 4.0 Manufacturing


Intelligent Defect Detection in Metal Parts using Optical System for  Industry 4.0 Manufacturing

Converting conventional quality control (QC) units into intelligent and automated detection systems is a high priority for manufacturing professionals. Intelligent QC systems reduce cost and time in most production lines in various stages of production and significantly improve the quality of the final products. Traditionally, defects are detected by either human operators or computer models trained by time-consuming  machine learning (ML) techniques.

Papavasileiou et al. from the department of mechanical engineering and aeronautics at the university of Patras in Greece devised an automated optical system, which can detect and classify aesthetic defects in different sizes, shapes, and positions immediately after the casting process. Employing this approach eliminates the need for training and minimizes the required resources,  resulting in a cost-effective solution with high flexibility in terms of introducing new parts/products of the same material in different stages of production(Figure 1). The authors published their results in the “international journal of computer integrated manufacturing”, 2022.

Optical System
Figure 1. Various automotive parts can be controlled for defects using the proposed approach. Source

Computer vision (CV)-based solutions in-line with the industry 4.0 paradigm will be immensely helpful for the existing industrial processes and techniques. However, numerous efforts by engineers and researchers Have failed to meet the manufacturing requirements until now, particularly in the field of intelligent visual inspection systems. Defect detection immediately after the casting process involves three main approaches:

  1. Visual inspection by human operators: Although this is the most common technique, it is highly prone to human error.
  2. Non-destructive (ND) testing techniques: Ultrasonic inspection and radiographic examination are two accurate ND testing techniques.  Their downside is the high price of the equipment required to implement them (e.g. laser scans) is quite high.
  3. CV techniques and algorithms: These approaches consist of machine learning techniques, signal-processing methods, etc. Among them, CV techniques based on deep learning algorithms for identifying and classifying defects have garnered much attention since they offer excellent accuracy, low-cost operation, and a high level of automation.

Steel industries, for example, have used CV models to detect potential cracks on steel billet surfaces, monitor the surface of hot-rolled flat steel sheets, and inspect the trimmed edges of steel billets. Some image processing techniques such as Top-Hat filter and Hough transform have also been employed for defect detection in steel industries. In other industries, the brightness, contrast, or contours of images have been used to identify defects. Nevertheless, none of these models exhibited the required flexibility as they focus strictly on specific parts or processes and fail to generalize the solution.

Recently, deep learning approaches have been used to detect defects in hot-rolled steel. Surface defects have also been classified using Scale Invariant Feature Transform, and the use of Convolutional Neural Networks (CNN) in detecting welding defects has been investigated. A pre-trained AlexNet architecture has been utilized to classify black lines and wrinkle defects in leather images. Deep CNN algorithms based on TensorFlow have been applied to detect defects in wooden parts, the surface of solar cells, tungsten inert gas welding, steel wire ropes, cylindrical roller bearing components, etc.

Although the results of such studies are promising, some critical shortcomings still exist that limit their usage in real manufacturing settings. These issues include (i) the trained model’s accuracy for identifying and classifying aesthetic defects highly depends on the quality of training data and the labeling procedure. (ii) the amount of required data may differ and, in some cases, training should be performed from scratch. (iii) training a large, deep-learning model, requires high computational power resulting in a large carbon footprint.

The Workflow for Intelligently Detecting Defects in Metal Parts

The workflow of the proposed optical system consists of six steps:

Image acquisition

The selected camera captures appropriate images. This procedure is performed through the following steps:

  1. Establishing a connection with the camera
  2. Setting up the parameters of the data acquisition device, including a) width, b) height, c) offset in X-axis, d) offset in Y-axis, and e) exposure value
  3. Obtaining the image
  4. Storing the captured image

Image Pre-processing

Image pre-processing involves three steps:

  1. Smoothing the image using Gaussian filters to remove excessive detail and noise
  2. Creating regions of interest (ROI)
  3. Calculating projections

Defects Detection

The authors propose an algorithm (Figure 2a) for detecting defects from selected vertical windows (Figure 2b) after the preprocessing step. The projections calculated in the pre-processing step are investigated using a custom sliding window search algorithm.

Optical System
Figure 2. (a) the proposed algorithm for defect detection, and (b) selected vertical windows. Source

Signal processing consists of identifying peaks and valleys in the selected ROIs (Figure 3).

Optical System
Figure 3. Projection values with a potential defect circled in red. Source

Image Classification

The coordinates of potential defects are obtained in the previous step. The acquired 2D images are classified based on the height and width measurements of each identified defect, which depends on the application’s requirements (Figure 4).

Optical System
Figure 4. Classification algorithm. Source

Visualizing Results

The obtained results from the previous steps are arrays of X and Y coordinates for each identified defect. These sets of coordinates are marked by visible circles inside the image, which can be communicated to the human operator using the Web Application UI.

System Implementation Architecture

The proposed optical system has been designed as a standalone application. Figure 5 shows the communication routes between the different layers.

Intelligent Defect Detection
Figure 5. Optical system architecture. Source

Presentation Layer

The Presentation Layer includes the UI controller and the Web Application UI. The UI controller provides the image with marked defects in a graphical window before communicating it using the Web Application UI.

Service Layer

The Service Layer performs interface services including the communication traffic handler.

Logic Layer

The Logic Layer contains four components: (i) Acquisition Module: to activate the camera and capture 2D images, (ii) Pre-processing Module: to filter/smooth the image and calculate the required values, (iii) Detection Algorithm, and (iv) Classification Algorithm: the two latter components include the core detection and classification algorithms for identifying and classifying defects on the product surface.

Data Layer

This layer groups/models the data used by the optical system. Data models are only used for the internal operations of the application. Thus, an internal reverse connection between the Logic Layer and the data models (Data Layers) has been established.

Software Implementation

HDevelop13 and Pylon Viewer software applications were employed to calibrate the camera and the threshold values (Table 1). The proposed algorithms were implemented using the C# programming language inside Microsoft Visual Studio IDE. The optical system runs on a combination of C# and MATLAB scripts. The MVTec Halcon Library (“HALCON – Comprehensive Software for Machine Vision with an IDE) has been used to control the selected camera and process various data used for detection.

Table 1. Selected parameters
Optical System

A user interface based on HTML5 principles has been deployed on top of a web application to display the results in devices like computers, tablets, and smartphones. Figure 6 shows the developed web UI.

Optical System
Figure 6. Web application UI. Source

Employing the Proposed Approach in the Automotive Industry

To evaluate the proposed approach for a real-life application, detecting aesthetic defects on the surface of metal parts for the automotive industry is considered as a case study. A set of 25 monoblock aluminum brake calipers are placed inside the field of view of a camera sensor one by one. An industrial robot manipulator has been used for automatic transportation (Figure 7). The brake calipers were mounted on the end effector of the robot manipulator, based on the caliper’s body geometry. A Basler 2D area scan camera (Ag 2020) equipped with Kowa lens (“Kowa 1” LM35HC Lens” 2020) is used to support exposure and focus calibration, and to capture images with a resolution of 3000 × 1700 pixels.

Optical System
Figure 7. Experiment setup for implementing the automatic defect detection. Source

The connection between the camera and the PC has been accomplished using a network switch device (“Linksys Business LGS308MP PoE+ Smart 8 Port Gigabit Network Switch (130W)” 2020). Figure 8 depicts an overview of the connectivity between different components.

Optical System
Figure 8. Connectivity overview. Source

Comparing Results with Other Works

Table 2 presents and compares the results of implementing the proposed approach on 25 monoblock aluminum brake calipers. Defects are identified with a success rate of 95.9% with an average execution time of 52 seconds. The total error was 8.7% including either failure to detect actual defects (12 out of 22) or points wrongly interpreted as defects (10 out of 22).

Table 2. Experimental results.
Optical System

Figure 9 demonstrates the identified defects in monoblock aluminum brake calipers using the proposed optical system.

Optical System
Figure 9. Identified defects from the optical system. Source

Images are grouped into three categories based on the size of their defects:: (A) OK, (B) MINOR DEFECTS, and (C) NOT OK. The classification algorithm categorizes results with a success rate of 92%. The mean error for the identified defects’ height and width was 1.48mm and 1.53mm, respectively (Figure 10).

Optical System
Figure 10. Classification accuracy for caliper images based on defects’ dimensions. Source

Table 3 presents and compares the detection and classification accuracy with other approaches. Hot-rolled steel plates and steel billets have been selected since their morphological state is similar, even though the presented metal caliper parts are proven to have even more difficult surface conditions. The proposed model exhibits a better performance in detecting actual defects for real applications.

Table 3. Comparison of the performance of the proposed model with other works
Optical System

Conclusions and future work

The proposed approach improves the quality of inspection for finished products using a simple and cost-effective optical system under the scope of Industry 4.0. The proposed identification and classification algorithms have been successfully implemented in the automotive industry. The results prove that the optical system is fast and cost-effective, and exhibits high reliability for precisely identifying and classifying aesthetic defects without training a model. Additionally, the system is highly flexible and can be used to inspect the quality of different products from the same material.

Authors claim that the proposed approach can be applied in various industries and for different materials such as plastic, paper, food, etc. Furthermore, to achieve minimum execution time, the proposed approach should be implemented on a multithreaded basis. Finally, to automatically select suitable threshold values for various applications, more research should be conducted.

If you have any inquiries about how the AIEX platform can be used to train state-of-the-art models for intelligent defect detection in special cases, please do not hesitate to contact us.

Related articles
Computer vision is a critical component of self-driving cars, a hot topic in recent years. We examine this topic...
Smart farming and artificial intelligence
The fourth agricultural revolution is already under way with the adoption of smart farm technology such as artificial intelligence,...
waste management
This article discusses the integration of artificial intelligence in the field of recycling. We will also train a model...
traffic management
As urban areas continue to grow, the number of vehicles on the road is increasing, which leads to congested...
Thanks to advancements in artificial intelligence, particularly in computer vision, deep learning models can now be utilized in various...
Trauma Detection on Pelvic Radiographs using Computer Vision Algorithms
A timely and accurate diagnosis of the proximal femur and pelvis injuries in trauma patients is essential to saving...
Subscribe to our newsletter and get the latest practical content.

You can enter your email address and subscribe to our newsletter and get the latest practical content. You can enter your email address and subscribe to our newsletter.