Quality control automation using computer vision: A case study of bottle caps

Workflow

The authors used the OpenCV library in python as the line detection and clustering algorithms. The TensorFlow library is employed for object detection and convolutional neural networks in Keras Library are applied for the pattern recognition.

Clustering

K-Means clustering is an algorithm to split the bottle image into many clusters. The cap can be identified by an unsupervised learning algorithm in a distinct cluster. This algorithm works as follows:

1) Reading the image.

2) Applying K-Means clustering to split the image into clusters. For liquids with a similar color to that of the cap, aggressive clustering (i.e., K>20) should be applied.

3) Clustering segments of images with a similar bottle cap color.

4) Converting image into grayscale and defining contour lines to find the most connected clusters. In the case of a well-packed bottle, this cluster includes the ring below the cap.

5) Any displacement of the cap fitting is recognized by the contour line shape and the bounding box.

6) The well-packed cap exhibits a higher area than that of a misfitted cap (Figure 3).

Line and object detection

The authors use a supervised learning algorithm based on a neural network for the segmentation of the bottle cap image. A single shot multibox detector (SSD) is employed for the cap detection. The feature maps can be extracted using the SSD before detecting the bottle cap by convolutional filters. This algorithm provides better performance compared to other object detection algorithms, the authors claimed.

The line detection module can examine whether the cap is correctly fitted or not. To achieve that, the following steps should be followed:

1. Receiving the bottle cap image from the object detection module.
2. Applying the Gaussian blur filter to the image.
3. Applying canny edge detection to the image.
4. Employing the probabilistic Hough line detection.
5. Discarding all lines with a slope larger than 0.05 or less than -0.05.
6. Applying natural partitioning to adjust group boundaries.
7. The line with a y-coordinate not included in the defined group indicates that the bottle cap is misfitted and must be discarded from the production line.

Pattern recognition

As mentioned above, a CNN algorithm is used by authors for pattern recognition in both correctly and incorrectly packaged bottles. Different filters are applied to the labeled data in order to extract features. An adaptive thresholding algorithm is employed to identify the rings, which makes the algorithm insensitive to the color differences and helps it focus on the differences in the bottle cap position.