TW202411883A - Automated defect detection methods - Google Patents

Abstract

The automated defect detection method provided by the present invention is to perform defect detection on a production line with a production defect rate of less than 20%, and includes a setting procedure and a detection procedure. The setting procedure includes the following steps: 1. Photographing a plurality of first test objects one by one to obtain a plurality of first images each containing only a single first test object; 2. Extracting a first feature information from each first image; 3. Grouping the first feature information into two pieces, and performing a difference comparison between the two first feature information in the same group. When the comparison results in the same group are substantially the same, the first feature information in the group is marked as normal, and when the comparison results are substantially different, the first feature information in the group is marked as abnormal; 4. Integrating the first feature information marked as normal into a standard feature set. The detection procedure includes the following steps: 1. Obtaining a second image of a second object to be tested on the production line; 2. Extracting second feature information of the second image and comparing it with the standard feature set. When the comparison result is substantially the same, the second image and/or the second feature information is marked as normal; when the comparison result is substantially different, the second image and/or the second feature information is marked as abnormal.

Description

Automated defect detection method

The present invention relates to defect detection technology, and in particular to an automated defect detection method.

According to the common production mode of products or components, they can be divided into high variety and high quantity or low variety and high quantity. For example, capacitor rubber sealing products are common components in circuits, but in order to meet different needs, they have different sizes according to the capacity of the capacitor, and the specifications of the sealing rubber also need to change accordingly, which is a product of high variety and high quantity.

Since the sealing rubber is one of the key components to maintain the normal operation of the capacitor, it can also be used as a basis for judging whether the product has defects. As shown in Figure 1(a), it shows the front and back images of the sealing rubber after the rubber is hot-pressed.

Furthermore, several types of defects that may occur in the production process of sealing rubber are listed in Figure 1(b). In addition, subsequent statistical results show that there are 38 types of defects, including 21 surface defects on the front or back, 7 pinhole defects, and 10 bottom or punching defects. In this way, if 100 million capacitors are produced per month, it is no longer possible to inspect defects manually, and improvements are needed, such as using an automated optical inspection system (AOI) for inspection.

However, developing an automatic optical inspection system suitable for different manufacturing lines is indeed complex. For example, the basic steps for traditionally developing an automatic optical inspection system are as follows:

1. Confirm the workpiece type and defect type: workpiece type, inspection quantity and inspection speed, defect type and pattern, etc.

2. Design the camera lighting shooting environment: plan the camera resolution, light source type, working distance, etc. according to the defect requirements.

3. Design test methods: Design and plan various defect detection methods.

4. Develop a testing system.

5. Testing and system tuning.

6. Education, training and acceptance.

From the above, we can see that if we want to customize and develop an automatic optical inspection system to meet the above requirements, we may face the following problems:

1. Long development time and high labor cost: System development requires considerable time and corresponding labor costs to achieve detection goals.

2. Testing and system tuning: The system needs to undergo complex and complete testing and verification to ensure that the system operates as expected.

3. Lack of flexibility in the system: When there are new models or new workpieces, various parameters need to be adjusted and the software program needs to be modified appropriately when necessary to avoid the lack of flexibility of the system and the inability to adapt to different workpiece styles.

In addition to the aforementioned traditional automatic optical inspection system to detect defects, in recent years, technologies such as artificial intelligence, machine learning, deep learning or neural networks have been developed and applied to image recognition, such as the following patent cases.

Taiwan's invention patent No. I667575, "Defect Detection System and Method Using Artificial Intelligence," is mainly about introducing artificial intelligence into industrial processes, such as automatic sampling, artificial intelligence image labeling, model training, classification test verification, etc., to reduce the burden of manpower.

Taiwan's invention patent No. I731565, "Integrated system for rapid defect detection of sheet materials and its use method," is aimed at detecting continuous rolls of materials (such as cloth) and uses artificial intelligence technology to detect various defects, such as missing edges, hooks, embossing marks, holes, etc.

Taiwan's invention patent No. I749714, "Defect Detection Method, Defect Classification Method and System thereof," uses machine learning to train models for each area of a good product in an image. Then, after taking one or more images of a good product, defect detection can be performed on each area. The detection threshold is also set for each area based on its position, area, color, brightness and shape to achieve defect detection.

However, all of these previous cases utilize artificial intelligence technology for identification, and a large number of samples must be collected for learning and training before testing can be performed. In particular, each new product requires a long period of time to collect a large number of positive or negative samples for learning, and usually requires manual image labeling.

Since artificial intelligence technology is mainly used for detection and classification, there are still the following problems in practical application:

1. It is impossible to find details or tiny defects in workpieces: it is difficult to identify whether by detection or classification.

2. It is difficult to collect images of defective workpieces: If the defect rate of the production line to be inspected is too low, it is time-consuming to collect a certain number of defective images.

3. Model verification takes a long time: The model goes through a cycle of image collection, training, and testing, which takes a long time.

4. Individual introduction is required when applied to different industries: Although the machine learning model is universal, different models should be trained and introduced.

5. Long recognition time: It may be several or even dozens of times longer than the detection time of traditional automatic optical inspection systems.

The main purpose of the present invention is to provide an automated defect detection method, which is aimed at components or workpieces manufactured in large batches. When using image recognition for defect detection, these repeated workpiece patterns can be cross-matched for defect detection. Compared with the customization of traditional automated optical inspection systems and the use of artificial intelligence that requires a large amount of data for calculation, the present invention eliminates the need for customization and the collection of a large number of samples, greatly shortens the introduction process and reduces costs, so as to achieve a faster and more effective defect detection method.

Therefore, in order to achieve the above-mentioned purpose, the automated defect detection method provided by the present invention is to perform defect detection on a production line with a production defect rate of less than 20%, and includes a setting procedure and a detection procedure, wherein:

The setup process includes the following steps:

Photographing a plurality of first objects to be tested one by one to obtain a plurality of first images each including only a single first object to be tested;

Extracting first feature information from each of the first images;

The first feature information is grouped into two pieces, and the two first feature information in the same group are respectively compared for differences. When the comparison result in the same group is substantially the same, the first feature information in the group is marked as normal, and when the comparison result is substantially different, the first feature information in the group is marked as abnormal;

Integrate the first feature information marked as normal into a standard feature set;

The testing procedure includes the following steps:

Acquire a second image of a second object to be tested on the production line;

The second feature information of the second image is extracted and compared with the standard feature set. When the comparison result is substantially the same, the second image and/or the second feature information is marked as normal; when the comparison result is substantially different, the second image and/or the second feature information is marked as abnormal.

Accordingly, the present invention compares two first feature information in the first test image belonging to the same group in a similarity-seeking-differences manner, thereby improving the shortcomings of learning customized design and collecting a large number of samples, thereby quickly obtaining the standard feature set as a reference for defect detection.

Furthermore, the production defect rate of the production line is preferably less than 1%, and more preferably less than 0.5%.

In one embodiment, the first characteristic information includes the outline, size, and color of the first object to be measured.

In one embodiment, the automatic defect detection method of the present invention further determines whether a first similarity between the first feature information in the same group falls within a preset first threshold interval, wherein when the first similarity does not fall within the first threshold interval, the first feature information of the group is marked as normal; when the first similarity falls within the first threshold interval, the first feature information of the group is marked as abnormal.

In one embodiment, the second feature information includes the outline, size, and color of the second object to be measured.

In one embodiment, the automatic defect detection method of the present invention further determines whether a second similarity between the second feature information and the standard feature set falls within a preset second threshold interval. When the second similarity does not fall within the second threshold interval, the second image and/or the second feature information is marked as normal; when the second similarity falls within the second threshold interval, the second image and/or the second feature information is marked as abnormal.

In one embodiment, in the setting procedure, background removal, binarization and/or sharpening processing are performed on each of the first images to obtain the first image after image pre-processing.

In one embodiment, in the setting procedure, object detection is further performed on the first image after image pre-processing to obtain the position, shape or size of the first object to be detected in the first image.

In one embodiment, in the setting procedure, the position of the first object to be detected in the first image is located according to the object detection result.

In one embodiment, in the detection procedure, object detection is performed on the second image to obtain the position, shape or size of the second object to be detected in the second image.

In one embodiment, in the detection procedure, the position of the second object to be detected in the second image is located according to the object detection result.

First of all, it is to be noted that the object to be tested described in the following preferred embodiment of the present invention is a workpiece on a production line, specifically a capacitor rubber sealing product. However, the part of the defect detection technology that does not hinder the disclosure of the technical features of the present invention will not be described in the following description. However, such omitted parts are the common knowledge in the technical field to which the present invention belongs, and the omission does not affect the completeness of the disclosure of the main technical features of the present invention.

The "shooting" referred to in the present invention refers to the use of an image capture module, such as a CCD grayscale/color camera, a line scan camera, etc., to capture an actual image of an object to be tested. In addition, during the operation of the image capture module, factors such as external environmental conditions and the external structure of the object to be tested will affect the light receiving area and reflection angle from the light source to the surface of the object to be tested, which may make the image unclear. Therefore, in order to obtain a clearer image, a light source can be used to supplement the light of the object to be tested with forward light, backlight, or coaxial light to improve the contrast of the image to be tested.

The "image pre-processing" referred to in the present invention refers to analyzing, processing and treating an image to obtain more and more useful information from the processed image. Among them, common image pre-processing includes background removal, binarization or sharpening technology.

The "object detection" referred to in the present invention refers to the simultaneous identification of the position and size of an object.

The "Region of Interest" (ROI) referred to in the present invention refers to the area to be processed in the image, which is outlined in the form of a box, circle, ellipse, irregular polygon, etc. through the result of the aforementioned object detection. The "normal" referred to in the present invention can also be understood as "defect-free", which means that the image of the object to be tested does not have any defects.

The term "abnormal" as used in the present invention refers to a defect in the image of the object under test being compared.

Please refer to FIG. 2 , which shows an automated defect detection method provided in a preferred embodiment of the present invention, which performs defect detection on a production line with a production defect rate of less than 20%, preferably less than 1% and more preferably less than 0.5%. The method includes a setup procedure and a detection procedure.

The setting procedure includes the following steps:

Step S101: Obtain the image to be tested

A plurality of objects to be tested are photographed one by one continuously to obtain a plurality of first images each containing only a single object to be tested.

Step S102: Image pre-processing

Perform image pre-processing on each of the first images respectively.

Step S103: Object detection

Object detection is performed on each of the first images processed in step S102. For example, as long as there is a black and white outline at the edge of the object to be detected in each of the pre-processed images, the object to be detected can be detected, and the position, shape or size of the first object to be detected in the first image can be obtained.

Step S104: Positioning calibration

Since the position of the object to be measured may deviate during shooting, the position of the first object to be measured in the first image is located according to the object detection result of step S103, and a first region of interest for the first object to be measured is outlined in the first image.

Step S105: Feature extraction

A first feature information is extracted from each of the first regions of interest in step S104, wherein the first feature information is the outline, size or color of the first object to be measured.

Step S106: Detection and comparison

The first feature information is grouped into two pieces, and the two first feature information in the same group are compared with each other. When the comparison results in the same group are substantially the same, the first feature information in the group is marked as normal, as shown in Figure 3(a); when the comparison results are substantially different, the first feature information in the group is marked as abnormal, as shown in Figure 3(b).

Specifically, in this example, the aforementioned comparison method is to determine whether a first similarity of the first object to be tested between the first feature information in the same group falls within a preset first threshold interval, and the first similarity is contour similarity, size similarity or color similarity, wherein, when the first similarity does not fall within the first threshold interval, the first feature information of the group is marked as normal, and represents that the first objects to be tested contained in the first images of the group are normal; when the first similarity falls within the first threshold interval, the first feature information of the group is marked as abnormal, and represents that one or both of the first objects to be tested contained in the first images of the group are abnormal.

Generally speaking, when the first similarity is either contour similarity or size similarity, the range of the first threshold interval is between 0 and 1, but due to the comparison based on different first feature information, there will be different corresponding first threshold interval ranges, for example, the first threshold interval range is between 0.5 and 1, the first threshold interval range is between 0 and 0.8, or the first threshold interval range is between 0.2 and 0.8.

In addition, when the first feature information is color (i.e., color RGB), correspondingly, the first similarity is color similarity, and the range of the first threshold interval is between 0 and 255. Furthermore, different ranges of the first threshold interval can be set according to actual needs, for example, the first threshold interval range is between 83 and 255, or the first threshold interval range is between 50 and 160.

Then, the comparison results are stored in a database.

Step S107: Standard feature set

Continuing to step S106, the first feature information of all first images marked as normal is collected and integrated to obtain a standard feature set, which is used as a detection reference in the subsequent detection process.

The test procedure has the following steps:

Step S201: Obtain a second image of a second object to be tested on the production line. Furthermore, the second image may be subjected to image pre-processing before proceeding to the next step.

Step S202: Perform object detection on the second image to obtain the position, shape or size of the second object to be detected in the second image.

Step S203: According to the object detection result of step S202 and in combination with the standard region of interest of the standard feature set, the second object to be detected is located in the second image, and a second region of interest for the second object to be detected is outlined in the second image.

Step S204: extracting second feature information from the second region of interest in the second image, wherein the second feature information is the outline, size or color of the second object to be measured.

Step S205: Compare the second feature information with the standard feature set. When the comparison result is substantially the same, mark the second image and/or the second feature information as normal; when the comparison result is substantially different, mark the second image and/or the second feature information as abnormal.

Specifically, in this example, the aforementioned comparison method is to determine whether a second similarity between the second feature information and the standard feature set falls within a preset second threshold interval, and the second similarity is contour similarity, size similarity or color similarity, wherein, when the second similarity does not fall within the second threshold interval, the second image and/or the second feature information is normal; when the second similarity falls within the second threshold interval, the second image and/or the second feature information is marked as abnormal.

Generally speaking, when the second similarity is either contour similarity or size similarity, the range of the second threshold interval is between 0 and 1, but due to the comparison based on different second feature information, there will be different corresponding second threshold interval ranges, for example, the second threshold interval range is between 0.5 and 1, the second threshold interval range is between 0 and 0.8, or the second threshold interval range is between 0.2 and 0.8.

In addition, when the second feature information refers to a color (e.g., color RGB), the second similarity is a color similarity, and the range of the second threshold interval is between 0 and 255. Furthermore, different ranges of the second threshold interval can be set according to actual needs, for example, the second threshold interval range is between 83 and 255, or the second threshold interval range is between 50 and 160.

Step S206: Finally, the comparison result is stored in the database, and the first feature information of all first images marked as normal is collected and integrated to obtain a standard feature set.

By using the above technology, the present invention is able to automatically detect and lock the objects to be tested, and to group any two objects to be tested and compare their images to achieve the purpose of automated defect detection.

Furthermore, the present invention can be directly applied to existing equipment, for example, applied to the original camera image acquisition communication interface, or packaged into a library for software developers to integrate.

In addition, the present invention eliminates the need for customization of traditional automated optical inspection systems during the inspection process, and does not use artificial intelligence technology for calculation or identification. It also avoids lengthy operations such as collecting, sorting, and calculating a large amount of data, and there is no need to classify different defects, thereby achieving the goal of shortening the introduction schedule and reducing costs.

As shown in FIG. 4 , the second embodiment of the present invention further provides an automated defect detection system, which includes a light source 10, an image capture module 20, a calculation module 30, and a database 40, wherein the image capture module 20 may be, but is not limited to, a CCD grayscale/color camera or a line scan camera, for photographing the first objects to be tested and the second objects to be tested, respectively.

The light source 10 may be, but is not limited to, an incandescent lamp, a low-pressure gas discharge lamp, a low-pressure sodium lamp, a cold cathode lamp (CCFL), a HID lamp (High Intensity Discharge Lamp), or an LED lamp, and is used to supplement the light for the first object to be tested or the second object to be tested in order to solve the problem of insufficient ambient light.

The computing module 30 may be, but is not limited to, a central processing unit (CPU), or other programmable general-purpose or special-purpose microprocessors, digital signal processors (DSP), programmable controllers, application-specific integrated circuits (ASIC), or other similar components or a combination of the above components, and the computing module 30 is respectively connected to the light source 10, the image acquisition module 20, and the database 30 to execute the aforementioned steps of the setting procedure and part of the detection procedure, such as image pre-processing, object detection, positioning correction, feature extraction, detection comparison, etc., so as to quickly detect whether the second object to be tested has defects.

The specific storage medium of the database 40 can be phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), flash memory disk, read-only memory (ROM), random access memory (RAM), magnetic disk or optical disk, etc., for storing the aforementioned data or the calculation results of the calculation module 30.

In summary, although the present invention has been disclosed as above by way of embodiments, it is not intended to limit the present invention. Those with ordinary knowledge in the technical field to which the present invention belongs can make various changes and modifications without departing from the spirit and scope of the present invention. Therefore, the scope of protection of the present invention shall be subject to the scope defined in the attached patent application.

10: Light source

20: Image capture module

30: Computational Module

40: Database

S101, S102, S103, S104, S105, S106, S107, S201, S202, S203, S204, S205: Steps

Figure 1(a) shows the front and back images of a traditional sealing rubber.

Figure 1(b) is a schematic diagram showing the types of defects in traditional sealing rubber.

FIG. 2 is a flow chart of a first embodiment of the present invention.

FIG. 3( a ) and FIG. 3( b ) are schematic diagrams of detecting whether the first object to be tested is normal or abnormal according to a first embodiment of the present invention.

FIG4 is a system block diagram of a second embodiment of the present invention.

S101, S102, S103, S104, S105, S106, S107, S201, S202, S203, S204, S205: Steps

Claims (11)

An automated defect detection method is to perform defect detection on a production line with a production defect rate of less than 20%, and includes a setting procedure and a detection procedure; The setting procedure includes the following steps: Photographing a plurality of first test objects one by one to obtain a plurality of first images each containing only a single first test object; Extracting a first feature information from each first image; Grouping the first feature information into two pieces, and performing a difference comparison between two first feature information in the same group. When the comparison results in the same group are substantially the same, the first feature information in the group is marked as normal, and when the comparison results are substantially different, the first feature information in the group is marked as abnormal; Integrating the first feature information marked as normal into a standard feature set; The detection procedure includes the following steps: Obtain a second image of a second object to be tested on the production line; Extract the second feature information of the second image and compare it with the standard feature set. When the comparison result is substantially the same, the second image and/or the second feature information is marked as normal; when the comparison result is substantially different, the second image and/or the second feature information is marked as abnormal. An automated defect detection method as described in claim 1, wherein the production defect rate of the production line is preferably less than 1%, and more preferably less than 0.5%. An automated defect detection method as described in claim 1, wherein the first feature information includes the outline, size, and color of the first object to be detected. An automated defect detection method as described in claim 3, wherein it is further determined whether a first similarity between the first feature information in the same group falls within a preset first threshold interval, and the first similarity is contour similarity, size similarity or color similarity, wherein when the first similarity does not fall within the first threshold interval, the first feature information of the group is marked as normal; when the first similarity falls within the first threshold interval, the first feature information of the group is marked as abnormal. An automated defect detection method as described in claim 1, wherein the second feature information includes the contour, size, and color of the second object to be detected. An automated defect detection method as described in claim 5, wherein it is further determined whether a second similarity between the second feature information and the standard feature set falls within a preset second threshold interval; when the second similarity does not fall within the second threshold interval, the second image and/or the second feature information is marked as normal; when the second similarity falls within the second threshold interval, the second image and/or the second feature information is marked as abnormal. As described in claim 1, the automated defect detection method, wherein, in the setting procedure, background removal, binarization and/or sharpening are performed on each of the first images to obtain a first image that has undergone image pre-processing. The automated defect detection method as described in claim 7, wherein, in the setup procedure, object detection is further performed on the first image after image pre-processing to obtain the position, shape or size of the first object to be detected in the first image. An automated defect detection method as described in claim 8, wherein, in the setting procedure, the position of the first object to be detected in the first image is located based on the object detection result. The automated defect detection method as described in claim 1, wherein, in the detection procedure, object detection is performed on the second image to obtain the position, shape or size of the second object to be detected in the second image. An automated defect detection method as described in claim 10, wherein, in the detection procedure, the position of the second object to be detected in the second image is located based on the object detection result and in combination with the standard feature set.

Images