JP6834126B2 - Information processing equipment, defect detection methods and programs - Google Patents

Description

The present invention relates to information processing devices, defect detection methods and programs.

In recent years, defect detection technology using images has been generally used mainly for applications in FA (Factory Automation). In such a defect detection technology, even when the number of sample images that can be used as prior knowledge is small or there is a disturbance such as an illuminance change, it is possible to perform defect detection with high accuracy and to deal with an unknown defect mode. It has become a big issue. Here, as a defect detection algorithm that is resistant to disturbance and can deal with unknown defect modes, we used semi-supervised anomaly detection that detects images that deviate from the normal image by learning only normal images by machine learning. The technology has been known for some time.

Further, for example, as shown in Patent Document 1, by performing binarization processing on the image information of the inspection target object, acquiring the region information of the defect candidate, and identifying whether or not the defect is a defect from the characteristics of the region. , A technique for detecting defects is known.

However, in the conventional technology, the number of normal samples, including disturbances, needs to be a certain number or more. For example, when only a few normal samples can be prepared, defect detection can be performed with good performance in a situation with disturbances. There is the problem that it is difficult to do.

Further, since the technique of Patent Document 1 uses simple binarization, erroneous detection may occur due to a change in the lighting state or the like. Further, even the technique of Patent Document 1 is vulnerable to disturbance, and it is difficult to detect defects with good performance under conditions with disturbance.

The present invention has been made in view of the above, and is an information processing apparatus capable of performing defect detection with high accuracy even when the number of sample images is small, when there is a disturbance, or even in a new defect mode. , The main purpose is to provide defect detection methods and programs.

In order to solve the above-mentioned problems and achieve the object, the present invention is an information processing apparatus that detects defects in the appearance of an object, and uses unsupervised learning from an image of a detection target in which the object is captured. The object is classified into clusters, and the cluster containing the object is extracted from the classified cluster group based on the value based on the original pixel of the object, and the area of the cluster is set as the seed area. A classification unit that extracts a normal part from the captured image to be detected, a defect detection unit that detects a defect from the image to be detected based on the extracted normal part and a predetermined normal template image, and a defect detection unit. Equipped with.

According to the present invention, there is an effect that defect detection can be performed with high accuracy even when the number of sample images is small, when there is disturbance, or even in the case of a new defect mode.

FIG. 1 is a configuration diagram of an example of an information processing system according to the present embodiment. FIG. 2 is a hardware configuration diagram of an example of the information processing device according to the present embodiment. FIG. 3 is a functional block diagram of an example of the information processing device according to the present embodiment. FIG. 4 is a flowchart showing an example of the procedure of the defect detection process according to the present embodiment. FIG. 5 is a diagram showing an example of an image input in the present embodiment. FIG. 6 is a diagram showing an example of an image of the seed region in the present embodiment. FIG. 7 is a diagram showing an example of an image extracted as a component shape in the present embodiment. FIG. 8 is a diagram showing an example of a template image in the present embodiment. FIG. 9 is a diagram showing an example of the detection result in the present embodiment.

Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings.

<System configuration>
First, the system configuration of the information processing system 1 according to the first embodiment will be described. FIG. 1 is a configuration diagram of an example of an information processing system according to the first embodiment.

As shown in FIG. 1, the information processing system 1 includes an information processing device 10 and an image pickup device 30. Further, the information processing device 10 and the image pickup device 30 are communicably connected via, for example, a USB (Universal Social Bus) cable or the like. The information processing device 10 and the image pickup device 30 may be communicably connected via a LAN (Local Area Network) or the like.

The information processing device 10 is, for example, a PC (personal computer) or the like. The defect detection program 20 is installed in the information processing device 10. By using the defect detection program 20, the information processing device 10 can detect defects in the appearance of the subject in the image data input by taking an image with the image pickup device 30. In the present embodiment, the subject imaged by the image pickup apparatus 30 is assumed to be, for example, a product mass-produced in a factory, and when there is a defect in the appearance, there is a scratch or the like on the appearance (surface) of the product or the like. Suppose there is a case.

Here, it is assumed that the defect detection program 20 detects defects in the appearance of the subject of the image data by abnormality detection using unsupervised learning. Here, unsupervised learning is one of machine learning, and is machine learning that uses only input data and does not require teacher data.

The information processing device 10 is not limited to a PC, and may be a mobile terminal device such as a smartphone or a tablet terminal.

The image pickup device 30 is a camera or the like that captures a subject and generates image data. The image data generated by the image pickup apparatus 30 is input to the information processing apparatus 10 via, for example, a USB cable or the like. The image pickup device 30 may be built in the information processing device 10.

<Hardware configuration>
Next, the hardware configuration of the information processing apparatus 10 according to the first embodiment will be described. FIG. 2 is a hardware configuration diagram of an example of the information processing device according to the first embodiment. As shown in FIG. 2, the information processing device 10 includes a CPU (Central Processing Unit) 11, an HDD (Hard Disk Drive) 12, a RAM (Random Access Memory) 13, and a ROM (Read Only Memory) 14. There is. Further, the information processing device 10 includes an input device 15, a display device 16, an external I / F 17, and a communication I / F 18. Each of these parts is connected by bus B.

The CPU 11 is an arithmetic unit that realizes control and functions of the entire information processing device 10 by reading a program or data from a storage device such as the ROM 14 or the HDD 12 onto the RAM 13 and executing processing.

The HDD 12 is a non-volatile storage device that stores programs and data. The stored programs and data include, for example, an OS (Operating System) which is basic software for controlling the entire information processing apparatus 10, application software (for example, a defect detection program 20) which provides various functions on the OS, and the like. .. The HDD 12 manages the stored programs and data by a predetermined file system and / or DB (database). The information processing device 10 may include an SSD (Solid State Drive) or the like instead of the HDD 12 or in combination with the HDD 12.

The RAM 13 is a volatile semiconductor memory (storage device) that temporarily holds programs and data. The ROM 14 is a non-volatile semiconductor memory (storage device) capable of holding programs and data even when the power is turned off.

The input device 15 is a device used by the user to input various operation signals. The input device 15 is, for example, various operation buttons, a touch panel, a keyboard, a mouse, and the like. The display device 16 is a device that displays the processing result of the information processing device 10. The display device 16 is, for example, a display or the like.

The external I / F 17 is an interface with an external device. The external device includes, for example, a USB (Universal Serial Bus) memory, an SD card, a CD, a DVD, and the like. The communication I / F 18 is an interface for performing data communication with the image pickup apparatus 30.

The information processing apparatus 10 according to the present embodiment can realize various processes described later by having the above hardware configuration.

<Functional configuration>
Next, the functional configuration of the information processing apparatus 10 according to the first embodiment will be described. FIG. 3 is a functional block diagram of an example of the information processing device according to the first embodiment. As shown in FIG. 3, the information processing apparatus 10 mainly includes a component image generation unit 101, a classification unit 102, a defect detection unit 104, and a template image 110. The component image generation unit 101, the classification unit 102, and the defect detection unit 104 are realized by, for example, the CPU 11 executing the defect detection program 20.

The template image 110 is an image (template) that exists for each component constituting the object and that shows a normal shape that is predetermined for each component. The template image 110 is stored in the HDD 12.

The component image generation unit 101 performs template matching with the template image of the image of the object captured by the imaging device 30, and cuts out one or a plurality of component images of the object from the captured image and generates the image. Here, when matching with the template image 110, the component image generation unit 101 inserts (complements) the template image for the portion outside the display screen when a part of the component is outside the display screen. Perform matching. As a result, even if a part of the component is cut off the screen, the defect detection unit 104 does not determine the defect.

The classification unit 102 classifies the object into clusters by using unsupervised learning from the captured image of the object, and extracts the cluster containing the object from the classified cluster group based on the value based on the original pixel of the object. Then, by clustering in which the cluster area is set as the seed area, the normal part is extracted from the image of the detection target in which the object is captured. Specifically, the classification unit 102 uses a k-means algorithm as a clustering algorithm for unsupervised learning to obtain two brightness histograms of a central portion of an image of one or more parts of an object as an image to be detected. Classify into clusters, and classify each pixel constituting each of one or more component images into clusters. Here, the classification unit 102 sets a cluster of pixels that approximate the brightness value of the original component among the pixels classified into clusters as a seed region. Here, as the pixel that approximates the brightness value of the original component, the pixel has a brightness value whose difference from the brightness value of the original component is within a predetermined value. Further, the classification unit 102 extracts the component shape from the component image by the Chan-Vese algorithm as a clustering algorithm based on the set seed region. Here, the details of the Chan-Vese algorithm are described in the non-patent document "TF Chan and LA Vese," Active contours without edges, "IEEE Transactions on Image Processing, 10 (2), 266-277 (2001)." Has been done.

The defect detection unit 104 detects defects from the image to be detected based on the normal portion extracted by the classification unit 102 and a predetermined normal template image. Specifically, the defect detection unit 104 compares the component shape extracted by the classification unit 102 with a template image having a normal shape predetermined for the component, and the difference is equal to or higher than a predetermined threshold value. In some cases, the part image identifies it as defective.

Further, when the defect detection unit 104 determines that there is a defect, the defect detection unit 104 determines that the region having a difference from the template image in the component image is a defect region.

<Details of processing>
Next, the defect detection process by the information processing apparatus 10 according to the present embodiment will be described. FIG. 4 is a flowchart showing an example of the procedure of the defect detection process of the present embodiment.

First, the component image generation unit 101 inputs an image of an object captured by the imaging device 30, which is an image to be inspected, and performs matching (template matching) with the template image 110 for each component from the image. , A part of a part constituting an object is cut out to generate one or a plurality of part images (S11).

FIG. 5 is a diagram showing an example of an image input in the present embodiment. In the example of this image, it is assumed that the black part near the center is a defect.

Next, the classification unit 102 classifies the luminance histogram at the center of each component into two clusters by the k-means algorithm, which is unsupervised learning, for each component image generated in S11, and classifies each pixel into two clusters. It is classified into the cluster to which it belongs (S12).

Next, the classification unit 102 sets a cluster of pixels classified into clusters that are close to the brightness value of the original component (pixels whose difference in brightness value is equal to or less than a predetermined value) as a seed region (S13).

FIG. 6 is a diagram showing an example of an image of the seed region in the present embodiment. In the example of FIG. 6, by clustering by the k-means algorithm of S12, only the region having the luminance value close to the original luminance value near the center of the component is extracted and set as the seed region.

Then, the classification unit 102 extracts the component shape by the Chan-Vese algorithm, which is a clustering algorithm, based on the set seed region (S14). FIG. 7 is a diagram showing an example of an image (normal region image) extracted as a component shape in the present embodiment.

Next, the defect detection unit 104 compares the component shape extracted in S14 with the template image of the normal component shape (S15). FIG. 8 is a diagram showing an example of the template image 110 in the present embodiment. The image of FIG. 7 and the image of FIG. 8 will be compared.

Then, the defect detection unit 104 determines whether or not the difference between the component shape and the template image of the normal component shape is equal to or greater than a predetermined threshold value (S16). When the difference is equal to or greater than the threshold value (S16: Yes), the defect detection unit 104 identifies the component as having a defect (S17). Then, the defect detection unit 104 determines that the portion having a difference from the template image 110 is a defect region (S18).

In S16, when the difference between the component shape and the template image of the normal component shape is less than a predetermined threshold value (S16: No), the component is determined to have no defect (S19).

The processes from S15 to S18 and S19 are repeatedly executed for each component. When the processes from S15 to S18 and S19 are executed for all the parts, the process ends.

According to S12, even if there is a defect in the component, only the region of the portion without the defect can be used as the seed region for the clustering in the next stage. As a result, even when a defect exists in the component, only the normal portion can be extracted by clustering in S14.

<Evaluation example>
An actual evaluation example is shown below. It is desirable that many images are input for evaluation of the detection result. In this evaluation example, since the number of images provided is small, Data augmentation is performed by randomly adding translation, rotation, and brightness change to the provided images, and the number of images for evaluation is increased to 55. It was. Table 1 shows the processing results of the evaluation image.

Here, the computer that performed the defect detection was a CPU: Core i7 870, a memory of 20 GB, and the OS was Windows (registered trademark) 7 64 bits.

The defect detection rate is the detection rate of 30 defect sites existing in the evaluation image. The false detection rate indicates the rate at which a normal part is mistakenly detected as a defective part in a total of 55 sample images. All of the sample images prepared this time are correct, and it can be seen that they have high defect detection performance.

FIG. 9 is a diagram showing an example of the detection result in the present embodiment. The broken line of reference numeral 902 is the detection result of the correct answer, and the solid line of reference numeral 901 is the detection result detected by the algorithm of the present embodiment. In the detection result shown in FIG. 9, it can be seen that the defective region can be normally detected so as to substantially overlap with the correct answer. Regarding the processing time, when the processing content was multithreaded and 55 sample images were processed, it was 777 ms per image.

As described above, in the present embodiment, the object is classified into clusters by using unsupervised learning from the captured image of the object to be detected, and the object is included from the classified cluster group based on the value based on the original pixel of the object. By clustering that extracts the cluster and sets the cluster as the seed area, the normal part is extracted from the image of the detection target in which the object is captured, and the detection target is based on the extracted normal part and the normal template image. Detect defects in the image. That is, in this embodiment, since it is a method that combines unsupervised learning and clustering processing, it is not necessary to prepare a plurality of samples in advance. Then, in the present embodiment, by using pixels close to the brightness value of the original pixels of the component from the cluster classified by unsupervised learning as the seed region, even if there is a defect in the component, there is no defect. Only the region can be extracted. As a result, according to the present embodiment, defect detection can be performed with high accuracy even when the number of sample images is small, when there is disturbance, or even in the case of a new defect mode.

The defect detection program 20 executed by the information processing apparatus 10 of the present embodiment is a file in an installable format or an executable format, and is a CD-ROM, a flexible disk (FD), a CD-R, or a DVD (Digital Versaille). It is recorded and provided on a recording medium that can be read by a computer such as a Disc).

Further, the defect detection program 20 executed by the information processing apparatus 10 of the present embodiment may be stored on a computer connected to a network such as the Internet and provided by downloading via the network. .. Further, the defect detection program 20 executed by the information processing apparatus 10 of the present embodiment may be provided or distributed via a network such as the Internet.

Further, the defect detection program 20 executed by the information processing apparatus 10 of the present embodiment may be configured to be provided by being incorporated in the ROM 14 or the like in advance.

Although some embodiments of the present invention have been described, these embodiments are presented as examples and are not intended to limit the scope of the invention. These novel embodiments can be implemented in various other embodiments, and various omissions, replacements, and changes can be made without departing from the gist of the invention. These embodiments and modifications thereof are included in the scope and gist of the invention, and are also included in the scope of the invention described in the claims and the equivalent scope thereof.

1 Information processing system 10 Information processing device 101 Parts image generation unit 102 Classification unit 104 Defect detection unit 110 Template image

Japanese Unexamined Patent Publication No. 2013-167596

Claims (10)

An information processing device that detects defects in the appearance of an object.
Each pixel of one or more partial images of the object cut out from the image to be detected in which the object is captured is classified into one of a plurality of clusters by using unsupervised learning, and the classified clusters are used. By clustering, clusters including pixels corresponding to the object are extracted from the group based on the brightness value of the original pixels of the object, and a region composed of the extracted pixels classified into the cluster is used as a seed region. , A classification unit that extracts the component shape of the object from the image of the detection target in which the object is captured,
A defect detection unit that detects defects from the image to be detected based on the extracted component shape and a predetermined normal template image, and
Information processing device equipped with. A component image generator for cutting out one or more component images of the object from the image to be detected is further provided.
The classification unit is an algorithm for the clustering of unsupervised learning based on a brightness histogram which is a histogram of the brightness values of pixels in the central portion of one or a plurality of component images of the object cut out from the image to be detected. By classifying each pixel included in the brightness histogram into one of two clusters using the k-means algorithm as described above, each pixel constituting each of the one or a plurality of component images is divided into the two. Classify into one of the clusters,
The information processing device according to claim 1. The classification unit extracts a cluster of pixels having a brightness value whose difference from the brightness value of the original component is within a predetermined value from the two clusters, and an area composed of pixels classified into the extracted cluster. Is the seed region.
The information processing device according to claim 2. Based on the seed region, the classification unit extracts the component shape from the component image by the Chan-Vese algorithm as the clustering algorithm.
The information processing device according to claim 3. The defect detection unit compares the extracted component shape with a template image having a normal shape predetermined for the component, and when the difference is equal to or greater than a predetermined threshold value, the component image. Identify as defective,
The information processing device according to claim 4. When the defect detection unit determines that there is a defect, the defect detection unit determines, in the component image, a region having a difference from the template image as a defect region.
The information processing device according to claim 5. The component image generation unit cuts out one or a plurality of component images of the object from the image to be detected by matching with the template image.
The information processing device according to claim 5 or 6. When matching with the template image, the component image generation unit complements the portion outside the display screen with the template image when a part of the component is outside the display screen, and performs the matching. Do, do
The information processing device according to claim 7. A defect detection method that detects defects in the appearance of an object.
Each pixel of one or more partial images of the object cut out from the image to be detected in which the object is captured is classified into one of a plurality of clusters by using unsupervised learning, and the classified clusters are used. A cluster containing pixels corresponding to the object is extracted from the group based on a value based on the brightness value of the original pixel of the object, and an area composed of the extracted pixels classified into the cluster is used as a seed area. The component shape of the object was extracted from the image of the detection target in which the object was imaged by the clustering.
Defects are detected from the image to be detected based on the extracted part shape and a predetermined normal template image.
Defect detection methods including that. A program to be executed by a computer that detects defects in the appearance of an object.
Each pixel of one or more partial images of the object cut out from the image to be detected in which the object is captured is classified into one of a plurality of clusters by using unsupervised learning, and the classified clusters are used. A cluster containing pixels corresponding to the object is extracted from the group based on the brightness value of the original pixel of the object, and a region composed of the extracted pixels classified into the cluster is used as a seed region by clustering. , The component shape of the object is extracted from the image of the detection target in which the object is captured.
Defects are detected from the image to be detected based on the extracted part shape and a predetermined normal template image.
A program that causes the computer to do this.