JP2017096853A - Information processing device, defect detection method and program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a method for performing defect detection with high accuracy when the number of sample images is small, when there is disturbance, and even when it is in a new defect mode.SOLUTION: An information processing device 10 for detecting apparent defects of an object includes a classification part 102 for extracting a cluster including an object on the basis of a value based on an original image of the object from a cluster group by using teacherless learning from an image of a detection object in which the object is imaged, and extracting a normal part from the image of the detection object in which the object is imaged from clustering set with the area of the cluster as a seed area, and a defect detection part 104 for detecting defects from the image of the detection object on the basis of the extracted normal part and a prescribed normal template image 110.SELECTED DRAWING: Figure 3

Description

  The present invention relates to an information processing apparatus, a defect detection method, and a program.

  In recent years, a defect detection technique using an image has been generally used mainly for applications in FA (Factory Automation) and the like. In such defect detection technology, even if there are few sample images that can be used as prior knowledge or there are disturbances such as illuminance changes, defect detection can be performed with high accuracy and compatibility with unknown defect modes. It has become a big issue. Here, as a defect detection algorithm that is resistant to disturbances and can handle unknown defect modes, semi-supervised anomaly detection is used to detect images that deviate from them by learning only normal images through machine learning. The technology is conventionally known.

  Further, for example, as shown in Patent Document 1, binarization processing is performed on image information of an inspection object, area information of a defect candidate is acquired, and whether or not the defect is identified from the characteristics of the area is obtained. A technique for detecting defects is known.

  However, in the conventional technology, the number of normal samples needs to be a certain number or more including disturbances.For example, when only a few normal samples can be prepared, defect detection can be performed with good performance in the presence of disturbances. There is a problem that it is difficult to do.

  Further, since the technique disclosed in Patent Document 1 uses simple binarization, there is a possibility that erroneous detection may occur due to a change in illumination state or the like. Further, even the technique of Patent Document 1 is vulnerable to disturbance, and it is difficult to detect defects with good performance under conditions where there is disturbance.

  The present invention has been made in view of the above, and is an information processing apparatus capable of performing defect detection with high accuracy even when the number of sample images is small, when there is a disturbance, or in a new defect mode. The main object is to provide a defect detection method and program.

  In order to solve the above-described problems and achieve the object, the present invention is an information processing apparatus that detects a defect in the appearance of an object, and uses unsupervised learning from an image to be detected in which the object is imaged. The cluster is extracted from a group of classified clusters based on the original pixel value of the object, and the cluster is set as a seed region by clustering the object. A defect detection unit that detects a defect from the detection target image based on a classification unit that extracts a normal part from the imaged detection target image, the extracted normal part, and a predetermined normal template image; Equipped with.

  According to the present invention, there is an effect that defect detection can be performed with high accuracy even when the number of sample images is small, when there is a disturbance, or in the case of a new defect mode.

FIG. 1 is a configuration diagram of an example of an information processing system according to the present embodiment. FIG. 2 is a hardware configuration diagram of an example of the information processing apparatus according to the present embodiment. FIG. 3 is a functional block diagram of an example of the information processing apparatus according to the present embodiment. FIG. 4 is a flowchart illustrating an example of a procedure of defect detection processing according to the present embodiment. FIG. 5 is a diagram illustrating an example of an image input in the present embodiment. FIG. 6 is a diagram illustrating an example of an image of a seed region in the present embodiment. FIG. 7 is a diagram illustrating an example of an image extracted as a component shape in the present embodiment. FIG. 8 is a diagram illustrating an example of a template image in the present embodiment. FIG. 9 is a diagram illustrating an example of a detection result in the present embodiment.

  Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings.

<System configuration>
First, the system configuration of the information processing system 1 according to the first embodiment will be described. FIG. 1 is a configuration diagram of an example of an information processing system according to the first embodiment.

  As illustrated in FIG. 1, the information processing system 1 includes an information processing device 10 and an imaging device 30. The information processing apparatus 10 and the imaging apparatus 30 are connected to be communicable via, for example, a USB (Universal Serial Bus) cable. Note that the information processing apparatus 10 and the imaging apparatus 30 may be communicably connected via a LAN (Local Area Network) or the like.

  The information processing apparatus 10 is, for example, a PC (personal computer). A defect detection program 20 is installed in the information processing apparatus 10. By using the defect detection program 20, the information processing apparatus 10 can detect a defect in the appearance of the subject in the image data input by imaging with the imaging apparatus 30. In the present embodiment, as a subject imaged by the imaging device 30, for example, a product mass-produced in a factory is assumed, and when there is a defect on the appearance, there is a scratch on the appearance (on the surface) of the product or the like. Say a certain case.

  Here, it is assumed that the defect detection program 20 detects defects on the appearance of the subject of the image data by abnormality detection using unsupervised learning. Here, unsupervised learning is one type of machine learning and is machine learning that requires only input data and does not require teacher data.

  The information processing apparatus 10 is not limited to a PC, and may be a mobile terminal apparatus such as a smartphone or a tablet terminal.

  The imaging device 30 is a camera or the like that images a subject and generates image data. The image data generated by the imaging device 30 is input to the information processing device 10 via, for example, a USB cable. The imaging device 30 may be built in the information processing device 10.

<Hardware configuration>
Next, the hardware configuration of the information processing apparatus 10 according to the first embodiment will be described. FIG. 2 is a hardware configuration diagram of an example of the information processing apparatus according to the first embodiment. As shown in FIG. 2, the information processing apparatus 10 includes a CPU (Central Processing Unit) 11, a HDD (Hard Disk Drive) 12, a RAM (Random Access Memory) 13, and a ROM (Read Only Memory) 14. Yes. The information processing apparatus 10 includes an input device 15, a display device 16, an external I / F 17, and a communication I / F 18. These units are connected by a bus B.

  The CPU 11 is an arithmetic device that realizes control and functions of the entire information processing apparatus 10 by reading a program and data from a storage device such as the ROM 14 and the HDD 12 onto the RAM 13 and executing processing.

  The HDD 12 is a non-volatile storage device that stores programs and data. The stored programs and data include, for example, an OS (Operating System) that is basic software for controlling the entire information processing apparatus 10 and application software (for example, a defect detection program 20) that provides various functions on the OS. . The HDD 12 manages stored programs and data by a predetermined file system and / or DB (database). The information processing apparatus 10 may include an SSD (Solid State Drive) or the like instead of the HDD 12 or in combination with the HDD 12.

  The RAM 13 is a volatile semiconductor memory (storage device) that temporarily stores programs and data. The ROM 14 is a nonvolatile semiconductor memory (storage device) that can retain programs and data even when the power is turned off.

  The input device 15 is a device used by a user to input various operation signals. The input device 15 is, for example, various operation buttons, a touch panel, a keyboard, a mouse, or the like. The display device 16 is a device that displays a processing result by the information processing device 10. The display device 16 is, for example, a display.

  The external I / F 17 is an interface with an external device. Examples of the external device include a USB (Universal Serial Bus) memory, an SD card, a CD, and a DVD. The communication I / F 18 is an interface for performing data communication with the imaging device 30.

  The information processing apparatus 10 according to the present embodiment has the above hardware configuration, and can implement various processes described later.

<Functional configuration>
Next, the functional configuration of the information processing apparatus 10 according to the first embodiment will be described. FIG. 3 is a functional block diagram of an example of the information processing apparatus according to the first embodiment. As illustrated in FIG. 3, the information processing apparatus 10 mainly includes a component image generation unit 101, a classification unit 102, a defect detection unit 104, and a template image 110. The component image generation unit 101, the classification unit 102, and the defect detection unit 104 are realized by causing the CPU 11 to execute the defect detection program 20, for example.

  The template image 110 is an image (template) that is present for each part constituting the object and shows a normal shape that is predetermined for each part. The template image 110 is stored in the HDD 12.

  The component image generation unit 101 performs template matching on the image of the object captured by the imaging device 30 with the template image, and generates one or a plurality of component images from the captured image. Here, when matching with the template image 110, the part image generation unit 101 inserts (complements) the template image for a part outside the display screen when a part of the part is outside the display screen. Perform matching. Thus, even when a part of the component is cut off the screen, the defect detection unit 104 does not determine that the defect is defective.

  The classification unit 102 classifies the detection target image obtained by capturing an object into clusters using unsupervised learning, and extracts a cluster including the object based on a value based on the original pixel of the object from the classified cluster group. Then, the normal part is extracted from the detection target image in which the object is imaged by clustering in which the cluster region is set as the seed region. Specifically, the classification unit 102 uses a k-means algorithm as a clustering algorithm for unsupervised learning to calculate a luminance histogram of a central portion of one or more component images of an object as an image to be detected. The pixels are classified into clusters, and each pixel constituting each of one or more component images is classified into a cluster. Here, the classification unit 102 sets a cluster of pixels that approximate the luminance value of the original component among the pixels classified into clusters as a seed region. Here, the pixel that approximates the luminance value of the original component is a pixel whose luminance value is within a predetermined value with respect to the luminance value of the original component. Further, the classification unit 102 extracts a part shape from the part image by the Chan-Vese algorithm as a clustering algorithm based on the set seed region. Here, the details of the Chan-Vese algorithm are described in the non-patent document “TF Chan and LA Vese,“ Active contours without edges, ”IEEE Transactions on Image Processing, 10 (2), 266-277 (2001).” Has been.

  The defect detection unit 104 detects a defect from the detection target image based on the normal part extracted by the classification unit 102 and a predetermined normal template image. Specifically, the defect detection unit 104 compares the part shape extracted by the classification unit 102 with a template image having a normal shape that is predetermined for each part, and the difference is equal to or greater than a predetermined threshold value. In some cases, the part image is identified as defective.

  In addition, when the defect detection unit 104 identifies that there is a defect, the defect detection unit 104 determines that an area having a difference from the template image in the component image is a defect area.

<Details of processing>
Next, the defect detection process by the information processing apparatus 10 according to the present embodiment will be described. FIG. 4 is a flowchart showing an example of the procedure of the defect detection process of the present embodiment.

  First, the component image generation unit 101 inputs an image of an object imaged by the imaging device 30 that is an image to be inspected, and performs matching (template matching) with the template image 110 for each component from the image. Then, one or a plurality of component images are generated by cutting out the parts of the component constituting the object (S11).

  FIG. 5 is a diagram illustrating an example of an image input in the present embodiment. In the example of this image, it is assumed that a black portion near the center is a defect.

  Next, the classification unit 102 classifies the luminance histogram at the center of each component into two clusters by the k-means algorithm which is unsupervised learning for each component image generated in S11, and each pixel is classified. (S12).

  Next, the classification unit 102 sets a cluster of pixels close to the luminance value of the original component (pixels having a luminance value difference equal to or less than a predetermined value) among the pixels classified into clusters as a seed region (S13).

  FIG. 6 is a diagram illustrating an example of an image of a seed region in the present embodiment. In the example of FIG. 6, in the clustering based on the k-means algorithm in S <b> 12, only the luminance value region near the original luminance value near the center of the component is extracted and set as the seed region.

  And the classification | category part 102 extracts a component shape with the Chan-Vese algorithm which is a clustering algorithm based on the set seed area | region (S14). FIG. 7 is a diagram illustrating an example of an image (normal region image) extracted as a component shape in the present embodiment.

  Next, the defect detection unit 104 compares the part shape extracted in S14 with a template image of a normal part shape (S15). FIG. 8 is a diagram illustrating an example of the template image 110 in the present embodiment. The image of FIG. 7 and the image of FIG. 8 are compared.

  Then, the defect detection unit 104 determines whether or not the difference between the component shape and the template image of the normal component shape is equal to or greater than a predetermined threshold (S16). If the difference is greater than or equal to the threshold (S16: Yes), the defect detection unit 104 identifies that the component has a defect (S17). Then, the defect detection unit 104 determines a portion having a difference from the template image 110 as a defective region (S18).

  If the difference between the component shape and the template image of the normal component shape is less than the predetermined threshold value in S16 (S16: No), it is determined that the component has no defect (S19).

  The processes from S15 to S18 and S19 are repeatedly executed for each part. When the processes from S15 to S18 and S19 are executed for all parts, the process ends.

  By S12, even when there is a defect in the part, only the area of the part having no defect can be used as a seed area for the next clustering. As a result, even when there is a defect in the part, only the normal part can be extracted by the clustering of S14.

<Evaluation example>
Hereinafter, an actual evaluation example will be described. For evaluation of detection results, it is desirable that more images be input. In this evaluation example, since the number of images provided is small, data augmentation is performed on the provided images by randomly translating, rotating, and changing the brightness, and the number of evaluation images is increased to 55. It was. Table 1 shows the processing results of the evaluation image.

  Here, the computer which performed the defect detection was CPU: Core i7 870 and memory 20 GB, and the OS used Windows (registered trademark) 7 64 bits.

  The defect detection rate is a detection rate of 30 defective parts existing in the evaluation image. Further, the false detection rate indicates the rate at which a normal part is mistakenly detected as a defective part in a total of 55 sample images. It can be seen that all the sample images prepared this time are correct and have high defect detection performance.

  FIG. 9 is a diagram illustrating an example of a detection result in the present embodiment. A broken line denoted by reference numeral 902 is the detection result of the correct answer, and a solid line denoted by reference numeral 901 is the detection result detected by the algorithm of the present embodiment. From the detection result shown in FIG. 9, it can be seen that the defect area can be normally detected in a form substantially overlapping with the correct answer. Regarding the processing time, the processing content was multi-threaded and 55 sample images were processed, resulting in 777 ms per image.

  As described above, in the present embodiment, the detection target image in which the object is imaged is classified into clusters using unsupervised learning, and the object is included based on a value based on the original pixel of the object from the classified cluster group. The cluster is extracted, and the normal part is extracted from the detection target image in which the object is imaged by clustering with the cluster set as a seed region. Based on the extracted normal part and the normal template image, the detection target Detect defects from the image. That is, in this embodiment, since it is a method that combines unsupervised learning and clustering processing, it is not necessary to prepare a plurality of samples in advance. In this embodiment, by using a pixel close to the luminance value of the original pixel of the component from the cluster classified by unsupervised learning as the seed region, even if there is a defect in the component, Only the region can be extracted. As a result, according to the present embodiment, defect detection can be performed with high accuracy even when the number of sample images is small, when there is a disturbance, or in the case of a new defect mode.

  The defect detection program 20 executed by the information processing apparatus 10 according to the present embodiment is a file in an installable format or an executable format, and is a CD-ROM, flexible disk (FD), CD-R, DVD (Digital Versatile). The program is recorded on a computer-readable recording medium such as a disk.

  Further, the defect detection program 20 executed by the information processing apparatus 10 according to the present embodiment may be provided by being stored on a computer connected to a network such as the Internet and downloaded via the network. . Further, the defect detection program 20 executed by the information processing apparatus 10 according to the present embodiment may be provided or distributed via a network such as the Internet.

  In addition, the defect detection program 20 executed by the information processing apparatus 10 according to the present embodiment may be provided by being incorporated in advance in the ROM 14 or the like.

  Although several embodiments of the present invention have been described, these embodiments are presented by way of example and are not intended to limit the scope of the invention. These novel embodiments can be implemented in various other forms, and various omissions, replacements, and changes can be made without departing from the scope of the invention. These embodiments and modifications thereof are included in the scope and gist of the invention, and are included in the invention described in the claims and the equivalents thereof.

DESCRIPTION OF SYMBOLS 1 Information processing system 10 Information processing apparatus 101 Component image generation part 102 Classification part 104 Defect detection part 110 Template image

JP2013-167596A

Claims (10)

An information processing apparatus for detecting defects on the appearance of an object,
Classifying into clusters using unsupervised learning from the image to be detected in which the object is imaged, and extracting a cluster including the object based on a value based on the original pixel of the object from the classified cluster group, A classification unit that extracts a normal part from an image of a detection target in which the object is imaged by clustering with the region of the cluster set as a seed region;
A defect detection unit that detects a defect from the image to be detected based on the extracted normal part and a predetermined normal template image;
An information processing apparatus comprising: The classification unit uses a k-means algorithm as a clustering algorithm of the unsupervised learning for a luminance histogram of a central portion of one or a plurality of component images of the object as the detection target image. Classify into clusters and classify each pixel constituting each of the one or more component images into the cluster.
The information processing apparatus according to claim 1. The classification unit sets, as the seed region, a cluster of pixels whose luminance value is within a predetermined value from a luminance value of an original component among the pixels classified into the cluster.
The information processing apparatus according to claim 2. The classification unit extracts a part shape from the part image by a Chan-Vese algorithm as the clustering algorithm based on the set seed region.
The information processing apparatus according to claim 3. The defect detection unit compares the extracted part shape with a template image having a normal shape predetermined corresponding to the part, and if the difference is equal to or greater than a predetermined threshold, the part image is Identify it as defective,
The information processing apparatus according to claim 4. When the defect detection unit identifies that there is a defect, an area having a difference from the template image in the component image is determined as a defect area.
The information processing apparatus according to claim 5. A component image generation unit that cuts out one or more component images of the object from the image of the object by matching with the template image;
The information processing apparatus according to claim 5 or 6, further comprising: The part image generation unit complements the part outside the display screen with the template image when the part is outside the display screen when matching with the template image, and performs the matching. Do,
The information processing apparatus according to claim 7. A defect detection method for detecting defects on the appearance of an object,
Classifying into clusters using unsupervised learning from the image to be detected in which the object is imaged, and extracting a cluster including the object based on a value based on the original pixel of the object from the classified cluster group, By clustering with the cluster region set as a seed region, a normal part is extracted from the detection target image in which the object is imaged,
Detecting a defect from the image to be detected based on the extracted normal part and a predetermined normal template image;
A defect detection method including: A program for causing a computer to detect defects in the appearance of an object,
Classifying into clusters using unsupervised learning from the image to be detected in which the object is imaged, and extracting a cluster including the object based on a value based on the original pixel of the object from the classified cluster group, By clustering with the cluster region set as a seed region, a normal part is extracted from the detection target image in which the object is imaged,
Detecting a defect from the image to be detected based on the extracted normal part and a predetermined normal template image;
A program for causing the computer to execute the above.