JP7453813B2 - Inspection equipment, inspection methods, programs, learning devices, learning methods, and learned datasets - Google Patents

Description

The present invention relates to an inspection device, an inspection method, a program, a learning device, a learning method, and a learned data set.

BACKGROUND ART In the manufacturing process of foods, pharmaceuticals, industrial products, etc., an inspection process is sometimes provided on the manufacturing line to perform inspections such as detecting defective products. Product inspections are performed visually by humans (visual inspection), which poses the problem of high human costs. For this reason, in order to automate part or all of the inspection process, development of systems for automatically inspecting products using machines is underway.

For example, Patent Document 1 discloses a technique for defect inspection using machine learning. Specifically, a first learning unit uses a set of normal data to learn a first model for discriminating normal data. A second learning unit learns a second model for discriminating between correct data and incorrect data, regarding an abnormal candidate area selected by a user among a plurality of abnormal candidate areas detected based on the first model from each of a plurality of photographed images prepared in advance as correct data and an abnormal candidate area not selected by the user as incorrect data. A detection unit detects an abnormal candidate area from a photographed image using the first model. A judgment unit judges whether an abnormal candidate area detected by the detection unit belongs to correct data or incorrect data using the second model.

Japanese Patent Application Publication No. 2018-120300

In the case of Patent Document 1, the first learning unit performs machine learning using only non-defective product images. On the other hand, for the second learning section, it is necessary to prepare not only normal sample images but also sample images containing defects as sample images used for learning. However, when the object to be inspected is a product with a low incidence of defects, it may be difficult to obtain sample images containing defects. For this reason, when the object to be inspected has a low incidence of abnormalities, it has been difficult to employ the conventional technique.

An object of the present invention is to provide a technique for successfully detecting abnormalities even in objects with a low incidence of abnormalities.

In order to solve the above problems, a first invention of the present application is an inspection device for inspecting an object, the defect detection device detecting defects in the object based on an image of the object obtained by imaging the object. and a defect determination unit that determines whether the defect detected by the defect detection unit is true or false, and the defect detection unit includes a learning model for defect detection that uses images as input and output, and a learning target for learning. Defect detection learning in which a specimen image, which is an image of an object, is input to the defect detection learning model, and the input specimen image is dimensionally compressed and reproduced to output a reconstructed image of the specimen image. the defect detection unit uses the learning model for defect detection to which the trained data for defect detection is applied to perform dimension compression and reproduction of the object image, and reproduces the object image. executing a process of generating a constituent image, a process of generating a difference image that is a difference between the object image and the reconstructed image, and a process of detecting the presence or absence of a defect based on the difference image; The defect determination unit is configured to use the defect determination learning model that receives the difference image as input , and the defect determination learning model that receives the difference image of the sample image that is the image of the learning object as input, and causes the defect determination learning model to learn the range of non-defective products. The defect determining section applies the learned data for defect determination to the difference image of the object image of the object in which the defect detection section has detected a defect. The defect determination process is input to the learning model for defect determination and determines whether the object belongs to the range of non-defective products.

A second invention of the present application is the inspection apparatus of the first invention, in which the trained data for defect judgment is obtained by inputting only the difference image of the sample image of a non-defective product to the learning model for defect judgment. This is trained data in which the range has been trained.

A third invention of the present application is the inspection apparatus according to the first invention or the second invention , wherein the trained data for defect detection is obtained by inputting the object image of a non-defective item to the learning model for defect detection. This is trained data that has been trained to output a reconstructed image of the target object image.

A fourth invention of the present application is an inspection method for inspecting a target object, which includes a) a defect detection step of detecting a defect in the target object based on an image of the target object obtained by imaging the target object, and b) a defect determination step of determining the authenticity of the defect detected in the step a), and the step a) is a step of converting the object image into a learning model for defect detection to which trained data for defect detection is applied. input, dimensionally compress and reproduce the object image to generate a reconstructed image of the object image, and generate a difference image that is a difference between the object image and the reconstructed image. , a step of detecting the presence or absence of a defect based on the difference image, the learning model for defect detection is a learning model that uses images as input and output, and the trained data for defect detection is a learning model for learning. A trained model that inputs a specimen image, which is an image of a target object, into the defect detection learning model, and trains the input specimen image to perform dimension compression and reproduction to output a reconstructed image of the specimen image. data, and in the step b), input the difference image of the object image of the object in which the defect was detected in the step a) to a learning model for defect judgment to which the learned data for defect judgment is applied; It is determined whether the object belongs to a range of non-defective products, and the learned data for defect judgment is obtained by inputting the difference image of the sample image, which is an image of the object for learning, to a learning model for defect judgment. , is trained data in which the range of non-defective products is learned.

A fifth invention of the present application is the inspection method according to the fourth invention, wherein the trained data for defect judgment is obtained by inputting only the difference image of the sample image of a non-defective product into a learning model for defect judgment. This is a model that has been trained.

A sixth invention of the present application is an inspection program for causing a computer to inspect an object, the computer having: a) detecting defects in the object based on an image of the object obtained by imaging the object; and b) a defect determination step of determining whether the defect detected in step a) is true or false. a step of inputting the object image into a learning model for defect detection to which the object image is applied and compressing and reproducing the object image to generate a reconstructed image of the object image; and a step of generating a reconstructed image of the object image; and a step of detecting the presence or absence of a defect based on the difference image, and the learning model for defect detection is a learning model that uses images as input and output, and The trained data for detection is obtained by inputting a specimen image, which is an image of a learning object, to the learning model for defect detection, and compressing and reproducing the input specimen image to obtain a reconstructed image of the specimen image. In step b), the computer converts the difference image of the object image of the object in which a defect was detected in step a) into trained data for defect determination. The trained data for defect judgment is input to a learning model for defect judgment that applies This is trained data obtained by inputting the difference image to a learning model for defect judgment and learning the range of non-defective products.

A seventh invention of the present application is a learning device for constructing an inspection device for inspecting an object, which inputs a sample image, which is an image of the object for learning, into a learning model for defect detection, and a defect detection learning unit that generates trained data for defect detection that is trained to output a reconstructed image of the sample image obtained by dimensionally compressing and reproducing the sample image ; and a sample image that is an image of the object for learning. , Defect determination learning in which a difference image between the sample image and the reconstructed image generated using the defect detection learning model is input into a defect determination learning model to learn the range of non-defective products. and a defect judgment learning unit that generates completed data.

The eighth invention of the present application is a learning method for constructing an inspection device for inspecting an object, which comprises x) inputting a sample image, which is an image of the object for learning, to a learning model for defect detection; y) a defect detection learning step of generating trained data for defect detection trained to output a reconstructed image obtained by dimensionally compressing and reproducing the sample image; , Defect determination learning in which a difference image between the sample image and the reconstructed image generated using the defect detection learning model is input into a defect determination learning model to learn the range of non-defective products. and a defect judgment learning step for generating completed data.

According to the first to eighth inventions of the present application, by checking the detection results of the defect detection section using the learning model for defect judgment and the trained data for defect judgment in which the range of non-defective products is learned, It is possible to determine the detected non-defective image. Thereby, it is possible to reduce over-detection in which a good product is judged as a defective product.

In particular, according to the second and fifth inventions of the present application, over-detection can be efficiently reduced even for objects for which defective products are difficult to obtain.

It is a figure showing an inspection device of a 1st embodiment. FIG. 1 is a diagram showing a hardware configuration of an information processing apparatus according to a first embodiment. 1 is a diagram showing a functional configuration of an information processing apparatus according to a first embodiment; FIG. It is a flowchart which shows the flow of the learning process for defect detection of 1st Embodiment. FIG. 2 is a diagram conceptually showing a learning model for defect detection according to the first embodiment. FIG. 3 is a diagram conceptually showing how a defective image is input to a trained learning model for defect detection. It is a flowchart which shows the flow of the defect detection process in the defect detection part of 1st Embodiment. It is a figure showing an example of an image obtained at each stage of a defect detection process of a 1st embodiment. It is a figure showing an example of an image obtained at each stage of a defect detection process of a 1st embodiment. It is an example of a diagram showing a difference image and a defect detection partial image for a non-defective object. It is an example of a diagram showing a difference image and a defect detection partial image for a defective object. It is a flowchart which shows the flow of the learning process for defect judgment of 1st Embodiment. It is a figure showing the example of the image obtained at each stage of the learning process for defect judgment of a 1st embodiment. It is a flowchart which shows the flow of an inspection process using the inspection device of a 1st embodiment. It is a flowchart which shows the flow of a defect judgment process using the inspection device of a 1st embodiment. FIG. 3 is a diagram showing examples of images and inspection results obtained at each stage of the inspection process of the first embodiment. It is a figure showing the functional composition with which an information processing device of an inspection device of a 2nd embodiment is provided.

Embodiments of the present invention will be described below with reference to the accompanying drawings. Note that the components described in this embodiment are merely examples, and the scope of the present invention is not intended to be limited thereto. In the drawings, dimensions and numbers of parts may be exaggerated or simplified as necessary to facilitate understanding.

<1. First embodiment>
<1-1. Configuration of inspection device>
FIG. 1 is a diagram showing an inspection apparatus 10 according to the first embodiment. The inspection device 10 detects defects in the object 90 by analyzing an image of the object 90. The object 90 is specifically a tablet, but is not limited to a tablet. The inspection device 10 includes a camera 110 and an information processing device 120. Camera 110 is electrically connected to information processing device 120. Camera 110 includes an image sensor. Camera 110 outputs an image signal obtained by capturing an image of object 90 using an image sensor to information processing device 120 . The object 90 imaged by the camera 110 may be stopped at a predetermined position, or may be moved in a predetermined direction by a conveyance mechanism such as a belt conveyor.

FIG. 2 is a diagram showing the hardware configuration of the information processing device 120 of the first embodiment. The information processing device 120 has a configuration as a computer. Specifically, the information processing device 120 includes a processor 121, a RAM 123, a storage section 125, an input section 127, a display section 129, a device I/F 131, and a communication I/F 133. Processor 121, RAM 123, storage section 125, input section 127, display section 129, device I/F 131, and communication I/F 133 are electrically connected to each other via bus 135.

Processor 121 specifically includes a CPU or GPU. The RAM 123 is a storage medium from which information can be read and written, and specifically, it is an SDRAM. The storage unit 125 is a recording medium from which information can be read and written, and specifically includes an HDD (hard disk drive) or an SSD (solid state drive). Note that the storage unit 125 may include a portable optical disk, magnetic disk, semiconductor memory, or the like. The storage unit 125 stores a program P. The processor 121 implements various functions by executing the program P using the RAM 123 as a work area. Note that the program P may be provided or distributed to the information processing device 120 via a network.

The input unit 127 is an input device that receives user operation input, and specifically, is a mouse, a keyboard, or the like. The display unit 129 is a display device that displays images representing various information, and specifically, is a liquid crystal display.

The device I/F 131 is an interface for electrically connecting the camera 110 to the information processing device 120. The communication I/F 133 is an interface for connecting the information processing device 120 to a network such as the Internet. Camera 110 may be connected to information processing device 120 via communication I/F 133. That is, the inspection device 10 does not necessarily need to include the camera 110, and may include only the information processing device 120.

FIG. 3 is a diagram showing the functional configuration of the information processing device 120 of the first embodiment. The information processing device 120 includes a defect detection section 20, a defect judgment section 30, a defect detection learning section 210, and a defect judgment learning section 310.

The defect detection unit 20 detects defects in the object 90 based on an object image obtained by capturing an image of the object 90 with the camera 110. The defect detection section 20 includes a learning model for defect detection 200, trained data for defect detection 21, a reconstructed image generation section 22, a difference image generation section 23, and a defect detection processing section 24.

The defect determining section 30 determines whether the defect detected by the defect detecting section 20 is true or false. The defect determination unit 30 includes a learning model 300 for defect determination, learned data 31 for defect determination, and a defect determination processing unit 32.

The reconstructed image generation section 22, the difference image generation section 23, the defect detection processing section 24, and the defect judgment processing section 32 are functions realized by the processor 121 operating according to the program P. The defect detection learning section 210 and the defect judgment learning section 310 are not necessarily provided in the information processing device 120, and may be provided in another computer.

The defect detection learning model 200 is stored in the storage unit 125. The defect detection learning model 200 is a learning model that uses images as input and images as output.

The defect detection learned data 21 is learned data obtained by the defect detection learning section 210 performing learning using the defect detection learning model 200. Specifically, the trained data 21 for defect learning is configured such that a sample image, which is an image of an object for learning, is input to the learning model 200 for defect detection, and a reconstructed image of the input sample image is output. This is trained data that has been trained. That is, the learning model 200 for defect detection to which the trained data 21 for defect detection is applied receives a sample image of a good product as input and operates to output a reconstructed image of the input sample image.

The defect detection learning model 200 is, for example, an autoencoder using a neural network or a variational autoencoder.

The non-defective specimen image is an image obtained by capturing an image of the object 90 with no defects using the camera 110. Hereinafter, the sample image of a non-defective product is also referred to as a "defective product image." The defect detection learning unit 210 generates trained data 21 for defect detection by machine learning using a plurality of non-defective product images, and stores it in the storage unit 125.

When inspecting an object, an object image captured by the camera 110 of the object 90 to be inspected is input to the defect detection section 20 as an input image. The reconstructed image generation unit 22 inputs the input image to the defect detection learning model 200 to which the defect detection trained data 21 is applied, and executes a process of generating a reconstructed image from the input image. The difference image generation unit 23 executes processing to generate a difference image that is the difference between the input image and the reconstructed image. The defect detection processing unit 24 executes a process of detecting the presence or absence of a defect based on the difference image.

The defect determination learning model 300 is stored in the storage unit 125. The learning model 300 for defect determination is a learning model that receives the processing data processed by the defect detection unit 20 as input and outputs a judgment as to whether the input processing data is processing data of a non-defective product.

The trained data for defect determination 31 is trained data obtained by the defect determination learning unit 310 performing learning using the learning model for defect determination 300. Specifically, the trained data 31 for defect judgment is obtained by inputting processed data obtained by processing a sample image for learning by the defect detection unit 20 into the learning model 300 for defect judgment, and the defect judgment learning unit 310 determines the range of non-defective products. This is the trained data that was trained on. That is, the defect judgment learning model 300 to which the defect judgment trained data 31 is applied receives the difference image generated by the defect detection unit 20 from the sample image of a non-defective product as input, and determines whether the difference image is included in the range of non-defective products. It operates to output something.

The defect determination learning model 300 is, for example, One Class SVM (One Class Support Vector Machine). One Class SVM can detect outliers by performing classification using supervised learning. Note that the defect determination learning model 300 may be a machine learning model that performs other supervised learning. Further, as the defect determination learning model 300, a machine learning model that can perform clustering by unsupervised learning, such as a One Class neural network, may be used.

The defect determination processing unit 32 inputs the difference image of the object 90 in which the defect detection unit 20 has detected a defect to the defect determination learning model 300 to which the defect determination trained data 31 is applied, and determines whether the target object 90 is a non-defective item. Defect determination processing is executed to determine whether the defect belongs to the range of . This determines whether there is a final defect. That is, for the object 90 that the defect determination processing unit 32 determines to belong to the range of non-defective products, the defect detected by the defect detection unit 20 is determined to be a false defect, and is not ultimately detected as a defective product. On the other hand, for the object 90 that the defect determination processing unit 32 determines does not belong to the range of non-defective products, the defect detected by the defect detection unit 20 is determined to be a true defect, and is detected as a final defective product.

The processor 121 may display information representing the final determination result output by the defect determination processing section 32 on the display section 129. The information representing the determination result may include information indicating the position in the target image where an abnormality has been detected, as well as the target image determined to have a defect.

Preferably, the plurality of sample images that are input when learning the trained data for defect detection 21 and the plurality of sample images that are input when learning the trained data for defect determination 31 do not include the same sample image.

<1-2. Defect detection learning process>
The defect detection learning process will be explained with reference to FIG. 4. FIG. 4 is a flowchart showing the flow of the defect detection learning step S10 of this embodiment. In the defect detection learning step S10, the defect detection learning model 200 is used to generate defect detection learned data 21, which is parameters for detecting defects.

As shown in FIG. 4, the information processing device 120 first performs a preparation process of preparing a plurality of non-defective product images (step S11). Specifically, the processor 121 captures images of the plurality of non-defective objects 90 with the camera 110, thereby acquiring a plurality of non-defective sample images (non-defective product images) D11. Then, the processor 121 obtains the plurality of non-defective product images D11. Then, the processor 121 causes the storage unit 125 to store the plurality of acquired non-defective product images.

When the preparation step S11 is completed, the defect detection learning unit 210 inputs the plurality of non-defective images obtained in the preparation step S11 as a data set to the defect detection learning model 200, and performs a step to generate trained data 21 for defect detection. 1. A learned data generation step is executed (step S12). Note that the term "generate learned data" includes updating already generated learned data.

Specifically, in the first trained data generation step S12, the defect detection learning unit 210 inputs a dataset of a plurality of non-defective images to the defect detection learning model 200, and causes the defect detection learning model 200 to generate a reconstructed image. Then, an evaluation index indicating how similar the input image and the reconstructed image are is calculated. For example, the sum of squares of differences (errors) between the input non-defective image and the output reconstructed image is used as the evaluation index. Thereafter, based on the calculated evaluation index, the trained data 21 for defect detection, including parameters for weighting the neural network constituting the learning model 200 for defect detection, is updated by back propagation.

Subsequently, the defect detection learning unit 210 determines whether learning of the defect detection learning model 200 has been completed (step S13). Specifically, it is determined whether the evaluation index calculated in the immediately preceding first learned data generation step S12 has converged within a predetermined range. That is, the defect detection learning model 200 to which the defect detection trained data 21 generated in the immediately preceding first learned data generation step S12 is applied, generates a reconstructed image that approximates a non-defective image with desired accuracy. Determine whether or not there is. Specifically, for example, when the evaluation index calculated in the first learned data generation step S12 is smaller than a predetermined threshold value, the defect detection learning model 200 determines that learning has ended. In addition, in step S13, it may be determined that learning has ended also when the number of times of repeated execution of the first learned data generation step S12 reaches a preset number of times.

In step S13, if it is determined that the learning of the defect detection learning model 200 is not completed (step S13: No), the processor 121 returns to the first learned data generation step S12.

When determining that learning of the defect detection learning model 200 is completed in step S13 (step S13: Yes), the processor 121 stores the defect detection trained data 21 generated by the defect detection learning unit 210 in the storage unit 125. A storage process is executed (step S14).

FIG. 5 is a diagram conceptually showing a defect detection learning model 200 to which the defect detection trained data 21 of this embodiment is applied. As shown in FIG. 5, the defect detection learning model 200 is specifically a neural network having an encoder and a decoder. The encoder obtains latent variables by dimensionally compressing the input image. The decoder recreates the original input image from the latent variables. Note that the number of elements in the input layer and output layer and the number of hidden layers of the defect detection learning model 200 shown in FIG. 5 are only examples, and are not limited to these.

The defect detection learning unit 210 performs learning so that the defect detection learning model 200 outputs a reconstructed image D21 that is the same as the original non-defective image D11 from the input non-defective image D11. Defect detection trained data 21, which is parameters and the like, is generated. In the defect detection learning step S10 of the present embodiment, the defect detection learning unit 210 uses, for example, 1000 non-defective images D11 to learn the learning model 200 for defect detection, and generates trained data 21 for defect detection. do. Below, the defect detection learning model 200 to which the defect detection trained data 21 is applied may be simply referred to as "the trained defect detection learning model 200."

FIG. 6 conceptually shows how an image of a defective product (hereinafter referred to as "defective image") D12 is input to the (trained) defect detection learning model 200 to which the defect detection trained data 21 is applied. FIG. The defective image D12 input in FIG. 6 is an image obtained by capturing an image of the object 90 including the defective portion 91 using the camera 110. The defective portion 91 is, for example, dirt or foreign matter attached to the object 90, a chip, or the like. The reconstructed image D22 that is reproduced from the defective image D12 by the trained defect detection learning model 21 is a non-defective image that does not include the defective portion 91.

<1-3. Defect detection process by defect detection section>
Next, the defect detection step S20 using the trained defect detection learning model 200 in the defect detection section 20 will be described with reference to FIGS. 7 to 9. In this defect detection step S20, defects in the object 90 are detected based on an object image obtained by imaging the object 90. FIG. 7 is a flowchart showing the flow of the defect detection process in the defect detection section 20 of this embodiment. FIGS. 8 and 9 are diagrams showing examples of images D1 to D3 obtained at each stage of the defect detection step S20. Note that FIG. 8 is an example in which the object 90 is a non-defective item, and FIG. 9 is an example in which the object 90 is a defective item.

As shown in FIG. 7, the information processing device 120 first performs an imaging step of photographing an object 90 to be inspected to obtain an image D1 of the object 90 (hereinafter referred to as "object image D1"). (Step S21). Specifically, the processor 121 uses the camera 110 to capture an image of the object to be inspected, thereby acquiring the object image D1. Then, the processor 121 causes the storage unit 125 to store the acquired object image D1.

Next, the reconstructed image generation unit 22 executes a reconstructed image generation step in which a reconstructed image D2 is generated using the trained defect detection learning model 200 from the object image D1 obtained in the imaging step S21. (Step S22). Specifically, the reconstructed image generation unit 22 inputs the object image D1 to the defect detection learning model 200 to which the defect detection trained data 21 is applied, and obtains the reconstructed image D2 as the output. . Then, the processor 121 causes the storage unit 125 to store the obtained reconstructed image D2.

Subsequently, the difference image generation unit 23 executes a difference image generation step of generating a difference image D3 from the object image D1 and the reconstructed image D2 stored in the storage unit 125 (step S23). Specifically, the difference image generation unit 23 generates the difference image D3 by taking the difference between the brightness of each pixel of the object image D1 and the brightness of each pixel of the reconstructed image D2. Then, the processor 121 causes the storage unit 125 to store the acquired difference image D3.

Here, if the object 90 is a good item, the reconstructed image D2 becomes an image that approximates the object image D1, as shown in FIG. That is, in this case, the brightness of each pixel of the reconstructed image D2 approximates the brightness of each pixel of the object image D1. Therefore, as shown in FIG. 8, the difference image D3 becomes an image in which the brightness of the approximate image is canceled out, and the brightness of each pixel is small.

On the other hand, if the object 90 is defective, a defective portion 91 appears in the object image D1, as shown in FIG. In such a case, as described above, the reconstructed image D2 generated from the object image D1 including the defective portion 91 becomes an image in which the defective portion 91 is removed from the object image D1.

Therefore, in the difference image D3, in the normal portion of the object 90, the brightness of the approximated image is canceled out, and the brightness of each pixel is small. On the other hand, regarding the defective portion 91 in the difference image D3, since the difference in brightness between the object image D1 and the reconstructed image D2 is large, the brightness of the defective portion 91 is greater than that of the normal portion. Therefore, as shown in FIG. 9, the defective portion 91 is displayed whitish in the difference image D3. In this way, by obtaining the difference image D3, the defective portion 91 can be specifically extracted.

Finally, the defect detection processing unit 24 executes a defect detection step of determining whether or not the object 90 has a defect based on the difference image D3 (step S24). Specifically, the defect detection processing unit 24 determines whether there is a portion in the difference image D3 whose brightness is higher than a predetermined threshold. This threshold value may be a common threshold value for the entire difference image D3, or a threshold value may be set depending on the part of the difference image D3 depending on the type of the object 90.

Here, with reference to FIGS. 10 and 11, a case will be described in which defects are detected using a common threshold value for the entire difference image D3. FIG. 10 is an example of a diagram showing a difference image D3 for a non-defective object 90 and a defect detection partial image D4 in the difference image D3. FIG. 11 is an example of a diagram showing a difference image D3 of a defective object 90 having a defective portion 91 and a defect detection portion image D4 in the difference image D3.

In FIGS. 10 and 11, the difference image D3 is shown in the upper part, and the part of the difference image D3 corresponding to the defect detection partial image D4 is shown in the solid line and the dashed white rectangle in the lower part. It is shown. Note that, for ease of understanding, the difference image D3 in FIG. 10 and the difference image D3 in FIG. 11 are images having the same brightness except for the defective portion 91.

In addition, in FIGS. 10 and 11, the difference image D3 is divided into 64 blocks of 8×8 in the vertical and horizontal directions, and a block having a portion with a brightness higher than the threshold value is defined as a defect detection partial image D4. Note that the method of dividing the difference image D3 is not limited to this. Moreover, it is not necessarily necessary to divide into multiple blocks.

As shown in FIG. 11, in the difference image D3, it is clear that the brightness of the defective part 91 is higher than that of other parts. However, in this object 90, as shown in FIGS. 10 and 11, in the difference image D3, there may be a portion with relatively high brightness in addition to the defective portion 91. For example, the object 90 of this embodiment is a tablet having a score line, and even if it is a good product, the degree of light reflection at the score line portion and the edge differs depending on the object 90. For this reason, the defect detection learning model 21 may not be able to sufficiently reconstruct the reflection of the light, and the brightness may increase in the difference image D3.

Therefore, in FIGS. 10 and 11, in addition to the part surrounded by a solid line, which is a true defect part, there is a possibility that a part surrounded by a broken line, which is a false defect part, may be selected as the defect detection partial image D4. . That is, the defect detection partial image D4 includes a true defect detection partial image D41 including a true defect and a false defect detection partial image D42 including a false defect.

Therefore, when a defect is detected in the defect detection unit 20, this inspection apparatus 10 determines whether the defect detected in the defect detection unit 20 (in this embodiment, the defect detection partial image D4) is a true defect. It has a defect determination section 30 that determines.

<1-4. Learning process for defect judgment>
The defect determination learning process will be described with reference to FIGS. 12 and 13. FIG. 12 is a flowchart showing the flow of the defect determination learning step S30 of this embodiment. FIG. 13 is a diagram showing examples of images D13 and D4 (D42) obtained at each stage of the defect determination learning step S30. In the defect determination learning step S30, the defect determination learning model 300 is used to perform defect determination learning, which is a parameter for determining whether the processed data output by the defect detection unit 20 is included in the range of non-defective products. generated data 31.

As shown in FIG. 12, the information processing device 120 first performs a preparation process of preparing a plurality of non-defective product images (step S31). Specifically, the processor 121 captures images of the plurality of non-defective objects 90 with the camera 110 to obtain a plurality of non-defective sample images (non-defective product images) D13 (not shown). Then, the processor 121 causes the storage unit 125 to store the plurality of acquired non-defective product images D13.

When the preparation step S31 is completed, the defect judgment learning section 310 inputs the plurality of non-defective images D13 obtained in the preparation step S31 to the defect detection section 20, and obtains the defect detection partial image D4 generated by the defect detection section 20. A defect detection result generation step is executed (step S32). In this defect detection result generation step S32, there are cases in which no defect detection partial images D4 are generated for one non-defective product image D13, and cases in which one or more defect detection partial images D4 are generated. . Here, since all the input images input to the defect detection unit 20 in the defect detection result generation step S32 are non-defective images D13, all of the defect detection partial images D4 output from the defect detection unit 20 at this time are false defects. This is a detected partial image D42. At this time, the processor sets the count number n to 1.

Subsequently, the defect judgment learning unit 310 inputs the n-th image of the plurality of false defect detection partial images D42 obtained in the defect detection result generation step S32 to the defect judgment learning model 300, and performs defect judgment learning. A second learned data generation step for generating learned data 31 is executed (step S33). Note that the term "generate learned data" includes updating already generated learned data.

Specifically, the defect judgment learning unit 310 applies the plurality of false defect detection partial images D42 obtained in the defect detection result generation step S32 to the defect judgment learning model 300, which is a machine learning model such as One Class SVM. , along with a label of "fake defect", are input to the defect judgment learning model 31 to learn the range of false defects (i.e. non-defective products). As a result, trained data 31 for defect determination is generated.

Thereafter, the processor 121 determines whether the false defect detection partial image D42 input in the immediately preceding second learned data generation step S33 is the final image prepared in the preparation step S31 (step S34). ). That is, the processor 121 determines whether the count number n matches the number of false defect detection partial images D42 prepared in the preparation step S31.

In step S34, if it is determined that the image is not the final image (step S34: No), the processor 121 increments n (step S35), and returns to the second learned data generation step S33.

In step S34, if it is determined that the image is the final image (step S34: Yes), the processor 121 executes a storage step of storing the defect judgment trained data 31 generated by the defect judgment learning unit 310 in the storage unit 125 ( Step S36).

Note that it is preferable that the plurality of non-defective images used for learning the trained data 21 for defect detection and the plurality of non-defective images used for learning the learned data 31 for defect determination are all different. That is, the non-defective image obtained in the preparation step S11 of the defect detection learning step S10 is different from the non-defective image obtained in the preparation step S31 of the defect determination learning step S30. When the same non-defective image used for learning the trained data 21 for defect detection is used for learning the trained data 31 for defect judgment, the learning model 300 for defect judgment to which the trained data 31 for defect judgment is applied is There is a possibility that the false positive detection rate (rate of detecting false defects) will decrease.

<1-5. Inspection process (defect detection)>
Finally, an inspection process for detecting defects in a product object using the inspection apparatus 10 will be described with reference to FIGS. 14 to 16. FIG. 14 is a flowchart showing the flow of the inspection process S30 using the inspection apparatus 10 of this embodiment. FIG. 15 is a flowchart showing the flow of the defect determination step S50 using the inspection apparatus 10 of this embodiment. FIG. 16 is a diagram showing examples of images D1 and D4 and classification results D5 obtained at each stage of the inspection process S30.

As shown in FIG. 14, the information processing device 120 first executes the defect detection step S20 by the defect detection unit 20 described above (step S20). This defect detection step S20 is the same as the defect detection step S20 described above. That is, the processor 121 generates a reconstructed image D2 using the defect detection learning model 21 by the defect detection unit 20 with respect to the object image D1 captured by the camera 110 of the object to be inspected, and then generates a difference image D3. Defect detection processing is performed by generating .

Following the defect detection step S20, the information processing device 120 determines whether the defect detection unit 20 has detected a defect (step S41). In the defect detection step S20, if the defect detection section 20 does not detect a defect (step S41: No), the defect detection section 20 does not output the defect detection partial image D4. In this case, the information processing device 120 concludes that no defect has been found and ends the inspection of the target object image D1.

On the other hand, in the defect detection step S20, when the defect detection unit 20 detects a defect (step S41: Yes), the defect detection unit 20 outputs a defect detection partial image D4 of one or more portions recognized as defects. do. In this case, the information processing device 120 delivers the defect detection partial image D4 to the defect determination unit 30, and proceeds to the defect determination step S50.

Here, the defect determination step S50 will be explained with reference to FIGS. 15 and 16. In the example of FIG. 16, the object image D1 has a defective portion 91. Further, in the example of FIG. 16, the defect detection section 20 outputs a plurality of defect detection partial images D4 in the defect detection step S41. Hereinafter, the defect determination step S50 will be explained with reference to the example of FIG. 16.

In the defect determination step S50, the defect determination processing unit 32 processes the defect detection partial image D4 outputted by the defect detection unit 20 in the defect detection step S20 by applying (already learned) defect determination learning data 31 to the defect detection partial image D4 outputted by the defect detection unit 20 in the defect detection step S20. The defect detection partial images D4 are input to the model 300 and classified into whether each defect detection partial image D4 is a true defect or a false defect (step S51).

In step S51, specifically, when the trained defect judgment learning model 300 determines that the input defect detection partial image D4 is within the range of non-defective products, it converts the defect detection partial image D4 into a fake defect detection part. The image is classified into image D42. On the other hand, when the trained defect determination learning model 300 determines that the input defect detection partial image D4 is not within the range of non-defective products, it classifies the defect detection partial image D4 as a true defect detection partial image D41. Then, a classification result D5 for the object image D1 to be detected as a defect is output. In the example of FIG. 16, the defect determination learning model 31 classifies one of the four input defect detection partial images D4 as a true defect detection partial image D41, and classifies the remaining three as false defects. It is classified as a detected partial image D42.

Subsequently, the defect determination unit 30 determines whether the true defect detection partial image D41 is included in the classification result D5 outputted by the trained defect determination learning model 300 (step S52). When determining that the classification result D5 includes the true defect detection partial image D41 (step S52: Yes), the defect determination unit 30 outputs the true defect detection partial image D41 and performs the inspection on the object image D1. (Step S53). On the other hand, if the defect determination unit 30 determines that the classification result D5 does not include the true defect detection partial image D41, it is assumed that no defect has been found and the inspection of the object image D1 is ended (step S54). .

In this way, the defect determining section 30 determines whether the defect of the object 90 detected by the defect detecting section 20 is over-detected.

Generally, in a defect detection device that detects a defect based on an object image, over-detection may occur. The defect determination unit 30 includes a learning model 300 for defect determination and trained data 31 for defect determination learned using processed data obtained by processing non-defective product images by the defect detection unit 20. That is, the defect determination unit 30 has a trained defect determination learning model 300 that is trained using processed data obtained by processing non-defective product images by the defect detection unit 20. Therefore, in this inspection apparatus 10, the detection result of the defect detection section 20, which detects a defect based on the object image, is checked by the defect judgment section 30, so that over-detected non-defective objects can be checked. Images can be identified. Therefore, it is possible to reduce over-detection in which non-defective products are judged as defective products.

The trained defect determination learning model 300 of the defect determination unit 30 learns the range of non-defective products using only non-defective product images. Therefore, it is possible to efficiently reduce overdetection even for objects for which defective products are difficult to obtain.

In particular, the defect detection unit 20 of this embodiment uses a learning model 200 for defect detection that reconstructs images, and learned data 21 for defect detection that is trained using only non-defective images. That is, this defect detection unit 20 has a trained defect detection learning model 200 that has been trained using only non-defective product images. In the training of the defect detection learning model 200, increasing the number of non-defective images for learning does not necessarily improve the detection accuracy. Therefore, by providing such a defect determining section 30 after the defect detecting section 20, overdetection can be reduced more efficiently than by relearning the defect detection learning model 200.

As described above, the defect detection section 20 and the defect judgment section 30 that constitute the inspection apparatus 10 according to the first embodiment are constructed by the defect detection learning section 210 and the defect judgment learning section 310. That is, a learning device that learns the inspection apparatus 10 that inspects the object 90 includes a defect detection learning section 210 and a defect judgment learning section 310. The learning method for learning the inspection apparatus 10 that inspects the object 90 includes a defect detection learning step S10 performed by the defect detection learning section 210 and a defect judgment learning step S30 performed by the defect judgment learning section 310. As a result, the trained data 21 for defect detection applied to the learning model 200 for defect detection, which constitutes the trained data set for constructing the inspection apparatus 10 that inspects the object 90, and the learning model 300 for defect judgment. The applied defect determination trained data 31 is generated.

<2. Second embodiment>
FIG. 17 is a diagram showing the functional configuration of the information processing device 120A of the inspection device according to the second embodiment. The information processing device 120A includes a defect detection section 20A, a defect judgment section 30A, and a defect judgment learning section 310A.

The defect detection unit 20A detects defects in the object based on the object image D1 obtained by capturing an image of the object with a camera. The defect detection section 20A includes a template image storage section 25A and a comparison processing section 26A. The template image storage unit 25A stores a plurality of template images that are samples of non-defective images. The comparison processing unit 26A compares the input object image D1 with the template image stored in the template image storage unit 25A, and if it is determined that the item is defective, the comparison processing unit 26A compares the input object image D1 with the template image stored in the template image storage unit 25A. is output to the defect determination section 30A as a defective image D6.

The defect determining section 30A determines whether the defect detected by the defect detecting section 20A is true or false. The defect determination section 30A includes a learning model for defect determination 300A, learned data for defect determination 31A, and a defect determination processing section 32A. The trained data for defect judgment 31A is trained data such as parameters that the defect judgment learning section 310A learns the range of non-defective products by inputting the sample image for learning. Specifically, the defect judgment learning model 300A to which the defect judgment trained data 31A is applied (already learned) is a model in which the range of non-defective products is learned using sample images of non-defective products as input.

The defect judgment processing unit 32A inputs the defective image D6, which is the object image D1 of the object in which the defect detection unit 20A has detected a defect, into the trained defect judgment learning model 31A, and determines whether the defective image D6 is within the range of non-defective products. Defect determination processing is executed to determine whether the defect belongs to the . As a result, a final determination result D7 is output as to whether or not there is a defect.

In the first embodiment, the defect detection unit 20 uses a trained learning model, but the inspection apparatus of the present invention is not limited to this. As in this embodiment, the defect detection section 20A may be a conventional inspection device. Further, in the first embodiment, the image input to the trained defect judgment learning model 31 of the defect judgment unit 30 is processed data processed by the defect detection unit 20, but the present invention is not limited to this. . As in the second embodiment, the image input to the trained defect judgment learning model 31A of the defect judgment unit 30A may be the object image itself of the material in which the defect detection unit 20A has detected a defect.

<3. Modified example>
Although the embodiments have been described above, the present invention is not limited to the above, and various modifications and combinations are possible.

Although all of the image groups used to generate the trained data 21 for defect detection in the first embodiment are non-defective images, the image group may include a small number of defective images. In the image group used to generate the trained data 21 for defect detection, if the number of defective images is sufficiently smaller than the number of non-defective images, the influence of the defective images will be small. Therefore, the defect detection learning unit 210 can generate trained data 21 for defect detection that allows the learning model 200 for defect detection to approximately reproduce a non-defective image. However, when the defect detection trained data 21 is generated using only non-defective images as in the first embodiment, the influence of abnormal features of defective images is effectively eliminated. Therefore, by using the learned defect detection data 21 generated only from non-defective images, the non-defective image can be reconstructed with high precision.

Further, although the image group used to generate the trained data for defect determination 31 in the first embodiment described above were all non-defective images, the image group may include defective images. The image group used to generate the trained data for defect determination 31 may include a group of non-defective images labeled as non-defective products and a group of defective images labeled as defective products.

Although this invention has been described in detail, the above description is illustrative in all aspects, and the invention is not limited thereto. It is understood that countless variations not illustrated can be envisaged without departing from the scope of the invention. The configurations described in each of the above embodiments and modifications can be appropriately combined or omitted as long as they do not contradict each other.

10 Inspection device 20, 20A Defect detection section 21 Learned data for defect detection 30, 30A Defect judgment section 31, 31A Learned data for defect judgment 90 Object 200 Learning model for defect detection 300, 300A Learning model for defect judgment D1 Target Object image D2 Reconstructed image D3 Difference image D4 Defect detection partial image

Claims (8)

An inspection device for inspecting an object,
a defect detection unit that detects defects in the object based on an object image obtained by imaging the object;
a defect determining unit that determines whether the defect detected by the defect detecting unit is true or false;
has
The defect detection section includes:
A learning model for defect detection that uses images as input and output,
A specimen image, which is an image of a learning object, is input to the defect detection learning model, and the model is trained to perform dimension compression and reproduction of the input specimen image to output a reconstructed image of the specimen image. Trained data for defect detection,
including;
The defect detection section includes:
using the defect detection learning model to which the defect detection trained data is applied, dimensionally compressing and reproducing the target object image to generate a reconstructed image of the target object image;
A process of generating a difference image that is a difference between the object image and the reconstructed image;
A process of detecting the presence or absence of a defect based on the difference image;
Run
The defect determination section includes:
a learning model for defect determination that receives the difference image as input;
trained data for defect judgment in which the learning model for defect judgment is made to learn a range of non-defective products by inputting the difference image of the sample image that is an image of the object for learning;
including;
The defect determination section includes:
The defect detection unit inputs the difference image of the target object image of the target object in which a defect has been detected to the defect determination learning model to which the defect determination trained data is applied, and the target object belongs to a range of non-defective products. An inspection device that executes defect determination processing to determine whether or not the defect occurs. The inspection device according to claim 1,
The defect determination trained data is trained data obtained by inputting only the difference image of the sample image of a non-defective product to the defect determination learning model to learn a range of non-defective products. The inspection device according to claim 1 or claim 2,
The trained data for defect detection is trained data obtained by inputting the target object image of a non-defective product into the defect detection learning model and learning to output a reconstructed image of the input target object image. , inspection equipment. An inspection method for inspecting an object,
a) a defect detection step of detecting defects in the object based on an object image obtained by imaging the object;
b) a defect determination step of determining the authenticity of the defect detected in step a);
has
The step a) is
inputting the object image into a learning model for defect detection to which trained data for defect detection is applied, compressing and reproducing the dimension of the object image to generate a reconstructed image of the object image;
generating a difference image that is a difference between the object image and the reconstructed image;
Detecting the presence or absence of a defect based on the difference image;
including;
The learning model for defect detection is a learning model that uses images as input and output,
The trained data for defect detection is obtained by inputting a sample image, which is an image of an object for learning, into the learning model for defect detection, compressing the dimensions of the input sample image, and reproducing the sample image to reconstruct the sample image. This is trained data that has been trained to output images.
In the step b), the difference image of the object image of the object in which the defect was detected in the step a) is input to a learning model for defect judgment to which trained data for defect judgment is applied, and the defect judgment is performed on the object. Determine whether the product falls within the range of good quality,
The trained data for defect judgment is trained data obtained by inputting the difference image of the sample image, which is an image of the object for learning, into a learning model for defect judgment and learning the range of non-defective products. . The testing method according to claim 4,
In the inspection method, the trained data for defect determination is a model obtained by inputting only the difference image of the sample image of a non-defective product to a learning model for defect determination to learn a range of non-defective products. An inspection program for causing a computer to inspect an object,
to the computer;
a) a defect detection step of detecting defects in the object based on an object image obtained by imaging the object;
b) a defect determination step of determining the authenticity of the defect detected in step a);
run the
The step a) is
inputting the object image into a learning model for defect detection to which trained data for defect detection is applied, compressing and reproducing the dimension of the object image to generate a reconstructed image of the object image;
generating a difference image that is a difference between the object image and the reconstructed image;
Detecting the presence or absence of a defect based on the difference image;
including;
The defect detection learning model is a learning model that uses images as input and output,
The trained data for defect detection is obtained by inputting a sample image, which is an image of an object for learning, into the learning model for defect detection, compressing the dimensions of the input sample image, and reproducing the sample image to reconstruct the sample image. This is trained data that has been trained to output images.
In the step b), the computer inputs the difference image of the object image of the object in which the defect was detected in the step a) to a learning model for defect judgment to which trained data for defect judgment is applied; Determining whether the object falls within the range of non-defective products;
The trained data for defect judgment is trained data obtained by inputting the difference image of the sample image, which is an image of the object for learning, into a learning model for defect judgment and learning the range of non-defective products. . A learning device for constructing an inspection device for inspecting an object,
Defect detection in which a sample image, which is an image of a learning object, is input into a learning model for defect detection, and the input sample image is dimensionally compressed and trained to output a reconstructed image of the sample image. a defect detection learning unit that generates trained data for use;
A difference image, which is a difference between a sample image that is an image of a learning object and the reconstructed image generated from the sample image using the defect detection learning model, is input to a defect judgment learning model. , a defect judgment learning unit that generates trained data for defect judgment in which the range of non-defective products is learned;
A learning device equipped with. A learning method for constructing an inspection device for inspecting an object, the method comprising:
x) Defect detection learning in which a sample image, which is an image of a learning object, is input into a defect detection learning model, and the input sample image is trained to perform dimension compression and output a reconstructed image that is reproduced. a defect detection learning process that generates completed data;
y) inputting a difference image, which is the difference between a sample image that is an image of the object for learning, and the reconstructed image generated from the sample image using the learning model for defect detection, into a learning model for defect determination; a defect judgment learning step of generating trained data for defect judgment in which the range of non-defective products is learned;
A learning method that prepares you.

Images