JP2021179907A - Image inspection device, image inspection method, and learned model creation device - Google Patents

Abstract

To provide an image inspection device, an image inspection method, and a learned model creation device that can restore a special pattern and can prevent the generation of a pattern of a defective item.SOLUTION: In an image inspection system in which an image inspection device is connected with a learned model creation device via a communication network, the image inspection device 20 comprises: image creation 26 that creates a plurality of restored divided images by inputting, to a plurality of learned models 55 that have learned to output the restored divided images with one or more of non-defective divided images being divided images of a non-defective object to be inspected as input, inspection divided images being divided images of the object to be inspected; and an inspection unit 270 that inspects the object to be inspected based on the plurality of restored divided images.SELECTED DRAWING: Figure 8

Description

The present invention relates to an image inspection device, an image inspection method, and a trained model generation device.

Conventionally, an image inspection device for inspecting an object based on an image obtained by photographing the object is known.

For example, in Patent Document 1, in order to reconstruct normal image data from feature quantities extracted from a normal image data group in an abnormality determination device that determines an abnormality based on input determination target image data. Reconstruction image data is generated from the feature amount of the judgment target image data using the reconstruction parameter of, and abnormality judgment is performed based on the difference information between the generated reconstructed image data and the judgment target image data. Those having a process execution means for executing an abnormality determination process are described. When the image data to be determined includes image data of a plurality of channels, the abnormality determination device of Patent Document 1 generates reconstructed image data for each channel from the feature amount of the image data of each channel using the reconstruction parameter. Abnormality determination is performed based on the difference information between each generated reconstructed image data and the image data of each channel of the determination target image data.

Japanese Unexamined Patent Publication No. 2018-5773

In Patent Document 1, a reconstructed image is generated from a determination target image by using a trained autoencoder which is a trained model. Here, for example, when a special pattern is locally present in the image of a non-defective inspection object, if the expressive ability of the trained model is low, the special pattern is restored in the image generated by the trained model. There was something I couldn't do. In this case, there is a risk that the image of the inspection target, which is a non-defective product, may be erroneously determined to be defective.

Further, in the image of the inspection object of a non-defective product, if the pattern of the non-defective product at one position or part is defective at another position or part, the pattern of the defective product is generated in the image generated by the trained model. , I sometimes overlooked the inspection target of defective products.

Therefore, it is an object of the present invention to provide an image inspection device, an image inspection method, and a trained model generation device that can restore a special pattern and suppress the generation of a defective pattern. I will do it.

The image inspection apparatus according to one aspect of the present disclosure is a plurality of trained models trained to output a restored divided image by inputting one or a plurality of non-defective divided images which are divided images of good products to be inspected. An image generation unit that inputs an inspection divided image, which is a divided image of an inspection object, to generate a plurality of restored divided images, and an inspection unit that inspects an inspection object based on a plurality of restored divided images. Be prepared.

According to this aspect, the inspection divided image is input to each of a plurality of trained models trained to output the restored divided image by inputting one or a plurality of non-defective divided images, and the plurality of restored divided images are input. Generate. As a result, each trained model can learn a pattern peculiar to each good product divided image, so that a local special pattern included in the good product divided image can be restored in the generated divided restored image. .. Further, even when a pattern that is a good product at a certain position or part is a defective product at another position or part in an image of another good product to be inspected, each trained model is in the good product image of the inspection target. Since it is possible to learn different judgment criteria depending on the position, it is possible to suppress the generation of defective patterns in the generated split-restored image. Therefore, the inspection accuracy of the inspection target can be improved by inspecting the inspection target TA based on a plurality of divided restored images in which the special pattern is restored and the generation of the defective product pattern is suppressed.

In the above-described embodiment, the inspection unit may generate a restored image based on a plurality of restored divided images, and inspect the inspection object based on the comparison between the inspection image of the inspection object and the restored image.

According to this aspect, a restored image is generated based on a plurality of restored divided images, and the inspection object is inspected based on the comparison between the restored image and the inspection image of the inspection object. As a result, inspection with high inspection accuracy can be easily realized.

In the above-described embodiment, the inspection unit may inspect the inspection object based on the difference between the restored image and the inspection image.

According to this aspect, the inspection object is inspected based on the difference between the restored image and the inspection image. As a result, inspection with high inspection accuracy can be realized more easily.

In the above-described embodiment, a learning unit that trains a learning model for each one or a plurality of non-defective divided images and generates a plurality of trained models may be further provided.

According to this aspect, a learning unit that trains a learning model for each one or a plurality of non-defective divided images and generates a plurality of trained models is further provided. This makes it possible to obtain a plurality of trained models without a trained model generator.

In the above-described embodiment, the learning unit may train a learning model by inputting one or a plurality of non-defective divided images subjected to data augmentation to generate a plurality of trained models.

According to this aspect, a learning model is trained by inputting one or a plurality of non-defective divided images to which data augmentation is applied, and a plurality of trained models are generated. As a result, a limited number of non-defective divided images can be increased, and each trained model can have a robust design with enhanced robustness against variations in the non-defective divided images.

In the embodiments described above, the data augmentation applies at least one of translation, rotation, enlargement, reduction, left / right inversion, upside down, and filtering to one or more non-defective split images. It may include that.

According to this aspect, data augmentation applies at least one of translation, rotation, enlargement, reduction, left / right inversion, upside down, and filtering to one or more non-defective split images. Including doing. As a result, variations of the non-defective divided image can be easily obtained.

In the above-described embodiment, a division portion for dividing the inspection image of the inspection object into a plurality of inspection division images may be further provided.

According to this aspect, a division portion for dividing the inspection image of the inspection object into a plurality of inspection division images is further provided. Thereby, the inspection divided image can be easily obtained.

In the above-described embodiment, an image pickup unit for acquiring an inspection image of an inspection object may be further provided.

According to this aspect, an image pickup unit for acquiring an inspection image of an inspection object is further provided. Thereby, the inspection image can be easily obtained.

In the above-described embodiment, a storage unit for storing a plurality of trained models may be further provided.

According to this aspect, a storage unit for storing a plurality of trained models is further provided. As a result, each trained model can be easily read out.

In the image inspection method according to another aspect of the present disclosure, a plurality of learned images which have been trained to output a restored divided image by inputting one or a plurality of non-defective divided images which are divided images of a non-defective product to be inspected. The model includes a step of inputting an inspection divided image, which is a divided image of the inspection object, to generate a plurality of restored divided images, and a step of inspecting the inspection object based on the plurality of restored divided images.

According to this aspect, the inspection divided image is input to each of a plurality of trained models trained to output the restored divided image by inputting one or a plurality of non-defective divided images, and the plurality of restored divided images are input. Generate. As a result, each trained model can learn a pattern peculiar to each good product divided image, so that a local special pattern included in the good product divided image can be restored in the generated divided restored image. .. Further, even when a pattern that is a good product at a certain position or part is a defective product at another position or part in an image of another good product to be inspected, each trained model is in the good product image of the inspection target. Since it is possible to learn different judgment criteria depending on the position, it is possible to suppress the generation of defective patterns in the generated split-restored image. Therefore, the inspection accuracy of the inspection target can be improved by inspecting the inspection target TA based on a plurality of divided restored images in which the special pattern is restored and the generation of the defective product pattern is suppressed.

The trained model generator according to another aspect of the present disclosure has been trained to output a restored divided image by inputting one or a plurality of non-defective product divided images which are divided images of non-defective products to be inspected. It has a model generator that generates a trained model.

According to this aspect, a plurality of trained models trained to output a restored divided image by inputting one or a plurality of non-defective divided images are generated. As a result, each trained model can learn a pattern peculiar to each good product divided image, so that a local special pattern included in the good product divided image can be restored in the output divided restored image. .. Further, even when a pattern that is a good product at a certain position or part is a defective product at another position or part in a good product image, each trained model can learn different judgment criteria depending on the position in the good product image. Therefore, it is possible to suppress the generation of defective patterns in the divided and restored image that is the output. Therefore, for example, by inputting an inspection divided image into each trained model and generating a divided and restored image, it is possible to restore a local special pattern in the divided and restored image and suppress the generation of a defective pattern. can do.

According to the present invention, it is possible to restore a special pattern and suppress the generation of a defective pattern.

FIG. 1 is a configuration diagram illustrating a schematic configuration of an image inspection system according to an embodiment. FIG. 2 is a configuration diagram showing a physical configuration of a trained model generation device and an image inspection device in one embodiment. FIG. 3 is a configuration diagram showing a configuration of a functional block of the trained model generator in one embodiment. FIG. 4 is a conceptual diagram for explaining the generation of a non-defective product divided image by the learning image generation unit shown in FIG. FIG. 5 is a conceptual diagram for explaining the learning model used by the model generation unit shown in FIG. FIG. 6 is a conceptual diagram for explaining an example of generation of a plurality of trained models by the model generation unit shown in FIG. FIG. 7 is a conceptual diagram for explaining another example of generation of a plurality of trained models by the model generation unit shown in FIG. FIG. 8 is a configuration diagram showing a configuration of a functional block of an image inspection device according to an embodiment. FIG. 9 is a conceptual diagram for explaining an example of processing by the division unit, the image generation unit, and the inspection unit shown in FIG. FIG. 9 is a conceptual diagram for explaining another example of processing by the division unit, the image generation unit, and the inspection unit shown in FIG. FIG. 11 is a diagram showing an example of a non-defective product image. FIG. 12 is a conceptual diagram for explaining the generation of the difference image by the inspection unit shown in FIG. FIG. 13 is a diagram showing an example of an inspection image. FIG. 14 is a diagram showing an example of the restored image. FIG. 15 is a diagram showing an example of a difference image. FIG. 16 is a flowchart for explaining an example of the trained model generation process performed by the trained model generation device in one embodiment. FIG. 17 is a flowchart for explaining an example of the image inspection process performed by the image inspection apparatus in one embodiment.

An embodiment of the present invention will be described below. In the description of the following drawings, the same or similar parts are represented by the same or similar reference numerals. However, the drawings are schematic. Therefore, the specific dimensions and the like should be determined in light of the following explanations. In addition, it goes without saying that parts having different dimensional relationships and ratios are included between the drawings. Furthermore, the technical scope of the present invention should not be construed as being limited to the embodiment.

First, the configuration of the image inspection system according to the embodiment will be described with reference to FIG. 1. FIG. 1 is a configuration diagram illustrating a schematic configuration of an image inspection system 1 according to an embodiment.

As shown in FIG. 1, the image inspection system 1 includes an image inspection device 20 and a lighting IL. The image inspection device 20 is connected to the trained model generation device 10 via the communication network NW. The illumination IL irradiates the inspection object TA with light L. The image inspection device 20 photographs the reflected light R and inspects the inspection target TA based on the image of the inspection target TA (hereinafter, also referred to as “inspection image”). The trained model generation device 10 generates a trained model used by the image inspection device 20 for inspection.

Next, with reference to FIG. 2, the physical configuration of the trained model generation device and the image inspection device according to the embodiment will be described. FIG. 2 is a configuration diagram showing a physical configuration of the trained model generation device 10 and the image inspection device 20 in one embodiment.

As shown in FIG. 2, the trained model generation device 10 and the image inspection device 20 include a processor 31, a memory 32, a storage device 33, a communication device 34, an input device 35, and an output device 36, respectively. To prepare for. Each of these configurations is connected to each other via a bus so that data can be transmitted and received.

In the example shown in FIG. 2, the case where the trained model generation device 10 and the image inspection device 20 are each composed of one computer will be described, but the present invention is not limited thereto. The trained model generation device 10 and the image inspection device 20 may be realized by combining a plurality of computers, respectively. Further, the configuration shown in FIG. 2 is an example, and the trained model generation device 10 and the image inspection device 20 may each have a configuration other than these, or may not include a part of these configurations. good.

The processor 31 is configured to control the operation of each part of the trained model generation device 10 and the image inspection device 20. The processor 31 is, for example, a CPU (Central Processing Unit), a DSP (Digital Signal Processor), an ASIC (Application Specific Integrated Circuit), a PLD (Programmable Logic Device), and a PLD (Programmable Logic Device). It is configured to include an integrated circuit such as a-Chip).

The memory 32 and the storage device 33 are configured to store programs, data, and the like, respectively. The memory 32 is composed of, for example, a ROM (Read Only Memory), an EPROM (Erasable Programmable ROM), an EEPROM (Electrically Erasable Programmary ROM), and / or a RAM (Random Access Memory). The storage device 33 is composed of, for example, a storage such as an HDD (Hard Disk Drive), an SSD (Solid State Drive) and / or an eMMC (embedded MultiMediaCard).

The communication device 34 is configured to communicate via at least one of a wired and wireless network. The communication device 34 includes, for example, a network card, a communication module, an interface for connecting to other devices, and the like.

The input device 35 is configured so that information can be input by a user's operation. The input device 35 includes, for example, a keyboard, a touch panel, a mouse, a pointing device, and / or a microphone.

The output device 36 is configured to output information. The output device 36 includes, for example, a liquid crystal display, an EL (Electro Luminescence) display, a plasma display, a display device such as an LCD (Liquid Crystal Display), and / or a speaker.

Next, a functional block of the trained model generator according to one embodiment will be described with reference to FIGS. 3 to 7. FIG. 3 is a configuration diagram showing a configuration of a functional block of the trained model generation device 10 in one embodiment. FIG. 4 is a conceptual diagram for explaining the generation of a non-defective product divided image by the learning image generation unit 131 shown in FIG. FIG. 5 is a conceptual diagram for explaining the learning model 50 used by the model generation unit 135 shown in FIG. FIG. 6 is a conceptual diagram for explaining an example of generation of a plurality of trained models 55 by the model generation unit 135 shown in FIG. FIG. 7 is a conceptual diagram for explaining another example of generation of a plurality of trained models 55 by the model generation unit 135 shown in FIG.

As shown in FIG. 3, the trained model generation device 10 includes a communication unit 110, a storage unit 120, and a learning unit 130.

The communication unit 110 is configured to be able to transmit and receive various types of information. The communication unit 110 transmits, for example, the learned model 55, which will be described later, to the image inspection device 20 via the communication network NW. Further, the communication unit 110 receives a good product image from another device, for example, via the communication network NW. That is, the communication unit 110 acquires a non-defective product image, which is a non-defective product image. A plurality of non-defective image images acquired by the communication unit 110 (hereinafter, one of them is also referred to as a “non-defective image 40” when they are not distinguished) are written and stored in the storage unit 120.

The storage unit 120 is configured to store various types of information. The storage unit 120 includes, for example, a non-defective image 40, a plurality of learning images (hereinafter, one of them is also referred to as a “learning image 124” when they are not distinguished), and a plurality of trained models (hereinafter, each). When no distinction is made, one of them is also referred to as "trained model 55").

The learning unit 130 is for training a learning model to generate a trained model. The learning unit 130 includes, for example, a learning image generation unit 131 and a model generation unit 135.

The learning image generation unit 131 is configured to generate a learning image 124. The training image 124 is an image used to generate the trained model 55. The learning image 124 is, for example, an image obtained by dividing one good product image into a plurality of images (hereinafter referred to as “good product divided image”).

More specifically, the learning image generation unit 131 reads out one of the plurality of non-defective product images 40 stored in the storage unit 120, divides the non-defective product image 40 into a plurality of non-defective product divided images, and is a learning image. To generate.

As shown in FIG. 4, the learning image generation unit 131 generates 16 non-defective product divided images by, for example, dividing the non-defective product image 40 into four vertically and horizontally, respectively. It is sufficient that the non-defective product image 40 is divided into two or more non-defective product divided images, and may be divided into 2 to 15 non-defective product divided images or 17 or more non-defective product divided images. ..

Further, the learning image generation unit 131 may apply data augmentation to one or a plurality of non-defective divided images, and generate the image to which the data augmentation has been applied as the learning image 124.

Data augmentation involves applying at least one of translation, rotation, enlargement, reduction, left / right inversion, upside down, and filtering to one or more non-defective split images. The filter used for the filter processing may be, for example, a filter that converts the brightness of the non-defective divided image, a smoothing filter that removes noise, or a filter that extracts edges, and any filter is used. be able to.

The data augmentation is not limited to the above-mentioned method. For example, the data augmentation may perform projective transformation or add random noise to one or more non-defective divided images.

The model generation unit 135 is configured to train a learning model for each one or a plurality of non-defective divided images and generate a plurality of trained models 55. Each trained model 55 is, for example, a model that inputs one or a plurality of non-defective divided images and outputs a restored divided image. Alternatively, each trained model 55 may be a model that inputs one or a plurality of non-defective divided images to which data augmentation is applied and outputs a restored divided image. Hereinafter, unless otherwise specified, each trained model 55 will be described as being generated by inputting one or a plurality of non-defective divided images. In this case, the restored divided image is an image for restoring a non-defective divided image.

As shown in FIG. 5, the learning model 50 used by the model generator 135 to generate the trained model 55 is, for example, one of the neural networks and one of the unsupervised machine learning methods. It is an encoder. The learning model 50 includes an input layer 501, an output layer 505, and an intermediate layer 503 arranged between the input layer 501 and the output layer 505.

The learning model 50 is not limited to the autoencoder. The learning model 50 may be, for example, a model using a PCA (Principal Component Analysis). Further, the intermediate layer 503 of the learning model 50 is not limited to the case of one layer, and may be two or more layers.

When the input image is input to the input layer 501, the dimension of the input image is reduced in the intermediate layer 503 and the feature vector is extracted. Then, the output image is output by returning the dimension in the output layer 505.

The learning model 50 learns weights (also referred to as coefficients) so that the difference between the input image and the output image is minimized. More specifically, each node represented by a circular shape is uniquely weighted at each edge represented by a line, and the weighted value is input to the node of the next layer. By learning the weights in this weighting, the difference between each pixel included in the input image and each corresponding pixel included in the output image is minimized.

In the present embodiment, the input image is, for example, a non-defective divided image, and the output image is a restored divided image. As shown in FIG. 6, for example, when the trained model 55 for the good product divided image 400 located at the upper left in the good product image 40 is generated, the corresponding learning model 50A has the upper left of each of the plurality of good product images 40. The non-defective product divided image 400 is input as an input image. Then, the learning model 50A learns each weight so as to output the restored divided image for restoring the good product divided image 400 on the upper left as an output image. As a result, the trained model 55A that restores the non-defective divided image 400 is generated. In this way, by training the corresponding learning models 50A, 50B, 50C, ... For each non-defective product divided image 400, 402, 404, ... In the non-defective product image 400, a plurality of model generation units 135 are provided. The trained models 55A, 55B, 55C, ... Of are generated.

The learning model 50 is not limited to the case where a non-defective product divided image at one position is used as an input image. In other words, the number of divisions of the non-defective image 40, for example, the number of divisions m (m is an integer of 2 or more), the number of learning models 50 used for training, and the number of trained models 55 generated, for example, the number of generations n (n). Is an integer of 2 or more) is not limited to the same number. The learning model 50 may be trained using a plurality of non-defective product divided images, for example, a non-defective product divided image 400 and a non-defective product divided image 402 as input images. In this case, a trained model 55 that restores the divided images of the two areas of the good product divided image 400 and the good product divided image 402 is generated.

Further, as shown in FIG. 7, among a plurality of non-defective product divided images 400', 402', 404', ... If they are similar, the learning model 50 used for each may be integrated. That is, the non-defective product divided image 402'and the non-defective product divided image 404'that are similar to each other are input to the same learning model 50B, and as a result of learning, one trained model 55B'is generated. In this case, the generated number n of the trained model 50 is smaller than the divided number m of the good product image 40 (generated number n <divided number m). It should be noted that the determination as to whether or not the two non-defective divided images are similar may be performed based on, for example, whether or not the degree of matching is equal to or higher than a predetermined threshold value.

In this way, by generating a plurality of trained models 55 trained so as to output the restored divided image by inputting one or a plurality of non-defective divided images, each trained model 55 is described as described later. Since the pattern peculiar to each good product divided image can be learned, the local special pattern included in the good product divided image can be restored in the output divided restored image. Further, even when a pattern that is a good product at a certain position or part is a defective product at another position or part in a good product image, as will be described later, each trained model 55 has different judgment criteria depending on the position in the good product image. Is possible, so that it is possible to suppress the generation of defective patterns in the output divided and restored image.

At least one of the learning image generation unit 131 and the model generation unit 135 may be realized by the processor 31 executing the program stored in the storage device 33. When executing a program, the program may be stored in a storage medium. The storage medium in which the program is stored may be a non-transitory computer readable medium that can be read by a computer. The non-temporary storage medium is not particularly limited, but may be, for example, a storage medium such as a USB memory or a CD-ROM (Compact Disc ROM).

Next, the functional block of the image inspection apparatus according to one embodiment will be described with reference to FIGS. 8 to 12. FIG. 8 is a configuration diagram showing a configuration of a functional block of the image inspection device 20 in one embodiment. FIG. 9 is a conceptual diagram for explaining an example of processing by the division unit 250, the image generation unit 260, and the inspection unit 270 shown in FIG. FIG. 10 is a conceptual diagram for explaining another example of processing by the division unit 250, the image generation unit 260, and the inspection unit 270 shown in FIG. FIG. 11 is a diagram showing an example of a non-defective product image. FIG. 12 is a conceptual diagram for explaining the generation of the difference image by the inspection unit 270 shown in FIG. FIG. 13 is a diagram showing an example of an inspection image. FIG. 14 is a diagram showing an example of the restored image. FIG. 15 is a diagram showing an example of a difference image.

As shown in FIG. 8, the image inspection device 20 includes a communication unit 210, a storage unit 220, a learning unit 230, an image pickup unit 240, a division unit 250, an image generation unit 260, and an inspection unit 270. Be prepared.

The communication unit 210 is configured to be able to transmit and receive various types of information. The communication unit 210 receives a plurality of trained models 55 from the trained model generator 10 via, for example, the communication network NW. Further, the communication unit 210 receives a plurality of learning images 124 from the trained model generation device 10 or other devices via, for example, the communication network NW. The plurality of received learning images 124 and the plurality of trained models 55 are written and stored in the storage unit 220. The communication unit 210 may receive only one of the plurality of learning images 124 and the plurality of trained models 55. When the communication unit 210 receives only a plurality of learning images 124, the learning unit 230, which will be described later, uses the plurality of learning images 124 to generate a plurality of trained models 55.

The storage unit 220 is configured to store various types of information. The storage unit 220 stores, for example, a plurality of learning images 124 and a plurality of trained models trained models 55. As described above, by including the storage unit 220 that stores the plurality of trained models 55, each trained model can be easily read out.

The learning unit 230 is configured to train a learning model for each one or a plurality of non-defective divided images and generate a plurality of trained models 55. Each trained model 55 is a model that inputs one or a plurality of non-defective divided images and outputs a restored divided image. The generated plurality of trained models 55 are written and stored in the storage unit 220. Since the method of training the learning model is the same as the description of the model generation unit 135 in the trained model generation device 10, the description thereof will be omitted.

In this way, by providing the learning unit 230 that learns the learning model for each one or a plurality of non-defective divided images and generates the plurality of trained models 55, a plurality of trained model generators 10 are provided without the trained model generation device 10. You can get a trained model.

Further, when each of the plurality of learning images stored in the storage unit 220 is a non-defective divided image, the learning unit 230 performs data augmentation on one or a plurality of non-defective divided images, and the data augmentation is performed. A learning model may be trained by using the applied one as an input, and a plurality of trained models 55 may be generated. As a result, a limited number of non-defective divided images can be increased, and each trained model 55 can have a robust design with enhanced robustness against variations in the non-defective divided images.

As mentioned above, data augmentation applies at least one of translation, rotation, enlargement, reduction, left / right inversion, upside down, and filtering to one or more non-defective split images. Including that. The filter used for the filter processing may be, for example, a filter that converts the brightness of the non-defective divided image, a smoothing filter that removes noise, or a filter that extracts edges, and any filter is used. be able to. In this way, data augmentation applies at least one of translation, rotation, enlargement, reduction, left-right inversion, up-down inversion, and filtering to one or more non-defective split images. By including the above, variations of the non-defective divided image can be easily obtained.

The data augmentation is not limited to the above-mentioned method. For example, the data augmentation may perform projective transformation or add random noise to one or more non-defective divided images.

The image pickup unit 240 is for acquiring an inspection image of the inspection object TA. The image pickup unit 240 includes an image pickup device such as a camera. The imaging unit 240 of the present embodiment receives the reflected light R from the inspection object TA and acquires an inspection image. Then, the imaging unit 240 outputs the acquired inspection image to the division unit 250 and the inspection unit 270. In this way, by acquiring the inspection image of the inspection target TA, the inspection image can be easily obtained.

The division unit 250 is configured to divide the inspection image of the inspection object TA into a plurality of inspection division images. More specifically, the division unit 250 divides the inspection image by the same method as the division of the good product image in the trained model generation device 10. Specifically, the division unit 250 divides the inspection image of the inspection object TA into four vertically and horizontally to generate 16 inspection division images. Then, the division unit 250 outputs the generated inspection division image to the image generation unit 260. In this way, by dividing the inspection image of the inspection target TA into a plurality of inspection division images, the inspection division image can be easily obtained.

The image generation unit 260 is configured to input inspection divided images into a plurality of trained models 55 and generate a plurality of restored divided images.

As shown in FIG. 9, the inspection image of the inspection object TA acquired by the imaging unit 240 is divided into a plurality of inspection division images 420, 422, 424, ... By the division unit 250. The image generation unit 260 inputs each inspection divided image 420, 422, 424, ... To the corresponding trained models 55A, 55B, 55C, ... Of the plurality of trained models 55. Each trained model 55A, 55B, 55C, ... Outputs an image by outputting a restored divided image 440, 442, 444, ... Corresponding to each inspection divided image 420, 422, 424, ... The generation unit 260 generates a plurality of restored divided images 440, 442, 444, ....

Further, the inspection unit 270, which will be described later, integrates the generated plurality of restored divided images 440, 442, 444, ..., And the restored image 44 having the same vertical and horizontal sizes (number of pixels) as the inspection image 42. To generate.

As described above, when a plurality of non-defective divided images similar to each other are integrated into one learning model 50 to generate a trained model 55, as shown in FIG. 10, the image generation unit 260 has the inspection image 42 in the inspection image 42. The inspection divided images at the positions corresponding to the plurality of non-defective divided images similar to each other, for example, the inspection divided image 422 and the inspection divided image 424, are input to the corresponding trained model 55B', respectively. The trained model 55B'outputs the restored divided image 442 corresponding to the inspection divided image 422 and outputs the restored divided image 444 corresponding to the inspection divided image 424. Each trained model 55A, 55B', ... Outputs each restored divided image 440, 442, 444, ..., So that the image generation unit 260 can generate a plurality of restored divided images 440, 442, 444. ... is generated. The restored divided image 442 output by the trained model 55B'is an image corresponding to the inspection divided image 422, and the restored divided image 444 output by the trained model 55B'is an image corresponding to the inspection divided image 424. .. That is, if the inspection divided image 422 and the inspection divided image 424 are the same, the restored divided image 442 and the restored divided image 444 are the same image, but if there is a difference between the inspection divided image 422 and the inspection divided image 424, the restored divided image 442 and the restored divided image 444 are the same image. , The restored divided image 442 and the restored divided image 444 may be different images.

Here, as shown in FIG. 11, the non-defective product image 60 of the non-defective product TA to be inspected is divided into, for example, six non-defective product divided images 600, 602, 604, 606, 608, and 610. Of these six non-defective divided images, the non-defective divided images 602,606 and 608 contain patterns similar to each other. Further, the non-defective product divided image 604 contains a special pattern different from other non-defective product divided images.

Unlike the plurality of trained models 55A, 55B, 55C, ... Of the present embodiment, when one trained model is generated using these six non-defective divided images, the trained model is output. Although the pattern of the good product divided image 602, 604 or 608 can be reproduced in the restored divided image, if the expressive ability of the trained model is low, the special pattern included in the good product divided image 604 may not be restored.

On the other hand, in the image inspection device 20 of the present embodiment, the non-defective product divided image 600 is applied to a plurality of trained models 55 trained to output the restored divided image by inputting one or a plurality of non-defective divided images. Input 602, 604, 606, 608, and 610, respectively, to generate a plurality of restored divided images. As a result, each trained model 55 can learn the patterns peculiar to each good product divided image 600, 602, 604, 606, 608, and 610. Therefore, in the generated divided restored image, the good product divided image 604 Local special patterns contained in can be restored.

Further, in a non-defective product image of another non-defective product TA, a pattern that is a non-defective product at a certain position or part may be a defective product at another position or part. Even in such a case, in the image inspection device 20 of the present embodiment, each trained model 55 can learn different judgment criteria depending on the position of the inspection target TA in the non-defective image, so that the generated split restoration can be performed. It is possible to suppress the generation of defective patterns in the image.

Returning to the description of FIG. 8, the inspection unit 270 is configured to inspect the inspection target TA based on the plurality of restored divided images generated by the image generation unit 260. As described above, the special pattern is restored in each restored divided image generated by the image generation unit 260, and the generation of the defective pattern is suppressed. Therefore, by inspecting the inspection target TA based on these plurality of divided and restored images, the inspection accuracy of the inspection target TA can be improved.

More specifically, the inspection unit 270 generates a restored image based on a plurality of restored divided images, and inspects the restored object TA based on the comparison between the restored image and the inspection image of the inspection object TA. It is configured in. As a result, inspection with high inspection accuracy can be easily realized.

Specifically, the inspection unit 270 inspects the inspection target TA based on the difference between the restored image and the inspection image. As a result, inspection with high inspection accuracy can be realized more easily.

As shown in FIG. 12, the inspection unit 270 obtains the difference between the inspection image 42 and the restored image 44 and generates the difference image 46, for example. Then, the inspection unit 270 determines whether or not the inspection target TA is a non-defective product based on the difference image 46.

As shown in FIG. 13, the inspection image 72 includes, for example, a linear defect image 700. The defect image is an image of the defect of the inspection object.

On the other hand, as shown in FIG. 12, in the restored image 74, since the non-defective image is restored, the defective image is removed.

Therefore, as shown in FIG. 15, the difference image 76 showing the difference between the inspection image 72 of the inspection target TA and the restored image 74 mainly includes the defect image 760. In the present embodiment, the inspection unit 270 determines whether or not the inspection target TA is a non-defective product based on the difference image 76 including the defect image 760. For example, when the size of the defective image 760 exceeds a predetermined threshold value, the inspection unit 270 determines that the inspection target TA is not a non-defective product, that is, a defective product.

Alternatively, the inspection unit 270 may detect the defect of the inspection target TA based on the presence / absence of the defect image 760 included in the difference image 76.

At least one of the learning unit 230, the dividing unit 250, the image generation unit 260, and the inspection unit 270 may be realized by the processor 31 executing a program stored in the storage device 33. When executing a program, the program may be stored in a storage medium. The storage medium in which the program is stored may be a non-temporary storage medium that can be read by a computer. The non-temporary storage medium is not particularly limited, but may be, for example, a storage medium such as a USB memory or a CD-ROM.

Next, with reference to FIG. 16, a processing procedure performed by the trained model generator according to the embodiment will be described. FIG. 16 is a flowchart for explaining an example of the trained model generation process S100 performed by the trained model generation device 10 in one embodiment.

As shown in FIG. 16, first, the communication unit 110 acquires a plurality of non-defective image 40s via the communication network NW (step S101). The acquired plurality of non-defective image 40s are stored in the storage unit 120.

Next, the learning image generation unit 131 reads out a plurality of good product images 40 from the storage unit 120, and generates a plurality of learning images 124 based on the plurality of good product images 40 (step S102). As described above, the learning image 124 may be a non-defective divided image, or may be one or a plurality of non-defective divided images subjected to data augmentation.

Next, the model generation unit 135 takes the plurality of learning images 124 generated in step S102 as inputs, and generates a plurality of trained models 55 trained so as to output the restored divided images. (Step S103). The generated plurality of trained models 55 are stored in the storage unit 120.

Next, the communication unit 110 transmits the plurality of learned models 55 generated in step S103 to the image inspection device 20 via the communication network NW (step S104). As a result, the image inspection device 20 can use a plurality of trained models generated by the trained model generation device 10.

After step S104, the trained model generation device 10 ends the trained model generation process S100.

Next, with reference to FIG. 17, a processing procedure performed by the image inspection apparatus according to the embodiment will be described. FIG. 17 is a flowchart for explaining an example of the image inspection process S200 performed by the image inspection apparatus 20 in one embodiment.

In the following example, it is assumed that the communication unit 210 receives a plurality of trained models 55 from the trained model generation device 10, and the storage unit 220 stores the plurality of trained models 55.

As shown in FIG. 17, first, the imaging unit 240 acquires an inspection image of the inspection target TA (step S201). The acquired inspection image is output to the division unit 250 and the inspection unit 270.

Next, the division unit 250 divides the inspection image acquired in step S201 and generates a plurality of inspection division images (step S202). The generated plurality of inspection divided images are output to the image generation unit 260.

Next, the image generation unit 260 reads out the plurality of trained models 55 stored in advance in the storage unit 220, inputs the plurality of inspection divided images generated in step S207 into the plurality of trained models 55, respectively. Generate a plurality of restored divided images (step S203). The generated plurality of restored divided images are output to the inspection unit 270.

Next, the inspection unit 270 integrates the plurality of restored divided images generated in step S203 to generate a restored image (step S204). Then, the inspection unit 270 calculates the difference between the inspection image acquired in step S201 and the restored image generated in step S204, and generates a difference image (step S205).

Next, the inspection unit 270 inspects the inspection target TA based on the difference image generated in step S205 (step S206).

After step S206, the image inspection device 20 ends the image inspection process S200.

The order of the sequences and flowcharts described in the present embodiment may be changed as long as there is no contradiction in the processing.

The exemplary embodiments of the present invention have been described above. According to the image inspection device 20 and the image inspection method in the present embodiment, the non-defective product divided image is applied to a plurality of trained models 55 trained to output the restored divided image by inputting one or a plurality of non-defective divided images. Input 600, 602, 604, 606, 608, and 610, respectively, to generate a plurality of restored divided images. As a result, each trained model 55 can learn the patterns peculiar to each good product divided image 600, 602, 604, 606, 608, and 610. Therefore, in the generated divided restored image, the good product divided image 604 Local special patterns contained in can be restored. Further, in the image of the inspection target TA of another good product, even when the pattern that is a good product at a certain position or part is a defective product at another position or part, each trained model 55 is the inspection target TA. Since it is possible to learn different judgment criteria depending on the position in the inspection image, it is possible to suppress the generation of defective patterns in the generated divided restoration image. Therefore, the inspection accuracy of the inspection target TA can be improved by inspecting the inspection target TA based on a plurality of divided restored images in which the special pattern is restored and the generation of the defective product pattern is suppressed.

Further, according to the trained model generation device 10 in the present embodiment, a plurality of trained models 55 trained so as to output a restored divided image by inputting one or a plurality of non-defective divided images are generated. As a result, each trained model 55 can learn a pattern peculiar to each good product divided image, so that the local special pattern included in the good product divided image can be restored in the output divided restored image. can. Further, even when a pattern that is a good product at a certain position or part is a defective product at another position or part in a good product image, each trained model 55 may learn different judgment criteria depending on the position in the good product image. Therefore, it is possible to suppress the generation of defective patterns in the divided and restored image that is the output. Therefore, for example, by inputting an inspection divided image into each trained model 55 and generating a divided and restored image, a local special pattern can be restored in the divided and restored image, and a defective pattern can be generated. It can be suppressed.

The embodiments described above are for facilitating the understanding of the present invention, and are not for limiting the interpretation of the present invention. Each element included in the embodiment and its arrangement, material, condition, shape, size, and the like are not limited to those exemplified, and can be appropriately changed. Further, it is possible to partially replace or combine the configurations shown in different embodiments.

[Appendix 1]
A plurality of trained models (55) trained to output a restored divided image by inputting one or a plurality of non-defective divided images, which are divided images of a non-defective product inspection object (TA), are subjected to an inspection object (inspection object (55). An image generation unit (260) that inputs inspection divided images, which are divided images of TA), and generates a plurality of restored divided images, and
It comprises an inspection unit (270) for inspecting an inspection object (TA) based on a plurality of restored divided images.
Image inspection device (20).
[Appendix 10]
A plurality of trained models (55) trained to output a restored divided image by inputting one or a plurality of non-defective divided images, which are divided images of a non-defective inspection object (TA), are subjected to an inspection object (inspection object (55). A step of inputting each inspection divided image which is a divided image of TA) to generate a plurality of restored divided images, and
A step of inspecting an object to be inspected (TA) based on multiple restored fragmented images, including.
Image inspection method.
[Appendix 11]
A model generation unit that generates a plurality of trained models (55) trained to output a restored divided image by inputting one or a plurality of non-defective divided images which are divided images of a non-defective product inspection object (TA). (135)
Trained model generator (10).

1 ... Image inspection system, 10 ... Trained model generator, 20 ... Image inspection device, 31 ... Processor, 32 ... Memory, 33 ... Storage device, 34 ... Communication device, 35 ... Input device, 36 ... Output device, 40 ... Good product image, 42 ... Inspection image, 44 ... Restored image, 46 ... Difference image, 50 ... Learning model, 55, 55A, 55B, 55C ... Trained model, 60 ... Good product image, 72 ... Inspection image, 74 ... Restored image, 76 ... difference image, 110 ... communication unit, 120 ... storage unit, 124 ... learning image, 130 ... learning unit, 131 ... learning image generation unit, 135 ... model generation unit, 210 ... communication unit, 220 ... storage unit, 230 ... learning unit, 240 ... imaging unit, 250 ... division unit, 260 ... image generation unit, 270 ... inspection unit, 400, 402, 404 ... good product division image, 420, 422, 424 ... inspection division image, 440,442. 444 ... restored divided image, 501 ... input layer, 503 ... intermediate layer, 505 ... output layer, 600,602,604,606,608,610 ... good product divided image, 700 ... defective image, 760 ... defective image, IL ... lighting , L ... light, NW ... communication network, R ... reflected light, S100 ... learned model generation processing, S200 ... image inspection processing, TA ... inspection target.

Claims (11)

Inspection division, which is a divided image of an inspection object, is applied to a plurality of trained models trained to output a restored divided image by inputting one or a plurality of non-defective divided images, which are divided images of a non-defective product. An image generator that inputs an image and generates a plurality of the restored divided images,
It comprises an inspection unit for inspecting the inspection object based on the plurality of restored divided images.
Image inspection equipment. The inspection unit generates a restored image based on the plurality of restored divided images, and inspects the inspection object based on a comparison between the inspection image of the inspection object and the restored image.
The image inspection apparatus according to claim 1. The inspection unit determines whether or not the inspection object is a non-defective product based on the difference between the inspection image and the restored image.
The image inspection apparatus according to claim 2. A learning unit that trains a learning model for each one or a plurality of the non-defective divided images and generates the plurality of trained models is further provided.
The image inspection apparatus according to any one of claims 1 to 3. The learning unit trains the learning model by inputting one or a plurality of the non-defective divided images to which data augmentation is applied, and generates the plurality of trained models.
The image inspection apparatus according to claim 4. The data augmentation comprises applying at least one of translation, rotation, enlargement, reduction, left / right inversion, upside down, and filtering to the non-defective split image.
The image inspection apparatus according to claim 5. A division portion for dividing the inspection image of the inspection object into the plurality of inspection division images is further provided.
The image inspection apparatus according to any one of claims 1 to 6. Further provided with an imaging unit for acquiring an inspection image of an inspection object.
The image inspection apparatus according to any one of claims 1 to 7. Further provided with a storage unit for storing the plurality of trained models.
The image inspection apparatus according to any one of claims 1 to 8. Inspection division, which is a divided image of an inspection object, is applied to a plurality of trained models trained to output a restored divided image by inputting one or a plurality of non-defective divided images, which are divided images of a non-defective inspection object. A step of inputting each image and generating a plurality of the restored divided images,
A step of inspecting the inspection object based on the plurality of restored divided images.
Image inspection method. It is provided with a model generation unit that generates a plurality of trained models trained to output a restored divided image by inputting one or a plurality of non-defective divided images which are divided images of a non-defective inspection object.
Trained model generator.

Images