JP2020181532A - Image determination device and image determination method - Google Patents

Abstract

To improve the determination accuracy of processing that determines the normality of an image.SOLUTION: A storage unit 711 stores a restoration model 721 that extracts a feature from input data and creates output data restored from the extracted feature. The storage unit 711 further stores a reference value 722 that is obtained by causing the restoration model to learn a normal image at a plurality of times, and is based on a plurality of values of a function indicating a difference between the input data and the output data. A learning processing unit 712 causes the restoration model 721 to learn an image to be determined at a plurality of times to acquire the plurality of values of the function. A determination unit 713 determines the normality of the image to be determined by using the reference value 722 and the plurality of values of the function acquired by causing the restoration model 721 to learn the image to be determined at a plurality of times, and an output unit 714 outputs a determination result indicating the normality of the image to be determined.SELECTED DRAWING: Figure 7

Description

The present invention relates to an image determination device and an image determination method.

Visual inspections are carried out on production lines of factories and the like in order to detect defective parts such as scratches, dents, and stains on manufactured products. In this visual inspection, an inspector assigned to each production line visually determines whether or not the product is defective based on a predetermined inspection standard.

However, the inspection criteria are often expressed in texts such as "no scratches can be seen at a distance of XX m with visual acuity XX" and "less than XX scratches in the XX mm range". In this case, since the determination is made depending on the individual's sense within the limited inspection time for the defective portion taking various forms, the determination result varies. For this reason, measures such as allocating a plurality of inspectors in stages to prevent erroneous judgments are often taken, which causes an increase in labor costs.

Therefore, in recent years, the introduction of an automatic determination system using image processing techniques such as a pattern matching method and a feature extraction method has been progressing.

In relation to abnormality detection, a technique for detecting an abnormality using a self-encoder is known (see, for example, Patent Document 1 and Patent Document 2). A technique for evaluating the construction quality of a concrete structure using deep learning or the like is also known (see, for example, Patent Document 3).

JP-A-2018-73258 Japanese Unexamined Patent Publication No. 2018-12863 Japanese Unexamined Patent Publication No. 2016-142601

With an automatic determination system using image processing technology, it is difficult to improve the accuracy of the determination result for the captured image of the product to a practical level in line with the inspection standard.

It should be noted that such a problem occurs not only in the visual inspection of the product on the production line of the factory but also in the process of determining the normality of the photographed images of various photographed objects.

In one aspect, the present invention aims to improve the determination accuracy of a process for determining the normality of an image.

In one proposal, the image determination device includes a storage unit, a learning processing unit, a determination unit, and an output unit. The storage unit stores a restored model that extracts features from the input data and generates output data restored from the extracted features. Further, the storage unit stores a reference value based on a plurality of values of a function indicating an error between the input data and the output data, which is acquired by having the restoration model learn the normal image a plurality of times.

The learning processing unit acquires a plurality of values of the function by causing the restoration model to learn the image to be determined a plurality of times. The judgment unit determines the normality of the judgment target image using the plurality of values of the function acquired by training the restoration model multiple times and the reference value, and the output unit determines the judgment target. Outputs the judgment result indicating the normality of the image.

In one aspect, the determination accuracy of the process for determining the normality of the image can be improved.

It is a functional block diagram at the time of learning of an image processing apparatus. It is a figure which shows the change of the value of a loss function. It is a figure which shows the restoration process of a normal image. It is a functional block diagram at the time of determination of an image processing apparatus. It is a figure which shows the determination process for a normal image. It is a figure which shows the determination process for an abnormal image. It is a functional block diagram of an image determination device. It is a flowchart of an image determination process. It is a functional block diagram of a visual inspection system. It is a figure which shows the image to be determined. It is a figure which shows the change of the value of the loss function with respect to a normal image and a judgment target image. It is a figure which shows the value of the loss function with respect to a normal image. It is a figure which shows the value of the loss function with respect to the image to be determined. It is a flowchart of a learning process. It is a flowchart of a determination process. It is a block diagram of an information processing apparatus.

Hereinafter, embodiments will be described in detail with reference to the drawings.
As described above, in recent visual inspections, the introduction of image processing techniques such as a pattern matching method and a feature extraction method is progressing.

In the pattern matching method, the presence or absence of a difference is determined by superimposing the captured image and the normal image. A normal image is an image showing a good product (good product) that does not include defective parts such as scratches, dents, and stains. In such a pattern matching method, it is desirable to prepare a shooting environment in which no positional deviation occurs between the captured image and the normal image, or to correct the positional deviation by another image processing, which increases the maintenance cost or processing load. To increase. Further, when a difference occurs between the captured image and the normal image depending on the brightness or direction of the illumination, the erroneous determination increases.

In the feature extraction method, a feature assumed in advance as an abnormal pattern is searched from the captured image. Therefore, when there are many abnormal patterns having different scratches or stains, thickness, inclination, area, etc., it is difficult to detect all the abnormal patterns without omission.

Recently, the introduction of image judgment technology utilizing artificial intelligence represented by machine learning is also being considered. This image determination technique can be roughly divided into the following two methods.
(M1) A classification method in which a model such as SVM (Support Vector Machine) or CNN (Convolutional Neural Network) is made to learn a normal image and an abnormal image and a teacher label, and the photographed image is classified using the model after learning.
(M2) A convolutional autoencoder (CAE) is trained to learn a normal image, and a determination is made to determine whether the input image is normal or not based on the difference between the input image and the output image for the learned CAE. Method.

An abnormal image is an image showing a defective product including a defective portion. The classification method of (M1) has the following problems.
(A) It is desirable to train a model with a large amount of normal images and a large number of abnormal images, but when the number of defective products is small, it is difficult to prepare a sufficient amount of abnormal images for learning.
(B) The classification accuracy of captured images including unlearned abnormal patterns is reduced.
(C) A teacher label for a normal image and an abnormal image indicates whether each image is a normal image or an abnormal image, and is given by an operator. For this reason, the same variation as in the case of visual inspection occurs in the teacher label, which affects the classification result.

On the other hand, the discrimination method (M2) has an advantage that the problems (a) to (c) do not occur because learning is performed using only normal images and no teacher label is required.

FIG. 1 shows an example of a functional configuration at the time of learning of an image processing apparatus using the discrimination method (M2). The image processing apparatus 101 of FIG. 1 includes a CAE 111 and a loss function 112, and the CAE 111 includes a coding unit 121 and a decoding unit 122.

The coding unit 121 extracts features from the input image and encodes the extracted features. The decoding unit 122 decodes the encoded feature and outputs the restored image restored from the feature as an output image. The coding unit 121 and the decoding unit 122 are generated using a neural network and include weights and biases as parameters.

At the time of learning, a normal image is input to CAE111 as an input image, and CAE111 outputs a restored image restored from the features of the normal image. The loss function 112 is a function indicating an error between the input image input to the CAE 111 and the output image output from the CAE 111. The image processing device 101 evaluates the error between the normal image and the restored image by using the loss function 112, and repeats learning for the normal image while adjusting the parameters inside the CAE 111 so that the value of the loss function becomes small. ..

FIG. 2 shows an example of a change in the value of the loss function during learning. The horizontal axis represents the number of times of learning for the same normal image, and the vertical axis represents the value of the calculated loss function. In this example, the value of the loss function decreases as the number of learnings increases, and when the number of learnings exceeds 5, the value of the loss function converges to a sufficiently small value.

FIG. 3 shows an example of a normal image restoration process in a state where learning has progressed. The CAE 111 learns the characteristics of each of the plurality of normal images 301 and restores the normal image 302. In this way, when the learning is sufficiently advanced, the parameters for restoring the normal image 302 are accumulated inside the CAE 111, and if the normal image 301 is input, the same normal image 302 is output as the restored image. Will be.

FIG. 4 shows an example of a functional configuration at the time of determination of the image processing device 101 of FIG. The image processing device 101 of FIG. 4 includes a CAE 111 after learning, an image difference extraction unit 401, and an abnormality determination unit 402.

At the time of determination, the determination target image is input to CAE111 as an input image, and CAE111 outputs the restored image restored from the features of the determination target image. The image difference extraction unit 401 calculates the direct difference between the pixel value of the determination target image and the pixel value of the restored image, and the abnormality determination unit 402 determines that the determination target image is based on the calculated difference magnitude. Determine whether it is a normal image or an abnormal image.

FIG. 5 shows an example of determination processing for a normal image. When the determination target image 501 is a normal image, the CAE 111 outputs a normal image substantially the same as the determination target image 501 as the restored image 502. In this case, the difference between the determination target image 501 and the restored image 502 is relatively small.

FIG. 6 shows an example of determination processing for an abnormal image. When the abnormal image in which the scratch 611 is shown is the determination target image 601, the CAE 111 manages to match the determination target image 601 with the normal image. However, the restored image 602 restored by CAE111 is an image in which an abnormal portion 612 showing the remnants of the scratch 611 remains to some extent. In this case, the difference between the determination target image 601 and the restored image 602 is larger than that in the case of FIG.

The abnormality determination unit 402 determines that the determination target image is a normal image when the difference between the determination target image and the restored image is less than a certain value, and when the difference is larger than a certain value, the determination target image is an abnormality image. Is determined to be.

However, the discrimination method of (M2) has the following problems.
As mentioned above, in the field of visual inspection, the product is inspected based on the inspection standard specified in advance. Therefore, for example, the product should be inspected in consideration of the degree of scratches (shade, depth, or reflection). Is desirable. The following are examples of inspection criteria that take into account the degree of scratches.
(C1) In the case of a conspicuous scratch, even if the length, width, or range of the scratch is small, it is regarded as abnormal.
(C2) If the scratch is small, such as a thin and faint scratch, it is not considered to be abnormal, and if the scratch spreads beyond a predetermined range or is scattered in multiple places, it is considered to be abnormal for the first time.

However, in the determination method (M2), the determination target image is determined based on the difference between the determination target image and the restored image. For this reason, if the judgment target image includes a part that is not a defective part but is different from the normal image, such as a product misalignment, inclination, difference in brightness, etc., the judgment target image and the restored image The difference becomes large, and it becomes easy to be erroneously determined as an abnormal image. In addition, the difference derived from the scratch that is the original detection target may be buried in the difference derived from the portion other than the scratch, and the abnormal image may be overlooked.

Therefore, it is difficult to improve the accuracy of the judgment result for the judgment target image to a practical level in line with the inspection standard considering the degree of scratches.

FIG. 7 shows an example of a functional configuration of the image determination device of the embodiment. The image determination device 701 of FIG. 7 includes a storage unit 711, a learning processing unit 712, a determination unit 713, and an output unit 714.

The storage unit 711 stores the restoration model 721 that extracts features from the input data and generates output data restored from the extracted features. Further, the storage unit 711 stores a reference value 722 based on a plurality of values of a function indicating an error between the input data and the output data, which is acquired by causing the restoration model 721 to learn the normal image a plurality of times. The learning processing unit 712 and the determination unit 713 perform image determination processing using the restoration model 721 and the reference value 722.

FIG. 8 is a flowchart showing an example of the image determination process performed by the image determination device 701 of FIG. 7. First, the learning processing unit 712 acquires a plurality of values of the function by causing the restoration model 721 to learn the image to be determined a plurality of times (step 801). Next, the determination unit 713 determines the normality of the determination target image by using the plurality of values of the function acquired by training the restoration model 721 a plurality of times and the reference value (step). 802). Then, the output unit 714 outputs a determination result indicating the normality of the determination target image (step 803).

According to the image determination device 701 of FIG. 7, the determination accuracy of the process for determining the normality of the image can be improved.

FIG. 9 shows a functional configuration example of the visual inspection system including the image determination device 701 of FIG. The visual inspection system of FIG. 9 includes an image determination device 901 and an image pickup device 902, and has two types of operation modes, a learning mode and a determination mode. The inspection target of the visual inspection is the product 903 on the production line 904.

The image determination device 901 corresponds to the image determination device 701 of FIG. 7, and includes a storage unit 911, an image acquisition unit 912, an image processing unit 913, a learning processing unit 914, a calculation unit 915, a determination unit 916, and an output unit 917. .. The storage unit 911, the learning processing unit 914, the determination unit 916, and the output unit 917 correspond to the storage unit 711, the learning processing unit 712, the determination unit 713, and the output unit 714 of FIG. 7, respectively.

The image pickup device 902 is a camera having an image pickup element such as a CCD (Charged-Coupled Device) or a CMOS (Complementary Metal-Oxide-Semiconductor), and photographs the product 903 in the determination mode. The image acquisition unit 912 acquires the captured image from the image pickup apparatus 902, and stores the acquired image as the determination target image 926 in the storage unit 911.

The image processing unit 913 divides the determination target image 926 into a plurality of regions. For example, as a plurality of regions, blocks of n × n (n is an integer of 2 or more) are used. The defective portion of the determination target shown in the determination target image 926 is often small compared to the size of the entire product. In this case, the determination accuracy can be further improved by dividing the determination target image 926 and evaluating the image for each region.

FIG. 10 shows an example of the determination target image 926 divided into a plurality of blocks. The determination target image 1001 is divided into 8 × 8 (64 pieces) blocks 1002 by vertical and horizontal boundaries.

The image processing unit 913 can also perform image conversion such as filter processing by a sobel filter, an edge filter, or the like in order to make the defective portion reflected in the determination target image 926 stand out.

The storage unit 911 stores the CAE921, the loss function 922, the image set 923-1 and the image set 923-2.

CAE921 is an example of a restoration model before learning, extracts features from an input image, and generates an output image restored from the extracted features. When the determination target image 926 is divided into a plurality of regions, CAE921 is also prepared for each region. For example, when the determination target image 926 is divided into 64 areas, the storage unit 911 stores 64 CAE921s.

The loss function 922 is the loss function of CAE921. For example, as the loss function 922, a function indicating a cross entropy error between an input image and an output image, a least squares error, and the like is used.

The image set 923-1 is a set of normal images for learning input to CAE921, and the image set 923-2 is a set of normal images input to CAE924 after learning. These normal images can be obtained by photographing a non-defective product that does not include a defective portion with the image pickup apparatus 902.

In the learning mode, the image processing unit 913 divides each normal image included in the image set 923-1 and the image set 923-2 into a plurality of regions, and generates a normal image for each region. The learning processing unit 914 determines the parameters inside the CAE921 by causing the CAE921 for each region to learn the normal image for each region generated from the image set 923-1 a plurality of times.

The learning processing unit 914 repeats learning while adjusting the parameters inside CAE921 so that the value of the loss function 922 for each region becomes small, and when the amount of change in the value of the loss function 922 becomes smaller than a predetermined value, it is obtained. The parameters are determined as the learning result. Then, the learning processing unit 914 generates a CAE 924 for each area after learning by setting the parameters determined in the CAE 921 for each area, and stores the CAE 924 in the storage unit 911.

Next, the learning processing unit 914 acquires a plurality of values of the loss function 922 for each region by causing the CAE 924 for each region to learn the normal image for each region generated from the image set 923-2 a plurality of times. .. These values represent the historical information of the loss function 922 in the training of the image set 923-2. The calculation unit 915 calculates a reference value 925 for each area using a plurality of values of the acquired loss function 922 for each area, and stores the reference value 925 for each area in the storage unit 911.

The CAE 924 and the reference value 925 correspond to the restoration model 721 and the reference value 722 of FIG. 7, respectively. By using CAE924 as the restoration model 721, it becomes possible to use the loss function 922 as a function indicating the error between the input image and the output image.

The number of normal images included in the image set 923-2 may be smaller than the number of normal images for learning included in the image set 923-1. For example, when 100 normal images are obtained by shooting 100 good products, 90 to 99 normal images are used as an image set 923-1 and the remaining 10 to 1 normal images are used. Can be used as the image set 923-2. The image set 923-2 may be a part of the image set 923-1. By training CAE921 with an image set 923-1 containing a large number of normal images, CAE924 having high restoration performance can be generated.

Further, the number of repetitions of learning (number of epochs) for each normal image of the image set 923-2 may be smaller than the number of learning epochs for each normal image of the image set 923-1. For example, even if the number of epochs for the image set 923-1 is several thousand to tens of thousands, the number of epochs for the image set 923-2 may be several to ten times.

In the determination mode, the image processing unit 913 divides the determination target image 926 into a plurality of regions and generates a determination target image for each region. The learning processing unit 914 acquires a plurality of values of the loss function 922 for each region by causing the CAE 924 for each region to learn the determination target image for each region a plurality of times. These values represent the history information of the loss function 922 in learning the image to be determined for each region. As the number of epochs for the image to be determined 926, the number of epochs for the image set 923-2 can be used.

The determination unit 916 calculates an index indicating the normality of the determination target image using a plurality of values of the loss function 922 acquired using the determination target image for each region and the reference value 925 for each region. .. Then, the determination unit 916 determines the normality of the determination target image 926 by comparing the index calculated for each region with the threshold value T1, and stores the determination result 927 indicating the normality of the determination target image 926. Store in 911. The determination result 927 indicates whether the determination target image 926 is a normal image or an abnormal image. The output unit 917 outputs the determination result 927.

FIG. 11 shows an example of the change in the value of the loss function 922 in the learning for the normal image for each region of the image set 923-2 and the learning for the determination target image for each region. The horizontal axis represents the number of learnings, and the vertical axis represents the calculated value of the loss function 922.

The judgment target image Q is an abnormal image, and the value of the loss function 922 for the judgment target image Q is the value of the loss function 922 for the normal images P1 to the normal image P5 included in the image set 923-2 at any number of learning times. Is bigger than.

FIG. 12 shows an example of the value of the loss function 922 in learning the normal image for each region of the image set 923-2. In FIG. 12, the values of the loss function 922 for the normal images P1 to P5 are shown, and the values of the loss function 922 for the normal images P6 to P10 are omitted. The numerical value in the column of m times (m = 1 to 10) represents the value of the loss function 922 at the time when the m-th learning is completed, and the cumulative total represents the sum of the numerical values in the columns of 1 to 10 times.

The calculation unit 915 calculates the reference value R for each region by the following equation, for example, using the cumulative statistical value S and the standard deviation σ of the loss function 922 for each normal image of the image set 923-2.
R = S + σ (1)

As the statistical value S of the equation (1), the average value, the median value, the total value, etc. of a plurality of cumulative totals are used. Then, the calculation unit 915 records the calculated reference value R as the reference value 925.

FIG. 13 shows an example of the value of the loss function 922 in learning for the determination target image Q for each region.

The determination unit 916 calculates an index K indicating the normality of the determination target image by the following equation using the cumulative L of the loss function 922 and the reference value R for the determination target image for each region.
K = L / R (2)

The closer the judgment target image is to the normal image, the smaller the index K, and the more different the judgment target image is from the normal image, the larger the index K. Therefore, the determination unit 916 calculates the index K for all the regions, and if the index K is larger than the threshold value T1 in any one or more regions, the determination target image 926 is determined to be an abnormal image. In this case, the product shown in the determination target image 926 is determined to be a defective product. By performing the determination based on the index K, it is possible to determine the normality of the image to be determined for each region by using the history information of the loss function 922 in the multiple learnings.

On the other hand, when the index K is equal to or less than the threshold value T1 in all the regions, the determination unit 916 obtains the sum of the index K in all the regions and compares the sum of the index K with the threshold value T2 to determine the image to be determined 926. Judge the normality of.

For example, the determination unit 916 determines that the determination target image 926 is an abnormal image when the total sum of the index K is larger than the threshold value T2, and the determination target image 926 is normal when the total sum of the index K is equal to or less than the threshold value T2. Judged as an image. When the determination target image 926 is determined to be a normal image, the product shown in the determination target image 926 is determined to be a non-defective product. By making a judgment based on the sum of the indexes K, it becomes possible to detect a defective product having scratches, stains, etc. that have spread thinly on the surface of the product.

The threshold value T1 and the threshold value T2 are tuning parameters that are finely adjusted when the inspection standard in the visual inspection is changed, and are set according to the product to be inspected. For example, the threshold value T1 and the threshold value T2 may be set according to the level of defects generated in the production line, the shooting environment of the product, and the like.

According to the image determination device 901 of FIG. 9, the determination is made by using the history information of the loss function 922 for the determination target image 926 instead of the direct difference between the pixel value of the determination target image 926 and the pixel value of the restored image. The normality of the target image 926 is determined. As a result, even if there is a difference in the shooting environment such as a position shift, an inclination, or a difference in brightness of the product in the determination target image 926, it is possible to absorb the influence. Therefore, it is suppressed that the index K exceeds the threshold value T1 and the total sum of the index K exceeds the threshold value T2, and the possibility of erroneous determination as an abnormal image is reduced.

Further, by using a plurality of normal images as the image set 923-2, the history information of the loss function 922 for various normal images can be reflected in the reference value R and the index K. As a result, the difference in the shade of the scratch, the range of the scratch, the reflection of another object, etc., which is difficult to distinguish only by the difference between the pixel value of the image to be judged 926 and the pixel value of the restored image, is reflected in the index K. Can be made to.

Therefore, according to the image determination device 901 of FIG. 9, the accuracy of determining a defective product in the visual inspection using the image of the product is improved as compared with the image processing device 101 of FIG.

FIG. 14 is a flowchart showing an example of learning processing performed by the image determination device 901 in the learning mode. Each normal image included in the image set 923-1 and the image set 923-2 is divided into n × n blocks by the image processing unit 913.

First, the learning processing unit 914 repeats the processing of steps 1401 to 1403 for each block for the normal image of the image set 923-1.

The learning processing unit 914 normalizes the color tone of the normal image for each block with a gradation value of 0 to 255 (step 1401), and causes CAE921 for each block to learn the normalized normal image (step 1402). The number of epochs in the learning of step 1402 is, for example, several thousand to tens of thousands, and the batch size is, for example, 16 or 32. Then, the learning processing unit 914 generates CAE924 for each block after learning by setting the parameter determined by learning to CAE921 for each block (step 1403).

Next, the learning processing unit 914 repeats the processes of steps 1404 to 1407 for each block with respect to the normal image of the image set 923-2.

The learning processing unit 914 normalizes the color tone of the normal image for each block in the same manner as in step 1401 (step 1404), and causes the CAE 924 for each block to learn the normalized normal image (step 1405). The number of epochs in the learning of step 1405 is, for example, several to 10 times. Then, the calculation unit 915 calculates the cumulative value of the loss function 922 for each block (step 1406). The processes of steps 1404 to 1406 are repeated for each normal image of the image set 923-2.

Next, the calculation unit 915 calculates the reference value 925 for each block by the equation (1) using the cumulative value of the loss function 922 for each block (step 1407).

FIG. 15 is a flowchart showing an example of the determination process performed by the image determination device 901 in the determination mode. The determination target image 926 is divided into n × n blocks by the image processing unit 913.

First, the learning processing unit 914 repeats the processes of steps 1501 to 1504 for each block of the determination target image 926.

The learning processing unit 914 normalizes the color tone of the judgment target image for each block (step 1501) in the same manner as in step 1401 of FIG. 14, and causes the CAE 924 for each block to learn the normalized judgment target image (step 1501). Step 1502). The number of epochs in the learning in step 1502 is the same as the number of epochs in step 1405. Then, the determination unit 916 calculates the cumulative value of the loss function 922 for each block (step 1503), and uses the calculated cumulative total and the reference value 925 for each block to use the index for each block according to the equation (2). Calculate K (step 1504).

Next, the determination unit 916 determines the normality of the determination target image 926 using the index K, the threshold value T1, and the threshold value T2 calculated for each block, and generates a determination result 927 (step 1505). Then, the output unit 917 outputs the determination result 927.

The configuration of the image processing device 101 of FIGS. 1 and 4 is only an example, and some components may be omitted or changed depending on the use or conditions of the image processing device 101.

The configuration of the image determination device 701 in FIG. 7 is only an example, and some components may be omitted or changed depending on the use or conditions of the image determination device 701.

The configuration of the visual inspection system of FIG. 9 is only an example, and some components may be omitted or changed depending on the use or conditions of the visual inspection system. For example, when the image set 923-1, the image set 923-2, and the determination target image 926 are stored in the storage unit 911 in advance, the image acquisition unit 912 can be omitted. When it is not necessary to divide the normal image of the image set 923-1, the normal image of the image set 923-2, and the determination target image 926, the image processing unit 913 can be omitted. If the reference value 925 is stored in the storage unit 911 in advance, the calculation unit 915 can be omitted.

As the restoration model 721 of FIG. 7, a model other than CAE may be used. For example, instead of CAE921 and CAE924 in FIG. 9, a self-encoder such as VAE (Variational Auto Encoder) can be used. Instead of the loss function 922, another function indicating the error between the input data and the output data can be used.

The image determination device 901 of FIG. 9 can be applied not only to the appearance inspection of the product but also to the process of determining the normality of the captured images obtained by capturing various imaging targets. For example, the captured image to be determined may be an image included in the image of the surveillance camera.

The flowcharts of FIGS. 8, 14 and 15 are merely examples, and some processes may be omitted or changed depending on the configuration or conditions of the image determination device. For example, in the image determination device 901 of FIG. 9, when it is not necessary to normalize the normal images of the image set 923-1 and the image set 923-2, the processes of steps 1401 and 1404 of FIG. 14 may be omitted. it can. When it is not necessary to normalize the determination target image 926, the process of step 1501 in FIG. 15 can be omitted. When the CAE 924 and the reference value 925 are generated by an external device, the learning process of FIG. 14 can be omitted.

The value of the loss function shown in FIGS. 2 and 11 to 13 is only an example, and the value of the loss function changes according to the image to be learned by CAE. The restoration process shown in FIG. 3 and the determination process shown in FIGS. 5 and 6 are merely examples, and the restoration process and the determination process change according to the image to be learned by CAE. The method of dividing the determination target image 1001 shown in FIG. 10 is only an example, and the image processing unit 913 can also divide the determination target image 1001 into regions having different shapes.

Equations (1) and (2) are merely examples, and the image determination device 901 may calculate the reference value R and the index K using another mathematical expression. For example, the standard deviation σ on the right side of the equation (1) may be omitted.

FIG. 16 shows a configuration example of an information processing device used as the image determination device 701 of FIG. 7 and the image determination device 901 of FIG. The information processing device of FIG. 16 includes a CPU (Central Processing Unit) 1601, a memory 1602, an input device 1603, an output device 1604, an auxiliary storage device 1605, a medium drive device 1606, and a network connection device 1607. These components are connected to each other by bus 1608. The image pickup apparatus 902 of FIG. 9 may be connected to the bus 1608.

The memory 1602 is, for example, a semiconductor memory such as a ROM (Read Only Memory), a RAM (Random Access Memory), or a flash memory, and stores a program and data used for processing. The memory 1602 can be used as the storage unit 711 of FIG. 7 or the storage unit 911 of FIG.

The CPU 1601 (processor) operates as the learning processing unit 712 and the determination unit 713 in FIG. 7 by executing a program using, for example, the memory 1602. The CPU 1601 also operates as the image acquisition unit 912, the image processing unit 913, the learning processing unit 914, the calculation unit 915, and the determination unit 916 in FIG. 9 by executing the program using the memory 1602.

The input device 1603 is, for example, a keyboard, a pointing device, or the like, and is used for inputting instructions or information from an operator or a user. The output device 1604 is, for example, a display device, a printer, a speaker, or the like, and is used for inquiries or instructions to an operator or a user, and output of a processing result. The processing result may be the determination result 927. The output device 1604 can be used as the output unit 917 of FIG.

The auxiliary storage device 1605 is, for example, a magnetic disk device, an optical disk device, a magneto-optical disk device, a tape device, or the like. The auxiliary storage device 1605 may be a hard disk drive or a flash memory. The information processing device can store programs and data in the auxiliary storage device 1605 and load them into the memory 1602 for use. The auxiliary storage device 1605 can be used as the storage unit 711 of FIG. 7 or the storage unit 911 of FIG.

The medium drive device 1606 drives the portable recording medium 1609 and accesses the recorded contents. The portable recording medium 1609 is a memory device, a flexible disk, an optical disk, a magneto-optical disk, or the like. The portable recording medium 1609 may be a CD-ROM (Compact Disk Read Only Memory), a DVD (Digital Versatile Disk), a USB (Universal Serial Bus) memory, or the like. The operator or the user can store programs and data in the portable recording medium 1609 and load them into the memory 1602 for use.

In this way, the computer-readable recording medium that stores the programs and data used for processing is physical (non-temporary) recording such as memory 1602, auxiliary storage device 1605, or portable recording medium 1609. It is a medium.

The network connection device 1607 is a communication interface circuit that is connected to a communication network such as a LAN (Local Area Network) or WAN (Wide Area Network) and performs data conversion associated with communication. The information processing device can receive programs and data from an external device via the network connection device 1607, load them into the memory 1602, and use them. The network connection device 1607 can be used as the output unit 917 of FIG.

It is not necessary for the information processing apparatus to include all the components of FIG. 16, and some components may be omitted depending on the application or conditions. For example, if an interface with an operator or a user is not required, the input device 1603 and the output device 1604 may be omitted. When the portable recording medium 1609 or the communication network is not used, the medium driving device 1606 or the network connecting device 1607 may be omitted.

Having described in detail the embodiments of the disclosure and its advantages, those skilled in the art will be able to make various changes, additions and omissions without departing from the scope of the invention as expressly stated in the claims. Let's do it.

The following additional notes will be further disclosed with respect to the embodiments described with reference to FIGS. 1 to 16.
(Appendix 1)
With the input data obtained by extracting features from the input data and generating the output data restored from the extracted features, storing the restored model, and training the restored model multiple times for normal images. A storage unit that stores a reference value based on a plurality of values of a function indicating an error from the output data, and a storage unit.
A learning processing unit that acquires a plurality of values of the function by causing the restoration model to learn the image to be determined a plurality of times.
A determination unit that determines the normality of the determination target image by using the plurality of values of the function acquired by training the restoration model a plurality of times and the reference value.
An output unit that outputs a judgment result indicating the normality of the judgment target image, and
An image determination device comprising.
(Appendix 2)
The reference value is calculated from a plurality of values of the function obtained by training the restoration model a plurality of times of the normal image.
The determination unit uses a plurality of values of the function acquired by causing the restoration model to learn the determination target image a plurality of times and the reference value to obtain an index indicating the normality of the determination target image. The image determination device according to Appendix 1, wherein the normality of the image to be determined is determined by calculating and comparing the index and the threshold value.
(Appendix 3)
The normal image is one of the normal images of each of the plurality of regions in the first image.
The determination target image is one of the determination target images of each of the plurality of regions in the second image.
The storage unit stores the restoration model for each area, and is based on a plurality of values of the function for each area, which is acquired by having the restoration model for each area learn a normal image for each area a plurality of times. Memorize each reference value and
The reference value for each region is calculated from a plurality of values of the function for each region.
The learning processing unit acquires a plurality of values of the function for each region by causing the restoration model for each region to learn the image to be determined for each region a plurality of times.
The determination unit uses a plurality of values of the function for each region acquired by using the determination target image for each region and a reference value for each region to determine the normality of the determination target image for each region. The normality of the second image is determined by calculating the indicated index and comparing the index with the first threshold value.
The image determination device according to Appendix 1, wherein the output unit outputs a determination result indicating the normality of the second image.
(Appendix 4)
The determination unit determines the normality of the second image by obtaining the sum of the indexes indicating the normality of the determination target image for each region and comparing the total of the indexes with the second threshold value. The image determination device according to Appendix 3, which is a feature.
(Appendix 5)
The image determination device according to any one of Supplementary note 1 to 4, wherein the restoration model is generated by training a set of normal images in a restoration model before learning.
(Appendix 6)
The image determination device according to any one of Supplementary note 1 to 5, wherein the restoration model is a self-encoder, and the function is a loss function of the self-encoder.
(Appendix 7)
An image determination method performed by a computer
The computer
A function that indicates an error between the input data and the output data by having the restored model train the image to be judged multiple times to extract features from the input data and generate the output data restored from the extracted features. Get multiple values of
A reference value based on a plurality of values of the function acquired by having the restoration model learn a normal image a plurality of times, and a plurality of the functions acquired by having the restoration model learn the determination target image a plurality of times. The normality of the image to be determined is determined by using the value of.
Outputs a judgment result indicating the normality of the judgment target image.
An image determination method characterized by this.
(Appendix 8)
The reference value is calculated from a plurality of values of the function obtained by training the restoration model a plurality of times of the normal image.
The computer calculates an index indicating the normality of the determination target image by using the plurality of values of the function acquired by having the restoration model learn the determination target image a plurality of times and the reference value. The image determination method according to Appendix 7, wherein the normality of the determination target image is determined by comparing the index with the threshold value.
(Appendix 9)
The normal image is one of the normal images of each of the plurality of regions in the first image.
The determination target image is one of the determination target images of each of the plurality of regions in the second image.
The computer stores the restoration model for each area and trains the restoration model for each area a plurality of times to learn a normal image for each area, and is based on a plurality of values of the function for each area. Memorize the reference value of
The reference value for each region is calculated from a plurality of values of the function for each region.
The computer acquires a plurality of values of the function for each region by causing the restoration model for each region to learn the image to be determined for each region a plurality of times, and obtains the image to be determined for each region. By using a plurality of values of the function for each region and a reference value for each region, an index indicating the normality of the image to be determined for each region is calculated, and the index is compared with the first threshold value. The image determination method according to Appendix 7, wherein the normality of the second image is determined, and a determination result indicating the normality of the second image is output.
(Appendix 10)
The computer is characterized in that the totality of indicators indicating the normality of the determination target image for each region is obtained, and the normality of the second image is determined by comparing the total sum of the indicators with the second threshold value. The image determination method according to Appendix 9.
(Appendix 11)
The image determination method according to any one of Supplementary note 7 to 10, wherein the restoration model is generated by training a set of normal images in a restoration model before learning.
(Appendix 12)
The image determination method according to any one of Supplementary note 7 to 11, wherein the restoration model is a self-encoder, and the function is a loss function of the self-encoder.

101 Image processing equipment 111, 921, 924 CAE
112 Loss function 121 Encoding unit 122 Decoding unit 301, 302 Normal image 401 Image difference extraction unit 402 Abnormality determination unit 501, 601 Judgment target image 502, 602 Restored image 701, 901 Image determination device 711, 911 Storage unit 712, 914 Learning Processing unit 713, 916 Judgment unit 714, 917 Output unit 721 Restoration model 722 Reference value 902 Image pickup device 903 Product 904 Production line 912 Image acquisition unit 913 Image processing unit 915 Calculation unit 922 Loss function 923-1, 923-2 Image set 925 Reference value 926, 1001 Judgment target image 927 Judgment result 1002 block 1601 CPU
1602 Memory 1603 Input device 1604 Output device 1605 Auxiliary storage device 1606 Media drive device 1607 Network connection device 1608 Bus 1609 Portable recording medium

Claims (6)

With the input data obtained by extracting features from the input data and generating the output data restored from the extracted features, storing the restored model, and training the restored model multiple times for normal images. A storage unit that stores a reference value based on a plurality of values of a function indicating an error from the output data, and a storage unit.
A learning processing unit that acquires a plurality of values of the function by causing the restoration model to learn the image to be determined a plurality of times.
A determination unit that determines the normality of the determination target image by using the plurality of values of the function acquired by training the restoration model a plurality of times and the reference value.
An output unit that outputs a judgment result indicating the normality of the judgment target image, and
An image determination device comprising. The reference value is calculated from a plurality of values of the function obtained by training the restoration model a plurality of times of the normal image.
The determination unit uses a plurality of values of the function acquired by having the restoration model learn the determination target image a plurality of times and the reference value to obtain an index indicating the normality of the determination target image. The image determination device according to claim 1, wherein the normality of the determination target image is determined by calculating and comparing the index with the threshold value. The normal image is one of the normal images of each of the plurality of regions in the first image.
The determination target image is one of the determination target images of each of the plurality of regions in the second image.
The storage unit stores the restoration model for each area, and is based on a plurality of values of the function for each area, which is acquired by having the restoration model for each area learn a normal image for each area a plurality of times. Memorize each reference value and
The reference value for each region is calculated from a plurality of values of the function for each region.
The learning processing unit acquires a plurality of values of the function for each region by causing the restoration model for each region to learn the image to be determined for each region a plurality of times.
The determination unit uses a plurality of values of the function for each region acquired by using the determination target image for each region and a reference value for each region to determine the normality of the determination target image for each region. The normality of the second image is determined by calculating the indicated index and comparing the index with the first threshold value.
The image determination device according to claim 1, wherein the output unit outputs a determination result indicating the normality of the second image. The determination unit determines the normality of the second image by obtaining the sum of the indexes indicating the normality of the determination target image for each region and comparing the total of the indexes with the second threshold value. The image determination device according to claim 3, which is characterized. The image determination device according to any one of claims 1 to 4, wherein the restoration model is generated by training a set of normal images in a restoration model before learning. An image determination method performed by a computer
The computer
A function that indicates an error between the input data and the output data by having the restored model train the image to be judged multiple times to extract features from the input data and generate the output data restored from the extracted features. Get multiple values of
A reference value based on a plurality of values of the function acquired by having the restoration model learn a normal image a plurality of times, and a plurality of the functions acquired by having the restoration model learn the determination target image a plurality of times. The normality of the image to be determined is determined by using the value of.
Outputs a judgment result indicating the normality of the judgment target image.
An image determination method characterized by this.

Images