JP2020030565A - Image determination method, image determination device and image determination program - Google Patents

Abstract

To simply perform image determination using machine learning with high accuracy.SOLUTION: An image determination device includes a first learning unit, a second learning unit, an evaluation unit, a first determination unit, and a second determination unit. The first learning unit generates a normal model obtained by modeling a normal image by first learning being machine learning using a learning image to be a positive example. The second learning unit generates an abnormal classification model obtained by modeling a pattern of an abnormal image by second learning being machine learning using a learning image to be a negative example. The evaluation unit evaluates the degree of deviation from a normal state of a determination object image on the basis of an output value of the normal model. The first determination unit determines whether the determination object image is abnormal on the basis of the degree of deviation. The second determination unit determines a pattern of an abnormal image on the basis of the output value of the abnormal classification model obtained by inputting the determination object image to the abnormal classification model in the case where the first determination unit determines that the determination object image is abnormal.SELECTED DRAWING: Figure 2

Description

  An embodiment of the present disclosure relates to an image determination method, an image determination device, and an image determination program.

  2. Description of the Related Art Conventionally, a technique for performing image analysis using machine learning is known. For example, the technology disclosed in Patent Literature 1 executes machine learning using a feature vector calculated for a plurality of images whose classification results are known, and the classification result is unknown based on the result obtained by the machine learning. This classifies a certain image.

  By using such a technique, it is possible to classify, for example, from an image of a manufactured product, whether or not the product is a normal product, and what is abnormal if the product is not a normal product.

JP-A-2015-011552

  However, the above-described conventional technology has room for further improvement in performing image determination using machine learning simply and accurately.

  Specifically, for example, when there are countless patterns such as scratches and color unevenness in the above-described product abnormalities, in order to generate a classification model capable of classifying all of the patterns, it is necessary to collect a large amount of learning data. It requires long learning and is complicated.

  Further, in normal operation, it is easy to acquire a normal image in a normal state, and it is difficult to obtain an abnormal image when an abnormality occurs. Therefore, it is difficult to sufficiently learn the classification pattern in the initial stage of operation or the like, and the accuracy of the classification model cannot be secured.

  In addition, even if it is a normal image, if there is a normal image corresponding to a so-called outlier, if this is learned as a normal part, accuracy of the classification model may be degraded.

  An aspect of the embodiment is made in view of the above, and provides an image determination method, an image determination device, and an image determination program that can easily and accurately perform image determination using machine learning. With the goal.

  The image determination method according to one aspect of the embodiment includes a first learning step, a second learning step, an evaluation step, a first determination step, and a second determination step. The first learning step generates a normal model in which a normal image is modeled by executing first learning which is machine learning using a learning image serving as a positive example. The second learning step generates an abnormality classification model in which a pattern of an abnormal image is modeled by executing second learning that is machine learning using the learning image, which is a negative example. The evaluation step evaluates a degree of deviation of the determination target image from a normal state based on an output value of the normal model obtained by inputting the determination target image to the normal model. The first determination step determines whether or not the determination target image is abnormal based on the degree of divergence. The second determination step is an output of the abnormality classification model obtained by inputting the determination target image to the abnormality classification model when the determination target image is determined to be abnormal by the first determination step. The pattern is determined based on the value.

  According to one aspect of the embodiment, image determination using machine learning can be performed easily and accurately.

FIG. 1A is a schematic explanatory diagram (part 1) of an image determination method according to the embodiment. FIG. 1B is a schematic explanatory diagram (part 2) of the image determination method according to the embodiment. FIG. 2 is a block diagram of the image determination device according to the embodiment. FIG. 3A is a diagram illustrating a specific example of a normal image. FIG. 3B is a diagram (part 1) illustrating a specific example of a detection result by the normal model. FIG. 3C is a diagram (part 2) illustrating a specific example of the detection result by the normal model. FIG. 3D is a diagram (part 3) illustrating a specific example of the detection result using the normal model. FIG. 3E is a diagram (part 4) illustrating a specific example of the detection result by the normal model. FIG. 3F is a diagram (part 5) illustrating a specific example of the detection result using the normal model. FIG. 3G is a diagram (part 6) illustrating a specific example of the detection result using the normal model. FIG. 4A is a flowchart (part 1) illustrating a processing procedure executed by the image determination device. FIG. 4B is a flowchart (part 2) illustrating a processing procedure executed by the image determination device. FIG. 5 is a hardware configuration diagram illustrating an example of a computer that implements the functions of the image determination device. FIG. 6A is a diagram (part 1) illustrating a specific example of a learning image according to another embodiment. FIG. 6B is a diagram (part 2) illustrating a specific example of a learning image according to another embodiment. FIG. 6C is a diagram (part 1) illustrating a specific example of a determination target image according to another embodiment. FIG. 6D is a diagram (part 2) illustrating a specific example of a determination target image according to another embodiment.

  Hereinafter, embodiments of an image determination method, an image determination device, and an image determination program disclosed in the present application will be described in detail with reference to the accompanying drawings. The present invention is not limited by the embodiments described below.

  First, an outline of an image determination method according to the embodiment will be described with reference to FIGS. 1A and 1B. 1A and 1B are schematic explanatory diagrams (part 1) and (part 2) of an image determination method according to the embodiment.

  In the following, a case where a gear is manufactured as a product and a defective product is detected in a pre-shipment inspection or the like of the gear will be described as an example.

  As shown in FIG. 1A, in the image determination method according to the embodiment, first, machine learning is performed using only a “normal image” which is a normal gear image as a learning data set (step S1), and the normal model 12c is converted to a normal model. Generate.

  Note that, even in the case of the “normal image”, there are cases where the illuminance and the like vary, so that it is preferable to prepare a plurality of normal patterns in the “normal image” as shown in FIG. 1A. This makes it possible to generate a normal model 12c that is robust against variations in illuminance and the like.

  In the image determination method according to the embodiment, a “determination target image” to be determined is input to the generated normal model 12c, and an error from normal (hereinafter, referred to as “normal”) based on an output value output by the normal model 12c. “Degree of deviation from normal state” or simply “degree of deviation” is calculated for each pixel (step S2). Then, if the sum of the deviations is large, it is determined that there is an abnormality (step S3).

  That is, in the image determination method according to the embodiment, the normal model 12c based on only the “normal image” first determines whether the image is normal or abnormal. That is, in the image determination method according to the embodiment, since an abnormality is detected based on the degree of deviation from a normal state, it is possible to detect an unknown abnormality as an abnormality without learning a large number of abnormal patterns. . Thereby, even in a stage where the amount of learning data is small in the initial stage of operation, it is possible to determine an abnormality using the normal model 12c.

  As will be described later with reference to FIG. 3B and the like, in the image determination method according to the embodiment, a mapping image to which the degree of deviation for each pixel calculated in step S2 is assigned is generated, and the mapping image is determined to be abnormal. The abnormal target area is extracted. Based on such an abnormality target area, an abnormality classification process using an abnormality classification model 12d described later can be performed. This point will be described in the abnormality classification process shown in FIG. 1B.

  In addition, even if it is determined that there is an abnormality in step S3 of FIG. 1A, there may be a case where an erroneous detection can be determined based on the extracted abnormality determination region or the knowledge of a person who visually recognizes the region. In such a case, the image determination method according to the embodiment performs additional learning using the corresponding determination target image (step S4), and updates the normal model 12c. Accordingly, the accuracy of the normal model 12c can be appropriately improved even during operation.

  By the way, depending on the normal model 12c described above, even if an abnormality can be detected, the pattern of the abnormality cannot be classified. Therefore, in the image determination method according to the embodiment, as shown in FIG. 1B, the abnormal target regions R1, R2, R3... Of the image of the defective product detected in the past (hereinafter referred to as “abnormal image”) are used as the learning data. The set machine learning is executed (step S5), and the abnormality classification model 12d is generated.

  That is, in the image determination method according to the embodiment, not only the normal model 12c that models the normal image but also the abnormality classification model 12d that models the abnormal pattern based on the abnormal image at the time of the past abnormality detection is used. Therefore, the classification of abnormal patterns was performed.

  Here, as shown in the figure, the “abnormal images” are clustered for each abnormal pattern such as “wrong character” or “missing”. An abnormal pattern of a defective product can be modeled by the abnormal classification model 12d generated based on the abnormal image.

  In the image determination method according to the embodiment, the abnormal target region RX of the “determination target image” determined to be abnormal based on, for example, the normal model 12c is input to the abnormal classification model 12d. As a result, the abnormal classification model 12d outputs a classification result (for example, “class ID” in the figure) and a degree of similarity to the classification result (Step S6).

  Here, when the degree of similarity to the classification result is small, there is a possibility that the pattern is an unknown abnormal pattern. Therefore, in such a case, the image determination method according to the embodiment additionally sets a new abnormal pattern classification class based on human knowledge and the like, and performs additional learning using the corresponding abnormal target area (step S7). The abnormality classification model 12d is updated. Thereby, the accuracy of the abnormality classification model 12d can be appropriately improved even during operation.

  On the other hand, when the degree of similarity to the classification result is large, there is a high possibility that the pattern is a known abnormal pattern. Therefore, the image determination method according to the embodiment notifies the classification result obtained from the abnormal classification model 12d (step S8).

  It should be noted that the abnormal classification model 12d can include a normal part indicating an outlier as shown by the class ID “004” in the figure. The normal part of the outlier is included as an abnormal pattern, and if the class ID “004” is obtained as a classification result by the abnormal classification model 12d, this is notified as normal rather than abnormal, thereby temporarily setting the outlier. The accuracy degradation of the normal model 12c which can occur when the normal model 12c is additionally learned with the normal component can be prevented.

  Hereinafter, the configuration of the image determination device 10 to which the above-described image determination method is applied will be described more specifically.

  FIG. 2 is a block diagram of the image determination device 10 according to the present embodiment. In FIG. 2, components necessary for explaining the features of the present embodiment are represented by functional blocks, and descriptions of general components are omitted.

  In other words, each component illustrated in FIG. 2 is a functional concept and does not necessarily need to be physically configured as illustrated. For example, the specific form of distribution / integration of each functional block is not limited to the illustrated one, and all or a part of the functional blocks may be functionally or physically distributed in arbitrary units according to various loads and usage conditions. -It is possible to integrate and configure.

  In the description using FIG. 2, the description of the components already described above may be simplified or omitted.

  As shown in FIG. 2, the image determination device 10 includes a control unit 11 and a storage unit 12. The control unit 11 includes an acquisition unit 11a, a first learning unit 11b, a second learning unit 11c, an evaluation unit 11d, a first determination unit 11e, and a second determination unit 11f.

  The storage unit 12 is a storage device such as a hard disk drive, a non-volatile memory, or a register, and includes a first learning data set 12a, a second learning data set 12b, a normal model 12c, an abnormal classification model 12d, The information 12e is stored. The evaluation information 12e includes divergence information 12ea, a mapping image 12eb, a classification result 12ec, and a similarity 12ed.

  The control unit 11 performs overall control of the image determination device 10. The acquisition unit 11a acquires a learning image that has been clustered. The learning image includes a first learning normal image for generating the normal model 12c and a second learning abnormal image for generating the abnormal classification model 12d.

  Further, the acquiring unit 11a acquires an abnormal target area in the second learning. Such an area can be designated from the operation unit 20 such as a keyboard, a mouse, and a touch panel connected to the image determination device 10, for example. Note that information regarding the identification of the abnormal target area may be stored in the storage unit 12 or the like in advance, and the acquiring unit 11a may extract the abnormal target area from the information. A classification class is associated with each of the abnormal target areas.

  The acquisition unit 11a stores a normal image among the acquired learning images in the first learning data set 12a. The acquisition unit 11a stores the abnormal target area in the abnormal image in the second learning data set 12b.

  Further, when receiving a determination result of erroneous detection from a first determination unit 11e described later, the obtaining unit 11a obtains the corresponding normal image for additional learning and stores it in the first learning data set 12a. .

  In addition, when the determination unit 11a receives a determination result indicating that additional learning is necessary from the second determination unit 11f described later, the obtaining unit 11a obtains the abnormal target region RX of the corresponding abnormal image for additional learning, and obtains the second learning target region RX. The data is stored in the data set 12b.

  The first learning unit 11b acquires the first learning data set 12a, executes machine learning using the data set 12a, and generates a normal model 12c. The second learning unit 11c acquires the second learning data set 12b, performs machine learning using the data set 12b, and generates the abnormality classification model 12d.

  Note that VAE (Variational Auto Encoder), GAN (Generative Adversarial Network), or the like can be used as an algorithm for machine learning. Since these algorithms are publicly known, detailed description thereof is omitted here.

  The evaluation unit 11d acquires the image to be determined, inputs the acquired image to the normal model 12c, and receives an output result from the normal model 12c. Then, the evaluation unit 11d calculates the divergence for each pixel of the determination target image based on the received output result, and stores the calculated divergence in the divergence information 12ea. Further, the evaluation unit 11d generates a mapping image 12eb in which the calculated divergence is assigned to each pixel.

  In addition, when the evaluation unit 11d receives a determination result that the determination target image is abnormal from the first determination unit 11e described below, the evaluation unit 11d inputs the abnormality target region RX included in the determination result to the abnormality classification model 12d, An output result from the abnormality classification model 12d is received. Then, the evaluation unit 11d stores the received output result in the classification result 12ec and the similarity 12ed.

  The first determination unit 11e refers to the divergence degree information 12ea and notifies an abnormality via the display unit 30 or the like connected to the image determination apparatus 10 when the sum of the divergence degrees is larger than a predetermined determination threshold. I do.

  When the sum of the deviations is greater than a predetermined determination threshold, the first determination unit 11e acquires the corresponding determination target image from the evaluation unit 11d and determines whether the image is a target of additional learning. That is, it is determined whether the detection is erroneous.

  This determination can be made based on, for example, separately providing a threshold for identifying erroneous detection, and then determining whether or not a statistic related to the divergence, such as the sum of the divergence or the variation in the divergence, is equal to or larger than the threshold. In addition, the first determination unit 11e causes the display unit 30 to display the corresponding determination target image or the mapping image 12eb, determines whether or not a false detection has been made based on human knowledge, and accepts the determination result. You may.

  In addition, when it is determined that the detection is erroneous, the first determination unit 11e notifies the acquisition unit 11a of the determination result, and sends the corresponding determination target image to the acquisition unit 11a for additional learning of the normal model 12c. Get it. The first learning unit 11b executes the first learning based on the image for the additional learning, and updates the normal model 12c.

  In addition, when it is determined that there is no false detection, the first determination unit 11e extracts the abnormality target region RX from the mapping image 12eb, and notifies the acquisition unit 11a of the determination result of the abnormality including the abnormality target region RX. .

  The second determination unit 11f refers to the classification result 12ec and the similarity 12ed, and notifies the classification result 12ec via the display unit 30 and the like when the similarity 12ed is larger than a predetermined determination threshold.

  Further, when the similarity 12ed is equal to or smaller than the predetermined determination threshold, the second determination unit 11f notifies the acquisition unit 11a of the determination result indicating that additional learning is necessary, and sets the abnormality target area RX of the corresponding abnormal image to abnormal. The acquisition unit 11a acquires the classification model 12d for additional learning.

  The acquiring unit 11a receives additional setting of a new abnormal pattern classification class via the operation unit 20 or the like based on the notification. The second learning unit 11c executes the second learning based on the abnormality target region RX for the additional learning and the content of the additional setting, and updates the abnormality classification model 12d.

  Next, in order to make the description so far easy to understand, FIGS. 3A to 3G show specific examples of a normal image and a detection result by the normal model 12c. FIG. 3A is a diagram illustrating a specific example of a normal image. FIGS. 3B to 3G are diagrams (part 1) to (part 6) showing specific examples of the detection results obtained by the normal model 12c.

  As described above, a normal image of a “normal product” is used to generate the normal model 12c, as shown in FIG. 3A. That is, a normal image as shown in FIG. 3A is stored in the first learning data set 12a, and the first learning unit 11b generates a normal model 12c using the normal image. Note that, as described above, it is preferable to prepare a plurality of normal patterns, such as a case where the illuminance differs even for a normal image. This makes it possible to generate a normal model 12c that is robust to variations in illuminance. What is different is not limited to the illuminance, but may be, for example, luminance, saturation, angle, or the like.

  Then, for the normal model 12c generated from such a normal image, the evaluation unit 11d inputs the determination target image. Then, based on the output value of the normal model 12c obtained as a result, the evaluation unit 11d calculates the degree of divergence for each pixel and generates the mapping image 12eb.

  Here, it is assumed that there is “defective product # 1” shown in FIG. 3B. As shown in the M1 part in the figure, "defective product # 1" is a defective product having a "character difference" in which characters to be engraved are different. When the image of “defective product # 1” is input as the image to be determined, the evaluation unit 11d inputs the image to the normal model 12c, and based on the output value, as shown in “Detection result” in FIG. Generate the mapping image 12eb.

  That is, in the example of the example of FIG. 3B, a mapping image 12eb in which an area corresponding to the position of engraving a character is clearly different from other areas is generated. If the sum of the divergence degrees is larger than a predetermined determination threshold value (that is, if it is abnormal), the first determination unit 11e extracts a region different from the others as the abnormality target region RX.

  Further, FIG. 3C shows “defective product # 2” of “with chipping” in which a part of the gear is missing (see M2 portion in the drawing). When the image of “defective product # 2” is input as the image to be determined, as shown in the “detection result” in the figure, the mapping image where the area corresponding to the missing position is clearly different from the other areas 12eb is generated and if abnormal, a region different from the others is extracted as the abnormal target region RX.

  FIG. 3D shows “defective product # 3” having “projection” in which a projection is formed on a part of the inner diameter (see the M3 part in the figure). When the image of “defective product # 3” is input as the image to be determined, as shown in the “detection result” in the drawing, the area corresponding to the position of the projection is clearly different from the other areas. 12eb is generated and if abnormal, a region different from the others is extracted as the abnormal target region RX.

  Further, FIG. 3E shows “defective product # 4” of “difference in inner diameter” whose inner diameter is not a regular size (see M4 part in the figure). When such an image of “defective product # 4” is input as the image to be determined, as shown in the “detection result” of the figure, the area corresponding to the inner diameter portion of the gear is clearly different from the other areas. If the image 12eb is generated and abnormal, an area different from the others is extracted as the abnormal target area RX.

  Also, FIG. 3F shows “defective product # 5” of “peripheral part difference” in which the peripheral part of the inner diameter is not a regular dimension (see M5 part in the figure). When the image of “defective product # 5” is input as the image to be determined, as shown in the “detection result” of FIG. Is generated, and if abnormal, an area different from the others is extracted as the abnormal target area RX.

  Also, FIG. 3G shows “defective product # 6” of “stained” (stained (see M6 in the figure)). When such an image of “defective product # 6” is input as the image to be determined, as shown in “Detection result” of FIG. 12, the mapping image 12eb where the area corresponding to the stained part is clearly different from the other areas Is generated, and if it is abnormal, a region different from the others is extracted as the abnormal target region RX.

  Then, each extracted abnormal target region RX is stored in the second learning data set 12b, and the second learning unit 11c generates or updates the abnormal classification model 12d using this.

  Next, a processing procedure executed by the image determination device 10 according to the embodiment will be described with reference to FIGS. 4A and 4B. FIGS. 4A and 4B are flowcharts (part 1) and (part 2) illustrating a processing procedure executed by the image determination device 10 according to the embodiment.

  As shown in FIG. 4A, first, it is determined whether or not it is a learning target period (step S101). The learning target period corresponds to, for example, the first operation of the image determination apparatus 10 or a predetermined period in which the normal model 12c and the abnormal classification model 12d are created before operation.

  Here, if it is the learning target period (step S101, Yes), the first learning unit 11b generates the normal model 12c by machine learning from the learning data set of the normal image (that is, the first learning data set 12a). (Step S102).

  Then, the acquiring unit 11a receives the setting of the abnormal target area and the classification class in the abnormal image detected in the past (Step S103). Then, the second learning unit 11c generates the abnormality classification model 12d by machine learning from the learning data set of the abnormal target area (that is, the second learning data set 12b) (Step S104).

  If the period is not the learning target period (No at Step S101), the process proceeds to Step S105 (see FIG. 4B).

  Subsequently, as illustrated in FIG. 4B, the evaluation unit 11d inputs the determination target image to the normal model 12c, and calculates the degree of deviation of each pixel from the normal state (Step S105). Then, the first determination unit 11e determines whether the sum of the degrees of deviation is larger than a predetermined determination threshold (Step S106).

  Here, if the sum of the divergence degrees is larger than the predetermined determination threshold value (step S106, Yes), the first determination unit 11e notifies an abnormality (step S107), and subsequently determines whether or not an erroneous detection is made. (Step S108). If the sum of the divergence degrees is equal to or smaller than the predetermined determination threshold value (No in step S106), the process ends.

  When it is determined in Step S108 that the detection is not erroneous (Step S108, No), the first determination unit 11e extracts the abnormal target area RX from the mapping image 12eb in which the dissociation degree is assigned to each pixel (Step S109). .

  On the other hand, if it is determined in step S108 that the detection is erroneous (step S108, Yes), the first learning unit 11b updates the normal model 12c by machine learning using the corresponding image as additional learning (step S110). , And the process ends.

  After step S109, the evaluation unit 11d inputs the abnormality target region RX to the abnormality classification model 12d, and calculates the classification result and the similarity (step S111).

  Then, the second determination unit 11f determines whether or not the similarity is greater than a predetermined determination threshold (Step S112). Here, when the similarity is larger than the predetermined determination threshold (Step S112, Yes), the second determination unit 11f notifies the classification result (Step S113).

  On the other hand, when the similarity is equal to or smaller than the predetermined determination threshold (No at Step S112), the acquiring unit 11a receives additional setting of the classification class of the corresponding abnormal target region RX via the operation unit 20 or the like (Step S114). .

  Then, the second learning unit 11c updates the abnormality classification model 12d by machine learning using the corresponding abnormality target region RX as additional learning (step S115), and ends the process.

  Note that the image determination device 10 according to the above-described embodiment is realized by, for example, a computer 60 having a configuration as shown in FIG. FIG. 5 is a hardware configuration diagram illustrating an example of a computer that realizes the functions of the image determination device 10. The computer 60 includes a CPU (Central Processing Unit) 61, a RAM (Random Access Memory) 62, a ROM (Read Only Memory) 63, a HDD (Hard Disk Drive) 64, a communication interface (I / F) 65, and an input / output interface (I / F) 66 and a media interface (I / F) 67.

  The CPU 61 operates based on a program stored in the ROM 63 or the HDD 64 and controls each unit. The ROM 63 stores a boot program executed by the CPU 61 when the computer 60 is started, a program depending on hardware of the computer 60, and the like.

  The HDD 64 stores a program executed by the CPU 61, data used by the program, and the like. The communication interface 65 receives data from another device via the communication network, sends the data to the CPU 61, and transmits the data generated by the CPU 61 to the other device via the communication network.

  The CPU 61 controls an output device such as a display and a printer and an input device such as a keyboard and a mouse via the input / output interface 66. The CPU 61 acquires data from an input device via the input / output interface 66. Further, the CPU 61 outputs the generated data to the output device via the input / output interface 66.

  The media interface 67 reads a program or data stored in the recording medium 68 and provides the program or data to the CPU 61 via the RAM 62. The CPU 61 loads the program from the recording medium 68 onto the RAM 62 via the media interface 67, and executes the loaded program. The recording medium 68 is, for example, an optical recording medium such as a DVD (Digital Versatile Disc), a PD (Phase Change Rewritable Disk), a magneto-optical recording medium such as an MO (Magneto-Optical disk), a tape medium, a magnetic recording medium, or a semiconductor memory. And so on.

  For example, when the computer 60 functions as the image determination device 10 according to the embodiment, the CPU 61 of the computer 60 implements each function of the control unit 11 by executing a program loaded on the RAM 62. The data in the storage unit 12 is stored in the HDD 64. Although the CPU 61 of the computer 60 reads these programs from the recording medium 68 and executes them, as another example, these programs may be obtained from another device via a communication network.

  As described above, the image determination device 10 according to the embodiment includes the first learning unit 11b, the second learning unit 11c, the evaluation unit 11d, the first determination unit 11e, and the second determination unit 11f. . The first learning unit 11b generates a normal model 12c that models a normal image by executing first learning that is machine learning using a learning image serving as a positive example. The second learning unit 11c generates an abnormality classification model 12d that models a pattern of an abnormal image by executing a second learning that is a machine learning using a learning image that is a negative example. The evaluation unit 11d evaluates the degree of deviation of the determination target image from the normal state based on the output value of the normal model 12c obtained by inputting the determination target image to the normal model 12c. The first determination unit 11e determines whether the determination target image is abnormal based on the degree of divergence. When the first determination unit 11e determines that the determination target image is abnormal, the second determination unit 11f outputs the output value of the abnormality classification model 12d obtained by inputting the determination target image to the abnormality classification model 12d. The above pattern is determined based on the pattern.

  Therefore, according to the image determination device 10 according to the present embodiment, it is possible to easily and accurately perform image determination using machine learning.

(Other embodiments)
By the way, in the above-described embodiment, the case where the image determination device 10 is used for detecting defective gears has been described as an example, but the application of the image determination device 10 is of course not limited. For example, the image determination device 10 can be used for determining whether a parking lot is full or empty.

  6A and 6B are diagrams (part 1) and (part 2) showing specific examples of a learning image according to another embodiment. 6C and 6D are diagrams (part 1) and (part 2) showing specific examples of the determination target image according to another embodiment.

  The vacancy determination is for determining the availability of a parking space in a parking lot. When the fully empty state of the parking lot is taken as a positive example in the full / vacancy determination, the learning images are as shown in FIGS. 6A and 6B.

  As shown in FIGS. 6A and 6B, even in the same completely empty state, for example, images having different illuminances are acquired as learning images, thereby absorbing variations in results due to differences in time zones, weather, and the like. This makes it possible to perform the determined image determination. Note that, as already described, the difference is not limited to the illuminance, but may be, for example, luminance, saturation, or angle.

  In the full / vacancy determination, the availability of the parking space in the parking lot is determined by the normal model 12c and the abnormal classification model 12d created based on the learning images. For example, assume that the image shown in FIG. 6C is input as a determination target image.

  In such a case, it can be determined at least that the vehicle is not completely empty based on the degree of deviation obtained from the normal model 12c. Further, if it is determined that there is an abnormality, the area where the vehicle is parked in FIG. 6C, that is, the areas corresponding to the areas R11 and R12 shown in FIG. 6A are extracted as the abnormality target area RX. Then, by inputting the abnormality target region RX to the abnormality classification model 12d, it is possible to obtain a classification result indicating that the vehicle is parked in the regions R11 and R12, for example. In addition, the pattern of the classification result is not limited to when the vehicle is parked, and can cover various situations such as presence of a person, presence of oil spots, and shadows.

  FIG. 6C shows an image of “illuminance: high”, which can be determined with high accuracy by learning the learning image of “illuminance: high” as shown in FIG. 6A. Becomes

  Also, for example, it is assumed that the image shown in FIG. 6D is input as the determination target image. In this case as well, it can be determined at least that the vehicle is not completely empty based on the degree of divergence obtained from the normal model 12c, and then the vehicle is parked in FIG. 6D, that is, the regions R13 and R14 shown in FIG. 6B. The corresponding area is extracted as the abnormal target area RX. Then, by inputting the abnormality target region RX into the abnormality classification model 12d, it is possible to obtain a classification result that the vehicle is parked in the regions R13 and R14, for example. FIG. 6D shows an image of “illuminance: low”, which can be accurately determined by learning the learning image of “illuminance: low” as shown in FIG. 6B. Becomes

  The examples shown in FIGS. 6C and 6D show a case where an ordinary passenger car is parked. However, an image in which different types of vehicles, such as light vehicles and trucks, are parked for each parking space. Is used as the learning image, it is possible to specify even the type of the vehicle parked in the corresponding parking space. It is also possible to notify the number of remaining parking spaces by counting the number of parking spaces where it is determined that the vehicle is parked.

  In the above-described embodiment, VAE or GAN is used as the machine learning algorithm, but the algorithm to be used is not limited. Therefore, the normal model 12c and the abnormal classification model 12d may be generated by executing machine learning by a regression analysis technique such as support vector regression using a pattern classifier such as an SVM (Support Vector Machine). Here, the pattern classifier is not limited to the SVM, and may be, for example, AdaBoost. Further, a random forest or the like may be used.

  Further effects and modifications can be easily derived by those skilled in the art. For this reason, the broader aspects of the present invention are not limited to the specific details and representative embodiments shown and described above. Accordingly, various modifications may be made without departing from the spirit or scope of the general inventive concept as defined by the appended claims and equivalents thereof.

Reference Signs List 10 image determination device 11a acquisition unit 11b first learning unit 11c second learning unit 11d evaluation unit 11e first determination unit 11f second determination unit 12a first learning data set 12b second learning data set 12c normal model 12d abnormality classification Model 12e Evaluation information 12ea Deviation information 12eb Mapping image 12ec Classification result 12ed Similarity

Claims (7)

A first learning step of generating a normal model in which a normal image is modeled by executing first learning which is machine learning using a learning image serving as a positive example;
A second learning step of generating an abnormal classification model that models a pattern of an abnormal image by executing second learning that is machine learning using the learning image as a negative example;
An evaluation step of evaluating a degree of deviation from a normal state of the determination target image based on an output value of the normal model obtained by inputting the determination target image to the normal model,
A first determination step of determining whether the determination target image is abnormal based on the divergence degree;
When the determination target image is determined to be abnormal by the first determination step, the pattern is determined based on an output value of the abnormality classification model obtained by inputting the determination target image to the abnormality classification model. A second determining step of determining. The first determination step includes:
When the determination result that the determination target image is abnormal is determined to be erroneous detection based on the divergence degree, the first learning using the determination target image as additional learning is performed in the first learning step. 2. The image determination method according to claim 1, wherein the normal model is updated by executing the method. The first determination step includes:
When the determination target image is abnormal, the degree of divergence is extracted from the mapping image assigned to each pixel based on the degree of divergence based on the degree of divergence, and the abnormal target area is input to the abnormality classification model,
The second determination step includes:
The image determination method according to claim 1, wherein the pattern is determined based on a classification result obtained by inputting the abnormal target area to the abnormality classification model and a degree of similarity to the classification result. . The second determination step includes:
When the similarity is equal to or less than a predetermined determination threshold, the abnormality classification model is updated by causing the second learning step to execute the second learning in which the abnormality target area is added learning. The image determination method according to claim 3. The second learning step includes:
Performing the second learning using the normal image indicating an outlier as the learning image serving as the negative example;
The second determination step includes:
The method according to claim 3, wherein when the similarity is greater than a predetermined determination threshold and the classification result indicates the normal image having the outlier, the determination target image is notified as being normal. Or the image determination method according to 4. A first learning unit that generates a normal model in which a normal image is modeled by executing first learning that is machine learning using a learning image serving as a positive example;
A second learning unit that generates an abnormality classification model that models an abnormal image pattern by executing second learning that is machine learning using the learning image that is a negative example;
An evaluation unit that evaluates a degree of deviation from a normal state of the determination target image based on an output value of the normal model obtained by inputting the determination target image to the normal model,
A first determination unit that determines whether the determination target image is abnormal based on the divergence degree;
When the first determination unit determines that the determination target image is abnormal, the pattern is determined based on an output value of the abnormality classification model obtained by inputting the determination target image to the abnormality classification model. And a second determination unit for determining. A first learning procedure of generating a normal model in which a normal image is modeled by executing first learning which is machine learning using a learning image serving as a positive example;
A second learning procedure of generating an abnormal classification model that models an abnormal image pattern by executing second learning that is machine learning using the learning image as a negative example;
An evaluation procedure for evaluating a degree of deviation from a normal state of the determination target image based on an output value of the normal model obtained by inputting the determination target image to the normal model,
A first determination procedure for determining whether the determination target image is abnormal based on the divergence degree;
When the determination target image is determined to be abnormal by the first determination procedure, the pattern is determined based on an output value of the abnormality classification model obtained by inputting the determination target image to the abnormality classification model. An image determination program for causing a computer to execute a second determination procedure for determining.