JP2020046731A - Determination result output device, determination result output method, and determination result output program - Google Patents

Abstract

To conduct quality determination of an object simply with high accuracy.SOLUTION: A determination result output device (1A) comprises: a primary quality determination unit (10) which conducts quality determination of an object, in each of learned CNN (11), by applying the plurality of learned CNN (11) different from each other, to image data; and a final quality determination unit (20) which finally determines that the object is a defective product, when at least one quality determination result of the quality determination results by the primary quality determination unit (10), indicates that the object is the defective product.SELECTED DRAWING: Figure 1

Description

  The present invention relates to a determination result output device and the like.

  Conventionally, an image recognition technology using a neural network has been developed. Examples of such an image recognition technique are disclosed in Patent Documents 1 and 2, for example.

  In the pattern recognition learning method of Patent Document 1, a plurality of representative local features are selected from a plurality of local features detected from input image data, and a learning data set including the representative local features as teacher data is used. You are building a trained model.

  In the image recognition system of Patent Document 2, an entire image of a subject and an image of a predetermined portion are extracted from image data, and learning is performed for each extracted image using an individual determination neural network. A learned model is constructed by inputting the recognition result of each individual determination neural network to the comprehensive determination neural network. The authentication of the actual subject is performed in the same manner as the method of constructing the learned model.

Japanese Patent No. 4532915 (registered June 18, 2010) JP-A-4-264985 (published on September 21, 1992)

  However, in Patent Literature 1, one learned model is constructed. For example, if a change occurs in the image input to the trained model, the output result is different, and as a result, the target included in the image may be erroneously determined to be good even though it is defective. There is. In order to eliminate the erroneous determination, it is necessary to recreate the learned model itself, and in this case, it takes much time and effort.

  In Patent Literature 2, in order to authenticate a subject using an image including an entire image and an image of a predetermined portion, a final determination result is input by inputting a determination result in an individual determination neural network to a comprehensive determination neural network. Have acquired. Therefore, it is difficult to output the determination result more easily.

  An object of one embodiment of the present invention is to realize a determination result output device or the like that can easily and accurately determine the acceptability of an object.

  In order to solve the above problem, a determination result output device according to an aspect of the present invention is a determination result output device that outputs a result of a pass / fail determination of a target included in input image data, By applying a plurality of different learned models to the data, in each of the learned models, a primary pass / fail determination unit that performs pass / fail determination of the object, and a pass / fail determination of the primary pass / fail determination unit by the primary pass / fail determination unit. When at least one of the results of the pass / fail determination indicates that the target is a defective product, the device further includes a final pass / fail determination unit that finally determines that the target is a defective product.

  Further, in order to solve the above problem, a determination result output method according to an aspect of the present invention is a determination result output method for outputting a result of a pass / fail determination of an object included in input image data, By applying a plurality of different learned models to the image data, in each of the learned models, a primary pass / fail determination step of performing pass / fail determination of the object, and the pass / fail in the primary pass / fail determination step When at least one of the determination results indicates that the object is defective, a final quality determination step for finally determining that the object is defective. .

  According to one embodiment of the present invention, the quality of an object can be easily and accurately determined.

FIG. 2 is a block diagram illustrating a main configuration of a determination result output device according to the first embodiment of the present invention. It is a conceptual diagram which shows the learning process in the production | generation of the learned CNN provided in the said determination result output apparatus, (a) is a conceptual diagram which shows the learning process in production | generation of the learned CNN, It is a conceptual diagram which shows a test process. It is a table | surface which shows the example of the determination result using the said determination result output apparatus. It is a table | surface which shows the other example of the determination result using the said determination result output apparatus.

[Embodiment 1]
Hereinafter, the determination result output device 1A according to the embodiment of the present invention will be described in detail.

  The determination result output device 1A is a device that determines whether the target object is a non-defective product or a defective product based on image data obtained by capturing the target object by the imaging device 2, and outputs the determination result. In other words, the determination result output device 1A is a device that outputs the result of the pass / fail determination of the object included in the input image data. In the present embodiment, a sheet metal is described as an example of the object. The criteria for determining the quality of the sheet metal include, for example, the presence / absence of a flaw, color, and chipping.

(Configuration of the determination result output device 1A)
FIG. 1 is a block diagram illustrating a main configuration of the determination result output device 1A. As shown in FIG. 1, the determination result output device 1 </ b> A includes a primary pass / fail determination unit 10, a final pass / fail determination unit 20, and a display unit 30.

  As shown in FIG. 1, the primary pass / fail determination unit 10 includes a plurality of learned CNNs (Convolutional Neural Network) 11. The learned CNN 11 outputs a result of the pass / fail determination of the target included in the input image data. Each of the learned CNNs 11 is a learned model created in advance by machine learning. Each of the learned CNNs 11 has a different model by making the teacher data input in the machine learning different.

  Here, a method of creating the learned CNN 11 (learning method) will be described. The following description is an example, and the present invention is not limited to the case where the learned CNN 11 is created by the method described below. 2A and 2B show a method of creating a learned CNN11. FIG. 2A is a conceptual diagram showing a learning process in creating a learned CNN11, and FIG. 2B shows a test process in creating a learned CNN11. FIG. The learned CNN 11 is created by the learning device 100 as shown in FIG.

  In the method of creating the learned CNN 11, first, a predetermined number (here, 1000 sheets) of sheet metal image data is prepared as teacher data (learning data). Of the teacher data, a predetermined percentage of image data is image data of non-defective sheet metal, and the remaining image data is image data of defective sheet metal. In the teacher data, a good or defective product is specified in advance by the user.

  Next, the learning device 100 creates an image data set 31 (learning data set) for making the CNN learn. The learning device 100 creates the image data set 31 by randomly extracting 200 pieces of teacher data from 1000 pieces of teacher data for each CNN, for example. The image data sets 31 as learning data in the respective CNNs are different image data sets. However, it is not necessary that all the image data of the image data set 31 be different from the image data of the other image data sets 31, and some of the image data of the image data set 31 May overlap. In the present embodiment, the ratio between the number of non-defective image data and the number of defective image data is constant in all the image data sets 31.

Next, the learning device 100 creates a learned model using the image data set 31 for each CNN.
Specifically, the learning device 100 first selects 160 images out of 200 image data as training image data for training the CNN (hereinafter, training image data). Further, the learning device 100 selects the remaining 40 sheets as cross-test image data (hereinafter, cross-test image data) for adjusting the CNN parameters. These selections are also made randomly.

  Next, as shown in FIG. 2A, the learning device 100 learns to the CNN by inputting 40 training image data randomly extracted from the 160 training image data to the CNN. Let it. Next, as shown in FIG. 2 (b), the learning apparatus 100 inputs the cross-validation image data to the learned CNN, so that the CNN parameters can be improved in order to improve the accuracy of the pass / fail determination. To adjust. Next, the learning apparatus 100 inputs 40 training image data different from the previous training image data out of the 160 training image data to further make the CNN learn. Next, the learning device 100 inputs the cross-validation image data to the learned CNN, and further adjusts the parameters of the CNN in order to improve the accuracy of the pass / fail determination. Thereafter, the learning device 100 creates a learned CNN 11 for the first CNN by repeating the process. The learning device 100 creates the learned CNN 11 by causing other CNNs to perform similar learning.

  Here, as described above, in the learning of each learned CNN 11, the image data sets as the learning data are different image data sets. Accordingly, each of the learned CNNs 11 becomes a different learned model from each other.

  Each of the learned CNNs 11 outputs, as a quality determination result for the input image data, whether the object included in the image data is a non-defective product or a defective product. Specifically, each learned CNN 11 calculates a probability (determination accuracy) that the target object can be determined to be a non-defective product, and compares the calculated result with a threshold value serving as a criterion for the non-defective product to determine whether the target product is non-defective or non-defective. The result of the determination as a good product is output.

  Each learned CNN 11 outputs, for example, a result of determination as a non-defective or defective product by comparing the target with a threshold value, which is an index for determining a non-defective product. Each of the learned CNNs 11 determines a non-defective product if the probability of being determined as a non-defective product is equal to or greater than a threshold value, and determines a defective product if the probability is less than the threshold value.

  For example, this is a case where the output result from any learned CNN 11 indicates that the probability of being determined to be good is 65% and the probability of being determined to be defective is 35%. Consider a case where the threshold value of the learned CNN 11 is set to 50%. In this case, since the output result (probability of being determined to be good: 65%) exceeds the threshold value (50%), the learned CNN 11 determines that the target object is good.

  However, the threshold value may be a threshold value which is an index for determining a target object as a defective product. In this case, each learned CNN 11 calculates the probability that the target object can be determined to be a defective product, and compares the calculation result with the threshold to output a determination result of a good or defective product. Each of the learned CNNs 11 determines that the product is defective if the probability that the product can be determined to be defective is equal to or larger than the threshold, and determines that the product is non-defective if the probability is less than the threshold.

  For example, this is a case where the output result from any learned CNN 11 indicates that the probability of being determined to be good is 65% and the probability of being determined to be defective is 35%. Consider a case where the threshold value of the learned CNN 11 is set to 50%. In this case, since the output result (probability of being determined to be defective: 35%) is less than the threshold value (50%), the learned CNN 11 determines that the target object is a non-defective product.

  The threshold is set for each learned CNN 11. As described above, since the image data sets 31 input to the CNN for creating each learned CNN 11 are different, each learned CNN 11 is a different learned model from each other. Therefore, even if the same image data is input to each learned CNN 11, the output results (the above probabilities) may be different. Therefore, in order to obtain a pass / fail judgment result with high accuracy in each learned CNN 11, a threshold suitable for each learned CNN 11 is set through an experiment or the like.

  As described above, since the plurality of learned CNNs 11 have different models, when the same image data is input, there is a possibility that the respective output results are different. Each of the plurality of learned CNNs 11 outputs the result of the pass / fail determination of the target included in the input image data to the final pass / fail determination unit 20.

  The final pass / fail determination unit 20 determines the final pass / fail of the target object in the determination result output device 1A. The final pass / fail judgment unit 20 makes a final pass / fail judgment using the pass / fail judgment results of the object output from the plurality of learned CNNs 11, respectively. Specifically, the final acceptability judgment unit 20 judges the target object as an acceptable item only when all the acceptability results input from the plurality of learned CNNs 11 indicate an acceptable item. In other words, if at least one of the determination results input from the plurality of learned CNNs 11 indicates a defective product, the final pass / fail determination unit 20 determines that the target object is a defective product. It is determined that there is. The final pass / fail judgment unit 20 outputs the result of the pass / fail judgment to the display unit 30.

  The display unit 30 is a device that displays the determination result input from the final pass / fail determination unit 20. The display unit 30 is not particularly limited, but is, for example, a liquid crystal display. The output device that outputs the determination result is not limited to the display unit 30, and may be a speaker or the like that outputs the determination result as sound. The output device does not necessarily need to be provided in the determination result output device 1A, and may be provided as an external device of the determination result output device 1A that can be connected to the determination result output device 1A.

(Operation of the determination result output device 1A)
Next, the operation of the determination result output device 1A in this embodiment (in other words, the determination result output method) will be described with reference to FIG.

  The determination result output method according to the present embodiment includes a primary pass / fail determination step and a final pass / fail determination step.

  In the primary pass / fail determination step, pass / fail determination of image data (object) is performed in each of the learned CNNs 11 by applying a plurality of different learned CNNs 11 to each other. In the final pass / fail judgment step, a final pass / fail judgment of the target object is performed based on a plurality of pass / fail judgment results in the primary pass / fail judgment step.

  In the determination result output method according to the embodiment, as shown in FIG. 1, first, image data of an object for which the quality determination is performed is input to each learned CNN 11 included in the primary quality determination unit 10. Next, each learned CNN 11 makes a pass / fail judgment on the image data (primary pass / fail judgment step). Next, each learned CNN 11 outputs a pass / fail determination result to the final pass / fail determination unit 20.

  Next, the final pass / fail judgment unit 20 judges the final pass / fail of the target object based on the pass / fail judgment results output from the plurality of learned CNNs 11 (final pass / fail judgment step). Specifically, the final acceptability judgment unit 20 judges the target object as an acceptable product only when all the acceptability results output from the plurality of learned CNNs 11 indicate that the product is acceptable. In other words, the final pass / fail determination unit 20 determines that the object is a defective product when at least one of the plurality of determination results input from the primary pass / fail determination unit 10 indicates a defective product. Is determined. A specific example will be described with reference to FIG.

  FIG. 3 is a table showing an example of a determination result using the determination result output device 1A in the present embodiment. In FIG. 3, in order to distinguish a plurality of (five) learned CNNs 11, the learned CNN 11A to the learned CNN 11E are described.

  As shown in FIG. 3, for the product A, all of the learned CNN 11A to the learned CNN 11E are determined to be good. For this reason, the final pass / fail determination unit 20 determines that the product A is a non-defective product. On the other hand, for the products B to D, at least one of the learned CNN 11A to the learned CNN 11E is determined to be defective. For this reason, the final pass / fail determination unit 20 determines that the products B to D are defective.

  Here, in the image data input to the learned CNN 11, the number of pixels of the image data of the image of the object or the direction of imaging the object in the image data may change from the time of learning. In other words, the image input to the learned CNN 11 may change. In such a case, conventionally, only one trained CNN is used to determine the quality, so if the image data changes from the time of learning, the defective product may be determined as a good product.

  On the other hand, in the determination result output device 1A, the quality determination result of the object is input from the plurality of learned CNNs 11 to the final quality determination unit 20. Then, the final good / bad judgment unit 20 judges the target object as a good product only when the result of all the good / bad judgments input from the plurality of learned CNNs 11 indicates a good product judgment. In other words, the final pass / fail determination unit 20 determines that the target is a defective product when at least one of the determination results input from the plurality of learned CNNs 11 indicates a defective product. Is determined.

  According to the above configuration, even when the image input to the learned CNN 11 changes, unless all of the plurality of learned CNNs 11 are determined to be non-defective, they are not finally determined to be non-defective. Can be reduced as a non-defective product.

  Next, a case in which the same object (product E) is changed in resolution (number of pixels) of image data and a non-defective item is judged by the judgment result output device 1A of the present invention will be described with reference to FIG. . FIG. 4 is a table showing an example of a determination result using the determination result output device 1A. The example illustrated in FIG. 4 illustrates a determination result when the resolution of the image data is reduced as going from Example 1 to Example 4. Here, description will be made assuming that the product E is defective.

  In the quality judgment using the CNN, the quality judgment may differ depending on the resolution of the image data. Therefore, when the pass / fail judgment is performed using one learned CNN as in the related art, the possibility of erroneous judgment increases.

  On the other hand, in the determination result output device 1A, as shown in FIG. 4, some of the learned CNNs 11 out of the plurality of learned CNNs 11 due to the decrease in the resolution of the image data as shown in Examples 2 and 3. Even if the target object is determined to be good, it can be determined that the target object is defective by determining that the other learned CNN 11 is defective. Therefore, the quality of the object can be determined with higher accuracy than before. In addition, the determination result output device 1A does not require a comprehensive determination neural network as in Patent Literature 2, so that the determination can be performed easily and accurately.

  In the present embodiment, when creating the learned CNN 11, the ratio between the number of non-defective image data and the number of defective image data included in all the image data sets 31 is fixed. This makes it easy to set parameters in the learned CNN 11.

  However, in one embodiment of the present invention, in creating the learned CNN 11, the ratio of the number of non-defective image data to the number of defective image data included in each image data set 31 may be different. . As a result, different characteristics can be provided for each of the learned CNNs 11. As a result, the determination accuracy of each learned CNN 11 can be made different depending on the image data, and the possibility of determining a defective product as a good product can be reduced.

The feature of the learned CNN 11 includes, for example, a feature that makes it easy to determine that it is a non-defective product or a defective product, or that performs a pass / fail judgment specialized for a specific index such as the presence or absence of a flaw, color, and chipping. For example,
(1) The number of image data to be included in the image data set 31,
(2) the ratio of the number of non-defective image data to the number of defective image data included in the image data set 31;
(3) The ratio of the number of training image data to the number of cross-validation image data to be included in the image data set 31,
(4) resolution of image data to be included in the image data set 31,
(5) The type of image data to be included in the image data set 31 (eg, a color image or a binary image), or
(6) Various parameters to be set at the time of learning (example: optimization function),
It is also possible to change the above characteristics by changing various conditions such as the above. By changing at least one of the above conditions, the learning device 100 learns, for example, the learned CNN 11 that is strong in identifying flaws (even a slight scratch is determined to be defective), or is strong in identifying chipping (slight chipping). However, it is also possible to create a learned CNN 11). Naturally, the characteristics of the learned CNN 11 also differ depending on the state of the target included in the prepared teacher data.

  In the present embodiment, the learning device 100 randomly selects a predetermined number of image data from the prepared image data. However, the present invention is not limited to this. For example, all of the prepared image data is input to each CNN. It does not matter. In addition, a predetermined number of image data groups (image data that is the basis of each image data set 31) selected to be input to each CNN may be the same. However, in this case, for example, at least one of the above (2) to (6) is made different in each of the image data sets 31 input to each CNN.

  The learning device 100 may create different learned CNNs 11 (learned CNNs 11 having different characteristics) by changing at least one of the above conditions for each learned CNN 11. In this case, the learning device 100 can also create, for example, a learned CNN 11 that is strong in identifying a flaw and a learned CNN 11 that is strong in identifying a chip, and have the primary good / bad judgment unit 10 have it.

  However, it is extremely difficult to specify which setting is changed and how much to give a predetermined characteristic. Therefore, in the present embodiment, the image data set 31 is created by randomly selecting teacher data in order to easily generate different learned CNNs 11 from each other.

  In the determination result output device 1A of the present embodiment, the primary acceptability determination unit 10 uses a plurality of learned CNNs 11. However, the determination result output device of the present invention is not limited to this. The determination result output device of the present invention may be any device in which the primary pass / fail determination unit applies a plurality of learned models for performing pass / fail determination. For example, instead of CNN, a support vector machine, a decision tree, a random forest, and the like May be used.

[Embodiment 2]
Another embodiment of the present invention will be described below. For convenience of description, members having the same functions as those described in the above embodiment are denoted by the same reference numerals, and description thereof will not be repeated.

  In the first embodiment, the threshold value used in each learned CNN 11 is predetermined. On the other hand, in the present embodiment, the threshold value used in each learned CNN 11 can be adjusted. That is, the threshold value of each learned CNN 11 is set to be adjustable (changeable). For example, the threshold value used in each learned CNN 11 is adjusted by an operation unit (not shown) receiving a user operation.

  For example, consider a case in which the output result from an arbitrary learned CNN 11 indicates that the probability of being determined as a non-defective product is 65% and the probability of being determined as a defective product is 35%.

  When the threshold value of the learned CNN 11 is an index for determining that the target object is non-defective, and the threshold value is set to 50%, the output result (probability of determining non-defective product: 65%) indicates the threshold value ( 50%). Therefore, the learned CNN 11 determines that the target object is good. On the other hand, when the threshold value of the learned CNN 11 is set to 70%, the output result (probability that it can be determined to be non-defective: 65%) is less than the threshold value (70%). Therefore, the learned CNN 11 determines that the target object is defective.

  In addition, when the threshold value of the learned CNN 11 is an index for determining the target object as a defective product and the threshold value is set to 50%, an output result (probability of determining the defective product: 35%) Is less than the threshold (50%), the learned CNN 11 determines that the target object is good. On the other hand, if the threshold value of the learned CNN 11 is set to 30%, the output result (probability of determining a defective product: 35%) exceeds the threshold value (30%). The object is determined to be defective.

  As described above, by adjusting the threshold value, it is possible to make the judgment result output from the learned CNN 11 different even when the image data to be judged (the object to be judged) is the same.

  For example, the threshold value (threshold value for non-defective product) which is an index for judging the target object as non-defective is increased, and the threshold value (threshold value for defective product) which is an index for judging the target object as non-defective product is decreased. Thus, the possibility that the target object is determined to be defective can be increased. That is, by adjusting the threshold value for non-defective products to be increased or the threshold value for defective products to be decreased, the determination result output device 1A can determine the target to be determined more strictly.

  In other words, the determination result output device 1A can reliably determine that the defective object is a defective product based on the image data to be determined by performing the adjustment as described above. Therefore, the determination result output device 1A can further suppress erroneous determinations such as outputting a determination result of a non-defective product when the target is a defective product. That is, the determination result output device 1A can output the pass / fail determination result with higher accuracy.

[Example of software implementation]
The control blocks (especially, the primary quality determination unit 10 and the final quality determination unit 20) of the determination result output device 1A may be realized by a logic circuit (hardware) formed on an integrated circuit (IC chip) or the like, It may be realized by software.

  In the latter case, the determination result output device 1A includes a computer that executes instructions of a program that is software for realizing each function. This computer includes, for example, one or more processors and a computer-readable recording medium storing the program. Then, in the computer, the object of the present invention is achieved by the processor reading the program from the recording medium and executing the program. As the processor, for example, a CPU (Central Processing Unit) can be used. As the recording medium, a "temporary tangible medium" such as a ROM (Read Only Memory), a tape, a disk, a card, a semiconductor memory, and a programmable logic circuit can be used. Further, a RAM (Random Access Memory) for expanding the program may be further provided. Further, the program may be supplied to the computer via an arbitrary transmission medium (a communication network, a broadcast wave, or the like) capable of transmitting the program. Note that one embodiment of the present invention can also be realized in the form of a data signal embedded in a carrier wave, in which the program is embodied by electronic transmission.

  The present invention is not limited to the embodiments described above, and various modifications are possible within the scope shown in the claims, and embodiments obtained by appropriately combining technical means disclosed in different embodiments. Is also included in the technical scope of the present invention.

1A Determination result output device 10 Primary pass / fail determination unit 11 Learned CNN (learned model)
20 Final pass / fail judgment unit 31 Image data set

Claims (5)

A determination result output device that outputs the result of the quality determination of the object included in the input image data,
For the image data, by applying a plurality of different learned models different from each other, in each of the learned models, a primary quality determination unit that determines the quality of the object,
When at least one of the results of the pass / fail determination by the primary pass / fail determination unit indicates that the target is defective, the target is finally determined to be defective. A determination result output device comprising: a final pass / fail determination unit.   The primary pass / fail judgment unit has image data as a plurality of learning data, and is created by inputting each of a plurality of learning data sets in which at least some of the image data are different from each other to a convolutional neural network. The determination result output device according to claim 1, wherein a learned model is used as each of the plurality of learned models.   The determination result output device according to claim 1, wherein a threshold value serving as a criterion of pass / fail determination is set to be changeable for each of the plurality of learned models. A determination result output method for outputting a result of the quality determination of the object included in the input image data,
By applying a plurality of different learned models to the image data, in each of the learned models, a primary pass / fail determination step of performing pass / fail determination of the object,
If at least one of the results of the pass / fail determination in the primary pass / fail determination step indicates that the target is a defective product, it is finally determined that the target is a defective product. And a final pass / fail judgment step.   A determination result output program for causing a computer to function as the determination result output device according to claim 1, wherein the determination result output program causes a computer to function as the primary pass / fail determination unit and the final pass / fail determination unit.

Images