JP7215030B2 - Judgment result output device, judgment result output method, and judgment result output program - Google Patents

Description

The present invention relates to a determination result output device and the like.

Conventionally, image recognition techniques using neural networks have been developed. Examples of such image recognition techniques are disclosed in Patent Documents 1 and 2, for example.

In the pattern recognition learning method of Patent Document 1, a plurality of representative local features are selected from a plurality of local features detected from input image data, and a learning data set containing the representative local features as teacher data is used. Building a trained model.

In the image recognition system disclosed in Patent Document 2, an image of the entire subject and an image of a predetermined portion are extracted from image data, and an individual judgment neural network is used for learning for each extracted image. A trained model is constructed by inputting the recognition result of each individual judgment neural network to the comprehensive judgment neural network. Authentication of an actual subject is also performed by a method similar to the method of constructing a learned model.

Japanese Patent No. 4532915 (registered June 18, 2010) Japanese Patent Application Laid-Open No. 4-264985 (published on September 21, 1992)

However, in Patent Literature 1, only one trained model is constructed. For example, if there is a change in the image input to the trained model, the output result will be different, and as a result, it is possible to erroneously determine that the object contained in the image is a good product even though it is a defective product. have a nature. In order to eliminate erroneous determinations, it is necessary to recreate the trained model itself, which is extremely time-consuming.

In Patent Document 2, in order to authenticate a subject using an image including an entire image and an image of a predetermined portion, the final determination result is obtained by inputting the determination result of the individual determination neural network to the overall determination neural network. have obtained. Therefore, it is difficult to output the determination result more easily.

An object of one aspect of the present invention is to realize a determination result output device or the like that can easily and accurately determine the quality of an object.

In order to solve the above problems, a determination result output device according to one aspect of the present invention is a determination result output device that outputs a result of quality determination of an object included in input image data, the image By applying a plurality of learned models different from each other to the data, in each of the learned models, a primary pass/fail judgment unit that judges the quality of the object, and the pass/fail judgment by the primary pass/fail judgment unit. a final quality determination unit that finally determines that the object is defective when at least one quality determination result out of the results indicates that the object is defective.

Further, in order to solve the above problems, a determination result output method according to one aspect of the present invention is a determination result output method for outputting a result of quality determination of an object included in input image data, By applying a plurality of different learned models to the image data, a primary pass/fail judgment step of judging pass/fail of the object in each of the trained models; a final pass/fail judgment step of finally judging that the object is defective when at least one pass/fail judgment result out of the judgment results indicates that the object is defective. .

According to one aspect of the present invention, it is possible to easily and accurately determine the quality of an object.

1 is a block diagram showing a configuration of a main part of a determination result output device according to Embodiment 1 of the present invention; FIG. It shows a method of creating a learned CNN provided in the determination result output device, (a) is a conceptual diagram showing a learning process in creating a learned CNN, and (b) is a diagram in creating a learned CNN. It is a conceptual diagram which shows a test process. It is a table|surface which shows the example of the determination result using the said determination result output apparatus. It is a table showing another example of determination results using the determination result output device.

[Embodiment 1]
A determination result output device 1A according to an embodiment of the present invention will be described in detail below.

The determination result output device 1A is a device that determines whether an object is a non-defective product or a defective product based on image data of an object imaged by the imaging device 2, and outputs the determination result. In other words, the determination result output device 1A is a device that outputs the result of quality determination of the object included in the input image data. In the present embodiment, a sheet metal will be described as an example of the object. Examples of criteria for judging the quality of sheet metal include the presence or absence of flaws, color, chipping, and the like.

(Configuration of determination result output device 1A)
FIG. 1 is a block diagram showing the essential configuration of the determination result output device 1A. As shown in FIG. 1, the determination result output device 1A includes a primary pass/fail determination unit 10, a final pass/fail determination unit 20, and a display unit 30. As shown in FIG.

The primary pass/fail determination unit 10 includes a plurality of learned CNNs (Convolutional Neural Networks) 11, as shown in FIG. The learned CNN 11 outputs the result of quality judgment of the object included in the input image data. Each trained CNN 11 is a trained model created in advance by machine learning. Each trained CNN 11 is a model different from each other by using different teacher data input in machine learning.

Here, a method of creating (learning method) the learned CNN 11 will be described. In addition, the following description is an example, and the learned CNN 11 in the present invention is not limited to the method described below. FIG. 2 shows a method for creating the trained CNN 11, (a) is a conceptual diagram showing the learning process in creating the trained CNN 11, and (b) shows the test process in creating the trained CNN 11. It is a conceptual diagram showing. The trained CNN 11 is created by the learning device 100 as shown in FIG.

In the method of creating the learned CNN 11, first, a predetermined number (here, 1000 sheets) of sheet metal image data are prepared as teacher data (learning data). Of the training data, a predetermined proportion of the image data is image data of non-defective sheet metals, and the remaining image data is image data of defective sheet metals. In the training data, whether the product is good or bad is specified in advance by the user.

Next, the learning device 100 creates an image data set 31 (learning data set) for CNN to learn. For example, the learning device 100 creates an image data set 31 by randomly extracting 200 sheets of teacher data from 1000 sheets of teacher data for each CNN. The image data sets 31 as learning data in each CNN are assumed to be different image data sets. However, it is not necessary that all the image data of the image data set 31 are different from the image data of the other image data sets 31. may overlap with In this embodiment, the ratio between the number of non-defective image data and the number of defective image data is constant in all the image data sets 31 .

Next, the learning device 100 creates a trained model using the image data set 31 for each CNN.
Specifically, the learning device 100 first selects 160 images out of 200 image data as training image data (hereinafter referred to as training image data) for making the CNN learn. Also, the learning device 100 selects the remaining 40 images as cross-validated image data (hereinafter referred to as cross-validated image data) for adjusting the parameters of the CNN. These selections are also made randomly.

Next, as shown in FIG. 2(a), the learning device 100 inputs 40 training image data randomly extracted from the 160 training image data to the CNN for learning. Let Next, as shown in FIG. 2(b), the learning device 100 inputs cross-validation image data to the learned CNN. to adjust. Next, the learning device 100 further learns the CNN by inputting 40 different training image data out of the 160 training image data. Next, the learning device 100 inputs the cross-validation image data to the learned CNN, and further adjusts the parameters of the CNN in order to improve the accuracy of pass/fail judgment. After that, the learning device 100 repeats the processing to create the learned CNN 11 for the first CNN. Learning device 100 creates learned CNN 11 by causing other CNNs to learn in the same way.

Here, as described above, in the learning of each trained CNN 11, different image data sets are used as learning data. Thereby, each of the trained CNNs 11 becomes a mutually different trained model.

Each learned CNN 11 outputs whether the object included in the image data is a non-defective product or a defective product as a quality judgment result for the input image data. Specifically, each learned CNN 11 calculates the probability (determination accuracy) that it is possible to determine that the object is a non-defective product, and compares the calculation result with a threshold value that is a criterion for determining whether the product is non-defective or defective. Outputs the judgment result of non-defective product.

Each learned CNN 11 outputs a determination result as a non-defective product or a defective product, for example, by comparing the object with a threshold, which is an index for determining a non-defective product. Each learned CNN 11 determines that the product is non-defective if the probability that it can be determined as a non-defective product is equal to or greater than the threshold, and determines that the product is defective if the probability is less than the threshold.

For example, if the output result from any learned CNN 11 indicates that the probability that it can be determined to be a non-defective product is 65% and the probability that it can be determined to be a defective product is 35%, Consider a case where the threshold for the learned CNN 11 is set to 50%. In this case, the learned CNN 11 determines that the target object is a non-defective product because the output result (probability of determining that the object is non-defective: 65%) exceeds the threshold value (50%).

However, the above threshold value may be a threshold value that is an index for determining an object as a defective product. In this case, each learned CNN 11 calculates the probability that the object can be determined to be defective, compares the calculation result with the threshold value, and outputs the determination result as good or bad. Each learned CNN 11 determines that the product is defective if the probability that it can be determined as a defective product is equal to or greater than the threshold, and determines that the product is non-defective if the probability is less than the threshold.

For example, if the output result from any learned CNN 11 indicates that the probability that it can be determined to be a non-defective product is 65% and the probability that it can be determined to be a defective product is 35%, Consider a case where the threshold for the learned CNN 11 is set to 50%. In this case, the learned CNN 11 determines that the target object is non-defective because the output result (probability of determining that the object is defective: 35%) is less than the threshold value (50%).

In addition, the said threshold value is set for every learned CNN11. As described above, since the image data sets 31 input to the CNN for creating each trained CNN 11 are different, each trained CNN 11 is a different trained model. Therefore, even if the same image data is input to each trained CNN 11, the output results (probabilities described above) may differ. Therefore, in order to obtain a pass/fail judgment result in each trained CNN 11 with high accuracy, a threshold suitable for each trained CNN 11 is set through experiments and the like.

As described above, the plurality of learned CNNs 11 are models that differ from each other, so when the same image data is input, there is a possibility that the respective output results will differ. Each of the plurality of trained CNNs 11 outputs the quality determination result of the object included in the input image data to the final quality determination unit 20 .

The final quality determination unit 20 determines the final quality of the object in the determination result output device 1A. The final pass/fail determination unit 20 makes a final pass/fail determination using the pass/fail determination results of the target object output from the plurality of learned CNNs 11 . Specifically, the final pass/fail judgment unit 20 judges the object as a non-defective product only when all the pass/fail judgment results input from the plurality of learned CNNs 11 indicate non-defective product judgment. In other words, when at least one pass/fail judgment result among the judgment results input from the plurality of learned CNNs 11 indicates that the object is defective, the final pass/fail judgment unit 20 determines that the object is a defective product. Determine that there is. The final pass/fail judgment unit 20 outputs the pass/fail judgment result to the display unit 30 .

The display unit 30 is a device that displays the determination result input from the final pass/fail determination unit 20 . The display unit 30 is not particularly limited, but is, for example, a liquid crystal display. The output device for outputting the determination result is not limited to the display unit 30, and a speaker or the like for outputting the determination result by sound may be used. Also, the output device does not necessarily have to be provided in the determination result output device 1A, and may be provided as an external device of the determination result output device 1A that can be connected to the determination result output device 1A.

(Operation of judgment result output device 1A)
Next, the operation of the determination result output device 1A (in other words, determination result output method) in this embodiment will be described with reference to FIG.

The determination result output method of the present embodiment includes a primary pass/fail determination step and a final pass/fail determination step.

In the primary pass/fail judgment step, by applying a plurality of learned CNNs 11 different from each other, pass/fail judgment of image data (object) is performed in each of the learned CNNs 11 . In the final pass/fail judgment step, a final pass/fail judgment of the object is made based on a plurality of pass/fail judgment results in the first pass/fail judgment step.

In the determination result output method of the embodiment, as shown in FIG. 1 , first, image data of an object for which quality determination is to be performed is input to each learned CNN 11 provided in the primary quality determination unit 10 . Next, each trained CNN 11 performs pass/fail judgment on the image data (primary pass/fail judgment step). Next, each trained CNN 11 outputs the quality determination result to the final quality determination section 20 .

Next, the final pass/fail judgment unit 20 judges the final pass/fail of the object based on the pass/fail judgment results output from the plurality of learned CNNs 11 (final pass/fail judgment step). Specifically, the final pass/fail judgment unit 20 judges the object as a non-defective product only when all the pass/fail judgment results output from the plurality of learned CNNs 11 indicate non-defective product judgment. In other words, when at least one pass/fail judgment result among the plurality of judgment results input from the primary pass/fail judgment unit 10 indicates that the object is defective, the final pass/fail judgment unit 20 regards the object as a defective product. It is determined that A specific example will be described with reference to FIG.

FIG. 3 is a table showing an example of determination results using the determination result output device 1A in this embodiment. In FIG. 3, in order to distinguish between a plurality of (five) learned CNNs 11, the learned CNN 11A to the learned CNN 11E are described.

As shown in FIG. 3, with regard to product A, all learned CNN 11A to learned CNN 11E are determined to be non-defective products. Therefore, the final quality determination unit 20 determines that the product A is a non-defective product. On the other hand, for product B to product D, at least one of learned CNN 11A to learned CNN 11E is determined to be defective. Therefore, the final quality determination unit 20 determines that the products B to D are defective.

Here, in the image data input to the learned CNN 11, the number of pixels of the image data of the target object or the direction of the target object in the image data may change from the time of learning. In other words, the images input to the trained CNN 11 may change. In such a case, conventionally, only one trained CNN is used for quality determination, so if the image data changes from the time of learning, there is a possibility that a defective product is determined to be a good product.

On the other hand, in the determination result output device 1A, the final quality determination unit 20 receives the quality determination results of the target object from the plurality of learned CNNs 11 respectively. Then, the final pass/fail judgment unit 20 judges the object as a non-defective product only when all the pass/fail judgment results input from the plurality of learned CNNs 11 indicate non-defective product judgment. In other words, the final quality determination unit 20 determines that the object is defective when at least one of the determination results input from the plurality of learned CNNs 11 indicates that the product is defective. I judge.

According to the above configuration, even if there is a change in the image input to the learned CNN 11, unless all of the learned CNNs 11 are judged to be non-defective, the product is not finally judged to be non-defective. can be reduced.

Next, with regard to the same object (product E), the case where the resolution (the number of pixels) of the image data is changed and non-defective product judgment is performed by the judgment result output device 1A of the present invention will be described with reference to FIG. . FIG. 4 is a table showing an example of determination results using the determination result output device 1A. The example shown in FIG. 4 shows the determination result when the resolution of the image data is lowered from example 1 to example 4. In FIG. Here, it is assumed that the product E is defective.

In pass/fail judgment using CNN, pass/fail judgment may differ depending on the resolution of image data. Therefore, in the case of making pass/fail judgments using one trained CNN as in the conventional art, the possibility of making erroneous judgments increases.

On the other hand, in the determination result output device 1A, as shown in FIG. Even if the target object is determined to be non-defective, it can be determined that the target is defective by determining that another learned CNN 11 is defective. Therefore, it is possible to determine the quality of the object with higher accuracy than in the conventional art. Moreover, since the determination result output device 1A does not require a comprehensive determination neural network as in Patent Document 2, it can be performed easily and accurately.

In the present embodiment, in creating the learned CNN 11, the ratio between the number of non-defective image data and the number of defective image data included in all the image data sets 31 is constant. This makes it easy to set parameters in the learned CNN 11 .

However, in one aspect of the present invention, in creating the trained CNN 11, the ratio between the number of non-defective image data and the number of defective image data included in each image data set 31 may be changed. . Thereby, further different characteristics can be provided for each learned CNN 11 . As a result, it is possible to vary the determination accuracy of each learned CNN 11 depending on the image data, and reduce the possibility of determining a defective product as a non-defective product.

Features of the learned CNN 11 include, for example, the ability to easily determine whether an item is good or defective, or the ability to perform pass/fail determination based on specific indicators such as the presence or absence of flaws, color, and chipping. for example,
(1) the number of image data to be included in the image data set 31;
(2) the ratio of the number of non-defective image data to the number of defective image data included in the image data set 31;
(3) the ratio of the number of training image data to the number of cross-validated image data included in the image dataset 31;
(4) the resolution of the image data included in the image data set 31;
(5) the type of image data to be included in the image data set 31 (eg, color image or binarized image), or
(6) Various parameters set during learning (eg optimization function),
It is also possible to change the above characteristics by changing various conditions such as. By changing at least one of the above conditions, the learning device 100 is, for example, a learned CNN 11 that is strong in identifying flaws (determines even a slight flaw as a defective product), or strong in identifying chipping (slight chipping). However, it is also possible to create a learned CNN 11 that is determined to be defective. Of course, the characteristics of the trained CNN 11 also differ depending on the state of the object included in the prepared teacher data.

Further, in the present embodiment, the learning device 100 randomly selects a predetermined number of image data from the prepared image data. I don't mind. Also, a predetermined number of image data groups (image data that form the basis of each image data set 31) selected to be input to each CNN may be the same. However, in this case, for example, at least one of the above (2) to (6) is made different for each image data set 31 input to each CNN.

Also, the learning device 100 may create different learned CNNs 11 (learned CNNs 11 having different features) by varying at least one of the above conditions for each learned CNN 11 . In this case, the learning device 100 can create, for example, a learned CNN 11 that is strong in identifying flaws and a learned CNN 11 that is strong in identifying defects, and allow the primary pass/fail judgment unit 10 to have them.

However, it is extremely difficult to specify which setting should be changed and how much to give a predetermined characteristic. Therefore, in the present embodiment, the image data set 31 is created by randomly selecting teacher data in order to easily generate different learned CNNs 11 .

In the determination result output device 1A of the present embodiment, the primary pass/fail determination unit 10 uses a plurality of learned CNNs 11, but the determination result output device of the present invention is not limited to this. In the judgment result output device of the present invention, the primary pass/fail judgment unit may apply a plurality of trained models that make pass/fail judgments. For example, instead of CNN, a support vector machine, a decision tree, a random forest, etc. model may be used.

[Embodiment 2]
Other embodiments of the invention are described below. For convenience of description, members having the same functions as those of the members described in the above embodiments are denoted by the same reference numerals, and description thereof will not be repeated.

In Embodiment 1, the threshold used in each learned CNN 11 is determined in advance. On the other hand, in this embodiment, the thresholds used in each learned CNN 11 are adjustable. That is, each learned CNN 11 is set with a threshold that can be adjusted (changed). For example, each threshold used in each learned CNN 11 is adjusted by an operation unit (not shown) receiving a user operation.

For example, consider a case where the output result from an arbitrary learned CNN 11 indicates that the probability of determining that the product is good is 65% and the probability of determining that it is defective is 35%.

When the threshold value of the learned CNN 11 is an index for determining the target object as a non-defective item, and the threshold value is set to 50%, the output result (probability of being able to determine that it is a non-defective item: 65%) is the threshold value ( 50%). Therefore, the learned CNN 11 determines that the object is non-defective. On the other hand, when the threshold value of the learned CNN 11 is set to 70%, the output result (probability of being determined as non-defective: 65%) is less than the threshold value (70%). Therefore, the learned CNN 11 determines that the object is defective.

In addition, when the threshold of the learned CNN 11 is an index for determining the target object as a defective product, and the threshold value is set to 50%, the output result (probability of determining that the product is defective: 35%) is less than the threshold value (50%), the learned CNN 11 determines that the object is non-defective. On the other hand, when the threshold of the learned CNN 11 is set to 30%, the output result (probability of determining that it is defective: 35%) exceeds the threshold (30%), so the learned CNN 11 is the target Determine that the item is defective.

By adjusting the threshold value in this way, even if the image data to be determined (target object to be determined) is the same, it is possible to vary the determination results output from the trained CNN 11 .

For example, as the threshold value (non-defective product threshold value), which is an index for determining a target object as a non-defective product, is increased, the threshold value (defective product threshold value), which is an index for determining a target object as a defective product, is decreased. The more, the more likely it is that the object will be determined to be defective. In other words, by increasing the non-defective product threshold value or decreasing the defective product threshold value, the determination result output device 1A can more strictly determine the object to be determined.

In other words, the determination result output device 1A can reliably determine that an object, which is a defective product, is defective based on the image data to be determined by adjusting as described above. Therefore, the determination result output device 1A can further suppress erroneous determination such as outputting a determination result of a non-defective product when the target object is a defective product. That is, the determination result output device 1A can output the pass/fail determination result with higher accuracy.

[Example of realization by software]
The control blocks of the judgment result output device 1A (in particular, the primary pass/fail judgment unit 10 and the final pass/fail judgment unit 20) may be realized by a logic circuit (hardware) formed in an integrated circuit (IC chip) or the like, It may be realized by software.

In the latter case, the determination result output device 1A is provided with a computer that executes instructions of a program, which is software that implements each function. This computer includes, for example, one or more processors, and a computer-readable recording medium storing the program. In the computer, the processor reads the program from the recording medium and executes it, thereby achieving the object of the present invention. As the processor, for example, a CPU (Central Processing Unit) can be used. As the recording medium, a "non-temporary tangible medium" such as a ROM (Read Only Memory), a tape, a disk, a card, a semiconductor memory, a programmable logic circuit, or the like can be used. In addition, a RAM (Random Access Memory) for developing the above program may be further provided. Also, the program may be supplied to the computer via any transmission medium (communication network, broadcast wave, etc.) capable of transmitting the program. Note that one aspect of the present invention can also be implemented in the form of a data signal embedded in a carrier wave in which the program is embodied by electronic transmission.

The present invention is not limited to the above-described embodiments, but can be modified in various ways within the scope of the claims, and can be obtained by appropriately combining technical means disclosed in different embodiments. is also included in the technical scope of the present invention.

1A Judgment result output device 10 Primary pass/fail judging unit 11 Learned CNN (learned model)
20 final pass/fail judgment unit 31 image data set

Claims (5)

A determination result output device that outputs a result of quality determination of an object included in input image data,
a primary pass/fail judgment unit that applies a plurality of different learned models to the image data to judge pass/fail of the object in each of the trained models;
When at least one pass/fail judgment result among the pass/fail judgment results by the primary pass/fail judging unit indicates that the object is defective, it is finally judged that the object is defective. and a final pass/fail judgment unit,
The primary pass/fail determination unit is a determination result output device that inputs the same image data to each of the plurality of trained models . The primary pass/fail determination unit has image data as a plurality of learning data, and is created by inputting each of a plurality of learning data sets in which at least a part of the image data is different from each other to a convolutional neural network. 2. The determination result output device according to claim 1, wherein a trained model is used as each of said plurality of trained models. 3. The determination result output device according to claim 1, wherein a threshold value serving as a criterion for determining whether the model is good or bad is set to be changeable for each of the plurality of trained models. A determination result output method for outputting a result of quality determination of an object included in input image data,
a primary pass/fail judgment step of applying a plurality of different learned models to the image data to judge pass/fail of the object in each of the trained models;
When at least one pass/fail judgment result among the pass/fail judgment results in the primary pass/fail judgment step indicates that the object is defective, it is finally judged that the object is defective. and a final pass/fail judgment step ,
A determination result output method , wherein in the primary pass/fail determination step, the same image data is input to each of the plurality of trained models . 2. A determination result output program for causing a computer to function as the determination result output device according to claim 1, the determination result output program for causing the computer to function as the primary pass/fail determination section and the final pass/fail determination section.

Images