JP2017151679A - Identification device and identification program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To calculate a value indicative of reliability different from likelihood in a class classification of machine learning.SOLUTION: An identification device includes: a characteristic-amount creation unit 1 that creates an amount of characteristic of input data; an identification unit 2 that classifies the input data to any of a plurality of classes from the amount of characteristic, and creates likelihood information on the plurality of classes; and a reliability degree calculation unit 4 that calculates a reliability degree of an identification result due to likelihood on the basis of the likelihood information on the plurality of classes. The identification device further includes a class independence determination unit 3 that calculates strength of independence of the likelihood among the plurality of classes on the basis of the likelihood information on the plurality of classes upon learning. When the strength of the independence is high, the reliability degree calculation unit is configured to calculate the reliability degree, or determine the identification result in a combination of the weighted reliability and likelihood based on a weighted coefficient in accordance with the strength of the independence.SELECTED DRAWING: Figure 1

Description

  The present invention relates to the field of predicting a class of input data in machine learning.

Conventionally, machine learning that makes a computer act without explicit programming has been studied. The supervised learning in which the target value to be predicted is included in the training data includes an identification (classification) problem for predicting a class. Improvements in reliability, speeding up of processing, and the like are issues.
For example, in the identification device described in Patent Literature 1, in the identification device that performs identification by integrating results using a plurality of weak classifiers, the feature amount is included in a predetermined range (suitable for identification). The weak discriminator is weighted, and the higher the discrimination performance, the higher the contribution to discrimination.
In the identification device described in Patent Document 2, the weak classifiers are rearranged in the order of high usefulness for identification, the outputs are integrated in order, and the calculation is efficiently performed by cutting off when the threshold value is exceeded.

JP 2013-33331 A JP2011-257805A

  However, in the prior art, when the classification results of a plurality of classifiers are divided into a plurality of classes, the likelihood of each class is calculated, but how reliable is the class with the highest likelihood as the correct answer? A value indicating whether it was good was not calculated other than the likelihood.

  The present invention has been made in view of the above problems in the prior art, and an object of the present invention is to calculate a value indicating reliability different from likelihood in class classification of machine learning.

The invention according to claim 1 for solving the above-described problem includes a feature value generation unit that generates a feature value of input data,
An identification unit that classifies input data into any of a plurality of classes from the feature amount, and generates likelihood information of the plurality of classes;
A reliability calculation unit that calculates the reliability of the identification result based on the likelihood based on the likelihood information;
It is an identification device characterized by having.

  The invention according to claim 2 is the identification device according to claim 1, wherein the identification result is determined by combining the reliability calculated by the reliability calculation unit and the likelihood.

The invention according to claim 3 has a class independence determining unit that calculates the strength of independence of likelihood between the plurality of classes based on the likelihood information of the plurality of classes at the time of learning,
The reliability calculation unit calculates the reliability when the identification result based on the likelihood indicates one class among a plurality of classes whose independence strength is lower than a predetermined value, and the likelihood The identification apparatus according to claim 1, wherein the reliability is not calculated when the identification result by indicates a class of a plurality of classes whose independence strength is higher than a predetermined value. is there.

The invention according to claim 4 includes a class independence determination unit that calculates the strength of independence of the likelihood between the plurality of classes based on the likelihood information of the plurality of classes at the time of learning,
The identification result is determined by weighting and combining the reliability calculated by the reliability calculation unit and the likelihood with a weighting coefficient corresponding to the strength of the independence. It is an identification apparatus of description.

  In the invention according to claim 5, the reliability calculation unit determines the class having the maximum likelihood from the difference between the maximum likelihood of any of the classes and the likelihood of the other class, and the identification result based on the likelihood. The identification device according to claim 1, wherein the degree of reliability is calculated.

  In the invention according to claim 6, the reliability calculation unit determines the class having the maximum likelihood from the identification result based on the likelihood based on the ratio between the maximum likelihood of any of the classes and the likelihood of the other classes. The identification device according to claim 1, wherein the degree of reliability is calculated.

  In the invention according to claim 7, the reliability calculation unit calculates the reliability from a difference between likelihood information of a plurality of classes at the time of recognition and likelihood information of a plurality of classes at the time of learning. The identification apparatus according to claim 1, wherein the identification apparatus is characterized in that:

  In the invention according to claim 8, the reliability calculation unit estimates the class having the maximum likelihood from the ratio of the maximum likelihood to the sum of all likelihoods in the likelihood information of the plurality of classes at the time of recognition. The identification device according to claim 1, wherein the reliability for calculating the identification result by degree is calculated.

The invention according to claim 9 is a feature value generation unit for generating a feature value of input data;
An identification unit that classifies input data into any of a plurality of classes from the feature amount, and generates likelihood information of the plurality of classes;
It is an identification program for causing a computer to function as a reliability calculation unit that calculates the reliability of an identification result based on likelihood based on the likelihood information.

  The invention according to claim 10 is the identification program according to claim 9, which causes a computer to realize a function of determining an identification result by combining the reliability calculated by the reliability calculation unit and the likelihood. .

The invention according to claim 11 causes a computer to function as a class independence determination unit that calculates the strength of independence of likelihood between a plurality of classes based on likelihood information of a plurality of classes at the time of learning,
The reliability calculation unit calculates the reliability when the identification result based on the likelihood indicates one class among a plurality of classes whose independence strength is lower than a predetermined value, and the likelihood 10. The function according to claim 9, for causing the computer to realize a function that does not calculate the reliability when the identification result by indicates a class of a plurality of classes whose independence strength is higher than a predetermined value. 10. It is an identification program.

The invention according to claim 12 causes a computer to function as a class independence determination unit that calculates the strength of independence of likelihood between a plurality of classes based on likelihood information of a plurality of classes at the time of learning,
Claims for causing a computer to realize a function of determining an identification result by weighting and combining the reliability calculated by the reliability calculation unit and the likelihood by a weighting coefficient corresponding to the strength of independence. Item 10. The identification program according to Item 9.

  The invention according to claim 13 is characterized in that the reliability calculation unit determines the class having the maximum likelihood from the difference between the maximum likelihood of any of the classes and the likelihood of the other class and the identification result based on the likelihood. The identification program according to claim 9, for causing a computer to realize a function of calculating the reliability of performing.

  The invention according to claim 14 is characterized in that the reliability calculation unit determines the class having the maximum likelihood from the classification result based on the likelihood based on the ratio of the maximum likelihood of any of the classes and the likelihood of the other classes. The identification program according to claim 9, for causing a computer to realize a function of calculating the reliability of performing.

  The invention according to claim 15 has a function in which the reliability calculation unit calculates the reliability from a difference between likelihood information of a plurality of classes at the time of recognition and likelihood information of a plurality of classes at the time of learning. The identification program according to claim 9, which is realized by a computer.

  In the invention according to claim 16, the reliability calculation unit estimates the class having the maximum likelihood from the ratio of the maximum likelihood to the sum of all likelihoods in the likelihood information of a plurality of classes at the time of recognition. The identification program according to claim 9 for causing a computer to realize a function of calculating the reliability with respect to an identification result by degree.

According to the present invention, since the reliability of the identification result based on the likelihood is calculated based on the likelihood information of a plurality of classes, a value indicating reliability different from the likelihood is calculated in the class classification of machine learning. Can do.
Since it is based not only on the maximum likelihood but also on the likelihood information of a plurality of classes including the second and subsequent likelihoods, a highly accurate reliability can be calculated.

It is a system conceptual diagram of the identification device concerning one embodiment of the present invention. FIG. 4A is a diagram showing class classification in an example with high independence between classes, and FIG. 4B is a diagram showing class classification in an example with low independence between classes. FIG. 6 is a likelihood information distribution diagram (a) in an example with high independence between classes, and a likelihood information distribution diagram (b) in an example with low independence between classes. FIG. 6 is a likelihood information distribution diagram (a) in an example with high independence between classes, and a likelihood information distribution diagram (b) in an example with low independence between classes. The distribution of likelihood information in an example with low reliability (a) and the distribution of likelihood information in an example with high reliability (b) show how the reliability is calculated from the likelihood difference. It is a distribution map of likelihood information in an example where the independence between classes is low, and shows “face detection considering angle” as an example. Comparison distribution diagram of likelihood information during learning and likelihood information during recognition in an example with low reliability (a), and comparison distribution of likelihood information during learning and likelihood information during recognition in an example with high reliability FIG. (B) shows how the reliability is calculated based on the likelihood distribution difference.

  An embodiment of the present invention will be described below with reference to the drawings. The following is one embodiment of the present invention and does not limit the present invention.

  FIG. 1 is a conceptual diagram of a system of the identification device according to the present embodiment. The identification apparatus is configured by installing an identification program for causing the computer to function as the following units. The input data is image data. Further, the calculation of the reliability and the like will be described by taking “face detection considering an angle” as an example. “Face detection in consideration of the angle” is a method of detecting the face of a person in the image in accordance with the direction of the face.

  The feature quantity generation unit 1 generates a feature quantity used by the identification unit 2 from the input data. Specifically, for example, in the case of an image, brightness, edge information, gradient information (HoG), and the like can be considered. Of course, the feature amount is not limited to that exemplified here.

The identification unit 2 identifies (classifies) which class the input data belongs to using the feature quantity obtained by the feature quantity generation unit 1. Specifically, after learning what features each class has by using the learning features of the identification target for identification methods such as “Random Forest” and “Support Vector Machine”, the learned classifier Identification is performed by giving a feature amount to. At the time of identification, an identification result indicating which class is identified and a likelihood of identification are obtained. For example, in the case of Random Forest, the number of trees indicating the class at the time of identification can be used.
In this case, for example, when completely different objects such as a cup and a dog shown in FIG. 2 (a) are distinguished from each other, there is no particular problem because there is a large difference in the feature amount, but the face shown in FIG. 2 (b) In the case of an object that changes continuously, such as the direction of, even if the class to be identified is divided discretely, it is continuous in the real space, so the classes are not completely independent. For this reason, there is a case where the data near the boundary shows a high likelihood in a plurality of classes, and as a result, the identification result cannot be trusted, and it is not desirable to use the likelihood as it is for the identification. The present invention addresses such a problem, and derives a new “reliability” index from the distribution information and the like based on the likelihood and uses it for identification.
The identification unit 2 outputs “the identification result by likelihood” and “the likelihood information of a plurality of classes”. The “classification result by likelihood” is, in principle, that the maximum likelihood class is the correct answer. The likelihood information of multiple classes includes the likelihood of the correct class and the likelihood of other classes.

The class independence determination unit 3 calculates the independence strength of the likelihood between the plurality of classes based on the likelihood information of the plurality of classes at the time of learning by the identification unit 2, and further determines the independence strength. It is determined whether the classes are independent by comparison with a predetermined value. The class independence determination unit 3 performs determination using data whose correct answer class is known. In the simplest case, it is conceivable to use the learning data used at the time of learning.
When classes are completely independent, if a certain class is mistakenly identified, the mistake is considered to be random, and the likelihood distribution of classes other than the correct answer becomes uniform as shown in FIG. 3 (a), for example. Conceivable. On the other hand, assuming that the object that changes continuously like the face direction (angle) is a class, assuming that the correct answer is “the face direction is 0 degrees”, the face direction is 10 degrees and 60 degrees. Then, 10 degrees is closer to 0 degrees, and it is considered that there is a high possibility of being mistaken. That is, it is considered that a bias occurs in the wrong way and the likelihood distribution of classes other than the correct answer also occurs as shown in FIG. 3B, for example.
In the case of the likelihood information shown in FIG. 3 (a), since the likelihoods of classes other than the correct answer are relatively equal, it is considered that the correct answer class and other classes are independent.
In the case of the likelihood information shown in FIG. 3B, since the likelihood of classes other than the correct answer is biased, the correct class and other classes are considered to be independent.

For this reason, the class independence determination unit 3 may consider a method of determining class independence with a bias in error when data with correct answers is input.
As the “strength of independence”, the simplest method is to use variance of likelihood of incorrect classes in likelihood information of a plurality of classes at the time of learning. “Intensity of independence” can be obtained by adding the variances of the likelihoods of incorrect answer classes obtained for each identification of one image with a correct answer. In this case, the closer the variance is to 0, the more independent the classes are (the more independent is considered).
As another method, a method using the likelihood difference between the correct answer class and the incorrect answer class can be considered. When the classes are independent, for example, the likelihood is considered to be concentrated on the correct class as shown in FIG. However, when the classes are not independent, for example, as shown in FIG. 4B, the likelihood is distributed to a plurality of classes, and the difference between the correct class and the incorrect class is considered to be small. In the case of the likelihood information shown in FIG. 4A, since the difference between the likelihood of the correct class and the likelihood of the other class is large, it is considered that the correct class and the other class are independent. In the case of the likelihood information shown in FIG. 4B, since the difference between the likelihood of the correct class and the likelihood of the other class is small, the correct class and the other class are considered to be independent.
Therefore, the class independence determination unit 3 calculates the “independence strength” from the difference between the likelihood of the correct class and the likelihood of other classes when data with a correct answer is input, and determines the class independence. Possible ways to do this.
The simplest method is to use the difference between the likelihood of the correct class and the maximum likelihood of the incorrect class in the likelihood information of a plurality of classes during learning. “Intensity of independence” can be obtained by adding the likelihood difference between the correct answer class and the incorrect answer class obtained for each identification of one correct answer image. In this case, the closer the total likelihood difference is to 0, the more considered to be non-independent between classes (it is considered to be less independent). In this embodiment, when it is determined that the independence of the class is lower than a predetermined value, the reliability is calculated by the reliability calculation unit 4 in the next item, and when it is determined that the independence of the class is higher than the predetermined value, The reliability is not calculated by the reliability calculation unit 4 in the next item. Thus, it is possible to stably calculate the reliability with high accuracy by automatically determining whether the class is suitable for the calculation of the reliability.

The reliability calculation unit 4 has one class among a plurality of classes in which the “classification result by likelihood” output from the identification unit 2 has a “independence strength” by the class independence determination unit 3 lower than a predetermined value. , The reliability of the “classification result based on likelihood” output from the identification unit 2 is calculated using the “likelihood information of a plurality of classes” output from the identification unit 2.
Specifically, as shown in FIG. 5, a method using the difference between the likelihood of the class with the highest likelihood and the second highest likelihood is conceivable. In this case, it is considered that the one with the highest likelihood protruding from the other is considered to have higher reliability. Therefore, the larger the difference, the higher the reliability (from FIG. 5A to FIG. 5B). Is more reliable).
In this case, the reliability calculation formula can be simply expressed as follows.

In addition to the second value, as shown in the following equation, all values after the second may be used for calculating the reliability.

A method using a ratio instead of a difference is also conceivable. The reliability calculation formula in this case can be expressed by the following formula, for example.

When the ratio is used, not only the second value but also all the second and subsequent values may be used for calculating the reliability as shown in the following equation.

Further, as shown in the following equation, a method of using a ratio to the sum of all likelihoods for calculating the reliability is also conceivable.

A method of using the shape of the distribution for calculation of reliability is also conceivable.
The identification unit 2 generally acquires identification performance by learning the characteristics of each class using learning data. If the classes to be identified are completely independent, the likelihood at the time of recognition is concentrated in one class, but in the case of `` face detection considering angle '' in this example, the classes are not completely independent, Likelihood occurs in addition to the correct class. At that time, the degree of occurrence of the likelihood varies depending on the class depending on the strength of the relationship between classes, and a class having a stronger relationship shows a higher likelihood. In the case of “face detection in consideration of angle”, for example, when a face with 0 degree orientation is learned in a problem of identifying a face with 0 degree to 180 degree orientation into classes every 30 degrees, it is finally obtained. In the discriminator, the face with the 0 degree orientation has the maximum likelihood, but since the angle change is continuous, the other classes also have a certain likelihood as shown in FIG. At that time, 30 degrees is closer to 0 degrees of the correct answer for the face with the orientation of 30 degrees and the face with the orientation of 60 degrees, so the order of likelihood is considered to be 0 degree> 30 degrees> 60 degrees. . Since this tendency is inherited at the time of recognition, it is considered that the reliability is high when the likelihood distribution is the same as that at the time of learning. For example, as shown in FIG. 7 (a), when there is a large difference between the likelihood at learning and the likelihood at recognition in each class, it can be evaluated that the reliability is low, as shown in FIG. 7 (b). In addition, when the difference between the likelihood during learning and the likelihood during recognition is small in each class, it can be evaluated that the reliability is high.
Based on the above idea, the formula for calculating the reliability from the difference between the likelihood information of multiple classes at the time of recognition and the likelihood information of multiple classes at the time of learning is simply as follows: Can express.

The above formulas are only a simple example and do not limit the calculation method of the reliability.
Since the likelihood distribution during learning varies depending on which class is correct, the above reliability calculation is performed for each class. That is, in the case of the 4-class discrimination problem, four reliability levels are calculated when each class is correct.
When obtaining the reliability from the distribution shape, the second and subsequent classes may have higher reliability than the class with the maximum likelihood. In that case, the form which outputs the class with the highest reliability as an identification result can be considered. That is, it is possible to employ a method of determining the identification result by combining the reliability calculated by the reliability calculation unit 4 and the likelihood calculated by the identification unit 2. As a combination method, a sum or product of likelihood and reliability may be used, and a weighting coefficient may be provided for each of likelihood and reliability. When providing the weighting coefficient, a method of variably setting the weighting coefficient according to the independence strength calculated by the class independence determination unit 3 can be implemented. The higher the degree of independence calculated by the class independence determination unit 3, the higher the likelihood weighting coefficient, and the lower the reliability weighting coefficient, and the independence calculated by the class independence determination unit 3 The lower the strength, the lower the likelihood weighting factor and the higher the reliability weighting factor. When the weighting coefficient is provided for the reliability as described above, the reliability may be calculated regardless of the degree of independence.
Note that these reliability calculation methods may be changed according to the strength of independence calculated by the class independence determination unit 3. For example, when the independence between classes is very low, it is difficult to trust the identification result based on the likelihood output from the identification unit 2 as it is, so it is desirable to use the reliability output from the reliability calculation unit 4 as it is. That is, the likelihood weighting coefficient is set to zero.
On the other hand, when there is some independence between classes, it is desirable to use the sum or product of the likelihood information output from the identification unit 2 and the reliability output from the reliability calculation unit 4. It is preferable to set the weighting coefficient according to the strength.
Regardless of the above-described embodiment, the class independence determination unit 3 may not be provided, and an identification apparatus including the feature amount generation unit 1, the identification unit 2, and the class independence determination unit 3 may be configured and implemented.

DESCRIPTION OF SYMBOLS 1 Feature-value production | generation part 2 Identification part 3 Class independence determination part 4 Reliability calculation part

Claims (16)

A feature value generation unit for generating a feature value of input data;
An identification unit that classifies input data into any of a plurality of classes from the feature amount, and generates likelihood information of the plurality of classes;
A reliability calculation unit that calculates the reliability of the identification result based on the likelihood based on the likelihood information;
An identification device comprising: The identification apparatus according to claim 1, wherein the identification result is determined by combining the reliability calculated by the reliability calculation unit and the likelihood. Based on the likelihood information of a plurality of classes at the time of learning, having a class independence determining unit that calculates the strength of independence of the likelihood between the plurality of classes,
The reliability calculation unit calculates the reliability when the identification result based on the likelihood indicates one class among a plurality of classes whose independence strength is lower than a predetermined value, and the likelihood 2. The identification device according to claim 1, wherein the reliability is not calculated when the identification result by indicates one class among a plurality of classes having an independence strength higher than a predetermined value. Based on the likelihood information of a plurality of classes at the time of learning, having a class independence determining unit that calculates the strength of independence of the likelihood between the plurality of classes,
The identification result is determined by weighting and combining the reliability calculated by the reliability calculation unit and the likelihood with a weighting coefficient corresponding to the strength of the independence. The identification device described. The reliability calculation unit calculates the reliability of the class having the maximum likelihood as the identification result based on the likelihood from the difference between the maximum likelihood of any of the classes and the likelihood of the other classes. The identification device according to claim 1, wherein: The reliability calculation unit calculates the reliability with respect to the class having the maximum likelihood as an identification result based on the likelihood from the ratio between the maximum likelihood of any of the classes and the likelihood of the other classes. The identification device according to claim 1, wherein: The said reliability calculation part calculates the said reliability from the difference of the likelihood information of the some class at the time of recognition, and the likelihood information of the some class at the time of learning, The claim 1 characterized by the above-mentioned. Identification device. The reliability calculation unit is configured to determine a class having the maximum likelihood as an identification result based on the likelihood from the ratio of the maximum likelihood to the sum of all likelihoods in the likelihood information of a plurality of classes at the time of recognition. The identification device according to claim 1, wherein the reliability is calculated. A feature value generation unit for generating a feature value of input data;
An identification unit that classifies input data into any of a plurality of classes from the feature amount, and generates likelihood information of the plurality of classes;
The identification program for functioning a computer as a reliability calculation part which calculates the reliability of the identification result by likelihood based on the said likelihood information. The identification program according to claim 9 for causing a computer to realize a function of determining an identification result by combining the reliability calculated by the reliability calculation unit and the likelihood. Based on likelihood information of a plurality of classes at the time of learning, the computer functions as a class independence determination unit that calculates the strength of independence of the likelihood between the plurality of classes,
The reliability calculation unit calculates the reliability when the identification result based on the likelihood indicates one class among a plurality of classes whose independence strength is lower than a predetermined value, and the likelihood 10. The function according to claim 9, for causing the computer to realize a function that does not calculate the reliability when the identification result by indicates a class of a plurality of classes whose independence strength is higher than a predetermined value. 10. Identification program. Based on likelihood information of a plurality of classes at the time of learning, the computer functions as a class independence determination unit that calculates the strength of independence of the likelihood between the plurality of classes,
Claims for causing a computer to realize a function of determining an identification result by weighting and combining the reliability calculated by the reliability calculation unit and the likelihood by a weighting coefficient corresponding to the strength of independence. Item 10. The identification program according to Item 9. The reliability calculation unit calculates the reliability of the class having the maximum likelihood as the identification result based on the likelihood from the difference between the maximum likelihood of any of the classes and the likelihood of the other classes. The identification program of Claim 9 for making a computer implement | achieve the function to perform. The reliability calculation unit calculates the reliability of the class having the maximum likelihood as the identification result based on the likelihood from the ratio between the maximum likelihood of any of the classes and the likelihood of the other classes. The identification program of Claim 9 for making a computer implement | achieve the function to perform. The said reliability calculation part makes a computer implement | achieve the function which calculates the said reliability from the difference of the likelihood information of the some class at the time of recognition, and the likelihood information of the some class at the time of learning. 9. The identification program according to 9. The reliability calculation unit is configured to determine the class having the maximum likelihood as the identification result by the likelihood from the ratio of the maximum likelihood to the sum of all likelihoods in the likelihood information of the plurality of classes at the time of recognition. The identification program according to claim 9 for causing a computer to realize the function of calculating the reliability.

Images