JP7268347B2 - IDENTIFICATION DEVICE, IDENTIFICATION METHOD AND PROGRAM - Google Patents

Description

The present invention relates to an identification device, identification method and program.

Conventionally, a classifier (also called a classifier) has been known that uses a neural network, an SVM (Support Vector Machine), or the like to determine which class among a plurality of classes input data belongs to. and is used in various fields. For example, there is a classifier that outputs the type of flower when a flower image is input with the type of flower as a class. There is also a classifier that outputs the name of a skin disease when a skin image is input with the name of a skin disease as a class.

Such classifiers are created by learning using a large amount of training data whose classes are known. However, if the composition ratio of the classes included in the training data is extremely skewed (large imbalance in the number of samples for each class), it will be biased to classes with a high ratio (classes with a large number of samples). , there is a problem that the accuracy of identification falls. In order to solve this problem, in Patent Document 1, even if the number of learning data belonging to one class is extremely smaller than the number of the other class, part of the learning data of the minority class is duplicated to increase it ( (oversampling), or by removing some of the learning data of many classes and reducing (undersampling) the learning data to adjust the balance of the learning data, thereby performing highly accurate classification.

JP 2013-161298 A

Conventionally, as a scale for adjusting the balance of learning data, a confusion matrix summarizing the classification results of each class is used. The balance between the numbers has been adjusted. However, the proper balance is greatly affected not only by the number of data belonging to each class, but also by the distribution of data near the boundaries between classes. Not easy to adjust.

SUMMARY OF THE INVENTION The present invention has been made to solve the above problems, and an object of the present invention is to provide an identification device, an identification method, and a program capable of improving the accuracy of identification.

In order to achieve the above object, the identification device of the present invention
a discriminator that outputs a predicted value for each class that discriminates each of a plurality of classes to which unknown data to be input belongs;
a correction unit that corrects the predicted value of each class output by the discriminator using a correction parameter of each class;
A creation unit that creates a confusion matrix based on the class identified by the predicted value corrected by the correction unit;
a modifying unit that modifies the correction parameter so that the deviation of the total number of samples of each class for each correct label of the classifier in the confusion matrix generated by the generating unit is smaller among the plurality of classes; ,
a result acquisition unit that acquires a final identification result by correcting the predicted value output by the classifier using the correction parameter corrected by the correction unit;
Prepare.

According to the present invention, it is possible to improve the accuracy of identification.

It is a figure which shows the functional structure of the identification device which concerns on Embodiment 1 of this invention. 5 is a flowchart of correction parameter acquisition processing according to the first embodiment; 4 is a flowchart of discriminator learning processing according to the first embodiment. 5 is a flowchart of identification processing according to the first embodiment; It is a figure explaining a confusion matrix. It is a figure explaining the process which acquires a correction parameter. FIG. 10 is a diagram illustrating a confusion matrix with values obtained by correcting predicted values with correction parameters; It is a figure explaining the process which corrects a correction parameter. FIG. 10 is a diagram illustrating a confusion matrix with values obtained by correcting predicted values with other correction parameters; It is a figure explaining the process which corrects a correction parameter further.

Hereinafter, identification devices and the like according to embodiments of the present invention will be described with reference to the drawings. The same reference numerals are given to the same or corresponding parts in the drawings.

(Embodiment 1)
The identification device 100 according to the first embodiment of the present invention reduces the influence of the bias in the number of samples by correcting the output of the classifier even if the number of sample data is biased between classes, and improves the accuracy. A high degree of discrimination can be achieved. Such an identification device 100 will be described below.

The identification device 100 according to Embodiment 1 includes a control unit 10, a storage unit 20, a data input unit 31, an output unit 32, a communication unit 33, and an operation input unit 34, as shown in FIG.

The control unit 10 is composed of a CPU (Central Processing Unit) or the like, and by executing a program stored in the storage unit 20, functions of each unit (discriminator 11, correction unit 12, result acquisition unit 13) to be described later. Realize.

The storage unit 20 includes a ROM (Read Only Memory), a RAM (Random Access Memory), etc., and stores programs executed by the CPU of the control unit 10 and necessary data.

The data input unit 31 is a device for inputting learning data, test data, or identifying (unknown) data. The control unit 10 acquires data via the data input unit 31 . Any device can be used as the data input unit 31 as long as the control unit 10 can acquire data. For example, when data is stored in the storage unit 20 and the control unit 10 acquires the data by reading the storage unit 20 , the storage unit 20 also serves as the data input unit 31 . Also, when the control unit 10 acquires data from an external server or the like via the communication unit 33 , the communication unit 33 also serves as the data input unit 31 .

The output unit 32 is a device for the control unit 10 to output the results of identification of data input from the data input unit 31 and the like. For example, the output unit 32 is a liquid crystal display or an organic EL (Electro-Luminescence) display. In this case, the output section 32 functions as a display section. However, the identification device 100 may include such a display (display unit) as the output unit 32, or may include the output unit 32 as an interface for connecting an external display. When the identification device 100 has the output unit 32 as an interface, the identification result and the like are displayed on an external display connected via the output unit 32 . The output unit 32 functions as output means.

The communication unit 33 is a device (network interface, etc.) for transmitting and receiving data to and from other external devices (for example, a server storing databases of learning data and test data). The control unit 10 can acquire data via the communication unit 33 .

The operation input unit 34 is a device that receives a user's operation input to the identification device 100, and is, for example, a keyboard, mouse, touch panel, or the like. The identification device 100 receives an instruction or the like from the user via the operation input unit 34 . The operation input unit 34 functions as operation input means.

Next, functions of the control unit 10 will be described. The control unit 10 realizes the functions of the discriminator 11 , the correction unit 12 and the result acquisition unit 13 .

The discriminator 11 is composed of logistic regression, DNN (Deep Neural Network), or the like, and outputs predicted values for discriminating the class to which given input data belongs. identify who belongs to Here, the number of classes identified by the discriminator 11 is N (the discriminator 11 is an N class discriminator). The discriminator 11 outputs a predicted value of each class as a number of 0 or more and 1 or less when input data is given. Since N (the number of classes to be identified) predicted values are output, the predicted values of each class can be collected and handled as an N-dimensional vector. Then, the class corresponding to the maximum predicted value among the N predicted values (each element of the vector) becomes the identification result (predicted class) of the input data. Logistic regression and DNN discriminators can be realized by the control unit 10 executing a program for realizing them. In this case, the control unit 10 also functions as the discriminator 11 . Since the discriminator 11 outputs the predicted value of each class as a number between 0 and 1, the SVM cannot be directly used as the discriminator 11. If converted into numbers, it is possible to configure the discriminator 11 using SVM.

The correction unit 12 acquires a correction parameter for correcting the predicted value of each class output by the discriminator 11 so as to enhance the symmetry of the confusion matrix. The confusion matrix is a table summarizing the results of class discrimination when the classifier 11 receives input data whose correct answer is known (referred to as test data here). When the class (predicted class) identified by the classifier 11 is taken in the horizontal direction and the correct class is taken in the vertical direction, the number of samples corresponding to each position becomes an element of the confusion matrix. A confusion matrix with high symmetry means that the total value of each column is less biased when the number of samples in each class of input data is equal.

More specifically, the correction unit 12 acquires (N) correction parameters corresponding to each class identified by the discriminator 11, and converts the predicted value of each class output by the discriminator 11 to the correction corresponding to the class. A corrected predicted value is obtained by correcting using the parameters. Then, a confusion matrix is created based on the classes identified by the corrected predicted values. Then, when the confusion matrix is normalized in the row direction (so that the number of correct data in each class is 1), the column direction sum (total prediction rate of each class) is as close to 1 as possible. modify the correction parameters (corresponding to each class) of Then, the correction parameter when the total prediction rate of each class is closest to 1 is obtained.

The result acquisition unit 13 corrects the predicted value of each class output by the classifier 11 using the correction parameter of each class acquired by the correction unit 12, and acquires the final classification result. The result acquisition unit 13 functions as result acquisition means.

The functional configuration of the identification device 100 has been described above. Next, a process (correction parameter acquisition process) in which the correction unit 12 of the identification device 100 acquires correction parameters will be described. This process is executed when the correction parameters are acquired. In addition, before executing this process, it is necessary to prepare a predetermined amount (in this case, M) of test input data (test data) with correct labels in advance in the storage unit 20 or the like. be.

A correct label indicates to which class the data with the correct label belongs. For example, if it is desired to prepare an identification device 100 that outputs a disease name when a disease image is input as input data, it is necessary to attach "disease name" as a correct label to each test data. As another example, if it is desired to prepare the identification device 100 that outputs the name of a flower when an image of a flower is input, it is necessary to attach the "name of the flower" as a correct label to each piece of test data. Now, a process (correction parameter acquisition process) in which the correction unit 12 of the identification device 100 acquires correction parameters will be described with reference to FIG.

First, the control unit 10 causes the discriminator 11 to undergo machine learning using a large amount of input data for learning (learning data) to which correct labels are attached in advance by discriminator learning processing described later (step S101). If the discriminator 11 has already been trained, step S101 may be skipped.

Next, the control unit 10 inputs the test data to the discriminator 11 and acquires the predicted value output from the discriminator 11 (step S102). If M pieces of test data are prepared, M prediction values (each being an N (number of classes) dimensional vector) are obtained.

Then, the correction unit 12 initializes the correction parameter of each class to 1, and initializes the counter variable CT to 0 (step S103).

Next, the correction unit 12 corrects each predicted value obtained in step S102 with the correction parameter of each class to obtain a corrected predicted value (step S104). Specifically, the predicted value of each class is converted to a value raised to the power of the correction parameter of the class, and the converted value is used as the corrected predicted value. Then, a confusion matrix is created from the corrected prediction values (step S105), and each row of the confusion matrix is normalized so that the sum becomes 1 (step S106).

Next, the correction unit 12 normalizes and acquires the total value of each column of the confusion matrix as the total prediction rate of each class (step S107). Then, the class with the maximum absolute difference between the total class prediction rate and 1 is searched for, and the searched class is set as a variable MC, and the maximum absolute difference value (the total class prediction rate of the class and 1 difference absolute value) is set in the variable MD, and the counter variable CT is incremented (step S108).

Then, the correction unit 12 determines whether or not the variable MD (the maximum value of the absolute difference values) is equal to or less than the reference value SD (for example, 0.02) (step S109). If the variable MD is equal to or less than the reference value SD (step S109; Yes), it is considered that the symmetry of the confusion matrix is sufficiently improved with the current correction parameters, so the correction unit 12 uses the current correction parameters in the identification process described later. It is determined as a correction parameter (step S114), and the correction parameter acquisition process ends. Step S114 is also called a correction parameter acquisition step.

If the variable MD is greater than the reference value SD (step S109; No), the correction unit 12 determines whether or not the counter variable CT is equal to or greater than the maximum search count LIMIT (eg, 100) (step S110). If the counter variable CT is equal to or greater than the maximum number of searches LIMIT (step S110; Yes), the correction unit 12 determines the current correction parameters as correction parameters used in the later-described identification process (step S114), and ends the correction parameter acquisition process. In this correction parameter acquisition process, there are cases where the value of the variable MD does not become equal to or less than the reference value SD and the variable MD vibrates. If not, the determination in step S110 is made to end the correction parameter acquisition process.

If the counter variable CT is less than the maximum search count LIMIT (step S110; No), the correction unit 12 determines whether or not the total class prediction rate of the class of the variable MC is greater than 1 (step S111). If the total class prediction rate of the class of variable MC is greater than 1 (step S111; Yes), the correction parameter of the class of variable MC is increased (step S112), and the process returns to step S104. The amount by which the correction parameter is increased may be a constant such as 0.05, or may be changed according to the value of the variable MD (maximum value of absolute difference) (if the value of MD is large, the correction parameter is The amount of increase is also increased, and if the value of MD is small, the amount of increase of the correction parameter is also decreased). However, if this value is large, the value of the variable MD tends to oscillate or diverge, so basically it is better to set the amount of increase as small as possible if the computation time is acceptable.

If the total class prediction rate of the class of the variable MC is 1 or less (step S111; No), the correction parameter of the class of the variable MC is decreased (step S113), and the process returns to step S104. As in step S112, the amount by which the correction parameter is reduced may be a constant value such as 0.05, or may be changed according to the value of the variable MD (maximum absolute value of difference) (when the value of MD is If the value of MD is large, the amount by which the correction parameter is decreased is also increased, and if the value of MD is small, the amount by which the correction parameter is decreased is also decreased). However, if this value is large, the value of the variable MD tends to oscillate or diverge, so basically it is better to set the amount to be reduced as small as possible if the calculation time is acceptable.

The above correction parameter acquisition processing has been described. Next, the discriminator learning process performed in step S101 will be described with reference to FIG.

First, the control unit 10 acquires learning data with a correct label through the data input unit 31 (step S201). Then, the control unit 10 inputs the learning data to the discriminator 11, and causes the discriminator 11 to learn based on the correct label attached to the learning data (step S202).

Then, the control unit 10 determines whether or not to end learning (step S203). For example, the learning is terminated when a predetermined number (for example, M) of learning data are learned. If learning is not finished (step S203; No), such as when learning data that has not been learned remains (step S203; No), the process returns to step S201 to acquire the next learning data and make the discriminator 11 learn. If learning is to be ended (step S203; Yes), the classifier learning process is ended.

The correction unit 12 acquires the correction parameters at the time when the correction parameter acquisition process (FIG. 2) described above is completed, and the identification device 100 corrects the output (predicted value) of the discriminator 11 with the correction parameters. to identify the input data. Now, the identification processing of the identification device 100 will be described with reference to FIG.

First, the control unit 10 acquires unknown data (identifying data) through the data input unit 31 (step S301). Then, the control unit 10 inputs the unknown data to the discriminator 11 and acquires the predicted value of each class output by the discriminator 11 (step S302).

Next, the result obtaining unit 13 corrects the predicted value of each class with the correction parameter obtained by the correction parameter obtaining process (FIG. 2) by the correcting unit 12 (step S303). Specifically, the predicted value of each class is converted to a value raised to the power of the correction parameter of the class, and the converted value is used as the corrected predicted value. Then, the result acquisition unit 13 acquires the class with the largest corrected predicted value as the identification result, and outputs it via the output unit 32 (step S304). Then, the identification processing ends. Step S304 is also called a result acquisition step.

Unknown data is identified by the above identification processing. Since this discrimination is performed based on the corrected predicted value obtained by correcting the predicted value output from the classifier 11, even if the number of learning data for each class is unbalanced, the influence thereof is eliminated as much as possible. Therefore, highly accurate identification can be performed.

Next, the correction parameter acquisition processing (FIG. 2) by the identification device 100 will be described with a specific example. Here, the discriminator 11 is a three-class discriminator that discriminates three classes of "A", "B", and "C", and is presumed to have been learned in step S101 of FIG. It is also assumed that 24 pieces of test data labeled with correct answers as shown on the left side of FIG. 5 are prepared. An output value (predicted value) obtained by inputting 24 pieces of test data to the discriminator 11 in step S102 of FIG. 2 is written to the right of the correct label. At step S103 in FIG. 2, the correction parameters of each class are initialized to 1.0 for all three classes, which is represented by a three-dimensional vector (1.0, 1.0, 1.0). . Also, the counter variable CT=0.

For example, in the case of the test data at the top, the correct label is "C" and the predicted values obtained by the discriminator 11 are (0.7, 0.2, 0.68). In this case, the maximum value of the elements of the predicted value vector is 0.7, and 0.7 is the first element (corresponding to A), so the identification result of the identifier 11 is "A". Also, as described above, the initial values of the correction parameters are (1.0, 1.0, 1.0), and in step S104 of FIG. ^ 1.0, 0.2 ^ 1.0, 0.68 ^ 1.0), and the same (0.7, 0.2, 0.68) as the original prediction values are obtained as corrected prediction values, The identification result based on this corrected predicted value is also "A", which is the same as the identification result based on the predicted value.

The confusion matrix shown on the right side of FIG. 5 is a table summarizing the relationship between the correct label and the identification result based on the corrected prediction value for all 24 pieces of test data in step S105 of FIG. Looking at this confusion matrix, among the test data with the correct answer label of "A", 10 have the classification result of "A", 2 have the classification result of "B", and the classification result of It can be seen that there are 0 "C". Similarly, among the test data with the correct answer label of "B", three have the identification result of "A", five have the identification result of "B", and have the identification result of "C". It turns out that there is one thing that has become. Furthermore, among the test data with the correct answer label of "C", there are 2 items with the identification result of "A", 0 items with the identification result of "B", and the identification result of "C". It turns out that there is one thing.

Next, the process of normalizing this confusion matrix and acquiring correction parameters will be described with reference to FIG. First, in step S106 of FIG. 2, in order to normalize the confusion matrix, first, the total number of pieces of test data with each correct label is obtained as shown in the top of FIG. Then, for each row of the mixing matrix, normalize the sum to 1 by dividing the element by the value of the sum, as in the second table from the top of FIG.

Then, in step S107 of FIG. 2, when the total value of the elements in the vertical direction for each column of this normalized confusion matrix (total prediction rate of each class) is calculated, as shown in FIG. is 1.833, the total prediction rate for class "B" is 0.722, and the total prediction rate for class "C" is 0.444. In this example, the proportion of the identification result being "A" is quite high, and the symmetry of the confusion matrix is low. Also, the average accuracy rate is as low as 0.574. Here, the average correct answer rate is a value obtained by averaging the proportion of test data whose correct answer is class "X" and whose identification result (predicted class) is "X" for all classes. In FIG. 6, the values existing on the diagonal line of the second table from the top are averaged to obtain (0.833+0.556+0.333)/3=0.574. The numerical value of this average accuracy rate is described.

In order to increase the symmetry of the confusion matrix, the predicted value of a class with a large vertical total value in each column should be corrected to a small value, or the predicted value of a class with a small total value should be corrected to a large value. It is considered to be good. As described above, the predicted value output by the discriminator 11 is a value between 0 and 1, and when this value is raised to a power greater than 1 (within the range between 0 and 1), it becomes a smaller value than the original value. , raised to a power less than 1 (between 0 and 1) yields a greater value than the original. Therefore, using this property, in order to increase the symmetry of the confusion matrix, the predicted value of the class with a large vertical total value in each column should be raised to a power greater than 1, or the total value is small It suffices to raise the predicted value of the class to a power smaller than one.

In the example of FIG. 6, class "A" has the highest total prediction rate at 1.833 and class "C" has the lowest total prediction rate at 0.444. Therefore, it is conceivable to raise the predicted value of class "A" to a power greater than 1, and to raise the predicted value of class "C" to a power of less than 1. However, if multiple parameters are changed at the same time, there is a possibility that the prediction value of each class will change abruptly and the classification result will change unexpectedly. Therefore, in the first embodiment, only one parameter is changed at a time. In this example, in step S108 of FIG. 2, the class "A" is searched as the class having the maximum absolute difference value from the reference value 1, MC="A", and the maximum absolute difference value MD= |1.833−1|=0.833. Then, the counter variable CT is incremented to one.

Next, in step S109 of FIG. 2, it is determined whether or not the variable MD (here, 0.833) is equal to or less than the reference value SD (eg, 0.02), the determination is No, and the process proceeds to step S110. In step S110, it is determined whether or not the counter variable CT (here, 1) is equal to or greater than the maximum search count LIMIT (eg, 100), the determination is No, and the process proceeds to step S111. In step S111, it is determined whether or not 1.833, which is the total class prediction rate of class "A", is greater than 1, the determination is Yes, and in step S113 the correction parameter of class "A" is increased. Here, the amount to be increased is assumed to be 0.05. Then, the correction parameter of class "A" becomes 1.05, and the correction parameters of class "B" and class "C" remain 1. Therefore, if the correction parameters are represented by a vector, (1.05, 1.0, 1 .0).

Although not shown, even if the process returns from step S113 to step S104 and corrects the predicted values of the discriminator shown in FIG. Therefore, the same confusion matrix is obtained again, and the same process proceeds to step S113, where the correction parameter for class A becomes 1.1. Then, returning to step S104, the predicted values of the discriminator shown in FIG. part of the identification results change. Then, when the confusion matrix is created in step S105, the confusion matrix shown on the right side of FIG. 7 is created.

Then, when normalized in step S106, a normalized confusion matrix as shown in the second table from the top in FIG. 8 is obtained. Then, when the total prediction rate of each class is calculated in step S107, the total prediction rate of class "A" is 1.167, the total prediction rate of class "B" is 1.056, and the total prediction rate of class "C" is It is 0.778, and it can be seen that the symmetry is higher than the previous confusion matrix. The average accuracy rate is 0.796, which is higher than the previous time.

In this case, in step S108, the class "C" is searched as the class having the maximum absolute difference from the reference value 1, MC="C", and the maximum absolute difference MD= |0.778−1|=0.222. Then, in step S111, it is determined whether or not 0.778, which is the total class prediction rate of class "C", is greater than 1, the determination is No, and in step S112, the correction parameter of class "C" is decreased. Assume here that the amount to be reduced is 0.05. Then, the correction parameter for class "C" is 0.95, and the correction parameter is expressed as a vector (1.1, 1.0, 0.95).

Then, returning to step S104, the predicted values of the discriminator shown in FIG. part of the identification results change. Then, when the confusion matrix is created in step S105, the confusion matrix shown on the right side of FIG. 9 is created.

Then, when normalized in step S106, a normalized confusion matrix as shown in the second table from the top in FIG. 10 is obtained. Then, when the total prediction rate of each class is calculated in step S107, the total prediction rate of class "A" is 0.833, the total prediction rate of class "B" is 1.056, and the total prediction rate of class "C" is 1.111, and it can be seen that the symmetry is higher than the previous confusion matrix. The average accuracy rate is 0.907, which is higher than the previous time.

In this case, in step S108, the class "A" is searched as the class having the maximum absolute difference value from the reference value 1, MC="A", and the maximum absolute difference value MD= |0.833−1|=0.166. Then, in step S111, it is determined whether or not 0.833, which is the total class prediction rate of class "A", is greater than 1, the determination is No, and in step S112, the correction parameter of class "A" is decreased. Assume here that the amount to be reduced is 0.05. Then, the correction parameter for class "A" is 1.05, and the correction parameter is expressed as a vector (1.05, 1.0, 0.95). In this way, the process is repeated until the maximum value MD of the absolute difference between the total class prediction rate and 1 becomes equal to or less than the reference value SD, or until the counter variable CT becomes equal to or greater than the maximum number of searches LIMIT. parameters will be obtained.

With the above-described identification processing, the identification device 100 corrects the prediction values of the classifier 11 so as to increase the symmetry of the confusion matrix, thereby reducing the influence of bias even if sample data is biased between classes. Therefore, highly accurate identification can be performed.

(Modification 1)
In the first embodiment, when correcting the predicted value of the discriminator 11, the correcting unit 12 performs the calculation of exponentiating the predicted value by the correction parameter. function determined by correction parameters) is not limited to this. There is a premise that the predicted value is a value between 0 and 1, and the value after correcting the predicted value with the correction parameter (corrected predicted value) must also be a value between 0 and 1, but the correction parameter is fixed. The magnitude relationship of the original predicted value is maintained even after correction (even the corrected predicted value), and the direction and degree of change when the predicted value is corrected according to the magnitude of the correction parameter value is determined, the correction unit 12 may correct the predicted value of the discriminator 11 using another function.

That is, the correction unit 12 is a monotonically non-decreasing function (monotonically increasing function in a broad sense) that converts a value between 0 and 1 as an input value (predicted value) into an output value (corrected predicted value) between 0 and 1. , the monotonically non-decreasing function (broadly defined monotonically increasing function) is a function determined by the correction parameter, and when the input value (predicted value) is fixed, the change in the output value (corrected predicted value) due to the correction parameter is also monotonous Any function that is non-decreasing or monotonically non-increasing can be used to correct the predicted value. For example, a function that exponentiates an input value by a correction parameter is a function that satisfies these conditions.

(Modification 2)
In the first embodiment, in the correction parameter acquisition process (FIG. 2), if the maximum value MD of the absolute difference value does not become equal to or less than the reference value SD, the correction parameter when the counter variable CT becomes equal to or greater than the maximum number of searches LIMIT is set to It is used as the final correction parameter. However, it is not limited to this. For example, between steps S107 and S108, the correction parameter for when the maximum value MD of the difference absolute value is the smallest is stored, and in step S109, the counter variable CT is set to the maximum number of searches LIMIT or more. If so, the stored correction parameters may be determined as the final correction parameters in step S114. By doing so, even if the value of the maximum value MD of the difference absolute value oscillates or diverges, the correction unit 12 uses the correction parameter when the symmetry of the confusion matrix becomes highest can be obtained.

(Modification 3)
In the first embodiment, in the correction parameter acquisition process (FIG. 2), processing is performed to increase only the symmetry of the confusion matrix regardless of the level of the correct identification rate of each class. may be considered. For example, when the confusion matrix is normalized in step S105, the average correct answer rate (as described above, the percentage of test data whose correct answer is class "X", the identification result (predicted class) is "X" is the average value for all classes, and the lower right numerical value of the second confusion matrix table in FIGS. Save parameters. Then, when the counter variable CT reaches or exceeds the maximum search count LIMIT in step S109, the stored correction parameter may be determined as the final correction parameter in step S114. By doing so, even if the value of the maximum value MD of the difference absolute value oscillates or diverges, the correction unit 12 acquires the correction parameter when the average accuracy rate is the highest. can do.

(Modification 4)
Depending on the input data (learning data and test data) given to the discriminator 11, in the correction parameter acquisition process (FIG. 2), there is a trade-off relationship between the high symmetry of the confusion matrix and the high average accuracy rate. There is Therefore, in the correction parameter acquisition process (FIG. 2), instead of deciding on one correction parameter, several correction parameter candidates are presented to the user in step S114, and the user selects the correction parameter. You can have it.

In order to perform such processing, the control unit 10 has a correction parameter when the confusion matrix has high symmetry (the maximum value MD of the difference absolute value is equal to or less than the difference reference value (for example, 0.1)), , the correction parameters when the average accuracy rate is high (the average accuracy rate is equal to or higher than the accuracy rate reference value (for example, 90%)) are stored in the storage unit 20 as correction parameter candidates. The confusion matrix and average accuracy rate created using the corrected parameters are presented to the user via the output unit 32 . Then, the control unit 10 causes the user to select a correction parameter via the operation input unit 34, and causes the correction unit 12 to acquire the correction parameter selected by the user.

Even if there is a trade-off relationship between the high symmetry of the confusion matrix and the high average accuracy rate, the identification device 100 according to Modification 4 allows the user to select the correction parameter to be adopted from among the correction parameter candidates. is selected, identification can be performed using appropriate correction parameters.

In the above-described embodiment and modified example, the learning data and the test data are described as different data. ) and correction parameter acquisition processing (FIG. 2) may be performed.

In the above-described embodiment and modification, the control unit 10 executes the program that implements the discriminator 11 so that the control unit 10 also functions as the discriminator 11. However, the present invention is not limited to this. The identification device 100 may include a device (for example, a GPU (Graphics Processing Unit), a dedicated IC (Integrated Circuit), etc.) that realizes the function of the identification device 11 , separately from the control unit 10 .

Also, the above-described embodiments and modifications can be combined as appropriate. For example, in the case of combining Modification 2 and Modification 3 to make the symmetry of the confusion matrix more important than the average correct answer rate, when the counter variable CT becomes equal to or greater than the maximum number of searches LIMIT in step S109, , in step S114, the correction parameter at which the value of the maximum value MD of the absolute difference is the smallest is established as the correction parameter to be used in the identification process. When the maximum value MD of the difference absolute value is the minimum in the correction parameters), the average accuracy rate for each correction parameter is compared, and the correction parameter with the highest average accuracy rate is used in the identification process. It is sufficient to perform processing to determine as .

Each function of the identification device 100 can also be performed by a computer such as a normal PC (Personal Computer). Specifically, in the above embodiment, the identification processing program performed by the identification device 100 is pre-stored in the ROM of the storage unit 20 . However, the program may be stored in a computer-readable storage medium such as a flexible disk, CD-ROM (Compact Disc Read Only Memory), DVD (Digital Versatile Disc), MO (Magneto-Optical Disc), memory card, USB (Universal Serial Bus) memory, etc. By storing and distributing the program in a recording medium, and reading and installing the program in the computer, a computer capable of realizing each of the functions described above may be configured.

Although the preferred embodiments of the present invention have been described above, the present invention is not limited to such specific embodiments, and the present invention includes the invention described in the claims and their equivalents. be The invention described in the original claims of the present application is appended below.

(Appendix 1)
a classifier that identifies the class to which the input data belongs and outputs a predicted value for each class;
a correction unit that acquires the correction parameter that enhances the symmetry of the confusion matrix created based on the class identified by the corrected predicted value obtained by correcting the predicted value of each class output from the classifier using the correction parameter for each class; ,
a result acquisition unit that acquires a final identification result by correcting the predicted value output by the discriminator using the correction parameter acquired by the correction unit;
identification device.

(Appendix 2)
The correcting unit reduces the total prediction rate of each class of the confusion matrix after normalization to 1 as much as possible when the confusion matrix is normalized so that the number of correct data in each class of the confusion matrix is 1. obtaining the correction parameters that approach;
The identification device according to appendix 1.

(Appendix 3)
The correction unit is
correcting the predicted value output from the discriminator using the monotonically non-decreasing function determined by the correction parameter, which converts an input value of 0 or more and 1 or less to an output value of 0 or more and 1 or less; to get the corrected predictions, and
creating the confusion matrix based on the classes identified by the corrected predicted values;
When the confusion matrix is normalized so that the number of correct data in each class of the created confusion matrix is 1, the total prediction rate of each class of the confusion matrix after normalization approaches 1 as much as possible get the correction parameters,
The identification device according to appendix 2.

(Appendix 4)
The monotone non-decreasing function is a function that performs an operation of exponentiating the predicted value output by the discriminator by the correction parameter,
The identification device according to appendix 3.

(Appendix 5)
The correction unit acquires an average accuracy rate by averaging the accuracy rate of each class, and acquires the correction parameter that maximizes the average accuracy rate among the correction parameters that increase the symmetry of the confusion matrix. ,
5. Identification device according to any one of appendices 1 to 4.

(Appendix 6)
A correction parameter acquisition step of acquiring the correction parameter that enhances the symmetry of the confusion matrix created based on the class identified by the corrected predicted value obtained by correcting the predicted value of each class output by the discriminator using the correction parameter for each class and,
a result obtaining step of obtaining a final identification result by correcting the predicted value output by the discriminator using the correction parameter obtained in the correction parameter obtaining step;
identification methods, including

(Appendix 7)
In the computer of the identification device equipped with the identifier,
Acquisition of correction parameter for enhancing symmetry of a confusion matrix created based on the class identified by the corrected predicted value obtained by correcting the predicted value of each class output from the classifier using a correction parameter for each class step, and
A result obtaining step of obtaining a final identification result by correcting the predicted value output by the classifier using the correction parameter obtained in the correction parameter obtaining step;
program to run the

DESCRIPTION OF SYMBOLS 10... Control part 11... Discriminator 12... Correction part 13... Result acquisition part 20... Storage part 31... Data input part 32... Output part 33... Communication part 34... Operation input part 100... identification device

Claims (7)

a discriminator that outputs a predicted value for each class that discriminates each of a plurality of classes to which unknown data to be input belongs;
a correction unit that corrects the predicted value of each class output by the discriminator using a correction parameter of each class;
A creation unit that creates a confusion matrix based on the class identified by the predicted value corrected by the correction unit;
a modifying unit that modifies the correction parameter so that the deviation of the total number of samples of each class for each correct label of the classifier in the confusion matrix generated by the generating unit is smaller among the plurality of classes; ,
a result acquisition unit that acquires a final identification result by correcting the predicted value output by the classifier using the correction parameter corrected by the correction unit;
An identification device comprising: The correction unit reduces the total prediction rate of each class of the confusion matrix after normalization to 1 as much as possible when the confusion matrix is normalized so that the number of correct data in each class of the confusion matrix is 1. characterized by modifying the correction parameters to approximate
2. Identification device according to claim 1. The correction unit is
correcting the predicted value output from the discriminator using the monotonically non-decreasing function determined by the correction parameter, which converts an input value of 0 or more and 1 or less to an output value of 0 or more and 1 or less; ,
The correction unit
When the confusion matrix created by the creation unit is normalized so that the number of correct data in each class of the confusion matrix is 1, the total prediction rate of each class of the confusion matrix after normalization is as much as possible characterized by modifying the correction parameter to approach 1,
3. Identification device according to claim 2. The monotone non-decreasing function is a function that performs an operation to raise the predicted value output by the classifier to the power of the correction parameter,
4. Identification device according to claim 3. The correcting unit obtains an average accuracy rate by averaging the accuracy rate of each class , and adjusts the bias of the total number of samples of each class for each correct label of the classifier in the confusion matrix between the plurality of classes. Obtaining the correction parameter that maximizes the average accuracy rate among the correction parameters to be made smaller by
Identification device according to any one of claims 1 to 4. a correction step of correcting the predicted value of each class output by a discriminator that outputs a predicted value of each class for identifying each of a plurality of classes to which unknown data to be input belongs, using a correction parameter of each class;
A creation step of creating a confusion matrix based on the classes identified by the predicted values corrected in the correction step;
A correction step of correcting the correction parameter so that the deviation of the total number of samples of each class for each correct label of the classifier in the confusion matrix created in the creation step is smaller among the plurality of classes. and,
a result obtaining step of obtaining a final identification result by correcting the predicted value output by the discriminator using the correction parameter corrected in the correcting step;
A method of identification comprising: A computer of an identification device equipped with a classifier that outputs a predicted value for each class that identifies each of a plurality of classes to which unknown data that is input belongs,
correction means for correcting the predicted value of each class output by the discriminator using a correction parameter of each class;
creating means for creating a confusion matrix based on the classes identified by the predicted values corrected by the correcting means;
Correction means for correcting the correction parameter so that the deviation of the total number of samples of each class for each correct label of the classifier in the confusion matrix created by the creation means is smaller among the plurality of classes;
result acquisition means for acquiring a final identification result by correcting the predicted value output by the classifier using the correction parameter corrected by the correction means;
A program to function as

Images