JP2022154442A - Learning apparatus, learning method, and program - Google Patents

Abstract

To provide a technology for generating a trained model configured to obtain a similar result with any explanation methods applied thereto.SOLUTION: A learning apparatus includes: an input unit which acquires leaning data and a correct answer value for the learning data; an inference unit which outputs a first inference value based on the learning data and a machine learning algorithm; an explanation unit which outputs multiple pieces of explanation information which are grounds for determining the first inference value; an inference evaluation unit which obtains an inference evaluation result based on the correct answer value and the first inference value; an explanation evaluation unit which obtains an explanation evaluation result based on the multiple pieces of explanation information; and an update unit which updates a first weight parameter of the machine learning algorithm based on the inference evaluation result and the explanation evaluation result.SELECTED DRAWING: Figure 1

Description

The present invention relates to a learning device, a learning method and a program.

A neural network (hereinafter also referred to as “NN”) has high performance in image recognition and the like. However, NNs are generally composed of a huge number of parameters and complicated models, and it is difficult to interpret the relationship between NN parameters and output results from NNs. Therefore, there is a problem that it is difficult to improve the accuracy of the NN by manually correcting the parameters of the NN. In order to solve this problem, several techniques have been proposed for explaining the basis of judgment of the NN.

For example, there is a known method that uses the gradient of the NN's input and output, calculates the degree of contribution to the inference result for each input region, and presents the calculated contribution to explain the basis for the NN's judgment ( For example, see Patent Document 1 and Non-Patent Document 1). Furthermore, in the classification problem, there is known a method of learning a typical example (prototype) of each classification class and displaying the degree of similarity with the prototype for each region of the input to explain the basis for the NN's decision (for example, , see Patent Document 2).

Japanese translation of PCT publication No. 2018-513507

Daniel Smilkov, 4 others, "SmoothGrad: removing noise by adding noise", [online], [searched on March 19, 2021], Internet <https://arxiv.org/abs/1706.03825> Chaofan Chen, 5 others, "This Looks Like That: Deep Learning for Interpretable Image Recognition", [online], [searched March 19, 2021], Internet <https://arxiv.org/abs/1806.10574> Hiroshi Fukui, 3 others, "AttentionBranch Network: Learning of Attention Mechanism for Visual Explanation", [online], [searched March 19, 2021], Internet <https://arxiv.org/abs/1812.10025>

However, although all of the above explanation methods present the regions of the input that contributed to the inference results, different results are obtained for each explanation method even if the explanation methods are applied to the same NN model. There was a problem. If different explanation methods yield different results, the explanation may become unreliable. Therefore, it is desired to provide a technique for generating a trained model (for example, a trained model of NN) that provides similar results regardless of which explanation method is applied.

In order to solve the above problem, according to one aspect of the present invention, an input unit that acquires learning data and a correct value of the learning data, and a first learning data based on the learning data and a machine learning algorithm an inference unit for outputting an inference value of; an explanation unit for outputting a plurality of pieces of explanation information of judgment grounds for said first inference value; and obtaining an inference evaluation result based on said correct value and said first inference value. an inference evaluation unit, an explanation evaluation unit that obtains an explanation evaluation result based on the plurality of pieces of explanation information, and updates a first weight parameter of the machine learning algorithm based on the inference evaluation result and the explanation evaluation result. and an updating unit for performing.

The explanation unit may output the plurality of pieces of explanation information based on a plurality of explanation methods.

At least one explanation technique among the plurality of explanation techniques may include a function capable of error backpropagation.

The explanation unit may have a second weight parameter, and the update unit may update the second weight parameter by error backpropagation.

The machine learning algorithms may include neural networks.

The inference unit outputs the first inference value and the feature amount based on the learning data and the neural network, and at least one of the plurality of explanation methods uses the first inference method. Explanation information may be output based on the value, the feature amount, and the neural network.

At least one explanation technique among the plurality of explanation techniques outputs explanation information and a second inference value based on the first inference value, the feature amount, and the neural network, and the inference evaluation unit , the inference evaluation result may be obtained based on the correct value, the first inference value and the second inference value.

The updating unit may update the first weighting parameter based on an addition result of the inference evaluation result and the explanation evaluation result.

Each of the plurality of explanatory information may be information indicating a degree of contribution of the learning data given to the first inference value.

The information indicating the degree of contribution may be a heat map indicating the degree of contribution of the learning data given to the first inference value for each region.

The information indicating the degree of contribution may be linguistic explanation information indicating the degree of contribution of the learning data given to the first inference value.

According to another aspect of the present invention, acquiring learning data and a correct value of the learning data, and outputting a first inference value based on the learning data and a machine learning algorithm outputting a plurality of pieces of explanation information as basis for judgment of the first inference value; obtaining an inference evaluation result based on the correct value and the first inference value; and producing the plurality of pieces of explanation information. and updating a first weight parameter of the machine learning algorithm based on the inference evaluation result and the explanation evaluation result. be.

According to another aspect of the present invention, a computer comprises an input unit for acquiring learning data and a correct value of the learning data, and a first inference based on the learning data and a machine learning algorithm. an inference unit for outputting a value; an explanation unit for outputting a plurality of pieces of explanation information of the basis for judgment of said first inference value; and an inference evaluation for obtaining an inference evaluation result based on said correct value and said first inference value. an explanation evaluation unit that obtains an explanation evaluation result based on the plurality of explanation information; and an update that updates a first weight parameter of the machine learning algorithm based on the inference evaluation result and the explanation evaluation result. A program functioning as a learning device comprising a part is provided.

As described above, according to the present invention, there is provided a technique for generating a trained model that provides similar results regardless of which explanation method is applied.

1 is a diagram showing a functional configuration example of a learning device according to a first embodiment of the present invention; FIG. FIG. 4 is a diagram showing the relationship between an inference value, an explanation method, explanation information, and a loss function; 4 is a flow chart showing an example of the operation of the learning device according to the embodiment; FIG. 7 is a diagram showing an example of functional configuration of a learning device according to a second embodiment of the present invention; 4 is a flow chart showing an example of the operation of the learning device according to the embodiment; 1 is a diagram showing a hardware configuration of an information processing device as an example of a learning device; FIG.

Preferred embodiments of the present invention will be described in detail below with reference to the accompanying drawings. In the present specification and drawings, constituent elements having substantially the same functional configuration are denoted by the same reference numerals, thereby omitting redundant description.

In addition, in this specification and drawings, a plurality of components having substantially the same functional configuration may be distinguished by attaching different numerals after the same reference numerals. However, when there is no particular need to distinguish between a plurality of constituent elements having substantially the same functional configuration, only the same reference numerals are used. Also, similar components in different embodiments may be distinguished by attaching different alphabets after the same reference numerals. However, when there is no particular need to distinguish between similar components of different embodiments, only the same reference numerals are used.

(0. Outline of embodiment)
An outline of an embodiment of the present invention will be described. In the embodiment of the present invention, a learning device that performs neural network learning based on a combination of learning data and correct values will be described. But neural networks are just one example of machine learning algorithms. Therefore, other machine learning algorithms may be used instead of neural networks. For example, SVM (Support Vector Machine) or the like may be used as another example of the machine learning algorithm.

(1. First embodiment)
First, a first embodiment of the present invention will be described. FIG. 1 is a diagram showing a functional configuration example of a learning device 10 according to the first embodiment of the present invention. As shown in FIG. 1, the learning device 10 according to the first embodiment of the present invention includes an input unit 115, an inference unit 121, an explanation unit 123, an inference evaluation unit 140, an explanation evaluation unit 150, and an updating unit 160 .

In the first embodiment of the present invention, it is mainly assumed that the inference unit 121 and the explanation unit 123 include neural networks. Below, a neural network is also written as "NN." More specifically, the inference unit 121 includes a neural network (hereinafter also referred to as “inference NN”), but the inference unit 121 may have any specific configuration.

For example, if the inference unit 121 includes a function capable of error backpropagation, part of the inference unit 121 may perform inference based on a specific inference algorithm.

The input unit 115, the inference unit 121, the explanation unit 123, the inference evaluation unit 140, the explanation evaluation unit 150, and the update unit 160 are arithmetic units such as a CPU (Central Processing Unit) or a GPU (Graphics Processing Unit). , and a program stored in a ROM (Read Only Memory) is developed in a RAM by an arithmetic unit and executed, thereby realizing its function. At this time, a computer-readable recording medium recording the program may also be provided. Alternatively, these blocks may be composed of dedicated hardware, or may be composed of a combination of multiple pieces of hardware. Data necessary for calculation by the calculation device are appropriately stored in a storage unit (not shown).

The data set 110, the weighting parameter 132 (first weighting parameter) of the inference NN, and the weighting parameter 133 (second weighting parameter) of the explanation section 123 are stored in a storage section (not shown). The storage unit may be composed of a memory such as a RAM (Random Access Memory), a hard disk drive, or a flash memory.

In the initial state, the weight parameter 132 of the inference NN and the weight parameter 133 of the explanation part 123 are each set to an initial value. For example, the initial values set to these may be random values, but may be any value. For example, the initial values set for these may be learned values obtained in advance through learning.

(Dataset 110)
The data set 110 includes a plurality of learning data (input data) and correct values for each of the plurality of learning data. In the embodiment of the present invention, it is mainly assumed that the learning data is image data (particularly still image data). However, the type of learning data is not particularly limited, and data other than image data can be used as learning data. For example, the learning data may be moving image data including a plurality of frames, or may be sound data.

(Input unit 115)
The input unit 115 sequentially acquires combinations of learning data and correct values from the data set 110 . Input unit 115 sequentially outputs combinations of learning data and correct values to inference unit 121 .

Note that, for example, when the input unit 115 has acquired all combinations of learning data and correct values from the data set 110, the input unit 115 repeats the operation of acquiring the combinations again from the beginning and outputting them again a predetermined number of times. good. In such a case, the blocks subsequent to the input unit 115 may sequentially repeat their processes based on the re-input.

(Inference unit 121)
The inference unit 121 obtains an inference value (first inference value) based on the input data input from the input unit 115 and the inference NN. The weight parameter 132 of the inference NN is stored in a storage unit (not shown). Therefore, the inference unit 121 acquires the weight parameter 132 from a storage unit (not shown), performs inference by the inference NN based on the acquired weight parameter 132 and the input data output from the input unit 115, and obtains an inference value.

In this specification, obtaining an output from a neural network based on an input to the neural network is broadly referred to as "inference".

The inference unit 121 outputs inference values to the explanation unit 123 and the inference evaluation unit 140, respectively. As will be described later, among the plurality of explanation methods included in the explanation unit 123, there may be an explanation method that requires the feature amount output from the inference NN. In such a case, the inference unit 121 may output the feature amount output from the intermediate layer of the inference NN to the explanation unit 123 together with the inference value.

A specific configuration of the inference NN is not particularly limited. However, it is preferable that the output format of the inference NN is set together with the format of the correct value corresponding to the learning data. For example, if the correct answer values are classes of a classification problem, the output of the inference NN may be a one-hot vector with a length equal to the number of classes.

(Description part 123)
Explanation unit 123 outputs to explanation evaluation unit 150 a plurality of pieces of explanation information of the basis for determining the inference value input from inference unit 121 . More specifically, the explanation part 123 is configured including a plurality of explanation methods. Then, the explanation unit 123 generates explanation information corresponding to each of the plurality of explanation techniques based on the plurality of explanation techniques, and sends the explanation information corresponding to each of the plurality of explanation techniques to the explanation evaluation unit 150 as a plurality of explanation information. Output.

Here, each of the plurality of explanation information is information indicating the degree of contribution of the input data given to the inference value input from the inference section 121 . The explanation technique may be any technique as long as it is a technique for generating such explanation information. For example, at least one explanation technique among the plurality of explanation techniques may generate explanation information based on an inference value input from the inference unit 121 .

More specifically, at least one of the plurality of explanation methods may obtain explanation information based on the inference value and the inference NN input from the inference unit 121 .

Alternatively, as described above, not only the inference value but also the feature amount may be input from the inference unit 121 to the explanation unit 123 . In such a case, at least one explanation technique among the plurality of explanation techniques may obtain explanation information based on the inference value, the feature amount, and the inference NN input from the inference unit 121 . That is, at least one of the plurality of explanation methods may obtain explanation information by performing inference by the inference NN based on the inference value and feature amount input from the inference unit 121 .

For example, at least one explanation technique among the plurality of explanation techniques may include a function capable of error backpropagation. At this time, as will be described later, the weight parameter 133 of the description unit 123 can be updated by the update unit 160 by error backpropagation. That is, at least one of the plurality of explanation methods may generate explanation information using the updated weight parameters 133 by the error backpropagation method.

Non-Patent Literature 1 describes an example of an explanation method for generating explanation information using updated weighting parameters by the error backpropagation method. The explanation method described in Non-Patent Document 1 is an explanation method that outputs, as explanation information, a heat map indicating regions of inputs to a neural network that have a high degree of contribution to an inference value. The explanation method described in Non-Patent Document 1 can also be applied to the embodiments of the present invention.

That is, in the embodiment of the present invention, at least one of the plurality of explanation methods generates a heat map showing the degree of contribution of input data to the inference value output from the inference unit 121 for each region. You may

For example, a plurality of explanation techniques included in the explanation section 123 may be prepared by using the explanation technique described in Non-Patent Document 1 and changing the parameter values of the explanation technique. Alternatively, a plurality of explanation techniques included in the explanation section 123 may be prepared by combining the explanation technique described in Non-Patent Document 1 and the explanation technique described in Patent Document 1. In addition, various explanation methods such as Vanilla Gradient and Grad-CAM can be applied.

As an example, let Fk (k=1, 2, . n) and the inference value is x, the relationship shown in the following formula (1) can be established.

Mk=Fk(x) (1)

2 shows the relationship between the inference value x, the explanation method Fk (k=1, 2, . . . , n), the explanation information Mk (k=1, 2, . . . , n), and the loss function G. It is shown.

(Inference evaluation unit 140)
The inference evaluation unit 140 obtains an inference evaluation result based on the inference value input from the inference unit 121 and the correct value obtained by the input unit 115 . More specifically, the inference evaluation unit 140 obtains an inference evaluation result by comparing the inference value input from the inference unit 121 and the correct value obtained by the input unit 115 . The inference evaluation unit 140 outputs the inference evaluation result to the update unit 160 .

In the embodiment of the present invention, it is assumed that the inference evaluation unit 140 calculates a loss function corresponding to the inference value input from the inference unit 121 and the correct value obtained by the input unit 115 as the inference evaluation result. Here, the loss function according to the inference value and the correct value is not limited to a specific function, and a loss function similar to loss functions used in general neural networks may be used. For example, the loss function according to the inferred value and the correct value may be the mean squared error based on the difference between the correct value and the inferred value.

(Description evaluation unit 150)
Explanation evaluation unit 150 obtains explanation evaluation results based on a plurality of pieces of explanation information input from explanation unit 123 . More specifically, explanation evaluation unit 150 obtains explanation evaluation results by comparing a plurality of pieces of explanation information input from explanation unit 123 . Explanation evaluation section 150 outputs the explanation evaluation result to update section 160 .

In the embodiment of the present invention, it is assumed that the explanation evaluation unit 150 calculates a loss function according to a plurality of pieces of explanation information input from the explanation unit 123 as explanation evaluation results. Here, the loss function corresponding to the multiple pieces of explanatory information is not limited to a specific function, and a loss function similar to loss functions used in general neural networks may be used. For example, the loss function corresponding to a plurality of explanatory information may be a mean squared error based on the difference between the average value of the plurality of explanatory information and each of the plurality of explanatory information.

As an example, let Mk (k=1, 2, . The relationship shown in can be established.

L = G (M1, M2, ..., Mn) ... (2)

(Update unit 160)
The update unit 160 updates the weight parameter 132 of the inference NN based on the inference evaluation result input from the inference evaluation unit 140 and the explanation evaluation result input from the explanation evaluation unit 150 . As a result, the weight parameter 132 of the inference NN can be updated so that the inference value output from the inference unit 121 approaches the correct value and the plurality of pieces of explanation information output from the explanation unit 123 approach each other. . The weight parameters 132 of the inference NN may be updated by backpropagation.

For example, the update unit 160 adds the inference evaluation result input from the inference evaluation unit 140 and the explanation evaluation result input from the explanation evaluation unit 150, and updates the weight parameter 132 of the inference NN based on the addition result. should be done. At this time, the update unit 160 may update the weight parameter 132 of the inference NN by error back propagation using the calculated addition result as an error.

Furthermore, the update unit 160 may update the weight parameter 133 that the description unit 123 has. More specifically, when at least one explanation technique among the plurality of explanation techniques includes a function capable of error backpropagation, the update unit 160 performs error backpropagation based on the inference evaluation result and the explanation evaluation result. Backpropagation) may update the weight parameter 133 of the explanation part 123 .

Note that the learning termination condition (that is, the weight parameter update termination condition) is not particularly limited as long as it indicates that the learning of the inference NN has been performed to some extent. Specifically, the learning termination condition may include a condition that the value of the loss function is smaller than a threshold. Alternatively, the learning end condition may include a condition that the change in the value of the loss function is smaller than a threshold (a condition that the value of the loss function has converged). Alternatively, the learning termination condition may include a condition that the weighting parameters have been updated a predetermined number of times. Alternatively, when the inference evaluation unit 140 calculates the accuracy (for example, the percentage of correct answers) based on the correct value and the inference value, the learning termination condition is that the accuracy exceeds a predetermined percentage (for example, 90%). may include the condition

(Operation of the first embodiment)
The flow of operation of the learning device 10 according to the first embodiment of the present invention will be described with reference to FIG. FIG. 3 is a flow chart showing an operation example of the learning device 10 according to the first embodiment of the present invention.

First, as shown in FIG. 3 , the input unit 115 acquires a combination of input data (that is, learning data) and correct values from the data set 110 . Furthermore, the inference unit 121 acquires the weight parameter 132 of the inference NN (S11). The inference unit 121 makes an inference based on the input data and the inference NN obtained by the input unit 115 (S12), and outputs the inference value obtained by the inference to the inference evaluation unit 140 and the explanation unit 123, respectively.

Based on the inference value input from the inference unit 121, the explanation unit 123 generates explanation information corresponding to each of the plurality of explanation methods as a plurality of pieces of explanation information (S13). The explanation unit 123 outputs the generated plural pieces of explanation information to the explanation evaluation unit 150 .

The inference evaluation unit 140 evaluates the inference value input from the inference unit 121 based on the correct value obtained by the input unit 115 to obtain an inference evaluation result. More specifically, the inference evaluation unit 140 calculates a loss function according to the correct value and the inference value as an inference evaluation result. The inference evaluation unit 140 outputs the calculated inference evaluation result to the update unit 160 .

Explanation evaluation unit 150 obtains explanation evaluation results based on a plurality of pieces of explanation information input from explanation unit 123 . More specifically, the explanation evaluation unit 150 calculates a loss function according to the difference between the multiple pieces of explanation information input from the explanation unit 123 as the explanation evaluation result. The explanation evaluation unit 150 outputs the calculated explanation evaluation result to the updating unit 160 (S14).

The update unit 160 updates the weight parameter 132 of the inference NN based on the inference evaluation result input from the inference evaluation unit 140 and the explanation evaluation result input from the explanation evaluation unit 150 (S15). More specifically, the updating unit 160 updates the weight parameter 132 of the inference NN by error backpropagation based on the inference evaluation result and the explanation evaluation result. Further, the update unit 160 updates the weight parameter 133 of the explanation unit 123 by error backpropagation based on the inference evaluation result and explanation evaluation result.

The updating unit 160 determines whether or not the learning termination condition is satisfied each time the updating of the weight parameter based on the input data is completed (S16). If it is determined that the learning termination condition is not satisfied ("NO" in S16), the operation proceeds to S11, the input unit 115 acquires the next data for learning, and the inference unit 121 and explanation unit 123 , the inference evaluation unit 140, the explanation evaluation unit 150, and the update unit 160, respectively, perform their respective processes again based on the next input data. On the other hand, when updating unit 160 determines that the end condition for learning is satisfied ("YES" in S16), learning ends.

The flow of operation of the learning device 10 according to the first embodiment of the present invention has been described above.

(Summary of the first embodiment)
As described above, according to the first embodiment of the present invention, a mechanism for generating a plurality of explanatory information for a single NN model and evaluating the difference between the plurality of explanatory information is the learning function of the NN. added to the mechanism. As a result, the NN can be trained such that the difference between the multiple explanations generated by the multiple explanation methods is small. As a result, even when different explanation methods are used, it is possible to obtain a model that generates a plurality of pieces of explanation information close to each other (that is, a model with consistent explanation).

The first embodiment of the present invention has been described above.

(2. Second embodiment)
Next, a second embodiment of the invention will be described. FIG. 4 is a diagram showing a functional configuration example of the learning device 20 according to the second embodiment of the present invention. As shown in FIG. 4, the learning device 20 according to the second embodiment of the present invention includes an input unit 115, an inference unit 121, and A description evaluation unit 150 and an update unit 160 are provided. Furthermore, the learning device 20 according to the second embodiment of the present invention includes an explanation unit 223 and an inference evaluation unit 240 .

Also in the second embodiment of the present invention, the functions of the data set 110, the input section 115, the inference section 121, the explanation evaluation section 150 and the update section 160 are exhibited as in the first embodiment of the present invention. Therefore, the explanation part 223 and the inference evaluation part 240 are mainly explained below.

Explanation unit 223 and reasoning evaluation unit 240 include an arithmetic device such as a CPU or GPU, and their functions can be realized by the arithmetic device expanding a program stored in ROM into RAM and executing the program. At this time, a computer-readable recording medium recording the program may also be provided. Alternatively, these blocks may be composed of dedicated hardware, or may be composed of a combination of multiple pieces of hardware. Data necessary for calculation by the calculation device are appropriately stored in a storage unit (not shown).

(Description part 223)
The explanation unit 223 outputs, to the explanation evaluation unit 150, a plurality of pieces of explanation information of the judgment basis of the inference value input from the inference unit 121, similarly to the explanation unit 123 according to the first embodiment of the present invention. For example, at least one explanation technique among the plurality of explanation techniques may generate explanation information based on an inference value input from the inference unit 121 .

At least one of the plurality of explanation methods may obtain explanation information based on the inference value and the inference NN input from the inference unit 121 . In the second embodiment of the present invention, it is assumed that not only an inference value but also a feature amount is input from the inference unit 121 to the explanation unit 223 . In such a case, at least one of the plurality of explanation methods obtains explanation information based on the inference value, the feature amount, and the inference NN input from the inference unit 121 .

As in the first embodiment of the present invention, at least one explanation technique among the plurality of explanation techniques may include a function capable of error backpropagation. At this time, the weight parameter 133 of the explanation unit 123 can be updated by the update unit 160 by the error backpropagation method. That is, at least one of the plurality of explanation methods may generate explanation information using the updated weight parameters 133 by the error backpropagation method.

Further, in the second embodiment of the present invention, it is assumed that at least one explanation technique out of a plurality of explanation techniques not only generates explanation information but also calculates an inference value (second inference value). In such a case, the explanation unit 123 outputs not only the plurality of pieces of explanation information but also the inference value thus calculated to the inference evaluation unit 240 .

In Non-Patent Document 3, based on the feature amount output from the neural network, a heat map indicating a region with a high degree of contribution to the inference value among the inputs to the neural network is output as explanatory information. An example of an explanation method is described in which inference is made based on feature values and an inference value is output. The explanation method described in Non-Patent Document 3 can also be applied to the embodiment of the present invention.

For example, a plurality of explanation techniques included in the explanation section 223 may be prepared by using the explanation technique described in Non-Patent Document 3 and changing the parameter values of the explanation technique. Alternatively, the plurality of explanation techniques included in the explanation section 223 may be prepared by combining the explanation technique described in Non-Patent Document 3 and other explanation techniques.

Inference evaluation unit 240 obtains an inference evaluation result based on the inference value input from inference unit 121 , the correct value obtained by input unit 115 , and the inference value input from explanation unit 223 . More specifically, the inference evaluation unit 240 compares the inference value input from the inference unit 121 and the correct value obtained by the input unit 115, and compares the inference value input from the explanation unit 223 with the inference value obtained by the input unit 115. An inference evaluation result is obtained by comparing with the obtained correct answer value. The inference evaluation unit 240 outputs the inference evaluation result to the update unit 160 .

In the embodiment of the present invention, the inference evaluation unit 240 infers a loss function according to the inference value input from the inference unit 121, the inference value input from the explanation unit 223, and the correct value obtained by the input unit 115. Assume a case where it is calculated as an evaluation result. Here, as in the first embodiment of the present invention, the loss function according to the inference value and correct value is not limited to a specific function.

(Operation of Second Embodiment)
The operation flow of the learning device 20 according to the second embodiment of the present invention will be described with reference to FIG. FIG. 5 is a flow chart showing an operation example of the learning device 20 according to the second embodiment of the present invention.

S11-S12 are executed in the same manner as S11-S12 (FIG. 3) according to the first embodiment of the present invention.

As in the first embodiment of the present invention, the explanation unit 223 generates a plurality of pieces of explanation information corresponding to each of the plurality of explanation methods based on the inference value input from the inference unit 121 . The explanation unit 223 outputs the generated plural pieces of explanation information to the explanation evaluation unit 150 . Furthermore, at least one of the plurality of explanation methods included in the explanation section 223 calculates an inference value (second inference value). The explanation unit 223 outputs the calculated inference value to the inference evaluation unit 240 (S23).

Inference evaluation unit 240 obtains an inference evaluation result based on the inference value input from inference unit 121 , the correct value obtained by input unit 115 , and the inference value input from explanation unit 223 . The inference evaluation unit 240 outputs the inference evaluation result to the update unit 160 . As in the first embodiment of the present invention, the explanation evaluation unit 150 obtains explanation evaluation results based on multiple pieces of explanation information input from the explanation unit 123 . Then, the explanation evaluation unit 150 outputs the calculated explanation evaluation result to the updating unit 160 (S14).

As in the first embodiment of the present invention, the update unit 160 calculates the weight of the inference NN based on the inference evaluation result input from the inference evaluation unit 240 and the explanation evaluation result input from the explanation evaluation unit 150. The parameter 132 is updated (S15). More specifically, the updating unit 160 updates the weight parameter 132 of the inference NN by error backpropagation based on the inference evaluation result and the explanation evaluation result. Further, the update unit 160 updates the weight parameter 133 of the explanation unit 123 by error backpropagation based on the inference evaluation result and explanation evaluation result. Further, the update unit 160 updates the weight parameter 133 of the explanation unit 123 by error backpropagation based on the inference evaluation result and explanation evaluation result.

The updating unit 160 determines whether or not the learning termination condition is satisfied each time the updating of the weight parameter based on the input data is completed (S16). If it is determined that the learning end condition is not satisfied ("NO" in S16), the operation proceeds to S11, the next data for learning is acquired by the input unit 115, and the inference unit 121 and explanation unit 223 , the inference evaluation unit 240, the explanation evaluation unit 150, and the update unit 160, respectively, perform their respective processes again based on the next input data. On the other hand, when updating unit 160 determines that the end condition for learning is satisfied ("YES" in S16), learning ends.

The flow of operation of the learning device 20 according to the second embodiment of the present invention has been described above.

(Summary of the second embodiment)
As described above, according to the second embodiment of the present invention, the same effects as those of the first embodiment of the present invention can be obtained. Further, according to the second embodiment of the present invention, by connecting the explanation unit 223 and the inference evaluation unit 240, it is possible to use an explanation technique that outputs both explanation information and inference values.

The second embodiment of the present invention has been described above.

(3. Hardware configuration example)
Next, a hardware configuration example of the learning device 10 according to the first embodiment of the present invention will be described. Note that the hardware configuration of the learning device 20 according to the second embodiment of the present invention can also be realized in the same manner.

A hardware configuration example of the information processing device 900 will be described below as a hardware configuration example of the learning device 10 according to the embodiment of the present invention. Note that the hardware configuration example of the information processing device 900 described below is merely an example of the hardware configuration of the learning device 10 . Therefore, as for the hardware configuration of the learning device 10, unnecessary configurations may be deleted from the hardware configuration of the information processing device 900 described below, or a new configuration may be added.

FIG. 6 is a diagram showing the hardware configuration of an information processing device 900 as an example of the learning device 10 according to the embodiment of the present invention. The information processing device 900 includes a CPU (Central Processing Unit) 901, a ROM (Read Only Memory) 902, a RAM (Random Access Memory) 903, a host bus 904, a bridge 905, an external bus 906, and an interface 907. , an input device 908 , an output device 909 , a storage device 910 and a communication device 911 .

The CPU 901 functions as an arithmetic processing device and a control device, and controls general operations within the information processing device 900 according to various programs. Alternatively, the CPU 901 may be a microprocessor. The ROM 902 stores programs, calculation parameters, and the like used by the CPU 901 . The RAM 903 temporarily stores programs used in the execution of the CPU 901, parameters that change as appropriate during the execution, and the like. These are interconnected by a host bus 904 comprising a CPU bus or the like.

The host bus 904 is connected via a bridge 905 to an external bus 906 such as a PCI (Peripheral Component Interconnect/Interface) bus. Note that the host bus 904, the bridge 905 and the external bus 906 do not necessarily have to be configured separately, and these functions may be implemented in one bus.

The input device 908 includes input means for the user to input information, such as a mouse, keyboard, touch panel, button, microphone, switch, and lever, and an input control circuit that generates an input signal based on the user's input and outputs it to the CPU 901 . etc. A user who operates the information processing apparatus 900 can input various data to the information processing apparatus 900 and instruct processing operations by operating the input device 908 .

The output device 909 includes, for example, a CRT (Cathode Ray Tube) display device, a liquid crystal display (LCD) device, an OLED (Organic Light Emitting Diode) device, a display device such as a lamp, and an audio output device such as a speaker.

The storage device 910 is a device for data storage. The storage device 910 may include a storage medium, a recording device that records data on the storage medium, a reading device that reads data from the storage medium, a deletion device that deletes data recorded on the storage medium, and the like. The storage device 910 is configured by, for example, an HDD (Hard Disk Drive). The storage device 910 drives a hard disk and stores programs executed by the CPU 901 and various data.

The communication device 911 is, for example, a communication interface configured with a communication device or the like for connecting to a network. Also, the communication device 911 may support either wireless communication or wired communication.

The hardware configuration example of the learning device 10 according to the embodiment of the present invention has been described above.

(4. Summary)
Although the preferred embodiments of the present invention have been described in detail above with reference to the accompanying drawings, the present invention is not limited to such examples. It is obvious that a person having ordinary knowledge in the technical field to which the present invention belongs can conceive of various modifications or modifications within the scope of the technical idea described in the claims. It is understood that these also naturally belong to the technical scope of the present invention.

In the first and second embodiments of the present invention, it is mainly assumed that each of the multiple explanation methods includes a function capable of backpropagating errors. However, some of the multiple explanation techniques may involve functions that cannot be backpropagated.

In the first and second embodiments of the present invention, it is mainly assumed that each of a plurality of explanation methods generates explanation information based on an inference value. However, at least some of the multiple descriptive techniques may output descriptive information labeled by humans.

In the first embodiment and the second embodiment of the present invention, each of the plurality of explanation methods creates a heat map showing the degree of contribution of input data to the inference value output from the inference unit 121 for each region. I mainly assumed the case of generating. However, each of the plurality of explanation methods may be other information indicating the degree of contribution of the input data given to the inference value output from the inference unit 121 .

Each of the plurality of explanation methods may be linguistic explanation information indicating the degree of contribution of the input data given to the inference value output from the inference unit 121 . For example, the linguistic explanation information may be explanation information such as "there is an object having characteristics of a predetermined kind of animal in the center of the image (input data)". Alternatively, each of the plurality of explanation methods may be information indicating the degree of contribution of each piece of input data given to the inference value output from the inference unit 121 .

10, 20 learning device 110 data set 115 input unit 121 inference unit 123, 223 explanation unit 132 weight parameter 133 weight parameter 140, 240 inference evaluation unit 150 explanation evaluation unit 160 update unit

Claims (13)

an input unit that acquires learning data and a correct value of the learning data;
an inference unit that outputs a first inference value based on the learning data and the machine learning algorithm;
an explanation unit that outputs a plurality of pieces of explanation information for the basis for judgment of the first inference value;
an inference evaluation unit that obtains an inference evaluation result based on the correct value and the first inference value;
an explanation evaluation unit that obtains an explanation evaluation result based on the plurality of pieces of explanation information;
an updating unit that updates a first weight parameter of the machine learning algorithm based on the inference evaluation result and the explanation evaluation result;
A learning device comprising: The explanation unit outputs the plurality of explanation information based on a plurality of explanation techniques.
A learning device according to claim 1. At least one explanation method among the plurality of explanation methods includes a function capable of back propagation,
3. A learning device according to claim 2. The description part has a second weighting parameter,
The update unit updates the second weight parameter by error backpropagation.
4. A learning device according to claim 3. the machine learning algorithm comprises a neural network;
A learning device according to any one of claims 2 to 4. The inference unit outputs the first inference value and the feature value based on the learning data and the neural network,
At least one of the plurality of explanation techniques outputs explanation information based on the first inference value, the feature amount, and the neural network.
The learning device according to claim 5. at least one explanation method among the plurality of explanation methods outputs explanation information and a second inference value based on the first inference value, the feature value, and the neural network;
The inference evaluation unit obtains the inference evaluation result based on the correct value, the first inference value, and the second inference value.
7. A learning device according to claim 6. The update unit updates the first weight parameter based on the addition result of the inference evaluation result and the explanation evaluation result.
A learning device according to any one of claims 1 to 7. Each of the plurality of explanatory information is information indicating a degree of contribution of the learning data given to the first inference value,
9. A learning device according to claim 8. The information indicating the degree of contribution is a heat map indicating the degree of contribution of the learning data given to the first inference value for each region,
10. A learning device according to claim 9. The information indicating the degree of contribution is linguistic explanation information indicating the degree of contribution of the learning data given to the first inference value,
10. A learning device according to claim 9. Acquiring learning data and a correct value of the learning data;
outputting a first inference value based on the learning data and a machine learning algorithm;
outputting a plurality of pieces of explanatory information of the basis for judgment of the first inference value;
obtaining an inference evaluation result based on the correct value and the first inference value;
obtaining an explanation evaluation result based on the plurality of pieces of explanation information;
updating a first weight parameter of the machine learning algorithm based on the inference evaluation result and the explanation evaluation result;
A learning method comprising: the computer,
an input unit that acquires learning data and a correct value of the learning data;
an inference unit that outputs a first inference value based on the learning data and the machine learning algorithm;
an explanation unit that outputs a plurality of pieces of explanation information for the basis for judgment of the first inference value;
an inference evaluation unit that obtains an inference evaluation result based on the correct value and the first inference value;
an explanation evaluation unit that obtains an explanation evaluation result based on the plurality of pieces of explanation information;
an updating unit that updates a first weight parameter of the machine learning algorithm based on the inference evaluation result and the explanation evaluation result;
A program that functions as a learning device with

Images