JP2023087266A - Machine learning program, machine learning method, and machine learning device - Google Patents

Abstract

To suppress occurrence of degradation in retraining of a machine learning model.SOLUTION: A plurality of pieces of fourth data among a plurality of pieces of third data, and a plurality of pieces of fifth data associated with the fourth data among a plurality of pieces of second data are selected, on the basis of difference between an output value of a first machine learning model with respect to data included in the plurality of pieces of third data, and a value of a correct label corresponding to the data included in the plurality of pieces of third data, and the difference between the output value of the first machine learning model with respect to each of the plurality of pieces of third data and the output value of the first machine learning model with respect to each of a plurality of pieces of second data. A second machine learning model is generated by identifying and updating a plurality of second parameters among a plurality of first parameters included in the first machine learning model, on the basis of a numerical value calculated during forward-propagation and the numerical value calculated during inverse-propagation when inputting the plurality of pieces of fourth data and the plurality of pieces of fifth data to the first machine learning model.SELECTED DRAWING: Figure 1

Description

The present invention relates to a machine learning program, a machine learning method and a machine learning apparatus.

In a system using a machine learning model such as a deep neural network (DNN), there are cases where the machine learning model is corrected when an output undesirable to the user occurs. Hereinafter, the DNN machine learning model may be referred to as a DNN model.

For example, in an automatic driving system using a camera, if a DNN model misrecognizes a road sign, the DNN model is corrected so that the road sign can be recognized correctly.

DNN models are built according to input training data rather than built according to specifications. Therefore, the DNN model is corrected by inputting training data.

JP-A-9-128358

However, in such a conventional DNN model correction method using training data, although data used for correction is collected, it is not always possible to correct errors using the collected data.

In addition, retraining of the DNN model may result in incorrect inference after retraining for data that was correctly inferred before retraining, which may cause degrade. Degradation is caused by the principle of CACE (Changing Anything Changes Everything).

In one aspect, an object of the present invention is to suppress occurrence of degradation in retraining of a machine learning model.

In one aspect, the machine learning program may cause the computer to perform the following processes. The processing includes a second plurality of data and an incorrect third prediction result according to the output value of the first machine learning model for each of the first plurality of data among the first plurality of data. may include a process of identifying a plurality of data of In addition, the processing includes output values of the first machine learning model for data included in the third plurality of data and correct label values corresponding to the data included in the third plurality of data. and the difference between the output value of the first machine learning model for each of the third plurality of data and the output value of the first machine learning model for each of the second plurality of data selecting a fourth plurality of data out of the third plurality of data and a fifth plurality of data related to the fourth data out of the second plurality of data based on may contain. Furthermore, the processing includes numerical values calculated during forward propagation and backward propagation when the fourth plurality of data and the fifth plurality of data are input to the first machine learning model. a plurality of second parameters among the first plurality of parameters included in the first machine learning model are specified based on the calculated numerical values, and the second plurality of parameters among the first plurality of parameters are specified A process of generating a second machine learning model by updating only a plurality of parameters may be included.

In one aspect, the present invention can suppress the occurrence of degradation in retraining of machine learning models.

1 is a diagram schematically showing the configuration of an information processing apparatus as an example of an embodiment; FIG. FIG. 4 is a diagram for explaining processing of a model training unit in the information processing device as an example of an embodiment; FIG. 10 is a diagram for explaining processing of the neighborhood success data search unit of the information processing apparatus as one example of the embodiment; FIG. 10 is a diagram for explaining processing of a neighborhood success data search unit in the information processing apparatus as one example of the embodiment; FIG. 10 is a diagram for explaining processing of a neighborhood success data search unit in the information processing apparatus as one example of the embodiment; FIG. 10 is a diagram for explaining processing of a failure data narrowing unit of the information processing apparatus as one example of the embodiment; FIG. 10 is a diagram for explaining processing of a successful data narrowing unit of the information processing apparatus as one example of the embodiment; FIG. 10 is a diagram for explaining processing by a weight influence measurement unit of the information processing apparatus as one example of the embodiment; FIG. 10 is a diagram for explaining processing by a weight influence measurement unit of the information processing apparatus as one example of the embodiment; FIG. 10 is a diagram for explaining processing by a weight influence measurement unit of the information processing apparatus as one example of the embodiment; FIG. 10 is a diagram for explaining processing by a weight influence measurement unit of the information processing apparatus as one example of the embodiment; FIG. 10 is a diagram for explaining a process of determining the number of selected weights by a correction target weight identification unit of the information processing apparatus as one example of the embodiment; FIG. 10 is a diagram for explaining processing for specifying correction target weights by the target weight specifying unit of the information processing apparatus as an example of the embodiment; FIG. 10 is a diagram illustrating a method of selecting correction target weights by a correction target weight specifying unit of the information processing apparatus as one example of the embodiment; 4 is a flowchart for explaining an overview of processing in an information processing apparatus as one example of an embodiment; 7 is a flowchart for explaining processing of a test data classification unit of an information processing device as an example of an embodiment; 8 is a flowchart for explaining processing of a neighborhood success data search unit of the information processing apparatus as one example of an embodiment; 8 is a flowchart for explaining processing by a failure data narrowing unit of the information processing apparatus as one example of the embodiment; 9 is a flowchart for explaining processing of a successful data narrowing unit of the information processing apparatus as an example of the embodiment; 7 is a flow chart for explaining processing of a weighted influence measuring unit of the information processing apparatus as an example of the embodiment; 8 is a flowchart for explaining processing of a correction target weight identifying unit of the information processing apparatus as an example of the embodiment; FIG. 4 is a diagram for explaining the relationship between modification and degradation of a DNN model; FIG. 4 is a diagram for explaining correction target weights determined in an information processing apparatus as an example of an embodiment; It is a figure which illustrates the simulation result in the information processing apparatus as an example of embodiment. It is a figure which illustrates the hardware constitutions of the information processing apparatus as an example of embodiment.

Embodiments of the present program, method and apparatus will be described below with reference to the drawings. However, the embodiments shown below are merely examples, and are not intended to exclude the application of various modifications and techniques not explicitly described in the embodiments. For example, this embodiment can be modified in various ways without departing from the spirit of the embodiment. Also, each drawing does not mean that it has only the constituent elements shown in the drawing, but can include other functions and the like.

(A) Configuration FIG. 1 is a diagram schematically showing the configuration of an information processing apparatus 1 as an example of an embodiment.

The information processing device 1 implements a model correction function for correcting a trained DNN model (machine learning model). As shown in FIG. 1, the information processing apparatus 1 includes a model training unit 101, a test data classification unit 102, a neighborhood success data search unit 103, a failure data narrowing unit 104, a successful data narrowing unit 105, a weighted influence measuring unit 106, and a It has a function as a correction target weight identification unit 107 . These blocks 101 to 107 are an example of a control section.

The model training unit 101 performs training (retraining, machine learning) by inputting training data to a DNN model to be trained. For example, the model training unit 101 inputs training data to a trained DNN model (first machine learning model), performs DNN model retraining (machine learning), and performs retrained DNN model (first machine learning model). 2 machine learning model).

FIG. 2 is a diagram for explaining the processing of the model training unit 101 in the information processing device 1 as an example of the embodiment.

Training data and a DNN model to be trained are input to the model training unit 101 . Training data may comprise, for example, test data (input data) and training labels (correct data, correct labels) corresponding to the test data. A plurality of training data may be referred to as a test data list.

The DNN model, for example, inputs input data such as images and sounds into the input layer, and sequentially executes predetermined calculations in hidden layers including convolution layers and pooling layers, so that the information obtained by the calculation is sent to the input side. Forward direction processing (forward propagation processing) is executed to sequentially transmit from to the output side. In each layer, nodes are connected by edges. Edges also have weights.

FIG. 2 shows layers, types, weights, and the number of weights as DNN model information.

A layer is identification information for specifying a layer, and natural numbers are used in the example shown in FIG. The type is the type of layer, and in the example shown in FIG. 4, Conv2D and Dense are used. A weight is a weight value used in each layer, and is flattened in one dimension in the example shown in FIG. Also, in the example shown in FIG. 2, the weights of the DNN model before training show the initial values of the weights. The number of weights is the number of weights provided for each layer.

After executing the forward processing, the model training unit 101 determines parameters to be used in the forward processing in order to reduce the value of the error function obtained from the output data output from the output layer and the correct data. Execute backward processing (backpropagation processing). Then, update processing is executed to update the weights (variables) based on the result of the backpropagation processing. For example, a gradient descent method may be used as an algorithm for determining the update width of weights used in backpropagation calculations. Hereinafter, the DNN model may be simply referred to as "model".

The model training unit 101 stores each information of the trained DNN model (first machine learning model, trained model) and the retrained DNN model (second machine learning model) in the storage device 13 (FIG. 25). reference). For example, the model training unit 101 may store a retrained DNN model created by retraining a trained DNN model in a storage area such as the information storage device 13 . The DNN model information stored in the storage area also includes weights.

The model training unit 101 calculates the forward propagation of the trained DNN model M using the test data as input, and calculates the output y of the final hidden layer.

The model training unit 101 updates (re-trains) only the parameters (correction target weights) specified by the correction target weight specifying unit 107, which will be described later, among the plurality of parameters (learning parameters) of the trained DNN model M. to generate a retrained DNN model (second machine learning model).

The test data classification unit 102 divides test data into success data and failure data based on the output obtained by inputting test data (input data) into a DNN model (trained model) and training labels corresponding to the input data. classified into

Success data is test data for which the DNN model outputs correct answers, and is test data for which the output matches the training label. Successful data is an example of a second plurality of data in which the prediction result (classification class) according to the output value of the trained DNN model (first machine learning model) is correct.

Failed data is test data for which the DNN model failed and whose output did not match the training label. Failure data is an example of a third plurality of data in which the prediction result (classification class) according to the output value of the trained DNN model (first machine learning model) is incorrect.

The test data classification unit 102 evaluates the test data with the DNN model M, and classifies the test data into a test data set X _pos in which inference succeeds and a test data set X _neg in which inference fails.

A set X _pos of test data for which inference is successful may be referred to as a successful data list X _pos . A set X _neg of test data that fails inference may also be referred to as a failed data list X _neg .

A trained DNN model M may be referred to as a trained model M. We may also denote the output of the trained model M by the symbol y. Furthermore, the correct label of the input data may be represented by the sign y^ (or ^ above y).

The test data classification unit 102 classifies the test data into the failure data set X _neg and the success data set X _pos for the trained model M using the output y of the trained model M and the correct label y^. .

The test data classification unit 102 stores the classified success data list X _pos and failure data list X _neg in a predetermined storage area such as the storage device 13 .

Neighboring success data search unit 103 searches for success data in the vicinity of failure data.

FIG. 3 is a diagram for explaining processing of the neighborhood success data search unit 103 of the information processing apparatus 1 as an example of the embodiment.

For each failure data in the failure data set X _neg , the neighborhood success data search unit 103 searches the classification boundary between the output y _neg obtained by inputting the failure data into the DNN model and the correct label y ^ (or classification Calculate the distance d _a (from the boundary). A classification boundary may be referred to as a decision boundary.

The distance d _a to the classification boundary can be obtained from the difference between the value of the maximum element of the output of the DNN model of the failure data and the index value of the correct label. That is, the distance d _a may be obtained by the following formula (1).

The distance d _a from the classification boundary is an example of the difference between the output value of the DNN model for failure data included in a plurality of failure data (third plurality of data) and the value of the correct label corresponding to the failure data. is.

FIG. 4 is a diagram for explaining the processing of the neighborhood success data search unit 103 in the information processing apparatus 1 as an example of the embodiment, and shows information generated by the neighborhood success data search unit 103. FIG.

In FIG. 4, symbol A illustrates an output and a correct label obtained by inputting failure data (test data) to the DNN model. The output of the DNN model may be the output of the last stage of the hidden layer.

Symbol B is the value of the maximum element of the DNN model output (y _neg [argmax(y _neg )]) and the value of the correct label (y _neg [y ^]) and Symbol C illustrates the distance d _a to the classification boundary of each failure data (test data) calculated based on the value indicated by symbol B. FIG.

Hereinafter, for convenience, the output obtained by inputting failure data to the DNN model may be referred to as failure data output, and the output obtained by inputting success data to the DNN model may be referred to as success data output. Also, the distance between the output of failure data and the output of success data may simply be referred to as the distance between failure data and success data.

The neighboring successful data searching unit 103 calculates the distance d _b between the output y _pos of each successful data and the output y _neg of the failed data. The distance d _b can be represented by the absolute value of the difference between the vectors of the values of the failure data and the values of the success data. That is, the distance d _b may be obtained by the following formula (2).

The distance d _b may be the "inter-data distance" between the failure data and the success data, and is the difference between the output value of the DNN model for each of the plurality of failure data and the output value of the DNN model for each of the plurality of success data. is an example.

Neighboring successful data search _{unit 103} finds successful data (see FIG _. 3 (see symbols P2 and P4 in ) to obtain a set X _near,i of near success data. The neighborhood coefficient c is a value less than 1 and may be arbitrarily set by the user. A range specified by the distance _{d a between the failed data x i and} the classification boundary and the neighborhood coefficient c may be referred to as a neighborhood range. The neighborhood range is a region represented by a circle of radius cd _a centered on the failed data x _i .

In addition, in FIG. 3, in order to distinguish from the failure data x _i indicated by the symbol P1 for convenience, the distance to the classification boundary related to the failure data x _i indicated by the symbol P3 is indicated by the symbol d _a ', and the failure indicated by the symbol P3 The distance between the data x _i and the output y _neg is denoted by symbol d _b ', and may be treated in the same manner as the failure data x _i denoted by symbol P1.

Being included in the neighborhood range centered on the failed data x _i can be said to be the vicinity of the failed data x _i . A set of successful data included in a neighborhood range centered on the failed data x _i constitutes a set X _near,i of nearby successful data. A set X _near,i of neighborhood success data may be expressed as a neighborhood success data list NN.

FIG. 5 is a diagram for explaining the processing of the neighborhood success data search unit 103 in the information processing apparatus 1 as an example of the embodiment, and shows information generated by the neighborhood success data search unit 103. FIG.

Hereinafter, in the drawings, each piece of test data (failure data, success data) may be represented by a unique integer.

In FIG. 5, symbol A illustrates an output and a correct label obtained by inputting failure data (test data) to the DNN model. Symbol B exemplifies an output and a correct label obtained by inputting successful data (test data) to the DNN model. The output of the DNN model is the output of the last stage of the hidden layer.

Symbol C exemplifies the distance d _b between each failure data (test data) indicated by symbol A and each successful data indicated by symbol B. FIG. Symbol D illustrates, for each failure data shown in symbol C, the set of successful data in the respective neighborhood.

The failure data narrowing unit 104 sorts the failure data sets X _neg in ascending order of the number |X _{niar, i |} get. In this way, a set of top k failure data with a small number of success data in the neighborhood range is represented as a failure data set X _k . A failed data set X _k may be referred to as a refined failed data list X _k .

Hereinafter, the failure data included in the failure data set X _neg may be referred to as failure data neg using the code neg. Also, the number |X _{niar, i | of successful data sets X niar} ,i in the vicinity of failure data x _i may be expressed as the number of neighboring successful data and represented by the code #NN[neg].

For example, in the example shown in FIG. 3, the number #NN[neg] of neighboring successful data (see symbol P2) for the failed data (see symbol P1) on the left side of the figure is 2, and the number of failed data (see symbol P3) on the right side of the figure is 2. reference), the number #NN[neg] of neighborhood success data (reference P4) is one.

FIG. 6 is a diagram for explaining processing of the failure data narrowing-down unit 104 of the information processing apparatus 1 as an example of the embodiment, and shows information generated by the failure data narrowing-down unit 104. FIG.

In FIG. 6, symbol A illustrates the set of successful data in the respective neighborhood for each failed data. The failed data narrowing unit 104 sorts the failed data in ascending order of the number of nearby successful data |X _niar,i |. Symbol B indicates an example in which the failure data indicated by symbol A are sorted in ascending order of the number of neighboring successful data sets.

Symbol C illustrates a failed data set X _k (restricted failed data list X _k ) created by extracting the top k sorted results shown in symbol B. FIG.

That is, the failure data narrowing unit 104 preferentially selects the highest number of failure data (k pieces) from among the plurality of failure data, in order of the failure data with the smallest number of success data within the neighborhood radius (cd _a ) from the failure data. data is determined as narrowed-down failure data. The plurality of failure data is an example of a third plurality of data, and the success data within a neighborhood radius (cd _a ) from the failure data is an example of a fifth plurality of data. The narrowed-down failure data is an example of the fourth plurality of data.

In this way, the failed data narrowing down unit 104 selects failed data that has been narrowed down from among a plurality of failed data based on the distance d _a from the classification boundary and the data-to-data distance d _b between the failed data and the successful data. do.

The success data narrowing unit 105 obtains a set of near success data X _near,i (a near success data list NN) for each of the failure data neg contained in the narrowed down failure data list X _k , thereby obtaining a set of success data Get X _near . This success data set X _near may be referred to as a narrowed success data set X _near or a narrowed success data list X _near . The successful data included in the narrowed-down successful data list _Xnear may be referred to as the narrowed-down successful data. The narrowed down success data is an example of the fifth plurality of data.

Success data narrowing unit 105 obtains the union of sets of success data near each failure data in failure data set X _k (filtered failure data list X _k ) created by failure data narrowing unit 104, thereby Create a set of data X _near . This success data set X _near may be represented by the following equation (3).

FIG. 7 is a diagram for explaining processing of the successful data narrowing unit 105 of the information processing apparatus 1 as an example of the embodiment, and shows information generated by the successful data narrowing unit 105. FIG.

In FIG. 7 , reference A indicates an example of a failed data set X _k (refined failed data list X _k ) created by the failed data narrowing unit 104 . Symbol B exemplifies a narrowed success data list X _near created based on the failure data set X _k shown in symbol A (narrowed down failure data list X _k ).

Success data in a range (neighborhood range) of radius cd _a centered on each failure data in the narrowed down failure data list X _k may be referred to as narrowed down success data. The radius cd _a may also be referred to as the neighborhood radius.

The successful data narrowing unit 105 determines the union of successful data in the vicinity of each failed data in the narrowed down failed data list X _k as an element of the narrowed down successful data list X _near .

In this way, the successful data narrowing unit 105 selects the narrowed successful data from among the plurality of successful data based on the distance d _a from the classification boundary and the inter-data distance d _b between the failure data and the successful data. do.

The weighted impact measurer 106 calculates the forward impact of failure data and the forward impact of successful data using narrowed down failure data X _k and narrowed down success data list X _near . The forward influence expresses the influence on the weight for the output in the forward direction processing of the DNN model. The forward influence represents the degree of influence of the value (forward propagation) obtained by multiplying the output of each layer of the DNN model by the weight connected to the next layer.

The weight influence measurement unit 106 inputs the failure data set X _k to the DNN model M (post-training model), and calculates the forward influence fwd _neg for each layer of the DNN model M. FIG.

The weight influence measurement unit 106 calculates the i-th value o _i ^(l-1) of the output in the layer l-1 of the DNN model M and the j-th row i-th column value W _ji ^(l) of the weight of the layer l. and the product o _i ^(l−1) W _ji ^(l) as the forward influence fwd _neg .

Weight influence measurement section 106 then creates list I ^(l) _{fwd_neg} in which the indices of weights W _ji ^(l) with large forward influence fwd _neg are arranged in order.

FIG. 8 is a diagram for explaining the processing by the weighted influence measurement unit 106 of the information processing apparatus 1 as an example of the embodiment, and is a diagram illustrating the calculation process of the forward influence of failure data.

Reference character A in FIG. 8 illustrates failed data in the narrowed-down failed data list X _k created by the failed data narrowing unit 104 . Symbol B exemplifies the weight for each layer as information of the DNN model (post-trained model) M .

The weighted influence measurement unit 106 inputs the failure data of the narrowed down failure data list X _k to such a DNN model M, and uses the output for calculating the forward influence fwd _neg described above.

In FIG. 8, symbol C indicates forward impact information about failed data. The forward influence information exemplified in FIG. 8 comprises the forward influence (the influence of the weight on the output) and the number of weights for each layer for each layer of failed data.

^The weight influence _measurement unit 106 inputs the failure _data ⁱⁿ the narrowed down failure data list X _k to the DNN model M, and calculates the code C Obtain the forward impact of the failure data exemplified in .

The influence of the weight on the output is calculated for each layer in the DNN model, and calculated for each value of the weight of the same layer. In the example shown in FIG. 8, for example, for the weights of layer 1 [0.054, 0.141, . calculated. Further, hereinafter, forward influence information about failure data may be referred to as forward influence information 210 by attaching a code 210 .

Weight influence measurement section 106 also generates an index list in which the weights are rearranged in descending order of the influence of the weight on the output for each layer.

In FIG. 8, symbol D indicates an index list in which indexes are sorted in order of forward influence.

Hereinafter, an index list obtained by sorting the forward impact information about failure data in descending order of impact may be referred to as a forward impact order index list 211 .

The forward impact order index list 211 indicated by symbol D indicates an index list sorted in the forward impact information 210 indicated by symbol C in descending order of the influence of the weight on the output. This forward impact order index list 211 comprises a number of layers, forward impact order indexes (indexes) of failure data, and weights.

This forward influence order index list 211 is an example of a list I ^(l) _{fwd_neg} in which the indices of the weights W _ji ^(l) with large forward influence fwd _neg are arranged in order.

Then, the forward influence order index is sorted in descending order of the influence of the weight on the output of the forward influence information 210 indicated by symbol C, and the corresponding weight index instead of the influence value. is represented by

As a result, by referring to the forward impact order index list 211, it is possible to easily know, for each layer, the weight with which the failure data has a high impact on the output in the DNN model.

The forward influence fwd _neg is an example of a numerical value calculated during forward propagation when narrowed down failure data (fourth plurality of data) is input to the DNN model.

Further, the forward influence fwd _neg is obtained based on the product of the output of the layer included in the DNN model obtained by inputting the failure data (the third plurality of data) into the DNN model and the weight in the layer. It is an example of the first forward influence degree that is calculated.

The weight influence measurement unit 106 inputs the success data set X _near to the DNN model M (post-training model), and calculates the forward influence fwd _pos for each layer of the DNN model M.

The weight influence measurement unit 106 calculates the i-th value o _i ^(l-1) of the output in the layer l-1 of the DNN model M and the j-th row i-th column value W _ji ^(l) of the weight of the layer l. and calculate the product o _i ^(l-1) W _ji ^(l) as the forward influence fwd _pos .

Then, weight influence measuring section 106 creates list I ^(l) _{fwd_pos} in which the indices of weights W _ji ^(l) with large forward influence fwd _pos are arranged in order.

FIG. 9 is a diagram for explaining the processing by the weighted influence degree measuring unit 106 of the information processing apparatus 1 as an example of the embodiment, and is a diagram illustrating the calculation process of the forward influence degree of the success data.

In FIG. 9 , symbol A exemplifies successful data in the narrowed successful data list X _near created by the successful data narrowing unit 105 . Symbol B exemplifies the weight for each layer as information of the DNN model (post-trained model) M .

The weighted influence measurement unit 106 inputs the successful data of the narrowed down successful data list X _near to such a DNN model M, and uses the output for calculating the forward influence fwd _pos described above.

In FIG. 9, symbol C indicates forward influence information for successful data. The forward influence information exemplified in FIG. 9 includes forward influence for each layer of successful data (influence of weight on output) and the number of weights for each layer.

^The weight influence measurement unit ₁₀₆ inputs ^the success data of the narrowed down success data list X _near to the DNN model M, and calculates _the code C Obtain the forward impact of the success data illustrated in .

The influence of the weight on the output is calculated for each layer in the DNN model, and calculated for each value of the weight of the same layer. In the example shown in FIG. 9, for example, for the weights of layer 1 [0.054, 0.141, . calculated. Further, hereinafter, forward influence information about successful data may be referred to as forward influence information 220 by attaching a code 220 .

Weight influence measurement section 106 also generates an index list in which the weights are rearranged in descending order of the influence of the weight on the output for each layer.

In FIG. 9, symbol D indicates an index list in which indexes are sorted in order of forward influence.

Hereinafter, an index list obtained by sorting forward impact information about successful data in descending order of impact may be referred to as a forward impact order index list 221 .

The forward impact order index list 211 indicated by symbol D indicates an index list sorted in descending order of the influence of the weight on the output in the forward impact information 220 indicated by symbol C. FIG. This forward influence order index list 221 comprises a number of layers, forward influence order indexes (indexes) of successful data, and weights.

This forward influence order index list 221 is an example of a list I ^(l) _{fwd_pos} in which the indices of the weights W _ji ^(l) having the largest forward influence fwd _pos are arranged in order.

Then, the forward influence order index is sorted in descending order of the influence of the weight on the output of the forward influence information 220 indicated by symbol C, and an index of the corresponding weight instead of the influence value. is represented by

As a result, by referring to the forward influence order index list 221, it is possible to easily know, for each layer, the weight having a high influence of successful data on the output in the DNN model.

Further, the weighted influence measurement unit 104 uses the narrowed down failure data X _k and the narrowed down success data list X _near to calculate the backward influence of the failure data and the backward influence of the successful data. The backward influence expresses the influence on the weight for the output in the backward processing of the DNN model. The backward influence represents the loss gradient (backpropagation) obtained by differentiating the loss with the weight.

The weight influence measurement unit 106 differentiates the respective outputs of the narrowed down failure data X _k and the narrowed down successful data list X _near by the DNN model, thereby quantifying (measuring) the influence of the weight on the output. .

The forward influence fwd _pos is an example of a numerical value calculated during forward propagation when successful narrowing down data (fifth plurality of data) is input to the DNN model.

Further, the forward influence fwd _pos is obtained based on the product of the output of the layer included in the DNN model obtained by inputting the success data (the second plurality of data) into the DNN model and the weight in the layer. It is an example of the second forward influence degree that is calculated.

The weight influence measurement unit 106 inputs the failure data set X _k to the DNN model M (post-training model), and calculates the backward influence grad _neg for each layer of the DNN model M. FIG.

そして、重み影響度測定部１０６は、後方影響度grad_negの大きい重みのindexを順に並べたリストI^(l) _{grad_neg}を作成する。 The weight influence measurement unit 106 automatically differentiates the output of the DNN model using the weight to calculate the influence (influence) that each weight value has on the output. The weight influence degree measuring unit 106 calculates the backward influence degree grad _neg by automatically differentiating the output Y _k when the narrowed down failure data X _k is input to the DNN model with the weight W ^(l) in the layer l. . The degree of backward influence grad _neg in layer l may be represented by the following equation (4).

Then, weight influence measuring section 106 creates a list I ^(l) _{grad_neg} in which indices of weights having large backward influence grad _neg are arranged in order.

FIG. 10 is a diagram for explaining the processing by the weighted influence measurement unit 106 of the information processing apparatus 1 as an example of the embodiment, and is a diagram illustrating the calculation process of the backward influence of failure data.

In FIG. 10 , symbol A exemplifies failed data in the narrowed down failed data list X _k created by the failed data narrowing unit 104 . Symbol B exemplifies the weight for each layer as information of the DNN model (post-trained model) M .

The weighted influence measurement unit 106 inputs the failure data of the narrowed down failure data list X _k to such a DNN model M, and uses the output to calculate the backward influence grad _neg .

In FIG. 10, symbol C indicates backward impact information about failed data. The backward influence information shown in FIG. 10 includes the backward influence of failed data (the influence of weight on output) and the number of weights for each layer.

The influence of the weight on the output is calculated for each layer in the DNN model, and calculated for each value of the weight of the same layer. In the example shown in FIG. 10, for example, for the weights of layer 1 [0.054, 0.141, . calculated. Further, hereinafter, backward influence information about failure data may be referred to as backward influence information 230 by attaching a code 230 .

Weight influence measurement section 106 also generates an index list in which the weights are rearranged in descending order of the influence of the weight on the output for each layer.

In FIG. 10, symbol D indicates an index list in which indexes are sorted in order of backward influence.

Hereinafter, an index list obtained by sorting the backward impact information about failure data in descending order of impact may be referred to as a backward impact order index list 231 .

A backward impact order index list 231 indicated by symbol D indicates an index list sorted in descending order of the value of the impact of the weight on the output in the backward impact information 230 indicated by symbol C. FIG. This backward impact order index list 231 comprises a number of layers, backward impact order indexes (indexes) of failed data, and weights.

This backward impact order index list 231 is an example of a list I ^(l) _{grad_neg} in which the indices of the weights having the largest backward impact grad _neg are arranged in order.

Then, the backward influence degree order index rearranges the influence degree of the weight on the output of the backward influence degree information 230 indicated by symbol C in descending order of the value, and replaces the value of the influence degree with the index of the corresponding weight. is represented by

As a result, by referring to the backward influence order index list 231, it is possible to easily know, for each layer, the weight that has a high influence of failure data on the output in the DNN model.

The degree of backward influence grad _neg is an example of a numerical value calculated during backpropagation when narrowed down failure data (fourth plurality of data) is input to the DNN model.

Further, the backward influence degree grad _neg is an example of a first backward influence degree obtained by differentiating the output obtained by inputting the failure data (the third plurality of data) into the DNN model with the weight.
The weight influence measurement unit 106 inputs the successful data set X _near to the DNN model M, and calculates the backward influence grad _pos for each layer of the DNN model M.

The weight influence degree measuring unit 106 calculates the backward influence degree grad _pos by automatically differentiating the output Y _k when the narrowed successful data list X _near is input to the DNN model with the weight W ^(l) in the layer l. . Weighted influence degree measurement section 106 calculates the backward influence degree grad _pos in layer l using Equation (4) described above.

Weight influence measuring section 106 then creates a list I ^(l) _{grad_pos} in which the indices of the weights having the largest backward influence grad _pos are arranged in order.

FIG. 11 is a diagram for explaining the processing by the weight influence measuring unit 106 of the information processing apparatus 1 as an example of the embodiment, and is a diagram illustrating the calculation process of the backward influence of success data.

In FIG. 11 , reference A indicates successful data in the narrowed successful data list X _near created by the successful data narrowing unit 105 . Symbol B exemplifies the weight for each layer as information of the DNN model (post-trained model) M .

The weight influence measurement unit 106 inputs the success data of the narrowed down success data list X _near to such a DNN model M, and uses the output to calculate the backward influence grad _pos .

In FIG. 11, symbol C indicates backward influence information for successful data. The backward influence information shown in FIG. 11 includes, for each layer, the backward influence of successful data (the influence of the weight on the output) and the number of weights for each layer.

The influence of the weight on the output is calculated for each layer in the DNN model, and calculated for each value of the weight of the same layer. In the example shown in FIG. 11, for example, for the weights of layer 1 [0.054, 0.141, . calculated. Further, hereinafter, backward influence information about successful data may be referred to as backward influence information 240 by attaching a code 240 .

Weight influence measurement section 106 also generates an index list in which the weights are rearranged in descending order of the influence of the weight on the output for each layer.

In FIG. 11, symbol D indicates an index list in which indexes are sorted in order of backward influence.

Hereinafter, an index list obtained by sorting the backward impact information about successful data in descending order of the impact may be referred to as a backward impact order index list 241 in some cases.

The backward impact order index list 241 denoted by D indicates an index list sorted in descending order of the impact of the weight on the output in the backward impact information 240 denoted by C. FIG. This backward influence order index list 241 comprises a number of layers, backward influence order indexes (index) of successful data, and weights.

This backward influence order index list 241 is an example of the list I ^(l) _{grad_pos} in which the indices of the weights having the largest backward influence grad _pos are arranged in order.

Then, the backward influence degree order index rearranges the degree of influence of the weight on the output of the backward influence degree information 240 indicated by symbol C in descending order of the value, and replaces the value of the degree of influence with the index of the corresponding weight. is represented by

As a result, by referring to the backward influence order index list 241, it is possible to easily know, for each layer, the weight having a high influence of successful data on the output in the DNN model.

The degree of backward influence grad _pos is an example of a numerical value calculated during backpropagation when narrowed-down success data (fifth plurality of data) are input to the DNN model.

Also, the backward influence grad _pos is an example of a second backward influence obtained by differentiating the output obtained by inputting the successful data (the second plurality of data) into the DNN model with the weight.

The correction target weight specifying unit 107 determines the number of selected weights for each layer of the DNN model, and determines (specifies) the weights to be corrected.

Correction target weight identification section 107 determines the number of selection weights for each layer based on forward influence information 210 and 220 and backward influence information 230 and 240 .

FIG. 12 is a diagram for explaining the process of determining the number of selected weights by the correction target weight specifying unit 107 of the information processing apparatus 1 as an example of the embodiment.

In FIG. 12, symbol A indicates forward impact information 210 for failed data. Based on the forward influence information 210, the correction target weight specifying unit 107 calculates the average value of the forward influence (the influence of the weight on the output) of the failure data for each layer.

Furthermore, the correction target weight specifying unit 107 normalizes the calculated average value of the forward influence degree of the failure data for each layer so that the sum of all layers becomes one. Let e ^(l) _k,fwd be the mean forward influence of the normalized failure data.

In FIG. 12, symbol B indicates an example of the normalized average forward impact of failed data in each layer.

The correction target weight identification unit 107 stores the calculated average of the forward influence degrees of the normalized failure data for each layer in a predetermined storage area such as the storage device 13 .

Further, in FIG. 12, symbol C indicates forward influence information 220 for successful data. Based on the forward influence information 220, the correction target weight specifying unit 107 calculates the average value of the forward influence (the influence of the weight on the output) of the successful data for each layer.

Furthermore, the correction target weight specifying unit 107 normalizes the calculated average value of the forward influence degree of the successful data for each layer so that the sum of all the layers becomes 1. Let e ^(l) _near,fwd be the average forward influence of the normalized success data.

In FIG. 12, symbol D indicates an example of the normalized average forward impact of success data of each layer.

The correction target weight identification unit 107 stores the calculated average of the forward impact degrees of the normalized success data for each layer in a predetermined storage area such as the storage device 13 .

Also, in FIG. 12, symbol E indicates backward influence information 230 for failed data. Based on the backward influence information 230, the correction target weight specifying unit 107 calculates the average value of the backward influence (the influence of the weight on the output) of the failure data for each layer.

Furthermore, the correction target weight specifying unit 107 normalizes the calculated average value of the degree of backward influence of the failure data for each layer so that the sum of all layers becomes one. Let e ^(l) _k,grad be the average of the normalized failure data backward influences.

In FIG. 12, symbol F indicates an example of the normalized average degree of backward influence of failed data in each layer.

The correction target weight specifying unit 107 stores the calculated average of the backward influence degrees of the normalized failure data for each layer in a predetermined storage area such as the storage device 13 .

Also, in FIG. 12, symbol G indicates backward influence information 240 for successful data. Based on the backward influence information 240, the correction target weight specifying unit 107 calculates the average value of the backward influence (the influence of the weight on the output) of the successful data for each layer.

Furthermore, the correction target weight specifying unit 107 normalizes the calculated average value of the degree of backward influence of the successful data for each layer so that the sum of all layers becomes one. Let e ^(l) _near,grad be the mean of the normalized success data backward impact.

In FIG. 12, symbol H indicates an example of the normalized average degree of backward influence of successful data of each layer.

The correction target weight identifying unit 107 stores the calculated average degree of backward influence of the normalized success data for each layer in a predetermined storage area such as the storage device 13 .

The correction target weight specifying unit 107 calculates the normalized average of the forward impact of failure data for each layer, the average of the forward impact of successful data for each layer, and the average backward impact of failure data for each layer. And based on the average backward influence degree of successful data for each layer, the following equation (5) is used to calculate the suspicion value susp ^(l) for each layer.

In FIG. 12, J indicates an example of the normalized suspicion value of each layer.

The correction target weight identification unit 107 stores the calculated suspicion value susp ^(l) for each layer in a predetermined storage area such as the storage device 13 .

The correction target weight identification unit 107 determines the number of weights to be selected as correction targets in each layer based on the suspicion value susp ^(l) . Correction target weight identifying section 107 determines the number of selected weights for each layer using the following equation (6).

where |W ^(l) _n | is the number of weights for layer l, susp ^(l) is the suspicion value for layer l, and ρ is the reduction rate. The reduction rate ρ may be arbitrarily set by the user.

In FIG. 12, K indicates an example of the calculated number of selection weights for each layer.

Further, the correction target weight specifying unit 107 determines (specifies) the weight to be corrected for each layer.

13A and 13B are diagrams for explaining the correction target weight specifying process by the correction target weight specifying unit 107 of the information processing apparatus 1 as an example of the embodiment.

Based on the forward impact order index list 211 based on the failure data, the correction target weight specifying unit 107 selects the indexes corresponding to the number of selection weights from the top of the forward impact order indexes of the failure data for each layer. Hereinafter, the upper indexes corresponding to the number of selection weights (see symbol E in FIG. 13), which are selected from the forward impact index of the failed data, may be referred to as the forward impact high index of the failed data. The forward impact upper index of failed data is a set of weights that have a large impact on the output. Symbol A in FIG. 13 shows an example of the forward influence high index of failure data.

The forward impact high-order index of failure data is a set I (l) ^' _{fwd_neg} obtained by selecting c ^(l) weights from the top of the forward impact order index list 211 (list I ^(l) _{fwd_neg} ).

The correction target weight specifying unit 107 stores the forward impact high index of the selected failure data in a predetermined storage area such as the storage device 13 .

Moreover, based on the forward impact order index list 221 based on the successful data, the correction target weight specifying unit 107 selects the indexes corresponding to the number of selection weights from the top of the forward impact order indexes of the successful data for each layer. Hereinafter, the upper indexes corresponding to the number of selection weights (see symbol E in FIG. 13), which are selected from the forward impact order indexes of successful data, may be referred to as forward impact higher indexes of successful data. The forward influence upper index of success data is a set of weights having a large influence on the output. Symbol B in FIG. 13 shows an example of the forward impact high-order index of the success data.

The forward influence upper index of the success data is a set I (l) ^' _{fwd_pos} in which c ^(l) weights are selected from the top of the forward influence order index list 221 (list I ^(l) _{fwd_pos} ).

The correction target weight identification unit 107 stores the forward impact high index of the selected success data in a predetermined storage area such as the storage device 13 .

Based on the backward impact order index list 231 based on the failure data, the correction target weight specifying unit 107 selects the indexes corresponding to the number of selected weights from the top of the backward impact order indexes of the failure data for each layer. Hereinafter, the upper indexes corresponding to the number of selection weights (see symbol E in FIG. 13), which are selected from the backward impact index of the failed data, may be referred to as the backward impact high index of the failed data. The backward impact high index of failed data is a set of weights that have a large impact on the output. Reference symbol C in FIG. 13 shows an example of the forward influence high index of failure data.

The backward impact high-order index of failure data is a set I (l)′ _{grad_neg} of c ^(l) weights selected from the top of the backward impact order index list 231 (list I ^{(l) grad_neg} ₎ .

The correction target weight specifying unit 107 stores the backward impact high-order index of the selected failure data in a predetermined storage area such as the storage device 13 .

Based on the backward impact order index list 241 based on the successful data, the correction target weight specifying unit 107 selects the indexes corresponding to the number of selected weights from the highest backward impact order indexes of the successful data for each layer. Hereinafter, the upper indices corresponding to the number of selection weights (see symbol E in FIG. 13), which are selected from the backward influence index of the successful data, may be referred to as the backward influence high index of the successful data. The backward influence degree upper index of the success data is a set of weights having a large influence of the weights on the output. Symbol D in FIG. 13 shows an example of the backward impact high order index of the success data.

The backward influence high index of the success data is a set I (l)′ _{grad_pos} of c ^(l) weights selected from the top of the backward influence order index list 241 (list I ^{(l) grad_pos} ₎ .

The correction target weight specifying unit 107 stores the backward impact high-order index of the selected successful data in a predetermined storage area such as the storage device 13 .

Then, the correction target weight identifying unit 107 determines (identifies) the weights that do not affect the success data X _near among the weights that affect the failure data X _k as the correction target weights. The correction target weight identification unit 107 selects, as correction target weights, weights that are included in the high-ranking degree of influence on failure data and are not included in the high-ranking degree of influence on successful data in each layer that constitutes the DNN model M. .

Specifically, in each layer that configures the DNN model M, the correction target weight specifying unit 107 sets the forward influence index of failure data (set I ^(l)′ _{fwd_neg} ) and the backward influence index of failure data ( For the intersection (∩) with the set I ^(l)' _{grad_neg} ), the forward influence index of successful data (set I ^(l)' _{fwd_pos} ) and the backward influence index of successful data (set I ^{(l )'} _{grad_pos} ) and the difference set (\) of the product set (∩) to specify the correction target weight (W ^(l) _localize ).

That is, the correction target weight identification unit 107 determines, as correction target weights, weights that satisfy the following equation (7) in each layer that configures the DNN model M.

The intersection (∩) of the upper index of the forward influence of the failed data (set I ^(l)' _{fwd_neg} ) and the upper index of the backward influence of the failed data (set I ^(l)' _{grad_neg} ) is _the A set of weights with high impact (high impact). On the other hand, the intersection (∩) of the upper index of forward influence of successful data (set I ^(l)' _{fwd_pos} ) and the upper index of backward influence of successful data (set I ^(l)' _{grad_pos} ) is the success data X _near is a set of weights that have a high impact on (high impact).

Therefore, in the above formula (7), among the set of weights to be corrected (W ^(l) _localize ) that have a high (higher order) influence on the failure data X _k , the influence on the success data X _near is Indicates to select a set of small weights.

Here, let m be the number of weights determined according to susp ^(l) in layer l. _{The correction} target weight identification unit 107 selects ^the subsets I ^{( l )} _{′ fwd_neg} ^, _I ⁽ For ^l)' _{grad_neg} , I ^(l)' _{fwd_pos} , I ^(l)' _{grad_pos} (I ^(l)' _{fwd_neg} ∩ I ^(l)' _{grad_neg} ) \ (I ^(l)' _{fwd_pos} ∩ ^{I (l)'} _{grad_pos} ) and let it be the index I ^(l) _localize of the weight to be modified.

Note that the correction target weight W ^(l) _localize of the layer l can be expressed by the following formula (7′), and the correction target weight W _localize of all layers can be expressed by the following formula (7″). .

FIG. 14 is a diagram illustrating a method of selecting correction target weights by the correction target weight specifying unit 107 of the information processing apparatus 1 as an example of the embodiment.

In FIG. 14, symbol A indicates the "upper index of forward influence of failed data (set I ^(l)' _{fwd_neg} )", and symbol B indicates the upper index of backward influence of failed data (set I ^(l)' _{grad_neg} )” is exemplified.

The weight of the intersection (∩) of these "upper index of forward influence of failure data (set I ^(l)' _{fwd_neg} )" and "upper index of backward influence of failure data (set I ^(l)' _{grad_neg} )" The elements are {77, 11, 425, 572}.

Further, in FIG. 14, symbol C indicates "upper index of forward influence of successful data (set I ^(l)' _{fwd_pos} )", and symbol D indicates "upper index of backward influence of successful data (set I ^{(l) '} _{grad_pos} )'.

The weight of the intersection (∩) of these "upper index of forward influence of successful data (set I ^(l)' _{fwd_pos} )" and "upper index of backward influence of successful data (set I ^(l)' _{grad_pos} )" The elements are {287, 11, 425}.

The difference set (\) between the weighting element set {77, 11, 425, 572} and the weighting element set {287, 11, 425} is the weighting element {77, 572}. In such a case, correction target weight identifying section 107 selects weight element {77, 572} as a correction target weight.

In this way, the correction target weight identification unit 107 determines that the numerical values calculated during forward propagation and the numerical values calculated during back propagation when the narrowed-down failed data and the narrowed-down successful data are input to the DNN model. Based on this, among the plurality of learning parameters (the first plurality of parameters) included in the DNN model, the correction target weights (the second plurality of parameters) are specified.

The correction target weight identification unit 107 stores the correction target weight W ^(l) _localize represented by the above equation (7′) in a predetermined storage area such as the storage device 13 . Then, correction target weight identification section 107 outputs the correction target weight W _localize represented by the above equation (7″).

Weight correction may be performed on W _localize output in this way, for example, using PSO (Particle swarm optimization). Note that the fitness may be obtained using the following equation (8).

Note that N _patched indicates the number of modified failed test data, and N _intact indicates the number of successful test data that remains unchanged. PSO is a known method, and detailed description thereof will be omitted.

(B) Operation Next, the outline of the processing in the information processing apparatus 1 configured as described above as an example of the embodiment will be described with reference to the flowchart (steps A1 to A8) shown in FIG.

At step A<b>1 , test data and a trained DNN model are input to the information processing device 1 .

At step A2, the test data classification unit 102 executes the test data classification process. The details of the test data classification processing by the test data classification unit 102 will be described later using the flowchart shown in FIG.

At step A3, the neighboring successful data search unit 103 executes a search process for successful data in the vicinity of the failed data. The details of the process of searching for successful data in the vicinity of failed data by the neighboring successful data search unit 103 will be described later with reference to the flowchart shown in FIG.

At step A4, the failed data narrowing unit 104 executes a process of creating the failed data set X _k (the narrowed down failed data list X _k ). The details of the failure data set X _k (restricted failure data list X _k ) creation processing by the failure data narrowing unit 104 will be described later with reference to the flowchart shown in FIG. 18 .

At step A5, the success data narrowing down unit 105 executes the process of creating the success data set X _near (the narrowed down success data list X _near ). The details of the process of creating the success data set X _near (the narrowed success data list X _near ) by the success data narrowing unit 105 will be described later with reference to the flowchart shown in FIG. 19 .

At step A6, the weighted impact measurer 106 calculates the forward impact of failure data and the forward impact of successful data. The details of the calculation process of the forward influence of failure data and the forward influence of successful data by the weighted influence measuring unit 106 will be described later with reference to the flowchart shown in FIG.

In step A7, the correction target weight specifying unit 107 determines the number of selected weights for each layer of the DNN model, and determines (specifies) the weights to be corrected. Details of the process of determining the number of selected weights and weights to be corrected by the correction target weight identification unit 107 will be described later with reference to the flowchart shown in FIG.

After that, in step A8, the weight to be modified is output, and the process is terminated.

Next, the processing of the test data classification unit 102 of the information processing device 1 as an example of the embodiment will be described according to the flowchart (steps B1 to B9) shown in FIG.

At step B1, the information processing apparatus 1 receives the test data list, the correct label of the test data, and the trained DNN model M. FIG.

In step B2, the test data classification unit 102 secures storage destinations of the failure data list X _neg and the success data list X _pos in the memory 12 (see FIG. 25).

At step B3, a loop process is started to repeat the control up to step B7 for all test data (input data) existing in the test data list.

In step B4, the test data classification unit 102 inputs the test data to the trained DNN model M and calculates estimated labels.

At step B5, the test data classification unit 102 confirms whether the estimated label output from the DNN model M matches the correct label of the test data.

As a result of confirmation, if the estimated label and the correct label match (YES in step B5), the process proceeds to step B6. At step B6, the test data classification unit 102 adds the test data to the successful data list X _pos .

If the result of confirmation in step B5 is that the estimated label and the correct label do not match (NO in step B5), the process proceeds to step B7. At step B7, the test data classification unit 102 adds test data to the failure data list Xneg.

Control then proceeds to step B8. At step B8, loop end processing corresponding to step B3 is performed. Here, when the processing for all test data is completed, the control proceeds to step B9.

At step B9, the failure data list X _neg and the success data list X _pos are output. After that, the process ends.

Next, processing of the neighborhood success data searching unit 103 of the information processing apparatus 1 as an example of the embodiment will be described according to the flowchart (steps C1 to C15) shown in FIG.

At step C1, the failed data list X _neg , the successful data list X _pos and the trained DNN model M are input.

In step C2, the neighborhood success data search unit 103 secures a storage destination of the neighborhood success data list NN for failure data on the memory 12. FIG.

In step C3, a loop process is started in which the processes up to step C13 are repeated for all the failed data neg (negεX _neg ) included in the failed data list X _neg .

In step C4, the neighborhood success data searching unit 103 inputs the failure data neg to the DNN model M, and calculates the output y _neg of the last layer.

In step C5, the neighborhood success data searching unit 103 calculates the distance of the failure data neg to the classification boundary layer using the above equation (1).

In step C6, the near success data search unit 103 secures a storage destination area for the near success data list X _near,nag for the failure data neg on the memory 12. FIG.

At step C7, a loop process is started to repeat the processes up to step C11 for all the successful data pos (posεX _pos ) included in the successful data list X _pos .

In step C8, the neighborhood success data searching unit 103 inputs the success data pos to the DNN model M, and calculates the output y _pos of the last layer.

At step C9, the neighboring successful data searching unit 103 calculates the distance from the failure data neg to the successful data pos using the above equation (2).

In step C10, the neighborhood success data searching unit 103 checks whether the neighborhood coefficient c satisfies the condition d _b >c×d _a . That is, the neighborhood success data searching unit 103 checks whether the success data pos is within the neighborhood range of the failure data neg.

If the condition d _b >c×d _a is satisfied (YES in step C10), the process proceeds to step C11.

In step C11, the near success data searching unit 103 adds the success data pos to the near success data list X _{near, neg} of the failure data neg.

As a result of confirmation in step C10, if the condition d _b >c×d _a is not satisfied (NO in step C10), the process proceeds to step C12.

At step C12, loop end processing corresponding to step C7 is performed. Here, when the processing for all successful data pos in all successful data lists X _pos is completed, the process proceeds to step C13.

In step C13, the neighborhood success data search unit 103 adds the neighborhood success data list X _{near, nag} for failure data neg to the neighborhood success data list NN[neg] for failure data neg.

Thereafter, in step C14, loop end processing corresponding to step C3 is performed. Here, when the processing for all failed data neg included in the failed data list X _neg is completed, the process proceeds to step C15.

At step C15, the neighborhood success data list NN is output, and then the process is terminated.

Next, processing by the failed data narrowing unit 104 of the information processing apparatus 1 as an example of the embodiment will be described according to the flowchart (steps D1 to D5) shown in FIG.

At step D1, a failed data list X _neg and a neighborhood successful data list NN are input.

In step D2, the failed data narrowing unit 104 secures a storage area of the narrowed failed data list _Xk in a storage area such as the memory 12 or the like.

In step D3, the failure data narrowing unit 104 sorts the failure data list X _neg in ascending order according to the number of neighboring successful data (NN[neg]) for each failure data neg.

In step D4, the failure data narrowing down unit 104 acquires the top k failure data in the sorted failure data list X _neg and stores them in the narrowed down failure data list X _k .

At step D5, the narrowed-down failure data list _Xk is output, and then the process is terminated.

Next, the processing of the successful data narrowing down unit 105 of the information processing apparatus 1 as an example of the embodiment will be described according to the flowchart (steps E1 to E6) shown in FIG.

At step E1, a narrowed failed data list X _k and a neighboring successful data list NN are input.

In step E2, the successful data narrowing unit 105 secures a storage area of the narrowed successful data list X _near in a storage area such as the memory 12 or the like.

At step E3, loop processing is started to repeat the processing at step E4 for all failure data neg ( _negεXk ) included in the narrowed down failure data list _Xk .

In step E4, the success data narrowing unit 105 adds the near success data list NN[neg] of the failure data neg to the narrowed success data list X _near so that there is no duplication of data.

At step E5, loop end processing corresponding to step E3 is performed (end of the first loop). Here, when the processing for all failure data neg (negεX _k ) included in the narrowed-down failure data list X _k is completed, the process proceeds to step E6.

At step E6, the narrowed-down success data list _Xnear is output, and the process ends.

Next, the processing of the weighted influence measuring unit 106 of the information processing apparatus 1 as an example of the embodiment will be described according to the flowchart (steps F1 to F4) shown in FIG.

In step F1, the narrowed down failure data list X _k , the narrowed down success data list X _near and the trained DNN model M are input.

In step F2, the weighted influence measurement unit 106 inputs the narrowed down failure data list X _k to the DNN model M. FIG. The weight influence measurer 106 calculates the forward influence fwd _neg of each weight and the backward influence grad _neg of each weight, and stores them in a storage area such as the memory 12 .

In step F3, the weighted influence measurement unit 106 inputs the narrowed-down success data list X _near to the DNN model M. FIG. The weight influence measurer 106 calculates the forward influence fwd _pos of each weight and the backward influence grad _pos of each weight, and stores them in a storage area such as the memory 12 .

In step F4, the forward influence fwd _neg of each weight on the extracted failure data X _k , the backward influence grad _neg of each weight on the extracted failure data X _k , the forward influence of each weight on the extracted successful data X _near The influence fwd _pos and the backward influence grad _pos of each weight for the extracted successful data X _near are output. After that, the process ends.

Next, the processing of the correction target weight identification unit 107 of the information processing apparatus 1 as an example of the embodiment will be described according to the flowchart (steps G1 to G8) shown in FIG.

In step G1, the forward influence fwd _neg of each weight on the extracted failure data X _k , the backward influence grad _neg of each weight on the extracted failure data X _k , the forward influence of each weight on the extracted successful data X _near The influence fwd _pos and the backward influence grad _pos of each weight on the extracted success data X _near and the trained DNN model M are input.

In step G2, modification target weight identifying section 107 determines the number of weights (selected weight number) m to be selected in each layer using the degree of influence of each weight (fwd _neg , grad _neg , fwd _pos , grad _pos ). do.

In step G3, the correction target weight specifying unit 107 sorts the weights of the DNN model M in descending order with respect to the forward influence fwd _neg to obtain a set of top m weights W _f,n . This weight set W _f,n is an example of a set I (l) ^′ _{fwd_neg} (forward impact high order index of failure data) selected from the top of the forward impact order index list 211 (list I ^(l) _{fwd_neg} ). .

In step G4, the correction target weight identification unit 107 sorts the weights of the DNN model M in descending order with respect to the degree of backward influence grad _neg to obtain a weight set W _g,n of top m weights. This weight set W _g,n is an example of a set I (l) ^′ _{grad_neg} (upper index of backward impact of failed data) selected from the top of the backward impact index list 231 (list I ^(l) _{grad_neg} ). .

In step G5, the correction target weight identification unit 107 sorts the weights of the DNN model M in descending order with respect to the forward influence fwd _pos , and obtains the top m weight set W _f,p . This weight set W _f,p is an example of a set I ^(l)' _{fwd_pos} (forward influence high order index of successful data) selected from the top of the forward influence order index list 221 (list I ^(l) _{fwd_pos} ). .

In step G6, the correction target weight identification unit 107 sorts the weights of the DNN model M in descending order with respect to the degree of backward influence grad _pos , and obtains the top m weight set W _g,p . This weight set W _g,p is an example of the set I (l ^)′ _{grad_pos} (upper index of backward influence of successful data) selected from the top of the backward influence order index list 241 (list I ^(l) _{grad_pos} ). .

In step G7, the correction target weight specifying unit 107 calculates the correction target weight W _localized using the following equation (9), and stores it in a storage area such as the memory 12 or the like.
W _localized = (W _f,n ∩ W _{g, n} ) ＼(W _{f, p} ∩ _{W g, p} ) (9)

At step G8, the weight W _localized to be modified is output, and the process ends.

Then, the correction target weight W _localize generated in this manner is subjected to weight correction using, for example, PSO.

(C) Effect of an Embodiment As _described above, according to the information processing apparatus 1 as an example of the embodiment, the failure data narrowing unit 104 selects, among a plurality of failure data, The top k pieces of data that are preferentially selected from failure data with a small number of success data are determined as narrowed-down failure data.

Also, the success data narrowing unit 105 selects the success data included in the vicinity range of the narrowed-down failure data as the narrowed-down success data.

The weighted influence measurement unit 106 calculates weighted influences (fwd _neg , grad _neg , fwd _pos , grad _pos ) based on these narrowed down failure data and narrowed down success data. Then, the correction target weight specifying unit 107 selects, as correction target weights, a set of weights that have a small effect on the success data from among the sets of weights that have a high (higher order) effect on the failure data.

As a result, when modifying the DNN model and updating the modification target weights selected as described above, the occurrence of degradation can be suppressed, and the accuracy of the DNN model can be improved.

FIG. 22 is a diagram for explaining the relationship between DNN model correction and degradation. Modifying the DNN model changes the classification boundary (decision boundary). As a result, data (successful data) inferred correctly in the DNN model before correction may not be inferred correctly in the DNN model after correction, that is, so-called degradation may occur. Such degradation often occurs in successful data near the failed data to be corrected.

The following points 1 and 2 can be seen in the occurrence of degradation accompanying modification of the DNN model.

Point 1: Failed data with a small number of nearby successful data are less likely to be degraded due to changes in the classification boundary accompanying modification of the DNN model.

Point 2: When correcting failed data far from the classification boundary, the change in the classification boundary is large, and degradation is likely to occur.

FIG. 23 is a diagram for explaining correction target weights determined in the information processing apparatus 1 as an example of the embodiment.

In the information processing apparatus 1, based on the above point 1, failure data that are not adjacent to success data are preferentially used for selection of weights to be corrected (see symbol P1). As a result, it is possible to reduce the success data affected by the modification of the DNN model, and to suppress the occurrence of degradation.

Further, based on Point 2, the neighborhood area set for failure data is widened by increasing the neighborhood radius as the failure data is further away from the classification boundary (see symbol P2). This makes it easier for successful data to be included in the neighboring area of failed data far from the classification boundary. As a result, failure data far from the classification boundary is less likely to be used for selection of correction target weights, and the occurrence of degradation can be suppressed. can.

The successful data narrowing unit 105 focuses on the failed data included in the narrowed down failed data among the plurality of successful data, and determines the neighborhood radius (cd _a ) determined by the distance d _a from the classification boundary and the neighborhood coefficient c. Data within the range are determined as narrowed-down successful data.

This makes it possible to easily identify successful data that may be degraded by correcting failed data.

The failure data narrowing-down unit 104 narrows down the top plurality (k pieces) of data preferentially selected from among the plurality of failure data, in descending order of the number of successful data within the neighborhood radius (cd _a ). Determined as failed data. As a result, it is possible to use the failure data, which is less likely to cause degradation to nearby success data due to the correction, to select weights to be corrected, thereby reducing the success data affected by the correction of the DNN model and causing the occurrence of degradation. can be deterred.

FIG. 24 is a diagram illustrating simulation results in the information processing apparatus 1 as an example of the embodiment.

FIG. 24 compares the correction target edge determination method by the information processing apparatus 1 with the conventional method (Arachne) using images of traffic signs of GTSRB (The German Traffic Sign Recognition Benchmark) as target data.

As shown in FIG. 24, according to the machine learning method by the information processing apparatus 1, it can be seen that the BreakRate is successfully suppressed on a sufficient RepairRate.

(D) Hardware Configuration Example FIG. 25 is a diagram illustrating the hardware configuration of the information processing apparatus 1 as an example of the embodiment.

The information processing device 1 is an example of a computer, and includes, for example, a processor 11, a memory 12, a storage device 13, a graphic processing device 14, an input interface 15, an optical drive device 16, a device connection interface 17, and a network interface 18. have as These components 11 to 18 are configured to communicate with each other via a bus 19 .

The processor 11 controls the information processing apparatus 1 as a whole. Processor 11 may be a multiprocessor. The processor 11 is, for example, a CPU (Central Processing Unit), an MPU (Micro Processing Unit), a DSP (Digital Signal Processor), an ASIC (Application Specific Integrated Circuit), a PLD (Programmable Logic Device), or an FPGA (Field Programmable Gate Array). or one. Also, the processor 11 may be a combination of two or more types of elements among CPU, MPU, DSP, ASIC, PLD, and FPGA. The processor 11 may be a GPU (Graphics Processing Unit).

Then, the processor 11 executes a control program (machine learning program: not shown) for the information processing apparatus 1 to perform the model training unit 101, the test data classification unit 102, and the neighborhood success data search unit 103 illustrated in FIG. , a failure data narrowing unit 104, a successful data narrowing unit 105, a weight influence degree measuring unit 106, and a correction target weight specifying unit 107, as a control unit.

Note that the information processing apparatus 1 may realize the function of the control unit by executing a program (machine learning program, OS program) recorded in a non-temporary computer-readable recording medium, for example.

A program describing the contents of processing to be executed by the information processing apparatus 1 can be recorded in various recording media. For example, a program to be executed by the information processing device 1 can be stored in the storage device 13 . The processor 11 loads at least part of the program in the storage device 13 into the memory 12 and executes the loaded program.

Also, the program to be executed by the information processing device 1 (processor 11) can be recorded in a non-temporary portable recording medium such as the optical disk 16a, the memory device 17a, the memory card 17c, or the like. A program stored in a portable recording medium becomes executable after being installed in the storage device 13 under the control of the processor 11, for example. Alternatively, the processor 11 can read and execute the program directly from the portable recording medium.

The memory 12 is a storage memory including ROM (Read Only Memory) and RAM (Random Access Memory). A RAM of the memory 12 is used as a main storage device of the information processing apparatus 1 . At least part of the program to be executed by the processor 11 is temporarily stored in the RAM. In addition, the memory 12 stores various data used for processing by the processor 11 .

The storage device 13 is a storage device such as a hard disk drive (HDD), SSD (Solid State Drive), storage class memory (SCM), etc., and stores various data. The storage device 13 is used as an auxiliary storage device for the information processing device 1 .

The storage device 13 stores an OS program, a control program, and various data. A control program includes a machine learning program.

A semiconductor storage device such as an SCM or flash memory can also be used as the auxiliary storage device. Alternatively, a plurality of storage devices 13 may be used to configure RAID (Redundant Arrays of Inexpensive Disks).

The storage device 13 may store information of the DNN model M, and may also store test data (failure data, success data). Further, the storage device 13 may store at least part of the neighborhood success data, narrowed-down failure data, narrowed-down success data, and each degree of influence. The storage device 13 may also store the number of selected weights for each layer determined by the weight influence measurer 106 .

A monitor 14 a is connected to the graphics processing unit 14 . The graphics processing unit 14 displays an image on the screen of the monitor 14a in accordance with instructions from the processor 11. FIG. Examples of the monitor 14a include a display device using a CRT (Cathode Ray Tube), a liquid crystal display device, and the like.

A keyboard 15 a and a mouse 15 b are connected to the input interface 15 . The input interface 15 transmits signals sent from the keyboard 15 a and the mouse 15 b to the processor 11 . Note that the mouse 15b is an example of a pointing device, and other pointing devices can also be used. Other pointing devices include touch panels, tablets, touch pads, trackballs, and the like.

The optical drive device 16 uses laser light or the like to read data recorded on the optical disk 16a. The optical disc 16a is a portable, non-temporary recording medium on which data is recorded so as to be readable by light reflection. The optical disk 16a includes DVD (Digital Versatile Disc), DVD-RAM, CD-ROM (Compact Disc Read Only Memory), CD-R (Recordable)/RW (ReWritable), and the like.

The device connection interface 17 is a communication interface for connecting peripheral devices to the information processing apparatus 1 . For example, the device connection interface 17 can be connected with a memory device 17a and a memory reader/writer 17b. The memory device 17a is a non-temporary recording medium equipped with a communication function with the device connection interface 17, such as a USB (Universal Serial Bus) memory. The memory reader/writer 17b writes data to the memory card 17c or reads data from the memory card 17c. The memory card 17c is a card-type non-temporary recording medium.

A network interface 18 is connected to a network. A network interface 18 transmits and receives data via a network. Other information processing devices, communication devices, and the like may be connected to the network.

In addition, regardless of the embodiment described above, various modifications can be made without departing from the gist of the present embodiment.

For example, in the above-described embodiment, an example of application to DNN is shown, but the present invention is not limited to this and may be applied to NN.

Moreover, the present embodiment can be implemented and manufactured by those skilled in the art based on the above disclosure.

(E) Supplementary Note Regarding the above embodiment, the following Supplementary Note will be disclosed.

(Appendix 1)
Among the first plurality of data, a second plurality of data and a third plurality of data having incorrect prediction results according to the output value of the first machine learning model for each of the first plurality of data and
the difference between the output value of the first machine learning model for the data included in the third plurality of data and the correct label value corresponding to the data included in the third plurality of data; Based on the difference between the output value of the first machine learning model for each of the plurality of data and the output value of the first machine learning model for each of the second plurality of data, the third selecting a fourth plurality of data of the plurality of data and a fifth plurality of data of the second plurality of data related to the fourth plurality of data;
Numerical values calculated during forward propagation and numerical values calculated during backpropagation when the fourth plurality of data and the fifth plurality of data are input to the first machine learning model Based on, identifying a second plurality of parameters among the first plurality of parameters included in the first machine learning model,
generating a second machine learning model by updating only the second plurality of parameters of the first plurality of parameters;
A machine learning program that makes a computer perform a process.

(Appendix 2)
In the process of selecting the fifth plurality of data, centering on the data of the fourth plurality of data among the second plurality of data, between the output values of the first machine learning model for the data A process of determining data within the range of the neighborhood radius determined by the difference and the neighborhood coefficient as the fifth plurality of data,
The machine learning program according to Appendix 1.

(Appendix 3)
In the process of selecting the fourth plurality of data, among the third plurality of data, the fifth plurality of data within the range of the neighborhood radius is preferentially selected in descending order of priority. A process of determining a plurality of data as the fourth plurality of data,
The machine learning program according to Appendix 2.

(Appendix 4)
In the process of identifying the second plurality of parameters, the degree of influence on the second plurality of data is selected from a first parameter group in which parameters having a high degree of influence on the third plurality of data are preferentially selected. A process of identifying a third parameter group obtained by subtracting a second parameter group in which a parameter with a smaller value is preferentially selected as the second plurality of parameters,
The machine learning program according to any one of Appendices 1 to 3.

(Appendix 5)
The degree of influence on the third plurality of data is
The first obtained based on the product of the output of the layer included in the first machine learning model obtained by inputting the third plurality of data to the first machine learning model and the weight in the layer 5. The machine learning program of Clause 4, including forward influence.

(Appendix 6)
The degree of influence on the third plurality of data is
A second obtained based on the product of the output of the layer included in the first machine learning model obtained by inputting the second plurality of data to the first machine learning model and the weight in the layer 6. The machine learning program of Appendix 4 or Appendix 5 including forward impact.

(Appendix 7)
The degree of influence on the third plurality of data is
Any one of appendices 4 to 6, including a first backward influence degree obtained by differentiating the output obtained by inputting the third plurality of data into the first machine learning model with a weight. machine learning program.

(Appendix 8)
The degree of influence on the third plurality of data is
Any one of appendices 4 to 7, including a second backward influence degree obtained by differentiating the output obtained by inputting the second plurality of data into the first machine learning model with a weight. machine learning program.

(Appendix 9)
Among the first plurality of data, a second plurality of data and a third plurality of data having incorrect prediction results according to the output value of the first machine learning model for each of the first plurality of data and
the difference between the output value of the first machine learning model for the data included in the third plurality of data and the correct label value corresponding to the data included in the third plurality of data; Based on the difference between the output value of the first machine learning model for each of the third plurality of data and the output value of the first machine learning model for each of the second plurality of data, selecting a fourth plurality of data of the three plurality of data and a fifth plurality of data of the second plurality of data related to the fourth plurality of data;
Numerical values calculated during forward propagation and numerical values calculated during backpropagation when the fourth plurality of data and the fifth plurality of data are input to the first machine learning model Based on, identifying a second plurality of parameters among the first plurality of parameters included in the first machine learning model,
generating a second machine learning model by updating only the second plurality of parameters of the first plurality of parameters;
A machine learning method in which the processing is performed by a computer.

(Appendix 10)
In the process of selecting the fifth plurality of data, centering on the data of the fourth plurality of data among the second plurality of data, between the output values of the first machine learning model for the data A process of determining data within the range of the neighborhood radius determined by the difference and the neighborhood coefficient as the fifth plurality of data,
The machine learning method according to Appendix 9.

(Appendix 11)
In the process of selecting the fourth plurality of data, among the third plurality of data, the fifth plurality of data within the range of the neighborhood radius is preferentially selected in descending order of priority. including a process of determining the plurality of data as a fourth plurality of data;
The machine learning method according to Appendix 10.

(Appendix 12)
In the process of specifying the second plurality of parameters, the degree of influence on the second plurality of data is selected from a first parameter group in which parameters having a high degree of influence on the third plurality of data are preferentially selected. A process of identifying a third parameter group obtained by subtracting a second parameter group in which a parameter with a smaller value is preferentially selected as the second plurality of parameters,
The machine learning method according to any one of appendices 9 to 11.

(Appendix 13)
The degree of influence on the third plurality of data is
The first obtained based on the product of the output of the layer included in the first machine learning model obtained by inputting the third plurality of data to the first machine learning model and the weight in the layer 13. The machine learning method of clause 12 including forward influence.

(Appendix 14)
The degree of influence on the third plurality of data is
A second obtained based on the product of the output of the layer included in the first machine learning model obtained by inputting the second plurality of data to the first machine learning model and the weight in the layer 14. The machine learning method of clause 12 or clause 13 comprising forward influence.

(Appendix 15)
The degree of influence on the third plurality of data is
Any one of appendices 12 to 14, including a first backward influence degree obtained by differentiating the output obtained by inputting the third plurality of data into the first machine learning model with a weight. machine learning method.

(Appendix 16)
The degree of influence on the third plurality of data is
Any one of appendices 12 to 15, including a second backward influence degree obtained by differentiating the output obtained by inputting the second plurality of data into the first machine learning model with a weight. machine learning method.

(Appendix 17)
Among the first plurality of data, a second plurality of data and a third plurality of data having incorrect prediction results according to the output value of the first machine learning model for each of the first plurality of data and
The difference (decision boundary and the difference between the output value of the first machine learning model for each of the third plurality of data and the output value of the first machine learning model for each of the second plurality of data and selecting a fourth plurality of data of the third plurality of data and a fifth plurality of data of the second plurality of data related to the fourth plurality of data based on
Numerical values calculated during forward propagation and numerical values calculated during backpropagation when the fourth plurality of data and the fifth plurality of data are input to the first machine learning model Based on, identifying a second plurality of parameters among the first plurality of parameters included in the first machine learning model,
generating a second machine learning model by updating only the second plurality of parameters of the first plurality of parameters;
An information processing apparatus comprising a control unit that executes processing.

(Appendix 18)
The control unit
of the neighborhood radius determined by the difference between the output values of the first machine learning model for the data and the neighborhood coefficient, centering on the data of the fourth plurality of data among the second plurality of data 18. The information processing apparatus according to appendix 17, wherein data within the range is determined as the fifth plurality of data.

(Appendix 19)
The control unit
Among the plurality of third data, the plurality of data with the highest priority selected from those having the smallest number of the plurality of fifth data within the range of the neighborhood radius are determined as the plurality of fourth data. 19. The information processing apparatus according to Supplementary Note 18.

(Appendix 20)
The control unit
From a first parameter group in which parameters having a high degree of influence on the third plurality of data are preferentially selected, a second parameter group in which parameters having a low degree of influence on the second plurality of data are preferentially selected 19. The information processing apparatus according to any one of appendices 17 to 19, wherein a third parameter group obtained by subtracting is specified as the second plurality of parameters.

1 Information Processing Device 11 Processor 12 Memory 13 Storage Device 14 Graphic Processing Device 14a Monitor 15 Input Interface 15a Keyboard 15b Mouse 16 Optical Drive Device 16a Optical Disk 17 Equipment Connection Interface 17a Memory Device 17b Memory Reader/Writer 17c Memory Card 18 Network Interface 18a Network 19 bus 101 model training unit 102 test data classification unit 103 neighborhood success data search unit 104 failure data narrowing unit 105 success data narrowing unit 106 weight influence measurement unit 107 correction target weight identification unit 210, 220 forward influence information 230, 240 backward influence degree information 211, 221 forward influence order index list 231, 241 backward influence order index list

Claims (10)

Among the first plurality of data, a second plurality of data and a third plurality of data having incorrect prediction results according to the output value of the first machine learning model for each of the first plurality of data and
the difference between the output value of the first machine learning model for the data included in the third plurality of data and the correct label value corresponding to the data included in the third plurality of data; Based on the difference between the output value of the first machine learning model for each of the third plurality of data and the output value of the first machine learning model for each of the second plurality of data, selecting a fourth plurality of data of the three plurality of data and a fifth plurality of data of the second plurality of data related to the fourth plurality of data;
Numerical values calculated during forward propagation and numerical values calculated during backpropagation when the fourth plurality of data and the fifth plurality of data are input to the first machine learning model Based on, identifying a second plurality of parameters among the first plurality of parameters included in the first machine learning model,
generating a second machine learning model by updating only the second plurality of parameters of the first plurality of parameters;
A machine learning program that makes a computer perform a process. In the process of selecting the fifth plurality of data, centering on the data of the fourth plurality of data among the second plurality of data, between the output values of the first machine learning model for the data A process of determining data within the range of the neighborhood radius determined by the difference and the neighborhood coefficient as the fifth plurality of data,
The machine learning program according to claim 1. In the process of selecting the fourth plurality of data, among the third plurality of data, the fifth plurality of data within the range of the neighborhood radius is preferentially selected in descending order of priority. including a process of determining the plurality of data as a fourth plurality of data;
The machine learning program according to claim 2. In the process of specifying the second plurality of parameters, the degree of influence on the second plurality of data is selected from a first parameter group in which parameters having a high degree of influence on the third plurality of data are preferentially selected. A process of identifying a third parameter group obtained by subtracting a second parameter group in which a parameter with a smaller value is preferentially selected as the second plurality of parameters,
The machine learning program according to any one of claims 1 to 3. The degree of influence on the third plurality of data is
The first obtained based on the product of the output of the layer included in the first machine learning model obtained by inputting the third plurality of data to the first machine learning model and the weight in the layer 5. The machine learning program of claim 4, comprising forward influence. The degree of influence on the third plurality of data is
A second obtained based on the product of the output of the layer included in the first machine learning model obtained by inputting the second plurality of data to the first machine learning model and the weight in the layer 6. A machine learning program according to claim 4 or 5, comprising forward influence. The degree of influence on the third plurality of data is
Any one of claims 4 to 6, comprising a first backward influence obtained by differentiating the output obtained by inputting the third plurality of data to the first machine learning model with a weight The described machine learning program. The degree of influence on the third plurality of data is
Any one of claims 4 to 7, comprising a second backward influence obtained by differentiating the output obtained by inputting the second plurality of data to the first machine learning model with a weight The described machine learning program. Among the first plurality of data, a second plurality of data and a third plurality of data having incorrect prediction results according to the output value of the first machine learning model for each of the first plurality of data and
the difference between the output value of the first machine learning model for the data included in the third plurality of data and the correct label value corresponding to the data included in the third plurality of data; Based on the difference between the output value of the first machine learning model for each of the third plurality of data and the output value of the first machine learning model for each of the second plurality of data, selecting a fourth plurality of data of the three plurality of data and a fifth plurality of data of the second plurality of data related to the fourth plurality of data;
Numerical values calculated during forward propagation and numerical values calculated during backpropagation when the fourth plurality of data and the fifth plurality of data are input to the first machine learning model Based on, identifying a second plurality of parameters among the first plurality of parameters included in the first machine learning model,
generating a second machine learning model by updating only the second plurality of parameters of the first plurality of parameters;
A machine learning method in which the processing is performed by a computer. Among the first plurality of data, a second plurality of data and a third plurality of data having incorrect prediction results according to the output value of the first machine learning model for each of the first plurality of data and
the difference between the output value of the first machine learning model for the data included in the third plurality of data and the correct label value corresponding to the data included in the third plurality of data; Based on the difference between the output value of the first machine learning model for each of the third plurality of data and the output value of the first machine learning model for each of the second plurality of data, selecting a fourth plurality of data of the three plurality of data and a fifth plurality of data of the second plurality of data related to the fourth plurality of data;
Numerical values calculated during forward propagation and numerical values calculated during backpropagation when the fourth plurality of data and the fifth plurality of data are input to the first machine learning model Based on, identifying a second plurality of parameters among the first plurality of parameters included in the first machine learning model,
generating a second machine learning model by updating only the second plurality of parameters of the first plurality of parameters;
An information processing apparatus comprising a control unit that executes processing.

Images