JP6735862B1 - Learning method of neural network - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a learning method of a neural network capable of determining the end of learning of a neural network at an appropriate timing with which an error becomes small. SOLUTION: The first condition that the absolute value of the error between the target value of the output data and the actual value of the output data is less than the error judgment value, and the actual value of the output data after the execution of the previous learning step and the current value The learning ends when at least one of the second conditions in which the absolute value of the amount of change from the actual value of the output data after execution of the learning step is less than the amount of change determination value is satisfied for all pairs of the supervised learning data set. A learning method for a neural network, which determines that the condition is satisfied, and otherwise determines that the learning end condition is not satisfied. [Selection diagram] Figure 2

Description

The present application relates to a learning method for a neural network.

The following Patent Document 1 discloses a learning method for a neural network. In the technique of Patent Document 1, the learning of a neural network is performed by a known back propagation method (back propagation method) using a learning data set with a teacher.

At this time, the end of learning is determined by the difference between the teacher data of the supervised learning data (the target value of the output data of the neural network) and the actual value of the output data of the neural network when the learning of all the supervised learning data sets It holds when the data falls below the judgment value.

Japanese Patent No. 3823554

The judgment value for judging the end of learning is determined from the allowable error, the configuration of the neural network, the numerical value range of the teacher data, and the like. If the judgment value is set to a large value, the learning end condition is likely to be met, but if learning is continued, the error could be made smaller, but learning was ended before that. The case may occur. On the other hand, if the judgment value is set to a small value, the error can be expected to decrease if the learning progresses well, but if the limit of the generalization ability of the neural network is reached, the error does not decrease. The termination condition is not satisfied at all, learning is continued, and useless consumption of computational resources may occur. This dilemma will be described using a specific example.

FIG. 8 shows the behavior of an error with respect to the number of learning trials in a neural network. The absolute value of the error decreases from the start of learning to about 1000 trials, and the maximum and minimum values of the error change toward 0.

However, after about 1000 trials, the absolute value of the error gradually decreases while increasing or decreasing, and the absolute value of the error becomes the smallest at about 5000 trials. However, the "smallest" is found as a result of performing up to 20000 trials, and it is unknown at this point whether or not it is the smallest. When learning is further performed after about 5,000 trials, the absolute value of the error slightly increases, and after about 9000 trials, the absolute value is almost unchanged.

In FIG. 8, if the judgment value of the learning end judgment is set to 5% of the error, the learning end condition is satisfied and the learned neural network with the error of 5% is obtained before 1000 trials. In this case, the number of trials of learning, that is, the amount of calculation is small, but learning is promptly abandoned before the error is minimized. That is, underlearning has occurred.

On the other hand, if the judgment value is set to 2% of the error, the learning end condition will not be satisfied even if the number of trials is calculated up to 20000. In preparation for such a case, conventionally, learning is configured to be forcibly terminated when the number of learning trials reaches the number of forced termination trials, and the number of forced termination trials is set to 10,000, for example. If set, learning is forcibly terminated after 10,000 trials. Then, despite consuming a large amount of computational resources, only a learned neural network with an error of about 4%, which is not the smallest error, can be obtained.

The judgment value of the learning end judgment and the judgment value of the forced termination trial number are influenced by the configuration of the neural network and the prepared learning data set with a teacher. Especially, when learning is performed by repeatedly collecting or generating a supervised learning data set, it is difficult to set the learning end determination value and the forced termination trial frequency determination value to optimal values in advance. .. Therefore, the above-mentioned under-learning or useless calculation resource consumption occurs.

An object of the present application is to obtain a learning method of a neural network that can determine the end of learning of the neural network at an appropriate timing when the error becomes small.

The second neural network learning method of the present application is
Using a supervised learning data set composed of a plurality of pairs of input data input to the neural network and target values of output data output from the neural network for the input data, parameters of the neural network are set. A learning step to update,
It is determined whether or not a learning end condition is satisfied. If the learning end condition is not satisfied, the learning step is executed again. If the learning end condition is satisfied, the learning step A learning end determination step of ending the execution,
In the learning end determination step, the absolute value of the error between the target value of the output data and the actual value of the output data output from the neural network, and the actual value of the output data after execution of the previous learning step. And the multiplication value obtained by multiplying the absolute value of the amount of change between the actual value of the output data after the execution of the learning step this time is below the multiplication value determination value, all the pairs of the supervised learning data set. Is satisfied, it is determined that the learning end condition is satisfied, and in other cases, it is determined that the learning end condition is not satisfied.

The third neural network learning method of the present application is
Using a supervised learning data set composed of a plurality of pairs of input data input to the neural network and target values of output data output from the neural network for the input data, parameters of the neural network are set. A learning step to update,
It is determined whether or not a learning end condition is satisfied. If the learning end condition is not satisfied, the learning step is executed again. If the learning end condition is satisfied, the learning step A learning end determination step of ending the execution,
In the learning end determination step, for each pair of the learning data set with teacher, the absolute value of the error between the target value of the output data and the actual value of the output data output from the neural network, and the previous learning A multiplication value is calculated by multiplying the absolute value of the change amount between the actual value of the output data after the execution of the step and the actual value of the output data after the execution of the learning step this time, and the multiplication of all pairs is performed. When the integrated value obtained by integrating the values is less than the integrated value determination value, it is determined that the learning end condition is satisfied, and in other cases, it is determined that the learning end condition is not satisfied. is there.

The small absolute value of the change amount of the actual value of the output data means that the actual value of the output data changes only a little even if the number of learning trials is advanced by one. In this state, it can be determined that the generalization ability of the neural network is approaching the limit, that is, the absolute value of the error cannot be further reduced and the error is close to the minimum value. Therefore, by adding to the learning end condition that the absolute value of the change amount of the actual value of the output data is small, it is possible to suppress the waste of calculation resources.

According to the second learning method of the neural network, the multiplication value between the absolute value of the error of the output data and the absolute value of the change amount of the actual value of the output data is small for all pairs of the supervised learning data set, Therefore, the learning can be ended in a state where the error is sufficiently reduced or in a state close to the minimum value of the error. Therefore, learning can be ended at an appropriate timing with a small error while suppressing waste of calculation resources.

According to the third learning method of the neural network, it is determined that the sum of all pairs of the multiplied values of the absolute value of the error of the output data and the absolute value of the change amount of the actual value of the output data is small. Therefore, the learning can be ended in a state where the error is sufficiently reduced or in a state close to the minimum value of the error. Therefore, learning can be ended at an appropriate timing with a small error while suppressing waste of calculation resources.

3 is a schematic configuration diagram of a neural network according to the first embodiment. FIG. 5 is a flowchart illustrating a learning method of the neural network according to the first embodiment. 4 is a flowchart illustrating in detail a part of the learning method of the neural network according to the first embodiment. 9 is a flowchart illustrating a learning method of the neural network according to the second embodiment. 7 is a flowchart for explaining in detail a part of the learning method of the neural network according to the second embodiment. 9 is a flowchart illustrating a learning method of the neural network according to the third embodiment. 9 is a flowchart for explaining in detail a part of the learning method of the neural network according to the third embodiment. It is a figure for demonstrating the subject of this application.

1. Embodiment 1
A learning method of the neural network according to the first embodiment will be described with reference to the drawings.

<Neural network>
In the present embodiment, as shown in FIG. 1, a forward propagation type neural network (FNN: Feedforward Neural Network, hereinafter referred to as FNN) in which a signal is transmitted from an input layer to an output layer is used. FNN is a network in which units (also called nodes or neurons) arranged in a hierarchy are connected between adjacent layers, and information is propagated in one direction from an input side to an output side. is there. In the unit, the value input from each unit in the previous layer is weighted, and a value to which a bias is applied is input to the activation function, and the output of the activation function is used as the output of the unit.

In the FNN, one or more input data is input to the input layer, one or more intermediate layers are provided, and one or more output data is output from the output layer. The number of input data, the number of intermediate layers, the number of units in each intermediate layer, and the number of output data may be arbitrary numbers.

<Neural network learning>
Changing the parameters of the FNN (in this example, the weight between units and the bias of the units) so that the output data output from the FNN with respect to the input data input to the FNN approaches the target value. Call it learning. In order to perform learning, it is necessary to prepare a supervised learning data set including a plurality of pairs of input data and target values of output data. In addition, a learning method such as a known back propagation method (back propagation method) is used for learning.

2 and 3 are flowcharts for explaining the FNN learning method according to the present embodiment. Each step of the FNN learning method is executed by a processing circuit included in the learning device. Specifically, the learning device includes, as processing circuits, an arithmetic processing device (computer) such as a CPU (Central Processing Unit) and a DSP (Digital Signal Processor), a storage device that exchanges data with the arithmetic processing device, and an arithmetic processing device. An input/output circuit for inputting/outputting signals to/from an external device is provided. The storage device includes a RAM (Random Access Memory), a ROM (Read Only Memory), an auxiliary storage device such as a hard disk, and the like. The input/output circuit includes a communication device, a signal input/output device, and the like.

In each step of the FNN learning method, the arithmetic processing unit executes software (program) stored in a storage device such as a ROM or a hard disk, and other hardware of the learning device such as the storage device or the input/output circuit. It is realized by cooperating with. Data such as FNN parameters, supervised learning data set, error, and variation used in each step are stored in a storage device such as a RAM or a hard disk. Hereinafter, each step of the learning method of FNN will be described in detail.

<Learning step>
In learning step S1, the parameters of FNN are updated using the supervised learning data set. As described above, the supervised learning data set includes a plurality of pairs of input data and target values of output data. In the present embodiment, the learning step S1 is composed of steps S12 to S16. First, in step S11, when starting a series of learning, the number N of learning trials is set to zero (N=0). For example, step S11 is executed when a new supervised learning data set is prepared and learning is started.

Then, in step S12, all pairs of the supervised learning data set are set unprocessed. Then, in step S13, one unprocessed pair is selected from the learning data set with teacher and set as the processed data.

In step S14, the input data of the pair set as the processing data (hereinafter referred to as the processing pair) is input to the FNN, and the output data is output by using the most recently updated parameters (weight and bias) of the FNN. Calculate the value. Then, for the input data of the processing pair, the error between the actual value of the output data output from the FNN and the target value of the output data of the processing pair is calculated. Then, based on the error, a known learning method such as an error back propagation method (back propagation method) is used to update, that is, learn the FNN parameters. A learning method such as an error back propagation method (back propagation method) is a known technique, and thus its description is omitted.

In step S15, it is determined whether or not the learning in step S14 has been performed for all pairs. If it is determined that the learning has not been performed for all pairs, the process returns to step S13 and learning is completed for all pairs. Up to step S13, when the processes of steps S13 to S14 are repeatedly executed and it is determined that all the pairs have been executed, the process proceeds to step S16.

In step S16, the FNN parameters are updated only once for all pairs of the learning data set with teacher. Therefore, the number N of learning trials is increased by 1 (N←N+1).

In step S21, it is determined whether or not the learning trial number N is less than a preset forced termination number J. If it is determined to be less than, the process proceeds to step S22. Kill learning. The number of forced terminations J gives an upper limit to the number of learning trials from the calculation resources allocated for learning, the time required for calculation, and the like.

<Learning end determination step>
Step S22 corresponds to the learning end determination step. In step S22, it is determined whether or not the learning end condition is satisfied. If the learning end condition is not satisfied, the learning step S1 is executed again, and if the learning end condition is satisfied, the learning step is performed. The execution of S1 is ended.

In step S22, as shown in equation (1), the absolute value |Oerr| of the error between the target value Ot of the output data and the actual value Or of the output data is The first value that is less than the error determination value JOerr, and the absolute value of the amount of change between the actual value Or of the output data after the execution of the previous learning step S1 and the actual value Or of the output data after the execution of the current learning step S1 | It is determined that the learning end condition is satisfied when at least one of the second conditions in which ΔOr| is smaller than the variation determination value JOr is satisfied for all pairs of the supervised learning data set, and otherwise, , It is determined that the learning end condition is not satisfied. In Expression (1), the pair number i is assumed to be from 1 to Iend, and the subscript i of each value represents the pair number. In addition, each value corresponding to after execution of the learning step S1 of this time (immediately before) (current learning trial count N) is represented by (N), and after execution of the previous learning step S1 (previous learning trial count N). Each value corresponding to (-1) is represented by (N-1).

For i=1 to Iend,
|Oerr _i |=|Ot _i −O r _i (N)|
|ΔOr _i |=|Or _i (N−1)−Or _i (N)|
And calculate
For all i=1 to Iend,
|Oerr _i |<JOerr
And |ΔOr _i |<JOr
At least one of the above holds... (1)

The process of step S22 can be configured as shown in the flowchart of FIG. In step S221, the pair number i is set to the initial value of 1.

In step S222, the input data of the pair number i is input to the FNN, and the output data after execution of the current learning step S1 is performed using the FNN parameters (weight and bias) updated in the current learning step S1. The actual value Or _i (N) of is calculated. Then, as shown in the equation (1), the actual value Or _i (N) of the output data output from the FNN for the input data of the pair number i and the target value Ot _i of the output data of the pair number _i. The absolute value of the error between |Oerr _i | is calculated. Each calculated value such as the calculated actual value Or _i of the output data is stored in a storage device such as a RAM in association with the current learning trial number N and the pair number i, and is used in step S223 and the like. ..

In step S223, for the pair number i, the actual value Or _i (N−1) of the output data after the execution of the previous learning step S1 and the current value after the execution of this learning step S1 are performed, as shown in equation (1). The absolute value |ΔOr _i | of the amount of change from the actual value Or _i (N) of the output data is calculated.

In step S224, it is determined whether the absolute value of the error |Oerr _i | of the output data of the pair number i calculated in step S222 is less than the error determination value JOerr. If it is determined that the difference is below the error determination value JOerr, the process proceeds to step S225, and if it is determined that it is not below the error determination value JOerr, the process proceeds to step S226. In step S226, it is determined whether or not the absolute value |ΔOr _i | of the change amount of the actual value of the output data of the pair number i calculated in step S223 is less than the change determination value JOr. If it is determined that it is below the change amount determination value JOr, the process proceeds to step S225, and if it is determined that it is not below the change amount determination value JOr, the process proceeds to step S229. In step S229, it is determined that the current pair number i is not below the error determination value JOerr and is not below the change amount determination value JOr, so the determination is not continued for the next pair number i, and learning is performed. The final determination is made that the termination condition is not satisfied, and the determination processing ends.

On the other hand, when it is determined that the current pair number i is below the error determination value JOerr in step S224 or below the change amount determination value JOr in step S226, the pair number i is determined in step S225. Is less than the final pair number Iend, and if it is less than the final pair number Iend, the pair number i is incremented by 1 in step S227, and then the process returns to step S222 to return to the next pair. The determination is continued for the number i of. On the other hand, if it is not less than the final pair number Iend, step S224 or step S226 is satisfied for all pairs, so it is determined in step S228 that the learning end condition is satisfied, and the determination process is ended.

When the generalization capability of the FNN is approaching the limit, the absolute value of the error |Oerr| of the output data may not decrease to the desired error (error determination value JOerr). Therefore, if the learning end condition is determined only by comparing the absolute value |Oerr| of the error of the output data and the error determination value JOerr, the learning does not end in the above case, and the calculation resources are wasted. On the other hand, when the FNN generalization capability is approaching its limit, the absolute value |ΔOr| of the change amount of the actual value of the output data approaches zero. According to the above configuration, for each pair, even if the absolute value of output data error |Oerr| is not sufficiently reduced, the absolute value |ΔOr| When the value is less than the value JOr, it is determined that the learning result is good. Therefore, for all pairs, the error in the output data is sufficiently reduced, or the error is not sufficiently reduced, but the learning end condition is satisfied when the error is approaching the limit of the FNN generalization ability. It can be determined that it did. Therefore, it is possible to appropriately end learning for all pairs, and it is possible to suppress waste of calculation resources.

2. Embodiment 2
Next, a learning method of the neural network according to the second embodiment will be described. The description of the same components as those in the first embodiment is omitted. In the present embodiment, the condition for satisfying the learning end condition in the learning end determination step is different from that in the first embodiment.

4 and 5 are flowcharts for explaining the FNN learning method according to the present embodiment. The learning step S1 (steps S11 to S16) of FIG. 4 is the same as the learning step S1 (steps S11 to S16) of FIG.

Similar to the first embodiment, in step S31, it is determined whether or not the learning trial number N is less than a preset forced termination number J, and if it is less than, the process proceeds to step S32. If it is determined that it is not less than, the learning is forcibly terminated.

<Learning end determination step>
In the present embodiment, step S32 corresponds to the learning end determination step. In step S32, it is determined whether or not the learning end condition is satisfied. If the learning end condition is not satisfied, the learning step S1 is executed again, and if the learning end condition is satisfied, the learning step is performed. The execution of S1 is ended.

In step S32, the absolute value |Oerr| of the error between the target value Ot of the output data and the actual value Or of the output data after execution of the learning step S1 of this time (immediately before), as shown in equation (2), The multiplication value M of the absolute value |ΔOr| of the change amount between the actual value Or of the output data after the execution of the previous learning step S1 and the actual value Or of the output data after the execution of the current learning step is the multiplication value determination. If the condition below the value Jm is satisfied for all pairs of the supervised learning data set, it is determined that the learning end condition is satisfied, and otherwise, it is determined that the learning end condition is not satisfied. .. In Expression (2), the pair number i is assumed to be from 1 to Iend, and the subscript i of each value represents the pair number. In addition, each value corresponding to after execution of the learning step S1 of this time (immediately before) (current learning trial count N) is represented by (N), and after execution of the previous learning step S1 (previous learning trial count N). Each value corresponding to (-1) is represented by (N-1).

For i=1 to Iend,
|Oerr _i |=|Ot _i −O r _i (N)|
|ΔOr _i |=|Or _i (N−1)−Or _i (N)|
M _i =|Oerr _i |×|ΔOr _i |
And calculate
For all i=1 to Iend,
M _i <Jm
Holds (2)

The process of step S32 can be configured as shown in the flowchart of FIG. In step S321, the pair number i is set to the initial value of 1.

In step S322, the input data of the pair number i is input to the FNN, and the output data after execution of the current learning step S1 is performed using the FNN parameters (weight and bias) updated in the current learning step S1. The actual value Or _i (N) of is calculated. Then, as shown in Expression (2), the actual value Or _i (N) of the output data output from the FNN with respect to the input data of the pair number i and the target value Ot _i of the output data of the pair number _i. The absolute value of the error between |Oerr _i | is calculated. Each calculated value such as the calculated actual value Or _i of the output data is stored in a storage device such as a RAM in association with the current learning trial number N and the pair number i, and is used in step S323 and the like. ..

In step S323, as shown in equation (2), for the pair number i, the actual value Or _i (N-1) of the output data after the execution of the previous learning step S1 and the current value after the execution of the learning step S1 this time are performed. The absolute value |ΔOr _i | of the amount of change from the actual value Or _i (N) of the output data is calculated.

In step S324, the absolute value of the error |Oerr _i | of the output data of the pair number i calculated in step S322 and the output data of the pair number i calculated in step S323 are calculated as shown in equation (2). The absolute value |ΔOr _i | of the change amount of the actual value of is multiplied by M _i .

In step S325, it is determined whether the multiplication value M _i of the pair number i calculated in step S324 is lower than the multiplication value determination value Jm. If it is determined that it is below the multiplication value determination value Jm, the process proceeds to step S326, and if it is determined that it is not below the multiplication value determination value Jm, the process proceeds to step S329. In step S329, it is determined that the multiplication value determination value Jm is not below the current pair number i, so the determination is not continued for the next pair number i and the final determination is made if the learning end condition is not satisfied. Then, the determination process ends.

On the other hand, if it is determined in step S325 that the current pair number i is less than the multiplication value determination value Jm, it is determined in step S326 whether the pair number i is less than the final pair number Iend. If it is determined that the number is smaller than the final pair number Iend, the pair number i is incremented by 1 in step S327, the process returns to step S322, and the determination is continued for the next pair number i. On the other hand, if it is not less than the final pair number Iend, step S325 is satisfied for all pairs, so it is determined in step S328 that the learning end condition is satisfied, and the determination process is ended.

When the generalization capability of the FNN is approaching the limit, the absolute value of the error |Oerr| of the output data may not decrease to the desired error (error determination value JOerr). Therefore, if the learning end condition is determined only by comparing the absolute value |Oerr| of the error of the output data and the error determination value JOerr, the learning does not end in the above case, and the calculation resources are wasted. On the other hand, when the FNN generalization capability is approaching its limit, the absolute value |ΔOr| of the change amount of the actual value of the output data approaches zero. According to the above configuration, for each pair, even if the absolute value of the error |Oerr| of the output data is not sufficiently decreased, the absolute value of the error of the output data is If the multiplication value M of |Oerr| and the absolute value |ΔOr| of the change amount of the actual value of the output data decreases and the multiplication value M is below the multiplication value determination value Jm, the learning result is good. Is determined. Therefore, for all pairs, the error in the output data is sufficiently reduced, or the error is not sufficiently reduced, but the learning end condition is satisfied when the error is approaching the limit of the FNN generalization ability. It can be determined that it did. Therefore, it is possible to appropriately end learning for all pairs, and it is possible to suppress waste of calculation resources.

3. Embodiment 3
Next, a learning method of the neural network according to the third embodiment will be described. The description of the same components as those in the first embodiment is omitted. In the present embodiment, the condition for satisfying the learning end condition in the learning end determination step is different from that in the first embodiment.

6 and 7 are flowcharts for explaining the FNN learning method according to the present embodiment. The learning step S1 (steps S11 to S16) of FIG. 6 is the same as the learning step S1 (steps S11 to S16) of FIG.

Similar to the first embodiment, in step S41, it is determined whether or not the learning trial number N is less than a preset forced termination number J, and if it is less than, the process proceeds to step S42. If it is determined that it is not less than, the learning is forcibly terminated.

<Learning end determination step>
In the present embodiment, step S42 corresponds to the learning end determination step. In step S42, it is determined whether or not the learning end condition is satisfied. If the learning end condition is not satisfied, the learning step S1 is executed again, and if the learning end condition is satisfied, the learning step is performed. The execution of S1 is ended.

In step S42, as shown in equation (3), after execution of the learning step S1 of this time (immediately before), the target value Ot of the output data and the actual value Or of the output data are set for each pair of the learning data set with teacher. Absolute value |Oerr| of the difference between the actual value Or of the output data after the execution of the previous learning step S1 and the actual value Or of the output data after the execution of the current learning step |ΔOr| If the integrated value Mint, which is calculated by multiplying the multiplied values M of all the pairs by and is less than the integrated value determination value Jint, determines that the learning end condition is satisfied, and otherwise Determines that the learning end condition is not satisfied. In Expression (3), the pair number i is assumed to be from 1 to Iend, and the subscript i of each value represents the pair number. In addition, each value corresponding to after execution of the learning step S1 of this time (immediately before) (current learning trial count N) is represented by (N), and after execution of the previous learning step S1 (previous learning trial count N). Each value corresponding to (-1) is represented by (N-1).

For i=1 to Iend,
|Oerr _i |=|Ot _i −O r _i (N)|
|ΔOr _i |=|Or _i (N−1)−Or _i (N)|
M _i =|Oerr _i |×|ΔOr _i |
Mint=ΣM _i
And calculate
Mint<Jint
Holds (3)

The process of step S42 can be configured as shown in the flowchart of FIG. In step S421, the pair number i is set to the initial value of 1. In step S422, the integrated value Mint is reset to 0.

In step S423, the input data of the pair number i is input to the FNN, and the output data after execution of the current learning step S1 is performed using the FNN parameters (weight and bias) updated in the current learning step S1. The actual value Or _i (N) of is calculated. Then, as shown in Expression (3), the actual value Or _i (N) of the output data output from the FNN for the input data of the pair number i and the target value Ot _i of the output data of the pair number _i. The absolute value of the error between |Oerr _i | is calculated. Each calculated value such as the calculated actual value Or _i of the output data is stored in a storage device such as a RAM in association with the current learning trial number N and the pair number i, and is used in step S424 and the like. ..

In step S424, as shown in equation (3), for the pair number i, the actual value Or _i (N-1) of the output data after the execution of the previous learning step S1 and the current value after the execution of this learning step S1 are performed. The absolute value |ΔOr _i | of the amount of change from the actual value Or _i (N) of the output data is calculated.

In step S425, as shown in Expression (3), the absolute value of the error |Oerr _i | of the output data of the pair number i calculated in step S423 and the output data of the pair number i calculated in step S424. The absolute value |ΔOr _i | of the change amount of the actual value of is multiplied by M _i .

In step S426, the integrated value Mint is integrated with the multiplied value M _i of the pair number i. In step S427, it is determined whether or not the pair number i is less than the final pair number Iend. If it is less than the final pair number Iend, in step S428, the pair number i is incremented by 1, Returning to step S423, the calculation of the multiplication value and integrated value for the number i of the next pair is continued, and if it is not less than the final pair number Iend, the process proceeds to step S429.

In step S429, it is determined whether the integrated value Mint is below the integrated value determination value Jint. If it is determined that it is below the integrated value determination value Jint, it proceeds to step S430, it is determined that the learning end condition is satisfied, and if it is determined that it is not below the integrated value determination value Jint, to step S431. Then, it is determined that the learning end condition is not satisfied, and the determination process ends.

When the generalization capability of the FNN is approaching the limit, the absolute value of the error |Oerr| of the output data may not decrease to the desired error (error determination value JOerr). Therefore, if the learning end condition is determined only by comparing the absolute value |Oerr| of the error of the output data and the error determination value JOerr, the learning does not end in the above case, and the calculation resources are wasted. On the other hand, when the FNN generalization capability is approaching its limit, the absolute value |ΔOr| of the change amount of the actual value of the output data approaches zero. According to the above configuration, for each pair, even if the absolute value of the error |Oerr| of the output data is not sufficiently decreased, the absolute value of the error of the output data is If the multiplication value M of |Oerr| and the absolute value |ΔOr| of the change amount of the actual value of the output data and its integrated value Mint decrease and the integrated value Mint is below the integrated value determination value Jint, learning The result is judged to be good. Therefore, for all pairs, the error in the output data is sufficiently reduced, or the error is not sufficiently reduced, but the learning end condition is satisfied when the error is approaching the limit of the FNN generalization ability. It can be determined that it did. Therefore, it is possible to appropriately end learning for all pairs, and it is possible to suppress waste of calculation resources.

[Other Embodiments]
(1) In each of the above-described embodiments, it is determined in step S21, step S31, and step S41 whether or not the learning trial number N is less than the forced termination number J, and when it is determined to be less than, In the learning end determination step, the case where the learning is forcibly terminated when it is determined that the difference is not less than is described as an example. However, the embodiment of the present application is not limited to this. That is, step S21, step S31, and step S41 may not be provided.

(2) In each of the above embodiments, in step S222, step S322, and step S423, the input data of the pair number i is input to the FNN, and the FNN parameters (weight and The case has been described as an example in which the actual value Or _i (N) of the output data after the execution of the learning step S1 of this time is calculated using (bias). However, the embodiment of the present application is not limited to this. That is, in step S222, step S322, and step S423, the actual value of the output data using FNN is not calculated. Instead, the actual value of the output data calculated in step S14 of this learning step S1 is used. You may be asked.

Although the present application describes various exemplary embodiments and examples, various features, aspects, and functions described in one or more of the embodiments are applicable to the particular embodiment. However, the present invention is not limited to the above, and can be applied to the embodiments alone or in various combinations. Therefore, innumerable variations not illustrated are envisioned within the scope of the technology disclosed herein. For example, it is assumed that at least one component is modified, added or omitted, and at least one component is extracted and combined with the components of other embodiments.

Oerr Error between actual value of output data and target value of output data, Iend final pair number, J forced termination count, JOerr error judgment value, JOr change amount judgment value, Jint integrated value judgment value, Jm multiplication value judgment value, M Multiply value, Mint integrated value, N number of learning trials, actual value of Or output data, target value of Ot output data, S1 learning step, S22, S32, S42 learning end determination step

Claims (2)

Using a supervised learning data set consisting of a plurality of pairs of input data input to a neural network and target values of output data output from the neural network for the input data, parameters of the neural network are set. A learning step to update,
It is determined whether or not a learning end condition is satisfied. If the learning end condition is not satisfied, the learning step is executed again. If the learning end condition is satisfied, the learning step A learning method for a neural network, comprising: a learning end determination step of ending execution,
In the learning end determination step, the absolute value of the error between the target value of the output data and the actual value of the output data output from the neural network, and the actual value of the output data after execution of the previous learning step. And the multiplication value obtained by multiplying the absolute value of the amount of change between the actual value of the output data after the execution of the learning step this time is less than the multiplication value determination value, all the pairs of the supervised learning data set. A learning method for a neural network, which determines that the learning end condition is satisfied when the above condition is satisfied, and otherwise determines that the learning end condition is not satisfied. Using a supervised learning data set consisting of a plurality of pairs of input data input to a neural network and target values of output data output from the neural network for the input data, parameters of the neural network are set. A learning step to update,
It is determined whether or not a learning end condition is satisfied. If the learning end condition is not satisfied, the learning step is executed again. If the learning end condition is satisfied, the learning step A learning method for a neural network, comprising: a learning end determination step of ending execution,
In the learning end determination step, for each pair of the supervised learning data set, the absolute value of the error between the target value of the output data and the actual value of the output data output from the neural network, and the previous learning A multiplication value obtained by multiplying the absolute value of the change amount between the actual value of the output data after execution of the step and the actual value of the output data after execution of the learning step this time is calculated, and the multiplication of all pairs is performed. If the integrated value obtained by integrating the values is less than the integrated value determination value, it is determined that the learning end condition is satisfied, and otherwise, it is determined that the learning end condition is not satisfied. Learning method.

Images