JP2000112914A - Learning method for neural circuit network - Google Patents

Abstract

PROBLEM TO BE SOLVED: To attain improvement in predictive accuracy by enabling the correction of only one of a bias value and a load value without interlocking them by separately learning the load value and bias value and executing the learning of load value and bias value while successively switching them. SOLUTION: An intermediate layer LM adds up inputs from a lot of neural circuit elements E11-E1m composing of an input layer LI after applying respectively different prescribed weighting processing, applies prescribed processing to this result and outputs it from an output layer LO. This learning method for neural circuit network can execute three modes of (1) correcting processing of a load value only to individual neural circuit elements E11-E1m, (2) correcting processing of a bias component and (3) correcting processing of both the load value and the bias component. While using at least two of these three correcting processing modes, correcting processing is executed while switching the modes so as to provide the scheduled output data by inputting prepared learning data.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a method for learning a neural network which enables effective fine adjustment without significantly destroying the contents learned in the past.

[0002]

2. Description of the Related Art A neural network is a computer technology developed as a technology for artificially realizing advanced information processing functions. In other words, this neural network is an artificial intelligence technology that simulates a neural circuit of the brain using a computer, and has an input layer for inputting information, and an output layer for determining which answer corresponds to the input. , And an intermediate layer intermediate those layers, each layer being connected by many bonds.

When attempting to realize the information processing function of the brain by using such a network, it is necessary to first model a nerve cell as a basic unit. Usually, a single nerve cell is modeled as a multi-input / 1-output analog element as shown in FIG.

That is, inputs are INP ₁ , INP ₂ , INP
₃ ,... INP _n , and the synapse loads as w ₁ , w ₂ , w ₃ ,.
w _n, threshold _{(bias) θ, Σw i INP i +} θ ( however, i is 1, 2, 3, ...) as y, the output OUT OU
T = f (y). The function f is, for example, a monotone function having a saturation characteristic.

[0005] To a nerve cell, another input is given to each of a plurality of input terminals. For example, the input INP ₁ is input to the _first input terminal, the input INP ₂ is input to the _second input terminal, the input INP ₃ is input to the third input terminal, and the input INP is input to the n-th input terminal. _n is input as follows. In the nerve cell, these inputs are multiplied by synaptic loads w ₁ , w ₂ , w ₃ ,..., W _n unique to the respective input systems to make corrections, and these corrected inputs w ₁ INP ₁ , w _{2
INP 2, w 3 INP 3,} ..., is added for w _n INP _n,
Further, a threshold (bias) θ is added. The output OUT is obtained by calculating the value Σw _i INP _i + θ thus obtained by a predetermined monotone function having a saturation characteristic.

As described above, since a nerve cell model has parameters such as synaptic load and bias, an arbitrary mapping can be created from a plurality of inputs by adjusting these parameters.

[0007]

The elements constituting a neural network are modeled on nerve cells. These elements have parameters called load values (synaptic loads) and biases. Any mapping can be created by adjusting.

(Calculation formula O _i = f (Σw _ij O _j + h _i )).
Here, w _ij is a load value, and h _i is a bias value. F is an output function. The bias value is the value of the element i when the input to the element i is zero.
, But learning is normally performed in the same manner as other load values. In a program, a dummy element that always outputs “1” is added, and the load value of the dummy element is treated as a bias.

The bias is more likely to become an inappropriate value during learning than the weight value, and if it becomes inappropriate, the learning efficiency of the entire neural network is reduced. .

When performing time series data processing using a neural network, prediction of future data is generally performed using past time series data as learning data. The fluctuation component needs to be corrected.

However, conventionally, when correction to the neural network system is required, (1) a numerical value is uniformly added to or subtracted from the output of the neural network, and (2) the neural network is calculated based on data near the predicted time. The method of re-learning the whole is adopted. In such a conventional method, the following problems remain.

That is, in the above method (1), the correction is insufficient when the fluctuation component is not a constant, and in the above method (2), the characteristics of the entire neural network are changed by the neighboring data. In other words, the correction becomes excessive.

Therefore, in predicting future data using past time-series data as learning data, when time-series data has a long-period fluctuation component, the fluctuation component can be optimally corrected. It is hoped that the development of technologies that will do this will work.

Therefore, the object of the present invention is to
It is an object of the present invention to provide a method for learning a neural network in which only one of a bias value and a load value can be corrected without being linked, thereby improving prediction accuracy.

[0015]

In order to achieve the above object, the present invention is configured as follows. That is, input data and desired output data for the input data are given to a neural network formed by connecting a plurality of neural circuit elements that perform required processing and output on the input data in a multi-stage manner. A neural network learning method for correcting a weight value and a bias component of a neural circuit element added to data so that an output value at the time coincides with the desired output data. It is possible to perform a correction process of only a value, a correction process of only a bias component for an individual neural circuit element, and a correction process of both a load value and a bias component for an individual neural circuit element. , Using at least two, switching the correction process so that the expected output data can be obtained by inputting the prepared learning data. Which comprises carrying out. Furthermore, new data similar to the already-learned data is given to the neural network for which learning has been completed, and re-learning is performed. At the time of this re-learning, only the bias value is corrected or the weight value is changed. Only make corrections.

When a correction to the neural network system is required, in the conventional method, [i] a numerical value is uniformly added to or subtracted from the output of the neural network. [ii] Re-learn the entire neural network with data near the predicted time. That method was adopted.

However, in the above method [i], correction is insufficient when the fluctuation component is not a constant, and in the above method [ii], the characteristics of the entire neural network are changed by the neighboring data. In addition, the correction becomes excessive.

In the present invention, the learning of the load value and the bias value can be performed separately, and the learning of the load value and the bias value can be switched at any time, so that the learning efficiency can be improved. By adding new data similar to the already learned data to the neural network for which learning has been completed, and performing re-learning, by correcting only the bias value or correcting only the weight value Fine adjustment of the mapping can be performed without greatly destroying the characteristics of the mapping realized by the neural network.

Therefore, according to the present invention, it is possible to correct only one of the bias value and the load value without interlocking the bias value and the load value, and therefore, it is possible to improve the prediction accuracy. A method of learning a net can be provided.

[0020]

Embodiments of the present invention will be described below with reference to the drawings. Here, an embodiment of the present invention will be described with an example in which a time series is simulated by using a trigonometric function and time series prediction is performed. The neural network used for the prediction is as shown in FIG. That is, in FIG.
LI is an input layer for inputting information, and is composed of a plurality of neural circuit elements E11, E12, E13,..., E1m (m is any positive integer). LM is an intermediate layer, and the input layer L
A number of neural circuit elements E11, E12, E13,
, E1m are subjected to different predetermined weighting processes and then added together, and then subjected to predetermined processing and output. LO is an output layer,
This is to determine which answer corresponds to from the information given from the intermediate layer LM and output the result.

More specifically, the input layer LI includes a plurality of neural circuit elements E11, E12, E13,.
Further, the intermediate layer LM includes neural circuit elements E21, E22, E23,
, E2n (n is an arbitrary positive integer), and the outputs of the neural network elements E11, E12, E13,. After the processing, the neural network elements E21, E22, E23,..., And E2n constituting the intermediate layer LM are respectively inputted.

Each of the neural circuit elements E21, E22, E23,..., E2n constituting the intermediate layer LM performs necessary processing on the input, and then, with respect to the neural circuit element E3 constituting the output layer LO. The neural circuit element E3 outputs information as a final answer based on the outputs from these neural circuit elements E21, E22, E23,..., E2n.
The output layer LO is composed of one neural circuit element E3.

The operation of the present system having such a configuration will be described with reference to the following example. FIG. 2 shows the function F (t) = 0.7 sin
(T / π) +0.2 cos (4πt) (where π is pi)
Shows the time series generated. As shown in FIG.
This function F (t) is a function that exhibits a change characteristic such that a small undulation is placed on a large undulation. Now, it is assumed that if the time series section is divided into 100 in the range of “0” to “9”, the resolution will be satisfactory. For simplicity of description, the target neural network has five input elements LI of E11 to E15 and an intermediate layer L
It is assumed that M has a ten-element configuration of E21 to E210.

For the function F (t), the interval 0 ≦
t ≦ 9 is divided into 100 equal parts, and 100 time-series data x
1, to create a _{~x100 (x t = F (0.9} × (t-
1)). Since the input layer LI has a five-element configuration, in this example, the time series data x1,..., X30 for the first 30 of these 100 pieces of time series data x1,. It is assumed that x31,..., x100 are predicted.

In this case, the neural network used for prediction is as shown in FIG. That is, in FIG.
LI is an input layer for inputting information, and is composed of a plurality of neural circuit elements E11, E12, E13, E14 and E15. Also, L
M is an intermediate layer. The input from the five neural circuit elements E11, E12, E13, E14 and E15 constituting the input layer LI is subjected to different predetermined weighting processes and then added together. A predetermined process is performed and output.
LO is an output layer, which determines which answer corresponds from information given from the intermediate layer LM and outputs the result.

The intermediate layer LM includes ten neural network elements E21,
E22, E23, E24, E25, E26, E27, E28, E29, E
210, and these neural circuit elements E21, E22, E
23, E24, E25, E26, E27, E28, E29, E210 respectively include the neural circuit elements E11, E12, E1 of the input layer LI.
The output of 3, E14, E15 is input to itself.

The neural network elements E21, E22, E23, E
24, E25, E26, E27, E28, E29, and E210 are configured to receive information obtained by subjecting the information input thereto to a required weighting process.

The neural network elements E21, E22, E23 to E27, E28, E29 and E210 constituting the intermediate layer LM are each subjected to a required processing for the input and then to the output layer LO.
Are output to the neural circuit element E3 which constitutes the neural network elements E21, E22, E23,.
Information as a final answer is output based on the outputs from E27, E28, E29 and E210. The output layer LO
Consists of one neural circuit element E3.

Learning of the neural network having such a configuration is first performed. In this learning, a time series up to five times before is input to the input layer LI as input information, and the time series at the current time is estimated. However, the load value and the bias value may be adjusted so that the input is processed so that the output becomes the same.

In this embodiment, 100 time-series data x
1, x2, x3, to x98, x99, x100, x
Since 1 to 30 are used as learning data and the remaining 70 x31 to x100 are predicted, the time series up to 5 hours before is taken as input information. The learning data is (x1, x2, x3, x4, x5) → x6 (x2, x3, x4, x5, x6) → x7 (x3, x4, x5, x6, x7) → x8 (x4, x5, x6). , X7, x8) → x9... (X24, x25, x26, x27, x28) → x29 (x25, x26, x27, x28, x29) → x30.

That is, the time series data x1, x2, x3
, X4, x5 are converted into parallel data, input to the corresponding one of the neural circuit elements E11, E12, E13, E14, E15 of the input layer I, processed by the intermediate layer LM, and processed by the output layer LO to obtain the time series data x6. Is output as

That is, the time series data x1 is input to the neural network element E11, and the time series data x2 is input to the neural circuit element E12.
, The time-series data x3 is input to the neural circuit element E13, the time-series data x4 is input to the neural circuit element E14, and the time-series data x5 is input to the neural circuit element E15. Is input to the corresponding element of the neural circuit element and processed, as a result, time-series data x5
X6, which is time series data at the next time point, is obtained and output from the output layer LO.

Next, the time series data x2, x3, x4, x
5 and x6 are converted into parallel data and input to the corresponding one of the neural circuit elements E11, E12, E13, E14 and E15 of the input layer I, processed by the intermediate layer LM, and time-series data x6 from the output layer LO. Is output.

That is, the time series data x2 is inputted to the neural network element E11, and the time series data x3 is inputted to the neural network element E12.
, The time-series data x4 is input to the neural circuit element E13, the time-series data x5 is input to the neural circuit element E14, and the time-series data x6 is input to the neural circuit element E15. Is input to the corresponding element of the neural circuit element and processed, as a result, time-series data x6
X7, which is time series data at the next time point, is obtained and output from the output layer LO.

Next, the time series data x3, x4, x5, x
6, x7 are converted into parallel data, input to the corresponding one of the neural circuit elements E11, E12, E13, E14, E15 of the input layer I, processed by the intermediate layer LM, and output from the output layer LO as time-series data x8. Is output.

That is, the time series data x3 is input to the neural network element E11, and the time series data x4 is input to the neural circuit element E12.
, The time-series data x5 is input to the neural circuit element E13, the time-series data x6 is input to the neural circuit element E14, and the time-series data x7 is input to the neural circuit element E15. Is input to the corresponding element of the neural circuit element and processed, as a result, time-series data x7
X8, which is time series data at the next time point, is obtained and output from the output layer LO.

Similarly, the time series data x25, x26, x
27, x28, x29, these time series data x25,
x26, x27, x28, x29 are converted into parallel data and input layer I
Is input to a corresponding one of the neural circuit elements E11, E12, E13, E14, and E15, processed by the intermediate layer LM, and output to the output layer LO.
Is output as time series data x30.

That is, the time series data x25 is input to the neural network element E11, and the time series data x26 is input to the neural circuit element E12.
, The time-series data x27 is input to the neural circuit element E13, the time-series data x28 is input to the neural circuit element E14, and the time-series data x29 is input to the neural circuit element E15. Is input to the corresponding element of the neural circuit element and processed, and the time series data x29
X30, which is time series data at the next time point, is obtained and output from the output layer LO.

Therefore, data from x1 to x30, which are known data, are used as learning data, and 5 data are sequentially assigned from x1.
When the future data for one data of the own set is obtained for 25 sets which are shifted by one data in time series for each set as one set, 25 kinds of outputs from x6 to x30 are obtained. Learning to the neural network can be performed by correcting the load values and bias components so that the values of the 25 types of outputs x6 to x30 become the values of the known data x6 to x30 used for learning.

In learning, as described above, known input data is given to the neural network, and the weight value and the bias component of the neural circuit element are corrected stepwise so that desired output data corresponding to the input data is obtained. I do.

The weight value and bias component of the neural circuit element added to the data are corrected stepwise so that the output value at the time of inputting the input data coincides with the desired output data. (1) Correction processing of only the weight value for individual neural circuit elements, (2) Correction processing of only bias components for individual neural circuit elements, (
3) Correction processing of both the weight value and bias component for each neural circuit element can be performed, and at least two of these correction processing modes are used to prepare the prepared learning data. The mode is switched and the correction processing is performed so that expected output data can be obtained by input.

In the learning, back propagation is performed. You
That is, the output value of the neural circuit element Ei is represented by o_i , Neural circuit element
The bias value of child Ei is h_i , Input to neural circuit element Ei
U is the sum of the values_i From the neural circuit element Ej to the neural circuit element E
The load value of the connection toward i_ij, Learning data (I:
T) (where I is the input, T is the desired output), load
Weight w_ijΔw_ijThe via of the neural circuit element Ei
Value h_i Δh_i And the error of the neural circuit element Ei
δ_i , When the output function is f (x), the neural circuit element
Output value o of Ei_i Is when the neural circuit element Ei is an input element
Then o_i = I_i ... (1) When the neural circuit element Ei is not an input element, o_i = F (u_i ) (2) Here, the sum u of the input values to the neural circuit element Ei
_i Is It is expressed as Also, the error δ of the neural circuit element Ei_i The god
When the transit element Ei is an output elementIf the neural circuit element Ei is not an output elementAnd the load value w_ijCorrection amount Δw_ijIs Δw_ij= −αδ_i o_j + ΒΔw_ij ... (6) Bias value h of neural circuit element Ei_i Correction amount Δh_i Is Δh_i = −αδ_i + ΒΔh_i ... (7) (where α and β are positive constants)
By performing arithmetic processing on the condition using the equation,
Load value of the connection from the road element Ej to the neural circuit element Ei
w_ijLoad correction amount Δw for_ijAnd the neural circuit element Ei
Bias value h_i Correction amount Δh for_i Seeking
Confuse. Then, thereafter, the obtained load value correction amount Δw
_ijAnd the bias value correction amount Δh_ij ← w_ij+ Δw_ij … (8) h_i ← h_i + Δh_i (9) By performing the following processing, the neural circuit element Ej
The load value w of the connection toward the circuit element Ei_ijOr neural circuit
The bias value h of the child Ei_i To update. Such processing
Error Err

[0043]

(Equation 1) Is repeated until is sufficiently small. This is learning by back propagation.

Thus, for each neural circuit element Ei of the neural network, the weight value w _ij and the bias value h
_i can learn. It should be noted here that, as can be seen from the above equations, each neural network element E of the neural network
For i, refers the load value w _ij individually, and a point referred to be able to learn the bias value h _i, learning and load value w _ij only, it is possible to learn only the bias values h _i Is a point.

As a result, each of the neural network elements can be corrected independently, and only one of them can be corrected without interlocking the bias value h and the load value w. The bias value h and the weight value w of the desired neural circuit element can be corrected without giving otherwise,
Subtle corrections can be made, and the prediction accuracy can be improved.

After learning is completed for each neural network element of the neural network, the process proceeds to time series prediction. Prior to that, new data similar to the learned data is given to the neural network. In addition, re-learning is performed, and in this re-learning, only the bias value or only the load value is corrected.

That is, according to the present invention, new data similar to the already-learned data is added to the neural network which has been trained by the back propagation as described above, and re-learning is performed. Do.

The re-learning is performed by giving new data similar to the already-learned data to the neural network for which learning has been completed, and at the time of the re-learning, only the bias value is corrected or only the weight value is changed. Make corrections.

The processing procedure of re-learning and time series prediction will be described. [Step 1] First, t ← 31 (31 for t)
Is substituted). This is because 30 points of time are used for learning, and the measurement is performed using the data at the subsequent points in time.

[Step 2] Next, the following five of the time-series data are used as fine adjustment data. ( _Xt-10 , _xt-9 , _xt-8 , _xt-7 , _xt-6 : _xt-5 ) ( _xt-9 , _xt-8 , _xt-7 , _{xt- 6} , _xt-5 : _xt-4 ) ( _xt-8 , _xt-7 , _xt-6 , _xt-5 , _xt-4 : _xt-3 ) ( _xt-7 , x _t-6 , _xt-5 , _xt-4 , _xt-3 : _xt-2 ) ( _xt-6 , _xt-5 , _xt-4 , _xt-3 , _xt-2 : x _t-1 ) [step 3] Next, only the bias value h is corrected for such five types of fine adjustment data. this is,
Equations (1), (2), (3), (4), (5),
The correction amount Δh of the bias value h is calculated using Expression (7),
Only the bias value h is calculated by the equation (9) to obtain the correction amount Δ
h, which is a process of correcting, and the process here is
This is a fine adjustment process by re-learning using data near the data used in the previous learning.

[Step 4] Next, with respect to such a finely adjusted neural network, the fine adjustment data (x _t-5 ,
_xt-4 , _xt-3 , _xt-2 , _xt-1 ), and calculates a predicted value _xt 'at time t.

[Step 5] Next, if the observed value x _t at the time _t is obtained, it is added to the database. [Step 6] When the prediction is continued, the process returns to step 2 with t ← t + 1. Otherwise, end.

In the re-learning process using the new data selected from the already-learned data, when the current time is t, the input data is time-series data up to time t-1, and the output data is When the data is after t (that is, data of the time series prediction), a learning method of correcting both the weight value and the bias component is applied to the past data, and a future time series is obtained from the current input data. At the time of prediction, the bias component is corrected by a learning method of correcting only the bias component using past data near the prediction time. In this way, fine adjustment can be efficiently performed on the learned neural network. Then, after completing such re-learning, the process proceeds to time-series prediction.

FIG. 3 shows the result of the above-described method for predicting the time-series data. That is, as shown in FIG. 3, in the case where only the bias value at the time of fine adjustment is corrected, the relative error average is “4.00” [%] and the relative error variance is “0.
0062 ”, and in the case of correcting both the load value and the bias value at the time of fine adjustment, the average relative error is“ 5.17 ”[%].
The relative error variance is “0.0112”, and in the case where fine adjustment is not performed, the relative error average is “18.57”.
In [%], the relative error variance was “0.0992”.

Here, the relative error is | x _i ′ −x _i | / x _i
Calculated by Also, the closer the error average is to zero and the smaller the variance, the higher the prediction accuracy. Therefore, it can be understood that the prediction accuracy is remarkably improved by adopting a method of correcting the bias value and the load value without interlocking as proposed in the present invention.

As described above, according to the present invention, the input data and the desired output with respect to the input data are provided for the neural network formed by connecting a plurality of neural circuit elements for performing required processing on the input data and outputting the processed data. A method for learning a neural network, in which, by providing data, a weight value and a bias component of a neural circuit element added to data in a stepwise manner so that an output value at the time of input data input matches the desired output data. In, the correction process of only the load value for the individual neural circuit element, the correction process of only the bias component for the individual neural circuit element, the correction process of both the load value and the bias component for the individual neural circuit element,
In addition, at least two of these correction processes are used, and the correction processes are switched to be performed so that expected output data can be obtained by inputting the prepared learning data. Further, the present invention provides new data similar to the already-learned data to the neural network for which learning has been completed, performs re-learning, and corrects or corrects only the bias value during this re-learning. Only the load value is corrected.

When a correction to the neural network system is necessary, in the conventional method, a numerical value is uniformly added to or subtracted from the output of the neural network, or the entire neural network is re-learned with data near the predicted time. In such a method, if the fluctuation component is not a constant, the correction is insufficient, and the characteristics of the entire neural network are changed by the neighboring data, so to speak, the correction is excessive. However, in the present invention, the learning of the load value and the bias value can be performed separately, and the learning of the load value and the bias value can be switched at any time. Learning efficiency can be improved, and new data similar to the previously learned data is added to the neural network for which learning has been completed, and re-learning is performed. At this time, by correcting only the bias value or correcting only the load value, it is possible to fine-tune the mapping without greatly destroying the characteristics of the mapping realized by the neural network. It becomes something.

Therefore, according to the present invention, it is possible to correct only one of the bias value and the load value without interlocking the bias value and the load value. It is possible to provide a method for learning a neural network, which allows fine adjustment to be performed, thereby improving prediction accuracy. Note that the present invention is not limited to the above-described embodiment, and can be implemented with various modifications.

[0059]

As described above, according to the present invention, it is possible to correct only one of the bias value and the load value without interlocking the bias value and the load value. Can be effectively fine-tuned without greatly destroying it, and a neural network system capable of improving prediction accuracy can be provided.

[Brief description of the drawings]

FIG. 1 is a block diagram showing an overall configuration of a neural network as an example used in the present invention.

FIG. 2 is a diagram showing time-series data as an example used in an embodiment of the present invention.

FIG. 3 is a neural network for explaining a specific example of the present invention.

FIG. 4 is a diagram showing prediction results obtained by a neural network as a result of applying the method of the present invention.

FIG. 5 is a diagram for explaining a nerve cell model constituting a neural network.

[Explanation of symbols]

LI ... input layer LM ... intermediate layer LO ... output layer E11, ~E1m, E21, ~E2n, E3 ... bias value u _i of the output values h _i ... neural elements Ei of neural elements o _i ... neural elements Ei ... sum w _ij ... load value of the connection from the neural elements Ej towards neural elements Ei Δw _ij ... w _ij correction amount Δh _i ... h _i of the correction amount [delta] _i ... neural elements of input values to the neural elements Ei Error of i f (x) ... output function

Claims (5)

[Claims] An input data and a desired output data corresponding to the input data are given to a neural network formed by connecting a plurality of neural circuit elements for performing required processing on the input data and outputting the input data. A neural network learning method for gradually correcting a weight value and a bias component of a neural circuit element added to data so that an output value at the time of input data input matches the desired output data. Correction processing of only the load value for the element, correction processing of only the bias component for the individual neural circuit element, and correction processing of both the load value and the bias component for the individual neural circuit element are enabled, and these corrections are performed. Using at least two of the processes, cut the correction process so that the expected output data can be obtained by inputting the prepared learning data. Learning neural network, characterized in that Ete implemented. 2. The neural network learning method according to claim 1, wherein new data similar to the already-learned data is provided to the neural network for which learning has been completed, and re-learning is performed. A method for learning a neural network, comprising correcting only a bias value or correcting only a weight value during learning. 3. The neural network learning method according to claim 1, wherein when the current time point is t, the input data is time-series data until time t-1, and the output data is data after time t. , A learning method that corrects both the weight value and the bias component is applied to past data, and when predicting a future time series from the current input data, the past time near the prediction time is used. A method for learning a neural network, comprising: correcting a bias component by a learning method of correcting only a bias component using the data of the above; 4. The method for learning a neural network according to claim 1, wherein a backpropagation algorithm is used for calculating the amount of correction of the weight value and the bias component. 5. The neural network learning method according to claim 1, wherein learning data given in advance is learned by a method of correcting both a weight value and a bias component, and a new input data is given. When calculating the output with respect to the data x, one that is similar to the input data x is selected from the learning data, and the bias value is corrected by a learning method in which only the bias value is corrected by using the similar data. A method for learning a neural network, comprising calculating an output value for input data x.