JPH10254846A - Learning method for regression-type neural network - Google Patents

Abstract

PROBLEM TO BE SOLVED: To suppress drop to a local minimum and to execute a stable learning operation with less errors by correcting a connection weight parameter in a direction where the errors are reduced the most, while a learning coefficient is adjusted so as not to increase the errors, from a stage where the errors are reduced to the value of not more than a reference value. SOLUTION: A neural network initialization part 7 constitutes a neural network from the parameter of the number of elements or the like, initializes connection weight by random numbers and sets a neuro gain parameter β and the learning coefficient η. Then, a βvalue deciding part 9 decides the value of the neuro gain parameter β. When all time sequential data are inputted, a steepest drop direction calculation part 11 calculates the steepest drop direction. A η value deciding part 10 decides the value of the learning coefficient ηand a connection weight matrix correction part 12 corrects connection weight, based on the decided learning coefficient η and the steepest drop direction.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention estimates the input / output relationship from specific input / output time-series data as a repetitive mapping function having an external input using a recursive neural network, and performs time-series pattern recognition. The present invention relates to a learning method of a recurrent neural network in which a neural network stores and learns time-series data in a technology for performing voice recognition, grammar analysis, finite state machine design, and the like.

[0002]

2. Description of the Related Art A neural network, also called a neutral network, is an information processing device devised by modeling a network of brain cells of an organism. The most commonly used neural networks are briefly described.

[0003] The neural network of the biological brain is considered to be a system composed of a number of nerve cells and connections between the elements.

[0004] Each neural element outputs an impulse according to its membrane potential. In a model in which the output of each element takes a continuous value, the output value represents the pulse density. In addition, since the membrane potential increases upon receiving an impulse from another element, it is considered that the membrane potential can be calculated by a weighted sum in consideration of the pulse density and the coupling efficiency with the element. The output value monotonically increases with an increase in the membrane potential, but the pulse density has an upper limit and the output value is saturated.

From these facts, if the output value of the i-th neural element at time t is S _i (t), the membrane potential is h _i (t), and the coupling efficiency from element j to element i is w _ij , In a discrete-time model,

(Equation 1) S _i (t + 1) = σ (h _i (t)) (2) Here, σ is a sigmoid function, specifically, a logistic function,

(Equation 2) Is most often used. Also, for the future,

(Equation 3) Is introduced as follows. This is called a neurogain parameter. In particular, when there is no description of β, β = 1.0.

[0006] An input is externally supplied to some or all of the neural elements of the neural network. This part is called an input part or input layer. Some or all of the neural elements output to the outside or are observed from the outside. This part is called an output unit or an output layer.

[0007] A feedforward type neural network has only one connection for transmitting a signal from the input layer to the output layer. On the other hand, a recurrent neural network that has a coupling in a feedback direction and can loop a signal flow is called a recurrent neural network.

Consider a neural network composed of three layers: an input layer, a middle layer, and an output layer. As shown in FIG. 5, the regression type structure has a structure in which an element in the input layer is coupled to an element in the intermediate layer, and a element in the intermediate layer is coupled to an element in the output layer and coupled to an element in the intermediate layer. The neural network is called an Elman net and is one of the recurrent neural networks having a widely used structure.

The set of elements in the input layer is I, the number of elements N _I , the set of elements in the intermediate layer is U, the number of elements N _U , the set of elements in the output layer is O, and the number of elements N _O. The total number of elements is N. When the output values of the output element, the intermediate element, and the input element are collectively expressed as a vector, they are N _O -order vector S ^O , N _U -order vector S ^U , and N _I -order vector S ^I , respectively. The vector is an N-order vector S.

[0010]

(Equation 4) Here, the connection weight from the j-th element in the intermediate layer to the i-th element in the output layer is w _ij ^OU, and the connection weight from the j-th element in the intermediate layer to the i-th element in the intermediate layer is w _ij and ^UU, the connection weight to the i-th element of the intermediate layer and w _ij ^UI from the j-th element of the input layer. At this time, w _ij , w _ij ^OU ,
w _ij ^UU, w _ij ^UI, respectively _{(N O + N U) ×} (N U +
N _i ), N _o × N _u , N _u × N _u , N _u × N _I matrix (W, W ^OU , W ^UU , W ^UI ) (i, j) elements, and their relationships are as follows: become.

[0011]

(Equation 5) Since there is no direct coupling from the input layer to the output layer, the matrix upper right is the zero matrix of N _O × N _I. From the above, the expression of the time evolution of the neural network is written as follows.

[0012]

(Equation 6) Here, σ is a vector function in which each element is calculated by a sigmoid function.

Next, learning of the neural network will be described. These neural networks are capable of approximating the system that realizes the relationship between this input and output value pair from several input value ξ (t) and output value ζ (t) pairs. . The kind of system that can be approximated depends on the configuration of the neural network, such as how elements are connected.For example, in a feedforward neural network, a mapping from a multidimensional real number function or feature space to a variable representing a classification is performed. Functions and regression-type neural networks can approximate mapping functions from dynamical systems and time-series data to variables representing classifications. In particular, recurrent neural networks are used when dealing with series data when the data changes over time.

In a neural network, approximation of various systems is realized by setting _wij , which is the coupling efficiency between elements, as a parameter and setting this value to an appropriate value.

However, since the neural network is a non-linear system, it is generally not possible to obtain optimum parameters at once. For this reason, the normal parameters w _ij are determined sequentially. This is called "learning".

A given set of input / output values is called learning data.

｛Ξ ^(k) (1), ξ ^(k) (2),…, ξ ^(k) (t)｝ (8) ｛ζ ^(k) (1), ζ ^(k) (2), .., Ζ ^(k) (t)｝ (9) where k = 1, 2,..., P Here, P sets of learning data are given.
Ξ ^(k) (t) and ζ ^(k) (t) are N _I order and N _O , respectively.
The next vector.

[0018] _{ξ (t) = (ξ 1} (t), ..., ξN O (t)) here 'ζ (t) = (ζ 1 (t), ..., ζN I (t))', ' is Represents transposition. The input learning data is given to the neural network.

[0019]

(Equation 7) Or, when written together:

[0020]

(Equation 8) The output value S ^{O (k)} (t) of the neural network and the output value of the learning data ζ
The difference between ^{(k) and} (t) is called an error. The sum of squares E of all errors is determined as follows. Let S ^{O (k)} (t) be simply written as S ^(k) (t).

[0021]

(Equation 9) When the error E becomes 0, it means that the input / output correspondence of the learning data has been completely acquired. Therefore, the problem is how to correct w _ij and reduce the error E. The most commonly used method is the steepest descent method, which corrects the direction in which the error E is reduced most, that is, the steepest descent w _ij . Since the steepest descent direction is determined by partially differentiating E with w _ij, correction range [Delta] w _ij of w _ij is

(Equation 10) Becomes η is called a learning coefficient and is a positive coefficient for stabilizing learning. How to calculate the ∂E / ∂w _ij are known some following two typical methods.

First, a real-time recurrent learning (RTRL: Re
al Time Recurrent Learning). Hereinafter, (k) representing the learning data number is omitted, but the final Δw _ij may be the sum of the correction values of w _{ij in} each learning data.

From equations (15) and (16),

[Equation 11] Becomes Here, from equations (13) and (14),

(Equation 12) And p _pq ⁱ (t) can be obtained sequentially. However, the initial condition is

(Equation 13) It is.

From the sequentially obtained p _pq ⁱ (t), Δw _ij (t) can be calculated. Thus, in the RTRL, when time-series data is given, a correction value of the connection weight can be calculated at that time,
The feature is that learning can be advanced.

Next, another calculation method for obtaining the steepest descent method is a back propagation through time (BPT).
T: Back Propagation Through Time) will be specifically described. Here, (k) representing the learning data number is omitted.

From equation (15),

[Equation 14] Becomes here,

(Equation 15) Then, equation (24) can be written as follows.

[0027]

(Equation 16) z _i (τ) can be obtained as follows.

[0028]

[Equation 17] As described above, in BPTT, z _i (t) is calculated in the time reverse direction to obtain Δw _ij . However, the termination condition is z _i (T) = e _i (T) (30).

Next, learning of a finite automaton will be described as an example of learning the time series data of the dissolution time and discrete values. First, the definition of the Moore finite automaton M is as follows.

[0030] M = (X, Y, S , f s, f o, s o) · X: input symbol set · Y: Output symbol set · S: state set · s _o ∈S: initial state · f _s: X × S → S state transition function f _o : S → Y output function Suppose that an input symbol x is given to a finite automaton whose current state is s. At this time, the finite state automaton state in accordance with the state transition function _{f s, s'= f s (} s, x) transitions to further outputs the output symbols _{y = f o (s')} . That is, in response to the input of the symbol, the automaton changes the state and outputs the symbol according to the state. In this way,
Given an input symbol sequence, the state transition and symbol output are repeated, and as a result, the finite state automaton returns an output symbol sequence.

In the learning of the finite state automaton, the input symbol and the output symbol are converted into a continuous value vector into an input vector input to the input layer and an output vector corresponding to the output of the output layer, respectively. In the learning of the finite automaton, learning is performed on these input / output vector sequences.

When the learning is successful, the trajectories of the output values of the elements of the intermediate layer are concentrated in a non-connected area in the phase space.
That is, it is known that some clusters are formed. Further, since this cluster corresponds to the “state” in the finite state automaton, it is possible to reconstruct the learned finite state automaton.

[0033]

In the steepest descent method in which a parameter is corrected in a direction in which an error is reduced to a minimum, once a valley of a local minimum value which is not the minimum value falls,
There is a problem that it is not possible to escape from there and to reach the overall minimum value. Note that a local minimum value is called a local minimum, and a minimum value of the entire area is called a global minimum.

Further, even in a neural network that has acquired the input / output relationship of learning data, stable automaton-like operation is not guaranteed, and there is a problem in that unstable behavior is exhibited for long series of data.

Further, when the neurogain parameter β is large, the output of the element may change sensitively with respect to learning, that is, the change of w _ij , and there is a problem that the error increases rapidly on the way.

The present invention has been made in view of the above,
An object of the present invention is to provide a learning method for a recurrent neural network that can suppress a drop to a local minimum, have a small error, and perform a stable learning operation.

[0037]

In order to achieve the above object, according to the present invention, there is provided an input time series data of discrete time and discrete value and a target time of discrete time and discrete value corresponding to the data. When several sets of series data are given, input time series data is input to a discrete time, continuous value regression type neural network, and an error between actual output data of the neural network and target output time series data. A regression-type neural network learning method in which the connection weight parameter is successively corrected in the steepest descent direction of the error plane so as to reduce and the function between the input and output time series data is obtained. From the stage where the error decreases below the reference value, the learning proceeds while increasing the neuro gain parameter so that the error does not increase, and the error decreases most while adjusting the learning coefficient so that the error does not increase. And summarized in that to correct the connection weights parameter countercurrent.

According to the first aspect of the present invention, the learning is advanced to some extent and the learning is advanced while increasing the value of the neurogain parameter from the stage where the error becomes equal to or less than the reference value. The increase width of the neurogain parameter and the value of the learning coefficient are adaptively adjusted so that the error does not increase beyond the reference value due to the increase and the modification of the coupling matrix.

[0039]

DESCRIPTION OF THE PREFERRED EMBODIMENTS Before describing the embodiments of the present invention, basic matters and principles relating to the present invention will be described.

The error E is a function of the connection weight w _ij of the neural network. Therefore, if the error E is regarded as a function using the connection weight w _ij as a variable, y = E ({w _ij | i, j = 1,... N}) (31) forms a plane, which is called an error plane.

In the steepest descent method, ∂E / ∂w _ij is calculated, and this vector value indicates the direction in which the slope of the slope of the error plane is the steepest. That, w _ij from w _ij initial values in the steepest descent method in the steepest direction of the error plane
Will move.

When the neurogain parameter β is 0,
The error plane is a flat plane having a slope of 0. Because σ
This is because β = o (x) = １ ／, and the output value of the element is always constant regardless of w _ij .

When the neurogain parameter β is ∞, the error plane is a fine stepped surface. Because the sigmoid function becomes a complete {0,1} step function,
Also takes discrete values.

When the value of β is increased from 0,
Although the shape of the error plane was flat at first, it gradually has a mountain-valley structure with a gentle inclination. Furthermore, as the value of β is increased, the slope increases and a steep mountain-valley appearance is exhibited.
Finally, β = 、, and it becomes a step-like surface of a cliff.

From these facts, when β is large, learning does not proceed on a flat ground portion, and on a steep slope at the boundary of a flat ground, the change in the output value of the element becomes sensitive to the change in w _ij , so that learning is difficult. You can see that there is. For a relatively small β, the error plane is a gentle slope, and learning can be expected to proceed reliably. Next, the relationship between the neurogain parameter β and the behavior of the neural network will be considered. The operation of the neural network as an automaton stabilizes when the neurogain parameter β becomes larger than a certain value β _O , so that a stable state transition and symbol output can be obtained for an input symbol string of any length. .

Here, the reason why the automaton operation is performed stably will be described. First, some words and symbols are defined.

Considering a N _U dimensional Euclidean space R ^NU , the output vector of the intermediate element can be regarded as a point on R ^NU . However, 0 because it is ^{_{≦ S i O ≦ 1, N}} U dimensional unit hypercube ([0,1] Nu) only present in the. It should be noted that, for the sake of simplicity in the future, S ^O is to be written as to omit the O S.
Let the vertex of this hypercube be v, ie v = (v ₁ ,
.., V _NU ) and v _i {0, 1}. Also, a set of all vertices is set to V (= {0, 1} Nu).

A coupling matrix W and a finite number of input vectors ξα
(Α = α ₁ ,..., Α _p ) is given. Plane on R ^NU

(Equation 18) think of. This will be referred to as _Hi α. R ^NU is divided into subspaces P _v, α by these planes H _i α (α = α ₁ ,..., Α _{p, i} = 1,... N ^U ).

[0049]

[Equation 19] Next, D _v is defined as follows.

Dv = fβ ( _Pv, α, α) (33) where fβ (S, α) has the following meaning.

[0051] fβ (S, α) = σβ ((W UU S + W UI ξα)) (34) D v is trying to note that does not depend on α and β. S∈P
Let us show that when _{v and} α, the range of fβ (S, α) does not depend on α or β. The mapping destination S ′ of S is

(Equation 20) From equation (32), when v _i = 0,

(Equation 21) 0 <(S ′) _i <１ ／. When v _i = 1,

(Equation 22) And 1/2 <(S ′) _i <1. Therefore, it can be seen that the mapping destination of S is a region independent of α and β. That is,

(Equation 23) It is.

F: {V × X} → V is called an inter-vertex transition function. When there exists an input symbol sequence α ₁ α ₂ ,..., Α _n that can reach v _q from v _p by F, v _p
And v _q are connected. A set of connected vertices is defined as V _c (⊆V). S∈D _v against g (S) = v
And a function g are defined. Specifically, F is defined as follows using g.

[0053] F (v, α) = gofβ (v, α) (37) a N _v is defined as follows.

N _v = D _v ∩P _{F (v,} α ₁₎ , α ₁ ∩... ∩P _{F (v,} α _P) , α _P (38) F (v, α
_As mapped to _i ), PF _(v, α ₁₎ , α ₁ means the area mapped to D _{v ′} by fβ.

That is, if N _v = 0, then gof (N _v , α _i ) = F (v, α _i ) where i = 1,..., P (39).

[Theorem] For all elements v of V _c , v
If ∈N _v ^c (N _v ^c is a region including a boundary of N _v.) Is, there are beta ₀ finite value as follows. When beta is any value or ^{_{β 0, fβα n o ... ofβα}} 2 ofβα 1 (D vo) ⊂N v1 'v 1 = Fα n o ... oFα 1 oFα 1 (v 0) (40) any v _o ∈V _c , input symbol string α1 α2 of arbitrary length
... holds true for α∈X ^* . Here, Fα (V) and f
βα has the same meaning as f (v, α) and fβ (S, ξα), respectively. Let β ₀ be called the critical neurogain parameter.

[Proof] For all input symbols α∈X, that is, for all types of input vectors ξα, a pusher is given as fβα (N _v ) ⊆TF _(v, α) ⊆NF _(v, α). It is shown that the following hypercube T _v (v∈V _c ) can be formed by the condition of β.

Assume that the vertices v ₀ → v ₁ → v ₂ →... Transition with respect to the input symbol strings α 1 α ₂ . At this time, if T _v as described above can be constructed,
The state vector created by the output vector of the intermediate element of the neural network goes through N _v0 , N _v1 , N _v2 ,.
Make a stable state transition.

For S and the input vector ξα,

(Equation 24) Is defined.

[0060] when the _{S∈N v, fβ (S, ξα} ) ∈D F (v,
α ₎ , if (F (v, α)) _i = 1, then _hi α> 0, and if (F (v, α)) _i = 0, then h
_i α <0. Therefore, it can be seen that when β approaches infinity, σβ (h _i α) approaches 1 and 0, respectively.

The vertex of N _v is z _k (k = 1,..., K), and φ _{v, i} and ψ _{v, i} are defined as follows.

[0062]

(Equation 25) T _v can be determined as follows using φ _{v, i} and ψ _{v, i} .

[0063]

(Equation 26) Note that T _v ⊆N _v .

[0064] In addition, as shown in the following β _i ^v, to define the α.

For each v, α, i and all S∈N _v ,

[Equation 27] Let β _i ^v, α be the minimum β that satisfies.

Here, V ′ = F (v, α). Here, since N _v is the convex space, the maximum value and the minimum value of h _i alpha occurs on the vertices of S is N _v. Thus, it is not necessary to examine all of S∈N _v, it is sufficient to examining whether the condition is satisfied only all vertices. Β ^v, α = max _i β _i ^v,
If α is set, it can be seen that fβ (S, ∈α) _' T _{v ′} (46) holds for β ≧ β ^v, α. Therefore,

[Equation 28] By selecting β ₁ as follows, it can be seen that fβ ₁ (N _v, α) is a subset of Tv ′ for all v∈V _c and α∈X.

Therefore, if β larger than β1 determined as described above is selected, for any S ₀ ∈N _vo , T _v1 ∋fβα (S ₀ ) = S ₁ , T _v2 ∋fβα (S ₁ ) S ₂ , T _v3 ∋fβα (S ₂ ) = S ₃ ,..., And for any input symbol string, the trajectory is always mapped into N _v , and the critical neurogain parameter β ₀ (≦ β ₁ ) exists, that is, the theorem has been proved.

In view of the above, first, the neurogain parameter β is set to a certain small value, and _wij is corrected so that the error becomes a minimum value. Efficient learning on gentle slopes and avoidance of local minimum can be expected.

Next, when β gradually increases and reaches β ₀ , a stable state transition can be obtained. When β is large, the output value of the element changes sensitively to the change in the connection weight, and the error may increase rapidly. However, since η is adaptively determined here, the correction width of the connection weight is carefully adjusted, and the error does not increase sharply.

Next, an embodiment of the present invention will be described with reference to the drawings. FIG. 1 is a block diagram showing a configuration of a time-series data learning device that implements a learning method for a recurrent neural network according to an embodiment of the present invention. In FIG.
Reference numeral 1 denotes a memory for storing parameters, data, results during calculation, etc., 2 a data input unit for inputting learning data, 3 a data output unit for outputting an output value of an output element, 4
Is a neural element value storage unit for storing values of neural elements, 5 is a connection weight storage unit for storing connection weight values, 6 is a control unit for controlling the operation of each unit, and 7 is a neural network based on parameters. , A neural network initialization unit that initializes connection weights by random numbers, 8 calculates an element time evolution, and updates an element value. 9 determines a value of a neurogain parameter β. A β value determination unit 10, a η value determination unit 10 for determining the value of the learning coefficient η, a steepest descent direction calculation unit 11 for calculating the steepest descent direction, and a connection weight matrix correction unit 12 for correcting the connection weight.

Next, the operation of the time-series data learning device configured as described above will be described with reference to the flowchart shown in FIG.

First, the neural network initializing unit 7 forms a neural network from parameters such as the number of elements, initializes connection weights with random numbers (step S21), and sets the neurogain parameter β and the learning coefficient η to initial values. It is set (step S22). Note that the initial value of the neuro gain parameter β is set to a small value.

Next, the value of the neurogain parameter β is determined by the β value determining section 9 (step S23). The process of determining the value of the neuro gain parameter β will be described later in detail with reference to FIG. Then, the element values are initialized (step S24). Time series data is input from the data input unit 2 to the elements of the input layer (step S2).
5). The element value is updated based on the time evolution formula (step S26). Then, it is checked whether or not all the patterns have been completed, that is, whether or not all the time-series data has been input (step S27).
Returning to 24, the process is repeated until all the time-series data is input. If all the time-series data has been input, the process proceeds to step 28, where the steepest descent direction calculation unit 11 calculates the steepest descent direction. .

Then, the value of the learning coefficient η is determined by the η value determination section 10 (step S29), and the connection weight is corrected by the connection weight matrix correction section 12 based on the determined learning coefficient η and the steepest descent direction. (Step S30). Then, it is checked whether or not the error is within an allowable range (step S31).
The process ends, but if not, step S2
3 and the same process is repeated.

Next, the process of adaptively determining the value of the neurogain parameter β in step S23 of FIG. 2 will be described with reference to the flowchart shown in FIG. In this process, the maximum error is

(Equation 29) It can also be used to determine whether all learning data has been learned correctly. If this value is smaller than 0.5, it means that all learning has been completed correctly.

In the process of FIG. 3, first, it is checked whether or not the maximum error Emax is less than 0.5 (step S4).
1) If it is less than 0.5, the value of the neurogain parameter β and the increase width Δ are initialized to β ₀ and Δ ₀ respectively (step S43), and if not less than 0.5, the process proceeds to step S42. Then, the neuro gain parameter β is output as it is, and the process is terminated.

Then, the maximum error E (β) becomes E (β +
Δ) is checked (step S4).
4) If it is larger, the value and increase width of the neurogain parameter β are increased to β ← β + Δ, Δ ← 2Δ (step S45). Then, the maximum error E (β) is E (β +
Δ) or not (step S4)
6). If the value is smaller, it is determined that the value of the neurogain parameter β has been determined, the value of the neurogain parameter β is output, and the process ends (step S47). If the maximum error E (β) is not smaller than E (β + Δ),
Returning to step S45, similarly, the value and the increase width of the neuro gain parameter β are increased to β ← β + Δ, Δ ← 2Δ, and the same processing is repeated.

If it is determined in step S44 that the maximum error E (β) is equal to or smaller than E (β + Δ), the process proceeds to step S48, and the increment Δ of the neurogain parameter β is reduced to Δ / 2. And the maximum error E (β)
Is greater than or equal to E (β + Δ) (step S49), and if so, the neurogain parameter (β + Δ) is output, the value of the neurogain parameter β is determined, and the process is terminated (step S50). ).

If it is determined in step S49 that the maximum error E (β) is equal to or smaller than E (β + Δ), the flow advances to step S51 to determine whether or not the number of processes in steps S48 and S49 exceeds a predetermined number. Check. If not, the process returns to step S48 to repeat the same processing. If it does, the process proceeds to step S52, where the value of the neurogain parameter B ₀ is output, and the process ends (the value of β Decision).

Next, the processing for adaptively determining the value of the learning coefficient η in step S29 of FIG. 2 will be described in detail with reference to the flowchart shown in FIG. In this process, E (η) refers to the sum of squares E (Equation (15)) of all errors calculated after changing the connection weight with the learning coefficient η.

In the process of FIG. 4, first, the value of the learning coefficient η is set to η
Initialized to ₀ (step S61). Then E (0)
Is smaller than E (η) (step S).
62) If it is smaller, the value of the learning coefficient η is reduced to η / 2 (step S66), and it is checked whether E (0) is larger than E (η) (step S67). If it is larger, the process proceeds to step S68, where the value of the learning coefficient η is output, and the process ends (determination of the value of η). E (0) is E
If (η) or less, the process returns to step S66, where the learning coefficient η is reduced to η / 2, and the same processing is repeated.

On the other hand, in the process of step S62, E
If (0) is equal to or larger than E (η), the process proceeds to step S63, and it is checked whether E (η) is smaller than E (2η). The value of η is increased to 2η, and the same process is repeated. If the value is small, however, the process proceeds to step S64, where the learning coefficient η is output, and the process is terminated (determination of the value of η).

[0083]

As described above, according to the present invention,
In the initial stage of learning the time series data of the neural network with regression coupling, the use of a small neuro-gain parameter β minimizes the drop to the local minimum, enabling efficient learning. From a stage where the learning has progressed to some extent, learning is performed while gradually increasing the neurogain parameter β, and the neural network can acquire a stable state transition. Furthermore, stable learning can be performed by adaptively adjusting the neurogain parameter β and the learning coefficient η so as not to destroy the input / output relationship once obtained.

[Brief description of the drawings]

FIG. 1 is a block diagram illustrating a configuration of a time-series data learning device that implements a learning method for a recurrent neural network according to an embodiment of the present invention.

FIG. 2 is a flowchart showing the operation of the time-series data learning device shown in FIG.

FIG. 3 is a flowchart showing a process of adaptively determining a value of a neurogain parameter β in step S23 of FIG.

FIG. 4 is a flowchart showing processing for adaptively determining a value of a learning coefficient η in step S29 of FIG. 2;

FIG. 5 shows Eima, which is a kind of a configuration of a recurrent neural network.
It is a figure showing composition of n net.

[Explanation of symbols]

 Reference Signs List 1 memory 2 data input unit 3 data output unit 4 neural element value storage unit 5 connection weight storage unit 6 control unit 7 neural network initialization unit 8 element time evolution calculation unit 9 β value determination unit 10 η value determination unit 11 steepest descent Direction calculation unit 12 Connection weight matrix correction unit

Claims (1)

[Claims] When given several sets of discrete-time, discrete-valued input time-series data and discrete-time, discrete-valued target output time-series data corresponding to the data, the input time-series data is divided into discrete-time, A continuous value is input to the regression type neural network, and the connection weight parameter is sequentially corrected in the steepest descent direction of the error plane so as to reduce the error between the actual output data of the neural network and the target output time series data. A learning method for a regression-type neural network that acquires a function between the input and output time-series data, wherein the learning has progressed to some extent and the neurogain has been set so that the error does not increase from the stage where the error decreases below a reference value. A regression-type neural circuit characterized in that learning is advanced while increasing parameters, and a connection weight parameter is corrected in a direction in which an error is reduced while adjusting a learning coefficient so that an error does not increase. Web learning method.