JPH1011413A - Recurrent net model production device for dynamics system - Google Patents

Abstract

PROBLEM TO BE SOLVED: To generate a recurrent net model belonging to such a wide group that can generate a finite number of orbits that are designated in an optional dynamics system by means of a special solution of a recurrent net that is obtained from the phase space learning and also an optional matrix. SOLUTION: A recurrent net model parameter production means 60 calculates and outputs a wait matrix W and an external input I of a recurrent net to lead ϕc, i.e., a recurrent net model of a dynamics system in an n-dimensional space where (m) orbits shown in the orbit information are generated based on an (n×d) matrix C which is optionally given and inputted from an input means 10, the parameter inputted from a vector field production means 30, and a dimension (d) and a base e1 ...ed } of an orthogonal complement which are inputted from a base production means 50. Furthermore, the number (k) of hidden units outputted from the means 30 is outputted as a number of hidden units of the recurrent units. Then the orbit of the dynamics system are generated based on the states of output units of the recurrent net.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to an apparatus for generating a recurrent net model used as a model expressing a dynamic system, and more particularly, to a dynamic system for which a model is desired to be obtained, and a trajectory generated by the dynamic system. The present invention relates to an apparatus for creating a recurrent net model of a dynamical system capable of generating a trajectory designated as important.

[0002]

2. Description of the Related Art A linear dynamic system can produce a controllable model, so that its analysis and control can be easily performed, and it has been used for various applications. However,
Dynamical systems generated by interaction with the environment are generally non-linear. Nonlinear dynamical systems include, for example, flow in a furnace, speech synthesis, robot motion, computer graphics (C
Appears in various fields such as animation in G). Therefore, obtaining a controllable model for an arbitrary nonlinear dynamical system is an important issue for the analysis and control of the nonlinear dynamical system and its engineering application.

[0003] A recurrent net, which is a neural net having a feedback type mechanism, has the ability to approximate an arbitrary dynamical system, and also has parallel computing properties. Therefore, as a controllable model of any dynamical system,
Attempts have been made to use recurrent nets.

A dynamic system is a collection of trajectories, and each trajectory is important information for analyzing an actual problem. Known literature, for example, J. Hertz, A. Krogh and R. Palmer; "I
ntroduction to the Theory of Neural Computation ",
Lecture Notes, Santa Fe Institute Studies in the S
As shown in ciences of Complexity, (1991) (Reference 1), as a method for causing a recurrent net to learn a finite number of orbits of a nonlinear dynamical system, an RTRL learning method (Real-Ti
me Recurrent Learning) and TDR
BP learning method (Time-Dependent Recurrent Back-Propagati
on: Real-time dependent recursive backpropagation learning method) is widely used. However, it has been pointed out that such a recurrent net learning method for directly learning a trajectory is not always effective when learning a complicated dynamical system. Then, for a dynamical system that generates a specified trajectory, a vector field that determines the dynamical system is learned by a feedforward net, which is a neural network without a feedback mechanism, and based on this feedforward net, A well-known document, FS. Tsunga, found that the framework of phase space learning, which is to find a recurrent net that generates the trajectory of a dynamical system, is more effective.
nd GW Cottrell; "Phase-Space Learning", Advance
s in Neural Information Processing Systems, Vol.
7, pp. 481-488, (1995) (Reference 2). As for the learning of the feedforward net, various efficient learning methods such as a back error propagation learning method having a momentum term as described in the above-mentioned document 1 are known.

[0005] The present inventors have also disclosed in Japanese Patent Application No. Hei 8-9389.
For a given dynamical system in n-dimensional space that can be completely represented by a recurrent net consisting of n units with no hidden units, a dynamics in n-dimensional space that generates a specified finite number of orbitals We have proposed a neural network generator that creates a recurrent net model of all dynamical systems that can be completely represented by a recurrent net consisting of n units without hidden units. In this device, a TDRBP learning method is used when calculating a special solution of a recurrent net model of a dynamic system that generates a specified trajectory.

[0006]

SUMMARY OF THE INVENTION In a practical problem,
For a dynamical system for which a model is to be obtained, a particularly important finite number of orbits among the orbits that are important information in the dynamical system are designated. Therefore, it is required to create a recurrent net model of a dynamical system for which a model is desired to be acquired based on the specified orbit information. However, a dynamic system that generates these trajectories on the assumption that a certain trajectory is designated does not always generate another trajectory of the original dynamic system.
That is, it is generally unknown whether or not a recurrent net model of the original dynamical system can be obtained from only the specified finite number of orbital information.

Therefore, for a wide family of dynamical systems capable of generating a specified finite number of orbits, it is required to realize an apparatus capable of creating a recurrent net model of a dynamical system belonging to this family. The challenge is to realize such a device not only for a particular type of dynamical system, but also for any dynamical system.

SUMMARY OF THE INVENTION An object of the present invention is to provide a recurrent net model for a dynamic system belonging to a dynamic family that can generate a specified finite number of orbitals in an arbitrary dynamic system. An object of the present invention is to provide a net model making device.

[0009]

According to the recurrent net model forming apparatus for a dynamical system of the present invention, when m orbits are given to an arbitrary dynamical system in an n-dimensional space, the m
In a device for creating a recurrent net model of a dynamical system on an n-dimensional space for generating orbits, orbit information of m orbits with a time step of Δt

[0010]

[Outside 6] , An n-dimensional vector field V (x) along each trajectory
Data generating means for calculating and outputting the data, and a subset of an n-dimensional space which is a domain of the n-dimensional vector field V (x) with respect to the n-dimensional vector field V (x) output from the teacher data generating means Two-layer feedforward net Ag (x) on Ω
And the three-layer feedforward net Bg (Hx + θ) having k hidden units, the vector field V (x)
In order to express + x, when an n-dimensional vector field V (x) is input, a weight matrix A expressed by an n × n matrix from an input unit to an output unit in a two-layer feedforward net, and a three-layer feedforward A weight matrix H expressed by a k × n matrix from an input unit to a hidden unit in a net, and an n × matrix from a hidden unit to an output unit in a three-layer feedforward net
vector field learning means for calculating and outputting a weight matrix B represented by a k matrix, a k-dimensional vector θ in which threshold values of hidden units in a three-layer feedforward net are collected, and orbit information of m orbits

[0011]

[Outside 7] And the number k of hidden units, the k-dimensional vector θ, and the weight matrix H output from the vector field learning means, and m curves in an n + k-dimensional space, which are images of the unit activation function g of the recurrent net

[0012]

[Outside 8]The mL n + k dimensional vector q expressing₁,…, Q_mL
A curve vector generating means for calculating and outputting
ML n + k dimensional vectors q output by the
₁,…, Q_mLAnd input them into n + k-dimensional space
Vector q₁,…, Q _mLIn the subvector space containing
For the smallest subvector space, its orthogonal complement
Dimension d and basis ｛e₁,…, E_dBase for calculating and outputting 基
Bottom creating means, arbitrarily given n × d matrix C and three-layer matrix
K and the number of hidden units in the forward network
Eight matrices A, H, B, k-dimensional vector θ and orthogonal complement space
Dimension d and basis ｛e₁,…, E_dOrbit as input
information

[0013]

[Outside 9] In order to derive φ _C , which is a recurrent net model of a dynamical system on n-dimensional space that generates m orbits represented by
The number k of hidden units in the layer feedforward net is defined as the number of hidden units in the recurrent net, and the weight matrix W and the external input I in the recurrent net are calculated and output. In order to generate the trajectory of the dynamical system, the initial value given to the recurrent net when the point x on the n-dimensional space is given as the initial value of the trajectory generated by the dynamical system is (x, Hx +
θ), and a recurrent net model parameter generating means for outputting a weight matrix H and a k-dimensional vector θ as parameters.

According to the present invention, for a dynamical system in an n-dimensional space capable of generating m orbits of an arbitrary dynamical system in the n-dimensional space, a special solution of the recurrent net model is efficiently calculated. To do so, a phase space learning framework is used, and a special solution is calculated by the teacher data creating means, the vector field learning means, and the recurrent net model parameter creating means. In the vector field learning means, it is desirable to obtain the weight matrices A, H, B and the k-dimensional vector θ by using the inverse error propagation learning method having the momentum term of the neural network. At this time, the vector field learning means calculates an error function while increasing the number k of hidden units in the three-layer feedforward net, and calculates the number of hidden units when the value of the error function falls within a predetermined allowable error. Preferably, k is output, so that the number k of hidden units is the number of hidden units of the recurrent net.

According to the present invention, a basis {e ₁ ,..., _Ed } obtained by the curve vector creation means and the basis creation means, and an arbitrarily designated n × d matrix From C, parameters for generating such a recurrent net model of a wide family of dynamical systems are calculated by the recurrent net model parameter creation means. That is, every time the n × d matrix C is specified, a different recurrent net is generated.

Hereinafter, the operation of the present invention will be described in detail. First, the dynamic system will be described. In the text of this specification, the n-dimensional space

[0017]

[Outside 10] Is simply abbreviated as R ^n, and a set of all real numbers

[0018]

[Outside 11] Is abbreviated as R.

A dynamical system 上 の in an n-dimensional space R ⁿ is

[0020]

[Outside 12] Mapping

[0021]

(Equation 1) And nature

[0022]

(Equation 2) Is satisfied. That is, a dynamical system ψ (dynamical system 6) on an n-dimensional space R ⁿ is defined as an n-dimensional space when an input unit 5 gives an n-dimensional vector x, as shown in FIG. It is considered that the system generates a trajectory ξ (t) represented by the following equation on R ⁿ and outputs it to the output unit 7.

[0023]

(Equation 3) In the following description, a dynamical system in an n-dimensional space R ⁿ for which a model is to be obtained is denoted by ψ, and of the trajectories generated by this dynamical system ψ,
Let m orbits designated as important be ξ ^{(1 )} (t),
..., ξ ^(m) (t).

Next, a recurrent net model of a dynamical system on an n-dimensional space handled in the present invention will be described. In the recurrent net, n + k units are provided, and n units from the first to the n-th are output units,
The k units from the (n + 1) th to the (n + k) th are hidden units. The recurrent net is (n + k) × (n
+ K) Parameterized by a weight matrix W = (W _ij ), which is an execution sequence, and an external input I = (I ₁ ,..., I _{n + k} ) ∈R ^{n + k} . Time t of unit i in the recurrent net
The state u _i (t) at ∈R, given an initial value for each unit, is determined by the solution of the following ODE:

[0025]

(Equation 4) Where the unit activation function g is a given bounded monotonic increase on R

[0026]

[Outside 13] A function, for example, tanh (βt), (β> 0). It is constituted by such a recurrent net,
Recurrent net model φ _C of a dynamical system on n-dimensional space R ⁿ ,
(C is an n × d execution sequence)

[0027]

[Outside 14] Mapping

[0028]

(Equation 5) As shown in FIG. 2, n is an initial value for the dynamical system.
A trajectory φ _C (t, x) is generated for a point x on the dimensional space R ⁿ . That is, the recurrent net model φ
_C (recurrent net model 6A) is input from input unit 5A to n
When the dimensional vector x is given, the trajectory φ _C (t, x) is output to the output unit 7A. Here, providing an initial value for the recurrent net, when the parameter H _i and theta ₁ ∈R ^k of k × n matrix is specified, the trajectory phi _C (t,
x) is calculated as the states (u ₁ (t),..., U _n (t)) of _n output units generated by substituting the initial values (x, H ₁ x + θ ₁ ) into the recurrent net. You.

For a wide family of dynamical systems producing the specified orbitals ξ ⁽¹⁾ (t),..., Ξ ^(m) (t), their recurrent net models are represented by φ _C , (C: n It is the aim of the present invention to create in the form of (* d execution sequence). Hereinafter, each step of the present invention will be described in detail.

<< Preparation of Special Solution >> Specified trajectory in dynamical system ψ

[0031]

[Outside 15] Is calculated as follows based on the framework of phase space learning.

First, an n-dimensional space R ⁿ for determining the dynamical system ψ
As a sample of the above vector field, a tangent vector field V (x), (x∈Ω) of a specified trajectory is calculated. That is,

[0033]

(Equation 6) Is calculated.

Next, a vector field V (x) + x is learned by two feedforward nets. Thereby, the number k of hidden units, the n × n execution sequence A ⁰ , the n × k execution sequence B ⁰ , the k × n execution sequence H, θ∈R ^k , which are the parameters of the special solution of the recurrent net model, are calculated. . That is,

[0035]

(Equation 7) Becomes Note that one of the two feedforward nets is a two-layer feedforward net, and the above-described n × n execution sequence A ⁰ is a weight matrix from an input unit to an output unit in the two-layer feedforward net. is there. The other feedforward net is a three-layer feedforward net having k hidden units, and the k × n execution sequence H is a weight matrix from an input unit to a hidden unit in this three-layer feedforward net, and has n × k The execution sequence B ⁰ is a weight matrix from the hidden unit to the output unit. The k-dimensional vector θ is a vector in which threshold values of k hidden units in the three-layer feedforward net are collected, that is, a vector having these threshold values as elements.

The weight matrix W ⁰ is

[0037]

(Equation 8) And the external input I is

[0038]

(Equation 9) To create a recurrent net. Using the output unit of this recurrent net, a dynamic system φ ₀ on the n-dimensional space R ⁿ is defined as follows, and this is a special solution of the recurrent net model of the intended dynamic system. That is, t∈
For R and x∈R ⁿ , φ ₀ (t, x) is (x, Hx + θ) ∈
^N of this recurrent net when R ^{n + k} is the initial value
This is the state of the output units at time t.

<< Creation of Solution Family >> As described above, assuming that the specific solution of the recurrent net model has been obtained, the dynamic system belonging to this family is The recurrent net model is calculated as follows.

The above-mentioned unit activation function g is expressed as follows in the n-dimensional space R ⁿ .

[0041]

[Outside 16] Extended to isomorphism:

[0042]

(Equation 10) Also, a subset of the vector space R ^{n + k}

[0043]

[Equation 11]This vector space R containing^{n + k}The smallest partial vector of
V (ξ⁽¹⁾,…, Ξ^(m)). Also, the vector space R
^{n + k}The subvector space V (ξ ⁽¹⁾,
…, Ξ^(m)) Orthogonal complement space

[0044]

[Outside 17] The basis of

[0045]

(Equation 12) And

At this time, an arbitrary n × d execution sequence C = (C _ij )
, N * n matrix A ^* = (A ^* _ij ) and n * k matrix B ^* =
(B ^* _ij ) and

[0047]

(Equation 13) Defined by

[0048]

[Equation 14] A recurrent net having the following weight matrix W and an external input of I is created.

Using the output unit of this Rikaren net, a dynamic system φ _C on an n-dimensional space R ⁿ is defined as follows:
This is the recurrent net model of the dynamical system that was the goal. That is, for t∈R and _{^{x∈R n, φ C (t,}} x)
Is a state at time t of n output units of this recurrent net when (x, Hx + θ) ∈R ^{n + k} is set as an initial value.

According to the present invention, an n-dimensional space RⁿUpper dynamics
指定 designated orbit ξ⁽¹⁾(t),…, ξ ^(m)generate (t)
Recurrent net model φ for a family of dynamical systems_CIs a special solution
Μ = (A⁰, B⁰, H, θ) and the vector
space

[0051]

[Outside 18] The underlying _{{e 1, ..., e d} } and the calculation of the formula (14), it is possible to create by (15). As is clear from equations (14) and (15), the n × d matrix C is used to identify a certain dynamical system in a wide family, and by changing the matrix C, Can be generated.

[0052]

Next, embodiments of the present invention will be described with reference to the drawings. FIG. 3 is a block diagram illustrating a configuration of a recurrent net model creation device for a dynamic system according to an embodiment of the present invention.

When a finite number (m) of orbits is given to an arbitrary dynamical system in an n-dimensional space, the recurrent net model creation device 1 for the dynamical system generates the given orbit. It is for generating a recurrent net of a family of a dynamical system in an n-dimensional space, and includes an input unit 10, a teacher data generation unit 20, a vector field generation unit 30, a curve vector generation unit 40, a basis generation unit 50, a recurrent net model. It comprises a parameter creation means 60 and an output means 70. It is assumed that m orbit information of the dynamic system in the n-dimensional space is stored in the input information storage unit 80 in advance, and this orbit information is input by the input means 10 to the input information storage unit 8.
0 is input to the recurrent net model creation device 1.

The teacher data creating means 20 is provided with the input means 10
Ｍ ⁽¹⁾ (t),…,
ｎ ^(m) Calculates and outputs an n-dimensional vector field V (x) along these orbits based on (t), and is used when calculating the n-dimensional vector field V (x). A teacher data creation rule database 21 for storing rules, an operation unit 22 for calculating an n-dimensional vector field V (x) with reference to the teacher data creation rule database 21, and a buffer area for temporarily holding the operation result 23.

The vector field creating means 30 generates a subset Ω of the n-dimensional space which is the domain of the n-dimensional vector field V (x) for the n-dimensional vector field V (x) output from the teacher data creating means 20. On the two-layer feedforward net Ag
(x) and the three-layer feedforward net Bg (Hx + θ) having k hidden units, the vector field V
In order to express (x) + x, when an n-dimensional vector field V (x) is input, parameters μ characterizing these feedforward nets are calculated, and each calculated parameter is output together with the number k of hidden units. Is what you do.
Specifically, the parameter μ is nx from the input unit to the output unit in the two-layer feedforward net.
A weight matrix A expressed by an n matrix, a weight matrix H expressed by a k × n matrix from an input unit to a hidden unit in a three-layer feedforward net, and an nx from a hidden unit to an output unit in a three-layer feedforward net Weight matrix B represented by k matrix
And a k-dimensional vector θ in which threshold values of hidden units in the three-layer feedforward net are collected. The vector field creating means 30 is used to calculate the weight matrices A, H, B and the k-dimensional vector θ while increasing the number k of hidden units from 0 based on the input n-dimensional vector field V (x). With reference to the vector field creation rule database 31 that stores rules to be executed, and the vector field creation rule database 31, an error function E _k (μ), weight matrices A, H, B, and k-dimensional vector θ described later are calculated. An operation unit 32 and a buffer area 33 for temporarily holding an operation result are provided.

The curve vector creating means 40 is provided with the input means 1
Orbit information input from 0

[0057]

[Outside 19] And the number k of hidden units, the k-dimensional vector θ, and the weight matrix H output from the vector field learning means 30 as inputs, and m number of images in an n + k-dimensional space, which is an image of a unit activation function g of the recurrent net, are input. curve

[0058]

[Outside 20] , Represents mL n + k-dimensional vectors q ₁ , ..., q _mL
, And outputs a curve vector creation rule database 41 for storing rules used for calculating these n + k-dimensional vectors q ₁ ,..., Q _mL , and a curve vector creation rule database 41. N +
an operation unit 42 for calculating k-dimensional vectors q ₁ ,..., q _mL ;
And a buffer area 43 for temporarily holding the operation result.

The basis creating means 50 outputs mL n + k dimensional vectors q ₁ ,
..., as input q _mL, these vectors q ₁ in the n + k-dimensional space, ..., with respect to partial vector space is the smallest among the partial vector space containing q _mL, dimension d and basis of the orthogonal complement _{{e 1, ..., e d} } be one that calculates and outputs, the dimension d and basis _{{e 1, ..., e d} } a base created rules database 51 which stores rules to be used when calculating the , Calculating a dimension d and a basis {e ₁ ,..., E _d } with reference to the basis creation rule database 51.
And a buffer area 5 for temporarily holding an operation result
3 is constituted.

The recurrent net model parameter creating means 60 is arbitrarily given and inputted from the input means 10.
Xd matrix C, parameters input from vector field creating means 30 (number k of hidden units in three-layer feedforward net, weight matrices A, H, B, and k-dimensional vector θ), and basis input means 50 Based on the dimension d of the orthogonal complement space to be input and the basis {e ₁ ,..., E _d }, the trajectory information

[0061]

[Outside 21] In order to derive φ _C , which is a recurrent net model of a dynamical system on an n-dimensional space that generates m orbits represented by the following formula, a weight matrix W and an external input I
The parameter creation rule database 61 stores rules used when calculating the weight matrix W and the external input I, and the weight matrix W and the external It comprises an operation unit 62 for calculating the input I and a buffer area 63 for temporarily holding the operation result. Further, the recurrent net model parameter creating means 60 outputs the number of hidden units in the recurrent net, using the number k of hidden units output from the vector field creating means 40 as the number of hidden units in the recurrent net, and outputs the number of hidden units in the recurrent net. In order to generate the trajectory of the dynamical system according to the state, when the point x in the n-dimensional space is given as the initial value of the trajectory generated by the dynamical system, the initial value given to the recurrent net is (x, Hx + θ) The weight matrix H and the k-dimensional vector θ, which are the parameters of

The output means 70 outputs the weight matrix W,
H, the number k of hidden units, the external input I, and the k-dimensional vector θ are output to a predetermined output information storage unit 90.

The recurrent net model creation device 1 for a dynamic system can be generally realized as a computer system. Therefore, the time t is discretized at intervals of Δt, and each value is calculated for each discretized time. When implemented as a computer system, each of the rule databases 21, 31, 41, 51, 61 is constructed on an auxiliary storage device (such as a hard disk device) of the computer, and each of the arithmetic units 22, 32, 42, 52, 62 is The CPU (Central Processing Unit) of the computer is in charge, and each buffer area 2
3, 33, 43, 53, 63 are secured in a main memory (RAM) or an auxiliary storage device of the computer. Input information storage unit 8
0 and the output information storage unit 90 are also constructed on the auxiliary storage device of the computer.

Next, the operation of the recurrent net model creation device 1 for a dynamic system will be described.

Here, since it is assumed that the time is discretized, m orbits of an arbitrary dynamical system ｎ in an n-dimensional space ξ
⁽¹⁾ (t),..., Ξ ^(m) (t) is equal to Δ at time [0, T].
It is assumed that the number is specified by t and the number of samples L. That is,

[0066]

(Equation 15) Is specified. here,

[0067]

(Equation 16) It is. For any n × d matrix C, these trajectories

[0068]

[Outside 22] Consider a recurrent net model φ _C of a dynamical system in an n-dimensional space that generates the following formulas. The number k of hidden units, the weight matrix W and the external input I, which are parameters for expressing the recurrent net model φ _C , and the recurrent net model φ _{A case} where a k × n matrix H, which is a parameter for giving an initial value to _C , and a k-dimensional vector θ (∈R ^k ) are calculated and output will be described.

First, m pieces of orbit information are input from the input information storage unit 80 through the input means 10.

[0070]

[Outside 23] , An arbitrary n × d matrix C given by the user is input. Then, the n-dimensional vector field V (x) is calculated by the teacher data generating means 20, whereby the number k of hidden units and the parameter μ [= (A ⁰ , B ⁰ ,
H, θ)] is calculated. Then, in the curve vector creation means 40, mL n values are obtained from the above-mentioned trajectory information, the number k of hidden units, the vector θ, and the weight matrix H.
+ K-dimensional vectors q ₁ ,..., Q _mL are calculated.
Number of vectors q _1, ..., a q _mL, in the basal creation unit 50, the number d and underlying basement _{{e 1, ..., e d} } are calculated. Finally, in the recurrent net model parameter creation means 60, each of the above-mentioned parameters, that is, C, A,
The weight matrix W and the external input I of the recurrent net are calculated by k, θ, H, B, d, {e ₁ ,..., e _d }, and the number k of hidden units and the weight matrix W , H, the external input I and the vector θ are stored in the output information storage unit 9.
0 is stored. Hereinafter, details of the processing in each unit will be described.

<< Processing by Teacher Data Creation Means 20 >> The specified trajectory information is collected, and a subset Ω of the n-dimensional space R ⁿ ;

[0072]

[Equation 17] Is defined. The mapping V from Ω to R ⁿ ;

[0073]

(Equation 18) Is calculated and output. The mapping V shown in Expression (19) represents an n-dimensional vector field V (x). Equations (18) and (19) correspond to discretized equations (7) and (6), respectively.

<< Process in Vector Field Learning Means 30 >> A vector field in an n-dimensional space R ⁿ that approximates the mapping V (x) from Ω to R ⁿ output by the teacher data creation means 20 on Ω.

[0075]

[Outside 24] To

[0076]

[Equation 19] The number k of hidden units and the parameter μ, which are the parameters of

[0077]

(Equation 20) Is calculated and output. Equation (20) is equivalent to discretization of equation (8). Here, A ⁰ is an n × n execution sequence, and B ⁰ is k × n
The execution sequence, H, is an n × k execution sequence, θ∈R ^k .

Learning of the vector field V (x) is performed by a hidden unit
Is given, the error function Ε _kTo

[0079]

(Equation 21) The integer η and a real number δ between 0 and 1 are defined by

[0080]

(Equation 22) Is performed by the inverse error propagation method having the momentum term of the neural network based on the steepest descent method. The number k of hidden units is sequentially increased from zero. This continues until the error Ε _k (μ) falls within the specified tolerance with the specified number of learnings. By learning the vector field V (x) and determining the number k of hidden units and the parameter μ in this manner, the special solution of the recurrent net is determined.

<< Processing in Curve Vector Creation Means 40 >>
From the discrete data of the orbit given in (16), RⁿML pieces
Vector q₁,…, Q _mLIs output. They are:
Is calculated as:

[0082]

(Equation 23) Where g is the value of R ⁿ defined in equation (11).

[0083]

[Outside 25] It is an isomorphism. That is, the vectors q ₁ ,..., Q _mL are images of the trajectory information based on the unit activation function g.

<< Processing in Basis Creating Means 50 >> mL vectors q obtained in the curve vector creating means 40
₁ ,…, q _mL

[0085]

(Equation 24) far. R ⁿ including these mL vectors q ₁ ,..., Q _mL
｛E ₁ , ..., the basis of the orthogonal complement space of the minimum partial vector space of
_ed } is calculated and output. At this time, when the dimension d of the orthogonal complement space is 0,
Only the information of d = 0 is sent to the recurrent net model parameter creation means 60 without outputting the basis. Hereinafter, a specific procedure for calculating the basis {e ₁ ,..., E _d } will be described.

For v∈R ⁿ and the substitution σ of n characters,

[0087]

[Outside 26] Is defined by:

[0088]

(Equation 25) Now,

[0089]

(Equation 26) And the substitution σ _{1 of} n characters is defined by the compatibility between 1 and j ₁ . Also,

[0090]

[Equation 27] age,

[0091]

[Equation 28] And Then, R ⁿ ∋q _i ⁽²⁾ , q _i ⁽³⁾ ,... Are calculated recursively as follows.

In general, for a natural number r,

[0093]

(Equation 29) When is calculated,

[0094]

[Equation 30] Is calculated as follows: First, an (r + 1) × (r + 1) matrix

[0095]

[Outside 27] To

[0096]

(Equation 31) Defined by

[0097]

(Equation 32) And define the permutation σ _{r + 1} of n characters by the compatibility of r + 1 and j _{r + 1} ,

[0098]

[Equation 33] Is calculated. This operation is sequentially continued with respect to r. If (1): r = n or (2): i _{r + 1} no longer exists, this operation is terminated. In the case of (1), d = 0 is output.
In the case of (2), d = n−r is output, and an r × n matrix Q = (q
_ij ^(r) ) and r × r matrix

[0099]

(Equation 34) And calculate

[0100]

(Equation 35) Is calculated. Where ε ₁ , ..., ε _n are

[0101]

[Equation 36] N × 1 matrix. Also, the substitution σ ₁ , ..., σ _{r of} n characters
The synthesized substituted σ = σ ₁ ... σ _r calculated in, further calculates the inverse permutation sigma ^-1. Finally, the following calculation yields the desired basis:

[0102]

(37) As described above, the basis ｛e ₁ ,.
e _{d 求}め is required.

<< Process in Recurrent Net Model Parameter Creation Means 70 >> n × arbitrarily given by the user
d execution sequence C = (C _ij ), the number of hidden units k obtained by the vector field learning means 30, the n × n matrix A ⁰ , the k × n matrix B ⁰ , the n × k matrix H, and θ∈R ^k And the base creation means 50
Dimension d and basal _{{e 1, ..., e d} } obtained in respect and, in order to guide the recurrent net model phi _C dynamical system,
A weight matrix W and external input I∈R ^{n + k} which are parameters of the recurrent net are calculated. Here, as the number of hidden units of the recurrent net, the number k of hidden units determined by the vector field creating means 30, that is, the number of hidden units in the special solution is adopted. Also, the weight matrix W
Is an (n + k) × (n + k) matrix. The external input I is given by the equation (1
0). When the number (dimension) d of the basis is 0, the weight matrix W is W ⁰ calculated by the equation (9). When the dimension d is positive, the weight matrix W is expressed by the equation (1
This is W calculated in 4) and (15).

Further, the recurrent net model parameter creating means 60 outputs the number k of hidden units, and generates a trajectory of a dynamic system based on the state of the output unit of the recurrent net. An n × k matrix H and a k-dimensional vector θ, which are parameters for setting an initial value given to the recurrent net when given as an initial value of the dynamic system to (x, Hx + θ), are output.

The trajectory designated by the integer k, (n + k) × (n + k) matrix W, I∈R ^{n + k} , n × k matrix H, and θ∈R ^k output from the recurrent net model parameter creation means 60 By generating ⁽ time series) ξ ⁽¹⁾ (t),..., Ξ ^(m) (t), a recurrent net model φ _C of the dynamical system is created.

[0106]

As described above, according to the present invention, a special solution of a recurrent net is obtained by phase space learning, and by using this special solution and an arbitrary matrix C, a recurrent net model of a wide family of dynamical systems can be obtained. By calculating, for a given dynamical system, when a finite number of orbits generated by the dynamical system are specified, create a recurrent net model of a wide family of dynamical systems that can generate these orbitals Is possible, and for a dynamic system that wants to acquire a model, the effect that a controllable model of the dynamic system that can generate a finite number of orbits designated as important among the orbits generated by the model can be obtained is obtained. is there.

[Brief description of the drawings]

FIG. 1 shows a dynamic system as a system having an input unit and an output unit.
FIG.

FIG. 2 is a diagram illustrating a recurrent net model φ _C of a dynamical system ψ as a system having an input unit and an output unit.

FIG. 3 is a block diagram showing a configuration of a recurrent net model creation device for a dynamic system according to an embodiment of the present invention.

[Explanation of symbols]

 DESCRIPTION OF SYMBOLS 1 Recurrent net model creation device of dynamics system 5,5A input part 6 Dynamic system 6A Recurrent net installation position 7,7A output part 10 Input means 20 Teacher data creation means 21 Teacher data creation rule database 22,32,42,52,62 Arithmetic units 23, 33, 43, 53, 63 Buffer area 30 Vector field creation means 31 Vector field creation rule database 40 Curve vector creation means 41 Curve vector creation rule database 50 Base creation means 51 Base creation rule database 60 Recurrent net model parameter creation Means 61 Parameter creation rule database 70 Output means 80 Input information storage unit 90 Output information storage unit

Claims (4)

[Claims] When a given dynamical system in an n-dimensional space is given m orbits, a recurrent net model of the dynamical system in the n-dimensional space that generates the m orbits is generated. In the device to be created, the orbit information of the m orbits in which the time interval is Δt , The n-dimensional vector field V along each trajectory
teacher data generating means for calculating and outputting (x); and an n-dimensional vector field V output by the teacher data generating means.
For (x), a two-layer feedforward net Ag (x) and a three-layer feedforward having k hidden units on a subset Ω of an n-dimensional space that is a domain of the n-dimensional vector field V (x) In order to express the vector field V (x) + x by the sum with the net Bg (Hx + θ), when the n-dimensional vector field V (x) is input, the input unit is changed from the input unit to the output unit in the two-layer feedforward net. A weight matrix A represented by an n × n matrix, a weight matrix H represented by a k × n matrix from the input unit to the hidden unit in the three-layer feedforward net, and a hidden unit in the three-layer feedforward net. A weight matrix B expressed by an n × k matrix to the output unit and a threshold value of a hidden unit in the three-layer feedforward net are Vector field learning means for calculating and outputting the collected k-dimensional vector θ, and trajectory information of the m trajectories And the number k of hidden units of the three-layer feedforward net output from the vector field learning means,
With the two-dimensional vector θ and the weight matrix H as inputs, m curves in an n + k-dimensional space, which are images of the recurrent net by the unit activation function g, , Represents mL n + k-dimensional vectors q ₁ , ..., q _mL
And curves vector generating means for calculating and outputting the said mL pieces of n + k-dimensional vector q ₁ the curve vector generating means outputs, ..., as input q _{m L,} these vectors q ₁ in the n + k-dimensional space, ... , q _mL , a basis creation means for calculating and outputting the dimension d and the basis {e ₁ ,..., e _d } of the orthogonal complement space for the smallest partial vector space of the partial vector spaces; N × d matrix C arbitrarily given, the number k of hidden units in the three-layer feedforward net, the weight matrices A, H, B, the k-dimensional vector θ, the dimension d of the orthogonal complement space, and the basis With {e ₁ , ..., _ed } as input, the orbit information In order to derive φ _C , which is a recurrent net model of a dynamical system on an n-dimensional space that generates the m orbits represented by The number of units, the weight matrix W and the external input I in the recurrent net are calculated and output, and furthermore, in order to generate a trajectory of a dynamical system based on the state of the output unit of the recurrent net, When the point x is given as the initial value of the trajectory generated by the dynamic system, the weight matrix H and the k-dimensional vector θ, which are parameters for setting the initial value given to the recurrent net to (x, Hx + θ), are output. A recurrent net model creation device for a dynamical system, comprising: 2. The weighting method according to claim 1, wherein the vector field learning means performs the weighting by a reverse error propagation learning method having a momentum term of a neural network using the n-dimensional vector field V (x) output from the teacher data generating means as teacher data. Matrices A, H,
B and the k-dimensional vector θ are calculated.
The recurrent net model creation device for a dynamic system according to claim 1. 3. The method according to claim 2, wherein the recurrent net model parameter creating means is configured to generate the trajectory information. For each dynamical system in the n-dimensional space that generates the m orbits represented by
The recurrent net model creation device for a dynamic system according to claim 1 or 2, wherein the device creates such a recurrent net model having a different dynamic system. 4. The vector field learning means calculates an error function while increasing the number k of hidden units in the three-layer feedforward net, and calculates the error function when the value of the error function is within a predetermined allowable error. The recurrent net model creation device for a dynamic system according to any one of claims 1 to 3, wherein the number k of hidden units is output.