CN108090559A - A kind of construction method of antithesis reserve pool neural network model - Google Patents
A kind of construction method of antithesis reserve pool neural network model Download PDFInfo
- Publication number
- CN108090559A CN108090559A CN201810004120.7A CN201810004120A CN108090559A CN 108090559 A CN108090559 A CN 108090559A CN 201810004120 A CN201810004120 A CN 201810004120A CN 108090559 A CN108090559 A CN 108090559A
- Authority
- CN
- China
- Prior art keywords
- mrow
- reserve pool
- msup
- reserve
- antithesis
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/12—Computing arrangements based on biological models using genetic models
- G06N3/126—Evolutionary algorithms, e.g. genetic algorithms or genetic programming
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- Biophysics (AREA)
- Health & Medical Sciences (AREA)
- Life Sciences & Earth Sciences (AREA)
- Computing Systems (AREA)
- Mathematical Physics (AREA)
- Data Mining & Analysis (AREA)
- Evolutionary Computation (AREA)
- General Health & Medical Sciences (AREA)
- Molecular Biology (AREA)
- Biomedical Technology (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Computational Linguistics (AREA)
- Software Systems (AREA)
- Artificial Intelligence (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Bioinformatics & Computational Biology (AREA)
- Evolutionary Biology (AREA)
- Physiology (AREA)
- Genetics & Genomics (AREA)
- Other Investigation Or Analysis Of Materials By Electrical Means (AREA)
Abstract
The invention discloses a kind of construction methods of antithesis reserve pool neural network model, a kind of antithesis reserve pool network of decoupling confrontation performance suitable for time series forecasting field of this method structure, the network separate two kinds of performance non-linear mapping capabilities and short term memory capacity originally in reserve pool computation model with antagonism.Traditional reserve pool computation model is based on single reserve pool network, and such as echo state network and liquid condition machine, although can be applied in dynamic system model, this computation model needs to weigh non-linear mapping capability and short term memory capacity.Hidden layer in echo state network is arranged to the reserve pool of two antithesis by the antithesis reserve pool network that the present invention is built, and the compression transmission of reserve pool internal information is realized by non-supervisory encoder principal component analysis.Antithesis reserve pool real-time performance obtains extraordinary effect with enhancing and in Chaotic time series forecasting to the separation of non-linear mapping capability and short term memory capacity.
Description
Technical field
The present invention relates to reserve pool calculating and neutral net studying technological domains, and in particular to a kind of antithesis reserve pool nerve
The construction method of network model.
Background technology
As viewed from the perspective of Modelling of Dynamic System, as reserve pool computation model represent echo state network gather around there are two
Important performance --- non-linear mapping capability and short term memory capacity.Nonlinear Mapping performance is higher to be meant to preferably
Fit non-linear data, and short term memory capacity relatively then means by force that system is more prone to the rule for reflecting data for the previous period
Rule, lacks and makes the ability precisely predicted to complex nonlinear data.Both the above ability is inherently presence of confronting with each other
, when reserve pool network is shown as compared with strong nonlinearity, memory capability can be weakened；Conversely, the stronger deposit of memory capability
Pond network can also show weaker non-thread sexuality.There is experiment to show in current reserve pool network, it is non-linear to reflect
Ability is penetrated with short term memory capacity there are a kind of relation for resisting balance, two can not increase or reduce simultaneously.2010
Verstraeten has studied the interaction of both confrontation performances in the l-G simulation test of a reserve pool.2013
Butcher from the impact analysis of action point z, by input scaling parameter IS and spectral radius ρ controlled by this problem, wherein action point z
System.As shown in Figure 1, Butcher has found：Give a small input signal, one less than 1 input scaling parameter IS with
And one close to 1 but the spectral radius ρ less than 1, action point z by the range of linearity in activation primitive, possess most in the region
High memory capability and minimum non-linear mapping capability；When increase inputs scaling parameter IS and spectral radius ρ, due to making
Nonlinear area is risen to point z, the memory capability of reserve pool will weaken and non-linear ability is strengthened.From
From the point of view of the analysis of Butcher, mutual restrict of non-linear mapping capability and short term memory capacity is since action point z is in single
The range of linearity or nonlinear area.In order to solve the problems, such as this, Butcher introduces two kinds of Gao Fei in echo state network
Linear static limit learning machine module, and the model is referred to as R2SP.Another similar work is Gallicchio in 2011
ψ-ESN the models of proposition, the model introduce an extreme learning machine module by echo state network.Two above model is all
So that the non-thread sexuality of whole system has obtained a degree of enhancing.Although existing technology can be realized in certain level
The promotion of non-linear mapping capability or short term memory capacity in echo state network, but substantially there is no propose make two kinds it is right
The method that anti-performance optimizes jointly, can not remove between both performances to anti-collision essence, which greatly limits currently have
The reserve pool calculating network of single reserve pool is for the modeling and forecasting ability of chaos time sequence.It therefore, can by which kind of technology
It realizes the decoupling and promotion of two kinds of confrontation performances, is a very universal, tool in reserve pool calculating network (echo state network)
There is studying a question for use value.The solution of the problem will largely improve reserve pool calculating network in dynamical system construction in a systematic way
Comprehensive performance in terms of mould.
The content of the invention
The purpose of the present invention is to solve drawbacks described above of the prior art, disclose a kind of antithesis reserve pool nerve net
The construction method of network model, wherein, the antithesis reserve pool neutral net of confrontation performance decoupling can decouple current reserve pool and calculate net
Present in network have antagonism non-linear mapping capability and short term memory capacity, and can to two kinds of antagonisms respectively into
Row is adjusted and optimization is promoted.Antithesis reserve pool neutral net proposed by the present invention, consider using two reserve pools come share this two
Kind ability, specific method are that two reserve pools are set in the hidden layer of echo state network, are known as antithesis reserve pool, pass through tool
The principal component analytical method for having unsupervised code capacity realizes that the compression of antithesis reserve pool internal information is transferred, and finally realizes pair
The decoupling and enhancing of non-linear mapping capability and short term memory capacity, and best prediction is obtained in chaotic time forecasting problem
Precision.
The purpose of the present invention can be reached by adopting the following technical scheme that：
A kind of construction method of antithesis reserve pool neural network model, the construction method comprise the following steps：
Hidden layer in echo state network is replaced with two antithesis reserve pools, determines first by S1, netinit
The size of reserve pool and second reserve pool generates the input weights of two reserve pools, the random connection weight inside reserve pool,
Determine activation primitive f (z), initialization input scaling parameter IS, spectral radius parameter ρ；
S2, signal input, input current signal u (n)；
S3, the state of first reserve pool update, by the state of first reserve pool of n moment n-1 moment reserve pools
(the 1- γ) of state times of add the moment f (z) γ times represent that wherein z is the weighting of n-1 moment reserve pool states and n moment
The sum of weighting scaling of input signal, the first portion of z are adjusted with spectral radius parameter ρ, and second portion is by inputting pantograph ratio
The IS controls of example parameter；Its function representation is：
Wherein, at the time of n is corresponding to reserve pool, x (n) is the state corresponding to reserve pool, and γ is leakage rate, and u (n) is
Input signal, ρ be spectral radius parameter, WinIt is projection matrix, W is transition matrix, by W0It generates, λmax(W0) be matrix maximum
Characteristic value, W0Element randomly generated between [- 0.5,0.5], in formula footnote (1) represent first reserve pool corresponding to ginseng
Amount；
S4, given input scaling parameter IS and spectral radius parameter are passed throughρ, the state table of first reserve pool of acquisition
Show, which represents the Nonlinear Mapping information and short-term memory information that include first reserve pool；
S5, dimensionality reduction coding is carried out to the state of first reserve pool by PCA encoders, extracts the reserve pool and imply shape
Abstract characteristics vector h (n) in state, using h (n) as the input signal of second reserve pool；
S6, the state of second reserve pool update, and current signal h (n) are inputted, by the shape of first reserve pool of n moment
State represents that wherein z lays in for the n-1 moment plus γ times of the moment f (z) again with (the 1- γ) of the state of n-1 moment reserve pools
The sum of the weighting of pond state and the weighting scaling of n moment input signals, the first portion available spectrum radius parameter ρ of z is adjusted, the
Two parts can be by inputting scaling parameter IS controls；Its function representation is
Wherein, at the time of n is corresponding to reserve pool, x (n) is the state corresponding to reserve pool,γFor leakage rate, u (n) is
Input signal, ρ are spectral radius, WinIt is projection matrix, W is transition matrix, by W0It generates, λmax(W0) be matrix maximum feature
Value, W0Element randomly generated between [- 0.5,0.5], in formula footnote (2) represent second reserve pool corresponding to parameter；
S7, will be directly connected to, feature connection and the information of output layer are collected into matrix M, and pass through linear regression technique
Learn corresponding connection weight, teacher signal is collected into matrix T, by introducing Thikhonov regularization terms, be built into one
A ridge regression problem is realized to antithesis reserve pool network model optimal weights matrix W*Solution.
Further, the optimal weights matrix W*Function expression be：
W*=TMT((MM T+βI)-1 (4)
Wherein, reserve pool hyper parameter is optimized including IS, ρ, γ by genetic algorithm, and the index of system performance height is
Root-mean-square error NRMSE by standardization, is represented by the following formula：
Wherein, T is the length of signal, and y (n) is the prediction output at n moment,It is the realistic objective value at n moment.
Further, based on the expression study in deep learning, first is realized by the use of non-supervisory encoder PCA as intermediary
The forward direction of ability information transfers between a reserve pool and second reserve pool.
Further, by constructing antithesis reserve pool in echo state network, then to the control of antithesis reserve pool state
System realizes the separation and automatic adjustment to non-linear mapping capability and short term memory capacity.
Further, hyper parameter --- the input to two reserve pools is realized as intelligent optimum operation using genetic algorithm
Scaling parameter IS, the optimization of spectral radius parameter ρ and then control non-linear mapping capability and short term memory capacity are in two storages
Decoupling on standby pond.
The present invention is had the following advantages compared with the prior art and effect：
The present invention realizes two kinds of ability -- non-linear mapping capability and short term memory capacities with confrontation performance originally
It is decoupled, and achievees the purpose that collaboration optimization.Unlike traditional echo state network, the present invention is by introducing two
A reserve pool is respectively used to realize that the ability for having two kinds confrontation performance is decoupled and optimized.Compared to traditional echo
State network, this method and corresponding application system achieve more preferably effect in terms of Chaotic time series forecasting.
Description of the drawings
Fig. 1 is a kind of activation primitive used by the prior art --- the action point and different zones of hyperbolic tangent function are drawn
Divide schematic diagram；
Fig. 2 is one simple Organization Chart of antithesis reserve pool network in the present invention；
Fig. 3 is the structure flow chart of the antithesis reserve pool neural network model of decoupling confrontation performance.
Specific embodiment
To make the purpose, technical scheme and advantage of the embodiment of the present invention clearer, below in conjunction with the embodiment of the present invention
In attached drawing, the technical solution in the embodiment of the present invention is clearly and completely described, it is clear that described embodiment is
Part of the embodiment of the present invention, instead of all the embodiments.Based on the embodiments of the present invention, those of ordinary skill in the art
All other embodiments obtained without making creative work belong to the scope of protection of the invention.
Embodiment one
Present embodiment discloses a kind of structure sides of antithesis reserve pool neutral net (DRN) model for decoupling antagonism
Method constructs antithesis reserve pool in echo state network, by realizing the control of antithesis reserve pool state to two kinds of antagonism
Can --- the separation and automatic adjustment of non-linear mapping capability and short term memory capacity；Learnt based on the expression in deep learning,
Non-supervisory encoder such as principal component analysis make use of to transmit intermediary as the information between antithesis reserve pool, realize two antithesis storages
The forward direction of ability information transfers between standby pond；The super ginseng to two reserve pools is realized using the intelligent optimum operation such as genetic algorithm
Two kinds of number --- input scaling parameter IS, the optimization of spectral radius parameter ρ and then control antagonisms are on two reserve pools
Decoupling.
Fig. 2 is one simple Organization Chart of antithesis reserve pool network of the present invention.Fig. 3 is the antithesis deposit of decoupling confrontation performance
Pond neural network model structure flow chart.Using Macket-Glass systems as example, footnote (1), (2) represent first deposit
Parameter corresponding to pond and second reserve pool, the construction step of antithesis reserve pool neural network model mainly include following step
Suddenly：
S1, netinit, determine the size of two reserve pools, generate the input weights of two reserve pools, in reserve pool
The random connection weight in portion determines activation primitive f (z), initialization input scaling parameter IS, spectral radius parameter ρ；
S2, signal input, input current signal u (n)；
S3, the state of first reserve pool update, by the state of first reserve pool of n moment n-1 moment reserve pools
(the 1- γ) of state times of add the moment f (z) γ times represent that wherein z is the weighting of n-1 moment reserve pool states and n moment
The sum of weighting scaling of input signal, the first portion available spectrum radius parameter ρ of z are adjusted, and second portion can be contracted by inputting
Put scale parameter IS controls；Its function representation is
Wherein, at the time of n is corresponding to reserve pool, x (n) is the state corresponding to reserve pool, and γ is leakage rate, and u (n) is
Input signal, ρ be spectral radius parameter, WinIt is projection matrix, W is transition matrix, by W0It generates, λmax(W0) be matrix maximum
Characteristic value, W0Element randomly generated between [- 0.5,0.5].
S4, input scaling parameter IS and spectral radius ρ by giving obtain the state expression of first reserve pool,
The state represents the Nonlinear Mapping information and short-term memory information that include first reserve pool；
S5, dimensionality reduction coding is carried out to the state of first reserve pool by PCA encoders, extracts the reserve pool and imply shape
Abstract characteristics vector h (n) in state, using h (n) as the input signal of second reserve pool；
S6, the state of second reserve pool update, and current signal h (n) are inputted, by the shape of first reserve pool of n moment
State represents that wherein z lays in for the n-1 moment plus γ times of the moment f (z) again with (the 1- γ) of the state of n-1 moment reserve pools
The sum of the weighting of pond state and the weighting scaling of n moment input signals, the first portion available spectrum radius parameter ρ of z is adjusted, the
Two parts can be by inputting scaling parameter IS controls；Its function representation is
Wherein, at the time of n is corresponding to reserve pool, x (n) is the state corresponding to reserve pool,γFor leakage rate, u (n) is
Input signal, ρ are spectral radius, WinIt is projection matrix, W is transition matrix, by W0It generates, λmax(W0) be matrix maximum feature
Value, W0Element randomly generated between [- 0.5,0.5].
S7, will be directly connected to, feature connection and the information of output layer are collected into matrix M, and pass through simple linear return
Return the corresponding connection weight of technological learning, teacher signal is collected into matrix T, by introducing Thikhonov regularization terms, structure
A ridge regression problem is built up to realize to antithesis reserve pool network model optimal weights matrix W*Solution.
Its function expression is：W*=TMT((MM T+βI)-1 (4)
Wherein, reserve pool hyper parameter is optimized including IS, ρ, γ by genetic algorithm, and the index of system performance height is
Root-mean-square error (NRMSE) by standardization, can be represented by the following formula：
Wherein, T is the length of signal, and y (n) is the prediction output at n moment,It is the realistic objective value at n moment.
The construction method of the antithesis reserve pool neural network model of separation confrontation performance disclosed above, by traditional echo shape
Hidden layer in state network replaces with two antithesis reserve pools, and realizes that antithesis stores up by the use of non-supervisory encoder PCA as intermediary
Information between standby pond is transferred.The experimental results showed that：Antithesis reserve pool network achieves prominent effect in the prediction of time series
Fruit.
Above-described embodiment is the preferable embodiment of the present invention, but embodiments of the present invention and from above-described embodiment
Limitation, other any Spirit Essences without departing from the present invention with made under principle change, modification, replacement, combine, simplification,
Equivalent substitute mode is should be, is included within protection scope of the present invention.
Claims (5)
1. a kind of construction method of antithesis reserve pool neural network model, which is characterized in that the construction method includes following
Step：
Hidden layer in echo state network is replaced with two antithesis reserve pools by S1, netinit, determines first deposit
Pond and the size of second reserve pool generate the input weights of two reserve pools, and the random connection weight inside reserve pool determines
Activation primitive f (z), initialization input scaling parameter IS, spectral radius parameter ρ；
S2, signal input, input current signal u (n)；
S3, the state of first reserve pool update, by the state of the state n-1 moment reserve pools of first reserve pool of n moment
(1- γ) times represent that wherein z is to input at the weighting of n-1 moment reserve pool states and n moment plus γ times of the moment f (z)
The sum of weighting scaling of signal, the first portion of z are adjusted with spectral radius parameter ρ, and second portion is joined by inputting scaling
Number IS controls；Its function representation is：
<mrow>
<msup>
<mi>x</mi>
<mrow>
<mo>(</mo>
<mn>1</mn>
<mo>)</mo>
</mrow>
</msup>
<mrow>
<mo>(</mo>
<mi>n</mi>
<mo>)</mo>
</mrow>
<mo>=</mo>
<mrow>
<mo>(</mo>
<mn>1</mn>
<mo>-</mo>
<msup>
<mi>&gamma;</mi>
<mrow>
<mo>(</mo>
<mn>1</mn>
<mo>)</mo>
</mrow>
</msup>
<mo>)</mo>
</mrow>
<msup>
<mi>x</mi>
<mrow>
<mo>(</mo>
<mn>1</mn>
<mo>)</mo>
</mrow>
</msup>
<mrow>
<mo>(</mo>
<mi>n</mi>
<mo>-</mo>
<mn>1</mn>
<mo>)</mo>
</mrow>
<mo>+</mo>
<msup>
<mi>&gamma;</mi>
<mrow>
<mo>(</mo>
<mn>1</mn>
<mo>)</mo>
</mrow>
</msup>
<msup>
<mi>f</mi>
<mrow>
<mo>(</mo>
<mn>1</mn>
<mo>)</mo>
</mrow>
</msup>
<mrow>
<mo>(</mo>
<msup>
<mi>W</mi>
<mrow>
<mo>(</mo>
<mn>1</mn>
<mo>)</mo>
</mrow>
</msup>
<msup>
<mi>x</mi>
<mrow>
<mo>(</mo>
<mn>1</mn>
<mo>)</mo>
</mrow>
</msup>
<mo>(</mo>
<mrow>
<mi>n</mi>
<mo>-</mo>
<mn>1</mn>
</mrow>
<mo>)</mo>
<mo>+</mo>
<msup>
<mi>IS</mi>
<mrow>
<mo>(</mo>
<mn>1</mn>
<mo>)</mo>
</mrow>
</msup>
<mo>&CenterDot;</mo>
<msubsup>
<mi>W</mi>
<mrow>
<mi>i</mi>
<mi>n</mi>
</mrow>
<mrow>
<mo>(</mo>
<mn>1</mn>
<mo>)</mo>
</mrow>
</msubsup>
<mi>u</mi>
<mo>(</mo>
<mi>n</mi>
<mo>)</mo>
<mo>)</mo>
</mrow>
<mo>-</mo>
<mo>-</mo>
<mo>-</mo>
<mrow>
<mo>(</mo>
<mn>1</mn>
<mo>)</mo>
</mrow>
</mrow>
<mrow>
<mi>W</mi>
<mo>=</mo>
<mfrac>
<mi>&rho;</mi>
<msub>
<mi>W</mi>
<mn>0</mn>
</msub>
</mfrac>
<msub>
<mi>&lambda;</mi>
<mrow>
<mi>m</mi>
<mi>a</mi>
<mi>x</mi>
</mrow>
</msub>
<mrow>
<mo>(</mo>
<msub>
<mi>W</mi>
<mn>0</mn>
</msub>
<mo>)</mo>
</mrow>
<mo>-</mo>
<mo>-</mo>
<mo>-</mo>
<mrow>
<mo>(</mo>
<mn>2</mn>
<mo>)</mo>
</mrow>
</mrow>
Wherein, at the time of n is corresponding to reserve pool, x (n) is the state corresponding to reserve pool, and γ is leakage rate, and u (n) is input
Signal, ρ be spectral radius parameter, WinIt is projection matrix, W is transition matrix, by W0It generates, λmax(W0) be matrix maximum feature
Value, W0Element randomly generated between [- 0.5,0.5], in formula footnote (1) represent first reserve pool corresponding to parameter；
S4, input scaling parameter IS and spectral radius parameter ρ by giving obtain the state expression of first reserve pool,
The state represents the Nonlinear Mapping information and short-term memory information that include first reserve pool；
S5, dimensionality reduction coding is carried out to the state of first reserve pool by PCA encoders, extracted in the reserve pool hidden state
Abstract characteristics vector h (n), using h (n) as the input signal of second reserve pool；
S6, the state of second reserve pool update, and input current signal h (n), the state of first reserve pool of n moment is used
(the 1- γ) of the state of n-1 moment reserve pools times of add the moment f (z) γ times represent that wherein z is n-1 moment reserve pool shapes
The sum of the weighting of state and the weighting scaling of n moment input signals, the first portion available spectrum radius parameter ρ of z is adjusted, second
Dividing can be by inputting scaling parameter IS controls；Its function representation is
<mrow>
<msup>
<mi>x</mi>
<mrow>
<mo>(</mo>
<mn>2</mn>
<mo>)</mo>
</mrow>
</msup>
<mrow>
<mo>(</mo>
<mi>n</mi>
<mo>)</mo>
</mrow>
<mo>=</mo>
<mrow>
<mo>(</mo>
<mn>1</mn>
<mo>-</mo>
<msup>
<mi>&gamma;</mi>
<mrow>
<mo>(</mo>
<mn>2</mn>
<mo>)</mo>
</mrow>
</msup>
<mo>)</mo>
</mrow>
<msup>
<mi>x</mi>
<mrow>
<mo>(</mo>
<mn>2</mn>
<mo>)</mo>
</mrow>
</msup>
<mrow>
<mo>(</mo>
<mi>n</mi>
<mo>-</mo>
<mn>1</mn>
<mo>)</mo>
</mrow>
<mo>+</mo>
<msup>
<mi>&gamma;</mi>
<mrow>
<mo>(</mo>
<mn>2</mn>
<mo>)</mo>
</mrow>
</msup>
<msup>
<mi>f</mi>
<mrow>
<mo>(</mo>
<mn>2</mn>
<mo>)</mo>
</mrow>
</msup>
<mrow>
<mo>(</mo>
<msup>
<mi>W</mi>
<mrow>
<mo>(</mo>
<mn>2</mn>
<mo>)</mo>
</mrow>
</msup>
<msup>
<mi>x</mi>
<mrow>
<mo>(</mo>
<mn>2</mn>
<mo>)</mo>
</mrow>
</msup>
<mo>(</mo>
<mrow>
<mi>n</mi>
<mo>-</mo>
<mn>1</mn>
</mrow>
<mo>)</mo>
<mo>+</mo>
<msup>
<mi>IS</mi>
<mrow>
<mo>(</mo>
<mn>2</mn>
<mo>)</mo>
</mrow>
</msup>
<mo>&CenterDot;</mo>
<msubsup>
<mi>W</mi>
<mrow>
<mi>i</mi>
<mi>n</mi>
</mrow>
<mrow>
<mo>(</mo>
<mn>2</mn>
<mo>)</mo>
</mrow>
</msubsup>
<mi>h</mi>
<mo>(</mo>
<mi>n</mi>
<mo>)</mo>
<mo>)</mo>
</mrow>
<mo>-</mo>
<mo>-</mo>
<mo>-</mo>
<mrow>
<mo>(</mo>
<mn>3</mn>
<mo>)</mo>
</mrow>
</mrow>
<mrow>
<mi>W</mi>
<mo>=</mo>
<mfrac>
<mi>&rho;</mi>
<msub>
<mi>W</mi>
<mn>0</mn>
</msub>
</mfrac>
<msub>
<mi>&lambda;</mi>
<mrow>
<mi>m</mi>
<mi>a</mi>
<mi>x</mi>
</mrow>
</msub>
<mrow>
<mo>(</mo>
<msub>
<mi>W</mi>
<mn>0</mn>
</msub>
<mo>)</mo>
</mrow>
<mo>-</mo>
<mo>-</mo>
<mo>-</mo>
<mrow>
<mo>(</mo>
<mn>2</mn>
<mo>)</mo>
</mrow>
</mrow>
Wherein, at the time of n is corresponding to reserve pool, x (n) is the state corresponding to reserve pool, and γ is leakage rate, and u (n) is input
Signal, ρ are spectral radius, WinIt is projection matrix, W is transition matrix, by W0It generates, λmax(W0) be matrix maximum eigenvalue, W0
Element randomly generated between [- 0.5,0.5], in formula footnote (2) represent second reserve pool corresponding to parameter；
S7, will be directly connected to, feature connection and the information of output layer are collected into matrix M, and pass through linear regression technique and learn
Corresponding connection weight collects teacher signal into matrix T, by introducing Thikhonov regularization terms, is built into a ridge
Regression problem is realized to antithesis reserve pool network model optimal weights matrix W*Solution.
2. the construction method of a kind of antithesis reserve pool neural network model according to claim 1, which is characterized in that described
Optimal weights matrix W*Function expression be：
W*=TMT((MMT+βI)-1 (4)
Wherein, reserve pool hyper parameter is optimized including IS, ρ, γ by genetic algorithm, the index of system performance height be by
The root-mean-square error NRMSE of standardization, is represented by the following formula：
<mrow>
<mi>N</mi>
<mi>R</mi>
<mi>M</mi>
<mi>S</mi>
<mi>E</mi>
<mo>=</mo>
<mfrac>
<msqrt>
<mrow>
<mfrac>
<mn>1</mn>
<mi>T</mi>
</mfrac>
<msubsup>
<mi>&Sigma;</mi>
<mrow>
<mi>n</mi>
<mo>=</mo>
<mn>1</mn>
</mrow>
<mi>T</mi>
</msubsup>
<msup>
<mrow>
<mo>&lsqb;</mo>
<mi>y</mi>
<mrow>
<mo>(</mo>
<mi>n</mi>
<mo>)</mo>
</mrow>
<mo>-</mo>
<mover>
<mi>y</mi>
<mo>^</mo>
</mover>
<mrow>
<mo>(</mo>
<mi>n</mi>
<mo>)</mo>
</mrow>
<mo>&rsqb;</mo>
</mrow>
<mn>2</mn>
</msup>
</mrow>
</msqrt>
<mrow>
<mi>V</mi>
<mi>a</mi>
<mi>r</mi>
<mrow>
<mo>(</mo>
<mover>
<mi>y</mi>
<mo>^</mo>
</mover>
<mo>(</mo>
<mi>n</mi>
<mo>)</mo>
</mrow>
<mo>)</mo>
</mrow>
</mfrac>
<mo>-</mo>
<mo>-</mo>
<mo>-</mo>
<mrow>
<mo>(</mo>
<mn>5</mn>
<mo>)</mo>
</mrow>
</mrow>
Wherein, T is the length of signal, and y (n) is the prediction output at n moment,It is the realistic objective value at n moment.
3. the construction method of a kind of antithesis reserve pool neural network model according to claim 1, which is characterized in that be based on
Expression study in deep learning realizes first reserve pool and second deposit by the use of non-supervisory encoder PCA as intermediary
The forward direction of ability information transfers between pond.
4. the construction method of a kind of antithesis reserve pool neural network model according to claim 1, which is characterized in that pass through
Antithesis reserve pool is constructed in echo state network, then the control of antithesis reserve pool state is realized to non-linear mapping capability
And the separation and automatic adjustment of short term memory capacity.
5. the construction method of a kind of antithesis reserve pool neural network model according to claim 1, which is characterized in that use
Genetic algorithm realizes the hyper parameter to two reserve pools as intelligent optimum operation --- input scaling parameter IS, spectral radius
Decoupling of the optimization and then control non-linear mapping capability and short term memory capacity of parameter ρ on two reserve pools.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201810004120.7A CN108090559A (en) | 2018-01-03 | 2018-01-03 | A kind of construction method of antithesis reserve pool neural network model |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201810004120.7A CN108090559A (en) | 2018-01-03 | 2018-01-03 | A kind of construction method of antithesis reserve pool neural network model |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN108090559A true CN108090559A (en) | 2018-05-29 |
Family
ID=62181514
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201810004120.7A Pending CN108090559A (en) | 2018-01-03 | 2018-01-03 | A kind of construction method of antithesis reserve pool neural network model |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN108090559A (en) |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109462821A (en) * | 2018-11-19 | 2019-03-12 | 东软集团股份有限公司 | Method, apparatus, storage medium and the electronic equipment of predicted position |
| CN114202032A (en) * | 2021-12-15 | 2022-03-18 | 中国科学院深圳先进技术研究院 | Gait detection method and device based on reservoir model and computer storage medium |
-
2018
- 2018-01-03 CN CN201810004120.7A patent/CN108090559A/en active Pending
Cited By (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109462821A (en) * | 2018-11-19 | 2019-03-12 | 东软集团股份有限公司 | Method, apparatus, storage medium and the electronic equipment of predicted position |
| CN109462821B (en) * | 2018-11-19 | 2021-07-30 | 东软集团股份有限公司 | Method, device, storage medium and electronic equipment for predicting position |
| CN114202032A (en) * | 2021-12-15 | 2022-03-18 | 中国科学院深圳先进技术研究院 | Gait detection method and device based on reservoir model and computer storage medium |
| CN114202032B (en) * | 2021-12-15 | 2023-07-18 | 中国科学院深圳先进技术研究院 | Gait detection method, device and computer storage medium based on reserve pool model |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| Papageorgiou et al. | Fuzzy cognitive map learning based on nonlinear Hebbian rule | |
| CN105279555B (en) | A kind of adaptive learning neural network implementation method based on evolution algorithm | |
| CN105243454A (en) | Big data-based electrical load prediction system | |
| CN105894379A (en) | System and method for generating financial product transaction strategy | |
| CN108090559A (en) | A kind of construction method of antithesis reserve pool neural network model | |
| CN105427241A (en) | Distortion correction method for large-field-of-view display device | |
| Alizadeh et al. | Content analysis using fuzzy cognitive map (FCM): a guide to capturing causal relationships from secondary sources of data | |
| CN115017178A (en) | Training method and device for data-to-text generation model | |
| CN107563518A (en) | A kind of learning method of the extreme learning machine based on social force model colony optimization algorithm | |
| CN109507891B (en) | Semi-active fuzzy control method | |
| CN109886405A (en) | It is a kind of inhibit noise based on artificial neural network structure's optimization method | |
| CN114202063A (en) | Fuzzy neural network greenhouse temperature prediction method based on genetic algorithm optimization | |
| Wu et al. | Forecasting stock market performance using hybrid intelligent system | |
| Lebiere et al. | Convergence and constraints revealed in a qualitative model comparison | |
| CN117807895B (en) | Magnetorheological damper control method and device based on deep reinforcement learning | |
| Grosan et al. | Fuzzy expert systems | |
| Zheng et al. | Generating and optimizing a funicular arch floor structure | |
| CN110910164A (en) | Product sales forecasting method, system, computer device and storage medium | |
| Wout et al. | Learning Gaussian policies from corrective human feedback | |
| CN102662916A (en) | Lagrange function based least-squares multi-objective optimization method | |
| CN118131634B (en) | Generalized second-class fuzzy self-adaptive control method and system based on chaotic system | |
| CN117172283B (en) | Method for constructing offspring model by selecting learning genes from ancestors through gradient sparsity | |
| Dunin-Barkowski et al. | Prognosis of dynamical systems behavior based on cerebellar-type neural technologies | |
| Chen et al. | Linguistic hedges and fuzzy rule based systems | |
| Johansson et al. | Adaptive Model-Free Control Applied to Truck Front Wheel Drive: Real time control with reinforcement learning utilising recurrent deterministic policy gradient |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20180529 |
|
| RJ01 | Rejection of invention patent application after publication |