CN202472729U - SIMF Elman network structure - Google Patents

Abstract

An SIMF (Seasonal Input Multilayer Feedback) Elman network structure relates to a network structure, and comprises a network flow monitoring device (1), a data processing device (2), and an SIMF Elman neural network (3); wherein, the network flow monitoring device (1) is connected with the data processing device (2), and the data processing device (2) is connected with the SIMF Elman neural network (3). The SIMF Elman network structure of the utility model has the beneficial effects that: the SIMF Elman network structure introduces a chaotic mechanism to the SIMF Elman network, and utilizes inherent global moving characteristic of the chaotic mechanism to escape from the local minimum point existing in a weight optimization process; in addition, the SIMF Elman network structure studies chaotic characteristics of a Tent mapping, and constructs a corresponding chaotic training algorithm, wherein, the algorithm has a better ergodic uniformity, can search a global optimal solution, and has a quicker search speed; and the SIMF Elman network structure also avoids the problems that a standard Elman neural network has a slow convergence speed and is easy to fall into a local minimum value.

Description

Seasonal input Multi-Layer Feedback Elman network structure

Technical field:

The utility model relates to a kind of network structure, is specifically related to a kind of seasonal input Multi-Layer Feedback Elman network structure.

Background technology:

The Elman neural network is a time delay feedback-type neural network in the typical local recursion, and on the basis of BP neural network, through introducing feedback signal, the storage internal state makes its tool respectively shine upon dynamic function.It is except having input layer, hidden layer, output layer unit; Also has a special layer of accepting; Accepting layer is the output valve that is used for remembering hidden layer unit previous moment, can think that a step postpones, and the delay of the output of hidden layer through accepting layer, storage, is linked to the input of hidden layer certainly.Thisly make it have susceptibility to historical data from the couplet mode, the adding of internal feedback network has increased network itself and has handled the ability of multidate information, thereby reaches the purpose of dynamic modeling.

The mathematical model of Elman neural network: like Fig. 1

If network be input as x (k), x ' is the output of hidden layer (k), x _c(k) for accepting layer output; w ¹, w ², w ³Be respectively input layer to hidden layer, accept layer to hidden layer, the connection weights of hidden layer to output layer; F () is the transport function of hidden layer neuron, often adopts the Sigmoid function, and α (0≤α＜1) is for accepting a layer self feed back gain factor.The mathematical model of network is:

y(k)＝g(W ³x′(k)) (1)

x′(k)＝f(W ¹x(k)+W ²x _c(k)) (2)

x _c(k)＝x′(k-1)+αx _c(k-1) (3)

The tradition neural network model is only studied the time series forecasting with taxis; In addition; The tradition neural network adopts the BP algorithm or basically based on the hybrid algorithm of neural network; These algorithms all have weak point, need find the solution the derivative of objective function based on the gradient descent algorithm of BP, and the scope of application has limitation; The simple problem complicacy is talked about by hybrid algorithm regular meeting based on neural network, also is easier to occur the local convergence problem.

The utility model content:

The purpose of the utility model provides a kind of seasonal input Multi-Layer Feedback Elman network structure; It can be introduced mechanism of chaos in the SIMF Elman network, utilizes the intrinsic overall situation of mechanism of chaos to move about, and escapes from the local minimum point that exists in the weights optimizing process; Studied the chaotic characteristic of Tent mapping; And constructed corresponding chaos training algorithm, and algorithm has better traversal homogeneity, and this algorithm can be searched for globally optimal solution; And search speed is faster arranged, also avoid the speed of convergence of standard Elman neural network slow, be prone to be absorbed in problem such as local minimum.

In order to solve the existing problem of background technology; The utility model is taked following technical scheme: it comprises network traffics monitoring device 1, data processing equipment 2 and SIMF Elman neural network 3; Network traffics monitoring device 1 links to each other with data processing equipment 2, and data processing equipment 2 links to each other with SIMF Elman neural network 3.

The SIMF Elman network Model of the utility model:

y(k)＝g(W ³x′(k)+W ⁵x _c2(k)) (4)

x′(k)＝f(W ¹x(k)+W ²x _c1(k)+W ⁴x _c2(k)) (5)

x _c1(k)＝x′(k-1)+αx _c1(k-1) (6)

x _c2(k)＝y(k-1)+βx _c2(k-1) (7)

W wherein ¹, w ², w ³, w ⁴, w ⁵Be respectively input layer to hidden layer, accept layer 1 to hidden layer, hidden layer is accepted layer 2 to hidden layer to output layer, accepts the layer 2 connection weights to output layer; X (k) is the input of network, and x ' is the output of hidden layer (k), x _C1(k) and x _C2(k) be respectively the output of accepting layer 1 and accepting layer 2, y (k) is the output of network; α (0≤α＜1) and β (0≤β＜1) are for accepting layer 1 and the self feed back gain factor of accepting layer 2.

Accept the introducing of layer 2 and seasonal periodicity algorithm in the SIMF Elman network; Realized the full recurrence of network; Make system not only can dynamically recall internal state information, and can dynamic tracking the variation of output, further improved the approximation capability of network and dynamic mapping ability.

The utility model can be introduced mechanism of chaos in the SIMF Elman network, utilizes the intrinsic overall situation of mechanism of chaos to move about, and escapes from the local minimum point that exists in the weights optimizing process; Studied the chaotic characteristic of Tent mapping; And constructed corresponding chaos training algorithm, and algorithm has better traversal homogeneity, and this algorithm can be searched for globally optimal solution; And search speed is faster arranged, also avoid the speed of convergence of standard Elman neural network slow, be prone to be absorbed in problem such as local minimum.

Description of drawings:

Fig. 1 is the structural representation in the utility model background technology;

Fig. 2 is the structural representation of the utility model;

Fig. 3 is the concrete structure synoptic diagram of Fig. 2.

Embodiment:

With reference to Fig. 1-3; This embodiment is taked following technical scheme: it comprises network traffics monitoring device 1, data processing equipment 2 and SIMF Elman neural network 3; Network traffics monitoring device 1 links to each other with data processing equipment 2, and data processing equipment 2 links to each other with SIMF Elman neural network 3.

The SIMF Elman network Model of this embodiment:

y(k)＝g(W ³x′(k)+W ⁵x _c2(k)) (4)

x′(k)＝f(W ¹x(k)+W ²x _c1(k)+W ⁴x _c2(k)) (5)

x _c1(k)＝x′(k-1)+αx _c1(k-1) (6)

x _c2(k)＝y(k-1)+βx _c2(k-1) (7)

W wherein ¹, w ², w ³, w ⁴, w ⁵Be respectively input layer to hidden layer, accept layer 1 to hidden layer, hidden layer is accepted layer 2 to hidden layer to output layer, accepts the layer 2 connection weights to output layer; X (k) is the input of network, and x ' is the output of hidden layer (k), x _C1(k) and x _C2(k) be respectively the output of accepting layer 1 and accepting layer 2, y (k) is the output of network; α (0≤α＜1) and β (0≤β＜1) are for accepting layer 1 and the self feed back gain factor of accepting layer 2.

Accept the introducing of layer 2 and seasonal periodicity algorithm in the SIMF Elman network; Realized the full recurrence of network; Make system not only can dynamically recall internal state information, and can dynamic tracking the variation of output, further improved the approximation capability of network and dynamic mapping ability.

The Learning Algorithm of this embodiment:

If the input layer of network is a m node; Accept layer 1 and be r node; Accept layer 2 and be n node, actual being output as

the definition error function of establishing k step system is:

$E\left(k\right)=\frac{1}{2}{(\hat{y}\left(k\right)-y\left(k\right))}^T(\hat{y}\left(k\right)-y\left(k\right))---\left(8\right)$

Calculate E (k) respectively each connected the partial derivative of authority credentials, adopt the dynamic BP algorithm that weights are revised, can obtain the basic studies algorithm of SIMF Elman network:

$\mathrm{\&Delta;}{\mathrm{<figure-callout\; id="1"\; label="w\; ji"\; filenames="HSA00000638968900011.png"\; state="\{\{state\}\}">w</figure-callout>}}_{\mathrm{ji}}^{}={\mathrm{\&sigma;}}_1\underset{h=1}{\overset{n}{\mathrm{\&Sigma;}}}\left[({\hat{y}}_h\left(k\right)-y_h\left(k\right))w_{\mathrm{hj}}^3{f}_j^{\&prime;}(\&CenterDot;)\right]x_i\left(k\right);j=\mathrm{1,2},...,r;i=\mathrm{1,2},...,m---\left(9\right)$

In like manner can obtain w ², w ³, w ⁴, w ⁵The study step-length.

Calculate the partial derivative of E (k) respectively, and make it equal 0, the Elman Learning Algorithms after can being improved the connection authority credentials:

$\mathrm{\&Delta;}{\mathrm{<figure-callout\; id="1"\; label="w\; ji"\; filenames="HSA00000638968900011.png"\; state="\{\{state\}\}">w</figure-callout>}}_{\mathrm{ji}}^{}={\mathrm{\&sigma;}}_1\underset{h=1}{\overset{n}{\mathrm{\&Sigma;}}}\left[({\hat{y}}_h\left(k\right)-y_h\left(k\right))w_{\mathrm{hj}}^3{f}_j^{\&prime;}(\&CenterDot;)\right]x_i\left(k\right);j=\mathrm{1,2},...,r;i=\mathrm{1,2},...,m---\left(10\right)$

$\mathrm{\&Delta;}{w}_{\mathrm{jl}}^2={\mathrm{\&sigma;}}_2\underset{h=1}{\overset{n}{\mathrm{\&Sigma;}}}\left[({\hat{y}}_h\left(k\right)-y_h\left(k\right))w_{\mathrm{hj}}^3{f}_j^{\&prime;}\right]\frac{{\&PartialD;x}_j^{\&prime;}\left(k\right)}{\&PartialD;w_{\mathrm{jl}}^2};j=\mathrm{1,2},...,r;l=\mathrm{1,2},..,r---\left(11\right)$

$\mathrm{\&Delta;}{\mathrm{<figure-callout\; id="3"\; label="wh\; hj"\; filenames="HSA00000638968900011.png"\; state="\{\{state\}\}">wh</figure-callout>}}_{\mathrm{hj}}^{}={\mathrm{\&sigma;}}_3({\hat{y}}_h\left(k\right)-y_h\left(k\right))x_j^{\&prime;}\left(k\right);h=\mathrm{1,2},...,n;j=\mathrm{1,2},...,r---\left(12\right)$

$\mathrm{\&Delta;}{w}_{\mathrm{jq}}^4={\mathrm{\&sigma;}}_4\underset{h=1}{\overset{n}{\mathrm{\&Sigma;}}}\left[({\hat{y}}_h\left(k\right)-y_h\left(k\right))w_{\mathrm{hj}}^3{f}_j^{\&prime;}\right]\frac{{\&PartialD;x}_j^{\&prime;}\left(k\right)}{{\&PartialD;w}_{\mathrm{jq}}^4};j=1.2.....r;q=\mathrm{1,2},...,n---\left(13\right)$

$\mathrm{\&Delta;}{w}_{\mathrm{hq}}^5={\mathrm{\&sigma;}}_5({\hat{y}}_h\left(k\right)-y_h\left(k\right))x_{c2,q}\left(k\right);h=\mathrm{1,2},...,n;q=\mathrm{1,2},...,n---\left(14\right)$

In the formula:

$\frac{{\&PartialD;x}_j^{\&prime;}\left(k\right)}{{\&PartialD;w}_{\mathrm{jl}}^2}=f^{\&prime;}(\&CenterDot;)x_l^{\&prime;}(k-1)+\mathrm{\&alpha;}\frac{{\&PartialD;x}_j^{\&prime;}(k-1)}{{\&PartialD;w}_{\mathrm{jl}}^2};j=\mathrm{1,2},...,r;l=\mathrm{1,2},...,r---\left(15\right)$

$\frac{{\&PartialD;x}_j^{\&prime;}\left(k\right)}{{\&PartialD;w}_{\mathrm{jq}}^4}=f_j^{\&prime;}(\&CenterDot;)y_q(k-1)+\mathrm{\&beta;}\frac{{\&PartialD;x}_j^{\&prime;}(k-1)}{{\&PartialD;w}_{\mathrm{jq}}^4};j=\mathrm{1,2},...,r;q=\mathrm{1,2},...,n---\left(16\right)$

σ wherein ₁, σ ₂, σ ₃, σ ₄, σ ₅, be respectively w ¹, w ², w ³, w ⁴, w ⁵The study step-length; M, n, r are respectively the number of input layer, output layer, hidden layer neuron.Formula (15) and (16) have constituted the dynamic recurrence relation of gradient

and , thereby can realize the effective identification to high order system.

In addition; Elman network after the improvement has increased from output layer and has been connected through the feedback of a step time-delay back to hidden layer and output layer; And the introducing of assembling the periodicity algorithm; Utilize the gradient descent method to ask the backpropagation of weights variation and error, make network possess the ability of handling multidate information, make things convenient for the modeling of dynamic process.

The weights of output layer change.Weights to being input to k output from i have:

$\mathrm{\&Delta;}{w}_{2_{\mathrm{Kj}}}=-\mathrm{\&eta;}\frac{\&PartialD;E}{{\&PartialD;w}_{2_{\mathrm{Kj}}}}=-\mathrm{\&eta;}\frac{\&PartialD;E}{{\&PartialD;a}_{2_k}}\frac{\&PartialD;a_{2_k}}{{\&PartialD;w}_{2_{\mathrm{Ki}}}}=\mathrm{\&eta;}(t_k-a_{2_k})f_{2^{\&prime;}}{a}_{1_i}=\mathrm{\&eta;}{\mathrm{\&delta;}}_{\mathrm{Ki}}{a}_{1_i},$ Wherein, ${\mathrm{\&delta;}}_{\mathrm{Kj}}=(t_k-a_{2_k}).$

In like manner can get: $\mathrm{\&Delta;}{b}_{2_{\mathrm{Kj}}}=-\mathrm{\&eta;}\frac{\&PartialD;E}{\&PartialD;b_{2_{\mathrm{Kj}}}}=-\mathrm{\&eta;}\frac{\&PartialD;E}{{\&PartialD;a}_{2_k}}\frac{\&PartialD;a_{2_k}}{{\&PartialD;b}_{2_{\mathrm{Ki}}}}=\mathrm{\&eta;}(t_k-a_{2_k})f_{2^{\&prime;}}=\mathrm{\&eta;}{\mathrm{\&delta;}}_{\mathrm{Ki}}.$

The hidden layer weights change.Weights to being input to i output from j have:

$\mathrm{\&Delta;}{w}_{1_{\mathrm{Kih}}}=-\mathrm{\&eta;}\frac{\&PartialD;E}{\&PartialD;w_{1_{\mathrm{Ij}}}}=-\mathrm{\&eta;}\frac{\&PartialD;E}{{\&PartialD;a}_{2_k}}\frac{\&PartialD;a_{2_k}}{{\&PartialD;a}_{1_i}}\frac{\&PartialD;a_{1_i}}{{\&PartialD;w}_{1_{\mathrm{Ij}}}}=\mathrm{\&eta;}\underset{k=1}{\overset{s_2}{\mathrm{\&Sigma;}}}(t_k-a_{2_k})f_{2^{\&prime;}}{w}_{2_{\mathrm{Ki}}}{\mathrm{<figure-callout\; id="1"\; label="f"\; filenames="HSA00000638968900011.png"\; state="\{\{state\}\}">f</figure-callout>}}_{1^{\&prime;}}{p}_j=\mathrm{\&eta;}{\mathrm{\&delta;}}_{\mathrm{Ki}}{p}_j,$ Wherein

δ _ij＝e _if ₁′， $e_i=\underset{k=1}{\overset{s_2}{\mathrm{\&Sigma;}}}{\mathrm{\&delta;}}_{\mathrm{ki}}{w}_{2_{\mathrm{ki}}}.$

The seasonal periodicity algorithm of the input and output node of this embodiment

(1) general choosing method

Object vector be P (i, j), i=1,2 ..., 24 totally 24 data; Input vector be P (i, j-1), i=1,2 ..., 24 totally 24 data.The input layer that is network training adopts 24 neurons, shines upon the prediction flow of 24 hours the previous days respectively, and promptly input matrix is the row vector of P (1 * 24); Output layer adopts 24 neurons, following 1 day 24 hours flow of mapping, and promptly output matrix is the row vector of T (1 * 24).The relation of input variable and output variable is shown in table 3-1:

Table 1 is generally chosen neural network input and output vector relation table

Input point is 48, and output point is 24, and these group data are called first group of data

(2) seasonal cycle choosing method

If hope promptly to carry out M step prediction with the value in following M (M >=1) the individual moment of the individual data prediction of N (N >=1) in the past, a desirable N adjacent sample, and they are mapped as M value; The sample predicted value in this M value representative M the moment this window after, network traffics time series are that to have with the sky be the periodic characteristics of unit, to the network traffics data modeling of collection; Following one day forecast model is following: object vector be P (i, j), i=1; 2 ..., 24 totally 24 data; Input vector be P (i, j-2), P (i, j-1), i=1,2 ..., 24 totally 48 data.The input layer that is network training adopts 48 neurons, and preceding 2 days 48 hours flow is predicted in mapping respectively, and the input window of seasonal periodicity is 2 cycles, and promptly input matrix is the row vector of P (1 * 48); Output layer adopts 24 neurons, following 1 day 24 hours flow of mapping, and promptly output matrix is the row vector of T (1 * 24).The relation of input variable and output variable is shown in table 3-2:

Table 3-2 seasonal cycle is chosen neural network input and output vector relation table

Input point is 48, and output point is 24, and these group data are called second group of data.

(3) training parameter is selected

Before adjustment connects power; Must connect power to each earlier and give initial value at random, hope that generally initial weight can make each neuronic state value approach zero when input adds up, but can not equal same number; Otherwise system can not train; In training process, can consider to adjust automatically learning rate, the formula of regulating learning rate is shown below:

Wherein, SSE be the network output error with, choosing of initial learn speed η (0) has very big randomness, generally is taken as 0.01; k _IncAnd k _DecBe the increment factor and the decrement factor, initial value is 1.05 and 0.7; Can reduce error through adjusting two variablees.

This embodiment can be introduced mechanism of chaos in the SIMF Elman network, utilizes the intrinsic overall situation of mechanism of chaos to move about, and escapes from the local minimum point that exists in the weights optimizing process; Studied the chaotic characteristic of Tent mapping; And constructed corresponding chaos training algorithm, and algorithm has better traversal homogeneity, and this algorithm can be searched for globally optimal solution; And search speed is faster arranged, also avoid the speed of convergence of standard Elman neural network slow, be prone to be absorbed in problem such as local minimum.

Claims (1)

1. seasonal input Multi-Layer Feedback Elman network structure; It is characterized in that it comprises network traffics monitoring device (1), data processing equipment (2) and SIMF Elman neural network (3); Network traffics monitoring device (1) links to each other with data processing equipment (2), and data processing equipment (2) links to each other with SIMF Elman neural network (3).

Images