CN111832623A - Echo state network time sequence prediction algorithm based on phase space reconstruction - Google Patents
Echo state network time sequence prediction algorithm based on phase space reconstruction Download PDFInfo
- Publication number
- CN111832623A CN111832623A CN202010533889.5A CN202010533889A CN111832623A CN 111832623 A CN111832623 A CN 111832623A CN 202010533889 A CN202010533889 A CN 202010533889A CN 111832623 A CN111832623 A CN 111832623A
- Authority
- CN
- China
- Prior art keywords
- echo state
- state network
- prediction
- phase space
- time
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/049—Temporal neural networks, e.g. delay elements, oscillating neurons or pulsed inputs
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
- G06F18/214—Generating training patterns; Bootstrap methods, e.g. bagging or boosting
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/25—Fusion techniques
- G06F18/254—Fusion techniques of classification results, e.g. of results related to same input data
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Data Mining & Analysis (AREA)
- Physics & Mathematics (AREA)
- Life Sciences & Earth Sciences (AREA)
- Artificial Intelligence (AREA)
- General Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- Evolutionary Computation (AREA)
- Bioinformatics & Computational Biology (AREA)
- Evolutionary Biology (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Computational Linguistics (AREA)
- Biomedical Technology (AREA)
- Biophysics (AREA)
- Health & Medical Sciences (AREA)
- General Health & Medical Sciences (AREA)
- Molecular Biology (AREA)
- Computing Systems (AREA)
- Mathematical Physics (AREA)
- Software Systems (AREA)
- Cable Transmission Systems, Equalization Of Radio And Reduction Of Echo (AREA)
Abstract
The invention discloses an echo state network time sequence prediction algorithm based on phase space reconstruction, which comprises the following steps: (1) determining two parameters of delay time and embedding dimension of phase space reconstruction, reconstructing a time sequence into a plurality of groups of data by utilizing the phase space reconstruction, and mapping the data from a low-dimensional space to a high-dimensional characteristic space; (2) determining parameters of an echo state network reserve pool, initializing an echo state network connection weight matrix, and constructing an echo state network model; (3) by utilizing the characteristic that the echo state network model can carry out multi-input, training and reconstructing a plurality of groups of data by using the model, and carrying out direct multi-step prediction and iterative multi-step prediction; (4) and directly fusing and iterating the multi-step prediction results by using weighted average to obtain a fused multi-step prediction value. The invention has better prediction precision in the multi-step prediction of the time sequence and can accurately predict the change trend of the time sequence.
Description
Technical Field
The invention relates to the technical field of neural networks and time sequence analysis and prediction, in particular to an echo state network time sequence prediction algorithm based on phase space reconstruction.
Background
The traditional echo state network overcomes the problem of low training efficiency of the recurrent neural network, avoids the problems of local optimization and the like in the training process, and on the other hand, along with the deep research, some limitations of the traditional echo state network gradually appear, and in many cases, for some complex time sequence prediction problems, satisfactory prediction precision cannot be obtained by simply using the traditional echo state network. Therefore, it is necessary to combine the echo state network with other prediction or signal preprocessing methods, and to propose a combined echo state network prediction method to improve the prediction accuracy.
Time series multi-step prediction estimates the sequence value for a longer period of time in the future based on the current value of the sequence, which is clearly more challenging than single-step prediction, which only needs to estimate the sequence value of the next step based on the current value of the sequence.
Disclosure of Invention
The technical problem to be solved by the invention is to provide an echo state network time sequence prediction algorithm based on phase space reconstruction, which can be applied to time sequence analysis and prediction.
In order to solve the above technical problem, the present invention provides an echo state network time series prediction algorithm based on phase space reconstruction, which comprises the following steps:
(1) determining two parameters of delay time and embedding dimension of phase space reconstruction, reconstructing a time sequence into a plurality of groups of data by utilizing the phase space reconstruction, and mapping the data from a low-dimensional space to a high-dimensional characteristic space;
(2) determining parameters of an echo state network reserve pool, initializing an echo state network connection weight matrix, and constructing an echo state network model;
(3) by utilizing the characteristic that the echo state network model can carry out multi-input, training and reconstructing a plurality of groups of data by using the model, and carrying out direct multi-step prediction and iterative multi-step prediction;
(4) and directly fusing and iterating the multi-step prediction results by using weighted average to obtain a fused multi-step prediction value.
Preferably, in the step (1), reconstructing the time series into a plurality of sets of data by using phase space reconstruction includes the following steps:
(11) with two discrete time sequences s1,…,s1},{q1,…,q1What is needed isThe information entropy of the system S and the system Q is as follows:
in the above formula PS(si),PQ(qi) Respectively representing events S in S and QiAnd q isiThe probability of (d);
recording events siAnd q isiHas a joint distribution probability of PS,Q(si,qi) The joint information entropy of system S and system Q is as follows:
a mutual information calculation formula of the systems S and Q can be obtained:
I(S,Q)=H(S)+H(Q)-H(S,Q)
(12) providing a set of time sequences (u (1), …, u (t), …, u (N)), wherein N is the sequence length;
defining (S, Q) ═ u (I), (I + τ)), (1 ≦ I ≦ N- τ), where S represents u (I) and Q represents u (I + τ), then I (S, Q) in step (11) is a function of the delay time τ, denoted as I (τ);
the delay time tau is calculated by the following steps: first, τ is taken as a value densely, and mutual information corresponding to different τ values is respectively calculated:
I(τ)=H(x)+H(x+τ)-H(x,x+τ)
when I (tau) reaches a minimum value point, the corresponding tau is the delay time of phase space reconstruction;
(13) constructing a vector yi(d) (i + (d-1) τ)), (1 ≦ i ≦ N- (d-1) τ), wherein τ is the delay time in step (11);
yi(d) there is a closest point of approach yn(i,d)(d) (N (i, d) ≠ i, N (i, d) ≦ N- (d-1) τ) 1, and the Euclidean distance between two points is as follows:
Ri(d)=||yi(d)-yn(i,d)(d)||2
when the dimension increases to d +1, the distance between two points in the new space changes, the new distance being Ri(d +1), and
when the dimension becomes larger, the Euclidean distance between two points also becomes larger, such that
If a is1(i,d)>RτThen yn(i,d)(d) Is yi(d) A false closest point of approach, here RτIs a threshold value;
calculating the proportion of the false nearest points to the whole data volume from the minimum value 2 of the dimension, and then gradually increasing the dimension m until the proportion of the false nearest points is less than 5%, wherein the dimension m is the embedding dimension;
(14) the time sequence { u (N) ═ 1,2, …, N }, after phase space reconstruction, an m-dimensional data combination can be obtained:
u(n)=[u(n),u(n+τ),…,u(n+(m-1)τ)]。
preferably, the step (2) specifically comprises the following steps:
(21) the predictive effect of the echo state network is influenced by pool parameters including: the method comprises the following steps of (1) obtaining a reserve pool sparsity degree SD, a reserve pool unit scale IS, a reserve pool scale N and a spectrum radius SR of a reserve pool internal connection weight;
(22) in the initialization phase, the input connection right between the input layer and the reserve pool is randomly generatedConnection right W inside reserve tankresThe weight matrixes are always fixed and do not need to be updated through training, and the feedback connection weights between the reserve pool and the output layerIs set to 0.
Preferably, the step (3) specifically comprises the following steps:
(31) setting the network input and output of the echo state as u (n) and y (n), respectively, the internal state updating equation of the reserve pool is as follows:
wherein f is an activation function of a reserve pool node, and specifically a hyperbolic tangent function is used;
updating of output connection right requires internal state vector data support, and internal state vectors (x) in the reserve pool are collected at each moment1(i),x2(i),…,xN(i) Group to form a matrix B; collecting output values y (n) at each moment to form a matrix T;
the model output equation is as follows:
y(t+1)=Wout(xT(n),u(n),y(n))
in the formula WoutIs the output connection right;
to make the model predict the valueMinimum mean square error between the sum of the true values y (n), WoutThe update is to solve the following formula:
solving the above formula by a least square method, the following can be obtained:
(Wout)T=B-1T
(32) direct multi-step prediction is that a multi-step predictor is directly constructed to satisfy
In the formula, h represents a prediction time interval, u (n) represents reconstructed data, u (n) is used as the model input in the step (31), and u (n + h) is used as the output to obtain a predicted value;
(33) iterative multi-step prediction firstly determines a single-step predictor satisfying
Taking u (n) as the model input in the step (31) and u (n +1) as the output, the predicted value at the next moment is obtainedCombining n data from the time 2 to the time n +1 to obtain an input value u (n +1) at the time n + 1; the above formula is used for multiple times to carry out single step prediction, the predicted value of a future time point is obtained each time, and after iteration is carried out for h times, the h-step predicted value of a time sequence is obtained
Preferably, in the step (4), a plurality of predicted values obtained in the step (3) are fused, that is
In the formula, f (-) represents a fusion algorithm, and a fusion algorithm with lower complexity is used, namely a plurality of predicted values are weighted and averaged:
a1、a2representing the weight.
The invention has the beneficial effects that: (1) the echo state network time sequence prediction algorithm based on phase space reconstruction can accurately predict the time sequence change trend; the phase space reconstruction is combined with the echo state network, the phase space reconstruction has good nonlinear analysis capability in processing the time sequence, and the echo state network has the advantages of low algorithm complexity, difficult falling into local optimum and the like in processing the time sequence, so that the problem of poor prediction accuracy of the existing model to the time sequence is solved; (2) experimental data show that when the prediction step length is 5, the average absolute error of the method is improved by 20% compared with the traditional echo state network; when the predicted step length is 15%, the average absolute error of the method is improved by 40% compared with the traditional echo state network; the invention has better prediction precision in the multi-step prediction of the time sequence and can accurately predict the change trend of the time sequence.
Drawings
FIG. 1 is a schematic diagram of the algorithm flow of the present invention.
Fig. 2 is a schematic diagram of an echo state network model according to the present invention.
FIG. 3 is a schematic diagram of the multi-step prediction error of the present invention.
Detailed Description
As shown in fig. 1, an echo state network time series prediction algorithm based on phase space reconstruction includes the following steps:
(1) determining two parameters of delay time and embedding dimension of phase space reconstruction, reconstructing a time sequence into a plurality of groups of data by utilizing the phase space reconstruction, and mapping the data from a low-dimensional space to a high-dimensional characteristic space;
(2) determining parameters of an echo state network reserve pool, initializing an echo state network connection weight matrix, and constructing an echo state network model;
(3) by utilizing the characteristic that the echo state network can input multiple inputs, training and reconstructing multiple groups of data by using a model, and performing direct multi-step prediction and iterative multi-step prediction;
(4) and directly fusing and iterating the multi-step prediction results by using weighted average to obtain a fused multi-step prediction value.
In the embodiment, the echo state network time sequence prediction algorithm based on phase space reconstruction is applied to voltage time sequence prediction to show the effectiveness of the echo state network time sequence prediction algorithm, and the advantage of the echo state network time sequence prediction algorithm is shown by comparing the prediction accuracy with the prediction accuracy of the traditional echo state network algorithm.
FIG. 1 is a drawing ofAnd (3) predicting the time sequence of the echo state network based on phase space reconstruction. FIG. 2 is a block diagram of an echo state network in which the output connection W from the pool to the output layeroutIs the only weight that needs to be updated by training, and WoutThe updating of (1) is to solve a simple linear regression problem, and the calculation amount is extremely small. The echo state network has a simpler structure than the traditional recurrent neural network and has an extremely simple network training process. Fig. 3 is a voltage multi-step prediction error graph including a 5-step prediction error, a 10-step prediction error, and a 15-step prediction error, where the abscissa represents a predicted time point and the ordinate represents an error value. It can be seen that the time sequence prediction effect of the echo state network model based on the phase space reconstruction is good, and the predicted value is consistent with the real value.
Table 1 is a comparison diagram of the voltage multi-step prediction accuracy between the method of the present invention and the conventional echo state network in this embodiment. Therefore, the method is obviously superior to the traditional echo state network method, and the prediction precision is improved more obviously along with the increase of the prediction step length. For example, when the prediction is carried out in 15 steps, the prediction accuracy of the method is improved by 40% compared with that of the traditional echo state network.
TABLE 1 comparison of voltage multistep prediction accuracy between the method of the present invention and the conventional echo state network
Claims (5)
1. An echo state network time sequence prediction algorithm based on phase space reconstruction is characterized by comprising the following steps:
(1) determining two parameters of delay time and embedding dimension of phase space reconstruction, reconstructing a time sequence into a plurality of groups of data by utilizing the phase space reconstruction, and mapping the data from a low-dimensional space to a high-dimensional characteristic space;
(2) determining parameters of an echo state network reserve pool, initializing an echo state network connection weight matrix, and constructing an echo state network model;
(3) by utilizing the characteristic that the echo state network model can carry out multi-input, training and reconstructing a plurality of groups of data by using the model, and carrying out direct multi-step prediction and iterative multi-step prediction;
(4) and directly fusing and iterating the multi-step prediction results by using weighted average to obtain a fused multi-step prediction value.
2. The echo state network time series prediction algorithm based on phase space reconstruction as claimed in claim 1, wherein in the step (1), the phase space reconstruction is used to reconstruct the time series into a plurality of groups of data, comprising the following steps:
(11) with two discrete time sequences s1,…,s1},{q1,…,q1And information entropy of the system S and the system Q is as follows:
in the above formula PS(si),PQ(qi) Respectively representing events S in S and QiAnd q isiThe probability of (d);
recording events siAnd q isiHas a joint distribution probability of PS,Q(si,qi) The joint information entropy of system S and system Q is as follows:
a mutual information calculation formula of the systems S and Q can be obtained:
I(S,Q)=H(S)+H(Q)-H(S,Q)
(12) providing a set of time sequences (u (1), …, u (t), …, u (N)), wherein N is the sequence length;
defining (S, Q) ═ u (I), (I + τ)), (1 ≦ I ≦ N- τ), where S represents u (I) and Q represents u (I + τ), then I (S, Q) in step (11) is a function of the delay time τ, denoted as I (τ);
the delay time tau is calculated by the following steps: first, τ is taken as a value densely, and mutual information corresponding to different τ values is respectively calculated:
I(τ)=H(x)+H(x+τ)-H(x,x+τ)
when I (tau) reaches a minimum value point, the corresponding tau is the delay time of phase space reconstruction;
(13) constructing a vector yi(d) (i + (d-1) τ)), (1 ≦ i ≦ N- (d-1) τ), wherein τ is the delay time in step (11);
yi(d) there is a closest point of approach yn(i,d)(d) (N (i, d) ≠ i, N (i, d) ≦ N- (d-1) τ) 1, and the Euclidean distance between two points is as follows:
Ri(d)=||yi(d)-yn(i,d)(d)||2
when the dimension increases to d +1, the distance between two points in the new space changes, the new distance being Ri(d +1), and
when the dimension becomes larger, the Euclidean distance between two points also becomes larger, such that
If a is1(i,d)>RτThen yn(i,d)(d) Is yi(d) A false closest point of approach, here RτIs a threshold value;
calculating the proportion of the false nearest points to the whole data volume from the minimum value 2 of the dimension, and then gradually increasing the dimension m until the proportion of the false nearest points is less than 5%, wherein the dimension m is the embedding dimension;
(14) the time sequence { u (N) ═ 1,2, …, N }, after phase space reconstruction, an m-dimensional data combination can be obtained:
u(n)=[u(n),u(n+τ),…,u(n+(m-1)τ)]。
3. the echo state network time series prediction algorithm based on phase space reconstruction as claimed in claim 1, wherein the step (2) specifically comprises the following steps:
(21) the predictive effect of the echo state network is influenced by pool parameters including: the method comprises the following steps of (1) obtaining a reserve pool sparsity degree SD, a reserve pool unit scale IS, a reserve pool scale N and a spectrum radius SR of a reserve pool internal connection weight;
(22) in the initialization phase, the input connection right between the input layer and the reserve pool is randomly generatedConnection right W inside reserve tankresThe weight matrixes are always fixed and do not need to be updated through training, and the feedback connection weights between the reserve pool and the output layerIs set to 0.
4. The echo state network time series prediction algorithm based on phase space reconstruction as claimed in claim 1, wherein the step (3) comprises the following steps:
(31) setting the network input and output of the echo state as u (n) and y (n), respectively, the internal state updating equation of the reserve pool is as follows:
wherein f is an activation function of a reserve pool node, and specifically a hyperbolic tangent function is used;
updating of output connection right requires internal state vector data support, and internal state vectors (x) in the reserve pool are collected at each moment1(i),x2(i),…,xN(i) Group to form a matrix B; collecting output values y (n) at each moment to form a matrix T;
the model output equation is as follows:
y(t+1)=Wout(xT(n),u(n),y(n))
in the formula WoutIs the output connection right;
to make the model predict the valueMinimum mean square error between the sum of the true values y (n), WoutThe update is to solve the following formula:
solving the above formula by a least square method, the following can be obtained:
(Wout)T=B-1T
(32) direct multi-step prediction is that a multi-step predictor is directly constructed to satisfy
In the formula, h represents a prediction time interval, u (n) represents reconstructed data, u (n) is used as the model input in the step (31), and u (n + h) is used as the output to obtain a predicted value;
(33) iterative multi-step prediction firstly determines a single-step predictor satisfying
Taking u (n) as the model input in the step (31) and u (n +1) as the output, the predicted value at the next moment is obtainedCombining n data from the time 2 to the time n +1 to obtain an input value u (n +1) at the time n + 1; the above formula is used for multiple times to carry out single step prediction, the predicted value of a future time point is obtained each time, and after iteration is carried out for h times, the h-step predicted value of a time sequence is obtained
5. The echo state network time series prediction algorithm based on phase space reconstruction as claimed in claim 1, characterized in that in step (4), the multiple predicted values obtained in step (3) are fused, i.e. the multiple predicted values are fused
In the formula, f (-) represents a fusion algorithm, and a fusion algorithm with lower complexity is used, namely a plurality of predicted values are weighted and averaged:
a1、a2representing the weight.
Priority Applications (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CN202010533889.5A CN111832623A (en) | 2020-06-12 | 2020-06-12 | Echo state network time sequence prediction algorithm based on phase space reconstruction |
Applications Claiming Priority (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CN202010533889.5A CN111832623A (en) | 2020-06-12 | 2020-06-12 | Echo state network time sequence prediction algorithm based on phase space reconstruction |
Publications (1)
Publication Number | Publication Date |
---|---|
CN111832623A true CN111832623A (en) | 2020-10-27 |
Family
ID=72899440
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
CN202010533889.5A Pending CN111832623A (en) | 2020-06-12 | 2020-06-12 | Echo state network time sequence prediction algorithm based on phase space reconstruction |
Country Status (1)
Country | Link |
---|---|
CN (1) | CN111832623A (en) |
Cited By (2)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN114118373A (en) * | 2021-11-25 | 2022-03-01 | 福州大学 | Multi-dimensional time sequence missing completion method based on echo state network |
CN114613372A (en) * | 2022-02-21 | 2022-06-10 | 北京富通亚讯网络信息技术有限公司 | Error concealment technical method for preventing packet loss in audio transmission |
-
2020
- 2020-06-12 CN CN202010533889.5A patent/CN111832623A/en active Pending
Cited By (3)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN114118373A (en) * | 2021-11-25 | 2022-03-01 | 福州大学 | Multi-dimensional time sequence missing completion method based on echo state network |
CN114613372A (en) * | 2022-02-21 | 2022-06-10 | 北京富通亚讯网络信息技术有限公司 | Error concealment technical method for preventing packet loss in audio transmission |
CN114613372B (en) * | 2022-02-21 | 2022-10-18 | 北京富通亚讯网络信息技术有限公司 | Error concealment technical method for preventing packet loss in audio transmission |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
CN112418482B (en) | Cloud computing energy consumption prediction method based on time series clustering | |
Wang et al. | Chaotic time series method combined with particle swarm optimization and trend adjustment for electricity demand forecasting | |
CN111553543B (en) | TPA-Seq2 Seq-based power load prediction method and related components | |
CN110413878B (en) | User-commodity preference prediction device and method based on adaptive elastic network | |
CN111260124A (en) | Chaos time sequence prediction method based on attention mechanism deep learning | |
CN111832623A (en) | Echo state network time sequence prediction algorithm based on phase space reconstruction | |
CN114169645A (en) | Short-term load prediction method for smart power grid | |
CN110874374A (en) | On-line time sequence prediction method and system based on granularity intuition fuzzy cognitive map | |
CN111160650A (en) | Adaboost algorithm-based traffic flow characteristic analysis and prediction method | |
Pinson | Introducing distributed learning approaches in wind power forecasting | |
CN111105005A (en) | Wind power prediction method | |
Yang et al. | Host load prediction based on PSR and EA-GMDH for cloud computing system | |
Gomez-Rosero et al. | Deep neural network for load forecasting centred on architecture evolution | |
CN116523001A (en) | Method, device and computer equipment for constructing weak line identification model of power grid | |
CN115809747A (en) | Pyramid cause-and-effect network-based coupling information flow long-term prediction method | |
Rasheduzzaman et al. | Study of different forecasting models on Google cluster trace | |
CN113010774B (en) | Click rate prediction method based on dynamic deep attention model | |
Rafik et al. | Learning and Predictive Energy Consumption Model based on LSTM recursive neural networks | |
Jiang et al. | Monthly electricity consumption forecasting by the fruit fly optimization algorithm enhanced Holt-Winters smoothing method | |
Borkar et al. | Reinforcement learning for matrix computations: PageRank as an example | |
CN112667394A (en) | Computer resource utilization rate optimization method | |
Zhang et al. | A novel hybrid forecasting system based on data augmentation and deep learning neural network for short-term wind speed forecasting | |
Gong et al. | Integrated multi-horizon power and energy forecast for aggregated electric water heaters | |
WO2023027128A1 (en) | Information processing device and recording medium | |
CN113011628B (en) | Power load prediction method based on topological structure and time convolution network |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
PB01 | Publication | ||
PB01 | Publication | ||
SE01 | Entry into force of request for substantive examination | ||
SE01 | Entry into force of request for substantive examination |