CN112949896A - Time sequence prediction method based on fusion sequence decomposition and space-time convolution - Google Patents
Time sequence prediction method based on fusion sequence decomposition and space-time convolution Download PDFInfo
- Publication number
- CN112949896A CN112949896A CN202011525943.8A CN202011525943A CN112949896A CN 112949896 A CN112949896 A CN 112949896A CN 202011525943 A CN202011525943 A CN 202011525943A CN 112949896 A CN112949896 A CN 112949896A
- Authority
- CN
- China
- Prior art keywords
- sequence
- time
- dimensional
- decomposition
- block
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
- G06Q10/00—Administration; Management
- G06Q10/04—Forecasting or optimisation specially adapted for administrative or management purposes, e.g. linear programming or "cutting stock problem"
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/048—Activation functions
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/049—Temporal neural networks, e.g. delay elements, oscillating neurons or pulsed inputs
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
- G06N3/084—Backpropagation, e.g. using gradient descent
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Computational Linguistics (AREA)
- Health & Medical Sciences (AREA)
- Biophysics (AREA)
- Artificial Intelligence (AREA)
- Data Mining & Analysis (AREA)
- Evolutionary Computation (AREA)
- General Health & Medical Sciences (AREA)
- Molecular Biology (AREA)
- Computing Systems (AREA)
- General Engineering & Computer Science (AREA)
- Life Sciences & Earth Sciences (AREA)
- Mathematical Physics (AREA)
- Software Systems (AREA)
- Biomedical Technology (AREA)
- Business, Economics & Management (AREA)
- Human Resources & Organizations (AREA)
- Economics (AREA)
- Strategic Management (AREA)
- Entrepreneurship & Innovation (AREA)
- Game Theory and Decision Science (AREA)
- Development Economics (AREA)
- Marketing (AREA)
- Operations Research (AREA)
- Quality & Reliability (AREA)
- Tourism & Hospitality (AREA)
- General Business, Economics & Management (AREA)
- Management, Administration, Business Operations System, And Electronic Commerce (AREA)
Abstract
The invention provides a time sequence prediction method based on fusion sequence decomposition and space-time convolution, which carries out prediction through a fusion sequence decomposition strategy and a neural network model (SDBRNN, Series-Decompose-Block RNN) of three-dimensional time sequence convolution. Aiming at the problem of uncertain period in the original STL decomposition, an improved algorithm based on trend seasonal decomposition is provided; when the time sequence block is constructed, the components are combined according to the correlation, so that the three-dimensional convolution module can better extract the space-time characteristics; meanwhile, the three-dimensional volume block is a module embedded in the LSTM cell, and replaces the gate updating operation of the LSTM, so that the LSTM can learn with the time-space characteristics of data. Experiments prove the rationality of the time sequence block structure and the effectiveness of the model.
Description
Technical Field
The invention belongs to the field of data analysis, and particularly relates to a time sequence prediction algorithm based on fusion sequence decomposition and space-time convolution.
Technical Field
The time sequence data is used for describing the characteristics of the change of the object along with the time, and the research on the time sequence can help people to know the historical development mode of the object and predict the future change trend of the object, such as stock price fluctuation, road and vehicle flow prediction, user behavior analysis and the like. The essential characteristics of the sequence can be drawn according to the mean, variance, covariance and the like of the sequence, and the sequence is divided into a stationary time sequence and a non-stationary time sequence. On the traditional time series prediction problem, a plurality of machine learning methods such as a support vector machine and a decision tree algorithm appear. Meanwhile, with the rapid development of deep learning technology, more and more researchers apply the neural network to the prediction problem of the time series. For example, a Recurrent Neural Network (RNN) is used to introduce the time characteristics of the sequence into the Neural Network, and predictions are made on the detrended time series data. Many neural network variants based on RNN, such as Long-Short Term Memory network (LSTM), not only solve the problem of gradient disappearance of RNN, but also utilize a forgetting gate mechanism to help the sequence have a Memory function for Long-Term dependence. In addition, the Convolutional Network is also applied to the time sequence prediction problem, although the Convolutional Network cannot capture the time sequence characteristics as well as the RNN Network due to the limitation of the size of a Convolutional kernel, the effect of the Convolutional Network on the time sequence prediction is sometimes better than that of the RNN through reasonable structural design, for example, a time domain Convolutional Network (TCN) is not only trained faster than an LSTM, but also has stronger capability of capturing local time domain characteristics. The prior time convolution sequence prediction method usually predicts different observation points separately, but the method is effective, but ignores the relation among the observation points, because the spatial dependence exists for the time sequence of multiple observation points with stronger correlation.
Disclosure of Invention
The invention aims to provide a novel time sequence prediction method of a neural network model (SDBRNN, Series-Decompose-Block RNN) fusing a sequence decomposition strategy and three-dimensional time sequence convolution.
Therefore, the invention adopts the following technical scheme:
a time sequence prediction method based on fusion sequence decomposition and space-time convolution is characterized in that one-dimensional time sequence data is subjected to time sequence decomposition and spliced into a three-dimensional data block with time and space based on an STL decomposition strategy, high-dimensional space-time characteristics of the three-dimensional data block are extracted by using a three-dimensional convolution model, and finally back propagation learning is carried out on the high-dimensional space-time characteristics by using an LSTM sequence module.
The method carries out time sequence prediction through a sequence Decomposition Block Recurrent Neural Network model SDBRNN (Series-Decomposition-Block Recurrent Neural Network); the SDBRNN model comprises two modules, namely a time sequence Block construction module and a space-time fusion prediction module based on a decomposition strategy, wherein the space-time fusion prediction module comprises a three-dimensional convolution module and a Block-LSTM module.
Further, the sequential block construction is divided into an STL decomposition stage and a component reassembly stage.
Further, in the STL decomposition stage, the period n (p) of the original one-dimensional input sequence is determined first.
Further, when the period n (p) is determined, firstly inputting an alternative period n (P)i(i ═ 1,2, …), performing STL decomposition on the original sequence at different periods, respectively, and obtaining different decomposition results as shown in formula (1):
to pairRespectively with the original sequence OvCalculating DTW distance diGet diMinimum period parameter n (P)k(k epsilon {1, 2. }) is taken as an STL decomposition parameter, and a trend component T of the original sequence is obtainedvSeasonal component SvAnd residual errorComponent RvExpressed as:
Ov=Tv+Sv+Rv,v=1,2,...,N (2)。
further, the STL decomposition specific operation includes the following six steps:
step 1, performing trend removing processing on an original sequence, namely subtracting the trend of the sequence at the last moment, as shown in formula (3):
step 2, periodic subsequence smoothing (Cycle-subseries smoothing); to DtMaking periodic subsequence smooth regression operation, making 1 period N (p) respectively extended before and after the sequence, and recording the sequence whose length is N +2N (p)Wherein v ═ 1-N (p), …, N + N (p);
step 3, periodic subsequence Low-throughput filteringRespectively carrying out the running average operation of the parameters N (p), and 3 for three times, and carrying out LOESS regression again to obtain the sequence with the length of NI.e. a periodic subsequenceLow flux trend of (d);
step 4, smoothing the periodic subsequence trend (regression of smoothed cycles-subseries), and seasonally formingIs composed ofAnda difference of (d); as shown in formula (4):
step 5, removing seasonality (desonalizing) and Trend smoothing (Trend smoothing), and carrying out comparison on the original sequence OtBy subtracting seasonal componentsThen, LOESS regression with smoothing coefficient n (t) is performed to obtain trend componentAs shown in formula (5):
step 6, for the original sequence ytObtained by subtracting the above stepsAndobtaining residual components; as shown in formula (6):
further, STL decomposition is carried out to obtain four equal-length sequences of three components and the original sequence, and a two-dimensional matrix M is constructedN×h=[Ot,Tv,Sv Rv]Where N is a single sequence length and h equals 4 represents 4 components;
and splicing the two-dimensional matrixes of different observation points to obtain the three-dimensional time sequence block with time and space characteristics.
Furthermore, in the space-time fusion prediction module, the three-dimensional convolution module extracts high-dimensional characteristics by performing three-dimensional convolution operation on the time sequence block, and replaces the gate updating operation of the LSTM; the Block-LSTM module takes the output of the three-dimensional convolution module as the input of the LSTM module, and at the moment, the cell data of the Block-LSTM has both temporal and spatial characteristics.
Further, the core portion of the LSTM module is the cellular state C transmitted in each LSTM celltAnd cell state CtThe three gating mechanisms are used to determine the forgetting gate ftInput gate ItAnd an output gate Ot(ii) a Wherein, forget the door ftActivation of the weight matrix W by a sigmoid activation functionfAnd the previous time hidden layer information ht-1And the current input xtDetermining whether the cell retains or discards the previous state information; input gate ItFirstly, activating a weight matrix W through a sigmoid functioniAnd ht-1And xtDetermines updated information by the product of the two, and activates the weight matrix W by the tanh activation functionCAnd ht-1And xtThe product of (a) yields candidate cell informationNew cell state CtFrom ftAnd Ct-1Sum of products ItAndthe sum of the products of (a); output gate OtActivation of the weight matrix W by sigmoidoThe product of ht-1 and xt, the current hidden layer state htFrom OtWith tanh activated CtObtaining the product of the two;
LSTM module passing hidden state ht、CtCalculating an error term for each step before the gradient delta t, and deducing to obtain a forgetting gate f according to a back propagation principletInput gate ItOutput gate OtParameter W off、WIAnd WoComputing propagation processesAs shown in formula (7):
further, the three-dimensional convolution kernel slides on the time sequence block and conducts convolution operation with pixel points on the time sequence block to obtain a high-order three-dimensional feature block, the feature block is evenly divided to obtain four small feature blocks which serve as forgetting gates f of the LSTM cellstInput gate ItOutput gate OtAnd candidate cell Ct(ii) a Each gating mechanism in each Block-LSTM cell carries the timing characteristics of the data Block, where input gate ItWith the hidden layer information h at the previous momentt-1Splicing and activating by using sigmoid; forget door ftSplicing with ht-1 and activating a sigmoid activation function; candidate cellSplicing with ht-1 and activating by using a tanh activation function; new cell state CtIs composed ofAnd ItProduct of (a) and (f)tAnd previous cell state Ct-1Summing; the cell operation of SDBRNN is shown in formula (8), where x represents the convolution operation.
The method carries out time sequence prediction through a sequence decomposition block recurrent neural network model SDBRNN; the method is suitable for the data sets with different observation points related in reality logic and strong correlation, and richer features can be extracted through time-space synchronous convolution, so that the prediction effect of the sequence is improved.
Drawings
FIG. 1 is a schematic diagram of a network structure of a recurrent neural network model of the present invention.
FIG. 2-1 is an exploded view of the STL structure of the timing block of the present invention, and FIG. 2-2 is a schematic view of the timing block structure of the present invention.
FIG. 3 is a schematic diagram of the structure of the SDBRNN cell of the present invention.
Detailed Description
The invention provides a time sequence prediction method based on fusion sequence decomposition and space-time convolution, which comprises the steps of firstly, carrying out time sequence decomposition on one-dimensional time sequence data based on an STL decomposition strategy, splicing the time sequence data into a three-dimensional data block with time and space, extracting high-dimensional space-time characteristics of the three-dimensional data block by using a three-dimensional convolution model, and finally carrying out back propagation learning on the high-dimensional space-time characteristics by using an LSTM sequence module.
Aiming at the problem of uncertain period in the original STL decomposition, the invention provides an improved algorithm based on trend seasonal decomposition; when the time sequence block is constructed, the components are combined according to the correlation, so that the three-dimensional convolution module can better extract the space-time characteristics; meanwhile, the three-dimensional convolution module is embedded in the LSTM cell and replaces the gate updating operation of the LSTM, so that the LSTM can learn the space-time characteristics of data. Experiments prove the rationality of the time sequence block structure and the effectiveness of the model.
As shown in FIG. 1, the invention provides a sequence Decomposition Block Recurrent Neural Network model (SDBRNN) by utilizing the correlation of space-time dimensions and the advantage of feature enhancement brought by multi-dimensional fusion. The SDBRNN model can be mainly divided into two modules, namely a time sequence Block construction module and a space-time fusion prediction module based on a decomposition strategy, wherein the space-time fusion prediction module comprises a three-dimensional convolution module and a Block-LSTM module.
The time sequence block structure is used for decomposing and combining the characteristics in the time sequence and splicing different sequences according to the sequence of observation points to form a three-dimensional data time sequence block structure module with time sequence characteristics, and comprises the following steps:
as shown in fig. 2-1 and 2-2, the sequential block construction can be divided into an STL decomposition stage and a component reassembly stage. In the STL decomposition stage, the period n (p) of the original one-dimensional input sequence needs to be determined first. Sequence period is often takenDepending on the data itself, if the sampling interval of the traffic speed data set is 5 minutes, there are 288 sample points in a day, and the subjective cycle is one day, n (p) is 288, and the longer cycle may be one week or even one month. How to select the period not only affects the sequence decomposition effect, but also affects the final prediction accuracy. In order to optimize the value of the period n (p), firstly, inputting an alternative period n (P)i(i ═ 1,2, …), the original sequence was subjected to STL decomposition in different periods, and the obtained different decomposition results were as shown in formula (1).
To pairRespectively with the original sequence OvCalculating DTW distance diIn the method, d is takeniMinimum period parameter n (P)k(k epsilon {1, 2. }) is taken as an STL decomposition parameter, and a trend component T of the original sequence is obtainedvSeasonal component SvAnd a residual component RvIt can be expressed as:
Ov=Tv+Sv+Rv,v=1,2,...,N (2)
the specific operation of STL decomposition comprises the following six steps:
step 1, Detrending, namely, Detrending the original sequence, namely subtracting the trend of the sequence at the last moment, as shown in formula (3):
step 2, Cycle-subsequences smoothingtMaking periodic subsequence smooth regression operation, making 1 period N (p) respectively extended before and after the sequence, and recording the sequence whose length is N +2N (p)Wherein v ═ 1-N (p), …, N + N (p);
step 3, periodic subsequence Low-throughput filteringRespectively carrying out the running average operation of the parameters N (p), and 3 for three times, and carrying out LOESS regression again to obtain the sequence with the length of NI.e. a periodic subsequenceLow flux trend of (d);
and 4, smoothing the periodic subsequence trend (trimming of smoothed cycles-subseries)Is composed ofAnda difference of (d); as shown in formula (4):
step 5, removing seasonality (desonalizing) and Trend smoothing (Trend smoothing), and carrying out O treatment on the original sequencetBy subtracting seasonal componentsThen, LOESS regression with smoothing coefficient n (t) is performed to obtain trend componentAs shown in formula (5):
step 6, for the original sequence ytObtained by subtracting the above stepsAndobtaining residual components; as shown in formula (6):
different from the existing method for independently predicting and learning components, the model constructs a two-dimensional matrix M by decomposing three components obtained by STL and four sequences with equal length of an original sequence in a component recombination stageN×h=[Ot,Tv,Sv Rv]Where N is a single sequence length and h equals 4 represents 4 components.
It is clear that the order of combining the 4 components will affect the two-dimensional matrix values formed, depending on the nature of the convolution operation, for the given M aboveN×hRow vector T in the middle of the matrixvAnd SvWill be compared to the row vector O at the edgetAnd RvAnd (4) participating in convolution calculation. The predicted effect of the different ranking methods will be demonstrated experimentally in the experimental part herein. Thus, a two-dimensional matrix of single observation points subjected to STL decomposition and component recombination is obtained. And finally, splicing the two-dimensional matrixes of different observation points to obtain the three-dimensional time sequence block with time and space characteristics.
The spatio-temporal fusion prediction module is described below, including a three-dimensional convolution module and an LSTM module. 1) A three-dimensional convolution module: the high-dimensional characteristics are extracted by performing three-dimensional convolution operation on the time sequence block, and the gate updating operation of the LSTM is replaced; 2) LSTM module: the output of the three-dimensional convolution module is used as the input of the LSTM module, and at the moment, the cell data of the LSTM has both time and space characteristics.
The core portion of the LSTM is the cellular state C transported in each LSTM celltAnd cell state CtThe three gating mechanisms are used to determine the forgetting gate ftInput gate ItAnd an output gate Ot. Wherein, forget the door ftActivation of the weight matrix W by a sigmoid activation functionfAnd the previous time hidden layer information ht-1And the current input xtDetermining whether the cell retains or discards the previous state information; input gate ItFirstly, activating a weight matrix W through a sigmoid functioniAnd ht-1And xtDetermines updated information by the product of the two, and activates the weight matrix W by the tanh activation functionCAnd ht-1And xtThe product of (a) yields candidate cell informationNew cell state CtFrom ftAnd Ct-1Sum of products ItAndthe sum of the products of (a); output gate OtActivation of the weight matrix W by sigmoidoThe product of ht-1 and xt, the current hidden layer state htFrom OtWith tanh activated CtThe product of the two is obtained.
LSTM passes through hidden state ht、CtAnd the gradient δ t calculates the error term towards each previous step, the key being that the calculation parameters are based on the partial derivatives of the loss function. According to the back propagation principle, deducing a forgetting gate ftInput gate ItOutput gate OtParameter W off、WIAnd WoThe calculation propagation process is shown in equation (7).
However, the vast spatial relationships that exist for multi-observation time series cannot be exploited by LSTM. The space-time fusion convolutional network provided by the invention combines the space extraction capability of the convolutional layer and the time sequence extraction capability of the LSTM, and is called Block-LSTM. Different from a simple convolutional layer + LSTM network, the three-dimensional convolution operation of Block-LSTM is realized in the bottom layer inside the LSTM cell and replaces the gate updating and cell updating operation of LSTM.
As shown in the three-dimensional convolution module of fig. 3, when the three-dimensional convolution kernel slides on the time sequence block and performs convolution operation with the pixel points on the time sequence block, a high-order three-dimensional feature block is obtained, and the feature block is uniformly divided to obtain four small feature blocks serving as the forgetting gate f of the LSTM celltInput gate ItOutput gate OtAnd candidate cell Ct. Thus each gating mechanism in each Block-LSTM cell carries the timing characteristics of the data Block. Wherein, the input gate ItWith the hidden layer information h at the previous momentt-1Splicing and activating by using sigmoid; forget door ftSplicing with ht-1 and activating a sigmoid activation function; candidate cellSplicing with ht-1 and activating by using a tanh activation function; new cell state CtIs composed ofAnd ItProduct of (a) and (f)tAnd previous cell state Ct-1And (4) summing. The cell operation of SDBRNN is shown in formula (8), where x represents the convolution operation.
By comparison, it can be found that the gate mechanism of Block-LSTM cells works in a manner similar to LSTM, with the most important change being the replacement of state updates between cells by three-dimensional convolution operations. The Block-LSTM can not only extract spatial features by utilizing the correlation among multiple sequences like a CNN network, but also establish the time sequence relation among the sequences by utilizing the LSTM, thereby achieving the space-time prediction.
The SDBRNN model carries out prediction verification on 3 data sets, including two traffic flow data sets PeMS-Bay and Seattle and a power data set Solar-Energy. The three data sets are all multi-observation point time series data sets. The comparison method and parameter settings were as follows:
1) LSTM is a special RNN network, which has a gate mechanism long-short term memory neural network and has better effect on solving the problem of long-term dependence. The effect can be contrasted with the STL decomposition module and the three-dimensional convolution module of the models herein. In the comparative experiment, the number of LSTM layers was 2 and the hidden layer size was [16,32 ].
2) Gru, a special RNN network, has a simpler network mechanism than LSTM, has better effect in solving long-term problems and the model is easier to train. In the comparative experiment, the number of GRU layers was 2 and the hidden layer size was [16,32 ].
3) Wavenet, a sequence generation model, has stronger time domain view than the common CNN structure, and has good effects on speech generation and text prediction. In a comparative experiment, the number of WaveNet layers is 4, the convolution kernel size is 2, the Residual channel coefficient Residual channel is 32, and the Skip channel coefficient Skip channel is 128.
4) Tcn time convolutional network. The method has the advantages that the method has a more flexible receptive field by utilizing the cavity convolution, and has a residual network structure, so that the gradient of the method in the training process is more stable. In the comparative experiment, the number of TCN layers is 2, the convolution kernel size is 2, and the hole coefficient size is 2.
5) ConvLSTM is a variant LSTM network for extracting spatial features by using two-dimensional convolution, has a good extraction effect on the spatial features, and has a good comparison effect with a three-dimensional convolution module of a text model. In the comparative experiment, the number of layers of ConvLSTM was 2, the convolution kernel size was 2 × 2, and the hidden layer size was [16,32 ].
The comparison parameters are 3 conventional evaluation indexes, namely RMSE, MAE and MAPE.
TABLE 1 results of the experimental predictions
As can be seen from the table, in the PeMS-Bay and Seattle traffic data sets, the TCN model with the best short-term prediction performance has MAPE of 2.94%, while the SDBRNN model has MAPE of 5.50%, the SDBRNN model with the best medium-long term prediction performance has SDBRNN proposed herein, predicts MAPE values of 3.47% and 2.54% for 6 and 12 time slices, which are about 1 and 2 percentage points better than TCN, respectively, which indicates that the extraction of different components of the sequence by the STL decomposition module can improve the long-term prediction effect, but is not obviously helpful in short-term prediction. It is not difficult to understand, because of the tendency component T of STL decompositionvAnd seasonal ingredient SvThe representative data is the characteristics of the data overall level, which means that the longer the data is, the more cycles the sequence contains, the better the decomposition effect is, and for the local characteristics, the STL has no good strategy for extraction. Therefore, the advantage of SDBRNN is more apparent when the predicted step size is longer in comparison of different models. Particularly, the results of comparing ConvLSTM and SDBRNN show that the effect of the SDBRNN model on long-term prediction is about 2 percent better than that of ConvLSTM, because the convolution module of ConvLSTM only performs two-dimensional convolution operation, namely spatial feature extraction, on a single moment, and the three-dimensional convolution module of SDBRNN simultaneously extracts space-time features, so that the effectiveness of the SDBRNN model on long-term time sequence prediction is verified.
Claims (10)
1. A time sequence prediction method based on fusion sequence decomposition and space-time convolution is characterized in that one-dimensional time sequence data is subjected to time sequence decomposition and spliced into a three-dimensional data block with time and space based on an STL decomposition strategy, high-dimensional space-time characteristics of the three-dimensional data block are extracted by using a three-dimensional convolution model, and finally back propagation learning is carried out on the high-dimensional space-time characteristics by using an LSTM sequence module.
2. The method of claim 1, wherein the method performs time sequence prediction by using a sequence-Decomposition Block Recurrent Neural Network (SDBRNN) model; the SDBRNN model comprises two modules, namely a time sequence Block construction module and a space-time fusion prediction module based on a decomposition strategy, wherein the space-time fusion prediction module comprises a three-dimensional convolution module and a Block-LSTM module.
3. The method of claim 2, wherein the time sequence block structure is divided into an STL decomposition stage and a component reconstruction stage.
4. A method of temporal prediction based on fused sequence decomposition with spatio-temporal convolution according to claim 3 characterized in that in the STL decomposition stage, the period n (p) of the original one-dimensional input sequence is determined first.
5. The method as claimed in claim 4, wherein when determining the period n (p), the alternative period n (p) is inputted firsti(i ═ 1,2, …), performing STL decomposition on the original sequence at different periods, respectively, and obtaining different decomposition results as shown in formula (1):
to pairRespectively with the original sequence OvCalculating DTW distance diGet diMinimum period parameter n (P)k(k epsilon {1, 2. }) is taken as an STL decomposition parameter, and a trend component T of the original sequence is obtainedvSeasonal component SvAnd a residual component RvExpressed as:
Ov=Tv+Sv+Rv,v=1,2,...,N (2)。
6. the method of claim 3, wherein the STL decomposition operation comprises the following six steps:
step 1, performing trend removing processing on an original sequence, namely subtracting the trend of the sequence at the last moment, as shown in formula (3):
step 2, periodic subsequence smoothing (Cycle-subseries smoothing); to DtMaking periodic subsequence smooth regression operation, making 1 period N (p) respectively extended before and after the sequence, and recording the sequence whose length is N +2N (p)Wherein v ═ 1-N (p), …, N + N (p);
step 3, periodic subsequence Low-throughput filteringRespectively carrying out the running average operation of the parameters N (p), and 3 for three times, and carrying out LOESS regression again to obtain the sequence with the length of NI.e. a periodic subsequenceLow flux trend of (d);
step 4, smoothing the periodic subsequence trend (regression of smoothed cycles-subseries), and seasonally formingIs composed ofAnda difference of (d); as shown in formula (4):
step 5, removing seasonality (desonalizing) and Trend smoothing (Trend smoothing), and carrying out comparison on the original sequence OtBy subtracting seasonal componentsThen, LOESS regression with smoothing coefficient n (t) is performed to obtain trend componentAs shown in formula (5):
step 6, for the original sequence ytObtained by subtracting the above stepsAndobtaining residual components; as shown in formula (6):
7. the time sequence prediction method based on fusion sequence decomposition and space-time convolution of claim 3, characterized in that STL decomposition obtains four sequences with the same length of three components and the original sequence itself, and constructs a two-dimensional matrix MN×h=[Ot,Tv,Sv Rv]Where N is a single sequence length and h equals 4 represents 4 components;
and splicing the two-dimensional matrixes of different observation points to obtain the three-dimensional time sequence block with time and space characteristics.
8. The time-series prediction method based on fusion sequence decomposition and space-time convolution of claim 3 is characterized in that in the space-time fusion prediction module, the three-dimensional convolution module extracts high-dimensional features by performing three-dimensional convolution operation on the time-series block instead of the gate updating operation of the LSTM; the Block-LSTM module takes the output of the three-dimensional convolution module as the input of the LSTM module, and at the moment, the cell data of the Block-LSTM has both temporal and spatial characteristics.
9. The method of claim 8 in which the core part of the LSTM module is the cell state C transmitted in each LSTM celltAnd cell state CtThe three gating mechanisms are used to determine the forgetting gate ftInput gate ItAnd an output gate Ot(ii) a Wherein, forget the door ftActivation of the weight matrix W by a sigmoid activation functionfAnd the previous time hidden layer information ht-1And the current input xtDetermining whether the cell retains or discards the previous state information; input gate ItFirstly, activating a weight matrix W through a sigmoid functioniAnd ht-1And xtDetermines updated information by the product of the two, and activates the weight matrix W by the tanh activation functionCAnd ht-1And xtThe product of (a) yields candidate cell informationNew cell state CtFrom ftAnd Ct-1Sum of products ItAndthe sum of the products of (a); output gate OtActivation of the weight matrix W by sigmoidoThe product of ht-1 and xt, the current hidden layer state htFrom OtWith tanh activated CtObtaining the product of the two;
LSTM module passing hidden state ht、CtCalculating an error term for each step before the gradient delta t, and deducing to obtain a forgetting gate f according to a back propagation principletInput gate ItOutput gate OtParameter W off、WIAnd WoThe calculation propagation process is shown in equation (7):
10. the method of claim 9, wherein a three-dimensional convolution kernel slides on the time sequence block and performs convolution operation with pixel points on the time sequence block to obtain a high-order three-dimensional feature block, and the feature block is uniformly divided to obtain four small feature blocks as the forgetting gate f of the LSTM celltInput gate ItOutput gate OtAnd candidate cell Ct(ii) a Each gating mechanism in each Block-LSTM cell carries the timing characteristics of the data Block, where input gate ItWith the hidden layer information h at the previous momentt-1Splicing and activating by using sigmoid; forget door ftSplicing with ht-1 and activating a sigmoid activation function; candidate cellSplicing with ht-1 and activating by using a tanh activation function; new cell state CtIs composed ofAnd ItProduct of (a) and (f)tAnd previous cell state Ct-1Summing; the cell working mode of the SDBRNN is shown in a formula (8), wherein the operation mode represents convolution operation;
Priority Applications (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CN202011525943.8A CN112949896B (en) | 2020-12-22 | 2020-12-22 | Time sequence prediction method based on fusion sequence decomposition and space-time convolution |
Applications Claiming Priority (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CN202011525943.8A CN112949896B (en) | 2020-12-22 | 2020-12-22 | Time sequence prediction method based on fusion sequence decomposition and space-time convolution |
Publications (2)
Publication Number | Publication Date |
---|---|
CN112949896A true CN112949896A (en) | 2021-06-11 |
CN112949896B CN112949896B (en) | 2022-06-17 |
Family
ID=76234825
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
CN202011525943.8A Active CN112949896B (en) | 2020-12-22 | 2020-12-22 | Time sequence prediction method based on fusion sequence decomposition and space-time convolution |
Country Status (1)
Country | Link |
---|---|
CN (1) | CN112949896B (en) |
Cited By (4)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN113569479A (en) * | 2021-07-27 | 2021-10-29 | 天津大学 | Long-term multi-step control method, device and storage medium for rock fracture development of stone cave temple |
CN114239948A (en) * | 2021-12-10 | 2022-03-25 | 浙江省交通投资集团有限公司智慧交通研究分公司 | Deep traffic flow prediction method, medium and equipment based on time sequence decomposition unit |
CN115545315A (en) * | 2022-10-12 | 2022-12-30 | 重庆移通学院 | PM2.5 prediction method based on three-dimensional convolutional neural network and gated cyclic unit |
CN117668477A (en) * | 2024-01-31 | 2024-03-08 | 山东科技大学 | Intelligent lightweight processing method and system for ocean big data |
Citations (4)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN109862585A (en) * | 2019-01-31 | 2019-06-07 | 湖北工业大学 | A kind of dynamic heterogeneous network flow prediction method based on depth Space-time Neural Network |
CN109977098A (en) * | 2019-03-08 | 2019-07-05 | 北京工商大学 | Non-stationary time-series data predication method, system, storage medium and computer equipment |
CN111260030A (en) * | 2020-01-13 | 2020-06-09 | 润联软件系统(深圳)有限公司 | A-TCN-based power load prediction method and device, computer equipment and storage medium |
CN111325976A (en) * | 2020-02-24 | 2020-06-23 | 交通运输部水运科学研究所 | Short-term traffic flow prediction method and system |
-
2020
- 2020-12-22 CN CN202011525943.8A patent/CN112949896B/en active Active
Patent Citations (4)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN109862585A (en) * | 2019-01-31 | 2019-06-07 | 湖北工业大学 | A kind of dynamic heterogeneous network flow prediction method based on depth Space-time Neural Network |
CN109977098A (en) * | 2019-03-08 | 2019-07-05 | 北京工商大学 | Non-stationary time-series data predication method, system, storage medium and computer equipment |
CN111260030A (en) * | 2020-01-13 | 2020-06-09 | 润联软件系统(深圳)有限公司 | A-TCN-based power load prediction method and device, computer equipment and storage medium |
CN111325976A (en) * | 2020-02-24 | 2020-06-23 | 交通运输部水运科学研究所 | Short-term traffic flow prediction method and system |
Non-Patent Citations (1)
Title |
---|
LAN QIN ET AL.: "Effective passenger flow forecasting using STL and ESN based on two improvement strategies", 《NEUROCOMPUTING》 * |
Cited By (7)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN113569479A (en) * | 2021-07-27 | 2021-10-29 | 天津大学 | Long-term multi-step control method, device and storage medium for rock fracture development of stone cave temple |
CN113569479B (en) * | 2021-07-27 | 2023-11-10 | 天津大学 | Long-term multi-step control method, device and storage medium for rock mass crack development of stone cave temple |
CN114239948A (en) * | 2021-12-10 | 2022-03-25 | 浙江省交通投资集团有限公司智慧交通研究分公司 | Deep traffic flow prediction method, medium and equipment based on time sequence decomposition unit |
CN114239948B (en) * | 2021-12-10 | 2023-07-21 | 浙江省交通投资集团有限公司智慧交通研究分公司 | Deep traffic flow prediction method, medium and equipment based on time sequence decomposition unit |
CN115545315A (en) * | 2022-10-12 | 2022-12-30 | 重庆移通学院 | PM2.5 prediction method based on three-dimensional convolutional neural network and gated cyclic unit |
CN117668477A (en) * | 2024-01-31 | 2024-03-08 | 山东科技大学 | Intelligent lightweight processing method and system for ocean big data |
CN117668477B (en) * | 2024-01-31 | 2024-04-26 | 山东科技大学 | Intelligent lightweight processing method and system for ocean big data |
Also Published As
Publication number | Publication date |
---|---|
CN112949896B (en) | 2022-06-17 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
CN112949896B (en) | Time sequence prediction method based on fusion sequence decomposition and space-time convolution | |
CN110223517B (en) | Short-term traffic flow prediction method based on space-time correlation | |
CN108510741B (en) | Conv1D-LSTM neural network structure-based traffic flow prediction method | |
CN108448610B (en) | Short-term wind power prediction method based on deep learning | |
CN108024158A (en) | There is supervision video abstraction extraction method using visual attention mechanism | |
CN107066445B (en) | The deep learning method of one attribute emotion word vector | |
CN110929092B (en) | Multi-event video description method based on dynamic attention mechanism | |
CN113487088A (en) | Traffic prediction method and device based on dynamic space-time diagram convolution attention model | |
CN109389244A (en) | Tourist's number prediction technique in a kind of short-term scenic spot of multifactor perception based on GRU | |
CN108876044B (en) | Online content popularity prediction method based on knowledge-enhanced neural network | |
CN112733444A (en) | Multistep long time sequence prediction method based on CycleGAN neural network | |
CN110458181A (en) | A kind of syntax dependency model, training method and analysis method based on width random forest | |
CN110570035B (en) | People flow prediction system for simultaneously modeling space-time dependency and daily flow dependency | |
CN109829495A (en) | Timing image prediction method based on LSTM and DCGAN | |
Dong et al. | An integrated deep neural network approach for large-scale water quality time series prediction | |
CN105447510B (en) | Fluctuating wind speed prediction technique based on artificial bee colony optimization LSSVM | |
CN113112791A (en) | Traffic flow prediction method based on sliding window long-and-short term memory network | |
CN109918493A (en) | A kind of dialogue generation method based on shot and long term Memory Neural Networks | |
CN110188200A (en) | A kind of depth microblog emotional analysis method using social context feature | |
CN113935489A (en) | Variational quantum model TFQ-VQA based on quantum neural network and two-stage optimization method thereof | |
CN115952424A (en) | Graph convolution neural network clustering method based on multi-view structure | |
CN115481784A (en) | Traffic flow prediction method based on improved combination model and application | |
Liu | Language database construction method based on big data and deep learning | |
CN114596726A (en) | Parking position prediction method based on interpretable space-time attention mechanism | |
CN113919593A (en) | ME-MBiGRU short-time traffic flow prediction method based on attention mechanism |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
PB01 | Publication | ||
PB01 | Publication | ||
SE01 | Entry into force of request for substantive examination | ||
SE01 | Entry into force of request for substantive examination | ||
GR01 | Patent grant | ||
GR01 | Patent grant |