CN112949896B - Time sequence prediction method based on fusion sequence decomposition and space-time convolution - Google Patents

Time sequence prediction method based on fusion sequence decomposition and space-time convolution Download PDF

Info

Publication number
CN112949896B
CN112949896B CN202011525943.8A CN202011525943A CN112949896B CN 112949896 B CN112949896 B CN 112949896B CN 202011525943 A CN202011525943 A CN 202011525943A CN 112949896 B CN112949896 B CN 112949896B
Authority
CN
China
Prior art keywords
sequence
time
dimensional
decomposition
block
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
CN202011525943.8A
Other languages
Chinese (zh)
Other versions
CN112949896A (en
Inventor
叶惠波
郭长丰
金苍宏
董腾然
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Zhejiang Baian Information Technology Co ltd
Original Assignee
Zhejiang Baian Information Technology Co ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Zhejiang Baian Information Technology Co ltd filed Critical Zhejiang Baian Information Technology Co ltd
Priority to CN202011525943.8A priority Critical patent/CN112949896B/en
Publication of CN112949896A publication Critical patent/CN112949896A/en
Application granted granted Critical
Publication of CN112949896B publication Critical patent/CN112949896B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
    • G06Q10/00—Administration; Management
    • G06Q10/04—Forecasting or optimisation specially adapted for administrative or management purposes, e.g. linear programming or "cutting stock problem"
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/045—Combinations of networks
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/048—Activation functions
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/049—Temporal neural networks, e.g. delay elements, oscillating neurons or pulsed inputs
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • G06N3/084—Backpropagation, e.g. using gradient descent

Landscapes

  • Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Computational Linguistics (AREA)
  • Health & Medical Sciences (AREA)
  • Biophysics (AREA)
  • Artificial Intelligence (AREA)
  • Data Mining & Analysis (AREA)
  • Evolutionary Computation (AREA)
  • General Health & Medical Sciences (AREA)
  • Molecular Biology (AREA)
  • Computing Systems (AREA)
  • General Engineering & Computer Science (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Mathematical Physics (AREA)
  • Software Systems (AREA)
  • Biomedical Technology (AREA)
  • Business, Economics & Management (AREA)
  • Human Resources & Organizations (AREA)
  • Economics (AREA)
  • Strategic Management (AREA)
  • Entrepreneurship & Innovation (AREA)
  • Game Theory and Decision Science (AREA)
  • Development Economics (AREA)
  • Marketing (AREA)
  • Operations Research (AREA)
  • Quality & Reliability (AREA)
  • Tourism & Hospitality (AREA)
  • General Business, Economics & Management (AREA)
  • Management, Administration, Business Operations System, And Electronic Commerce (AREA)

Abstract

The invention provides a time sequence prediction method based on fusion sequence decomposition and space-time convolution, which carries out prediction through a fusion sequence decomposition strategy and a neural network model (SDBRNN, Series-Decompose-Block RNN) of three-dimensional time sequence convolution. Aiming at the problem of uncertain period in the original STL decomposition, an improved algorithm based on trend seasonal decomposition is provided; when the time sequence block is constructed, the components are combined according to the correlation, so that the three-dimensional convolution module can better extract the space-time characteristics; meanwhile, the three-dimensional volume block is a module embedded in the LSTM cell, and replaces the gate updating operation of the LSTM, so that the LSTM can learn the space-time characteristics of data. Experiments prove the rationality of the time sequence block structure and the effectiveness of the model.

Description

Time sequence prediction method based on fusion sequence decomposition and space-time convolution
Technical Field
The invention belongs to the field of data analysis, and particularly relates to a time sequence prediction algorithm based on fusion sequence decomposition and space-time convolution.
Technical Field
The time sequence data is used for describing the characteristics of the change of the object along with the time, and the research on the time sequence can help people to know the historical development mode of the object and predict the future change trend of the object, such as stock price fluctuation, road and vehicle flow prediction, user behavior analysis and the like. The essential characteristics of the sequence can be drawn according to the mean, variance, covariance and the like of the sequence, and the sequence is divided into a stationary time sequence and a non-stationary time sequence. On the traditional time series prediction problem, a plurality of machine learning methods such as a support vector machine and a decision tree algorithm appear. Meanwhile, with the rapid development of deep learning technology, more and more researchers apply the neural network to the prediction problem of the time series. For example, a Recurrent Neural Network (RNN) is used to introduce the time characteristics of the sequence into the Neural Network, and predictions are made on the detrended time series data. Many neural network variants based on RNN, such as Long-Short Term Memory network (LSTM), not only solve the problem of gradient disappearance of RNN, but also utilize a forgetting gate mechanism to help the sequence have a Memory function for Long-Term dependence. In addition, the Convolutional Network is also applied to the time sequence prediction problem, although the Convolutional Network cannot capture the time sequence characteristics as well as the RNN Network due to the limitation of the size of a Convolutional kernel, the effect of the Convolutional Network on the time sequence prediction is sometimes better than that of the RNN through reasonable structural design, for example, a time domain Convolutional Network (TCN) is not only trained faster than an LSTM, but also has stronger capability of capturing local time domain characteristics. The prior time convolution sequence prediction method usually predicts different observation points separately, but the method is effective, but ignores the relation among the observation points, because the spatial dependence exists for the time sequence of multiple observation points with stronger correlation.
Disclosure of Invention
The invention aims to provide a novel time sequence prediction method of a neural network model (SDBRNN, Series-Decompose-Block RNN) fusing a sequence decomposition strategy and three-dimensional time sequence convolution.
Therefore, the invention adopts the following technical scheme:
a time sequence prediction method based on fusion sequence decomposition and space-time convolution is characterized in that one-dimensional time sequence data is subjected to time sequence decomposition and spliced into a three-dimensional data block with time and space based on an STL decomposition strategy, high-dimensional space-time characteristics of the three-dimensional data block are extracted by using a three-dimensional convolution model, and finally back propagation learning is carried out on the high-dimensional space-time characteristics by using an LSTM sequence module.
The method carries out time sequence prediction through a sequence Decomposition Block Recurrent Neural Network model SDBRNN (Series-Decomposition-Block Recurrent Neural Network); the SDBRNN model comprises two modules, namely a time sequence Block construction module and a space-time fusion prediction module based on a decomposition strategy, wherein the space-time fusion prediction module comprises a three-dimensional convolution module and a Block-LSTM module.
Further, the sequential block construction is divided into an STL decomposition stage and a component reassembly stage.
Further, in the STL decomposition stage, the period n (p) of the original one-dimensional input sequence is determined first.
Further, when the period n (p) is determined, firstly inputting an alternative period n (P)i(i ═ 1,2, …), performing STL decomposition on the original sequence at different periods, respectively, and obtaining different decomposition results as shown in formula (1):
Figure BDA0002850798430000021
to pair
Figure BDA0002850798430000022
Respectively with the original sequence OvCalculating DTW distance diGet diMinimum period parameter n (P)k(k epsilon {1, 2. }) is taken as an STL decomposition parameter, and a trend component T of the original sequence is obtainedvSeasonal component SvAnd a residual component RvExpressed as:
Ov＝Tv+Sv+Rv,v＝1,2,...,N (2)。
further, the STL decomposition specific operation includes the following six steps:
step 1, performing trend removing processing on an original sequence, namely subtracting the trend of the sequence at the last moment, as shown in formula (3):
Figure BDA0002850798430000023
step 2, periodic subsequence smoothing (Cycle-subseries smoothing); to DtMaking periodic subsequence smooth regression operation, respectively extending 1 period n (p) before and after the sequence to reconstitute lengthThe sequence of N +2N (p) is given
Figure BDA0002850798430000031
Wherein v ═ 1-N (p), …, N + N (p);
step 3, periodic subsequence Low-throughput filtering
Figure BDA0002850798430000032
Respectively carrying out the running average operation of the parameters N (p), and 3 for three times, and carrying out LOESS regression again to obtain the sequence with the length of N
Figure BDA0002850798430000033
I.e. a periodic subsequence
Figure BDA0002850798430000034
Low flux trend of (d);
step 4, smoothing the periodic subsequence trend (regression of smoothed cycles-subseries), and seasonally forming
Figure BDA0002850798430000035
Is composed of
Figure BDA0002850798430000036
And
Figure BDA0002850798430000037
a difference value of (a); as shown in formula (4):
Figure BDA0002850798430000038
step 5, removing seasonality (desonasalizing) and Trend smoothing (Trend smoothing), and carrying out comparison on the original sequence OtBy subtracting seasonal components
Figure BDA0002850798430000039
Then, LOESS regression with smoothing coefficient n (t) is performed to obtain trend component
Figure BDA00028507984300000310
As shown in formula (5):
Figure BDA00028507984300000311
step 6, for the original sequence ytObtained by subtracting the above steps
Figure BDA00028507984300000312
And
Figure BDA00028507984300000313
obtaining residual components; as shown in formula (6):
Figure BDA00028507984300000314
further, STL decomposition is carried out to obtain four equal-length sequences of three components and the original sequence, and a two-dimensional matrix M is constructedN×h＝[Ot,Tv,Sv Rv]Where N is a single sequence length and h equals 4 represents 4 components;
and splicing the two-dimensional matrixes of different observation points to obtain the three-dimensional time sequence block with time and space characteristics.
Furthermore, in the space-time fusion prediction module, the three-dimensional convolution module extracts high-dimensional characteristics by performing three-dimensional convolution operation on the time sequence block, and replaces the gate updating operation of the LSTM; the Block-LSTM module takes the output of the three-dimensional convolution module as the input of the LSTM module, and at the moment, the cell data of the Block-LSTM has both time and space characteristics.
Further, the core portion of the LSTM module is the cellular state C transported in each LSTM celltAnd cell state CtThe three gating mechanisms are used to determine the forgetting gate ftInput gate ItAnd an output gate Ot(ii) a Wherein, forget the door ftActivation of the weight matrix W by a sigmoid activation functionfAnd the previous time hidden layer information ht-1And the current input xtDetermining whether the cell retains or discards the previous state information; input gate ItFirstly, activating a weight matrix W through a sigmoid functioniAnd ht-1And xtDetermines updated information by the product of the two, and activates the weight matrix W by the tanh activation functionCAnd ht-1And xtThe product of (a) yields candidate cell information
Figure BDA0002850798430000041
New cell state CtFrom ftAnd Ct-1Sum of products ItAnd with
Figure BDA0002850798430000042
The sum of the products of (a); output gate OtActivation of the weight matrix W by sigmoidoThe product of ht-1 and xt, the current hidden layer state htFrom OtWith tanh activated CtObtaining the product of the two;
LSTM module passing hidden state ht、CtCalculating an error term for each step in the forward direction of the sum gradient deltat, and deducing to obtain a forgetting gate f according to a back propagation principletInput gate ItOutput gate OtParameter W off、WIAnd WoThe calculation propagation process is shown in equation (7):
Figure BDA0002850798430000043
further, the three-dimensional convolution kernel slides on the time sequence block and conducts convolution operation with pixel points on the time sequence block to obtain a high-order three-dimensional feature block, the feature block is evenly divided to obtain four small feature blocks which serve as forgetting gates f of the LSTM cellstInput gate ItOutput gate OtAnd candidate cell Ct(ii) a Each gating mechanism in each Block-LSTM cell carries the timing characteristics of the data Block, where input gate ItWith the hidden layer information h at the previous momentt-1Splicing and activating by using sigmoid; forget door ftSplicing with ht-1 and sigmoid activation function activation(ii) a Candidate cell
Figure BDA0002850798430000044
Splicing with ht-1 and activating by using a tanh activation function; new cell state CtIs composed of
Figure BDA0002850798430000045
And ItProduct of (a) and (f)tAnd previous cell state Ct-1Summing; the cell operation of SDBRNN is shown in formula (8), where x represents the convolution operation.
Figure BDA0002850798430000046
According to the method, time sequence prediction is carried out through a sequence decomposition block recurrent neural network model SDBRNN; the method is suitable for the data sets with different observation points related in reality logic and strong correlation, and richer features can be extracted through time-space synchronous convolution, so that the prediction effect of the sequence is improved.
Drawings
FIG. 1 is a schematic diagram of a network structure of a recurrent neural network model of the present invention.
FIG. 2-1 is an exploded view of the STL structure of the timing block of the present invention, and FIG. 2-2 is a schematic view of the timing block structure of the present invention.
FIG. 3 is a schematic diagram of the structure of the SDBRNN cell of the present invention.
Detailed Description
The invention provides a time sequence prediction method based on fusion sequence decomposition and space-time convolution, which comprises the steps of firstly, carrying out time sequence decomposition on one-dimensional time sequence data based on an STL decomposition strategy, splicing the time sequence data into a three-dimensional data block with time and space, extracting high-dimensional space-time characteristics of the three-dimensional data block by using a three-dimensional convolution model, and finally carrying out back propagation learning on the high-dimensional space-time characteristics by using an LSTM sequence module.
Aiming at the problem of uncertain period in the original STL decomposition, the invention provides an improved algorithm based on trend seasonal decomposition; when the time sequence block is constructed, the components are combined according to the correlation, so that the three-dimensional convolution module can better extract the space-time characteristics; meanwhile, the three-dimensional convolution module is embedded in the LSTM cell and replaces the gate updating operation of the LSTM, so that the LSTM can learn the space-time characteristics of data. Experiments prove the rationality of the time sequence block structure and the effectiveness of the model.
As shown in FIG. 1, the invention provides a sequence Decomposition Block Recurrent Neural Network model (SDBRNN) by utilizing the correlation of space-time dimensions and the advantage of feature enhancement brought by multi-dimensional fusion. The SDBRNN model can be mainly divided into two modules, namely a time sequence Block construction module and a space-time fusion prediction module based on a decomposition strategy, wherein the space-time fusion prediction module comprises a three-dimensional convolution module and a Block-LSTM module.
The time sequence block structure is used for decomposing and combining the characteristics in the time sequence and splicing different sequences according to the sequence of observation points to form a three-dimensional data time sequence block structure module with time sequence characteristics, and comprises the following steps:
as shown in fig. 2-1 and 2-2, the sequential block construction can be divided into an STL decomposition stage and a component reassembly stage. In the STL decomposition stage, the period n (p) of the original one-dimensional input sequence needs to be determined first. The sequence period often depends on the data itself, for example, if the sampling interval of the traffic speed data set is 5 minutes, there are 288 sample points in one day, subjective is one day period, n (p) is 288, and longer period can be one week or even one month period. How to select the period not only affects the sequence decomposition effect, but also affects the final prediction accuracy. In order to optimize the value of the period n (p), firstly, inputting an alternative period n (P)i(i ═ 1,2, …), the original sequence was subjected to STL decomposition in different periods, and the obtained different decomposition results were as shown in formula (1).
Figure BDA0002850798430000061
To pair
Figure BDA0002850798430000062
Respectively with the original sequence OvCalculating DTW distance diIn the method, d is takeniMinimum period parameter n (P)k(k epsilon {1, 2. }) is taken as an STL decomposition parameter, and a trend component T of the original sequence is obtainedvSeasonal component SvAnd a residual component RvIt can be expressed as:
Ov＝Tv+Sv+Rv,v＝1,2,...,N (2)
the specific operation of STL decomposition comprises the following six steps:
step 1, Detrending, namely, Detrending the original sequence, namely subtracting the trend of the sequence at the last moment, as shown in formula (3):
Figure BDA0002850798430000063
step 2, Cycle-subsequences smoothingtMaking periodic subsequence smooth regression operation, making 1 period N (p) respectively extended before and after the sequence, and recording the sequence whose length is N +2N (p)
Figure BDA0002850798430000064
Wherein v ═ 1-N (p), …, N + N (p);
step 3, periodic subsequence Low-throughput filtering
Figure BDA0002850798430000065
Respectively carrying out the moving average operation of the parameters N (p), and 3 for three times, carrying out LOESS regression again, and recording the obtained sequence with the length of N
Figure BDA0002850798430000066
I.e. a periodic subsequence
Figure BDA0002850798430000067
Low flux trend of (a);
step 4, smoothing the periodic subsequence trend (Detrending ofsmoothened cycles-subseries), apparently, seasonally formed
Figure BDA0002850798430000071
Is composed of
Figure BDA0002850798430000072
And
Figure BDA0002850798430000073
a difference of (d); as shown in formula (4):
Figure BDA0002850798430000074
step 5, removing seasonality (desonalizing) and Trend smoothing (Trend smoothing), and carrying out O treatment on the original sequencetBy subtracting seasonal components
Figure BDA0002850798430000075
Then, LOESS regression with smoothing coefficient n (t) is performed to obtain trend component
Figure BDA0002850798430000076
As shown in formula (5):
Figure BDA0002850798430000077
step 6, for the original sequence ytObtained by subtracting the above steps
Figure BDA0002850798430000078
And
Figure BDA0002850798430000079
obtaining residual components; as shown in formula (6):
Figure BDA00028507984300000710
different from the existing method, the independent prediction learning is carried out on the componentsIn the component recombination stage, the model constructs a two-dimensional matrix M by dividing three components obtained by STL decomposition into four sequences with equal length with the original sequenceN×h＝[Ot,Tv,Sv Rv]Where N is a single sequence length and h equals 4 represents 4 components.
It is clear that the order of combining the 4 components will affect the two-dimensional matrix values formed, depending on the nature of the convolution operation, for the given M aboveN×hRow vector T in the middle of the matrixvAnd SvWill be compared to the row vector O at the edgetAnd RvAnd (4) taking part in one convolution calculation. The predicted effect of the different ranking methods will be demonstrated experimentally in the experimental part herein. Thus, a two-dimensional matrix of single observation points subjected to STL decomposition and component recombination is obtained. And finally, splicing the two-dimensional matrixes of different observation points to obtain the three-dimensional time sequence block with time and space characteristics.
Figure BDA00028507984300000711
Figure BDA0002850798430000081
The spatio-temporal fusion prediction module is described below, including a three-dimensional convolution module and an LSTM module. 1) A three-dimensional convolution module: the high-dimensional characteristics are extracted by performing three-dimensional convolution operation on the time sequence block, and the gate updating operation of the LSTM is replaced; 2) LSTM module: the output of the three-dimensional convolution module is used as the input of the LSTM module, and at the moment, the cell data of the LSTM has both time and space characteristics.
The core portion of the LSTM is the cellular state C transported in each LSTM celltAnd cell state CtThe three gating mechanisms are used to determine the forgetting gate ftInput gate ItAnd an output gate Ot. Wherein, forget the door ftActivation of the weight matrix W by a sigmoid activation functionfAnd the previous time hidden layer information ht-1And at presentInput xtDetermining whether the cell retains or discards the previous state information; input gate ItFirstly, activating a weight matrix W through a sigmoid functioniAnd ht-1And xtDetermines updated information by the product of the two, and activates the weight matrix W by the tanh activation functionCAnd ht-1And xtThe product of (a) yields candidate cell information
Figure BDA0002850798430000082
New cell state CtFrom ftAnd Ct-1Sum of products ItAnd
Figure BDA0002850798430000083
the sum of the products of (a); output gate OtActivation of the weight matrix W by sigmoidoThe product of ht-1 and xt, the current hidden layer state htFrom OtWith tanh activated CtThe product of the two is obtained.
LSTM passes through hidden state ht、CtAnd the gradient δ t calculates the error term towards each previous step, the key being that the calculation parameters are based on the partial derivatives of the loss function. Deducing a forgetting gate f according to a back propagation principletInput gate ItOutput gate OtParameter W off、WIAnd WoThe calculation propagation process is shown in equation (7).
Figure BDA0002850798430000084
However, the vast spatial relationships that exist for multi-observation time series cannot be exploited by LSTM. The space-time fusion convolutional network provided by the invention combines the space extraction capability of the convolutional layer and the time sequence extraction capability of the LSTM, and is called Block-LSTM. Different from a simple convolutional layer + LSTM network, the three-dimensional convolution operation of Block-LSTM is realized in the bottom layer inside the LSTM cell and replaces the gate updating and cell updating operation of LSTM.
As shown in the three-dimensional convolution block of FIG. 3, when the three-dimensional convolution kernel is slipped over the time sequence blockPerforming convolution operation on the motion vector and pixel points on the time sequence block to obtain a high-order three-dimensional feature block, and uniformly dividing the feature block to obtain four small feature blocks serving as forgetting gates f of the LSTM cellstInput gate ItOutput gate OtAnd candidate cell Ct. Thus each gating mechanism in each Block-LSTM cell carries the timing characteristics of the data Block. Wherein, the input gate ItWith the hidden layer information h at the previous momentt-1Splicing and activating by using sigmoid; forget door ftSplicing with ht-1 and activating a sigmoid activation function; candidate cell
Figure BDA0002850798430000091
Splicing with ht-1 and activating by using a tanh activation function; new cell state CtIs composed of
Figure BDA0002850798430000092
And ItProduct of (a) and (f)tAnd previous cell state Ct-1And (4) summing. The cell operation of SDBRNN is shown in formula (8), where x represents the convolution operation.
Figure BDA0002850798430000093
By comparison, it can be found that the gate mechanism of Block-LSTM cells works in a manner similar to LSTM, with the most important change being the replacement of state updates between cells by three-dimensional convolution operations. The Block-LSTM can not only extract spatial features by utilizing the correlation among multiple sequences like a CNN network, but also establish the time sequence relation among the sequences by utilizing the LSTM, thereby achieving the space-time prediction.
The SDBRNN model carries out prediction verification on 3 data sets, including two traffic flow data sets PeMS-Bay and Seattle and a power data set Solar-Energy. The three data sets are all multi-observation point time series data sets. The comparison method and parameter settings were as follows:
1) LSTM is a special RNN network, which has a gate mechanism long-short term memory neural network and has better effect on solving the problem of long-term dependence. The effect can be contrasted with the STL decomposition and three-dimensional convolution modules of the models herein. In the comparative experiment, the number of LSTM layers was 2 and the hidden layer size was [16,32 ].
2) Gru, a special RNN network, has a simpler network mechanism than LSTM, has better effect in solving long-term problems and the model is easier to train. In comparative experiments, the number of GRU layers was 2 and the hidden layer size was [16,32 ].
3) Wavenet, a sequence generation model, has stronger time domain view than the common CNN structure, and has good effects on speech generation and text prediction. In a comparative experiment, the number of WaveNet layers is 4, the convolution kernel size is 2, the Residual channel coefficient Residual channel is 32, and the Skip channel coefficient Skip channel is 128.
4) Tcn time convolutional network. The method has the advantages that the method has a more flexible receptive field by utilizing the cavity convolution, and has a residual network structure, so that the gradient of the method in the training process is more stable. In the comparative experiment, the number of TCN layers is 2, the convolution kernel size is 2, and the void coefficient size is 2.
5) ConvLSTM is a variant LSTM network for extracting spatial features by using two-dimensional convolution, has a good extraction effect on the spatial features, and has a good comparison effect with a three-dimensional convolution module of a text model. In the comparative experiment, the number of layers of ConvLSTM was 2, the convolution kernel size was 2 × 2, and the hidden layer size was [16,32 ].
The comparison parameters are 3 conventional evaluation indexes, namely RMSE, MAE and MAPE.
Figure BDA0002850798430000101
TABLE 1 Experimental prediction results
Figure BDA0002850798430000111
As can be seen, in the PeMS-Bay and Seattle traffic data sets, the TCN model with the best short-term prediction performed, the MAPE was 2.94%, while the SDBRNN model had a MAPE of 2.94%5.50%, the best performance of the medium-long term prediction is the SDBRNN proposed herein, which predicts MAPE values of 3.47% and 2.54% in 6 and 12 time slices, which are respectively better than TCN by about 1 and 2 percentage points, which shows that the extraction of different components of the sequence by the STL decomposition module can improve the long-term prediction effect, but does not help obviously in the short-term prediction. It is not difficult to understand the trend component T due to STL decompositionvAnd seasonal ingredient SvThe representative data is the characteristics of the data overall level, which means that the longer the data is, the more cycles the sequence contains, the better the decomposition effect is, and for the local characteristics, the STL has no good strategy for extraction. Therefore, the advantage of SDBRNN is more apparent when the predicted step size is longer in comparison of different models. Particularly, the results of comparing ConvLSTM and SDBRNN show that the effect of the SDBRNN model on long-term prediction is about 2 percent better than that of ConvLSTM, because the convolution module of ConvLSTM only performs two-dimensional convolution operation, namely spatial feature extraction, on a single moment, and the three-dimensional convolution module of SDBRNN simultaneously extracts space-time features, so that the effectiveness of the SDBRNN model on long-term time sequence prediction is verified.

Claims (5)

1. A time sequence prediction method based on fusion sequence decomposition and space-time convolution is characterized in that the method firstly carries out time sequence decomposition on one-dimensional traffic flow time sequence data based on an STL decomposition strategy and splices the time sequence decomposition data into a three-dimensional data block with time and space, then extracts high-dimensional space-time characteristics of the three-dimensional data block by using a three-dimensional convolution model, and finally carries out back propagation learning on the high-dimensional space-time characteristics by using an LSTM sequence module;
the method carries out time sequence prediction through a sequence Decomposition Block Recurrent Neural Network model SDBRNN (Series-Decomposion-Block Recurrent Neural Network); the SDBRNN model comprises two modules, namely a time sequence Block construction module and a space-time fusion prediction module based on a decomposition strategy, wherein the space-time fusion prediction module comprises a three-dimensional convolution module and a Block-LSTM module;
STL decomposition is carried out to obtain four sequences with the same length of three components and the original sequence, and a two-dimensional matrix M is constructedN×h＝[Ot,Tv,Sv Rv]Where N is a single sequence length and h equals 4 represents 4 components;
splicing the two-dimensional matrixes of different observation points to obtain a three-dimensional time sequence block with time and space characteristics; m for the given two-dimensional matrixN×hRow vector T in the middle of the matrixvAnd SvWill be compared to the row vector O at the edgetAnd RvMore than one convolution calculation is involved;
in the space-time fusion prediction module, a three-dimensional convolution module extracts high-dimensional characteristics by performing three-dimensional convolution operation on a time sequence block, and replaces the gate updating operation of LSTM; the Block-LSTM module takes the output of the three-dimensional convolution module as the input of the LSTM module, and the cell data of the Block-LSTM has both time and space characteristics;
the core part of the LSTM module is the cellular state C transported in each LSTM celltAnd cell state CtThe three gating mechanisms are used to determine the forgetting gate ftInput gate ItAnd an output gate Ot(ii) a Wherein, forget the door ftActivation of the weight matrix W by a sigmoid activation functionfAnd the previous time hidden layer information ht-1And the current input xtDetermining whether the cell retains or discards the previous state information; input gate ItFirstly, activating a weight matrix W through a sigmoid functioniAnd ht-1And xtDetermines updated information by the product of the two, and activates the weight matrix W by the tanh activation functionCAnd ht-1And xtThe product of (a) yields candidate cell information
Figure FDA0003615663450000011
New cell state CtFrom ftAnd Ct-1Sum of products ItAnd
Figure FDA0003615663450000012
the sum of the products of (a); output gate OtActivation of the weight matrix W by sigmoidoAnd ht-1 and xtIs obtained whenFront hidden layer state htFrom OtWith tanh activated CtObtaining the product of the two;
LSTM module passing hidden state ht、CtCalculating an error term for each step before the gradient delta t, and deducing to obtain a forgetting gate f according to a back propagation principletInput gate ItOutput gate OtParameter W off、WIAnd WoThe calculation propagation process is shown in equation (7):
Figure FDA0003615663450000021
sliding the three-dimensional convolution kernel on the time sequence block and carrying out convolution operation on the three-dimensional convolution kernel and pixel points on the time sequence block to obtain a high-order three-dimensional feature block, and uniformly dividing the feature block to obtain four small feature blocks serving as a forgetting gate f of the LSTM celltInput gate ItOutput gate OtAnd candidate cell Ct(ii) a Each gating mechanism in each Block-LSTM cell carries the timing characteristics of the data Block, where input gate ItWith the hidden layer information h at the previous momentt-1Splicing and activating by using sigmoid; forget door ftAnd ht-1 concatenation and sigmoid activation function activation; candidate cell
Figure FDA0003615663450000022
And ht-1 splicing is activated using a tanh activation function; new cell state CtIs composed of
Figure FDA0003615663450000023
And ItProduct of (a) and (f)tAnd previous cell state Ct-1Summing; the cell working mode of the SDBRNN is shown as a formula (8), wherein a star represents convolution operation;
Figure FDA0003615663450000024
2. the time-series prediction method based on fusion sequence decomposition and space-time convolution of claim 1, characterized in that the time-series block construction is divided into an STL decomposition stage and a component reconstruction stage.
3. The method of claim 2, wherein in the STL decomposition stage, the period n (p) of the original one-dimensional input sequence is determined;
4. the method as claimed in claim 3, wherein when determining the period n (p), the alternative period n (p) is inputted firsti(i ═ 1,2, …), performing STL decomposition on the original sequence at different periods, respectively, and obtaining different decomposition results as shown in formula (1):
Figure FDA0003615663450000025
to pair
Figure FDA0003615663450000031
Respectively with the original sequence OvCalculating DTW distance diGet diMinimum period parameter n (P)k(k epsilon {1, 2. }) is taken as an STL decomposition parameter, and a trend component T of the original sequence is obtainedvSeasonal component SvAnd a residual component RvExpressed as:
Ov＝Tv+Sv+Rv,v＝1,2,...,N (2)。
5. the method for time-series prediction based on fusion sequence decomposition and space-time convolution of claim 2, wherein the STL decomposition operation includes the following six steps:
step 1, performing trend removing processing on an original sequence, namely subtracting the trend of the sequence at the last moment, as shown in formula (3):
Figure FDA0003615663450000032
step 2, periodic subsequence smoothing (Cycle-subseries smoothing); to DtMaking periodic subsequence smooth regression operation, making 1 period N (p) respectively extended before and after the sequence, and recording the sequence whose length is N +2N (p)
Figure FDA0003615663450000033
Wherein v ═ 1-N (p), …, N + N (p);
step 3, periodic subsequence Low-flux filtration (Low-pass filtration); for is to
Figure FDA0003615663450000034
Respectively carrying out the running average operation of the parameters N (p), and 3 for three times, and carrying out LOESS regression again to obtain the sequence with the length of N
Figure FDA0003615663450000035
I.e. a periodic subsequence
Figure FDA0003615663450000036
Low flux trend of (d);
step 4, smoothing the periodic subsequence trend (regression of smoothed cycles-subseries), and seasonally forming
Figure FDA0003615663450000037
Is composed of
Figure FDA0003615663450000038
And
Figure FDA0003615663450000039
a difference of (d); as shown in formula (4):
Figure FDA00036156634500000310
step 5, removing seasonality (desonalizing) and Trend smoothing (Trend smoothing), and carrying out comparison on the original sequence OvBy subtracting seasonal components
Figure FDA00036156634500000311
Then, LOESS regression with smoothing coefficient n (t) is performed to obtain trend component
Figure FDA00036156634500000312
As shown in formula (5):
Figure FDA00036156634500000313
step 6, for the original sequence OvObtained by subtracting the above steps
Figure FDA00036156634500000314
And
Figure FDA00036156634500000315
obtaining residual components; as shown in formula (6):
Figure FDA00036156634500000316
CN202011525943.8A 2020-12-22 2020-12-22 Time sequence prediction method based on fusion sequence decomposition and space-time convolution Active CN112949896B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202011525943.8A CN112949896B (en) 2020-12-22 2020-12-22 Time sequence prediction method based on fusion sequence decomposition and space-time convolution

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202011525943.8A CN112949896B (en) 2020-12-22 2020-12-22 Time sequence prediction method based on fusion sequence decomposition and space-time convolution

Publications (2)

Publication Number Publication Date
CN112949896A CN112949896A (en) 2021-06-11
CN112949896B true CN112949896B (en) 2022-06-17

Family

ID=76234825

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202011525943.8A Active CN112949896B (en) 2020-12-22 2020-12-22 Time sequence prediction method based on fusion sequence decomposition and space-time convolution

Country Status (1)

Country Link
CN (1) CN112949896B (en)

Families Citing this family (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN113569479B (en) * 2021-07-27 2023-11-10 天津大学 Long-term multi-step control method, device and storage medium for rock mass crack development of stone cave temple
CN114239948B (en) * 2021-12-10 2023-07-21 浙江省交通投资集团有限公司智慧交通研究分公司 Deep traffic flow prediction method, medium and equipment based on time sequence decomposition unit
CN115545315A (en) * 2022-10-12 2022-12-30 重庆移通学院 PM2.5 prediction method based on three-dimensional convolutional neural network and gated cyclic unit
CN117668477B (en) * 2024-01-31 2024-04-26 山东科技大学 Intelligent lightweight processing method and system for ocean big data

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN109862585A (en) * 2019-01-31 2019-06-07 湖北工业大学 A kind of dynamic heterogeneous network flow prediction method based on depth Space-time Neural Network
CN109977098A (en) * 2019-03-08 2019-07-05 北京工商大学 Non-stationary time-series data predication method, system, storage medium and computer equipment
CN111260030A (en) * 2020-01-13 2020-06-09 润联软件系统(深圳)有限公司 A-TCN-based power load prediction method and device, computer equipment and storage medium
CN111325976A (en) * 2020-02-24 2020-06-23 交通运输部水运科学研究所 Short-term traffic flow prediction method and system

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN109862585A (en) * 2019-01-31 2019-06-07 湖北工业大学 A kind of dynamic heterogeneous network flow prediction method based on depth Space-time Neural Network
CN109977098A (en) * 2019-03-08 2019-07-05 北京工商大学 Non-stationary time-series data predication method, system, storage medium and computer equipment
CN111260030A (en) * 2020-01-13 2020-06-09 润联软件系统(深圳)有限公司 A-TCN-based power load prediction method and device, computer equipment and storage medium
CN111325976A (en) * 2020-02-24 2020-06-23 交通运输部水运科学研究所 Short-term traffic flow prediction method and system

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
Effective passenger flow forecasting using STL and ESN based on two improvement strategies;Lan Qin et al.;《Neurocomputing》;20190511(第356期);244-256 *

Also Published As

Publication number Publication date
CN112949896A (en) 2021-06-11

Similar Documents

Publication Publication Date Title
CN112949896B (en) Time sequence prediction method based on fusion sequence decomposition and space-time convolution
CN108448610B (en) Short-term wind power prediction method based on deep learning
CN108510741B (en) Conv1D-LSTM neural network structure-based traffic flow prediction method
CN109597891B (en) Text emotion analysis method based on bidirectional long-and-short-term memory neural network
CN113487088A (en) Traffic prediction method and device based on dynamic space-time diagram convolution attention model
CN111612243B (en) Traffic speed prediction method, system and storage medium
CN107066445B (en) The deep learning method of one attribute emotion word vector
Liu et al. Time series prediction based on temporal convolutional network
CN110390397B (en) Text inclusion recognition method and device
CN107463609A (en) It is a kind of to solve the method for video question and answer using Layered Space-Time notice codec network mechanism
CN112733444A (en) Multistep long time sequence prediction method based on CycleGAN neural network
CN106897268A (en) Text semantic understanding method, device and system
CN108876044B (en) Online content popularity prediction method based on knowledge-enhanced neural network
CN110570035B (en) People flow prediction system for simultaneously modeling space-time dependency and daily flow dependency
CN110458181A (en) A kind of syntax dependency model, training method and analysis method based on width random forest
CN109829495A (en) Timing image prediction method based on LSTM and DCGAN
CN113112791A (en) Traffic flow prediction method based on sliding window long-and-short term memory network
CN110321805A (en) A kind of dynamic expression recognition methods based on sequential relationship reasoning
CN113935489A (en) Variational quantum model TFQ-VQA based on quantum neural network and two-stage optimization method thereof
CN113919593A (en) ME-MBiGRU short-time traffic flow prediction method based on attention mechanism
Wang et al. TATCN: time series prediction model based on time attention mechanism and TCN
Niu et al. ALSTM: adaptive LSTM for durative sequential data
CN116959252A (en) Space-time convolution traffic flow prediction method and system based on R-C-R structure
CN116543289A (en) Image description method based on encoder-decoder and Bi-LSTM attention model
Sun et al. Time series prediction based on time attention mechanism and lstm neural network

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant