CN116610919A - A Spatial Time Series Forecasting Method Based on Recurrent Graph Operator Neural Network - Google Patents

Abstract

The invention discloses a spatial time series prediction method based on a cyclic graph operator neural network, comprising: modeling the traffic road network to obtain a graph structure model, and abstracting the sensors in the traffic road network into the graph structure model nodes; standardize the data collected by sensors; build a variety of graph operator networks, as data feature capturers, which use different methods to aggregate the information of each node in the graph structure model; build a GGRU unit, respectively The operator network is embedded in the GRU unit of the neural network; the feature information captured by different graph operator networks is aggregated through the integrator; an encoder-decoder architecture is constructed by using multiple integrators to realize the spatial multivariate time series predict. This method has the ability to understand information from multiple angles, effectively improves the understanding of spatial multivariate time series data, and then improves the accuracy of prediction.

Description

A Spatial Time Series Forecasting Method Based on Recurrent Graph Operator Neural Network

technical field

The invention relates to the technical field of space time series prediction, in particular to a space time series prediction method based on a cycle graph operator neural network.

Background technique

Spatial time series forecasting has a very wide range of application requirements in modern times, such as: (1) Meteorological forecasting: Meteorological forecasting is an important application of space time series forecasting. By modeling meteorological data, it is possible to predict the changing trend of meteorological variables such as temperature, rainfall, and wind speed in the future, and provide decision support for weather forecasting, disaster warning, and agricultural production. (2) Energy demand forecast: For energy suppliers and consumers, it is very important to understand the changing trend of future energy demand. By modeling historical energy demand data, future energy demand can be forecasted so that production planning and supply chain management can be adjusted. (3) Traffic forecasting: Traffic forecasting can help urban planners and traffic managers to better manage traffic flow and improve traffic efficiency. By modeling historical traffic data, it is possible to predict future road congestion, traffic accident rates, etc., so that corresponding measures can be taken. (4) Prediction of population mobility: population mobility is an important factor in urban planning and social policy formulation. By modeling historical population flow data, future population flow trends can be predicted for rational planning of urban infrastructure and social resource allocation. (5) Stock price prediction: Stock price prediction is an important application in the financial field. By modeling historical stock price data, it is possible to predict future stock price trends and provide decision support for investors.

At present, deep learning methods are mainly used for multivariate time series forecasting. This neural network based model can be effectively used for multivariate time series forecasting. For example: (1) Recurrent neural network RNN: It is a neural network that can process sequence data, and it captures the temporal relationship of the sequence by taking the output of the previous moment as the input of the current moment. In multivariate time series forecasting, multiple RNNs can be used to process each series separately, and then their outputs are concatenated to get the final forecast. RNN is widely used in multivariate time series forecasting, for example, it has achieved good results in traffic flow forecasting, stock price forecasting and other fields. (2) Long-short-term memory network LSTM: It is a special RNN that can better handle long sequences and long-term dependencies. In multivariate time series forecasting, multiple LSTMs can be used to process each series, and then their outputs are concatenated to get the final forecast. Compared to RNNs, LSTMs perform better at handling long sequences and long-term dependencies, so they may be more suitable in some applications. (3) Convolutional neural network CNN: It is a neural network mainly used for image processing, but it can also be used for multivariate time series prediction in some cases. In multivariate time series forecasting, multiple sequences can be regarded as multiple channels, and then multiple convolution kernels are used to perform convolution operations on each channel, and finally the outputs of all channels are connected to obtain the final prediction result. CNNs have relatively few applications in multivariate time series forecasting, but in some applications they may be better suited. (4) Attention mechanism Attention: It is a mechanism that can give different weights according to different parts of the input data, and can be used in multivariate time series prediction. In multivariate time series forecasting, an attention mechanism is used to automatically learn the importance of different parts of the sequence, and the attention weights are applied to the prediction of each sequence. The application of attention mechanisms to multivariate time series forecasting is relatively new, but has achieved good results in some applications.

However, the above multivariate time series forecasting either does not consider the spatial dependence between the series at all, or only considers the spatial dependence between the series from a single perspective. In fact, the underlying relationships between these sequences are complex. Therefore, a new prediction method is urgently needed to ensure that it can fully mine these underlying relationships from multiple different dimensions. Just like humans read words, the metaphors of words are often deeper than the surface meaning.

Contents of the invention

The purpose of the present invention is to provide a spatial time series prediction method based on a cyclic graph operator neural network, which has the ability to understand information from multiple angles, effectively improves the understanding of spatial multivariate time series data, and then improves the accuracy of prediction.

In order to achieve the above purpose, this application proposes a spatial time series prediction method based on a cyclic graph operator neural network, including:

Modeling the traffic road network to obtain a graph structure model, abstracting the sensors in the traffic road network into nodes in the graph structure model;

Standardize the data collected by the sensor;

Construct a variety of graph operator networks as data feature capturers, which use different methods to aggregate the information of each node in the graph structure model;

Construct the GGRU unit, and embed various graph operator networks into the GRU unit of the neural network;

The feature information captured by different graph operator networks is aggregated through the integrator;

Using multiple integrators, a sequence-to-sequence encoder-decoder architecture is constructed to realize the prediction of spatially multivariate time series.

Further, use Indicates the traffic flow data collected by all sensors at time r, where N represents the number of sensors, and P represents the number of traffic indicators detected by each sensor; use /> Represents the historical observations within T' time stamps, use /> Represents the predicted value of T timestamps in the future; and then determines the function of the neural network learning graph structure model:

in, is the neural network to be fitted.

Further, the specific way to normalize the data collected by the sensor is as follows:

Among them, X represents the training set sample, X ^* represents the normalized data, E(X) represents the mean value of the training set sample, and D(X) represents the variance of the training set sample.

Further, build a variety of graph operator networks as data feature capturers, specifically:

Construct a graph diffusion convolution operator DC Operator, which is a graph operator network based on a static graph;

Construct a graph-gated attention operator GA Operator, which is a dynamic graph-based graph operator network.

Furthermore, the specific method of constructing the graph diffusion convolution operator DC Operator is as follows:

The graph diffusion convolution operator uses a random walk strategy:

Among them, X represents the input of the model, H represents the output of the model; its diffusion process uses a limited S-step truncation; D _O , D _I represent the out-degree matrix and in-degree matrix in the figure, Represents the forward state transition matrix and reverse state transition matrix; α, β∈[0,1] represent the restart probability of random walk respectively; /> Represents model parameters, and Θ _{O[q, p, s]} = α(1-α) ^s , Θ _{I[q, p, k]} = β(1-β) ^s ;

Use a distance-based Gaussian kernel to get the critical matrix W:

Here dist(v _i , v _j ) denotes the distance between sensors v _i and v _j ; σ is the standard deviation of the set of distances.

Furthermore, the specific method of constructing the graph-gated attention operator GA Operator is as follows:

Using the K-head attention mechanism in the graph-gated attention operator, there is a K-dimensional gating vector g _i for each node i:

where, /> Represents the set of all neighbor nodes of node i; x _i =X _i,: represents the feature vector of node i; /> the reference vectors of all neighbors of node i, and /> Max means to take the maximum value by element; /> Indicates that the final result is mapped to K dimensions and scaled to [0, 1];

Dynamically obtain the attention weight matrix between nodes:

in, Represents a linear transformation whose parameter is θ _xa ; /> Represents a linear transformation whose parameter is θ _za ; ie /> denotes a linear layer; θ denotes different parameters.

Get the output vector y _{i of node i} :

Among them, K represents the number of attention heads, and k represents that the kth attention head is currently being executed; Represents a linear transformation whose parameter is θ ₀ ;/> Indicates that the parameter is /> linear transformation.

Furthermore, the GGRU unit uses Represent different graph operators:

H ^(t) ＝u ^(t) ⊙H ^(t-1) +(1-u ^(t) )⊙C ^(t)

Among them, X ^(t) and H ^(t) represent the input and output of the t-th time stamp; r ^(t) and u ^(t) represent the reset gate (Reset gate) and the update gate (Update gate) of the t-th time stamp. gate); Θ _r , Θ _u , Θ _C are different filter parameters; Γ _g represents the specified graph network execute in /> Operator; ⊙ means Hadamard product.

As a further step, the feature information captured by different graph operator networks is aggregated through the integrator, specifically:

in, Indicates the GRU unit embedded with graph operators, I indicates the number of types of graph operators, /> Indicates that the output dimension is d ₀ feed-forward neural network, and τ indicates that the tanh activation function is used in the last layer.

As a further step, the input to the encoder is historical observations The output of the decoder is the future prediction value />

Compared with the prior art, the above technical solution adopted by the present invention has the advantage that the present invention uses a deep learning model and combines multiple graph operator networks to model the spatial dependencies between nodes from multiple angles, and at the same time through the integration The device enables the effective aggregation of information captured by multiple graph operators. It enhances the model's ability to understand the underlying dependencies between multivariate time series, and improves the prediction accuracy.

Description of drawings

Figure 1 is a structure diagram using parallel single input and single output;

Figure 2 is a structure diagram using parallel multi-input and multi-output;

Figure 3 is a structure diagram using parallel single input, single output and class residuals;

Figure 4 is a structure diagram using parallel multi-input, multi-output and class residuals;

Figure 5 is a structure diagram using serial single input and single output;

Fig. 6 is a structural diagram of multi-input and multi-output using serial connection;

Fig. 7 is a structure diagram using series single input, single output and class residual;

Fig. 8 is a structure diagram using serial multi-input, multi-output and class residuals;

Figure 9 is a structural diagram of the GGRU model;

FIG. 10 is an architecture diagram of an encoder decoder.

Detailed ways

In order to make the purpose, technical solution and advantages of the present application clearer, the present application will be further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described here are only used to explain the present application, and are not intended to limit the present application, that is, the described embodiments are only part of the embodiments of the present application, rather than all the embodiments.

Accordingly, the following detailed description of the embodiments of the application provided in the accompanying drawings is not intended to limit the scope of the claimed application, but merely represents selected embodiments of the application. Based on the embodiments of the present application, all other embodiments obtained by those skilled in the art without making creative efforts belong to the scope of protection of the present application.

This application provides a spatial time series prediction method based on a cyclic graph operator neural network, which specifically includes:

Step 1: Modeling the traffic road network to obtain a graph structure model, abstracting the sensors in the traffic road network into nodes in the graph structure model;

Specifically, use Indicates the traffic flow data collected by all sensors at time t, where N represents the number of sensors, and P represents the number of traffic indicators detected by each sensor; use /> Indicates the historical observations within T′ time stamps, use /> Represents the predicted value of T timestamps in the future; and then determines the function of the neural network learning graph structure model:

in, is the neural network to be fitted.

Step 2: Normalize the data collected by the sensor:

Among them, X represents the training set sample, X ^* represents the normalized data, E(X) represents the mean value of the training set sample, and D(X) represents the variance of the training set sample.

Step 3: Construct a variety of graph operator networks as data feature capturers, which use different methods to aggregate the information of each node in the graph structure model;

Specifically, the present invention includes two types of graph operators, which use different methods to aggregate the information of each node in the graph structure model, and embed it into the GRU unit as a replacement for the original linear unit.

Step 3.1: Construct a graph diffusion convolution operator DC Operator, which is a graph operator network based on a static graph;

Step 3.1.1: The graph diffusion convolution operator adopts a random walk strategy:

Here, X represents the input of the model, and H represents the output of the model. Its diffusion process uses a finite K-step truncation. D _O , D _I represent the out-degree and in-degree matrices in the graph, Represents the forward state transition matrix and the reverse state transition matrix. α, β ∈ [0, 1] represent the restart probability of the random walk, respectively. /> denote model parameters, and Θ _{O[q, p, s]} = α(1-α) ^s , Θ _{I[q, p, s]} = β(1-β) ^s . /> Represents the probability value of performing S-step diffusion from the i-th node to its neighbor nodes, the former represents the forward propagation probability, and the latter represents the back propagation probability.

Step 3.1.2: Use the distance-based Gaussian kernel to get the critical matrix W:

Here dist(v _i , v _j ) denotes the distance between sensors v _i and v _j ; σ is the standard deviation of the set of distances.

Step 3.2: Build a graph-gated attention operator GA Operator, which is a graph operator network based on dynamic graphs;

Step 3.2.1: Using the K-head attention mechanism in the graph-gated attention operator, there is a K-dimensional gating vector g _i for each node i:

in, Represents the set of all neighbor nodes of node i; x _i =X _i,: represents the feature vector of node i; the reference vectors of all neighbors of node i, and /> Max means to take the maximum value by element; /> Indicates that the final result is mapped to K dimensions and scaled to [0, 1];

Step 3.2.2: Dynamically obtain the attention weight matrix between nodes:

in, Represents a linear transformation whose parameter is θ _xa ; /> Represents a linear transformation whose parameter is θ _za ;

Step 3.2.3: Get the output vector y _i of node i:

Step 4: Construct the GGRU unit, and embed various graph operator networks into the GRU unit of the neural network, see Figure 9, here Represent different graph operators:

H ^(t) ＝u ^(t) ⊙H ^(t-1) +(1-u ^(t) )⊙C ^(t)

Among them, X ^(t) and H ^(t) represent the input and output of the t-th time stamp; r ^(t) and u ^(t) represent the reset gate (Reset gate) and the update gate (Update gate) of the t-th time stamp. gate); Θ _r , Θ _u , Θ _C are different filter parameters; Represents a network in the specified graph /> execute in /> Operator; ⊙ means Hadamard product.

Step 5: aggregate feature information captured by different graph operator networks through the integrator;

in, Represents a GRU unit embedded with graph operators. Figures 1-8 show eight aggregator structures.

Step 6: Use multiple integrators to construct a sequence-to-sequence encoder-decoder architecture to realize the prediction of spatial multivariate time series, see Figure 10;

Specifically, the input of the encoder is the historical observation value The output of the decoder is the future prediction value />

In this embodiment, the Ubuntu system is used as the development environment, Python is used as the development language, and Pytorch is used to build the framework. Adopt a kind of spatial time series forecasting method based on cycle graph operator neural network of the present invention, carry out the forecast of traffic flow: obtain the graph structure model representation of traffic road network, adopt public METR-LA data set; Divide data set into The ratio of training set, validation set and test set is 7:1:2. In order to compare the performance of the algorithm, it is compared with some commonly used multivariate time series forecasting methods. These include statistical learning method ARIMA; machine learning method LSVR; deep learning method FC-LSTM; and graph-based deep learning methods DCRNN, STGCN, GaAN, ASTGCN, GMAN. At the same time, a variety of evaluation indicators are used to evaluate the performance of the model comprehensively, including MAE, RMSE, MAPE, etc.

Among them, _xi represents the actual data, Represents the data predicted by the model.

The comparison results are shown in Table 1: the best evaluation indicators among all algorithms will be bolded.

Table 1: Performance comparison of different traffic speed prediction models on the METR-LA dataset

Table 1 compares the prediction results of the iGoRNN model with other baseline models. By comparing the scores of each model on long-term and short-term time series forecasting, it can be seen that the graph-based neural network model achieves better forecast accuracy. For the smoother sequence PeMS-BAY, some conventional models achieve good results even in short-term forecasts (15 minutes), but poorly in long-term forecasts (60 minutes). For the unstable sequence Metr-LA, traditional models perform poorly in both short-term and long-term forecasts. On the other hand, the GaAN model performs well on the Metr-LA dataset. GMAN is more suitable for smoother sequences. All of these illustrate the importance of introducing graph structures in complex time series forecasting. In addition, the model designed by the present invention shows optimal or suboptimal performance in long-term and short-term forecasting, especially for more complex time series, because it simultaneously introduces the advantages of multiple graph structures.

To sum up, compared with other more advanced methods, a spatial time series prediction method based on cyclic graph operator neural network proposed by the present invention has better performance and robustness, and can adapt to more complex multivariate Time series forecasting problems.

The foregoing descriptions of specific exemplary embodiments of the present invention have been presented for purposes of illustration and description. These descriptions are not intended to limit the invention to the precise form disclosed, and obviously many modifications and variations are possible in light of the above teaching. The exemplary embodiments were chosen and described in order to explain the specific principles of the invention and its practical application, thereby enabling others skilled in the art to make and use various exemplary embodiments of the invention, as well as various Choose and change. It is intended that the scope of the invention be defined by the claims and their equivalents.

Claims (9)

1. The space time sequence prediction method based on the cyclic graph algorithm sub-neural network is characterized by comprising the following steps of:

modeling a traffic road network to obtain a graph structure model, and abstracting a sensor in the traffic road network into nodes in the graph structure model;

carrying out standardization processing on data acquired by a sensor;

constructing various graph calculation sub-networks as feature traps of data, wherein the feature traps respectively use different modes to aggregate information of each node in the graph structure model;

constructing GGRU units, and respectively embedding various graph calculation sub-networks into GRU units of the neural network;

aggregating the characteristic information captured by different graph sub-networks through an integrator;

a sequence-to-sequence encoder-decoder architecture is constructed using a plurality of integrators to implement prediction of spatial multi-element time series.

2. The method for predicting the space-time sequence based on the cyclic graph sub-neural network according to claim 1, whereinThe traffic flow data acquired by all the sensors at the time t are represented, wherein N represents the number of the sensors, and P represents the number of traffic indexes detected by each sensor; use->Representing historical observations within T' time stamps, with +.>Predicted values representing T time stamps in the future; and then determining a function of the neural network learning graph structure model:

wherein ,is the neural network to be fitted.

3. The space-time sequence prediction method based on the cyclic graph sub-neural network according to claim 1, wherein the specific mode of carrying out normalization processing on the data acquired by the sensor is as follows:

wherein X represents training set sample, X ^* Representing normalized data, E (X) represents training set sample mean and D (X) represents training set sample variance.

4. The space-time sequence prediction method based on the cyclic graph sub-neural network according to claim 1, wherein a plurality of graph sub-networks are constructed as feature traps of data, specifically:

constructing a graph diffusion convolution Operator DC Operator, which is a graph calculation sub-network based on a static graph;

a graph gating attention Operator GA Operator is constructed, which is a graph computation sub-network based on dynamic graphs.

5. The space-time sequence prediction method based on the cyclic graph Operator neural network according to claim 4, wherein the specific mode of constructing the graph diffusion convolution Operator DC Operator is as follows:

the graph diffusion convolution operator adopts a random walk strategy:

wherein X represents the input of the model and H represents the output of the model; the diffusion process uses limited S steps for cutting; d (D) _O ,D _I The out-degree matrix and in-degree matrix in the graph are represented,representing a forward state transition matrix and a reverse state transition matrix; alpha, beta E [0,1 ]]Respectively representing the restarting probability of random walk; />Represents model parameters, and Θ _O[q,p,s] ＝α(1-α) ^s ,Θ _I[q,p,k] ＝β(1-β) ^s ；

The critical matrix W is obtained using a distance-based gaussian kernel:

here dist (v) _i ,v _j ) Representing sensor v _i and v_j A distance therebetween; σ is the standard deviation of the distance set.

6. The space-time sequence prediction method based on the cyclic graph Operator neural network according to claim 4, wherein the specific mode of constructing the graph gating attention Operator GA Operator is as follows:

using the K-head attention mechanism in the graph gating attention operator, there is one K-dimensional gating vector g for each node i _i ：

wherein ,representing all neighbor node sets of node i; x is x _i ＝X _i,: A feature vector representing node i; />Reference vectors of all neighboring nodes of node i, and z _i ＝/>Max represents taking the maximum value by element; />Representing the final result mapped to K-dimension and scaled to [0,1 ]]Between them;

dynamically acquiring an attention weight matrix among nodes:

wherein ,representing the parameter theta _xa Is a linear transformation of (2); />Representing the parameter theta _za Is a linear transformation of (2);

obtaining an output vector y of the node i _i :

Where K represents the number of attention heads and K represents the kth note currently being performedA force-imparting head;representing the parameter theta ₀ Is a linear transformation of (2); />The expression parameter is->Is a linear transformation of (a).

7. The method for spatial-temporal sequence prediction based on a cyclic graph sub-neural network of claim 1, wherein the GGRU unit is used inRepresenting different graphic operators:

H ^(t) ＝u ^(t) ⊙H ^(t-1) +(1-u ^(t) )⊙C ^(t)

wherein ,H^(t) ,H ^(t) Input and output representing a t-th timestamp; r is (r) ^(t) ,u ^(t) A reset gate and a gate state of the update gate representing a t-th timestamp; theta (theta) _r ,Θ _u ,Θ _C Is a different filter parameter;represented in the specified graph network +.>Middle execution->An operator; as indicated by the letter Hadamard product.

8. The space-time sequence prediction method based on cyclic graph sub-neural network according to claim 1, wherein the integrator is used for aggregating the characteristic information captured by different graph sub-networks, specifically:

wherein ,representing GRU units embedded with graphic operators, I representing the number of categories of graphic operators,/-, etc.>Representing an output dimension d ₀ Feedforward neural network, τ, means using the tanh activation function in the last layer.

9. The method of claim 1, wherein the encoder input is a historical observationThe output of the decoder is the future prediction value +.>