CN116011638A - Urban space-time prediction method based on space-time attention - Google Patents
Urban space-time prediction method based on space-time attention Download PDFInfo
- Publication number
- CN116011638A CN116011638A CN202211715849.8A CN202211715849A CN116011638A CN 116011638 A CN116011638 A CN 116011638A CN 202211715849 A CN202211715849 A CN 202211715849A CN 116011638 A CN116011638 A CN 116011638A
- Authority
- CN
- China
- Prior art keywords
- space
- time
- attention
- self
- module
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 40
- 239000013598 vector Substances 0.000 claims abstract description 64
- 230000004927 fusion Effects 0.000 claims abstract description 12
- 239000011159 matrix material Substances 0.000 claims description 74
- 230000002123 temporal effect Effects 0.000 claims description 10
- 238000006243 chemical reaction Methods 0.000 claims description 9
- 230000008569 process Effects 0.000 claims description 8
- 239000013604 expression vector Substances 0.000 claims 1
- 230000007246 mechanism Effects 0.000 description 6
- 230000008447 perception Effects 0.000 description 6
- 206010039203 Road traffic accident Diseases 0.000 description 2
- 238000013528 artificial neural network Methods 0.000 description 2
- 238000010586 diagram Methods 0.000 description 2
- 230000000694 effects Effects 0.000 description 2
- 238000002474 experimental method Methods 0.000 description 2
- 230000003993 interaction Effects 0.000 description 2
- 230000003068 static effect Effects 0.000 description 2
- 230000009466 transformation Effects 0.000 description 2
- 101001095088 Homo sapiens Melanoma antigen preferentially expressed in tumors Proteins 0.000 description 1
- 235000008694 Humulus lupulus Nutrition 0.000 description 1
- 102100037020 Melanoma antigen preferentially expressed in tumors Human genes 0.000 description 1
- 230000002159 abnormal effect Effects 0.000 description 1
- 230000003044 adaptive effect Effects 0.000 description 1
- 238000013473 artificial intelligence Methods 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 238000004364 calculation method Methods 0.000 description 1
- 238000007418 data mining Methods 0.000 description 1
- 238000013135 deep learning Methods 0.000 description 1
- 230000001934 delay Effects 0.000 description 1
- 238000013461 design Methods 0.000 description 1
- 238000011156 evaluation Methods 0.000 description 1
- 230000006870 function Effects 0.000 description 1
- 230000009191 jumping Effects 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000010606 normalization Methods 0.000 description 1
- 230000000750 progressive effect Effects 0.000 description 1
Images
Classifications
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02A—TECHNOLOGIES FOR ADAPTATION TO CLIMATE CHANGE
- Y02A30/00—Adapting or protecting infrastructure or their operation
- Y02A30/60—Planning or developing urban green infrastructure
Landscapes
- Management, Administration, Business Operations System, And Electronic Commerce (AREA)
Abstract
The invention discloses a city space-time prediction method based on space-time attention, which comprises the steps of constructing and utilizing a city space-time prediction model based on space-time attention to carry out city space-time prediction; the city space-time prediction model comprises a space-time encoder, wherein the space-time encoder comprises a semantic space self-attention module, a geographic space self-attention module, a time self-attention module and a heterogeneous attention fusion module, and the space-time encoder is used for obtaining space-time feature vectors according to high-dimensional space-time representation vectors. The urban space-time prediction method disclosed by the invention can acquire long-distance semantic neighborhood information, short-distance geographic neighborhood information and long-distance time dependency relationship in the historical data, and accurately performs urban space-time prediction by simulating the complex and dynamic space-time dependency relationship of the historical data.
Description
Technical Field
The invention relates to the technical field of urban space-time prediction, in particular to a method for urban space-time prediction based on space-time attention.
Background
Urban space-time prediction is the "main battlefield" where artificial intelligence is combined with smart cities. Common city data prediction tasks include traffic condition prediction, population density prediction, riding service demand prediction, air quality prediction, and the like.
At present, urban space-time dynamic prediction based on deep learning neural network is a mainstream method in the field of urban data mining at present due to higher accuracy; however, GNN-based models have three main limitations in urban space-time prediction:
(1) The spatial correlation between spatial locations in spatio-temporal data, as affected by travel patterns and accidents in cities, is time-varying, not static. For example, the correlation between two spatial nodes becomes stronger during early peaks and weaker during other periods. The existing method mainly builds a spatial correlation model in a static mode, and limits the ability of learning a dynamic city space-time mode whether the spatial correlation model is predefined or adaptive;
(2) Due to the functional division of cities, two remote sites may reflect similar spatio-temporal patterns, which means that the spatial dependency between sites is long distance. However, the existing method is often based on local design, and cannot capture the long-distance dependency relationship. For example, GNN-based models suffer from excessive smoothness, making it difficult to capture long-range spatial dependencies;
(3) In a space-time system, the propagation of spatial information between different locations may be subject to time delays. For example, when a traffic accident occurs at one place, a delay time of several minutes is required to affect traffic conditions at neighboring places. However, in a typical GNN model-based instant messaging mechanism, such features are omitted.
In this regard, the present invention proposes a dynamic long-distance urban space-time prediction model based on propagation delay perception, and utilizes the model to perform urban space-time prediction, so as to overcome the above-mentioned drawbacks existing in the prior art.
Disclosure of Invention
In view of this, the present invention provides a space-time prediction method for cities based on space-time attention, which is implemented by constructing a space-time prediction model of cities, wherein the space-time prediction model includes a space-time encoder based on a self-attention mechanism, captures dynamic spatial correlation of short distance and long distance through a geospatial self-attention module and a semantic space self-attention module, and captures time patterns of dynamic and long distance through a time self-attention module, thereby accurately simulating complex and dynamic space-time dependency of historical data.
In order to achieve the above purpose, the present invention adopts the following technical scheme:
a city space-time prediction method based on space-time attention comprises the steps of constructing a city space-time prediction model based on space-time attention, and carrying out city space-time prediction by utilizing the city space-time prediction model; the city space-time prediction model comprises a space-time encoder which comprises a semantic space self-attention module, a geographic space self-attention module, a time self-attention module and a heterogeneous attention fusion module, and is used for obtaining a space-time characteristic vector according to a high-dimensional space-time representation vector,
the semantic space self-attention module is used for acquiring long-distance semantic neighborhood information according to the high-dimensional space-time representation vector and the binary semantic mask matrix;
the geographic space self-attention module is used for acquiring short-distance geographic neighborhood information according to the high-dimensional space-time representation vector and the binary geographic mask matrix;
the time self-attention module is used for acquiring a long-distance time dependency relationship according to the high-dimensional space-time representation vector;
and the heterogeneous attention fusion module is used for carrying out serial projection on the output results of the semantic space self-attention module, the geographic space self-attention module and the time self-attention module to obtain the space-time feature vector.
Preferably, the high-dimensional space-time representation vector is obtained by converting historical space-time data and urban traffic road network structures through a data embedding layer.
Preferably, the high-dimensional space-time representation vector is input to the semantic space self-attention module, the geographic space self-attention module and the time self-attention module after being divided according to the number of the attention modules.
Preferably, the semantic space self-attention module, the geospatial self-attention module and the temporal self-attention module map the high-dimensional space-time representation vector into a query matrix, a key matrix and a value matrix when calculating.
Preferably, the expression of the calculation process of the semantic space self-attention module and the geospatial self-attention module is as follows:
in the formula ,
respectively a query matrix, a key matrix and a value matrix mapped according to the high-dimensional space-time representation vector, t represents time, s represents space, and a represents Hadamard product, M sem Representing a binary semantic mask matrix, M geo Representing a binary geographic mask matrix, d' representing the dimensions of the query, key, and value matrix.
Preferably, the geospatial self-attention module, prior to computation, the key matrix
Firstly, obtaining a key matrix of time delay capable of simulating space information propagation through a delay sensing characteristic conversion module>
The acquisition step comprises the following steps:
s1, acquiring a short-term space-time mode according to input historical space-time data, and converting the short-term space-time mode into a memory vector m i ，
S2, obtaining a high-dimensional representation u of the S-step historical space-time data sequence of the node n from the time slice (t-S+1) to t t,n And the memory vector m i Comparing to obtain a similarity vector w i ，
S3, according to the similarity vector w i The comprehensive history sequence representation r is obtained according to the following formula t,n ，
in the formula ,Wc Is a parameter matrix which can be learned;
s4, integrating the history sequence characteristics r of N nodes t,n After concatenation, adding with the key matrix to obtain a modified key matrix
/>
Preferably, the step of acquiring the short-term spatio-temporal pattern comprises:
slicing the historical space-time data by utilizing a sliding window to obtain a historical space-time data sequence;
k-Shape clustering is carried out on the historical space-time data sequence to obtain the short-term space-time mode
Preferably, the binary geographic mask matrix is constructed according to the relation between the distance between two nodes and a preset threshold value; the binary semantic mask matrix is constructed according to the following steps,
k nodes with highest similarity with the current node are selected as semantic neighbors;
and enabling the weight between the current node and the semantic neighbor to be 1 and the rest to be 0.
Preferably, the space-time encoder has multiple layers, and the space-time feature vector is obtained by performing jump connection on the output result of each layer of space-time encoder.
Preferably, the urban space-time prediction model further comprises an output layer for obtaining predicted urban space-time data according to the space-time feature vector,
preferably, the output layer comprises two fully connected layers.
Preferably, the heterogeneous attention fusion module performs a serial projection process, and the expression formula is:
wherein ,
representing a connector, Z geo ,Z sem ,Z t Is output connection, h geo ，h sem ，h t Respectively represent the number of the three attention modules, W O ∈R d×d Is a learnable projection matrix, d represents the dimensions of the high-dimensional spatiotemporal representation vector.
According to the technical scheme, the urban space-time prediction method based on space-time attention is provided, and the problems caused by complex features of space-time data, namely dynamic, long distance and time delay, are fully solved.
Compared with the prior art, the space-time encoder based on the self-attention mechanism is constructed, the geographic space self-attention module and the semantic space self-attention module are specifically arranged, the local geographic neighborhood and the global semantic neighborhood are modeled through different shielding methods so as to capture the dynamic space correlation of short distance and long distance, and the time self-attention module is used for capturing the dependency relationship of the dynamic and long distance;
another beneficial effect of the invention includes: the feature conversion module of delay perception is arranged in front of the geospatial self-attention module, and the geospatial self-attention module is expanded through the feature conversion module of delay perception so as to explicitly simulate the time delay of spatial information propagation.
The urban space-time prediction model provided by the invention can execute multi-step prediction and single-step prediction, and simultaneously releases the limit of the existing GNN model in urban space-time prediction, and has accurate prediction performance and good interpretability.
Drawings
In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings that are required to be used in the embodiments or the description of the prior art will be briefly described below, and it is obvious that the drawings in the following description are only embodiments of the present invention, and that other drawings can be obtained according to the provided drawings without inventive effort for a person skilled in the art.
FIG. 1 is a schematic diagram of a space-time encoder according to the present invention;
FIG. 2 is a schematic diagram of a city space-time prediction model structure provided by the invention.
Detailed Description
The following description of the embodiments of the present invention will be made clearly and completely with reference to the accompanying drawings, in which it is apparent that the embodiments described are only some embodiments of the present invention, but not all embodiments. All other embodiments, which can be made by those skilled in the art based on the embodiments of the invention without making any inventive effort, are intended to be within the scope of the invention.
The embodiment of the invention discloses a city space-time prediction method based on space-time attention, which comprises the following steps: constructing a city space-time prediction model based on space-time attention, and carrying out city space-time prediction by utilizing the constructed city space-time prediction model;
the space-time encoder based on the self-attention mechanism can simulate complex and dynamic space-time dependency.
Structurally, as shown in fig. 1, the space-time encoder includes a semantic space self-attention module, a geospatial self-attention module, a temporal self-attention module, and a heteroattention fusion module, respectively, for obtaining space-time feature vectors according to high-dimensional space-time representation vectors, specifically,
the semantic space self-attention module is used for acquiring long-distance semantic neighborhood information according to the high-dimensional space-time representation vector and the binary semantic mask matrix;
the geographic space self-attention module is used for acquiring short-distance geographic neighborhood information according to the high-dimensional space-time representation vector and the binary geographic mask matrix;
the time self-attention module is used for acquiring a long-distance time dependency relationship according to the high-dimensional space-time representation vector;
and the heterogeneous attention fusion module is used for carrying out serial projection on the output results of the semantic space self-attention module, the geographic space self-attention module and the time self-attention module to obtain a space-time feature vector.
Firstly, the high-dimensional space-time representation vector is obtained by converting historical space-time data and urban traffic road network structures through a data embedding layer. The data embedding layer can retain as much spatial structure information and time sequence information as possible in the original data in the process of converting the original input into a high-dimensional representation. Specifically, the input of the embedding layer is the original space-time data, and the original space-time data is projected into a space-time data embedding vector X epsilon R through the full connection layer T×N×d Where N is the number of spatial nodes, T is the time step and d is the feature dimension.
In one embodiment, after the high-dimensional space-time representation vector is obtained, the high-dimensional space-time representation vector is divided according to the number of self-attention modules and then is respectively input into the semantic space self-attention module, the geographic space self-attention module and the time self-attention moduleA force module. If you h geo ，h sem ，h t Representing the number of three attention modules, respectively, the acquired high-dimensional spatiotemporal representation vector is divided into (h geo +h sem +h t ) The dimensions d' of each module are, correspondingly:
d′＝d/(h geo +h sem +h t )
further, the semantic space self-attention module, the geospatial self-attention module, and the temporal self-attention module map the high-dimensional spatio-temporal representation vectors into a query matrix Q, a key matrix K, and a value matrix V prior to computation.
The spatial self-attention module, namely the semantic spatial self-attention module and the geographic spatial self-attention module, designed by the invention is used for capturing dynamic spatial correlation in space-time data. Formally, at time t, a query, key and value matrix from the attention operation is first obtained from the high-dimensional spatiotemporal representation vector, respectively:
in the formula, wherein
D' is the dimension of the query, key, and value matrix, which is a learnable parameter. />
Then, self-attention operations are applied in the spatial dimension to model interactions between nodes and to derive spatial correlations (attention scores) of all nodes over time, i.e.:
it can be seen from this that the spatial correlation between nodes
Are different, i.e. dynamic, on different time slices. Thus, the spatial self-attention module may be adjusted to capture dynamic spatial correlation; and multiplying the attention score by a value matrix to obtain the output of the spatial self-attention module, wherein the specific formula is as follows:
for spatial self-attention in the above formula, each node interacts with all nodes, which is equivalent to treating the spatial map as a fully connected map. However, only a few interactions between node pairs are necessary, including nearby node pairs and node pairs that are far apart but functionally similar. Thus, the present application introduces two graph mask matrices M geo and Msem To capture both short-range and long-range spatial dependencies in the spatiotemporal data.
Specifically, a binary geographic mask matrix M is constructed according to the relation between the distance between two nodes and a preset threshold value geo The method comprises the steps of carrying out a first treatment on the surface of the I.e. the weight is reset to 1 only if the distance between two nodes (number of hops in the figure) is smaller than the threshold lambda, otherwise to 0. In this way, the attention of the node pairs farther away can be masked.
Further, a binary semantic mask matrix M is constructed according to the following steps sem ，
K nodes with highest similarity with the current node are selected as semantic neighbors;
and enabling the weight between the current node and the semantic neighbor to be 1 and the rest to be 0.
That is, from a remote perspective, the present application uses a Dynamic Time Warping (DTW) algorithm to calculate the similarity of historical spatiotemporal data between nodes. In this way, node pairs that exhibit similar spatiotemporal patterns due to similar city functions are found.
Based on the two mask matrices obtained, the expression of the computation process of the semantic space self-attention module and the geospatial self-attention module can be defined as:
in the formula ,
respectively a query matrix, a key matrix and a value matrix mapped according to the high-dimensional space-time representation vector, t represents time, s represents space, and a represents Hadamard product, M sem Representing a binary semantic mask matrix, M geo Representing a binary geographic mask matrix, d' representing the dimensions of the query, key, and value matrix.
Meanwhile, in the propagation of real-world spatiotemporal data, there is a propagation delay, for example, in the case of traffic data, when a traffic accident occurs in one region, it may take several minutes to affect the traffic conditions in the neighboring region. Thus, in one embodiment provided by the present invention, the geospatial self-attention module employs a key matrix prior to computation
Firstly, obtaining a key matrix of time delay capable of simulating space information propagation through a delay sensing characteristic conversion module>
The space propagation delay sensing feature conversion module can capture propagation delay from short-term historical space-time data of each node, incorporate delay information into a key matrix of the geospatial self-attention module and can clearly simulate the time delay of the propagation of the space information.
Further, a key matrix is acquired
The method comprises the following steps:
s1, acquiring a short-term space-time mode according to input historical space-time data, and utilizing an embedded matrix W in a delay perception feature conversion module m Converting short-term spatiotemporal patterns into memory vectors m i ，
In one embodiment, representative short-term spatiotemporal patterns are derived from input historical spatiotemporal data
The acquisition process comprises the following steps: historical spatiotemporal data X epsilon R entered using a sliding window of length S T×N×C Slicing on a time axis T, (C is the dimension of the spatiotemporal feature, such as c=2 representing inflow and outflow) to obtain a cluster of historical spatiotemporal data sequences, i.e. several time sequences of length S;
k-Shape clustering is carried out on the obtained plurality of historical space-time data sequences to obtain a short-term space-time mode
Specifically, firstly, slicing the input historical space-time data by using a sliding window with the size of S to obtain a cluster of historical space-time data sequences, and then, carrying out k-Shape clustering on the historical space-time data sequences. The k-Shape clustering algorithm is a time series clustering method, and the Shape of the time series is kept and is not influenced by scaling and movement. The present application uses the center point p of each cluster i To represent the cluster, where p i Also a time series of length S. Then use
Representing the clustering result, where N p Is the total number of clusters, i.e. can +.>
Considered as a set of short-term spatio-temporal patterns.
Further, embedding matrix W in feature transformation module using delay perception m Space-time pattern set
Is converted into a memory vector, namely:
m i ＝p i W m
s2, acquiring an S-step historical space-time data sequence from a time slice (t-S+1) to t of a node n, and setting the sequence as x (t-S+1:t),n Further through embedding matrix W in the delay-aware feature transformation module u Obtaining a high-dimensional representation u t,n The method comprises the following steps:
u t,n ＝x (t-S+1:t),n W u
similar spatio-temporal patterns may have similar effects on nearby spatio-temporal patterns, especially abnormal spatio-temporal patterns, such as congestion. Thus, the present application compares the historical spatiotemporal data sequence of each node with the extracted spatiotemporal pattern set, fusing information of similar patterns into the historical traffic sequence representation of each node. Specifically, the similarity vector is obtained by the following formula,
s3, according to the similarity vector w i The time space mode set is weighted and summed to obtain the comprehensive history sequence representation r t,n The formula is as follows,
in the formula ,Wc Is a parameter matrix which can be learned; integrated historical sequence representation r t,n Comprising nodes n from time slices (t-S+1) to tIs provided.
S4, integrating the history sequence characteristics r of N nodes t,n After concatenation, adding with the key matrix to obtain a modified key matrix
Namely: />
wherein ,Rt ∈R N×d′ Comprehensive history sequence characteristic r representing N nodes t,n And d' is the dimension of the key matrix as a result of the concatenation.
Novel key matrix
The historical spatiotemporal data information of all nodes from time slice (t-S+1) to t is integrated at time slice t. The query matrix may take into account historical spatiotemporal data of other nodes when calculating the product of the query matrix and the new key matrix to obtain the spatial dependency of the time slices t in the formula. This process explicitly simulates the time delay in the propagation of spatial information. This module is not added to the semantic space self-attention module because short-term spatiotemporal data of distant nodes has little impact on the current node.
For temporal self-attention, there is a dependency (e.g., periodicity, trending) between spatiotemporal data in different time slices, and the dependency is different in different cases. Thus, the present application employs a temporal self-attention module to discover dynamic temporal patterns. Formally, for node n, a query, key and value matrix are obtained first, respectively:
wherein 
T represents time, which is a learnable parameter. Then, as before, the self-attention operation is applied in the time dimension, and the time dependency relationship among all the time slices of the node n is obtained as follows:
it can be seen that the temporal self-attention can discover the dynamic temporal patterns of different nodes in the spatiotemporal data. In addition, the self-attention in time has global information modeling capability, and long-distance time dependency relationship among all time slices can be modeled.
In one embodiment, the output of the temporal self-attention module is:
when three types of attention mechanisms are defined, the present application merges heterogeneous attention into a multi-headed self-attention block to reduce the computational complexity of the model. Specifically, the heterogeneous attention fusion module is arranged at the end and is used for carrying out serial projection on the output result of the attention mechanism corresponding to the geography, the semantics and the time head, so that the space and the time information are integrated at the same time. Formally, the process of serial projection can be expressed by the following formula:
wherein ,
representing a connector, Z geo ,Z sem ,Z t Is output connection, h geo ,h sem ,h t For the number of three attention modules, W O ∈R d×d Is a learnable projection matrix.
In one embodiment, a feed-forward network of fully-connected locations is employed after the heterogeneous attention fusion module to obtain an output X o ∈R T×N×d The method comprises the steps of carrying out a first treatment on the surface of the And the position fully connected feed forward network includes layer normalization and residual connection.
In one embodiment, the space-time encoder is provided with a plurality of layers, and the space-time characteristic vector is obtained by jumping and connecting the output result of each layer of space-time encoder.
Furthermore, the urban space-time prediction model disclosed by the invention further comprises an output layer, wherein the output layer is used for obtaining predicted urban space-time data according to the space-time feature vector, namely, the input of the output layer is the space-time feature vector obtained by the space-time encoder, and the output is the prediction of the space-time data under a plurality of time steps in the future.
The output layer comprises two full-connection layers, and multi-step prediction and feature dimension conversion into required output dimensions are respectively realized through the two full-connection layers.
In one embodiment, an overall urban spatiotemporal predictive model is shown in FIG. 2, and predicts future spatiotemporal data based on input historical spatiotemporal data and the structure of the urban traffic network, the model comprising, structurally, a data embedding layer, a plurality of stacked spatiotemporal encoders and an output layer.
And the executing steps of the model specifically comprise:
s1: converting the historical space-time data and the urban traffic road network structure into high-dimensional space-time representation vectors through a data embedding layer;
s2: dividing the high-dimensional space-time representation vector input into three types of attention heads of a layer 1 'space-time encoder' according to the size proportion of the dimension;
s3: each attention header maps the input high-dimensional spatiotemporal representation vector into a query, key and value matrix (Q, K, V)
S4: the method comprises the steps of inputting original space-time data, inquiry, keys and a value matrix into a delay perception feature conversion module, inputting the output of the module into a geographic space self-attention module, and acquiring short-distance geographic neighborhood information by combining a binary geographic mask matrix;
s5: inputting the query, the key and the value matrix into a semantic space self-attention module, and acquiring long-distance semantic neighborhood information by combining a binary semantic mask matrix;
s6: inputting the query, the key and the value matrix into a 'time self-attention module', and acquiring a long-distance time dependency relationship;
s7: the outputs of the three types of self-attention modules are spliced by a heterogeneous attention fusion module to obtain space-time feature vectors;
s8: inputting the space-time characteristic vector into a 'fully connected feedforward neural network';
s9: inputting the space-time characteristic vector output by the 1 st layer space-time encoder as a high-dimensional space-time representation vector to the 2 nd layer space-time encoder, repeating the operations of S2 to S8, and the like until the output of the L th layer space-time encoder is obtained;
s10: the outputs of the space-time encoders of the 1 st layer to the L th layer are connected through jump, so that a final space-time feature vector is obtained;
s11: and inputting the final space-time feature vector into an output layer to obtain predicted space-time data.
In order to verify the effect achieved by the technical solution of the invention, peMS04, peMS07, peMS08 are the sets of spatiotemporal data in the United states with only one dimension of flow, respectively; and on a Beijing city taxi flow data set T-Drive of two dimensions of input flow and output flow, a New York city taxi flow data set NYCTaxi and a Chicago city bicycle flow data set BikeCHI, city space-time prediction experiments are carried out, and the invention uses average absolute error MAE, mean square error RMSE and average absolute percentage error MAPE as evaluation indexes to evaluate, and experimental results are shown in the following table, wherein the method provided by the invention is named PDFormer, and other rows are different existing space-time prediction methods.
Table one: american dataset comparison results
And (II) table: beijing, new York and Chicago dataset comparison-inflow flow
Table three: beijing, new York and Chicago dataset comparison-inflow flow
Experiments prove that the performance of the model provided by the invention exceeds that of a plurality of existing methods, and accurate urban space-time prediction is realized.
In the present specification, each embodiment is described in a progressive manner, and each embodiment is mainly described in a different point from other embodiments, and identical and similar parts between the embodiments are all enough to refer to each other.
The previous description of the disclosed embodiments is provided to enable any person skilled in the art to make or use the present invention. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the generic principles defined herein may be applied to other embodiments without departing from the spirit or scope of the invention. Thus, the present invention is not intended to be limited to the embodiments shown herein but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.
Claims (10)
1. A city space-time prediction method based on space-time attention is characterized in that a city space-time prediction model based on space-time attention is constructed, and city space-time prediction is carried out by utilizing the city space-time prediction model; the city space-time prediction model comprises a space-time encoder which comprises a semantic space self-attention module, a geographic space self-attention module, a time self-attention module and a heterogeneous attention fusion module, and is used for obtaining a space-time characteristic vector according to a high-dimensional space-time representation vector,
the semantic space self-attention module is used for acquiring long-distance semantic neighborhood information according to the high-dimensional space-time representation vector and the binary semantic mask matrix;
the geographic space self-attention module is used for acquiring short-distance geographic neighborhood information according to the high-dimensional space-time representation vector and the binary geographic mask matrix;
the time self-attention module is used for acquiring a long-distance time dependency relationship according to the high-dimensional space-time representation vector;
and the heterogeneous attention fusion module is used for carrying out serial projection on the output results of the semantic space self-attention module, the geographic space self-attention module and the time self-attention module to obtain the space-time feature vector.
2. The method for urban space-time prediction based on space-time attention according to claim 1, wherein the high-dimensional space-time representation vector is obtained by converting historical space-time data and urban traffic road network structures through a data embedding layer.
3. The method according to claim 1, wherein the high-dimensional spatiotemporal expression vector is divided by the number of attention modules and is input to the semantic spatial self-attention module, the geospatial self-attention module and the temporal self-attention module, respectively.
4. The method of claim 1, wherein the semantic spatial self-attention module and the geospatial self-attention module are calculated as follows:
in the formula ,
respectively a query matrix, a key matrix and a value matrix mapped according to the high-dimensional space-time representation vector, t represents time, s represents space, and a represents Hadamard product, M sem Representing a binary semantic mask matrix, M geo Representing a binary geographic mask matrix, d' representing the dimensions of the query, key, and value matrix.
5. The method of claim 4, wherein the geospatial self-attention module, prior to computation, uses the key matrix to predict the city space-time
Firstly, obtaining a key matrix of time delay capable of simulating space information propagation through a delay sensing characteristic conversion module>
The acquisition step comprises the following steps:
s1, acquiring a short-term space-time mode according to input historical space-time data, and converting the short-term space-time mode into a memory vector m i ，
S2, obtaining a high-dimensional representation u of the S-step historical space-time data sequence of the node n from the time slice (t-S+1) to t t,n And the memory vector m i Comparing to obtain a similarity vector w i ，
S3, according to the similarity vector w i The comprehensive history sequence representation r is obtained according to the following formula t,n ，
in the formula ,Wc Is a parameter matrix which can be learned;
s4, integrating the history sequence characteristics r of N nodes t,n After concatenation, adding with the key matrix to obtain a modified key matrix
6. The method of spatiotemporal attention-based urban spatiotemporal prediction of claim 5, wherein the step of acquiring short term spatiotemporal patterns comprises:
slicing the input historical space-time data by utilizing a sliding window to obtain a historical space-time data sequence;
k-Shape clustering is carried out on the historical space-time data sequence to obtain the short-term space-time mode
7. The method for urban space-time prediction based on space-time attention according to claim 1, wherein the binary geographic mask matrix is constructed according to the relation between the distance between two nodes and a preset threshold value; the binary semantic mask matrix is constructed according to the following steps,
k nodes with highest similarity with the current node are selected as semantic neighbors;
and enabling the weight between the current node and the semantic neighbor to be 1 and the rest to be 0.
8. The method of claim 1, wherein the space-time encoder has a plurality of layers, and the space-time feature vector is obtained by jumping-connection of the output result of the space-time encoder of each layer.
9. The method of claim 1, wherein the model further comprises an output layer for obtaining predicted urban spatiotemporal data based on the spatiotemporal feature vectors.
10. The urban space-time prediction method based on space-time attention according to claim 1, wherein the heterogeneous attention fusion module performs a series projection process, and the expression is as follows:
wherein ,
representing a connector, Z geo ,Z sem ,Z t Is output connection, h geo ，h sem ，h t The number of the three attention modules, W O ∈R d×d Is a learnable projection matrix, d is the dimension of the high-dimensional spatiotemporal representation vector. />
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202211715849.8A CN116011638A (en) | 2022-12-29 | 2022-12-29 | Urban space-time prediction method based on space-time attention |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202211715849.8A CN116011638A (en) | 2022-12-29 | 2022-12-29 | Urban space-time prediction method based on space-time attention |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN116011638A true CN116011638A (en) | 2023-04-25 |
Family
ID=86036818
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202211715849.8A Pending CN116011638A (en) | 2022-12-29 | 2022-12-29 | Urban space-time prediction method based on space-time attention |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN116011638A (en) |
-
2022
- 2022-12-29 CN CN202211715849.8A patent/CN116011638A/en active Pending
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN109887282B (en) | Road network traffic flow prediction method based on hierarchical timing diagram convolutional network | |
| CN110781838B (en) | Multi-mode track prediction method for pedestrians in complex scene | |
| CN113313947B (en) | Road condition evaluation method of short-term traffic prediction graph convolution network | |
| CN114299723B (en) | Traffic flow prediction method | |
| CN113570859B (en) | Traffic flow prediction method based on asynchronous space-time expansion graph convolution network | |
| CN113362368B (en) | Crowd trajectory prediction method based on multi-level space-time diagram neural network | |
| CN115510174A (en) | Road network pixelation-based Wasserstein generation countermeasure flow data interpolation method | |
| CN115600022A (en) | Training and processing method, device and medium of spatio-temporal data processing model | |
| CN114461931A (en) | User trajectory prediction method and system based on multi-relation fusion analysis | |
| Yang et al. | Long-short term spatio-temporal aggregation for trajectory prediction | |
| Chen et al. | Pedestrian behavior prediction model with a convolutional LSTM encoder–decoder | |
| CN116108983A (en) | Urban carbon emission prediction method and equipment based on collaborative space-time convolution network | |
| CN115082896A (en) | Pedestrian trajectory prediction method based on topological graph structure and depth self-attention network | |
| CN116030537B (en) | Three-dimensional human body posture estimation method based on multi-branch attention-seeking convolution | |
| Zhang et al. | Off-deployment traffic estimation—a traffic generative adversarial networks approach | |
| CN116011638A (en) | Urban space-time prediction method based on space-time attention | |
| CN116246465A (en) | Traffic flow prediction method based on generated space-time evolution modeling | |
| CN114781696B (en) | Model-free accident influence range prediction method for urban road network | |
| CN115018134A (en) | Pedestrian trajectory prediction method based on three-scale spatiotemporal information | |
| CN115457081A (en) | Hierarchical fusion prediction method based on graph neural network | |
| CN117475638B (en) | Traffic OD passenger flow prediction method and system based on multichannel hypergraph convolutional network | |
| CN117635216B (en) | Shared bicycle demand prediction method based on improved GCN (generalized communication network) | |
| CN117688257B (en) | Long-term track prediction method for heterogeneous user behavior mode | |
| Chen et al. | Spatial-temporal attention networks for vehicle trajectory prediction | |
| CN117671952A (en) | Traffic flow prediction method and system based on time-space synchronous dynamic graph attention network |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |