CN112532439B - Network flow prediction method based on attention multi-component space-time cross-domain neural network model - Google Patents

Network flow prediction method based on attention multi-component space-time cross-domain neural network model Download PDF

Info

Publication number
CN112532439B
CN112532439B CN202011326835.8A CN202011326835A CN112532439B CN 112532439 B CN112532439 B CN 112532439B CN 202011326835 A CN202011326835 A CN 202011326835A CN 112532439 B CN112532439 B CN 112532439B
Authority
CN
China
Prior art keywords
data
cross
domain
attention
wireless cellular
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
CN202011326835.8A
Other languages
Chinese (zh)
Other versions
CN112532439A (en
Inventor
陈赓
曾庆田
孙强
段华
邵睿
徐先杰
张旭
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Shandong University of Science and Technology
Original Assignee
Shandong University of Science and Technology
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Shandong University of Science and Technology filed Critical Shandong University of Science and Technology
Priority to CN202011326835.8A priority Critical patent/CN112532439B/en
Publication of CN112532439A publication Critical patent/CN112532439A/en
Application granted granted Critical
Publication of CN112532439B publication Critical patent/CN112532439B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L41/00—Arrangements for maintenance, administration or management of data switching networks, e.g. of packet switching networks
    • H04L41/14—Network analysis or design
    • H04L41/147—Network analysis or design for predicting network behaviour
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/045—Combinations of networks
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/048—Activation functions
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/049—Temporal neural networks, e.g. delay elements, oscillating neurons or pulsed inputs
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L41/00—Arrangements for maintenance, administration or management of data switching networks, e.g. of packet switching networks
    • H04L41/14—Network analysis or design
    • H04L41/145—Network analysis or design involving simulating, designing, planning or modelling of a network
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04W—WIRELESS COMMUNICATION NETWORKS
    • H04W24/00—Supervisory, monitoring or testing arrangements
    • H04W24/06—Testing, supervising or monitoring using simulated traffic
    • Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
    • Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
    • Y02D—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE
    • Y02D30/00—Reducing energy consumption in communication networks
    • Y02D30/70—Reducing energy consumption in communication networks in wireless communication networks

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Health & Medical Sciences (AREA)
  • Computing Systems (AREA)
  • Biomedical Technology (AREA)
  • Biophysics (AREA)
  • Computational Linguistics (AREA)
  • Data Mining & Analysis (AREA)
  • Evolutionary Computation (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Molecular Biology (AREA)
  • Artificial Intelligence (AREA)
  • General Engineering & Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Mathematical Physics (AREA)
  • Software Systems (AREA)
  • Health & Medical Sciences (AREA)
  • Computer Networks & Wireless Communication (AREA)
  • Signal Processing (AREA)
  • Management, Administration, Business Operations System, And Electronic Commerce (AREA)
  • Mobile Radio Communication Systems (AREA)

Abstract

The invention discloses a network flow prediction method based on an attention multi-component space-time cross-domain neural network model, belongs to the technical field of intelligent communication, and solves the problem of prediction of wireless cellular network flow. Firstly, dividing wireless cellular traffic data into neighboring data, daily cycle data and weekly cycle data according to the periodic characteristics of the wireless cellular traffic data; then modeling the neighboring data, the daily cycle data and the weekly cycle data through a conv-LSTM structure or a conv-GRU structure; different weights are adaptively distributed to the three kinds of characteristic data through an attention layer, so that the characteristic extraction capability of the three kinds of characteristic data is improved, and the characteristic information which interferes with the prediction moment is inhibited; and finally, combining the embedding of the timestamp characteristics, fusing various cross-domain data and carrying out flow prediction by using a common auxiliary model. The model can effectively utilize the periodic characteristics of wireless cellular flow data, saves model training time, greatly reduces workload, and further improves the prediction performance of network flow.

Description

Network flow prediction method based on attention multi-component space-time cross-domain neural network model
Technical Field
The invention belongs to the technical field of intelligent communication, and particularly relates to a network flow prediction method based on an attention multi-component space-time cross-domain neural network model.
Background
With the advent of the age of 5G/B5G, the number of mobile devices and internet of things has exponentially increased around the world, and the demand for wireless mobile data has rapidly increased. How to scientifically and reasonably allocate and optimize the existing cellular network resources, improve the utilization rate of the resources and reduce the energy consumption of the cellular base station is a problem to be considered and solved in the communication industry.
The method and the device can accurately predict the wireless cellular flow, and are beneficial to the development of base station site selection, urban area planning, regional flow prediction and other works. However, accurate prediction of wireless traffic flow is a very challenging problem, mainly for the following 3 reasons. First, the generation source of wireless communication network traffic is a user having mobility, and the mobility of the wireless user causes spatial dependency of traffic among a plurality of areas. In particular, the advent of new types of traffic has enabled people to reach from one end of a city to the other in a short period of time. This makes the spatial dependence of wireless traffic not only local, but rather has a large-scale global dependence. On the other hand, the wireless traffic flow also has a dependency in the time dimension, and the traffic value at a certain time has a high correlation with the traffic values at its neighboring time (short-term dependency) and the corresponding time of a certain day (periodicity). Second, the spatial constraint problem that multi-source cross-domain data creates for wireless traffic. The reasons for affecting the generation of wireless traffic in a certain area are diverse. When traffic prediction is performed, not only is a regular pattern implied by wireless service traffic mined only from the perspective of historical data, but also spatial constraint factors generated by other cross-domain and cross-source data on the traffic should be considered. Factors such as base station data of a certain area, point of interest information, social activity level of an area, etc. all affect the change of traffic. Therefore, how to efficiently fuse the multi-source cross-domain data which seems not to have a direct relation with the wireless service flow is a difficult problem to be solved at present. Third, how to achieve high accuracy of wireless cellular traffic prediction in consideration of space-time factors and in combination with cross-domain data is also a difficult problem.
Currently, the main model methods for wireless cellular traffic prediction are: (1) an integrated moving average autoregressive model (ARIMA); (2) an exponential smoothing method (ES); (3) linear regression method (LR); (4) support vector machine regression method (SVR); (5) multi-layer perceptron Method (MLP); (6) based on long-short time memory network method (LSTM); (7) convolutional neural network based approach (CNN); (7) an LSTM model; (8) a space-time convolutional network STDenseNet model; (9) an STNet model; (10) a STMNet model; (11) STCNet model, etc. These model methods for solving the wireless cellular network traffic prediction are considered from the aspects of space factors, time factors, space-time factors and the like.
Disclosure of Invention
The invention provides a multi-component space-time cross-domain neural network model method based on an attention mechanism, which is used for efficiently fusing multi-source cross-domain data, fully utilizing the time-space characteristics of the multi-source cross-domain data, introducing the attention mechanism and further improving the accuracy of wireless cellular flow prediction.
In order to achieve the purpose, the invention adopts the following technical scheme:
a network flow prediction method based on attention multi-component space-time cross-domain neural network model comprises the following steps:
(1) performing Pearson correlation analysis and matrixing processing on three service data, namely short message service data, telephone service data and Internet service data;
(2) gridding and dividing different areas, and clustering and classifying the areas;
(3) wireless cellular flow of the three services is divided into neighbor data, day cycle data and week cycle data according to the characteristics of the neighbor, day cycle and week cycle;
(4) performing correlation analysis and matrixing processing on the cross-domain data, and fusing;
(5) extracting the characteristics of the timestamp of the wireless cellular traffic;
(6) and fusing and inputting various cross-domain data and service data into the attention multi-component space-time cross-domain neural network model, and finally outputting a wireless cellular network flow prediction result.
Preferably, the step (1) specifically comprises the following steps:
(1-1) analyzing the correlation among three service data, namely short messages, telephone and the Internet, and analyzing the periodicity and difference of different service data and the difference of different regional data;
(1-2) processing three service data of short message, telephone and Internet into three 100 x 100 matrixes with the same size, wherein each element in the matrixes represents a flow data value of a certain service.
Preferably, the step (2) specifically comprises the following steps:
(2-1) dividing the predicted area into 100 x 100 grid areas, each grid corresponding to a data value of wireless cellular traffic of a certain service;
(2-2) according to the similarity and the difference of wireless cellular traffic of different areas, the similar areas are gathered together to obtain three different classes, and then model training is carried out on the different classes.
Preferably, the step (3) specifically comprises the following steps:
(3-1) extracting neighbor data of the wireless cellular traffic data, the neighbor data being a cellular traffic data sequence segment of a history time period directly adjacent to the predicted time t;
(3-2) extracting daily cycle data of the wireless cellular traffic data, the daily cycle data being a cellular traffic data sequence segment which is the same as the predicted target time in n days before the predicted t time;
and (3-3) extracting cycle-per-cycle data of the wireless cellular traffic data, wherein the cycle-per-cycle data is a cellular traffic data sequence segment which has the same attribute and the same time as the n previous weeks of the predicted t time and the predicted target week.
Preferably, the step (4) specifically includes the following steps:
and (4-1) performing Pearson correlation coefficient analysis on the acquired three cross-domain data sets of Social information Social, base station distribution BS and interest point distribution POI to obtain the correlation, similarity and correlation characteristics of cross-domain data and different service data.
(4-2) processing the three kinds of cross-domain data into three matrices of 100 × 100;
and (4-3) packaging the three kinds of cross-domain data subjected to the matrixing processing into a tensor.
Preferably, the step (5) specifically comprises the following steps:
(5-1) extracting four characteristic attributes of week, hour, workday and weekend from the timestamp of the wireless cellular traffic and processing the four characteristic attributes into a vector;
(5-2) transforming the processed vector into a 100 x 100 matrix.
Preferably, the step (6) specifically comprises the following steps:
(6-1) cellular traffic D with close proximity t h Day-periodic cellular traffic D t d And periodic cellular traffic D t w Three parts of characteristic input data are led into a conv-GRU structure with two layers, pass through an attention layer and distribute different weights to the flow data with the adjacency, the daily periodicity and the weekly periodicity through the attention layer;
(6-2) putting the timestamp feature matrix into a two-layer fully-connected neural network for embedded learning, and performing timestamp feature modeling;
(6-3) fusing three kinds of cross-domain data including base station distribution BS, Social information Social and POI (point of interest) of the prediction area into a set D cross Introducing the two-layer convolutional neural network into a two-layer convolutional neural network for spatial correlation modeling;
and (6-4) splicing the initial characteristic outputs of the steps into a new tensor according to the specified dimensionality, inputting the new tensor into a dense connection convolutional network DenseNet, and finally outputting a flow prediction result.
Preferably, in the step (6-1), the attention model is divided into an input layer, a hidden layer and an attention layer, and the specific process is as follows:
1) firstly, a conv-GRU structure is utilized to obtain a hidden layer state (h) 1 ，h 2 ，…，h t )；
2) Calculating the influence e of each current input position on the current position i t The formula is as follows:
Figure BDA0002794519290000031
wherein v is a 、W a 、U a Is the weight value of the attention network, relu (-) is the activation function, T is the total number of time intervals, S is the current input shapeState;
3) to e t Performing softmax normalization to obtain an attention weight distribution, wherein the formula is as follows:
Figure BDA0002794519290000032
wherein exp (·) is an exponential function with a natural constant e as the base;
4) using alpha t Weighted summation is carried out to obtain a vector c t The formula is as follows:
Figure BDA0002794519290000033
preferably, in the step (6-1), the conv-GRU structure may be replaced with a conv-LSTM structure.
Preferably, in said step (6-4), the DenseNet network comprises L layers, each layer implementing a complex function transformation, the complex function comprising the batch regularization BN, the activation function Relu, and the convolution operation Conv.
The invention has the following beneficial technical effects:
the method of the invention starts from the characteristics of time and space factors, carries out fine grained division on wireless cellular traffic data, fully utilizes the time-space characteristics thereof, introduces an attention mechanism layer to adaptively distribute weight to the divided wireless cellular traffic, improves the characteristic extraction capability thereof, inhibits the characteristic information which generates interference to the prediction moment, and further improves the prediction efficiency and accuracy of the model to the wireless cellular traffic.
Drawings
FIG. 1 is a flow chart of data preprocessing according to an embodiment of the present invention;
fig. 2(a), (b), and (c) are dynamic characteristic graphs of three different services of Sms, Call, and Internet in different areas in the time dimension, respectively, according to an embodiment of the present invention;
FIG. 3 is a diagram illustrating Pearson correlation between wireless traffic and a cross-domain data set according to an embodiment of the present invention;
FIG. 4 is a diagram illustrating a tensor T according to an embodiment of the present invention;
FIG. 5 is a schematic diagram of a multi-component spatiotemporal cross-domain neural network model (att-MCSTCNet) based on an attention mechanism according to an embodiment of the present invention;
FIG. 6 is a diagram of a wireless cellular traffic time series segment as input by an embodiment of the present invention;
FIG. 7 is a schematic diagram of an LSTM structure according to an embodiment of the present invention;
FIG. 8 is a schematic structural diagram of a GRU according to an embodiment of the present invention;
fig. 9 is a schematic structural diagram of an attention according to an embodiment of the invention.
Detailed Description
The invention is described in further detail below with reference to the following figures and detailed description:
the method comprises the following specific steps:
(1) performing Pearson correlation analysis and matrixing processing on three data including short messages, telephone calls and the Internet: analyzing the correlation among three service data, namely short messages, telephone and the Internet, and analyzing the periodicity and the difference of different service data and the difference of different regional data; processing three kinds of service data of short messages, telephones and the Internet into three matrixes with the same size, namely 100 multiplied by 100; wherein each element in the matrix represents a traffic data value for a certain service.
(2) Gridding and dividing the Milan city, and clustering and classifying the divided different areas: dividing a predicted region (Milan city) into 100 x 100 grid regions, enabling each grid to correspond to a data value of wireless cellular traffic of a certain service of the matrix, gathering similar regions together according to similarity and difference of the wireless cellular traffic of different regions to obtain three different classes, and then performing model training on the different classes.
(3) Firstly, extracting neighbor data of wireless cellular traffic data, namely cellular traffic data sequence segments of a period of historical time directly adjacent to the predicted t moment; then, extracting daily cycle data of the wireless cellular traffic data, namely cellular traffic data sequence segments which are the same as the predicted target time in n days before the predicted t time; and finally, extracting cycle data of the wireless cellular traffic data, namely cellular traffic data sequence segments which have the same attribute and the same time as the target week of the previous n weeks of the predicted time t.
(4) Performing correlation analysis and matrixing processing on cross-domain data, and fusing: performing Pearson correlation coefficient analysis on the acquired three cross-domain data sets of Social information (Sociai), base station distribution (BS) and point of interest (POI) to obtain the characteristics of correlation, similarity, correlation degree and the like of cross-domain data and different service data; then, the three kinds of cross-domain data are also processed into three matrixes of 100 multiplied by 100, and the sizes of the matrixes are kept consistent with those of the other services; and encapsulating the three types of cross-domain data subjected to matrixing into a tensor.
(5) Performing feature extraction on the time stamp of the wireless cellular traffic: extracting four characteristic attributes of week, hour, working day and weekend from the timestamp of the wireless cellular flow and processing the four characteristic attributes into a vector; the processed vector is transformed into a 100 x 100 matrix, which is guaranteed to be the same size as the matrix.
(6) First, for neighbor cellular traffic D t h Day-cycle cellular traffic D t d Periodic cellular traffic D t w Modeling, wherein three part characteristic inputs are led into a conv-LSTM or conv-GRU structure of two layers, and then the weight of historical cellular traffic information which is more critical to the target moment is increased through an attention layer, so that the weight of other information is reduced, the purpose of filtering irrelevant information is achieved, and the efficiency and the accuracy of wireless cellular traffic prediction are further improved. Secondly, performing feature modeling on the timestamp, putting a timestamp feature matrix into a two-layer fully-connected neural network for embedded learning, and then performing spatial correlation modeling on cross-domain data; wherein D cross Three types of base station distribution BS, Social information Social and interest point distribution POI in the area are fused into a set of three types of cross-domain data, and a fused cross-domain data set D cross A two-layer convolutional neural network is introduced to process the data and is used for assisting the prediction of wireless cellular traffic. Finally, the preliminary characteristic outputs are spliced and combined into a new one according to the specified dimensionalityTensor, and input to a dense connected convolutional network (DenseNet) which contains L layers in total, each layer implementing a complex function transformation, which is the same as the operation in cross-domain data feature learning, and contains batch regularization (BN), activation function (Relu), and convolution operation (Conv).
Further details are provided below.
(1) Data matrixing and correlation analysis of wireless cellular traffic.
Fig. 1 is a flow chart of data processing. The first step is data cleaning. In the experiment, wireless cellular flow data of short messages, telephones and the Internet are extracted, and for the missing traffic data in a certain time period in a certain area, the average traffic value of the surrounding area or the time period is used for filling. And step two, screening data. Since the recording time interval of the original data is 10 minutes, the recorded data value is mostly 0, which results in sparseness of data values. The data is divided by the number of hours, normalized by min-max, to speed up the training process. And thirdly, aligning data. In order to make the following data, the cleaned wireless cellular traffic data, cross-domain data and the milan city are divided into 100 × 100 grid areas for one-to-one correspondence. The wireless traffic data type is represented as k, where k belongs to { Sms, Call, Internet }, and taking Internet as an example, the wireless traffic of a certain city can be represented as a tensor of T dimension according to the wireless traffic data timestamp, where T is the total number of time intervals, T is {1,2, …, T }, X and Y respectively represent a coordinate point of the city, and a city regional traffic matrix D of the T-th time slot k,t Can be expressed as:
Figure BDA0002794519290000061
fig. 2 is a graph of wireless cellular traffic data after being preprocessed and cleaned, and (a), (b) and (c) specifically show the dynamic characteristics of three services in the time dimension under three different areas. The three differently shaped curves in fig. 2 represent three different locations, respectively. From fig. 2, we can conclude that:
the data is periodic. Wireless cellular traffic for different services exhibits the same periodicity, for example: fig. 2(a), (b), and (c) show that the flow curves of three different services in the area of bocconi university have the same change rule. In addition, similar periodicity also exists for wireless cellular traffic in different regions, such as: in fig. 1(a), under the Sms service, the change law of the traffic curve representing three different areas shows similarity.
And difference of regional data. There is a large difference in the amount of data of wireless cellular traffic in different areas, for example: the data volume of wireless cells in this area is not very different in one week because the navelli area is a night living area of milan, while the area of bocconi university is a suburban area of milan city, so the data volume of wireless cells is relatively small.
And differences of the business data. The amount of data in wireless cellular traffic varies between different services, such as the Internet, and the duration of the traffic peaks is shorter than in the other two services.
(2) And performing matrixing processing and correlation analysis on cross-domain data.
The invention considers three data sets which have larger influence on wireless service flow, namely Social information (Social), base station distribution (BS) and point of interest (POI). Since the three data types have small variation on the time axis, the invention treats them as static data sets, and after processing, maps the data to specific areas according to the coordinate information. Referring to equation (1), equation (2), a cross-domain data set D, can be obtained cross Is represented as follows:
Figure BDA0002794519290000062
for analyzing the correlation between different service flows and cross-domain data sets, a Pearson correlation coefficient is calculated, as shown in formula (3):
Figure BDA0002794519290000063
where cov (·) denotes covariance, and σ denotes standard deviation.
Fig. 3 is a correlation thermodynamic diagram between different services and cross-domain data sets (BS, POI, Social) obtained after Pearson correlation coefficient calculation, and the following conclusions can be drawn from fig. 3:
data correlation. The correlation of three different service data of Sms, Call and Internet is high, which shows that the model can be trained by introducing transfer learning between different service flow data.
Data similarity. The similarity between cross-domain data and wireless service flow is higher, so that the similarity can be regarded as a spatial characteristic constraint condition of the wireless service flow, and the service flow can be predicted more accurately.
And III, data correlation degree. The POI and the BS are relatively high in correlation in the cross-domain data set relative to three services (Sms, Call and Internet), and the Social is the second correlation, which shows that the influence of the POI and the BS cross-domain data set on the accurate prediction of the service flow is relatively larger than that of the Social.
Finally, we get several matrixes D with consistent size and corresponding information of each element t 、D meta 、D cross Synthesizing a multidimensional tensor T, wherein the data form is shown in fig. 4, each element (such as a black square in fig. 4) in the tensor represents the traffic information of the area coordinate corresponding to a certain moment of a certain service, the timestamp information and the information of the cross-domain data set of the area, and is convenient for the following model to use, and the cross-domain data set is added as
Figure BDA0002794519290000071
(3) And (5) performing time stamp feature analysis processing.
In order to fully utilize the characteristics of the timestamp to perform auxiliary prediction, 4 characteristics are extracted from the timestamp, the 4 characteristics are processed into a vector m, and the vector m is processed into a tensor T with the same size as the wireless cellular traffic data set and the cross-domain data set through a full connection layer. The extracted 4 features are shown in table 1:
table 14 characteristics of time stamps
Feature(s) Name(s) Value taking
1 Week 0,1,2…6
2 Hour(s) 0,1，…23
3 Working day 0,1
4 Weekend 0,1
For example: the four feature values extracted from 12, 14, 15 in 2013 are respectively: week is 5, hour is 14, workday is 0, weekend is 1.
(4) A multi-component spatio-temporal cross-domain neural network model based on an attention mechanism.
The invention provides a multi-component space-time cross-domain neural network model based on an attention mechanism, which comprises the following 4 parts as shown in figure 5:
the first part is neighbor cellular traffic D t h Day-cycle cellular traffic D t d Periodic cellular traffic DD t w And (6) modeling. The time sequence segment of the input wireless cellular traffic is shown as a graph. The three characteristic inputs are introduced into a conv-LSTM or conv-GRU structure with two layers, the weight of historical cellular traffic information which is more critical to the target moment is increased through an attention layer, the weight of other information is reduced, the purpose of filtering irrelevant information is achieved, and the efficiency and the accuracy of wireless cellular traffic prediction are further improved.
The second part models the display time characteristics, the input is a matrix D characterized by time stamps meta And putting the feature matrix into a two-layer fully-connected neural network for embedded learning.
The third part is modeling cross-domain data, and the input is a cross-domain data set D cross In which D is cross Distributing a set of three kinds of cross-domain data including BS, Social information Social and POI for a base station; finally, the cross-domain data set D is processed cross A two-layer convolutional neural network is introduced to process such data.
The fourth part is a feature fusion layer, the input is a new tensor formed by splicing the initial feature outputs according to the specified dimensionality, and the new tensor is input into a dense connection convolution network (Densenet), the network totally comprises L layers, each layer realizes a compound function transformation, and the compound functions are the same as the operations in cross-domain data feature learning and comprise batch regularization (BN), activation functions (Relu) and convolution operations (Conv).
The specific training process is as follows: as shown in fig. 6, the neighbor cellular traffic D associated with the predicted target time t t h Day-cycle cellular traffic D t d Periodic cellular traffic D t w The traffic matrix (2) can be seen as a plurality of single-channel pictures input to a single conv-LSTM or conv-GRU network with two layers, so that not only can a time sequence relation be obtained, but also the characteristics can be extracted like a convolution layer to extract spatial characteristics. As shown in FIG. 7, each element of the conv-LSTM network layer has a storage unit C for storing status information, and the unit C controls the deletion and addition of data information by three gates, respectively, the input gate i g Forgetting door f g And an output gate o g . Wherein, the input gate i g A forgetting gate f for selectively storing required data information g The redundant information is selectively 'forgotten', and the final hidden state is output from the output gate o g And controlling and deciding important data information required by output. The key operation of conv-LSTM is as follows:
Figure BDA0002794519290000081
wherein σ (·) is an activation function, a convolution operation, an Hadamard product operation, W (·) For the weight of training, H (·) Is an output gate o g Hidden state of (b) (·) For the bias of training, tanh (-) is a hyperbolic tangent function,
Figure BDA0002794519290000082
c τ 、
Figure BDA0002794519290000083
H τ are all three-dimensional tensors, the output obtained through conv-LSTM network layer
Figure BDA0002794519290000084
H is the number of feature maps.
The Gate Recovery Unit (GRU) is a kind of Recurrent Neural Network (RNN), and is also a variant of LSTM, compared with LSTM, the use of GRU can achieve a comparable effect, and compared with LSTM, training is easier, and training efficiency can be improved to a great extent, so the invention adopts the conv-GRU structure. As shown in FIG. 8, r t A reset gate for controlling the reset, called reset gate for short, the reset gate being used to control the extent to which status information of a previous moment is ignored, z t For controlling the update gate (update gate), referred to as update gate for short, the update gate is used to control the degree to which the state information of the previous time is brought into the current state, compared to the three gates of LSTM, the parameters are reduced, but the final performance is matched, relativelyLess parameters save more resources and have faster convergence speed.
Equation (5) includes resetting gate r t And an update gate z t The calculation process of (2). Wherein the content of the first and second substances,
Figure BDA0002794519290000085
mainly comprising the current input x t Data, is pertinently directed to
Figure BDA0002794519290000086
Adding to the current hidden state is equivalent to memorizing the state at the current moment. (1-z) t )⊙h t-1 Indicating selective "forgetting" of an originally hidden state. 1-z t Can be regarded as a forgetting gate (forget gate), forgetting h t-1 Some unimportant information in the dimension.
Figure BDA0002794519290000091
Indicating that the pair contains current node information
Figure BDA0002794519290000098
To perform selective memory, which can be regarded as a pair
Figure BDA0002794519290000092
Some information in the dimensions is selected. Therefore, one gate z t The two operations of forgetting and selecting memory can be simultaneously carried out, which is also the advantage of the GRU structure.
Figure BDA0002794519290000093
Wherein h is t-1 Is the hidden state of the previous node, which contains the relevant information of the previous node. x is the number of t Is the current input, σ (-) is the sigmoid activation function, W (·) For the trained weight, tanh (-), which is a hyperbolic tangent function, is a Hadamard product operation.
The attention mechanism is a solution proposed to imitate human attention, that is, a mechanism of aligning internal experience and external feeling to increase the fineness of observation of a partial region. For example, when a picture is processed by human vision, a target area needing important attention, namely an attention focus, is obtained by rapidly scanning a global image. More attention is then devoted to this area to get more detailed information about the objects that need attention and to suppress other unwanted information. For the wireless cellular traffic time sequence of the invention, for the output y at a certain moment, different attention is allocated on the hidden layer h corresponding to the input x, i.e. different weights are given to the characteristics with different importance degrees, and the characteristics are associated with the output, thereby achieving the purpose of information screening. The attention model structure is expanded as shown in fig. 9. The attention model is roughly divided into three layers: input layer, hidden layer, attention layer.
1) Firstly, a conv-GRU structure is utilized to obtain a hidden layer state (h) 1 ，h 2 ，...，h t )；
2) Calculating the influence e of each current input position on the current position i t The formula is as follows:
Figure BDA0002794519290000094
wherein v is a ，W a ，U a Is the weight value of the attention network, relu (·) is the activation function, T is the total number of time intervals, S is ·;
3) to e t Performing softmax normalization to obtain an attention weight distribution, wherein the formula is as follows:
Figure BDA0002794519290000095
wherein exp (·) is ·;
4) using alpha t Weighted summation is carried out to obtain a vector c t The formula is as follows:
Figure BDA0002794519290000096
D meta preliminary characteristic of (A) O meta The treatment process comprises the following steps:
Figure BDA0002794519290000097
o meta ＝Reshape(o meta ) (10)
where σ (-) is the activation function,
Figure BDA0002794519290000101
and
Figure BDA0002794519290000102
is the parameter to be learned. After being processed by a full-connection layer with two layers,
Figure BDA0002794519290000103
reshape has the function of converting O meta Matrix transformation into and t a uniform size tensor.
D cross Preliminary characteristics of (1) cross Can be expressed as:
o cross ＝f(W cross *D cross ) (11)
Figure BDA0002794519290000104
wherein the content of the first and second substances,
Figure BDA0002794519290000105
is a splicing operation, W cross For the parameter to be learned, f (-) is a complex function, including batch regularization, Relu activation function and convolution operation, for D cross Performing convolution and nonlinear transformation to obtain a product t 、O meta Are consistent in size so as to facilitate the next splicing process.
The resulting Frobenius norm calculation of the final output:
Figure BDA0002794519290000106
where θ is the set of all parameters of STC-N.
The following table 2 gives the algorithm for the att-MCSTCNet model training process. A training instance is first constructed from the original sequence (lines 1-5) and then trained by back-propagation and Adam (lines 6-11).
Table 2 att-MCSTCNet model training process algorithm
Figure BDA0002794519290000107
It is to be understood that the above description is not intended to limit the present invention, and the present invention is not limited to the above examples, and those skilled in the art may make modifications, alterations, additions or substitutions within the spirit and scope of the present invention.

Claims (7)

1. A network flow prediction method based on attention multi-component space-time cross-domain neural network model is characterized by comprising the following steps:
(1) performing Pearson correlation analysis and matrixing processing on three service data, namely short message service data, telephone service data and Internet service data;
(2) carrying out gridding division on different areas, and carrying out cluster classification on the areas;
(3) wireless cellular flow of the three services is divided into neighbor data, day cycle data and week cycle data according to the characteristics of the neighbor, day cycle and week cycle;
(4) performing correlation analysis and matrixing processing on the cross-domain data, and fusing; the method specifically comprises the following steps:
(4-1) performing Pearson correlation coefficient analysis on the three acquired cross-domain data sets of Social information Social, base station distribution BS and POI (point of interest) to obtain correlation, similarity and correlation characteristics of cross-domain data and different service data;
(4-2) processing the three kinds of cross-domain data into three matrices of 100 × 100;
(4-3) packaging the three kinds of cross-domain data subjected to matrixing into a tensor;
(5) extracting the characteristics of the timestamp of the wireless cellular traffic;
(6) fusing and inputting various cross-domain data and service data into the attention multi-component space-time cross-domain neural network model, and finally outputting a wireless cellular network flow prediction result; the method specifically comprises the following steps:
(6-1) cellular traffic D with close proximity t h Day-periodic cellular traffic D t d And periodic cellular traffic D t w Three parts of characteristic input data are led into a conv-GRU structure with two layers, and different weights are distributed to flow data with the adjacency, day periodicity and week periodicity through an attention layer;
in the step (6-1), the attention model is divided into an input layer, a hidden layer and an attention layer, and the specific process is as follows:
1) firstly, a conv-GRU structure is utilized to obtain a hidden layer state (h) 1 ，h 2 ，…，h t )；
2) Calculating the influence e of each current input position on the current position i t The formula is as follows:
Figure FDA0003726118910000011
wherein v is a 、W a 、U a Is the weight value of the attention network, relu (·) is the activation function, T is the total number of time intervals, S is the current input state;
3) to e t Performing softmax normalization to obtain attention weight distribution, wherein the formula is as follows:
Figure FDA0003726118910000012
wherein exp (·) is an exponential function with a natural constant e as the base;
4) using alpha t Weighted summation is carried out to obtain a vector c t The formula is as follows:
Figure FDA0003726118910000013
(6-2) putting the timestamp feature matrix into a two-layer fully-connected neural network for embedded learning, and performing timestamp feature modeling;
(6-3) fusing three kinds of cross-domain data including base station distribution BS, Social information Social and POI (point of interest) of the prediction area into a set D cross Introducing the two layers of convolution neural networks into a two-layer convolution neural network for spatial correlation modeling;
and (6-4) splicing the initial characteristic outputs of the steps into a new tensor according to the specified dimensionality, inputting the new tensor into a dense connection convolutional network DenseNet, and finally outputting a flow prediction result.
2. The method for predicting network traffic based on the attention multi-component spatiotemporal cross-domain neural network model according to claim 1, wherein the step (1) comprises the following steps:
(1-1) analyzing the correlation among three service data, namely short messages, telephone and the Internet, and analyzing the periodicity and difference of different service data and the difference of different regional data;
(1-2) processing three service data of short message, telephone and internet into three 100 x 100 matrixes with the same size, wherein each element in the matrixes represents a flow data value of a certain service.
3. The method for predicting network traffic based on the attention multi-component spatiotemporal cross-domain neural network model according to claim 2, wherein the step (2) comprises the following steps:
(2-1) dividing the predicted area into 100 x 100 grid areas, each grid corresponding to a data value of wireless cellular traffic of a certain service;
(2-2) according to the similarity and difference of wireless cellular traffic of different regions, the similar regions are gathered together to obtain three different classes, and then model training is carried out on the different classes.
4. The method for predicting network traffic based on the attention multi-component spatiotemporal cross-domain neural network model according to claim 1, wherein the step (3) comprises the following steps:
(3-1) extracting neighbor data of the wireless cellular traffic data, the neighbor data being a cellular traffic data sequence segment of a history time period directly adjacent to the predicted time t;
(3-2) extracting day cycle data of the wireless cellular traffic data, wherein the day cycle data is a cellular traffic data sequence segment which is the same as the predicted target time in n days before the predicted t time;
and (3-3) extracting cycle-per-cycle data of the wireless cellular traffic data, wherein the cycle-per-cycle data is a cellular traffic data sequence segment which has the same attribute and the same time as the n previous weeks of the predicted t time and the predicted target week.
5. The method for predicting network traffic based on the attention multi-component spatiotemporal cross-domain neural network model according to claim 1, wherein the step (5) comprises the following steps:
(5-1) extracting four characteristic attributes of week, hour, workday and weekend from the timestamp of the wireless cellular traffic and processing the four characteristic attributes into a vector;
(5-2) converting the processed vector into a 100 x 100 matrix.
6. The method for predicting network traffic based on attention multi-component spatiotemporal cross-domain neural network model according to claim 1, wherein in the step (6-1), the conv-GRU structure can be replaced by a conv-LSTM structure.
7. The method for predicting network traffic based on the attention multi-component space-time cross-domain neural network model according to claim 1, wherein in the step (6-4), the DenseNet network comprises L layers, each layer implements a complex function transformation, and the complex function comprises batch regularization BN, activation function Relu and convolution operation Conv.
CN202011326835.8A 2020-11-24 2020-11-24 Network flow prediction method based on attention multi-component space-time cross-domain neural network model Active CN112532439B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202011326835.8A CN112532439B (en) 2020-11-24 2020-11-24 Network flow prediction method based on attention multi-component space-time cross-domain neural network model

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202011326835.8A CN112532439B (en) 2020-11-24 2020-11-24 Network flow prediction method based on attention multi-component space-time cross-domain neural network model

Publications (2)

Publication Number Publication Date
CN112532439A CN112532439A (en) 2021-03-19
CN112532439B true CN112532439B (en) 2022-08-23

Family

ID=74994532

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202011326835.8A Active CN112532439B (en) 2020-11-24 2020-11-24 Network flow prediction method based on attention multi-component space-time cross-domain neural network model

Country Status (1)

Country Link
CN (1) CN112532439B (en)

Families Citing this family (12)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN113365298B (en) * 2021-04-22 2022-08-02 山东师范大学 MR network signal intensity prediction method and system based on high-order fuzzy cognitive map
CN113094357B (en) * 2021-04-23 2022-05-10 大连理工大学 Traffic missing data completion method based on space-time attention mechanism
CN113222239B (en) * 2021-05-08 2022-07-01 浙江科技学院 Short-time traffic flow prediction method based on CNN-LSTM-At neural network
CN115051925B (en) * 2021-06-18 2023-07-18 北京邮电大学 Time-space sequence prediction method based on transfer learning
CN113762485B (en) * 2021-09-29 2024-04-02 西安交通大学 Attention mechanism-based multi-dataset joint prediction method
CN114039871B (en) * 2021-10-25 2022-11-29 中山大学 Method, system, device and medium for cellular traffic prediction
CN114221876B (en) * 2021-11-16 2024-03-12 山东师范大学 Carrier network flow prediction method and system based on fuzzy control and multi-feature fusion
CN114048783B (en) * 2021-11-17 2024-04-16 东南大学 Cellular signal map construction method based on mobile group perception
CN114158085B (en) * 2021-11-18 2023-07-21 湖北工业大学 Mobile cellular flow high-efficiency prediction method based on space-time aggregation graph convolution network
CN115150840B (en) * 2022-05-18 2024-03-12 西安交通大学 Mobile network flow prediction method based on deep learning
CN115913996A (en) * 2022-12-07 2023-04-04 长春理工大学 Mobile flow prediction system and method based on regional space-time characteristics
CN116451118B (en) * 2023-04-19 2024-01-30 西安电子科技大学 Deep learning-based radar photoelectric outlier detection method

Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN110619430A (en) * 2019-09-03 2019-12-27 大连理工大学 Space-time attention mechanism method for traffic prediction
WO2020027619A1 (en) * 2018-08-02 2020-02-06 네오사피엔스 주식회사 Method, device, and computer readable storage medium for text-to-speech synthesis using machine learning on basis of sequential prosody feature

Family Cites Families (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN109919358B (en) * 2019-01-31 2021-03-02 中国科学院软件研究所 Real-time station flow prediction method based on neural network space-time attention mechanism
CN111314121A (en) * 2020-02-03 2020-06-19 支付宝(杭州)信息技术有限公司 Link abnormity detection method and device
CN111343650B (en) * 2020-02-14 2021-11-30 山东大学 Urban scale wireless service flow prediction method based on cross-domain data and loss resistance
CN111651504B (en) * 2020-06-03 2021-10-08 湖南大学 Multi-element time sequence multilayer space-time dependence modeling method based on deep learning
CN111918321B (en) * 2020-07-22 2022-08-05 湖北工业大学 Mobile flow prediction method based on space-time attention convolutional network

Patent Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2020027619A1 (en) * 2018-08-02 2020-02-06 네오사피엔스 주식회사 Method, device, and computer readable storage medium for text-to-speech synthesis using machine learning on basis of sequential prosody feature
CN110619430A (en) * 2019-09-03 2019-12-27 大连理工大学 Space-time attention mechanism method for traffic prediction

Also Published As

Publication number Publication date
CN112532439A (en) 2021-03-19

Similar Documents

Publication Publication Date Title
CN112532439B (en) Network flow prediction method based on attention multi-component space-time cross-domain neural network model
CN112291807B (en) Wireless cellular network traffic prediction method based on deep migration learning and cross-domain data fusion
Li et al. A deep learning method based on an attention mechanism for wireless network traffic prediction
Ke et al. Hexagon-based convolutional neural network for supply-demand forecasting of ride-sourcing services
Wang et al. Deep learning for real-time crime forecasting and its ternarization
CN109508360B (en) Geographical multivariate stream data space-time autocorrelation analysis method based on cellular automaton
Zeng et al. Traffic prediction of wireless cellular networks based on deep transfer learning and cross-domain data
CN111860951A (en) Rail transit passenger flow prediction method based on dynamic hypergraph convolutional network
CN110267292B (en) Cellular network flow prediction method based on three-dimensional convolutional neural network
CN109344992B (en) Modeling method for user control behavior habits of smart home integrating time-space factors
CN112910711B (en) Wireless service flow prediction method, device and medium based on self-attention convolutional network
Alizadeh et al. Network traffic forecasting based on fixed telecommunication data using deep learning
CN111339818A (en) Face multi-attribute recognition system
CN115565369B (en) Space-time hypergraph convolution traffic flow prediction method and system based on hypergraph
CN109255469A (en) Merge the Forecasting Flood method of stack self-encoding encoder and support vector regression
CN113688253B (en) Hierarchical perception temporal knowledge graph representation learning method
CN110889560A (en) Express delivery sequence prediction method with deep interpretability
Pramanik et al. Deep learning based resource availability prediction for local mobile crowd computing
Jin et al. Urban hotspot forecasting via automated spatio-temporal information fusion
CN117116048A (en) Knowledge-driven traffic prediction method based on knowledge representation model and graph neural network
Zhao et al. Developing a multiview spatiotemporal model based on deep graph neural networks to predict the travel demand by bus
Zhang et al. Adaptive spatio-temporal convolutional network for traffic prediction
ABBAS A survey of research into artificial neural networks for crime prediction
Liu et al. Acfm: A dynamic spatial-temporal network for traffic prediction
CN115456238A (en) Urban trip demand prediction method based on dynamic multi-view coupling graph convolution

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant