CN112532439A - Network flow prediction method based on attention multi-component space-time cross-domain neural network model - Google Patents

Network flow prediction method based on attention multi-component space-time cross-domain neural network model Download PDF

Info

Publication number
CN112532439A
CN112532439A CN202011326835.8A CN202011326835A CN112532439A CN 112532439 A CN112532439 A CN 112532439A CN 202011326835 A CN202011326835 A CN 202011326835A CN 112532439 A CN112532439 A CN 112532439A
Authority
CN
China
Prior art keywords
data
cross
domain
attention
neural network
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Granted
Application number
CN202011326835.8A
Other languages
Chinese (zh)
Other versions
CN112532439B (en
Inventor
陈赓
曾庆田
孙强
段华
邵睿
徐先杰
张旭
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Shandong University of Science and Technology
Original Assignee
Shandong University of Science and Technology
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Shandong University of Science and Technology filed Critical Shandong University of Science and Technology
Priority to CN202011326835.8A priority Critical patent/CN112532439B/en
Publication of CN112532439A publication Critical patent/CN112532439A/en
Application granted granted Critical
Publication of CN112532439B publication Critical patent/CN112532439B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L41/00—Arrangements for maintenance, administration or management of data switching networks, e.g. of packet switching networks
    • H04L41/14—Network analysis or design
    • H04L41/147—Network analysis or design for predicting network behaviour
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/045—Combinations of networks
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/048—Activation functions
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/049—Temporal neural networks, e.g. delay elements, oscillating neurons or pulsed inputs
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L41/00—Arrangements for maintenance, administration or management of data switching networks, e.g. of packet switching networks
    • H04L41/14—Network analysis or design
    • H04L41/145—Network analysis or design involving simulating, designing, planning or modelling of a network
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04W—WIRELESS COMMUNICATION NETWORKS
    • H04W24/00—Supervisory, monitoring or testing arrangements
    • H04W24/06—Testing, supervising or monitoring using simulated traffic
    • Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
    • Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
    • Y02D—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE
    • Y02D30/00—Reducing energy consumption in communication networks
    • Y02D30/70—Reducing energy consumption in communication networks in wireless communication networks

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Health & Medical Sciences (AREA)
  • Computing Systems (AREA)
  • Biomedical Technology (AREA)
  • Biophysics (AREA)
  • Computational Linguistics (AREA)
  • Data Mining & Analysis (AREA)
  • Evolutionary Computation (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Molecular Biology (AREA)
  • Artificial Intelligence (AREA)
  • General Engineering & Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Mathematical Physics (AREA)
  • Software Systems (AREA)
  • Health & Medical Sciences (AREA)
  • Computer Networks & Wireless Communication (AREA)
  • Signal Processing (AREA)
  • Management, Administration, Business Operations System, And Electronic Commerce (AREA)
  • Mobile Radio Communication Systems (AREA)

Abstract

The invention discloses a network flow prediction method based on an attention multi-component space-time cross-domain neural network model, belongs to the technical field of intelligent communication, and solves the problem of prediction of wireless cellular network flow. Firstly, dividing wireless cellular traffic data into neighboring data, daily cycle data and weekly cycle data according to the periodic characteristics of the wireless cellular traffic data; then modeling the neighboring data, the daily cycle data and the weekly cycle data through a conv-LSTM structure or a conv-GRU structure; different weights are adaptively distributed to the three kinds of characteristic data through an attention layer, so that the characteristic extraction capability of the three kinds of characteristic data is improved, and the characteristic information which generates interference on the prediction moment is restrained; and finally, combining the embedding of the timestamp characteristics, fusing various cross-domain data and carrying out flow prediction by using a common auxiliary model. The model can effectively utilize the periodic characteristics of wireless cellular flow data, saves model training time, greatly reduces workload, and further improves the prediction performance of network flow.

Description

Network flow prediction method based on attention multi-component space-time cross-domain neural network model
Technical Field
The invention belongs to the technical field of intelligent communication, and particularly relates to a network flow prediction method based on an attention multi-component space-time cross-domain neural network model.
Background
With the advent of the age of 5G/B5G, the number of mobile devices and internet of things has increased exponentially around the world, and the demand for wireless mobile data has increased rapidly. How to scientifically and reasonably allocate and optimize the existing cellular network resources, improve the utilization rate of the resources and reduce the energy consumption of the cellular base station is a problem to be considered and solved in the communication industry.
The method and the device can accurately predict the wireless cellular flow, and are beneficial to the development of base station site selection, urban area planning, regional flow prediction and other works. However, accurate prediction of wireless traffic flow is a very challenging problem, mainly for the following 3 reasons. First, the generation source of wireless communication network traffic is a user having mobility, and the mobility of the wireless user causes spatial dependency of traffic among a plurality of areas. In particular, the advent of new types of traffic has enabled people to reach from one end of a city to the other in a short period of time. This makes the spatial dependence of wireless traffic not only local, but rather has a large-scale global dependence. On the other hand, the wireless traffic flow also has a dependency in the time dimension, and the traffic value at a certain time has a high correlation with the traffic values at its neighboring time (short-term dependency) and the corresponding time of a certain day (periodicity). Second, the spatial constraint problem that multi-source cross-domain data creates on wireless traffic. The reasons for affecting the generation of wireless traffic in a certain area are diverse. When traffic prediction is performed, not only is a regular pattern implied by wireless service traffic mined only from the perspective of historical data, but also spatial constraint factors generated by other cross-domain and cross-source data on the traffic should be considered. Factors such as base station data of a certain area, point of interest information, social activity level of an area, etc. all affect the change of traffic. Therefore, how to efficiently fuse the multi-source cross-domain data which seems not to have a direct relation with the wireless service flow is a difficult problem to be solved at present. Third, how to achieve high accuracy of wireless cellular traffic prediction in consideration of space-time factors and in combination with cross-domain data is also a difficult problem.
Currently, the main model methods for wireless cellular traffic prediction are: (1) an integrated moving average autoregressive model (ARIMA); (2) an exponential smoothing method (ES); (3) linear regression method (LR); (4) support vector machine regression method (SVR); (5) multi-layer perceptron Method (MLP); (6) based on long-short time memory network method (LSTM); (7) convolutional neural network based approach (CNN); (7) an LSTM model; (8) a space-time convolutional network STDenseNet model; (9) an STNet model; (10) a STMNet model; (11) STCNet model, etc. These model methods for solving the wireless cellular network traffic prediction are considered from the aspects of space factors, time factors, space-time factors and the like.
Disclosure of Invention
The invention provides a multi-component space-time cross-domain neural network model method based on an attention mechanism, which is used for efficiently fusing multi-source cross-domain data, fully utilizing the time-space characteristics of the multi-source cross-domain data, introducing the attention mechanism and further improving the accuracy of wireless cellular flow prediction.
In order to achieve the purpose, the invention adopts the following technical scheme:
a network flow prediction method based on attention multi-component space-time cross-domain neural network model comprises the following steps:
(1) performing Pearson correlation analysis and matrixing processing on three service data, namely short message service data, telephone service data and Internet service data;
(2) gridding and dividing different areas, and clustering and classifying the areas;
(3) wireless cellular flow of the three services is divided into neighbor data, day cycle data and week cycle data according to the characteristics of the neighbor, day cycle and week cycle;
(4) performing correlation analysis and matrixing processing on the cross-domain data, and fusing;
(5) extracting the characteristics of the timestamp of the wireless cellular traffic;
(6) and fusing and inputting various cross-domain data and service data into the attention multi-component space-time cross-domain neural network model, and finally outputting a wireless cellular network flow prediction result.
Preferably, the step (1) specifically comprises the following steps:
(1-1) analyzing the correlation among three service data, namely short messages, telephone and the Internet, and analyzing the periodicity and difference of different service data and the difference of different regional data;
(1-2) processing three service data of short message, telephone and internet into three 100 x 100 matrixes with the same size, wherein each element in the matrixes represents a flow data value of a certain service.
Preferably, the step (2) specifically comprises the following steps:
(2-1) dividing the predicted area into 100 x 100 grid areas, each grid corresponding to a data value of wireless cellular traffic of a certain service;
(2-2) according to the similarity and difference of wireless cellular traffic of different regions, the similar regions are gathered together to obtain three different classes, and then model training is carried out on the different classes.
Preferably, the step (3) specifically comprises the following steps:
(3-1) extracting neighbor data of the wireless cellular traffic data, the neighbor data being a cellular traffic data sequence segment of a history time period directly adjacent to the predicted time t;
(3-2) extracting daily cycle data of the wireless cellular traffic data, the daily cycle data being a cellular traffic data sequence segment which is the same as the predicted target time in n days before the predicted t time;
and (3-3) extracting cycle-per-cycle data of the wireless cellular traffic data, wherein the cycle-per-cycle data is a cellular traffic data sequence segment which has the same attribute and the same time as the n previous weeks of the predicted t time and the predicted target week.
Preferably, the step (4) specifically includes the following steps:
and (4-1) carrying out Pearson correlation coefficient analysis on the three acquired cross-domain data sets of Social information Social, base station distribution BS and POI (point of interest) to obtain the correlation, similarity and correlation characteristics of the cross-domain data and different service data.
(4-2) processing the three kinds of cross-domain data into three matrices of 100 × 100;
and (4-3) packaging the three kinds of cross-domain data subjected to the matrixing processing into a tensor.
Preferably, the step (5) specifically comprises the following steps:
(5-1) extracting four characteristic attributes of week, hour, workday and weekend from the timestamp of the wireless cellular traffic and processing the four characteristic attributes into a vector;
(5-2) transforming the processed vector into a 100 x 100 matrix.
Preferably, the step (6) specifically comprises the following steps:
(6-1) cellular traffic D with close proximityt hDay-periodic cellular traffic Dt dAnd periodic cellular traffic Dt wThree parts of characteristic input data are led into a conv-GRU structure with two layers, pass through an attention layer and distribute different weights to the flow data with the adjacency, the daily periodicity and the weekly periodicity through the attention layer;
(6-2) putting the timestamp feature matrix into a two-layer fully-connected neural network for embedded learning, and performing timestamp feature modeling;
(6-3) fusing three kinds of cross-domain data including base station distribution BS, Social information Social and POI (point of interest) of the prediction area into a set DcrossIntroducing the two-layer convolutional neural network into a two-layer convolutional neural network for spatial correlation modeling;
and (6-4) splicing the initial characteristic outputs of the steps into a new tensor according to the specified dimensionality, inputting the new tensor into a dense connection convolutional network DenseNet, and finally outputting a flow prediction result.
Preferably, in the step (6-1), the attention model is divided into an input layer, a hidden layer and an attention layer, and the specific process is as follows:
1) firstly, a conv-GRU structure is utilized to obtain a hidden layer state (h)1，h2，…，ht)；
2) Calculating the influence e of each current input position on the current position itThe formula is as follows:
Figure BDA0002794519290000031
wherein v isa、Wa、UaIs the weight value of the attention network, relu (-) is the activation function, T is the total number of time intervals, S is the current input state;
3) to etPerforming softmax normalization to obtain an attention weight distribution, wherein the formula is as follows:
Figure BDA0002794519290000032
wherein exp (·) is an exponential function with a natural constant e as the base;
4) using alphatWeighted summation is carried out to obtain a vector ctThe formula is as follows:
Figure BDA0002794519290000033
preferably, in the step (6-1), the conv-GRU structure may be replaced with a conv-LSTM structure.
Preferably, in the step (6-4), the DenseNet network comprises L layers, each layer implements a complex function transformation, and the complex function comprises the batch regularization BN, the activation function Relu, and the convolution operation Conv.
The invention has the following beneficial technical effects:
the method of the invention starts from the characteristics of time and space factors, carries out fine grained division on wireless cellular traffic data, fully utilizes the time-space characteristics thereof, introduces an attention mechanism layer to adaptively distribute weight to the divided wireless cellular traffic, improves the characteristic extraction capability thereof, inhibits the characteristic information which generates interference to the prediction moment, and further improves the prediction efficiency and accuracy of the model to the wireless cellular traffic.
Drawings
FIG. 1 is a flow chart of data preprocessing according to an embodiment of the present invention;
fig. 2(a), (b), and (c) are dynamic characteristic graphs of three different services of Sms, Call, and Internet in different areas in the time dimension, respectively, according to an embodiment of the present invention;
FIG. 3 is a diagram illustrating Pearson correlation between wireless traffic and a cross-domain data set according to an embodiment of the present invention;
FIG. 4 is a diagram illustrating a tensor T according to an embodiment of the present invention;
FIG. 5 is a schematic diagram of a multi-component spatiotemporal cross-domain neural network model (att-MCSTCNet) based on an attention mechanism according to an embodiment of the present invention;
FIG. 6 is a diagram of a wireless cellular traffic time series segment as input by an embodiment of the present invention;
FIG. 7 is a schematic diagram of an LSTM structure according to an embodiment of the present invention;
FIG. 8 is a schematic structural diagram of a GRU according to an embodiment of the present invention;
fig. 9 is a schematic structural diagram of an attention according to an embodiment of the invention.
Detailed Description
The invention is described in further detail below with reference to the following figures and detailed description:
the method comprises the following specific steps:
(1) performing Pearson correlation analysis and matrixing processing on three data including short messages, telephone calls and the Internet: analyzing the correlation among three service data, namely short messages, telephone and the Internet, and analyzing the periodicity and the difference of different service data and the difference of different regional data; processing three kinds of service data of short messages, telephones and the Internet into three matrixes with the same size, namely 100 multiplied by 100; wherein each element in the matrix represents a traffic data value for a certain service.
(2) Gridding and dividing Milan city, and clustering and classifying the divided different areas: dividing a predicted region (Milan city) into 100 x 100 grid regions, enabling each grid to correspond to a data value of wireless cellular traffic of a certain service of the matrix, gathering similar regions together according to similarity and difference of the wireless cellular traffic of different regions to obtain three different classes, and then performing model training on the different classes.
(3) Firstly, extracting neighbor data of wireless cellular traffic data, namely a cellular traffic data sequence segment of a period of historical time directly adjacent to the predicted time t; then, extracting daily cycle data of wireless cellular traffic data, namely cellular traffic data sequence segments which are the same as the predicted target time in n days before the predicted t time; and finally, extracting cycle data of the wireless cellular traffic data, namely cellular traffic data sequence segments which have the same attribute and the same time as the target week of the previous n weeks of the predicted time t.
(4) Performing correlation analysis and matrixing processing on cross-domain data, and fusing: performing Pearson correlation coefficient analysis on the acquired three cross-domain data sets, namely, Social information (Social), base station distribution (BS) and point of interest (POI), to obtain the characteristics of correlation, similarity, correlation degree and the like of cross-domain data and different service data; then, the three kinds of cross-domain data are also processed into three matrixes of 100 multiplied by 100, and the sizes of the matrixes are kept consistent with those of the other services; and packaging the three kinds of cross-domain data subjected to the matrixing into a tensor.
(5) Performing feature extraction on the time stamp of the wireless cellular traffic: extracting four characteristic attributes of week, hour, working day and weekend from the timestamp of the wireless cellular flow and processing the four characteristic attributes into a vector; the processed vector is transformed into a 100 x 100 matrix, which is guaranteed to be the same size as the matrix.
(6) First, for neighbor cellular traffic Dt hDay-cycle cellular traffic Dt dAll the year roundPeriodic cellular traffic Dt wModeling, wherein three part characteristic inputs are led into a conv-LSTM or conv-GRU structure of two layers, and then the weight of historical cellular traffic information which is more critical to the target moment is increased through an attention layer, so that the weight of other information is reduced, the purpose of filtering irrelevant information is achieved, and the efficiency and the accuracy of wireless cellular traffic prediction are further improved. Secondly, performing feature modeling on the timestamp, putting a timestamp feature matrix into a two-layer fully-connected neural network for embedded learning, and then performing spatial correlation modeling on cross-domain data; wherein DcrossThree types of base station distribution BS, Social information Social and interest point distribution POI in the area are fused into a set of three types of cross-domain data, and a fused cross-domain data set DcrossA two-layer convolutional neural network is introduced to process the data and is used for assisting the prediction of wireless cellular traffic. And finally, splicing the initial characteristic outputs into a new tensor according to the specified dimensionality, and inputting the new tensor into a dense connection convolution network (DenseNet), wherein the network totally comprises L layers, each layer realizes a compound function transformation, and the compound function is the same as the operation in cross-domain data characteristic learning and comprises batch regularization (BN), an activation function (Relu) and convolution operation (Conv).
Further details are provided below.
(1) Data matrixing and correlation analysis of wireless cellular traffic.
Fig. 1 is a flowchart of data processing. The first step is data cleaning. In the experiment, wireless cellular flow data of short messages, telephones and the Internet are extracted, and for the missing traffic data in a certain time period in a certain area, the average traffic value of the surrounding area or the time period is used for filling. And step two, screening data. Since the recording time interval of the original data is 10 minutes, the recorded data value is mostly 0, which results in sparseness of data values. The data is divided by the number of hours, normalized by min-max, to speed up the training process. And thirdly, aligning data. In order to facilitate the formulation of data below, the cleaned wireless cellular traffic data, cross-domain data and the milan city are divided into 100 × 100 grid areas for one-to-one correspondence. Will be wirelessThe traffic data type is represented as k, where k is ∈ { Sms, Call, Internet }, and taking Internet as an example, the wireless traffic flow of a certain city can be represented as a tensor of T dimension according to the wireless traffic data timestamp, where T is the total number of time intervals, T ═ 1,2, …, T }, X, Y represent the coordinate point of the city, and the city area traffic matrix D of the T-th time slot represents the coordinate point of the city, respectivelyk,tCan be expressed as:
Figure BDA0002794519290000061
fig. 2 is a graph of wireless cellular traffic data after being preprocessed and cleaned, and (a), (b) and (c) specifically show the dynamic characteristics of three services in the time dimension under three different areas. The three differently shaped curves in fig. 2 represent three different locations, respectively. From fig. 2, we can conclude that:
the data is periodic. Wireless cellular traffic for different services exhibits the same periodicity, for example: fig. 2(a), (b), and (c) show that the flow curves of three different services in the area of bocconi university have the same change rule. In addition, similar periodicity also exists for wireless cellular traffic in different regions, such as: in fig. 1(a), under the Sms service, the change law of the traffic curve representing three different areas shows similarity.
And difference of regional data. There is a large difference in the amount of data of wireless cellular traffic in different areas, for example: the data volume of wireless cells in this area is not very different in one week because the navelli area is a night living area of milan, while the area of bocconi university is a suburban area of milan city, so the data volume of wireless cells is relatively small.
And differences of service data. The amount of data in wireless cellular traffic varies between different services, such as the Internet, and the duration of the traffic peaks is shorter than in the other two services.
(2) And performing matrixing processing and correlation analysis on cross-domain data.
The invention considers three data sets with large influence on wireless service flow, and divides the data sets intoThe others are Social information (Social information), base station distribution (BS), and point of interest (POI). Since the three data types have small variation on the time axis, the invention treats them as static data sets, and after processing, maps the data to specific areas according to the coordinate information. Referring to equation (1), equation (2), a cross-domain data set D, can be obtainedcrossIs represented as follows:
Figure BDA0002794519290000062
for analyzing the correlation between different service flows and cross-domain data sets, a Pearson correlation coefficient is calculated, as shown in formula (3):
Figure BDA0002794519290000063
wherein cov (·) represents covariance, and σ represents standard deviation.
Fig. 3 is a correlation thermodynamic diagram between different service and cross-domain data sets (BS, POI, Social) obtained by Pearson correlation coefficient calculation, and the following conclusions can be drawn from fig. 3:
data correlation. The correlation of three different service data of Sms, Call and Internet is high, which shows that the transfer learning can be introduced between the different service flow data to train the model.
And II, data similarity. The similarity between cross-domain data and wireless service flow is higher, so that the similarity can be regarded as a spatial characteristic constraint condition of the wireless service flow, and the service flow can be predicted more accurately.
And III, data correlation degree. The POI and the BS are relatively high in correlation in the cross-domain data set relative to three services (Sms, Call and Internet), and the Social is the second correlation, which shows that the influence of the POI and the BS cross-domain data set on the accurate prediction of the service flow is relatively larger than that of the Social.
Finally, we get several matrixes D with consistent size and corresponding information of each elementt、Dmeta、DcrossSynthesizing a multidimensional tensor T, wherein the data form is shown in fig. 4, each element (such as a black square in fig. 4) in the tensor represents the traffic information of the area coordinate corresponding to a certain moment of a certain service, the timestamp information and the information of the cross-domain data set of the area, and is convenient for the following model to use, and the cross-domain data set is added as
Figure BDA0002794519290000071
(3) And (5) performing time stamp feature analysis processing.
In order to fully utilize the characteristics of the timestamp to perform auxiliary prediction, 4 characteristics are extracted from the timestamp, the 4 characteristics are processed into a vector m, and the vector m is processed into a tensor T with the same size as the wireless cellular traffic data set and the cross-domain data set through a full connection layer. The extracted 4 features are shown in table 1:
table 14 characteristics of time stamps
Feature(s) Name (R) Value taking
1 Week 0,1,2…6
2 Hour(s) 0,1，…23
3 Working day 0,1
4 Weekend 0,1
For example: four feature values extracted from 12 months, 14 days and 15 days in 2013 are respectively as follows: week is 5, hour is 14, workday is 0, weekend is 1.
(4) A multi-component spatio-temporal cross-domain neural network model based on an attention mechanism.
The invention provides a multi-component space-time cross-domain neural network model based on an attention mechanism, which comprises the following 4 parts as shown in figure 5:
the first part is neighbor cellular traffic Dt hDay-cycle cellular traffic Dt dPeriodic cellular traffic DDt wAnd (6) modeling. The time sequence segment of the input wireless cellular traffic is shown as a graph. The three characteristic inputs are introduced into a conv-LSTM or conv-GRU structure of two layers, the weight of historical cellular traffic information which is more critical to the target moment is increased through an attention layer, the weight of other information is reduced, the purpose of filtering irrelevant information is achieved, and the efficiency and the accuracy of wireless cellular traffic prediction are further improved.
The second part models the display time characteristics, the input is a matrix D characterized by time stampsmetaAnd putting the feature matrix into a two-layer fully-connected neural network for embedded learning.
The third part is modeling cross-domain data, and the input is a cross-domain data set DcrossWherein D iscrossDistributing a set of three kinds of cross-domain data including BS, Social information Social and POI for a base station; finally, the cross-domain data set D is collectedcrossA two-layer convolutional neural network is introduced to process such data.
The fourth part is a feature fusion layer, the input is a new tensor formed by splicing the initial feature outputs according to the specified dimensionality, and the new tensor is input into a dense connection convolution network (Densenet), the network totally comprises L layers, each layer realizes a compound function transformation, and the compound functions are the same as the operations in cross-domain data feature learning and comprise batch regularization (BN), activation function (Relu) and convolution operation (Conv).
The specific training process is as follows: as shown in fig. 6, the neighbor cellular traffic D associated with the predicted target time tt hDay-cycle cellular traffic Dt dPeriodic cellular traffic Dt wThe traffic matrix (2) can be seen as a plurality of single-channel pictures input into a two-layer conv-LSTM or conv-GRU network, so that not only can a time sequence relation be obtained, but also the characteristics can be extracted like a convolutional layer to extract spatial characteristics. As shown in FIG. 7, each element of the conv-LSTM network layer has a storage unit C for storing status information, and the unit C controls the deletion and addition of data information by three gates, respectively, the input gate igForgetting door fgAnd an output gate og. Wherein, the input gate igA forgetting gate f for selectively storing required data informationgThe redundant information is selectively 'forgotten', and the final hidden state is output from the output gate ogAnd controlling and deciding important data information required by output. The key operation of conv-LSTM is as follows:
Figure BDA0002794519290000081
wherein σ (·) is an activation function, a convolution operation, an Hadamard product operation, W(·)For the weight of training, H(·)Is an output gate ogHidden state of (b)(·)For the bias of training, tanh (-) is a hyperbolic tangent function,
Figure BDA0002794519290000082
cτ、
Figure BDA0002794519290000083
Hτall being a three-dimensional tensor passing through conv-LSTM network layerThe obtained output
Figure BDA0002794519290000084
H is the number of feature maps.
The Gate Recovery Unit (GRU) is a kind of Recurrent Neural Network (RNN), and is also a variant of LSTM, compared with LSTM, the use of GRU can achieve a comparable effect, and compared with LSTM, training is easier, and training efficiency can be improved to a great extent, so the invention adopts the conv-GRU structure. As shown in FIG. 8, rtA reset gate for controlling the reset, called reset gate for short, the reset gate being used to control the extent to which status information of a previous moment is ignored, ztCompared with the three gates of the LSTM, the gate control system has the advantages that the parameters are reduced, the final performance is matched, relatively few parameters save resources more, and the convergence speed is higher.
Equation (5) includes resetting gate rtAnd an update gate ztThe calculation process of (2). Wherein the content of the first and second substances,
Figure BDA0002794519290000085
mainly containing the current input xtData, is pertinently directed to
Figure BDA0002794519290000086
Adding to the current hidden state is equivalent to memorizing the state at the current moment. (1-z)t)⊙ht-1Indicating selective "forgetting" of an originally hidden state. 1-ztCan be regarded as a forgetting gate (forget gate), forgetting ht-1Some unimportant information in the dimension.
Figure BDA0002794519290000091
Indicating that pairs contain current node information
Figure BDA0002794519290000098
To perform selective memory, which can be regarded as a pair
Figure BDA0002794519290000092
Some information in the dimensions is selected. Therefore, one gate ztThe two operations of forgetting and selecting memory can be simultaneously carried out, which is also the advantage of the GRU structure.
Figure BDA0002794519290000093
Wherein h ist-1Is the hidden state of the previous node, which contains the relevant information of the previous node. x is the number oftIs the current input, σ (-) is the sigmoid activation function, W(·)For the trained weight, tanh (-), which is a hyperbolic tangent function, is a Hadamard product operation.
The attention mechanism is a solution proposed to imitate human attention, that is, a mechanism of aligning internal experience and external feeling to increase the fineness of observation of a partial region. For example, when a picture is processed by human vision, a target area needing important attention, namely an attention focus, is obtained by rapidly scanning a global image. More attention is then devoted to this area to obtain more detailed information about the objects that need attention and to suppress other unwanted information. For the wireless cellular traffic time sequence of the invention, for the output y at a certain moment, different attention is allocated on the hidden layer h corresponding to the input x, i.e. different weights are given to the characteristics with different importance degrees, and the characteristics are associated with the output, thereby achieving the purpose of information screening. The attention model structure is expanded as shown in fig. 9. The attention model is roughly divided into three layers: input layer, hidden layer, attention layer.
1) Firstly, a conv-GRU structure is utilized to obtain a hidden layer state (h)1，h2，...，ht)；
2) Calculating the influence e of each current input position on the current position itThe formula is as follows:
Figure BDA0002794519290000094
wherein v isa，Wa，UaIs the weight value of the attention network, relu (·) is the activation function, T is the total number of time intervals, S is ·;
3) to etPerforming softmax normalization to obtain an attention weight distribution, wherein the formula is as follows:
Figure BDA0002794519290000095
wherein exp (·) is ·;
4) using alphatWeighted summation is carried out to obtain a vector ctThe formula is as follows:
Figure BDA0002794519290000096
Dmetapreliminary characteristic of (A) OmetaThe treatment process comprises the following steps:
Figure BDA0002794519290000097
ometa＝Reshape(ometa) (10)
where σ (-) is the activation function,
Figure BDA0002794519290000101
and
Figure BDA0002794519290000102
is the parameter to be learned. After being processed by a full-connection layer with two layers,
Figure BDA0002794519290000103
reshape has the function of converting OmetaMatrix transformation into andta uniform size tensor.
DcrossPreliminary characteristic of (A) OcrossCan be expressed as：
ocross＝f(Wcross*Dcross) (11)
Figure BDA0002794519290000104
Wherein the content of the first and second substances,
Figure BDA0002794519290000105
is a splicing operation, WcrossFor the parameter to be learned, f (-) is a complex function, including batch regularization, Relu activation function and convolution operation, for DcrossPerforming convolution and nonlinear transformation to process into Ot、OmetaAre consistent in size so as to facilitate the next splicing process.
The resulting Frobenius norm calculation of the final output:
Figure BDA0002794519290000106
where θ is the set of all parameters of STC-N.
The following table 2 gives the algorithm for the att-MCSTCNet model training process. A training instance is first constructed from the original sequence (lines 1-5) and then trained by back-propagation and Adam (lines 6-11).
TABLE 2 att-MCSTCNet model training process algorithm
Figure BDA0002794519290000107
It is to be understood that the above description is not intended to limit the present invention, and the present invention is not limited to the above examples, and those skilled in the art may make modifications, alterations, additions or substitutions within the spirit and scope of the present invention.

Claims (10)

1. A network flow prediction method based on attention multi-component space-time cross-domain neural network model is characterized by comprising the following steps:
(1) performing Pearson correlation analysis and matrixing processing on three service data, namely short message service data, telephone service data and Internet service data;
(2) gridding and dividing different areas, and clustering and classifying the areas;
(3) wireless cellular flow of the three services is divided into neighbor data, day cycle data and week cycle data according to the characteristics of the neighbor, day cycle and week cycle;
(4) performing correlation analysis and matrixing processing on the cross-domain data, and fusing;
(5) extracting the characteristics of the timestamp of the wireless cellular traffic;
(6) and fusing and inputting various cross-domain data and service data into the attention multi-component space-time cross-domain neural network model, and finally outputting a wireless cellular network flow prediction result.
2. The method for predicting network traffic based on the attention multi-component spatiotemporal cross-domain neural network model according to claim 1, wherein the step (1) comprises the following steps:
(1-1) analyzing the correlation among three service data, namely short messages, telephone and the Internet, and analyzing the periodicity and difference of different service data and the difference of different regional data;
(1-2) processing three service data of short message, telephone and internet into three 100 x 100 matrixes with the same size, wherein each element in the matrixes represents a flow data value of a certain service.
3. The method for predicting network traffic based on the attention multi-component spatiotemporal cross-domain neural network model according to claim 2, wherein the step (2) comprises the following steps:
(2-1) dividing the predicted area into 100 x 100 grid areas, each grid corresponding to a data value of wireless cellular traffic of a certain service;
(2-2) according to the similarity and difference of wireless cellular traffic of different regions, the similar regions are gathered together to obtain three different classes, and then model training is carried out on the different classes.
4. The method for predicting network traffic based on the attention multi-component spatiotemporal cross-domain neural network model according to claim 1, wherein the step (3) comprises the following steps:
(3-1) extracting neighbor data of the wireless cellular traffic data, the neighbor data being a cellular traffic data sequence segment of a history time period directly adjacent to the predicted time t;
(3-2) extracting daily cycle data of the wireless cellular traffic data, the daily cycle data being a cellular traffic data sequence segment which is the same as the predicted target time in n days before the predicted t time;
and (3-3) extracting cycle-per-cycle data of the wireless cellular traffic data, wherein the cycle-per-cycle data is a cellular traffic data sequence segment which has the same attribute and the same time as the n previous weeks of the predicted t time and the predicted target week.
5. The method for predicting network traffic based on the attention multi-component spatiotemporal cross-domain neural network model according to claim 1, wherein the step (4) comprises the following steps:
and (4-1) carrying out Pearson correlation coefficient analysis on the three acquired cross-domain data sets of Social information Social, base station distribution BS and POI (point of interest) to obtain the correlation, similarity and correlation characteristics of the cross-domain data and different service data.
(4-2) processing the three kinds of cross-domain data into three matrices of 100 × 100;
and (4-3) packaging the three kinds of cross-domain data subjected to the matrixing processing into a tensor.
6. The method for predicting network traffic based on the attention multi-component spatiotemporal cross-domain neural network model according to claim 1, wherein the step (5) comprises the following steps:
(5-1) extracting four characteristic attributes of week, hour, workday and weekend from the timestamp of the wireless cellular traffic and processing the four characteristic attributes into a vector;
(5-2) transforming the processed vector into a 100 x 100 matrix.
7. The method for predicting network traffic based on the attention multi-component spatiotemporal cross-domain neural network model according to claims 4-6, wherein the step (6) comprises the following steps:
(6-1) cellular traffic D with close proximityt hDay-periodic cellular traffic Dt dAnd periodic cellular traffic Dt wThree parts of characteristic input data are led into a conv-GRU structure with two layers, and different weights are distributed to flow data with the adjacency, day periodicity and week periodicity through an attention layer;
(6-2) putting the timestamp feature matrix into a two-layer fully-connected neural network for embedded learning, and performing timestamp feature modeling;
(6-3) fusing three kinds of cross-domain data including base station distribution BS, Social information Social and POI (point of interest) of the prediction area into a set DcrossIntroducing the two-layer convolutional neural network into a two-layer convolutional neural network for spatial correlation modeling;
and (6-4) splicing the initial characteristic outputs of the steps into a new tensor according to the specified dimensionality, inputting the new tensor into a dense connection convolutional network DenseNet, and finally outputting a flow prediction result.
8. The method for predicting network traffic based on the attention multi-component spatiotemporal cross-domain neural network model according to claim 7, wherein in the step (6-1), the attention model is divided into an input layer, a hidden layer and an attention layer by the following specific processes:
1) firstly, a conv-GRU structure is utilized to obtain a hidden layer state (h)1，h2，…，ht)；
2) Calculating the influence e of each current input position on the current position itThe formula is as follows:
Figure FDA0002794519280000021
wherein v isa、Wa、UaIs the weight value of the attention network, relu (-) is the activation function, T is the total number of time intervals, S is the current input state;
3) to etPerforming softmax normalization to obtain an attention weight distribution, wherein the formula is as follows:
Figure FDA0002794519280000031
wherein exp (·) is an exponential function with a natural constant e as the base;
4) using alphatWeighted summation is carried out to obtain a vector ctThe formula is as follows:
Figure FDA0002794519280000032
9. the method for predicting network traffic based on attention multi-component spatiotemporal cross-domain neural network model according to claim 7, wherein in the step (6-1), the conv-GRU structure can be replaced by a conv-LSTM structure.
10. The method for predicting network traffic based on the attention multi-component space-time cross-domain neural network model according to claim 7, wherein in the step (6-4), the DenseNet network comprises L layers, each layer implements a complex function transformation, and the complex function comprises batch regularization BN, activation function Relu and convolution operation Conv.
CN202011326835.8A 2020-11-24 2020-11-24 Network flow prediction method based on attention multi-component space-time cross-domain neural network model Active CN112532439B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202011326835.8A CN112532439B (en) 2020-11-24 2020-11-24 Network flow prediction method based on attention multi-component space-time cross-domain neural network model

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202011326835.8A CN112532439B (en) 2020-11-24 2020-11-24 Network flow prediction method based on attention multi-component space-time cross-domain neural network model

Publications (2)

Publication Number Publication Date
CN112532439A true CN112532439A (en) 2021-03-19
CN112532439B CN112532439B (en) 2022-08-23

Family

ID=74994532

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202011326835.8A Active CN112532439B (en) 2020-11-24 2020-11-24 Network flow prediction method based on attention multi-component space-time cross-domain neural network model

Country Status (1)

Country Link
CN (1) CN112532439B (en)

Cited By (12)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN113094357A (en) * 2021-04-23 2021-07-09 大连理工大学 Traffic missing data completion method based on space-time attention mechanism
CN113222239A (en) * 2021-05-08 2021-08-06 浙江科技学院 Short-time traffic flow prediction method based on CNN-LSTM-At neural network
CN113365298A (en) * 2021-04-22 2021-09-07 山东师范大学 MR network signal intensity prediction method and system based on high-order fuzzy cognitive map
CN113762485A (en) * 2021-09-29 2021-12-07 西安交通大学 Multi-dataset joint prediction method based on attention mechanism
CN114039871A (en) * 2021-10-25 2022-02-11 中山大学 Method, system, device and medium for cellular traffic prediction
CN114048783A (en) * 2021-11-17 2022-02-15 东南大学 Cellular signal map construction method based on mobile group perception
CN114158085A (en) * 2021-11-18 2022-03-08 湖北工业大学 Mobile cellular traffic high-efficiency prediction method based on spatio-temporal aggregation graph convolution network
CN114221876A (en) * 2021-11-16 2022-03-22 山东师范大学 Carrier network flow prediction method and system based on fuzzy control and multi-feature fusion
CN115051925A (en) * 2021-06-18 2022-09-13 北京邮电大学 Time-space sequence prediction method based on transfer learning
CN115150840A (en) * 2022-05-18 2022-10-04 西安交通大学 Mobile network flow prediction method based on deep learning
CN115913996A (en) * 2022-12-07 2023-04-04 长春理工大学 Mobile flow prediction system and method based on regional space-time characteristics
CN116451118A (en) * 2023-04-19 2023-07-18 西安电子科技大学 Deep learning-based radar photoelectric outlier detection method

Citations (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN109919358A (en) * 2019-01-31 2019-06-21 中国科学院软件研究所 A kind of real-time site traffic prediction technique based on neural network space-time attention mechanism
CN110619430A (en) * 2019-09-03 2019-12-27 大连理工大学 Space-time attention mechanism method for traffic prediction
WO2020027619A1 (en) * 2018-08-02 2020-02-06 네오사피엔스 주식회사 Method, device, and computer readable storage medium for text-to-speech synthesis using machine learning on basis of sequential prosody feature
CN111314121A (en) * 2020-02-03 2020-06-19 支付宝(杭州)信息技术有限公司 Link abnormity detection method and device
CN111343650A (en) * 2020-02-14 2020-06-26 山东大学 Urban scale wireless service flow prediction method based on cross-domain data and loss resistance
CN111651504A (en) * 2020-06-03 2020-09-11 湖南大学 Multi-element time sequence multilayer space-time dependence modeling method based on deep learning
CN111918321A (en) * 2020-07-22 2020-11-10 湖北工业大学 Mobile flow prediction method based on space-time attention convolutional network

Patent Citations (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2020027619A1 (en) * 2018-08-02 2020-02-06 네오사피엔스 주식회사 Method, device, and computer readable storage medium for text-to-speech synthesis using machine learning on basis of sequential prosody feature
CN109919358A (en) * 2019-01-31 2019-06-21 中国科学院软件研究所 A kind of real-time site traffic prediction technique based on neural network space-time attention mechanism
CN110619430A (en) * 2019-09-03 2019-12-27 大连理工大学 Space-time attention mechanism method for traffic prediction
CN111314121A (en) * 2020-02-03 2020-06-19 支付宝(杭州)信息技术有限公司 Link abnormity detection method and device
CN111343650A (en) * 2020-02-14 2020-06-26 山东大学 Urban scale wireless service flow prediction method based on cross-domain data and loss resistance
CN111651504A (en) * 2020-06-03 2020-09-11 湖南大学 Multi-element time sequence multilayer space-time dependence modeling method based on deep learning
CN111918321A (en) * 2020-07-22 2020-11-10 湖北工业大学 Mobile flow prediction method based on space-time attention convolutional network

Cited By (20)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN113365298A (en) * 2021-04-22 2021-09-07 山东师范大学 MR network signal intensity prediction method and system based on high-order fuzzy cognitive map
CN113094357A (en) * 2021-04-23 2021-07-09 大连理工大学 Traffic missing data completion method based on space-time attention mechanism
CN113222239B (en) * 2021-05-08 2022-07-01 浙江科技学院 Short-time traffic flow prediction method based on CNN-LSTM-At neural network
CN113222239A (en) * 2021-05-08 2021-08-06 浙江科技学院 Short-time traffic flow prediction method based on CNN-LSTM-At neural network
CN115051925B (en) * 2021-06-18 2023-07-18 北京邮电大学 Time-space sequence prediction method based on transfer learning
CN115051925A (en) * 2021-06-18 2022-09-13 北京邮电大学 Time-space sequence prediction method based on transfer learning
CN113762485A (en) * 2021-09-29 2021-12-07 西安交通大学 Multi-dataset joint prediction method based on attention mechanism
CN113762485B (en) * 2021-09-29 2024-04-02 西安交通大学 Attention mechanism-based multi-dataset joint prediction method
CN114039871A (en) * 2021-10-25 2022-02-11 中山大学 Method, system, device and medium for cellular traffic prediction
CN114221876A (en) * 2021-11-16 2022-03-22 山东师范大学 Carrier network flow prediction method and system based on fuzzy control and multi-feature fusion
CN114221876B (en) * 2021-11-16 2024-03-12 山东师范大学 Carrier network flow prediction method and system based on fuzzy control and multi-feature fusion
CN114048783A (en) * 2021-11-17 2022-02-15 东南大学 Cellular signal map construction method based on mobile group perception
CN114048783B (en) * 2021-11-17 2024-04-16 东南大学 Cellular signal map construction method based on mobile group perception
CN114158085A (en) * 2021-11-18 2022-03-08 湖北工业大学 Mobile cellular traffic high-efficiency prediction method based on spatio-temporal aggregation graph convolution network
CN114158085B (en) * 2021-11-18 2023-07-21 湖北工业大学 Mobile cellular flow high-efficiency prediction method based on space-time aggregation graph convolution network
CN115150840A (en) * 2022-05-18 2022-10-04 西安交通大学 Mobile network flow prediction method based on deep learning
CN115150840B (en) * 2022-05-18 2024-03-12 西安交通大学 Mobile network flow prediction method based on deep learning
CN115913996A (en) * 2022-12-07 2023-04-04 长春理工大学 Mobile flow prediction system and method based on regional space-time characteristics
CN116451118B (en) * 2023-04-19 2024-01-30 西安电子科技大学 Deep learning-based radar photoelectric outlier detection method
CN116451118A (en) * 2023-04-19 2023-07-18 西安电子科技大学 Deep learning-based radar photoelectric outlier detection method

Also Published As

Publication number Publication date
CN112532439B (en) 2022-08-23

Similar Documents

Publication Publication Date Title
CN112532439B (en) Network flow prediction method based on attention multi-component space-time cross-domain neural network model
CN112291807B (en) Wireless cellular network traffic prediction method based on deep migration learning and cross-domain data fusion
Ke et al. Hexagon-based convolutional neural network for supply-demand forecasting of ride-sourcing services
Li et al. A deep learning method based on an attention mechanism for wireless network traffic prediction
CN109508360B (en) Geographical multivariate stream data space-time autocorrelation analysis method based on cellular automaton
Fonseca et al. Artificial neural networks for job shop simulation
CN107423442A (en) Method and system, storage medium and computer equipment are recommended in application based on user's portrait behavioural analysis
CN110267292B (en) Cellular network flow prediction method based on three-dimensional convolutional neural network
CN109344992B (en) Modeling method for user control behavior habits of smart home integrating time-space factors
CN112910711B (en) Wireless service flow prediction method, device and medium based on self-attention convolutional network
Zhang et al. Deep-AIR: A hybrid CNN-LSTM framework for fine-grained air pollution estimation and forecast in metropolitan cities
WO2019214455A1 (en) Data sequence prediction method and computing apparatus
CN111339818A (en) Face multi-attribute recognition system
CN111861028A (en) Method for predicting crime number based on spatio-temporal data fusion
CN115565369A (en) Hypergraph-based time-space hypergraph convolution traffic flow prediction method and system
Wang et al. On prediction of traffic flows in smart cities: a multitask deep learning based approach
Pramanik et al. Deep learning based resource availability prediction for local mobile crowd computing
CN115810149A (en) High-resolution remote sensing image building extraction method based on superpixel and image convolution
Gammelli et al. Recurrent flow networks: A recurrent latent variable model for density estimation of urban mobility
Alghamdi et al. Multi-stage deep probabilistic prediction for travel demand
CN117764227A (en) Customer loss prediction device for gas station
CN116957260A (en) Intelligent allocation method for carrier vehicles based on vehicle demand prediction
Zhao et al. Developing a multiview spatiotemporal model based on deep graph neural networks to predict the travel demand by bus
CN115456238A (en) Urban trip demand prediction method based on dynamic multi-view coupling graph convolution
CN114492886A (en) Network taxi appointment demand prediction method and system considering departure place and destination

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant