CN111464354A - Fine-grained network flow calculation method and device and storage medium - Google Patents

Fine-grained network flow calculation method and device and storage medium Download PDF

Info

Publication number
CN111464354A
CN111464354A CN202010247708.2A CN202010247708A CN111464354A CN 111464354 A CN111464354 A CN 111464354A CN 202010247708 A CN202010247708 A CN 202010247708A CN 111464354 A CN111464354 A CN 111464354A
Authority
CN
China
Prior art keywords
network flow
fine
network traffic
grained
network
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Granted
Application number
CN202010247708.2A
Other languages
Chinese (zh)
Other versions
CN111464354B (en
Inventor
刘川
张刚
陶静
刘世栋
李伯中
马睿
黄在朝
卜宪德
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
State Grid Corp of China SGCC
State Grid Information and Telecommunication Co Ltd
State Grid Zhejiang Electric Power Co Ltd
Global Energy Interconnection Research Institute
Original Assignee
State Grid Corp of China SGCC
State Grid Information and Telecommunication Co Ltd
State Grid Zhejiang Electric Power Co Ltd
Global Energy Interconnection Research Institute
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by State Grid Corp of China SGCC, State Grid Information and Telecommunication Co Ltd, State Grid Zhejiang Electric Power Co Ltd, Global Energy Interconnection Research Institute filed Critical State Grid Corp of China SGCC
Priority to CN202010247708.2A priority Critical patent/CN111464354B/en
Publication of CN111464354A publication Critical patent/CN111464354A/en
Application granted granted Critical
Publication of CN111464354B publication Critical patent/CN111464354B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L41/00—Arrangements for maintenance, administration or management of data switching networks, e.g. of packet switching networks
    • H04L41/14—Network analysis or design
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L41/00—Arrangements for maintenance, administration or management of data switching networks, e.g. of packet switching networks
    • H04L41/14—Network analysis or design
    • H04L41/142—Network analysis or design using statistical or mathematical methods
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L41/00—Arrangements for maintenance, administration or management of data switching networks, e.g. of packet switching networks
    • H04L41/14—Network analysis or design
    • H04L41/145—Network analysis or design involving simulating, designing, planning or modelling of a network
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L43/00—Arrangements for monitoring or testing data switching networks
    • H04L43/50—Testing arrangements

Landscapes

  • Engineering & Computer Science (AREA)
  • Computer Networks & Wireless Communication (AREA)
  • Signal Processing (AREA)
  • Physics & Mathematics (AREA)
  • Algebra (AREA)
  • General Physics & Mathematics (AREA)
  • Mathematical Analysis (AREA)
  • Mathematical Optimization (AREA)
  • Mathematical Physics (AREA)
  • Probability & Statistics with Applications (AREA)
  • Pure & Applied Mathematics (AREA)
  • Data Exchanges In Wide-Area Networks (AREA)

Abstract

The invention discloses a fine-grained network flow calculation method, a fine-grained network flow calculation device and a storage medium, wherein the method comprises the following steps: acquiring end-to-end network flow; sampling the network flow to obtain discrete network flow; decomposing the discrete network flow according to an empirical mode decomposition algorithm, and calculating to obtain the decomposed network flow; and recovering the decomposed network flow according to a cubic interpolation algorithm, and calculating to obtain the fine-grained network flow. The fine-grained network traffic calculation method provided by the embodiment of the invention researches how to estimate and recover an end-to-end network traffic matrix in a fine-grained size from sampled traffic tracking. The calculation method can reconstruct the network flow with fine granularity by utilizing an EMD method and cubic interpolation. The fine-grained network traffic calculation method provided by the embodiment of the invention can calculate and obtain accurate end-to-end network traffic, and solves the technical problem that accurate measurement and analysis of the network traffic cannot be realized in the prior art.

Description

Fine-grained network flow calculation method and device and storage medium
Technical Field
The invention relates to the technical field of network flow measurement, in particular to a fine-grained network flow calculation method, a fine-grained network flow calculation device and a storage medium.
Background
With the wide application of new-generation information technologies, applications such as smart cities, internet of things and Software Defined Networks (SDNs) are growing explosively. The high-speed backbone networks that support these applications carry a tremendous network traffic load. Due to the continuous enlargement of the scale of the backbone network, the speed of the backbone network is continuously improved, so that the measurement technology of the network flow brings huge challenges.
In order to measure the performance of a network, a network operator needs to collect a large amount of traffic data from network test nodes. However, only the OC48 link can collect up to 600GB of traffic per hour, while expending significant resources to store, transmit and process traffic data. Therefore, in next generation networks such as SDN, large scale and high speed sampling technology has become one of the main options for measuring and monitoring communication networks. These techniques significantly reduce the amount of measurement data and also avoid the additional overhead of network measurement, and thus have attracted much attention.
However, the sampling technique of network traffic can only obtain incomplete measurement data, which may affect the effect of network monitoring, correct analysis of network management and performance evaluation, and may result in incorrect final network management decision. Therefore, how to realize accurate measurement and analysis of network traffic is a problem to be solved urgently at present.
Disclosure of Invention
In view of this, embodiments of the present invention provide a fine-grained network traffic calculation method, apparatus, and storage medium, so as to solve the problem in the prior art that accurate measurement and analysis of network traffic cannot be achieved.
The technical scheme provided by the invention is as follows:
a first aspect of an embodiment of the present invention provides a fine-grained network traffic calculation method, where the calculation method includes: acquiring end-to-end network flow; sampling the network flow to obtain discrete network flow; decomposing the discrete network flow according to an empirical mode decomposition algorithm, and calculating to obtain decomposed network flow; and recovering the decomposed network flow according to a cubic interpolation algorithm, and calculating to obtain the fine-grained network flow.
Further, decomposing the discrete network traffic according to an empirical mode decomposition algorithm, and calculating to obtain decomposed network traffic, including: decomposing the discrete network flow into the sum of component components and residual errors according to an empirical mode decomposition algorithm; calculating the correlation coefficient of the component and the original network flow according to the decomposition times; determining effective component components according to the correlation coefficients; and determining the decomposed network flow according to the effective component and the residual error.
Further, recovering the decomposed network traffic according to a cubic interpolation algorithm, and calculating to obtain fine-grained network traffic, including: calculating according to the time interval of network flow sampling to obtain a coefficient of a cubic interpolation function; reconstructing the decomposed network flow according to the coefficient of the cubic interpolation function to obtain a cubic interpolation result of the network flow component; and calculating the result of the cubic interpolation of the network flow component according to a weighted average algorithm to obtain the network flow with fine granularity.
Further, the fine-grained network traffic calculation method further includes: and carrying out error calculation on the fine-grained network flow according to a relative error algorithm.
A second aspect of the embodiments of the present invention provides a fine-grained network traffic calculation apparatus, where the apparatus includes: the acquisition module is used for acquiring end-to-end network flow; the sampling module is used for sampling the network flow to obtain discrete network flow; the decomposition module is used for decomposing the discrete network flow according to an empirical mode decomposition algorithm and calculating the decomposed network flow; and the recovery module is used for recovering the decomposed network flow according to a cubic interpolation algorithm and calculating to obtain the fine-grained network flow.
Further, the decomposition module comprises: the decomposition submodule is used for decomposing the discrete network flow into the sum of component components and residual errors according to an empirical mode decomposition algorithm; the correlation coefficient calculation module is used for calculating the correlation coefficients of the component components and the original network flow according to the decomposition times; the effective component calculation module is used for determining the effective component according to the correlation coefficient; and the decomposition calculation module is used for determining the decomposed network flow according to the effective component components and the residual errors.
Further, the recovery module includes: the interpolation coefficient calculation module is used for calculating the coefficient of the cubic interpolation function according to the time interval of network flow sampling; the interpolation calculation module is used for reconstructing the decomposed network flow according to the coefficient of the cubic interpolation function to obtain a cubic interpolation result of the network flow component; and the weighted calculation module is used for calculating the result of the cubic interpolation of the network flow component according to a weighted average algorithm to obtain the network flow with fine granularity.
Further, the fine-grained network traffic calculation apparatus further includes: and the error calculation module is used for performing error calculation on the fine-grained network flow according to a relative error algorithm.
A third aspect of the embodiments of the present invention provides a computer-readable storage medium, where computer instructions are stored, and the computer instructions are configured to cause a computer to execute the fine-grained network traffic calculation method according to any one of the first aspect and the first aspect of the embodiments of the present invention.
A fourth aspect of an embodiment of the present invention provides an electronic device, including: the fine-grained network traffic calculation method comprises a memory and a processor, wherein the memory and the processor are connected in communication with each other, the memory stores computer instructions, and the processor executes the computer instructions to execute the fine-grained network traffic calculation method according to the first aspect and any one of the first aspect of the embodiments of the present invention.
The technical scheme provided by the invention has the following effects:
the fine-grained network traffic calculation method, the fine-grained network traffic calculation device and the storage medium provided by the embodiment of the invention research how to estimate and recover an end-to-end network traffic matrix in a fine-grained size from sampled traffic tracking. The calculation method can reconstruct the network flow with fine granularity by utilizing an EMD method and cubic interpolation. Firstly, the fractal and self-similar characteristics of end-to-end network flow are utilized, an EMD method is used for decomposing the network flow, and the decomposed components can well reflect the relevant characteristics of the network flow; and then recover them at a finer time granularity using a cubic interpolation method. Therefore, the fine-grained network traffic calculation method provided by the embodiment of the invention can calculate and obtain accurate end-to-end network traffic, and solves the technical problem that accurate measurement and analysis of the network traffic cannot be realized in the prior art.
The fine-grained network traffic calculation method, the fine-grained network traffic calculation device and the storage medium provided by the embodiment of the invention adopt a weighted geometric mean algorithm to calculate the weight of each network traffic component, and obtain the fine-grained network traffic by using a weighted summation method so as to improve the reconstruction accuracy of the end-to-end network traffic. The fine-grained network traffic calculation method provided by the embodiment of the invention has the best reconstruction performance, can extract an accurate end-to-end network traffic matrix, and has profound influence on network planning, network optimization and network scale in SDN application.
Drawings
In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the embodiments or the prior art will be briefly described below, and it is obvious that the drawings in the following description are some embodiments of the present invention, and other drawings can be obtained by those skilled in the art without creative efforts.
FIG. 1 is a flow diagram of a fine-grained network traffic calculation method according to an embodiment of the invention;
FIG. 2 is a block diagram of some direct measurement of end-to-end network traffic according to an embodiment of the present invention;
FIG. 3 is a block diagram of a sampled end-to-end network traffic according to an embodiment of the invention;
FIG. 4 is a schematic diagram of prediction, filling and reconstruction from a pair using the EMD method according to an embodiment of the present invention;
fig. 5 is a result of the EMD method decomposing network traffic according to an embodiment of the present invention;
FIG. 6 is a flow diagram of a fine-grained network traffic calculation method according to another embodiment of the invention;
FIG. 7 is a flow diagram of a fine-grained network traffic calculation method according to another embodiment of the invention;
FIG. 8 is a flow diagram of a fine-grained network traffic calculation method according to another embodiment of the invention;
fig. 9 is a comparative graph of network traffic results predicted using the EMD-SP method, the SRSVD method, and the ARMA method according to an embodiment of the present invention;
FIG. 10 is a graphical comparison of probability distribution functions of relative error of network traffic in accordance with an embodiment of the present invention;
FIG. 11 is a box plot comparison of network traffic relative error according to an embodiment of the present invention;
FIG. 12 is a mean and variance comparison of network traffic relative errors according to an embodiment of the present invention;
fig. 13 is a block diagram of a fine-grained network traffic computation device according to an embodiment of the invention;
fig. 14 is a block diagram of a fine-grained network traffic computation apparatus according to another embodiment of the present invention;
fig. 15 is a block diagram of a fine-grained network traffic computation apparatus according to another embodiment of the present invention;
fig. 16 is a block diagram of a fine-grained network traffic computation apparatus according to another embodiment of the present invention;
FIG. 17 is a schematic structural diagram of a computer-readable storage medium provided in accordance with an embodiment of the present invention;
fig. 18 is a schematic structural diagram of an electronic device provided in an embodiment of the present invention.
Detailed Description
In order to make the objects, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are some, but not all, embodiments of the present invention. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
An embodiment of the present invention provides a fine-grained network traffic calculation method, as shown in fig. 1, the calculation method includes the following steps:
step S101: acquiring end-to-end network flow; the obtained end-to-end network traffic may be as shown in fig. 2. In particular, a system model of end-to-end (source-to-destination or origin-destination) network traffic may be established to model the network traffic. The traffic y (t) in the link, i.e. end-to-end, may be represented by a traffic matrix, and may be specifically represented by formula (1).
y (t) ax (t) formula (1)
Where y (t) represents the traffic of all links end-to-end. y isi(t) (i ═ 1, 2.., v) may be used to represent traffic in link i. v may be used to represent the total number of links in a given network. x (t) ═ x1(t),x2(t),...,xu(t)) is all end-to-end network traffic in a given network (i.e., traffic matrix); u is the total number of origin-target node pairs; x is the number ofj(t) is the flow of the origin-destination j stream. A ═ aji)u×vA routing matrix is represented that describes routing configuration information in a given network.
Step S102: sampling the network flow to obtain discrete network flow; as shown in fig. 3, is the sampled end-to-end network traffic. Specifically, because the existing network traffic is large and cannot be measured, the acquired network traffic can be sampled and then measured. Wherein, at a given measurement duration [0, T]The measured value of the inner end-to-end network traffic is xg(t), therefore, the end-to-end network traffic sample sequence can be expressed as equation (2).
xg(t)＝{(t,xg(t))|t∈[0,T],xg(t)∈R+Equation (2)
Wherein R is+Representing a non-negative real number field.
For sampled data traffic is a sequence discretized in the time dimension, i.e. time t is discretized. Then, the formula (2) can be expressed by the formula (3).
xg(n)＝{(n,xg(n))|n∈Z+,xg(n)∈R+Formula (3)
Where N ∈ {0,1, …, N } represents a time series of samples and N ∈ Z+，Z+Represents a non-negative integer; n is the total number of time slots of the sample sequence during the measurement, R+Representing a non-negative real number field.
Step S103: decomposing the discrete network flow according to an empirical mode decomposition algorithm, and calculating to obtain the decomposed network flow; specifically, by using the fractal and self-similar characteristics of the end-to-end network traffic, the network traffic can be decomposed by using an EMD (empirical mode decomposition) algorithm, and the decomposed components can well reflect the relevant characteristics of the network traffic. Where fig. 4 shows a schematic diagram of prediction, filling and reconstruction using EMD, where the black lines (Actual) represent the true measurement of network traffic and the grey lines represent the residual components. Fig. 5 shows the result of decomposing the network traffic by using the EMD method, and after multiple decompositions, the residual component fluctuation of the network traffic becomes smaller and smaller, which shows that the overall stability of the network traffic is stronger and stronger.
Step S104: and recovering the decomposed network flow according to a cubic interpolation algorithm, and calculating to obtain the fine-grained network flow. Specifically, the method of cubic interpolation can be used for filling and smoothing components of network traffic. Meanwhile, the interpolation result of the network flow is interpolated again by utilizing a cubic interpolation method, and the network flow with fine granularity can be obtained through calculation.
The fine-grained network traffic calculation method provided by the embodiment of the invention researches how to estimate and recover an end-to-end network traffic matrix in a fine-grained size from sampled traffic tracking. The calculation method can reconstruct the network flow with fine granularity by utilizing an EMD method and cubic interpolation. Firstly, the fractal and self-similar characteristics of end-to-end network flow are utilized, an EMD method is used for decomposing the network flow, and the decomposed components can well reflect the relevant characteristics of the network flow; and then recover them at a finer time granularity using a cubic interpolation method. Therefore, the fine-grained network traffic calculation method provided by the embodiment of the invention can calculate and obtain accurate end-to-end network traffic, and solves the technical problem that accurate measurement and analysis of the network traffic cannot be realized in the prior art
In an embodiment, as shown in fig. 6, the step S103 decomposes the discrete network traffic according to an empirical mode decomposition algorithm, and calculates to obtain decomposed network traffic, including the following steps:
step S301: decomposing the discrete network flow into the sum of component components and residual errors according to an empirical mode decomposition algorithm; specifically, the network traffic may be decomposed by using an EMD method, and the time series of the network traffic is preliminarily expressed as a sum of the component components and the residual, which may be expressed by formula (4).
Figure BDA0002433741950000081
Step S302: and calculating the correlation coefficient of the component and the original network flow according to the decomposition times. In particular, since not all IMFs (Intrinsic Mode functions) are significant components. In general, a wrong mode component is likely to occur in a low frequency portion, and a noise component is likely to occur in a high frequency portion. Therefore, it is necessary to select an effective network traffic component between the decomposed network traffic and the original network traffic according to the correlation coefficient. The correlation coefficient indicates the relationship between the components of the network traffic and the original traffic, and the larger the correlation coefficient is, the higher the component effectiveness of the traffic is.
In one embodiment, a threshold value of the correlation coefficient may be set to λ, and if the correlation coefficient between each decomposed component and the original network traffic data is greater than the threshold value λ, the corresponding IMF is considered as a valid component, otherwise, the corresponding IMF is considered as an invalid network traffic component.
A specific calculation method of the correlation coefficient between the IMF component c and the raw network traffic data x (t) of the SDN network may be represented by formula (5).
Figure BDA0002433741950000082
Where m represents the number of decompositions in the time series x (t).
Step S303: determining effective component components according to the correlation coefficients; specifically, the effective network traffic can be determined by decomposing a plurality of component components of the obtained network traffic through the network traffic, and comparing the correlation coefficient thereof with the original sequence with a threshold value.
Step S304: and determining the decomposed network flow according to the effective component and the residual error. Specifically, the network traffic may be reconstructed using the effective component and the residual error, which may be represented by equation (6).
Figure BDA0002433741950000091
Where n is the number of IMFs extracted, rn(t) are the final residuals, which together reflect the main trends x (t), ciAre mutually orthogonal and have an almost zero mean value of the extracted IMF.
In an embodiment, as shown in fig. 7, the step S104 recovers the decomposed network traffic according to a cubic interpolation algorithm, and calculates to obtain a fine-grained network traffic, including the following steps:
step S401: calculating according to the time interval of network flow sampling to obtain a coefficient of a cubic interpolation function; specifically, the cubic interpolation method is a polynomial interpolation method, and includes dividing an interpolation interval into q small intervals equally, then constructing an interpolation function, and calculating interpolation at each division point by using the interpolation function. Without loss of generality, for the kth sampling interval [ k, k +1] of network traffic, q-1 points of network traffic can be interpolated, namely:
Figure BDA0002433741950000092
corresponding to the interval [ k, k +1]]Cubic interpolation function ck(t) satisfies the following equation:
Figure BDA0002433741950000093
wherein the content of the first and second substances,
Figure BDA0002433741950000094
the representation corresponds to a point in time
Figure BDA0002433741950000095
Is measured. F (-) does not exceed each seedInterval(s)
Figure BDA0002433741950000096
Of (4) is a cubic polynomial. Then, for the kth sampling interval [ k, k +1] of network traffic]Cubic interpolation function sk(t) can be represented by formula (9).
Figure BDA0002433741950000097
Wherein the content of the first and second substances,
Figure BDA0002433741950000101
i is 0,1, …, q, and
Figure BDA0002433741950000102
and
Figure BDA0002433741950000103
are coefficients of a cubic interpolation function. Function sk(t) at the sampling point
Figure BDA0002433741950000104
Has a continuous second derivative, and is therefore based on the boundary value points
Figure BDA0002433741950000105
And
Figure BDA0002433741950000106
the value of (2) and the coefficient of cubic interpolation obtained by the cubic natural interpolation method can be expressed by formula (10).
Figure BDA0002433741950000107
Where i is 0,1, …, q. From equation (10), it can be determined that the corresponding interval [ k, k +1] is]Of the cubic interpolation function sk(t), wherein k is 1, 2.
Step S402: reconstructing the decomposed network flow according to the coefficient of the cubic interpolation function to obtain the cubic interpolation node of the network flow componentIf, in particular, it can be assumed that the time unit of the reconstructed fine-grained traffic is v when the sampling interval is v × j (where j is>1 and j ∈ Z) time units, the sampled network traffic is gammav,j. If t (0) is continuously acquired<t<v × j × N) time units of data, each end-to-end flow having a total of N sampled data the reconstruction of the end-to-end communication of v time units requires (j-1) × N interpolation points, and therefore equation (11) can be derived.
Figure BDA0002433741950000108
Wherein
Figure BDA0002433741950000109
And
Figure BDA00024337419500001010
respectively representing time slots and corresponding sampled network traffic xv,jAnd fine-grained cubic interpolation results. In addition to this, the present invention is,
Figure BDA00024337419500001011
j and k ∈ {1, 2., N }.
The reconstruction is performed using a cubic interpolation method, and equation (12) can be obtained.
Figure BDA00024337419500001012
Wherein the content of the first and second substances,
Figure BDA00024337419500001013
representing the result of cubic interpolation of the network traffic component.
Step S403: and calculating the result of the cubic interpolation of the network flow component according to a weighted average algorithm to obtain the network flow with fine granularity. In particular, the interpolation result for each component (IMF) of the network traffic may be calculated using a weighted average
Figure BDA0002433741950000111
For triple insertionReconstructed network traffic of value method
Figure BDA0002433741950000112
Without loss of generality, assume that equation (13) holds:
Figure BDA0002433741950000113
then, a weighted average is performed for each component. The result of the weighted average can be expressed by equation (14).
Figure BDA0002433741950000114
Wherein r isi,cA coefficient representing each component;
Figure BDA0002433741950000115
and (4) carrying out interpolation on the c component of the network flow.
In one embodiment, the coefficients of the network traffic components may be found by correlation calculations. To calculate a weighting coefficient riAssuming x (t) represents sample values of end-to-end network traffic, where 0 ≦ t ≦ j × N, the target fine-grained inference and estimation network traffic function may be represented using equation (15).
Figure BDA0002433741950000116
Constructing the objective function for t ≦ j ≦ N, for 0 ≦ t ≦ j, according to equation (15), resulting in equation (16):
Figure BDA0002433741950000117
then, equation (16) may be converted to equation (17):
Figure BDA0002433741950000118
by solving for the commonEquation (17) can obtain the optimal weighting coefficient r of the network traffic componenti,cExpressed by equation (18).
Figure BDA0002433741950000121
And obtaining an accurate reconstruction result x (t) of the end-to-end network flow according to the formula (18) and the formula (14).
In an embodiment, the fine-grained network traffic calculation method further includes: and carrying out error calculation on the network flow with fine granularity according to a relative error algorithm. Specifically, in the reconstruction of the end-to-end traffic, in order to verify the result of the network traffic calculation, the result of the network traffic may be analyzed by using a relative error. RE represents the relative network traffic error, the definition of which can be expressed by equation (19).
Figure BDA0002433741950000122
Wherein REiIndicating a relative error.
The fine-grained network traffic calculation method provided by the embodiment of the invention adopts a weighted geometric mean algorithm to calculate the weight of each network traffic component, and obtains the fine-grained network traffic by using a weighted summation method so as to improve the reconstruction precision of the end-to-end network traffic. The fine-grained network traffic calculation method provided by the embodiment of the invention has the best reconstruction performance, can extract an accurate end-to-end network traffic matrix, and has profound influence on network planning, network optimization and network scale in SDN application.
In an embodiment, as shown in fig. 8, the fine-grained network traffic calculation method provided in the embodiment of the present invention may be implemented according to the following processes: firstly, coarse-grained sampling is carried out, namely network flow is obtained, flow sampling is carried out, and reconstruction of the network flow is realized; performing fine-grained inverse sampling, namely decomposing by using EMD network traffic to obtain a plurality of network traffic components; and obtaining fine-grained interpolation by adopting a cubic interpolation method for each network flow component, and summing a plurality of calculated results to realize the recovery and reconstruction of the network flow, thereby obtaining the fine-grained network flow.
In an embodiment, the network traffic predicted by the fine-grained network traffic calculation method (EMP-SP), the SRSVD method and the ARMA method provided by the embodiment of the present invention may be compared, as shown in fig. 9, it can be seen that the reconstruction result of the EMP-SP is better than that of the ARMA and the SRSVD. Fig. 10, 11, and 12 show the results of comparative analysis of the relative errors of the network traffic by the three methods. FIG. 10 is a graph of a probability distribution function of relative error of network traffic; FIG. 11 is a boxed graph of network traffic relative error; FIG. 12 is a comparative analysis of the mean and variance of the relative error of network traffic.
From fig. 10, it can be seen that 90% of the relative error of the network traffic estimated and filled by the EMD-SP method is less than 0.3, while the relative error of the ARMA and SRSVD methods 90% is less than 0.715 and 0.682, respectively. From the steepness of the three functions, the EMD-SP method is also significantly superior to the ARMA and SRSVD methods.
From the box plot in fig. 11, it can be seen that the median of EMD-SP is much smaller than the network traffic error values estimated by the ARMA and SRSVD methods, which are 0.21, 0.39 and 0.52, respectively. The relative error distribution range of the EMD-SP is far smaller than the network flow estimation results of the ARMA method and the SRSVD method.
The average value and the relative error of the network traffic relative errors of the EMD-SP, ARMA and SRSVD methods can be seen from fig. 12, and the network traffic average values of the three methods are 0.215, 0.384 and 0.513 respectively; relative errors of the network traffic estimation results are 0.09, 0.35 and 0.21 respectively, and the EMD-SP estimation result is most stable in the three methods, while the ARMA method is relatively least stable.
An embodiment of the present invention further provides a fine-grained network traffic calculation apparatus, as shown in fig. 13, the apparatus includes:
an obtaining module 1, configured to obtain an end-to-end network traffic; for details, refer to the related description of step S101 in the above method embodiment.
The sampling module 2 is used for sampling the network traffic to obtain discrete network traffic; for details, refer to the related description of step S102 in the above method embodiment.
The decomposition module 3 is used for decomposing the discrete network traffic according to an empirical mode decomposition algorithm and calculating the decomposed network traffic; for details, refer to the related description of step S103 in the above method embodiment.
And the recovery module 4 is used for recovering the decomposed network traffic according to a cubic interpolation algorithm and calculating to obtain the fine-grained network traffic. For details, refer to the related description of step S104 in the above method embodiment.
The fine-grained network traffic calculation device provided by the embodiment of the invention researches how to estimate and recover an end-to-end network traffic matrix in a fine-grained size from sampled traffic tracking. The computing device can reconstruct network flow with fine granularity by using an EMD method and cubic interpolation. Firstly, the fractal and self-similar characteristics of end-to-end network flow are utilized, an EMD method is used for decomposing the network flow, and the decomposed components can well reflect the relevant characteristics of the network flow; and then recover them at a finer time granularity using a cubic interpolation method. Therefore, the fine-grained network traffic calculation device provided by the embodiment of the invention can calculate and obtain accurate end-to-end network traffic, and solves the technical problem that accurate measurement and analysis of the network traffic cannot be realized in the prior art.
In one embodiment, as shown in fig. 14, the decomposition module 3 includes:
the decomposition submodule 31 is used for decomposing the discrete network traffic into the sum of the component components and the residual error according to an empirical mode decomposition algorithm; for details, refer to the related description of step S301 in the above method embodiment.
A correlation coefficient calculation module 32, configured to calculate a correlation coefficient between the component and an original network traffic according to the decomposition times; for details, refer to the related description of step S302 in the above method embodiment.
An effective component calculation module 33, configured to determine an effective component according to the correlation coefficient; for details, refer to the related description of step S303 in the above method embodiment.
And the decomposition calculation module 34 is used for determining the decomposed network flow according to the effective component and the residual error. For details, refer to the related description of step S304 in the above method embodiment.
In one embodiment, as shown in fig. 15, the recovery module 4 includes:
an interpolation coefficient calculation module 41, configured to calculate a coefficient of a cubic interpolation function according to a time interval of network traffic sampling; for details, refer to the related description of step S401 in the above method embodiment.
The interpolation calculation module 42 is configured to reconstruct the decomposed network traffic according to the coefficient of the cubic interpolation function, so as to obtain a cubic interpolation result of the network traffic component; for details, refer to the related description of step S402 in the above method embodiment.
And the weighted calculation module 43 is configured to calculate a result of the cubic interpolation of the network traffic component according to a weighted average algorithm, so as to obtain a fine-grained network traffic. For details, refer to the related description of step S403 in the above method embodiment.
In one embodiment, as shown in fig. 16, the fine-grained network traffic calculation apparatus further includes:
and the error calculation module 5 is used for performing error calculation on the fine-grained network flow according to a relative error algorithm. Specifically, in the reconstruction of the end-to-end traffic, in order to verify the result of the network traffic calculation, the result of the network traffic may be analyzed by using a relative error. RE represents the relative network traffic error, the definition of which can be expressed by equation (19).
Figure BDA0002433741950000151
Wherein REiIndicating a relative error.
The fine-grained network traffic calculation device provided by the embodiment of the invention adopts a weighted geometric mean algorithm to calculate the weight of each network traffic component, and obtains the fine-grained network traffic by using a weighted summation method so as to improve the reconstruction precision of the end-to-end network traffic. The fine-grained network flow calculation device provided by the embodiment of the invention has the best reconstruction performance, can extract an accurate end-to-end network flow matrix, and has profound influence on network planning, network optimization and network scale in SDN application.
The detailed description of the function of the fine-grained network traffic calculation apparatus provided by the embodiment of the present invention refers to the description of the fine-grained network traffic calculation method in the above embodiment.
An embodiment of the present invention further provides a storage medium, as shown in fig. 17, on which a computer program 601 is stored, where the instructions, when executed by a processor, implement the steps of the screen capture method in the foregoing embodiments. The storage medium is also stored with audio and video stream data, characteristic frame data, an interactive request signaling, encrypted data, preset data size and the like. The storage medium may be a magnetic Disk, an optical Disk, a Read-Only Memory (ROM), a Random Access Memory (RAM), a flash Memory (FlashMemory), a Hard Disk (Hard Disk Drive, abbreviated as HDD) or a Solid State Drive (SSD), etc.; the storage medium may also comprise a combination of memories of the kind described above.
It will be understood by those skilled in the art that all or part of the processes of the methods of the embodiments described above can be implemented by a computer program, which can be stored in a computer-readable storage medium, and when executed, can include the processes of the embodiments of the methods described above. The storage medium may be a magnetic Disk, an optical Disk, a Read-Only Memory (ROM), a Random Access Memory (RAM), a flash Memory (FlashMemory), a Hard Disk (Hard Disk Drive, abbreviated as HDD), a Solid State Drive (SSD), or the like; the storage medium may also comprise a combination of memories of the kind described above.
An embodiment of the present invention further provides an electronic device, as shown in fig. 18, the electronic device may include a processor 51 and a memory 52, where the processor 51 and the memory 52 may be connected by a bus or in another manner, and fig. 18 takes the connection by the bus as an example.
The processor 51 may be a Central Processing Unit (CPU). The Processor 51 may also be other general purpose processors, Digital Signal Processors (DSPs), Application Specific Integrated Circuits (ASICs), Field Programmable Gate Arrays (FPGAs) or other Programmable logic devices, discrete Gate or transistor logic devices, discrete hardware components, or combinations thereof.
The memory 52, which is a non-transitory computer readable storage medium, may be used to store non-transitory software programs, non-transitory computer executable programs, and modules, such as the corresponding program instructions/modules in the embodiments of the present invention. The processor 51 executes various functional applications and data processing of the processor by running non-transitory software programs, instructions and modules stored in the memory 52, that is, implements the screen-capture method in the above-described method embodiments.
The memory 52 may include a storage program area and a storage data area, wherein the storage program area may store an operating system, an application program required for at least one function; the storage data area may store data created by the processor 51, and the like. Further, the memory 52 may include high speed random access memory, and may also include non-transitory memory, such as at least one magnetic disk storage device, flash memory device, or other non-transitory solid state storage device. In some embodiments, the memory 52 may optionally include memory located remotely from the processor 51, and these remote memories may be connected to the processor 51 via a network. Examples of such networks include, but are not limited to, the internet, intranets, local area networks, mobile communication networks, and combinations thereof.
The one or more modules are stored in the memory 52 and, when executed by the processor 51, perform a fine-grained network traffic calculation method as in the embodiments of fig. 1-12.
The details of the electronic device may be understood by referring to the corresponding descriptions and effects in the embodiments shown in fig. 1 to 12, which are not described herein again.
Although the embodiments of the present invention have been described in conjunction with the accompanying drawings, those skilled in the art may make various modifications and variations without departing from the spirit and scope of the invention, and such modifications and variations fall within the scope defined by the appended claims.

Claims (10)

1. A fine-grained network traffic calculation method is characterized by comprising the following steps:
acquiring end-to-end network flow;
sampling the network flow to obtain discrete network flow;
decomposing the discrete network flow according to an empirical mode decomposition algorithm, and calculating to obtain decomposed network flow;
and recovering the decomposed network flow according to a cubic interpolation algorithm, and calculating to obtain the fine-grained network flow.
2. The fine-grained network traffic calculation method according to claim 1, wherein the discrete network traffic is decomposed according to an empirical mode decomposition algorithm, and the calculation of the decomposed network traffic includes:
decomposing the discrete network flow into the sum of component components and residual errors according to an empirical mode decomposition algorithm;
calculating the correlation coefficient of the component and the original network flow according to the decomposition times;
determining effective component components according to the correlation coefficients;
and determining the decomposed network flow according to the effective component and the residual error.
3. The fine-grained network traffic calculation method according to claim 1, wherein the fine-grained network traffic is calculated by recovering the decomposed network traffic according to a cubic interpolation algorithm, and the method comprises the following steps:
calculating according to the time interval of network flow sampling to obtain a coefficient of a cubic interpolation function;
reconstructing the decomposed network flow according to the coefficient of the cubic interpolation function to obtain a cubic interpolation result of the network flow component;
and calculating the result of the cubic interpolation of the network flow component according to a weighted average algorithm to obtain the network flow with fine granularity.
4. The fine-grained network traffic calculation method according to claim 1, further comprising:
and carrying out error calculation on the fine-grained network flow according to a relative error algorithm.
5. A fine-grained network traffic computing apparatus, comprising:
the acquisition module is used for acquiring end-to-end network flow;
the sampling module is used for sampling the network flow to obtain discrete network flow;
the decomposition module is used for decomposing the discrete network flow according to an empirical mode decomposition algorithm and calculating to obtain the decomposed network flow;
and the recovery module is used for recovering the decomposed network flow according to a cubic interpolation algorithm and calculating to obtain the fine-grained network flow.
6. The fine-grained network traffic computing device of claim 5 wherein the decomposition module comprises:
the decomposition submodule is used for decomposing the discrete network flow into the sum of component components and residual errors according to an empirical mode decomposition algorithm;
the correlation coefficient calculation module is used for calculating the correlation coefficients of the component components and the original network flow according to the decomposition times;
the effective component calculation module is used for determining the effective component according to the correlation coefficient;
and the decomposition calculation module is used for determining the decomposed network flow according to the effective component components and the residual errors.
7. The fine-grained network traffic computing device of claim 5 wherein the recovery module comprises:
the interpolation coefficient calculation module is used for calculating the coefficient of the cubic interpolation function according to the time interval of network flow sampling;
the interpolation calculation module is used for reconstructing the decomposed network flow according to the coefficient of the cubic interpolation function to obtain a cubic interpolation result of the network flow component;
and the weighted calculation module is used for calculating the result of the cubic interpolation of the network flow component according to a weighted average algorithm to obtain the network flow with fine granularity.
8. The fine-grained network traffic computing device of claim 5 further comprising:
and the error calculation module is used for performing error calculation on the fine-grained network flow according to a relative error algorithm.
9. A computer-readable storage medium storing computer instructions for causing a computer to perform the fine-grained network traffic calculation method of any one of claims 1-4.
10. An electronic device, comprising: a memory and a processor, the memory and the processor being communicatively coupled, the memory storing computer instructions, the processor executing the computer instructions to perform the fine-grained network traffic calculation method of any of claims 1-4.
CN202010247708.2A 2020-03-31 2020-03-31 Fine-grained network flow calculation method and device and storage medium Active CN111464354B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202010247708.2A CN111464354B (en) 2020-03-31 2020-03-31 Fine-grained network flow calculation method and device and storage medium

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202010247708.2A CN111464354B (en) 2020-03-31 2020-03-31 Fine-grained network flow calculation method and device and storage medium

Publications (2)

Publication Number Publication Date
CN111464354A true CN111464354A (en) 2020-07-28
CN111464354B CN111464354B (en) 2023-02-28

Family

ID=71680985

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202010247708.2A Active CN111464354B (en) 2020-03-31 2020-03-31 Fine-grained network flow calculation method and device and storage medium

Country Status (1)

Country Link
CN (1) CN111464354B (en)

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN115022191A (en) * 2022-05-26 2022-09-06 电子科技大学 Quick inversion method for end-to-end flow in IPv6 network
CN115442246A (en) * 2022-08-31 2022-12-06 武汉烽火技术服务有限公司 Flow prediction method, device, equipment and storage medium of data plane network

Citations (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN102568205A (en) * 2012-01-10 2012-07-11 吉林大学 Traffic parameter short-time prediction method based on empirical mode decomposition and classification combination prediction in abnormal state
CN103729688A (en) * 2013-12-18 2014-04-16 北京交通大学 Section traffic neural network prediction method based on EMD
US20160164721A1 (en) * 2013-03-14 2016-06-09 Google Inc. Anomaly detection in time series data using post-processing
CN108845230A (en) * 2018-06-22 2018-11-20 国网陕西省电力公司电力科学研究院 A kind of sub-synchronous oscillation random time-dependent modal identification method
CN109802862A (en) * 2019-03-26 2019-05-24 重庆邮电大学 A kind of combined network flow prediction method based on set empirical mode decomposition
CN110381523A (en) * 2019-06-17 2019-10-25 盐城吉大智能终端产业研究院有限公司 A kind of network of cellular basestations method for predicting based on TVF-EMD-LSTM model
CN110740063A (en) * 2019-10-25 2020-01-31 电子科技大学 Network flow characteristic index prediction method based on signal decomposition and periodic characteristics
CN110933023A (en) * 2019-10-16 2020-03-27 电子科技大学 Network flow abnormity detection method for networking communication of multimedia medical equipment

Patent Citations (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN102568205A (en) * 2012-01-10 2012-07-11 吉林大学 Traffic parameter short-time prediction method based on empirical mode decomposition and classification combination prediction in abnormal state
US20160164721A1 (en) * 2013-03-14 2016-06-09 Google Inc. Anomaly detection in time series data using post-processing
CN103729688A (en) * 2013-12-18 2014-04-16 北京交通大学 Section traffic neural network prediction method based on EMD
CN108845230A (en) * 2018-06-22 2018-11-20 国网陕西省电力公司电力科学研究院 A kind of sub-synchronous oscillation random time-dependent modal identification method
CN109802862A (en) * 2019-03-26 2019-05-24 重庆邮电大学 A kind of combined network flow prediction method based on set empirical mode decomposition
CN110381523A (en) * 2019-06-17 2019-10-25 盐城吉大智能终端产业研究院有限公司 A kind of network of cellular basestations method for predicting based on TVF-EMD-LSTM model
CN110933023A (en) * 2019-10-16 2020-03-27 电子科技大学 Network flow abnormity detection method for networking communication of multimedia medical equipment
CN110740063A (en) * 2019-10-25 2020-01-31 电子科技大学 Network flow characteristic index prediction method based on signal decomposition and periodic characteristics

Cited By (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN115022191A (en) * 2022-05-26 2022-09-06 电子科技大学 Quick inversion method for end-to-end flow in IPv6 network
CN115022191B (en) * 2022-05-26 2023-10-03 电子科技大学 End-to-end flow quick inversion method in IPv6 network
CN115442246A (en) * 2022-08-31 2022-12-06 武汉烽火技术服务有限公司 Flow prediction method, device, equipment and storage medium of data plane network
CN115442246B (en) * 2022-08-31 2023-09-26 武汉烽火技术服务有限公司 Traffic prediction method, device, equipment and storage medium of data plane network

Also Published As

Publication number Publication date
CN111464354B (en) 2023-02-28

Similar Documents

Publication Publication Date Title
CN110839016B (en) Abnormal flow monitoring method, device, equipment and storage medium
CN111464354B (en) Fine-grained network flow calculation method and device and storage medium
WO2022141871A1 (en) Time sequence data anomaly detection method, apparatus and device, and storage medium
CN108804392B (en) Traffic data tensor filling method based on space-time constraint
KR20170056687A (en) Actually-measured marine environment data assimilation method based on sequence recursive filtering three-dimensional variation
CN104459696A (en) SAR interference baseline precise estimating method based on flat-earth phase
CN108965017B (en) Network traffic prediction method and device
CN110082823B (en) Seismic data interpolation method and device
KR102319145B1 (en) Method and device for generating high-resolution ocean data
US9559911B1 (en) Forecasting link utilization between points of presence in an IP network
CN113723716B (en) Passenger flow classification early warning abnormity warning method, device and storage medium
CN115830866A (en) Traffic jam inference method, system, device and medium based on time sequence dynamic graph
CN116505653A (en) Transmission line monitoring system, method, device, computer equipment and storage medium
Devaraj et al. Uncertainty propagation in analytic availability models
CN116680635A (en) Power grid fault position inference method and system
CN115376309B (en) Missing traffic data restoration method based on multi-view time matrix decomposition
Xuegang et al. Missing Data Reconstruction Based on Spectral k-Support Norm Minimization for NB-IoT Data
Cressie et al. Mapping rates associated with polygons
Xu et al. The measurement of road traffic states under high data loss rate
JP2005249662A (en) Weather radar system
CN116963156B (en) Wireless signal transmission capability detection method and device
CN117172997B (en) Flood control early warning method and device, computer equipment and storage medium
Wang et al. The study of short-term traffic flow forecasting based on theory of chaos
Parker et al. Design of experiments for categorical repeated measurements in packet communication networks
CN115225528B (en) Network flow data distributed measurement scheduling method, system and medium

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant