CN102724078B - End-to-end network flow reconstruction method based on compression sensing in dynamic network - Google Patents

Abstract

The invention discloses an end-to-end network flow reconstruction method based on compression sensing in a dynamic network. In a large-scale IP backbone network, the OD flow is selected by a random walk method; the flow value of partial OD flow acquired by a router subset is described by constructing a sparse flow matrix; a compressing sensing reconstruction module is established by adopting a main information analysis method; a relationship between the OD flow generated by the router sublet and all the end-to-end OD flow in the IP backbone network is descried by using the module, and further all the end-to-end OD flow of the whole IP backbone network is determined. The method can more accurately acquire end-to-end network flow detail characteristic, cannot consume mass hardware resources, can track the dynamic changes of the OD flow in a real time manner, and the reconstruction error is less.

Description

End to end network flow reconstructing method based on compressed sensing under a kind of dynamic network

Technical field

The present invention belongs to dynamic network down-off to measure and analysis field, is specifically related to the end to end network flow reconstructing method based on compressed sensing under a kind of dynamic network.

Background technology

In recent years, along with the develop rapidly of the Internet, although the service that the network application of being on the increase provides users with the convenient.But also make network become day by day complicated.For Virtual network operator, the management of network and control are also more and more difficult.Traffic matrix is most important input parameter in network traffic engineering, represent all end-to-end fluxes in network, and the distribution of flow has intactly been described, provide current network state to network manager, but be difficult in actual applications accurately obtain traffic matrix.

Even if traffic matrix is extremely important for network operator, the estimated value that still obtains exactly traffic matrix is very difficult.Therefore, traffic matrix estimates that becoming one has challenging research topic.In recent years, emerged a large amount of achievements in research.Vardi proposes to use network tomography method to solve end-to-end flux reconstruction, and this method is found broad application subsequently, and is used to study IP network internal feature.For network tomography method, end-to-end flux can, by Poisson distribution and Gaussian Profile reconstruct, still can not be caught the correlation of the room and time of end-to-end flux; The people such as P.Conti utilize maximal possibility estimation algorithm calculating Hurst value to carry out estimated flow matrix.The people such as Nucci add weight to deriving from the routing configuration of multiple routing iinformations, obtain little traffic matrix evaluated error with this.Y.Zhang etc. have proposed to describe with gravity field model the characteristic of current end-to-end flux, by obtaining extra constraint information, to overcome the highly problem of morbid state; A.Lakhina etc. have proposed PCA and have directly measured and built end-to-end flux reconstruction model; The independent same distribution Poisson model hypothesis based on end-to-end flux such as A.Soule, proposes iteration Bayes inversion algorithm and carrys out reconstruct end-to-end flux; G.Liang etc. have proposed a kind of pseudo-likelihood reconstructing method, and using improved EM algorithm is that several comprise a subproblem that OD is right by PROBLEM DECOMPOSITION, and the error of estimation precision is decreased.

Although statistical model correlation technique can obtain the estimated value of traffic matrix, and is widely used in the middle of reality, still estimated flow matrix exactly of these methods.In fact directly measure end-to-end network traffics specific discharge and estimate more accurately, and equipment manufacturers provide a large amount of measuring technique and equipment, and this is wherein most typical is exactly Netflow.Netflow can be by extracting and analyze source, the object IP address of packet, and source, destination slogan and protocol number obtain network flow value end to end.But it is unpractical using NetFlow to measure each OD stream, or even not attainable.This is because the port of router, in the time of operation NetFlow program, will consume a large amount of hardware resources, and directly measures and will cause extra communication overhead.So just reduce the storing and forwarding efficiency of router, so increased network delay and easily caused network congestion.

Summary of the invention

For the deficiencies in the prior art, the present invention proposes the end to end network flow reconstructing method based on compressed sensing under a kind of dynamic network, to reach minimizing hardware resource, and the dynamic change of real-time tracking OD stream, the object of less reconstructed error.

An end to end network flow reconstructing method based on compressed sensing under dynamic network, comprises the following steps:

Step 1, the system that arranges flow length, the random walk number of times of random walk for OD;

Step 2, in large-scale ip backbone network, adopt the mode of random walk to select OD stream, adopt the method that builds boolean's sparseness measuring matrix to describe the opening and closing state of whole routers according to selected OD stream;

The method of step 3, employing structure rarefied flow moment matrix is described the flow value of the part OD stream of router subset collection, described router subset is the router of whole openings, and according to boolean's sparseness measuring matrix and the rarefied flow moment matrix computation and measurement value of constructing;

Step 4, according to the sparseness measuring matrix of constructing in step 3 with calculate gained measured value, adopt principal component analysis method to build compressed sensing reconstruction model, describe the relation of whole end-to-end OD streams in OD stream that router subset produces and large-scale ip backbone network with this model, and then determine in whole IP backbone all OD stream flow end to end.

Structure boolean sparseness measuring matrix described in step 2, comprises the following steps:

Step 2-1, OD stream complete graph of structure, complete graph comprises summit and limit, the number on summit is equal to the number of OD stream;

Step 2-2: random walk is set and measures number of times initial value;

Step 2-3: select a summit as unified starting point;

Step 2-4: complete graph is carried out to equally distributed random walk, the point finding according to random walk, the router state that this point is corresponding is opening, otherwise is closed condition;

Step 2-5: if random walk measurement number of times is less than initialization, random walk measurement number of times is set, returns to step 2-3; If random walk measurement number of times is greater than or equal to initialization, random walk measurement number of times is set, performs step 2-6;

Step 2-6: preserve boolean's sparseness measuring matrix.

Boolean's sparseness measuring matrix described in step 1, this matrix meets the equidistant criterion of constraint; Line number in boolean's sparseness measuring matrix represents the number of times of random walk large-scale ip backbone network, and the columns in boolean's sparseness measuring matrix represents the sum of OD stream in large-scale ip backbone network.

The calculating of the measured value described in step 3 meets following formula:

Measured value=boolean sparseness measuring matrix × rarefied flow moment matrix

Advantage of the present invention:

End to end network flow reconstructing method based on compressed sensing under a kind of dynamic network of the present invention, is realized large-scale IP backbone is carried out to network traffics reconstruct end to end by the direct metering system of part.Can obtain more accurately network traffics details characteristic end to end, can't consume a large amount of hardware resources, can real-time tracking OD the dynamic change of stream, less reconstructed error.

Brief description of the drawings

Fig. 1 is the end to end network flow reconstructing method flow chart based on compressed sensing under the dynamic network of an embodiment of the present invention;

Fig. 2 is an embodiment of the present invention stream sensing reconstructing framework schematic diagram;

Fig. 3 is that an embodiment of the present invention builds boolean's sparseness measuring matrix flow chart;

Fig. 4 is that an embodiment of the present invention random walk builds boolean's sparseness measuring matrix schematic diagram;

Wherein, 4-1 is complete graph;

Fig. 5 is that an embodiment of the present invention NMAE changes schematic diagram with the order of traffic matrix;

Fig. 6 is that an embodiment of the present invention NMAE is with random walk length variations schematic diagram;

Fig. 7 is that an embodiment of the present invention needs the OD flow amount schematic diagram of directly measuring;

Fig. 8 is that an embodiment of the present invention is estimated schematic diagram to the 30th article, the 60th article OD stream traffic matrix; A) be that the 30th article of OD stream traffic matrix estimated schematic diagram; B) be that the 60th article of OD stream traffic matrix estimated schematic diagram;

Fig. 9 is an embodiment of the present invention space relative error and time relative error schematic diagram; A) be space relative error schematic diagram; B) be time relative error schematic diagram;

Figure 10 is an embodiment of the present invention space relative error and time error cumulative distribution function schematic diagram; A) be space relative error cumulative distribution function schematic diagram; B) be time relative error cumulative distribution function schematic diagram.

Embodiment

Below in conjunction with accompanying drawing, specific embodiment of the invention is described further.

First the embodiment of the present invention builds boolean's sparseness measuring matrix according to random walk, obeys RIP(constraint equidistant) criterion; Then corresponding boolean's sparseness measuring matrix is collected part end-to-end flux with less router port, according to the linear relationship computation and measurement value between boolean's sparseness measuring matrix, rarefied flow moment matrix and measured value; Finally utilizing PCA(principal component to analyse) model carries out singular value decomposition to meet rarefaction condition to traffic matrix, based on all OD stream of compressed sensing reconstruct traffic matrix.

In the embodiment of the present invention, use the real traffic data of Abilene backbone network, it has 12 nodes, 30 interior links, and 24 outer links, 144 end-to-end fluxes, emulated data adopts the 5min time interval, amounts to 2016 moment.

Traffic matrix in Abilene backbone network can be as shown in Equation (1),

$M=\left[\begin{array}{cccc}{m}_1\left(1\right)& m_1\left(2\right)& ...& m_1\left(T\right)\\ m_2\left(1\right)& m_2\left(2\right)& ...& m_2\left(T\right)\\ .& .& & .\\ .& .& & .\\ .& .& & .\\ m_N\left(1\right)& m_N\left(2\right)& ...& m_N\left(T\right)\end{array}\right]---\left(1\right)$

M in formula (1) represents traffic matrix, and the row of matrix M represents in extensive Abilene backbone network that source node is to the flow total amount of destination node.

Wherein, number of network node is n, OD fluxion N=n ².Therefore, M is the traffic matrix of a N × T.

M _j(t): the t moment that represents j article of OD stream;

M (t): the t row that represent M;

j∈{1,2,…,N}；

t∈{1,2,…,T}。

Compressive sensing theory must meet two characteristics: the one, measure matrix and meet RIP(constraint equidistantly) criterion; The one, traffic matrix is sparse matrix.Under regard to the requirement for matrix in compressive sensing theory and describe in conjunction with formula (2).

Due to the redundancy of signal, suppose that vectorial s can be expressed as:

Wherein,

S is discrete signal N dimension space R ⁿin rank, N × 1 column vector;

X _ifor every column element of s, i=1,2,3 ..., N;

base vector, r ⁿone group of orthogonal basis.

And if only if, and signal is by K(K<<N) rank base vector form linear combination time signal s be only sparse, i.e. { x in formula (2) _ito only have K element be non-zero.Signal s is considered to compressible in compressive sensing theory so.K also becomes the degree of rarefication of signal.

The length that random walk is set in the embodiment of the present invention is r, and measurement number of times is p, wherein random walk number of times p=19 in the embodiment of the present invention, duration of random walk r=11, the order q=5(q=K of traffic matrix), boolean's sparseness measuring matrix that the method builds is obeyed RIP criterion.

Fig. 1 is the end to end network flow reconstructing method flow chart based on compressed sensing under the dynamic network of the embodiment of the present invention, comprises the following steps:

Step 1, the order of the length, random walk number of times and the traffic matrix that are directed to router random walk in system is set;

The order of described traffic matrix is the degree of rarefication that meets compressive sensing theory matrix, the order q=5 of traffic matrix in the present embodiment;

Step 2, in large-scale ip backbone network, adopt the mode of random walk to select OD stream, adopt the method that builds boolean's sparseness measuring matrix to describe the opening and closing state of whole routers according to selected OD stream;

In the embodiment of the present invention, adopt the mode of random walk to select router, the router composition router subset being selected, adopt the method that builds boolean's sparseness measuring matrix to describe the unlatching of router subset or the closed condition that are selected, build boolean's sparseness measuring matrix A of a p × N according to p random walk.Fig. 2 is that the present invention flows sensing reconstructing framework schematic diagram, network operator utilizes NetFlow collection unit to divide the data on flows of the end to end network on the router of operation by controlling the network equipment (comprising hardware and software), be the unlatching of network management center control router or close, and the NetFlow moving from each router interface collects data on flows.Collect after part end to end network flow, recover whole OD stream traffic matrixs at network management center.Fig. 3 is that the embodiment of the present invention builds boolean's sparseness measuring matrix flow chart, specifically comprises the following steps:

Step 2-1, structure OD stream complete graph G (V, E) (as complete graph 4-1 in Fig. 4), wherein, the set on V and E difference representative of graphics summit and limit.The number on summit is equal to the number N of OD stream;

Step 2-2: arrange and measure number of times parameter c _iteration=1;

Step 2-3: select a summit as unified starting point;

Step 2-4: profit is carried out equally distributed random walk to G (V, E), the point finding according to random walk, the router state that this point is corresponding is opening (being 1), otherwise is closed condition (being 0);

Fig. 4 is that random walk of the present invention builds boolean's sparseness measuring matrix schematic diagram, and wherein, the summit of black represents N bar OD stream, and random walk is carried out in summit.Summit 10,15 in the embodiment of the present invention, 18,70 and 72 is summits that random walk is found, and uses NetFlow to measure the 10th, 15,18,70 and 72 OD streams.A _ifor the row of boolean's sparseness measuring matrix that random walk builds, pass through p(p=19) inferior random walk, obtain p × N(19 × 144) boolean's sparseness measuring matrix A, as follows:

$A=\left[\begin{array}{c}{a}_1\\ a_2\\ .\\ .\\ .\\ a_p\end{array}\right]---\left(3\right)$

A _ithat a row vector is:

a _i=[a _i1,a _i2,a _i3,…,a _iN],i∈{1,2,3,…,p}. (4)

In the time that a summit is found by random walk, its corresponding matrix element value is 1.In Fig. 3, a _ithe 10th, 15,18,70,72 element values are " 1 ", and all the other are " 0 ", and a is set _i10, a _i15, a _i18, a _i70and a _i72" 1 ".

Step 2-5: if c _iteration<p, arranges c _iteration=c _iteration+ 1, carry out and return to step 204; If c _iteration>p, performs step 207;

Step 2-6: preserve boolean's sparseness measuring matrix.

The method of step 3, employing structure rarefied flow moment matrix is described the flow value of the part OD stream of router subset collection, described router subset is the router of whole openings, and according to boolean's sparseness measuring matrix and the rarefied flow moment matrix computation and measurement value of constructing;

In the embodiment of the present invention, according to formula (5)～(7), all network traffics are carried out to zero-mean processing.

In the embodiment of the present invention, intercept one section of historical data on flows as prior information, use M _hisrepresent.Zero-mean processing procedure is expressed as:

M'=M _his-M ^mean (5)

Wherein, M ' is zero-mean rarefied flow moment matrix after treatment;

M _hisfor historical data on flows matrix;

$M^{\mathrm{mean}}=\left[\begin{array}{c}{m}_1^{\mathrm{mean}},m_1^{\mathrm{mean}},m_1^{\mathrm{mean}},...,m_1^{\mathrm{mean}}\\ .\\ .\\ .\\ m_N^{\mathrm{mean}},m_N^{\mathrm{mean}},m_N^{\mathrm{mean}},...,m_N^{\mathrm{mean}}\end{array}\right]---\left(6\right)$

Formula (6) is the matrix of a N × T, and its element is just like giving a definition:

$m_j^{\mathrm{mean}}=\frac{1}{T}\underset{t=1}{\overset{T}{\mathrm{\&Sigma;}}}{m}_j\left(t\right)---\left(7\right)$

Wherein: T is the length that intercepts historical flow;

In the embodiment of the present invention, utilize historical traffic matrix computation and measurement value Y according to formula (8)～(10).

The measured value y (t) in t moment.

$\left[\begin{array}{c}{y}_1\left(t\right)\\ y_2\left(t\right)\\ .\\ .\\ .\\ y_p\left(t\right)\end{array}\right]=\left[\begin{array}{c}{a}_1\\ a_2\\ .\\ .\\ .\\ a_p\end{array}\right]\&times;\left[\begin{array}{c}{m}_1\left(t\right)\\ m_2\left(t\right)\\ .\\ .\\ .\\ m_N\left(t\right)\end{array}\right].---\left(8\right)$

Formula (8) is equivalent to,

y(t)=A×m(t) (9)

To all moment, there is following formula:

$Y=A\&times;M^{\&prime;}$

$=\left[\begin{array}{c}{a}_1\\ a_2\\ .\\ .\\ .\\ a_p\end{array}\right]\&times;\left[\begin{array}{cccc}{m}_1\left(1\right)& m_1\left(2\right)& ...& m_1\left(T\right)\\ m_2\left(1\right)& m_2\left(2\right)& ...& m_2\left(T\right)\\ .& .& & .\\ .& .& & .\\ .& .& & .\\ m_N\left(1\right)& m_N\left(2\right)& ...& m_N\left(T\right)\end{array}\right]---\left(10\right)$

Wherein, measured value Y, boolean's sparseness measuring matrix A and rarefied flow moment matrix M ' they are respectively p × T, the matrix of p × N and N × T.

Because A is a sparse random matrix, therefore we only need to know that element value corresponding with A in the M ' matrix in formula (10) can obtain measured value Y.

In the embodiment of the present invention, obtain the union of the OD adfluxion of all random walks measurements by formula (11).

Suppose the OD adfluxion that representative need to be measured by the i time random walk, all essential OD stream is expressed as:

$\left\{\begin{array}{c}{S}^f={\&cup;}_{i=1}^p{S}_i^f\\ p\&le;N\end{array}\right.---\left(11\right)$

Wherein: S ^frepresentative is asked union to the OD stream generating from random walk for the first time to the p time random walk;

the OD adfluxion that representative need to be measured by the i time random walk;

Therefore, generate all essential OD stream the number that corresponding essential OD flows is traffic matrix so

$\stackrel{\stackrel{\&OverBar;}{\&OverBar;}}{M}=\left[\begin{array}{c}{m}_1\left(t\right)\\ m_2\left(t\right)\\ .\\ .\\ .\\ m_N\left(t\right)\end{array}\right]---\left(12\right)$

Wherein: for comprising the sparse matrix of essential OD stream;

every a line represents an OD stream time series, and extraction need to directly be measured bar OD stream, all the other OD streams are corresponding the behavior 0 of matrix.In formula (10) because A is boolean's sparseness measuring matrix, so as long as know the indivedual row in M ' just can obtain Y.That is to say that Y=A × M ' is equivalent to

Describe for example the method for selecting directly to measure OD stream subset below in detail.As shown in Equation (13):

$\begin{array}{c}\begin{array}{ccc}Y\left(t\right)& A_{(5\&times;10)}& m\left(t\right)\end{array}\\ \left[\begin{array}{c}{y}_1\left(t\right)\\ y_2\left(t\right)\\ y_3\left(t\right)\\ y_4\left(t\right)\\ y_5\left(t\right)\end{array}\right]=\left[\begin{array}{c}0001010100\\ 0001100000\\ 1000010000\\ 1000110000\\ 0001000000\end{array}\right]\&times;\left[\begin{array}{c}{\mathrm{OD}}_1\\ 0\\ 0\\ {\mathrm{OD}}_4\\ {\mathrm{OD}}_5\\ {\mathrm{OD}}_6\\ 0\\ {\mathrm{OD}}_8\\ 0\\ 0\end{array}\right]\end{array}---\left(13\right)$

In formula, boolean's sparseness measuring matrix A is the sparse matrix of one " 1 " or " 0 " composition, and the line number of A represents that the number of times of random walk, columns represent the OD stream of each random walk traversal.T at any time, the OD stream that utilizes " 1 " index in matrix A column element to measure.In Fig. 4, A(5 capable 10 is listed as so) boolean's sparseness measuring matrix, i.e. random walk 5 times, the OD fluxion that travel through is 10.Utilize for the first time column element in A " 1 " manipulative indexing to OD flow point be OD4, OD6, OD8; The OD stream indexing is for the second time OD4, OD5; By that analogy, the OD stream that altogether obtains measuring through 5 random walks is respectively: OD1, OD4, OD5, OD6, OD8.

Step 4, according to the sparseness measuring matrix of constructing in step 3 with calculate gained measured value, adopt PCA(principal component to analyse) method structure compressed sensing reconstruction model, describe the relation of whole end-to-end OD streams in OD stream that router subset produces and large-scale ip backbone network with this model, and then determine in whole IP backbone all OD stream flow end to end.

For meeting the requirement of compressed sensing traffic matrix rarefaction characteristic, in the embodiment of the present invention, use pca model to historical traffic matrix carry out singular value decomposition.

Step 4-1: measured value Y is carried out to singular value decomposition;

Below in conjunction with formula (14)～(17), the principle of singular value decomposition is described further.

Principal component analysis (PCA) main thought is that high dimensional data is projected to compared with lower dimensional space.In pca model, m × n rank traffic matrix L is expressed as

L=U∑V ^T， (14)

Wherein: U is orthogonal matrix;

UU ^tor U ^tu is unit matrix;

∑ is that diagonal element class value is the singular value of the diagonal matrix of L;

V is L ^tthe characteristic vector of L;

L ^tfor the transposition of L.

In fact, formula (14) is that L utilizes orthogonal basis U ∑ to space projection.L matrix column is base vector σ _ku _klinear combination, normalization base vector is expressed as:

$u_k=\frac{{\mathrm{Lv}}_k}{{\mathrm{\&sigma;}}_k},k=\mathrm{1,2},...,\min (m,n),---\left(15\right)$

Wherein: u _k, v _krepresent respectively the k row of U and V;

Min (m, n) represents the order of matrix L;

σ _kit is the energy feature that the singular value of matrix L represents L.

In order conveniently to understand low-rank approximate procedure, formula (14) can be expressed as:

$L={\mathrm{U\&Sigma;V}}^T=\underset{k=1}{\overset{\min (m,n)}{\mathrm{\&Sigma;}}}{\mathrm{\&sigma;}}_k{u}_k{v}_k^T,---\left(16\right)$

Therefore, can select large singular value to remove approximate ewal matrix, this process can be explained with mathematical formulae:

$\hat{L}=\underset{k=1}{\overset{q}{\mathrm{\&Sigma;}}}{\mathrm{\&sigma;}}_k{u}_k{v}_k^T.---\left(17\right)$

Wherein: constant q<min (m, n);

σ _kq maximum singular value;

Q is order, it equals degree of rarefication.

In the embodiment of the present invention, suppose historical traffic matrix M _hislength d >N.According to formula (14), can be decomposed into:

$M_{\mathrm{his}}^T=U_{\mathrm{his}}{\mathrm{\&Sigma;}}_{\mathrm{his}}{V}_{\mathrm{his}}^T,---\left(18\right)$

Wherein, U _his, ∑ _hiswith respectively d × N, the matrix of N × N and N × N.

Step 4-2: according to historical traffic matrix and singular value decomposition method, be further analyzed calculating.

Can calculate V according to formula (18) _his∑ _his.Because every OD stream in traffic matrix all has long correlation, principal component remains unchanged substantially.Therefore,, in conjunction with formula (18), formula (10) can be similar to and be converted into:

Y=A×V _his∑ _hisU ^T=ΘU ^T, (19)

Wherein, AV _his∑ _his=Θ.

According to formula (19), intercept the front q row of Θ, there is new matrix Θ _colrepresent;

According to formula (16), Y can approximate representation be:

$Y=\mathrm{\&Theta;}{U}^T=\mathrm{\&Theta;}\left[\begin{array}{c}{u}_{\mathrm{PC}}^T\\ u^T\end{array}\right]\&ap;\mathrm{\&Theta;}\left[\begin{array}{c}{u}_{\mathrm{PC}}^T\\ 0\end{array}\right]=\mathrm{\&Theta;}{U}_{\mathrm{PC}}^T,---\left(20\right)$

Wherein, and u ^trespectively U ^tq × T (q<p) and (N-q) × T rank submatrix;

it is a sparse matrix that degree of rarefication is q.

Step 4-3: realize the matrix rarefaction after singular value decomposition by protruding optimization method:

${\hat{U}}_{\mathrm{PC}}^T=\arg \min {\left|\right|U_{\mathrm{PC}}^T\left|\right|}_{1}s.t.\mathrm{\&Theta;}{U}_{\mathrm{PC}}^T=Y,---\left(21\right)$

Wherein: for making while getting minimum value value;

So, formula (20) can be transformed to:

$Y={\mathrm{\&Theta;}}_{\mathrm{col}}{u}_{\mathrm{PC}}^T,---\left(22\right)$

Wherein, Θ _colthe q<p row of Θ.Therefore, l ₂norm minimum problem can be used for solving u _pC, that is,

${\hat{u}}_{\mathrm{PC}}^T=\arg \min {\left|\right|u_{\mathrm{PC}}^T\left|\right|}_{2}s.t.{\mathrm{\&Theta;}}_{\mathrm{col}}{u}_{\mathrm{PC}}^T=Y,---\left(23\right)$

With this, formula (22) is an overdetermined problem, is easy to find optimization to solve scheme.

From formula (22), can know Θ _coland u _pCfor row non-singular matrix.Therefore, can obtain following formula,

$Y=u_y{\mathrm{\&Sigma;}}_y{v}_y^T.---\left(24\right)$

Wherein, matrix u _ysingular value decomposition by Y obtains;

Matrix y ^tthe characteristic vector of Y;

Because Θ _coland u _pCall row non-singular matrix, Θ _colorder equal u _yorder (rank (Θ _col)=rank (u _y)) and with Θ _colthe space of launching for base equals with u _pCfor the space (span (Θ of base expansion _col)=span (u _y)).

Step 4-4: according to orthogonal transform to the nearest step analysis of matrix.

Calculate R according to formula (25);

Orthogonal transform matrix is obeyed following constraints, exists matrix R to meet following formula,

u _y=Θ _colR. (25)

According to formula (26), (27), calculate

Hypothesis matrix W meets with W Θ _col=I, I is unit matrix.

Therefore,

$\mathrm{WY}=W{\mathrm{\&Theta;}}_{\mathrm{col}}{u}_{\mathrm{PC}}^T=u_{\mathrm{PC}}^T.---\left(26\right)$

In addition,

Wu _y=WΘ _colR=R. (27)

According to (28), calculate

Can obtain according to formula (26) and (27):

$\left\{\begin{array}{c}\mathrm{WY}=u_{\mathrm{PC}}^T\\ {\mathrm{Wu}}_y=R\end{array}\right..---\left(28\right)$

$u_{\mathrm{PC}}^T=\left({\mathrm{Ru}}_y^T\right)Y---\left(29\right)$

Obtain according to above step

By calculating obtain traffic matrix M according to formula (18) _hisestimated value.

By above step, we can, according to directly measuring part network traffics end to end, obtain the estimated result of network traffics matrix.This matrix description all uninterrupteds end to end in large scale IP backbone.

Performance Evaluation result:

Fig. 5 is that embodiment of the present invention NMAE changes schematic diagram with the order of traffic matrix, assesses performance of the present invention by normalization mean absolute error (NMAE), and in the embodiment of the present invention, the value of order q is respectively 1,2,3,4,5,6,7,8(as shown in Figure 6).

NMAE computing formula is:

$\mathrm{NMAE}=\frac{\underset{j,t}{\mathrm{\&Sigma;}}|{\hat{m}}_j\left(t\right)-m_j\left(t\right)|}{\underset{j,t}{\mathrm{\&Sigma;}}\left|m_j\left(t\right)\right|}.---\left(29\right)$

Wherein, represent the j article of OD flow estimated value in t moment;

M _j(t) represent the j article of OD flow actual value in t moment.

Simulation result shows along with q changes, not too large variation of algorithm performance of the present invention.But q is subject to some constraints, first order q is equivalent to degree of rarefication.That is, in formula (20), in the time that order is q, U ^tcan be by it is approximate, row be that q is sparse.Boolean's sparseness measuring matrix A is the matrix of a p × N, p=O (Klog (N/K)).In emulation, K=q, therefore, p=O (qlog (N/q)).Measure number of times p and directly affect the total amount that needs the OD stream of measuring.In the embodiment of the present invention, according to p, q is set and equals the total amount that OD flows.In Fig. 5, there is a fluctuation in NMAE, is because now p is tending towards flowing border qlog (N/q).In the case, p-qlog (N/q) is less, and the probability of reconstruct failure is larger.Fig. 6 be embodiment of the present invention NMAE with random walk length variations schematic diagram, NMAE is along with the length of random walk increases on a declining curve substantially.If r is enough little, robustness will decline so; If r is too large, can produces again a part of OD that we have to measure and flow.

Shown in Fig. 7, illustrate that OD stream chooses number Normal Distribution, the intercept of straight line represents that OD stream chooses the average of number, and slope represents standard deviation.Simulation result shows: the about 105-120 bar of our needs OD diffluence recovers traffic matrix.In flow compression reconfiguration, in the embodiment of the present invention, adopt PCA method, and do not carried out other preliminary treatment, therefore, the embodiment of the present invention needs 60% link operation NetFlow.

In Fig. 8, front 500 points are used for calculating V as prior information in algorithm _his∑ _his.Random the 30th article and the 60th article OD stream of selection of the embodiment of the present invention is tested.We find that three kinds of methods can track the dynamic change of flow in time.But, in Fig. 8 (a), the sparse reconstruct singular value decomposition of SRSVD() and TomoG(Gravity Models) occur larger fluctuation, FSR(flow sensing reconstructing) can reach and obtain exactly the 30th article of OD and flow; Fig. 8 (b) clearly finds out that SRSVD and TomoG cannot flow by 60 OD of Accurate Reconstruction, and FSR can obtain the trend of changes in flow rate more accurately.Therefore, the better network traffics of estimating peer-to-peer of our method of the results show.

In order to assess more accurately the advantage of algorithm of the present invention, the embodiment of the present invention contrasts this three kinds of methods by another one standard.Because network traffics present time variation and temporal correlation, the embodiment of the present invention goes to analyze the feature of FSR with reference to space relative error (SREs) and time relative error (TREs).

SREs and TREs are expressed as:

$\mathrm{SRE}\left(n\right)=\frac{{\left|\right|{\hat{x}}_T\left(n\right)-x_T\left(n\right)\left|\right|}_2}{{\left|\right|x_T\left(n\right)\left|\right|}_2},n=\mathrm{1,2},...,N---\left(29\right)$

Wherein: represent the n article of OD flow estimated value in T moment;

X _t(n) the n article of OD flow actual value in expression T moment.

$\mathrm{TRE}\left(t\right)=\frac{{\left|\right|{\hat{x}}_N\left(t\right)-x_N\left(t\right)\left|\right|}_2}{{\left|\right|x_N\left(t\right)\left|\right|}_2},t=\mathrm{1,2},...,T---\left(30\right)$

Wherein: represent the estimated value of N article of OD flow in the t moment;

X _n(t) represent the actual value of N article of OD flow in the t moment.

Here nonnegative integer N and T represent respectively OD stream sum and measure the moment.|| || ₂represent l ₂norm.The space error that SERs has embodied different OD streams distributes.TREs has represented time error distribution simultaneously.

In Fig. 9 (a), SRSVD and TomoG method indicate that the 10th article and the 40th article of OD stream have very high fluctuation.Because the first half segment data of this OD stream is very little, the energy accounting in whole network traffics is especially little.So be difficult to estimate exactly these OD stream.And then obtain FSR, and SRSVD, the mean space correlativity analysis error of TomoG is not 0.47,0.77,0.68.But the FSR method of the embodiment of the present invention has minimum space correlation error in the first half section of OD stream.This explanation, has very stable and effective evaluation characteristic to the flow FSR of small scale.In Fig. 9 (b), all methods have sudden change in time-domain.But FSR has very low sudden change within the little time interval.Meanwhile, FSR, SRSVD, correlation error average time of TomoG is respectively 0.20,0.29,0.26, so FSR has minimum error with relation to time.

In the embodiment of the present invention, remove to obtain traffic matrix evaluation characteristic by the space relative error of three kinds of methods and the cumulative distribution function of time relative error.Figure 10 (a) shows FSR, SRSVD, and tri-kinds of methods of TomoG, the space relative error that about 92%, 73%, 84% OD stream obtains is 0.83.And the embodiment of the present invention is to estimation moment of about 80%, Figure 10 (b) shows that the time relative error obtaining is 0.25,0.31,0.30.Therefore this measurement result has represented the identical estimated result of three kinds of methods, and the evaluated error of the FSR method that wherein embodiment of the present invention adopts is minimum.

In sum, illustrate that the traffic matrix obtaining by the reconstruct of FSR method is more accurate.

Claims (3)

1. the end to end network flow reconstructing method based on compressed sensing under dynamic network, is characterized in that: comprise the following steps:

Step 1, the system that arranges flow length, the random walk number of times of random walk for OD;

Step 2, in large-scale ip backbone network, adopt the mode of random walk to select OD stream, adopt the method that builds boolean's sparseness measuring matrix to describe the opening and closing state of whole routers according to selected OD stream;

Step 2-1, OD stream complete graph of structure, complete graph comprises summit and limit, the number on summit is equal to the number of OD stream;

Step 2-2: random walk is set and measures number of times initial value;

Step 2-3: select a summit as unified starting point;

Step 2-4: complete graph is carried out to equally distributed random walk, the point finding according to random walk, the router state that this point is corresponding is opening, otherwise is closed condition;

Step 2-5: if random walk measurement number of times is less than initialization, random walk measurement number of times is set, returns to step 2-3; If random walk measurement number of times is greater than or equal to initialization, random walk measurement number of times is set, performs step 2-6;

Step 2-6: preserve boolean's sparseness measuring matrix;

The method of step 3, employing structure rarefied flow moment matrix is described the flow value of the part OD stream of router subset collection, described router subset is the router of whole openings, and according to boolean's sparseness measuring matrix and the rarefied flow moment matrix computation and measurement value of constructing;

Step 4, according to the sparseness measuring matrix of constructing in step 3 with calculate gained measured value, adopt principal component analysis method to build compressed sensing reconstruction model, describe the relation of whole end-to-end OD streams in OD stream that router subset produces and large-scale ip backbone network with this model, and then determine in whole IP backbone all OD stream flow end to end; Specific as follows:

Step 4-1: measured value is carried out to singular value decomposition;

Measured value is carried out to singular value decomposition, can obtain following formula:

$Y=u_y{\mathrm{\&Sigma;}}_y{v}_y^T.---\left(1\right)$

Wherein, matrix u _ysingular value decomposition by Y obtains;

Matrix y ^tthe characteristic vector of Y;

Step 4-2: according to historical traffic matrix and singular value decomposition method, be further analyzed calculating;

To historical traffic matrix M _hisdecompose:

$M_{\mathrm{his}}^T=U_{\mathrm{his}}{\mathrm{\&Sigma;}}_{\mathrm{his}}{V}_{\mathrm{his}}^T,---\left(2\right)$

Wherein, U _hisbe the matrix of d × N, d is historical flow M _hishypothesis length, d > N, N is the number of OD stream, U _hisorthogonal matrix, or it is unit matrix; Σ _histhe matrix of N × N, Σ _hisfor diagonal element class value, be M _histhe singular value of diagonal matrix; it is the matrix of N × N;

Calculate V according to formula (2) _hisΣ _his, and measured value computational process is similar to and is converted into:

Y＝A×V _hisΣ _hisU ^T＝ΘU ^T， (3)

Wherein, AV _hisΣ _his=Θ; $A=\left[\begin{array}{c}{a}_1\\ a_2\\ .\\ .\\ .\\ a_p\end{array}\right],$ A _ifor the row of boolean's sparseness measuring matrix that random walk builds, a _ithat a row vector is: a _i=[a _i1, a _i2, a _i3..., a _iN], i ∈ 1,2,3 ..., p};

Y approximate representation is:

$Y={\mathrm{\&Theta;U}}^T=\mathrm{\&Theta;}\left[\begin{array}{c}{u}_{\mathrm{PC}}^T\\ u^T\end{array}\right]\&ap;\mathrm{\&Theta;}\left[\begin{array}{c}{u}_{\mathrm{PC}}^T\\ 0\end{array}\right]=\mathrm{\&Theta;}{U}_{\mathrm{PC}}^T,---\left(4\right)$

Wherein, and u ^trespectively U ^tq × T and (N-q) × T rank submatrix; Q < p, p is for measuring number of times; T is the length that intercepts historical flow; it is a sparse matrix that degree of rarefication is q;

Step 4-3: realize the matrix rarefaction after singular value decomposition by protruding optimization method;

$\begin{array}{ccc}{\hat{U}}_{\mathrm{PC}}^T=\arg \min {\left|\right|U_{\mathrm{PC}}^T\left|\right|}_1& s.t.& \mathrm{\&Theta;}{U}_{\mathrm{PC}}^T=Y,\end{array}---\left(5\right)$

Wherein: for making while getting minimum value value;

Adopt l ₂norm minimum problem solving u _pC, that is,

$\begin{array}{ccc}{\hat{u}}_{\mathrm{PC}}^T=\arg \min {\left|\right|u_{\mathrm{PC}}^T\left|\right|}_2& s.t.& {\mathrm{\&Theta;}}_{\mathrm{col}}{u}_{\mathrm{PC}}^T=Y\end{array}---\left(6\right)$

Wherein, Θ _colrepresent to intercept the front q row of Θ; for making while getting minimum value value;

Step 4-4: according to orthogonal transform to the nearest step analysis of matrix;

Orthogonal transform matrix is obeyed following constraints, exists matrix R to meet following formula:

u _y＝Θ _colR. (7)

Can obtain according to formula (6) and formula (7):

$u_{\mathrm{PC}}^T=\left({\mathrm{Ru}}_y^T\right)Y---\left(8\right)$

According to obtaining by calculating obtain traffic matrix M according to formula (2) _hisestimated value.

2. the end to end network flow reconstructing method based on compressed sensing under dynamic network according to claim 1, is characterized in that: the boolean's sparseness measuring matrix described in step 1, and this matrix meets the equidistant criterion of constraint; Line number in boolean's sparseness measuring matrix represents the number of times of random walk large-scale ip backbone network, and the columns in boolean's sparseness measuring matrix represents the sum of OD stream in large-scale ip backbone network.

3. the end to end network flow reconstructing method based on compressed sensing under dynamic network according to claim 1, is characterized in that: the calculating of the measured value described in step 3 meets following formula:

Measured value=boolean sparseness measuring matrix × rarefied flow moment matrix.