CN103607292A - Fast distributed monitoring method for electric-power communication network services - Google Patents

Abstract

The invention relates to a fast distributed monitoring method for electric power communication network services, comprising the following steps: generating a Bernoulli matrix, selecting OD flows to be directly measured through the matrix, and constructing a flow matrix; obtaining an observation matrix; constructing an optimized greedy self Adaptation dictionary; traffic matrix reconstruction by compressed sensing. According to the power-law distribution of the power communication network flow, the present invention uses a random matrix to select some directly measured OD flows, and runs a flow collection function in some routers according to requirements to obtain flow information of these OD flows. Based on these traffic information, an end-to-end network traffic reconstruction model is constructed, and the compressed sensing reconstruction algorithm is used to solve the model to obtain the monitoring value of traffic. It can accurately obtain the flow monitoring value in real time, and can effectively reduce the loss of network flow collection.

Description

A Fast Distributed Monitoring Method for Power Communication Network Services

technical field

The invention relates to the field of large-scale network end-to-end flow measurement and analysis, in particular to a fast distributed monitoring method for power communication network services.

Background technique

With the rapid development of power communication networks, the network provides basic communication technology support for production practices such as dispatching telephones, relay protection, and automation. In order to realize the intelligence of the power grid, the access terminals and bearer services of the power communication network become diversified. This puts forward higher requirements for network traffic engineering management such as network management, network monitoring, network design and network planning of power communication network. End-to-end network traffic (source-destination flow or OD flow) monitoring is an important input parameter and implementation basis for operations such as network management and traffic engineering, so end-to-end network traffic monitoring has received extensive attention.

Traditional network monitoring technology can realize real-time reproduction and prediction of large-scale backbone network traffic, and describe network traffic status through traffic matrix. Network traffic monitoring techniques can be divided into two categories, namely direct measurement and traffic estimation. Compared with traffic estimation methods, direct measurement can more accurately describe the dynamic changes of network traffic. Therefore, network equipment suppliers provide network traffic collection functions in network equipment (for example, NetFlow of Cisco routers). As shown in Figure 1, the traditional network traffic monitoring method collects traffic status information by running the traffic collection function on each router, and sends it to the network management station through the backbone network. This is the most straightforward way to get traffic status information. However, running the traffic collection function occupies the CPU and memory resources of the router, thus reducing the store-and-forward capability of the router. In addition, the transmission of traffic status information additionally increases the load of the network. To sum up, this method greatly increases the network loss, so it is rarely used in practice.

Through the network management station to control the traffic collection function of the router, a distributed traffic monitoring system is built to collect part of the end-to-end network traffic status information, and all the end-to-end network traffic monitoring values can be obtained through the traffic reconstruction method, as shown in Figure 2. Show. Reduce network operations energy consumption and network load by reducing the amount of directly measured end-to-end network traffic. How to determine the part of the end-to-end network traffic that needs to be directly measured, and how to reconstruct all the traffic status information at the network management station are the main problems when building a distributed network traffic monitoring system. Most of the existing methods are to optimize the distribution of routers running the collection function, and run the flow collection function on the least routers to measure most of the end-to-end network flow information. Although this method can significantly reduce network loss, part of the traffic information will be lost. Therefore, it is of great significance to study the distributed flow monitoring system to obtain all the network flow information.

Contents of the invention

To solve the problem of end-to-end network traffic real-time monitoring for the shortcomings of the prior art, the present invention provides a fast distributed monitoring method for power communication network services. The method selects some directly measured OD flows according to the Bernoulli measurement matrix and power-law distribution characteristics; according to these OD flows, a linear inference problem is established, and then, using the optimized greedy adaptive dictionary (OGAD) learning algorithm, the linear inference problem obeys Compressed Sensing Technical Requirements. Finally, the end-to-end network traffic is recovered using a compressive sensing reconstruction algorithm.

In order to realize the purpose of the present invention, the technical scheme adopted in the present invention is:

A fast distributed monitoring method for power communication network services, comprising the following steps:

Step 1: Generate a Bernoulli matrix, select the OD flow to be measured directly through the matrix, and construct a flow matrix;

Step 2: Find the observation matrix;

The construction of the observation matrix depends on the non-zero rows in the flow matrix X _part , and the rows with other zero elements represent unknown OD flows that need to be reconstructed. The method of calculating the observation matrix Y _m is as follows:

Y _m =B·X _part (2) where B is an M×N Bernoulli matrix, and X _part is an N×T traffic matrix. Then a linear system is formed by B, X _part and observation matrix Y _m ;

Step 3: Build an optimized greedy adaptive dictionary;

Step 4: Reconstruct the traffic matrix by compressed sensing.

Described step 1 specifically comprises the following steps:

Step 1-1: Generate Bernoulli matrix;

A Bernoulli random matrix B of M×N (M<N) is generated, and N is the number of OD flows in the network, which is equal to the square of the number of network nodes. The elements b(m,n) of the Bernoulli matrix are independent and identically distributed, the probability that the element is equal to 1 is Pr, and the probability that the element is equal to 0 is 1-Pr.

Step 1-2: Determine the number of OD flows that need to be measured;

令S＝[S(1),S(2),...,S(N)]^T为一个列向量，则需要测量的OD流数目为L＝||S||₁，||·||₁表示l₁范数。Boolean 'or' operation is performed on each column of the M×N Bernoulli matrix,

Let S=[S(1),S(2),...,S(N)] ^T is a column vector, then the number of OD streams to be measured is L=||S|| ₁ ，||·| | ₁ means the l ₁ norm.

Steps 1-3: Direct measurement of OD flow;

Calculate the mean value of each OD flow of the known historical flow matrix X ₀ :

$\mathrm{<span\; class="notranslate">\; aver</span>}<span\; class="notranslate">\; \_</span>\mathrm{<span\; class="notranslate">\; x</span>}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; no</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; =</span>\underset{\mathrm{<span\; class="notranslate">\; t</span>}<span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; 1</span>}}{\overset{{\mathrm{<span\; class="notranslate">\; T</span>}}_{\mathrm{<span\; class="notranslate">\; 0</span>}}}{\mathrm{<span\; class="notranslate">\; \&Sigma;</span>}}}{\mathrm{<span\; class="notranslate">\; x</span>}}_{\mathrm{<span\; class="notranslate">\; 0</span>}}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; no</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; t</span>}<span\; class="notranslate">\; )</span>\mathrm{<span\; class="notranslate">\; no</span>}<span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; 1,2</span>}<span\; class="notranslate">\; ,</span><span\; class="notranslate">\; .</span><span\; class="notranslate">\; .</span><span\; class="notranslate">\; .</span><span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; N</span>}<span\; class="notranslate">\; -</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; 1</span>}<span\; class="notranslate">\; )</span>$

Among them, T ₀ is the length of the historical traffic matrix, and N is the number of OD flows. According to the mean value, the network management station controls the on/off state of the flow collection function on the router, so as to achieve the purpose of measuring the largest L=||S|| ₁ OD flow. Record the measured L=||S|| ₁ OD flows as a set {x _mea (l)}, l=1,2,...,L;

Step 1-4: Construct flow matrix according to known OD flow {x _mea (l)}, l=1,2,....,L;

Described step 1-4 specifically comprises the following steps:

Step 1-4-1: Generate a Bernoulli random matrix B according to Step 1-1 and Step 1-2, and perform a Boolean 'or' operation on each column;

令l＝1，迭代次数j＝1，最大迭代次数N；Step 1-4-2: Initialize the flow matrix as an empty matrix, namely

Let l=1, the number of iterations j=1, the maximum number of iterations N;

否则， Step 1-4-3: When S(j)=1, the flow matrix becomes

otherwise,

Step 1-4-4: Add 1 to the number of iterations j, if j<N, return to step 1-4-3) until N iterations. Get the traffic matrix X _part ,

Step 1-4-5: The traffic matrix construction is completed.

Described step 3 specifically comprises the following steps:

Step 3-1: Initialize the digital dictionary D ₀ to be empty, that is, D ₀ =[];

Step 3-2: Singular value decomposition is performed on the historical traffic, and K principal components are extracted;

为历史流量的K个主成分，并确定迭代次数iter＝N，令j＝1， Step 3-3: Set up redundant items

are the K principal components of historical traffic, and determine the number of iterations iter=N, let j=1,

Step 3-4: Calculate R _j sparse index ξ _a , and column index a _j corresponding to the sparse index;

${\mathrm{<span\; class="notranslate">\; a</span>}}_{\mathrm{<span\; class="notranslate">\; j</span>}}<span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; arg</span>}\underset{\mathrm{<span\; class="notranslate">\; a</span>}<span\; class="notranslate">\; \&NotElement;</span>{\mathrm{<span\; class="notranslate">\; I</span>}}_{\mathrm{<span\; class="notranslate">\; j</span>}}}{\mathrm{<span\; class="notranslate">\; min</span>}}<span\; class="notranslate">\; \{</span>{\mathrm{<span\; class="notranslate">\; \&xi;</span>}}_{\mathrm{<span\; class="notranslate">\; a</span>}}<span\; class="notranslate">\; =</span>{<span\; class="notranslate">\; |</span><span\; class="notranslate">\; |</span>{\mathrm{<span\; class="notranslate">\; R</span>}}_{\mathrm{<span\; class="notranslate">\; j</span>}}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; a</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; |</span><span\; class="notranslate">\; |</span>}_{\mathrm{<span\; class="notranslate">\; 1</span>}}<span\; class="notranslate">\; /</span>{<span\; class="notranslate">\; |</span><span\; class="notranslate">\; |</span>{\mathrm{<span\; class="notranslate">\; R</span>}}_{\mathrm{<span\; class="notranslate">\; j</span>}}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; a</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; |</span><span\; class="notranslate">\; |</span>}_{\mathrm{<span\; class="notranslate">\; 2</span>}}<span\; class="notranslate">\; \}</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; 5</span>}<span\; class="notranslate">\; )</span>$

Where R _j (a) is the a-th column of the redundant item obtained after the j-th iteration, and ||·|| ₂ represents the l ₂ norm;

Step 3-5: According to step 3-4, get the column R _j (a _j ) with the smallest sparsity index, and unitize the a _jth column with the smallest sparsity index _ξa in the redundancy item, and set it as data The jth column of the dictionary, that is, d _j =R _j (a _j )/||R _j (a _j )|| ₂ ;

Step 3-6: Obtain the data dictionary according to step 3-5: D _j =[D _j-1 |d _j ], I _j =I _j-1 ∪{a _j }; then for all columns a;

Step 3-7: Update redundant items, each column: R _j+1 (a)=R _j (a)-d _j <d _j ,R _j (a)>, where <·> represents the inner product;

Step 3-8: the number of iterations j=j+1, if j<N, return to step 3-4 until N iterations are performed. An optimized greedy adaptive dictionary D is obtained, D=D _j .

Steps 3-9: Optimizing the construction of the greedy adaptive dictionary is completed.

Described step 3-2 comprises the following steps:

Step 3-2-1: Obtain historical traffic and perform singular value decomposition on historical traffic;

The obtained historical traffic matrix is denoted as X _0his . Perform singular value decomposition on X _0his ,

${\mathrm{<span\; class="notranslate">\; x</span>}}_{\mathrm{<span\; class="notranslate">\; 0</span>}\mathrm{<span\; class="notranslate">\; his</span>}}<span\; class="notranslate">\; =</span>\underset{\mathrm{<span\; class="notranslate">\; k</span>}<span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; 1</span>}}{\overset{\mathrm{<span\; class="notranslate">\; N</span>}}{\mathrm{<span\; class="notranslate">\; \&Sigma;</span>}}}{\mathrm{<span\; class="notranslate">\; \&sigma;</span>}}_{\mathrm{<span\; class="notranslate">\; k</span>}}{\mathrm{<span\; class="notranslate">\; u</span>}}_{\mathrm{<span\; class="notranslate">\; k</span>}}{\mathrm{<span\; class="notranslate">\; v</span>}}_{\mathrm{<span\; class="notranslate">\; k</span>}}^{\mathrm{<span\; class="notranslate">\; T</span>}}<span\; class="notranslate">\; -</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; 3</span>}<span\; class="notranslate">\; )</span>$

Among them, σ _k is the singular value, u _k is called the feature flow, and v _k is called the feature vector

Step 3-2-2: Extract the first K large singular values, and set the other small singular values to 0;

Then according to formula (3), we have,

${\mathrm{<span\; class="notranslate">\; x</span>}}_{\mathrm{<span\; class="notranslate">\; 0</span>}\mathrm{<span\; class="notranslate">\; his</span>}}^{\mathrm{<span\; class="notranslate">\; pc</span>}}<span\; class="notranslate">\; =</span>\underset{\mathrm{<span\; class="notranslate">\; k</span>}<span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; 1</span>}}{\overset{\mathrm{<span\; class="notranslate">\; K</span>}}{\mathrm{<span\; class="notranslate">\; \&Sigma;</span>}}}{\mathrm{<span\; class="notranslate">\; \&sigma;</span>}}_{\mathrm{<span\; class="notranslate">\; k</span>}}{\mathrm{<span\; class="notranslate">\; u</span>}}_{\mathrm{<span\; class="notranslate">\; k</span>}}{\mathrm{<span\; class="notranslate">\; v</span>}}_{\mathrm{<span\; class="notranslate">\; k</span>}}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; K</span>}<span\; class="notranslate">\; <</span>\mathrm{<span\; class="notranslate">\; N</span>}<span\; class="notranslate">\; -</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; 4</span>}<span\; class="notranslate">\; )</span>$

为提取了K个最大的奇异值的流量矩阵近似矩阵；

is the flow matrix approximation matrix with the K largest singular values extracted;

Described step 4 specifically comprises the following steps:

Step 4-1: According to the dictionary D obtained in step 3, solve the N×1 column vector through the following l ₁ norm minimization problem

${\stackrel{<span\; class="notranslate">\; ^</span>}{\mathrm{<span\; class="notranslate">\; \&theta;</span>}}}_{\mathrm{<span\; class="notranslate">\; t</span>}}\mathrm{<span\; class="notranslate">\; arg</span>}\mathrm{<span\; class="notranslate">\; min</span>}{<span\; class="notranslate">\; |</span><span\; class="notranslate">\; |</span>{\mathrm{<span\; class="notranslate">\; \&theta;</span>}}_{\mathrm{<span\; class="notranslate">\; t</span>}}<span\; class="notranslate">\; |</span><span\; class="notranslate">\; |</span>}_{\mathrm{<span\; class="notranslate">\; 1</span>}}\mathrm{<span\; class="notranslate">\; the\; s</span>}<span\; class="notranslate">\; .</span>\mathrm{<span\; class="notranslate">\; t</span>}<span\; class="notranslate">\; .</span>\mathrm{<span\; class="notranslate">\; BD</span>}{\mathrm{<span\; class="notranslate">\; \&theta;</span>}}_{\mathrm{<span\; class="notranslate">\; t</span>}}<span\; class="notranslate">\; =</span>{\mathrm{<span\; class="notranslate">\; Y</span>}}_{\mathrm{<span\; class="notranslate">\; t</span>}}<span\; class="notranslate">\; -</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; 6</span>}<span\; class="notranslate">\; )</span>$

Among them, Y _t is the tth column of the observation matrix Y;

Step 4-2: Obtain N×T matrix through T iterations

Step 4-3: Computing Flow Matrix Estimates

Advantages of the present invention:

According to the power-law distribution of power communication network traffic, the present invention uses a random matrix to select part of directly measured OD flows, and operates a flow collection function in part of routers according to requirements to obtain part of the directly measured OD flow flow information. An end-to-end network traffic reconstruction model is constructed based on the measured traffic information, and the compressed sensing reconstruction algorithm is used to solve the model to obtain traffic monitoring values. Utilizing the method of the present invention, the traffic monitoring value can be obtained accurately in real time, and at the same time, the loss of network traffic collection can be effectively reduced, which is beneficial for network researchers to conduct network behavior analysis and for network operators to better perform network management, network monitoring, network design and Network planning and other network traffic engineering management.

Description of drawings

Fig. 1 is a typical power communication network business flow monitoring system block diagram among the present invention;

Fig. 2 is a block diagram of the distributed flow monitoring system proposed by the present invention;

Fig. 3 is the flowchart of constructing new flow matrix of the embodiment of the present invention;

Fig. 4 is the flowchart of constructing the optimal greedy adaptive dictionary of the embodiment of the present invention;

5 is a schematic diagram of reconstructed values and real values of any OD stream CSOR method according to an embodiment of the present invention;

Fig. 6 is a schematic diagram of reconstruction value and real value of any OD stream SRSVD method according to the embodiment of the present invention;

7 is a schematic diagram of the relative root mean square error between the CSOR reconstruction method and the SRSVD method according to the embodiment of the present invention;

Fig. 8 is a schematic diagram of the estimated deviation between the CSOR reconstruction method and the SRSVD method according to the embodiment of the present invention;

9 is a schematic diagram of the standard deviation of the estimated deviation between the CSOR reconstruction method and the SRSVD method according to the embodiment of the present invention;

Fig. 10 is a schematic diagram of the cumulative distribution of the number of OD flows that need to be directly measured by the CSOR reconstruction method according to the embodiment of the present invention.

Detailed ways

The present invention will be further described in detail below in conjunction with the accompanying drawings and specific embodiments.

The simulation experiment carried out to the method of the present invention adopts the data of the backbone network of Abilene in the United States. The Abilene backbone network is mainly used for education and scientific research. It has 12 nodes, 30 internal links, 24 external links, and a total of 144 end-to-end flows. That is, in the embodiment of the present invention, M=54, N= 144. The simulation data is collected through NetFlow with a time interval of 5 minutes, a total of 2016 moments, of which the flow data of the first 500 moments are used as the pre-measured flow matrix, that is, in the embodiment of the present invention, T=2016, T ₀ =500, so that Calculate the mean value of the OD flow and establish the optimal reconstruction model (CSOR) based on compressed sensing, as shown in Figure 2.

The present invention is oriented to the fast distributed monitoring method of electric power communication network business, and the steps are as follows:

Step 1: Generate a Bernoulli matrix, select the OD flow to be measured directly through the matrix, and construct a flow matrix;

Specifically include the following steps:

Step 1-1: Generate Bernoulli matrix;

Generate a 54×144 Bernoulli random matrix B. The elements b(m,n) of the Bernoulli matrix are independent and identically distributed. In this embodiment, Pr=0.01, that is, the probability that the element is equal to 1 is 0.01, and the probability that the element is equal to 0 is 0.99.

Step 1-2: Determine the number of OD flows that need to be measured;

令S＝[S(1),S(2),...,S(144)]^T为一个列向量，则需要测量的OD流数目为L＝||S||₁，||·||₁表示l₁范数。Boolean 'or' operation is performed on each column of the 54×144 Bernoulli matrix,

Let S=[S(1),S(2),...,S(144)] ^T is a column vector, then the number of OD streams to be measured is L=||S|| ₁ ，||·| | ₁ means the l ₁ norm.

Steps 1-3: Direct measurement of OD flow;

Calculate the mean value of each OD flow of the known historical flow matrix X ₀ :

$\mathrm{<span\; class="notranslate">\; aver</span>}<span\; class="notranslate">\; \_</span>\mathrm{<span\; class="notranslate">\; x</span>}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; no</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; =</span>\underset{\mathrm{<span\; class="notranslate">\; t</span>}<span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; 1</span>}}{\overset{{\mathrm{<span\; class="notranslate">\; T</span>}}_{\mathrm{<span\; class="notranslate">\; 0</span>}}}{\mathrm{<span\; class="notranslate">\; \&Sigma;</span>}}}{\mathrm{<span\; class="notranslate">\; x</span>}}_{\mathrm{<span\; class="notranslate">\; 0</span>}}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; no</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; t</span>}<span\; class="notranslate">\; )</span>\mathrm{<span\; class="notranslate">\; no</span>}<span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; 1,2</span>}<span\; class="notranslate">\; ,</span><span\; class="notranslate">\; .</span><span\; class="notranslate">\; .</span><span\; class="notranslate">\; .</span><span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; 144</span>}<span\; class="notranslate">\; -</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; 1</span>}<span\; class="notranslate">\; )</span>$

Wherein, T ₀ is the length of the historical traffic matrix. In the embodiment, we intercept the historical traffic at time 500, that is, T ₀ =500. According to the mean value, the network management station controls the on/off state of the flow collection function on the router, so as to achieve the purpose of measuring the largest L=||S|| ₁ OD flow. The measured L=||S|| ₁ OD flows are recorded as a set {x _mea (l)}, l=1,2,...,L.

Step 1-4: Construct flow matrix according to the known OD flow {x _mea (l)}, l=1, 2, ..., L, as shown in Figure 3, the specific steps are as follows:

Step 1-4-1: Generate a Bernoulli random matrix B according to Step 1-1 and Step 1-2, and perform a Boolean 'or' operation on each column;

Step 1-4-2: Initialize the flow matrix as an empty matrix, namely Let l=1, the number of iterations j=1, and the maximum number of iterations is 144;

Step 1-4-3: When S(j)=1, the flow matrix becomes l=l+1; otherwise,

Step 1-4-4: Add 1 to the number of iterations j, if j<144, return to step 1-4-3 until 144 iterations. Get the traffic matrix X _part ,

Step 1-4-5: The traffic matrix construction is completed.

Step 2: Find the observation matrix;

The construction of the observation matrix in the present invention only depends on the non-zero rows in the flow matrix X _part , and the rows with other zero elements represent unknown OD flows that need to be reconstructed. The method of calculating the observation matrix Y _m is as follows:

Y _m =B·X _part (2) where B and X _part are obtained through Step 1. Then a linear system is formed by B, X _part and observation matrix Y _m .

Step 3: Build an optimized greedy adaptive dictionary, as shown in Figure 4, the specific steps are as follows:

Step 3-1: Initialize the digital dictionary D ₀ to be empty, that is, D ₀ =[];

Step 3-2: Singular value decomposition is performed on the historical traffic, and K principal components are extracted. In the embodiment, K=6, it specifically includes:

Step 3-2-1: Obtain historical traffic, and perform singular value decomposition on historical traffic;

The obtained historical traffic matrix is denoted as X _0his . Perform singular value decomposition on X _0his ,

${\mathrm{<span\; class="notranslate">\; x</span>}}_{\mathrm{<span\; class="notranslate">\; 0</span>}\mathrm{<span\; class="notranslate">\; his</span>}}<span\; class="notranslate">\; =</span>\underset{\mathrm{<span\; class="notranslate">\; k</span>}<span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; 1</span>}}{\overset{\mathrm{<span\; class="notranslate">\; 144</span>}}{\mathrm{<span\; class="notranslate">\; \&Sigma;</span>}}}{\mathrm{<span\; class="notranslate">\; \&sigma;</span>}}_{\mathrm{<span\; class="notranslate">\; k</span>}}{\mathrm{<span\; class="notranslate">\; u</span>}}_{\mathrm{<span\; class="notranslate">\; k</span>}}{\mathrm{<span\; class="notranslate">\; v</span>}}_{\mathrm{<span\; class="notranslate">\; k</span>}}^{\mathrm{<span\; class="notranslate">\; T</span>}}<span\; class="notranslate">\; -</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; 3</span>}<span\; class="notranslate">\; )</span>$

Among them, σ _k is the singular value, u _k is called the feature flow, and v _k is called the feature vector

Step 3-2-2: Extract the first 6 large singular values, and set the other small singular values to 0;

Then according to formula (3), we have,

${\mathrm{<span\; class="notranslate">\; x</span>}}_{\mathrm{<span\; class="notranslate">\; 0</span>}\mathrm{<span\; class="notranslate">\; his</span>}}^{\mathrm{<span\; class="notranslate">\; pc</span>}}<span\; class="notranslate">\; =</span>\underset{\mathrm{<span\; class="notranslate">\; k</span>}<span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; 1</span>}}{\overset{\mathrm{<span\; class="notranslate">\; 6</span>}}{\mathrm{<span\; class="notranslate">\; \&Sigma;</span>}}}{\mathrm{<span\; class="notranslate">\; \&sigma;</span>}}_{\mathrm{<span\; class="notranslate">\; k</span>}}{\mathrm{<span\; class="notranslate">\; u</span>}}_{\mathrm{<span\; class="notranslate">\; k</span>}}{\mathrm{<span\; class="notranslate">\; v</span>}}_{\mathrm{<span\; class="notranslate">\; k</span>}}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; K</span>}<span\; class="notranslate">\; <</span>\mathrm{<span\; class="notranslate">\; N</span>}<span\; class="notranslate">\; -</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; 4</span>}<span\; class="notranslate">\; )</span>$

为提取了6个最大的奇异值的流量矩阵近似矩阵；

is the flow matrix approximation matrix with the 6 largest singular values extracted;

为历史流量的6个主成分，并确定迭代次数iter＝144，令j＝1，

Step 3-3: Set up redundant items

are the 6 principal components of historical traffic, and determine the number of iterations iter=144, let j=1,

Step 3-4: Calculate R _j sparse index ξ _a , and column index a _j corresponding to the sparse index;

${\mathrm{<span\; class="notranslate">\; a</span>}}_{\mathrm{<span\; class="notranslate">\; j</span>}}<span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; arg</span>}\underset{\mathrm{<span\; class="notranslate">\; a</span>}<span\; class="notranslate">\; \&NotElement;</span>{\mathrm{<span\; class="notranslate">\; I</span>}}_{\mathrm{<span\; class="notranslate">\; j</span>}}}{\mathrm{<span\; class="notranslate">\; min</span>}}<span\; class="notranslate">\; \{</span>{\mathrm{<span\; class="notranslate">\; \&xi;</span>}}_{\mathrm{<span\; class="notranslate">\; a</span>}}<span\; class="notranslate">\; =</span>{<span\; class="notranslate">\; |</span><span\; class="notranslate">\; |</span>{\mathrm{<span\; class="notranslate">\; R</span>}}_{\mathrm{<span\; class="notranslate">\; j</span>}}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; a</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; |</span><span\; class="notranslate">\; |</span>}_{\mathrm{<span\; class="notranslate">\; 1</span>}}<span\; class="notranslate">\; /</span>{<span\; class="notranslate">\; |</span><span\; class="notranslate">\; |</span>{\mathrm{<span\; class="notranslate">\; R</span>}}_{\mathrm{<span\; class="notranslate">\; j</span>}}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; a</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; |</span><span\; class="notranslate">\; |</span>}_{\mathrm{<span\; class="notranslate">\; 2</span>}}<span\; class="notranslate">\; \}</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; 5</span>}<span\; class="notranslate">\; )</span>$

Where R _j (a) is the a-th column of the redundant item obtained after the j-th iteration, and ||·|| ₂ represents the l ₂ norm;

Step 3-5: According to step 3-4, get the column R _j (a _j ) with the smallest sparsity index, and unitize the a _jth column with the smallest sparsity index _ξa in the redundancy item, and set it as data The jth column of the dictionary, that is, d _j =R _j (a _j )/||R _j (a _j )|| ₂ ;

Step 3-6: Obtain the data dictionary according to step 3-5: D _j =[D _j-1 |d _j ], I _j =I _j-1 ∪{a _j }; then for all columns a;

Step 3-7: Update redundant items, each column: R _j+1 (a)=R _j (a)-d _j <d _j ,R _j (a)>, where <·> represents the inner product;

Step 3-8: the number of iterations j=j+1, if j<144, return to step 3-4 until 144 iterations are performed. An optimized greedy adaptive dictionary D is obtained, D=D _j .

Steps 3-9: Optimizing the construction of the greedy adaptive dictionary is completed.

Step 4: Reconstruct the traffic matrix by compressed sensing;

Specific steps are as follows:

Step 4-1: According to the dictionary D obtained in step 3, the column vector of NX1 is solved by the following l ₁ norm minimization problem

${\stackrel{<span\; class="notranslate">\; ^</span>}{\mathrm{<span\; class="notranslate">\; \&theta;</span>}}}_{\mathrm{<span\; class="notranslate">\; t</span>}}\mathrm{<span\; class="notranslate">\; arg</span>}\mathrm{<span\; class="notranslate">\; min</span>}{<span\; class="notranslate">\; |</span><span\; class="notranslate">\; |</span>{\mathrm{<span\; class="notranslate">\; \&theta;</span>}}_{\mathrm{<span\; class="notranslate">\; t</span>}}<span\; class="notranslate">\; |</span><span\; class="notranslate">\; |</span>}_{\mathrm{<span\; class="notranslate">\; 1</span>}}\mathrm{<span\; class="notranslate">\; the\; s</span>}<span\; class="notranslate">\; .</span>\mathrm{<span\; class="notranslate">\; t</span>}<span\; class="notranslate">\; .</span>\mathrm{<span\; class="notranslate">\; BD</span>}{\mathrm{<span\; class="notranslate">\; \&theta;</span>}}_{\mathrm{<span\; class="notranslate">\; t</span>}}<span\; class="notranslate">\; =</span>{\mathrm{<span\; class="notranslate">\; Y</span>}}_{\mathrm{<span\; class="notranslate">\; t</span>}}<span\; class="notranslate">\; -</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; 6</span>}<span\; class="notranslate">\; )</span>$

where Y _t is the tth column of the observation matrix Y.

Step 4-2: Obtain a 144×2016 matrix through T=2016 iterations

Step 4-3: Computing Flow Matrix Estimates

In order to better evaluate the reconstruction performance of the present invention, we calculate the relative root mean square error, reconstruction bias and bias standard deviation.

$\mathrm{<span\; class="notranslate">\; RRMSE</span>}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; t</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; =</span>\frac{\sqrt{\underset{\mathrm{<span\; class="notranslate">\; no</span>}<span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; 1</span>}}{\overset{\mathrm{<span\; class="notranslate">\; N</span>}}{\mathrm{<span\; class="notranslate">\; \&Sigma;</span>}}}{<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; x</span>}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; no</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; t</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; -</span>\stackrel{<span\; class="notranslate">\; ^</span>}{\mathrm{<span\; class="notranslate">\; x</span>}}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; no</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; t</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; )</span>}^{\mathrm{<span\; class="notranslate">\; 2</span>}}}}{\sqrt{\underset{\mathrm{<span\; class="notranslate">\; no</span>}<span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; 1</span>}}{\overset{\mathrm{<span\; class="notranslate">\; N</span>}}{\mathrm{<span\; class="notranslate">\; \&Sigma;</span>}}}\mathrm{<span\; class="notranslate">\; x</span>}{<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; no</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; t</span>}<span\; class="notranslate">\; )</span>}^{\mathrm{<span\; class="notranslate">\; 2</span>}}}}<span\; class="notranslate">\; -</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; 7</span>}<span\; class="notranslate">\; )</span>$

$\mathrm{<span\; class="notranslate">\; bias</span>}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; no</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; =</span>\frac{\mathrm{<span\; class="notranslate">\; 1</span>}}{\mathrm{<span\; class="notranslate">\; T</span>}}\underset{\mathrm{<span\; class="notranslate">\; t</span>}<span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; 1</span>}}{\overset{\mathrm{<span\; class="notranslate">\; T</span>}}{\mathrm{<span\; class="notranslate">\; \&Sigma;</span>}}}<span\; class="notranslate">\; (</span>\stackrel{<span\; class="notranslate">\; ^</span>}{\mathrm{<span\; class="notranslate">\; x</span>}}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; no</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; t</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; -</span>\mathrm{<span\; class="notranslate">\; x</span>}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; no</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; t</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; )</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; 8</span>}<span\; class="notranslate">\; )</span>$

$\mathrm{<span\; class="notranslate">\; SD</span>}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; no</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; =</span>\sqrt{\frac{\mathrm{<span\; class="notranslate">\; 1</span>}}{\mathrm{<span\; class="notranslate">\; T</span>}<span\; class="notranslate">\; -</span>\mathrm{<span\; class="notranslate">\; 1</span>}}\underset{\mathrm{<span\; class="notranslate">\; t</span>}<span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; 1</span>}}{\overset{\mathrm{<span\; class="notranslate">\; T</span>}}{\mathrm{<span\; class="notranslate">\; \&Sigma;</span>}}}{<span\; class="notranslate">\; (</span><span\; class="notranslate">\; (</span>\stackrel{<span\; class="notranslate">\; ^</span>}{\mathrm{<span\; class="notranslate">\; x</span>}}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; no</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; t</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; -</span>\mathrm{<span\; class="notranslate">\; x</span>}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; no</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; t</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; )</span><span\; class="notranslate">\; -</span>\mathrm{<span\; class="notranslate">\; bisa</span>}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; no</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; )</span>}^{\mathrm{<span\; class="notranslate">\; 2</span>}}}<span\; class="notranslate">\; -</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; 9</span>}<span\; class="notranslate">\; )</span>$

In the simulation experiment, the proposed algorithm (CSOR) of the present invention is compared with the Sparse Regularized Singular Value Decomposition (SRSVD) algorithm, and the results show that the proposed algorithm of the present invention can accurately reconstruct end-to-end network traffic. Figure 5 is the reconstruction result of the 20th OD stream by the CSOR algorithm, and Figure 6 is the reconstruction result of the 20th OD stream by the SRSVD algorithm. Among them, the solid line represents the real traffic, and the dotted line represents its reconstructed value. It can be seen from the figure that although the CSOR reconstruction method has more or less overestimation and underestimation phenomena, CSOR can track the dynamic change trend of the OD flow very well, and is very close to the true value of the OD flow. Figure 7 shows the relative root mean square error of the CSOR and SRSVD algorithms. The solid line is the CSOR algorithm, and the dashed line is the SRSVD algorithm. It can be seen from the figure that the relative root mean square error of the method CSOR of the present invention is small. To further evaluate the proposed method in this invention, we introduce reconstruction bias and bias standard deviation. It can be seen from Fig. 8 and Fig. 9 that the reconstruction deviation of the CSOR method is small, and the standard deviation is also small, which shows that the method proposed by the present invention has strong stability. Next, evaluate the economic benefits of CSOR. Figure 10 shows the cumulative distribution function of the number of OD flows that we need to measure directly. It can be seen that in about 96% of cases, we only need to measure 70 OD flows, that is to say, we only need Measuring 49% of the OD streams provides monitoring results for all 144 OD streams.

Claims (6)

1. towards the fast distributed monitoring method of power communication network service, it is characterized in that comprising the steps:

Step 1: generate Bernoulli matrix, select to need the OD directly measuring to flow by this matrix, and build traffic matrix;

Step 2: obtain observing matrix;

The structure of observing matrix depends on traffic matrix X _partin non-zero capable, the row of other neutral element represent that the unknown OD that needs reconstruct flows.Calculating observation matrix Y _mmethod is as follows:

Y _m＝B·X _part （2）

Wherein, B is the Bernoulli matrix of M * N, X _partit is the traffic matrix of N * T.By B, X _partwith observing matrix Y _mformed a linear system;

Step 3: build and optimize greedy self-adapting dictionary;

Step 4: by compressed sensing reconstruct traffic matrix.

2. the fast distributed monitoring method towards power communication network service according to claim 1, is characterized in that described step 1 specifically comprises the steps:

Step 1-1: generate Bernoulli matrix;

Generate the Bernoulli random matrix B of M * N (M < N), N is the number of OD stream in network, its equal network node quantity square; Bernoulli entry of a matrix element b (m, n) is independent identically distributed, and it is Pr that element equals 1 probability, and equaling 0 probability is 1-Pr;

Step 1-2: determine the number that needs the OD of measurement stream;

Each row of the Bernoulli matrix of M * N are carried out respectively to boolean's ' or ' computing,

make S=[S (1), S (2) ..., S (N)] ^tbe a column vector, needing the OD flow amount of measuring is L=||S|| ₁, || || ₁represent l ₁norm;

Step 1-3: directly measure OD stream;

Calculate known historical traffic matrix X ₀the average of every OD stream:

$\mathrm{aver}\_X\left(n\right)=\underset{t=1}{\overset{T_0}{\mathrm{\&Sigma;}}}{X}_0(n,t)n=\mathrm{1,2},...,N---\left(1\right)$

Wherein, T ₀for the length of historical traffic matrix, N is OD flow amount, and according to this average, Network Management Station is controlled the on off state of flow acquisition function on router, to reach, measures maximum L=||S|| ₁the object of individual OD stream, by the L=||S|| having measured ₁individual OD stream is designated as set { x _mea(l) }, l=1,2 ..., L;

Step 1-4: according to known OD stream { x _mea(l) }, l=1,2 ...., L builds traffic matrix.

3. the fast distributed monitoring method towards power communication network service according to claim 2, is characterized in that described step 1-4 specifically comprises the steps:

Step 1-4-1: generate Bernoulli random matrix B according to step 1-1, step 1-2, and its each row are carried out respectively to boolean's ' or ' computing;

Step 1-4-2: initialization traffic matrix is empty matrix,

make l=1, iterations j=1, maximum iteration time N;

Step 1-4-3: when S (j)=1, traffic matrix becomes

; Otherwise,

Step 1-4-4: iterations j adds 1, if j < is N, returns to step 1-4-3, until iteration N time obtains traffic matrix X _part,

Step 1-4-5: traffic matrix builds and finishes.

4. the fast distributed monitoring method towards power communication network service according to claim 1, is characterized in that described step 3 specifically comprises the steps:

Step 3-1: initialization numeral dictionary D ₀for sky, i.e. D ₀=[];

Step 3-2: historical flow is carried out to singular value decomposition, and extract K principal component;

Step 3-3: redundancy is set

for K principal component of historical flow, and definite iterations iter=N, make j=1,

Step 3-4: calculate R _jsparse index ξ _a, and column index a corresponding to sparse index _j;

$a_j=\arg \underset{a\&NotElement;I_j}{\min }\{{\mathrm{\&xi;}}_a={\left|\right|R_j\left(a\right)\left|\right|}_1/{\left|\right|R_j\left(a\right)\left|\right|}_2\}---\left(5\right)$ R wherein _j(a) be a row of the redundancy that obtains after the j time iteration, || || ₂represent l ₂norm;

Step 3-5: the row R that obtains having minimum sparse index according to step 3-4 _j(a _j), and will in redundancy, there is minimum sparse index ξ _aa _jthe positionization of itemizing, and it is made as to the j row of data dictionary, i.e. d _j=R _j(a _j)/|| R _j(a _j) || ₂;

Step 3-6: obtaining data dictionary according to step 3-5 is: D _j=[D _j-1| d _j], I _j=I _j-1∪ { a _j; Then to all row a;

Step 3-7: upgrade redundancy, each row: R _j+1(a)=R _j(a)-d _j< d _j, R _j(a) >, wherein < > represents inner product;

Step 3-8: iterations j=j+1, if j < is N, returns to step 3-4, until carry out iteration N time, the greedy self-adapting dictionary D that is optimized, D=D _j;

Step 3-9: optimize greedy self-adapting dictionary and built.

5. the fast distributed monitoring method towards power communication network service according to claim 4, is characterized in that described step 3-2 comprises the steps:

Step 3-2-1: obtain historical flow, historical flow is carried out to singular value decomposition;

The historical traffic matrix obtaining is designated as X _0his.To X _0hiscarry out singular value decomposition,

$X_{0\mathrm{his}}=\underset{k=1}{\overset{N}{\mathrm{\&Sigma;}}}{\mathrm{\&sigma;}}_k{u}_k{v}_k^T---\left(3\right)$

σ wherein _kfor singular value, u _kbe called feature stream, v _kbe called characteristic vector

Step 3-2-2: K large singular value before extracting, other little singular values are set to 0;

According to formula (3), have,

$X_{0\mathrm{his}}^{\mathrm{pc}}=\underset{k=1}{\overset{K}{\mathrm{\&Sigma;}}}{\mathrm{\&sigma;}}_k{u}_k{v}_k,K<N---\left(4\right)$

for having extracted the traffic matrix approximate matrix of K maximum singular value.

6. the fast distributed monitoring method towards power communication network service according to claim 1, is characterized in that described step 4 comprises the steps:

Step 4-1: the dictionary D obtaining according to step 3, by following l ₁the column vector of Norm minimum problem solving N * 1

${\widehat{\mathrm{\&theta;}}}_t\arg \min {\left|\right|{\mathrm{\&theta;}}_t\left|\right|}_{1}s.t.\mathrm{BD}{\mathrm{\&theta;}}_t=Y_t---\left(6\right)$

Wherein, Y _tthe t row of observing matrix Y;

Step 4-2: by T iteration, obtain N * T matrix

Step 4-3: calculated flow rate Matrix Estimation value

Images