CN106789262A - A kind of complex network community method for detecting abnormality - Google Patents

Abstract

A kind of a kind of complex network community method for detecting abnormality, it is proposed that complex network community method for detecting abnormality based on HMM.Positive research shows that most real network not only has uncalibrated visual servo and small world, also with corporations' characteristic.The ANOMALOUS VARIATIONS that quick detection network occurs in evolution process, complex network is applied has very real meaning in various fields.The thought that communication of the present invention based on the overwhelming majority in network occurs all in corporations, in units of corporations, builds HMM, propose a kind of large complicated network community method for detecting abnormality, first, using complex network community probe algorithm, different corporations are splitted the network into；Then in units of corporations, build HMM and train relevant parameter；The entropy of monitored sequence, the abnormal corporations of positioning are finally calculated using the model for training.

Description

A kind of complex network community method for detecting abnormality

Technical field

The invention belongs to the technical field of computer software, it is related to a kind of network detecting method.

Background technology

The scale-free model of Barab á si and Albert and the small-world network model of Watts and Strogatz are disclosed The essential laws of network structure, in past nearly 20 years, have promoted what complex network studied to develop rapidly.Further grind Study carefully discovery, most reality networks are uneven, are made up of many sub-networks.Connection between sub-network interior nodes is tighter It is close, and the connection of the intermediate node of subnet is than sparse, this phenomenon is all relatively common in artificial network and natural network, referred to as It is the community structure (community structure) in network.Community structure become after worldlet and uncalibrated visual servo characteristic it One of most universal and most important topological structure attribute in complex network afterwards.With the continuous maturation of Complex Networks Theory, research Person is to many complicated interconnection systems, including the various networks such as community network, Internet and World Wide Web (WWW), All go to study its statistical nature and practical application from the visual angle of complex network.Modern society is in big data, big flow In the epoch, it is highly dependent on the normally and efficiently operation of these network systems.How rapidly to detect these networks in the process for developing In whether there is exception, cause the extensive concern of researchers, current method is mainly from network entirety, by than Relatively adjacent to the network change amount of time slot, dependent thresholds are set, judge whether network is abnormal, and the deficiency of these methods essentially consists in net Network is too big, it is impossible to real-time judge, and threshold value sets difficulty greatly, and after detecting exception, positioning abnormal ranges are also extremely difficult.Therefore, The present invention has the thought that the communication of the characteristic and the overwhelming majority of community structure occurs all in corporations based on network, is with corporations Unit, builds HMM, it is proposed that a kind of large complicated network community method for detecting abnormality, first, using complexity Network community probe algorithm, splits the network into different corporations；Then in units of corporations, HMM is built simultaneously Training relevant parameter；The entropy of monitored sequence, the abnormal corporations of positioning are finally calculated using the model for training.

The content of the invention

The present invention has the thought that the communication of the characteristic and the overwhelming majority of community structure occurs all in corporations based on network, In units of corporations, HMM is built, it is proposed that a kind of large complicated network community method for detecting abnormality.Divide below Technical method proposed by the present invention is not illustrated in terms of corporations and HMM etc..

First, corporations：Inside same corporations, relative close is connected between node, and between node between corporations Connection is relatively sparse, with modularity in networkMinimum principle is divided and splits the network into different societies Group, wherein Nc represents the number of corporations in network, and M represents the sum connected in network, m_cRepresent the company between corporations' c interior nodes Connect number, d_cRepresent all node number of degrees sums in corporations c.

2nd, the construction method of corporations' traffic model

(1) basic definition：

Observation is the sequence node of the generation traffic in corporations c, is expressed asWhereinRepresent in t The node that moment corporations c communicates with other corporations, especially, if two nodes of communication are all in corporations c, only considers hair Send the node of data.Observation space is：V=1,2 ..., N }.

State value is the corporations being connected with corporations c with t, is expressed as y=y₁,y₂,...y_T, state value space is S= {1,2,...,M}。

The parameter model of corporations' traffic model is expressed as：θ={ π, A, B }, wherein, π is general for the original state of initial model Rate, A is state transition probability, and B is observation probability.

(2) parameter estimation techniques of the corporations' traffic model based on forward-backward algorithm algorithm

Corporations' traffic model parameter Estimation task of corporations c is that the sequence of observations by collecting estimates corresponding hidden half The parameter of Markov model.The present invention is asked using the parameter Estimation that famous forward-backward algorithm algorithm solves corporations' traffic model Topic, it is described in detail below.

1) forward-backward algorithm variable is defined：

α_t(j)=P [S_t=j, o_1:t|θ]

β_t(j)=P [o_t+1:T|S_t=j, θ]

2) initialization of forward-backward algorithm algorithm：

α₁(j)=π_j,

β_T(j)=1.

3) iteration derivation：

4) intermediate variable is calculated：

ξ_t(i, j)=P [S_t=i, S_t+1=j, o₁:_T| λ]=α_t(i)a_ijb_j(o_t+1)β_t+1(j)

5) parameter more new formula

Wherein, o is worked as_t=v_kWhen, I (o_t=v_k)=1, otherwise I (o_t=v_k)=0.

(3) network community method for detecting abnormality

The entropy of calculating observation sequence：

The standard variance for calculating the entropy of the observation sequence under normal condition is σ₀, average is μ₀,

During abnormality detection, the average that the entropy of monitoring data sequent is calculated first is μ, then with | μ-μ₀| it is abnormality detection amount, if | μ-μ₀|≥3σ₀, then it is abnormality.

Brief description of the drawings

Fig. 1 is the corporations' Network anomaly detection model schematic based on hidden Markov model.

Specific embodiment

Flow is as shown in figure 1, step is as follows：

Step 1：Training data is pre-processed, the training dataset of generation corporations communication sequence sequence；

Step 2：The parameter of model is estimated using forward-backward algorithm algorithm；

Step 3：Collection real-time network communication sequence；

Step 4：The entropy of monitored sequence is calculated using the model for training；

Step 5：Calculate abnormality detection amount | μ-μ₀|；

Step 6：Judge | μ-μ₀|≥3σ₀Whether set up, it is otherwise no abnormal if set up, then it represents that abnormal.

Claims (2)

1. a kind of complex network community method for detecting abnormality, it is characterized in that：

First, corporations：Inside same corporations, relative close is connected between node, and the connection between node between corporations It is relatively sparse, with modularity in networkMinimum principle is divided and splits the network into different corporations, its Middle Nc represents the number of corporations in network, and M represents the sum connected in network, m_cThe connection number between corporations' c interior nodes is represented, d_cRepresent all node number of degrees sums in corporations c；

2nd, the construction method of corporations' traffic model

(1) basic definition：

Observation is the sequence node of the generation traffic in corporations c, is expressed asWhereinRepresent in t The node that corporations c communicates with other corporations, especially, if two nodes of communication are all in corporations c, only considers to send number According to node.Observation space is：V=1,2 ..., N }；

State value is the corporations being connected with corporations c with t, is expressed as y=y₁,y₂,...y_T, state value space be S=1, 2,...,M}；

The parameter model of corporations' traffic model is expressed as：θ={ π, A, B }, wherein, π is the initial state probabilities of initial model, A It is state transition probability, B is observation probability；

(2) parameter estimation techniques of the corporations' traffic model based on forward-backward algorithm algorithm

Corporations' traffic model parameter Estimation task of corporations c is that the sequence of observations by collecting estimates corresponding hidden half Ma Er Can husband's model parameter.The present invention solves the Parameter Estimation Problem of corporations' traffic model, tool using famous forward-backward algorithm algorithm Body is as described below.

1) forward-backward algorithm variable is defined：

α_t(j)=P [S_t=j, o_1:t|θ]

β_t(j)=P [o_t+1:T|S_t=j, θ]

2) initialization of forward-backward algorithm algorithm：

α₁(j)=π_j,

β_T(j)=1.

3) iteration derivation：

${\mathrm{\α}}_t\left(j\right)=\underset{i\∈S\backslash \left\{j\right\}}{\Σ}{\mathrm{\α}}_{t-1}\left(i\right)a_{ij}{b}_j\left(o_t\right),$

${\mathrm{\β}}_t\left(j\right)=\underset{i\∈S\backslash \left\{j\right\}}{\Σ}{a}_{ji}{b}_i\left(o_{t+1:}\right){\mathrm{\β}}_{t+1}\left(i\right);$

4) intermediate variable is calculated：

ξ_t(i, j)=P [S_t=i, S_t+1=j, o₁:_T| λ]=α_t(i)a_ijb_j(o_t+1)β_t+1(j)

${\mathrm{\γ}}_t\left(j\right)=P\[S_t=j,o_{1:T}|\mathrm{\λ}\]=\underset{j}{\Σ}{\mathrm{\ξ}}_t(i,j)$

5) parameter more new formula

${\widehat{\mathrm{\π}}}_j={\mathrm{\γ}}_1\left(j\right)$

${\hat{a}}_{ij}=\frac{\underset{t=1}{\overset{T-1}{\Σ}}{\mathrm{\ξ}}_t(i,j)}{\underset{t=1}{\overset{T-1}{\Σ}}{\mathrm{\γ}}_t\left(j\right)}$

${\hat{b}}_j\left(v_k\right)=\frac{\underset{t=1}{\overset{T}{\Σ}}{\mathrm{\γ}}_t\left(j\right)I(o_t=v_k)}{\underset{t}{\Σ}{\mathrm{\γ}}_t\left(j\right)}$

Wherein, o is worked as_t=v_kWhen, I (o_t=v_k)=1, otherwise I (o_t=v_k)=0；

(3) network community method for detecting abnormality

The entropy of calculating observation sequence：

$P\[o_{1:T}|\mathrm{\λ}\]=\underset{j}{\Σ}{\mathrm{\α}}_t\left(j\right)$

The standard variance for calculating the entropy of the observation sequence under normal condition is σ₀, average is μ₀,

During abnormality detection, the average that the entropy of monitoring data sequent is calculated first is μ, then with | μ-μ₀| it is abnormality detection amount, if | μ-μ₀| ≥3σ₀, then it is abnormality.

2. complex network community method for detecting abnormality according to claim 1, it is characterized in that flow is：

Step 1：Training data is pre-processed, the training dataset of generation corporations communication sequence sequence；

Step 2：The parameter of model is estimated using forward-backward algorithm algorithm；

Step 3：Collection real-time network communication sequence；

Step 4：The entropy of monitored sequence is calculated using the model for training；

Step 5：Calculate abnormality detection amount | μ-μ₀|；

Step 6：Judge | μ-μ₀|≥3σ₀Whether set up, it is otherwise no abnormal if set up, then it represents that abnormal.