CN111786935B - Service flow abnormity detection method for optical cable fiber core remote intelligent scheduling exchange - Google Patents

Service flow abnormity detection method for optical cable fiber core remote intelligent scheduling exchange Download PDF

Info

Publication number
CN111786935B
CN111786935B CN201911090284.7A CN201911090284A CN111786935B CN 111786935 B CN111786935 B CN 111786935B CN 201911090284 A CN201911090284 A CN 201911090284A CN 111786935 B CN111786935 B CN 111786935B
Authority
CN
China
Prior art keywords
factor
flow
network traffic
network
function
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
CN201911090284.7A
Other languages
Chinese (zh)
Other versions
CN111786935A (en
Inventor
李欢
孟凡博
刘扬
陈得丰
卢盛阳
王东东
杨智斌
耿洪碧
任帅
南洋
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
State Grid Corp of China SGCC
Electric Power Research Institute of State Grid Liaoning Electric Power Co Ltd
Original Assignee
State Grid Corp of China SGCC
Electric Power Research Institute of State Grid Liaoning Electric Power Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by State Grid Corp of China SGCC, Electric Power Research Institute of State Grid Liaoning Electric Power Co Ltd filed Critical State Grid Corp of China SGCC
Priority to CN201911090284.7A priority Critical patent/CN111786935B/en
Publication of CN111786935A publication Critical patent/CN111786935A/en
Application granted granted Critical
Publication of CN111786935B publication Critical patent/CN111786935B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L63/00—Network architectures or network communication protocols for network security
    • H04L63/14—Network architectures or network communication protocols for network security for detecting or protecting against malicious traffic
    • H04L63/1408—Network architectures or network communication protocols for network security for detecting or protecting against malicious traffic by monitoring network traffic
    • H04L63/1425—Traffic logging, e.g. anomaly detection
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04B—TRANSMISSION
    • H04B10/00—Transmission systems employing electromagnetic waves other than radio-waves, e.g. infrared, visible or ultraviolet light, or employing corpuscular radiation, e.g. quantum communication
    • H04B10/80—Optical aspects relating to the use of optical transmission for specific applications, not provided for in groups H04B10/03 - H04B10/70, e.g. optical power feeding or optical transmission through water
    • H04B10/85—Protection from unauthorised access, e.g. eavesdrop protection

Landscapes

  • Engineering & Computer Science (AREA)
  • Computer Security & Cryptography (AREA)
  • Computer Networks & Wireless Communication (AREA)
  • Signal Processing (AREA)
  • Physics & Mathematics (AREA)
  • Electromagnetism (AREA)
  • Computer Hardware Design (AREA)
  • Computing Systems (AREA)
  • General Engineering & Computer Science (AREA)
  • Data Exchanges In Wide-Area Networks (AREA)

Abstract

The invention belongs to the network anomaly detection of an optical fiber communication access network, and particularly relates to a service flow anomaly detection method for optical fiber core remote intelligent scheduling exchange. Including giving network traffic
Figure DDA0002266645800000011
And the number of the public factors k, and constructing a random matrix Y; normalizing the original data to obtain a normalized data matrix R (R ═ cov (Y)), obtaining an eigenvalue and an eigenvector of the matrix R, calculating variance and accumulated variance contribution rate, and determining a common factor Y in network flowcAnd special factor Y in network traffics(ii) a After the public factor is solved, the factor is rotated to obtain a main factor meeting proper rotation; establishing a factor analysis model, and evaluating the state of each sample in the whole model; obtaining the score of each factor to obtain the flow
Figure DDA0002266645800000012
Middle common factor
Figure DDA0002266645800000013
Another time series with a particular factor of the network traffic
Figure DDA0002266645800000014
Obtaining another time series
Figure DDA0002266645800000015
Set of characteristic functions of (a): extracting characteristics; and (5) filtering the characteristics, finding out the abnormal flow part and storing the result. The method and the device are used for detecting the abnormal components in the network service flow, and have higher accuracy and timeliness.

Description

Service flow abnormity detection method for optical cable fiber core remote intelligent scheduling exchange
Technical Field
The invention belongs to the network anomaly detection of an optical fiber communication access network, in particular to a service flow anomaly detection method for optical fiber core remote intelligent scheduling exchange, and particularly relates to an anomaly detection method for network service flow.
Background
With the rapid development of new generation network technologies, network applications in the optical fiber communication access network present new types of traffic and cause rapid growth of network traffic, and it follows that new traffic flow in the optical fiber communication access network is abnormal. Traffic anomalies in fiber optic communications affect network performance and user quality of experience. How to effectively detect and find anomalies in network traffic has become a major challenge. More importantly, an anomaly in traffic flow implies an abnormal operation of the user or network device. If abnormal traffic is detected, the operator can effectively implement active defense of the network. Therefore, the detection of network traffic anomaly is of great significance in current network operation, and has become a very important research topic, and has received extensive attention from both academic and industrial circles.
With the development of information technology, the concealment of network traffic anomaly is stronger and stronger. From relatively large and constantly changing normal flows, a relatively small abnormal flow is detected, like a large sea fishing needle, and a new detection technology, method and mechanism are required.
The difficulty of flow anomaly detection is mainly reflected in the accuracy of anomaly detection time. How to highlight the characteristics of network traffic anomalies has been widely studied, and various methods have been proposed. A time-frequency domain method is proposed to find abnormal components in network traffic, and the method is relatively accurate in detecting abnormal network traffic. Wei Xiong describes changes in network traffic states through a collaborative neural network and mutation theory to detect anomalies in the degree of state deviation. And the Yang Yue utilizes a butterfly mutation series model to model the network flow according to the nonlinear dynamic characteristics of the network flow, and detects the mutation of the flow through the jump of mutation series. Jianren Lin et al use a cusp mutation model to model normal and abnormal data of traffic to achieve a certain effect, but the attribute statistical characteristic parameters of the model cannot effectively characterize the traffic characteristics. Thottan uses a statistical distribution of individual MIB variables to detect sudden changes in network traffic. Among the various anomaly statistical detection techniques, entropy-based methods have proven accuracy and efficiency in detecting anomalous traffic matrix time series. Zhang navigation and the like establish an anomaly detection method based on behaviors by utilizing the maximum value and the relative entropy. The baseline distribution based on maximum entropy is made up of pre-labeled training data, but the mechanism by which the baseline accommodates network traffic dynamics is still unclear. A jiang also proposes to use spectral kurtosis analysis (spectral kurtosis analysis) to analyze and identify abnormal network traffic. A jiang also proposed to characterize network traffic using compressive sensing theory, which motivated us to use signal processing techniques to look for traffic anomalies.
Disclosure of Invention
Aiming at the defects in the prior art, the invention provides a service flow anomaly detection method facing optical cable fiber core remote intelligent scheduling exchange, aiming at capturing service flow anomaly components in an optical fiber communication access network by a new rapid detection method, combining a factor analysis method and an empirical mode decomposition method and having higher timeliness and accuracy.
Based on the above purpose, the invention is realized by the following technical scheme:
a service flow abnormity detection method facing optical cable fiber core remote intelligent scheduling exchange comprises the following steps:
step 1: giving network traffic
Figure BDA0002266645780000021
And the number of the common factors k, constructing a random matrix Y;
step 2: normalizing the original data to obtain a correlation matrix R (R ═ cov (Y)) of the normalized data, obtaining an eigenvalue and an eigenvector of the matrix R, calculating the contribution rate of variance and the contribution rate of accumulated variance, and determining a common factor Y in the network flowcAnd special factor Y in network traffics；
And step 3: after the public factor solution is obtained, factor rotation is carried out, and a main factor meeting proper rotation is obtained;
and 4, step 4: establishing a factor analysis model for the obtained main factors, and evaluating the state of each sample in the whole model by using the factor analysis model; obtaining the score of each factor by adopting a regression estimation method, a Batterest estimation method and a Thomson estimation method to obtain the flow
Figure BDA0002266645780000022
Common factor in
Figure BDA0002266645780000023
Another time series of special factors with network traffic
Figure BDA0002266645780000024
And 5: respectively obtained by empirical mode decompositionFlow rate
Figure BDA0002266645780000025
Common factor in
Figure BDA0002266645780000026
Another time series of special factors with network traffic
Figure BDA0002266645780000027
Set of characteristic functions of (a):
step 6: according to gc(t) and hs(t) pairs
Figure BDA0002266645780000028
And
Figure BDA0002266645780000029
carrying out feature extraction;
and 7: through gc(t) and hs(t) filtering the features, finding out abnormal flow parts and storing the results.
The step 5: obtaining another time series of common factors in the traffic and special factors of the network traffic respectively by using empirical mode decomposition method
Figure BDA00022666457800000210
The set of feature functions of (a) is:
gc(t)＝{g1,c(t),g2,c(t) } and hs(t)＝{h1,s(t),h2,s(t),...}；
Wherein, { gi,c(t) } flow rate after decomposition by empirical mode decomposition
Figure BDA0002266645780000031
Common factor in
Figure BDA0002266645780000032
Characteristic function component of { h }i,s(t) } flow rate after decomposition by empirical mode decomposition
Figure BDA0002266645780000033
Specific factor in
Figure BDA0002266645780000034
The characteristic function component of (2).
Step 1, constructing the random matrix Y, wherein the method comprises the following steps:
taking the network traffic as a time sequence, representing the change of the network traffic along with time by y (t) | t ═ 1,2
Figure BDA0002266645780000035
Where n is an integer, the following random matrix is obtained:
Y＝{yi}n×1＝{y(1),y(2),...,y(n)} (1)
wherein, yi(i ═ 1,2, …, n) is the dominant random vector whose mean vector e (y) is 0.
Step 2, determining the common factor Y in the network flowcAnd special factor Y in network trafficsThe determination method comprises the following steps:
according to the factorial theory, Y is decomposed into the following equation:
Figure BDA0002266645780000036
wherein, Yci(i ═ 1,2, …, p, and p ≦ n) is an implicit random vector whose mean vector E (Y) isc)＝0(Yc＝{Yc1,Yc2,...,Ycp}), covariance matrix Cov (Y)c) 1, represents YciAre independent of one another, Ysj(j-1, 2, …, n) is a complementary random vector in factorization, Ysj(j ═ 1,2, …, n) and Yci(i ═ 1,2, …, p) (p ≦ n) independently of one another, equation E (Y)s)＝0(Ys＝{Ys1,Ys2,...,Ysn}) are true, and YsThe factors in (a) are independent of each otherij( i 1,2, …, n, j 1,2, …, p and p ≦ n)Representing an implicit random vector YciThe coefficient of (a).
Flow rate in step 4
Figure BDA0002266645780000041
Common factor in
Figure BDA0002266645780000042
Another time series of special factors with network traffic
Figure BDA0002266645780000043
The determination method comprises the following steps:
the k most important common factors are chosen, as follows:
Y＝{yi}n×1＝A·Yc+Ys (3)
where Y is the currently obtained network traffic matrix, { Yi}n×1To express a random matrix of Y, YcBeing a common factor in network traffic, YsA is a factor load matrix, which is a special factor in the network traffic.
From said equations (2) - (3), a new time series is obtained:
Figure BDA0002266645780000044
wherein the content of the first and second substances,
Figure BDA0002266645780000045
is the flow rate of
Figure BDA0002266645780000046
The common factor of (a) is,
Figure BDA0002266645780000047
is a common factor
Figure BDA0002266645780000048
Time series characterization of (a), yi,c(i ═ 1,2, p) as a common factor
Figure BDA0002266645780000049
Each time component of (a), another time series of specific factors of the network traffic
Figure BDA00022666457800000410
By the same way obtain
Figure BDA00022666457800000411
Figure BDA00022666457800000412
Wherein the content of the first and second substances,
Figure BDA00022666457800000413
is the flow rate of
Figure BDA00022666457800000414
By a specific factor of (a) or (b),
Figure BDA00022666457800000415
is a special factor
Figure BDA00022666457800000416
Time series characterization of (a), yi,s(i-1, 2, …, n) is a special factor
Figure BDA00022666457800000417
Each time component of (a);
the state of each sample in the entire model was evaluated using a factorial analysis model, and the factorial score was calculated using a regression estimation method, a bartlett estimation method, or a thomson estimation method.
Respectively obtaining the flow by using an empirical mode decomposition method as described in step 5
Figure BDA00022666457800000418
Common factor in
Figure BDA00022666457800000419
Another time series of special factors with network traffic
Figure BDA00022666457800000420
Set of characteristic functions of (a): the method comprises the following steps:
step (1): is provided with
Figure BDA00022666457800000421
And c is 1; r is0(t) denotes a common factor
Figure BDA00022666457800000422
C is a judgment factor;
step (2): setting i to be 1, initializing a threshold value a and a maximum iteration number S;
and (3): initial setting k is 0 and ei+1,k(t)＝ri(t), let spline function s (t) be a cubic spline, s ═ 3, v ═ P, and P0; e.g. of the typei+1,k(t) is ri(t) expressing a polynomial function, wherein s is the highest degree of the polynomial in the spline interpolation function, and v is the number of interpolation points;
and (4): find out ei+1,k(t) local maxima and local minima, creating two spline curves s using a spline interpolation method based on s (t)u(t) and sl(t) obtaining mi+1,k＝(su(t)+sl(t))2, and ei+1,k+1(t)＝ei+1,k(t)-mi+1,k；mi+1,kAs a spline mean curve, ei+1,k(t)、ei+1,k+1(t) is a flow function ri(t) a polynomial function representation;
and (5): judgment ei+1,k+1(t) whether the conditions for the eigenmode function components are met, if so, going to step (9); if not, the next step is carried out;
and (6): judging v > mi+1,kIf yes, setting v as mi+1,k，e(t)＝ei+1,k+1(t) if not, proceeding the next step; v represents the maximum value of the spline mean of the current cycle;
and (7): and (3) judging: if s ═ 3 is true, the spline function s (t) is set as a B-spline function, and if s ═ B, the step (4) is carried out, otherwise, the next step is carried out;
and (8): and (3) judging: if k is ≦ S and
Figure BDA0002266645780000051
if yes, k +1, s + 3 are set and the procedure returns to step (4), or e is seti+1,k+1(t) ═ e (t); otherwise, carrying out the next step; s is the maximum number of iterations, a is a threshold, ek-1(t)、ek(t) is the flow rate
Figure BDA0002266645780000052
A polynomial function representation of (a);
and (9): obtaining the eigenmode function component fi+1(t)＝ei+1,k+1(t) and ri+1(t)＝ri(t)-fi+1(t)；ri(t)、ri+1(t) is a common factor
Figure BDA0002266645780000053
The ith and i +1 components of the time function;
step (10): and (3) judging: if the residual error ri+1(t) if the monotone function is true, setting i to i +1, and returning to step (3); otherwise, carrying out the next step;
step (11): judging whether c is 1, if so, calculating a common factor to enable gi,c(t)＝fi(t)，rm,c(t)＝ri+1(t) obtaining a set of characteristic functions gc(t)＝{g1,c(t),g2,c(t) }, make
Figure BDA0002266645780000054
c is 2, returning to the step (2), and recalculating the special factor part; otherwise, entering the next step; r ism,c(t) recording a non-monotonic function ri+1(t)，{gi,c(t) } flow rate after decomposition by empirical mode decomposition
Figure BDA0002266645780000055
Common factor in
Figure BDA0002266645780000056
A characteristic function component of (a);
step (12): let hi,s(t)＝fi(t)，sm,s(t)＝ri+1(t) obtaining a set of characteristic functions
hs(t)＝{h1,s(t),h2,s(t),...}；sm,s(t) recording a non-monotonic function ri+1(t)，{hs(t) } flow rate after decomposition by empirical mode decomposition
Figure BDA0002266645780000057
Specific factor in
Figure BDA0002266645780000058
The characteristic function component of (2).
The invention has the following advantages and beneficial effects:
the invention discloses a service flow abnormity detection method for intelligent optical cable fiber core dispatching exchange based on factor analysis, which combines a factor analysis method and empirical mode decomposition. And converting the network flow sequence into a flow matrix, and performing principal component decomposition on the matrix to determine a public component and a special component of the network flow. The flow is divided into two parts, so that empirical mode decomposition is conveniently carried out on k factors of each part, and the analysis accuracy is improved; respectively establishing different empirical mode functions to capture and characterize the factors, and effectively capturing the characteristics of the factors; the invention provides a new rapid detection method based on the thought, which can be used for detecting abnormal components in the network service flow and has higher accuracy and timeliness.
Drawings
The technical solution in the embodiment of the present invention will be clearly and completely described below with reference to the accompanying drawings in the embodiment of the present invention. It is to be understood that the described embodiments are merely exemplary of the invention, and not restrictive of the full scope of the invention. Other embodiments, which can be derived by one of ordinary skill in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
FIG. 1 is a general flowchart of an embodiment of a method for detecting abnormal traffic flow in an optical fiber core intelligent scheduling switch based on factor analysis according to the present invention;
FIG. 2 is a flow chart of steps in an embodiment of the empirical mode decomposition method of the present invention;
FIG. 3a is a normal network flow diagram of the present invention;
FIG. 3b is a graph of abnormal network traffic for the present invention;
FIG. 3c is a diagram of the common parts extracted from the abnormal network traffic of FIG. 3b in accordance with the present invention;
FIG. 3d is a special portion of the graph extracted from the abnormal network traffic of FIG. 3b according to the present invention;
FIG. 4 is a result of empirical mode decomposition of a traffic common component of the present invention;
FIG. 5 is a result of empirical mode decomposition of a particular portion of the flow of the present invention;
fig. 6 shows the flow anomaly detection result of the present invention, in this simulation, the determined detection threshold is 0.6, and the point-pulse curve shows the time when the anomaly flow is injected.
Detailed Description
The invention relates to a service flow anomaly detection method for optical cable fiber core remote intelligent scheduling exchange, which is used for detecting abnormal components in network flow, and is shown in figure 1, wherein figure 1 is a general flow chart of an embodiment of the service flow anomaly detection method for optical cable fiber core intelligent scheduling exchange based on factor analysis. The method comprises the following steps:
step 101: starting;
step 102: given network traffic as a known condition, determining the number of common factors k, network traffic in a fiber optic communications access network varies over time, and therefore we can treat them as a time series. Let y (t) represent network traffic at time t. Then the time series y (t) t 1,2, represents the change in network traffic over time. Without loss of generality, network traffic of length n is set
Figure BDA0002266645780000071
Where n is an integer. According to network traffic
Figure BDA0002266645780000072
We can get the following random matrix:
Y＝{yi}n×1＝{y(1),y(2),...,y(n)} (1)
wherein, yi(i ═ 1,2, …, n) is the dominant random vector whose mean vector e (y) is 0.
Step 103: the covariance matrix cov (Y) R of the matrix Y is determined, and the eigenvalues and eigenvectors of the covariance matrix R are obtained. Then, the flow rate Y in equation (1) is factorized, and the contribution rate of variance and the contribution rate of cumulative variance are calculated, and the factor Y is determinedcAnd factor Ys. According to factorial theory, Y can be decomposed into the following equation:
Figure BDA0002266645780000073
wherein, Yci(i ═ 1,2, …, p, and p ≦ n) is an implicit random vector whose mean vector E (Y) isc)＝0(Yc＝{Yc1,Yc2,...,Ycp}), covariance matrix Cov (Y)c) 1. This represents YciAre independent of each other. Y issj(j-1, 2, …, n) is a complementary random vector in factorization, Ysj(j ═ 1,2, …, n) and Yci(i ═ 1,2, …, p) (p ≦ n) independently of one another, equation E (Y)s)＝0(Ys＝{Ys1,Ys2,...,Ysn}) are true, and YsThe factors in (1) are independent of each other. a isij( i 1,2, …, n, j 1,2, …, p and p ≦ n) represents an implicit random vector YciThe coefficient of (a).
Step 104: typical representative amounts of each common factor are not significant after the common factor solution is obtained. And then, performing an orthogonal factor rotation method with the largest variance to ensure that each row of elements in the common factor sequence is separated by the distance as much as possible and obtain a main factor meeting the appropriate rotation. Since the k most important common factors have been selected, there is the following equation:
Y＝{yi}n×1＝A·Yc+Ys (3)
where Y is the currently obtained network traffic matrix, { Yi}n×1To express a random matrix of Y, YcRepresenting a common factor in network traffic, YsRepresenting a particular factor in the network traffic, a is called a factor load matrix. The model in equation (3) may be used to characterize network traffic.
Step 105: the state of each sample in the entire model is evaluated using a factorial analysis model. The score for each sample for the common factor is calculated from a linear combination of factors represented by the variable y as a factor score function. The number p of equations in the factor score function is less than the number n of variables, so that the factor score cannot be accurately calculated, and can only be estimated. A regression estimation method, a butteret estimation method or a thomson estimation method is used.
Therefore, according to equations (2) - (3), a new time series can be obtained as follows:
Figure BDA0002266645780000081
wherein the content of the first and second substances,
Figure BDA0002266645780000082
representative of the flow
Figure BDA0002266645780000083
The common factor of (a) is,
Figure BDA0002266645780000084
is a common factor
Figure BDA0002266645780000085
Time series characterization of (a), yi,c(i ═ 1,2, p) as a common factor
Figure BDA0002266645780000086
Each of (1)A time component. Then another time series of special factors for describing the network traffic
Figure BDA0002266645780000087
The construction was as follows:
Figure BDA0002266645780000088
wherein the content of the first and second substances,
Figure BDA0002266645780000089
is the flow rate of
Figure BDA00022666457800000810
By a specific factor of (a) or (b),
Figure BDA00022666457800000811
is a special factor
Figure BDA00022666457800000812
Time series characterization of (a), yi,s(i-1, 2, n) is a special factor
Figure BDA00022666457800000813
Each time component of (a).
In this case, the network traffic is divided into
Figure BDA00022666457800000814
And
Figure BDA00022666457800000815
two components.
Step 106: separate derivation of flow using empirical mode decomposition
Figure BDA00022666457800000816
Common factor in
Figure BDA00022666457800000817
With another of the special factors of the network trafficIntermediate sequence
Figure BDA00022666457800000818
Set g of feature functions ofc(t)＝{g1,c(t),g2,c(t) } and hs(t)＝{h1,s(t),h2,s(t),. Wherein, { gi,c(t) } flow rate after decomposition by empirical mode decomposition
Figure BDA00022666457800000819
Common factor in
Figure BDA00022666457800000820
Characteristic function component of { h }i,s(t) } flow rate after decomposition by empirical mode decomposition
Figure BDA00022666457800000821
Specific factor in
Figure BDA00022666457800000822
The characteristic function component of (2).
This step is relatively complex and is illustrated in detail in the flowchart and explanation of fig. 2.
Step 107: through gc(t) and hs(t) filtering the characteristics, finding out abnormal parts of the service flow and storing the results.
FIG. 2 is a flow chart showing the steps of the empirical mode decomposition method according to the present invention. The process comprises the following steps:
step 201: starting;
step 202: is provided with
Figure BDA0002266645780000091
And c is 1. r is0(t) denotes a common factor
Figure BDA0002266645780000092
C is a judgment factor.
Step 203: let i equal to 1. A threshold a and a maximum number of iterations S are then initialized.
Step 204: initial setting k is 0 and ei+1,k(t)＝ri(t) of (d). Let spline function s (t) be a cubic spline, s-3, v-P and P0. e.g. of the typei+1,k(t) is riAnd (t) expressing a polynomial function, wherein s is the highest degree of the polynomial in the spline interpolation function, and v is the number of interpolation points.
Step 205: find out ei+1,k(t) local maxima and local minima, creating two spline curves s using a spline interpolation method based on s (t)u(t) and sl(t) obtaining mi+1,k＝(su(t)+sl(t))2, and ei+1,k+1(t)＝ei+1,k(t)-mi+1,k。mi+1,kAs a spline mean curve, ei+1,k(t)、ei+1,k+1(t) is a flow function ri(t) is a polynomial function representation.
Step 206: judgment ei+1,k+1(t) whether the conditions for the eigenmode function components are met, and if so, proceeding to step 210; if not, the next step is carried out.
Step 207: judging v > mi+1,kIf yes, setting v as mi+1,k，e(t)＝ei+1,k+1(t) otherwise, proceeding to the next step. v represents the maximum spline mean for the current cycle.
Step 208: and (3) judging: if s ═ 3 is true, then spline s (t) is assumed to be a B-spline, and go to step 205, otherwise proceed to the next step.
Step 209: and (3) judging: if k is ≦ S and
Figure BDA0002266645780000093
if true, k +1 and s 3 are set and the process returns to step 205. Or is provided with ei+1,k+1(t) e (t). Otherwise, the next step is carried out. S is the maximum number of iterations, a is a threshold, ek-1(t)、ek(t) is the flow rate
Figure BDA0002266645780000094
Is expressed by a polynomial function of (1).
Step 210: obtaining the eigenmode function component fi+1(t)＝ei+1,k+1(t)，And is provided with ri+1(t)＝ri(t)-fi+1(t)。ri(t)、ri+1(t) is a common factor
Figure BDA0002266645780000095
The ith and i +1 components of the time function.
Step 211: and (3) judging: if the residual error ri+1If (t) is the monotone function is established, i is set to i +1, and the process returns to step 205. Otherwise, the next step is carried out.
Step 212: and judging whether c is 1, and if so, indicating that the calculation is to calculate the common factor. Let g bei,c(t)＝fi(t)，rm,c(t)＝ri+1(t) obtaining a set of characteristic functions gc(t)＝{g1,c(t),g2,c(t) }, make
Figure BDA0002266645780000101
c is 2, the process returns to step 203 to recalculate the special factor part. Otherwise, the next step is carried out. r ism,c(t) recording a non-monotonic function ri+1(t)，{gi,c(t) } flow rate after decomposition by empirical mode decomposition
Figure BDA0002266645780000102
Common factor in
Figure BDA0002266645780000103
The characteristic function component of (2).
Step 213: the calculation result is a special factor part, so that h isi,s(t)＝fi(t)，sm,s(t)＝ri+1(t) obtaining a set of eigenfunctions hs(t)＝{h1,s(t),h2,s(t),...}。sm,s(t) recording a non-monotonic function ri+1(t)，{hs(t) } flow rate after decomposition by empirical mode decomposition
Figure BDA0002266645780000104
Specific factor in
Figure BDA0002266645780000105
The characteristic function component of (2).
In the actual simulation process, in order to better verify the detection capability of abnormal traffic of the service flow intelligently scheduled and exchanged at the fiber core of the optical cable, the simulation uses real data from an Abilene backbone network. In the simulation, abnormal network traffic is injected into normal background network traffic at four different time slots, which are 300,700,1100 and 1500 respectively, and the duration is 80. To avoid random errors, 50 simulations were run to obtain an average detection result, and the detection threshold was automatically determined according to the detection algorithm. The factorization-based feature extraction capability, the empirical mode decomposition-based feature extraction capability, and the anomaly detection capability are evaluated.
The network traffic and the factorization results of the present invention are shown in fig. 3 a-3 d, where fig. 3a and 3b represent normal and abnormal network traffic, respectively, and fig. 3c and 3d depict the common and special portions extracted from the abnormal network traffic in fig. 3b, respectively. As can be seen from fig. 3a and 3b, there is no significant difference between the normal flow and the abnormal flow, which makes the detection difficult. Fig. 3c and 3d show that the algorithm of the present invention can correctly extract the common features and the special features of the abnormal network traffic. It can be seen that the traffic of the common part reflects the common characteristics of this network traffic. This demonstrates the effectiveness of the present invention.
Fig. 4 shows the result of empirical mode decomposition on the common part of the flow. The common part flow can be accurately characterized by 10 empirical mode functions, and different empirical mode functions can capture different characteristics of the common part of the flow.
Fig. 5 shows the result of empirical mode decomposition of a particular portion of the flow. The particular portion of flow may be accurately characterized by 10 empirical mode functions, and different empirical mode functions may capture different characteristics of the particular portion of flow. This further demonstrates the reliability of the invention.
Fig. 6 shows the flow anomaly detection result of the present invention, in this simulation, the determined detection threshold is 0.6, and the point-pulse curve shows the time when the anomaly flow is injected. As can be seen from the figure, the detection curve can accurately mark the time when the abnormal network traffic occurs, and the abnormal network traffic can be correctly found out by using threshold detection. The invention can accurately detect the abnormal traffic of the fiber core of the optical cable. The invention is proved to be reliable, accurate and timely.
Although specific embodiments of the present invention have been described above, it will be appreciated by those skilled in the art that these are merely illustrative and that various changes or modifications may be made to these embodiments without departing from the principles and spirit of the invention. The scope of the invention is only limited by the appended claims.

Claims (7)

1. A service flow abnormity detection method facing optical cable fiber core remote intelligent scheduling exchange is characterized in that: the method comprises the following steps:
step 1: giving network traffic
Figure FDA0003474108900000011
And the number of the common factors k, constructing a random matrix Y; r (R ═ cov (y))
Step 2: standardizing the original data to obtain a correlation matrix of the standardized data, obtaining an eigenvalue and an eigenvector of a matrix R, calculating the contribution rate of variance and the contribution rate of accumulated variance, and determining a common factor Y in the network flowcAnd special factor Y in network traffics；
And step 3: after the public factor solution is obtained, factor rotation is carried out, and a main factor meeting proper rotation is obtained;
and 4, step 4: establishing a factor analysis model for the obtained main factors, and evaluating the state of each sample in the whole model by using the factor analysis model; obtaining the score of each factor by adopting a regression estimation method, a Batterest estimation method and a Thomson estimation method to obtain the flow
Figure FDA0003474108900000012
Common factor in
Figure FDA0003474108900000013
And network flowAnother time series of special factors of the quantity
Figure FDA0003474108900000014
And 5: separate derivation of flow using empirical mode decomposition
Figure FDA0003474108900000015
Common factor in
Figure FDA0003474108900000016
Another time series of special factors with network traffic
Figure FDA0003474108900000017
Set of characteristic functions of (a):
step 6: according to gc(t) and hs(t) pairs
Figure FDA0003474108900000018
And
Figure FDA0003474108900000019
carrying out feature extraction;
and 7: through gc(t) and hs(t) filtering the features, finding out abnormal flow parts and storing the results.
2. The method for detecting the abnormal traffic flow of the optical cable fiber core remote intelligent dispatching exchange as claimed in claim 1, wherein: the step 5: obtaining another time series of common factors in the traffic and special factors of the network traffic respectively by using empirical mode decomposition method
Figure FDA00034741089000000110
The set of feature functions of (a) is:
gc(t)＝{g1,c(t),g2,c(t) } and hs(t)＝{h1,s(t),h2,s(t),...}；
Wherein, { gi,c(t) } flow rate after decomposition by empirical mode decomposition
Figure FDA00034741089000000111
Common factor in
Figure FDA00034741089000000112
Characteristic function component of { h }i,s(t) } flow rate after decomposition by empirical mode decomposition
Figure FDA00034741089000000113
Specific factor in
Figure FDA00034741089000000114
The characteristic function component of (2).
3. The method for detecting the abnormal traffic flow of the optical cable fiber core remote intelligent dispatching exchange as claimed in claim 1, wherein: step 1, constructing the random matrix Y, wherein the method comprises the following steps:
taking the network traffic as a time sequence, representing the change of the network traffic along with time by y (t) | t ═ 1,2
Figure FDA0003474108900000021
Where n is an integer, the following random matrix is obtained:
Y＝{yi}n×1＝{y(1),y(2),...,y(n)} (1)
wherein, yiN is a dominant random vector whose mean vector e (y) is 0.
4. The method for detecting the abnormal traffic flow of the optical cable fiber core remote intelligent dispatching exchange as claimed in claim 1, wherein: step 2, determining the common factor Y in the network flowcAnd special factor Y in network trafficsThe determination method comprises the following steps:
according to the factorial theory, Y is decomposed into the following equation:
Figure FDA0003474108900000022
wherein, Yci(i 1, 2.. p and p.ltoreq.n) is an implicit random vector whose mean vector E (Y) isc)＝0(Yc＝{Yc1,Yc2,...,Ycp}), covariance matrix Cov (Y)c) 1, represents YciAre independent of one another, Ysj(j ═ 1, 2.. times, n) is the complementary random vector in the factorization, Ysj(j ═ 1,2,. n) and Yci(i ═ 1, 2.. times, p) (p ≦ n), independent of one another, equation E (Y)s)＝0(Ys＝{Ys1,Ys2,...,Ysn}) are true, and YsThe factors in (a) are independent of each otherij(i 1, 2.., n, j 1, 2.. 7., p, and p ≦ n) represents an implicit random vector YciThe coefficient of (a).
5. The method for detecting the abnormal traffic flow of the optical cable fiber core remote intelligent dispatching exchange as claimed in claim 1, wherein: flow rate in step 4
Figure FDA0003474108900000023
Common factor in
Figure FDA0003474108900000024
Another time series of special factors with network traffic
Figure FDA0003474108900000025
The determination method comprises the following steps:
the k most important common factors are chosen, as follows:
Y＝{yi}n×1＝A·Yc+Ys (3)
wherein Y is the currently obtained network flowQuantity matrix, { yi}n×1To express a random matrix of Y, YcBeing a common factor in network traffic, YsA is a factor load matrix, which is a special factor in the network traffic.
6. The method for detecting traffic flow abnormality of optical cable fiber core remote intelligent dispatching exchange according to claim 4 or 5, wherein: from said equations (2) - (3), a new time series is obtained:
Figure FDA0003474108900000031
wherein the content of the first and second substances,
Figure FDA0003474108900000032
is the flow rate of
Figure FDA0003474108900000033
The common factor of (a) is,
Figure FDA0003474108900000034
is a common factor
Figure FDA0003474108900000035
Time series characterization of (a), yi,c(i ═ 1, 2.. times, p) is a common factor
Figure FDA0003474108900000036
Each time component of (a), another time series of specific factors of the network traffic
Figure FDA0003474108900000037
By the same way obtain
Figure FDA0003474108900000038
Figure FDA0003474108900000039
Wherein the content of the first and second substances,
Figure FDA00034741089000000310
is the flow rate of
Figure FDA00034741089000000311
By a specific factor of (a) or (b),
Figure FDA00034741089000000312
is a special factor
Figure FDA00034741089000000313
Time series characterization of (a), yi,s(i ═ 1, 2.. times.n) is a special factor
Figure FDA00034741089000000314
Each time component of (a);
the state of each sample in the entire model was evaluated using a factorial analysis model, and the factorial score was calculated using a regression estimation method, a bartlett estimation method, or a thomson estimation method.
7. The method for detecting the abnormal traffic flow of the optical cable fiber core remote intelligent dispatching exchange as claimed in claim 1, wherein: respectively obtaining the flow by using an empirical mode decomposition method as described in step 5
Figure FDA00034741089000000315
Common factor in
Figure FDA00034741089000000316
Another time series of special factors with network traffic
Figure FDA00034741089000000317
Set of characteristic functions of (a): the method comprises the following steps:
step (1): is provided with
Figure FDA00034741089000000318
And c is 1; r is0(t) denotes a common factor
Figure FDA00034741089000000319
C is a judgment factor;
step (2): setting i to be 1, initializing a threshold value a and a maximum iteration number S;
and (3): initial setting k is 0 and ei+1,k(t)＝ri(t) if the spline function s (t) is a cubic spline, s is 3, v is P and P > 0; e.g. of the typei+1,k(t) is ri(t) expressing a polynomial function, wherein s is the highest degree of the polynomial in the spline interpolation function, and v is the number of interpolation points;
and (4): find out ei+1,k(t) local maxima and local minima, creating two spline curves s using a spline interpolation method based on s (t)u(t) and sl(t) obtaining mi+1,k＝(su(t)+sl(t))/2, and ei+1,k+1(t)＝ei+1,k(t)-mi+1,k；mi+1,kAs a spline mean curve, ei+1,k(t)、ei+1,k+1(t) is a flow function ri(t) a polynomial function representation;
and (5): judgment ei+1,k+1(t) whether the conditions for the eigenmode function components are met, if so, going to step (9); if not, the next step is carried out;
and (6): judging v > mi+1,kIf yes, setting v as mi+1,k，e(t)＝ei+1,k+1(t) if not, proceeding the next step; v represents the maximum value of the spline mean of the current cycle;
and (7): and (3) judging: if s ═ 3 is true, the spline function s (t) is set as a B-spline function, and if s ═ B, the step (4) is carried out, otherwise, the next step is carried out;
and (8): and (3) judging: if k is ≦ S and
Figure FDA0003474108900000041
if yes, k +1, s + 3 are set and the procedure returns to step (4), or e is seti+1,k+1(t) ═ e (t); otherwise, carrying out the next step; s is the maximum number of iterations, a is a threshold, ek-1(t)、ek(t) is the flow rate
Figure FDA0003474108900000042
A polynomial function representation of (a);
and (9): obtaining the eigenmode function component fi+1(t)＝ei+1,k+1(t) and ri+1(t)＝ri(t)-fi+1(t)；ri(t)、ri+1(t) is a common factor
Figure FDA0003474108900000043
The ith and i +1 components of the time function;
step (10): and (3) judging: if the residual error ri+1(t) if the monotone function is true, setting i to i +1, and returning to step (3); otherwise, carrying out the next step;
step (11): judging whether c is 1, if so, calculating a common factor to enable gi,c(t)＝fi(t)，rm,c(t)＝ri+1(t) obtaining a set of characteristic functions gc(t)＝{g1,c(t),g2,c(t) }, make
Figure FDA0003474108900000044
c is 2, returning to the step (2), and recalculating the special factor part; otherwise, entering the next step; r ism,c(t) recording a non-monotonic function ri+1(t)，{gi,c(t) } flow rate after decomposition by empirical mode decomposition
Figure FDA0003474108900000045
Common factor in
Figure FDA0003474108900000046
A characteristic function component of (a);
step (ii) of(12): let hi,s(t)＝fi(t)，sm,s(t)＝ri+1(t) obtaining a set of eigenfunctions hs(t)＝{h1,s(t),h2,s(t),...}；sm,s(t) recording a non-monotonic function ri+1(t)，{hs(t) } flow rate after decomposition by empirical mode decomposition
Figure FDA0003474108900000047
Specific factor in
Figure FDA0003474108900000048
The characteristic function component of (2).
CN201911090284.7A 2019-11-08 2019-11-08 Service flow abnormity detection method for optical cable fiber core remote intelligent scheduling exchange Active CN111786935B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN201911090284.7A CN111786935B (en) 2019-11-08 2019-11-08 Service flow abnormity detection method for optical cable fiber core remote intelligent scheduling exchange

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN201911090284.7A CN111786935B (en) 2019-11-08 2019-11-08 Service flow abnormity detection method for optical cable fiber core remote intelligent scheduling exchange

Publications (2)

Publication Number Publication Date
CN111786935A CN111786935A (en) 2020-10-16
CN111786935B true CN111786935B (en) 2022-03-01

Family

ID=72755748

Family Applications (1)

Application Number Title Priority Date Filing Date
CN201911090284.7A Active CN111786935B (en) 2019-11-08 2019-11-08 Service flow abnormity detection method for optical cable fiber core remote intelligent scheduling exchange

Country Status (1)

Country Link
CN (1) CN111786935B (en)

Families Citing this family (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN112910567B (en) * 2021-01-25 2022-07-01 北京邮电大学 Interception classification monitoring method based on recurrent neural network and related equipment

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN104079452A (en) * 2014-06-30 2014-10-01 电子科技大学 Data monitoring technology and network traffic abnormality classifying method
CN106972966A (en) * 2017-03-28 2017-07-21 国网辽宁省电力有限公司阜新供电公司 The real-time detection method of exception of network traffic in a kind of communication network
CN107317701A (en) * 2017-06-13 2017-11-03 电子科技大学 A kind of network flow abnormal detecting method based on empirical mode decomposition
CN109067725A (en) * 2018-07-24 2018-12-21 成都亚信网络安全产业技术研究院有限公司 Network flow abnormal detecting method and device
CN109697630A (en) * 2018-11-21 2019-04-30 浙江大学城市学院 A kind of businessman's volume of the flow of passengers multiplicity and prediction technique based on sparse regression

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN104079452A (en) * 2014-06-30 2014-10-01 电子科技大学 Data monitoring technology and network traffic abnormality classifying method
CN106972966A (en) * 2017-03-28 2017-07-21 国网辽宁省电力有限公司阜新供电公司 The real-time detection method of exception of network traffic in a kind of communication network
CN107317701A (en) * 2017-06-13 2017-11-03 电子科技大学 A kind of network flow abnormal detecting method based on empirical mode decomposition
CN109067725A (en) * 2018-07-24 2018-12-21 成都亚信网络安全产业技术研究院有限公司 Network flow abnormal detecting method and device
CN109697630A (en) * 2018-11-21 2019-04-30 浙江大学城市学院 A kind of businessman's volume of the flow of passengers multiplicity and prediction technique based on sparse regression

Also Published As

Publication number Publication date
CN111786935A (en) 2020-10-16

Similar Documents

Publication Publication Date Title
Kurt et al. Online cyber-attack detection in smart grid: A reinforcement learning approach
CN111163057B (en) User identification system and method based on heterogeneous information network embedding algorithm
Li et al. Linearity evaluation and variable subset partition based hierarchical process modeling and monitoring
CN111652496B (en) Running risk assessment method and device based on network security situation awareness system
Mestav et al. Bayesian state estimation for unobservable distribution systems via deep learning
Xu et al. Digital twin-based anomaly detection in cyber-physical systems
Pedarsani et al. A bayesian method for matching two similar graphs without seeds
CN109145516B (en) Analog circuit fault identification method based on improved extreme learning machine
CN113255835A (en) Hydropower station pump equipment anomaly detection method
CN111783845B (en) Hidden false data injection attack detection method based on local linear embedding and extreme learning machine
Oozeer et al. Cognitive dynamic system for control and cyber-attack detection in smart grid
CN110264041A (en) A kind of non-intrusion type load decomposition and monitoring method based on difference
CN111786935B (en) Service flow abnormity detection method for optical cable fiber core remote intelligent scheduling exchange
CN114528190B (en) Single index abnormality detection method and device, electronic equipment and readable storage medium
CN117171619A (en) Intelligent power grid terminal network anomaly detection model and method
Cong et al. Minimizing efforts in reconciling participatory sensing data
Gao et al. The prediction role of hidden markov model in intrusion detection
CN113642029B (en) Method and system for measuring correlation between data sample and model decision boundary
Casas et al. Efficient methods for traffic matrix modeling and on-line estimation in large-scale IP networks
CN116662866A (en) End-to-end incomplete time sequence classification method based on data interpolation and characterization learning
CN115965149A (en) Water quality index prediction method based on LSTM algorithm model
Ke et al. A model for degradation prediction with change point based on Wiener process
CN105404973A (en) Power transmission and transformation equipment state prediction method and system
CN112597699B (en) Social network rumor source identification method integrated with objective weighting method
Ni et al. A refined instrumental variable method for Hammerstein-Wiener continuous-time model identification

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant