CN101534305A - Method and system for detecting network flow exception - Google Patents

Method and system for detecting network flow exception Download PDF

Info

Publication number
CN101534305A
CN101534305A CN200910082913A CN200910082913A CN101534305A CN 101534305 A CN101534305 A CN 101534305A CN 200910082913 A CN200910082913 A CN 200910082913A CN 200910082913 A CN200910082913 A CN 200910082913A CN 101534305 A CN101534305 A CN 101534305A
Authority
CN
China
Prior art keywords
flow indicator
network
principal component
indicator data
flow
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN200910082913A
Other languages
Chinese (zh)
Inventor
袁小坊
谢高岗
裴唯
陈楠楠
王东
张大方
闵应骅
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Institute of Computing Technology of CAS
Original Assignee
Institute of Computing Technology of CAS
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Institute of Computing Technology of CAS filed Critical Institute of Computing Technology of CAS
Priority to CN200910082913A priority Critical patent/CN101534305A/en
Publication of CN101534305A publication Critical patent/CN101534305A/en
Pending legal-status Critical Current

Links

Images

Landscapes

  • Data Exchanges In Wide-Area Networks (AREA)

Abstract

The invention provides a method and a system for detecting network flow exception. The method comprises the following steps of collecting flow index data and establishing a flow index data matrix; establishing a pivot element model of the flow index data using a principal component analytic method; and detecting exception of the network flow according to the multi-variable statistical control graph of the pivot element model. The method and system provided in the invention have strong operability and effectively improve the accuracy of detecting network flow exception.

Description

Network flow abnormal detecting method and system
Technical field
The present invention relates to network management and safe practice field, be specifically related to a kind of for the unusual detection method of network.
Background technology
When high-speed backbone network aspect was carried out the detection of macro network Traffic Anomaly, the real-time processing of huge flow and the detection of unknown attack had brought very big challenge for traditional Intrusion Detection Technique.In the Traffic Anomaly context of detection, multiple detection method is constantly inquired into and proposed in academic institution both domestic and external and enterprise.
The researcher mainly adopts source-purpose (Origin-Destination OD) stream matrix and ASSOCIATE STATISTICS analytical method to carry out the network abnormality detection both at home and abroad at present, in the articles such as Structural Analysis of Network Traffic Flows of people such as Lakhina in 2004 in ACMSIGMETRICS network abnormality detection analytical method as described below being disclosed: at first utilizes principal component analytical method (PCA), the OD stream matrix that OD between source and target stream is formed carries out the PCA analyzing and processing, and the network traffics data are divided into the data that the cycle changes substantially, the data of short-term sudden change and the noise data of Gaussian Profile; The data that will change in the cycle are summed up in the point that on 3 main compositions then, come the feature of reconstructed network stream with 3 new composite variables, thereby have constituted the proper space of network traffics data, and other remaining composition composite variable has constituted the abnormal space of network traffics data; Adopt subspace (Subspace) method to come the check and analysis network unusual at last.But in said method, the obtaining of OD stream matrix, computational methods are very complicated, and the cost height is also very complicated to the determination methods of Exception Type, need carry out the study of standard off-note in advance, so operability is relatively poor in actual applications for this method.
Summary of the invention
The technical problem to be solved in the present invention provides a kind of workable network flow abnormal detecting method and system.
For achieving the above object, the invention provides a kind of network flow abnormal detecting method, comprise the following steps:
1) acquisition stream figureofmerit data are set up the flow indicator data matrix;
2) adopt principal component analytical method to set up the principal component model of described flow indicator data matrix;
3) the multivariate statistics control chart by described principal component model carries out abnormality detection to network traffics.
In the said method, described flow indicator packet includes network application type, the right byte number indication information of port or IP address, the bag number indication information that network application type, port or IP address are right, packet byte size information, and/or flow path direction information.
In the said method, described step 1) further comprises:
Gather described flow indicator data, described flow indicator data are carried out statistic of classification based on application layer protocol;
Set up the flow indicator data matrix.
In the said method, described step 1) also comprises carries out normalized step to described flow indicator matrix.
In the said method, described step 3) is: adopt the square prediction error statistical chart to detect the time that described exception of network traffic occurs.
In the said method, described step 3) also comprises step:
Adopt the pivot contribution plot to analyze to the maximum flow indicator of described exception of network traffic contribution.
According to a further aspect in the invention, also provide a kind of exception of network traffic detection system, comprised following parts:
Network probe is used for acquisition stream figureofmerit data;
Analysis component is used to set up the flow indicator data matrix, adopts principal component analytical method to set up the principal component model of described flow indicator data matrix, and the multivariate statistics control chart by described principal component model carries out abnormality detection to network traffics.
In the said detecting system, described network probe comprises application layer traffic identification and sort module, is used for based on the described flow indicator data of application layer protocol statistic of classification.
Said method provided by the invention is workable, and has effectively improved the accuracy that exception of network traffic detects.
Description of drawings
Fig. 1 is the flow chart of the network anomaly detection method of a specific embodiment according to the present invention;
Fig. 2 is the rubble figure of the byte number flow indicator of the specific embodiment according to the present invention;
Fig. 3 is the SPE statistical chart of principal component model of the byte number flow indicator data matrix of the specific embodiment according to the present invention;
Fig. 4 is the 160th each byte number flow indicator contribution plot of the moment of the specific embodiment according to the present invention;
Fig. 5 is each agreement departure byte-rate of unusual moment base application of the specific embodiment according to the present invention;
Fig. 6 is the rubble figure of the bag number flow indicator of the specific embodiment according to the present invention;
Fig. 7 is the SPE statistical chart of principal component model of the bag number flow indicator data matrix of the specific embodiment according to the present invention;
Fig. 8 respectively wraps number flow indicator contribution plot in the 160th moment of the specific embodiment according to the present invention;
Fig. 9 is each agreement departure packet rate of unusual moment base application of the specific embodiment according to the present invention;
Figure 10 is the network traffics detection system of the specific embodiment according to the present invention.
Embodiment
In order to make purpose of the present invention, technical scheme and advantage clearer,, exception of network traffic check and analysis method is according to an embodiment of the invention further described below in conjunction with accompanying drawing.Should be appreciated that specific embodiment described herein only in order to explanation the present invention, and be not used in qualification the present invention.
Current those of ordinary skills think that basic network traffics achievement data can not satisfy the requirement to the in-depth analysis of network, must have detailed stream information.But in fact for the network manager, there is not detailed stream information, as long as but inform that the feature and the scope of the unusual stream information of appearance just can help its investigation reason fast.For example onlinely detect certain unusual generation, Realtime Alerts, and point out that this is to be caused by the basic application traffic of leaving the country unusually, the size of packet is mainly between 256 bytes and 512 bytes, such information can be very fast the help keeper scope of dwindling problem analysis, the keeper can inquire about the departure bit rate figure of each agreement of base application at that time at once, if find the bit rate surge of http protocol at that time, it can be investigated like this, thereby checks whether the WEB server in the net occurs unusually.
Owing between the network traffics index, exist certain correlation, this makes and utilizes less variable to reflect that the information of whole variablees becomes possibility, therefore, can adopt the PCA method that the flow indicator data matrix is carried out dimensionality reduction and feature extraction, be translated into the separate low-dimensional variable space, set up corresponding principal component model, realize feature extraction complicated flow indicator data.Principal component model has been given up the part residual error and has been kept the main direction that embodies data variation.The specific measured value that can make unusually changes according to specific rule, and principal component model has then comprised unusual change direction in the variable space.
Before specifically describing method and system of the present invention, PCA method of the present invention is described at first.The PCA method is a kind of method in the multivariate statistics, and it attempts to lose under the minimum principle at the data message of trying hard to keep, and multivariable cross-section data table is carried out optimum comprehensive simplification, that is to say, high dimensional variable space is carried out dimension-reduction treatment.
Suppose to have p stochastic variable, be designated as X 1, X 2..., X p, PCA will change the problem of this p stochastic variable into linear combination F that p stochastic variable is discussed 1, F 2..., F pProblem, it is called as principal component, and is separate between these linear combinations.F wherein 1, F 2..., F k(k≤p) fully reflects the information of former index according to the principle that keeps main amount of information.This process of reducing to a few overall target by a plurality of indexs is called dimensionality reduction on mathematics.The way that PCA is general is to seek the linear combination F of former index i, wherein:
F 1=u 11X 1+u 21X 2+…+u p1X p
F 2=u 12X 1+u 22X 2+…+u p2X p
……
F p=u 1pX 1+u 2pX 2+…+u ppX p
And,
u 1 i 2 + u 2 i 2 + · · · + u pi 2 = 1 .
Separate between the principal component, promptly non-overlapping information, that is:
Cov(F i,F j)=0,i≠j,i,j=1,2,…,p
The variance of principal component is successively decreased successively, and importance is successively decreased successively, promptly
Var(F 1)≥Var(F 2)≥…≥Var(F p)
By the introduction of front as can be known PCA can produce the statistical model of a compression---principal component model, model has provided the linear combination of stochastic variable, has described the main trend of data variation.Principal component model makes square redistribution of former stochastic variable data standard difference, and most of stochastic variable data standard differences square can be distributed on the first main composition, and next is distributed on the second main composition, and the rest may be inferred.By certain criterion last several main compositions are considered as decomposing residual error and are ignored, then might utilize less main composition that more information is described.For the data matrix X of a capable sampled value of given m and n row stochastic variable, every row is the sampling of n the variable of preset time.
The covariance of X is:
COV ( X ) = X ′ X m - 1
If X is normalization, covariance just becomes correlation matrix.PCA with data matrix X be decomposed into k (k≤min|m, n) vector product of individual vector and and the principal component model of a residual matrix E:
X=t 1p 1+t 2p 2+...+t mp m+E=X p+E
E is a residual matrix in the formula, and t is a score vector, and p is the characteristic vector of covariance.
The orthogonal matrix that score vector t forms
Figure A200910082913D00071
How inter-related described sampled value is.The orthonormal matrix that characteristic vector p forms ( p i T p j = 0 , When i ≠ j, p i T p j = 1 When i=j) to have described stochastic variable be how to be mutually related.
And:
COV(X)P i=λ iP i
P in the formula iBe the characteristic vector of covariance, λ iBe characteristic vector P iCharacteristic value.
The score of a main composition is meant resulting value when the characteristic value of all stochastic variables of given number of samples being carried out main composition valuation.For example: the first main composition has been weighed the linear combination degree of stochastic variable, and the variation that stochastic variable has been obtained maximum data in this direction is because P 1Be relevant with the eigenvalue of maximum of covariance.The second main composition has time many data variation, and is relevant with the second largest characteristic value of covariance, and and the first main composition quadrature.Illustrated that with the incoherent linear variable displacement combination of first main composition degree some variations and the first main composition are irrelevant.Briefly, for the m dimension variable space first characteristic vector P 1Defined maximum change direction, and the first score vector t 1Represented the projection on the first main composition axle of each sampling or observed value.To the capable n column matrix of a m X, can calculate n main composition, but because correlation and noise, the most preceding k main composition just is enough to explain the main of data have been changed, and can obtain the principal component model of k main composition.
Fig. 1 is the flow chart of the network anomaly detection method of a specific embodiment according to the present invention, and as shown in Figure 1, this method comprises the steps: that acquisition stream figureofmerit data set up the flow indicator data matrix by the regular hour window; This flow indicator data matrix is carried out normalized; Adopt the PCA method to set up the principal component model of described flow indicator data matrix; Adopt the SPE statistical chart of principal component model that thereby real data and normal model are compared the time point of finding that automatically tested exception of network traffic occurs; Find flow indicator component by the pivot contribution plot, thereby find to cause the unusual main cause that occurs the contribution maximum of exception of network traffic.Being described in detail as follows of this network anomaly detection method:
Network traffics application layer protocol Network Based is classified, and it comprises traditional bare flow, P2P flow, instant message flow, recreation flow, flow media flux, voip traffic and based on the flow of privately owned other agreement.It is which kind of application has caused that network is unusual that keeper's understanding is convenient in described classification.One of ordinary skill in the art will appreciate that, can also further segment network traffics based on arbitrary application layer protocol according to the degree of depth that detects.Acquisition stream figureofmerit data, it comprises the byte number indication information that network application type, port or IP address are right; The bag number indication information that network application type, port or IP address are right; The packet byte size information; And/or flow path direction information.The flow indicator data of being gathered are carried out statistic of classification based on the classification of above-mentioned flow.
Utilize the flow indicator data of statistic of classification to set up data matrix X---the flow indicator data matrix, a rational flow indicator data matrix can reflect the overall variation of network traffics, and will comprise the reason that abundant information helps the observer to note abnormalities and produce.Consider and detect emphasis and computational complexity that some that can select the flow indicator data makes up sets up the flow indicator data matrix.From above-mentioned flow indicator as can be seen, can add up according to byte and bag number respectively the flow indicator data, both are unanimous on the whole, but also there are differences, and are preferred, with both in conjunction with to reflect the feature of network traffics better.
According to one embodiment of present invention, adopt byte number flow indicator as shown in table 1 to set up the flow indicator data matrix according to above-mentioned classification.
The capable vector of table 1 byte number flow indicator data matrix constitutes
The index sequence number Index name Explanation
1 basic_in_byte Tradition flow immigration byte number
2 basic_out_byte Conventional flow measures the border byte number
3 p2p_in_byte The p2p byte number of entering a country
4 p2p_out_byte The p2p byte number of leaving the country
5 im_in_byte Instant message immigration byte number
6 im_out_byte Instant message departure byte number
7 game_in_byte Recreation immigration byte number
8 game_out_byte Recreation departure byte number
9 streaming_in_byte Streaming Media immigration byte number
10 streaming_out_byte Streaming Media departure byte number
The index sequence number Index name Explanation
11 voip_in_byte The VOIP byte number of entering a country
12 voip_out_byte VOIP situation byte number
13 others_in_byte Other byte number of entering a country
14 others_out_byte Other byte number of leaving the country
15 SUM(B_I_64) Immigration total amount of byte less than the bag of 64byte
16 SUM(B_O_64) Departure total amount of byte less than the bag of 64byte
17 SUM(B_I_128) Greater than 64 immigration total amount of bytes less than 128 bag
18 SUM(B_O_128) Greater than 64 departure total amount of bytes less than 128 bag
19 SUM(B_I_256) Greater than 128 immigration total amount of bytes less than 256 bag
20 SUM(B_O_256) Greater than 128 departure total amount of bytes less than 256 bag
21 SUM(B_I_512) Greater than 256 immigration total amount of bytes less than 512 bag
22 SUM(B_O_512) Greater than 256 departure total amount of bytes less than 512 bag
23 SUM(B_I_1024) Greater than 512 immigration total amount of bytes less than 1024 bag
24 SUM(B_O_1024) Greater than 512 departure total amount of bytes less than 1024 bag
25 SUM(B_I_1518) Greater than 1024 immigration total amount of bytes less than 1518 bag
26 SUM(B_O_1518) Greater than 1024 departure total amount of bytes less than the bag of 1518e
27 SUM(B_I_L1518) Immigration total amount of byte greater than the bag of 1518byte
28 SUM(B_O_L1518) Departure total amount of byte greater than the bag of 1518byte
The row vector of flow indicator data matrix is made of these 28 flow indicators, considers that flow is the cycle with the sky, adopt be spaced apart 5 minutes 24 hours sampled data altogether 288 samples as modeling space.
For eliminating the influence of actual dimension, above-mentioned flow indicator data are carried out normalization.Each flow indicator is as a stochastic variable, and the sampled data of each stochastic variable is deducted its average then divided by its standard deviation.Represent the data matrix of all stochastic variables with Xs through obtaining after the normalized:
X ‾ s = [ X - ( 1 . . . . . . . 1 ) T M ] diag [ 1 s 1 1 s 2 . . . . . 1 s m ] ;
M is the average of stochastic variable in the formula, M=[m 1m 2... m m],
S is the stochastic variable standard deviation, s = [ 1 s 1 1 s 2 . . . . . 1 s m ] .
One section network traffics between normal epoch the flow indicator sampled data after having carried out above-mentioned normalization, utilize aforesaid PCA method to set up principal component model, flow indicator data to be detected are detected being used for.
Carrying out principal component analysis according to the data matrix after the normalization obtains:
X ‾ s = t 1 p 1 T + t 2 p 2 T + . . . + t m p m T
As can (k<m) individual main composition comes the main variation in the representative data, can obtain principal component model: X=X with preceding k p+ E.
Table 2 is formation parameter lists of preceding 9 the main compositions of this principal component model, and C1-C28 represents raw bytes number flow indicator the 1st to No. 28.Table 3 is characteristic value, contribution rate and accumulation contribution counting rate meters of preceding 9 main compositions of this principal component model.The contribution rate that can find out the first main composition PC1 from table 3 is 74%, and to PC3, contribution rate of accumulative total has reached 86.7%.
Table 2 byte number flow indicator data matrix master composition constitutes table
Table 3 byte number flow indicator data matrix master compositional characteristic analytical table
Figure A200910082913D00103
Fig. 2 is the rubble figure of byte number flow indicator, shows the corresponding relation of principal component and its characteristic value, and for example the characteristic value of first principal component is 20.73.Curve among the figure presents comparatively desirable pattern, and only the main body that needs preceding 3 main compositions can describe flow indicator has changed, so set up the principal component model that comprises 3 main compositions.
Set up after the principal component model, utilize the multivariate statistics control chart that network traffics are carried out abnormality detection.One of ordinary skill in the art will appreciate that the multivariate statistics control chart comprises: square prediction error (SPE) statistical chart, T^2 statistical chart, pivot contribution plot etc.According to embodiments of the invention, adopt the SPE statistical chart of principal component model that thereby real data and normal model are compared the time of finding the unusual appearance of measured flux automatically, one of ordinary skill in the art will appreciate that the multivariate statistics control chart that can also adopt other carries out abnormality detection.Preferably, embodiments of the invention also comprise: by the flow indicator component of pivot contribution plot analysis to unusual contribution maximum, thereby find to cause the unusual main cause that occurs.
The SPE statistical chart claims the Q statistical chart again.What the SPE statistic was represented is the departure degree of the principal component model of network traffics to be detected with respect to network traffics principal component model just often, is to weigh measuring of model external data variation.The SPE statistic is also referred to as the Q statistic.Calculate the SPE statistic SPE of each sampling instant principal component model to be detected δ, and when calculating insolation level and being α, SPE statistic control limit SPE a
If SPE δSPE a, it is unusual to represent that this moment statistic occurs.Preferably, SPE aInsolation level α value be 0.95, to reach higher statistical test confidence level.The SPE statistic is formed by the comprehensive function of a plurality of stochastic variables, can monitor simultaneously a plurality of variablees.Generally SPE statistic and the control limit with each sampled value is drawn on the SPE statistical chart, and the point that exceeds the control limit then is possible abnormity point.
Fig. 3 is the SPE statistical chart of the byte number flow indicator data matrix of the specific embodiment according to the present invention, can see obviously that locating one at 160 exceeds a lot of unusual of control limit, SPE 160=297.416446.
When the SPE statistic exceeds its control in limited time, can judge that abnormal conditions have appearred in data to be tested, be that problem has appearred in what data but can not find out from the SPE statistical chart actually.One can help to determine that it is pivot contribution plot (Contributionplot) that unusual effective tool appears in which original component data.
According to the analysis of the pivot contribution plot of network traffics index to be detected, can determine to cause that the SPE statistic exceeds the control limit is the variation of which flow indicator.These analysis results are combined with relevant knowledge, will easily find out and cause unusual reason.Know 160 located unusual after, the contribution plot of 160 each stochastic variables of locating that draws, as shown in the figure.Shown in Fig. 4 to unusual contribution herein maximum be No. 2 and No. 22 original variables, just basic_out_byte in the row matrix vector and SUM (B_O_512) index, this explanation is to be caused by the basic application traffic of leaving the country herein unusually, mainly between 256byte and 512byte, such information can the very fast scope that helps the keeper to dwindle problem analysis for the size of packet.
Owing to know that abnormal flow is the departure flow of base application, can inquire about the departure bit rate figure of each agreement of base application at that time, as figure.Fig. 5 illustrates the bit rate surge of http protocol at that time.The keeper can investigate like this, checks whether the WEB server in the net occurs unusually.
Consider that byte number and number of data packets have reflected the different situations of network traffics,, adopt the bag information of number in the network traffics indication information to set up the flow indicator data matrix again according to embodiments of the invention.The flow indicator that this flow indicator data matrix comprises is similar to table 1, as showing:
The capable vector of table 4 bag number flow indicator data matrix constitutes
The index sequence number Index name Explanation
1 basic_in_pkt Tradition flow immigration bag number
2 basic_out_pkt Conventional flow measures border bag number
3 p2p_in_pkt P2p immigration bag number
27 SUM(P_1_L1518) The immigration bag sum of bag greater than 1518
28 SUM(P_O_L1518) The departure bag sum of bag greater than 1518
Same adopt be spaced apart 5 minutes 24 hours sampled data altogether 288 samples as modeling space.Fig. 6 is the rubble figure of bag number flow indicator, can find out that therefrom the main body that only needs preceding 3 main compositions can describe flow has changed.Fig. 7 is the SPE statistical chart of the principal component model of bag number flow indicator data matrix, its and Fig. 4 be illustrated in equally 160 locate one exceed the control limit a lot of unusually, SPE 160=299.074614.Fig. 8 respectively wraps number flow indicator contribution plot in the 160th moment of the specific embodiment according to the present invention, similar with Fig. 4, it is No. 2 and No. 22 original variables that Fig. 8 also shows what contribute maximum herein unusually, just basic_out_pkt in the row matrix vector and SUM (P_O_512) index, same explanation is to be caused by the basic application traffic of leaving the country herein unusually, and the size of packet is mainly between 256byte and 512byte.
Inquire about the departure packet rate figure of each agreement of base application at that time,, find the departure packet rate surge of http protocol at that time, notice that simultaneously the packet rate of base application proprietary protocol also has very big variation as Fig. 9.Compare the byte-rate figure in the same unusual moment of front, the byte-rate of base application proprietary protocol does not have big variation, and the data flow that can infer the base application proprietary protocol that these are unusual all is a small data packets.Hence one can see that, if can set up the flow indicator data matrix more careful, then the present invention will find more accurately and unusual reason occur.
According to one embodiment of present invention, provide a kind of network abnormality detection system, it is deployed in certain metropolitan area network exit of domestic certain operator, as shown in figure 10.This detection system is the complete distributed system that 2 to 7 layer network performance tests and flow analysis are provided of a cover, and it is made up of distributed network probe and the centralized analysis component of supporting multiple speed and interface.
Network probe is deployed on the outbound of metropolitan area network, and it is used for form 7*24 hour acquisition stream figureofmerit data by beam split, and these flow indicator data can reflect the operation integral status of network.Preferably, comprise application layer traffic identification and sort module (MPI) in the network probe, this module is used to discern the flow indicator data based on application layer, and statistic of classification is based on the flow indicator data of application layer.
The flow indicator data that analysis component periodically receives, preserves and the phase-split network probe is gathered detect to carry out exception of network traffic.Concrete, it sets up the flow indicator data matrix, adopts principal component analytical method to set up the principal component model of flow indicator data matrix, and the multivariate statistics control chart by principal component model carries out abnormality detection to network traffics.
The present invention carries out network traffics modeling and abnormality detection from network traffics index aspect, this is because detection system provides very abundant network index information, comprise the IP address to, port numbers, application layer type, byte number, bag number, flow amount and inflow and outflow direction etc., and these indication informations can reflect the operation integral status of network.Therefore the obtaining cost stream is much smaller than obtaining OD of flow indicator data, method operability provided by the invention is stronger.And, the accuracy that the exception of network traffic that suitable flow indicator data matrix can effectively improve detects.
Should be noted that and understand, under the situation that does not break away from the desired the spirit and scope of the present invention of accompanying Claim, can make various modifications and improvement the present invention of foregoing detailed description.Therefore, the scope of claimed technical scheme is not subjected to the restriction of given any specific exemplary teachings.

Claims (8)

1. a network flow abnormal detecting method comprises the following steps:
1) acquisition stream figureofmerit data are set up the flow indicator data matrix;
2) adopt principal component analytical method to set up the principal component model of described flow indicator data matrix;
3) the multivariate statistics control chart by described principal component model carries out abnormality detection to network traffics.
2. detection method according to claim 1, it is characterized in that, described flow indicator packet includes network application type, the right byte number indication information of port or IP address, the bag number indication information that network application type, port or IP address are right, packet byte size information, and/or flow path direction information.
3. detection method according to claim 1 and 2 is characterized in that, described step 1) further comprises:
Gather described flow indicator data, described flow indicator data are carried out statistic of classification based on application layer protocol;
Set up the flow indicator data matrix.
4. detection method according to claim 1 and 2 is characterized in that, described step 1) also comprises carries out normalized step to described flow indicator matrix.
5. detection method according to claim 1 and 2 is characterized in that, described step 3) is: adopt the square prediction error statistical chart to detect the time that described exception of network traffic occurs.
6. detection method according to claim 1 and 2 is characterized in that, described step 3) also comprises step:
Adopt the pivot contribution plot to analyze to the maximum flow indicator of described exception of network traffic contribution.
7. exception of network traffic detection system comprises following parts:
Network probe is used for acquisition stream figureofmerit data;
Analysis component is used to set up the flow indicator data matrix, adopts principal component analytical method to set up the principal component model of described flow indicator data matrix, and the multivariate statistics control chart by described principal component model carries out abnormality detection to network traffics.
8. detection system according to claim 7 is characterized in that, described network probe comprises application layer traffic identification and sort module, is used for based on the described flow indicator data of application layer protocol statistic of classification.
CN200910082913A 2009-04-24 2009-04-24 Method and system for detecting network flow exception Pending CN101534305A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN200910082913A CN101534305A (en) 2009-04-24 2009-04-24 Method and system for detecting network flow exception

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN200910082913A CN101534305A (en) 2009-04-24 2009-04-24 Method and system for detecting network flow exception

Publications (1)

Publication Number Publication Date
CN101534305A true CN101534305A (en) 2009-09-16

Family

ID=41104693

Family Applications (1)

Application Number Title Priority Date Filing Date
CN200910082913A Pending CN101534305A (en) 2009-04-24 2009-04-24 Method and system for detecting network flow exception

Country Status (1)

Country Link
CN (1) CN101534305A (en)

Cited By (18)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN102014031A (en) * 2010-12-31 2011-04-13 湖南神州祥网科技有限公司 Method and system for network flow anomaly detection
CN103259742A (en) * 2012-01-04 2013-08-21 国际商业机器公司 Activity-based virtual machine availability in a networked computing environment
CN104734894A (en) * 2013-12-18 2015-06-24 中国移动通信集团甘肃有限公司 Flow data screening method and device
CN105119734A (en) * 2015-07-15 2015-12-02 中国人民解放军防空兵学院 Full network anomaly detection positioning method based on robust multivariate probability calibration model
WO2016037579A1 (en) * 2014-09-12 2016-03-17 北京神州绿盟信息安全科技股份有限公司 Ddos attack detection method and apparatus
CN107070952A (en) * 2017-05-27 2017-08-18 郑州云海信息技术有限公司 A kind of network node Traffic Anomaly analysis method and system
CN107086944A (en) * 2017-06-22 2017-08-22 北京奇艺世纪科技有限公司 A kind of method for detecting abnormality and device
CN107683586A (en) * 2015-06-04 2018-02-09 思科技术公司 Method and apparatus for rare degree of the calculating in abnormality detection based on cell density
CN107733921A (en) * 2017-11-14 2018-02-23 深圳中兴网信科技有限公司 Network flow abnormal detecting method, device, computer equipment and storage medium
CN108055149A (en) * 2017-12-08 2018-05-18 国网辽宁省电力有限公司本溪供电公司 End-to-end Traffic Anomaly feature extracting method in a kind of Time and Frequency Synchronization application
CN108141349A (en) * 2015-10-02 2018-06-08 华为技术有限公司 Improve the method for abnormality detection rate
CN108737406A (en) * 2018-05-10 2018-11-02 北京邮电大学 A kind of detection method and system of abnormal flow data
CN109063533A (en) * 2018-04-30 2018-12-21 李泽中 A kind of dynamic face Fast Recognition Algorithm
CN110022248A (en) * 2019-04-19 2019-07-16 山东浪潮云信息技术有限公司 Link flow statistical method and system, traffic statistics host and statistics request end
CN112825506A (en) * 2019-11-21 2021-05-21 中国移动通信有限公司研究院 Flow mirror image detection method and device
CN114499997A (en) * 2021-12-30 2022-05-13 深圳供电局有限公司 Attack behavior detection method, apparatus, device, medium, and computer program product
CN114666117A (en) * 2022-03-17 2022-06-24 国网浙江省电力有限公司信息通信分公司 Network security situation measuring and predicting method for power internet
CN115988558A (en) * 2023-03-21 2023-04-18 中汽研软件测评(天津)有限公司 Intelligent vehicle data exit detection device, method, equipment and storage medium

Cited By (26)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN102014031A (en) * 2010-12-31 2011-04-13 湖南神州祥网科技有限公司 Method and system for network flow anomaly detection
CN103259742A (en) * 2012-01-04 2013-08-21 国际商业机器公司 Activity-based virtual machine availability in a networked computing environment
CN103259742B (en) * 2012-01-04 2016-12-28 国际商业机器公司 The method and system controlled based on movable virtual machine availability is carried out in networked computing environment
CN104734894A (en) * 2013-12-18 2015-06-24 中国移动通信集团甘肃有限公司 Flow data screening method and device
WO2016037579A1 (en) * 2014-09-12 2016-03-17 北京神州绿盟信息安全科技股份有限公司 Ddos attack detection method and apparatus
US11140197B2 (en) 2014-09-12 2021-10-05 NSFOCUS Information Technology Co., Ltd. Method and apparatus for DDoS attack detection
CN107683586A (en) * 2015-06-04 2018-02-09 思科技术公司 Method and apparatus for rare degree of the calculating in abnormality detection based on cell density
CN107683586B (en) * 2015-06-04 2021-07-20 思科技术公司 Method and apparatus for calculating cell density based dilution for use in anomaly detection
CN105119734A (en) * 2015-07-15 2015-12-02 中国人民解放军防空兵学院 Full network anomaly detection positioning method based on robust multivariate probability calibration model
CN105119734B (en) * 2015-07-15 2018-04-17 中国人民解放军防空兵学院 Whole network abnormality detection localization method based on healthy and strong multivariate probability calibrating patterns
CN108141349A (en) * 2015-10-02 2018-06-08 华为技术有限公司 Improve the method for abnormality detection rate
CN107070952A (en) * 2017-05-27 2017-08-18 郑州云海信息技术有限公司 A kind of network node Traffic Anomaly analysis method and system
CN107086944A (en) * 2017-06-22 2017-08-22 北京奇艺世纪科技有限公司 A kind of method for detecting abnormality and device
CN107086944B (en) * 2017-06-22 2020-04-21 北京奇艺世纪科技有限公司 Anomaly detection method and device
CN107733921A (en) * 2017-11-14 2018-02-23 深圳中兴网信科技有限公司 Network flow abnormal detecting method, device, computer equipment and storage medium
CN108055149A (en) * 2017-12-08 2018-05-18 国网辽宁省电力有限公司本溪供电公司 End-to-end Traffic Anomaly feature extracting method in a kind of Time and Frequency Synchronization application
CN109063533A (en) * 2018-04-30 2018-12-21 李泽中 A kind of dynamic face Fast Recognition Algorithm
CN108737406A (en) * 2018-05-10 2018-11-02 北京邮电大学 A kind of detection method and system of abnormal flow data
CN108737406B (en) * 2018-05-10 2020-08-04 北京邮电大学 Method and system for detecting abnormal flow data
CN110022248A (en) * 2019-04-19 2019-07-16 山东浪潮云信息技术有限公司 Link flow statistical method and system, traffic statistics host and statistics request end
CN112825506A (en) * 2019-11-21 2021-05-21 中国移动通信有限公司研究院 Flow mirror image detection method and device
CN114499997A (en) * 2021-12-30 2022-05-13 深圳供电局有限公司 Attack behavior detection method, apparatus, device, medium, and computer program product
CN114499997B (en) * 2021-12-30 2024-03-15 深圳供电局有限公司 Attack behavior detection method, apparatus, device, medium and computer program product
CN114666117A (en) * 2022-03-17 2022-06-24 国网浙江省电力有限公司信息通信分公司 Network security situation measuring and predicting method for power internet
CN115988558A (en) * 2023-03-21 2023-04-18 中汽研软件测评(天津)有限公司 Intelligent vehicle data exit detection device, method, equipment and storage medium
CN115988558B (en) * 2023-03-21 2023-11-24 中汽研软件测评(天津)有限公司 Intelligent vehicle data departure detection device, method, equipment and storage medium

Similar Documents

Publication Publication Date Title
CN101534305A (en) Method and system for detecting network flow exception
CN111898691B (en) River burst water pollution early warning and tracing method, system, terminal and medium
JP6141235B2 (en) How to detect anomalies in time series data
CN111506478A (en) Method for realizing alarm management control based on artificial intelligence
CN104360677B (en) Cigarette processing quality evaluation and diagnosis method
US20100071061A1 (en) Method and Apparatus for Whole-Network Anomaly Diagnosis and Method to Detect and Classify Network Anomalies Using Traffic Feature Distributions
CN105241680A (en) Health state assessment method for rotary machine based on probability density function
CN102789676B (en) Method for designing industrial alarm on basis of alarm evidence fusion
CN112861350B (en) Temperature overheating defect early warning method for stator winding of water-cooled steam turbine generator
CN112414694A (en) Equipment multistage abnormal state identification method and device based on multivariate state estimation technology
CN109359234B (en) Multi-dimensional network security event grading device
Ceschini et al. A Comprehensive Approach for Detection, Classification and Integrated Diagnostics of Gas Turbine Sensors (DCIDS)
CN112000081B (en) Fault monitoring method and system based on multi-block information extraction and Mahalanobis distance
CN104777115A (en) Water quality abnormal event detection method based on spectral statistical characteristics
CN111191720B (en) Service scene identification method and device and electronic equipment
Perry Identifying the time of polynomial drift in the mean of autocorrelated processes
CN117319047A (en) Network path analysis method and system based on network security anomaly detection
CN101106487A (en) A method and device for detecting exception of network traffic
CN108345289B (en) Industrial process stability detection method based on alternative data method
CN103389360A (en) Probabilistic principal component regression model-based method for soft sensing of butane content of debutanizer
CN117675230A (en) Knowledge-graph-based oil well data integrity identification method
Xiong et al. Nonuniversality of the horizontal visibility graph in inferring series periodicity
Ceschini et al. Resistant Statistical Methodologies for Anomaly Detection in Gas Turbine Dynamic Time Series: Development and Field Validation
CN114295162A (en) Environmental monitoring system based on data acquisition
CN115700553A (en) Anomaly detection method and related device

Legal Events

Date Code Title Description
C06 Publication
PB01 Publication
C10 Entry into substantive examination
SE01 Entry into force of request for substantive examination
C02 Deemed withdrawal of patent application after publication (patent law 2001)
WD01 Invention patent application deemed withdrawn after publication

Open date: 20090916