CN110445680A - Network flow abnormal detecting method, device and server - Google Patents
Network flow abnormal detecting method, device and server Download PDFInfo
- Publication number
- CN110445680A CN110445680A CN201910687114.0A CN201910687114A CN110445680A CN 110445680 A CN110445680 A CN 110445680A CN 201910687114 A CN201910687114 A CN 201910687114A CN 110445680 A CN110445680 A CN 110445680A
- Authority
- CN
- China
- Prior art keywords
- network flow
- reference value
- sample sequence
- data sample
- valid data
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L41/00—Arrangements for maintenance, administration or management of data switching networks, e.g. of packet switching networks
- H04L41/14—Network analysis or design
- H04L41/142—Network analysis or design using statistical or mathematical methods
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L41/00—Arrangements for maintenance, administration or management of data switching networks, e.g. of packet switching networks
- H04L41/14—Network analysis or design
- H04L41/147—Network analysis or design for predicting network behaviour
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L43/00—Arrangements for monitoring or testing data switching networks
- H04L43/08—Monitoring or testing based on specific metrics, e.g. QoS, energy consumption or environmental parameters
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L43/00—Arrangements for monitoring or testing data switching networks
- H04L43/08—Monitoring or testing based on specific metrics, e.g. QoS, energy consumption or environmental parameters
- H04L43/0876—Network utilisation, e.g. volume of load or congestion level
- H04L43/0888—Throughput
Abstract
The embodiment of the present invention proposes a kind of network flow abnormal detecting method, device and server, is related to network flow early warning technology field.This method and device are by judge whether extremely Normal Distribution determines the real network flow of object time using the Outlier Detection Algorithm being adapted with judging result and according to valid data sample sequence to valid data sample sequence.The present invention is by carrying out abnormality detection the valid data sample sequence being distributed in a manner of different distributions using different Outlier Detection Algorithms, so that detection process is more for specific aim, and then obtains good prediction effect.
Description
Technical field
The present invention relates to network flow early warning technology field, in particular to a kind of network flow abnormal detecting method,
Device and server.
Background technique
With the continuous development of network, the network user is more and more.Each company is for the body good, stable to user
Test, it will usually network flow is monitored in real time, so as in network link exception (such as by network attack, link congestion,
Flow is uprushed bust etc. in short-term), existing exception can be had found that it is likely that in time, reduce abnormal bring loss as far as possible.
Summary of the invention
In view of this, the purpose of the present invention is to provide a kind of network flow abnormal detecting method, device and servers.
To achieve the goals above, technical solution used in the embodiment of the present invention is as follows:
In a first aspect, the embodiment of the present invention provides a kind of network flow abnormal detecting method, the exception of network traffic inspection
Survey method includes:
Judge whether the valid data sample sequence obtained Normal Distribution and generates judging result, wherein described to have
Imitate the web-based history flow that data sample sequence includes multiple historical junctures；
According to the valid data sample sequence, mesh is determined using the Outlier Detection Algorithm being adapted with the judging result
Whether the real network flow for marking the moment is abnormal.
Second aspect, the embodiment of the present invention provide a kind of exception of network traffic detection device, the exception of network traffic inspection
Surveying device includes:
Judgment module, for judge obtain valid data sample sequence whether Normal Distribution and generate judgement knot
Fruit, wherein the valid data sample sequence includes the web-based history flow of multiple historical junctures；
Detection module, for utilizing the exception being adapted with the judging result according to the valid data sample sequence
Detection algorithm determines whether the real network flow of object time is abnormal.
The third aspect, the embodiment of the present invention provide a kind of server, including processor and memory, the memory storage
There is the machine-executable instruction that can be executed by the processor, the machine-executable instruction can be performed with reality in the processor
The existing any method of aforementioned embodiments.
Fourth aspect, the embodiment of the present invention provide a kind of computer readable storage medium, are stored thereon with computer program,
The exception of network traffic detection as described in any one of aforementioned embodiments is realized when the computer program is executed by processor
Method.
Network flow abnormal detecting method, device and server provided in an embodiment of the present invention, by judging valid data
Whether Normal Distribution is utilized and abnormal is examined with what judging result was adapted and according to valid data sample sequence sample sequence
Method of determining and calculating determines whether the real network flow of object time is abnormal.The present invention passes through effective to what is be distributed in a manner of different distributions
Data sample sequence is carried out abnormality detection using different Outlier Detection Algorithms, so that detection process is more for specific aim, in turn
Obtain good prediction effect.
To enable the above objects, features and advantages of the present invention to be clearer and more comprehensible, preferred embodiment is cited below particularly, and cooperate
Appended attached drawing, is described in detail below.
Detailed description of the invention
In order to illustrate the technical solution of the embodiments of the present invention more clearly, below will be to needed in the embodiment attached
Figure is briefly described, it should be understood that the following drawings illustrates only certain embodiments of the present invention, therefore is not construed as pair
The restriction of range for those of ordinary skill in the art without creative efforts, can also be according to this
A little attached drawings obtain other relevant attached drawings.
Fig. 1 shows the block diagram of server provided in an embodiment of the present invention.
Fig. 2 shows the flow charts of network flow abnormal detecting method provided in an embodiment of the present invention.
Fig. 3 shows the specific flow chart of S202 in Fig. 2.
Fig. 4 shows the specific flow chart of S2021 in Fig. 3.
Fig. 5 shows normal distribution curve.
Fig. 6 shows the further flow chart of network flow abnormal detecting method provided in an embodiment of the present invention.
Fig. 7 shows the functional block diagram of exception of network traffic detection device provided in an embodiment of the present invention.
Icon: 100- server；110- memory；120- processor；130- communication module；The inspection of 200- exception of network traffic
Survey device；210- judgment module；220- detection module；230- exception score determining module；240- exception level determining module；
250- update module.
Specific embodiment
Below in conjunction with attached drawing in the embodiment of the present invention, technical solution in the embodiment of the present invention carries out clear, complete
Ground description, it is clear that described embodiments are only a part of the embodiments of the present invention, instead of all the embodiments.Usually exist
The component of the embodiment of the present invention described and illustrated in attached drawing can be arranged and be designed with a variety of different configurations herein.
Therefore, the detailed description of the embodiment of the present invention provided in the accompanying drawings is not intended to limit below claimed
The scope of the present invention, but be merely representative of selected embodiment of the invention.Based on the embodiment of the present invention, those skilled in the art
Member's every other embodiment obtained without making creative work, shall fall within the protection scope of the present invention.
It should be noted that the relational terms of term " first " and " second " or the like be used merely to an entity or
Operation is distinguished with another entity or operation, and without necessarily requiring or implying between these entities or operation, there are any
This actual relationship or sequence.Moreover, the terms "include", "comprise" or its any other variant be intended to it is non-exclusive
Property include so that include a series of elements process, method, article or equipment not only include those elements, but also
Further include other elements that are not explicitly listed, or further include for this process, method, article or equipment it is intrinsic
Element.In the absence of more restrictions, the element limited by sentence "including a ...", it is not excluded that including described
There is also other identical elements in the process, method, article or equipment of element.
With the continuous development of network, each company is more and more based on the product line that Network stretch goes out, can be accurately
Obtaining the curve that network flow changes over time seems most important.In the prior art, scheme passes through training set training network
The regression model of flow sequence, and the network flow sequence prediction for the previous period based on the regression model and current time is worked as
The network flow magnitude at preceding moment, the network flow magnitude at the current time then obtained according to prediction obtain actual observation current
The network flow magnitude at moment carries out abnormality detection.
With this method the shortcomings that, is, first is that being not aware that the network flow sequence for the previous period at current time
Distribution mode, directly adopt that single method is predicted more general and there is not specific aim, leading to prediction result, there are larger
Error；Second is that model cannot be adjusted in time according to current prediction result, accumulated error always is understood, cause to predict that error is larger.
To which the present invention provides a kind of server, network flow abnormal detecting method and devices to solve the above problems.
Fig. 1 is please referred to, is the block diagram of server 100.The server 100 includes memory 110, processor
120 and communication module 130.The memory 110, processor 120 and each element of communication module 130 between each other directly or
Ground connection is electrically connected, to realize the transmission or interaction of data.For example, these elements can pass through one or more communication between each other
Bus or signal wire, which are realized, to be electrically connected.
Wherein, memory 110 is for storing program or data.The memory 110 may be, but not limited to, at random
It accesses memory (Random Access Memory, RAM), read-only memory (Read Only Memory, ROM), may be programmed
Read-only memory (Programmable Read-Only Memory, PROM), erasable read-only memory (Erasable
Programmable Read-Only Memory, EPROM), electricallyerasable ROM (EEROM) (Electric Erasable
Programmable Read-Only Memory, EEPROM) etc..
Data or program of the processor 120 for being stored in read/writable memory device 110, and execute corresponding function.
Communication module 130 is used to establish the communication connection between the server 100 and other communication terminals by network,
And for passing through the network sending and receiving data.
It should be understood that structure shown in FIG. 1 is only the structural schematic diagram of server 100, the server 100 may be used also
Including component more perhaps more less than shown in Fig. 1 or with the configuration different from shown in Fig. 1.Each group shown in Fig. 1
Part can be realized using hardware, software, or its combination.
The present invention also provides a kind of network flow abnormal detecting methods, are applied to above-mentioned server 100.Referring to Fig. 2,
For the flow chart of network flow abnormal detecting method provided by the invention.The network flow abnormal detecting method includes:
S201, judges whether the valid data sample sequence obtained Normal Distribution and generates judging result.
Wherein, valid data sample sequence includes corresponding web-based history flow of multiple historical junctures, can also include each
A historical juncture.In a kind of optional embodiment, which is that server 100 is real-time from database
It obtains.Wherein, which both can be set in the server 100, also can be set and communicates to connect in the server 100
Other servers 100.
It should be noted that valid data sample sequence is real-time update.Distinguishingly, if current time is first
It carves (at the time of i.e. first time obtains valid data sample sequence from database), then the valid data sample sequence at current time is
Pass through what is obtained after pre-processing to historical data sample sequence.And if current time is not first moment, when current
The valid data sample sequence at quarter is on the basis of the valid data sample sequence of the previous moment at current time, based on current
The real network flow of the previous moment at moment or the network flow correction value of the previous moment at current time are updated to obtain
's.
Wherein, historical data sample sequence includes the primitive network flow of multiple historical junctures；Meanwhile historical data sample
Sequence also can also include multiple historical junctures.Multiple primitive network flows that historical data sample sequence includes can be adopted by flow
Collection module collects from more interchangers, and is stored in a manner of time series to database.In a kind of optional implementation
In mode, flow collection module is primary original every acquisition in 1 minute (or other times interval, such as 2,3,4 minutes)
Network flow, and store to database.Meanwhile each of the links, divide uplink and downlink both direction to acquire, uplink traffic indicates single
The position time is injected into the byte number of the link, that is, uploads；Downlink traffic indicates that the unit time leaves the byte number of the link, i.e., under
It carries.That is, each historical juncture corresponding primitive network flow includes uplink traffic and downlink traffic.
For example, the data format for the historical data sample sequence that server 100 obtains can be as shown in table 1:
Table 1
Wherein, time 2018/10/1716:51 is a historical juncture, which corresponds in primitive network flow, under
Row network flow is 9131184bit/s, and uplink network flow is 3532293bit/s.
Due to directly carrying out exception of network traffic detection using historical data sample sequence, there can be biggish error.Cause
This, for improve testing result accuracy, first time judge current time real network flow whether exception when, need from
Database reads historical data sample sequence, and is pre-processed to obtain valid data sample sequence to historical data sample sequence
Column.It is alternatively possible to also be stored in obtained valid data sample sequence is pre-processed for the first time in database, but with history number
It is stored separately according to sample sequence.Alternatively, historical data sample sequence can also be covered valid data sample sequence.
Common pretreatment mode include but are not limited to data cleansing, normalized, standardization, at exceptional value
Reason and Supplementing Data processing etc..In the present embodiment, outlier processing and Supplementing Data processing are substantially carried out.That is,
Carrying out outlier processing and Supplementing Data processing to historical data sample sequence just can obtain valid data sample sequence.
In a kind of optional embodiment, the method that coarseness can be used filters out different in historical data sample sequence
Constant value.For example, first seek in historical data sample sequence, the mean value and standard deviation of all primitive network flows, and will meet l >
The primitive network flow of u'+3 σ ' or l <u'-3 σ ' are determined as exceptional value；Wherein, l is primitive network flow, and u' is all original
The mean value of network flow, σ ' are the standard deviation of all primitive network flows.
Exceptional value may be as caused by calculating mistake or the fault of flow collection module collection process, for this kind of exception
Value, needs to carry out outlier processing to it.Specifically, exceptional value is smoothed.For example, when where can taking exceptional value
The mean value of the front and back moment at quarter corresponding primitive network flow substitutes the exceptional value, and as the moment corresponding web-based history stream
Amount；Or the intermediate value of all primitive network flows can also be replaced to the exceptional value, and as the moment corresponding web-based history
Flow.
Such as, however, it is determined that the corresponding primitive network flow l of t moment1For exceptional value, and t-1 moment corresponding primitive network
Flow is l2, t+1 moment corresponding primitive network flow is l3, then by (l2+l3)/2 are as the corresponding web-based history stream of t moment
Amount.
Meanwhile during flow collection modules acquiring data, it is likely that do not collect the network flow at certain moment；
Or during data transmission, the network flow data at certain moment is lost.In short, loss of data equally will cause detection
As a result inaccurate.It is then desired to which the primitive network stream for whether lacking some historical juncture in historical data sample sequence first determined
Amount the case where missing if it exists, then needs to carry out completion to data.
It should be noted that Supplementing Data processing method includes but are not limited to averaging method, median method, incremental method, pre-
Survey method etc..For example, the web-based history flow using the mean value of all primitive network flows, median as the moment.
After obtaining valid data sample sequence, need to judge valid data sample sequence whether Normal Distribution.This
The normal distribution-test method that inventive embodiments provide can be but not limited to χ2Test of fitness of fot method, K-S Normal distribution test method,
Kurtosis skewness test etc..In the present embodiment, it by taking kurtosis skewness test as an example, illustrates how to judge valid data sample
Sequence whether Normal Distribution.Kurtosis skewness test is also known as Jarque-Bera (JB) method of inspection, and the principle is as follows:
Coefficient of skewness S is calculated first, and coefficient of skewness S is the probability density function symmetry to valid data sample sequence
Measurement.Wherein, coefficient of skewness S meets formula:
Wherein, xt' it is web-based history flow, and t=1,2,3 ...,For the mean value of valid data sample sequence, σ is
The standard deviation of valid data sample sequence, n are the sample size of valid data sample sequence.
Then coefficient of kurtosis K is calculated, coefficient of kurtosis K is the measurement fat or thin to probability density function.Wherein, coefficient of kurtosis K
Meet formula:
Test statistics JB is finally calculated according to coefficient of skewness S and coefficient of kurtosis K:
If the value of test statistics JB is greater than presetting critical value χ2, then show that the valid data sample sequence is disobeyed
Normal distribution.If the value of test statistics JB is less than or equal to presetting critical value χ2, then show the valid data sample sequence
Column Normal Distribution.Distinguishingly, when test statistics JB is 0, show that the valid data sample sequence is obeying standard just
State distribution.
It should be noted that when primitive network flow includes uplink traffic and downlink traffic, to historical data sample
When this sequence is pre-processed, needs to pre-process uplink traffic therein and downlink traffic, have obtained from
Imitate actually includes uplink traffic sequence and downlink traffic sequence in data sample sequence；Therefore, it is necessary to utilize the above method, point
Do not judge whether uplink traffic sequence that valid data sample sequence includes and downlink traffic sequence meet normal distribution and life
At judging result.
S202 determines target using the Outlier Detection Algorithm being adapted with judging result according to valid data sample sequence
Whether the real network flow at moment is abnormal.
Wherein, real network flow is flow collection module in the collected network flow of object time.One kind can
In the embodiment of choosing, if judging result is valid data sample sequence Normal Distribution, the abnormality detection adapted to it
Algorithm meets K-sigma principle: if judging result is that valid data sample sequence disobeys normal distribution, adapting to it different
Normal detection algorithm is self-adaptive averaging factor algorithm.
Referring to Fig. 3, being the further flow chart of network flow abnormal detecting method provided by the invention.The S202 packet
It includes:
S2021 determines first using the Outlier Detection Algorithm being adapted with judging result according to valid data sample sequence
A reference value and the second a reference value.
Similarly, determine that first reference value and the Outlier Detection Algorithm of the second a reference value are associated with judging result.Judgement knot
Fruit includes that effective data sample sequence Normal Distribution and valid data sample sequence disobey both knots of normal distribution
Fruit.In a kind of optional embodiment, if judging result is valid data sample sequence Normal Distribution, K- is utilized
Sigma principle determines first reference value and the second a reference value；If judging result is that valid data sample sequence disobeys normal state point
Cloth then determines first reference value and the second a reference value using adaptive exponential smoothing algorithm.Wherein, first reference value is greater than second
A reference value.
To, if referring to Fig. 4, S2021 include: judging result be valid data sample sequence Normal Distribution,
Execute S20211；If judging result is that valid data sample sequence disobeys normal distribution, S20213 is executed.
S20211, according to the web-based history flow of multiple historical junctures, calculate valid data sample sequence desired value and
Standard deviation.
Wherein, K-sigma principle is to contain in ± 3 σ of u when time series data Normal Distribution (as shown in Figure 5)
99.73% time series data, being distributed in the data other than ± 3 σ of u is abnormal data.
Therefore, the multiple web-based history flow rate calculation valid data sample sequences for first including based on valid data sample sequence
Desired value and standard deviation.
S20212 determines first reference value and the second a reference value according to desired value and standard deviation.
Specifically, in valid data sample sequence Normal Distribution, first reference value B1Meet: B1=μ+3 σ, second
A reference value B2Meet: B2=μ -3 σ；Wherein, μ is the desired value of valid data sample sequence, and σ is the mark of valid data sample sequence
It is quasi- poor.
S20213 calculates the standard deviation of valid data sample sequence according to the web-based history flow of multiple historical junctures.
S20214 determines valid data sample sequence in the smoothing factor of the previous moment of object time.
Adaptive exponential smoothing algorithm provided in an embodiment of the present invention is automatic adjustment on the basis of single exponential smoothing
Smoothing factor, can the variation of tracking prediction target in time finally to predict that error reaches to reduce prediction accumulated error
It is minimum.Wherein, the principle of single exponential smoothing is the reality that the exponential smoothing value of any moment is all the previous moment at the moment
The weighted average of the exponential smoothing value of value and previous moment, fundamental formular are as follows:
yt+1=α * yt'+(1-α)yt
Wherein, yt+1For the predicting network flow value at t+1 moment, yt' be t moment network flow actual value, ytFor t moment
Predicting network flow value (namely exponential smoothing value of t moment), α is smoothing factor, also referred to as weighted factor, and having reacted has
The severe degree for imitating data sample sequence variation, generally takes (0.05,0.3).
Normally, the predicting network flow value y at the first moment1It can be equal to the network flow actual value y at the first moment1', i.e.,
y1=y1'。
And the determination method of smoothing factor is as follows in Self adaptive thrice exponential smoothing:
Assuming that in the Smoothing Prediction error e of t momenttMeet: et=yt'-yt；
Wherein, yt' be t moment network flow actual value, ytFor the predicting network flow value of t moment.
Integrated forecasting error EtMeet: Et+1=β et+1+(1-β)Et(t=2,3,4 ...)；
Wherein, β is the weight for predicting error, generally takes empirical value (0.1 or 0.2), and take E1=0.
Absolute smoothing error MtMeet: Mt+1=β | et+1|+(1-β)Mt(M1=0, and t=2,3,4 ...)；
Then, self-adaptive averaging factor αtMeet: αt=| Et|/Mt(t=2,3,4 ... and α1=0)；
After finding out adaptive smooth parameter, so that it may use dynamic alphatInstead of constant α, to obtain adaptive exponential smoothing
Prediction model it is as follows:
yt+1=αtyt'+(1-αt)yt；
To determine that the process of the smoothing factor of the previous moment of object time is as follows:
A kind of possible embodiment is, according to α1=0, y1=y1' and formula yt+1=αtyt'+(1-αt)ytIt calculates
The predicting network flow value y at the second moment out2, in conjunction with the network flow actual value y at the second moment2' calculate the second moment finger
Number smoothing prediction error e2；Then according to the Smoothing Prediction error e at the second moment2The integrated forecasting for calculating for the second moment misses
Poor E2And absolutely smoothing error M2, and calculate with this smoothing factor α at the second moment2；Then according to the smooth system at the second moment
Number α2, the second moment network flow actual value y2' and the second moment predicting network flow value y2Calculate the net at third moment
Network traffic prediction value y3... and so on, the smoothing factor of the previous moment of object time is calculated, thereby may be ensured that
The accuracy for the smoothing factor determined.
Alternatively possible embodiment is, according to above-mentioned smoothing factor correlation formula, to precompute each moment pair
The smoothing factor answered, then stores into database, can be to the history net in valid data sample sequence when being stored
Network flow is numbered, and after the smoothing factor for then determining each moment again, establishes the corresponding relationship of number with smoothing factor.
In this way, when it needs to be determined that object time network second prediction network flow when, then can determine object time it is previous when
The number being engraved in valid data sample sequence is then based on the number and determines that the number is corresponding flat from above-mentioned corresponding relationship
The smoothing factor of the previous moment of sliding coefficient namely object time.It is gone through for example, valid data sample sequence includes 1440 altogether
History moment and corresponding web-based history flow of each historical juncture, if the historical juncture position determined by the previous moment of object time
The 128th in 1440 historical junctures, i.e. the number of the previous moment of object time is 128, then it is above-mentioned right to be based on
It should be related to and determine the corresponding smoothing factor α of number 128128, and by α128It is determined as valid data sample sequence in the target
The smoothing factor of the previous moment at moment, and α128=| E128|/M128.It is possible thereby to quickly determine object time it is previous when
The smoothing factor at quarter, and then can quickly determine the second prediction network flow of object time, it accelerates to a certain extent
Determine object time real network whether Yi Chang speed.
S20215 obtains target according to smoothing factor, the first web-based history flow and the first prediction predicting network flow
The second prediction network flow at moment.
Wherein, the first prediction network flow is the network flow of the previous moment for the object time that prediction obtains.First goes through
History network flow is the corresponding web-based history flow of previous moment of object time.
By taking object time was the 129th moment as an example, then the second network flow of object time is y129, then y129=α128y128'+(1-α128)y128, wherein α128It can be determined by any embodiment provided in step S20214.
S20216 determines first reference value and the second a reference value according to the second prediction network flow and standard deviation.
Specifically, when valid data sample sequence disobeys normal distribution, first reference value B1Meet: B1=yt+ 3 σ,
Second a reference value B2Meet: B2=yt-3σ；Wherein, ytFor the second prediction network flow, σ is the standard of valid data sample sequence
Difference.
S2022, judges whether real network flow is greater than first reference value, if it is, executing S2024；If it is not, then
Execute S2023.
Specifically, if judging result is valid data sample sequence Normal Distribution, judge that real network flow is
It is no to be greater than+3 σ of μ；If judging result is that valid data sample sequence disobeys normal distribution, whether real network flow is judged
Greater than yt+3σ。
S2023 judges real network flow whether less than the second a reference value, if it is, continuing to execute S2024；If
It is no, then execute S2025.
Specifically, if judging result is valid data sample sequence Normal Distribution, judge that real network flow is
It is no to be less than μ -3 σ；If judging result is that valid data sample sequence disobeys normal distribution, whether real network flow is judged
Less than yt-3σ。
S2024 determines real network Traffic Anomaly.
Specifically, if real network flow is greater than first reference value or less than the second a reference value, show the real network stream
Amount is abnormal data.
S2025 determines that real network flow is normal.
Specifically, if real network flow is no more than first reference value and is not less than the second a reference value, it is determined that practical net
Network flow is normal.
In order to more intuitively allow user to know the intensity of anomaly of abnormal network flow, the exception of network flow can be determined
Grade, referring to Fig. 6, network flow abnormal detecting method provided by the invention further include:
S203, if real network flow is greater than first reference value or less than the second a reference value, according to real network flow
And first reference value or real network flow and the second a reference value determine abnormal score.
Wherein, it if real network flow is greater than first reference value, is determined according to first reference value and real network flow
Abnormal score；If real network flow determines abnormal less than the second a reference value according to the second a reference value and real network flow
Score.
Specifically, if real network flow is greater than first reference value, real network flow and first reference value are formed
Difference be determined as abnormal score with the ratio of first reference value.That is, real network flow, first reference value and abnormal score
Meet formula:
Wherein, score is abnormal score, xtFor real network flow, B1For first reference value.
If the difference that less than the second a reference value, the second a reference value and real network flow are formed for real network flow and the
The ratio of two a reference values is determined as abnormal score.It is calculated that is, real network flow, the second a reference value and abnormal score meet
Formula:
Wherein, score is abnormal score, xtFor real network flow, B2For the second a reference value.
S204 determines the exception level of the network flow of object time according to abnormal score.
It should be noted that abnormal score is higher, then the exception level of the network flow of object time is higher.
In a kind of optional embodiment, if s1<socre≤s2, then the exception level of the network flow of object time be
Slight abnormality；If s2<socre≤s3, then the exception level of the network flow of object time is medium exception；If socre > s3, then
The exception level of the network flow of object time is severely subnormal.In a kind of optional embodiment, server 100 can basis
Different exception levels generates different prompt informations, to remind the intensity of anomaly of user's real network flow.In this way, user can
Different counter-measures to be performed quickly according to prompt information, to guarantee that communication link normally provides service.
Wherein, s1For the first presetting score threshold, s2For the second presetting score threshold, s3It is presetting
Three score thresholds, and s1<s2<s3.In a kind of optional embodiment, the first presetting score threshold is 0, presetting
Second score threshold is 20, and presetting third score threshold is 50.
S205 determines network flow correction value according to the web-based history flow of multiple historical junctures.
If the real network Traffic Anomaly of object time, also need to be determined according to the web-based history flow of multiple historical junctures
The network flow correction value of object time.The network flow correction value can with but be not limited only to multiple in valid data sample sequence
The intermediate value etc. of multiple web-based history flows in the mean value of web-based history flow, valid data sample sequence.
S206 is updated valid data sample sequence based on network flow correction value and object time.
Specifically, after carrying out abnormality detection to object time, which will become the historical juncture, then by target
The network flow correction value at moment is updated valid data sample sequence as the corresponding web-based history flow of object time,
And store into database, so that the subsequent time of object time can judge object time based on new valid data sample sequence
Subsequent time real network flow it is whether abnormal.
S207 determines abnormal be scored at if real network flow is no more than first reference value and is not less than the second a reference value
0。
In the case where real network flow is no more than first reference value and is not less than the second a reference value, real network flow
Normally, 0 is scored at so as to directly determine exception.
S208 is updated valid data sample sequence based on real network flow and object time.
Since the real network flow of object time is normal, there is no need to handle it, directly by object time
Real network flow is updated valid data sample sequence as the corresponding web-based history flow of object time, and store to
In database, so that the subsequent time of object time can judge lower a period of time of object time based on new valid data sample sequence
Whether the real network flow at quarter is abnormal.
To which exception of network traffic detection algorithm provided by the invention first determines whether valid data sample sequence takes
From normal distribution, if valid data sample sequence Normal Distribution, first reference value and are determined using K-sigma principle
Two a reference values；If valid data sample sequence disobeys normal distribution, the first base is determined using adaptive exponential smoothing algorithm
Quasi- value and the second a reference value, thus according to first reference value and the second a reference value judge object time real network flow whether
It is abnormal.Due to that can determine the first base using different algorithms when valid data sample sequence obeys or disobeys normal distribution
Quasi- value and the second a reference value；Namely different abnormality detections is used to the valid data sample sequence being distributed in a manner of different distributions
Algorithm carries out abnormality detection, so that detection process is more for specific aim, and then obtains more accurately prediction effect.
In order to execute the corresponding steps in above-described embodiment and each possible mode, it is different that a kind of network flow is given below
The implementation of normal detection device 200, optionally, which can use above-mentioned shown in FIG. 1
The device architecture of processor 120.Further, referring to Fig. 7, Fig. 7 is that a kind of network flow provided in an embodiment of the present invention is different
The functional block diagram of normal detection device 200.It should be noted that exception of network traffic detection device provided by the present embodiment
200, the technical effect of basic principle and generation is identical with above-described embodiment, and to briefly describe, the present embodiment part is not referred to
Place can refer to corresponding contents in the above embodiments.The exception of network traffic detection device 200 include: judgment module 210,
Detection module 220, abnormal score determining module 230, exception level determining module 240 and update module 250.
Judgment module 210 be used for judge obtain valid data sample sequence whether Normal Distribution and generate judgement tie
Fruit.
It is to be appreciated that judgment module 210 can be used for executing S201 in a kind of optional embodiment.
Detection module 220 is used to be calculated according to valid data sample sequence using the abnormality detection being adapted with judging result
Method determines whether the real network flow of object time is abnormal.
Specifically, detection module 220 is used to utilize the exception being adapted with judging result according to valid data sample sequence
Detection algorithm determines first reference value and the second a reference value, and if real network flow be greater than first reference value or less than second
A reference value, it is determined that real network Traffic Anomaly；If real network flow is no more than first reference value and is not less than the second benchmark
Value, it is determined that real network flow is normal.
Wherein, if judging result is valid data sample sequence Normal Distribution, according to going through for multiple historical junctures
History network flow calculates the desired value and standard deviation of valid data sample sequence, and determines the first base according to desired value and standard deviation
Quasi- value and the second a reference value.
If judging result is that valid data sample sequence disobeys normal distribution, according to the history net of multiple historical junctures
The standard deviation of network flow rate calculation valid data sample sequence, and determine valid data sample sequence in the previous moment of object time
Smoothing factor, then according to smoothing factor, the first web-based history flow and first prediction predicting network flow obtain target
The second prediction network flow at moment, finally determines first reference value and the second base according to the second prediction network flow and standard deviation
Quasi- value.
It is to be appreciated that in a kind of optional embodiment, detection module 220 can be used for executing S202, S2021,
S20211, S20212, S20213, S20214, S20215, S20216, S2022, S2023, S2024 and S2025.
If abnormal score determining module 230 for real network flow is greater than first reference value or less than the second a reference value,
Abnormal score is then determined according to real network flow and first reference value or real network flow and the second a reference value.
It is to be appreciated that abnormal score determining module 230 can be used for executing S203 in a kind of optional embodiment.
Exception level determining module 240 is used to determine the exception level of the network flow of object time according to abnormal score.
It is to be appreciated that exception level determining module 240 can be used for executing S204 in a kind of optional embodiment.
If update module 250 is used for the real network Traffic Anomaly of object time, according to the history of multiple historical junctures
Network flow determines network flow correction value, and based on network flow correction value and object time to valid data sample sequence into
Row updates.
It is to be appreciated that update module 250 can be used for executing S205 and S206 in a kind of optional embodiment.
If abnormal score determining module 230 is also used to real network flow no more than first reference value and not less than the second base
Quasi- value, it is determined that abnormal to be scored at 0.
It is to be appreciated that abnormal score determining module 230 can be used for executing S207 in a kind of optional embodiment.
If the real network flow that update module 250 is also used to object time is normal, it is based on real network flow and mesh
The mark moment is updated valid data sample sequence.
It is to be appreciated that update module 250 can be used for executing S205 and S208 in a kind of optional embodiment.
Optionally, above-mentioned module can be stored in memory 110 shown in FIG. 1 in the form of software or firmware (Firmware)
In or solidify in the operating system (Operating System, OS) of the server 100, and can be by the processor 120 in Fig. 1
It executes.Meanwhile the code etc. of data needed for executing above-mentioned module, program can store in the memory 110.
In conclusion network flow abnormal detecting method provided by the invention, device and server, by judging significant figure
According to sample sequence, whether Normal Distribution utilizes the exception being adapted with judging result and according to valid data sample sequence
Detection algorithm determines whether the real network flow of object time is abnormal.The present invention to what is be distributed in a manner of different distributions by having
Effect data sample sequence is carried out abnormality detection using different Outlier Detection Algorithms, so that detection process is more for specific aim, into
And obtain good prediction effect.
In several embodiments provided herein, it should be understood that disclosed device and method can also pass through
Other modes are realized.The apparatus embodiments described above are merely exemplary, for example, flow chart and block diagram in attached drawing
Show the device of multiple embodiments according to the present invention, the architectural framework in the cards of method and computer program product,
Function and operation.In this regard, each box in flowchart or block diagram can represent the one of a module, section or code
Part, a part of the module, section or code, which includes that one or more is for implementing the specified logical function, to be held
Row instruction.It should also be noted that function marked in the box can also be to be different from some implementations as replacement
The sequence marked in attached drawing occurs.For example, two continuous boxes can actually be basically executed in parallel, they are sometimes
It can execute in the opposite order, this depends on the function involved.It is also noted that every in block diagram and or flow chart
The combination of box in a box and block diagram and or flow chart can use the dedicated base for executing defined function or movement
It realizes, or can realize using a combination of dedicated hardware and computer instructions in the system of hardware.
In addition, each functional module in each embodiment of the present invention can integrate one independent portion of formation together
Point, it is also possible to modules individualism, an independent part can also be integrated to form with two or more modules.
It, can be with if the function is realized and when sold or used as an independent product in the form of software function module
It is stored in a computer readable storage medium.Based on this understanding, technical solution of the present invention is substantially in other words
The part of the part that contributes to existing technology or the technical solution can be embodied in the form of software products, the meter
Calculation machine software product is stored in a storage medium, including some instructions are used so that a computer equipment (can be a
People's computer, server or network equipment etc.) it performs all or part of the steps of the method described in the various embodiments of the present invention.
And storage medium above-mentioned includes: that USB flash disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), arbitrary access are deposited
The various media that can store program code such as reservoir (RAM, Random Access Memory), magnetic or disk.
The foregoing is only a preferred embodiment of the present invention, is not intended to restrict the invention, for the skill of this field
For art personnel, the invention may be variously modified and varied.All within the spirits and principles of the present invention, made any to repair
Change, equivalent replacement, improvement etc., should all be included in the protection scope of the present invention.
Claims (11)
1. a kind of network flow abnormal detecting method, which is characterized in that the network flow abnormal detecting method includes:
Judge whether the valid data sample sequence obtained Normal Distribution and generates judging result, wherein the significant figure
It include the web-based history flow of multiple historical junctures according to sample sequence；
According to the valid data sample sequence, when determining target using the Outlier Detection Algorithm being adapted with the judging result
Whether the real network flow at quarter is abnormal.
2. network flow abnormal detecting method according to claim 1, which is characterized in that described according to the valid data
Sample sequence, using with the judging result be adapted Outlier Detection Algorithm determine object time real network flow whether
Abnormal step includes:
According to the valid data sample sequence, the first base is determined using the Outlier Detection Algorithm being adapted with the judging result
Quasi- value and the second a reference value, wherein the first reference value is greater than second a reference value；
If the real network flow is greater than the first reference value or is less than second a reference value, it is determined that the practical net
Network Traffic Anomaly；
If the real network flow is no more than the first reference value and is not less than second a reference value, it is determined that the reality
Internet flow is normal.
3. network flow abnormal detecting method according to claim 2, which is characterized in that if the judging result is described
Valid data sample sequence Normal Distribution, then it is described according to the valid data sample sequence, it is tied using with the judgement
The step of adaptable Outlier Detection Algorithm of fruit determines first reference value and the second a reference value include:
According to the web-based history flow of the multiple historical juncture, the desired value and standard of the valid data sample sequence are calculated
Difference；
The first reference value and second a reference value are determined according to the desired value and the standard deviation.
4. network flow abnormal detecting method according to claim 2, which is characterized in that the multiple historical juncture is gone through
History network flow includes the first web-based history flow of the previous moment of the object time；
If the judging result is that the valid data sample sequence disobeys normal distribution, described according to the valid data
Sample sequence determines the step of first reference value and the second a reference value using the Outlier Detection Algorithm being adapted with the judging result
Suddenly include:
According to the web-based history flow of the multiple historical juncture, the standard deviation of the valid data sample sequence is calculated；
Determine the smoothing factor of previous moment of the valid data sample sequence in the object time；
The target is obtained according to the smoothing factor, the first web-based history flow and the first prediction predicting network flow
The second prediction network flow at moment, wherein the first prediction network flow be before predicting the obtained object time
The network flow at one moment；
The first reference value and second a reference value are determined according to the second prediction network flow and the standard deviation.
5. the network flow abnormal detecting method according to any one of claim 2-4, which is characterized in that the method
Further include:
If the real network flow is greater than the first reference value, according to the first reference value and the real network stream
Amount determines abnormal score；
If the real network flow is less than second a reference value, according to second a reference value and the real network stream
Amount determines the abnormal score.
6. network flow abnormal detecting method according to claim 5, which is characterized in that described according to first benchmark
Value and the real network flow determine that the step of abnormal score includes:
The ratio of difference and the first reference value that the real network flow and the first reference value are formed is determined as institute
State abnormal score；
Described the step of determining the abnormal score according to second a reference value and the real network flow includes:
The ratio of difference and second a reference value that second a reference value and the real network flow are formed is determined as institute
State abnormal score.
7. network flow abnormal detecting method according to claim 5, which is characterized in that the method also includes:
The exception level of the real network flow of the object time is determined according to the abnormal score.
8. network flow abnormal detecting method described in any one of -4 according to claim 1, which is characterized in that the method
Further include:
If the real network Traffic Anomaly of the object time, determined according to the web-based history flow of the multiple historical juncture
Network flow correction value；
The valid data sample sequence is updated based on the network flow correction value and the object time.
9. a kind of exception of network traffic detection device, which is characterized in that the exception of network traffic detection device includes:
Judgment module, for judging whether the valid data sample sequence obtained Normal Distribution and generates judging result,
In, the valid data sample sequence includes the web-based history flow of multiple historical junctures；
Detection module, for utilizing the abnormality detection being adapted with the judging result according to the valid data sample sequence
Algorithm determines whether the real network flow of object time is abnormal.
10. a kind of server, which is characterized in that including processor and memory, the memory is stored with can be by the place
The machine-executable instruction that device executes is managed, the machine-executable instruction can be performed to realize claim 1-8 in the processor
Any network flow abnormal detecting method.
11. a kind of computer readable storage medium, is stored thereon with computer program, which is characterized in that the computer program
Such as network flow abnormal detecting method of any of claims 1-8 is realized when being executed by processor.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910687114.0A CN110445680B (en) | 2019-07-29 | 2019-07-29 | Network traffic anomaly detection method and device and server |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910687114.0A CN110445680B (en) | 2019-07-29 | 2019-07-29 | Network traffic anomaly detection method and device and server |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN110445680A true CN110445680A (en) | 2019-11-12 |
| CN110445680B CN110445680B (en) | 2021-06-08 |
Family
ID=68431913
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201910687114.0A Active CN110445680B (en) | 2019-07-29 | 2019-07-29 | Network traffic anomaly detection method and device and server |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN110445680B (en) |
Cited By (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111092891A (en) * | 2019-12-20 | 2020-05-01 | 杭州安恒信息技术股份有限公司 | Method, system and related device for detecting abnormal point in network |
| CN111245684A (en) * | 2020-01-13 | 2020-06-05 | 智者四海（北京）技术有限公司 | Traffic scheduling method and device, electronic equipment and computer readable medium |
| CN111884874A (en) * | 2020-07-15 | 2020-11-03 | 中国舰船研究设计中心 | Programmable data plane-based ship network real-time anomaly detection method |
| CN112911627A (en) * | 2019-11-19 | 2021-06-04 | 中国电信股份有限公司 | Wireless network performance detection method, device and storage medium |
| CN113645215A (en) * | 2021-08-03 | 2021-11-12 | 恒安嘉新（北京）科技股份公司 | Method, device, equipment and storage medium for detecting abnormal network traffic data |
| CN114741377A (en) * | 2022-04-01 | 2022-07-12 | 深圳市爱路恩济能源技术有限公司 | Method and device for identifying and processing natural gas abnormal data |
| CN116519021A (en) * | 2023-06-29 | 2023-08-01 | 西北工业大学 | Inertial navigation system fault diagnosis method, system and equipment |
Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN1617512A (en) * | 2004-11-25 | 2005-05-18 | 中国科学院计算技术研究所 | Adaptive network flow forecasting and abnormal alarming method |
| CN101651568A (en) * | 2009-07-01 | 2010-02-17 | 青岛农业大学 | Method for predicting network flow and detecting abnormality |
| CN107483455A (en) * | 2017-08-25 | 2017-12-15 | 国家计算机网络与信息安全管理中心 | A kind of network node abnormality detection method and system based on stream |
| CN107579981A (en) * | 2017-09-08 | 2018-01-12 | 北京神州绿盟信息安全科技股份有限公司 | A kind of network flow monitoring method and system |
| CN109558295A (en) * | 2018-11-15 | 2019-04-02 | 新华三信息安全技术有限公司 | A kind of performance indicator method for detecting abnormality and device |
| CN109726198A (en) * | 2018-12-06 | 2019-05-07 | 中科恒运股份有限公司 | Method for processing abnormal data and device |
| CN109873712A (en) * | 2018-05-18 | 2019-06-11 | 新华三信息安全技术有限公司 | A kind of network flow prediction method and device |
-
2019
- 2019-07-29 CN CN201910687114.0A patent/CN110445680B/en active Active
Patent Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN1617512A (en) * | 2004-11-25 | 2005-05-18 | 中国科学院计算技术研究所 | Adaptive network flow forecasting and abnormal alarming method |
| CN101651568A (en) * | 2009-07-01 | 2010-02-17 | 青岛农业大学 | Method for predicting network flow and detecting abnormality |
| CN107483455A (en) * | 2017-08-25 | 2017-12-15 | 国家计算机网络与信息安全管理中心 | A kind of network node abnormality detection method and system based on stream |
| CN107579981A (en) * | 2017-09-08 | 2018-01-12 | 北京神州绿盟信息安全科技股份有限公司 | A kind of network flow monitoring method and system |
| CN109873712A (en) * | 2018-05-18 | 2019-06-11 | 新华三信息安全技术有限公司 | A kind of network flow prediction method and device |
| CN109558295A (en) * | 2018-11-15 | 2019-04-02 | 新华三信息安全技术有限公司 | A kind of performance indicator method for detecting abnormality and device |
| CN109726198A (en) * | 2018-12-06 | 2019-05-07 | 中科恒运股份有限公司 | Method for processing abnormal data and device |
Cited By (12)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112911627A (en) * | 2019-11-19 | 2021-06-04 | 中国电信股份有限公司 | Wireless network performance detection method, device and storage medium |
| CN112911627B (en) * | 2019-11-19 | 2023-03-21 | 中国电信股份有限公司 | Wireless network performance detection method, device and storage medium |
| CN111092891A (en) * | 2019-12-20 | 2020-05-01 | 杭州安恒信息技术股份有限公司 | Method, system and related device for detecting abnormal point in network |
| CN111092891B (en) * | 2019-12-20 | 2022-04-01 | 杭州安恒信息技术股份有限公司 | Method, system and related device for detecting abnormal point in network |
| CN111245684A (en) * | 2020-01-13 | 2020-06-05 | 智者四海（北京）技术有限公司 | Traffic scheduling method and device, electronic equipment and computer readable medium |
| CN111884874A (en) * | 2020-07-15 | 2020-11-03 | 中国舰船研究设计中心 | Programmable data plane-based ship network real-time anomaly detection method |
| CN111884874B (en) * | 2020-07-15 | 2022-02-01 | 中国舰船研究设计中心 | Programmable data plane-based ship network real-time anomaly detection method |
| CN113645215A (en) * | 2021-08-03 | 2021-11-12 | 恒安嘉新（北京）科技股份公司 | Method, device, equipment and storage medium for detecting abnormal network traffic data |
| CN114741377A (en) * | 2022-04-01 | 2022-07-12 | 深圳市爱路恩济能源技术有限公司 | Method and device for identifying and processing natural gas abnormal data |
| CN114741377B (en) * | 2022-04-01 | 2023-07-21 | 深圳市爱路恩济能源技术有限公司 | Method and device for identifying and processing natural gas abnormal data |
| CN116519021A (en) * | 2023-06-29 | 2023-08-01 | 西北工业大学 | Inertial navigation system fault diagnosis method, system and equipment |
| CN116519021B (en) * | 2023-06-29 | 2023-09-15 | 西北工业大学 | Inertial navigation system fault diagnosis method, system and equipment |
Also Published As
| Publication number | Publication date |
|---|---|
| CN110445680B (en) | 2021-06-08 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN110445680A (en) | Network flow abnormal detecting method, device and server | |
| CN104202329B (en) | Ddos attack detection method and device | |
| CN106951984B (en) | Dynamic analysis and prediction method and device for system health degree | |
| Bams et al. | An evaluation framework for alternative VaR-models | |
| US7243049B1 (en) | Method for modeling system performance | |
| CN109948669A (en) | A kind of abnormal deviation data examination method and device | |
| CN108491310A (en) | A kind of daily record monitoring method and system | |
| CN104994539B (en) | A kind of wireless sensor network Traffic anomaly detection method based on ARIMA models | |
| CN102355381B (en) | Method and system for predicting flow of self-adaptive differential auto-regression moving average model | |
| CN107480028B (en) | Method and device for acquiring usable residual time of disk | |
| CN106096226B (en) | A kind of data assessment method, apparatus and server | |
| CN107301570B (en) | Traffic prediction method, abnormal traffic detection device and electronic equipment | |
| CN108389631A (en) | Varicella morbidity method for early warning, server and computer readable storage medium | |
| CN110633893B (en) | Policy effectiveness monitoring method and device and computer equipment | |
| CN108550047A (en) | The prediction technique and device of trading volume | |
| CN104636874B (en) | Detect the method and apparatus of service exception | |
| CN110890998B (en) | Method and device for determining threshold | |
| CN108665096A (en) | Flow of the people alarm method and device | |
| CN110874674A (en) | Anomaly detection method, device and equipment | |
| KR101793625B1 (en) | Evaluation method and system of rainfall quantile considering climate change in nonstationary regional frequency analysis | |
| CN110059293A (en) | The determination method, apparatus and server of the quality of data of fund valuation data | |
| CN110795324B (en) | Data processing method and device | |
| CN108376292A (en) | A kind of crowd's method for predicting, system and equipment | |
| CN106357445B (en) | A kind of user experience monitoring method and monitoring server | |
| CN109960626A (en) | Recognition methods, device, equipment and the medium of port exception |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |