CN1588893A - Method for real timed monitoring stream data sudden change - Google Patents

Method for real timed monitoring stream data sudden change Download PDF

Info

Publication number
CN1588893A
CN1588893A CN 200410066238 CN200410066238A CN1588893A CN 1588893 A CN1588893 A CN 1588893A CN 200410066238 CN200410066238 CN 200410066238 CN 200410066238 A CN200410066238 A CN 200410066238A CN 1588893 A CN1588893 A CN 1588893A
Authority
CN
China
Prior art keywords
windowsize
rth
value
temp2
length
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Granted
Application number
CN 200410066238
Other languages
Chinese (zh)
Other versions
CN1312881C (en
Inventor
秦首科
周傲英
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Fudan University
Original Assignee
Fudan University
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Fudan University filed Critical Fudan University
Priority to CNB2004100662380A priority Critical patent/CN1312881C/en
Publication of CN1588893A publication Critical patent/CN1588893A/en
Application granted granted Critical
Publication of CN1312881C publication Critical patent/CN1312881C/en
Anticipated expiration legal-status Critical
Expired - Fee Related legal-status Critical Current

Links

Images

Landscapes

  • Complex Calculations (AREA)

Abstract

This invention relates to a method for real-time monitoring jump of data which first of all expands the definition of the current monitoring method on data jump and puts forward a definition for dynamicly monitoring jump, secondly, it introduces BOIH of an inverted drum series to utilze very small space to compress sample data streams x1, 2.....xn accurately, and adopt stream data monitor algorithm of error positive and negative to test the jump after adding xn to the BOIH, which can realize adaptively monitoring data jump and monitor jump in any multiple different time length windows.

Description

The method of real timed monitoring stream data sudden change
Technical field
The present invention relates to the technology in a kind of flow data management, relate to the method for a kind of real timed monitoring stream data sudden change (Burst) or rather, mainly be applicable to the monitoring of network traffic data and the monitoring of website visiting amount, with fault and the security problems that exists in the networks such as timely discovery and solution network congestion and network attack.
Background technology
Be different from the traditional data application model, flow data (Data Stream) model has following 4 general character: (1) data in real time arrives; (2) data arrive order independently, not controlled by application system; (3) data scale is grand and can not predict its maximum, for example company since 100 years stock exchange information and flow through data on the backbone network router or the like; (4) data one are treated, unless specially preserve, handled otherwise can not take out once more, perhaps extract data once more and cost dearly.The sudden change of Monitoring Data mainly is meant catches the behavior that data exception changes in a period of time in the process of data monitoring, and reports to the police, and is carried out the maintenance of system or other processing according to this abnormal information by corresponding personnel or software again.The technology of Monitoring Data sudden change almost is the important component part in all data acquisitions, data analysis and the data management software system, and its application is quite extensive.The method of Monitoring Data sudden change has determined the ability that software systems discover problems and solve them.The patent of invention that the implementation that formerly has a kind of advanced person's Monitoring Data sudden change in the technology: application number is 02107602.8, by name " be used to happen suddenly allow the method and apparatus that excessive bit error rate alarm detects and removes ".Yet this invention is having the following disadvantages aspect the monitoring sudden change:
1, can only use the Monitoring Data sudden change statically of absolute door limit value, can't dynamically monitor sudden change, therefore, the sudden change kind of its discovery is single.
2, can only find a sudden change that has defined in the length time window, therefore, it can't be in time or fully finds all sudden changes of current data.
3, the treatable data volume of institute is limited, and promptly Zui Da time window limited length can't be handled flow data.
4, sudden change on the monitoring target counting (Count) can only be found, and other qualitative sudden change of monitoring target can not be found, for example sudden change of mean value (Mean).
5, do not provide any method of jolting of resisting in its sudden change monitoring method, thereby the sudden change of being found can not guarantee that they are entirely true.
Summary of the invention
The objective of the invention is to propose a kind of method that can dynamically monitor the flow data sudden change of random length window, and this method can get rid of the interference that data are jolted and brought, to guarantee execution efficient, compatibility and the reliability of software systems.
Technical solution of the present invention:
At first, the present invention expands about the definition of data sudden change existing monitoring method, has proposed the dynamically definition of monitoring sudden change.If the sampled data stream of monitoring is sequence x 1.., x n, wherein any one is worth x iBe i sampled value constantly, i is a timestamp, and big more this sampled value that shows of timestamp is new more, x 1Be sampled value the earliest, x nBe up-to-date sampled value, n's is big or small unrestricted, and just the length of this sequence is unrestricted, if monitoring door is limited to RTH relatively, when RTH greater than 1 the time, when promptly the gathering result of calculation of current sequence enlarges markedly than the gathering result of calculation of previous adjacent equal length sequence, be forward mutation assay.When RTH less than 1 the time, when promptly the gathering result of calculation of current sequence significantly reduces than the gathering result of calculation of previous adjacent equal length sequence, be inverse transition.If the maximum length of the sequence of monitoring be j (1≤j≤n/2), and the gathering of carrying out to calculate be Sum.Definition so:
When &Sigma; k = n - j + 1 k = n x k &Sigma; k = n - 2 j + 1 k = n - j x k &GreaterEqual; RTH ( RTH > 1 ) The time, the length of current monitoring is the sequence generation forward mutation assay of j, when &Sigma; k = n - j + 1 k = n x k &Sigma; k = n - 2 j + 1 k = n - j x k &le; RTH ( 0 < RTH < 1 ) The time, the length of current monitoring is the sequence generation inverse transition of j.
The dynamic monitoring sudden change with the difference of static monitoring sudden change maximum is: 1. replace once the changeless absolute monitoring thresholding in back is set with relative monitoring thresholding (RTH); 2. can monitor the sequence of unrestricted several different lengths of number simultaneously; 3. can distinguish or monitor forward and inverse transition simultaneously.
Secondly, the present invention has introduced histogram (Histogram) technology, and a kind of new histogramming algorithm is provided---and the histogramming algorithm (BOIH) of invert barrel sequence, it can utilize accurately compression sampling data flow x of very little space 1.., x nWith the gathering of this approximate data estimation flow data, the accuracy of result of calculation can be up to more than 99.99%.The Data Structures that this algorithm is used is called bucket (Bucket).The histogram that algorithm makes up is exactly the sequence of a bucket.Each bucket (b i) all be the structure variable that a length is 20 bytes, be used to deposit data traffic on certain IP address in unit interval of collecting or certain website receives in the unit interval visit capacity, the number of the sampled value of being preserved in each barrel is unrestricted.Wherein, that the 1st to the 4th byte preservation is sampled value (b minimum in this barrel i a).The the 5th to the 8th byte preserved is the minimum value (MinD) of difference between two neighbouring sample values in this barrel.What the 9th to the 12nd byte preserved is the width (Wid) of bucket, just is kept at the number of the sampled value in this barrel.The the 13rd to the 16th byte preserved is the maximum (MaxD) of difference between two neighbouring sample values in this barrel.That the 17th to the 20th byte preserved is sampled value (b maximum in this barrel i b).Whenever a sampled value x newly arrives nJust move once this algorithm.If it is Sum that the gathering of carrying out is calculated, specific algorithm is as follows:
(1) when obtaining a new sampled value x n, the total B of current bucket is added 1, and the value of B is composed to j;
(2) a newly-built bucket, and with new sampled value x nPut into wherein;
(3) from i=B-1 to i=1 descending ground, be done as follows:
(a) to bucket b iWith b i aAnd b i bIncrease x respectively n
(b) judge b i bWhether greater than (1+ δ) b j aIf b i bGreater than (1+ δ) b j a, j--skips to the execution of the 3rd step then so; Otherwise, with b i bValue compose to b j b, again with b iWid add b to jWid on, if b iMaxD greater than b jMaxD, just use b iMaxD replace b jMaxD, if b iMinD less than b jMinD, just use b iMinD replace b jMinD, delete b at last i, and the total B of current bucket subtracted 1, skipped to for the 3rd step then to carry out.
All have the identical relative error upper limit, i.e. δ according to each barrel in the histogram of above algorithm structure.Because the existence of error is exactly an actual value so can not guarantee return value, but can guarantee that the value of being returned is the value big or slightly littler slightly than actual value, the error between they and the actual value is subjected to the influence of δ.This data structure provides two external interfaces, and one is BOIH.getLargerValue (windowSize), and it is the big slightly value of gathering result of calculation of the sequence of windowSize than length that this function can return one.Another is BOIH.getSmallerValue (windowSize), and it is the slightly little value of gathering result of calculation of the sequence of windowSize than length that this function returns one.
Once more, the present invention provides the wrong positive (False Positive) and false negative (False Negative) flow data monitoring algorithm respectively, guarantee 100% sudden change recall ratio (Recall) and 100% sudden change precision ratio (Precision) respectively, they can satisfy the demand of different application.Promptly finish an x by abovementioned steps nAfter adding histogram to, adopt wrong flow data monitoring algorithm positive and false negative to detect the sudden change situation.
Introduce False Positive algorithm earlier, specific algorithm is as follows:
(1) the counter burstNum that will suddenly change is changed to 0, and the length windowSize of the window that will detect is changed to 1;
(2) from windowSize=1 to windowSize = n 2 Ascending order ground is that the window of windowSize carries out following operation to length:
(a) if will monitor forward mutation assay, then the return value of BOIH.getLargerValue (windowSize) is composed to temporary variable temp1, the return value of BOIH.getSmallerValue (2*windowSize) is composed to temporary variable temp2; If temp2 〉=(1+RTH) (temp2-temp1) arranged, RTH＞1, length is the sequence generation forward mutation assay of windowSize so, and burstNum is added 1, otherwise, if burstNum is 0 then skips to the execution of the 2nd step;
(b) if will monitor inverse transition, then the return value of BOIH.getSmallerValue (windowSize) is composed to temporary variable temp1, the return value of BOIH.getLargerValue (2*windowSize) is composed to temporary variable temp2; If temp2≤(1+RTH) (temp2-temp1) arranged, 0＜RTH＜1, length is the sequence generation inverse transition of windowSize so, and burstNum is subtracted 1, otherwise skips to the execution of the 2nd step;
(3) will the suddenly change value of counter burstNum is returned.
Only need revise a little and just can access False Negative algorithm False Positive algorithm, specific practice is that the assignment to temp1 changes BOIH.getSmallerValue (windowSize) in going on foot FalsePositive algorithm the 2nd a), and the temp2 assignment is changed into BOIH.getLargerValue (2*windowSize).Again with b) in the assignment of temp1 is changed into BOIH.getLargerValue (windowSize), the temp2 assignment is changed into BOIH.getSmallerValue (2*windowSize) gets final product.Its algorithm is as follows:
(1) the counter burstNum that will suddenly change is changed to 0, and the length windowSize of the window that will detect is changed to 1;
(2) from windowSize=1 to windowSize = n 2 Ascending order ground is that the window of windowSize carries out following operation to length:
(a) if will monitor forward mutation assay, then the return value of BOIH.getSmallerValue (windowSize) is composed to temporary variable temp1, the return value of BOIH.getLargerValue (2*windowSize) is composed to temporary variable temp2; If temp2 〉=(1+RTH) (temp2-temp1) arranged, RTH＞1, length is the sequence generation forward mutation assay of windowSize so, and burstNum is added 1, otherwise, if burstNum is 0 then skips to the execution of the 2nd step;
(b) if will monitor inverse transition, then the return value of BOIH.getLargerValue (windowSize) is composed to temporary variable temp1, the return value of BOIH.getSmallerValue (2*windowSize) is composed to temporary variable temp2; If temp2≤(1+RTH) (temp2-temp1) arranged, 0＜RTH＜1, length is the sequence generation inverse transition of windowSize so, and burstNum is subtracted 1, otherwise skips to the execution of the 2nd step;
(3) will the suddenly change value of counter burstNum is returned.
The detected sudden change on big window of existing method often is not sudden change on less window, this be by a plurality of continuous jolt cause, jolting is that relative normal data is comparatively outstanding, but does not reach the data of sudden change degree.The present invention has considered the influence that data are jolted and brought to the sudden change monitoring, provide and got rid of the mechanism of jolting and disturbing, and respectively it is integrated in top two flow datas monitoring algorithm, specific practice such as False Positive algorithm the 2nd go on foot a) and b) in, when the sequence of sequence detection different length from small to large, if the sequence that to run into a length be windowSize is not undergone mutation, will no longer continue to detect length so is windowSize+1 and longer sequence.
In our method framework, the maintenance of data structure and the execution of algorithm do not need to consider terminal use's behavior, so system can carry out the optimization of efficient to greatest extent; Our method framework has no requirement to the characteristic of data, can handle the data of any distribution, has therefore improved the compatibility of software systems; The space that the data structure of compression is shared and the size of data volume are sublinear relations, and therefore when facing huge data, the space expense of data structure is still very little, and this reliability for software systems provides guarantee.
Advantage of the present invention and technical progress:
1, Monitoring Data sudden change adaptively.
2, can monitor the interior sudden change of any a plurality of different time length window simultaneously.
3, can handle the huge like this data volume of flow data, its maximum time length of window without limits.
4, the sudden change on can gathering (Aggregation) result of calculation of monitoring target, therefore, the sudden change kind of object be can find, counting (Count), mean value (Mean), summation (Sum), intermediate value (Median), maximum (Max), minimum value (Min) etc. comprised.
5, can in the monitoring sudden change, get rid of data and jolt the influence of monitoring result.
6, execution efficient, compatibility and the reliability for software systems provides guarantee.
Description of drawings
The bucket sequence schematic diagram that the existing histogramming algorithm of Fig. 1 makes up.
The bucket sequence schematic diagram that Fig. 2 histogramming algorithm provided by the invention makes up.
The structure of Fig. 3 bucket provided by the invention is formed schematic diagram.
Fig. 4 histogramming algorithm flow chart provided by the invention.
Fig. 5 sudden change detection algorithm provided by the invention flow chart.
The model structure schematic diagram of Fig. 6 real timed monitoring stream data sudden change method provided by the invention.
Fig. 7 data schematic diagram that jolts.
The performance test figure of Fig. 8 real timed monitoring stream data sudden change method provided by the invention.
Embodiment
Method of the present invention adopts following user's configurable parameter variable: monitor thresholding (RTH) relatively, the maximum relative error upper limit (δ) of each barrel.When RTH greater than 1 the time, the forward mutation assay that monitoring has ascendant trend, when RTH less than 1 the time, monitoring has the inverse transition of downward trend, two RTH also can be set to monitor this two kinds of sudden changes simultaneously.The sample sequence length variations that can monitor is continuous, and the upper limit of its quantity is unrestricted, and base is unrestricted when promptly monitoring.Can monitor the time window of all length in the duration scope total from the minimal sampling time to the signal simultaneously, increasing progressively of each time window length is minimal sampling time.
Referring to Fig. 1: the bucket sequence that constructs for existing histogramming algorithm shown in the figure, along the direction of time shaft 10 is direction near the current time, as seen increasing along the width of this direction bucket, the bucket that makes up the earliest is numbered bucket 1, the bucket of up-to-date structure is numbered a barrel n, bucket I be from bucket 1 to bucket n the numbering of any one barrel.
Referring to Fig. 2: the bucket sequence that constructs for histogramming algorithm of the present invention (BOIH) shown in the figure, along the direction of time shaft 20 is direction near the current time, as seen the width along this direction bucket is more and more littler, the bucket that makes up the earliest is numbered bucket 1, the bucket of up-to-date structure is numbered a barrel n, bucket I be from bucket 1 to bucket n the numbering of any one barrel.
Referring to Fig. 3: the structure that shown in the figure is bucket in the histogramming algorithm of the present invention (BOIH) is formed, and wherein, that the 1st to the 4th byte preserved is sampled value (b minimum in this barrel i a).The the 5th to the 8th byte preserved is the minimum value (MinD) of difference between two neighbouring sample values in this barrel.What the 9th to the 12nd byte preserved is the width (Wid) of bucket, just is kept at the number of the sampled value in this barrel.The the 13rd to the 16th byte preserved is the maximum of poor (MaxD) between two neighbouring sample values in this bucket.That the 17th to the 20th byte preserved is sampled value (b maximum in this barrel i b).
Referring to Fig. 7: jolting for data shown in the figure can cause false sudden change on than the time window of length.If the aggregate function of monitoring is Sum, RTH=1.1.From (a), can see on length is 1 window x 4And x 6All jolt.From (b), can see on length is 2 window (x 3+ x 4)=39.5＞RTH (x 1+ x 2)=1.1*30, x therefore jolts 4Cause this length window that false sudden change takes place during arrival.From (c), can see when jolting x 6During arrival, be to detect wrong sudden change on 3 the window in length.
Referring to Fig. 8: (a) is depicted as the monitoring accuracy of real timed monitoring stream data sudden change method provided by the invention among the figure.Can see that δ is more little, precision is high more.When δ≤0.01, the monitoring accuracy under various RTH are provided with can be up to more than 99.9%.(b) is depicted as the space expense of real timed monitoring stream data sudden change method provided by the invention among the figure, can see that δ is big more, and requisite space is more little.(c) is depicted as the processing time of real timed monitoring stream data sudden change method provided by the invention among the figure.The used data set of above performance test is taken from the True Data collection (WorldCup98) on the http://ita.ee.lbl.gov.Algorithm all adopts Microsoft Visual C++6.0 to realize.The hardware environment of test is that a CPU is 2.4GHz, the DellPC of 256 MB of memory.
Detect the process of sudden change below in conjunction with description of drawings.
Embodiment 1. is provided with RTH for greater than 1 value, only detects forward mutation assay.RTH=1.1 is set, δ=0.01.At first make up histogram according to algorithm shown in Figure 4.Concrete steps are as follows: when obtaining a new sampled value x n, the total B of current bucket is added 1, and the value of B is composed to j.A newly-built bucket, and with new sampled value x nPut into wherein., be done as follows to i=1 descending ground from i=B-1: to bucket b iWith b i aAnd b i bIncrease x respectively nJudge b i bWhether greater than (1+ δ) b j aIf b i bGreater than (1+ δ) b j a, j--so; Otherwise, with b i bValue compose to b j b, again with b iWid add b to jWid on, if b iMaxD greater than b jMaxD just use b iMaxD replace b jMaxD, if b iMinD less than b jMinD just use b iMinD replace b jMinD, delete b at last i, and the total B of current bucket subtracted 1, then j--.When finishing an x nAfter adding the work in the histogram to, next detect sudden change according to algorithm shown in Figure 5, concrete steps are as follows: the counter burstNum that at first will suddenly change is changed to 0, and the length windowSize of the window that will detect is changed to 1.Next from windowSize=1 to windowSize = n 2 Ascending order ground, to length is that the window of windowSize detects, the return value of BOIH.getLargerValue (windowSize) is composed to temporary variable temp1, the return value of BOIH.getSmallerValue (2*windowSize) is composed to temporary variable temp2.If temp2 〉=(1+RTH) (temp2-temp1) arranged, length is the sequence generation forward mutation assay of windowSize so, and burstNum is added 1, otherwise the value of the counter burstNum that will suddenly change is returned.Return value is that positive number represents that forward mutation assay is arranged.So far, x nSudden change testing after the arrival has just been finished.If concrete window size that detects sudden change then will undergo mutation and the window number of undergoing mutation report to the user, remind the user to have abnormal behaviour to occur.
Embodiment 2. is provided with RTH for less than 1 value, only detects inverse transition.RTH=0.5 is set, δ=0.01.In embodiment 1, earlier the new-comer is worth x nAdd in the histogram.Detect sudden change according to algorithm shown in Figure 5 then, concrete steps are as follows: the counter burstNum that at first will suddenly change is changed to 0, and the length windowSize of the window that will detect is changed to 1.Next from windowSize=1 to windowSize = n 2 Ascending order ground is that the window of windowSize detects to length.The return value of BOIH.getSmallerValue (windowSize) is composed to temporary variable temp1, the return value of BOIH.getLargerValue (2*windowSize) is composed to temporary variable temp2.If temp2≤(1+RTH) (temp2-temp1) arranged, length is the sequence generation inverse transition of windowSize so, and burstNum is subtracted 1, otherwise the value of the counter burstNum that will suddenly change is returned.Return value is that negative number representation has inverse transition.So far, x nSudden change testing after the arrival has just been finished.If concrete window size that detects sudden change then will undergo mutation and the window number of undergoing mutation report to the user, remind the user to have abnormal behaviour to occur.
Embodiment 3. is provided with RTH 1For less than 1 value, RTH 2For greater than 1 value, detect forward mutation assay and inverse transition simultaneously.RTH is set 1=0.5, RTH 2=1.1, δ=0.01.In embodiment 1, earlier the new-comer is worth x nAdd in the histogram.Detect sudden change according to algorithm shown in Figure 5 then, concrete steps are as follows: the counter burstNum that at first will suddenly change is changed to 0, and the length windowSize of the window that will detect is changed to 1.Next from windowSize=1 to windowSize = n 2 Ascending order ground, to length is that the window of windowSize carries out following two steps operation: the first step, if monitor forward mutation assay, then the return value of BOIH.getLargerValue (windowSize) is composed to temporary variable temp1, the return value of BOIH.getSmallerValue (2*windowSize) is composed to temporary variable temp2.If temp2 〉=(1+RTH is arranged 2) (temp2-temp1), RTH 2＞1, length is the sequence generation forward mutation assay of windowSize so, and burstNum is added 1, and burstNum is not that will the suddenly change value of counter burstNum of 0 words is returned else if.Second step, if will monitor inverse transition, then the return value of BOIH.getSmallerValue (windowSize) is composed to temporary variable temp1, the return value of BOIH.getLargerValue (2*windowSize) is composed to temporary variable temp2.If temp2≤(1+RTH is arranged 1) (temp2-temp1), 0＜RTH 1＜1, length is the sequence generation inverse transition of windowSize so, and burstNum is subtracted 1, otherwise the value of the counter burstNum that will suddenly change is returned.Return value is that positive number represents that forward mutation assay is arranged, and return value is that negative number representation has inverse transition.So far, x nSudden change testing after the arrival has just been finished.If concrete window size that detects sudden change then will undergo mutation and the window number of undergoing mutation report to the user, remind the user to have abnormal behaviour to occur.

Claims (1)

1, a kind of method of real timed monitoring stream data sudden change is characterized in that concrete steps are as follows:
If the sampled data stream of monitoring is sequence x 1.., x n, wherein any one is worth x iBe i sampled value constantly, i is a timestamp, x 1Be sampled value the earliest, x nBe up-to-date sampled value, n's is big or small unrestricted, establish relative monitoring door and be limited to RTH, when RTH greater than 1 the time, be forward mutation assay, when RTH less than 1 the time, be inverse transition; If the maximum length of the sequence of monitoring be j (1≤j≤n/2), and the gathering of carrying out to calculate be Sum, definition so:
When &Sigma; k = n - j + 1 k = n x k &Sigma; k = n - 2 j + 1 k = n - j x k &GreaterEqual; RTH ( RTH > 1 ) The time, the length of current monitoring is the sequence generation forward mutation assay of j,
When &Sigma; k = n - j + 1 k = n x k &Sigma; k = n - 2 j + 1 k = n - j x k &le; RTH ( 0 < RTH < 1 ) The time, the length of current monitoring is the sequence generation inverse transition of j;
(2) for each new sampled value x n, adopt the histogramming algorithm (being designated as BOIH) of invert barrel sequence to assemble calculating, the steps include:
1. ought obtain a new sampled value x n, the total B of current bucket is added 1, and the value of B is composed to j;
2. a newly-built bucket, and with new sampled value x nPut into wherein;
3. from i=B-1 to i=1 descending ground, be done as follows:
(a) to bucket b iWith b i aAnd b i bIncrease x respectively n
(b) judge b i bWhether greater than (1+ δ) b j aIf, b i bGreater than (1+ δ) b j a, j--skips to the execution of the 3rd step then so; Otherwise, with b i bValue compose to b j b, again with b iWid add b to iWid on, if b iMaxD greater than b jMaxD, just use b iMaxD replace b jMaxD, if b iMinD less than b jMinD, just use b iMinD replace b jMinD, delete b at last i, and the total B of current bucket subtracted 1, skipped to for the 3rd step then to carry out;
Wherein, b iBe bucket, b i aBe b iIn minimum sampled value, b i bBe b iIn maximum sampled value, MaxD is the maximum of difference of two neighbouring sample values in the bucket, MinD be the minimum value of the difference of two neighbouring sample values in the bucket, δ be barrel in the relative error upper limit;
(3) finish an x by above-mentioned steps nAfter adding histogram to, adopt wrong flow data positive and false negative to monitor algorithm, detect the sudden change situation; Wherein, the positive algorithm of mistake is as follows:
1. the counter burstNum that will suddenly change is changed to 0, and the length windowSize of the window that will detect is changed to 1;
2. from windowSize=1 to windowSize = n 2 Ascending order ground is that the window of windowSize carries out following operation to length:
(a) if will monitor forward mutation assay, then the return value of BOIH.getLargerValue (windowSize) is composed to temporary variable temp1, the return value of BOIH.getSmallerValue (2*windowSize) is composed to temporary variable temp2; If temp2 〉=(1+RTH) (temp2-temp1) arranged, RTH＞1, length is the sequence generation forward mutation assay of windowSize so, and burstNum is added 1, otherwise, if burstNum is 0 then skips to the execution of the 4th step;
(b) if will monitor inverse transition, then the return value of BOIH.getSmallerValue (windowSize) is composed to temporary variable temp1, the return value of BOIH.getLargerValue (2*windowSize) is composed to temporary variable temp2; If temp2≤(1+RTH) (temp2-temp1) arranged, 0＜RTH＜1, length is the sequence generation inverse transition of windowSize so, and burstNum is subtracted 1, otherwise skips to the execution of the 4th step;
3. the value of counter burstNum of will suddenling change is returned;
The algorithm of false negative is as follows:
1. the counter burstNum that will suddenly change is changed to 0, and the length windowSize of the window that will detect is changed to 1;
2. from windowSize=1 to windowSize = n 2 Ascending order ground is that the window of windowSize carries out following operation to length:
(a) if will monitor forward mutation assay, then the return value of BOIH.getSmallerValue (windowSize) is composed to temporary variable temp4, the return value of BOIH.getLargerValue (2*windowSize) is composed to temporary variable temp2; If temp2 〉=(1+RTH) (temp2-temp1) arranged, RTH＞1, length is the sequence generation forward mutation assay of windowSize so, and burstNum is added 1, otherwise, if burstNum is 0 then skips to the execution of the 2nd step;
(b) if will monitor inverse transition, then the return value of BOIH.getLargerValue (windowSize) is composed to temporary variable temp4, the return value of BOIH.getSmallValue (2*windowSize) is composed to temporary variable temp2; If temp2≤(1+RTH) (temp2-temp1) arranged, 0＜RTH＜1, length is the sequence generation inverse transition of windowSize so, and burstNum is subtracted 1, otherwise skips to the execution of the 2nd step;
3. the value of counter burstNum of will suddenling change is returned.
CNB2004100662380A 2004-09-09 2004-09-09 Method for real timed monitoring stream data sudden change Expired - Fee Related CN1312881C (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CNB2004100662380A CN1312881C (en) 2004-09-09 2004-09-09 Method for real timed monitoring stream data sudden change

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CNB2004100662380A CN1312881C (en) 2004-09-09 2004-09-09 Method for real timed monitoring stream data sudden change

Publications (2)

Publication Number Publication Date
CN1588893A true CN1588893A (en) 2005-03-02
CN1312881C CN1312881C (en) 2007-04-25

Family

ID=34603960

Family Applications (1)

Application Number Title Priority Date Filing Date
CNB2004100662380A Expired - Fee Related CN1312881C (en) 2004-09-09 2004-09-09 Method for real timed monitoring stream data sudden change

Country Status (1)

Country Link
CN (1) CN1312881C (en)

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101908065A (en) * 2010-07-27 2010-12-08 浙江大学 On-line attribute abnormal point detecting method for supporting dynamic update
CN101669037B (en) * 2007-04-30 2012-12-05 国际商业机器公司 Monitoring reliability of a digital system

Family Cites Families (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US6775237B2 (en) * 2001-03-29 2004-08-10 Transwitch Corp. Methods and apparatus for burst tolerant excessive bit error rate alarm detection and clearing
US20020169880A1 (en) * 2001-04-19 2002-11-14 Koninklijke Philips Electronics N.V. Method and device for robust real-time estimation of the bottleneck bandwidth in the internet
AU2003243634A1 (en) * 2002-06-18 2003-12-31 Reveo, Inc. Burst communications system and method

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101669037B (en) * 2007-04-30 2012-12-05 国际商业机器公司 Monitoring reliability of a digital system
CN101908065A (en) * 2010-07-27 2010-12-08 浙江大学 On-line attribute abnormal point detecting method for supporting dynamic update

Also Published As

Publication number Publication date
CN1312881C (en) 2007-04-25

Similar Documents

Publication Publication Date Title
CN101267313A (en) Flooding attack detection method and detection device
CN101051952A (en) Self adaption sampling stream measuring method under high speed multilink logic channel environment
CN100352208C (en) Detection and defence method for data flous of large network station
CN101227318B (en) Method for overtrick real-time detection of high speed network flow quantity
CN114500092B (en) Industrial Internet identification abnormal flow identification method based on SDN
CN1941716A (en) Method, device and system for accounting application flow
CN111782700B (en) Data stream frequency estimation method, system and medium based on double-layer structure
CN103685224A (en) A network invasion detection method
CN115102907B (en) Active large flow accurate identification method and system based on small flow filtering
CN104506356A (en) Method and device for determining credibility of IP (Internet protocol) address
CN115687038B (en) Hard disk fault prediction system and method based on big data
CN102801548A (en) Intelligent early warning method, device and information system
CN101043363A (en) Method for processing abnormal overturn of host machine
CN1812394A (en) Method for using immediate information software by data detection network address switching equipment
CN116760649B (en) Data security protection and early warning method based on big data
CN100558058C (en) Packet measuring method based on stream aggregation arbitrary sampling
CN1588893A (en) Method for real timed monitoring stream data sudden change
CN1170175A (en) Image histogram equalization circuit and method therefor
CN1902631A (en) Clustering technique for cyclic phenomena
CN117614693A (en) Cloud internal security threat detection method based on behavior traffic
CN117061170A (en) Intelligent manufacturing industry big data analysis method based on feature selection
CN1968135A (en) Flow distribution system and method
CN108460424B (en) Spatial motion trajectory identification method based on line point conversion and cluster comparison
CN100589423C (en) The method and the device of identification data streams in multi-person online role playing game
CN1510890A (en) Statistical method for realtime business information

Legal Events

Date Code Title Description
C06 Publication
PB01 Publication
C10 Entry into substantive examination
SE01 Entry into force of request for substantive examination
C14 Grant of patent or utility model
GR01 Patent grant
C17 Cessation of patent right
CF01 Termination of patent right due to non-payment of annual fee

Granted publication date: 20070425