JP2008211541A - Traffic leap detecting device and traffic leap detection method - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To detect a traffic leap in a large amount of traffic time series data at higher speed and with higher accuracy than before, by automatically setting a threshold. <P>SOLUTION: This traffic leap detection device is provided for detecting a traffic leap as abnormality for time series data of a traffic volume caused to flow in an internet backbone, the traffic leap detecting device has a noise filter for converting the traffic time series data x(t) to be input into correction time series data z(t) in which a traffic leap is detected easily, and a leap judging device for automatically setting a threshold and determing the presence/absence of a traffic leap for the time series data output by the noise filter. The noise filter has a low-frequency noise filter for eliminating the long period traffic variation in a normal state from the traffic time series data to be input, and a high-frequency noise filter for eliminating short-period traffic variations. <P>COPYRIGHT: (C)2008,JPO&INPIT

Description

  The present invention relates to a traffic leap detection apparatus and a traffic leap detection method, and in particular, detects fluctuations caused by DoS (Denial of Service) attacks, network equipment failures, etc. as time-series data of traffic volume flowing on the Internet backbone. Related to technology.

In order to operate a safe and secure network, ISPs (Internet Service Providers), hosting providers, and backbone communication providers monitor the traffic flowing through the network, and rapidly change the traffic volume (hereinafter referred to as track leap). It is necessary to take measures.
However, since the amount of traffic change due to a DoS attack or the like is smaller than the amount of traffic flowing through the Internet backbone, the traffic to be monitored is divided into granularities that can detect traffic leaps, and the resulting multiple traffic It is necessary to monitor time series data.
Further, the traffic volume on the Internet changes even in a normal state such that it becomes larger than the daytime at night, and it is necessary to distinguish between steady fluctuation of the traffic volume and traffic leap due to abnormal traffic.
The simplest method for detecting traffic leap from traffic time series data is a method of setting a certain value as a threshold value and determining that traffic leap has occurred when the traffic volume exceeds the threshold value.
However, since the normal traffic volume and the fluctuation range of the traffic volume differ depending on the traffic time-series data to be monitored, the appropriate threshold value also differs for each traffic time-series data. It is difficult to set manually.

As a conventional technique for automatically setting an appropriate threshold value for traffic time series data, a Bollinger band using a moving average value ± α × standard deviation as a threshold value is frequently used. (See Non-Patent Document 1 below)
However, when the Bollinger band is used for traffic time-series data on the Internet, day and night fluctuations are also detected as traffic leaps, making it difficult to detect traffic leaps with high accuracy.
In addition, as a technology for detecting traffic leap of traffic time series data with high accuracy, prediction is performed from past data using a time series statistical model or maximum likelihood estimation method, and the difference between the predicted value and the measured value is evaluated. There is a way. (See Patent Document 1 below)
However, since these conventional methods require a lot of processing for calculation, it is difficult to use them for traffic leap detection with respect to a plurality of traffic time-series data.

As prior art documents related to the invention of the present application, there are the following.
JP 2004-54370 A Introduction to Bollinger Band, John Bollinger, Pan Rolling Co., 2002

In the conventional technology, complex calculations were performed for single or a small number of time series data, and traffic leap detection was performed with high accuracy. However, traffic leap detection was performed in parallel for a large amount of traffic time series data. It is difficult to do.
However, on the Internet backbone through which a large amount of traffic flows, it is essential to simultaneously monitor a large amount of traffic time series data.
The present invention has been made to solve the problems of the prior art, and the object of the present invention is to automatically set a threshold value and to perform traffic leap in a large amount of traffic time-series data faster than before. And it is providing the technique which becomes possible [detecting with high precision].
The above and other objects and novel features of the present invention will become apparent from the description of this specification and the accompanying drawings.

Of the inventions disclosed in this application, the outline of typical ones will be briefly described as follows.
(1) A traffic leap detection device that detects traffic leap, which is a rapid change in traffic volume, as an anomaly with respect to time series data of traffic volume flowing in the Internet backbone, and is input traffic time series data x (t) Is converted to corrected time-series data z (t) that is easy to detect traffic leap, and a leap determination that determines the presence or absence of traffic leap by automatically setting a threshold for the time-series data output by the noise filter Device.
(2) In (1), the noise filter includes a low-frequency noise filter that removes long-period traffic fluctuations in a steady state from input traffic time-series data, and a high-frequency noise filter that removes short-period traffic fluctuations. .
(3) In (2), the low frequency noise filter stores the traffic amount x (t) at the time point (t) and the traffic amount x (t−1) at the time point immediately before the time point (t). Storage means, and arithmetic means for calculating and outputting an absolute value y (t) of a difference between the traffic amount x (t) and the traffic amount x (t−1) stored in the storage means.

(4) In (3), the noise filter has an average value calculation device, and the average value calculation device stores a plurality of absolute values y (t) of differences output from the low frequency noise filter. Storage means, and arithmetic means for calculating and outputting an average value β (t) of the absolute values of the plurality of differences stored in the storage means, wherein the high frequency noise filter is derived from the low frequency noise filter. Computation means for dividing the output absolute value y (t) of the difference by the average value β (t) output from the average value calculation device and outputting the result as corrected time series data z (t).
(5) In (4), when the storage means of the average value calculation device sets y (t ′) as the absolute value of the difference at the time when it was determined in the past that the traffic leap was generated by the leap determination device. , Absolute value y (k) of past M differences excluding absolute value y (t ′) of difference from time (t−1) to time (t ′) (k = t,..., T -M) is stored, and the arithmetic means of the average value calculation device calculates and outputs an average value of absolute values of the M differences stored in the storage means.

(6) In (4) or (5), a threshold setting device is provided, the threshold setting device, storage means for storing a plurality of corrected time series data z (t) that is an output of the noise filter, and the storage The computing means 1 for calculating an average value μ (t) of the plurality of corrected time series data z (t) stored in the means, and the plurality of corrected time series data z (t) stored in the storage means. The threshold value B is calculated using the calculation means 2 for calculating the standard deviation σi (t), the average value μ (t) calculated by the calculation means 1 and the standard deviation σi (t) calculated by the calculation means 2. The leap determination device compares the magnitude relationship between the corrected time series data z (t) and the threshold value B (t) and calculates the corrected time series data. Traffic leap occurred when z (t) exceeded threshold B (t) And judging.
(7) In (6), when the storage means of the threshold value setting device sets z (t ′) as the corrected time-series data at the time point when it was determined in the past that the traffic leap occurred by the leap determination device, N corrected past time series data z (j) (j = t,..., T−N) excluding the corrected time series data z (t ′) at time (t ′) from (t−1). ), The calculation means 1 of the threshold setting device calculates an average value of the N corrected time series data stored in the storage means, and the calculation means 2 of the threshold setting device calculates the storage The standard deviation of the N corrected time series data stored in the means is calculated.

(8) A traffic leap detection method for detecting, as an abnormality, traffic leap that is a sudden change in traffic volume with respect to time series data of traffic volume flowing in the Internet backbone, and input traffic time series data x (t) Is converted to corrected time-series data z (t) that is easy to detect traffic leap, and a step of automatically setting a threshold for the time-series data converted in step 1 to determine the presence or absence of traffic leap 2.
(9) In (8), the step 1 is the absolute value of the difference between the traffic amount x (t) at the time point (t) and the traffic amount x (t−1) at the time point immediately before the time point (t). Step 11 for calculating y (t), Step 12 for calculating the average value β (t) of the absolute value y (t) of the difference calculated in Step 11, and the absolute value y calculated in Step 11 (T) is divided by the average value β (t) calculated in step 12 and output as corrected time series data z (t).
(10) In (8) or (9), in the step 2, the average value μ (t) of the corrected time series data z (t) converted in the step 1 is calculated, and in the step 1 A step 22 for calculating the standard deviation σi (t) of the converted corrected time series data z (t), an average value μ (t) of the time series data calculated in the step 21, and the step 22 Step 23 for calculating the threshold value B (t) using the standard deviation σi (t), the corrected time series data z (t) converted in Step 1, and the threshold value B ( t) and comparing the magnitude relationship with t) and determining that traffic leap has occurred when the corrected time-series data z (t) exceeds a threshold value B (t).

In the present invention, by installing a noise filter in the traffic leap detection device that removes the steady fluctuation of the traffic amount that causes the loss of accuracy from the traffic time series data and converts the traffic leap to a time series data that is easy to detect. Realizes highly accurate traffic leap detection.
Within the noise filter, the low-frequency noise filter and the high-frequency noise filter remove the effects of normal traffic fluctuations by simple calculations such as calculating the traffic volume difference and dividing by the average value for the traffic time-series data. This makes it possible to reduce the processing time required for one piece of traffic time-series data.
In addition, the leap determination device automatically sets the threshold used for determination from the average value and standard deviation of each of the time series data received from the noise filter, and thus does not require individual settings for a large amount of traffic time series data.

The effects obtained by the representative ones of the inventions disclosed in the present application will be briefly described as follows.
According to the present invention, it is possible to automatically set a threshold value, and to detect a traffic leap in a large amount of traffic time-series data at a higher speed and with higher accuracy than before.

Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings.
In all the drawings for explaining the embodiments, parts having the same functions are given the same reference numerals, and repeated explanation thereof is omitted.
FIG. 1 is a block diagram showing a schematic configuration of a traffic leap detection apparatus according to an embodiment of the present invention. In FIG. 1, 100 is a traffic leap detection device, 200 is a traffic monitoring device, and 300 is a control device. The traffic leap detection device 100 includes a noise filter 110, a leap determination device 120, and a threshold setting device 121.
FIG. 2A is a block diagram illustrating a schematic configuration of the noise filter 110 illustrated in FIG. 1.
As illustrated in FIG. 2A, the noise filter 110 includes a low frequency noise filter 111, a high frequency noise filter 112, and an average value calculation device 113.
The low frequency noise filter 111 includes a storage unit 10 and a calculation unit 11. The high frequency noise filter 112 includes a calculation unit 12. The average value calculation device 113 includes a storage unit 13 and a calculation unit 15.
FIG. 2-2 is a block diagram illustrating a schematic configuration of the threshold value setting device 121 illustrated in FIG. 1.
As illustrated in FIG. 2B, the threshold setting device 121 includes a storage unit 20 and arithmetic units (21, 22, 23).

It is assumed that the traffic monitoring apparatus 200 installed outside counts the traffic volume on the Internet backbone to be monitored at regular intervals and outputs a plurality of traffic time-series data.
In this specification, a plurality of traffic time-series data x (t) output from the traffic monitoring apparatus 200 is distinguished by integers i (i = 0, 1, 2,...), And traffic time-series data xi (t). Call it. In this specification, the time t (t = 1, 2, 3,...) In the traffic time series data xi (t) is obtained from the traffic monitoring apparatus 200 after the traffic leap detection apparatus 100 is activated. It is defined as the number of times data xi has been received.
Hereinafter, the processing procedure of the traffic leap detection apparatus according to the present embodiment will be described with reference to FIG.
First, the traffic leap detection apparatus 100 of the present embodiment receives the traffic time series data xi (t) at the latest time point (t) from the traffic monitoring apparatus 200.
The low frequency noise filter 111 stores the traffic amount xi (t) at the time point (t) and the traffic amount xi (t-1) at the time point immediately before the time point (t) in the storage unit 10, and the calculation unit. 11, the absolute value yi (t) = | xi (t) −xi (t) of the difference between the traffic amount xi (t) at the time point (t) stored in the storage means 10 and the traffic amount xi (t−1). −1) | is calculated and output to the high frequency noise filter 112. However, when t = 1, yi (t) = 0 is output to the high-frequency noise filter 112.

The storage means 13 of the average value calculation device 113 stores absolute values yi (k) (k = t,..., T−M) for the past M pieces from the time point (t−1). Note that M is a value that is manually set in advance. However, if the leap determination device 120 determines that traffic leap has occurred at the time (t ′) in the traffic time series data xi (t), the absolute value yi (t ′) of the difference at that time is stored. Do not do.
Based on the following equation (1), the arithmetic means 15 of the average value calculator 113 calculates the absolute value yi (k) (k = t,..., T−) of the past M differences stored in the storage means 13. The average value βi (t) of M) is calculated and output. However, when t <M, the average value βi (t) is not calculated and output.
The high frequency noise filter 112 divides by the absolute value yi (t) of the difference output from the low frequency noise filter 111 and the average value βi (t) of the absolute value of the difference output from the average value calculation device 113. As a result of the calculation, corrected time-series data zi (t) [= yi (t) / βi (t)]) is output to the leap determination device 120. However, when t <M, zi (t) is not calculated and output.

・・・・・・・・・・・・・・・・・ （１）

.... (1)

The storage means 20 of the threshold value setting device 121 stores the corrected N series of corrected time series data zi (j) (j = t,..., T−N) from the time point (t−1). N is a value set manually in advance. However, if the leaping determination device 120 determines that the traffic leap has occurred at the time (t ′) in the traffic time series data xi (t), the corrected time series data zi (t ′) at that time is accumulated. Do not do.
The calculation means 21 of the threshold value setting device 121 calculates the average value μi (t) of the past N corrected time series data zi (t) stored in the storage means 20 based on the following equation (2).
The calculation means 22 of the threshold setting device 121 calculates the standard deviation σi (t) of the past N corrected time series data zi (t) stored in the storage means 20.
The computing means 23 of the threshold setting device 121 calculates and outputs a threshold Bi (t) [= μi (t) + α · σi (t)]. Α is a predetermined constant. However, when t <(M + N), the calculation and output of the above-described threshold value Bi (t) are not performed.

・・・・・・・・・・・・・・・・・ （２）

(2)

The leap determination device 120 compares the magnitude relationship between the corrected time series data zi (t) output from the high frequency noise filter 113 and the threshold value Bi (t) output from the threshold value setting device 121.
If zi (t)> Bi (t) holds, it is determined in the traffic time series data xi (t) that a traffic leap has occurred at time (t), and this is notified to the control device 300 as an alert. Notice.
Further, an alert is also notified to the average value calculation device 113 and the threshold value setting device 121 to prevent learning of data when a traffic leap occurs. However, when t <(M + N), the above-described leap determination is not performed.
The control device 300 installed outside receives the alert output from the leap setting device 120 and specifies a device that performs detailed analysis and traffic control for specifying the cause.
As described above, according to the traffic leap detection apparatus of the present embodiment, the traffic time series data is detected by automating the threshold estimation for individual traffic time series data, and the traffic time series data that causes the loss of the traffic leap detection accuracy. It is possible to improve the accuracy by removing from the above, and to increase the speed by performing a calculation for improving the accuracy by a simple calculation such as an average value and a standard deviation.
As a result, it becomes possible to detect traffic leaps for a large amount of traffic time-series data in real time, and to monitor traffic flowing through the Internet backbone.

Hereinafter, the results of evaluating the effectiveness of the present invention will be described.
4 and 5 are diagrams showing the results of applying the embodiment of the present invention to traffic time-series data obtained by counting a certain traffic every 5 minutes.
The horizontal axis in FIG. 4 is time, and the vertical axis is a value obtained by normalizing the value of x (t) by [0, 1]. Similarly, the horizontal axis in FIG. 5 is time, and the vertical axis is a value obtained by normalizing the values of z (t) and B (t) with [0, 1].
Parameters that need to be manually set such as the number of data stored in the average value calculation device 113 (M), the number of data stored in the threshold value setting device 121 (N), and the threshold calculation coefficient (α) are M = 24 ( 2 hours), N = 288 (24 hours), and α = 6.
FIG. 4 is a diagram showing a part of the traffic time series data x (t) and the traffic time series data x (t) determined by the traffic detection apparatus according to the present embodiment as determined to have traffic leap. FIG. 5 is a diagram showing fluctuations in z-correction time series data (t), which is a determination criterion for occurrence of traffic leap in the traffic detection apparatus of the present embodiment, and threshold value B (t) at the same time as FIG.

4 and 5, the traffic time series data x (t) fluctuates greatly between day and night, but the normal time fluctuation does not change the corrected time series data z (t). It can be seen that the corrected time-series data z (t) exceeds the threshold value B (t) only at locations where the series data x (t) shows abnormal fluctuations.
As described above, the traffic leap detection apparatus according to the present embodiment can detect the traffic leap with high accuracy with a simple calculation.
By implementing this embodiment on a computer having a CPU clock number of 3.06 GHz and a memory capacity of 1 GByte, it is possible to detect traffic leaps simultaneously for 100,000 traffic time-series data.
At that time, the total time required for one traffic leap judgment with respect to all 100,000 traffic time-series data was about 3 seconds.
As mentioned above, the invention made by the present inventor has been specifically described based on the above embodiments. However, the present invention is not limited to the above embodiments, and various modifications can be made without departing from the scope of the invention. Of course.

It is a block diagram which shows schematic structure of the traffic leap detection apparatus of the Example of this invention. It is a block diagram which shows schematic structure of the noise filter shown in FIG. It is a block diagram which shows schematic structure of the threshold value setting apparatus shown in FIG. It is a figure for demonstrating the process sequence of the traffic leap detection apparatus of the Example of this invention. It is a figure explaining the result of having evaluated the effectiveness of the present invention. It is a figure explaining the result of having evaluated the effectiveness of the present invention.

Explanation of symbols

10, 13, 20 Storage means 11, 12, 15, 21, 22, 23 Arithmetic means 100 Traffic leap detection device 110 Noise filter 111 Low frequency noise filter 112 High frequency noise filter 113 Average value calculation device 120 Leap determination device 121 Threshold setting device 200 traffic monitoring device 300 control device

Claims (10)

A traffic leap detection device that detects traffic leap, which is a rapid change in traffic volume, with respect to time-series data of traffic volume flowing through the Internet backbone,
A noise filter that converts input traffic time-series data x (t) into corrected time-series data z (t) that is easy to detect traffic leap;
A traffic leap detection apparatus, comprising: a leap determination apparatus that automatically sets a threshold for time series data output from the noise filter and determines the presence or absence of traffic leap. The noise filter is a low-frequency noise filter that removes long-period traffic fluctuations in a steady state from input traffic time-series data;
2. The traffic leap detection apparatus according to claim 1, further comprising a high frequency noise filter for removing short-period traffic fluctuations. The low-frequency noise filter includes a storage unit that stores a traffic amount x (t) at a time point (t) and a traffic amount x (t−1) at a time point immediately before the time point (t).
3. An arithmetic unit that calculates and outputs an absolute value y (t) of a difference between the traffic amount x (t) and the traffic amount x (t−1) stored in the storage unit. The traffic leap detection device described in 1. The noise filter has an average value calculation device,
The average value calculating device stores a plurality of absolute values y (t) of differences output from the low frequency noise filter;
Computing means for calculating and outputting an average value β (t) of the absolute values of the plurality of differences stored in the storage means;
The high-frequency noise filter divides the absolute value y (t) of the difference output from the low-frequency noise filter by the average value β (t) output from the average value calculation device to obtain corrected time series data z ( 4. The traffic leap detection apparatus according to claim 3, further comprising arithmetic means for outputting as t). The storage means of the average value calculation device starts from time (t−1), where y (t ′) is the absolute value of the difference when it is determined in the past that traffic leap has occurred by the leap determination device. Stores absolute values y (k) (k = t,..., T−M) of the past M differences excluding the absolute value y (t ′) of the difference at the time point (t ′),
5. The traffic leap detection according to claim 4, wherein the arithmetic means of the average value calculation device calculates and outputs an average value of absolute values of the M differences stored in the storage means. apparatus. Having a threshold setting device;
Storage means for storing a plurality of corrected time series data z (t) which is an output of the threshold value setting device and the noise filter;
Computing means 1 for calculating an average value μ (t) of the plurality of corrected time series data z (t) stored in the storage means;
Computing means 2 for calculating a standard deviation σi (t) of the plurality of corrected time series data z (t) stored in the storage means;
An arithmetic means 3 that calculates and outputs a threshold value B (t) using the average value μ (t) calculated by the arithmetic means 1 and the standard deviation σi (t) calculated by the arithmetic means 2; Have
The leap determination device compares the magnitude relationship between the corrected time series data z (t) and the threshold value B (t), and the traffic is detected when the corrected time series data z (t) exceeds the threshold value B (t). 6. The traffic leap detection apparatus according to claim 4, wherein it is determined that a leap has occurred. When the corrected time-series data at the time point when it was determined that the traffic leap occurred in the past by the leap determination device is z (t ′), the storage means of the threshold value setting device sets the time point (t−1) to the time point ( storing the corrected N series of corrected time series data z (j) (j = t,..., t−N) excluding the corrected time series data z (t ′) at time t ′);
The calculation means 1 of the threshold value setting device calculates an average value of the N corrected time series data stored in the storage means,
7. The traffic leap detection apparatus according to claim 6, wherein the calculation means 2 of the threshold value setting apparatus calculates a standard deviation of the N corrected time series data stored in the storage means. A traffic leap detection method for detecting traffic leap that is a sudden change in traffic volume as an anomaly with respect to time-series data of traffic volume flowing through the Internet backbone,
Step 1 of converting the input traffic time series data x (t) into corrected time series data z (t) that is easy to detect traffic leaps;
A traffic leap detection method comprising: step 2 of automatically setting a threshold value for the time-series data converted in step 1 and determining the presence or absence of traffic leap. The step 1 is a step of calculating an absolute value y (t) of a difference between the traffic amount x (t) at the time point (t) and the traffic amount x (t−1) at the time point immediately before the time point (t). 11 and
Calculating an average value β (t) of the absolute value y (t) of the difference calculated in step 11;
Dividing the absolute value y (t) calculated in the step 11 by the average value β (t) calculated in the step 12 and outputting the corrected time series data z (t) as a step 13; The traffic leap detection method according to claim 8, wherein: The step 2 includes a step 21 of calculating an average value μ (t) of the corrected time series data z (t) converted in the step 1;
Calculating a standard deviation σi (t) of the corrected time series data z (t) converted in step 1;
Calculating a threshold value B (t) using the average value μ (t) of the time series data calculated in step 21 and the standard deviation σi (t) calculated in step 22;
The magnitude relationship between the corrected time series data z (t) converted in step 1 and the threshold value B (t) calculated in step 23 is compared, and the corrected time series data z (t) 10. The traffic leap detection method according to claim 8, further comprising a step of determining that a traffic leap has occurred when t) is exceeded.

Images