JP4420850B2 - Flow class estimation method, apparatus and program - Google Patents

Description

  The present invention relates to a flow class estimation method and apparatus for estimating a flow class passing through a certain link in a communication network and managing the flow.

  As IP networks are widely used, the importance of network traffic management technology is increasing. In particular, by using a flow as a traffic unit, fine network management can be realized. A flow is defined as a set of packets having common characteristics, such as a TCP (Transmission Control Protocol) connection and a pair of sending and receiving hosts.

It is useful if the flow coming into the network is monitored and the flow characteristics can be grasped at the same time as the flow arrives. For example, if a huge flow (abnormal flow) that squeezes the line can be quickly identified as it arrives, the line can be prevented from being congested by quickly limiting the transmission rate of the flow.
As described above, by classifying flows according to flow characteristics (for example, normal flow, abnormal flow, etc.) and performing control according to the classes, effective network management can be performed.

  As a method for realizing the rapid classification of the flow as described above, there is a method of performing static classification based on a predetermined rule. For example, for a flow that should be prioritized, classifying information in a header or the like (for example, a ToS (Type of Service) field) of a packet that constitutes the flow by a transmission source enables quick classification. It becomes possible.

However, such a method needs to have a mechanism for notifying all transmission sources of the class, and such a mechanism exists in the current Internet composed of many autonomous networks under different management. In addition, there is a problem that the introduction of such a mechanism to the entire network has a very low possibility of realization.
Furthermore, since the class is determined statically at the transmission source, there is a problem that it is not possible to dynamically classify a flow that occurs unexpectedly, such as abnormal traffic.

In this way, if a mechanism for notifying all transmission sources of a class is provided, flow classification can be performed dynamically at the transmission destination, but it is currently composed of many autonomous networks under different management. In the Internet, it is impossible to introduce such a mechanism to the entire network.
In addition, since the class is determined statically at the transmission source, it is not possible to dynamically classify a flow that occurs unexpectedly, such as abnormal traffic.

(the purpose)
The object of the present invention is to solve such a conventional problem, and to dynamically and accurately estimate which class a newly arrived flow belongs to a predetermined class, An object of the present invention is to provide a flow class estimation method and apparatus capable of estimating a class even for flows having combinations of attributes that do not exist in the past.

  In the flow estimation method of the present invention, a statistical method called a naive Bayes classifier is used, and based on a stochastic relationship between a flow attribute learned in the past and its class, based on a newly arrived flow attribute. , Probabilistically estimate the class to which the flow belongs. By using a naive Bayes classifier, it is possible to estimate a likely class even for a flow having a combination of attributes that does not exist in the past, thereby enabling robust estimation.

  In the first flow class estimation method of the present invention, the flow statistics management unit measures the flow passing through the link (or a plurality of links), or acquires flow statistics from a separately installed flow measuring instrument. Then, the acquired flow statistical data is registered in the flow management table. Further, the flow learning unit calculates a probabilistic relationship between the attributes of the flow and the class to which the flow belongs based on the information registered in the flow management table, and records it in the learning data. In addition, the flow class estimation unit probabilistically estimates a class to which the flow belongs from a newly arrived flow attribute based on the learning data by using a naive Bayes classifier.

  In the second flow class estimation method of the present invention, the flow management table is managed by 1) holding the information obtained by referring to and counting the header part of the packet passing through the target link in the flow management table. Or 2) Acquire a statistical value output by a flow measuring device separately installed for the link and register it in the flow management table.

In the third flow class estimation method of the present invention, the flow management table includes a time at which a packet constituting the flow first arrives, a time at which packets finally arrive, a cumulative transmission packet number, a cumulative transmission byte in a predetermined measurement period. It consists of number, incoming / outgoing interface, flow attribute, etc.
The flow attributes include, for example, a calling IP address, a calling AS number, a calling prefix, a calling port number, a protocol type, a TTL (Time-to-live), a TCP flag, and a TCP described in the header of a packet constituting the flow. It can be defined by any combination of data that can be measured for the flow, such as window size, presence or absence of window scale options, and other values and statistics of those values. It is also possible to record in an aggregated form. By using a hash function, the number of data to be managed can be reduced.

  In the fourth flow class estimation method of the present invention, the flow learning unit refers to the flow statistical values described in the flow management table in a predetermined period, uses a predetermined threshold value, or averages the statistical values. Each flow is classified into classes Ci (C1, C2, C3,... Cn) (n classes) using values such as values and standard deviations, or any other statistical method.

In the fifth flow class estimation method of the present invention, the flow learning unit includes a class Ci in which flows are classified, and an attribute A = {A1, A2, A3,..., Am} (m attributes) ), A possible value of each attribute Aj is Aj = ajk,
Referring to the flow management table, count the number of times ni that class i has occurred, and the number of times nijk that the flow classified into class i has attribute ajk,
Using these values ni and nijk, the probability that the attribute Aj has the attribute value ajk under the condition where the class Ci occurred is calculated as P (Aj = ajk | Ci) = nijk / ni and recorded in the learning data To do.

  In the sixth flow class estimation method of the present invention, in the calculation of the probability P (Aj = ajk | Ci) in the flow learning unit in the fifth flow class estimation method, the class Ci and the attribute Aj exist simultaneously in the flow management table. In consideration of the case where the number of flows existing in the flow management table is n and its inverse f = 1 / n, the number of values that the attribute j can take is nj, and (Aj = ajk | Ci) by Laplace correction ) = (Nijk + f) / (ni + f * nj). Further, the correction method is not limited to the Laplace correction.

  In the seventh flow class estimation method of the present invention, the flow class estimation unit acquires the attribute A for each newly arrived flow based on link measurement or output from a separately installed flow measurement device, Using the probability P (Aj = ajk | Ci) recorded in the learning data of the fifth and sixth flow class estimation methods and the probability P (Ci) = ni / n that the class Ci occurs, the discriminant function f (Ci) = P (Ci) * ΠjP (Aj = ajk | Ci) (where Πj represents an operation of multiplying the subsequent terms for all j) for all classes Ci, Ci that gives the maximum value of the discriminant function is estimated as a class of the flow (naive Bayes classifier).

  In the eighth flow class estimation method of the present invention, the estimation of the flow class may be performed every time all flows arrive, or the flow to be estimated is limited by a technique such as sampling or filtering. Is also possible.

According to the present invention, it is possible to dynamically and accurately estimate which class a newly arrived flow belongs to a predetermined class. Further, by using a statistical technique called naive Bayes classifier, based on the probabilistic relation between the attributes of the flow learned in the past and their class, newly arrived flow attributes of Motoniso flow The class to which it belongs can be estimated probabilistically. As a result, since the class can be estimated even for a flow having a combination of attributes that does not exist in the past, robust estimation is possible.

Embodiments of the present invention will be described below with reference to the drawings.
In the embodiment, for the sake of simplicity, 1) the flow statistics are acquired by the flow class estimation device itself, and 2) the flow is defined as a packet having the same call origination IP address, call origination port number, and protocol type. 3) The case where the flow class is two classes determined by the size of the flow size will be described as an example, but the scope of application of the present invention is not limited to this embodiment.

FIG. 1 is a configuration diagram showing an example of a basic configuration of an IP network to which the present invention is applied.
As shown in FIG. 1, the flow class estimation device 11 is used in a form of being attached to a node such as a router 10 constituting a main link 12 of a network.
The flow class estimation device 11 can measure the traffic passing on the router 10 by any of the following methods.
1) By using a device such as an optical coupler or a tap on the link 12, all packets flowing on the link 12 are branched toward the flow class estimation device 11.
2) Using the port mirroring function or the like in the router 10, copy all packets passing through the router 10 to the flow class estimation device 11.

  The flow class measurement apparatus 11 can construct and acquire statistics of all flows generated on the link 12 in FIG. 1 by the method as described in 1) or 2). The method is not limited to the above-described method. For example, the flow class estimation device 11 itself can be used by inserting an optical coupler or a tap function into the line.

FIG. 2 is a configuration diagram of a flow class estimation apparatus according to an embodiment of the present invention.
When a packet arrives at the flow class estimation device 11, it is first input to the flow statistics management unit 111. After performing processing necessary for managing the flow, necessary items are written in the flow management table 112. The flow learning unit 113 reads the content of the written flow management table 112, analyzes it, writes it once in the learning data 114, transfers it to the flow class estimation unit 115, and estimates the flow class.
In this embodiment, the input to the flow class estimation device 11 is the header information of the packet acquired by means such as the above 1) or 2), but as another example, NetFlow provided by Cisco, etc. It may be data output from a flow measurement device installed on a separate link.

FIG. 3 is a configuration diagram of a flow management table according to an embodiment of the present invention.
For the sake of simplicity, FIG. 3 shows an example in which an incoming / outgoing IP address, an incoming / outgoing port number, and a protocol type are used as flow attributes, and only the cumulative number of transmitted bytes is used as a flow statistical value. In addition, the first arrival time and the last arrival time are also recorded.

FIG. 4 is a diagram showing an example of learning data configured in the present invention.
In the upper database of FIG. 4, the number of flows belonging to the class Ci and the number of flows having the value ajk for the class Ci and the attribute Aj are recorded.
In the lower database of FIG. 4, the probability that the flow of class Ci has the attribute A is calculated using the upper data and Laplace correction.
In this example, for the sake of simplicity, the number of attributes is two (A1, A2), and each possible value is limited to four (a11, a12,... A24) as shown in the figure. An example is shown. For example, from the figure, the total number of flows belonging to class C1 is n1 = 37, and the number of flows whose class A2 and attribute A2 value is A2 = a23 is n223 = 4.

FIG. 5 is a flowchart illustrating a procedure when the flow class estimation unit according to an embodiment of the present invention determines a class.
As shown in FIG. 2, when a packet arrives at the flow class estimation device 11 (step 101), the flow statistics management unit 111 reads the value described in the header and records the statistical value in the flow management table 112 for each flow. And update. In FIG. 5, as an example of the statistical value, the time when the packet arrived first, the time when the packet arrived last, and the cumulative number of bytes are shown. The flow statistics management unit 111 newly registers each time a new flow that is not registered in the flow management table 112 arrives. If a flow that has already been registered in the flow management table 112 arrives, the flow statistics management unit 111 displays the statistical value. Update.

  In the flow management table 112, 1) entries may be increased as long as the memory capacity is free, and 2) some flow statistics may be deleted at any time according to predetermined timings and rules. For example, in the case of 1), when the memory capacity reaches a limit at a certain timing, all the entered flow information at that time is deleted. In the case of 2), for example, for each flow, the time of the last arriving packet is managed, and when the difference between the time and the current time becomes a predetermined timeout value = T seconds or more, By deleting the statistical value of the corresponding flow according to other timings and rules, the number of flows simultaneously held can be appropriately managed.

The flow learning unit 113 constructs flow learning data every time period or timing that can be set to an arbitrary value in advance. That is, based on a predetermined rule, all the flows registered in the flow management table 112 are classified for each class, and ni and nijk are recorded in the learning data using the attribute values of each flow. . An example of this is shown in the upper diagram of FIG.
The flow learning unit 113 further calculates probabilities P (Ci) and P (Aj = ajk | Ci) based on the data recorded in the learning data.

At this time, the flow learning unit 113 sets the total number of flows existing in the flow management table 112 as n, its reciprocal number f = 1 / n, the number of values that the attribute j can take as nj, and by correcting the Laplace, P (Aj = ajk | Ci) = (nijk + f) / (ni + f * nj).
An example is shown in the lower part of FIG.

  The flow class estimation unit 115 acquires the attribute A for each newly arrived flow based on the link measurement or the output from the separately installed flow measurement device, and examines the attribute A (step 102). That is, using the probability P (Aj = ajk | Ci) recorded in the learning data 114 and the probability P (Ci) = ni / n that the class Ci occurs, the discriminant function f (Ci) = P (Ci) * ΠjP (Aj = ajk | Ci) (Πj represents an operation of multiplying the subsequent term for all j) is calculated for all classes Ci (step 103), and Ci giving the maximum value of the discriminant function Is estimated as the class of the flow (step 104).

In the present embodiment, the flow statistics management unit 111 of the flow class estimation device 11 shown in FIG. 2 includes a read / write control circuit that writes information to or reads information from the flow management table 112, and a flow learning unit. The information 113 is read each time the information of the incoming flow is registered in the flow management table 112, and is recorded in the learning data 114 by being calculated by the arithmetic circuit. Further, the flow class estimation unit 115 includes a naive Bayes classifier in order to probabilistically estimate the class to which the flow belongs.
And the flow class estimation apparatus 11 of this invention implements a process by control of a computer.

  If the procedure of processing due to the arrival of the flow shown in FIG. 5 is programmed and stored in a recording medium such as a CD-ROM, the recording medium is attached to the computer of the flow class estimation apparatus connected to the router, and the computer The flow class estimation method according to the present invention can be easily realized by installing and executing the program. Further, the program can be generalized by downloading from the computer to the user who desires it via the network.

It is a block diagram which shows an example of the basic composition of the IP network to which this invention is applied. It is a block diagram of the flow class estimation apparatus which concerns on one Example of this invention. It is a block diagram of the flow management table which concerns on one Example of this invention. It is explanatory drawing of the learning data which the flow learning part which concerns on one Example of this invention processed. It is a flowchart which shows the process sequence of the flow class estimation part which concerns on one Example of this invention.

Explanation of symbols

DESCRIPTION OF SYMBOLS 10: Router 11: Flow class estimation apparatus 12: Link 111: Flow statistics management part 112: Flow management table 113: Flow learning part 114: Learning data 115: Flow class estimation part

Claims (6)

A method of probabilistically estimating a class of a flow that passes through a link in a communication network under computer control,
The flow statistics management unit registers the measurement value acquired by measuring the flow passing through the link, or by reading the flow statistics value acquired by a separately installed flow meter and writing it in the flow management table,
The flow learning unit reads information registered in the flow management table, calculates a probabilistic relationship between an attribute of the flow and a class to which the flow belongs based on the information, and calculates the calculation result as learning data When recording to
Class Ci in which flows are classified, attributes A = {A1, A2, A3,..., Am} (m attributes) of each flow, and possible values of each attribute Aj are Aj = ajk (k = 1, 2, ...)
Referring to the flow management table, count the number n of flows existing in the flow management table, the number of times ni occurs in class i, and the number of times nijk in which the flows classified into class i have attribute ajk,
Using the values ni and nijk, the probability that the attribute Aj has the attribute value ajk under the condition where the class Ci occurred is calculated as P (Aj = ajk | Ci) = nijk / ni and recorded in the learning data. ,
The flow class estimation unit probabilistically estimates a class to which the flow belongs by reading a classification result of the classifier from a newly arrived flow attribute based on the learning data using a naive Bayes classifier. When
Based on the link measurement or the output from a separately installed flow measurement device, acquire the attribute A for each newly arrived flow,
Using the probability P (Aj = ajk | Ci) recorded in the learning data and the probability P (Ci) = ni / n that the class Ci occurs, the discriminant function f (Ci) = P (Ci) × ΠjP (Aj = ajk | Ci) (Πj represents an operation for multiplying the subsequent terms for all j) for all classes Ci,
A flow class estimation method characterized in that Ci giving the maximum value of the discriminant function is estimated as a class of the flow by a naive Bayes classifier. The flow class estimation method according to claim 1 ,
The calculation of the probability P (Aj = ajk | Ci) in the flow learning unit takes into account the case where the class Ci and the attribute Aj do not exist in the flow management table at the same time, the number n of flows existing in the flow management table, And the reciprocal number f = 1 / n, the number of values that the attribute j can take is nj, and Laplace correction,
P (Aj = ajk | Ci) = (nijk + f) / (ni + f × nj )
Flow class estimating wherein the benzalkonium be calculated as. In the flow class estimation method according to claim 1 or 2 ,
Flow The flow class estimator, for each of all the flows arrives, an estimate of the class the flow belongs, or which is characterized in that to limit the flow of more sampling or filter-ring, to implement the estimation Class estimation method. In the flow class estimation method according to any one of claims 1 to 3 ,
Information registered in the flow management table includes at least a time at which a packet constituting the flow first arrives, a time at which a packet constituting the flow arrives at the end, a cumulative number of transmitted packets, a cumulative number of transmitted bytes, an arrival / departure interface, Consists of flow attributes,
The flow attribute is, arrival and departure IP address described in the header of the packet that constitutes the person said flow, departure AS number, departure prefix, departure port number, protocol type, TTL, TCP flags, TCP window size, Mu perforated window scale option the defined by any combination of including each data flow class estimating method comprising Rukoto such with more aggregated hash function number further. As a means of performing programmed computer processing,
A flow statistics management unit, a flow learning unit, and a flow class estimation unit in the flow class estimation method according to any one of claims 1 to 4,
Registration of information in the flow management table by the flow statistics management unit,
Recording of the learning data by the flow learning unit;
By estimating the class to which the flow belongs by the flow class estimation unit,
A flow class estimation device characterized by probabilistically estimating a class of a flow passing through a link in a communication network.   The program for making a computer perform each process in the flow class estimation method in any one of Claims 1-4.

Images