JPH1188328A - Traffic gathering and analyzing system - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a traffic gathering/analyzing system capable of gathering and analyzing traffic without omission in the transmission line of a wide band even at the time of using a traffic gathering/analyzing device with insufficient throughput. SOLUTION: This system is provided with the plural traffic gathering/ analyzing devices (personal computers PC1-PC3) connected on the same network segment and respectively provided with traffic gathering/analyzing means (gathering objects 1-3) and a partial charge adjustment means (adjustment object or the like) for adjusting partial charges in the traffic gathering/analyzing means. By the partial charge adjustment means, the respective traffic gathering/ analyzing means are made to take the partial charges of the traffic on the network segment without the omission and gather and analyze it.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a traffic collection / analysis system for managing traffic in a computer network such as the Internet, and more particularly, to coordinating a plurality of inexpensive machines with low processing capacity by distributing a load. The present invention relates to a traffic collection / analysis system that realizes high processing capability and prevents loss in a wideband transmission path.

[0002]

2. Description of the Related Art In traffic management of a computer network such as the Internet, it is necessary to collect packets flowing on a network and analyze them to create various statistical data, and in some cases, to save the packet contents. There is. The creation of statistical data includes, for example, the creation of statistical information such as the number of packets at the source and destination of a packet.
For example, there is storage of a TCP / IP header of a packet.

[0003] In traffic collection, RMON (Remote network MONitorin) is used as a standard of an interface for accessing collection items and collected data.
g), RMON2 is present. Most products for traffic collection and analysis in recent years conform to these standards. The traffic collection and analysis methods based on RMON and RMON2 are classified as follows. Conventional method 1: A method of collecting data with a network device such as a router or a hub (concentrator). Conventional method 2: Traffic collection / analysis device (probe)
How to collect in. Here, a traffic collection / analysis device (probe) is a device that examines all packets on a segment and analyzes them to create statistical information, or stores the packets so that they can be analyzed later. Possible dedicated hardware, workstation, or PC (personal computer).

[0004]

Generally, traffic collection / analysis is a high-load process, and the above-mentioned conventional methods 1 and 2 are not necessarily effective methods. (1) Since the traffic collection / analysis is a process with a high load, as in the above-mentioned conventional method 1, the network device such as a router or a hub (concentrator) is provided with the original work (routing process in the case of a router, etc.). ), It is difficult to perform these heavy traffic collection processes in view of their processing capabilities. In the future, it is expected that the network will become more and more wideband, so that the conventional method 1 is considered to be less effective. (2) Dedicated traffic collection / analysis device (probe)
It is thought that the processing capacity of the traffic collecting / analyzing device cannot keep up with the method of the above-mentioned conventional method 2 using the method. The reason for this is that the traffic collection / analysis processing will have a much higher load in the future, such as a broader network and a wider range of traffic collection / analysis items.

(3) Even if the traffic collection / analysis device has the ability to perform traffic collection / analysis processing of a specific network, the use of the traffic collection / analysis device requires a great deal of cost. There's a problem. The reason for this is that traffic collection and analysis requires high-load processing, and high-performance traffic collection and analysis devices (probes) must be used for broadband transmission lines such as backbones (backbone networks). This is because individual traffic collection / analysis devices (probes) become expensive.
If a traffic collection / analysis device (probe) with insufficient processing capacity is used to avoid this, collection / analysis processing cannot be performed on all packets, and missing packets will occur, resulting in accurate statistics. The problem that it cannot be obtained arises.

(4) Further, in the Internet, since traffic is generally distributed without being concentrated at a specific location, the number of installed traffic collection / analysis devices increases.
Therefore, the cost of each traffic collecting / analyzing device becomes a big problem. An object of the present invention is to solve these problems by collecting and processing traffic with insufficient processing capacity.
An object of the present invention is to provide a traffic collection / analysis system that can collect and analyze traffic without fail over a wide band transmission path even when an analysis device is used.

[0007]

In order to achieve the above object, a traffic collection / analysis system of the present invention is connected on the same network segment, and each of them is a traffic collection / analysis means (collection object, collection process). A plurality of traffic collecting / analyzing apparatuses (for example, personal computers PC) each comprising: and a sharing adjusting means for adjusting the sharing in the traffic collecting / analyzing means (collection object, collection process). It is characterized in that the traffic on the network segment is collected and analyzed by being shared among the respective traffic collecting and analyzing means.

[0008] Further, the sharing adjustment means includes an attribute of a packet to be collected and analyzed by each traffic collecting / analyzing device (time when the packet passes through the network segment,
(The information written in the header of the packet) is determined in advance so as not to be missed, and when the same packet is collected by a plurality of traffic collection / analysis devices, the duplicated packet is detected. (Embodiments 1 and 2: In this case, the sharing adjustment means corresponds to some functions of the comparison object or the collection object).

In addition, the sharing adjusting means dynamically adjusts the sharing so that the traffic collecting / analyzing apparatuses start collecting after communicating with each other so that no data is missed. In addition, when the same packet is collected by the plurality of traffic collecting / analyzing devices, the duplicated packet is detected and removed (Embodiment 3: In this case, , Sharing adjustment means corresponds to an adjustment object).

[0010]

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS A preferred embodiment of the present invention has the following basic configuration. A plurality of traffic collection / analysis devices are installed on the same segment of the network that performs traffic collection / analysis. A traffic collection / analysis process (collection process) is operated on each traffic collection / analysis device. Each collection process is assigned a packet to be collected and analyzed, and each collection process selectively collects and analyzes a packet flowing through the segment to be measured according to the assignment. Aggregate traffic data collected and analyzed in each collection process.

[0011] In the above, each collection process collects
The following method is used for assigning a packet to be analyzed. (A) A method in which the allocation of packets to be collected and analyzed by each collection process is determined in advance (a method in which the allocation is statically determined in advance). As an example of this case, a method of pre-allocating a time zone for collecting a packet for each collection process, a method of pre-allocating an attribute of a packet to be collected (for example, a protocol written in a header) for each collection process, or the like is available. is there. (B) A method of deciding the sharing by communicating and adjusting between the collecting processes after each collection process is started (a method of dynamically adjusting and deciding the sharing during operation). As an example of this case, there is a method of dynamically deciding the allocation of packets to be collected between the collection processes while referring to the arrival order of the arriving packets.

(Regarding Role Sharing and Its Effect) As described above, as a method of allocating a packet to be collected to each collection process, the “method of determining the sharing in advance” and the “method of adjusting and determining the sharing” are described. is there. (A) In the “method of pre-determining allocation”, allocation of packets to be collected to each collection process is determined in advance, and roles are allocated. According to this method, it is possible to reduce the load per traffic collection / analysis device and collect and analyze all traffic as a whole. (B) In the “method of adjusting and determining the sharing”, the assignment of each collection process is dynamically changed during packet collection. According to this method, the load of each traffic collection / analysis device can be easily changed, so that scalability can be increased.

(Regarding Missing) Since the role of each collection process is assigned so that no missing occurs,
Aggregating the data from each collection process creates traffic data for that segment.

Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings. In the present invention, as a method of assigning the role of packet collection to each collection process, there are a “method of determining the assignment in advance” and a “method of adjusting and assigning the assignment”.
Further, as a “method of determining the sharing in advance”, there are a “method of determining the sharing by the passage time of the packet” and a “method of determining the sharing by the header attribute of the packet”. Hereinafter, as a first embodiment, a “method of deciding sharing based on a packet transit time”
In the following, a description will be given of a case where the “method of deciding the assignment by the header attribute of the packet” is used as the second embodiment, and a case where the “method of deciding the assignment by adjusting” is used as the third embodiment.

(Embodiment 1) Description of Embodiment Using "Method of Determining Allocation Based on Packet Passing Time" FIG. 1 is a diagram for explaining an embodiment using a "method of determining sharing based on packet transit time". It is. The "method of deciding the sharing by the transit time of the packet" is a method of synchronizing the time between the traffic collecting / analyzing devices and allocating the time zone for collecting the traffic to each collecting process (collecting object) to determine the sharing. It is. That is,
Each collection process (collection object) is determined to perform collection / analysis processing only for packets that have passed that segment during the assigned time zone.

FIG. 1A shows a 100 M Ethernet.
t (registered trademark) (Ethernet with specifications available at 100 Mbit / s), three PCs (personal computers),
An example is shown in which one collection object that communicates on a general-purpose distributed object platform is operated on PC2 and PC3 to collect and analyze traffic. An NTP (Net) is used between personal computers PC.
It is assumed that time synchronization with guaranteed accuracy is performed by using a work time protocol. It is assumed that the collection object 1 exists in the personal computer PC1, the collection object 2 exists in the personal computer PC2, and the collection object 3 exists in the personal computer PC3.

(Configuration of Collection Objects) FIG. 2 is a block diagram showing the internal configuration of each of the collection objects 1 to 3. In the figure, 20 is a collection object, 22 is a primary buffer, and 24 is storage means. The storage means 24
Database (DB) of network management information standardized by the TCP / IP protocol, that is, MIB (Manag)
ement Information Base). Reference numeral 21 denotes a process 1 for storing packets in the primary buffer 22; 24, processing of the packets stored in the primary buffer 22 to create various statistical data; 2 shows a process 2 for saving.

FIG. 1B shows collection objects 1 to 3 of three personal computers PC1 to PC3.
3 is a diagram showing the operation content of No. 3 along a time axis. FIG. In the figure, (a) shows "continuous collection time", and (b) shows "processing time". The “continuous collection time” is a time for accumulating packets passing through the segment in the primary buffer 22 (a time during which the process 1 operates), and the “processing time” is for creating various statistical data from the collected packets. And the time for storing the packet and the created statistical data (the time during which the process 2 operates) (see FIG. 2).

The role assignment in the present embodiment is performed by assigning a "continuous collection time" to each of the collection objects 1 to 3. In this case, if the synchronization between the respective successive acquisition times is not accurate, a dropout occurs at the time of switching. In order to prevent this, it is necessary to provide a margin, that is, an overlap, between the continuous acquisition times of the respective acquisition objects by the synchronization accuracy.

In this embodiment, the capacity of the primary buffer 22 of the collection object is determined so as to satisfy the condition of equation (1). Primary buffer capacity> Bandwidth of transmission path to be measured × Guaranteed accuracy of time synchronization × 2 (1)

The reason for this will be described with reference to the continuous acquisition time (a) and the processing time (b) in FIG. In FIG. 1, (X) represents the overlap time (degree of synchronization) between the collection objects, and (Y) represents the continuous collection time (a). These (X) and (Y) are determined as follows. (X) Determination Method (X) is determined as follows. (X) = guaranteed accuracy of time synchronization (2) Determination method of equation (Y) (Y) is one continuous collection time of the collection object (a)
Is represented. This continuous collection time needs to satisfy at least the condition of the following expression (3). (Y)> (X) × 2 (3) where the continuous collection time (Y) is: continuous collection time = capacity of primary buffer / bandwidth of transmission path to be measured. expressed. It can be seen from the condition of the expression (3) and the expression (4) that the condition of the above expression (1) needs to be satisfied.

From the above equation (1), the capacity of the primary buffer can be determined in consideration of the bandwidth of the transmission path to be measured and the guaranteed time synchronization accuracy. For example, 100M
When a synchronization accuracy of 10 ms is obtained in the transmission path of bps, (X) = 10 ms Expression (5) Expression (Y)> 20 ms Expression (6), which satisfies Expression (6). It can be seen that a primary buffer of 2000 Kbits or 250 Kbytes (= 2000/8) Kbits or more may be provided for this purpose.

With the above configuration, the following becomes possible. The collection (continuous collection) can be performed continuously within the allocated time of each collection object. There is no missing when switching roles between PCs.

(Method of Summarizing Data) A method of calculating the union of information collected by each collection object will be described below. (A) Packets collected during the time (X) during which each collection object is synchronized with the accuracy of synchronization, that is, when data is repeatedly collected, are stored. (B) The packets collected during (X) are compared between the collection objects that are collected in duplicate, and the duplicated packets are obtained. (C) The same analysis as for the collected packets is performed on the packets collected in duplicate, and the analysis data is stored. (D) The stored analysis data is subtracted from the sum of the data collected by the collection objects 1 and 2.

The comparison process (B) and the comparison process (C)
The analysis and storage processing of the packets collected in duplicate
There are two types of methods: a method assigned by an object provided exclusively for this purpose (hereinafter referred to as a comparison object) and a method assigned to each collection object. Hereinafter, each will be described. Method for Providing Dedicated Comparison Object A dedicated comparison object is provided, and the comparison object (B) and the analysis / storage process (C) are assigned to the comparison object. Each collection object sends the packets collected during (X) to the comparison object. The comparison object compares the data between the objects collected in duplicate, calculates the duplicated packet, and performs the analysis and storage processing of (C).

Method of Comparison by Each Collection Object The comparison process (B) and the analysis / storage process (C) are performed only for one of the two collection objects that are collecting packets in duplicate. In charge. At this time, the other collection object of the two collection objects that are being collected in duplicate performs the comparison processing of (B) and the analysis and storage processing of (C) with the packet collected during the period of (X). Send to the collection object in charge. The method of selecting one of the collection objects responsible for the comparison process (B) and the analysis / storage process (C) is to distribute the load as much as possible. In the case of FIG. 1 including three collection objects, for example, the comparison between the packet collected by the collection object 1 and the packet collected by the collection object 2 is as follows.
The comparison between the packet collected by the collection object 2 and the packet collected by the collection object 3 is as follows.
However, the collection object 3 compares the packets collected by the collection object 3 with the packets collected by the collection object 1.

Next, a method for obtaining the number of packets that have passed through the segment to be collected when the collection is duplicated between the collection object 1 and the collection object 2 will be described. The number of packets p to be obtained is as follows. p = x + yz Here, x: number of packets collected by the collection object 1 y: number of packets collected by the collection object 2 z: number of packets collected by duplication Data collection is performed by a dedicated control line If there is no problem, use a network that collects and analyzes traffic. In that case, the data is compressed and transmitted in order to reduce the load on the network for collecting and analyzing the traffic.

According to the present embodiment, since the configuration as described above is employed, it is possible to perform traffic collection and analysis by distributing the load by a plurality of traffic collection and analysis devices. Further, in the present embodiment, P with low processing capacity is used.
Even for C, shorten the time to collect traffic,
It can be used by increasing the processing time.

(Embodiment 2) Description of Embodiment Using "Method of Determining Allocation by Packet Header Attribute""Method of determining allocation by packet header attribute" collects information by protocol, source address, destination address, etc. Although it is conceivable to determine the sharing of objects, it is considered that the load may be greatly biased by the load generated by a single user on the Internet.
For practical use, it is necessary to use an attribute that does not cause bias, and a method of distributing the load by using an ID identifier, a header checksum, or the like is also conceivable. in this case,
It is necessary to determine the sharing so that the load is not biased in a very short time.

When the roles are shared by using the ID identifier, the header checksum, and the like, unlike the case of the first embodiment, the packets collected by each collection object do not overlap. This means that collection object 1
Indicates that the last one bit of the ID identifier is “0”, and the collection object 2 indicates that the last one bit of the ID identifier is “1”.
It is clear from the example that if packets are collected, the packets collected by the collection objects 1 and 2 will not overlap. Therefore, when obtaining a statistical value, it is only necessary to sum the data of each collected object. Further, the same method as in the first embodiment is used for data collection.

According to the present embodiment, it becomes possible to collect and analyze traffic by distributing the load among a plurality of general-purpose computers. Further, even when a PC having a low processing capability is used, the amount of collected packets of the PC is reduced (ID
When the last two digits of the identifier are "00", it becomes available.

(Embodiment 3) Description of Embodiment Using "Method of Adjusting and Determining Allocation" FIG. 3 shows an example of the "method of adjusting and determining allocation".
It is a figure for explaining an example of “a method of deciding sharing based on arrival order”. The “method of deciding sharing based on arrival order” is a method in which each collection object dynamically determines sharing using arrival order information on a segment. In other words, this is a method of collecting and analyzing packets sequentially arriving at each collection object. This method
The present invention can be applied to those in which the arrival sequence of packets in the same segment is the same, such as shared media of multiple access. In this method, a coordination (coordination) mechanism for preventing the same packet from being collected between each collection object is essential.

The reason is that, since the packets themselves do not include the order information, the packets to be collected by each collection object cannot be known. For example, 2
It is assumed that when one collection object collects every other packet, the same packet is collected by both collection objects. An adjustment object is provided to monitor these, and when it is detected that the same packet has been collected by both collection objects, an instruction is given to one of the collection objects to shift the packet to be collected by one. The method is conceivable.

In FIG. 3, for 100M Ethernet,
Traffic collection and analysis are performed using two personal computers, and each personal computer operates one collection object that communicates on a general-purpose distributed object platform. The adjustment object is operated by one of the personal computers. The example of FIG. 3A shows a case where the collection object 1 and the adjustment object are operated on the personal computer PC1, and the collection object 2 is operated on the personal computer PC2.

(Description of Operation) Before starting, the frequency n / m of packets collected for each collection object is determined according to the processing capacity of each PC (n and m are natural numbers). The frequency n / m means that when m packets arrive, exactly n packets are collected. This operation is performed at a frequency of 2/5, that is, n = 2,
FIG. 4 shows an example where m = 5. In FIG. 4, packets p (1), p (2), p (3),... P (5a +
When 1), p (5a + 2), p (5a + 3),... Arrive, 2/5 of the packets, that is, p (1), p (3),.
a + 1), p (5a + 3),...

Each collection object starts simultaneously,
Collection is started from an arbitrary packet, and n packets are collected into m packets. After each collection object has begun collection, the coordination object examines the collection packets of each collection object. When a plurality of collection objects are collecting the same packet, an adjustment method for each collection object is calculated, and an instruction is issued to the collection object. In the calculation of the adjustment method, how to shift packets collected by each collection object in order to eliminate missing data is calculated.

(Method of Determining Frequency n / m of Collected Packets) Among the PCs used for collection, n of the collection object on the PC having the lowest processing capacity is set to 1, and m corresponding to the capacity is determined. The collection object of another PC uses the m determined above to determine n corresponding to the processing capacity. Also, the number of PCs is prepared so that the sum of n assigned to each collection object is equal to or greater than m.

This mechanism will be specifically described with reference to FIG. It is assumed that both the collection objects 1 and 2 have n = 1 and m = 2. Assume that two collection objects happen to collect the same packets p (1), p (3), and p (5) at the time of activation (see FIG. 3B). The coordinating object examines the packets collected by each of the collection objects 1 and 2, finds out that the two collection objects are collecting the same packet, and instructs the collection object 2 to shift the packet to be collected by one. Put out. The collection object 1 continues collection in the same order as before (p (7), p (9), p (11),...).
On the other hand, the collection object follows the instruction of shifting the collection object 2 by one, and thereafter, the packet p shifted by one.
(8), p (10),...). By shifting the packets to be collected by one collection object one by one in this manner, subsequent missing is eliminated (see FIG. 3C).

(Data Aggregation) In this embodiment, no data duplication occurs after the adjustment is completed. Therefore, when obtaining a statistical value, it is sufficient to simply take the sum of the data of each collected object. The data collection is performed in the same manner as in the first embodiment.

According to this embodiment, with the above configuration, it is possible to collect and analyze traffic by distributing the load among a plurality of general-purpose computers. Also, when the load is distributed by a PC with low processing capacity,
C can be used by reducing the collection frequency of the packet C (for example, by preparing 10 PCs, each of which collects one packet for every 10 packets).

[0041]

According to the present invention, a PC having a low processing capacity is used.
(Personal computers) can also collect and analyze traffic by operating a plurality of them in cooperation, so that the cost of traffic collection and analysis can be reduced (cost reduction). Also, since the scalability is high, even if the bandwidth increases, it can be easily handled by increasing the number of general-purpose computers (can be used for a broadband transmission path).

[Brief description of the drawings]

FIG. 1 is a diagram illustrating a system configuration according to a first embodiment of the present invention and an operation example of each collection object.

FIG. 2 is a diagram illustrating a configuration example of a collection object according to the first embodiment of the present invention.

FIG. 3 is a diagram illustrating a system configuration according to a third embodiment of the present invention and an operation example of each object.

FIG. 4 is a diagram illustrating an operation example of each collection object when the collection frequency of a packet is 2/5 in the third embodiment of the present invention.

[Explanation of symbols]

20: collection object, 21: process processing 1, 2
2: Primary buffer, 23: Process processing 2, 24: Storage means (MIB; Management Information Base)

 ────────────────────────────────────────────────── ─── Continued on the front page (72) Inventor Shigeyuki Komatsubara 3-19-2 Nishishinjuku, Shinjuku-ku, Tokyo Inside Nippon Telegraph and Telephone Corporation

Claims (6)

[Claims] 1. A plurality of traffic collecting / analyzing devices connected on the same network segment, each including a traffic collecting / analyzing unit, and a sharing adjusting unit for adjusting the sharing of the traffic collecting / analyzing unit. A traffic collection / analysis system for a computer network, wherein the sharing / adjusting means causes the traffic on the network segment to be shared / analyzed without being missed by the traffic collection / analysis means. Analysis system. 2. The traffic collection / analysis system according to claim 1, wherein the sharing adjustment unit determines in advance that the attribute of a packet to be collected / analyzed by each of the traffic collection / analysis devices does not occur. And a means for detecting and removing the duplicate packet when the plurality of traffic collection / analysis devices are collecting the same packet. 3. The traffic collection / analysis system according to claim 2, wherein the attribute of the packet is a time at which the packet passes through a network segment. 4. The traffic collection / analysis system according to claim 2, wherein the attribute of the packet is information written in a header of the packet. 5. The traffic collecting / analyzing system according to claim 1, wherein the sharing adjusting means communicates with each other after the traffic collecting / analyzing apparatuses start collecting, so as to prevent the data from being missed. And a means for detecting and removing the duplicated packet when the plurality of traffic collection / analysis devices are collecting the same packet.
Analysis system. 6. The traffic collecting / analyzing system according to claim 5, wherein the dynamic adjustment of the sharing by the sharing adjusting means causes the traffic processing to be shared at a predetermined frequency for each traffic collecting / analyzing means. A traffic collection / analysis system characterized in that when the processing of the same traffic is duplicated by a plurality of traffic collection / analysis means, the order of traffic handled by the plurality of traffic collection / analysis means is shifted from each other.