JP2015046683A - Traffic scanning device and method - Google Patents

Abstract

PROBLEM TO BE SOLVED: To quickly and flexibly discriminate whether or not data included in a packet to be transferred on a communication network are belonging to a predetermined data group.SOLUTION: A hash value is generated with a predetermined entry pattern to be detected as a key, and the hash value is stored as collation data. Also, a mask for using data having desired inspection length included in traffic as an entry is generated and stored. Data from a desired position designated by an offset value are segmented from a traffic pattern, and the mask is applied to the segmented data such that a hash value is generated as an entry having the inspection length. Thus obtained hash value of each packet is compared with the stored collation data such that whether or not a suspicious pattern is included is determined.

Description

  The present invention relates to a traffic scanning apparatus and method for identifying data included in a packet transferred on a communication network.

  The data matching problem that checks whether two pieces of data match as a bit string, a byte string, a character string, etc. is an important problem that frequently appears in many applications, such as text search, database engineering, communication engineering, etc. It is used as an elemental technology for important technical issues in the field. In particular, a technique for comparing packets in a communication network and detecting and discriminating matching packets is an important elemental technique used for ensuring security and stabilizing infrastructure by detecting abnormal traffic.

  A Bloom filter shown in Non-Patent Document 1 is well known as a technique used for collating data to determine what matches. The Bloom filter is a kind of classic data structure devised in the field of data science, and is intended to improve spatial calculation efficiency by allowing a certain amount of false-positive detection. The Bloom filter is generally used also in the communication field because it can perform efficient data search and match determination.

As an example of using the Bloom filter in the communication field, the one shown in Non-Patent Document 2 can be cited. In the non-patent document 2, a plurality of bloom filters are arranged in parallel as shown in FIG. 1 in order to cope with packets of various data lengths while maintaining the high speed of the bloom filter. In this configuration, assuming that the minimum value L _min of the packet data length is 3 bytes and the maximum value L _max is w bytes, the length is L _max bytes and the packet data is shifted by 1 byte per clock. 91 and L _max −L _min +1 Bloom filters 92 provided in parallel with each other. Each of these Bloom filters 92 is a Bloom filter whose data length inputted by the first 3 bytes of the window cache 91 is L _min (= 3 bytes), and whose data length inputted by the first 4 bytes of the window cache 91 is 4 Bloom filter for bytes,..., Bloom filter for which the data length input by all data in the window cache 91 is L _max (= w bytes). In the configuration shown in FIG. 1, by providing a Bloom filter for each data length, packet data of various data lengths can be processed at high speed by hardware.

JP-T-2005-522781

Burton H. Bloom, "Space / Time Trade-offs in Hash Coding with Allowable Errors", Communications of the ACM, Vol. 13, No. 7, pp. 422-426 (July, 1970). S. Dharmapurikar, et al., "Deep Packet Inspection Using Parallel Bloom Filters", IEEE Micro, Vol. 24, Issue 1, pp. 52-61 (January / February 2004).

  The traffic flowing on the communication network includes packets of various data lengths, and in order to determine whether or not the data included in the packet is included in a predetermined data set for the purpose of traffic inspection or the like, As shown in Patent Document 1, there is a configuration in which a plurality of Bloom filters having different target data lengths are arranged. In this configuration, processing can be performed at high speed, but hardware is prepared for each data length of the packet to be inspected, which causes a problem that the circuit scale and wiring complexity increase. In addition, this configuration cannot cope with inspection lengths other than the register length prepared in advance, and it is difficult to say that the flexibility is sufficiently large.

  On the other hand, although it is possible to determine whether or not the packet data is included in a predetermined data set by software processing, the software processing takes time compared to the processing by hardware, and the packet in communication traffic having a wider bandwidth With respect to performing comparison of data and determination of coincidence, sufficient processing performance cannot be obtained by processing by software.

  An object of the present invention is a traffic scanning method for identifying data transferred on a communication network, and discriminates whether or not data included in a transferred packet belongs to a predetermined data set at high speed and flexibly. It is an object of the present invention to provide a traffic scanning apparatus and method that can be used.

  The traffic scanning device of the present invention is a traffic scanning device that determines whether or not data included in a packet transferred on a communication network belongs to a predetermined data set, and is given a key to be a mapping of the key. A hash calculation means for executing a hash calculation for generating a hash value, a storage means for holding a result obtained by executing a hash calculation by a hash calculation unit using a predetermined entry pattern as a key, as positive matching data, and a traffic pattern , An extraction means that extracts data of an arbitrary inspection length from the arbitrary position as an entry, and gives the entry calculated by the extraction means as a key to the hash calculation means to execute a hash operation to obtain a result And an inspection means for comparing and collating the result with the collation data held in the storage means.

  The traffic scanning method of the present invention is a traffic scanning method for determining whether or not data included in a packet transferred on a communication network belongs to a predetermined data set, and performs a hash operation using a predetermined entry pattern as a key. Executing and holding the result of the hash operation as positive collation data, extracting the data of an arbitrary inspection length from the arbitrary position from the traffic pattern as an entry, and using the entry as a key A step of executing a hash operation and using the result as comparison data, and a step of comparing and comparing the comparison data and the verification data.

  According to the present invention, from the traffic pattern, data of an arbitrary inspection length from an arbitrary position is extracted, a hash value is calculated as an entry, and the hash value and the hash value for verification are compared and verified. Therefore, it is possible to cope with an arbitrary inspection length without using a plurality of Bloom filters, and it is possible to quickly and flexibly discriminate whether or not data included in a packet belongs to a predetermined data set. Is obtained.

It is a block diagram which shows an example of the circuit which collates a packet using a some Bloom filter. It is a block diagram which shows the structure of the traffic scanning apparatus of one Embodiment of this invention. It is a block diagram which shows an example of a structure of a RISC processor. It is a figure explaining the process of traffic scanning.

  Next, a preferred embodiment of the present invention will be described with reference to the drawings.

  The traffic scanning apparatus according to the embodiment of the present invention shown in FIG. 2 is for discriminating whether or not data included in a packet transferred on a communication network belongs to a predetermined data set. This traffic scanning device records the entry pattern used for the hash value generation, and the hash calculation unit 21 that generates a hash value that becomes the mapping using the traffic pattern and the value of a predetermined reference entry pattern as a key. An entry registration unit 22 that realizes output and bit manipulation; a collation data storage unit 23 that stores an arbitrary hash value obtained in the hash calculation unit 21 as positive collation data for use as a comparison target; An offset cutout unit 24 and a scan window setting unit 25 for extracting an entry of an arbitrary inspection length from a traffic pattern arriving one packet at a time; a pattern matching unit 26 for matching a pattern represented by a hash value; A collation result discriminating unit 27 for discriminating collation in the unit 26; The signal input / output unit 28 for transferring data including traffic and control commands between these functional units 21 to 27 and between them and an external memory, and the entire functional units 21 to 28 are controlled. For this purpose, a signal control unit 29 is provided that includes a general CPU (Central Processing Unit) instruction and an extended instruction set, and executes settings and operations in accordance with arbitrary instructions from the user. In particular, the hash calculation unit 21 corresponds to a hash calculation unit, the collation data storage unit 23 corresponds to a storage unit, the offset cutout unit 24 and the scanning window setting unit 25 correspond to an extraction unit, and the pattern matching unit 26 serves as an inspection unit. Correspond. Each functional unit, that is, hash calculation unit 21, entry registration unit 22, collation data storage unit 23, offset cutout unit 24, scanning window setting unit 25, pattern collation unit 26, collation result discrimination unit 27, signal input / output unit 28, and signal The control unit 29 can exchange data and signals with each other by connecting to the signal line 20 in the bus form.

  In the above configuration, specifically, the pattern matching unit 26 gives an entry extracted from the traffic pattern by the offset cutout unit 24 and the scanning window setting unit 25 to the hash calculation unit 21 to calculate the hash value. The hash value peculiar to each packet (that is, the hash value of the traffic pattern) obtained in this way is compared with the above-described positive matching data stored in the matching data storage unit 23, and is thereby extracted from the traffic pattern. Check whether the entry matches the target pattern. The collation result discriminating unit 27 discriminates that the case where the suspicious pattern is detected is positive and the case where the suspicious pattern is not detected is negative according to the inspection result of the pattern collating unit 26. For example, when a pattern that a packet to be discarded is supposed to be an entry pattern and the hash value of the entry pattern is stored as a positive value in the verification data storage unit 23, the pattern verification unit 26 When it is detected that the hash value obtained from the pattern matches that stored in the collation data storage unit 23, the collation result discriminating unit 27 determines that the suspected pattern has been detected and determines that the hash value is positive.

  Next, extraction of entries from the traffic pattern by the offset cutout unit 24 and the scanning window setting unit 25 will be described.

  In the traffic scanning device of the present embodiment, an arbitrary data length (referred to as inspection length) portion of data included in a packet is used as an entry and designated in advance by a user setting or by a program. Since this entry is not necessarily at the beginning of the packet, when the start of the field value corresponding to the entry is specified as an offset value in the traffic pattern that sequentially arrives one packet at a time, the offset extraction unit 24 For each packet, the length of the field indicated by the offset value is removed from the beginning of the packet, and the remaining field portion is cut out and sent to the scanning window setting unit 25. The scanning window setting unit 25 has a mask function for generating the above-described inspection length entry, and performs a mask operation on the field cut out by the offset cutout unit 24. For example, by using a mask in which “1” bits are continued for the number of bits corresponding to the inspection length, followed by “0” bits, the logic for each bit is applied to the field cut out by the offset cutout unit 24. By executing the product (AND) operation, the scanning window setting unit 25 generates an entry having the inspection length. As described above, this entry is sent to the hash calculation unit 21 by the pattern matching unit 26 and subjected to hash calculation.

  The traffic scanning device according to the present embodiment can be configured such that each of the function units 21 to 29 is a separate hardware element, or processing corresponding to the operation of each of the function units 21 to 29 is performed by a computer. It can also be realized by software by a program to be executed. For example, a general RISC (Reduced Instruction Set Computer) processor can be used as a computer.

  By the way, the processing performed by the traffic scanning device of the present embodiment is specific to a specific application called traffic scanning, such as hash calculation, hash value comparison, field extraction based on an offset value, and mask operation with an arbitrary bit length. Although operations are included, these operations are difficult to execute at high speed with general-purpose hardware resources in a computer. Therefore, application-specific processor design methods as disclosed in Patent Document 1 are applied to a general RISC processor to compare hash operations, hash values, cut out fields based on offset values, and use arbitrary bit lengths. An arithmetic unit or a register for executing an operation such as a masking operation as a dedicated instruction for a specific application can be added to the processor. A person skilled in the art can easily understand that a traffic scanning device that executes arithmetic processing for traffic scanning at high speed while being software-controlled can be realized by adding arithmetic units or registers that execute dedicated instructions for specific applications. It can be done.

  FIG. 3 shows the configuration of a general RISC processor that can be used in this embodiment. The RISC processor 30 includes a plurality of registers 31, an ALU set 32 composed of a plurality of ALUs (Arithmetic Logical Units), a selector 33 that selects a register for sending data to the ALU set 32, and a memory ( RAM) 34, an external interface (I / F) 35, a load / store unit 36 for executing processing of loading and storing to the RAM 34 and the external interface 35, and data from the ALU set 32 and the load / store unit 36 And a selector 37 for selecting a register supplied with the. In general, a processor involves frequent data movement between a memory and a register file, and this becomes a bottleneck when high-performance processing is required. The RISC processor used in the present embodiment configures an execution stage as an ALU set 32 in which an ALU that has been expanded and optimized for specific processing including hardware assist and a general ALU are combined, and the execution stage is further improved. By providing the processing instruction including the logic to be operated automatically, it is possible to greatly reduce the number of required cycles in the operation required for the traffic operation processing.

  The RISC processor used in the present embodiment includes an instruction set that controls the function for a specific application as described above, but the bit width of the instruction set and the data width is not limited to a specific value. Depending on the design technology, there may be multiple implementation solutions for the instruction set.

  Next, the operation of the traffic scanning device of this embodiment will be described.

  In order to detect a target traffic pattern by scanning communication traffic, the entry registration unit 22 registers the pattern to be detected as an entry pattern in advance in the verification data storage unit 23. In the present embodiment, since it implies that the pattern to be detected is changed, it is assumed that the entry registration unit 22 and the collation data storage unit 23 correspond to deletion of the entry pattern.

  Since the hash calculation unit 21 has a configuration specialized for hash calculation, it enables high-speed calculation and assists calculation by a general-purpose ALU. The hash calculation unit 21 calculates the number of combinations of single or multiple types of hash calculation commands as an arbitrary number of times using the entry value input from the entry registration unit 22 as a key, and becomes a hash that becomes a mapping of the value as the key A value is generated and output to the entry registration unit 22. In the example shown in FIG. 4, “101001010101111” in binary notation is given in advance to the entry registration unit 22 as an entry pattern, which is sent as a key to the hash calculation unit 21, and the hash calculation unit 21 Is calculated and output to the entry registration unit 22. As a matter of course, a plurality of entry patterns can be prepared and the hash values of each of them can be calculated and stored in the collation data storage unit 23.

  On the other hand, for the traffic pattern input from the signal input / output unit 28 to the pattern matching unit 26, the cutout position is designated by an offset command according to the offset value designated by the program or the user, and the offset cutout unit 24 is described above. Cutout processing is performed as described above. Subsequently, a mask process is executed by the scanning window setting unit 25 in response to the mask command on the extracted traffic pattern. As described above, the mask used for the mask processing is, for example, one in which “1” of the number of bits corresponding to the inspection length designated by the program or the user is continuous, followed by “0” bits. . As a result of these processes, an entry having a length matching the inspection length is extracted from the first traffic pattern starting from the cut-out position specified by the offset command. In the example shown in FIG. 4, “1000”, that is, 8 bits is specified as an offset value in binary representation, and “11111111 11111111 11110000 00000000” corresponding to an inspection length of 20 bits is used as a mask. As a result, when the traffic pattern represented by “00000000 10101010 11101110 01010101 00000000” is input, the first 8 bits are removed as an offset, and the subsequent 20-bit portion, that is, “10101010 11101110 0101” is a valid value by masking. That is, it is extracted as a field corresponding to the entry pattern. The hash calculation unit 21 calculates the number of combinations of single or plural types of hash calculation instructions any number of times using the value of the entry pattern extracted in this way as a key, and generates a hash value that becomes a mapping to perform pattern matching. To the unit 26.

  In the present embodiment, in order to be able to determine whether or not an entry of an arbitrary inspection length in the traffic pattern is included in a predetermined data set, designation of an offset for the traffic pattern and a mask operation are used. As a result, it is possible to obtain a hash value for an entry at an arbitrary length and position in the traffic pattern, and to obtain a higher flexibility with respect to the inspection length than a circuit realized only with conventional hardware.

  The traffic scanning of this embodiment is based on matching hash values calculated from entries, and basically depends on the same processing principle as the Bloom filter. However, as described above, since it is possible to obtain a hash value for an entry at an arbitrary length and position in the traffic pattern, the traffic scanning of this embodiment does not provide a plurality of bloom filters. By using a single hash calculation unit and a single pattern matching unit, there is an advantage that an arbitrary inspection length can be handled. Thus, according to the present embodiment, the hardware scale can be greatly reduced while maintaining the same processing speed as when a plurality of Bloom filters are provided.

  The hash value input to the entry registration unit 22 can be regarded as a contracted expression in which the entry pattern is mapped to the hash value, and data (positive value) for representing positive when the communication traffic pattern is verified. Become. The entry registration unit 22 registers this positive value in the collation data storage unit 23 by bit operation.

  The pattern matching unit 26 compares and matches this positive value with the hash value calculated from the traffic for each packet. The value of the result of the comparison and collation is determined by the collation result discriminating unit 27. If the result is determined to be positive, detection of the suspicious pattern is completed, and traffic is separated or blocked by an instruction from the signal control unit 29. It becomes possible to perform control such as disposal.

  In this embodiment, since it is based on the same processing principle as the Bloom filter, there is no possibility of occurrence of false negatives, but there is a possibility of occurrence of false positives. Although details are omitted, in data matching using a Bloom filter, the number of hash operations that can minimize false positives when the number of entries and the size of the hash table are determined is well known. . Also in this embodiment, by maintaining an appropriate ratio between the size of the hash table and the number of entries, a false positive appearance probability that is sufficiently small in practice can be obtained. The size of the hash table can be arbitrarily determined according to the application. In this embodiment, since entries can be registered and deleted, even if the size of the hash table is arbitrarily determined, by changing the number of entries according to the size of the hash table, false The probability of positive occurrence can be maintained at an acceptable level. Since the possibility of false negatives does not exist in principle, the signal control unit 29 performs control such as passage of packets that match negative when the collation result discriminating unit 27 determines negative. To do.

  A number of algorithms have been proposed for the hash value calculation procedure in the Bloom filter, but these already proposed algorithms can also be used as the algorithm used in the hash calculation unit 21 in the present embodiment. It is also possible to improve the distribution characteristics of the hash value by adopting an algorithm other than these.

  After all, in the traffic scanning device of the present embodiment, in summary, a hash value is generated in the hash calculation unit 21 using a predetermined entry pattern to be detected as a key, and this hash value is stored in the verification data storage unit 23 as verification data. . In addition, a mask for using data of a desired inspection length included in the traffic as an entry is generated and stored. Data from a desired position specified by the offset value is cut out from the traffic pattern by the offset cutout unit 24, and a mask is applied to the cut out data by the scanning window setting unit 25 to set the length of the inspection length. This entry hash value is generated by the hash calculator 21 as an entry having. The pattern matching unit 26 compares the hash value of each packet obtained in this way with the stored matching data, and the matching result discrimination unit 27 determines whether or not a suspicious pattern is included.

  In the traffic scanning apparatus according to the present embodiment shown in FIG. 2, the function units 21 to 29 (that is, the hash calculation unit 21, the entry registration unit 22, the collation data storage unit 23, the offset cutout unit 24, the scanning window setting unit 25, the pattern) The collating unit 26, the collation result discriminating unit 27, the signal input / output unit 28, and the signal control unit 29) or each means may be a circuit, a device, a device, or the like, or realized by software processing. It may be. Furthermore, these functional units or means may be realized by firmware, or may be realized by hardware such as a reconfigurable device, element, substrate, and wiring. When the traffic scanning device is realized by software or firmware, the software or firmware is stored in a storage medium as a program, and is read and executed by the CPU. Therefore, this program is nothing but a program that causes a computer to function as the functional unit or means described above, and is included in the scope of the present invention.

DESCRIPTION OF SYMBOLS 20 Signal line 21 Hash calculating part 22 Entry registration part 23 Collation data storage part 24 Offset extraction part 25 Scan window setting part 26 Pattern collation part 27 Collation result discrimination part 28 Signal input / output part 29 Signal control part 30 RISC processor 31 Register 32 ALU Set 33, 37 Selector 34 Memory 35 External interface 36 Load / Store part

Claims (6)

A traffic scanning device for determining whether data included in a packet transferred on a communication network belongs to a predetermined data set,
Hash operation means for performing a hash operation for generating a hash value that is given a key and is a mapping of the key;
Storage means for holding a result obtained by executing the hash calculation by the hash calculation unit using a predetermined entry pattern as the key as positive matching data;
An extraction means for extracting data of an arbitrary inspection length from an arbitrary position from the traffic pattern as an entry;
The entry extracted by the extraction means is given as the key to the hash calculation means, the hash calculation is executed to obtain a result, and the result is compared with the verification data held in the storage means Inspection means;
A traffic scanning device. The extraction means includes
An offset cutout unit that cuts out a field from the position represented by the offset value from the beginning of the traffic pattern in which an offset value is specified;
A scanning window setting unit that performs mask processing using the mask for the field that is specified by a mask for extracting data corresponding to the inspection length and is cut out by the offset cutout unit;
The traffic scanning device according to claim 1, comprising:   The traffic scanning device according to claim 1, further comprising a processor that implements at least one function of the hash calculation unit, the extraction unit, and the inspection unit as a dedicated instruction for a specific application by hardware. A traffic scanning method for determining whether data included in a packet transferred on a communication network belongs to a predetermined data set,
Performing a hash operation using a predetermined entry pattern as a key and holding the result of the hash operation as positive matching data;
An extraction stage that extracts data of an arbitrary inspection length from an arbitrary position from a traffic pattern as an entry;
Performing the hash operation with the entry as a key and using the result as comparison data;
Comparing and comparing the comparison data and the verification data;
A traffic scanning method. The extraction step includes
Cutting out a field from the position represented by the offset value from the beginning of the traffic pattern with an offset value specified;
A mask for extracting data corresponding to the inspection length is designated, and a mask process using the mask is performed on the cut-out field;
The traffic scanning method according to claim 4, comprising:   The program which makes a computer perform the process of each step in the traffic scanning method of Claim 5 or 6.

Images