KR20090065313A - Hardware based method and system for high performance abnormal traffic detection - Google Patents

Abstract

An abnormal traffic detecting method based on hardware and a detecting device with an abnormal traffic detecting engine based on hardware are provided to help performance of a high performance abnormal traffic detecting function guaranteeing real time detection of line speed in a gigabit network environment, thereby enhancing security of an invasion detecting apparatus. A field for traffic measurement is extracted by extracting an IP(Internet Protocol) packet from inflow traffic(S120). A flow based traffic characteristic of the inflow traffic is measured based on the extracted field(S130). It is detected whether the IP packet is abnormal by comparing the measured characteristic with a threshold value according to each characteristic(S140). The detection result is transmitted to a software base security engine.

Description

Hardware-based abnormal traffic detection method and detection device equipped with hardware-based abnormal traffic detection engine {Hardware Based Method and System for High Performance Abnormal Traffic Detection}

The present invention relates to an intrusion detection technique of a network security system. Network-based intrusion detection technology is largely divided into misuse detection and abnormal detection technology. Misuse detection technology mainly uses pattern matching techniques to detect intrusions by comparing incoming packets with signatures of already found and established attack patterns. As a result, behavioral and statistical techniques are used to detect intrusions when sudden changes or low probability events occur.

The present invention relates to detecting intrusion using an abnormality detection technique, and more particularly, to a hardware-based high performance abnormal traffic detection technology that guarantees a real-time processing capability of network wire speed.

The present invention is derived from the research conducted by the Ministry of Information and Communication and the Institute of Information and Communications Research and Development for IP growth engine technology. [Task Management Number: 2006-S-042-02, Title: Real time attack signature generation and management technology development].

Existing anomaly detection techniques mostly extract flow-based traffic characteristics and measure the corresponding values per unit time, create a profile based on this, and when exceeding a certain threshold or greatly deviating from the profile Detect as abnormal traffic.

However, for real-time detection, most of the software techniques are used instead of the hardware-based technique used in the misuse detection technique. Misuse detection techniques have already been actively studied and excellent solutions such as Parallel Bloom Filter have been developed and commercialized. However, fault detection techniques are mostly software based due to the difficulty of implementing functions with limited hardware resources. . Most existing network intrusion detection systems increase detection rates by adopting complementary misuse detection and abnormality detection. In other words, it performs a real-time intrusion detection function through a hardware-based pattern matching engine and complements to detect dynamic attacks such as Internet worms and DoS / DDoS that could not be detected by pattern matching through a software-based abnormal detection module. have.

However, since the software-based fault detection module adopted in the conventional intrusion detection system has very limited processing performance, real-time processing at wire speed is virtually impossible, and in order to solve this problem, a sampling method, etc. Since traffic is measured using, the accuracy is falling considerably. In addition, the evolution to the increasingly large gigabit network environment is making software-based solutions more difficult to overcome the limitations of processing speed.

In order to perform a real-time intrusion detection function while managing a huge number of flows as the traffic volume increases, hardware-based solution must be developed first. However, traffic measurement and abnormal traffic detection through full inspection of all packets in real time while guaranteeing the speed are almost impossible due to technical difficulties to implement with limited hardware resources.

SUMMARY OF THE INVENTION The present invention has been made in an effort to provide a hardware-based high performance abnormal traffic detection technique and apparatus to which an abnormality detection technique is applied in order to solve the above-described problems of the related art. Measures address dispersion, session success rate, bits per second, packets per second (PPS) through full flow-based traffic inspection in real time while guaranteeing the speed in Gigabit network environment By detecting abnormal traffic, it is possible to detect attacks such as Internet worms and DoS / DDoS, which are difficult to detect by pattern matching techniques.

The present invention relates to a hardware-based abnormal traffic detection method, the process of extracting the IP packet from the incoming traffic to extract the field for traffic measurement, the process of measuring the flow-based traffic characteristics of the incoming traffic based on the extracted field, And detecting the abnormality of the IP packet by comparing the measured characteristic with a threshold for each characteristic and delivering the abnormality to the software-based security engine.

The present invention also relates to a hardware-based abnormal traffic detection engine, comprising: an IP packet extractor extracting an IP packet from an inflow traffic, a table manager for generating and managing a traffic characteristic table based on the flow of the extracted IP packet; And extracting a field for measuring traffic from the IP packet, calling the traffic characteristic table with different field values as input values according to the flow unit of the IP packet to be measured, and comparing the called value with a threshold. An abnormal traffic detector for detecting whether an IP packet is abnormal traffic.

According to the configuration of the present invention, it is possible to design and implement a software-based abnormality detection technique that is generally applied in the conventional intrusion detection apparatus based on hardware to significantly increase the processing speed. Therefore, it is possible to further enhance the security of the intrusion detection device by helping to perform a high performance abnormal traffic detection function that guarantees the real-time detection of the line speed in the Gigabit network environment that is becoming more common.

Specific details of other embodiments are included in the detailed description and the drawings.

Advantages and features of the present invention and methods for achieving them will be apparent with reference to the embodiments described below in detail with the accompanying drawings. However, the present invention is not limited to the embodiments disclosed below, but can be implemented in various different forms, only the embodiments are to make the disclosure of the present invention complete, the general knowledge in the art to which the present invention belongs It is provided to fully inform the person having the scope of the invention, which is defined only by the scope of the claims. Like reference numerals refer to like elements throughout.

Hereinafter, a hardware-based high performance abnormal traffic detection technique and apparatus will be described in more detail with reference to the accompanying drawings.

1 is a structural diagram of a system to which an abnormal traffic detection engine of the present invention is applied. When a network packet is introduced from the PHY 100, which is an input / output interface of the packet, the hardware-based security engine 110 detects an intrusion and transmits the result to the software-based security engine 150 through the PCI 140. Most existing commercial products perform only functions by placing only the pattern matching engine 120 in the hardware-based security engine, and the abnormal detection engine 130 is located in the software-based security engine and has a limitation in processing speed. However, in the present invention, in order to solve this problem, the abnormal detection engine 130 is also located on the hardware-based security engine to ensure the line speed while performing the abnormal traffic detection function in real time.

The software-based security engine 150 transmits the data input through the PCI 140 to the S / W manager 170, the S / W manager 170, the threshold value of the appropriate policy and measurement traffic characteristics according to the network environment Set the appropriately.

2 is an internal block diagram for each function of a detection apparatus equipped with a hardware-based abnormal traffic detection engine according to the present invention. The network packet is introduced through the PHY 200, which is a system input / output interface, and transmitted to the next stage. The PL3 interface 210, which is an IP packet extracting unit, recognizes a 2-layer Ethernet frame received through PL3 (Packet of SONET Level 3), which is a physical layer interface standard, extracts an IP packet, and delivers the IP packet to a higher inner block. At this time, in order to distinguish each packet, a packet ID is generated for each packet and delivered together. This is very useful when you need a per packet response. The delivered IP packet is basically parsed through the de-fragmentation process through the IP packet merging unit 231 of the abnormal traffic detector 230. The packet parser 232 extracts fields necessary for flow-based traffic measurement through this process. Basically, 5-Tuple (Source IP, Destination IP, Source Port, Destination Port, Protocol) information and TCP flag information necessary for flow management are extracted. Based on the extracted information, flow-based traffic measurement is performed through the flow-based traffic measurement unit 233, and when the detection result of the abnormality detection unit 234 exceeds a specific threshold per unit time, it is detected as abnormal traffic.

The traffic characteristics measured here are address dispersion, session success rate, SYN Count, Stealth Scan Count, BPS, PPS, etc. and are processed in real time and used to determine abnormality. The detected result is transmitted to the S / W manager 270 via the PCI 260 through the report interface 250. The S / W manager 270 uses the configuration interface 280 to appropriately set threshold values of policy and measurement traffic characteristics according to the network environment. In order to measure traffic based on the flow, the table manager 240 stores and manages the corresponding traffic characteristic values in three tables, 4-Tuple table, session table, and flow table.

Here, all modules except the S / W manager may be implemented as hardware such as an FPGA or an ASIC and a memory device such as an SRAM.

3 shows an example of abnormal traffic to be detected in the present invention. Internet worm 300 can be classified into DoS / DDoS 310. Internet worms generally attempt to scan from one source IP to multiple destination IPs in order to find the propagation target, and do many DoS / DDoS attacks. A large amount of traffic is sent from a source IP of one to a destination IP. Due to the above characteristics of traffic, the unit of flow managed to detect Internet worm is composed of <Source IP, Destination Port, Protocol>, and the unit of flow managed to detect DoS / DDoS is <Destination IP, Destination Port, Protocol Consists of> Once an attack is attempted, whether it is an Internet worm or DoS / DDoS, the values of address distribution, session success rate, SYN Count, Stealth Scan Count, BPS, PPS, etc., measured in flow units will be markedly different from normal traffic. Traffic detector 230 detects this.

Traffic characteristic Explanation Address distribution In case of Internet Worm: The number of other Hosts (Destination IP) in which a flow in which <Flow Source IP, Destination Port, Protocol> is sent during a specific time period. The number of other hosts (source IP) that received a packet for a certain time by a flow Session success rate TCP Session Connection Count / Number of SYN Packets Except Duplicated SYN Packets SYN Count Number of TCP SYN Packets Stealth scan count Number of Stealth Scan Attempts Other Than SYN Scanning BPS Bits per second PPS Packets per Second

Table 1 is a description of traffic characteristic values measured in units of flows. Hereinafter, an Internet worm among abnormal traffic will be described as an example to help understand the respective traffic characteristic values. In order to detect Internet worms, address distribution is measured based on the flow in the unit of <Source IP, Destination Port, Protocol>. In general, Internet worms go through a scanning step before they spread the attack code. In the scanning process of worms, address distribution is much higher than normal traffic. In order to detect such abnormal traffic, address distribution is measured. Table manager 240 is a 4-Tuple table based on <Source IP, Destination IP, Destination Port, Protocol> other than <Source IP, Destination Port, Protocol> based flow table 243 to measure address distribution. ).

When a packet is input, address distribution is measured by searching the flow table and 4-Tuple table and comparing them with each other. Address distribution will increase. In the example of the Internet worm of Fig. 3, the address distribution is six. If the address distribution exceeds the specified threshold during the unit measurement time, the result is reported to the upper module considering other characteristic values.

In order to detect the Internet worm, not only the address distribution but also the session secrecy rate is measured based on the flow of <Source IP, Destination Port, Protocol>. As mentioned earlier, in the case of worms, session success rate is significantly lower than normal traffic during scanning. The session table 242 is needed to measure the session success rate. Since TCP Session is established through 3-Way Handshaking process, session success rate is measured by comparing the connection establishment count through session tracking and the number of SYN packets except Duplicated SYN packet. If the session success rate exceeds the specified threshold during the unit measurement time, the result is reported to the upper module considering the address dispersion value.

In addition, SYN Count, BBP, and PPS will not be described, and the Stealth Scan Count will be described. Consider the case of the Internet worm mentioned earlier. If the scanning process is SYN scanning using SYN packets, the worm, which is abnormal traffic, can be detected by measuring SYN Count and session success rate. However, it cannot be detected by scanning using FIN packet or ACK packet other than SYN scanning. Therefore, in this case, abnormal traffic is detected by measuring the Stealth Scan Count. Through session tracking, only the legitimate connection through 3-Way Handshaking process exists in the session table. If there is no entry in the session table search, the Stealth Scan Count of the flow is increased.

Figure 4 shows a common table structure applied to the flow table, 4-Tuple table, session table. It includes multiple hash functions and N-Way Set Associative table structure for efficiently managing table entries with limited hardware resources.

First, the search key is a basic flow unit for each table. The flow table becomes <Source IP, Destination Port, Protocol> for Internet worms, and <Destination IP, Destination Port, Protocol> for DoS / DDoS. The 4-Tuple table becomes <Source IP, Destination IP, Destination Port, Protocol>, and the session table becomes <Source IP, Source Port, Destination IP, Destination Port, Protocol>. The hash key generator 410 has a double hash structure. The search key information is received as an input, and the corresponding table entry 440 is indexed and managed using the Hash1 (x) 420 and Hash2 (x) 430 functions. The Hash1 (x) 420 and Hash2 (x) 430 functions used herein may be any suitable functions such as an XOR function or a CRC function. The first hash function, Hash1 (x), is used to create an index that points to each Hash Set that allows Hash Collision for faster table searching. The second hash function, Hash2 (x), is used to generate a Hash Address that is used to distinguish each table entry in the Hash Set pointed to by Hash1 (x). Each hash set of the table can contain N entries, which is an N-Way Set Associative table structure. Here, each table is generally designed and implemented on SRAM to improve the processing speed.

5 shows each table entry field. Here, the Hash Address is an address generated by the Hash2 (x) 430 function to distinguish table entries in the Hash Set, and the Time Stamp is a field for storing time inserted or modified for entry management.

The flow table 500 includes bps, pps, address distribution, SYN Count, Session Success Count, and Stealth Scan Count, which are fields for storing traffic characteristic values to be measured. Since the 4-Tuple table 510 is for measuring address dispersion, only the Time Stamp and Hash Address fields are required. The session table 520 includes a current state, a time stamp, and a hash address. The Current State contains connection state information of the current session, and the Time Stamp is used to determine which session entry is deleted when the session table becomes full by applying a Least Recently Used (LRU) algorithm.

6 is a flowchart illustrating the flow of a hardware-based abnormal traffic detection method according to the present invention.

After extracting the IP packet from the inflow traffic and transmitting it to the abnormal traffic detector 230 along with the packet ID (S100), the packet S110 that has undergone the de-fragmentation process through the IP packet merger 231 passes through a packet parsing process. A traffic measurement field for measuring flow-based traffic is extracted (S120).

The flow-based traffic characteristic measurement unit 233 measures traffic characteristics such as address distribution or session success rate based on the flow (S130), and the abnormality detector 234 detects whether the traffic is abnormal using the measured traffic characteristics. (S140).

7 is a flowchart illustrating a method of increasing an accuracy of misuse detection and abnormality detection by transmitting an abnormally measured packet to a pattern matching engine by a hardware-based abnormal traffic detection method according to the present invention.

As shown, the packet is detected as abnormal traffic on the hardware based on the pattern matching engine to detect whether the signature pattern matching (S200), and transmits the result to the software-based security engine (S210). In this way, not only hardware-based misuse detection and abnormality detection are possible, but also hardware and software-based double traffic detection technology can increase the accuracy of intrusion detection.

The invention can also be embodied as computer readable code on a computer readable recording medium. The computer-readable recording medium includes all kinds of recording devices in which data that can be read by a computer system is stored. Examples of computer-readable recording media include ROM, RAM, CD_ROM, magnetic tape floppy disks, optical data suits, and the like, and may also be implemented in the form of carrier waves (for example, transmission over the Internet). The computer readable recording medium can also be distributed over network coupled computer systems so that the computer readable code is stored and executed in a distributed fashion.

Although embodiments of the present invention have been described above with reference to the accompanying drawings, those skilled in the art to which the present invention pertains may implement the present invention in other specific forms without changing the technical spirit or essential features thereof. I can understand that. Therefore, it should be understood that the embodiments described above are exemplary in all respects and not restrictive.

1 is a general structural diagram of a system equipped with a detection apparatus equipped with a hardware-based abnormal traffic detection engine according to the present invention;

2 is a diagram illustrating a structure of a hardware-based abnormal traffic detection engine according to the present invention;

3 is a diagram illustrating an example of abnormal traffic;

4 is a diagram illustrating a structure of a memory table used for traffic characteristic measurement and abnormal detection;

6 and 7 are flowcharts illustrating the flow of a hardware-based abnormal traffic detection method according to the present invention.

Claims (14)

The hardware-based abnormal traffic detection method, Extracting an IP packet from an inflow traffic and extracting a field for measuring a traffic; Measuring a flow-based traffic characteristic of the inflow traffic based on the extracted field; And And detecting the abnormality of the IP packet by comparing the measured characteristic with a threshold of each characteristic and transmitting the detected characteristic to the software-based security engine. The method of claim 1, The field extraction process includes merging fragmented IP packets to be suitable for transmission; And extracting 5-Tuple information and TCP flag information necessary for flow-based traffic measurement from the merged IP packet. The method of claim 1, The traffic characteristic measurement process is a hardware-based abnormal traffic detection method characterized in that for measuring the traffic characteristics including at least one of the address distribution value, session connection success rate, SYN Count, Stealth Scan Count, BPS, PPS. The method of claim 1, The hardware-based abnormal traffic detection method includes a 4-Tuple table storing 4-Tuple information on incoming traffic, a session table storing session information, and information on the traffic characteristic values to measure the traffic characteristics. Hardware-based abnormal traffic detection method further comprising the step of generating a flow table. The method of claim 4, wherein The traffic characteristic measurement process is a hardware-based abnormal traffic detection method, characterized in that for generating the table using at least two hash functions having different field values as input values according to the flow unit of the IP packet to be measured. The method of claim 5, wherein 4-Tuple information, session information, and traffic characteristic values of the IP packet include a first hash value for the input value as a row address value, and a second hash value for the input value as a column address value. Hardware-based abnormal traffic detection method characterized in that the stored in the hash table. The method of claim 1, The hardware-based abnormal traffic detection method further comprises performing a misuse detection through signature pattern matching in case of an IP packet whose traffic characteristics are abnormally detected by comparing with the threshold value. . The method of claim 1, The hardware-based abnormal traffic detection method further comprises adjusting a threshold to be compared with the traffic characteristics according to the network environment of the hardware in which the abnormal traffic detection algorithm is detected. Hardware-based abnormal traffic detection engine, An IP packet extracting unit extracting an IP packet from an inflow traffic; A table manager for generating and managing a traffic characteristic table based on the flow of the extracted IP packet; And Extracting a field for traffic measurement from the IP packet, calling the traffic characteristic table with a different field value as an input value according to the flow unit of the IP packet to be measured, and comparing the called value with a threshold to the IP A detection device with a hardware-based abnormal traffic detection engine including an abnormal traffic detector for detecting whether a packet is abnormal traffic. The method of claim 9, The abnormal traffic detector may include: an IP packet merging unit for merging when an IP packet fragmented to be suitable for transmission is extracted from the IP packet extracting unit; A packet parser configured to parse IP packets merged by the IP packet merger and extract a field suitable for measuring a traffic characteristic; A flow-based traffic measuring unit for calling the flow-based traffic characteristics stored in the table manager based on the field values extracted by the packet parsing unit to measure the traffic characteristics of the IP packet; And an abnormality detection unit configured to detect an abnormality by comparing the called traffic characteristic with a threshold value. The method of claim 9, The traffic manager generates and manages a 4-Tuple table in which 4-Tuple information on inflow traffic is stored, a session table in which session information is stored, and a flow table in which information on the traffic characteristic values to be measured is stored. Beacon with hardware-based anomaly traffic detection engine. The method of claim 11, The traffic manager generates the table using at least two hash functions having different field values as input values according to the flow unit of the IP packet to measure traffic. Detector. The method of claim 12, 4-Tuple information, session information, and traffic characteristic values of the IP packet include a first hash value for the input value as a row address value, and a second hash value for the input value as a column address value. Detection device equipped with a hardware-based abnormal traffic detection engine, characterized in that stored in the hash table. The method of claim 9, The detection apparatus equipped with the hardware-based abnormal traffic detection engine may further include a pattern matching engine configured to perform signature pattern matching on the IP packet detected abnormally by the abnormal traffic detector. Onboard detector.

Images