KR101498696B1 - System and method for detecting harmful traffic - Google Patents

Abstract

The present invention is a harmful traffic detection system that captures a packet transmitted through a monitoring target network, performs pattern matching inspection of an arbitrary position of the packet, adds a hash key, and outputs the packet- And a software-based packet collecting device for performing a pattern matching check of a specific location in a packet for each flow outputted by the hardware-based packet collecting device and outputting it to a network flow (Netflow).

Description

TECHNICAL FIELD [0001] The present invention relates to a system and a method for detecting harmful traffic,

The present invention relates to a system and method for detecting harmful traffic, and more particularly, to a system and method for detecting harmful traffic, more specifically, a method of preliminarily matching suspicious traffic patterns using a network card, pattern- To detect traffic.

Today, with the advancement of network technology and the increasing number of users, the information society is evolving, while the negative side is increasing due to harmful traffic that spreads or attacks viruses to other users through the network.

Harmful traffic refers to aggressive traffic that the system will operate abnormally when the corresponding traffic is received at the receiver side, or unnecessary traffic or data flow or normal packet in attack information or normal network flow, but the packet is excessively generated Traffic that interrupts the normal network communication flow of the receiver or interrupts the network connection. When the harmful traffic flows into the user's computer through the network, problems such as deterioration of the computer performance occur. Therefore, attempts have been made to block harmful traffic.

Generally, only the signature detection engine or software is used to detect harmful traffic. If the number of signatures increases when all the packets passing through the network are matched with the signatures, the time required for detection of harmful traffic increases, the detection performance decreases, and the performance of the entire system deteriorates due to an increase in the CPU usage rate .

The present invention solves the problem of using only software to detect harmful traffic on a network. Even if the number of signatures increases, the signature matching time is not delayed, and the performance of the entire system is kept constant without deteriorating And to provide a method and system that can enhance the performance of harmful traffic detection while ensuring stable operation of the system.

The present invention is a harmful traffic detection system that captures a packet transmitted through a monitoring target network, performs pattern matching inspection of an arbitrary position of the packet, adds a hash key, and outputs the packet- And a software-based packet collecting device for performing a pattern matching check of a specific location in a packet for each flow outputted by the hardware-based packet collecting device and outputting it to a network flow (Netflow).

The present invention is a harmful traffic detection method comprising the steps of capturing a packet transmitted through a network to be monitored, pattern matching inspection of an arbitrary position of the packet based on hardware, adding a hash key, And performing a pattern matching check of a specific location in the packet for each flow based on software and outputting it to a network flow (Netflow).

Since hardware and software are interlocked to detect harmful traffic, even if the number of signatures increases, the signature matching time is not delayed, and the performance of the entire system is maintained without being deteriorated, Detection performance can be improved.

1 is a block diagram of a harmful traffic detection system according to an embodiment of the present invention.
2 is a configuration diagram of a hardware-based packet collecting apparatus according to an embodiment of the present invention.
3 is a configuration diagram of a packet analyzing unit according to an embodiment of the present invention.
4 is a diagram for explaining a ruleset matching operation using multiple registers implemented in a ruleset matching unit according to an embodiment of the present invention.
5 is a diagram illustrating a multi-thread implementation through multiple memories according to an embodiment of the present invention.
6 is a configuration diagram of a software-based packet collecting apparatus according to an embodiment of the present invention.
FIG. 7 is a flow chart for explaining a harmful traffic detection method according to an embodiment of the present invention.

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS Reference will now be made in detail to embodiments of the present invention, examples of which are illustrated in the accompanying drawings, wherein like reference numerals refer to the like elements throughout.

In the following description of the present invention, a detailed description of known functions and configurations incorporated herein will be omitted when it may make the subject matter of the present invention rather unclear.

The terms used throughout the specification are defined in consideration of the functions in the embodiments of the present invention and can be sufficiently modified according to the intentions and customs of the user or the operator. It should be based on the contents of.

1 is a block diagram of a harmful traffic detection system according to an embodiment of the present invention.

As shown in FIG. 1, the harmful traffic detection system captures traffic generated in a monitoring target network, such as the backbone network 10 and the subscriber network 20, and detects whether the traffic is harmful. According to the embodiment of the present invention, the hardware-based packet collecting apparatus 100 and the software-based packet collecting apparatus 200 interwork with each other for 10G real-time packet collection / analysis and high- Traffic harmful detection can be performed.

The hardware-based packet capture device 100 is fabricated in the form of a network card and is plugged into the software-based packet capture device 200.

The hardware-based packet capturing apparatus 100 captures a packet transmitted through a network to be monitored, performs a pattern matching check to determine whether the captured packet is harmful, and performs a hash key generation processing for flow identification.

According to an embodiment of the present invention, the hardware-based packet collecting apparatus 100 performs pattern matching with no offset. Pattern matching without an offset is to detect a rule set existing at an arbitrary position of a packet. Since it takes much time to detect by a software, it is possible to perform hardware-based processing . Also, multi-flow management is performed for high-speed processing.

The software-based packet collecting apparatus 200 processes the data processed by the hardware-based packet collecting apparatus 100 by multi-thread processing to check whether the traffic is abnormal, and outputs the checked result to the network flow (Netflow). In accordance with an embodiment of the present invention, the software-based packet capture device 200 performs pattern matching with offsets. Here, pattern matching with an offset is to detect a rule set existing at a specific position of a packet, which can be easily detected in software, so that it is processed in a software basis for real-time high-speed detection.

The software-based packet collecting apparatus 200 provides a network flow (Netflow) to the server 40 and the server 40 stores it in the DB 50 or outputs it in GUI form via the user computer 60 Allow users to analyze directly. Then, the server 40 may feed back the harmful detection policy according to the analysis result to the software-based packet collecting apparatus 200.

2 is a configuration diagram of a hardware-based packet collecting apparatus according to an embodiment of the present invention.

Referring to FIG. 2, the hardware-based packet collecting apparatus 100 may be fabricated in the form of a network card and connected to the software-based packet collecting apparatus 200 in a plug-in manner as described above.

The hardware-based packet collecting apparatus 100 includes an optical module 110, an input interface unit 120, an EMAC 130, a packet analyzing unit 140, a multiple memory unit 150, and a computer interface unit (PCIe IP) 160 ). The input interface unit 120, the EMAC 130, the packet analysis unit 140, the multiple memory unit 150, and the output interface unit 160 may be implemented as the FPGA 170, .

The optical module 110 collects packets transmitted through the network to be monitored. According to an embodiment of the present invention, it may be an optical module that collects packets from an optical fiber through two ports. According to the present invention, the optical module 110 divides a packet input through a 10G optical fiber 30 into four 3.125G packets, and divides the divided 3.125G packets into four pipelines And outputs it to the FPGA 170. This makes it possible to process 10G or more packets in real time.

The input interface unit 120 sequentially stores the packets divided by the optical module 110, converts the packet into a form that can be input to the EMAC, and outputs the packet. According to one embodiment of the present invention, data of 3.125 G is converted into 64 bits and output.

The EMAC 130 implements an interface between the core processor and the external Ethernet network by providing one complete on-chip subsystem and interfaces between the interface unit 120 and the packet analysis unit 140.

The packet analyzing unit 140 performs pattern matching of input packets to primarily discriminate whether or not a harmful item is present, analyzes the packet, and outputs the separated flow type information. This will be described in detail with reference to FIG.

The multiple memory 150 stores the packets output from the packet analyzing unit 140 for each flow type for the multi-flow process of the present invention and transmits the packets to the software-based packet collecting apparatus 200 in cooperation with the hardware- Which will be described later in detail with reference to FIG. According to one embodiment of the present invention, a DMA (Direct Memory Access) memory can be used, which allows an IO device controller to move data without executing a program by the CPU. As a result, the speed of the input / output can be improved and the speed difference between the CPU and the peripheral device can be reduced.

The connection processing unit 160 is a PCIe IP core and is an essential IP technology used when developing a PCIe compatible device or a peripheral device in a computer peripheral such as a PC and a server.

3 is a configuration diagram of a packet analyzing unit according to an embodiment of the present invention.

3, the packet analyzing unit 140 includes a network module 141, a pattern matching unit 142, a hash key generating unit 143, a packet generating unit 144, and a control register 145 .

The network module 141 converts the 64-bit packet transmitted from the EMAC 130 into a frame link format and transmits the packet to the pattern matching unit 142. For example, the frame link format (frame link format) converted according to the netcope platform is shown in Table 1 below.

FrameLink Header FrameLink Payload Field Dst MAC Src MAC Type /
Length Data FCS Bytes 16 2 6 2 46-1500 4

The pattern matching unit 142 performs a pattern matching check on a packet converted into a frame link format. In accordance with an embodiment of the present invention, a pattern matching unit 142 performs a pattern matching check on a rule set existing in an arbitrary position in a packet. . The pattern matching unit 142 includes a rule set storage unit 142a and a rule set matching unit 142b in detail.

Several rule sets for detecting harmful traffic are stored in the rule set storage unit 142a. As described above, a ruleset existing at an arbitrary position is stored, and such rule sets can be downloaded from software.

The rule set matching unit 142b matches the transmitted packet with the ruleset pattern stored in the ruleset storage unit 142a to determine whether they match or not. The rule set matching unit 142b takes charge of one rule set and outputs the ruleset matching flag as a one-bit matching flag on a frame basis. That is, the input frame data is shifted by one byte, and it is determined whether there is a portion coinciding with the rule set. In accordance with one embodiment of the present invention, the length of the ruleset may be 1 byte to 16 bytes and may be preset by the software-based packet capture device 200. To this end, the rule set matching unit 142b may be implemented with multiple registers for 1 byte to 16 bytes.

4 is a diagram for explaining a ruleset matching operation using multiple registers implemented in a ruleset matching unit according to an embodiment of the present invention.

Referring to FIG. 4, each rule set register includes a ruleset storage register 410, a case number storage register 420, and a temporary storage register 430. Here, a case of matching a 3-byte rule set within 8 bytes of a payload stream input by using a register will be described as an example.

The payload is inputted continuously to form a stream, and the rule set matching unit 142b inputs 8 bytes to the temporary storage register 430 in one clock. 8 bytes of data stored in the temporary storage register 430 are separated by 3 bytes in the next clock and stored in the number register 420 of eight cases. Then, the value stored in the 8-case number registers 420 is compared with the value stored in the 3-byte rule set register 410, and if there is at least one identical value, a match flag is driven.

The hash key generation unit 143 analyzes the packet in which the match flag is driven by pattern matching, distinguishes the flow, and generates a hash key corresponding to the distinction.

Here, the term flow refers to a TCP or UDP packet flow that is generated according to the use of a specific communication application. Each packet flow can be distinguished by a 5-tuple including Source IP, Destination IP, Protocol of L3 (IP Packet), and Source Port and Destination Port of L4 (TCP Packet). However, this is only an embodiment, and the flow distinguishing method is not limited thereto.

However, when pattern matching and a hash key are generated as a packet header and transmitted, overhead is generated when the header value is increased, so that some loss of the packet may occur. Therefore, a hash key it is preferable to reduce the hash key.

The packet generating unit 144 generates a header as a result value output from the pattern matching unit 142 and the hash key generating unit 143 and attaches the resultant value to a payload to be transmitted to the multiple memory 150 send. That is, it receives the data transmitted in the frame link format, removes the source MAC and the destination MAC, and adds a match flag and a hash key.

An example of a packet format regenerated by the packet generator 144 is shown in Table 2 below.

FrameLink Header FrameLink Payload Field Type /
Length Data FCS Match
Flag Hash
Key Bytes 16 2 46-1500 4 4 4

Comparing Table 2 with the frame link format in Table 1, it can be seen that the source MAC and the destination MAC are removed and a match flag and a hash key are added.

The control register 145 stores the value transmitted from the software-based packet collecting apparatus 200 as control information for rule set information and ruleset matching, hash key generation control information, and packet control information, and transmits the control information, hash key generation control information and packet control information to the pattern matching unit 142, Key generation unit 143 and the packet generation unit 144, as shown in FIG.

5 is a diagram illustrating a multi-thread implementation through multiple memories according to an embodiment of the present invention.

Referring to FIG. 5, the multiple memory stores hardware-processed packets separately for each hash key, and outputs the processed data to be processed by the multithreaded software. According to an embodiment of the present invention, the multiple memories can be composed of eight channels.

6 is a configuration diagram of a software-based packet collecting apparatus according to an embodiment of the present invention.

6, the software-based packet collecting apparatus 200 includes a packet collecting unit 210, an abnormal traffic detecting unit 220, a network flow generating unit 230, a system managing unit 240, an environment setting unit 250, And a flow output unit 260.

The packet collecting unit 210 collects packets stored in the multiple memory 150 for each flow through a multi-thread process.

The abnormal traffic detection unit 220 performs pattern matching of the packets collected by the packet collection unit 210 to determine whether the packets are harmful or not. At this time, pattern matching is performed with reference to the rule addition / deletion / change information set by the environment setting unit 250. The matching of the ruleset pattern existing at a specific position of the packet is performed for each flow, so that the speed can be improved. The pattern-matched result by the abnormal traffic detector 220 is stored in a hash table (not shown) for each flow.

The network flow generating unit 230 converts the flows into network flow capable (Netflow). At this time, the Netflow template input from the environment setting unit 250 is referenced and converted.

The flow output unit 260 bundles network flows generated by the network flow generation unit 230 and transmits them to the outside.

The system management unit 240 periodically transmits the system management information to the analyst, receives the feedback information corresponding thereto, and performs an analyzer interface for managing the entire system. Here, the system management information may include config information including CPU usage, memory usage, disk usage, rule set information, hardware based or rule set matching location information to be processed based on software, or log information according to abnormal traffic detection .

The environment setting unit 250 performs addition / deletion / modification of a rule set to an abnormal traffic detection unit that processes pattern matching according to the information fed back by the system management unit 240, You can download the information.

FIG. 7 is a flow chart for explaining a harmful traffic detection method according to an embodiment of the present invention.

Referring to FIG. 7, the harmful traffic detection method can perform a traffic harm detection by interlocking a hardware-based process and a software-based process for 10G real-time packet collection / analysis and high-speed traffic hazard detection process extracted through a monitoring target network .

The hardware-based packet capturing apparatus 100 captures packets transmitted through the network to be monitored (S710). According to one embodiment of the present invention, packets can be collected from an optical fiber through two ports. According to the present invention, a packet input through a 10G optical fiber 30 can be divided into four 3.125G packets, and a divided 3.125G packet can be output through four pipelines. This makes it possible to process 10G or more packets in real time.

The hardware-based packet collecting apparatus 100 performs pattern matching of an arbitrary position of the captured packet (S720). The input frame data is shifted by one byte, and it is determined whether there is a portion coinciding with the rule set. If there is a matching portion, a match flag is driven. In accordance with one embodiment of the present invention, the length of the ruleset may be 1 byte to 16 bytes and may be preset by the software-based packet capture device 200. Further, multiple registers can be used for pattern matching at arbitrary positions.

The hardware-based packet collecting apparatus 100 analyzes the packet in which the match flag is driven by pattern matching, distinguishes the flow, and generates a hash key corresponding to the distinction (S730).

The hardware-based packet collecting apparatus 100 stores the packet to which the match flag and the hash key are added for each flow (S740). This is for storing the packets for each flow type for the multi-flow processing and interworking with the hardware-based packet collecting apparatus 100 to allow the software-based packet collecting apparatus 200 to perform multi-thread processing.

The software-based packet collecting apparatus 200 collects the packets stored for each flow through the multithread processing (S750).

The software-based packet collecting apparatus 200 performs pattern matching of a specific position of the collected packet to determine whether the packet is harmful (S760). That is, the matching of the ruleset pattern existing at a specific position of the packet is performed for each flow, so that the speed can be improved. The pattern-matched result is stored in a hash table (not shown) for each flow (S770).

The software-based packet collecting apparatus 200 converts the flows into a network transferable network flow (Netflow) (S780). At this time, the input Netflow template is referenced and converted. The software-based packet collecting apparatus 200 bundles network flows and transmits them to the outside (S790).

Claims (19)

A hardware-based packet collecting device for capturing a packet transmitted through a monitoring target network, performing a pattern matching check on an arbitrary position of the packet, and outputting only matched packets;
Based packet collecting device for performing a pattern matching check of a specific location in a packet output by the hardware-based packet collecting device and outputting it to a network flow (Netflow)
The hardware-based packet collecting device
And a packet analyzer for analyzing the packet to determine whether the packet is malicious or not, and analyzing the packet to determine whether the packet is harmful,
The packet analyzer
A pattern matching unit for detecting a rule set existing at an arbitrary position in the packet,
A hash key generation unit for analyzing a packet in which a match flag is driven by pattern matching to discriminate a flow and generating a hash key corresponding to the distinction;
And a packet generating unit for generating a header as a result value output from the pattern matching unit and the hash key generating unit, and attaching the resultant value to a payload.
The apparatus of claim 1, wherein the hardware-based packet collection device
Wherein the computer-readable medium is fabricated in the form of a computer card and is plugged into a software-based packet collection device in a plug-in form.
The apparatus of claim 1, wherein the hardware-based packet collection device
And an optical module for capturing a packet transmitted through a monitoring target network,
Further comprising a plurality of memories for storing a plurality of packets output from the packet analyzing unit for each flow type and for interworking with a hardware-based packet collecting apparatus and for performing a multithread processing in a software-based packet collecting apparatus.
4. The apparatus of claim 3, wherein the packet analyzer and the multiple memory
And the FPGA is implemented as a harmful traffic detection system.
The optical module according to claim 3,
Wherein the packet input through the 10G optical fiber is divided into four 3.125G packets and the divided 3.125G packets are output through four pipelines.
delete 4. The apparatus of claim 3, wherein the packet analyzer
Further comprising a network module for converting a packet captured by the optical module into a frame link format and outputting the packet to the pattern matching unit.
The apparatus of claim 3, wherein the pattern matching unit
A rule set storage unit for storing a number of rule sets for detecting harmful traffic,
And a rule set matching unit for shifting input frame data of the transmitted packet by one byte to check whether there is a portion coinciding with the rule set stored in the rule set storage unit and outputting the check result as a match flag Features harmful traffic detection system.
The method of claim 8, wherein the length of the ruleset is
1 byte to 16 bytes.
9. The apparatus of claim 8, wherein the ruleset matching unit
A rule set register for storing the ruleset,
A temporary storage register for storing data of a predetermined length to be inspected among packets to be inputted;
And a number register for storing data of a predetermined length stored in the temporary storage register into a rule set length,
And compares the values stored in the number registers in the case with the values stored in the rule set register to drive a match flag if there is at least one of the same values.
11. The method of claim 10, wherein the number of the number registers in the case is
And the number of bytes of data stored in the temporary storage register.
The method of claim 3, wherein the hash key generation unit
Each packet flow generates a hash key according to 5 tuple including Source IP of the L3 (IP Packet), Destination IP, Protocol, and Source Port and Destination Port of L4 (TCP Packet) Traffic detection system.
2. The apparatus of claim 1, wherein the software-based packet collection device
A packet collecting unit for collecting packets stored in the hardware-based packet collecting apparatus on a flow-by-multithread basis;
And an unauthorized traffic detection unit for performing ruleset matching at a specific location of the packet collected by the packet collection unit to determine whether the packet is harmful.
14. The apparatus of claim 13, wherein the software-based packet collection device
Further comprising a configuration unit for performing addition / deletion / modification of a rule set to the abnormal traffic detection unit and for downloading information related to ruleset matching to the hardware-based packet collection apparatus for hardware-based pattern matching, system.
Capturing a packet transmitted through a monitoring target network;
Performing a pattern matching check on an arbitrary location of the packet based on hardware;
Performing a pattern matching check of a specific location in a packet based only on the hardware-based pattern-matched packet,
The step of performing the pattern matching check of the arbitrary position
Performing Rule Set matching at an arbitrary position in a packet converted into a frame link format;
Generating a hash key for generating a hash key corresponding to the distinction by analyzing a packet in which the match flag is driven by the ruleset matching to distinguish the flow; Harmful traffic detection method.
delete 17. The method of claim 15, wherein performing the pattern matching check comprises:
Further comprising the step of converting the packet into a frame link format before rule set matching.
16. The method of claim 15, wherein performing ruleset matching comprises:
The method comprising: shifting frame data of the converted packet one byte at a time, checking whether there is a portion coinciding with a rule set stored in advance, and driving a match flag of the check result.
16. The method of claim 15, wherein performing pattern matching of the specific location comprises:
Collecting packets stored on a hardware-by-flow basis through multithread processing;
And performing ruleset matching at a specific location of the collected packet to determine whether the packet is harmful.

Images