KR101141183B1 - System and method for distributed processing and fast filtering of security log and event using gpu - Google Patents

Abstract

PURPOSE: A distributed processing/high speed filtering system and a method thereof are provided to perform high speed filtering and analysis about a security log/event based on low cost hardware by supporting a task of a CPU. CONSTITUTION: An information collecting unit(10) collects security log and event information. A memory unit(30) uploads the security log/event information. A processing unit(40) compares the security log/event information of the memory unit with a classifying policy of classification policy database. The processing unit includes a CPU(41) and a GPU(42) and compares the security log/event information with a classification policy. The GPU arranges a grouped classification policy group to an internal SP(streaming processor).

Description

SYSTEM AND METHOD FOR DISTRIBUTED PROCESSING AND FAST FILTERING OF SECURITY LOG AND EVENT USING GPU

The present invention relates to a system and method for distributed processing and high-speed filtering of security logs and events using a GPU. More particularly, the present invention relates to low-cost hardware by performing real-time filtering of security logs and events using a GPU to support CPU operations. The present invention relates to a distributed processing and fast filtering system and method for security logs and events using a GPU that enables high-speed filtering and analysis of a large-capacity security log and events.

As society develops and competition among companies intensifies, awareness of the importance of security for internal confidential data is also increasing rapidly. Therefore, the Enterprise Security Management (ESM), which manages security on and offline The importance and size are also increasing, and accordingly, the amount of data transmission on the integrated security management system is also rapidly increasing.

The existing integrated security management system collects security logs or events and inserts the formalized security logs and events into a relational DB to index and detect events that violate the complex security policy. Done.

In addition, the Security Information and Event Management System (SIEM), similar to this Integrated Security Management System, can enter security logs and events into a file DB and immediately retrieve information based on them. In order to query the data, the data is being moved back to the relational DB.

Therefore, in order to perform correlation analysis and filtering of complex conditions, there is a delay in performing a task by the time required for input, indexing, or selecting (lookup) into the DB.

In the structure using the existing relational DB, the processing time increases rapidly when the number of data is large or the size of data is large, and the violation of security policy that requires the collection of large-scale data (log / event, etc.) and real-time inquiry and analysis ( This structure is not suitable for DDoS attack.

Therefore, it is necessary to minimize the structural problems of the existing system for real-time analysis of large data (security log / event), and in particular, to increase the amount of data without increasing the cost of expensive system updates each time. There is a need for the construction of a highly efficient system that can be processed.

The present invention has been made to solve the above problems, the object of which is to perform a real-time filtering of security logs and events using a GPU to support the operation of the CPU by using a low-cost hardware security log and The present invention provides a system and method for distributed processing and fast filtering of security logs and events using a GPU that enables high-speed filtering and analysis of events.

In particular, an object of the present invention is to provide a parallel lookup acceleration technology for collecting and collecting security logs and events using a GPU.

It is also an object of the present invention to provide a parallel processing technique of filtering rules for security logs and events.

According to an aspect of the invention, the information collecting unit for collecting the security log and event information; A memory unit to which the collected security log and event information are uploaded; And a processing unit for filtering the security log and event information of the memory unit by comparing with a classification policy of a classification policy DB. The processor may include: a CPU configured to directly compare and filter the security log and event information with a classification policy; And a GPU that distributes the grouped classification policy groups to the internal SP so that individual security logs and event information are compared in parallel with each classification policy group. It provides a distributed processing and high-speed filtering system of the security log and events, characterized in that the collected security log and event information, including at the same time is filtered at the CPU and GPU.

Preferably, the CPU compares the set of security log and event information with the classification policy of the classification policy DB and compares the classification policy with respect to the classification policy DB. The comparison process is repeated for all classification policies of the classification policy DB. Characterized in that it becomes.

Preferably, the security log and event information stored and managed in the memory unit is converted to a parallel processing data type for filtering on the GPU.

Preferably, the security log and event information is converted into a binary value for conversion to the parallel processing data type.

Preferably, the security log and event information stored and managed in the classification policy DB is converted into a binary value for filtering on the GPU.

Preferably, the comparison of the security log and event information and the classification policy in the GPU is characterized in that the bit operation of the bit unit.

Meanwhile, according to another aspect of the present invention, the method includes: (a) collecting security log and event information; (b) uploading the collected security log and event information to the memory unit; (c) filtering and comparing the security log and event information of the memory unit with a classification policy of a classification policy DB in the CPU; (d) distributing the classification policy groups grouped by the GPU to the internal SPs so that individual security log and event information can be compared in parallel with each classification policy group and filtered; And (c) and (d) are simultaneously performed on the CPU and the GPU, respectively.

Preferably, in the step (c), the CPU compares the security log and the event information set with the classification policy of the classification policy DB, and compares all of the classification policy DB with each other. It is characterized in that it is repeatedly performed for the classification policy.

Preferably, the step (d) comprises: (d-1) dividing and classifying classification policies according to the number of SPs of the GPU and distributing the classification policy groups for each SP; (d-2) converting security log and event information in the memory unit into a parallel processing data type; (d-3) disposing security log and event information converted to parallel processing data types in all SPs of the GPU, and performing filtering by comparing each classification policy with security log and event information; Characterized in that it comprises a.

Preferably, in step (d-2), the security log and event information is converted into a binary value for conversion to the parallel processing data type.

Preferably, the security log and event information stored and managed in the classification policy DB is converted into a binary value for filtering on the GPU.

Preferably, the comparison of the security log and event information and the classification policy in the GPU is characterized in that the bit operation of the bit unit.

According to the present invention, by performing the real-time filtering of the security log and events using the GPU to support the operation of the CPU, it is possible to enable high-speed filtering and analysis of the large-capacity security log and events using low-cost hardware have.

In particular, by providing a parallel log acceleration technology for the collection and correlation of security logs and events using the GPU, and parallel processing of filtering rules for the security log and events, it is possible to collect large amounts of security logs and events without delay. As a result, a large number of security logs and events generated during DDoS attacks can be handled without leakage.

In addition, high-performance, expensive collection server can be replaced by a number of low-end fish (GPU equipped) class, resulting in a high-efficiency processing efficiency.

1 is a diagram for explaining an internal SM structure of a GPU.
2 is a view for explaining the internal structure of the GPU.
3 is a block diagram illustrating a distributed processing and fast filtering system for security logs and events using a GPU according to the present invention.
4 is a flowchart illustrating a distributed processing and fast filtering method of a security log and an event using a GPU according to the present invention.
5 is a diagram illustrating a filtering method of a CPU according to the present invention;
6 is a view for explaining an example of a CPU filtering algorithm according to the present invention.
7 is a view for explaining a filtering method of a GPU according to the present invention;

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENT Hereinafter, an embodiment of a distributed processing and fast filtering system and method for security logs and events using a GPU according to the present invention will be described in detail with reference to the accompanying drawings.

Recent research on microprocesses is a multi-core processor that integrates multiple cores onto a single CPU. However, in response to this trend, the graphics processor unit (GPU) has already integrated 128 cores for NVIDIA in 2004, and after publishing the CUDA library for utilizing these parallel processing cores in 2007, the general purpose that GPUs can be regarded as parallel processors Development is underway to provide a programming environment. In addition to most operations such as integer and bitwise operations, it provides transcendental functions such as trigonometric functions. While graphics are not being processed, the general purpose computations on GPU (GPGPU), a study that utilizes the computing power of an effective resource GPU for general purpose computation, is rapidly spreading in mathematics and science. In particular, it shows excellent performance in parallel operation using parallel arrangement of small processors, which is a structural feature of GPU. That is, since transistors prioritize data processing over data cache or flow control, not only basic operations such as matrix operation and data alignment, but also applications such as N-body problem and bioinformatics It is proven to be more effective.

In order to use the GPU for parallel computing, a technology that executes programming code used on the CPU on the GPU is required. The Compute Unified Device Architecture (CUDA) enables this. CUDA is one of the technologies for coding algorithms using the C programming language on the GPU. CUDA is a general purpose GPU programming library released by NVIDIA in 2007. It works on GeForce 8 series and above by interworking with existing C compiler in Linux, Windows, and Mac environment.

Now, the structure of the GPU will be described with reference to FIGS. 1 and 2.

1 is a view for explaining the internal SM structure of the GPU, Figure 2 is a view for explaining the internal structure of the GPU.

The GPU starts off with a very simple processor called a streaming processor (SP), with more than 100 threads running simultaneously. The SP is a simple CPU with registers and computational units connected in a pipeline, and a special function unit (SFU) is bundled to form a streaming processor (SM) as a building block. This includes cache and shared memory.

The SMs are then bundled together to form a Texture Processor Cluster (TPC). For example, the video card G8800 series includes two such SMs, and the video card GT 200 includes three such SMs. In addition, NVIDIA's latest GT 280 gathers 10 TPCs of GT 200 and has a total of 240 streaming processors (SM) (see Figure 2). Finally, the TPC family, the memory (DRAM), and the PCIe interface are connected to complete the GPU.

In the present invention, by using the GPU to perform the filtering of the security log and events with the CPU has a feature that enables high-speed filtering and analysis of large-capacity security log and events using low-cost hardware.

A description will now be given of a distributed processing and fast filtering system for security logs and events using the GPU according to the present invention.

In the distributed processing and high-speed filtering system of security logs and events using the GPU according to the present invention, a router system, a firewall system, a computer system, and the like collect security logs and events in a network environment connected to each other through a predetermined communication network such as the Internet. By filtering, including information collecting unit 10, file DB 20, memory unit 30, processing unit 40, classification policy DB 50, statistical DB 60 and transmission unit 70 It is composed.

The information collecting unit 10 collects security log and event information generated in the corresponding network and system, and the file DB 20 stores and manages the security log and event information collected through the information collecting unit 10. .

The security log and event information are records of all operations performed through the corresponding computer system. When a user or an external system accesses the computer system, all operations after access and connection are logged in a predetermined location of the server. Remains in form. That is, the security log and event information include information about the tasks performed by the computer system. These include not only a specific task request and success, but also information about the solution if that specific task fails.

The security log and event information stored in this way are accumulated in the file DB 20 for a predetermined inquiry time, and the security log and event information inquired by querying the accumulated security log and event information are uploaded to the memory unit 30. .

In this case, the inquiry about the security log and event information accumulated in the file DB 20 is to remove unnecessary data in advance and to reduce the probability of false positives in the analysis, and to normalize and abbreviate the stored security log and event information ( Aggregation), processing of duplicated data, and processing of invalid data may be included.

The memory unit 30 stores and manages the security log and event information queried from the file DB 20, and stores the stored security log and event information according to the processing of the controller 50. 41 to the GPU 42 to support filtering through the CPU 41 and the GPU 42 of the control unit 50.

At this time, the security log and event information stored and managed in the memory unit 30 is converted into a parallel processing data type for filtering in the GPU 42. As described above, the GPU 42 is a processor composed of a large number of Streaming Processors (SPs) and performs parallel processing of massive security log and event information uploaded from the memory unit 30 at each SP operated in parallel. The security log and event information needs to be preprocessed to convert to parallel data type.

Therefore, the memory unit 30 manages the security log and event information of the original format that can be filtered by the CPU 41, and at the same time converts the security log and event information of the original format in the GPU 42 The security log and event information of the parallel processing data types that can be filtered are managed.

On the other hand, the processing unit 40 includes a CPU 42 and GPU 42, and distributes the security log and event information of the original format transmitted from the memory unit 30 to the CPU 42, the memory unit The security log and event information of the parallel processing data type delivered from 30 are distributed to the GPU 42, so that each processing unit performs filtering (classification) on the security log and event information.

In order to filter (classify) the security log and event information of the processing unit 40, the security policy for classification of the security log and event information is stored and managed in the classification policy DB 50.

In addition, the security log and event information detected through the filtering operation in the processing unit 40 may be uploaded to the statistics DB 60 for statistical reporting. In addition, the security log and event information detected through the transmission unit 70 may be output to the monitor in real time to be notified to the administrator or may be reported in real time to a separate security control operation management system (SOC) through a network interface.

The filtering (classification) operation of the security log and event information of the above-described processing unit 40 will now be described in detail with reference to FIGS. 5 to 7.

First, FIG. 5 is a diagram illustrating a filtering method of a CPU according to the present invention.

Here, the CPU 41 performs filtering on the security log and event information y [i] according to the classification policy x [i] of the classification policy DB 50.

In FIG. 5, the classification policy x is x [0 ... m-1], and x [i] means the i th classification policy. The set y of the security log and event information is y [0 ... n-1], and y [j] is the j th security log and event information.

The filtering method using the CPU 41 first compares the set y of the security log and event information transmitted from the memory unit 30 with the specific classification policy x [i] of the classification policy DB 50 to match. Proceed with the comparison. The comparison process is repeated for all classification policies x of the classification policy DB 50 with respect to the set y of the security log and event information.

The filtering method of the CPU 41 is represented by an algorithm as shown in FIG. 6.

Therefore, the security log and event information can be filtered by repeating the process of comparing the classification policy x [i] with respect to the set y of the security log and event information. Here, as shown in FIG. 6, the filtering method using the CPU 41 may be configured to repeatedly perform a one-to-one comparison of a string of security log and event information with a string of a classification policy.

Next, FIG. 7 is a diagram illustrating a filtering method of a GPU according to the present invention.

In the filtering method using the GPU 42, the classification policies are grouped and the grouped classification policy groups are distributed to each SP (Streaming Processor) in the GPU 42, and then the same security log and event information is placed on each SP. Filtering is performed by repeating the comparison process for the number of security log and event information.

To this end, as a classification policy arrangement step, m classification policies are grouped by m / l with respect to the number of SPs of the GPU 42 so that the classification policy group is arranged at each SP of the GPU 42. Where m is greater than l. Then, the entire classification policy x (where x is x [0 ... m-1]) of the classification policy DB 50 is converted into a binary value xb = xb [0 ... m-1]. The binarized classification policy group is distributed to all SPs of the GPU 42. Therefore, different classification policy groups are arranged in each SP in the GPU 42 as shown in FIG. 7.

Thereafter, as a security log and event information arrangement step, the security log and event information in the memory unit 30 are first converted into a parallel data type. This parallelism type conversion converts all security log and event information y (where y is y [0 ... n-1]) to the binary value yb [0 ... n-1]. The security log and event information yb [j] converted into binary values is placed in all SPs of the corresponding GPU 42 so that each security log and event information can be compared with all classification policies in parallel. .

Thereafter, as a parallel filtering step for each SP, each of the SPs of the GPU 42 compares a classification policy, a security log, and event information therein. Such a comparison operation may be performed by bitwise performing xb [i] and yb [j] bit by bit (xb [i] ^ yb [j]) inside the SP. If the result of the operation is 1, the security log and event information of the comparison target are classified (filtered) by the classification policy.

In this parallel filtering step for each SP, bit operations are used, so the operation speed is faster than string operations, and since the bit operations are independently performed on l SPs, the speed improvement can be expected as much as l times. The performance improvement can be expected as much as the product (a × l) of the calculation speed increase (a) and the calculation speed increase (l) through parallel processing.

Now, with reference to Figure 4 will be described a method for distributed processing and fast filtering of security logs and events using the GPU according to the present invention.

First, through the information collecting unit 10 collects the security log and event information generated in the network and the system (S10). In this case, the drop processing is performed on the packet which does not arrive at the receiving port (S12 to S14).

Thereafter, the security log and event information collected by the information collecting unit 10 are input to the file DB 20 and stored and managed (S16). The security log and event information stored here are accumulated in the file DB 20 for a predetermined inquiry time (S18).

Subsequently, the security log and event information accumulated during the inquiry time is inquired in the file DB 20 (S20), and the inquired security log and event information is uploaded to the memory unit 30 (S22).

Thereafter, the security log and event information of the memory unit 30 are distributed to the CPU 41 and the GPU 42 to be simultaneously filtered.

First, the CPU 41 filters the security log and event information according to the classification policy of the classification policy DB 50. Specifically, the filtering method using the CPU 41 is transmitted from the memory unit 30. The set y of the security log and event information is compared with a specific classification policy x [i] of the classification policy DB 50 and compared with each other. This comparison process is performed by comparing the set y of the security log and event information. With respect to all the classification policy x of the classification policy DB (50) to be repeated (S24).

Here, the filtering method using the CPU 41 may be configured to repeatedly perform a one-to-one comparison of a string of security log and event information with a string of a classification policy.

On the other hand, the GPU 42 groups the classification policies for security log and event information, distributes the grouped classification policy groups to each SP in the GPU 42, and then assigns the same security log and event information to each SP. Filtering is performed by repeating the process of arranging and comparing security log and event information.

Specifically, first, the classification policy is divided and grouped according to the number of SPs of the GPU 42 so that the classification policy group is arranged at each SP of the GPU 42, and the entire classification policy of the classification policy DB 50 is described. Convert to a binary value (S26).

Thereafter, the binarized classification policy group is distributed to all SPs of the GPU 42 (S28).

After that, the security log and event information in the memory unit 30 is converted into a parallel processing data type (S30).

The security log and event information converted into binary values are placed in all SPs of the GPU 42 (S32), and each SP of the GPU 42 compares the classification policy with the security log and event information therein. The filtering process is performed (S34). Here, the comparison operation may be performed by bitwise comparing the comparison policy, security log, and event information bit by bit in the SP.

Thereafter, the GPU 42 transmits the detected security log and event information to the CPU 41 (S36).

Thereafter, the CPU 41 uploads the security log and event information detected through its own filtering operation and the security log and event information detected and transmitted through the GPU to the statistical DB 60 for statistical reporting. The real time output to the monitor through the transmission unit 70 or via a network interface to report to a separate security control operation management system (SOC) in real time (S38).

As described above, an optimal embodiment has been disclosed in the drawings and specification. Although specific terms have been employed herein, they are used for purposes of illustration only and are not intended to limit the scope of the invention as defined in the claims or the claims. Therefore, those skilled in the art will appreciate that various modifications and equivalent embodiments are possible without departing from the scope of the present invention. Therefore, the true technical protection scope of the present invention will be defined by the technical spirit of the appended claims.

10: information collecting unit 20: file DB
30 memory unit 40 processing unit
41: CPU 42: GPU
50: classification policy DB 60: statistics DB
70: transmission unit

Claims (12)

An information collecting unit collecting security log and event information;
A memory unit to which the collected security log and event information are uploaded; And
A processing unit for filtering the security log and event information of the memory unit by comparing with a classification policy of a classification policy DB; Including;
Wherein,
A CPU which directly compares and filters the security log and event information with the classification policy; And
A GPU that distributes the grouped classification policy groups to internal SPs so that individual security log and event information can be compared in parallel with each classification policy group; Distributed processing and high-speed filtering system of the security log and events, characterized in that the collected security log and event information, including filtering in the CPU and GPU at the same time.
The method of claim 1,
The CPU compares the set of security log and event information with the classification policy of the classification policy DB and compares them with each other. The comparison process is repeated for all classification policies of the classification policy DB. Distributed processing and high speed filtering system of security logs and events.
The method of claim 1,
The security log and event information stored and managed in the memory unit is converted to a parallel processing data type for filtering in the GPU, distributed processing and high-speed filtering system of the security log and event.
The method of claim 3, wherein
Distributed processing and high-speed filtering system of the security log and event, characterized in that for converting the security log and event information to a binary value for conversion to the parallel processing data type.
The method of claim 4, wherein
The security log and event information stored and managed in the classification policy DB is converted to a binary value for filtering in the GPU, distributed processing and high-speed filtering system of the security log and event.
6. The method of claim 5,
Comparing the security log and event information and the classification policy in the GPU is a bit operation of the bit unit, distributed processing and high-speed filtering system of the security log and event.
(a) collecting security log and event information;
(b) uploading the collected security log and event information to the memory unit;
(c) filtering and comparing the security log and event information of the memory unit with a classification policy of a classification policy DB in the CPU;
(d) distributing the classification policy groups grouped by the GPU to the internal SPs so that individual security log and event information can be compared in parallel with each classification policy group and filtered; Including;
The steps (c) and (d) are performed simultaneously on the CPU and the GPU, respectively.
The method of claim 7, wherein
In step (c),
The CPU compares the set of security log and event information with the classification policy of the classification policy DB and compares them. The comparison process is repeated for all classification policies of the classification policy DB. Distributed processing and fast filtering of security logs and events.
The method of claim 7, wherein
In step (d),
(d-1) dividing and classifying classification policies according to the number of SPs of a GPU and distributing the classification policy groups for each SP;
(d-2) converting security log and event information in the memory unit into a parallel processing data type;
(d-3) disposing security log and event information converted to parallel processing data types in all SPs of the GPU, and performing filtering by comparing each classification policy with security log and event information; Distributed processing and high-speed filtering method of the security log and event comprising a.
The method of claim 9,
In the step (d-2),
Method for distributed processing and fast filtering of the security log and event, characterized in that for converting the security log and event information to a binary value for conversion to the parallel processing data type.
The method of claim 10,
Security log and event information stored and managed in the classification policy DB is converted to a binary value for filtering on the GPU, distributed processing and high-speed filtering method of the security log and event.
12. The method of claim 11,
Comparing the security log and event information and the classification policy in the GPU is a bit operation of a bit unit, distributed processing and fast filtering method of the security log and event.

Images