KR20150091775A - Method and System of Network Traffic Analysis for Anomalous Behavior Detection - Google Patents

Abstract

According to an embodiment of the present invention, a system for analyzing network traffic comprises: a data collection unit for collecting network traffic data; a characteristic factor extraction unit for extracting network characteristic factors from the collected data; a cluster generation unit for clustering the extracted characteristic factors; an analysis unit for extracting outliers by analyzing the cluster; and an abnormal host determination unit for determining abnormal hosts from the outliers.

Description

TECHNICAL FIELD [0001] The present invention relates to an apparatus and method for analyzing network traffic,

The present invention relates to a method for analyzing network traffic for detecting an abnormal behavior, and more particularly, to collecting network data, extracting a characteristic factor for recognizing a cyber attack or the like from the collected data, The present invention relates to a technique for detecting a host suspected of an abnormal activity among hosts in a network by periodically learning data on the network.

Generally, a method of detecting a network intrusion is divided into a misuse detection method and anomaly detection according to intrusion detection contents. The misuse detection method is to perform intrusion detection by applying a hundred or more known attack patterns already known in advance and generate a signature or a rule database for an already known attack pattern by an expert, . Techniques used in such misuse detection methods include, for example, pattern matching, state definition analysis, expert systems, key stroke monitoring, conditional probability, and the like.

The anomaly detection method is based on statistical data and is a technique to detect various non-ideal behaviors that are not known from statistics. The abnormal behavior detection method requires a lot of learning data for the learning process to distinguish normal or abnormal from statistical data. The abnormal behavior detection methods include statistical methods, feature extraction, combination of abnormal behavior measurement methods, predictable pattern generation, and neural network techniques.

In the case of existing rule-based network intrusion detection systems, rules are defined based on known attacks or malicious actions, and because they recognize external intrusions by these rules, new types of malicious attacks I can not detect it. For example, an attacker may infiltrate a vaccine program update server to change the redirection address of the vaccine update server, to update the vaccine from the malicious server designated by the attacker himself, Can not be prevented by existing security equipment.

Meanwhile, the latest intelligent cyber security technology aims at an integrated security management technology that utilizes a large amount of data of various sources and configures network and system events into one integrated security infrastructure. Unknown cyber attacks, such as hacking and malicious program evolving APT (Advanced Persistent Threat) attacks, can cause many harms both at the organization and at the national level, as well as through information leakage, system down, site and network paralysis. . Therefore, intelligent security technology is a next generation security information technology that improves security intelligence by analyzing the association between data and security events occurring in host, network, etc., . ≪ / RTI >

An IDS (Intrusion Detection System) that detects intrusion of a network in general can recognize this when an attack such as DDoS, port scan, or an attempt to crack a computer occurs, but it is a recent intelligent It is necessary to recognize and detect secret attacks by analyzing the relation of various data collected on the host and network, not merely blocking one attack element, because it reveals limitations in recognizing and defending cyber target attacks have.

In order to solve the above-described problems of the related art, the present invention has been made to collect network data using a network data collection device to detect a cyber target attack, extract characteristic parameters for recognizing a cyber target attack from the collected data, , Analyzing the change state of the cluster diagram for each extracted characteristic factor on a time basis, and periodically learning it to detect a host suspected of anomaly activity among the hosts existing on the network, And to provide a system and method for intuitively and visually expressing a relationship between a characteristic parameter of a host and abnormal hosts.

A network traffic analysis system according to an embodiment of the present invention includes: a data collection unit for collecting network traffic data; A characteristic parameter extracting unit for extracting a network characteristic parameter from the collected data; A cluster generating unit for performing clustering on the extracted characteristic factors; An analysis unit for analyzing the cluster and extracting outliers; And an abnormal host determination unit for determining an abnormal host from the abnormal host.

According to another aspect of the present invention, there is provided a network traffic analyzing method for detecting a host that performs an abnormal operation, comprising: collecting network traffic data in a network traffic analyzing system; Extracting a network characteristic factor from the collected data; Performing clustering on the extracted characteristic factors; Analyzing the cluster to extract an abnormal host; And determining an abnormal host from the abnormal host.

According to an embodiment of the present invention, network data is collected using a network collecting device to detect a cyber target attack, a characteristic factor for recognizing a cyber target attack is extracted from the collected data, Based on time-based analysis of the state of change in the degree of aggregation, and periodically learns the hosts that are suspicious of anomaly behavior among the hosts in the network through the network data analysis method, And the relationship between the abnormal hosts can be intuitively and visually expressed.

In addition, the next generation intelligent security system needs to detect and block new cyber target attacks that are secretly proceeding with a continuous period of time rather than existing known attack patterns or rules. A method for collecting and analyzing data on the network according to the present invention, The system can be utilized as a technology and system for detecting a suspicious host with a cyber-target attack.

FIG. 1 is a conceptual diagram of a network analysis system for detecting abnormal behavior according to an embodiment of the present invention.
2 illustrates an exemplary structure of a network analysis system for abnormal behavior detection according to an embodiment of the present invention.
Figure 3 shows an exemplary list of network data characterization factors in accordance with one embodiment of the present invention.
4 shows clustering and cluster shape changes according to an embodiment of the present invention.
FIG. 5 illustrates an example of extracting an abnormal host list through a cluster analysis process according to an embodiment of the present invention.
FIG. 6 shows a visualization screen of a network analysis system according to an embodiment of the present invention.
FIG. 7 is a flowchart illustrating an abnormal behavior determination according to an exemplary embodiment of the present invention.

Various embodiments of the present invention will now be described in detail with reference to the accompanying drawings. Note that, in the drawings, the same components are denoted by the same reference symbols as possible. Further, the detailed description of well-known functions and constructions that may obscure the gist of the present invention will be omitted. In other words, it should be noted that only parts necessary for understanding the operation according to the embodiment of the present invention will be described, and descriptions of other parts will be omitted so as not to disturb the gist of the present invention.

An embodiment of the present invention described below collects network data, extracts a characteristic factor for recognizing a cyber target attack from the collected data, periodically learns data about the extracted characteristic factor, And a network data analysis method and system for detecting a suspicious host suspicious of an existing host. More specifically, the present invention includes a process for analyzing network data, a method for detecting a suspicious host through a characteristic parameter, a structure of a network analysis system including the process and method, and a screen configuration for expressing analysis results have.

FIG. 1 is a conceptual diagram of a network analysis system for detecting abnormal behavior according to an embodiment of the present invention.

Referring to FIG. 1, the network environment 10 may include an internal network 100 such as a corporate network, a network data analysis system 200, and an external network 300 such as the Internet. The internal network 100, the network data analysis system 200, and the external network 300 may be connected through a switch 110, a switch 310, a router 320, and the like. Such a configuration is exemplary and there may be various modified network environments.

A network data analysis system 200 (hereinafter also referred to as an analysis system 200) is located at a connection point between an external network and an internal network and detects network traffic generated between the external network and the internal network in order to detect malicious activity occurring from the outside And a network data collection device 210. In the example of FIG. 1, each separate device is shown as wired, but a data collection module or device located within one system 200 may perform the function of the network data collection device 210. This will be described with reference to FIG.

2 illustrates an exemplary structure of a network analysis system for abnormal behavior detection according to an embodiment of the present invention. The structure shown in FIG. 2 is exemplary, and the analysis system 200 does not necessarily have to have the same structure. Some of the components may be omitted, and functions shown as at least two units may be performed in one unit.

2, the analysis system 200 may include a data collection module 210, a cluster generation module 220, a model analysis module 230, and an abnormal recognition module 240. It may further comprise a data store 250 for storing the data obtained in each module.

The data collection module 210 may collect network data and perform preprocessing. The data collection module 210 may correspond to the above-described network data collection device. The data collection module 210 includes a network data collection unit 211 for collecting data, a network data statistics unit 213 for obtaining statistical information from the collected data, a characteristic parameter extraction unit 215 for extracting characteristic parameters from the collected data, And a data storage unit 217 for storing data in a big data platform or the like. The types and characteristics of the characteristic factors will be described later with reference to Fig.

The cluster generation module 220 may perform clustering using a cluster algorithm and defined characteristic factors. The cluster generation module 220 performs clustering and learning using an IP layer cluster unit 221 for performing clustering and learning using IP layer parameter among extracted network characteristic parameters and a transport layer parameter And an application layer analyzer 225 for performing clustering on application layer parameter of the application layer.

The model analysis module 230 can detect a suspicious host by analyzing clusters generated in the cluster generation module 220. The model analysis module 230 analyzes the shape of the cluster and detects an outlier (or an abnormal host) not belonging to the cluster. The cluster analysis unit 231 analyzes the detected anomaly by comparing the result of the previous clustering (clustering) A cluster time series analyzing unit 233 for performing filtering on a point, an abnormality point extracting unit 235 for finally extracting an abnormality point by combining the above processes, and an analyzing unit 235 for analyzing a past behavior history of the host corresponding to the extracted abnormality from stored data And a behavior analysis unit 237 for analyzing the behavior of the user.

The anomaly recognition module 240 may determine an abnormality with respect to the suspect host extracted from the model analysis module 230 and display the result on the screen supported by the system or the system. The abnormal recognition module 240 includes an abnormal host determination unit 241 for determining whether an action of a host classified as a suspicious host resembles or matches the behavior of a cyber target attack, an analyzed data display unit 243 for displaying a determined abnormal host, And an abnormal host tracking management unit 245 for managing the history of the host.

The data collection unit 211 for collecting network data may collect data related to the network elements from the network collection equipment and the user host. More specifically, when the host communicates, the IP address and port of the sender, the IP address and port of the destination, the duration of the data flow (i.e., start time and end time), the number of received packets The number of transmission packets, the size of received data, and the size of transmission data can be collected. Also, a list of TCP flags in a data flow, a service ID, an application protocol ID, a device ID, a service provider ID, an observation domain ID, and the like may be collected.

In addition, the data collection unit 211 may collect a payload of the packet when the unknown service or the unknown signature is included, while maintaining the database of the service and the signature. The raw data on the collected network is statistically analyzed by the network data statistics unit 213, and the network characteristic parameter extraction unit 215 can extract the characteristic parameters.

Figure 3 shows an exemplary list of network data characterization factors in accordance with one embodiment of the present invention.

Referring to FIG. 3, characteristic factors for analyzing network data can be divided into four categories. That is, the characteristic parameter can be divided into an IP layer characteristic parameter, a transport layer characteristic parameter, an application layer characteristic parameter, and a behavior characteristic parameter. Each extracted characteristic factor may be clustered in the corresponding cluster part 221, 223, 225, and analyzed by the cluster analysis part 231 and / or the behavior analysis part 237.

The IP layer characteristic factor may include five characteristic factors: TCHC, TCPC, FS, BDIP, and PDIP. More specifically, the network characteristic parameter extracting unit 215 extracts the number of connection hosts (TCHC) within 2 seconds, the number of connection ports (TCPC) within 2 seconds, The total number of flows (FS), the amount of data for each destination address (BPID), and the number of packets for each destination address (PDIP) can be extracted.

The transport layer characteristic parameters can include six characteristic parameters: PTYPE, FBYTE, TFLAG, DIP, DIPP, and TSUB. More specifically, the network characteristic parameter extracting unit 215 extracts, from the protocol type list (PTYPE), the source byte (src_bytes), and the destination byte (dst_bytes) of the received packet for analysis of the transport layer, (DIPP) of the destination IP address, and the destination address (TFLAG), the distribution of the destination IP address (DIP), the destination TCP / The total traffic statistics (TSUB) of the subnet can be extracted and calculated.

The application layer characteristic parameter may include five characteristic parameters: SAUP, SADP, FSID, FAID, and FSPID. More specifically, the network characteristic parameter extracting unit 215 extracts a network characteristic factor extracting unit 215 for analyzing an application layer, such as a user service and an application usage frequency (SAUP), a used service and application concentration (SADP), a service flow statistics (FSID) The flow statistics (FAID), and the flow statistics (FSPID) for the service provider ID.

Behavioral factors may include three characteristic factors: GF, USID, and USP. More specifically, the network characteristic parameter extracting unit 215 can extract information of connected destinations (GF) for analyzing user behavior, information of an unknown service ID, information (USP) about unknown signatures in the payload have.

For example, referring again to FIG. 2, the IP layer cluster unit 221 determines the number of hosts, the number of ports, the number of flows, the amount of data for the destination IP address, and the number of packets We first calculate the mean value of the BDIP and PDIP characteristic factors. In addition, clustering can be performed through the clustering algorithm using the parameter values of five types (TCHC, TCPC, FS, BDIP, and PDIP) as variables. Clustering generally means generating a dataset with clusters of similar characteristics by repeatedly using algorithms such as K-Means clustering algorithm and EM algorithm. Clustering can detect clusters and outliers that do not belong to clusters. Also, the result can be expressed visually, which will be described later with reference to FIG. As a result of clustering, the clusters, that is, the clusters and the center values and distance values within each cluster, can be calculated. As a result, periodic clustering process can be performed on clusters resulting in periodic clustering. And it is possible to extract outliers which are data not belonging to the cluster. For example, the IP layer analyzing unit included in the cluster analyzing unit 231 or the cluster analyzing unit 231 compares with the value of some other host or the average through the clustering, The host exchanging a large amount of packets or data can be extracted.

In this manner, the transport layer cluster unit 223 can perform clustering on the extracted characteristic parameters PTYPE, FBYTE, TFLAG, DIP, DIPP, and TSUB for a plurality of hosts. In the case where the difference between the transmission data and the reception data is remarkably large or the distribution of the destination address is different from that of the host through the clustering, the transport layer analyzing unit included in the cluster analyzing unit 231 or the cluster analyzing unit 231, And a case where a distribution difference of SYN, ACK, and FIN is large in the network can be detected.

In addition, the application layer cluster unit 225 may perform clustering on SAUP, SADP, FSID, FAID, and FSPID, which are extracted characteristic parameters for a plurality of hosts. The application layer analyzing unit included in the cluster analyzing unit 231 or the cluster analyzing unit 231 can find a case where the flow of the application differs from other hosts through clustering of the application and the service. For example, in the case of a vaccine update, a host with a different flow than the flow in the normal case can be found as a suspicious host.

4 shows clustering and cluster shape changes according to an embodiment of the present invention.

Referring to FIG. 4, clustered results are shown in cluster generation module 220. Result 410 shows an example of clustering. In the result 410, as a result of clustering, five clusters 401 and a plurality of outliers 402 that do not belong to the cluster 401 appear. The number of anomalies 402 may be adjusted by a clustering algorithm and other settings.

Such clustering may be performed periodically at a specific time T seconds. In addition, the rate of change between the clustering result of the previous period and the current clustering result is analyzed, and the abnormal point data not belonging to each cluster can be managed as a candidate for abnormal behavior detection (anomaly detection) in the cluster time series analyzing unit 233 . Through the analysis of the shape change rate of clustering, the number of extracted abnormal points can be variably adjusted when the ideal point increases sharply. In this case, the distance from the center point of the cluster to the outer boundary of the cluster may be increased.

As shown in Fig. 4, when clustering is performed periodically, a result 420A having almost no cluster change, or a cluster 420B having a large cluster change can be derived. The cluster time series analyzer 233 can track the rate of change and lower the false rate. The anomaly point extracting unit 235 extracts an anomaly point derived from the analysis results of the IP layer cluster unit 211, the transport layer cluster unit 223, the application layer cluster unit 225, and the cluster analysis unit 231 corresponding thereto, The ideal point can be finally determined.

FIG. 5 illustrates an example of extracting an abnormal host list through a cluster analysis process according to an embodiment of the present invention.

5, the clusters generated by the clustering algorithm in the IP layer clusters 221, the transport layer clusters 223, and the application layer clusters 225 of the cluster generating module 220 are divided into IP layers The cluster analysis unit 501, the transport layer cluster analysis unit 503, and the application layer cluster analysis unit 505 can extract the abnormal point 510, respectively.

In a modified embodiment, each of the cluster and the analysis unit can be integrated. For example, the IP layer cluster unit 221 and the IP layer cluster analysis unit 501 may be integrated into the IP layer analysis unit to perform abnormal point extraction through clustering and cluster analysis.

Referring again to FIG. 5, the IP layer cluster analyzer 501 can classify hosts 2, 5, 8, 12, and 18 (H2, H5, H8, H12, and H17) as abnormal points. In this way, the abnormal host candidate group 510 can be derived from each of the analysis units 501, 503, and 505. [ The anomaly extracting unit 235 integrates the results of the analyzers 501, 503, and 505 and the cluster movement rate based on the clustering result of the previous period provided by the cluster time series analyzing unit 233, H5, H8, H11) can be extracted. The behavior analysis unit 237 can determine the final unhealthy host set 530 (e.g., H5, H8) based on the behavior pattern database 507 and the geographic information database 509. [ The behavior analysis unit 237 may include a list of hosts in which an abnormal point has occurred in the non-ideal host set by time zone. The databases 507 and 509 are shown as separate databases, but may be co-located in the data store 250 shown in FIG.

For example, the behavior analysis unit 237 can inquire the connection IP address of each suspect host 520 to the geographical information database 507 to determine the connection location of each suspect host 520. In general, an attacker or a hacker often connects via a national system which is difficult to track in order to hide his / her IP. Therefore, if the access IP is an IP of the same country, the suspicious host performs an abnormal behavior It is likely to be a host. In this case, the connected destination address is a place that does not access much in the present network, and can be classified as an abnormal host if the connected country of the country is commonly used as the stopover destination of the attacker.

In another example, the behavior analysis unit 237 may determine whether to send a keep-alive message periodically to an external system based on the behavior pattern database 507, If it does not happen well, you can classify it as an abnormal host.

FIG. 6 shows a visualization screen of a network analysis system according to an embodiment of the present invention.

The visualization screen 600 of FIG. 6 may be provided on the screen of the network data analysis system by the analysis data presentation unit 243 of the abnormal recognition module 240. However, in a modified embodiment, the visualization screen 600 may be displayed on a screen of an administrator computing system in an enterprise internal network.

6, the visualization screen 600 includes a feature (parameter) display panel 601 showing a list of characteristic parameters, a host display panel 602 indicating the current status of the hosts, And a relation display panel 603 indicating the relationship between the characteristic factors.

In the host display area 602, the IP address or ID of all the hosts on the network may be indicated by a point 604. The region 602 may be composed of a two-dimensional plane. As the X coordinate value 605 increases, the number of times that the host is classified as an abnormal host increases, and as the Y coordinate value 606 increases, the suspicious value of the past data analysis result of the host may increase.

That is, the lower left area on the host display area 602, that is, the so-called safe zone 507, has the smallest number of times of being classified as an abnormal host and the result of past data analysis is low. can do. On the contrary, the rightmost upper region, the so-called 'most suspicious zone' 608, has the highest number of abnormal hosts, and has a high suspicion of the past data analysis results.

In addition, the connection line 609 between each host and the characteristic parameter is classified as an abnormal host by the characteristic parameter connected to the host, and the thicker the line, the more frequent the classification as the abnormal host due to the characteristic parameter have. Also, it can be seen that as a specific host (for example, Host # 1) moves from the normal host region to the abnormal host region over time. (E. G., Movement 610)

FIG. 7 is a flowchart illustrating an abnormal behavior determination according to an exemplary embodiment of the present invention. In the following description, descriptions corresponding to or overlapping with those described above will be omitted.

Referring to FIG. 7, when the network data is collected from the network data collecting device in S701, the collected data is stored in S702, the data statistics on the collected data are calculated in S703, and stored in the statistical data storage in S704. The characteristic parameters are extracted from the characteristic parameter profile in the data collected in S705 and the statistical data. The characteristic factor profile may be stored in the data store 250 in advance. In addition, the repositories described in connection with FIG. 7 may be located separately, or may be integrated into the data repository 250.

In S707, the data on which the characteristic parameter is extracted is clustering performed for each characteristic factor set by the clustering algorithm, and the result data clustered in S708 can be stored in the cluster data store. The analysis system 200 can analyze the result data after cluster execution in S711 to extract a host having abnormal data that is not included in the cluster. In this case, it is possible to extract suspicious hosts based on the cluster shape change rate trend by comparing and analyzing the cluster shape change rate over time with the previous cluster shape result (S712).

The long-term history data and the past collected data for the suspected hosts extracted in S717 are extracted, and the history of the suspicious hosts in the past T-time and the history of the past abnormal hosts can be analyzed . An abnormal host can be determined based on the analysis result in S718. The analysis system 200 may store the determination result in the abnormal host repository (or data repository 250) and output it to the screen at S720. In addition, such a series of processes may be performed periodically at a time interval having a predetermined period (S720).

It should be noted that all embodiments and conditional examples disclosed herein are intended to assist the reader in understanding the principles and concepts of the present invention by those of ordinary skill in the art, It will be understood that the invention may be embodied in various other forms without departing from the spirit or essential characteristics thereof. Therefore, the disclosed embodiments should be considered in an illustrative rather than a restrictive sense. The scope of the present invention is defined by the appended claims rather than by the foregoing description, and all differences within the scope of equivalents thereof should be construed as being included in the present invention.

Claims (15)

A network traffic analysis system comprising:
A data collection unit for collecting network traffic data;
A characteristic parameter extracting unit for extracting a network characteristic parameter from the collected data;
A cluster generating unit for performing clustering on the extracted characteristic factors;
An analysis unit for analyzing the cluster and extracting outliers; And
And an abnormal host determination unit that determines an abnormal host from the abnormal host. The method according to claim 1,
Wherein the data collector is located at a connection point connecting the internal network and the external network. The method according to claim 1,
Wherein the characteristic parameter comprises at least one of an IP layer characteristic parameter, a transport layer characteristic parameter, an application layer characteristic parameter, and a behavior characteristic parameter. The method according to claim 1,
Wherein the analysis unit extracts a host not included in the cluster for the extracted characteristic factor as an abnormal host. The method according to claim 1,
Wherein the abnormal host determination unit determines an abnormal host based on the behavior pattern or the geographic database of the abnormal host. The method according to claim 1,
And periodically performs an abnormal host determination process. The method according to claim 1,
Further comprising an analysis data presentation unit for visually presenting on the display screen whether or not an abnormal host is present for each host existing on the network. 8. The method of claim 7,
The display screen displays,
A specific parameter display area for displaying a characteristic parameter list,
A host display area indicating whether or not each host is abnormal, and
And a relationship display area for indicating a relationship between each host and a characteristic parameter. 9. The method of claim 8,
Wherein the host display area is configured in a two-dimensional plane, one axis represents the number of times each host is classified as an abnormal host, and the other axis represents a result of previous analysis for each host. The method according to claim 1,
Wherein the abnormal host determination unit determines the abnormal host based on at least one of past clustering result and past history data for the extracted abnormal host. A network traffic analysis method for detecting a host performing an abnormal behavior,
In a network traffic analysis system, collecting network traffic data includes:
Extracting a network characteristic factor from the collected data;
Performing clustering on the extracted characteristic factors;
Analyzing the cluster to extract an abnormal host; And
Determining an abnormal host from the abnormal host. 12. The method of claim 11,
Wherein the series of steps is performed according to a predetermined period. 12. The method of claim 11,
Further comprising displaying on the screen of the system whether an abnormal host is present for each host present on the network. 12. The method of claim 11,
Wherein the step of determining the abnormal host determines the abnormal host based on the geographical information obtained through the IP address of the abnormal host. 12. The method of claim 11,
Wherein the step of determining the abnormal host determines an abnormal host based on whether the abnormal host periodically transmits a keep-alive message to an external system.

Images