KR100950079B1 - Network abnormal state detection device using HMMHidden Markov Model and Method thereof - Google Patents

Abstract

Disclosed are an apparatus and method for detecting probabilistic network abnormal symptoms using hidden Markov model. The present invention probabilistically displays the cause of the network abnormality on the network by using a hidden Markov model so that the network administrator can determine the network abnormality more accurately and based on the manager's ability. A detection apparatus and a method thereof are provided. At present, the abnormal symptom detection method is a deterministic prediction, which does not involve the administrator's judgment and does not guarantee network scalability and has a high false positive rate. Accordingly, the present invention applies a hidden Markov model to probabilistically indicate the presence or absence of network traffic, thereby providing an administrator with a probability of predicting network abnormalities.

Hidden Markov Model, Network Traffic, Probability, Network Traffic Characteristics

Description

Network abnormal state detection device using HMM (Hidden Markov Model) and Method according to Hidden Markov Model

The present invention relates to an apparatus and method for analyzing network traffic for analyzing network traffic conditions on a network.

Network attacks not only make many services available over the Internet normally available, but they can also steal important economic or financial losses such as personal, financial, and military information. Therefore, it is important to detect and analyze network attacks in order to find out more about the current situation in which all life is connected and integrated around the network.

So far, there are rule-based and abnormal symptom-based detection methods for detecting network attacks.

Among them, the anomaly-based detection method detects an abnormal state when the network subscriber access level exceeds the traffic volume-based threshold set by the administrator by measuring the volume of the traffic to classify and analyze the traffic and determine the abnormal state of the network. Judging by. However, this traffic analysis method is not easy to determine the abnormal condition that may affect the performance of the network as a whole, and the threshold based on the traffic volume is an absolute value for the traffic volume. There is a problem that it is difficult to set the value properly.

Currently, abnormal symptom-based detection systems are in the spotlight due to problems such as lack of scalability and increased load of system administrators.However, abnormal symptom-based detection systems show only network occurrences as occurrences and non-occurrences. Diagnosis of abnormal conditions occurs.

The present invention aims to increase the accuracy of network abnormality determination using the existing abnormal symptom-based detection method by introducing a hidden Markov model, which is a learning-based model.

In addition, the present invention aims to provide a probabilistic abnormal symptom detection method to avoid the deterministic abnormal symptom detection method of the prior art and to emphasize the judgment authority of the administrator.

Other objects of the present invention will be readily understood through the following description of the embodiments.

According to an aspect of the present invention, a method for determining an abnormality of network traffic, comprising the steps of: a) inputting the network traffic; and b) deriving network traffic characteristics by analyzing the network traffic; and c) Selecting an observation set from the network traffic characteristics; and d) setting a traffic state of the network expected from the observation set to a state set; And e) applying a decision matrix of a Hidden Markov Model to the observation set and the state set to display all time series state states of the state set as probabilities. There is provided a method of determining whether there is an abnormality of a network.

Here, step b) provides a method of determining whether there is an abnormality in the network for deriving the network traffic characteristics by analyzing the network traffic in packet units.

The network traffic characteristics may include destination IP usage, number of ports, packet arrival rate, payload size, and payload size variation. Port number variation, Session time, Payload size per session, Number of Sessions per port, Source Destination IP ratio ), A method for determining whether there is an abnormality of a network including at least one of a destination IP distribution and reflection traffic.

Here, the observation set may include port scanning, IP scanning, IP spoofing (black IP), packet spoofing (checksum error), sudden increment, Provided is a method of determining whether an abnormality of a network including at least one of reflection is present.

Here, a method of determining whether there is an abnormality in a network for selecting the observation set by examining the frequency of occurrence of the network traffic characteristics for a unit time based on the association between the network traffic characteristics and the observation sets.

Here, the state set includes Denial of Service (DoS) / Distributed Denial of Service (DDoS), Worm-based Virus (Worm), Unknown Attack (Unknown), and Normal (Clear). Provided is a method for determining whether or not an abnormality of a network including at least one of a) state is present.

In addition, the apparatus for determining the abnormality of the network traffic, the apparatus comprising: a symptom extractor for outputting network traffic characteristics by analyzing the network traffic; And a symptom determination unit for outputting each state of the time series of the network as a probability value using the network traffic characteristic output from the symptom extraction unit as an input value.

Here, the symptom extraction unit is a traffic input unit to which the network traffic is input; and a packet processing unit for processing the input network traffic in a packet unit to determine whether there is an abnormality of the network traffic; and an abnormality of the network due to the processed packet. Log record unit for recording the presence or absence in a log file; and a network traffic characteristic output unit for analyzing the packet unit for a unit time for outputting the statistical data value for each unit time as the network traffic characteristics An apparatus for determining is provided.

The network traffic characteristics may include destination IP usage, number of ports, packet arrival rate, payload size, and payload size variation. Port number variation, Session time, Payload size per session, Number of Sessions per port, Source Destination IP ratio An apparatus for determining an abnormality of a network including at least one of a destination IP distribution and reflection traffic is provided.

The symptom determination unit may include a statistical data input unit that takes statistical data values of network traffic characteristics output from the symptom extractor as an input value, and an observation set derivation unit that derives the observation set using the input statistical data; A Hidden Markov Model (HMM) processing unit for deriving the state set by applying the Hidden Markov model from the observation set; And a state probability display unit for displaying all state strings of the state sets derived through the hidden Markov model processing unit as probability.

In addition, the recording medium recording a method for determining the abnormality of the network traffic, the step of inputting the network traffic; and analyzing the network traffic to derive network traffic characteristics; and observation from the network traffic characteristics Selecting a set, and setting a traffic state of a network expected from the set of observations as a state set; And applying a decision matrix of a Hidden Markov Model to the observation set and the state set to display all time series state sequences of the state set as probabilities. There is provided a recording medium on which a method of determining the presence or absence of an abnormality is recorded.

According to the present invention, there is an effect of displaying a traffic state (time series state) during a unit time as a probability by using a hidden Markov model.

In addition, since the network traffic status is not displayed as abnormal or normal, the traffic status generated during a unit time is displayed as a probability, and thus, there is an effect that the abnormality of the network can be indicated by the more accurate selection of the network administrator.

As the invention allows for various changes and numerous embodiments, particular embodiments will be illustrated in the drawings and described in detail in the written description. However, this is not intended to limit the present invention to specific embodiments, it should be understood to include all transformations, equivalents, and substitutes included in the spirit and scope of the present invention. In the following description of the present invention, if it is determined that the detailed description of the related known technology may obscure the gist of the present invention, the detailed description thereof will be omitted.

The terminology used herein is for the purpose of describing particular example embodiments only and is not intended to be limiting of the present invention. Singular expressions include plural expressions unless the context clearly indicates otherwise. In this application, the terms "comprise" or "consist" are intended to indicate that there is a feature, number, step, action, component, part, or combination thereof described in the specification, and one or more other It is to be understood that the present invention does not exclude the possibility of the presence or the addition of features, numbers, steps, operations, components, parts, or a combination thereof.

First of all, before describing an embodiment of the present invention based on the drawings, the network traffic characteristics, observation sets, and state sets used in the present invention will be defined.

In the present invention, the above 12 network traffic characteristics may be defined as shown in Table 1 below.

Twelve network traffic characteristics are numbered in order, and then network traffic characteristics will be described as numbers matching the network traffic characteristics.

<Table 1>

Network traffic characteristics Justice 1) Destination IP usage Create a histogram of the top 10 IPs for the frequency of destination IP per unit time. At this time, every time the corresponding destination IP appears for each packet, the number of occurrences is increased by 1, and the number is expressed according to the number of uses. The ratio of the top 10 destination IPs. 2) Number of Ports Number of destination ports used per unit time. 3) Packet Arrival rate Packet arrival rate per unit time. 4) Payload size Payload size per unit time 5) Payload size variation Distribution of the top 10 payload sizes per unit time. 6) Port number variation Number of top 10 destination ports per unit time. 7) Session time Distribution of session time per session of unit time. 8) Payload size per session Distribution of cumulative payload size per session in unit time. 9) Number of Sessions per port Distribution of cumulative number of packets per session in unit time. 10) Source Destination IP ratio The ratio of Source IP and Destination IP per unit time. 11) Destination IP distribution Frequency of use of IP bands per unit time. Frequency of use of class A. 12.Reflection traffic Number of packets of RST and Unreachable per unit time.

The twelve network traffic characteristics are described in more detail below.

1) Destination IP usage

Network attackers usually use zombies on a network to attack a single target host. Attacks can be suspected when network traffic is concentrated on a single destination IP. Especially in the case of DDoS, network traffic is likely to be concentrated on a single destination IP.

Create a histogram of the top 10 IPs for the frequency of destination IP per unit time. Whenever the corresponding destination IP appears for each packet, the number of occurrences is increased by one, and it is expressed according to the number of uses. In this case, the actual IP address is hidden as a privacy exposure problem and is simply expressed as an index.

2) Number of open port

Common network attacks target vulnerable services. Therefore, if there are a large number of open ports per hour, and network traffic destined for those ports increases, there is a high probability that an attack is occurring on the network.

In the present invention, the number of occurrences of the destination port is increased every unit time, and each occurrence is calculated as a probability relative to the total unit time traffic. In the case of DDoS and Witty worms, the probability increases rapidly at the time of network attack. Other abnormal symptom traffic is characterized by a large standard deviation of the number of open ports.

3) Packet arrival rate

Normal network attacks use more packets than normal networks, which increases the packet arrival rate. Similarly, the interval of arrival time (Inter packet time) is reduced.

In the present invention, the packet inter-arrival time is measured. In addition, in the present invention, the network attack occurrence time is more clearly distinguished, and the average packet is analyzed in units of 1,000 to reduce the computational load of the network.

In the case of denial of service (DoS), network attack occurs at a time point, and the fluctuation of the packet arrival rate per unit time is severe. Codered worms, Slammer worms, and Whitty worms all have a longer packet arrival time when network attacks occur. This is because multiple sessions / packets are sent for generation of network attack packets.

4) Payload size

Typical network users use FTP and HTTP to send text, multimedia, and images intermittently. On the other hand, network attackers such as DoS and Worm have a relatively constant payload size, so they send packets at a quick interval.

In the present invention, payload size was measured for each observation time. Payload size tends to increase when an attack occurs on network traffic. Since traffic with worms often uses a fixed length payload size, the standard deviation tends to be smaller than that of normal traffic. That is, the standard deviation of payload size is the largest when network traffic is attacked by DoS and the least normal traffic is the least. Since the difference between the absolute volume of normal traffic and attack traffic is the capacity difference of the network itself, the absolute comparison is meaningless.

5) Payload size variation per packet

The packet used in the present invention is transmitted with the maximum capacity that can be transmitted. The maximum transmission unit (MTU) of a typical network is 1500 bytes. Network attacks such as DoS / DDoS aim to cause network denial of service by sending meaningless data, so if small packets are continuously transmitted, network attack can be suspected. Worm series, on the other hand, use the same payload size because it contains attack codes. In this case, the variation in packet size is very low.

6) Port number variation

It is decided according to the use used for each packet. Thus, if network traffic is concentrated only on specific ports, network attacks can be suspected.

In the present invention, similar to the payload size per packet, the usage distribution of the port number per packet was recorded. .

7) Session time

Session time generally means that the network user starts session-connected TCP 3-way handshaking and then takes time to end with TCP 4-way handshaking. In general, a user has a relatively long session time, such as accessing a server, downloading texts, images, multimedia, etc., or continuing a session through a P2P connection with the user. However, when a network attack occurs, a large number of sessions are created within a short time to attack the target host, so that the change in session time occurs more than in a normal network situation.

8) Payload size per session

Sessions established for general communication come and go with various packets. If a certain length of packets is observed continuously in each session, a network attack can be suspected.

9) Number of session per port

Under normal circumstances, the number of packets per session varies greatly. The reason for this is that a network user sends many packets for user data in addition to the packets for a TCP connection per session. On the other hand, in the case of network attack, the number of sessions per session is not as high as the general situation because the purpose is to create multiple sessions.

10) Source, Destination IP ratio

Typically, network attackers use a number of zombies to attack a single target host. Therefore, the ratio of source IP and destination IP is high. Under normal circumstances, network users are connected to multiple servers, and each server has sessions with multiple clients, so the ratio of source IP and destination IP is relatively constant.

In the present invention, the ratio of the source IP and the destination IP is recorded for each observation time. In general, since multiple Client Source IPs are connected to the Destination IP, a ratio of 1.0 to 1.2 is recorded as in the case of normal traffic.

However, in case of network attack by DDoS, the attack is concentrated to the destination IP, so the ratio of source IP and destination IP is 0.4 or 0.01 at the time of the attack. Tends to decrease.

The ratio of Source IP and Destination IP was 1.2 when there was no network attack by DoS and 1.0 when network attack by DoS occurred.

On the other hand, in the case of Slammer and Witty worm, the activity of Worm-infected Client IP is prominent, so the ratio of Source IP and Destination IP tends to increase to 2.0 and even 6.7. In other words, the ratio of source IP and destination IP shows a large standard deviation compared to normal traffic when the network is attacked.

11) Destination IP distribution

As mentioned earlier, Destination IP, targeted as an attack target, is continuously attacked by zombies and serially infected by surroundings. This creates a number of specific zombies, which in turn increase their targets through their paths. The attack can be suspected by observing the frequency of the destination IP band to be attacked.

In the present invention, the frequency of use of the destination IP is expressed as a Cumulative Density Function (CDF). Whenever the destination IP appears in the packet, the number is increased by one, and it is expressed according to the number of uses.

12) Reflection traffic

Since a network attacker using TCP periodically sends a SYN packet to a target host, retransmission of the SYN packet increases more than normal. It also intentionally sends TCP RST and ICMP unreachable packets to congestion on the user's network.

In the present invention, the occurrence frequency of TCP RST / ICMP unreachable packets was observed for each observation time. For traffic with network attacks, the increase in RST / ICMP is noticeable because IP / Port Scanning is more than Normal Traffic.

In addition, the observation set defined in the present invention consists of six signs, the definitions of each of which are shown in Table 2 below.

<Table 2>

Matching symbol Observation set Justice A Port Scanning Port scanning is usually a way to check if the server is working properly and if a port is open on the network interface. The purpose of performing port scanning depends on the point of view, but if it is intended to attempt an attack, network intruders can see if there is a gateway to enter. B IP Scanning IP scanning, like port scanning, also refers to the act of scanning the network's IP for a specific purpose. C IP Spoofing (Black IP) IP spoofing means creating a packet that appears to be sent from a different IP address. The method is mainly used for DoS attacks. If the packet appears to be sent from a computer on the local network, the packet passes through firewall security and can be a threat to the network. D Packet Spoofing (Checksum Error) Packet spoofing refers to a network attacker modifying a packet for a specific purpose, or arbitrarily modifying a payload. In this case, a checksum error occurs. E Sudden Increase Sudden increase in packet requests or responses in the network can be a suspicious network attack. It is likely that an attacker is collecting your information for malicious purposes. F Reflection If the transmitted packet is returned, it may suspect that collecting information for the attack. A network attacker could attack by intercepting a message created by the sender and resending the message back to the sender to gain access.

In addition, the state set used in the present invention includes the following four states, the definition of which is shown in Table 3 below.

<Table 3>

State set Justice Denial of Service Attack / Distributed Denial of Service Attack (Dos / DDos) Status Network traffic anomalies caused by Denial of Service (DoS) / Distribute Denial of Service (DDoS) Worm Status by Worm Virus Worm series (Codered, slammer, Witty) Worm network network abnormal status Unknown state Unexpected network traffic anomalies Clear state Network traffic steady state

Hereinafter, embodiments of the present invention will be described in detail with reference to the accompanying drawings.

1 is a diagram illustrating a process of analyzing a network traffic of the present invention and applying a hidden Markov model to display a state set as a time series probability.

In step S110, network traffic is input.

Step S120 is a step of classifying time series network traffic characteristics by analyzing the input network traffic.

The input network traffic is divided into packets and classified into network traffic characteristics having 12 characteristics in a unit time in accordance with the above-described definition of each network traffic characteristic.

Since the above 12 network traffic characteristics are characteristics having a time series characteristic measured at any moment of traffic, the network traffic characteristic values are extracted every unit time.

That is, at every unit time, a specific threshold value determined as an abnormality of the network on the network is set, and it is determined whether or not the time series network traffic characteristics described above exceed the specific threshold value. In this case, exponential smoothing method was used among time series prediction algorithms. Exponential smoothing is a technique well known to those skilled in the art, so a detailed description thereof will be omitted.

Step S130 is to derive the observation set from the network traffic characteristics.

The method to derive the observation set from the 12 network traffic characteristics is as follows.

In order to obtain the state set from the observation set in the hidden Markov model, the observation set is highly related to the state set and must be directly observable from the state set through the emission probability matrix. However, although the observation set defined in the present invention is highly related to the state set, it cannot be directly observed from network traffic.

For example, Port Scannjng, the observation set of the present invention, can be judged by various situations on traffic, but cannot directly determine that Port Scannjng is generated by directly observing network traffic. Therefore, there is a need for network traffic characteristics capable of observing directly from network traffic and deriving the observation set of the present invention.

 In other words, the observation set of the present invention ({Port scanning, IP Scanning, IP Spoofing, Packet Spoofing, Sudden Increase, Reflection}) is actually the traffic information itself is not observable from the network, in the present invention to determine such observation set With the network traffic characteristics directly obtained from the traffic information, the observation set is derived from the network traffic characteristics based on the correlation between the network traffic characteristics and the observation set.

The following Table 4 shows the correlation between the traffic characteristics of the network and the observation set of the present invention.

<Table 4>

Observation set Network Traffic Characteristics A) Port Scanning 2.3.4.5.6.7.8.9 B) IP Scanning 1.3.4.5.6.7.8.10 C) IP Spoofing (Black IP) 1.10.11 D) Packet Spoofing (Checksum error) 3.5.6.7 E) Sudden Increase 2.3.4.5.10 F) Reflection 2.7.12

When each of the 12 network traffic characteristics occurs, the number of occurrences of the corresponding network traffic characteristics per unit time is counted, and the observation set is derived from the network traffic characteristics generated per unit time using Table 4 above.

For example, suppose that events of network traffic characteristics 2, 3, 4, and 5 occur within a unit time. This means that observation set A or E has occurred. At this time, if the network traffic characteristics 10 times occur within the same unit time, it can be determined that there was an attack of observation set E on the network within the same unit time.

The process of deriving observation sets from 12 network traffic characteristics is expressed as follows.

를 다음 수식과 같이 계산하고, First, the probability of each observation set

Calculate the following formula,

는 네트워크 트래픽 특성의 발생유무이고 {0,1}의 값을 갖는다.here,

Is the presence or absence of network traffic characteristics and has a value of {0,1}.

는 각 네트워크 트래픽 특성에 부과되는 가중치이고

의 범위를 갖는다. 이 가중치는 각 네트워크 트래픽 특성, 관측집합의 중요도를 고려하여 부과한다.here,

Is the weight imposed on each network traffic characteristic

Has a range of. This weight is applied considering the characteristics of each network traffic and the importance of the observation set.

는 각 관측집합 Y에 배정된 네트워크 트래픽 특성의 개수이다. here,

Is the number of network traffic characteristics assigned to each observation set Y.

를 계산한 결과가, 해당 단위시간의 관측집합이 된다. 단위시간마다 위 과정을 수행한 결과가 관측집합이 된다.In other words,

The result of the calculation is the observation set of the unit time. The result of performing the above process every unit time becomes the observation set.

Step S140 is a process of deriving a state set by applying a hidden Markov model to the observation set derived in step S130.

A method of deriving a state set using the hidden Markov model from the observation set will be described in detail with reference to FIG. 3.

In step S150, the state set derived in step S140 is displayed as a time series probability value according to each state string.

In the original hidden Markov model, the value of the state set is calculated and the state with the largest value (probability value) is determined as the state occurring in unit time.

However, in the present invention, instead of determining the state of the network traffic in unit time as one state, the probability of generating a state string of each state set is displayed so as to optimize the hidden markoff decision process for a probabilistic anomaly detection method. .

2 is a diagram illustrating a process of applying a hidden Markov model to the present invention.

To determine the network state, we put the predicted state of the network in state set N. Here are four features:

N = {DoS / DDoS, Worm, Unknown, Clear}

In order to determine the prediction of the traffic condition of the network, '12 network traffic characteristics' are extracted from the network traffic and placed as observation set M.

M is

M = {Port scanning, IP Scanning, IP Spoofing, Packet Spoofing, Sudden Increase, Reflection}

The process of deriving the state set from the observation set in the hidden Markov model uses the Decision Matrix of the hidden Markov model, where the decision matrix is the emission probability, the state transition probability, and the initial probability.

State set N and observation set M are applied to all prediction matrices as information for prediction. Unique stochastic information should be additionally given in network units of each matrix as follows.

행렬인 상태천이(state transition) 행렬 S는 다음과 같다. The probability of going from one state to the next

A state transition matrix S that is a matrix is

행렬인 방출분포행렬(emission distribution matrix) T는 다음과 같이 정의 된다. In addition, as the probability distribution that each attack can occur among the signs assigned to each state in each state.

The emission distribution matrix T, which is a matrix, is defined as follows.

행렬인 초기확률 ∏는 아래와 같다. It also represents the probability distribution that each state is an initial state to indicate which attack originates from the network from which observations.

The initial probability 인, which is a matrix, is

In order to apply the hidden Markov model to the present invention, first, a probability of a state transition path is calculated in an observation sequence.

In Fig. 2, the state string at observation time k is indicated by a circle. In addition, the path of change of the state from the observation time k to the observation time k + 1 is indicated by an arrow.

을 통해 상태열 Q=q_1,q_2,……,q_K+1 을 얻을 수 있다. Observation column O = O _1, O ₂ ,. … , O _K and Hidden Markov Models

Through the status string Q = q _1, q _2, ... … , q _{K + 1} can be obtained.

The route thus obtained can predict the attack on the network (that is, the probability of a set of states representing the state of the network).

를 최적화 하면 가장 큰 P(O|H)를 만드는 은닉마코프 모델

를 얻을 수 있다. However, an optimization process is needed to increase the probability of the result obtained. Hidden Markov Models through Optimization Process

Hidden Markov Models Create Optimal P (O | H)

Can be obtained.

The optimization process to produce the optimal hidden Markov model is as follows.

The state transition matrix S is the entire observation sequence O = O ₁ O ₂ . … The relative ratio of the number of moves to state q _j during .O _K and the number of times in state q _j is calculated as:

,

이다.here,

,

to be.

인 횟수와, 전체 관측열 동안 상태 q_i에 있는 횟수의 비로 아래와 같이 계산된다. The emission distribution matrix E has a state q _i at time K and the m observed feature at the same time.

The ratio of the number of times and the number of times in state q _i during the entire observation sequence is calculated as:

이다.here,

to be.

Through this optimization process, the hidden Markov model is optimized. As iteration progresses, the probability of generating a given observation sequence always increases. In the optimization process, the optimization process ends when there are no changes in the values of the parameters or when the change is small.

Based on the optimized emission probability matrix, the transition probability matrix, and the initial probability matrix, the hidden Markov model that receives the new observation sequence performs the following decision process.

Referring to FIG. 2, the hidden Markov model has the observation string O = O ₁ O ₂ . … Q = Q ₁ Q ₂ ..., Including the most likely state to produce .O _K and the prediction of the next column. … . Q _K Q _{K + 1} The decision process of the hidden Markov model takes into account the state transition probability and the emission probability, and calculates the probability of each state for the corresponding observation sequence as shown in the following equation.

는 k 번째 관측시간에서의 상태n 의 확률이다.) (here,

Is the probability of state n at the kth observation time.)

(단, k=1,2,…,K 그리고 n=1,2,…,N)가 최대인 상태열을 취득한 뒤, 관측시간(k)마다 가장 큰 값을 갖는

을 선정하여 확률 경로(path)를 결정한다. In the original hidden Markov model

(Where k = 1,2,…, K and n = 1,2,…, N) have the maximum value and then have the largest value for each observation time (k).

To determine the probability path.

을 표기한다. However, in the present invention, in order to optimize the hidden markoff decision process in the stochastic anomaly detection system, the maximum in each state string in each state string is optimized.

Indicate

For example, it is assumed that the hidden Markov model is applied to indicate the current weather condition in the case of weather.

In this case, if the original hidden Markov model is applied, the current weather conditions are 30% more likely to rain, 40% more likely to be sunny, and 30% less likely to know the current weather conditions. Confirm that it is sunny and sunny. But in reality it may be raining.

When the present invention is applied to the above-described weather example, the present invention displays all the probabilities of each state of the weather. In other words, instead of definitely presenting the current state, all possibilities that may exist in the current state are displayed as probabilities so that a person who is proficient in forecasting the weather can make a more accurate judgment.

3 is a diagram illustrating a process of deriving a state set from an observation set by applying a hidden Markov model of the present invention.

Figure 3 is a simplified illustration of the traffic state of a time series network using the hidden Markov model of the present invention as one embodiment of the present invention. In practice, the observed set of sequences can be observed continuously and randomly during the traffic analysis time.

For convenience of explanation, it is assumed as follows.

The probability of state transition is

The emission probability matrix is

It is assumed that the initial probability matrix is

At this time, it is assumed that the observation set sequence is determined as {C, D} by deriving the observation set from the above-described network traffic characteristics.

In practice, the observation set is {C, D, A, E, D,... } Is randomly repeated according to the observation time, but for convenience of explanation, it is assumed that the observation set sequence is {C, D}.

Using the hidden Markov model, the state sequence of the network during the unit time in which the observation set sequence {C, D} is observed is expressed as a probability value as follows.

The network traffic conditions during the unit time in which observation set C is observed are represented by the following four states.

First, the probability of DoS / DDoS is [initial probability value of DoS / DDoS (0.062)] * [emission probability matrix value of C to DoS / DDoS (0.105)] = 0.0065

Second, the probability of Worm is [Worm's initial probability value (0.103)] * [C's emission probability matrix from Worm (0.096)] = 0.0098

Third, the probability of Unknown is [Initial probability value of Unknown (0.0427)] * [Emission probability matrix value from C to Unknown (0.266)] = 0.1135

Fourth, the probability of Clear is [initial probability value of Clear (0.406)] * [Emission probability matrix value from C to Clear (0.101)] = 0.041

In addition, the probability of DoS / DDoS when the observation set sequence is observed from C-> D among the network traffic conditions during the unit time in which the observation set D is observed is calculated as follows.

The probability that observation set D is DoS / DDoS is the probability value obtained by calculating the state transition probability value from DoS / DDoS to DoS / DDoS (0.0065 * 0.444 = 0.002886) and the probability value obtained by calculating the state transition probability value from Worm to DoS / DDoS. (0.0098 * 0.104 = 0.0010192) and the probability value obtained by calculating the state transition probability value from Unknown to DoS / DDoS (0.1135 * 0.343 = 0.0389305) and the probability value calculated by calculating the state transition probability value from Clear to DoS / DDoS (0.041 * 0.122) = 0.005002) plus (0.002886 + 0.0010192 + 0.0389305 + 0.005002 = 0.047838).

In this case, when the observation set sequence is observed as C-> D, the probability that the observation set D is DoS / DDoS during the unit observation time is the probability that the observation set D is DoS / DDoS (0.047838) to D-> C. It is the product of emission probability matrix multiplied by 0.109 (0.047838 * 0.109 = 0.005214342).

Therefore, the probability value that the state of network traffic during each observation time when the observation set sequence is {C, D} is DoS / DDoS is {0.065, 0.005214342}.

In the same manner as described above, when the observation set sequence is {C, D}, the probability values of Worm, Unknown, and Clear of network traffic during each observation time are {0.0098, 0.0017}, {0.11, 0}, {0.041, 0.003}.

In the above example, the observation set sequence at two observation times is {C, D}. However, in the case where there are several observation times, that is, the observation set sequence is randomly repeated {A, C, D , B, A, E, F, C, B, F,... … … It is obvious that the same can be applied to}.

Figure 4 is a block diagram of an apparatus for detecting the presence of network traffic abnormality of the present invention.

Apparatus 400 for detecting the presence of abnormal network traffic of the present invention is configured as follows.

Sign extraction unit 410 and the indication determination unit 420 is composed of a sign extraction unit 410 extracts the abnormal signs of the network. The indication determination unit 420 determines the traffic state of the network from the abnormal indication of the network.

The symptom extractor 410 includes a traffic input unit 412, a packet processing unit 414, a log recording unit 416, and a network traffic characteristic output unit 418.

The traffic input unit 412 is where network traffic is input.

The packet processor 414 analyzes the input network traffic as a packet. The packet processor 414 calculates network traffic characteristics as statistical data by dividing the traffic packets into packets that meet the above-described definition of the network traffic characteristics.

The log recording unit 416 is a place for recording the statistical data of the calculated network traffic characteristics in a log.

The network traffic characteristic output unit 418 outputs the calculated network traffic characteristic to the symptom determination unit 420.

The indication determination unit 420 includes a network traffic characteristic input unit 422, an observation set derivation unit 424, an HMM (hidden Markov model) processing unit 426, and a network state probability display unit 428.

The network traffic characteristic input unit 422 is where the network traffic characteristic transmitted from the symptom extractor 410 is input.

The observation set deriving unit 422 derives the observation set from the network traffic characteristics.

The HMM processing unit 426 is a place for deriving a state set using a hidden Markov model from the observation set derived through the observation set deriving unit 424.

The network state probability display unit 428 is a place for displaying each state of network traffic derived through the HMM processing unit 426 as a probability value for a unit time.

5 shows the result of applying the present invention to normal network traffic.

The result of 510 shows in time series each probability of four sets of state sets {DoS / DDoS, Worm, Unknown, Clear}. Overall, the normal network traffic state of 'Clear' is up to 25.5%, an average of 16.37%, and the time when the probability of 'Worm' is increased is locally distributed. The probability of occurrence of 'Worm' is 15.3% on average, and the probability of 'DoS / DDoS' and 'Unknown' states is very small at 13.0% and 15.3% on average. Overall, the probability of 'Clear' is high.

520 results show the results of applying the present invention to network traffic exposed to DoS attacks.

The 520 results show that the overall probability distribution decreases the fluctuation of the traffic itself. In the case of DoS, the probability of 'DoS / DDoS' is high.

530 results show the results of applying the present invention to network traffic exposed to Slammer worm attack.

The 530 results show a maximum of 33.9% warning in the 20s. In addition, there was an unstable characteristic in the initial state, which was analyzed because 'F' (Reflection) learned from Normal Traffic was generated continuously in the observation set sequence. In general, the probability of 'Worm' is the majority, and the probability of occurrence of 'Worm' increases in the early 20s.

It will be apparent to those skilled in the art that various modifications and variations can be made in the present invention without departing from the spirit or scope of the invention as defined in the appended claims. It will be understood that the invention may be varied and varied without departing from the scope of the invention.

1 is a diagram illustrating a process of analyzing a network traffic of the present invention and applying a hidden Markov model to display a state set as a time series probability.

2 is a diagram illustrating a process of applying a hidden Markov model to the present invention.

3 is a diagram illustrating a process of deriving a state set from an observation set by applying a hidden Markov model of the present invention.

Figure 4 is a block diagram of an apparatus for detecting the presence of network traffic abnormality of the present invention.

5 shows the result of applying the present invention to network traffic.

<Description of the symbols for the main parts of the drawings>

400; Traffic abnormality detection device

410: sign extraction unit

412: traffic input unit

414: packet processing unit

416: log record

418: network traffic characteristic output unit

420: indication judgment part

422: network traffic characteristic input unit

424: observation set derivation unit

426: HMM processing unit

428: network state probability display unit

Claims (11)

In the method of determining the abnormality of network traffic, a) inputting the network traffic; b) deriving network traffic characteristics by analyzing the network traffic; c) selecting an observation set from the network traffic characteristics; d) setting a traffic state of the network expected from the observation set as a state set; e) applying a decision matrix of a Hidden Markov Model to the observation set and the state set and displaying all the time series state sequences of the state set as probabilities; , The network traffic characteristics may include destination IP usage, number of ports, packet arrival rate, payload size, payload size variation, and port. Port number variation, Session time, Payload size per session, Number of Sessions per port, Source Destination IP ratio, At least one of a destination IP distribution and reflection traffic, The state set includes Denial of Service (DoS) / Distributed Denial of Service (DDoS), Worm by Worm Virus, Unknown, and Clear. And at least one of the network traffic conditions of the network. The method of claim 1, Step b) is to determine whether there is an abnormality in the network to derive the network traffic characteristics by analyzing the network traffic in packet units. delete The method of claim 1, The observation set includes Port Scanning, IP Scanning, IP Spoofing (Black IP), Packet Spoofing (Checksum error), Sudden Increase, and Reflection ( Method of determining the abnormality of the network including at least one of (Reflection). The method of claim 1, And determining the abnormality of the network selecting the observation set by examining the number of occurrences of the network traffic characteristics during a unit time based on the association between the network traffic characteristics and the observation sets. delete In the device for determining the abnormality of network traffic, A symptom extraction unit for analyzing the network traffic and outputting network traffic characteristics; And A symptom judging unit which outputs each state of time series of a network as a probability value by using the network traffic characteristic output from the symptom extracting unit as an input value, The network traffic characteristics may include destination IP usage, number of ports, packet arrival rate, payload size, payload size variation, and port. Port number variation, Session time, Payload size per session, Number of Sessions per port, Source Destination IP ratio, At least one of a destination IP distribution and reflection traffic, The state set includes Denial of Service (DoS) / Distributed Denial of Service (DDoS), Worm by Worm Virus, Unknown, and Clear. And at least one of said network traffic conditions. The method of claim 7, wherein The symptom extraction unit A traffic input unit to which the network traffic is input; A packet processing unit which processes the input network traffic by packet unit and checks whether there is an abnormality of the network traffic; A log recording unit for recording an abnormality of the network due to the processed packet into a log file; A network traffic characteristic output unit for analyzing the packet unit for a unit time and outputs a statistical data value for each unit time as the network traffic characteristic. Device for determining whether there is an abnormality in the network. delete The method of claim 7, wherein The indication judgment part A statistical data input unit which takes a statistical data value of network traffic characteristics output from the symptom extractor as an input value; An observation set derivation unit for deriving the observation set using the input statistical data; A Hidden Markov Model (HMM) processing unit for deriving the state set by applying the Hidden Markov model from the observation set; And State probability display unit for displaying all the state strings of the state set derived through the hidden Markov model processing unit as a probability Device for determining whether there is an abnormality in the network. A recording medium for recording a method of determining whether there is an abnormality in network traffic, Inputting the network traffic; Analyzing the network traffic to derive network traffic characteristics; Selecting an observation set from the network traffic characteristics; Setting a traffic state of a network expected from the observation set as a state set; Applying a decision matrix of a hidden markov model to the observation set and the state set and displaying all time series state sequences of the state set as probabilities; The network traffic characteristics may include destination IP usage, number of ports, packet arrival rate, payload size, payload size variation, and port. Port number variation, Session time, Payload size per session, Number of Sessions per port, Source Destination IP ratio, At least one of a destination IP distribution and reflection traffic, The state set includes Denial of Service (DoS) / Distributed Denial of Service (DDoS), Worm by Worm Virus, Unknown, and Clear. And a method for determining whether there is an abnormality in the network, wherein the network traffic state includes at least one of the network traffic conditions.

Images