KR102055363B1 - System for performing anomaly detection using traffic classification - Google Patents

Abstract

An apparatus for classifying service traffic according to an embodiment of the present invention includes a flow processing unit for obtaining flow information from a data packet transmitted or received through a network by a server system providing a plurality of application services, and each of the applications. A service definition unit for receiving network information of a service and defining service-flow correspondence information for each application service based on the network information, and based on the data packet communication based on the flow information and the service-flow correspondence information; It may include a traffic analysis unit for classifying the traffic statistics information for each application service.

Description

System for performing anomaly detection using traffic classification {SYSTEM FOR PERFORMING ANOMALY DETECTION USING TRAFFIC CLASSIFICATION}

The present invention relates to an abnormality detection system for effectively detecting whether an abnormality has occurred in a server system including a plurality of servers for providing an application service in a data center and the cause of the abnormality through traffic analysis. .

In order to maintain the computerized system and the services provided therein, it is essential to quickly and accurately detect whether an abnormality occurs and the cause of the abnormality. Failure to detect and resolve such system abnormalities as soon as possible can result in serious disruption to the operation of the system and the provision of services.

Recently, a server system for providing various application services including a plurality of physical or logical servers, such as a cloud server system, has become popular. In such a server system, various service providers may provide their respective application services through the server system. Each application service may be serviced using one or more of hosts, which are servers included in the server system.

In the server system as described above, the provider of each application service may want to monitor the transmission status of data related to its service. However, in the case of the cloud environment, a plurality of servers belonging to the server system may share one physical port of a switch or a router for data transmission. In addition, since a plurality of logical servers implemented through virtual machines (VMs) may exist in one physical server, the number of services or hosts sharing the one physical port may be increased.

In such a situation where a large number of application services or hosts transmit data through one physical port, it is difficult to monitor data transmission status by service or host by monitoring packet transmission statistics of a physical port. . In addition, even when an abnormal symptom of the server system, such as a sudden increase or decrease in traffic, appears, it is also difficult to find a service or a host that caused the abnormal symptom and cope with the problem.

Korean Registered Patent Publication No. 10-1661743 (August 11, 2016)

The problem to be solved by the present invention, in providing an application service through a server system consisting of a plurality of servers, whether the abnormality in the server system and the service or host causing the abnormality is effectively detected through traffic analysis It is to provide an apparatus and a system for.

However, the problem to be solved by the present invention is not limited to those mentioned above, another problem to be solved is not mentioned can be clearly understood by those skilled in the art from the following description. will be.

An apparatus for classifying service traffic according to an embodiment of the present invention includes a flow processing unit for obtaining flow information from a data packet transmitted or received through a network by a server system providing a plurality of application services, and each of the applications. A service definition unit for receiving network information of a service and defining service-flow correspondence information for each application service based on the network information, and based on the data packet communication based on the flow information and the service-flow correspondence information; It may include a traffic analysis unit for classifying the traffic statistics information for each application service.

According to an embodiment of the present invention, a system for detecting an abnormality through traffic classification includes flow information of a data packet transmitted or received within a predetermined period by a server system providing a plurality of application services, and a network of the respective application services. A service traffic classification device for classifying and generating traffic statistics information about data packet communication for each application service using service-flow correspondence information based on the information, and data packets transmitted and received after a predetermined reference time point among the traffic statistics information. An abnormality detection for performing a comparison between the first part and the second part of the data packet transmitted and received before the reference time point for each application service, and determining whether an abnormality of each application service occurs based on the comparison result. Device It can hamhal.

According to an embodiment of the present invention, in a server system, traffic information of a specific service or a host may be obtained by classifying flow information of a data packet by an application service or by a host. . This enables traffic monitoring by application service or host, and further, through the monitoring, it is possible to quickly and accurately identify and cope with an abnormality of the server system and the cause of the abnormality. Accordingly, it is possible to stably perform application service and server system operation and maintenance.

1 is a view for explaining the configuration of the abnormality detection system according to an embodiment of the present invention.
2 is a diagram for describing a service traffic classification method according to an embodiment of the present invention in more detail.
3 is a diagram for describing traffic statistics information according to an embodiment of the present invention.
4 is a diagram illustrating each step of the service traffic classification method performed by the anomaly detection system according to an embodiment of the present invention.
5 is a diagram showing each step of the abnormality detection method performed by the abnormality detection system according to an embodiment of the present invention.

Advantages and features of the present invention and methods for achieving them will be apparent with reference to the embodiments described below in detail with the accompanying drawings. However, the present invention is not limited to the embodiments disclosed below, but can be implemented in various forms, and only the embodiments are to make the disclosure of the present invention complete, and common knowledge in the art to which the present invention pertains. It is provided to fully inform the person having the scope of the invention, which is defined only by the scope of the claims.

In describing the embodiments of the present invention, if it is determined that a detailed description of a known function or configuration may unnecessarily obscure the gist of the present invention, the detailed description thereof will be omitted. The terms to be described below are terms defined in consideration of functions in the embodiments of the present invention, which may vary according to the intention or custom of the subscriber or the operator. Therefore, the definition should be made based on the contents throughout the specification.

1 is a view for explaining the configuration of the abnormality detection system according to an embodiment of the present invention. The anomaly detection system 10 of FIG. 1 monitors a server system (not shown), such as a cloud system, including a plurality of physical or logical servers to provide various application services, and whether or not an abnormality of the server system occurs. As a system for detecting a cause, the service traffic classification apparatus 100 and the abnormality detection apparatus 200 may be included. However, since the abnormality detection system 10 of FIG. 1 is only one embodiment of the present invention, the spirit of the present invention is not limitedly interpreted by FIG. 1.

First, the service traffic classification apparatus 100 will be described. The apparatus 100 for classifying traffic of services may include a flow processor 110, a service definer 120, and a traffic analyzer 130. Each component of the service traffic classification apparatus 100 and the abnormality detection apparatus 200 to be described later may include a communication function for transmitting or receiving data and an operation function for reading and processing the data. Accordingly, each of the above components may be implemented by including a communication device such as a data bus, a wired / wireless communication module or a data input / output port, and a computing device such as a microprocessor.

The flow processor 110 may acquire flow information from data packets received from the server system or transmitted from the server system through a network. In order to perform such a function, the flow processor 110 may receive a duplicate packet generated by mirroring the data packets from a network switch connected to a server system.

The flow processor 110 may obtain flow information of each packet from the header of each of the duplicate packets. A flow corresponds to a flow of data packets, in other words, a data transmission path through which data packets can be transmitted, and one flow can be connected to one physical or virtual server, that is, a host, in a server system. In other words, obtaining the flow information of a data packet means knowing which flow (transmission path) the data packet is transmitted or received, and further knowing which host the data packet was transmitted or received. It can mean paying.

The flow information includes so-called five-tuples, such as source IP address information, destination IP address information, source port information, destination port information, protocol information, and the like. ) May include information. In addition, the flow information may include virtual network information such as a virtual LAN (VLAN) or a virtual extensible LAN. In other words, the flow information can be anything as long as it is for specifying the flow on which the data packet was sent.

The flow processing unit 110 may calculate a transmission amount of data packets for each flow at predetermined intervals and report the result to the service defining unit 120 to be described later. In detail, the flow processor 110 may classify data packets transmitted and received within a predetermined period such that data packets having the same flow information, that is, data packets having the same flow key, belong to the same group. In addition, the flow processor 110 may generate flow statistics information on how many data packets are transmitted and received during the predetermined period by each flow by measuring the total amount or number of data packets belonging to each group. Can be.

The flow processor 110 may transmit the flow statistics information to the traffic analyzer 130. The period of such transmission may also be the same period as the aforementioned predetermined period (for example, 5 seconds), and the period may be appropriately adjusted. If the period is short, more accurate flow statistics information may be generated, but there is a disadvantage in that an excessive load may be applied to hardware of the abnormality detection system 10. On the other hand, if the period is long, flow statistics information may become uncertain, and there is a disadvantage in that a larger amount of memory is required for the flow processor 110. Therefore, it is important to set an appropriate period.

The service defining unit 120 may receive network information of each application service performed by the server system from a service operator, and define service-flow correspondence information for each application service based on the network information. The server system may include a plurality of hosts, and each of the application services may be serviced using one or more of the plurality of hosts, as described above.

The service defining unit 120 may obtain IP, use port, protocol, virtual network information, etc. of hosts used for each application service from the network information, and thereby correspond to the correspondence between each application service and each flow. Service-flow response information, which is information, can be defined. That is, the service defining unit 120 provides the service-flow correspondence information to the traffic analysis unit 130, so that the traffic analysis unit 130 calculates the traffic statistics information for each application service. Correspondence information can be used.

The traffic analyzer 130 maps each group of data packets having the same flow information to each application service by using the above-described flow information, flow statistics information, and the service-flow correspondence information. Traffic statistics information related to each application service may be classified and generated. The traffic statistics information may include information about how many data packets have been transmitted and received for each application service during the predetermined period, and include the total amount of data (bytes), the number of packets, and bps (bit per bit) of the data packet. Second), pps (packet per second), etc. can be expressed in various ways. The traffic analyzer 130 may express an aspect in which the quantity of data packets transmitted / received within the predetermined period varies with time through the traffic statistics information, and the traffic statistics information may be stored in a separate database (not shown). ) Can be stored.

Traffic statistics information generated every predetermined period may be aggregated to be stored for each other period having a longer length. At this time, the shorter the length of the other period, the more severe the change in the traffic statistics information, and it will be necessary to ensure the capacity of the database, the other period needs to be appropriately selected.

2 is a diagram for describing a service traffic classification method according to an embodiment of the present invention in more detail. 2, duplicate packets are input to the flow processor 110, and the flow processor 110 generates flow statistics information FA, FB, and FC from the duplicate packets PA, PB, and PC. Can be. The flow statistics information is generated every predetermined period (t _p ), FA of the flow statistics information can be generated using a packet corresponding to the PA of the duplicate packet, FB, FC of the flow statistics information of the duplicate packet, respectively It may be generated using a packet corresponding to the PB, PC. The traffic analyzer 130 may generate traffic statistics information TS for each application service from the flow statistics information. As described above, the traffic statistics information may be collected according to a period longer than the predetermined period t _p .

1 again, the abnormality detection apparatus 200 will be described next. The abnormality detection apparatus 200 may detect an abnormality indication of the server system and analyze the cause of the abnormality using the traffic statistics information for each application service generated by the service traffic classification apparatus 100. The abnormality detecting apparatus 200 may include a service abnormality detecting unit 210, a flow abnormality detecting unit 220, and a connection analyzing unit 230.

The service abnormality detection unit 210 detects a sudden increase or decrease of traffic by using the traffic statistics information stored in the database by the traffic analysis unit 130 of the service traffic classification apparatus 100 to determine whether an abnormality occurs for each application service. You can judge. In detail, the service abnormality detection unit 210 may obtain the traffic statistics information every predetermined period (for example, one minute), and the part currently measured (first portion) of the traffic statistics information for each application service has been previously recorded. By comparing with the measured part (second part), if it is determined that the current traffic statistics information does not match the trend of the past statistical information, it may be determined that there are abnormal signs in relation to the corresponding application service. In addition, the service abnormality detection unit 210 may generate an alarm when an abnormal symptom occurs.

3 is a diagram for describing traffic statistics information according to an embodiment of the present invention. Referring to FIG. 3, it can be seen that there are two graphs A and B respectively showing traffic statistics information for two application services with respect to time.

First, referring to FIG. 3A, it can be seen that the traffic before a predetermined reference time point t _r , that is, the value of traffic that has already been measured in the past (corresponding to the second part described above) is substantially constant at all times. have. Such a graph may appear in an application service that continues to operate regardless of a user's life pattern, such as CCTV. Accordingly, the value I (_) _t of the traffic after the reference time point t _r that can be expected through the past traffic can also be estimated as shown in FIG. However, the actually measured value of the current traffic (I _t , corresponding to the first portion described above) may be significantly lower than the expected value of the traffic (I (_) _t ).

On the contrary, in (B) of FIG. 3, it can be seen that the value of the traffic before the reference time point t _r , that is, the traffic value previously measured in the past, fluctuates with time similarly to a sine wave. Such a graph may appear in an application service that is related to a user's life pattern, such as a streaming service and a navigation service. Accordingly, the value I (_) _t of the traffic after the reference time point t _r that can be expected through the past traffic can also be estimated as shown in FIG. 3B. However, the actually measured value of the current traffic I _t may be significantly higher than the expected traffic value I (_) _t .

In this way, the absolute value of the difference (d) between the values of the estimated traffic by past trends (I (_) _t) and the actually measured value of the current traffic (I _t) exceeds a predetermined threshold value (W) In this case, the service abnormality detection unit 210 may determine that an abnormal symptom occurs in the application data in which the phenomenon is observed.

On the other hand, the example of FIG. 3 is a diagram for easy description, and in practice, various prediction models may be used to estimate the value of the expected traffic (I (_) _t ). As such a prediction model, a moving average model, a least square regression model, or the like may be used. In addition, as described with reference to two graphs (A) and (B) in FIG. 3, the characteristics of the trend of the traffic statistics information for each application service may be different. Can be set.

1 again, when the service abnormality detection unit 210 detects an abnormality indication for a specific application service, the flow abnormality detection unit 220 detects an abnormality indication from the service abnormality detection unit 210. By receiving key information of a specific host serving a specific application and receiving and analyzing only statistical information of a flow corresponding to the key information from the flow processing unit 110, an abnormal flow suspected of an abnormality can be detected. . That is, the flow abnormality detection unit 220 may narrow the range of the abnormality detection result performed by the service abnormality detection unit 210 at the application service level to the flow level.

That is, the flow abnormality detection unit 220 extracts flows corresponding to the key information among a plurality of flows, and an abnormality occurs in each of the extracted flows based on the information on packet transmission / reception statistics for each flow. The abnormal flow determined to be detected can be detected. Specifically, the abnormality detection at such a flow level measures and measures metrics such as the amount of data packets (bps), number (pps), transmission duration, etc., transmitted for each flow. Each indicator may be compared with a predefined threshold value.

Information about each of the extracted flows may be stored in a separate database in real time for use of the flow anomaly detection unit 220. The real time storage of such information is performed only for the extracted flows, not all flows. As a result, system resources of the abnormality detection system 10 can be saved. Meanwhile, the flow abnormality detector 220 may provide key information of the detected abnormal flow to the connection analyzer 230, and the key information of the abnormal flow may include source / destination IP address information and source / destination port information. , Virtual network information, and the like.

The connection analyzer 230 may more specifically identify the root cause of the abnormality by using the abnormality detection result at the flow level performed by the flow abnormality detector 220. That is, the connection analyzer 230 may obtain detailed information on the cause of the abnormality by analyzing the performance of the session connection regarding the transmission of the data packet transmitted and received through the abnormal flow with respect to the abnormal flow. For example, the connection analysis unit 230 determines whether there is a problem in the application side of the host or client related to the abnormal flow, or whether the problem occurs in the network between the host and the client. Can diagnose the root cause of occurrence. In addition, the connection analysis unit 230 may provide a diagnosis result for the cause of the occurrence of the abnormality in the form of a notification.

For the operation as described above, the connection analyzer 230 may perform a network session analysis on the duplicated packet mirrored by the data packet transmitted and received through the abnormal flow. Through the analysis, the connection analyzer 230 may determine parameters such as connection per second (CPS), response time, retransmission rate, and the like in the data packet transmission through the abnormal flow. By performing the measurement for the above, it can be used for diagnosing the cause of the abnormal occurrence. For example, in the case of TCP sessions, the number of zero window occurrences can be checked to determine that there is a problem on the receiving side of the client. Can be.

4 is a diagram illustrating each step of the service traffic classification method performed by the anomaly detection system according to an embodiment of the present invention. 5 is a diagram showing each step of the abnormality detection method performed by the abnormality detection system according to an embodiment of the present invention. That is, the operation of the abnormality detection system 10 according to an embodiment of the present invention can be schematically viewed through FIGS. 4 and 5.

First, the service traffic classification method of FIG. 4 will be described. First, flow information may be obtained from a data packet through the flow processor 110 (S110). In addition, service-flow correspondence information may be defined for each application service through the service definition unit 120 (S120). Finally, traffic statistics information may be generated for each application service based on the information through the traffic analyzer 130 (S130).

Next, the abnormality detection method of FIG. 5 will be described. First, an abnormality of an application service may be detected using traffic statistics information through the service abnormality detecting unit 210 (S210). In operation S220, an abnormality of a flow corresponding to the abnormal application service may be detected through the flow abnormality detector 220. Finally, the connection analysis unit 230 may analyze the connection state of the abnormal flow (S230) and diagnose the root cause of the abnormality (S240).

Combinations of each block of the block diagrams and respective steps of the flowcharts attached to the present invention may be performed by computer program instructions. These computer program instructions may be mounted on an encoding processor of a general purpose computer, special purpose computer, or other programmable data processing equipment such that the instructions executed by the encoding processor of the computer or other programmable data processing equipment are not included in each block or block diagram. It will create means for performing the functions described in each step of the flowchart. These computer program instructions may be stored in a computer usable or computer readable memory that can be directed to a computer or other programmable data processing equipment to implement functionality in a particular manner, and thus the computer usable or computer readable memory. It is also possible for the instructions stored in to produce an article of manufacture containing instruction means for performing the functions described in each block or flowchart of each step of the block diagram. Computer program instructions may also be mounted on a computer or other programmable data processing equipment, such that a series of operating steps may be performed on the computer or other programmable data processing equipment to create a computer-implemented process to create a computer or other programmable data. Instructions that perform processing equipment may also provide steps for performing the functions described in each block of the block diagram and in each step of the flowchart.

In addition, each block or step may represent a portion of a module, segment or code that includes one or more executable instructions for executing a specified logical function (s). It should also be noted that in some alternative embodiments, the functions noted in the blocks or steps may occur out of order. For example, the two blocks or steps shown in succession may in fact be executed substantially concurrently, or the blocks or steps may sometimes be performed in the reverse order, depending on the functionality involved.

The above description is merely illustrative of the technical idea of the present invention, and those skilled in the art to which the present invention pertains may make various modifications and changes without departing from the essential quality of the present invention. Therefore, the embodiments disclosed in the present invention are not intended to limit the technical idea of the present invention but to describe the present invention, and the scope of the technical idea of the present invention is not limited by these embodiments. The scope of protection of the present invention should be interpreted by the following claims, and all technical ideas that fall within the scope of equivalents should be construed as being included in the scope of the present invention.

According to an embodiment of the present invention, it is possible to monitor traffic per application service or per host, and furthermore, through the monitoring, it is possible to quickly and accurately find out and cope with the occurrence of an abnormality of the server system and the cause of the abnormality. Accordingly, it is possible to stably perform the provision of the application service and the operation and maintenance of the server system.

10: abnormal detection system
100: service traffic classification device
110: flow processing unit
120: service definition
130: traffic analysis unit
200: abnormal detection device
210: service anomaly detection unit
220: flow anomaly detection unit
230: connection analysis unit

Claims (9)

A flow processing unit for obtaining flow information from data packets transmitted or received through a network by a server system providing a plurality of application services;
A service defining unit which receives network information of each application service and defines service-flow correspondence information used for each application service for calculating traffic statistics information for each application service based on the network information; And
Based on the flow information and the service-flow correspondence information, a traffic analysis unit for classifying and generating traffic statistics information about data packet communication for each application service,
The traffic statistics information is used to determine whether an abnormality of each application service occurs by using a difference between a traffic value after a reference point predicted by using information before a reference point and a traffic value actually measured after the reference point.
Service traffic classification device. The method of claim 1,
The flow processing unit obtains the flow information from a header of a duplicate packet in which the data packet is mirrored,
The flow information includes source IP address information, destination IP address information, source port information, destination port information, protocol information, and virtual network information of the data packet. Containing at least one
Service traffic classification device. The method of claim 1,
The flow processing unit classifies the data packet so that the flow information belongs to the same group among the same data packets with respect to the data packet transmitted and received within a predetermined period, and calculates the total data amount or number of data packets belonging to the respective groups. ,
The traffic analysis unit maps each group to the application service based on the flow information and the service-flow correspondence information to generate the traffic statistics information for each application service.
Service traffic classification device. The method of claim 1,
The traffic analyzer may generate, as the traffic statistics information, a change pattern according to time of the total data amount or the number of data packets transmitted and received within the predetermined period for each application service.
Service traffic classification device. The method of claim 1,
The server system is a cloud server system,
Each application service is serviced using one or more hosts among a plurality of hosts connected one-to-one with each flow information.
The traffic analysis unit may be configured to calculate the traffic statistics information for each application service based on a result of calculating, for each application service, the total data amount or number of data packets transmitted within the predetermined period through flow information connected to a serving host. To generate
Service traffic classification device. Traffic statistics about data packet communication using flow information of data packets transmitted or received within a predetermined period by a server system providing a plurality of application services and service-flow correspondence information based on network information of the respective application services. A service traffic classification apparatus for classifying and generating information for each application service; And
The first measurement value for the data packet transmitted and received during the first time after the predetermined reference time point among the traffic statistics information and the second measurement value for the data packet transmitted and received during the second time before the reference time point are respectively used. An abnormality detection device for determining whether an abnormality of the application service occurs,
The abnormality detection apparatus predicts a data packet to be transmitted and received during the first time using the second measured values, and determines that an abnormality has occurred when a difference between the predicted value and the first measured value exceeds a threshold.
System for anomaly detection through traffic classification. delete The method of claim 6,
The abnormality detection apparatus extracts a flow corresponding to an application service that is determined that the abnormality has occurred among a plurality of flows defined by the flow information, and includes information on the quantity of data packets transmitted and received for each of the extracted flows. On the basis of detecting an abnormal flow that is determined to have occurred in the flow
System for anomaly detection through traffic classification. The method of claim 8,
The abnormality detecting apparatus obtains detailed information on a cause of an abnormality in the abnormal flow based on an analysis result of a network session for transmission of data packets transmitted and received through the abnormal flow.
System for anomaly detection through traffic classification.

Images