KR20190066741A - System for performing anomaly detection using traffic classification - Google Patents

Abstract

According to an embodiment of the present invention, a service traffic classifying device comprises: a flow processing unit for obtaining flow information from data packets transmitted and received via a network by a server providing a plurality of application services; a service defining unit for receiving network information of each of the application services and defining service-flow correspondence information for each of the application services based on the network information; and a traffic analyzing unit for classifying and generating traffic statistics information for data packet communication for each of the application services based on the flow information and the service-flow correspondence information.

Description

TECHNICAL FIELD [0001] The present invention relates to a system for performing anomaly detection using traffic classification,

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to an abnormality detection system for effectively detecting whether an abnormality occurs in a server system including a plurality of servers for providing an application service in a data center, .

In order to maintain the computerized system and the services provided by the computerized system, it is indispensably required to quickly and accurately detect whether or not an abnormality occurs in the system and the cause of the abnormality. Failure to detect and resolve such anomalies quickly can lead to serious disruptions in the operation of the system and in the provision of services.

In recent years, a server system for providing various application services including a plurality of physical or logical servers, such as a cloud server system, has become popular. In such a server system, various service providers can provide each application service through the server system. Each application service can be serviced using one or more of the hosts, which are servers included in the server system.

In the server system as described above, the provider of each application service may want to monitor the transmission status of data associated with its service. However, in the case of the above-described cloud environment, a plurality of servers belonging to the server system may share one physical port of a switch or a router for data transmission. In addition, since there can exist a plurality of logical servers implemented in a single physical server through a virtual machine (VM), the number of services or hosts sharing the one physical port can be further increased.

In such a situation where a very large number of application services or hosts transmit data through one physical port, it is difficult to monitor the data transmission status by service or host as a method of monitoring the packet transmission statistics of the physical port . Also, when an abnormal symptom of the server system occurs such as a surge or a sudden decrease in traffic, it is also difficult to find a service or a host that caused the abnormal symptom and to deal with the problem.

Korean Registered Patent Publication No. 10-1661743 (issued October 10, 2016)

SUMMARY OF THE INVENTION It is an object of the present invention to provide a method and system for providing an application service through a server system constituted by a plurality of servers by effectively detecting whether or not an error has occurred in the server system, And to provide a device and system for performing the method.

It is to be understood, however, that the present invention is not limited to the above-mentioned embodiments, but other and further objects which need not be mentioned can be clearly understood by those skilled in the art from the following description will be.

A service traffic classification apparatus according to an embodiment of the present invention includes a flow processor for obtaining flow information from a data packet transmitted or received through a network by a server system that provides a plurality of application services, A service definition unit for receiving network information of a service and defining service-flow correspondence information for each of the application services based on the network information; and a service definition unit for determining, based on the flow information and the service- And a traffic analysis unit for classifying and generating traffic statistical information for each application service.

The system for abnormality detection through traffic classification according to an embodiment of the present invention is characterized in that flow information of a data packet transmitted or received within a predetermined period by a server system providing a plurality of application services, A service traffic classifying unit for classifying traffic statistical information related to data packet communication by each application service using information-based service-flow correspondence information, and a data traffic classifying unit for classifying data packets transmitted / And a second part of a data packet transmitted and received before the reference time point, for each of the application services, and for determining whether the application service is abnormal based on the comparison result, Device It can hamhal.

According to an embodiment of the present invention, traffic information about a specific service or a host can be obtained by classifying flow information of a data packet by application service or host in a server system . As a result, it is possible to monitor traffic by application service or host, and furthermore, it is possible to quickly and accurately find out the cause of abnormality of the server system and the cause of the abnormality through the monitoring. Accordingly, provision of application services and operation and maintenance of the server system can be performed stably.

1 is a diagram for explaining a configuration of an anomaly detection system according to an embodiment of the present invention.
FIG. 2 is a diagram for explaining a service traffic classification method according to an embodiment of the present invention in more detail.
3 is a diagram for explaining traffic statistical information according to an embodiment of the present invention.
4 is a diagram illustrating each step of a service traffic classification method performed by the anomaly detection system according to an embodiment of the present invention.
5 is a diagram illustrating each step of an anomaly detection method performed by an anomaly detection system according to an embodiment of the present invention.

BRIEF DESCRIPTION OF THE DRAWINGS The advantages and features of the present invention and the manner of achieving them will become apparent with reference to the embodiments described in detail below with reference to the accompanying drawings. The present invention may, however, be embodied in many different forms and should not be construed as limited to the embodiments set forth herein. Rather, these embodiments are provided so that this disclosure will be thorough and complete, and will fully convey the scope of the invention to those skilled in the art. To fully disclose the scope of the invention to those skilled in the art, and the invention is only defined by the scope of the claims.

In the following description of the present invention, a detailed description of known functions and configurations incorporated herein will be omitted when it may make the subject matter of the present invention rather unclear. The terms used in the following description are defined in consideration of the functions of the embodiments of the present invention and may vary depending on the subscriber, the intention or custom of the operator, and the like. Therefore, the definition should be based on the contents throughout this specification.

1 is a diagram for explaining a configuration of an anomaly detection system according to an embodiment of the present invention. The abnormality detection system 10 of FIG. 1 monitors a server system (not shown) such as a cloud system including a plurality of physical or logical servers to provide various application services, And may include a service traffic classification apparatus 100 and an anomaly detection apparatus 200. [ However, the anomaly detection system 10 of FIG. 1 is only an embodiment of the present invention, and therefore the spirit of the present invention is not limited to FIG.

First, the service traffic classification apparatus 100 will be described. The service traffic classification apparatus 100 may include a flow processing unit 110, a service defining unit 120, and a traffic analysis unit 130. Each component of the service traffic classification apparatus 100 and an anomaly detection apparatus 200 described later may include a communication function for transmitting or receiving data and a calculation function for reading and processing data. Accordingly, each of the above components may be implemented by including a communication device such as a data bus, a wire / wireless communication module or a data input / output port, and a computing device such as a microprocessor.

The flow processing unit 110 may obtain flow information from data packets received by the server system through the network or transmitted from the server system. In order to perform such a function, the flow processor 110 may receive a duplicate packet generated by mirroring the data packets from a network switch connected to the server system.

The flow processor 110 can obtain the flow information of each packet from the header of each of the duplicate packets. A flow corresponds to a flow of data packets, in other words, a data transmission path through which data packets can be transmitted, and one flow can be connected to one physical or virtual server, i.e., a host, in the server system. That is, obtaining flow information of a certain data packet means knowing what flow (transmission path) the corresponding data packet is transmitted or received, and furthermore, knowing by which host the data packet is transmitted or received It can mean to put out.

The flow information may include a so-called 5-tuple (e.g., tuple) such as source IP address information, destination IP address information, source port information, destination port information, ) Information. In addition, the flow information may include virtual network information such as a virtual LAN (VLAN) or a virtual extensible LAN. That is, the flow information can be any one for specifying a flow in which a data packet is transmitted.

The flow processor 110 may calculate the amount of data packets transmitted for each flow at predetermined intervals and report the result to the service defining unit 120, which will be described later. Specifically, the flow processor 110 can classify data packets transmitted and received within a predetermined period such that data packets having the same flow information, that is, data packets having the same flow key, belong to the same group. The flow processor 110 measures the total data amount or the number of data packets belonging to each group and generates flow statistical information about how much data packets are transmitted and received during the predetermined period by each flow .

The flow processing unit 110 may transmit the flow statistical information to the traffic analysis unit 130. The transmission period may be the same as the predetermined period (for example, 5 seconds), and the period may be appropriately adjusted. If the period is short, more accurate flow statistical information can be generated, but the load of the hardware of the abnormality detection system 10 may be excessive. On the other hand, if the period is long, the flow statistical information may become uncertain, and the flow processor 110 requires a larger amount of memory. Therefore, it is important to set an appropriate period.

The service defining unit 120 can receive the network information of each application service performed by the server system from the service operator and define service-flow correspondence information for each application service based on the network information. The server system may include a plurality of hosts, and each of the application services may be serviced using one or more of the plurality of hosts.

The service defining unit 120 can acquire the IPs, ports, protocols, virtual network information, and the like of the hosts used for each application service from the network information, thereby obtaining the correspondence between each application service and each flow Information-service-flow correspondence information can be defined. That is, the service defining unit 120 provides the service-flow correspondence information to the traffic analyzing unit 130 so that the traffic analyzing unit 130 analyzes the service- It is possible to use the corresponding information.

The traffic analyzing unit 130 maps each group of data packets having the same flow information to each application service by using the flow information, the flow statistical information, and the service-flow corresponding information described above, Can be generated for each application service. The traffic statistic information may include information on how many data packets are transmitted / received for each application service during the predetermined period. The total number of data packets (number of bytes), the number of packets, bps second, and packet per second (pps). The traffic analyzing unit 130 may express the pattern of the amount of data packets transmitted and received within the predetermined period of time according to the time through the traffic statistic information and may transmit the traffic statistical information to a separate database ). ≪ / RTI >

The traffic statistic information generated for each predetermined period may be aggregated to be stored for another period having a longer length. At this time, the shorter the length of the other period, the greater the change in the traffic statistical information, and it is necessary to secure a large capacity of the database, so that the other period also needs to be appropriately selected.

FIG. 2 is a diagram for explaining a service traffic classification method according to an embodiment of the present invention in more detail. Referring to FIG. 2, duplicate packets are input to the flow processor 110 and the flow processor 110 generates flow statistical information (FA, FB, FC) from the duplicate packets PA, PB, . The flow statistical information is generated every predetermined period (t _p ). The FA among the flow statistical information may be generated using a packet corresponding to the PA among the duplicate packets. The FB and FC among the flow statistical information PB, and PC, respectively. The traffic analysis unit 130 can generate traffic statistical information (TS) for each application service from the flow statistical information. It has been described above that the traffic statistic information can be collected according to a longer period than the predetermined period t _p .

Returning to Fig. 1, the abnormality detection device 200 will be described next. The anomaly detection device 200 can detect an anomaly of the server system using the traffic statistical information of each application service generated by the service traffic classification apparatus 100 and analyze the cause of the anomaly. The anomaly detection device 200 may include a service anomaly detection unit 210, a flow anomaly detection unit 220, and a connection analysis unit 230.

The service abnormality detection unit 210 detects a surge or a sudden decrease in traffic by using the traffic statistical information stored in the database by the traffic analyzing unit 130 of the service traffic classifying apparatus 100, It can be judged. Specifically, the service anomaly detection unit 210 can acquire the traffic statistic information every predetermined period (for example, one minute), and the currently measured portion (first portion) of the traffic statistical information for each application service By comparison with the measured portion (second portion), if it is determined that the current traffic statistic information does not match the trend of the past statistical information, it can be determined that there is an abnormal symptom associated with the application service. In addition, the service abnormality detection unit 210 may generate an alarm when an abnormal symptom occurs.

3 is a diagram for explaining traffic statistical information according to an embodiment of the present invention. Referring to FIG. 3, it can be seen that there are two graphs (A, B) showing traffic statistic information for two application services with respect to time.

3 (A), it can be seen that the traffic before the predetermined reference time point (t _r ), that is, the value of the traffic already measured in the past (corresponding to the second part described above) is substantially constant at all times have. These graphs can appear in application services that continue to work regardless of the user's lifestyle, such as CCTV. Accordingly, the traffic value I (_) _t after the reference time t _r that can be expected through the past traffic can also be estimated as shown in FIG. 3A. However, the actually measured current traffic value (I _t , corresponding to the first part described above) may be significantly lower than the value of the expected traffic I (_) _t .

In contrast, in FIG. 3B, it can be seen that the traffic before the reference time t _r , that is, the traffic value measured in the past, fluctuates with time similar to a sine wave. Such a graph may appear in an application service related to a user's life pattern, such as a streaming service or a navigation service. Accordingly, the traffic value I (_) _t after the reference time t _r that can be expected through the past traffic can also be estimated as shown in FIG. 3 (B). However, the actually measured current traffic value I _t may be significantly higher than the estimated traffic value I (_) _t .

In this way, the absolute value of the difference (d) between the values of the estimated traffic by past trends (I (_) _t) and the actually measured value of the current traffic (I _t) exceeds a predetermined threshold value (W) , The service anomaly detection unit 210 can determine that an abnormal symptom has occurred in the application data in which the phenomenon is observed.

In the meantime, the example of FIG. 3 is an illustrative diagram, and various prediction models may be used to estimate the estimated traffic value I () _t . As such a prediction model, a moving average model, a least square regression model, or the like can be used. In addition, as described with reference to the two graphs (A) and (B) in FIG. 3, the characteristics of trends of the traffic statistical information may vary from one application service to another. Therefore, Can be set.

1, if the service anomaly detection unit 210 detects an abnormal symptom for a specific application service, the flow anomaly detection unit 220 may detect the abnormal application from the service anomaly detection unit 210 to the specific application service A specific host that is to serve a specific application) and receives and analyzes only the statistical information of the flow corresponding to the key information from the flow processor 110, thereby detecting an abnormal flow suspected of the occurrence of an abnormality . That is, the flow anomaly detection unit 220 can narrow the range of the anomaly detection result performed by the service anomaly detection unit 210 at the application service level to the flow level.

That is, the flow anomaly detection unit 220 extracts flows corresponding to the key information from among a plurality of flows, and based on information on packet transmission / reception statistics for each of the extracted flows, It is possible to detect an abnormal flow which is judged to be abnormal. Specifically, abnormality detection at such a flow level is performed by measuring metrics such as the amount (in units of bps), the number (in units of pps), and the transmission duration (duration) of data packets transmitted for each flow, And comparing each of the indexes with the predefined threshold value.

The extracted information about each flow can be stored in a separate database in real time for use by the flow anomaly detection unit 220. The real time storage of the information can be performed only for the extracted flow, System resources of the anomaly detection system 10 can be saved. The flow anomaly detection unit 220 may provide the key information of the detected abnormal flow to the connection analysis unit 230. The key information of the abnormal flow may include source / destination IP address information, source / destination port information , Virtual network information, and the like.

The connection analysis unit 230 can more specifically identify the above root cause by using the abnormality detection result at the flow level performed by the flow anomaly detection unit 220. [ That is, in connection with the abnormal flow, the connection analyzing unit 230 can obtain detailed information on the cause of the abnormality by analyzing the performance of the session connection related to the transmission of the data packet transmitted / received through the abnormal flow. For example, the connection analysis unit 230 determines whether a problem occurs in the application side of the host or the client related to the abnormal flow, or whether a problem occurs in the network between the host and the client, The root cause of the outbreak can be diagnosed. In addition, the connection analyzing unit 230 may provide the diagnosis result of the cause of the abnormality in the form of a notification.

In order to perform the above operation, the connection analysis unit 230 may analyze the network session for the duplicated packet in which the data packet transmitted / received through the abnormal flow is mirrored. Through the analysis, the connection analyzing unit 230 analyzes parameters such as connection per second (CPS), response time, and retransmission rate in the data packet transmission through the abnormal flow By performing the measurement for the cause of the abnormality. For example, in the case of a TCP session, the number of zero window occurrences can be checked to see if there is a problem on the client side. In the case of an HTTP session, the state code is analyzed. If a 500 error occurs, .

4 is a diagram illustrating each step of a service traffic classification method performed by the anomaly detection system according to an embodiment of the present invention. And FIG. 5 is a diagram illustrating steps of an anomaly detection method performed by the anomaly detection system according to an embodiment of the present invention. 4 and 5, the operation of the anomaly detection system 10 according to an embodiment of the present invention can be schematically illustrated.

First, the service traffic classification method of FIG. 4 will be described. First, flow information can be obtained from a data packet through the flow processor 110 (S110). The service-flow correspondence information may be defined for each application service through the service defining unit 120 (S120). Lastly, the traffic analyzing unit 130 may generate traffic statistical information for each application service based on the information (S130).

Next, the abnormality detection method of FIG. 5 will be described. First, an abnormality of the application service can be detected using the traffic statistical information through the service abnormality detection unit 210 (S210). An abnormality of the flow corresponding to the abnormal application service can be detected through the flow abnormality detecting unit 220 (S220). Finally, the connection state of the abnormal flow is analyzed through the connection analysis unit 230 (S230), and the root cause can be diagnosed (S240).

Combinations of each step of the flowchart and each block of the block diagrams appended to the present invention may be performed by computer program instructions. These computer program instructions may be embedded in an encoding processor of a general purpose computer, special purpose computer, or other programmable data processing apparatus, so that the instructions, performed through the encoding processor of a computer or other programmable data processing apparatus, Thereby creating means for performing the functions described in each step of the flowchart. These computer program instructions may also be stored in a computer usable or computer readable memory capable of directing a computer or other programmable data processing apparatus to implement the functionality in a particular manner so that the computer usable or computer readable memory It is also possible for the instructions stored in the block diagram to produce a manufacturing item containing instruction means for performing the functions described in each block or flowchart of the block diagram. Computer program instructions may also be stored on a computer or other programmable data processing equipment so that a series of operating steps may be performed on a computer or other programmable data processing equipment to create a computer- It is also possible that the instructions that perform the processing equipment provide the steps for executing the functions described in each block of the block diagram and at each step of the flowchart.

Also, each block or each step may represent a module, segment, or portion of code that includes one or more executable instructions for executing the specified logical function (s). It should also be noted that in some alternative embodiments, the functions mentioned in the blocks or steps may occur out of order. For example, two blocks or steps shown in succession may in fact be performed substantially concurrently, or the blocks or steps may sometimes be performed in reverse order according to the corresponding function.

The above description is merely illustrative of the technical idea of the present invention, and various modifications and changes may be made by those skilled in the art without departing from the essential characteristics of the present invention. Therefore, the embodiments disclosed in the present invention are intended to illustrate rather than limit the scope of the present invention, and the scope of the technical idea of the present invention is not limited by these embodiments. The scope of protection of the present invention should be construed according to the following claims, and all technical ideas within the scope of equivalents thereof should be construed as falling within the scope of the present invention.

According to an embodiment of the present invention, it is possible to monitor traffic by application service or host, and furthermore, it is possible to quickly and accurately find out the cause of abnormality of the server system and the cause of the abnormality through the monitoring. Accordingly, provision of application services and operation and maintenance of the server system can be performed stably.

10: Abnormal detection system
100: service traffic classifier
110:
120: service definition section
130: Traffic analysis unit
200: Abnormal detection device
210: service abnormality detection unit
220: Flow abnormality detection unit
230: Connection Analysis Unit

Claims (9)

A flow processor for acquiring flow information from a data packet transmitted or received through a network by a server system providing a plurality of application services;
A service defining unit for receiving network information of each application service and defining service-flow correspondence information for each application service based on the network information; And
And a traffic analyzer for classifying and generating traffic statistical information related to data packet communication for each application service based on the flow information and the service-flow correspondence information
Service traffic classification device. The method according to claim 1,
The flow processor obtains the flow information from a header of a mirrored replication packet of the data packet,
The flow information includes at least one of a source IP address information, a destination IP address information, a source port information, a destination port information, a protocol information, and a virtual network information At least one
Service traffic classification device. The method according to claim 1,
Wherein the flow processor classifies the data packets so that the same data packets belong to the same group as the data packets transmitted and received within a predetermined period and calculates a total data amount or number of data packets belonging to each group In addition,
The traffic analyzing unit maps each group to the application service based on the flow information and the service-flow correspondence information, and generates the traffic statistical information for each application service
Service traffic classification device. The method according to claim 1,
The traffic analyzing unit generates, as the traffic statistical information, a change pattern of the total data amount or the number of data packets transmitted and received within the predetermined period, over time for each application service
Service traffic classification device. The method according to claim 1,
The server system is a cloud server system,
Wherein each of the application services is served using one or more hosts among a plurality of hosts connected to the flow information on a one-
The traffic analyzing unit analyzes traffic statistics information for each application service based on a result of calculating the total data amount or the number of data packets transmitted within the predetermined period on the basis of the flow information connected to the serving host, To generate
Service traffic classification device. A method of controlling traffic statistics for data packet communication using flow information of a data packet transmitted or received within a predetermined period by a server system providing a plurality of application services and service-flow correspondence information based on network information of each application service, A service traffic classifying unit for classifying and generating information for each application service; And
A comparison is made between a first part of a data packet transmitted and received after a predetermined reference time point of the traffic statistical information and a second part of a data packet transmitted and received before the reference time point for each application service, And an abnormality detection device for determining whether an abnormality has occurred in each of the application services
A system for anomaly detection through traffic classification. The method according to claim 6,
The abnormality detection device may calculate a difference between a predicted data packet transmission amount after the reference time predicted using the second portion and an actual data packet transmission amount after the reference time derived from the first portion, When the difference exceeds a predetermined threshold value, it is determined that an abnormality has occurred
A system for anomaly detection through traffic classification. The method according to claim 6,
Wherein the abnormality detection device extracts a flow corresponding to an application service that is determined to have generated the abnormality among a plurality of flows defined by the flow information and stores information on the number of data packets transmitted and received for each of the extracted flows On the basis of the detected abnormal flow,
A system for anomaly detection through traffic classification. 9. The method of claim 8,
The abnormality detection device acquires detailed information on the cause of abnormality of the abnormal flow, based on the analysis result of the network session for transmission of the data packet transmitted and received through the abnormal flow
A system for anomaly detection through traffic classification.

Images