WO2021008296A1

WO2021008296A1 - Traffic abnormality detection method and apparatus, network device, and storage medium

Info

Publication number: WO2021008296A1
Application number: PCT/CN2020/096847
Authority: WO
Inventors: 蒋勇; 彭鑫; 叶德忠
Original assignee: 中兴通讯股份有限公司
Priority date: 2019-07-16
Filing date: 2020-06-18
Publication date: 2021-01-21
Also published as: CN112242971A; CN112242971B

Abstract

Embodiments of the present invention provide a traffic abnormality detection method and apparatus, a network device, and a storage medium. The method comprises: acquiring the reception traffic and transmission traffic of each port of a network element device at the current detection moment; then determining a real-time traffic deviation ratio of the network element device in the current time window according to the acquired reception traffic and transmission traffic; determining the abrupt change slope of the current detection moment according to the real-time traffic deviation ratio and a historical traffic deviation ratio; and subsequently, based on the abrupt change slope of the current detection moment, determining whether the current detection moment is a moment when traffic is abnormal. Compared with a traffic monitoring solution in relevant prior art, a traffic abnormality detection solution provided by the embodiments of the present invention can more effectively discover an abnormality that does not make traffic exceed a limit, improve the comprehensiveness of the traffic monitoring of the network element device, and enhance the accuracy and reliability of a detection result.

Description

Method, device, network equipment and storage medium for detecting abnormal flow

Technical field

The present invention relates to the field of communications, and in particular to a method, device, network equipment and storage medium for detecting abnormal flow.

Background technique

With the increasing scale and complexity of communication networks, operators are facing increasing pressure and challenges in network operation and maintenance. Traditional network element equipment abnormality monitoring solutions mainly rely on the monitoring of alarm events. For example, the monitoring of network traffic generally compares port bandwidth utilization and CPU utilization with fixed thresholds set based on manual experience. Determine whether the index values of port bandwidth utilization and CPU utilization are within a reasonable range defined by the corresponding fixed threshold. These methods are effective for monitoring some peak traffic, but it is difficult to find some hidden abnormalities in the network element equipment that affect the quality of service operation, such as a large number of packet loss or a large number of illegally copied packets in the network. When this kind of abnormality occurs, the port traffic may not exceed the limit. Therefore, the traditional traffic monitoring solution cannot perceive this abnormality that does not cause the port traffic to exceed the limit.

Summary of the invention

The flow abnormality detection method, device, network equipment and storage medium provided by the embodiments of the present invention mainly solve the technical problem of: solving the problem that the related flow monitoring solution cannot detect the abnormal flow that will not cause the port flow to exceed the limit.

In order to solve the above technical problems, an embodiment of the present invention provides a method for detecting abnormal traffic, including:

Collect the receiving and sending traffic of each port of the inspected network element at the current inspection moment;

Determine the real-time window deviation ratio of the detected network element in the current time window according to the collected receiving and sending traffic. The current time window is the time window corresponding to the current detection moment. The real-time window deviation ratio is, and the flow deviation ratio can characterize the receiving The degree of balance between traffic and sending traffic;

Determine the steep slope of the current detection time according to the real-time window deviation ratio and the historical window deviation ratio. The historical window deviation ratio is the traffic deviation ratio of the detected network element in the time window corresponding to the previous detection time;

Based on the steep slope of the current detection time, it is determined whether the current detection time is an abnormal flow time.

The embodiment of the present invention also provides a flow abnormality detection device, including:

The traffic collection module is set to collect the receiving and sending traffic of each port of the detected network element at the current detection moment;

The deviation determination module is set to determine the real-time window deviation ratio of the inspected network element in the current time window based on the collected received and sent traffic. The current time window is the time window corresponding to the current detection moment, and the flow deviation ratio can represent the received The degree of balance between traffic and sending traffic;

The slope determination module is set to determine the steep slope of the current detection time according to the real-time window deviation ratio and the historical window deviation ratio. The historical window deviation ratio is the traffic deviation ratio of the detected network element in the time window corresponding to the previous detection time;

The abnormality determination module is configured to determine whether the current detection time is the abnormal flow time based on the steep slope of the current detection time.

The embodiment of the present invention also provides a network device, which includes a processor, a memory, and a communication bus;

The communication bus is set to realize the connection and communication between the processor and the memory;

The processor is configured to execute one or more programs stored in the memory to implement the steps of the method for detecting abnormal flow.

The embodiment of the present invention also provides a computer-readable storage medium, the computer-readable storage medium stores one or more programs, and the one or more programs can be executed by one or more processors to realize the above-mentioned traffic abnormality detection method A step of.

The beneficial effects of the present invention are:

The method, device, network equipment, and storage medium for detecting abnormal traffic provided by the embodiments of the present invention collect the received and sent traffic of each port of the detected network element at the current detection moment, and then determine the received and sent traffic according to the collected received and sent traffic. Check the real-time flow deviation ratio of the network element in the current time window, and determine the steep slope of the current detection time based on the real-time flow deviation ratio and the historical flow deviation ratio, and then determine whether the current detection time is an abnormal flow based on the steep slope of the current detection time . The traffic anomaly detection solution provided by the embodiment of the present invention is based on the fact that the total flow in and out of all ports of the inspected network element is basically balanced under normal working conditions, but when the inspected network element is in data routing and switching processing When abnormalities such as packet loss or illegal duplication occur, the balance of the received and received traffic will be broken. Therefore, the traffic anomaly detection solution provided by the embodiment of the present invention can measure the balance of the received and received traffic of the inspected network element and determine the inspected network element. The moment when the traffic balance changes sharply, so as to detect the moment when the traffic of the detected network element is abnormal. Compared with related traffic monitoring solutions, the traffic anomaly detection solution provided by the embodiments of the present invention can more effectively find those abnormalities that will not cause the traffic to exceed the limit, improve the comprehensiveness of the traffic monitoring of the detected network element, and increase the detection result. Accuracy and reliability.

Other features and corresponding beneficial effects of the present invention are described in the latter part of the specification, and it should be understood that at least some of the beneficial effects will become apparent from the description in the specification of the present invention.

Description of the drawings

FIG. 1 is a flowchart of a method for detecting abnormal traffic according to an embodiment of the present invention;

2 is a schematic diagram of the traffic deviation ratio of the checked network element in one day according to the embodiment of the present invention;

3 is a schematic diagram of a traffic deviation ratio of another checked network element in one day according to an embodiment of the present invention;

4 is a schematic diagram of a relationship between a time window and a detection time according to an embodiment of the present invention;

FIG. 5 is a flowchart of adjusting the normal slope range by a network device according to an embodiment of the present invention;

FIG. 6 is a network device according to an embodiment of the present invention determined to automatically mark an abnormal set;

FIG. 7 is a flowchart of a method for detecting abnormal traffic according to an embodiment of the present invention;

8 is a schematic diagram of a traffic deviation ratio of another checked network element in one day according to an embodiment of the present invention;

FIG. 9 is a schematic structural diagram of a flow abnormality detection device provided by an embodiment of the present invention;

10 is a schematic diagram of another structure of a flow abnormality detection device provided by an embodiment of the present invention;

11 is a schematic diagram of another structure of a flow abnormality detection device provided by an embodiment of the present invention;

FIG. 12 is a schematic diagram of a hardware structure of a network device provided by an embodiment of the present invention.

Detailed ways

In order to make the objectives, technical solutions, and advantages of the present invention clearer, the following further describes the embodiments of the present invention in detail through specific implementations in conjunction with the accompanying drawings. It should be understood that the specific embodiments described here are only used to explain the present invention, but not to limit the present invention.

With the continuous development of communication networks, the pressure on network operation and maintenance has gradually increased. The advanced nature of the traditional operation and maintenance model based on manual labor has been unable to meet the requirements, because the investment in operation and maintenance in the traditional operation and maintenance model is constantly increasing. The network failure rate and the timeliness of failure response processing have not improved.

Traditional traffic monitoring solutions generally set fixed thresholds for indicators such as port bandwidth utilization and CPU utilization based on manual experience. If an indicator of the detected network element port is found to exceed the fixed threshold corresponding to the indicator during the detection process, then It is determined that the monitoring is abnormal, and an alarm can be issued. Obviously, this traffic monitoring method is simply to determine whether the detected index value is normal based on the threshold. This is effective for monitoring the abnormality of the peak traffic exceeding the limit, but if the occurrence of the abnormality does not cause the port traffic to exceed the limit, then This traditional traffic monitoring solution cannot be perceived. For example, if a large number of packet loss or a large number of illegally copied packets occur in the inspected network element, but these abnormalities do not cause the traffic to exceed the limit, the traditional traffic monitoring solution will not identify these abnormalities.

In order to solve the above problem, this embodiment provides a method for detecting abnormal traffic. Please refer to the flowchart shown in FIG. 1, which includes the following steps:

S102: Collect the receiving traffic and the sending traffic of each port of the inspected network element at the current inspection moment.

For an inspected network element, under normal working conditions, the total traffic in and out of all ports should be basically balanced, that is to say, each inspected network element sends traffic in a unicast-based network Roughly the same as the received traffic. When and when the network element under inspection has packet loss or illegal duplication in the process of data routing and exchange processing, the balance of the traffic sent and received by the network element under inspection will be broken.

Therefore, in this embodiment, in the process of detecting abnormal traffic of a detected network element, the receiving traffic and the sending traffic of each port of the detected network element can be collected. For example, suppose that a detected network element has 4 ports, namely port a, port b, port c, and port d. In the process of detecting abnormal traffic on the detected network element, the network device can collect port a Receive traffic and send traffic, collect the receive traffic and send traffic of port b. For port c and port d, the network device will also collect the send and receive traffic.

In some examples of this embodiment, the network device may periodically collect the receiving traffic and sending traffic of each port of the inspected network element during the process of detecting abnormal traffic of a inspected network element. For example, in an example Among them, the network device can use 15 minutes as the detection granularity, that is, collect the receiving and sending traffic of each port of the inspected network element every 15 minutes. Optionally, assume that the network device collects the inspected network for the first time at 00:00. The sending and receiving traffic of each port of the element, the next time, the network device will collect the sending and receiving traffic of each port of the inspected network element at 00:15, and the timing of the third traffic collection is at 00:30... 00:00, 00:15, and 00:30, etc., which are referred to as detection time in this embodiment. Assuming that the current time is 00:15, 00:15 is the current detection time, and 00:00 is the historical detection time.

It can be understood that, in some other examples of this embodiment, when the network device detects abnormal traffic of the detected network element, it may also not need to periodically collect traffic. That is, when the network device collects the receiving and sending traffic of the inspected network element, the time interval between each inspection time is not completely consistent.

S104: Determine the real-time window deviation ratio of the checked network element in the current time window according to the collected receiving traffic and sending traffic.

After collecting the receiving traffic and sending traffic of each port of the inspected network element at the current inspection time, the network device can determine the real-time window deviation ratio of the inspected network element in the current time window. The so-called real-time window deviation ratio is the network under inspection. The flow deviation ratio of the yuan in the current time window.

First, explain the "traffic deviation ratio": for the inspected network element, the traffic deviation ratio at a certain detection time can be the sum of the received traffic and the sent traffic of each port of the inspected network element at the detection time. And the ratio of, for example,

bias=N _recv /N _send ;

Among them, bias is the traffic deviation ratio, and N _recv is the sum of all received traffic at the time when each port of the detected network element is detected. It can be calculated by the following formula:

Among them, n is the total number of ports in the inspected network element, and i represents the i-th port,

Represents the received traffic of the i-th port of the inspected network element at the time of inspection.

N _send is the sum of all received traffic at the time when each port of the inspected network element is detected, and can be calculated by the following formula:

Indicates the sending traffic of the i-th port of the inspected network element at the time of inspection.

The traffic deviation ratio of an inspected network element is mainly used to characterize the balance between the sending and receiving traffic of each port of the inspected network element. Therefore, there is no doubt that the traffic deviation ratio of the inspected network element is not necessarily the receiving The ratio of the sum of traffic to the sum of sent traffic may also be the ratio of the sent traffic to the received traffic, that is,

bias=N _send /N _recv ;

However, it should be noted that each time the network device determines the traffic deviation ratio of the inspected network element, it should choose a unified calculation method for the traffic deviation ratio. For example, in some examples, if the network device calculates the traffic deviation ratio for the first time When checking the traffic deviation ratio of the network element, the ratio of the sum of the received traffic and the sum of the sending traffic corresponding to the ports of the checked network element at the first detection time is calculated, and then at the subsequent detection time, the network device calculates When the flow deviation ratio of the spare part network element is calculated, it should also calculate the ratio of the sum of the received flow and the sum of the transmitted flow. It should not suddenly become the calculation of the sum of the transmitted flow and the received flow of each port of the inspected network element during a certain calculation process. The ratio of the sum. Figures 2 and 3 respectively show schematic diagrams of the traffic deviation ratios of two checked network elements in the same day, where the vertical axis bias represents the traffic deviation ratio, and the horizontal axis represents time.

The so-called current time window refers to the time window corresponding to the current detection time. The so-called "real-time window deviation ratio" is actually the traffic deviation ratio of the detected network element in the current time window. A time window includes at least one detection time. For example, in an example of this embodiment, there is only one detection time in the time window, and the real-time window deviation ratio of the detected network element in the current time window is actually the time window. Check the flow deviation ratio of the network element at the current detection time. However, if a time window includes two or more detection moments at the same time, the real-time window deviation ratio of the detected network element in the current time window is the traffic deviation of the detected network element at each detection time in the current time window. The mean of the ratio. For example, in an example, the time window includes three detection times. Please refer to the schematic diagram of the relationship between the time window and the detection time shown in Figure 4, where the vertical axis bias represents the flow deviation ratio, and the horizontal axis represents time:

Assuming that the current detection time is the nth detection time, the current time window 401 is the time window corresponding to the nth detection time, which also includes the nth detection time, the n-1th detection time and the n-2th detection time. Detection time. As for the historical time window 402, it is the time window corresponding to the previous detection time (that is, the n-1th detection time), which includes the n-1th detection time, the n-2th detection time, and the n-3th detection time. The time window of the detection time.

It is further assumed that the traffic deviation ratios of the detected network element at the nth detection time, the n-1th detection time, the n-2th detection time, and the n-3th detection time are b _n , b _n-1 , b _n-2 and b _n-3 , the real-time window deviation ratio of the checked network element is (b _n + b _n-1 + b _n-2 )/3. The historical window deviation ratio of the checked network element is (b _n-1 + b _n-2 ) + b _n-3 )/3.

It is understandable that if the current detection time is the nth detection time, when the network device determines the real-time window deviation ratio, it only needs to calculate the flow at the nth detection time based on the received flow and the sent flow collected in the nth flow. Deviation ratio b _n . As for the calculation of the real-time window deviation ratio, the other flow deviation ratios b _n-1 and b _n-2 have been calculated in the previous detection process (b _n-1 is the calculation of the real-time window deviation at the n-1th detection time It is calculated when comparing, b _n-2 is calculated when calculating the real-time window deviation ratio at the n-2th detection time), there is no need to calculate it again here.

S106: Determine the steep slope of the current detection moment according to the real-time window deviation ratio and the historical window deviation ratio.

According to the definition given by Hawkin, anomalies are data that deviate from most of the data in the data set. Therefore, anomalies are also called outliers. Therefore, in this embodiment, the network device determines whether the current detection time is an abnormal point (that is, the abnormal flow time) according to whether the flow deviation ratio corresponding to the current detection time deviates from the flow deviation ratio of most detection times.

The traffic deviation ratio of the inspected network element belongs to a time series indicator. The main goal of monitoring this indicator is to find the time point when it deviates from the normal value in time, which is a change point detection problem for the time series. Change point theory is a classic branch of statistics. Its basic definition is that in a sequence or process, when a certain statistical characteristic (distribution type, distribution parameter) changes at a certain point in time by systemic factors rather than accidental factors , We call this point in time the change point. The change point detection is to use statistics or statistical methods to find out the position of the change point.

Therefore, after calculating the real-time window deviation ratio corresponding to the current detection time of the inspected network element, the network device can determine the steep slope of the inspected network element at the current detection time based on the real-time window deviation ratio of the inspected network element and the historical window deviation ratio , The steep slope can characterize the degree of change of the real-time window deviation ratio at the current detection time relative to the historical window deviation ratio.

In this embodiment, the steep slope of the current detection moment can be determined according to the following formula:

Among them, n represents the nth detection time, and M _n represents the traffic deviation ratio in the time window corresponding to the nth detection time of the detected network element, and M _n-1 represents the detected network element at the n-1th detection time The flow deviation ratio in the time window corresponding to the time, K _n is the steep slope of the nth detection time. If it is the nth detection time, _Kn is the steep slope corresponding to the current detection time.

It is understandable that the historical window deviation ratio of the inspected network element will be calculated after the network device performs the n-1th transmission and reception traffic collection for the inspected network element. Therefore, in this embodiment, after the network device calculates the real-time window deviation ratio corresponding to the nth detection time, it records it so as to participate in the calculation as the historical window deviation ratio at the n+1th time.

S108: Determine whether the current detection time is an abnormal flow time based on the steep slope of the current detection time.

After calculating the steep slope corresponding to the detected network element at the current detection time, the network device can determine whether the detected network element has abnormal traffic at the current detection time according to the steep slope, that is, whether the current detection time is the abnormal traffic time.

It is understandable that if the traffic of the inspected network element at the nth detection time is normal, its steep slope at the nth detection time will be close to 1. In some examples of this embodiment, the network device stores the parameters that can divide the normal slope threshold. Here, it is assumed that the normal slope range (1/Q, Q), where Q is a positive number, so (1/Q, Q) is The values are relatively close to 1.

Naturally, the union of the range smaller than 1/Q and the range larger than Q belongs to the abnormal slope range. Therefore, in some examples of this embodiment, when the network device determines whether the traffic of the inspected network element is abnormal at the nth detection moment, it can determine whether the steep slope of the inspected network element at the nth detection moment is at a normal slope. Within the range, if yes, it is determined that the detection time is not the time of abnormal flow; if not, it is determined that the detection time is the time of abnormal flow of the detected network element.

In some examples of this embodiment, the value of Q may be fixed, for example, it is set by network operation and maintenance personnel based on a large number of experience values. It is understandable that the Q value set by network operation and maintenance personnel should ensure Detect all network abnormalities of the inspected network elements as accurately as possible. In some other examples of this embodiment, the value of Q can be adjusted adaptively. For example, the initial value of Q is set by network operation and maintenance personnel based on experience, but as the network equipment continues to check the network element For traffic anomaly detection, the network device can adjust the value of Q according to the accuracy of its detection results, thereby reducing false detections and/or missed detections during the flow anomaly detection process. Please refer to the adjustment shown in Figure 5 A flow chart of the normal slope range:

S502: Add each abnormal time of flow detected in a certain time period to the automatic marking abnormality set.

In this embodiment, the network device may adjust the value of Q at regular intervals. There is no doubt that adjusting the value of Q actually means adjusting the normal slope range. It is assumed that the network equipment set here adjusts the normal slope range every two hours.

In these two hours, if the time difference between two adjacent detection moments of the network device is 15 minutes, the network device may have performed 8 detections on the detected network element, and part of the detection moments of the 8 detections It will be judged as an abnormal flow time. The network device can add the abnormal traffic moments among the 8 detection moments to the automatic anomaly marking set. The automatic anomaly marking set is a set of abnormal traffic moments marked by the network equipment mechanized.

It is understandable that if the steep slope of the real-time window deviation ratio and the historical window deviation ratio at a detection moment is relatively large, there may be two situations:

The first is that, compared with the historical window deviation ratio, the real-time window deviation is better than the flow situation represented, that is, the absolute difference between the real-time window deviation ratio and 1 is less than the absolute difference between the historical window deviation ratio and 1, which means that although the current There was an abnormal flow at the time of detection, but this is because the abnormal flow is gradually recovering. Therefore, the abnormal flow at the current detection time is actually in the recovery state.

The second type, compared with the historical window deviation ratio, the real-time window deviation is worse than the flow situation represented, that is, the absolute difference between the real-time window deviation ratio and 1 is greater than the absolute difference between the historical window deviation ratio and 1, indicating the current detection There is an abnormal flow at the moment, and this abnormality is deteriorating. Therefore, the abnormal flow at the current detection time is actually in a deteriorating state.

In the process of detecting abnormal traffic on the inspected network elements, the network equipment only needs to pay attention to the points where the abnormality occurs or the points where the abnormality deteriorates, and the network equipment does not need to pay attention to the abnormal traffic moments that have a trend of recovery. Therefore, In some examples of this embodiment, the network device may refer to the flowchart shown in FIG. 6 to determine the automatic annotated abnormal set:

S602: The network device determines all abnormal traffic moments within the time period;

S602: For each abnormal flow time within the time period, the network device determines whether the abnormal flow at the abnormal flow time is in a recovery state or a deteriorating state according to the abrupt slope and historical abrupt slope of the abnormal flow;

S606: The network device removes the abnormal traffic moments that are in the recovery state, and uses the remaining abnormal traffic moments as an automatic anomaly set.

S504: Compare each abnormal flow time in the manually marked abnormal set with each abnormal flow time in the automatically marked abnormal set.

On the other hand, in order to check the accuracy of the network equipment's detection of abnormal traffic in these two hours, the network equipment will also obtain the manually marked abnormal set corresponding to the automatically marked abnormal set. The marked result of the abnormal traffic time within the hour. Here, the abnormal traffic moments in the artificially marked abnormal set can be regarded as completely correct, and there is no mislabeling; and it is considered that the artificially marked abnormal set contains all the abnormal traffic moments in the past two hours, and there is no missing label. Happening.

After obtaining the artificially marked anomaly set and the automatically marked anomaly set, the network device can compare each abnormal flow time in the artificially marked anomaly set with each abnormal flow time in the automatically marked anomaly set.

If the network equipment needs to check the error labeling in the automatic labeling anomaly set, that is, to determine the false detection rate of the automatic labeling anomaly set, the network device can determine the false detection exception in the automatic labeling anomaly set. The false detection exception is actually Automatically mark the abnormal traffic moments that exist in the abnormal set, but manually mark the abnormal traffic that does not exist in the abnormal set.

If the network equipment needs to check the omissions in the automatic anomaly set, that is, determine the missed detection rate of the automatic anomaly set, the network equipment can determine the missed anomalies in the automatically marked anomaly set. The missed anomalies are manually marked Exist in the abnormal set, but automatically mark the abnormal traffic moment that does not exist in the abnormal set.

S506: Adjust the normal slope range according to the comparison result.

In some examples of this embodiment, the network device can determine its own false detection rate according to the following formula:

If the network device determines that the false detection rate in the automatically marked abnormal set reaches the preset false detection threshold, it can determine the steep slope corresponding to each false detection abnormality, and then the network device determines the Q value corresponding to each steep slope, and then selects the largest one The Q value is used as the adjusted Q value. For example, if the network device determines that the false detection rate in the automatically marked anomaly set reaches the preset false detection threshold, including 3 false detection anomalies, the steep slopes corresponding to these three false detection anomalies are 1.5, 2 and 2.5 respectively, then this The Q corresponding to the three steep slopes are 1.5, 2 and 2.5 respectively, so the updated Q value is 2.5. For another example, the abrupt slopes corresponding to three false detection abnormalities are 1/4, 1/3, and 1/2 respectively, and the Q corresponding to the three abrupt slopes are 4, 3, and 2, respectively. Therefore, the updated Q value Is 4.

Through this adjustment, the Q value is increased, and the normal slope range is also increased, thereby reducing the possibility that the network device detects the abnormal flow when the flow is normal.

The network device can also determine its own missed detection rate according to the following formula:

If the network device determines that the missed detection rate in the automatically marked anomaly set reaches the preset missed detection threshold, it can determine the steep slope corresponding to each missed anomaly, and then update the Q value according to the minimum value of the steep slopes. Optionally, the network device may determine the Q value corresponding to each steep slope, and then select the smallest Q value as the updated Q value. For example, if the network device determines that the missed detection rate in the automatically marked anomaly set reaches the preset missed detection threshold, including 3 missed detections, the steep slopes corresponding to the three missed detections are 1/2, 1/3, and 3. The Q values corresponding to the three steep slopes are 2, 3, and 3 respectively. In order to identify these three missed abnormalities as abnormal traffic moments, the network equipment can adjust the value of Q to 2. It should be understood Yes, the value of Q must be greater than 3 before adjustment. Therefore, this adjustment actually reduces the value of Q, which also increases the range of abnormal slopes, thereby reducing the amount of abnormal traffic that cannot be correctly detected by network equipment. possibility.

The flow anomaly detection method provided in this embodiment analyzes the flow deviation ratio of the detected network element, and identifies the moment when the flow deviation ratio changes greatly based on the steep slope as the flow abnormal moment. This flow anomaly detection scheme can effectively identify Those abnormal points that will not cause the traffic of the inspected network element to exceed the limit, provide a basis for network optimization.

Furthermore, because the network equipment can adjust the parameters used to determine the abnormal time of the flow according to the result of marking the abnormal flow, the parameters used to determine the abnormal time of the flow are more accurate and more in line with the actual situation of the network, thereby improving the detection of the abnormal flow. Accuracy, reduce false detections and missed detections.

This embodiment will continue to introduce the flow abnormality detection method in the foregoing embodiment. Please refer to the flowchart shown in FIG. 7, which includes the following steps:

S702: Collect receiving traffic and sending traffic of each port of the inspected network element at the current inspection moment.

It is understandable that a network device used for traffic abnormality monitoring may monitor the traffic of two or more detected network elements at the same time. Therefore, when the network device performs port traffic collection, it is for all the monitored network elements. All ports of the inspected network element are performed. Therefore, after obtaining the collection result, the network device needs to determine which of the inspected network elements the collection result belongs to, according to the asset relationship data (which can characterize the corresponding relationship between the inspected network element and the port). Then, the time of abnormal traffic is determined for each detected network element.

In this embodiment, the network device may perform detection every 15 minutes, that is, the detection granularity is 15 minutes. It is understandable that if the detection granularity is set too large, the network device will not be able to detect which abnormalities that appear and recover in a short time. For example, if the detection granularity is set to 3 hours, the network device cannot detect the graph. The abnormality shown in 8.

Of course, although the smaller the detection granularity is set by the network device, the more subtle anomalies can be detected, but this will also cause the detection frequency of the network device to become higher, thus occupying more processing resources. Therefore, when the network device sets the detection granularity, it can be set according to its own processing capability.

S704: Determine whether the receiving traffic and the sending traffic of at least two ports of the inspected network element are collected.

It is understandable that after the network equipment collects the receiving traffic and the sending traffic on each port of the inspected network element, there may be a situation where the inspected network element contains a centralized port, and only one port has valid traffic data. . This situation is itself an abnormal phenomenon in the actual network, because the normal network element equipment as a data exchange node has at least two working ports to complete the source and destination data exchange. The phenomenon of single-port self-switching may actually be due to Certain failures caused some port traffic to not be collected. At the same time, if a network device conducts data observation on such a network element that only has single-port data, it will find that the received and sent traffic data fluctuates greatly, which also causes the traffic deviation to fluctuate greatly from the index data. It is understandable that a large abrupt slope is easily generated during the fluctuation process. Therefore, the current detection time is easily recognized by the network device as an abnormal flow time. However, when some port data is missing, the abnormal flow deviation ratio on a single port is not actually the abnormal flow deviation ratio of the network element we want to obtain. Therefore, in some examples of this embodiment, the network device It is necessary to exclude network elements that only have valid single-port data.

Therefore, if the judgment result of the network device is yes, the network device continues to perform S706, otherwise the process ends.

S706: Determine whether the receiving traffic and the sending traffic of each port of the inspected network element at the current inspection moment are all zero.

Due to various factors, the flow data collected by the network equipment will inevitably have deficiencies. For example, a detected network element may have a situation where the received and sent traffic on each port are both zero. According to the traditional big data processing method, for this kind of data missing, missing value filling is a common method. Common missing value filling methods include: before and after mean filling, mode filling, linear regression filling and so on. For machine learning training data, missing value filling is generally effective, but for the traffic anomaly detection scheme in this embodiment, missing value filling often has a great impact on the detection result. In fact, most filling methods make the data smoother, so that the network equipment cannot detect the original abnormal points based on the steep rise and fall of the traffic deviation ratio.

Therefore, in this embodiment, if the network device determines that the receiving traffic and the sending traffic of each port of the detected network element at the current detection moment are all zero values, S718 is executed.

S708: Determine the real-time window deviation ratio of the inspected network element in the current time window according to the collected receiving traffic and sending traffic.

After collecting the receiving traffic and sending traffic of each port of the inspected network element at the current inspection moment, the network device can determine the real-time window deviation ratio of the inspected network element in the current time window. In an example of this embodiment, one time window includes three detection moments. Therefore, the network device can determine the current detection time corresponding to the current detection time based on the traffic deviation ratio at the current detection time and the traffic deviation ratio between the previous two detection times. Real-time window deviation ratio.

S710: Determine the steep slope of the current detection moment according to the real-time window deviation ratio and the historical window deviation ratio.

After calculating the real-time window deviation ratio corresponding to the current detection time of the inspected network element, the network device can calculate the ratio of the real-time window deviation ratio of the inspected network element to the historical window deviation ratio to obtain the sharp change of the inspected network element at the current inspection time. Slope.

S712: Determine whether the current detection time is an abnormal flow time based on the steep slope of the current detection time.

After calculating the steep slope corresponding to the detected network element at the current detection time, the network device can determine whether the flow of the detected network element is abnormal at the current detection time according to the steep slope. Optionally, the network device determines whether the steep slope corresponding to the current detection time is within the normal slope range (1/Q, Q), if yes, then it is determined that the detection time is not the time of abnormal flow; if not, it is determined that the detection time is The time when the network element's traffic is abnormal.

In some examples of this embodiment, if the network device determines that the current detection time is the abnormal flow time of the inspected network element, the network device will further determine whether the abnormal flow time is a deteriorating abnormal flow time:

S714: Determine whether the abnormality at the time of the abnormal flow is in a deteriorating state according to the abrupt slope and the historical abrupt slope at the time of the abnormal flow.

If the judgment result is yes, S716 is executed, otherwise, S718 is not executed.

S716: Record the time when the traffic is abnormal.

If it is judged that the abnormality of the abnormal flow is in a deteriorating state based on the abrupt slope and the historical abrupt slope of the abnormal flow, the network device can record the abnormal flow for use in the subsequent network optimization process.

S718: Determine whether a new detection time is reached.

If the judgment result is yes, continue to execute S702, otherwise continue to judge.

In this embodiment, the network device can also evaluate its own false detection rate and missed detection rate once a period of time, and adjust the normal slope range according to the duration of the evaluation result, so as to reduce the error of abnormal traffic in the subsequent detection process. Detection rate and missed detection rate. The specific evaluation and adjustment process has been described in more detail in the foregoing embodiment, and will not be repeated here.

The traffic abnormality detection method provided in this embodiment automatically monitors the traffic of the inspected network element by the network device, and marks the abnormal time of the traffic, which not only reduces the demand for human resources, but also the detection process is fast and efficient, and can find conventional Traffic abnormalities that cannot be detected by the method. At the same time, it can also feedback and adjust the parameters for judging when the flow is abnormal according to the detection result, so that the detection result can achieve higher accuracy and reliability.

The embodiment of the present invention also provides a flow abnormality detection device. Please refer to the schematic structural diagram shown in FIG. 9, in which:

The flow abnormality detection device 90 includes a flow collection module 902, a deviation determination module 904, a slope determination module 906, and an abnormality determination module 908. The flow collection module 902 is configured to collect the received and sent traffic of each port of the inspected network element at the current detection moment. Flow; the deviation determination module 904 is set to determine the real-time window deviation ratio of the inspected network element in the current time window based on the collected received and sent traffic, and the slope determination module 906 is set to determine based on the real-time window deviation ratio and the historical window deviation ratio The steep slope of the current detection time, the historical window deviation ratio is the traffic deviation ratio of the detected network element in the time window corresponding to the previous detection time; the abnormality determination module 908 is set to determine whether the current detection time is based on the steep slope of the current detection time It is the moment of abnormal flow.

In some other examples of this embodiment, referring to FIG. 10, the flow anomaly detection device 90 further includes a preprocessing module 910, which is configured to determine whether the flow collection module 902 has collected at least two ports of received traffic of the detected network element. And send traffic. Only when the judgment result of the preprocessing module 910 is yes, the deviation determination module 904 will calculate the real-time window deviation ratio.

Alternatively, the preprocessing module 910 may also be configured to determine whether the receiving traffic and the sending traffic of each port of the inspected network element at the current inspection moment are all zero values. Only when the judgment result of the preprocessing module 910 is negative, the deviation determination module 904 will calculate the real-time window deviation ratio.

In some examples of this embodiment, a time window includes at least two detection moments, the deviation determination module 904 can determine the flow deviation ratio at the current detection moment based on the received traffic and the sent traffic collected at the current detection moment, and obtain the current time The flow deviation ratio of other detection moments in the window is then determined according to the flow deviation ratio of each detection time in the current time window to determine the average value of the flow deviation ratio of the current time window as the real-time window deviation ratio.

The traffic deviation ratio at the current detection time is the ratio of the sum of the received traffic and the sum of the sent traffic of each port of the detected network element collected at the current detection time.

In some examples of this embodiment, the abnormality determination module 908 can determine whether the steep slope of the current detection moment is outside the normal slope range, the normal slope range (1/Q, Q), where Q is a positive number; if so, it determines The current detection time is the time of abnormal flow. If not, it is determined that the current detection time is not the time of abnormal flow.

In some examples of this embodiment, as shown in FIG. 11, the flow anomaly detection device 90 may further include a parameter adjustment module 912, which is configured to evaluate the multiple determination results of the abnormality determination module 908, according to the false detection rate and At least one of the missed detection rates is adjusted to adjust the abnormality determination module 908 to determine whether a detection time is the normal slope range of the abnormal flow time.

Optionally, the parameter adjustment module 912 may add each abnormal time of traffic detected in a certain period of time to the automatic marking abnormality set, and then manually marking each abnormal flow time in the abnormal set and automatically marking each abnormal flow in the abnormal set Perform comparison at all times, and adjust the normal slope range according to the comparison result.

In an example of this embodiment, the parameter adjustment module 912 determines the false detection abnormalities in the automatically marked abnormal set, and then determines whether the false detection rate in the automatically marked abnormal set reaches the preset false detection threshold, and if so, the parameter adjustment The module 912 further determines the steep slope corresponding to each false detection abnormality, and adjusts the Q value according to the maximum value of the steep slope.

In an example of this embodiment, the parameter adjustment module 912 determines the missed abnormalities in the automatically labeled abnormal set, and then determines that the missed detection rate in the automatically labeled abnormal set reaches the preset missed detection threshold. If so, the parameter adjustment module 912 further determines the steep change slope corresponding to each missed abnormality, and adjusts the Q value according to the maximum value of each steep change slope.

Optionally, the parameter adjustment module 912 may first determine all the abnormal flow moments in a time period, and then for each abnormal flow moment in the time period, determine the abnormal flow rate according to the steep slope of the abnormal flow moment and the historical steep slope. Whether the abnormality is in the recovery state or the deteriorating state, then the abnormal flow moments in the recovery state are eliminated, and the remaining abnormal flow moments are regarded as the automatic anomaly set.

The traffic anomaly detection device 90 in this embodiment can be deployed on a network device, such as a network device in a bearer network, where the function of the traffic collection module 902 can be implemented by the processor of the network device and the communication Danyun, and the deviation determination module The functions of 904, the slope determination module 906, the abnormality determination module 908, the preprocessing module 910, and the parameter adjustment module 912 can all be implemented by the processor of the network device.

For other details of the method for detecting an abnormal flow of the flow anomaly detection device, please refer to the introduction of the foregoing embodiment, which will not be repeated here.

The flow abnormality detection device provided in this embodiment analyzes the flow deviation ratio of the detected network element, and identifies the moment when the flow deviation ratio changes greatly based on the steep slope as the flow abnormality moment. This flow abnormality detection scheme can effectively identify Those abnormal points that will not cause the traffic of the inspected network element to exceed the limit, provide a basis for network optimization.

Furthermore, because the traffic anomaly detection device can adjust the parameters for determining the abnormal time of the traffic according to the result of marking the abnormal traffic, so that the parameters for judging the abnormal time of the traffic are more accurate, more in line with the actual situation of the network, and thereby increase the traffic. The accuracy of anomaly detection reduces false detections and missed detections.

The embodiment of the present invention also provides a computer-readable storage medium. The computer-readable storage medium may store one or more computer programs that can be read, compiled, and executed by one or more processors. In an embodiment, the computer-readable storage medium may store a flow anomaly detection program, and the flow anomaly detection program can be used by one or more processors to execute a process for implementing any of the flow anomaly detection methods introduced in the foregoing embodiments.

In addition, this embodiment provides a network device, as shown in FIG. 12: the network device 120 includes a processor 121, a memory 122, and a communication bus 123 configured to connect the processor 121 and the memory 122, where the memory 122 may be the aforementioned storage A computer-readable storage medium for the flow anomaly detection program. The processor 121 may read the flow anomaly detection program, compile and execute the flow of the flow anomaly detection method introduced in the foregoing embodiment:

The processor 121 collects the receiving traffic and sending traffic of each port of the inspected network element at the current detection time, and determines the real-time window deviation ratio of the inspected network element in the current time window according to the collected receiving and sending traffic, and then according to the real-time window The deviation ratio and the historical window deviation ratio determine the steep slope of the current detection time, and determine whether the current detection time is an abnormal flow time based on the steep slope of the current detection time.

In some other examples of this embodiment, the processor 121 is further configured to determine whether at least the receiving traffic and the sending traffic of at least two ports of the inspected network element are collected. Only when the judgment result is yes, the processor 121 will calculate the real-time window deviation ratio.

Alternatively, the processor 121 may also determine whether the receiving traffic and the sending traffic of each port of the detected network element at the current detection moment are all zero values. Only when the judgment result is negative, the processor 121 will calculate the real-time window deviation ratio.

In some examples of this embodiment, a time window includes at least two detection moments, and the processor 121 may determine the flow deviation ratio at the current detection moment according to the received traffic and the sent traffic collected at the current detection moment, and obtain the current time window Then, according to the flow deviation ratio of each detection time in the current time window, the average value of the flow deviation ratio of the current time window is determined as the real-time window deviation ratio.

In some examples of this embodiment, the processor 121 may determine whether the steep slope at the current detection time is outside the normal slope range, the normal slope range (1/Q, Q), where Q is a positive number; if so, it determines that the current The detection time is the time of abnormal flow. If not, it is determined that the current detection time is not the time of abnormal flow.

In some examples of this embodiment, the processor 121 may also evaluate its own multiple determination results, and adjust the value used to determine whether a detection moment is an abnormal flow according to at least one of the false detection rate and the missed detection rate. Adjust the normal slope range.

Optionally, the processor 121 may add each abnormal time of traffic detected in a certain period of time to the automatic marking abnormality set, and then manually marking each abnormal flow time in the abnormal set and automatically marking each abnormal flow time in the abnormal set Perform comparison, and adjust the normal slope range according to the comparison result.

In an example of this embodiment, the processor 121 determines the false detection abnormalities in the automatically marked abnormal set, and then determines whether the false detection rate in the automatically marked abnormal set reaches the preset false detection threshold, and if so, the processor 121 Further determine the steep slope corresponding to each false detection abnormality, and adjust the Q value according to the maximum value of each steep slope.

In an example of this embodiment, the processor 121 determines the missed abnormalities in the automatically marked abnormal set, and then determines that the missed detection rate in the automatically marked abnormal set reaches the preset missed detection threshold, and if so, the processor 121 further Determine the steep slope corresponding to each missed abnormality, and adjust the Q value according to the steep slope.

Optionally, the processor 121 may first determine all abnormal flow moments within a time period, and then for each abnormal flow moment in the time period, determine the abnormal flow abnormal moment according to the abrupt slope of the abnormal flow moment and the historical abrupt slope. Whether it is in a recovering state or a deteriorating state, then the abnormal traffic moments in the recovering state are eliminated, and the remaining abnormal traffic moments are regarded as the automatic marking abnormal set.

The network device provided in this embodiment can automatically monitor the traffic of the inspected network element and mark the time when the traffic is abnormal, which not only reduces the demand for human resources, but also the detection process is fast and efficient, and it can be found that the conventional method cannot be detected. Abnormal traffic conditions. At the same time, it can also feedback and adjust the parameters for judging when the flow is abnormal according to the detection result, so that the detection result can achieve higher accuracy and reliability.

It can be understood that, in the case of no conflict, the features in the embodiments of the present invention can be used in combination.

Obviously, those skilled in the art should understand that all or some of the steps in the method disclosed above, the functional modules/units in the system, and the device can be implemented as software (which can be implemented by program code executable by a computing device) , Firmware, hardware and their appropriate combination. In hardware implementations, the division between functional modules/units mentioned in the above description does not necessarily correspond to the division of physical components; for example, one physical component may have multiple functions, or one function or step may consist of several physical components. The components are executed cooperatively. Some physical components or all physical components can be implemented as software executed by a processor, such as a central processing unit, a digital signal processor, or a microprocessor, or as hardware, or as an integrated circuit, such as an application specific integrated circuit . Such software may be distributed on a computer-readable medium and executed by a computing device, and in some cases, the steps shown or described may be executed in a different order than here. The computer-readable medium may include computer storage Medium (or non-transitory medium) and communication medium (or temporary medium). As is well known to those of ordinary skill in the art, the term computer storage medium includes volatile and non-volatile memory implemented in any method or technology for storing information (such as computer-readable instructions, data structures, program modules, or other data). Flexible, removable and non-removable media. Computer storage media include but are not limited to RAM, ROM, EEPROM, flash memory or other memory technologies, CD-ROM, digital versatile disk (DVD) or other optical disk storage, magnetic cassette, tape, magnetic disk storage or other magnetic storage devices, or Any other medium used to store desired information and that can be accessed by a computer. In addition, as is well known to those of ordinary skill in the art, communication media usually contain computer-readable instructions, data structures, program modules, or other data in a modulated data signal such as carrier waves or other transmission mechanisms, and may include any information delivery media . Therefore, the present invention is not limited to any specific combination of hardware and software.

The above content is a further detailed description of the embodiments of the present invention in combination with specific implementations, and it cannot be considered that the specific implementations of the present invention are limited to these descriptions. For those of ordinary skill in the technical field to which the present invention belongs, several simple deductions or substitutions can be made without departing from the concept of the present invention, which should be regarded as falling within the protection scope of the present invention.

Industrial applicability

As described above, the method, device, network device, and storage medium for detecting traffic anomalies provided by the embodiments of the present invention have the following beneficial effects: they can more effectively find abnormalities that do not cause the traffic to exceed the limit, and increase the traffic to the network element equipment. The comprehensiveness of monitoring increases the accuracy and credibility of the test results.

Claims

A flow abnormality detection method, including:

Collect the receiving and sending traffic of each port of the inspected network element at the current inspection moment;

Determine the real-time window deviation ratio of the detected network element in the current time window according to the collected received traffic and the sent traffic, and the current time window is the time window corresponding to the current detection moment, and the real-time window The deviation ratio is as described above, and the flow deviation ratio can represent the degree of balance between the received flow and the sent flow;

Determining the steep slope of the current detection time according to the real-time window deviation ratio and the historical window deviation ratio, where the historical window deviation ratio is the traffic deviation ratio of the detected network element in the time window corresponding to the previous detection time;

Based on the steep slope of the current detection time, it is determined whether the current detection time is an abnormal flow time.
The method for detecting abnormal traffic according to claim 1, wherein before determining the real-time window deviation ratio of the inspected network element in the current time window according to the collected receiving traffic and the sending traffic, the method further comprises :

It is determined that at least the receiving traffic and the sending traffic of at least two ports of the inspected network element are collected.
The method for detecting abnormal traffic according to claim 1, wherein before determining the real-time window deviation ratio of the inspected network element in the current time window according to the collected receiving traffic and the sending traffic, the method further comprises :

It is determined that the receiving traffic and the sending traffic of each port of the checked network element at the current detection moment are not all zero values.
The traffic abnormality detection method according to claim 1, wherein a time window includes at least two detection moments, and the detected network element is determined at the current time according to the collected receiving traffic and the sending traffic. The real-time window deviation ratio in the window includes:

Determine the flow deviation ratio at the current detection time according to the received flow rate collected at the current detection time and the transmission flow rate, and obtain the flow deviation ratios at other detection times in the current time window;

The average value of the flow deviation ratio of the current time window is determined as the real-time window deviation ratio according to the flow deviation ratio at each detection time in the current time window.
The method for detecting abnormal traffic according to claim 4, wherein the traffic deviation ratio at the current detection time is the ratio of the sum of the received traffic and the sum of the sent traffic of each port of the detected network element collected at the current detection time.
The method for detecting abnormal flow according to any one of claims 1 to 5, wherein the determining whether the current detection time is the abnormal flow time based on the steep slope of the current detection time comprises:

Judging whether the steep slope at the current detection moment is outside the normal slope range, the normal slope range (1/Q, Q), where Q is a positive number;

If yes, it is determined that the current detection time is the time of abnormal flow; if not, it is determined that the current detection time is not the time of abnormal flow.
8. The method for detecting abnormal traffic according to claim 6, wherein the method for detecting abnormal traffic further comprises:

Add each abnormal flow detected in a certain period of time to the automatic anomaly set;

Comparing each abnormal flow time in the manually marked abnormal set with each abnormal flow time in the automatically marked abnormal set;

Adjust the normal slope range according to the comparison result.
The method for detecting abnormal traffic according to claim 7, wherein comparing each abnormal time in the manually marked abnormal set with each abnormal time in the automatically marked abnormal set comprises: determining that the abnormal time in the automatically marked abnormal set The misdetection anomaly of, the misdetection anomaly is a traffic abnormal moment that exists in the automatically marked anomaly set but does not exist in the manually marked anomaly set;

The adjusting the normal slope range according to the comparison result includes:

Determining that the false detection rate in the automatically marked anomaly set reaches a preset false detection threshold, and the false detection rate is the number of falsely detected anomalies in the automatically marked anomaly set/the total number of abnormal traffic moments in the automatically marked anomaly set;

Determine the steep slope corresponding to each of the misdetected abnormalities;

The Q value is adjusted according to each of the steep slopes.
8. The method for detecting abnormal traffic according to claim 7, wherein the comparing the abnormal time of each traffic in the set of manually marked abnormalities with the abnormal time of each traffic in the set of automatically marked abnormalities comprises: determining the automatically marked abnormal A missed abnormality in the set, where the missed anomaly is a traffic abnormal moment that exists in the manually marked abnormal set but does not exist in the automatically marked abnormal set;

The adjusting the normal slope range according to the comparison result includes:

It is determined that the missed detection rate in the automatically labeled anomaly set reaches a preset missed detection threshold, and the missed detection rate is the number of missed anomalies in the automatically labeled anomaly set/(the total number of abnormal traffic moments in the automatically labeled anomaly set +Number of missed abnormalities);

Determine the steep slope corresponding to each of the missed abnormalities;

The Q value is adjusted according to each of the steep slopes.
8. The method for detecting abnormal traffic according to claim 7, wherein the adding each abnormal time of traffic detected in a certain period of time to the automatic marking abnormal set comprises:

Determine all abnormal traffic moments within the time period;

For each abnormal flow time in this time period, determine whether the abnormal flow at the abnormal flow time is in a recovery state or a deteriorating state according to the steep change slope and the historical steep change slope of the flow abnormal time;

Eliminate the abnormal flow moments in the recovery state, and use the remaining abnormal flow moments as the automatic anomaly set.
A flow abnormality detection device, including:

The traffic collection module is set to collect the receiving and sending traffic of each port of the detected network element at the current detection moment;

The deviation determining module is configured to determine the real-time window deviation ratio of the detected network element in the current time window based on the collected received traffic and the sent traffic, where the current time window is the time corresponding to the current detection moment Window, the traffic deviation ratio can represent the degree of balance between the received traffic and the sent traffic;

The slope determination module is configured to determine the steep slope of the current detection time according to the real-time window deviation ratio and the historical window deviation ratio, and the historical window deviation ratio is the time window corresponding to the previous detection time of the detected network element The flow deviation ratio;

The abnormality determination module is configured to determine whether the current detection time is an abnormal flow time based on the steep slope of the current detection time.
A network device, the network device including a processor, a memory, and a communication bus;

The communication bus is configured to realize connection and communication between the processor and the memory;

The processor is configured to execute one or more programs stored in the memory to implement the steps of the flow abnormality detection method according to any one of claims 1 to 10.
A computer-readable storage medium, the computer-readable storage medium stores one or more programs, and the one or more programs can be executed by one or more processors to implement any of claims 1 to 10 The steps of the method for detecting abnormal flow.