CN112242971B - Traffic abnormality detection method and device, network equipment and storage medium - Google Patents
Traffic abnormality detection method and device, network equipment and storage medium Download PDFInfo
- Publication number
- CN112242971B CN112242971B CN201910640994.6A CN201910640994A CN112242971B CN 112242971 B CN112242971 B CN 112242971B CN 201910640994 A CN201910640994 A CN 201910640994A CN 112242971 B CN112242971 B CN 112242971B
- Authority
- CN
- China
- Prior art keywords
- flow
- time
- detection
- abnormal
- anomaly
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L9/00—Cryptographic mechanisms or cryptographic arrangements for secret or secure communications; Network security protocols
- H04L9/40—Network security protocols
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L63/00—Network architectures or network communication protocols for network security
- H04L63/14—Network architectures or network communication protocols for network security for detecting or protecting against malicious traffic
- H04L63/1408—Network architectures or network communication protocols for network security for detecting or protecting against malicious traffic by monitoring network traffic
- H04L63/1425—Traffic logging, e.g. anomaly detection
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02D—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE
- Y02D30/00—Reducing energy consumption in communication networks
- Y02D30/50—Reducing energy consumption in communication networks in wire-line communication networks, e.g. low power modes or reduced link rate
Landscapes
- Engineering & Computer Science (AREA)
- Computer Security & Cryptography (AREA)
- Computer Networks & Wireless Communication (AREA)
- Signal Processing (AREA)
- Computer Hardware Design (AREA)
- Computing Systems (AREA)
- General Engineering & Computer Science (AREA)
- Data Exchanges In Wide-Area Networks (AREA)
Abstract
The embodiment of the invention provides a flow anomaly detection method, a device, network equipment and a storage medium, wherein receiving flow and sending flow of each port of network element equipment at the current detection moment are collected, then a real-time flow deviation ratio of the network element equipment in a current time window is determined according to the collected receiving flow and sending flow, an abrupt change slope of the current detection moment is determined according to the real-time flow deviation ratio and a historical flow deviation ratio, and then whether the current detection moment is the flow anomaly moment is determined based on the abrupt change slope of the current detection moment. Compared with the related traffic monitoring scheme, the traffic abnormality detection scheme provided by the embodiment of the invention can more effectively find the abnormality which does not cause traffic out-of-limit, improves the comprehensiveness of traffic monitoring of network element equipment, and increases the accuracy and the credibility of detection results.
Description
Technical Field
The present invention relates to the field of communications, and in particular, to a method, an apparatus, a network device, and a storage medium for detecting traffic anomalies.
Background
With the increasing size and complexity of communication networks, operators are facing increasing pressures and challenges in network operation. The traditional network element equipment anomaly monitoring scheme mainly relies on monitoring alarm events, for example, monitoring network traffic generally compares index values such as port bandwidth utilization and CPU utilization with fixed thresholds set according to manual experience, so as to determine whether the index values such as port bandwidth utilization and CPU utilization are within a reasonable range defined by corresponding fixed thresholds. These means are effective for monitoring peak traffic, but are difficult to find for some anomalies in the network element equipment that are hidden but affect the quality of service operation, such as a large number of lost packets or a large number of illegally duplicated packets that may occur in the network, but when such anomalies occur, the port traffic may not be out of limit, so that conventional traffic monitoring schemes cannot sense such anomalies that do not cause port traffic out of limit.
Disclosure of Invention
The flow anomaly detection method, the device, the network equipment and the storage medium provided by the embodiment of the invention mainly solve the technical problems that: the method and the device solve the problem that the related flow monitoring scheme can not detect flow abnormality which can not cause port flow out-of-limit.
In order to solve the above technical problems, an embodiment of the present invention provides a method for detecting a traffic abnormality, including:
collecting the received flow and the transmitted flow of each port of the network element to be detected at the current detection moment;
determining a real-time window deviation ratio of the detected network element in a current time window according to the acquired received flow and the transmission flow, wherein the current time window is a time window corresponding to the current detection moment, and the real-time window deviation ratio is that the flow deviation ratio represents the equilibrium degree of the received flow and the transmission flow;
determining the steep slope of the current detection moment according to the real-time window deviation ratio and the historical window deviation ratio, wherein the historical window deviation ratio is the flow deviation ratio of the detected network element in the time window corresponding to the previous detection moment;
and determining whether the current detection time is the abnormal flow time or not based on the abrupt change slope of the current detection time.
The embodiment of the invention also provides a flow abnormality detection device, which comprises:
the flow acquisition module is used for acquiring the received flow and the transmitted flow of each port of the network element to be detected at the current detection moment;
the deviation determining module is used for determining the real-time window deviation ratio of the detected network element in the current time window according to the acquired received flow and the transmission flow, wherein the current time window is a time window corresponding to the current detection moment, and the flow deviation ratio can represent the equilibrium degree of the received flow and the transmission flow;
The slope determining module is used for determining the steep slope of the current detection moment according to the real-time window deviation ratio and the historical window deviation ratio, wherein the historical window deviation ratio is the flow deviation ratio of the detected network element in the time window corresponding to the previous detection moment;
and the abnormality judgment module is used for determining whether the current detection moment is the abnormal moment of the flow or not based on the abrupt change slope of the current detection moment.
The embodiment of the invention also provides a network device, which comprises a processor, a memory and a communication bus;
the communication bus is used for realizing connection communication between the processor and the memory;
the processor is configured to execute one or more programs stored in the memory to implement the steps of the flow anomaly detection method described above.
The embodiment of the invention also provides a storage medium, which stores one or more programs, and the one or more programs can be executed by one or more processors to realize the steps of the flow anomaly detection method.
The beneficial effects of the invention are as follows:
according to the flow anomaly detection method, the flow anomaly detection device, the network equipment and the storage medium, the receiving flow and the sending flow of each port of the network element to be detected at the current detection moment are collected, then the real-time flow deviation ratio of the network element to be detected in the current time window is determined according to the collected receiving flow and sending flow, the abrupt change slope of the current detection moment is determined according to the real-time flow deviation ratio and the historical flow deviation ratio, and then whether the current detection moment is the flow anomaly moment is determined based on the abrupt change slope of the current detection moment. The flow anomaly detection scheme provided by the embodiment of the invention is based on that under the normal working condition of the network element to be detected, the total flow of all ports flowing in and out of the network element to be detected is basically balanced, but when the network element to be detected is abnormal in data routing and exchange processing, such as packet loss, illegal copying and the like, the balance degree of the receiving and transmitting flow of the network element to be detected is broken, so that the flow anomaly detection scheme provided by the embodiment of the invention can measure the balance degree of the receiving and transmitting flow of the network element to be detected, and determine the moment when the flow balance degree of the network element to be detected is changed suddenly, thereby detecting the moment when the flow of the network element to be detected is abnormal. Compared with the related flow monitoring scheme, the flow abnormality detection scheme provided by the embodiment of the invention can more effectively find the abnormality which does not cause flow out-of-limit, improves the comprehensiveness of monitoring the flow of the network element to be detected, and increases the accuracy and the credibility of the detection result.
Additional features and corresponding advantages of the invention will be set forth in the description which follows, and in part will be apparent from the description, or may be learned by practice of the invention.
Drawings
FIG. 1 is a flowchart of a flow anomaly detection method according to a first embodiment of the present invention;
fig. 2 is a schematic diagram of a flow deviation ratio of a detected network element in a day according to a first embodiment of the present invention;
fig. 3 is a schematic diagram of a flow deviation ratio of another network element under test in a day according to the first embodiment of the present invention;
FIG. 4 is a diagram showing a relationship between a time window and a detection time according to the first embodiment of the present invention;
FIG. 5 is a flowchart illustrating the network device adjusting the normal slope range according to the first embodiment of the present invention;
FIG. 6 is a diagram illustrating a network device determining an automatic labeling anomaly set in accordance with a first embodiment of the present invention;
FIG. 7 is a flowchart of a flow anomaly detection method according to a second embodiment of the present invention;
fig. 8 is a schematic diagram of a flow deviation ratio of another network element under test in a day according to the second embodiment of the present invention;
fig. 9 is a schematic structural diagram of a flow anomaly detection device according to a third embodiment of the present invention;
Fig. 10 is a schematic diagram of another structure of a flow anomaly detection device according to a third embodiment of the present invention;
fig. 11 is a schematic diagram of another structure of a flow anomaly detection device according to a third embodiment of the present invention;
fig. 12 is a schematic hardware structure of a network device according to a fourth embodiment of the present invention.
Detailed Description
In order to make the objects, technical solutions and advantages of the present invention more apparent, the following detailed description of the embodiments of the present invention is given with reference to the accompanying drawings. It should be understood that the specific embodiments described herein are for purposes of illustration only and are not intended to limit the scope of the invention.
Embodiment one:
with the continuous development of communication networks, the pressure faced by network operation and maintenance is gradually increased, and the advancement of the traditional operation and maintenance mode mainly comprising manpower cannot meet the requirements, because the operation and maintenance investment in the traditional operation and maintenance mode is continuously increased, but the network failure rate and the timeliness of failure response processing are not improved.
The traditional flow monitoring scheme generally sets a fixed threshold for indexes such as port bandwidth utilization rate and CPU utilization rate according to manual experience, and if a certain index of a detected network element port is found to exceed the fixed threshold corresponding to the index in the detection process, the abnormal condition is judged to be monitored, and an alarm can be given. It is apparent that this flow monitoring method simply determines whether the detected index value is normal according to the threshold value, which is effective for monitoring the abnormality of the flow peak value out of limit, but if the occurrence of the abnormality does not cause the port flow out of limit, the conventional flow monitoring scheme cannot sense. For example, if a large number of packet losses or a large number of illegally duplicated packets occur in the network element under test, these anomalies do not cause traffic overrun, and conventional traffic monitoring schemes do not recognize these anomalies.
In order to solve the above-mentioned problems, the present embodiment provides a flow anomaly detection method, please refer to the flowchart shown in fig. 1:
s102: and collecting the received flow and the transmitted flow of each port of the network element to be detected at the current detection moment.
For a network element to be tested, under normal operation, the total traffic flowing in and out of all ports should be basically balanced, that is, the sending traffic and the receiving traffic of each network element to be tested are approximately equal in the network with the unicast service as the main. When the detected network element has the phenomena of packet loss, illegal copying and the like in the process of data routing and switching processing, the balance of the receiving and transmitting flow of the detected network element can be broken.
Therefore, in this embodiment, in the process of detecting the traffic abnormality of one network element to be detected, the receiving traffic and the sending traffic of each port of the network element to be detected may be collected. For example, assuming that there are 4 ports, namely, port a, port b, port c and port d, in a network element to be tested, in the process of detecting the abnormal flow of the network element to be tested, the network device may collect the received flow and the transmitted flow of the port a, collect the received flow and the transmitted flow of the port b, and also collect the transmitted and received flows of the port c and the port d.
In some examples of this embodiment, in the process of detecting a traffic anomaly of a network element under test, the network device may periodically collect the received traffic and the transmitted traffic of each port of the network element under test, for example, in one example, the network device may use 15 minutes as the detection granularity, that is, collect the transmitted traffic and the received traffic of each port of the network element under test once every 15 minutes, alternatively, assuming that the network device collects the transmitted traffic and the received traffic of each port of the network element under test for the first time at 00:00, the network device will collect the transmitted traffic and the received traffic of each port of the network element under test at 00:15 next time, and the time of collecting the third traffic is referred to as the detection time in this embodiment at the time of 00:00, 00:15, and 00:30. Assuming that the current time is 00:15, 00:15 is the current detection time, and 00:00 is the historical detection time.
It can be appreciated that, in other examples of the present embodiment, the network device may not need to periodically perform traffic collection when detecting the traffic abnormality of the network element under test. That is, when the network device performs the traffic receiving and transmitting collection on the network element to be detected, the time interval between each detection time is not completely consistent.
S104: and determining the real-time window deviation ratio of the detected network element in the current time window according to the acquired received flow and the acquired transmitted flow.
After the received traffic and the transmitted traffic of each port of the network element to be detected at the current detection moment are collected, the network device can determine the real-time window deviation ratio of the network element to be detected in the current time window, wherein the real-time window deviation ratio is the traffic deviation ratio of the network element to be detected in the current time window.
First, explanation is made on "flow deviation ratio": for a network element to be tested, the flow deviation ratio of the network element to be tested at a certain testing moment can be the ratio of the sum of the received flows and the sum of the transmitted flows of each port of the network element to be tested at the testing moment, for example,
bias＝N recv /N send ；
wherein bias is the flow deviation ratio, N recv The sum of all received flows at the moment when each port of the network element to be detected detects can be calculated by the following formula:
where n is the total number of ports in the network element under test, and i represents the i-th port,
and the received flow of the ith port of the network element to be detected at the detection moment is indicated.
N send The sum of all received flows at the moment when each port of the network element to be detected detects can be calculated by the following formula:
where n is the total number of ports in the network element under test, and i represents the i-th port, 
And the transmission flow of the ith port of the network element to be detected at the detection moment is indicated.
The flow deviation ratio of a network element to be tested is mainly capable of characterizing the degree of equalization of the sending flow and the receiving flow of each port of the network element to be tested, so that it is needless to say that the flow deviation ratio of the network element to be tested is not necessarily the ratio of the sum of the receiving flows to the sum of the sending flows, but may be the ratio of the sending flow to the receiving flow, that is,
bias＝N send /N recv ；
however, it should be noted that, when determining the flow deviation ratio of the detected network element, the network device should select a uniform flow deviation ratio calculating manner, for example, in some examples, if the network device calculates, when calculating the flow deviation ratio of a certain detected network element for the first time, a ratio of a sum of received flows and a sum of transmitted flows corresponding to each port of the detected network element at the first detection moment, when calculating the flow deviation ratio of the spare network element at the subsequent detection moment, the network device should also calculate a ratio of a sum of received flows and a sum of transmitted flows, and should not suddenly become a ratio of calculating a sum of transmitted flows and a sum of received flows of each port of the detected network element in a certain calculation process. Fig. 2 and 3 show schematic diagrams of flow deviation ratios of two network elements under test in the same day, respectively, wherein the vertical axis bias represents the flow deviation ratio and the horizontal axis represents time.
The current time window is a time window corresponding to the current detection time, and the "real-time window deviation ratio" is actually the flow deviation ratio of the network element to be detected in the current time window. In one example of the present embodiment, only one detection time is included in the time window, and the real-time window deviation ratio of the detected network element in the current time window is actually the flow deviation ratio of the detected network element at the current detection time. However, if one time window includes two or more detection moments at the same time, the real-time window deviation ratio of the detected network element in the current time window is the average value of the flow deviation ratios of the detected network element in the current time window at each detection moment. For example, in one example, the time window includes three detection times, and please combine a schematic diagram of the relationship between the time window and the detection times shown in fig. 4, wherein the vertical axis bias represents the flow deviation ratio, and the horizontal axis represents time:
assuming that the current detection time is the nth detection time, the current time window 401 is a time window corresponding to the nth detection time, which includes both the nth detection time, the (n-1) th detection time, and the (n-2) th detection time. As for the history time window 402, it is a time window corresponding to the previous detection time (i.e., the n-1 th detection time), which includes the n-1 st detection time, the n-2 nd detection time, and the n-3 rd detection time.
Further assume that the flow deviation ratio of the detected network element at the nth detection time, the (n-1) th detection time, the (n-2) th detection time and the (n-3) th detection time is b respectively n 、b n-1 、b n-2 、b n-3 The real-time window deviation ratio of the detected network element is (b) n +b n-1 +b n-2 )/3. The history window deviation ratio of the detected network element is (b) n-1 +b n-2 )+b n-3 )/3。
It can be understood that if the current detection time is the nth detection time, when the network device determines the real-time window deviation ratio, it is only necessary to calculate the flow deviation ratio b of the nth detection time according to the received flow and the transmitted flow acquired by the nth flow n . As for other flow deviation ratio b for calculating the real-time window deviation ratio n-1 And b n-2 Has been calculated in the previous detection process (b n-1 Is calculated when the real-time window deviation ratio is calculated at the n-1 detection time, b n-2 Is calculated at the time of calculating the real-time window deviation ratio at the n-2 th detection instant), and need not be calculated again here.
S106: and determining the steep slope of the current detection moment according to the real-time window deviation ratio and the historical window deviation ratio.
An anomaly is data that deviates from most of the data in the dataset, as defined by Hawkin, and thus outliers are also called outliers. Therefore, in this embodiment, the network device determines whether the current detection time is an abnormal point (i.e. a traffic abnormality time) according to whether the traffic deviation ratio corresponding to the current detection time deviates from the traffic deviation ratio of most detection times.
The flow deviation ratio of the detected network element belongs to a time sequence index, and the main aim of monitoring the index is to find the moment point of deviation from a normal value in time, namely a change point detection problem aiming at the time sequence. The transformation point theory is a classical branch in statistics, and basically defined as a time point when a certain statistical property (distribution type, distribution parameter) is changed by a systematic factor instead of an accidental factor in a sequence or a process, and we refer to the time point as a transformation point. The variable point detection is to find out the variable point by using statistics or a statistical method.
Therefore, after the real-time window deviation ratio corresponding to the current detection time of the network element to be detected is calculated, the network equipment can determine the abrupt slope of the network element to be detected at the current detection time according to the real-time window deviation ratio and the historical window deviation ratio of the network element to be detected, and the abrupt slope can represent the change degree of the real-time window deviation ratio at the current detection time relative to the historical window deviation ratio.
In this embodiment, the steep slope of the current detection time may be determined according to the following formula:
wherein n represents the nth detection time and M n Representing the flow deviation ratio of the detected network element in the time window corresponding to the nth detection moment, and M n-1 Represents the flow deviation ratio, K, of the detected network element in a time window corresponding to the n-1 th detection moment n Is the steep slope of the nth detection time. If the current n-th detection time is the nth detection time, K n The steep slope corresponding to the current detection moment is obtained.
It can be understood that, for the history window deviation ratio of the network element to be detected, the history window deviation ratio is calculated after the network device performs the n-1 th time of receiving and transmitting traffic collection on the network element to be detected. Therefore, in this embodiment, after calculating the real-time window deviation ratio corresponding to the nth detection time, the network device records the real-time window deviation ratio so as to participate in calculation as the historical window deviation ratio at the (n+1) th time.
S108: and determining whether the current detection time is the abnormal flow time or not based on the abrupt change slope of the current detection time.
After calculating the steep slope corresponding to the detected network element at the current detection time, the network device may determine, according to the steep slope, whether the detected network element is abnormal in flow at the current detection time, that is, whether the current detection time is abnormal in flow.
It can be understood that if the flow of the network element under test at the nth detection time is normal, the steep slope at the nth detection time is close to 1. In some examples of this embodiment, parameters are stored in the network device that can divide the normal slope threshold, here assuming a normal slope range (1/Q, Q), where Q is a positive number, so the values in (1/Q, Q) are all relatively close to 1.
Naturally, the union of the range less than 1/Q and the range greater than Q belongs to the abnormal slope range. Therefore, in some examples of this embodiment, when determining whether the detected network element is abnormal in flow at the nth detection time, the network device may determine whether the steep slope of the detected network element at the nth detection time is within a normal slope range, and if so, determine that the detection time is not abnormal in flow; if not, the detection time is determined to be the abnormal flow time of the detected network element.
In some examples of this embodiment, the value of Q may be fixed, for example, set by a network operator according to a large number of experience values, and it is understood that Q set by the network operator should ensure that all network anomalies of the network element under test can be detected as accurately as possible. In other examples of this embodiment, the value of Q may be adaptively adjusted, for example, the initial value of Q is set by the network operation and maintenance personnel according to experience, but as the network device continuously detects the abnormal flow rate of the network element to be detected, the network device may adjust the value of Q according to the accuracy of its detection result, so as to reduce the false detection and/or missing detection in the abnormal flow rate detection process, please refer to a flowchart for adjusting the normal slope range shown in fig. 5:
S502: and adding each abnormal flow moment detected in a certain time period to the automatic labeling abnormal set.
In this embodiment, the network device may adjust the value of Q once at intervals, and it is needless to say that adjusting the value of Q actually adjusts the normal slope range. It is assumed that the network device set here adjusts the normal slope range every two hours.
Within these two hours, if the time difference between two adjacent detection times of the network device is 15 minutes, the network device may have detected the network element to be detected 8 times, and a part of detection times in these 8 times may be determined as traffic abnormality times. The network device may add the traffic anomaly time of the 8 detection times to an automatic anomaly labeling set, where the automatic anomaly labeling set is a set of traffic anomaly times that are labeled by the network device in a mechanized manner.
It will be appreciated that if the ratio of real-time window deviation at one detection instant is relatively large compared to the steep slope of the ratio of historical window deviation, two situations may exist:
first, the real-time window deviation is better than the characterized flow condition than the historical window deviation ratio, that is, the absolute difference between the real-time window deviation ratio and 1 is smaller than the absolute difference between the historical window deviation ratio and 1, which indicates that the flow abnormality is detected at the current detection moment, but the flow abnormality is gradually recovered. Therefore, the abnormal flow rate at the current detection time is actually in the recovery state.
Second, the real-time window deviation is worse than the characterized flow condition, i.e., the absolute difference of the real-time window deviation ratio from 1 is greater than the absolute difference of the historical window deviation ratio from 1, than the historical window deviation ratio, indicating that there is a flow anomaly at the current detection time and that the anomaly is deteriorating, so the flow anomaly at the current detection time is actually deteriorating.
In the process of detecting the traffic anomalies of the network element to be detected, the network device only needs to pay attention to the points where the anomalies occur or the points where the anomalies are worsened, and the network device does not need to pay attention to the traffic anomaly moments with the restoration trend, so in some examples of the embodiment, the network device may determine the automatic labeling anomaly set by referring to the flowchart shown in fig. 6:
s602: the network equipment determines all abnormal flow moments in a time period;
s602: for each flow abnormal time in the time period, the network equipment determines whether the abnormality of the flow abnormal time is in a recovery state or a deterioration state according to the abrupt slope of the flow abnormal time and the historical abrupt slope;
s606: the network equipment eliminates the abnormal flow time in the recovery state, and takes the rest abnormal flow time as an automatic labeling abnormal set.
S504: and comparing each flow abnormal time in the manual labeling abnormal set with each flow abnormal time in the automatic labeling abnormal set.
On the other hand, in order to check the accuracy of the network device in detecting the abnormal flow time in the two hours, the network device also obtains a manual abnormal labeling set corresponding to the automatic abnormal labeling set, wherein the manual abnormal labeling set is a labeling result of manually labeling the abnormal flow time in the same two hours. The abnormal flow moments in the abnormal manual labeling set can be regarded as completely correct, and the condition of error labeling does not exist; and the set of manually marked anomalies is considered to contain all flow anomaly moments within the two hours, and no condition of missing marks exists.
After the manual labeling abnormal set and the automatic labeling abnormal set are obtained, the network equipment can compare each flow abnormal time in the manual labeling abnormal set with each flow abnormal time in the automatic labeling abnormal set.
If the network device needs to check the error labeling condition in the automatic labeling abnormal set, namely determining the false detection rate of the automatic labeling abnormal set, the network device can determine the false detection abnormality in the automatic labeling abnormal set, wherein the false detection abnormality is actually the flow abnormality moment existing in the automatic labeling abnormal set but not existing in the manual labeling abnormal set.
If the network equipment needs to check the missed labeling condition in the automatic labeling abnormal set, namely determining the missed detection rate of the automatic labeling abnormal set, the network equipment can determine that the missed detection abnormality in the automatic labeling abnormal set exists in the manual labeling abnormal set, but the missed detection abnormality is the flow abnormality moment which does not exist in the automatic labeling abnormal set.
S506: and adjusting a normal slope range according to the comparison result.
In some examples of this embodiment, the network device may determine its false positive rate according to the following formula:
if the network equipment determines that the false detection rate in the automatic labeling anomaly set reaches the preset false detection threshold, the steep slope corresponding to each false detection anomaly can be determined, then the network equipment determines the Q value corresponding to each steep slope, and then the largest Q value is selected as the adjusted Q value. For example, if the network device determines that the false detection rate in the automatic labeling anomaly set reaches the preset false detection threshold, including 3 false detection anomalies, the steep slopes corresponding to the three false detection anomalies are 1.5, 2 and 2.5, respectively, and Q corresponding to the three steep slopes is 1.5, 2 and 2.5, respectively, so that the updated Q value is 2.5. For another example, if the steep slopes corresponding to the 3 false detection anomalies are 1/4, 1/3 and 1/2, respectively, then the Q corresponding to the three steep slopes are 4, 3 and 2, respectively, and therefore, the updated Q value is 4.
By this adjustment, the Q value is increased, and the normal slope range is also increased, thereby reducing the possibility that the traffic normal time is detected as an abnormal traffic time by the network device.
The network device may also determine its own miss rate according to the following formula:
if the network equipment determines that the omission ratio in the automatic labeling anomaly set reaches a preset omission threshold, the steep slope corresponding to each omission anomaly can be determined, and then the Q value is updated according to the minimum value in each steep slope. Alternatively, the network device may determine the Q value corresponding to each steep slope, and then select the smallest one of the Q values as the updated Q value. For example, if the network device determines that the miss rate in the automatically labeled anomaly set reaches the preset miss threshold, including 3 miss anomalies, the steep slopes corresponding to the three miss anomalies are 1/2, 1/3, and 3, respectively, and the Q values corresponding to the three steep slopes are 2, 3, and 3, respectively, in order to be able to identify the 3 miss anomalies as traffic anomaly moments, the network device may adjust the Q value to 2, it should be understood that the Q value must be greater than 3 before adjustment, so that the adjustment actually reduces the Q value, and increases the anomaly slope range, thereby reducing the likelihood that the traffic anomaly moments cannot be detected correctly by the network device.
According to the flow anomaly detection method provided by the embodiment, the flow deviation ratio of the detected network element is analyzed, the moment with larger change of the flow deviation ratio is identified based on the abrupt change slope to serve as the flow anomaly moment, and the flow anomaly detection scheme can effectively identify the anomaly points which cannot cause the flow out-of-limit of the detected network element, so that a foundation is provided for network optimization.
Furthermore, the network device can adjust the parameters for judging the abnormal flow time according to the labeling result of the abnormal flow, so that the parameters for judging the abnormal flow time are more accurate and more fit with the actual condition of the network, thereby improving the detection accuracy of the abnormal flow and reducing the false detection and omission.
Embodiment two:
the present embodiment will be further described with reference to the flow anomaly detection method in the foregoing embodiment, and please refer to the flowchart shown in fig. 7:
s702: and collecting the received flow and the transmitted flow of each port of the network element to be detected at the current detection moment.
It can be understood that one network device for monitoring abnormal traffic may monitor traffic of two or more network elements to be tested at the same time, so when the network device collects port traffic, the network device performs port traffic collection for all ports of all network elements to be tested monitored by the network device, and after obtaining the collection result, the network device needs to determine which network element to which the collection result belongs according to the asset relationship data (which can represent the correspondence between the network elements to be tested and the ports). Then, the abnormal flow moment is determined for each checked network element.
In this embodiment, the network device may perform detection every 15 minutes, that is, the detection granularity is 15 minutes. It will be appreciated that if the detection granularity is set too large, it may result in the network device failing to detect which anomalies occur and recover in a short time, for example, if the detection granularity is set to 3 hours as shown in fig. 8, the network device fails to detect the anomalies shown in fig. 8.
Of course, although the smaller the network device sets the detection granularity, the more fine anomalies can be detected, this also results in the detection frequency of the network device becoming higher, thus occupying more processing resources. Thus, the network device may set according to its own processing power when setting the detection granularity.
S704: and judging whether the receiving flow and the sending flow of at least two ports of the network element to be detected are acquired or not.
It will be appreciated that, after the network device performs the receiving traffic collection and the sending traffic collection on each port of the network element to be inspected, there may be a case that: the inspected network element contains a set of ports, only one of which has valid traffic data. This situation is an anomaly in the actual network itself, because the normal network element device acts as a data switching node, there are at least two working ports to complete the source and destination data switching, and the phenomenon of single port self-switching may actually be caused by some faults that cause some port traffic to be not collected. Meanwhile, if the network device observes the data of the network element which is only valid by the single port data, the network device finds that the fluctuation of the received and transmitted traffic data is larger, and the traffic deviation is larger than the fluctuation of the index data. It can be appreciated that a large steep slope is easily generated in the fluctuation process, so that the current detection moment is easily recognized as the abnormal moment of the flow by the network equipment. However, in the case of partial port data loss, such a traffic deviation ratio anomaly on a single port is not actually a network element traffic deviation ratio anomaly that we want to acquire, and therefore, in some examples of this embodiment, the network device excludes a network element for which only single port data is valid when needed.
Therefore, if the determination result of the network device is yes, the network device continues to execute S706, otherwise the flow ends.
S706: and judging whether the received traffic and the transmitted traffic of each port of the network element to be detected at the current detection moment are all zero values.
Due to various factors, the traffic data collected by the network device is inevitably missed, for example, a detected network element may have the situation that the receiving traffic and the sending traffic of each port are zero. According to the traditional big data processing mode, for the situation of data missing, missing value filling is a common means. Common missing value filling methods are: front-to-back mean filling, mode filling, linear regression filling, etc. For machine learning training data, the missing value filling is generally effective, but for the flow anomaly detection scheme in this embodiment, the missing value filling tends to have a great influence on the detection result. In fact, most filling methods make the data smoother, so that the network device cannot detect the original abnormal point according to the steep rise and fall of the flow deviation ratio.
Therefore, in the present embodiment, if the network device determines that the received traffic and the transmitted traffic of each port of the network element to be inspected at the current detection time are all zero values, S718 is performed.
S708: and determining the real-time window deviation ratio of the detected network element in the current time window according to the acquired received flow and the acquired transmitted flow.
After collecting the received traffic and the transmitted traffic of each port of the network element to be detected at the current detection moment, the network device can determine the real-time window deviation ratio of the network element to be detected in the current time window. In one example of this embodiment, three detection times are included in one time window, so the network device may determine the real-time window deviation ratio corresponding to the current detection time according to the flow deviation ratio of the current detection time and the flow deviation ratio of the previous two detection times.
S710: and determining the steep slope of the current detection moment according to the real-time window deviation ratio and the historical window deviation ratio.
After calculating the real-time window deviation ratio corresponding to the current detection time of the network element to be detected, the network equipment can calculate the ratio of the real-time window deviation ratio to the historical window deviation ratio of the network element to be detected, so as to obtain the abrupt change slope of the network element to be detected at the current detection time.
S712: and judging whether the current detection time is the abnormal flow time or not based on the abrupt change slope of the current detection time.
After calculating the steep slope corresponding to the detected network element at the current detection time, the network device can determine whether the flow of the detected network element is abnormal at the current detection time according to the steep slope. Optionally, the network device judges whether the steep slope corresponding to the current detection time is in a normal slope range (1/Q, Q), if yes, the detection time is not the abnormal flow time; if not, the detection time is determined to be the abnormal flow time of the detected network element.
In some examples of this embodiment, if the network device determines that the current detection time is a traffic abnormality time of the detected network element, the network device further determines whether the traffic abnormality time is a traffic abnormality time in a deteriorated state:
s714: and judging that the abnormality of the flow abnormal moment is in a deteriorated state according to the abrupt change slope and the historical abrupt change slope of the flow abnormal moment.
If yes, S716 is executed, otherwise S718 is not executed.
S716: recording the abnormal flow moment.
If the abnormality of the flow abnormal moment is in a deteriorated state according to the abrupt change slope and the historical abrupt change slope of the flow abnormal moment, the network equipment can record the flow abnormal moment for use in the subsequent network optimization process.
S718: and judging whether a new detection time is reached.
If yes, continuing to execute S702, otherwise continuing to judge.
In this embodiment, the network device may further evaluate the false detection rate and the missing detection rate at intervals, and adjust the normal slope range according to the duration of the evaluation result, so as to reduce the false detection rate and the missing detection rate at abnormal moments of the flow in the subsequent detection process. The specific evaluation adjustment process has been described in more detail in the foregoing embodiments, and will not be described here again.
According to the flow abnormality detection method provided by the embodiment, the network equipment automatically monitors the flow of the detected network element and marks the flow abnormality time, so that the demand for human resources is reduced, the detection process is rapid and efficient, and the flow abnormality condition which cannot be detected by the conventional mode can be found. Meanwhile, parameters for judging abnormal flow moment can be adjusted according to feedback of the detection result, so that the detection result can reach higher accuracy and reliability.
Embodiment III:
the present embodiment provides a flow anomaly detection device, please refer to the structural schematic diagram shown in fig. 9:
the flow anomaly detection device 90 includes a flow acquisition module 902, a deviation determination module 904, a slope determination module 906, and an anomaly determination module 908, where the flow acquisition module 902 is configured to acquire a received flow and a transmitted flow of each port of a network element under test at a current detection time; the deviation determining module 904 is configured to determine a real-time window deviation ratio of the detected network element in the current time window according to the collected received traffic and the collected transmission traffic, and the slope determining module 906 is configured to determine a steep slope at the current detection time according to the real-time window deviation ratio and a historical window deviation ratio, where the historical window deviation ratio is a traffic deviation ratio of the detected network element in a time window corresponding to the previous detection time; the anomaly determination module 908 is configured to determine whether the current detection time is a flow anomaly time based on a steep slope of the current detection time.
In some other examples of this embodiment, referring to fig. 10, the traffic anomaly detection device 90 further includes a preprocessing module 910, configured to determine whether the traffic acquisition module 902 acquires at least the received traffic and the transmitted traffic of at least two ports of the network element under test. Only if the result of the determination by the preprocessing module 910 is yes, the deviation determining module 904 performs calculation of the real-time window deviation ratio.
Alternatively, the preprocessing module 910 may be configured to determine whether the received traffic and the transmitted traffic of each port of the network element to be detected at the current detection time are all zero values. Only if the determination result of the preprocessing module 910 is no, the deviation determining module 904 performs calculation of the real-time window deviation ratio.
In some examples of this embodiment, one time window includes at least two detection times, and the deviation determining module 904 may determine a flow deviation ratio at the current detection time according to the received flow and the transmitted flow acquired at the current detection time, obtain flow deviation ratios at other detection times in the current time window, and then determine a flow deviation ratio average value of the current time window as a real-time window deviation ratio according to the flow deviation ratios at the detection times in the current time window.
The flow deviation ratio at the current detection moment is the ratio of the sum of the received flow and the sum of the transmitted flow of each port of the detected network element acquired at the current detection moment.
In some examples of this embodiment, the anomaly determination module 908 may determine whether the steep slope at the current detection time is outside of a normal slope range (1/Q, Q), where Q is a positive number; if yes, judging the current detection time as the abnormal flow time, and if not, judging that the current detection time is not the abnormal flow time.
In some examples of the present embodiment, as shown in fig. 11, the flow anomaly detection apparatus 90 may further include a parameter adjustment module 912, where the parameter adjustment module 912 is configured to evaluate the multiple determination results of the anomaly determination module 908, and adjust the normal slope range of the anomaly determination module 908 for determining whether a detection moment is a flow anomaly moment according to at least one of the false detection rate and the omission rate.
Optionally, the parameter adjustment module 912 may add each abnormal flow time detected in a certain period to the automatic labeling abnormal set, then compare each abnormal flow time in the manual labeling abnormal set with each abnormal flow time in the automatic labeling abnormal set, and adjust the normal slope range according to the comparison result.
In an example of this embodiment, the parameter adjustment module 912 determines that the false detection in the automatic labeling anomaly set is abnormal, then determines whether the false detection rate in the automatic labeling anomaly set reaches a preset false detection threshold, if so, the parameter adjustment module 912 further determines a steep slope corresponding to each false detection anomaly, and adjusts the Q value according to a maximum value in each steep slope.
In an example of this embodiment, the parameter adjustment module 912 determines that the failure detection in the automatic labeling failure set reaches the preset failure detection threshold, if yes, the parameter adjustment module 912 further determines a steep slope corresponding to each failure detection failure, and adjusts the Q value according to a maximum value in each steep slope.
Alternatively, the parameter adjustment module 912 may determine all abnormal flow moments in a period, then determine, for each abnormal flow moment in the period, whether the abnormality of the abnormal flow moment is in a recovery state or a worsened state according to the steep slope of the abnormal flow moment and the steep slope of the history, and then reject the abnormal flow moment in the recovery state, and use the remaining abnormal flow moment as the automatic labeling abnormality set.
In this embodiment, the traffic anomaly detection apparatus 90 may be deployed on a network device, for example, a network device in a carrier network, where the functions of the traffic acquisition module 902 may be implemented by a processor of the network device together with communication clouds, and the functions of the deviation determination module 904, the slope determination module 906, the anomaly determination module 908, the preprocessing module 910, and the parameter adjustment module 912 may be implemented by a processor of the network device.
For other details of the flow anomaly detection device implementing the flow anomaly detection method, please refer to the description of the foregoing embodiment, and the details are not repeated here.
According to the flow anomaly detection device provided by the embodiment, the flow deviation ratio of the detected network element is analyzed, the moment with larger flow deviation ratio change is identified based on the abrupt change slope to serve as the flow anomaly moment, and the flow anomaly detection scheme can effectively identify the anomaly points which cannot cause the flow out-of-limit of the detected network element, so that a foundation is provided for network optimization.
Furthermore, the flow abnormality detection device can adjust parameters for judging flow abnormality time according to the labeling result of the flow abnormality, so that the parameters for judging the flow abnormality time are more accurate and more fit with the actual condition of the network, thereby improving the flow abnormality detection accuracy and reducing the false detection and omission detection conditions.
Embodiment four:
the present embodiment provides a storage medium in which one or more computer programs that can be read, compiled and executed by one or more processors may be stored, and in this embodiment, the storage medium may store a flow anomaly detection program that can be used by one or more processors to execute a flow for implementing any of the flow anomaly detection methods described in the foregoing embodiments.
In addition, the present embodiment provides a network device, as shown in fig. 12: the network device 120 includes a processor 121, a memory 122, and a communication bus 123 for connecting the processor 121 and the memory 122, where the memory 122 may be a storage medium storing the foregoing traffic abnormality detection program. The processor 121 may read the flow anomaly detection program, compile and execute the flow of implementing the flow anomaly detection method in the foregoing embodiment:
the processor 121 collects the received traffic and the transmitted traffic of each port of the network element to be detected at the current detection time, determines the real-time window deviation ratio of the network element to be detected in the current time window according to the collected received traffic and the transmitted traffic, then determines the abrupt slope of the current detection time according to the real-time window deviation ratio and the historical window deviation ratio, and determines whether the current detection time is the abnormal traffic time based on the abrupt slope of the current detection time.
In other examples of this embodiment, the processor 121 is further configured to determine whether at least the received traffic and the transmitted traffic of at least two ports of the network element to be inspected are collected. Only if the determination is yes, the processor 121 performs calculation of the real-time window deviation ratio.
Alternatively, the processor 121 may further determine whether the received traffic and the transmitted traffic of each port of the network element to be detected at the current detection time are all zero values. Only if the determination is negative, the processor 121 calculates the real-time window deviation ratio.
In some examples of this embodiment, one time window includes at least two detection times, and the processor 121 may determine a flow deviation ratio at the current detection time according to the received flow and the transmitted flow acquired at the current detection time, obtain flow deviation ratios at other detection times in the current time window, and then determine a flow deviation ratio average value of the current time window as a real-time window deviation ratio according to the flow deviation ratios at the detection times in the current time window.
The flow deviation ratio at the current detection moment is the ratio of the sum of the received flow and the sum of the transmitted flow of each port of the detected network element acquired at the current detection moment.
In some examples of this embodiment, the processor 121 may determine whether the steep slope at the current detection time is outside the normal slope range (1/Q, Q), where Q is a positive number; if yes, judging the current detection time as the abnormal flow time, and if not, judging that the current detection time is not the abnormal flow time.
In some examples of the present embodiment, the processor 121 may further evaluate the own multiple determination results, adjust a normal slope range for determining whether one detection timing is a flow anomaly timing according to at least one of the false positive rate and the false negative rate, and adjust.
Alternatively, the processor 121 may add each abnormal flow time detected in a certain period to the automatic labeling abnormal set, then compare each abnormal flow time in the manual labeling abnormal set with each abnormal flow time in the automatic labeling abnormal set, and adjust the normal slope range according to the comparison result.
In one example of the present embodiment, the processor 121 determines that the false detection in the automatic labeling anomaly set is abnormal, then determines whether the false detection rate in the automatic labeling anomaly set reaches a preset false detection threshold, if so, the processor 121 further determines a steep slope corresponding to each false detection anomaly, and adjusts the Q value according to the maximum value of each steep slope.
In one example of the present embodiment, the processor 121 determines that the missing detection in the automatic labeling anomaly set is abnormal, then determines that the missing detection rate in the automatic labeling anomaly set reaches a preset missing detection threshold, if yes, the processor 121 further determines a steep slope corresponding to each missing detection anomaly, and adjusts the Q value according to each steep slope.
Alternatively, the processor 121 may determine all abnormal flow moments within a period, then determine, for each abnormal flow moment within the period, whether the abnormality of the abnormal flow moment is in a recovery state or a deteriorated state according to the steep slope of the abnormal flow moment and the steep slope of the history, and then reject the abnormal flow moment in the recovery state, and use the remaining abnormal flow moment as the automatically marked abnormality set.
For other details of the flow anomaly detection device implementing the flow anomaly detection method, please refer to the description of the foregoing embodiment, and the details are not repeated here.
The network equipment provided by the embodiment can automatically monitor the flow of the network element to be detected and label the abnormal flow time, so that the demand for human resources is reduced, the detection process is rapid and efficient, and abnormal flow conditions which cannot be detected by the conventional mode can be found. Meanwhile, parameters for judging abnormal flow moment can be adjusted according to feedback of the detection result, so that the detection result can reach higher accuracy and reliability.
It is to be understood that features of embodiments of the invention may be used in combination without conflict.
It will be apparent to one skilled in the art that all or some of the steps of the methods, systems, functional modules/units in the apparatus disclosed above may be implemented as software (which may be implemented in program code executable by a computing apparatus), firmware, hardware, and suitable combinations thereof. In a hardware implementation, the division between the functional modules/units mentioned in the above description does not necessarily correspond to the division of physical components; for example, one physical component may have multiple functions, or one function or step may be performed cooperatively by several physical components. Some or all of the physical components may be implemented as software executed by a processor, such as a central processing unit, digital signal processor, or microprocessor, or as hardware, or as an integrated circuit, such as an application specific integrated circuit. Such software may be distributed on computer-readable media, which may include computer storage media (or non-transitory media) and communication media (or transitory media), for execution by a computing device, and in some cases, the steps shown or described may be performed in a different order than that described herein. The term computer storage media includes both volatile and nonvolatile, removable and non-removable media implemented in any method or technology for storage of information such as computer readable instructions, data structures, program modules or other data, as known to those skilled in the art. Computer storage media includes, but is not limited to, RAM, ROM, EEPROM, flash memory or other memory technology, CD-ROM, digital Versatile Disks (DVD) or other optical disk storage, magnetic cassettes, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other medium which can be used to store the desired information and which can be accessed by a computer. Furthermore, as is well known to those of ordinary skill in the art, communication media typically embodies computer readable instructions, data structures, program modules or other data in a modulated data signal such as a carrier wave or other transport mechanism and includes any information delivery media. Therefore, the present invention is not limited to any specific combination of hardware and software.
The foregoing is a further detailed description of embodiments of the invention in connection with the specific embodiments, and it is not intended that the invention be limited to the specific embodiments described. It will be apparent to those skilled in the art that several simple deductions or substitutions may be made without departing from the spirit of the invention, and these should be considered to be within the scope of the invention.
Claims (12)
1. A traffic anomaly detection method comprising:
collecting the received flow and the transmitted flow of each port of the network element to be detected at the current detection moment;
determining a real-time window deviation ratio of the detected network element in a current time window according to the acquired receiving flow and the transmitting flow, wherein the current time window is a time window corresponding to the current detection moment, the real-time window deviation ratio is a flow deviation ratio of the detected network element in the current time window, and the flow deviation ratio can represent the balance degree of the receiving flow and the transmitting flow; the flow deviation ratio at a certain detection moment is the ratio of the sum of the received flow and the sum of the transmitted flow of each port of the detected network element at the certain detection moment;
determining the steep slope of the current detection moment according to the real-time window deviation ratio and the historical window deviation ratio, wherein the historical window deviation ratio is the flow deviation ratio of the detected network element in a time window corresponding to the previous detection moment;
And determining whether the current detection time is the abnormal flow time or not based on the steep slope of the current detection time.
2. The traffic anomaly detection method according to claim 1, wherein before determining a real-time window deviation ratio of the detected network element in a current time window according to the acquired received traffic and the transmission traffic, the method further comprises:
and determining the received traffic and the transmitted traffic of at least two ports of the network element to be detected.
3. The traffic anomaly detection method according to claim 1, wherein before determining a real-time window deviation ratio of the detected network element in a current time window according to the acquired received traffic and the transmission traffic, the method further comprises:
and determining that the received traffic and the transmitted traffic of each port of the network element to be detected at the current detection moment are not all zero values.
4. The traffic anomaly detection method according to claim 1, wherein a time window includes at least two detection moments, and the determining a real-time window deviation ratio of the detected network element in a current time window according to the collected received traffic and the transmission traffic includes:
Determining a flow deviation ratio of the current detection moment according to the received flow and the transmitted flow acquired at the current detection moment, and acquiring flow deviation ratios of other detection moments in the current time window;
and determining the average value of the flow deviation ratios of the current time window as the real-time window deviation ratio according to the flow deviation ratio of each detection moment in the current time window.
5. The flow anomaly detection method according to any one of claims 1 to 4, wherein the determining whether the current detection time is a flow anomaly time based on the steep slope of the current detection time includes:
judging whether the steep slope at the current detection moment is out of a normal slope range or not, wherein Q is a positive number in the normal slope range (1/Q, Q);
if yes, judging the current detection time as the abnormal flow time, and if not, judging that the current detection time is not the abnormal flow time.
6. The flow anomaly detection method according to claim 5, further comprising:
adding each abnormal flow moment detected in a certain time period to an automatic labeling abnormal set;
comparing each flow abnormal time in the manual labeling abnormal set with each flow abnormal time in the automatic labeling abnormal set;
And adjusting the normal slope range according to the comparison result.
7. The flow anomaly detection method of claim 6, wherein comparing each flow anomaly time in the manually labeled anomaly set with each flow anomaly time in the automatically labeled anomaly set comprises: determining false detection abnormality in the automatic labeling abnormality set, wherein the false detection abnormality is flow abnormality moment which exists in the automatic labeling abnormality set but does not exist in the manual labeling abnormality set;
the adjusting the normal slope range according to the comparison result comprises:
determining that the false detection rate in the automatic labeling abnormal set reaches a preset false detection threshold, wherein the false detection rate is the number of false detection abnormalities in the automatic labeling abnormal set/the total number of flow abnormal moments in the automatic labeling abnormal set;
determining the steep slope corresponding to each false detection abnormality;
and adjusting the Q value according to each abrupt change slope.
8. The flow anomaly detection method of claim 6, wherein comparing each flow anomaly time in the manually labeled anomaly set with each flow anomaly time in the automatically labeled anomaly set comprises: determining missed detection anomalies in the automatic labeling anomaly set, wherein the missed detection anomalies are flow anomalies which exist in the manual labeling anomaly set but do not exist in the automatic labeling anomaly set;
The adjusting the normal slope range according to the comparison result comprises:
determining that the miss rate in the automatic labeling abnormal set reaches a preset miss threshold, wherein the miss rate is the number of miss abnormalities in the automatic labeling abnormal set/(the total number of flow abnormality moments+the number of miss abnormalities in the automatic labeling abnormal set);
determining the steep slope corresponding to each missing detection abnormality;
and adjusting the Q value according to each abrupt change slope.
9. The flow anomaly detection method according to claim 6, wherein adding each flow anomaly time detected within a certain period of time to the automatic labeling anomaly set comprises:
determining all abnormal flow moments in the time period;
for each abnormal flow time in the time period, determining whether the abnormality of the abnormal flow time is in a recovery state or a deterioration state according to the abrupt slope and the historical abrupt slope of the abnormal flow time;
and eliminating the abnormal flow time in the recovery state, and taking the rest abnormal flow time as an automatic labeling abnormal set.
10. A flow anomaly detection device comprising:
the flow acquisition module is used for acquiring the received flow and the transmitted flow of each port of the network element to be detected at the current detection moment;
The deviation determining module is used for determining a real-time window deviation ratio of the detected network element in a current time window according to the acquired received flow and the transmission flow, wherein the current time window is a time window corresponding to the current detection moment, the real-time window deviation ratio is a flow deviation ratio of the detected network element in the current time window, and the flow deviation ratio can represent the equalization degree of the received flow and the transmission flow; the flow deviation ratio at a certain detection moment is the ratio of the sum of the received flow and the sum of the transmitted flow of each port of the detected network element at the certain detection moment;
the slope determining module is used for determining the steep slope of the current detection moment according to the real-time window deviation ratio and the historical window deviation ratio, wherein the historical window deviation ratio is the flow deviation ratio of the detected network element in the time window corresponding to the previous detection moment;
and the abnormality judgment module is used for determining whether the current detection moment is the abnormal moment of the flow or not based on the abrupt change slope of the current detection moment.
11. A network device comprising a processor, a memory, and a communication bus;
the communication bus is used for realizing connection communication between the processor and the memory;
The processor is configured to execute one or more programs stored in a memory to implement the steps of the flow anomaly detection method according to any one of claims 1 to 9.
12. A storage medium storing one or more programs executable by one or more processors to implement the steps of the flow anomaly detection method of any one of claims 1 to 9.
Priority Applications (2)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910640994.6A CN112242971B (en) | 2019-07-16 | 2019-07-16 | Traffic abnormality detection method and device, network equipment and storage medium |
| PCT/CN2020/096847 WO2021008296A1 (en) | 2019-07-16 | 2020-06-18 | Traffic abnormality detection method and apparatus, network device, and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910640994.6A CN112242971B (en) | 2019-07-16 | 2019-07-16 | Traffic abnormality detection method and device, network equipment and storage medium |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN112242971A CN112242971A (en) | 2021-01-19 |
| CN112242971B true CN112242971B (en) | 2023-06-16 |
Family
ID=74166749
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201910640994.6A Active CN112242971B (en) | 2019-07-16 | 2019-07-16 | Traffic abnormality detection method and device, network equipment and storage medium |
Country Status (2)
| Country | Link |
|---|---|
| CN (1) | CN112242971B (en) |
| WO (1) | WO2021008296A1 (en) |
Families Citing this family (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114373308B (en) * | 2021-11-30 | 2023-03-31 | 深圳市顺易通信息科技有限公司 | Method and device for determining total effective parking space quantity and storage medium |
| CN114285612B (en) * | 2021-12-14 | 2023-09-26 | 北京天融信网络安全技术有限公司 | Method, system, device, equipment and medium for detecting abnormal data |
| CN114745304B (en) * | 2022-04-27 | 2024-02-27 | 北京广通优云科技股份有限公司 | Service mutation point identification method based on network behavior parameters in IT operation and maintenance system |
| CN114979828B (en) * | 2022-05-18 | 2023-03-10 | 成都安讯智服科技有限公司 | Internet of things communication module flow control method and system based on Modbus |
| CN116915517B (en) * | 2023-09-14 | 2023-11-24 | 厦门快快网络科技有限公司 | Cloud service resource risk security management method |
| CN117650791B (en) * | 2024-01-30 | 2024-04-05 | 苏芯物联技术(南京)有限公司 | Welding history airflow data compression method integrating welding process mechanism |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2018035765A1 (en) * | 2016-08-24 | 2018-03-01 | 深圳天珑无线科技有限公司 | Method and apparatus for detecting network abnormity |
| CN108390864A (en) * | 2018-02-01 | 2018-08-10 | 杭州安恒信息技术股份有限公司 | A kind of Trojan detecting method and system based on attack chain behavioural analysis |
| CN108989135A (en) * | 2018-09-29 | 2018-12-11 | 新华三技术有限公司合肥分公司 | Network equipment failure detection method and device |
| CN109327345A (en) * | 2017-08-01 | 2019-02-12 | 中国移动通信集团湖北有限公司 | The detection method and device of exception flow of network, computer readable storage medium |
| CN109951491A (en) * | 2019-03-28 | 2019-06-28 | 腾讯科技（深圳）有限公司 | Network attack detecting method, device, equipment and storage medium |
Family Cites Families (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101399709B (en) * | 2007-09-28 | 2011-06-29 | 福建星网锐捷网络有限公司 | Method, device and system for network monitoring |
| CN101184097A (en) * | 2007-12-14 | 2008-05-21 | 北京大学 | Method of detecting worm activity based on flux information |
| US9843488B2 (en) * | 2011-11-07 | 2017-12-12 | Netflow Logic Corporation | Method and system for confident anomaly detection in computer network traffic |
| CN104506482B (en) * | 2014-10-10 | 2018-09-11 | 香港理工大学 | Network attack detecting method and device |
| CN107332723B (en) * | 2016-04-28 | 2020-09-04 | 华为技术有限公司 | Detection method and detection equipment for hidden channel |
-
2019
- 2019-07-16 CN CN201910640994.6A patent/CN112242971B/en active Active
-
2020
- 2020-06-18 WO PCT/CN2020/096847 patent/WO2021008296A1/en active Application Filing
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2018035765A1 (en) * | 2016-08-24 | 2018-03-01 | 深圳天珑无线科技有限公司 | Method and apparatus for detecting network abnormity |
| CN109327345A (en) * | 2017-08-01 | 2019-02-12 | 中国移动通信集团湖北有限公司 | The detection method and device of exception flow of network, computer readable storage medium |
| CN108390864A (en) * | 2018-02-01 | 2018-08-10 | 杭州安恒信息技术股份有限公司 | A kind of Trojan detecting method and system based on attack chain behavioural analysis |
| CN108989135A (en) * | 2018-09-29 | 2018-12-11 | 新华三技术有限公司合肥分公司 | Network equipment failure detection method and device |
| CN109951491A (en) * | 2019-03-28 | 2019-06-28 | 腾讯科技（深圳）有限公司 | Network attack detecting method, device, equipment and storage medium |
Also Published As
| Publication number | Publication date |
|---|---|
| CN112242971A (en) | 2021-01-19 |
| WO2021008296A1 (en) | 2021-01-21 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN112242971B (en) | Traffic abnormality detection method and device, network equipment and storage medium | |
| CN111126824B (en) | Multi-index correlation model training method and multi-index anomaly analysis method | |
| US9921943B2 (en) | Predicting anomalies and incidents in a computer application | |
| US9015312B2 (en) | Network management system and method for identifying and accessing quality of service issues within a communications network | |
| KR102418969B1 (en) | System and method for predicting communication apparatuses failure based on deep learning | |
| US8868736B2 (en) | Estimating a severity level of a network fault | |
| US8837302B2 (en) | Mapping a network fault | |
| CN101189895B (en) | Abnormality detecting method and system, and upkeep method and system | |
| US9003460B2 (en) | Network monitoring with estimation of network path to network element location | |
| US9871582B2 (en) | Optical channel telemetry | |
| US10404535B2 (en) | Method for managing the configuration of a wireless connection used to transmit sensor readings from a sensor to a data collection facility | |
| CN107872457B (en) | Method and system for network operation based on network flow prediction | |
| EP3667952B1 (en) | Method, device, and storage medium for locating failure cause | |
| CN115038088B (en) | Intelligent network security detection early warning system and method | |
| CN115687447A (en) | Marine environment monitoring system and method based on Internet of things | |
| CN112752172B (en) | Optical channel fault diagnosis method and system based on transfer learning | |
| WO2017059904A1 (en) | Anomaly detection in a data packet access network | |
| US11140067B2 (en) | Discovering cross-domain links based on traffic flow | |
| US8566634B2 (en) | Method and system for masking defects within a network | |
| US11265237B2 (en) | System and method for detecting dropped aggregated traffic metadata packets | |
| CN113438116A (en) | Power communication data management system and method | |
| KR100812946B1 (en) | System and Method for Managing Quality of Service in Mobile Communication Network | |
| CN117234806B (en) | Automatic restarting method and system for network card | |
| WO2024066331A1 (en) | Network abnormality detection method and apparatus, electronic device, and storage medium | |
| JP2009296336A (en) | Optical receiver, remote monitoring device and remote monitoring program |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |