CN113746688B - Method and device for updating anomaly detection model and computing equipment - Google Patents

Method and device for updating anomaly detection model and computing equipment Download PDF

Info

Publication number
CN113746688B
CN113746688B CN202010477620.XA CN202010477620A CN113746688B CN 113746688 B CN113746688 B CN 113746688B CN 202010477620 A CN202010477620 A CN 202010477620A CN 113746688 B CN113746688 B CN 113746688B
Authority
CN
China
Prior art keywords
kpi
abnormal
detection model
suspected
data
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
CN202010477620.XA
Other languages
Chinese (zh)
Other versions
CN113746688A (en
Inventor
龚旭
王仲宇
张彦芳
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Huawei Technologies Co Ltd
Original Assignee
Huawei Technologies Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Huawei Technologies Co Ltd filed Critical Huawei Technologies Co Ltd
Priority to CN202010477620.XA priority Critical patent/CN113746688B/en
Publication of CN113746688A publication Critical patent/CN113746688A/en
Application granted granted Critical
Publication of CN113746688B publication Critical patent/CN113746688B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L43/00—Arrangements for monitoring or testing data switching networks
    • H04L43/08—Monitoring or testing based on specific metrics, e.g. QoS, energy consumption or environmental parameters
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L41/00—Arrangements for maintenance, administration or management of data switching networks, e.g. of packet switching networks
    • H04L41/06—Management of faults, events, alarms or notifications
    • H04L41/0631—Management of faults, events, alarms or notifications using root cause analysis; using analysis of correlation between notifications, alarms or events based on decision criteria, e.g. hierarchy, tree or time analysis
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L43/00—Arrangements for monitoring or testing data switching networks
    • H04L43/08—Monitoring or testing based on specific metrics, e.g. QoS, energy consumption or environmental parameters
    • H04L43/0823—Errors, e.g. transmission errors
    • H04L43/0829—Packet loss
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L43/00—Arrangements for monitoring or testing data switching networks
    • H04L43/08—Monitoring or testing based on specific metrics, e.g. QoS, energy consumption or environmental parameters
    • H04L43/0852—Delays
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L43/00—Arrangements for monitoring or testing data switching networks
    • H04L43/50—Testing arrangements

Abstract

The application provides a method, a device and computing equipment for updating an anomaly detection model, and belongs to the technical field of network communication. The method comprises the following steps: and acquiring suspected abnormal KPI data determined by the detection equipment by using a suspected abnormal KPI detection model, updating the current abnormal KPI sample library according to the suspected abnormal KPI data, and updating the current abnormal KPI detection model according to the updated abnormal KPI sample library. By adopting the method and the device, the abnormal KPI detection model can continuously keep higher abnormal detection performance.

Description

Method and device for updating anomaly detection model and computing equipment
Technical Field
The present application relates to the field of network communications, and in particular, to a method, an apparatus, and a computing device for implementing an update of an anomaly detection model.
Background
The network fault triggering of operators and enterprises is mostly realized by means of active report exception of business side customers, and belongs to passive response, so that the network exception discovery time is late, and further the customer satisfaction is influenced. In order to improve the capability of actively sensing and discovering network faults, network abnormalities are required to be intelligently detected through Key Performance Indicators (KPIs) of network devices. In the anomaly detection method, the ideal performance required by operators and enterprise customers can keep high accuracy and high recall rate for a long time, and under the condition that the ideal performance is not achieved, the high accuracy rate is preferentially ensured without missing report or misinformation.
In the related KPI anomaly detection technology based on supervised learning, normal or abnormal manual labeling is performed on KPI data acquired in a network usually based on expert experience in the network operation and maintenance field. And then training a supervised learning model based on the labeled abnormal KPI sample library, and performing online abnormal KPI detection by using the model.
Complex KPIs such as Network Processor (NP) and Traffic Management (TM) packet loss tend to have diverse waveforms, and since the above models can only learn existing KPI abnormal patterns in the abnormal KPI sample library, the flexibility is poor, and the new KPI abnormal patterns cannot be adapted, thereby resulting in the decrease of abnormal detection performance.
Disclosure of Invention
The embodiment of the application provides a method, a device and a computing device for updating an abnormal detection model, and the abnormal KPI detection model can be continuously kept at high abnormal detection performance by adopting the method and the device.
In a first aspect, a method for implementing an update of an anomaly detection model is provided, where the method includes: acquiring suspected abnormal KPI data determined by the detection equipment by using a current suspected abnormal KPI detection model; updating the current abnormal KPI sample base according to the suspected abnormal KPI data; and updating the current abnormal KPI detection model according to the updated abnormal KPI sample library.
According to the scheme, the method is executed by the analysis equipment, and the analysis equipment acquires suspected abnormal KPI data determined by a current suspected abnormal KPI detection model. And the analysis equipment determines abnormal KPI data in the suspected abnormal KPI data, updates the current abnormal KPI sample library based on the abnormal KPI data and obtains the updated abnormal KPI sample library. And the analysis equipment updates the current abnormal KPI detection model by using the updated abnormal KPI sample library. Therefore, through continuously acquiring suspected abnormal KPI data, the abnormal KPI sample base is updated, and then the abnormal KPI detection model can be continuously updated, so that the abnormal KPI detection model continuously keeps higher detection performance.
In a possible implementation manner, the recall rate of the suspected abnormal KPI detection model is higher than that of the abnormal KPI detection model, or the false negative rate of the suspected abnormal KPI detection model is lower than that of the abnormal KPI detection model. Therefore, the abnormal KPI sample base can be rapidly expanded, and abnormal KPI data detected by the abnormal KPI detection model can be more accurate.
In a possible implementation manner, the accuracy of the abnormal KPI detection model is higher than that of the suspected abnormal KPI detection model, or the false alarm rate of the abnormal KPI detection model is lower than that of the suspected abnormal KPI detection model. Therefore, the abnormal KPI sample base can be rapidly expanded, and abnormal KPI data detected by the abnormal KPI detection model can be more accurate.
In a possible implementation manner, updating a current abnormal KPI sample according to suspected abnormal KPI data includes: acquiring KPI data confirmed to be abnormal in suspected abnormal KPI data; and adding the KPI data confirmed to be abnormal as an abnormal KPI sample to the current abnormal KPI sample library. In this way, KPI data that is confirmed to be abnormal is obtained by the analysis device, and can be added to the abnormal KPI sample library to augment the abnormal KPI sample library.
In one possible implementation, acquiring KPI data confirmed to be abnormal in suspected abnormal KPI data includes: sending suspected abnormal KPI data to a management device, and receiving KPI data confirmed to be abnormal in the suspected abnormal KPI data fed back by the management device; or displaying suspected abnormal KPI data, and acquiring KPI data confirmed to be abnormal in the displayed suspected abnormal KPI data.
According to the scheme of the embodiment of the application, suspected abnormal KPI data are provided for the management equipment by the analysis equipment, abnormal KPI data are determined in the suspected abnormal KPI data through the management equipment by a user and are provided for the analysis equipment through the management equipment, and the KPI data confirmed to be abnormal are obtained by the analysis equipment. Or the analysis equipment displays abnormal KPI data, a user confirms the abnormal KPI data in the displayed suspected abnormal KPI data and submits the abnormal KPI data, and the analysis equipment acquires the KPI data confirmed to be abnormal. Therefore, the analysis equipment can acquire accurate abnormal KPI data.
In a possible implementation manner, before updating the current abnormal KPI detection model according to the updated abnormal KPI sample library, the method further includes: and determining that the current abnormal KPI detection model does not meet the detection performance requirement according to the updated abnormal KPI sample library. The abnormal KPI detection model not meeting the detection performance requirement means that the abnormal KPI data detected by the current abnormal KPI detection model is inaccurate and cannot adapt to newly generated abnormal KPI data. Therefore, the updating is carried out only when the current abnormal KPI detection model does not meet the detection performance requirement, and the processing resource of the analysis equipment can be saved.
In a possible implementation manner, before determining that the current abnormal KPI detection model does not meet the detection performance requirement according to the updated abnormal KPI sample library, the method further includes: acquiring abnormal KPI data determined by detection equipment by using an abnormal KPI detection model; determining that the current abnormal KPI detection model does not meet the detection performance requirement according to the updated abnormal KPI sample library, including: and determining that the abnormal KPI detection model does not meet the detection performance requirement according to the abnormal KPI data and the updated abnormal KPI sample library.
According to the scheme, the analysis equipment acquires abnormal KPI data determined by using a current abnormal KPI detection model from the detection equipment, and then judges whether the current abnormal KPI detection model does not meet the detection performance requirement by using the updated abnormal KPI sample library and the abnormal KPI data determined by using the current abnormal KPI detection model. Therefore, whether the current abnormal KPI detection model does not meet the detection performance requirement or not can be accurately judged through the abnormal KPI data detected by the current abnormal KPI detection model.
In a possible implementation manner, determining that the abnormal KPI detection model does not meet the detection performance requirement according to the abnormal KPI data and the updated abnormal KPI sample library includes: determining performance indexes of the abnormal KPI detection model according to the abnormal KPI data and the updated abnormal KPI sample library, wherein the performance indexes comprise at least one of a missing report rate, a false report rate, a recall rate and an accuracy rate; and determining that the abnormal KPI detection model does not meet the detection performance requirement according to the performance index. In this way, since the performance index of the current abnormal KPI detection model is used to judge whether the detection performance requirement is not met, the current abnormal KPI detection model can be accurately judged to not meet the detection performance requirement.
In one possible implementation, the suspected abnormal KPI detection model is an unsupervised learning model.
In one possible implementation manner, before obtaining suspected abnormal KPI data determined by the detection device using the current suspected abnormal KPI detection model, the method further includes: according to a preset performance requirement, training to obtain a suspected abnormal KPI detection model, wherein the suspected abnormal KPI detection model is an initial suspected abnormal KPI detection model, and the initial suspected abnormal KPI detection model is a suspected abnormal KPI detection model which is issued by analysis equipment for the first time and is used for detecting suspected abnormal KPI data; and providing the suspected abnormal KPI detection model for the detection equipment. In this way, a suspected abnormal KPI detection model may be provided to the detection device for detecting suspected abnormal KPI data.
In a possible implementation manner, before training to obtain a suspected abnormal KPI detection model according to a first preset performance requirement, the method further includes: obtaining target suspected abnormal KPI data in the initial KPI data; according to a first preset performance requirement, training to obtain a suspected abnormal KPI detection model, comprising: acquiring KPI data confirmed to be abnormal in the target suspected abnormal KPI data; and training to obtain a suspected abnormal KPI detection model according to the first preset performance requirement and KPI data confirmed to be abnormal in the target suspected abnormal KPI data. The initial KPI data is KPI data provided by a KPI data acquisition device to an analysis device, and is used for determining an initial suspected abnormal KPI detection model, and the initial KPI suspected abnormal KPI detection model is a suspected abnormal KPI detection model issued by the analysis device to the detection device for the first time. In this way, an initial suspected abnormal KPI detection model may be obtained.
In one possible implementation, obtaining target suspected abnormal KPI data in the initial KPI data includes: determining target suspected abnormal KPI data in the initial KPI data according to a preset unsupervised abnormal KPI detection model; training to obtain an initial suspected abnormal KPI detection model according to a first preset performance requirement and KPI data confirmed to be abnormal in target suspected abnormal KPI data, wherein the method comprises the following steps: and updating the unsupervised abnormal KPI detection model according to the first preset performance requirement and the KPI data confirmed to be abnormal in the target suspected abnormal KPI data to obtain an initial suspected abnormal KPI detection model. In this way, an initial suspected abnormal KPI detection model may be obtained.
In a possible implementation manner, after updating the current abnormal KPI sample library according to the suspected abnormal KPI data, the method further includes: updating a suspected abnormal KPI detection model according to the updated abnormal KPI sample library; and providing the updated suspected abnormal KPI detection model for the detection equipment. Therefore, after the abnormal KPI sample library is updated every time, the suspected abnormal KPI detection model is updated in time, and the detection performance of the suspected abnormal KPI detection model can be improved.
In one possible implementation, the abnormal KPI detection model is a supervised learning model.
In a possible implementation manner, before updating the current abnormal KPI detection model according to the updated abnormal KPI sample library, the method further includes: training to obtain a current abnormal KPI detection model according to a second preset performance requirement, wherein the current abnormal KPI detection model is an initial abnormal KPI detection model; and providing the current abnormal KPI detection model to the detection equipment. In this way, an initial abnormal KPI detection model may be provided to the detection device for detecting abnormal KPI data.
In one possible implementation, the method further includes: and providing the updated abnormal KPI detection model for the detection equipment. In this way, the detection device can acquire the latest abnormal KPI detection model.
In one possible implementation, the suspected abnormal KPI data includes a suspected abnormal KPI time series, and the abnormal KPI data includes an abnormal KPI time series.
In one possible implementation, the suspected abnormal KPI data further includes an abnormal time period and/or an abnormal time point in the suspected abnormal KPI time series. KPIs in the abnormal time period are all suspected abnormal KPIs. The KPI at the abnormal time point is suspected to be abnormal KPI. The abnormal KPI data also includes abnormal time periods and/or abnormal time points in the abnormal KPI time series.
In a second aspect, a method for implementing an update of an anomaly detection model is provided, the method including: and acquiring a suspected abnormal KPI detection model, determining suspected abnormal KPI data in KPI data through the suspected abnormal KPI detection model, and reporting the suspected abnormal KPI data to analysis equipment, wherein the suspected abnormal KPI data is used for updating the abnormal KPI detection model.
According to the scheme, the method is executed by the detection equipment, the detection equipment obtains a suspected abnormal KPI detection model, KPI data are input into the suspected abnormal KPI detection model, and suspected abnormal KPI data are obtained. And the detection equipment sends the suspected abnormal KPI data to the analysis equipment, and the analysis equipment updates the abnormal KPI detection model. Therefore, the suspected abnormal KPI data are continuously provided for the analysis equipment, the abnormal KPI sample base is updated by the analysis equipment, and the abnormal KPI detection model can be continuously updated, so that the abnormal KPI detection model continuously keeps higher detection performance.
In one possible implementation, the method further includes: acquiring an abnormal KPI detection model; determining abnormal KPI data through the abnormal KPI detection model; and reporting the abnormal KPI data to the analysis equipment, wherein the abnormal KPI data is used for judging whether the abnormal KPI detection model meets the detection performance requirements.
According to the scheme, the detection equipment acquires the abnormal KPI detection model, and KPI data are input into the abnormal KPI detection model to acquire the abnormal KPI data. And the detection equipment sends the abnormal KPI data to the analysis equipment, and the analysis equipment is used for judging whether the current abnormal KPI detection model still meets the detection performance requirement. Therefore, data are provided for judging whether the current abnormal KPI detection model still meets the detection performance requirements.
In one possible implementation, determining suspected abnormal KPI data in KPI data by a suspected abnormal KPI detection model includes: and (3) carrying out one or more of the following treatments on each KPI time sequence in the KPI data through a suspected abnormal KPI detection model: filtering KPI time sequences with KPIs at all moments lower than a first value, filtering KPI time sequences with KPI sudden increase amount at any moment lower than a second value, filtering KPI time sequences with periodicity, or filtering KPI time sequences in a steady state; and performing abnormal sudden increase detection on the KPI time sequences remained in the KPI data through a suspected abnormal KPI detection model, and determining suspected abnormal KPI data in the KPI data. Therefore, the detection equipment carries out filtering operation on the KPI time sequence and filters the KPI time sequence without abnormal KPI data, so that suspected abnormal KPI data used for updating an abnormal KPI sample base is determined in the filtered KPI data, high recall rate can be ensured, and missed reports can be reduced.
In a third aspect, the present application provides an apparatus for implementing an update of an anomaly detection model, where the apparatus includes a plurality of modules, and the modules implement the method for implementing an update of an anomaly detection model provided in the first aspect by executing instructions.
In a fourth aspect, the present application provides an apparatus for implementing an update of an anomaly detection model, where the apparatus includes a plurality of modules, and the modules implement the method for implementing an update of an anomaly detection model provided in the second aspect by executing instructions.
In a fifth aspect, the present application provides a computing device comprising a memory and a processor, wherein the processor executes computer instructions stored in the memory, so that the computing device executes the method for implementing the anomaly detection model update according to the first aspect.
In a sixth aspect, the present application provides a computing device comprising a memory and a processor, wherein the processor executes computer instructions stored in the memory, so that the computing device executes the method for implementing the update of the anomaly detection model according to the second aspect.
In a seventh aspect, a computer-readable storage medium is provided, where computer instructions are stored, and when executed by a computing device, the computer instructions cause the computing device to execute the method for implementing the update of the anomaly detection model according to the first aspect, or cause the computing device to implement the functions of the apparatus according to the third aspect.
In an eighth aspect, a computer-readable storage medium is provided, where the computer-readable storage medium stores computer instructions, and when the computer instructions in the computer-readable storage medium are executed by a computing device, the computing device is caused to execute the method for implementing the update of the anomaly detection model according to the second aspect, or the computing device is caused to implement the functions of the apparatus according to the fourth aspect.
In a ninth aspect, the present application provides a computer program product comprising computer instructions which, when executed by a computing device, perform the method for implementing an anomaly detection model update of the first aspect.
In a tenth aspect, the present application provides a computer program product comprising computer instructions which, when executed by a computing device, perform the method for implementing an anomaly detection model update according to the first aspect.
In an eleventh aspect, the present application provides a system for implementing an update of an anomaly detection model, where the system includes an analysis device and a detection device, where the analysis device is the apparatus according to the third aspect, and the detection device is the apparatus according to the fourth aspect.
Drawings
FIG. 1 is a schematic block diagram of a computing device provided in an exemplary embodiment of the present application;
FIG. 2 is a schematic diagram of an application scenario for implementing an update of an anomaly detection model according to an exemplary embodiment of the present application;
FIG. 3 is a schematic diagram of an application scenario for implementing an update of an anomaly detection model according to an exemplary embodiment of the present application;
FIG. 4 is a schematic diagram of an application scenario for implementing an update of an anomaly detection model according to an exemplary embodiment of the present application;
FIG. 5 is a flow diagram providing suspected abnormal KPI data provided by an exemplary embodiment of the present application;
FIG. 6 is a flow chart for determining suspected abnormal KPI data provided by an exemplary embodiment of the present application;
FIG. 7 is a flowchart illustrating a method for implementing an update to an anomaly detection model according to an exemplary embodiment of the present application;
FIG. 8 is an interaction diagram of a method for deploying a suspected abnormal KPI detection model according to an exemplary embodiment of the present application;
FIG. 9 is an interaction diagram of a method for deploying an abnormal KPI detection model, provided by an exemplary embodiment of the present application;
FIG. 10 is an interactive schematic diagram of a method for updating suspected abnormal KPI detection models, provided by an exemplary embodiment of the present application;
FIG. 11 is an interaction diagram of a method for updating an abnormal KPI detection model according to an exemplary embodiment of the present application;
FIG. 12 is a schematic structural diagram of an apparatus for implementing an update of an anomaly detection model according to an exemplary embodiment of the present application;
FIG. 13 is a schematic structural diagram of an apparatus for implementing an update of an anomaly detection model according to an exemplary embodiment of the present application;
fig. 14 is a schematic structural diagram of an apparatus for implementing an update of an anomaly detection model according to an exemplary embodiment of the present application.
Detailed Description
To make the objects, technical solutions and advantages of the present application more clear, the following detailed description of the embodiments of the present application will be made with reference to the accompanying drawings.
The application provides a method for updating an anomaly detection model, which can be executed by an analysis device and/or a detection device.
The analysis apparatus may be a hardware device (may be referred to as an analysis device), such as a server, a terminal computing device, or the like, or may be a software device (such as a set of software programs running on the hardware device), and the hardware device run by the analysis apparatus may be referred to as an analysis device. For example, the analysis apparatus may operate in a cloud computing device system (which may include at least one cloud computing device, such as a server, etc.), may also operate in an edge computing device system (which may include at least one edge computing device, such as a server, a desktop computer, etc.), and may also operate in various terminal computing devices (such as a notebook computer, a personal desktop computer, etc.). The analysis device may logically be a device composed of various parts, for example, the analysis device may include an acquisition module and an update module. The various components of the analysis apparatus may be deployed in different systems or servers, respectively. Each part of the analysis device can be respectively operated in any two of the cloud computing equipment system, the edge computing equipment system and the terminal computing equipment. The cloud computing device system, the edge computing device system and the terminal computing device are connected through communication paths, and can communicate with each other and transmit data.
The detecting device may be a hardware device (which may be referred to as a detecting device), such as a server, a network device, etc., or may be a software device (which may be a set of software programs running on the hardware device), and the hardware device run by the detecting device may be referred to as a detecting device. For example, the detection apparatus may operate in a cloud computing device system (which may include at least one cloud computing device, such as a server, etc.), may also operate in an edge computing device system (which may include at least one edge computing device, such as a server, a desktop computer, etc.), and may also operate in various terminal computing devices (such as a notebook computer, a personal desktop computer, etc.). The detection device may be logically formed by various parts, such as an acquisition module, a determination module, a transmission module, and the like. The various components of the detection apparatus may be deployed in different systems or servers, respectively. Each part of the detection device can be respectively operated in any two of the cloud computing equipment system, the edge computing equipment system and the terminal computing equipment. The cloud computing device system, the edge computing device system and the terminal computing device are connected through communication paths, and can communicate with each other and transmit data.
The embodiment of the application further provides a computing device for updating the anomaly detection model. Fig. 1 illustratively provides one possible architecture diagram for a computing device 100. The computing device includes a memory 101, a processor 102, a transceiver 103, and a bus 104. The memory 101, the processor 102 and the transceiver 103 are connected to each other through a bus 104.
Memory 101 may be ROM, static storage, dynamic storage, or RAM. The memory 101 may store computer instructions that, when executed by the processor 102 stored in the memory 101, the processor 102 and the transceiver 103 are used to perform a method of implementing an anomaly detection model update. The memory may also store data, e.g., a portion of memory 101 used to store data needed to implement an exception detection model update, as well as intermediate or result data during program execution.
The processor 102 may be a general purpose CPU, an application ASIC, a Graphics Processing Unit (GPU), or any combination thereof. The processor 102 may include one or more chips.
The transceiver 103 enables communication between the computing device and other devices or communication networks using transceiver modules such as, but not limited to, transceivers.
Bus 104 may include a path that transfers information between various components of the computing device (e.g., memory 101, processor 102, transceiver 103).
The embodiment of the application can be applied to various scenes, and three possible scenes are given as follows:
scenario one, as shown in fig. 2, the analysis device is used to manage a network, where the analysis device may be a network manager, a network analyzer, a network controller, and the like, and the detection device is a network device in the network. The analysis device may establish a wired or wireless communication connection with a network device in the network, which may be a router, switch, etc. The analysis equipment builds a suspected abnormal KPI detection model and an abnormal KPI detection model, and updates the suspected abnormal KPI detection model and the abnormal KPI detection model in a long-time operation process. The analysis device may issue the suspected abnormal KPI detection model and the abnormal KPI detection model to all network devices or a part of network devices in the network. Network devices in a network may collect KPI data, determine suspected abnormal KPI data in the KPI data using a suspected abnormal KPI detection model, and determine abnormal KPI data in the KPI data using an abnormal KPI detection model. The network equipment in the network provides suspected abnormal KPI data, abnormal KPI data and the like for the analysis equipment. The analysis device may further provide the suspected abnormal KPI data to the management device, so that the user may confirm the abnormal KPI data included in the suspected abnormal KPI data, and may further send an abnormal KPI alarm to the user terminal. The analysis device may also receive user adjustment information for the accuracy (also known as precision) and recall (also known as recall) of the expected abnormal KPI detection model. In particular, the interaction of the analysis device and the detection device is described hereinafter.
Scenario two, as shown in fig. 3, the analyzing device is used to manage a plurality of networks, the analyzing device is connected to a management apparatus of each network, and the management apparatus may be a network manager, a network analyzer, a network controller, or the like. The detection device is a network device in the network. Fig. 3 shows that the management apparatus 1 manages the network 1, and the management apparatus n manages the network n and the like. In this case, the analysis device may be a cloud analysis platform connecting all the management apparatuses, and the analysis device interacts with the management apparatuses of each network. The analysis equipment issues the suspected abnormal KPI detection model and the abnormal KPI detection model of each network to the management device of each network, so that the management device of the network issues the suspected abnormal KPI detection model and the abnormal KPI detection model to the managed network. Network equipment in a network acquires KPI data, a suspected abnormal KPI detection model is used for determining the suspected abnormal KPI data in the KPI data, and an abnormal KPI detection model is used for determining the abnormal KPI data in the KPI data. And the network equipment sends suspected abnormal KPI data and abnormal KPI data to a management device of the network. The management device reports suspected abnormal KPI data and abnormal KPI data to the analysis equipment. And the analysis equipment updates the suspected abnormal KPI detection model and the abnormal KPI detection model based on the received suspected abnormal KPI data and the received abnormal KPI data. The analysis device may further provide the suspected abnormal KPI data to the management device, so that the user may confirm the abnormal KPI data included in the suspected abnormal KPI data, and may further send an abnormal KPI alarm to the user terminal. The analysis device may also receive user adjustment information for the accuracy (also known as precision) and recall (also known as recall) of the expected abnormal KPI detection model. In particular, the interaction of the analysis device and the detection device is described hereinafter.
In a third scenario, as shown in fig. 4, the analyzing device is used to manage a plurality of networks, the analyzing device is connected to a management apparatus of each network, the detecting device is a management apparatus, and the management apparatus may be a network manager, a network analyzer, a network controller, or the like. Fig. 4 shows that the management apparatus 1 manages the network 1, and the management apparatus n manages the network n and the like. In this case, the analysis device may be a cloud analysis platform connecting all the detection devices, and the analysis device interacts with the management apparatus of each network. The analysis equipment builds a suspected abnormal KPI detection model and an abnormal KPI detection model, and updates the suspected abnormal KPI detection model and the abnormal KPI detection model in a long-time operation process. And the analysis equipment issues the suspected abnormal KPI detection model and the abnormal KPI detection model of each network to the management device of each network. The management apparatus of each network acquires KPI data from network devices in the managed network. The management device of each network determines suspected abnormal KPI data in the KPI data by using a suspected abnormal KPI detection model and returns the suspected abnormal KPI data to the analysis equipment, and determines abnormal KPI data in the KPI data by using an abnormal KPI detection model and returns the abnormal KPI data to the analysis equipment. The analysis device may further provide the suspected abnormal KPI data to the management device, so that the user may confirm the abnormal KPI data included in the suspected abnormal KPI data, and may further send an abnormal KPI alarm to the user terminal. The analysis device may also receive user adjustment information for the accuracy (also known as precision) and recall (also known as recall) of the expected abnormal KPI detection model. In particular, the interaction of the analysis device and the detection device is described hereinafter.
It should be noted that, for the scenarios one to three, the management device may be an Operation Support System (OSS) or other devices connected to the analysis device. The network may be a local point network, a core network, or an edge network, and a user of each local point network may be an operator or an enterprise customer. The different office networks may be different networks divided according to corresponding dimensions, such as networks of different regions, networks of different operators, different service networks, different network domains, and the like.
Network devices in a network may provide KPI data, and specifically, the KPI data provided by the network devices may be used to characterize the performance of the network devices, and may also be used to characterize the performance of a certain module in the network devices. In the case where KPI data is used to characterize the performance of a module in a network, KPI data may be collected by that module, or by other modules. For example, in the case that KPI data is used to characterize the performance of a module in a Network device, the module may be a Network Processor (NP) or Traffic Management (TM) module, etc.
KPI data includes KPI time series, which describe the correspondence of time to KPI. The KPI may be a packet loss type KPI (for example, the KPI is an accumulated packet loss amount or an instantaneous packet loss amount), a time delay, and the like. For example, the KPI is a packet loss class KPI, the KPI time sequence is [ (t 1, k 1), (t 2, k 2), (t 3, k 3), …, (ti, ki), …, (tn, kn) ], (ki, ki) is the instantaneous packet loss number detected at the ti time point, or ki (ti, ki) is the sum of the packet loss numbers counted from the t1 time point to the ti time point (i.e., the cumulative packet loss number). It should be noted here that, for the above-described "instantaneous packet loss number", when the network device collects KPI data, the packet loss number is collected periodically, the instantaneous packet loss number corresponding to each period refers to the packet loss number of the period (i.e. the total of the packet loss numbers in the period), and accordingly, the time point corresponding to the instantaneous packet loss number is a specific time point of the period, such as the end time of the period.
Before describing the flow of the method for updating the abnormal detection model, a suspected abnormal KPI detection model and an abnormal KPI detection model are introduced, wherein the suspected abnormal KPI detection model is used for determining suspected abnormal KPI data in KPI data, and the abnormal KPI detection model is used for determining abnormal KPI data in the KPI data. The suspected abnormal KPI detection model is an unsupervised learning model (namely, a sample of the suspected abnormal KPI detection model is obtained as unlabeled KPI data). The abnormal KPI detection model is a supervised learning model (i.e. the samples from which the abnormal KPI detection model is obtained are labeled KPI data).
In the embodiment of the application, in order to recall all suspected abnormal KPI data by using the suspected abnormal KPI detection model as much as possible, the recall rate of the suspected abnormal KPI detection model is relatively high, or the false negative rate of the suspected abnormal KPI detection model is relatively low. In addition, in order to enable abnormal KPI data detected by the abnormal KPI detection model to be accurate, the recall rate of the abnormal KPI detection model is low, or the report missing rate of the abnormal KPI detection model is high. Therefore, the recall rate of the suspected abnormal KPI detection model is higher than that of the abnormal KPI detection model, or the false negative rate of the suspected abnormal KPI detection model is lower than that of the abnormal KPI detection model.
In the embodiment of the application, in order to recall all suspected abnormal KPI data by using a suspected abnormal KPI detection model as much as possible, the accuracy rate of the suspected abnormal KPI detection model is relatively low, or the false alarm rate of the suspected abnormal KPI detection model is relatively high. In addition, in order to enable abnormal KPI data detected by the abnormal KPI detection model to be accurate, the accuracy rate of the abnormal KPI detection model is high, or the false alarm rate of the abnormal KPI detection model is low. Therefore, the accuracy of the abnormal KPI detection model is higher than that of the suspected abnormal KPI detection model, or the false alarm rate of the abnormal KPI detection model is lower than that of the suspected abnormal KPI detection model.
In addition, in the embodiment of the present application, the description of "a and/or B" appears, and it can be understood that A, B, and a and B are three cases.
The following describes a flow of a method for implementing the update of the anomaly detection model with reference to the application scenario shown in fig. 2:
as shown in fig. 5, a flow is provided in which a detection device provides suspected abnormal KPI data to an analysis device:
step 501, the detection device obtains a current suspected abnormal KPI detection model.
Wherein, corresponding to the application scenario of fig. 2, the detection device is a network device in the network. The current suspected abnormal KPI detection model refers to a suspected abnormal KPI detection model currently used for detecting suspected abnormal KPI data.
In this embodiment, the analysis device updates the suspected abnormal KPI detection model, and the analysis device sends update information of the suspected abnormal KPI detection model to the detection device. And the detection equipment deploys the latest suspected abnormal KPI detection model based on the updated information to serve as the current suspected abnormal KPI detection model.
Step 502, the detection device determines suspected abnormal KPI data in KPI data through a suspected abnormal KPI detection model.
The suspected abnormal KPI data refers to KPI data with abnormal possibility in KPI data.
In this embodiment, the detection device obtains KPI data during operation, where the KPI data includes one or more KPI time series. And then the detection equipment inputs KPI data to be detected (the KPI data to be detected is the KPI data of which the existence of suspected abnormal KPI data is not detected at the current time point) into a suspected abnormal KPI detection model to obtain output, wherein the output is the KPI time sequence (which can be called as suspected abnormal KPI time sequence) marked as suspected abnormality in the KPI data and the abnormal time point and/or abnormal time segment in the KPI time sequence marked as suspected abnormality. The abnormal time point is the time point of suspected abnormal KPI in the suspected abnormal KPI time sequence, and the abnormal time period is the time period formed by the continuous time points of suspected abnormal KPI in the suspected abnormal KPI time sequence. Of course the output may also include a KPI time series that is nominally normal.
It should be noted here that the KPI time series in the KPI data that are marked as suspected abnormal are one or more suspected abnormal KPI time series, that is, the suspected abnormal KPI data include one or more suspected abnormal KPI time series.
Step 503, the detection device reports suspected abnormal KPI data to the analysis device.
In one possible implementation, a process for determining suspected abnormal KPI data using a suspected abnormal KPI detection model is also provided, as shown in fig. 6, the process flow of step 502 is:
step 5021, the detection equipment obtains a KPI time sequence in KPI data, wherein the KPI time sequence is a corresponding relation between each time point and KPI.
Step 5022, the detection equipment judges whether each KPI in the KPI time sequence meets normal distribution.
Step 5023, under the condition that the KPI time sequence meets normal distribution, the detection equipment carries out abnormal sudden increase detection on the KPI time sequence, and suspected abnormal KPI data in the KPI time sequence are determined.
In this embodiment, when the KPI time sequence satisfies normal distribution, the detecting device performs sudden abnormal increase detection on the KPI time sequence to obtain suspected abnormal KPI data in the KPI time sequence, and specifically, assuming that one KPI time sequence includes suspected abnormal KPI, after performing the sudden abnormal increase detection, a result obtained by calibrating the KPI time sequence as suspected abnormal, and an abnormal time point and/or an abnormal time period included in the KPI time sequence are obtained. The algorithm used for detecting abnormal sudden increase can be a double window sliding abnormal detection algorithm, a smooth z-score abnormal detection algorithm or other unsupervised abnormal detection algorithms common in the industry.
The principle of the double-sliding-window anomaly detection algorithm is as follows: respectively establishing an equal-length time window forwards and backwards along a time axis at the time t, taking the ratio of the KPI sum contained in the window close to the back on the time axis to the KPI sum contained in the window close to the front as the KPI sudden increase degree of the time t, determining whether the KPI corresponding to the time t is abnormal according to the sudden increase degree, sliding the time t along the time axis, continuously calculating the sudden increase degree of the corresponding time and judging whether the KPI data is suspected to be abnormal. The principle of the smooth z-score anomaly detection method is as follows: and continuously updating the mean value and the variance of the KPI on the time axis of the KPI time sequence based on a plurality of the latest KPIs, determining whether the KPI corresponding to the time point to be detected is suspected abnormal or not according to the distance between the time point to be detected and the mean value, sliding on the time axis in the same way, and continuously judging whether the KPI corresponding to the time point is abnormal or not.
Step 5024, under the condition that the KPI time sequence does not meet normal distribution, the detection device judges whether to filter the KPI time sequence, if the KPI time sequence is not filtered, the detection device performs abnormal sudden increase detection on the KPI time sequence (see the description in step 5023), and determines suspected abnormal KPI data in the KPI time sequence, specifically, if a KPI time sequence includes suspected abnormal KPI, after the abnormal sudden increase detection is performed, a result that the KPI time sequence is calibrated to be suspected abnormal, and abnormal time points and/or abnormal time periods included in the KPI time sequence are obtained. The detecting device then obtains the next undetected KPI time series from the KPI data (i.e., returns to step 5021).
Here, the flow of fig. 6 is an internal execution flow of the suspected abnormal KPI detection model.
For example, for the process in fig. 6, assuming that the KPI data includes 100 KPI time series, 40 KPI time series are left through filtering, and in these 40 KPI time series, it is determined that 25 KPI time series include suspected abnormal KPI, and in addition, 15 KPI time series do not include suspected abnormal KPI, the process in fig. 6 is executed to obtain 40 KPI time series, where the 25 KPI time series are calibrated as suspected abnormal KPI time series, and correspond to abnormal time points and/or abnormal time periods, and the 15 KPI time series are calibrated as normal.
When the detection equipment acquires the KPI data, the KPI data may not meet the input requirements of the suspected abnormal KPI detection model, so that the KPI data needs to be preprocessed. The pre-processing may include: 1. taking KPI as a packet loss class KPI as an example, KPI data includes a correspondence between a time point and an accumulated packet loss quantity, and the detection device can convert the correspondence into a correspondence between a time point and an instantaneous packet loss quantity, and the processing is as follows: for a target time point of a certain KPI time sequence (the target time point is any time point in the KPI time sequence), the detection equipment determines the difference value of the first KPI minus the second KPI, and determines the difference value as the instantaneous KPI of the target time point. The first KPI is the accumulated packet loss number corresponding to the target time point, and the second KPI is the accumulated packet loss number corresponding to the last time point of the target time point.
2. Because the detection device may have non-uniform time intervals when recording the KPI, KPI data filling processing can be performed on KPI time series with non-uniform sampling intervals to obtain KPI time series with uniform time intervals. The specific filling process is as follows: for each KPI time sequence, the detection equipment uses the KPIs at the existing time points in the KPI time sequence to determine the KPIs at equal time intervals. For example, time points t1, t2, t3 and t4 exist in a KPI time sequence, time intervals of the 4 time points are different, and assuming that the specified time interval is t1 to t2 and the time interval from t2 to t3 is greater than the specified time interval, time point t5 may be inserted between t2 and t3, where t5 is equal to the sum of t2 and the specified time interval. the KPI corresponding to t5 may be determined using the KPI corresponding to t2 and the KPI corresponding to t3, specifically: if t5 corresponds to KPI that is the accumulated KPI, the accumulated KPI is ((k 3-k 2) × (t 5-t 2)/(t 3-t 2)) + k2. If t5 corresponds to a KPI that is an instantaneous KPI, the instantaneous KPI is (k 2 × (t 5-t 2)/(t 3-t 2)) + (k 3 × (t 3-t 5)/(t 3-t 2)). k2 is the KPI corresponding to t2, and k3 is the KPI corresponding to t 3. This is merely an example and other means may of course be used.
Here, the algorithm for preprocessing the KPI time series may be integrated into the suspected abnormal KPI detection model, or may be disposed outside the suspected abnormal KPI detection model.
In one possible implementation, the KPI time series may be filtered as follows:
for any KPI time series, the KPI time series are filtered based on a threshold value (i.e., a first numerical value) of the KPI. The method specifically comprises the following steps: if the KPI at any time in the KPI time series is lower than the first value (namely the KPI at all times is lower than the first value), filtering the KPI time series. If the KPI at least one moment in the KPI time sequence is greater than or equal to the first value, the KPI time sequence is reserved to enter a subsequent filtering process. This first value may be preset (which may be obtained by analyzing a large amount of KPI data) and stored at the detection device. For the time sequence of the lost packet KPIs, the logical reason of the above operation is that a large amount of burst loss phenomena shown in a certain time period can represent abnormal packet loss behaviors. For example, if it is determined that the packet loss number of at least one time point in a certain time period in a certain KPI time sequence is greater than a first value (the packet loss number is not the accumulated packet loss number, but is an instantaneous packet loss number), it indicates that the KPI time sequence cannot exclude the possibility of the existence of suspected abnormal KPIs, and cannot be filtered out.
For any KPI time series, filtering the KPI time series based on the KPI surge amount of two adjacent time windows. The method specifically comprises the following steps: in the double-sliding-window abnormal detection algorithm, time windows with the same length are respectively arranged at the front side and the rear side of a time t according to the evolution sequence of a time axis, the KPI sum in the two time windows is respectively calculated, and the ratio of the KPI sum in the rear time window (t to t 2) to the KPI sum in the front time window (t 1 to t) (the difference value of t and t1 is equal to the difference value of t2 and t) is used as the KPI abrupt increment of the time t. And if the sum of the KPIs in the previous time window and the sum of the KPIs in the next time window are zero, setting the KPI sudden increase amount at the time t to be 0. And if the sum of the KPIs in the previous time window is zero and the sum of the KPIs in the next time window is not zero, setting the KPI sudden increase amount at the time t as the sum of the KPIs in the next time window. If the KPI sudden increase of a KPI time sequence at all times is lower than the second value, the KPI time sequence is considered to reflect normal packet loss behavior, and the detection equipment filters the KPI time sequence without subsequent filtering and suspected abnormal KPI detection processes. And if the KPI sudden increase quantity at the moment t is greater than or equal to a second numerical value, the KPI time sequence is not filtered by the detection equipment. The second value may be preset and stored to the detection device.
For any KPI time series, the detection device filters the KPI time series based on the periodic KPIs. The method specifically comprises the following steps: the detection device may extract periodic KPI components from the KPI time series, which are considered to be periodic if all or most of the periodic KPI components have the same or very similar periodic waveforms. The method for detecting the periodic KPI component may be a method commonly used in the industry, such as a periodic component decomposition method based on a moving average. For the lost packet type KPI, since the periodic packet loss is considered as a normal packet loss behavior, if the KPI time sequence has periodicity, the KPI time sequence is filtered, otherwise, the KPI time sequence is retained to be ready for a subsequent filtering process. It should be noted that if most of the KPIs with data volume less than two cycles and the periodic KPIs are zero, the periodic detection is not performed, and the subsequent filtering process is directly performed to ensure a high recall rate.
For any KPI time series, the detection device filters the KPI time series if the KPI time series has steady-state characteristics, and retains the KPI time series if the KPI time series does not have steady-state characteristics. The method specifically comprises the following steps: whether a KPI time series has steady-state characteristics is judged based on a time series steady-state analysis method commonly used in the industry. For example, an assumed value (which may also be referred to as an assumed probability (p-value)) obtained from an enhanced diky-Fuller (ADF) test of a KPI time series indicates that the KPI time series does not have a steady-state property if the assumed value is greater than 5%, and indicates that the KPI time series has a steady-state property if the assumed value is less than or equal to 5%. It should be noted here that, if a KPI time sequence includes a small number of KPI samples (for example, less than 20 samples), or if a sudden increase component included in the KPI time sequence is not negligible (for example, a KPI sample value exceeding 5% is too large), which may affect the accuracy of steady state determination, the KPI time sequence skips the steady state determination, and directly performs subsequent suspected abnormal KPI detection, so as to ensure a high recall rate. The concept of p-value is: in statistics, p-value belongs to the interval of [0,1], which is an important evidence for judging whether the original hypothesis is correct. A small p-value (e.g., within 5%) indicates that the original hypothesis is correct with a large probability, otherwise the hypothesis is incorrect. For example, in the present embodiment, it is originally assumed that a KPI time series has a steady-state characteristic, and p-value is greater than 5%, which indicates that the KPI time series does not have a steady-state characteristic.
The 4 filtering modes are described above, when any KPI time sequence is filtered, one or more of the above filtering modes can be used for filtering, and the multiple filtering modes can be arbitrarily ordered in the serial flow. Therefore, after the KPI time sequence is filtered by the detection equipment, suspected abnormal KPI data used for updating the abnormal KPI sample library is determined, so that high recall rate can be ensured, namely missing reports are reduced.
As shown in fig. 7, a flow is provided for updating the abnormal KPI detection model by the analysis device based on the suspected abnormal KPI provided by the detection device:
step 701, the analysis device obtains suspected abnormal KPI data determined by the detection device using the suspected abnormal KPI detection model.
In this embodiment, the analysis device receives suspected abnormal KPI data determined by the detection device using the suspected abnormal KPI detection model. The suspected abnormal KPI data comprises one or more suspected abnormal KPI time sequences.
In step 702, the analysis device updates the current abnormal KPI sample library according to the suspected abnormal KPI data.
The current abnormal KPI sample library refers to an abnormal KPI sample library stored by the analysis equipment at the current moment. The abnormal KPI sample library comprises abnormal KPI time sequences (the abnormal KPI time sequences are KPI time sequences with abnormal KPI), and abnormal time points and/or abnormal time periods corresponding to the KPI time sequences. Of course, the abnormal KPI sample library may also include an identifier of a detection device to which the abnormal KPI time series belongs, an identifier of a module to which the abnormal KPI time series belongs, and the like.
In this embodiment, the analysis device determines abnormal KPI data in the suspected abnormal KPI data according to the suspected abnormal KPI data. And adding the abnormal KPI data to the current abnormal KPI sample library to update the current abnormal KPI sample library. Specifically, the abnormal KPI data comprises one or more abnormal KPI time series, and the abnormal KPI data may further comprise abnormal time periods and/or abnormal time points in the abnormal KPI time series for indicating abnormal KPIs in the abnormal KPI time series. The abnormal time point is the time point of the abnormal KPI in the abnormal KPI time sequence, and the abnormal time section is the time section formed by the continuous time points of the abnormal KPI in the abnormal KPI time sequence. Specifically, the process of adding the abnormal KPI data to the current abnormal KPI sample library is as follows: each abnormal KPI time sequence in the abnormal KPI data and the abnormal time point and/or the abnormal time period corresponding to the abnormal KPI time sequence are used as an abnormal KPI sample (namely, one abnormal KPI sample comprises one KPI time sequence and the abnormal time point and/or the abnormal time period corresponding to the KPI time sequence), and are added to the current abnormal KPI sample library.
And 703, updating the current abnormal KPI detection model by the analysis equipment according to the updated abnormal KPI sample library.
In this embodiment, the analysis device updates the current abnormal KPI detection model according to the updated abnormal KPI sample library, so as to ensure that abnormal KPI data detected by the abnormal KPI detection model used by the detection device is relatively accurate. Here, the "update processing" may be to update parameters in the current abnormal KPI detection model based on the updated abnormal KPI sample library to obtain the updated abnormal KPI detection model, or to perform supervised training based on the updated abnormal KPI sample library and a machine learning algorithm to obtain the updated abnormal KPI detection model, and of course, other manners may also be used, which is not limited in the embodiment of the present application.
Therefore, the updating of the abnormal KPI sample base is realized by continuously acquiring suspected abnormal KPI data, and then the abnormal KPI detection model can be continuously updated, so that the abnormal KPI detection model continuously keeps higher detection performance.
In one possible implementation manner, when the current abnormal KPI sample library is updated in step 702, the following process is performed:
in step 7021, the analysis device obtains KPI data that is confirmed to be abnormal from the suspected abnormal KPI data.
In this embodiment, there are many ways to implement step 7021, and two possible ways are given below:
the analysis equipment sends suspected abnormal KPI data to the management equipment and receives KPI data confirmed to be abnormal in the suspected abnormal KPI data fed back by the management equipment; or displaying suspected abnormal KPI data, and acquiring KPI data confirmed to be abnormal in the displayed suspected abnormal KPI data.
Specifically, the analysis device sends the suspected abnormal KPI data to the management device after acquiring the suspected abnormal KPI data. And displaying the suspected abnormal KPI data by the management equipment, selecting the abnormal KPI data from the suspected abnormal KPI data displayed by the management equipment by a user, and submitting the abnormal KPI data, wherein the user is a person for confirming the abnormal KPI data in the suspected abnormal KPI data. And the management equipment sends the abnormal KPI data selected by the user to the analysis equipment. The analysis equipment receives KPI data confirmed to be abnormal in the suspected abnormal KPI data. Or, in the case that the analysis device has a function of displaying suspected abnormal KPI data for the user, after the analysis device obtains the suspected abnormal KPI data, the analysis device displays the suspected abnormal KPI data, and the user selects and submits the abnormal KPI data from the displayed suspected abnormal KPI data, so that the analysis device obtains the KPI data confirmed to be abnormal in the suspected abnormal KPI data.
In addition, after the user confirms abnormal KPI data in the suspected abnormal KPI data, the time point or the time segment of the confirmed abnormal KPI data can be modified, so that the confirmed abnormal KPI data can be used as an abnormal KPI sample.
Of course, the analysis device may also provide the user with KPI-related information such as suspected abnormal KPI time series, KPI surge degree time series, KPI data probability distribution, and the like. The suspected abnormal KPI time sequence refers to a KPI time sequence including suspected abnormal KPI data in the KPI time sequence. The KPI surge degree time series includes the surge degree of KPIs at any two adjacent time points.
At step 7022, the analysis device adds the KPI data that is confirmed to be abnormal as an abnormal KPI sample to the current abnormal KPI sample library.
In this embodiment, after acquiring KPI data confirmed to be abnormal from among suspected abnormal KPI data, the analysis device determines the KPI data confirmed to be abnormal as an abnormal KPI sample. And the analysis equipment adds the abnormal KPI sample to the current abnormal KPI sample library to obtain an updated abnormal KPI sample library.
In this way, the library of abnormal KPI samples can be updated, and the updated abnormal KPI samples are the true abnormal KPI data. More importantly, when the abnormal KPI sample library is updated, only a small amount of suspected abnormal KPI data needs to be marked manually, and screening is not performed on a large amount of KPI data, so that the efficiency of manually marking and expanding the abnormal KPI sample library can be obviously improved.
It should be noted here that, when determining abnormal KPI data in the suspected abnormal KPI data, the user may confirm one by one or selectively confirm. The step of confirming one by one refers to that the user confirms suspected abnormal KPI data reported by all detection equipment. The selective confirmation may be that the user confirms only suspected abnormal KPI data reported by some detection devices, or that the user confirms only suspected abnormal KPI data, which has a larger waveform difference from samples existing in the current abnormal KPI sample library, in the suspected abnormal KPI data.
Here, it should be noted that, after the user confirms that the suspected abnormal KPI data is abnormal KPI data, the abnormal KPI data is added to the abnormal KPI sample library, and the abnormal KPI sample library is expanded in terms of the type and number of samples. For an abnormal KPI data newly added into an abnormal KPI sample base, if the waveform is the same as or similar to the waveform of one or more existing abnormal KPI samples, the abnormal KPI data is considered to be the expansion of the number of the abnormal KPI sample base, and if the waveform is different from the waveform of any existing abnormal KPI sample, the abnormal KPI data is considered to be the expansion of the type of the abnormal KPI sample base.
In a possible implementation manner, after step 702, in order to enable the suspected abnormal KPI detection model to continuously maintain the detection performance, after each abnormal KPI sample library is updated, the analysis device may update the current suspected abnormal KPI detection model, and provide the updated suspected abnormal KPI detection model to the detection device, where the processing is as follows:
and the analysis equipment updates the suspected abnormal KPI detection model according to the updated abnormal KPI sample library, and provides the updated suspected abnormal KPI detection model for the detection equipment.
In this embodiment, the analysis device updates parameters in the current suspected abnormal KPI detection model by using the updated abnormal KPI sample library and using a preset performance requirement as a constraint condition, so as to obtain an updated suspected abnormal KPI detection model. The current suspected abnormal KPI detection model may also be referred to herein as a pre-update suspected abnormal KPI detection model.
The analysis device sends a first update message to the detection device, where the first update message may include the updated parameter, or may include a variation between the updated parameter and a parameter before updating, or may be an updated suspected abnormal KPI detection model, that is, an entire model file of the updated suspected abnormal KPI detection model.
The detection device and the analysis device agree on the type of the content carried by the first update message. After the detection device receives a first update message sent by the analysis device, if the first update message includes updated parameters, the detection device directly replaces the parameters of the current suspected abnormal KPI detection model with the parameters included in the first update message to obtain the updated suspected abnormal KPI detection model.
If the first update message includes the updated parameter and the variation of the parameter before updating, the detection device determines the updated parameter by using the variation and the parameter of the current suspected abnormal KPI detection model, and replaces the parameter of the current suspected abnormal KPI detection model with the updated parameter to obtain the updated suspected abnormal KPI detection model.
If the first update message includes the updated suspected-abnormal KPI detection model, the detection device may directly use the updated suspected-abnormal KPI detection model to replace the current suspected-abnormal KPI detection model, so as to obtain the updated suspected-abnormal KPI detection model. Specifically, the detection device loads a complete model file of the updated suspected abnormal KPI detection model to obtain the updated suspected abnormal KPI detection model.
In this way, the suspected abnormal KPI detection model can be continuously updated, so that the suspected abnormal KPI detection model can be kept at a high detection performance.
It should be noted here that the reason why the first update message carries parameters to instruct the detection device to update the current suspected abnormal KPI detection model is as follows: in the updating process of the suspected abnormal KPI detection model, the frame of the suspected abnormal KPI detection model is not changed, and only the parameters in the suspected abnormal KPI detection model are changed. The parameters of the suspected abnormal KPI detection model include parameters (such as a first value) in filtering the KPI time series, and/or parameters in an abnormal sudden increase detection process.
In one possible implementation manner, after step 703, in order to enable the detection apparatus to obtain the latest abnormal KPI detection model, after each abnormal KPI detection model update, the analysis apparatus may provide the updated abnormal KPI detection model to the detection apparatus, and the process is as follows:
and the analysis equipment provides the updated abnormal KPI detection model to the detection equipment.
In this embodiment, the analysis device sends a second update message of the abnormal KPI detection model to the detection device, where the second update message may include an updated parameter, may also include a variation between the updated parameter and a parameter before updating, and may also be an updated abnormal KPI detection model, that is, an entire model file of the updated abnormal KPI detection model.
The detection device and the analysis device agree on the type of the content carried by the second update message. After the detection device receives a second update message sent by the analysis device, if the second update message includes updated parameters, the detection device directly replaces the parameters of the current abnormal KPI detection model with the parameters included in the second update message to obtain the updated abnormal KPI detection model.
If the second update message includes the updated parameter and the variation of the parameter before updating, the detection device determines the updated parameter by using the variation and the parameter of the current abnormal KPI detection model, and replaces the parameter of the current abnormal KPI detection model with the updated parameter to obtain the updated abnormal KPI detection model.
If the updated abnormal KPI detection model is included in the second update message, the detection device may directly use the updated abnormal KPI detection model to replace the current abnormal KPI detection model, so as to obtain the updated abnormal KPI detection model. Specifically, the detection device loads a complete model file of the updated abnormal KPI detection model to obtain the updated abnormal KPI detection model. Therefore, the abnormal KPI detection model can be updated in time, and the detection performance of the abnormal KPI detection model is continuously guaranteed.
In a possible implementation manner, in order to reduce unnecessary updating of the abnormal KPI detection model, before step 703, it is further detected whether the current abnormal KPI detection model meets the detection performance requirement, if the detection performance requirement is not met, the updating process of step 703 is executed, and if the detection performance requirement is met, the updating process of step 703 is not executed. Specifically, the process of determining whether the abnormal KPI detection model meets the detection performance requirement is as follows: the analysis equipment acquires abnormal KPI data determined by the detection equipment by using a current abnormal KPI detection model, wherein the abnormal KPI data refers to the abnormal KPI data determined by the detection equipment which updates the abnormal KPI detection model last time to a current time point. And the analysis equipment determines whether the abnormal KPI detection model meets the detection performance requirements by using the abnormal KPI data and the updated abnormal KPI sample library.
Specifically, the process of acquiring abnormal KPI data determined by the detection device using the current abnormal KPI detection model by the analysis device is as follows:
the detection equipment acquires an abnormal KPI detection model; determining abnormal KPI data through an abnormal KPI detection model; and reporting the abnormal KPI data to the analysis equipment.
In this embodiment, the detection device stores an abnormal KPI detection model, and the abnormal KPI detection model is issued to the detection device by the analysis device. And the detection equipment acquires KPI data, inputs the KPI data into an abnormal KPI detection model and acquires abnormal KPI data. The abnormal KPI data comprises a KPI time-series (which may be referred to as abnormal KPI time-series) which is marked as abnormal, and the abnormal KPI data may further comprise abnormal time periods and/or abnormal time points in the KPI time-series which are marked as abnormal, for indicating abnormal KPIs in the abnormal KPI time-series. The abnormal time point is the time point of the abnormal KPI in the abnormal KPI time sequence, and the abnormal time section is the time section formed by the continuous time points of the abnormal KPI in the abnormal KPI time sequence.
The detection device then sends the abnormal KPI data to the analysis device. And the analysis equipment receives the abnormal KPI data sent by the detection equipment so as to obtain the abnormal KPI data.
It is noted herein that the abnormal KPI data includes one or more abnormal KPI time series.
Specifically, the analysis device determines whether the current abnormal KPI detection model meets the detection performance requirement according to the following method:
the analysis equipment determines the performance indexes of the current abnormal KPI detection model according to the abnormal KPI data and the updated abnormal KPI sample library, wherein the performance indexes comprise at least one of a missing report rate, a false report rate, a recall rate and an accuracy rate; and determining that the abnormal KPI detection model does not meet the detection performance requirement according to the performance index.
In this embodiment, in the abnormal KPI data detected by the detection device in the target time period, the analysis device determines the number of abnormal KPI time sequences that belong to the updated abnormal KPI sample library (i.e., the number of abnormal KPI time sequences that are correctly reported), and the number of abnormal KPI time sequences that do not belong to the updated abnormal KPI sample library (i.e., the number of abnormal KPI time sequences that are incorrectly reported (abbreviated as false reports)). And the analysis equipment determines KPI data confirmed to be abnormal in suspected abnormal KPIs reported by the detection equipment in the target time period, and determines the number of abnormal KPI time sequences (namely the number of abnormal KPI time sequences which are not reported) belonging to the abnormal KPI data detected by the detection equipment in the target time period in the KPI data. The target time segment here may be a time segment from a time point when the current abnormal KPI detection model starts to be used to a current time point, or may be a current cycle in which whether the abnormal KPI detection model meets the detection performance requirement is periodically determined after the current abnormal KPI detection model starts to be used, and an end time point of the current cycle is the current time point (in the latter case, each cycle is determined in this manner).
The analysis device then uses the number of the correctly reported abnormal KPI time series to divide the first number (the first number is the number of KPI time series confirmed to be abnormal in suspected abnormal KPIs detected by the detection device in the target time period) to obtain the recall rate.
And the analysis equipment divides the number of the abnormal KPI time sequences which are not reported by the first number by the number of the abnormal KPI time sequences which are not reported by the first number to obtain a report missing rate.
The analysis device uses the number of the abnormal KPI time sequences reported correctly to divide by a second number (the second number is the number of the abnormal KPI time sequences included in the abnormal KPI data detected by the detection device in the target time period) to obtain the accuracy.
And the analysis equipment divides the number of the abnormal KPI time sequences with false alarm by the second number to obtain the false alarm rate.
And the analysis equipment determines whether the abnormal KPI detection model does not meet the detection performance requirement or not according to the performance index. Specifically, when determining whether the abnormal KPI detection model does not meet the detection performance requirement, one of the conditions that the false alarm rate is higher than the first threshold, the false alarm rate is higher than the second threshold, the recall rate is higher than the third value, and the accuracy rate is lower than the fourth threshold may be used, and when the condition is met, it is determined that the abnormal KPI detection model does not meet the detection performance requirement, and when the condition is not met, it is determined that the abnormal KPI detection model meets the detection performance requirement. When judging whether the abnormal KPI detection model does not meet the detection performance requirement, at least two conditions of a false alarm rate higher than a first threshold, a false alarm rate higher than a second threshold, a recall rate higher than a third value and an accuracy rate lower than a fourth threshold can be used. When at least two conditions are used, it may be determined that the abnormal KPI detection model does not meet the detection performance requirement when one or more of the at least two conditions are met, and it is determined that the abnormal KPI detection model meets the detection performance requirement when none of the at least two conditions are met; or, when the at least two conditions are both satisfied, determining that the abnormal KPI detection model does not satisfy the detection performance requirement, and otherwise, determining that the abnormal KPI detection model satisfies the detection performance requirement.
In addition, the analysis equipment can generate an abnormal KPI alarm under the condition that the current abnormal KPI detection model meets the detection performance, and then sends the abnormal KPI alarm to a user terminal for abnormal investigation, so that a user can know the abnormal condition of the network in time. The abnormal KPI alarms may comprise abnormal KPI data, an identification of the detection device to which the abnormal KPI data belongs. If the abnormal KPI data includes abnormal KPI data for characterizing the performance of a certain module in the detection device, the abnormal KPI alarm further includes an identifier of the module.
It should be noted that, in scenario one, the analysis device is directly connected to the detection devices in one network, and when determining whether the current abnormal KPI detection model meets the detection performance requirement, the abnormal KPI data provided by all the detection devices in the network to which the abnormal KPI detection model belongs may be used. When the abnormal KPI detection model is updated, abnormal KPI data provided by all detection devices in a network to which the abnormal KPI detection model belongs can also be used. The fact that the current abnormal KPI detection model does not satisfy the detection performance requirement (which may also be considered as the performance degradation of the current abnormal KPI detection model) means that: the current abnormal KPI detection model cannot adapt to the current KPI data of the network, and the detection performance requirements cannot be met when abnormal KPI data in the current KPI data are detected. For example, the accuracy in the detection performance requirement is 95%, and the accuracy of the current abnormal KPI detection model for detecting abnormal KPI data in the current KPI data is 80%, which indicates that the current abnormal KPI detection model cannot adapt to the current KPI data of the network.
In this embodiment of the present application, the suspected abnormal KPI data provided by the detection device for the analysis device may further include an identifier of the detection device to which the suspected abnormal KPI data belongs. And if the KPI data provided by the detection equipment is used for representing the performance of the module in the detection equipment, identifying the module in the suspected abnormal KPI data. Of course, the abnormal KPI data provided by the detection device for the analysis device may further include an identifier of the detection device to which the abnormal KPI data belongs. If the KPI data provided by the detection device is used to characterize the performance of a module in the detection device, the abnormal KPI data also includes the identifier of the module.
In one possible implementation, the process of the analyzing device providing the suspected abnormal KPI detection model (i.e. the initial suspected abnormal KPI detection model) for the detecting device for the first time is as follows:
the analysis equipment trains to obtain a suspected abnormal KPI detection model according to a first preset performance requirement; and providing the suspected abnormal KPI detection model for the detection equipment.
The first preset performance requirement comprises one or more of a missed report rate lower than a missed report threshold, a recall rate higher than a recall threshold, an accuracy rate higher than an accurate threshold or a false report rate lower than a false report threshold.
In this embodiment, the analysis device may train to obtain an initial suspected abnormal KPI detection model by using the first preset performance requirement as a constraint condition. And the analysis equipment sends the initial suspected abnormal KPI detection model to the detection equipment, and the detection equipment determines suspected abnormal KPI data in the KPI data by using the initial suspected abnormal KPI detection model.
It should be noted that the suspected abnormal KPI detection model is mainly used to provide KPI data that may have abnormality, so when training the suspected abnormal KPI detection model, it is prioritized that the recall rate is higher than the recall threshold, and/or the false-negative rate is lower than the false-negative threshold.
In a possible implementation manner, the process of training the analysis device to obtain the suspected abnormal KPI detection model includes: the analysis equipment acquires target suspected abnormal KPI data in the initial KPI data; acquiring KPI data confirmed to be abnormal in the target suspected abnormal KPI data; and training to obtain an initial suspected abnormal KPI detection model according to the first preset performance requirement and KPI data confirmed to be abnormal in the target suspected abnormal KPI data.
In this embodiment, after acquiring KPI data (i.e., initial KPI data), the detecting device sends the KPI data to the analyzing device, and the analyzing device acquires suspected abnormal KPI data (i.e., target suspected abnormal KPI data) in the initial KPI data. The analysis equipment acquires KPI data confirmed to be abnormal in the target suspected abnormal KPI data. And then the analysis equipment takes the first preset performance requirement as a constraint condition, and trains and obtains an initial abnormal KPI detection model by using the KPI data confirmed to be abnormal in the target suspected abnormal KPI data.
In a possible implementation manner, the analysis device updates a preset unsupervised abnormal KPI detection model to obtain an initial suspected abnormal KPI detection model, and the processing is as follows:
the analysis equipment determines target suspected abnormal KPI data in the initial KPI data according to a preset unsupervised abnormal KPI detection model, and updates the unsupervised abnormal KPI detection model according to a first preset performance requirement and the KPI data confirmed to be abnormal in the target suspected abnormal KPI data to obtain the initial suspected abnormal KPI detection model.
In this embodiment, the unsupervised abnormal KPI detection model is an abnormal KPI detection model obtained based on an unsupervised learning algorithm, and is used for detecting suspected abnormal KPI data in the initial KPI data, where the framework of the unsupervised abnormal KPI detection model is the same as that of the suspected abnormal KPI detection model in the flow of fig. 6, except that the parameter values of the parameters in the unsupervised abnormal KPI detection model are initial parameter values, and the parameter values of the parameters in the suspected abnormal KPI detection model in fig. 6 are updated at least once on the basis of the initial parameter values. The analysis device uses an unsupervised abnormal KPI detection model to determine suspected abnormal KPI data in the initial KPI data (see fig. 6 for processing, which is not described here). The analysis device then composes the suspected abnormal KPI data in the initial KPI data into a library of abnormal KPI samples. And the analysis equipment takes the first preset performance requirement as a constraint condition, updates the unsupervised abnormal KPI detection model based on the abnormal KPI sample library, and obtains an initial suspected abnormal KPI detection model.
In a possible implementation manner, the process of the analyzing device providing the abnormal KPI detection model for the detecting device for the first time is as follows:
and the analysis equipment trains and obtains a current abnormal KPI detection model according to a second preset performance requirement, wherein the current abnormal KPI detection model is an initial abnormal KPI detection model, and the current abnormal KPI detection model is provided for the detection equipment.
The second preset performance requirement comprises one or more of a missing report rate not higher than a first threshold, a false report rate not higher than a second threshold, a recall rate not higher than a third value or an accuracy rate not lower than a fourth threshold.
In this embodiment, when there are enough abnormal KPI data in an abnormal KPI sample library established based on suspected abnormal KPI data, the analysis device performs supervised learning by using the abnormal KPI sample library to obtain an abnormal KPI detection model (i.e., an initial abnormal KPI detection model) that meets a second preset performance requirement. Here, sufficient "means that the supervised training and updating requirements of the machine learning model (i.e. the abnormal KPI detection model) can be satisfied.
It should be noted here that the abnormal KPI detection model is mainly used to provide accurate abnormal KPI data, so when training the abnormal KPI detection model, it is prioritized that the accuracy is not lower than the fourth threshold, and/or the false alarm rate is not higher than the second threshold.
In the embodiment of the application, the analysis equipment can update the abnormal KPI sample library in time, so that the abnormal KPI detection model is updated in time, high accuracy is ensured, and false alarms are reduced.
In addition, in the embodiment of the application, in the deployment stage of the abnormal KPI detection model, the user is assisted to label the abnormal KPI samples, and in the abnormal detection reasoning stage, suspected abnormal KPI data in the network is automatically sent to the analysis equipment, so that the accumulation of the types and the quantity of the samples of the abnormal KPI sample library is promoted.
It should be noted that the above is described by taking the scenario shown in fig. 2 as an example, and when the present application is applied to the scenario shown in fig. 3, the analysis device manages a plurality of networks, and detects whether the device is a network device in the network. The analysis equipment also determines an abnormal KPI detection model and a suspected abnormal KPI detection model, and sends the abnormal KPI detection model and the suspected abnormal KPI detection model to the detection equipment through a management device of a management network. And the detection equipment determines suspected abnormal KPI data and abnormal KPI data, and reports the data to the analysis equipment through the management device. The management device for managing the network is only used for transferring between the analysis device and the detection device, and the specific processing flow may refer to the scenario in fig. 2, which is not described herein again.
When the present application is applied to the scenario of fig. 4, the analysis device manages a plurality of networks, and the detection device is a management apparatus that manages the networks. The analysis equipment also determines an abnormal KPI detection model and a suspected abnormal KPI detection model and sends the abnormal KPI detection model and the suspected abnormal KPI detection model to a management device of a management network. And the network equipment in the network sends the KPI data to a management device of the affiliated network. And the management device determines suspected abnormal KPI data and reports the data to the analysis equipment. The specific processing flow can refer to the scenario in fig. 2, and is not described herein again.
It should be noted that, when the present application is applied to the scenarios shown in fig. 3 and fig. 4, each network may use a respective abnormal KPI detection model and suspected abnormal KPI detection model. When the abnormal KPI detection model is trained and updated, only the abnormal KPI data of the network to which the abnormal KPI detection model belongs can be used, and the abnormal KPI data of each network in the abnormal KPI sample library can also be used. When training and updating the suspected abnormal KPI detection model, only the abnormal KPI data of the network to which the suspected abnormal KPI detection model belongs in the abnormal KPI sample library may be used, or the abnormal KPI data of each network in the abnormal KPI sample library may be used. When the abnormal KPI data of each network in the abnormal KPI sample library is used to train or update a model (such as an abnormal KPI detection model or a suspected abnormal KPI detection model) of a certain network, the weight of the abnormal KPI data of the network is relatively high. When judging whether the abnormal KPI detection model meets the detection performance requirement, the abnormal KPI data of the network to which the abnormal KPI detection model belongs is used. In this case, in the abnormal KPI sample library, each abnormal KPI data indicates a belonging network, and specifically, the network identifier of the belonging network of the abnormal KPI data may be used for indicating. It should be noted here that, when each network uses its own abnormal KPI detection model and suspected abnormal KPI detection model, when the analysis device issues the abnormal KPI detection model for the first time, the same abnormal KPI detection model is likely to be issued for each network, and when the analysis device issues the suspected abnormal KPI detection model for the first time, the same suspected abnormal KPI detection model is likely to be issued for each network.
Of course, each network may use the same abnormal KPI detection model and suspected abnormal KPI detection model in common. And when the suspected abnormal KPI detection model is trained and updated, the abnormal KPI data of each network in the abnormal KPI sample library is used. When judging whether the abnormal KPI detection model meets the detection performance requirement, the abnormal KPI data of each network in the abnormal KPI database is also used.
In addition, in the embodiment of the present application, in order to better understand the embodiment of the present application, a deployment process of a suspected abnormal KPI detection model is further provided corresponding to the scenario of fig. 2, and an interaction diagram between an analysis device and a detection device is provided. As shown in fig. 8, step 801, the detection device provides initial KPI data to the analysis device; step 802, the analysis device determines suspected abnormal KPI data in the initial KPI data; step 803, the analysis equipment acquires KPI data confirmed to be abnormal in suspected abnormal KPI data, and establishes an abnormal KPI sample library; step 804, the analysis equipment updates an unsupervised learning algorithm based on the abnormal KPI sample library to obtain an initial suspected abnormal KPI monitoring model; step 805, the analysis device issues the initial suspected abnormal KPI detection model to the detection device; step 806, the detection device deploys an initial suspected abnormal KPI detection model, and detects suspected abnormal KPI data.
In order to better understand the embodiment of the present application, corresponding to the scenario of fig. 2, a deployment process of the abnormal KPI detection model is also provided, and an interaction diagram between the analysis device and the detection device is provided. As shown in fig. 9, in step 901, the analysis device performs supervised learning based on the abnormal KPI sample library established in step 803 to obtain an initial abnormal KPI detection model; step 902, the analysis device issues the initial abnormal KPI detection model to the detection device; and 903, deploying an initial abnormal KPI detection model by the detection equipment, and detecting abnormal KPI data.
In addition, in the embodiment of the present application, in order to better understand the embodiment of the present application, an update process of a suspected abnormal KPI detection model and an interaction diagram between an analysis device and a detection device are also provided corresponding to the scenario of fig. 2. As shown in fig. 10, step 1001, the detection device provides suspected abnormal KPI data to the analysis device; step 1002, an analysis device acquires KPI data confirmed to be abnormal in suspected abnormal KPI data; step 1003, updating an abnormal KPI sample base based on the KPI data confirmed to be abnormal; step 1004, updating the current suspected abnormal KPI detection model by the analysis equipment based on the updated abnormal KPI sample library to obtain an updated suspected abnormal KPI detection model; step 1005, the analysis equipment issues the updated suspected abnormal KPI detection model to the detection equipment; step 1006, the detection device deploys the updated suspected abnormal KPI detection model, and detects suspected abnormal KPI data.
In addition, in the embodiment of the present application, in order to better understand the embodiment of the present application, corresponding to the scenario of fig. 2, an update process of an abnormal KPI detection model is further provided, and an interaction diagram between an analysis device and a detection device is provided. As shown in fig. 11, in step 1101, the analysis device receives abnormal KPI data reported by the detection device; 1102, determining that a current abnormal KPI detection model does not meet the detection performance requirement by the analysis equipment based on abnormal KPI data reported by the detection equipment and a current abnormal KPI sample library, and updating the current abnormal KPI detection model into an updated abnormal KPI detection model based on the current abnormal KPI sample library; 1103, the analysis equipment issues the updated abnormal KPI detection model to the detection equipment; and 1104, deploying the updated abnormal KPI detection model by the detection equipment, and detecting abnormal KPI data.
It should be further noted that, in the above embodiments, the process of updating the abnormal KPI detection model and the suspected abnormal KPI detection model only once is described. The abnormal KPI detection model and the suspected abnormal KPI detection model may be continuously updated in the manner described above.
Fig. 12 is a schematic structural diagram for implementing update of an anomaly detection model according to an embodiment of the present application. The apparatus may be implemented as part or all of the apparatus by software, hardware or a combination of both. The apparatus provided in this embodiment of the present application may implement the process described in fig. 7 in this embodiment of the present application, and the apparatus includes: an obtaining module 1210 and an updating module 1220, wherein:
an obtaining module 1210, configured to obtain suspected abnormal KPI data determined by a detection device using a current suspected abnormal KPI detection model, and specifically may be configured to implement the obtaining function of step 701 and an implicit step included in step 701;
an update module 1220, configured to:
updating the current abnormal KPI sample base according to the suspected abnormal KPI data;
according to the updated abnormal KPI sample library, the current abnormal KPI detection model is updated, which may be specifically used to implement the updating function of step 702 and step 703 and the implicit steps included in step 702 and step 703.
In a possible implementation manner, a recall rate of the suspected abnormal KPI detection model is higher than a recall rate of the abnormal KPI detection model, or a false-positive rate of the suspected abnormal KPI detection model is lower than a false-positive rate of the abnormal KPI detection model.
In a possible implementation manner, the accuracy of the abnormal KPI detection model is higher than that of the suspected abnormal KPI detection model, or the false alarm rate of the abnormal KPI detection model is lower than that of the suspected abnormal KPI detection model.
In a possible implementation manner, the updating module 1220 is configured to:
acquiring KPI data confirmed to be abnormal in the suspected abnormal KPI data;
and taking the KPI data confirmed to be abnormal as abnormal KPI samples, and adding the abnormal KPI samples to the current abnormal KPI sample library.
In a possible implementation manner, the updating module 1220 is configured to:
sending the suspected abnormal KPI data to a management device, and receiving KPI data confirmed to be abnormal in the suspected abnormal KPI data fed back by the management device; alternatively, the first and second liquid crystal display panels may be,
displaying the suspected abnormal KPI data, and acquiring KPI data confirmed to be abnormal in the displayed suspected abnormal KPI data.
In a possible implementation manner, the updating module 1220 is further configured to:
and according to the updated abnormal KPI sample library, before updating the current abnormal KPI detection model, determining that the current abnormal KPI detection model does not meet the detection performance requirement according to the updated abnormal KPI sample library.
In a possible implementation manner, the updating module 1220 is further configured to:
according to the updated abnormal KPI sample library, acquiring abnormal KPI data determined by the detection equipment by using the abnormal KPI detection model before determining that the current abnormal KPI detection model does not meet the detection performance requirement;
the update module is configured to:
and determining that the abnormal KPI detection model does not meet the detection performance requirement according to the abnormal KPI data and the updated abnormal KPI sample library.
In a possible implementation manner, the updating module 1220 is configured to:
determining a performance index of the abnormal KPI detection model according to the abnormal KPI data and the updated abnormal KPI sample library, wherein the performance index comprises at least one of a missing report rate, a false report rate, a recall rate and an accuracy rate;
and determining that the abnormal KPI detection model does not meet the detection performance requirement according to the performance index.
In one possible implementation, the suspected abnormal KPI detection model is an unsupervised learning model.
In one possible implementation, as shown in fig. 13, the apparatus further includes:
the training module 1230 is configured to train to obtain a suspected abnormal KPI detection model according to a first preset performance requirement before obtaining suspected abnormal KPI data determined by a detection device using a current suspected abnormal KPI detection model, where the suspected abnormal KPI detection model is an initial suspected abnormal KPI detection model;
a sending module 1240, configured to provide the suspected abnormal KPI detection model to the detection device.
In a possible implementation manner, the obtaining module 1210 is further configured to:
according to a first preset performance requirement, before the suspected abnormal KPI detection model is obtained through training, target suspected abnormal KPI data in initial KPI data are obtained;
the training module 1230 is configured to:
acquiring KPI data confirmed to be abnormal in the target suspected abnormal KPI data;
and training to obtain an initial suspected abnormal KPI detection model according to a first preset performance requirement and KPI data confirmed to be abnormal in the target suspected abnormal KPI data.
In a possible implementation manner, the obtaining module 1210 is further configured to:
according to a preset unsupervised abnormal KPI detection model, determining target suspected abnormal KPI data in the initial KPI data;
the training module 1230 is further configured to:
and updating the unsupervised abnormal KPI detection model according to a first preset performance requirement and KPI data confirmed to be abnormal in the target suspected abnormal KPI data to obtain an initial suspected abnormal KPI detection model.
In a possible implementation manner, the updating module 1220 is further configured to:
updating a current abnormal KPI sample base according to the suspected abnormal KPI data, and then updating the suspected abnormal KPI detection model according to the updated abnormal KPI sample base;
the device further comprises a sending module, configured to provide the updated suspected abnormal KPI detection model to the detection device.
In one possible implementation, the abnormal KPI detection model is a supervised learning model.
In one possible implementation, as shown in fig. 13, the apparatus further includes:
the training module 1230 is configured to train and obtain the current abnormal KPI detection model according to a second preset performance requirement before updating the current abnormal KPI detection model according to the updated abnormal KPI sample library, where the current abnormal KPI detection model is an initial abnormal KPI detection model;
a sending module 1240, configured to provide the current abnormal KPI detection model to the detection device.
In one possible implementation, the apparatus further includes:
a sending module 1240, configured to provide the updated abnormal KPI detection model to the detection device.
In one possible implementation, the suspected abnormal KPI data includes a suspected abnormal KPI time series.
In one possible implementation, the suspected abnormal KPI data further includes an abnormal time period and/or an abnormal time point in the suspected abnormal KPI time series.
The division of the modules in the embodiments of the present application is illustrative, and only one logical function division is provided, and in actual implementation, there may be another division manner, and in addition, each functional module in each embodiment of the present application may be integrated in one processor, may also exist alone physically, or two or more modules are integrated into one module. The integrated module can be realized in a hardware mode, and can also be realized in a software functional module mode.
Fig. 14 is a schematic structural diagram for implementing an update of an anomaly detection model according to an embodiment of the present application. The apparatus may be implemented as part or all of the apparatus by software, hardware or a combination of both. The apparatus provided in the embodiment of the present application may implement the process described in fig. 5 in the embodiment of the present application, and the apparatus includes: an obtaining module 1410, a determining module 1420, and a sending module 1430, wherein:
an obtaining module 1410, configured to obtain a suspected abnormal KPI detection model, which may be specifically used to implement the obtaining function of step 501 and the implicit steps included in step 501;
a determining module 1420, configured to determine, through the suspected abnormal KPI detection model, suspected abnormal KPI data in KPI data, which may be specifically used to implement the determining function in step 502 and the implicit step included in step 502;
a sending module 1430, configured to report the suspected abnormal KPI data to an analysis device, where the suspected abnormal KPI data is used to update an abnormal KPI detection model, and specifically may be used to implement the sending function of step 503 and the implicit step included in step 503.
In a possible implementation manner, the obtaining module 1410 is further configured to obtain the abnormal KPI detection model;
the determining module 1420 is further configured to determine abnormal KPI data through the abnormal KPI detection model;
the sending module 1430 is further configured to report the abnormal KPI data to the analysis device, where the abnormal KPI data is used to determine whether the abnormal KPI detection model meets the detection performance requirement.
In a possible implementation manner, the determining module 1420 is configured to:
and through the suspected abnormal KPI detection model, carrying out one or more of the following treatments on each KPI time sequence in KPI data: filtering KPI time series with KPI at all moments lower than a first value, filtering KPI time series with KPI sudden increase smaller than a second value at any moment, filtering KPI time series with KPI having periodicity, or filtering KPI time series in steady state;
and performing abnormal sudden increase detection on the KPI time sequences remained in the KPI data through the suspected abnormal KPI detection model, and determining suspected abnormal KPI data in the KPI data.
The division of the modules in the embodiments of the present application is illustrative, and only one logical function division is provided, and in actual implementation, there may be another division manner, and in addition, each functional module in each embodiment of the present application may be integrated in one processor, may also exist alone physically, or two or more modules are integrated into one module. The integrated module can be realized in a hardware mode, and can also be realized in a software functional module mode.
In the above embodiments, all or part of the implementation may be realized by software, hardware, firmware or any combination thereof, and when the implementation is realized by software, all or part of the implementation may be realized in the form of a computer program product. The computer program product comprises one or more computer program instructions which, when loaded and executed on a server or terminal, cause the processes or functions described in accordance with embodiments of the application to be performed, in whole or in part. The computer instructions may be stored in a computer readable storage medium or transmitted from one computer readable storage medium to another, for example, the computer instructions may be transmitted from one website, computer, server, or data center to another website, computer, server, or data center by wire (e.g., coaxial cable, fiber optics, digital subscriber line) or wirelessly (e.g., infrared, wireless, microwave, etc.). The computer readable storage medium can be any available medium that can be accessed by a server or a terminal or a data storage device, such as a server, a data center, etc., that includes one or more of the available media. The usable medium may be a magnetic medium (such as a floppy Disk, a hard Disk, a magnetic tape, etc.), an optical medium (such as a Digital Video Disk (DVD), etc.), or a semiconductor medium (such as a solid state Disk, etc.).

Claims (40)

1. A method for implementing anomaly detection model updates, the method comprising:
obtaining suspected abnormal KPI data determined by a current suspected abnormal key performance index KPI detection model used by detection equipment;
updating the current abnormal KPI sample base according to the suspected abnormal KPI data;
updating the current abnormal KPI detection model according to the updated abnormal KPI sample library;
the determining suspected abnormal KPI data using the current suspected abnormal KPI detection model includes:
for any KPI time sequence in KPI data, when the KPI time sequence meets normal distribution, or when the KPI time sequence does not meet normal distribution and the KPI time sequence does not meet filtering conditions, performing abnormal sudden increase detection on the KPI time sequence to obtain suspected abnormal KPI data in the KPI time sequence;
wherein the filtration conditions include one or more of: the KPI at all time instants in the KPI time series is below a first value, the KPI surge at all time instants in the KPI time series is below a second value, the KPI time series is periodic or the KPI time series is in a steady state.
2. The method of claim 1, wherein the recall rate of the suspected abnormal KPI detection model is higher than the recall rate of the abnormal KPI detection model, or wherein the false-positive rate of the suspected abnormal KPI detection model is lower than the false-positive rate of the abnormal KPI detection model.
3. The method according to claim 1 or 2, characterized in that the accuracy of the abnormal KPI detection model is higher than the accuracy of the suspected abnormal KPI detection model, or the false alarm rate of the abnormal KPI detection model is lower than the false alarm rate of the suspected abnormal KPI detection model.
4. The method according to claim 1 or 2, wherein the updating of the current abnormal KPI sample according to the suspected abnormal KPI data comprises:
acquiring KPI data confirmed to be abnormal in the suspected abnormal KPI data;
and taking the KPI data confirmed to be abnormal as abnormal KPI samples, and adding the abnormal KPI samples to the current abnormal KPI sample library.
5. The method of claim 4, wherein said obtaining KPI data that are identified as abnormal in said suspected abnormal KPI data comprises:
sending the suspected abnormal KPI data to a management device, and receiving KPI data confirmed to be abnormal in the suspected abnormal KPI data fed back by the management device; alternatively, the first and second electrodes may be,
displaying the suspected abnormal KPI data, and acquiring KPI data confirmed to be abnormal in the displayed suspected abnormal KPI data.
6. The method according to claim 1 or 2, wherein before updating the current abnormal KPI detection model according to the updated abnormal KPI sample library, the method further comprises:
and determining that the current abnormal KPI detection model does not meet the detection performance requirement according to the updated abnormal KPI sample library.
7. The method according to claim 6, wherein before determining that the current abnormal KPI detection model does not meet detection performance requirements according to the updated abnormal KPI sample library, further comprising:
acquiring abnormal KPI data determined by the detection equipment by using the abnormal KPI detection model;
the determining that the current abnormal KPI detection model does not meet the detection performance requirements according to the updated abnormal KPI sample library comprises:
and determining that the abnormal KPI detection model does not meet the detection performance requirement according to the abnormal KPI data and the updated abnormal KPI sample library.
8. The method of claim 7, wherein determining that the abnormal KPI detection model does not meet detection performance requirements based on the abnormal KPI data and the updated abnormal KPI sample library comprises:
determining a performance index of the abnormal KPI detection model according to the abnormal KPI data and the updated abnormal KPI sample library, wherein the performance index comprises at least one of a missing report rate, a false report rate, a recall rate and an accuracy rate;
and determining that the abnormal KPI detection model does not meet the detection performance requirement according to the performance index.
9. The method according to claim 1 or 2, wherein the suspected abnormal KPI detection model is an unsupervised learning model.
10. The method according to claim 1 or 2, wherein the obtaining suspected abnormal KPI data determined by the detection device using the current suspected abnormal KPI detection model further comprises:
training to obtain the suspected abnormal KPI detection model according to a first preset performance requirement, wherein the suspected abnormal KPI detection model is an initial suspected abnormal KPI detection model;
and providing the suspected abnormal KPI detection model to the detection equipment.
11. The method according to claim 10, wherein before training to obtain the suspected abnormal KPI detection model according to the first preset performance requirement, further comprising:
obtaining target suspected abnormal KPI data in the initial KPI data;
the training to obtain the suspected abnormal KPI detection model according to the first preset performance requirement comprises:
acquiring KPI data confirmed to be abnormal in the target suspected abnormal KPI data;
and training to obtain an initial suspected abnormal KPI detection model according to a first preset performance requirement and KPI data confirmed to be abnormal in the target suspected abnormal KPI data.
12. The method of claim 11, wherein obtaining target suspected abnormal KPI data in the initial KPI data comprises:
determining target suspected abnormal KPI data in the initial KPI data according to a preset unsupervised abnormal KPI detection model;
the training to obtain an initial suspected abnormal KPI detection model according to a first preset performance requirement and KPI data confirmed to be abnormal in the target suspected abnormal KPI data comprises:
and updating the unsupervised abnormal KPI detection model according to a first preset performance requirement and KPI data confirmed to be abnormal in the target suspected abnormal KPI data to obtain an initial suspected abnormal KPI detection model.
13. The method according to claim 1 or 2, wherein after updating the current abnormal KPI sample base according to the suspected abnormal KPI data, the method further comprises:
updating the suspected abnormal KPI detection model according to the updated abnormal KPI sample library;
and providing the updated suspected abnormal KPI detection model for the detection equipment.
14. The method according to claim 1 or 2, wherein the abnormal KPI detection model is a supervised learning model.
15. The method according to claim 1 or 2, wherein before updating the current abnormal KPI detection model according to the updated abnormal KPI sample library, the method further comprises:
training to obtain the current abnormal KPI detection model according to a second preset performance requirement, wherein the current abnormal KPI detection model is an initial abnormal KPI detection model;
and providing the current abnormal KPI detection model for the detection equipment.
16. The method according to claim 1 or 2, characterized in that the method further comprises:
and providing the updated abnormal KPI detection model for the detection equipment.
17. The method of claim 1 or 2, wherein the suspected abnormal KPI data comprises a suspected abnormal KPI time series.
18. The method of claim 17, wherein the suspected abnormal KPI data further comprises an abnormal time period and/or an abnormal time point in the suspected abnormal KPI time series.
19. A method for implementing anomaly detection model updates, the method comprising:
obtaining a suspected abnormal key performance index KPI detection model;
determining suspected abnormal KPI data in KPI data through the suspected abnormal key performance index KPI detection model;
reporting the suspected abnormal KPI data to an analysis device, wherein the suspected abnormal KPI data is used for updating an abnormal KPI detection model;
the step of determining suspected abnormal KPI data in KPI data through the suspected abnormal KPI detection model comprises the following steps:
for any KPI time sequence in the KPI data, when the KPI time sequence meets normal distribution, or when the KPI time sequence does not meet normal distribution and the KPI time sequence does not meet filtering conditions, performing abnormal sudden increase detection on the KPI time sequence to obtain suspected abnormal KPI data in the KPI time sequence;
wherein the filtration conditions include one or more of: the KPI at all time instants in the KPI time series is below a first value, the KPI surge at all time instants in the KPI time series is below a second value, the KPI time series is periodic or the KPI time series is in a steady state.
20. The method of claim 19, further comprising:
acquiring the abnormal KPI detection model;
determining abnormal KPI data through the abnormal KPI detection model;
and reporting the abnormal KPI data to the analysis equipment, wherein the abnormal KPI data is used for judging whether the abnormal KPI detection model meets the detection performance requirements.
21. An apparatus for implementing anomaly detection model updates, the apparatus comprising:
the acquisition module is used for acquiring suspected abnormal KPI data determined by the detection equipment by using a current suspected abnormal key performance index KPI detection model;
an update module to:
updating the current abnormal KPI sample base according to the suspected abnormal KPI data;
updating the current abnormal KPI detection model according to the updated abnormal KPI sample library;
the determining suspected abnormal KPI data using the current suspected abnormal KPI detection model includes:
for any KPI time sequence in KPI data, when the KPI time sequence meets normal distribution, or when the KPI time sequence does not meet normal distribution and the KPI time sequence does not meet filtering conditions, performing abnormal sudden increase detection on the KPI time sequence to obtain suspected abnormal KPI data in the KPI time sequence;
wherein the filtration conditions include one or more of: the KPI at all time instants in the KPI time series is below a first value, the KPI surge at all time instants in the KPI time series is below a second value, the KPI time series is periodic or the KPI time series is in a steady state.
22. The apparatus of claim 21, wherein a recall rate of the suspected abnormal KPI detection model is higher than a recall rate of the abnormal KPI detection model, or wherein a false-positive rate of the suspected abnormal KPI detection model is lower than a false-positive rate of the abnormal KPI detection model.
23. The apparatus according to claim 21 or 22, wherein the accuracy of the abnormal KPI detection model is higher than the accuracy of the suspected abnormal KPI detection model, or the false alarm rate of the abnormal KPI detection model is lower than the false alarm rate of the suspected abnormal KPI detection model.
24. The apparatus of claim 21 or 22, wherein the update module is configured to:
acquiring KPI data confirmed to be abnormal in the suspected abnormal KPI data;
and taking the KPI data confirmed to be abnormal as abnormal KPI samples, and adding the abnormal KPI samples to the current abnormal KPI sample library.
25. The apparatus of claim 24, wherein the update module is configured to:
sending the suspected abnormal KPI data to a management device, and receiving KPI data confirmed to be abnormal in the suspected abnormal KPI data fed back by the management device; alternatively, the first and second electrodes may be,
displaying the suspected abnormal KPI data, and acquiring KPI data confirmed to be abnormal in the displayed suspected abnormal KPI data.
26. The apparatus of claim 21 or 22, wherein the update module is further configured to:
and according to the updated abnormal KPI sample library, before updating the current abnormal KPI detection model, determining that the current abnormal KPI detection model does not meet the detection performance requirement according to the updated abnormal KPI sample library.
27. The apparatus of claim 26, wherein the update module is further configured to:
according to the updated abnormal KPI sample library, acquiring abnormal KPI data determined by the detection equipment by using the abnormal KPI detection model before determining that the current abnormal KPI detection model does not meet the detection performance requirement;
the update module is configured to:
and determining that the abnormal KPI detection model does not meet the detection performance requirement according to the abnormal KPI data and the updated abnormal KPI sample library.
28. The apparatus of claim 27, wherein the update module is configured to:
determining a performance index of the abnormal KPI detection model according to the abnormal KPI data and the updated abnormal KPI sample library, wherein the performance index comprises at least one of a missing report rate, a false report rate, a recall rate and an accuracy rate;
and determining that the abnormal KPI detection model does not meet the detection performance requirement according to the performance index.
29. The apparatus according to claim 21 or 22, wherein the suspected abnormal KPI detection model is an unsupervised learning model and the abnormal KPI detection model is a supervised learning model.
30. The apparatus of claim 21 or 22, further comprising:
the training module is used for training to obtain a suspected abnormal KPI detection model according to a first preset performance requirement before suspected abnormal KPI data determined by a current suspected abnormal KPI detection model used by detection equipment is obtained, wherein the suspected abnormal KPI detection model is an initial suspected abnormal KPI detection model;
and the sending module is used for providing the suspected abnormal KPI detection model for the detection equipment.
31. The apparatus of claim 30, wherein the obtaining module is further configured to:
according to a first preset performance requirement, before the suspected abnormal KPI detection model is obtained through training, target suspected abnormal KPI data in initial KPI data are obtained;
the training module is configured to:
acquiring KPI data confirmed to be abnormal in the target suspected abnormal KPI data;
and training to obtain an initial suspected abnormal KPI detection model according to a first preset performance requirement and KPI data confirmed to be abnormal in the target suspected abnormal KPI data.
32. The apparatus of claim 31, wherein the obtaining module is further configured to:
determining target suspected abnormal KPI data in the initial KPI data according to a preset unsupervised abnormal KPI detection model;
the training module is further configured to:
and updating the unsupervised abnormal KPI detection model according to a first preset performance requirement and KPI data confirmed to be abnormal in the target suspected abnormal KPI data to obtain an initial suspected abnormal KPI detection model.
33. The apparatus of claim 21 or 22, wherein the update module is further configured to:
updating a current abnormal KPI sample base according to the suspected abnormal KPI data, and then updating the suspected abnormal KPI detection model according to the updated abnormal KPI sample base;
the device further comprises a sending module, configured to provide the updated suspected abnormal KPI detection model to the detection device.
34. The apparatus of claim 21 or 22, further comprising:
the training module is used for training to obtain a current abnormal KPI detection model according to a second preset performance requirement before updating the current abnormal KPI detection model according to the updated abnormal KPI sample library, wherein the current abnormal KPI detection model is an initial abnormal KPI detection model;
and the sending module is used for providing the current abnormal KPI detection model for the detection equipment.
35. The apparatus of claim 21 or 22, further comprising:
and the sending module is used for providing the updated abnormal KPI detection model for the detection equipment.
36. The apparatus of claim 21 or 22, wherein the suspected abnormal KPI data comprises a suspected abnormal KPI time series.
37. The apparatus of claim 36, wherein the suspected abnormal KPI data further comprises an abnormal time period and/or an abnormal time point in the suspected abnormal KPI time series.
38. An apparatus for implementing anomaly detection model updates, the apparatus comprising:
the acquisition module is used for acquiring a suspected abnormal key performance indicator KPI detection model;
the determining module is used for determining suspected abnormal KPI data in KPI data through the suspected abnormal key performance index KPI detection model;
a sending module, configured to report the suspected abnormal KPI data to an analysis device, where the suspected abnormal KPI data is used to update an abnormal KPI detection model;
determining suspected abnormal KPI data in KPI data through the suspected abnormal KPI detection model, including:
for any KPI time sequence in the KPI data, when the KPI time sequence meets normal distribution, or when the KPI time sequence does not meet normal distribution and the KPI time sequence does not meet filtering conditions, performing abnormal sudden increase detection on the KPI time sequence to obtain suspected abnormal KPI data in the KPI time sequence;
wherein the filtration conditions include one or more of: the KPI at all the time instants in the KPI time series is lower than a first value, the KPI sudden increase amount at all the time instants in the KPI time series is lower than a second value, the KPI time series has periodicity, or the KPI time series is in a steady state.
39. The apparatus according to claim 38, wherein said obtaining module is further configured to obtain said abnormal KPI detection model;
the determining module is further used for determining abnormal KPI data through the abnormal KPI detection model;
the sending module is further configured to report the abnormal KPI data to the analysis device, where the abnormal KPI data is used to determine whether the abnormal KPI detection model meets the detection performance requirement.
40. A computer-readable storage medium storing computer instructions which, when executed by a computing device, cause the computing device to perform the method of any of claims 1-20 or to implement the functionality of the apparatus of any of claims 21-39.
CN202010477620.XA 2020-05-29 2020-05-29 Method and device for updating anomaly detection model and computing equipment Active CN113746688B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202010477620.XA CN113746688B (en) 2020-05-29 2020-05-29 Method and device for updating anomaly detection model and computing equipment

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202010477620.XA CN113746688B (en) 2020-05-29 2020-05-29 Method and device for updating anomaly detection model and computing equipment

Publications (2)

Publication Number Publication Date
CN113746688A CN113746688A (en) 2021-12-03
CN113746688B true CN113746688B (en) 2023-02-28

Family

ID=78724831

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202010477620.XA Active CN113746688B (en) 2020-05-29 2020-05-29 Method and device for updating anomaly detection model and computing equipment

Country Status (1)

Country Link
CN (1) CN113746688B (en)

Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN109032829A (en) * 2018-07-23 2018-12-18 腾讯科技（深圳）有限公司 Data exception detection method, device, computer equipment and storage medium
CN110362612A (en) * 2019-07-19 2019-10-22 中国工商银行股份有限公司 Abnormal deviation data examination method, device and the electronic equipment executed by electronic equipment
CN111143102A (en) * 2019-12-13 2020-05-12 东软集团股份有限公司 Abnormal data detection method and device, storage medium and electronic equipment

Family Cites Families (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US9547719B2 (en) * 2014-06-04 2017-01-17 Netscout Systems Texas, Llc Bandwidth efficient processing and filtering across distributed databases

Patent Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN109032829A (en) * 2018-07-23 2018-12-18 腾讯科技（深圳）有限公司 Data exception detection method, device, computer equipment and storage medium
CN110362612A (en) * 2019-07-19 2019-10-22 中国工商银行股份有限公司 Abnormal deviation data examination method, device and the electronic equipment executed by electronic equipment
CN111143102A (en) * 2019-12-13 2020-05-12 东软集团股份有限公司 Abnormal data detection method and device, storage medium and electronic equipment

Also Published As

Publication number Publication date
CN113746688A (en) 2021-12-03

Similar Documents

Publication Publication Date Title
CN111126824B (en) Multi-index correlation model training method and multi-index anomaly analysis method
US6973415B1 (en) System and method for monitoring and modeling system performance
CN110169016A (en) Handle method, control node, network element and the system of network event in telecommunication network
CN103081403A (en) Method and apparatus for analysis of the operation of a communication system using events
CN107094086A (en) A kind of information acquisition method and device
CN112543465A (en) Abnormity detection method, abnormity detection device, terminal and storage medium
CN112188534A (en) Anomaly detection method and device
CN113037562A (en) Gateway fault assessment method and device and server
CN101345656B (en) global fault rate measuring method
CN113746688B (en) Method and device for updating anomaly detection model and computing equipment
CN110262955B (en) Application performance monitoring tool based on pinpoint
CN110609761B (en) Method and device for determining fault source, storage medium and electronic equipment
CN105429825B (en) Monitoring method and device for data flow system
EP4199449A1 (en) Automatic classification of correlated anomalies from a network through interpretable clustering
CN113835961B (en) Alarm information monitoring method, device, server and storage medium
CN116232851A (en) Early warning method and device for network abnormality, electronic equipment and storage medium
CN109450103A (en) Condition detection method, device and the intelligent terminal of pressing plate
CN116522213A (en) Service state level classification and classification model training method and electronic equipment
CN113300914A (en) Network quality monitoring method, device, system, electronic equipment and storage medium
JP2000151605A (en) Communication network management support system
CN112650644A (en) Monitoring method and system based on prometheus
CN107678905B (en) Monitoring method and device
CN114579347B (en) Page abnormity detection method and device, computer equipment and readable storage medium
CN116628508B (en) Model training process anomaly detection method, device, equipment and storage medium
CN117395198B (en) Congestion alarm method and system for power communication network

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant