CN113822421A - Neural network based anomaly positioning method, system, equipment and storage medium - Google Patents

Neural network based anomaly positioning method, system, equipment and storage medium Download PDF

Info

Publication number
CN113822421A
CN113822421A CN202111197387.0A CN202111197387A CN113822421A CN 113822421 A CN113822421 A CN 113822421A CN 202111197387 A CN202111197387 A CN 202111197387A CN 113822421 A CN113822421 A CN 113822421A
Authority
CN
China
Prior art keywords
neural network
trained
model
historical
fault
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Granted
Application number
CN202111197387.0A
Other languages
Chinese (zh)
Other versions
CN113822421B (en
Inventor
吴毓霖
代本辉
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Ping An Technology Shenzhen Co Ltd
Original Assignee
Ping An Technology Shenzhen Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Ping An Technology Shenzhen Co Ltd filed Critical Ping An Technology Shenzhen Co Ltd
Priority to CN202111197387.0A priority Critical patent/CN113822421B/en
Publication of CN113822421A publication Critical patent/CN113822421A/en
Application granted granted Critical
Publication of CN113822421B publication Critical patent/CN113822421B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06NCOMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00Computing arrangements based on biological models
    • G06N3/02Neural networks
    • G06N3/04Architecture, e.g. interconnection topology
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00Pattern recognition
    • G06F18/20Analysing
    • G06F18/21Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
    • G06F18/214Generating training patterns; Bootstrap methods, e.g. bagging or boosting
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06NCOMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00Computing arrangements based on biological models
    • G06N3/02Neural networks
    • G06N3/08Learning methods
    • G06N3/084Backpropagation, e.g. using gradient descent

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • Data Mining & Analysis (AREA)
  • Evolutionary Computation (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Artificial Intelligence (AREA)
  • General Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • General Health & Medical Sciences (AREA)
  • Software Systems (AREA)
  • Molecular Biology (AREA)
  • Computing Systems (AREA)
  • Biophysics (AREA)
  • Biomedical Technology (AREA)
  • Mathematical Physics (AREA)
  • Computational Linguistics (AREA)
  • Health & Medical Sciences (AREA)
  • Bioinformatics & Cheminformatics (AREA)
  • Bioinformatics & Computational Biology (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Evolutionary Biology (AREA)
  • Testing And Monitoring For Control Systems (AREA)

Abstract

The invention relates to the technical field of artificial intelligence, and provides an anomaly positioning method, system, equipment and storage medium based on a neural network, wherein the method comprises the following steps: acquiring current fault alarm information preprocessed by a target service system; acquiring current key characteristics according to the preprocessed current fault warning information and the trained restricted Boltzmann machine model; acquiring risk prediction grade and abnormal position information according to the current key characteristics and the trained fault prediction neural network model; and pre-alarming the target service system according to the risk prediction grade and the abnormal position information. In the embodiment of the invention, the key characteristics in the fault alarm information are extracted by the limited Boltzmann machine, and the key factors influencing the abnormity of the target service system can be obtained by performing probability processing on the fault alarm factors, so that the subsequent abnormity positioning precision of the target service system is improved.

Description

Neural network based anomaly positioning method, system, equipment and storage medium
Technical Field
The invention relates to the technical field of artificial intelligence, in particular to an anomaly positioning method, system, equipment and storage medium based on a neural network.
Background
In the big data fields of banking industry, insurance industry, e-commerce and the like, various application systems generate massive transaction data every day, high reliability of important service systems is kept, and rapid positioning and repairing are needed for abnormal conditions of the systems, so that major loss caused by failure of the service systems is avoided.
The conventional fault warning or risk early warning mechanism, for example, a method for realizing system early warning by methods such as an artificial neural network and a fuzzy algorithm, realizes timely response and processing of abnormality by judging a custom rule or strategy and feeding back the abnormality to operation and maintenance personnel, and by manually judging the abnormality. Although the methods can help to diagnose uncertain fault information of application services, the service system is usually large in scale and has more factors influencing system stability, so that the positioning accuracy is not high, the situation of false alarm often occurs, and operation and maintenance personnel sometimes have to face the situation of false alarm.
Therefore, how to realize effective early warning of the service system fault is one of the main problems in the operation and maintenance process.
Disclosure of Invention
The invention provides an anomaly positioning method based on a neural network, which mainly aims to carry out risk grade division on fault alarm and effectively improve the precision of fault pre-alarm.
In a first aspect, an embodiment of the present invention provides an anomaly positioning method based on a neural network, including:
acquiring current fault alarm information preprocessed by a target service system;
acquiring current key characteristics according to the preprocessed current fault warning information and the trained restricted Boltzmann machine model;
acquiring risk prediction grade and abnormal position information according to the current key characteristics and the trained fault prediction neural network model;
and pre-alarming the target service system according to the risk prediction grade and the abnormal position information.
Preferably, the trained restricted boltzmann model is obtained by:
acquiring a preset number of historical fault alarm information training sets, wherein each historical fault alarm information training set comprises a plurality of historical fault alarm information corresponding to the same historical abnormal position information;
training each initial limited Boltzmann machine by using each historical fault warning information training set to obtain a preset number of trained initial limited Boltzmann machines;
and cascading a preset number of trained initial limited Boltzmann machines to obtain a trained limited Boltzmann machine model.
Preferably, the trained failure prediction neural network model is obtained by:
inputting each historical fault alarm information training set into the trained restricted Boltzmann machine model, and acquiring a historical key feature corresponding to each historical fault alarm information;
and training the initial fault prediction neural network model according to the historical key features corresponding to each historical fault alarm information, the preset historical risk level corresponding to each historical fault alarm information and the historical abnormal position information corresponding to each historical fault alarm information, and acquiring the trained fault prediction neural network model.
Preferably, the initial restricted boltzmann machine is a two-tiered restricted boltzmann machine.
Preferably, the current key feature is obtained according to the preprocessed current fault warning information and the trained restricted boltzmann model, and a specific calculation formula is as follows:
ftr=[ftr1,…,ftrk,…,ftrn],
Figure BDA0003303574770000021
wherein ftr represents the current key feature, n represents the preset number, ftrkRepresenting the current key features extracted by the trained initial limited Boltzmann machine corresponding to the kth class,
Figure BDA0003303574770000022
for the interaction items between the hidden layer units and the visible units in the first-layer restricted boltzmann model corresponding to the kth category,
Figure BDA0003303574770000023
for the k-th class corresponding to the deviations of the visible layer in the first layer-constrained boltzmann model,
Figure BDA0003303574770000024
corresponding to the hidden layer deviation in the first layer restricted boltzmann model for the kth category,
Figure BDA0003303574770000025
for the interaction items between the hidden layer units and the visible units in the first-layer restricted boltzmann model corresponding to the kth category,
Figure BDA0003303574770000031
for the k-th class corresponding to the deviations of the visible layer in the second layer of the restricted boltzmann model,
Figure BDA0003303574770000032
and sigma (x) is a sigmoid activation function, and the k-th class corresponds to the deviation of the hidden layer in the second-layer restricted Boltzmann machine model.
Preferably, the acquiring current fault alarm information preprocessed by the target service system specifically includes:
acquiring monitoring index data of the target service system in real time, wherein the monitoring index data comprises basic resource monitoring data, application performance monitoring data and network safety detection data;
carrying out abnormal value processing, missing value processing, unified formatting and desensitization pretreatment on the monitoring index data to obtain pretreated monitoring index data;
and carrying out abnormal value detection on the preprocessed monitoring index data to acquire preprocessed current fault warning information.
Preferably, the outlier processing includes one or more of a fixed threshold method, a dynamic threshold method, and an index data prediction method.
In a second aspect, an embodiment of the present invention provides an anomaly locating system based on a neural network, including:
the preprocessing module is used for acquiring current fault warning information preprocessed by the target service system;
the feature extraction module is used for acquiring current key features according to the preprocessed current fault warning information and the trained restricted Boltzmann machine model;
the prediction module is used for acquiring risk prediction grade and abnormal position information according to the current key characteristics and the trained fault prediction neural network model;
and the warning module is used for pre-warning the target service system according to the risk prediction grade and the abnormal position information.
In a third aspect, an embodiment of the present invention provides a computer device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, where the processor implements the steps of the above-mentioned neural network-based anomaly locating method when executing the computer program.
In a fourth aspect, an embodiment of the present invention provides a computer storage medium, where a computer program is stored, and when the computer program is executed by a processor, the computer program implements the steps of the above-mentioned neural network-based anomaly locating method.
According to the abnormity positioning method, system, equipment and storage medium based on the neural network, due to the fact that a plurality of factors influencing the abnormity of the target business system are provided, the precision of abnormity positioning according to the monitored current fault alarm information in the subsequent positioning process is not high.
And according to the extracted key characteristics, combining with a fault prediction neural network model, obtaining the risk prediction grade of the current fault warning information so as to realize the risk grade division of the fault of the target service system, carrying out abnormal positioning, and carrying out division through risk probability, thereby improving the fault prediction warning precision, providing a reference basis for uncertain current fault warning information diagnosis, facilitating operation and maintenance personnel to determine the abnormal emergency degree of the target service system according to the risk grade, and conveniently allocating reasonable time and manpower for subsequent fault treatment.
Drawings
Fig. 1 is an application scenario diagram of an anomaly positioning method based on a neural network according to an embodiment of the present invention;
fig. 2 is a flowchart of an anomaly locating method based on a neural network according to an embodiment of the present invention;
fig. 3 is a schematic structural diagram of an anomaly locating system based on a neural network according to an embodiment of the present invention;
fig. 4 is a schematic structural diagram of a computer device provided in this embodiment.
The implementation, functional features and advantages of the objects of the present invention will be further explained with reference to the accompanying drawings.
Detailed Description
It should be understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention.
Fig. 1 is an application scenario diagram of an abnormal location method based on a neural network according to an embodiment of the present invention, as shown in fig. 1, a target service system is monitored in real time by corresponding monitoring software in a client, monitoring information is extracted, if the target service system fails, current failure alarm information of the target service system is collected and sent to a server, and after receiving the current failure alarm information, the server executes the abnormal location method based on the neural network, thereby implementing a pre-alarm for the target service system.
It should be noted that the server may be implemented by an independent server or a server cluster composed of a plurality of servers, or may be a cloud server that provides basic cloud computing services such as cloud service, a cloud database, cloud computing, a cloud function, cloud storage, network service, cloud communication, middleware service, domain name service, security service, content distribution network, and a big data and artificial intelligence platform. The client may be, but is not limited to, a smart phone, a tablet computer, a notebook computer, a desktop computer, and the like. The client and the server may be connected through bluetooth, USB (Universal Serial Bus), or other communication connection manners, which is not limited in this embodiment of the present invention.
Fig. 2 is a flowchart of an anomaly locating method based on a neural network according to an embodiment of the present invention, and as shown in fig. 2, the method includes:
s210, acquiring current fault warning information preprocessed by a target service system;
in the embodiment of the invention, the target business system is a system which needs fault monitoring and abnormity positioning, can be a bank system, an insurance system, an e-commerce system and the like, and is selected according to actual conditions.
The method comprises the steps of firstly collecting monitoring information of a target service system, wherein the monitoring information comprises MVC monitoring information, data information monitored by a system running log cloud and the like, the MVC monitoring information can record the execution time of actions under each controller and the view rendering completion time, and log4net is adopted to record the execution time of each action of MVC and the view rendering completion time, and data sent or received when the action is requested. The system log is used for recording the information of hardware, software and problems of the target service system and monitoring events in the system. Through which the user can check the cause of the error or look for traces left by the attacker when under attack. The system log includes a system log, an application log, and a security log.
Some abnormal indexes are filtered from the collected monitoring information, for example, normal ranges corresponding to all encountered indexes in a target service system are generally stored, and after real-time data of the indexes are collected, the real-time values of the indexes are compared with the normal ranges to judge whether the indexes are abnormal or not. For a certain index, the range of the normal operation of the index is generally within a value section, and if the real-time value of the index is greater than the maximum value of the value section or less than the minimum value of the value section, the index is abnormal. All indexes of the target service system are judged according to the method, and all indexes in abnormal states are filtered out to be used as current fault warning information of the target service system.
In the process of monitoring a target service system, some abnormal values, missing values and the like inevitably exist in the monitored current fault alarm information due to the interference of external environmental factors, the existence of noise, system jitter and the like, so that the current fault alarm information needs to be preprocessed first to remove some abnormal data in the current fault alarm information, and the current fault alarm information is subjected to standardized sample processing to form a matrix, so that the processed current fault alarm information meets the subsequent processing requirements.
S220, acquiring current key characteristics according to the preprocessed current fault warning information and the trained restricted Boltzmann machine model;
inputting the preprocessed current fault warning information into the trained restricted Boltzmann machine model, and extracting key indexes, namely current key characteristics, from all abnormal indexes of the current fault warning information after probability processing. In the embodiment of the invention, as a plurality of factors influencing the abnormity of the target service system are provided, the precision of abnormity positioning according to the monitored current fault alarm information in the subsequent positioning process is not high.
It should be noted that a Restricted Boltzmann Machine (RBM) is a special topology structure of a Boltzmann Machine (BM). The BM is a symmetrically coupled random feedback type binary unit neural network and consists of a visible layer and a plurality of hidden layers, network nodes are divided into visible units (visible units) and hidden units (hidden units), the visible units and the hidden units are used for expressing a learning model of the random network and a random environment, and the relevance between the units is expressed through weights.
As the name suggests, the constrained-botzmann machine is a variation of the botzmann machine, but the constrained model must be a bipartite graph. The model comprises an input (visible) unit corresponding to input parameters and a hidden unit corresponding to training results, and each edge must be connected with a visible unit and a hidden unit. This limitation makes training of the limited boltzmann machine more efficient than a general boltzmann machine.
The limited Bozmann machine is widely applied to dimensionality reduction, classification, collaborative filtering, feature learning and topic modeling, and can be trained by using a supervised learning or unsupervised learning method according to different tasks.
In the embodiment of the invention, the limited Boltzmann machine is trained in an unsupervised learning mode to obtain a trained limited Boltzmann machine model.
S230, acquiring risk prediction grade and abnormal position information according to the current key features and the trained fault prediction neural network model;
then inputting the current key features into a trained failure prediction neural network model to obtain risk prediction grades and abnormal position information of the target service system, wherein the risk grades represent the abnormal probability of the target service system, and are divided into different layers according to the probability, such as normal, medium and serious, and the like, and the classification can be specifically carried out according to actual needs; the anomaly location information may indicate a specific location of an anomaly in the target business system, such as a system, middleware, or database.
The risk grade is to grade the probability of occurrence of the fault according to a preset rule, the higher the risk grade is, the more dangerous the fault of the target service system is, the lower the risk grade is, the less dangerous the fault of the target service system is, and the degree of the risk grade of the fault of the target service system is quantified through the risk grade, so that the current emergency condition of the target service system can be obtained very conveniently, and the follow-up operation and maintenance personnel can conveniently handle the situation.
In addition, different abnormal indexes have different fault positions of the target service system, and some indexes can be only acquired at specific positions of the target service system, so that the fault positions of the target service system can be conveniently positioned according to the abnormal indexes, and the abnormal position information of the target service system can be obtained.
In the embodiment of the invention, the failure prediction neural network model belongs to one of neural networks, the failure prediction neural network also needs to be trained before being used, and the failure prediction neural network is trained through a pre-obtained sample and a label. The training process of the failure prediction neural network can be divided into three steps: defining the structure of a failure prediction neural network and an output result of forward propagation; defining a loss function and a back propagation optimization algorithm; finally, a session is generated and a back propagation optimization algorithm is run repeatedly on the training data.
The neuron is the minimum unit forming the neural network, one neuron can have a plurality of inputs and one output, and the input of each neuron can be the output of other neurons or the input of the whole neural network. The output of the neural network is the weighted sum of the inputs of all the neurons, the weight of different inputs is the neuron parameter, and the optimization process of the neural network is the process of optimizing the value of the neuron parameter.
The effect and optimization goal of the neural network are defined by a loss function, the loss function gives a calculation formula of the difference between the output result of the neural network and the real label, and supervised learning is a way of training the neural network, and the idea is that on a labeled data set of known answers, the result given by the neural network is as close as possible to the real answer (namely, the label). The training data is fitted by adjusting parameters in the neural network so that the neural network provides predictive power to unknown samples.
The back propagation algorithm realizes an iterative process, when each iteration starts, a part of training data is taken first, and the prediction result of the neural network is obtained through the forward propagation algorithm. Because the training data all have correct answers, the difference between the predicted result and the correct answer can be calculated. Based on the difference, the back propagation algorithm can correspondingly update the value of the neural network parameter, so that the neural network parameter is closer to the real answer.
After the training process is completed by the method, the trained failure prediction neural network can be used for application.
In the embodiment of the present invention, the failure prediction Neural network may specifically be a Convolutional Neural Network (CNN), a bp (back prediction) Neural network, and the like, and is specifically determined according to an actual situation.
S240, pre-alarming is carried out on the target service system according to the risk prediction grade and the abnormal position information.
And then pre-alarming the target service system according to the risk prediction grade and the abnormal position information, and specifically, feeding back the risk prediction grade and the abnormal position information of the target service system to related operation and maintenance personnel to realize the pre-alarming of the target service system.
In summary, according to the anomaly positioning method based on the neural network provided by the embodiment of the present invention, because there are many factors influencing the anomaly of the target service system, the accuracy of anomaly positioning according to the monitored current fault alarm information in the subsequent positioning process is not high.
And according to the extracted key characteristics, combining with a fault prediction neural network model, obtaining the risk prediction grade of the current fault warning information so as to realize the risk grade division of the fault of the target service system, carrying out abnormal positioning, and carrying out division through risk probability, thereby improving the fault prediction warning precision, providing a reference basis for uncertain current fault warning information diagnosis, facilitating operation and maintenance personnel to determine the abnormal emergency degree of the target service system according to the risk grade, and conveniently allocating reasonable time and manpower for subsequent fault treatment.
On the basis of the above embodiment, preferably, the trained restricted boltzmann model is obtained by:
acquiring a preset number of historical fault alarm information training sets, wherein each historical fault alarm information training set comprises a plurality of historical fault alarm information corresponding to the same historical abnormal position information;
training each initial limited Boltzmann machine by using each historical fault warning information training set to obtain a preset number of trained initial limited Boltzmann machines;
and cascading a preset number of trained initial limited Boltzmann machines to obtain a trained limited Boltzmann machine model.
Specifically, since the boltzmann machine model is also a neural network model in nature, similarly, before the boltzmann machine model is used, the boltzmann machine model needs to be trained first, and when the boltzmann machine model is trained, various historical fault alarm information occurring before the target service system needs to be collected in advance, the collection method and the collection process of the historical fault alarm information are the same as those of the current fault alarm information, and please refer to the above process in detail, which is not detailed herein.
After collecting enough historical fault alarm information, classifying the historical fault alarm information according to the position of the historical fault alarm information, and dividing the historical fault alarm information belonging to the same fault position into a training set, so that a plurality of categories of historical fault alarm information training sets can be obtained, and the abnormal position corresponding to each training set is different.
Assuming that C historical fault alarm information training sets exist in total, C identical initial limited Boltzmann machines are obtained, each historical fault alarm information training set corresponds to one initial limited Boltzmann machine, and the historical fault alarm information training sets are utilized to train the corresponding initial limited Boltzmann machines to obtain the trained initial limited Boltzmann machines. And C trained initial limited Boltzmann machines are cascaded to obtain a trained limited Boltzmann machine model.
In the embodiment of the invention, the fault alarm information of different abnormal positions is classified, and the fault alarm information of different abnormal positions trains different initial limited Boltzmann machines, so that the separate training mode can accelerate the training speed, the initial limited Boltzmann machines can be converged more quickly in the training process, and the trained parameters are more accurate. In specific application, all the trained initial limited Boltzmann machines are cascaded to obtain the trained limited Boltzmann machine model, and the training parameters of each trained initial limited Boltzmann machine are more targeted, so that the accuracy of feature extraction can be improved by the mode of singly training and then cascading.
On the basis of the foregoing embodiment, preferably, the trained failure prediction neural network model is obtained by:
inputting each historical fault alarm information training set into the trained restricted Boltzmann machine model, and acquiring a historical key feature corresponding to each historical fault alarm information;
and training the initial fault prediction neural network model according to the historical key features corresponding to each historical fault alarm information, the preset historical risk level corresponding to each historical fault alarm information and the historical abnormal position information corresponding to each historical fault alarm information, and acquiring the trained fault prediction neural network model.
In particular, the failure prediction neural network model is also one of the neural network models, and the failure prediction neural network model also needs to be trained before being used. Besides obtaining enough historical fault alarm information, a preset historical risk level corresponding to the historical fault alarm information and a historical abnormal position corresponding to the historical fault alarm information need to be obtained.
Inputting historical fault alarm information into a trained restricted Boltzmann machine model, extracting historical key features corresponding to the historical fault alarm information, taking the historical key features corresponding to the historical fault alarm information as input of an initial fault prediction neural network model, taking a preset historical risk level and a historical abnormal position corresponding to the historical fault alarm information as labels of the initial fault prediction neural network model, and training the initial fault prediction neural network model to obtain the trained fault prediction neural network model.
On the basis of the above embodiment, preferably, the initial limited boltzmann machine is a two-layer limited boltzmann machine.
Specifically, the number of layers of the initial limited boltzmann machine is 2. Correspondingly, for the historical fault alarm information of the C categories, a two-layer restricted Boltzmann machine model is adopted during training, and the C two-layer restricted Boltzmann machine model is obtained. Wherein the ith type of historical fault alarm information
Figure BDA0003303574770000111
The parameters of the restricted Boltzmann machine obtained by training are
Figure BDA0003303574770000112
And
Figure BDA0003303574770000113
wherein,
Figure BDA0003303574770000114
for the interactive item between the hidden unit and the visible unit in the first-layer limited Boltzmann machine model corresponding to the ith type of historical fault warning information,
Figure BDA0003303574770000115
for the ith type sampleThe deviation of a visible layer in a layer of the restricted boltzmann model,
Figure BDA0003303574770000116
the deviation of the hidden layer in the first layer limited Boltzmann machine model corresponding to the ith type sample,
Figure BDA0003303574770000117
for the interactive item between the hidden unit and the visible unit in the second-layer limited Boltzmann machine model corresponding to the ith type of historical fault warning information,
Figure BDA0003303574770000118
the deviations of the visible layers in the restricted boltzmann model of the second layer corresponding to the ith type sample,
Figure BDA0003303574770000119
and the deviation of the hidden layer in the second-layer limited Boltzmann machine model corresponding to the ith type sample is obtained.
Experiments prove that the two-layer restricted Boltzmann machine model can improve the extraction precision of key features, is compatible with the training efficiency of the model, and is the most appropriate model when the precision and the real-time performance are ensured.
On the basis of the foregoing embodiment, preferably, the current key feature is obtained according to the preprocessed current fault warning information and the trained limited boltzmann model, and a specific calculation formula is as follows:
ftr=[ftr1,…,ftrk,…,ftrn],
Figure BDA00033035747700001110
wherein ftr represents the current key feature, n represents the preset number, ftrkRepresenting the current key features extracted by the trained initial limited Boltzmann machine corresponding to the kth class,
Figure BDA00033035747700001111
for the interaction items between the hidden layer units and the visible units in the first-layer restricted boltzmann model corresponding to the kth category,
Figure BDA00033035747700001112
for the k-th class corresponding to the deviations of the visible layer in the first layer-constrained boltzmann model,
Figure BDA00033035747700001113
corresponding to the hidden layer deviation in the first layer restricted boltzmann model for the kth category,
Figure BDA00033035747700001114
for the interaction items between the hidden layer units and the visible units in the first-layer restricted boltzmann model corresponding to the kth category,
Figure BDA00033035747700001115
for the k-th class corresponding to the deviations of the visible layer in the second layer of the restricted boltzmann model,
Figure BDA00033035747700001116
and sigma (x) is a sigmoid activation function, and the k-th class corresponds to the deviation of the hidden layer in the second-layer restricted Boltzmann machine model.
On the basis of the foregoing embodiment, preferably, the acquiring current fault alarm information preprocessed by the target service system specifically includes:
acquiring monitoring index data of the target service system in real time, wherein the monitoring index data comprises basic resource monitoring data, application performance monitoring data and network safety detection data;
carrying out abnormal value processing, missing value processing, unified formatting and desensitization pretreatment on the monitoring index data to obtain pretreated monitoring index data;
and carrying out abnormal value detection on the preprocessed monitoring index data to acquire preprocessed current fault warning information.
Specifically, the monitoring index data of the target service system is collected in real time, where the monitoring index data includes basic resource monitoring data, application performance monitoring data, and network security monitoring data, the basic resource monitoring data refers to data related to software and hardware basic parameters of the target service system, such as target service system temperature, target service system memory size, target service system processor operation rate, and the like, the application performance monitoring data refers to application operation conditions of the target service system, taking e-commerce system as an example, such as throughput rate, abnormal burst frequency, and the like, and the network security monitoring data refers to data related to network security of the target service system, such as attacked frequency, success rate against attacks, and the like.
Abnormal value detection processing, missing value processing, uniform formatting processing, desensitization processing and the like are carried out on the monitoring index data, and the abnormal value processing comprises the following steps: and detecting an abnormal value of the acquired data, eliminating the abnormal value, and replacing the abnormal value by using the data mean value of two adjacent moments before and after the abnormal point. Missing value processing includes: and carrying out interpolation filling on the missing values to complete the data set. The unified formatting includes: carrying out unified formatting treatment on monitoring index data of different types and different dimensions; data desensitization: and encrypting the acquired data, and then performing network transmission and storage.
By preprocessing the current fault warning information, the abnormity prediction precision of a subsequent target service system can be improved.
In summary, the embodiments of the present invention provide an anomaly positioning method based on a neural network, in which, because the factors are subjected to probability processing, key factors influencing the anomaly of a target service system can be obtained, thereby improving the accuracy of subsequent anomaly positioning of the target service system.
According to the extracted key features, the risk grade division is carried out on the faults of the target service system by combining the fault prediction neural network model, the abnormity location is carried out, the fault prediction alarm precision is improved by dividing through the risk probability, a reference basis is provided for uncertain current fault alarm information diagnosis, and therefore operation and maintenance personnel can determine the abnormity emergency degree of the target service system according to the risk grade, and reasonable time and manpower can be conveniently distributed for follow-up fault processing.
In addition, in the training process of the initial limited Boltzmann machine, fault warning information of different abnormal positions is classified, and the fault warning information of different abnormal positions trains different initial limited Boltzmann machines. In specific application, all the trained initial limited Boltzmann machines are cascaded to obtain the trained limited Boltzmann machine model, and the training parameters of each trained initial limited Boltzmann machine are more targeted, so that the accuracy of feature extraction can be improved by the mode of singly training and then cascading.
And finally, preprocessing the monitoring data of the target service system to remove abnormal data and noise in the monitoring data, and improving the subsequent prediction precision.
Fig. 3 is a schematic structural diagram of an anomaly locating system based on a neural network according to an embodiment of the present invention, and as shown in fig. 3, the system includes a preprocessing module 310, a feature extraction module 320, a prediction module 330, and an alarm module 340, where:
the preprocessing module 310 is configured to obtain current fault warning information preprocessed by the target service system;
the feature extraction module 320 is configured to obtain a current key feature according to the preprocessed current fault warning information and the trained restricted boltzmann model;
the prediction module 330 is configured to obtain a risk prediction grade and abnormal position information according to the current key feature and the trained failure prediction neural network model;
the warning module 340 is configured to perform a pre-warning on the target service system according to the risk prediction level and the abnormal location information.
Specifically, in the feature extraction module 320, the trained restricted boltzmann model is obtained by the following steps:
acquiring a preset number of historical fault alarm information training sets, wherein each historical fault alarm information training set comprises a plurality of historical fault alarm information corresponding to the same historical abnormal position information;
training each initial limited Boltzmann machine by using each historical fault warning information training set to obtain a preset number of trained initial limited Boltzmann machines;
and cascading a preset number of trained initial limited Boltzmann machines to obtain a trained limited Boltzmann machine model.
Specifically, in the prediction module 330, the trained neural network model for fault prediction is obtained through the following steps:
inputting each historical fault alarm information training set into the trained restricted Boltzmann machine model, and acquiring a historical key feature corresponding to each historical fault alarm information;
and training the initial fault prediction neural network model according to the historical key features corresponding to each historical fault alarm information, the preset historical risk level corresponding to each historical fault alarm information and the historical abnormal position information corresponding to each historical fault alarm information, and acquiring the trained fault prediction neural network model.
Specifically, the initial restricted boltzmann machine is a two-tiered restricted boltzmann machine.
Specifically, in the feature extraction module 320, the current key feature is obtained according to the preprocessed current fault warning information and the trained restricted boltzmann model, and a specific calculation formula is as follows:
ftr=[ftr1,…,ftrk,…,ftrn],
Figure BDA0003303574770000141
wherein ftr represents the current key feature, n represents the preset number, ftrkRepresenting the current key features extracted by the trained initial limited Boltzmann machine corresponding to the kth class,
Figure BDA0003303574770000142
for the interaction items between the hidden layer units and the visible units in the first-layer restricted boltzmann model corresponding to the kth category,
Figure BDA0003303574770000143
for the k-th class corresponding to the deviations of the visible layer in the first layer-constrained boltzmann model,
Figure BDA0003303574770000144
corresponding to the hidden layer deviation in the first layer restricted boltzmann model for the kth category,
Figure BDA0003303574770000145
for the interaction items between the hidden layer units and the visible units in the first-layer restricted boltzmann model corresponding to the kth category,
Figure BDA0003303574770000146
for the k-th class corresponding to the deviations of the visible layer in the second layer of the restricted boltzmann model,
Figure BDA0003303574770000147
and sigma (x) is a sigmoid activation function, and the k-th class corresponds to the deviation of the hidden layer in the second-layer restricted Boltzmann machine model.
Specifically, the preprocessing module includes a monitoring unit, a preprocessing unit and a detecting unit, wherein:
the monitoring unit is used for acquiring monitoring index data of the target service system in real time, wherein the monitoring index data comprises basic resource monitoring data, application performance monitoring data and network safety detection data;
the preprocessing unit is used for carrying out abnormal value processing, missing value processing, unified formatting and desensitization preprocessing on the monitoring index data to obtain preprocessed monitoring index data;
the detection unit is used for detecting abnormal values of the preprocessed monitoring index data and acquiring preprocessed current fault alarm information.
Specifically, the abnormal value processing includes one or more of a fixed threshold method, a dynamic threshold method, and an index data prediction method.
The modules in the above-mentioned neural network based anomaly locating system can be wholly or partially implemented by software, hardware and a combination thereof. The modules can be embedded in a hardware form or independent from a processor in the computer device, and can also be stored in a memory in the computer device in a software form, so that the processor can call and execute operations corresponding to the modules.
The present embodiment is a system embodiment corresponding to the method, the specific implementation process is the same as the method embodiment, please refer to the method embodiment for details, and the system embodiment is not described herein again.
Fig. 4 is a schematic structural diagram of a computer device provided in this embodiment, where the computer device may be a server, and an internal structural diagram of the computer device may be as shown in fig. 4. The computer device includes a processor, a memory, a network interface, and a database connected by a system bus. Wherein the processor of the computer device is configured to provide computing and control capabilities. The memory of the computer device comprises a computer storage medium and an internal memory. The computer storage medium stores an operating system, a computer program, and a database. The internal memory provides an environment for the operation of an operating system and computer programs in the computer storage media. The database of the computer device is used for storing data generated or acquired during execution of a neural network-based anomaly locating method, such as current fault warning information, a trained restricted boltzmann model, a trained fault prediction neural network model, and the like. The network interface of the computer device is used for communicating with an external terminal through a network connection. The computer program is executed by a processor to realize an anomaly positioning method based on a neural network, and the method specifically comprises the following steps:
acquiring current fault alarm information preprocessed by a target service system;
acquiring current key characteristics according to the preprocessed current fault warning information and the trained restricted Boltzmann machine model;
acquiring risk prediction grade and abnormal position information according to the current key characteristics and the trained fault prediction neural network model;
and pre-alarming the target service system according to the risk prediction grade and the abnormal position information.
In one embodiment, a computer device is provided, which includes a memory, a processor, and a computer program stored on the memory and executable on the processor, and when the processor executes the computer program, the processor implements the steps of one of the above-mentioned embodiments of the method for neural network-based anomaly location. Alternatively, the processor, when executing the computer program, implements the functions of the modules/units in an embodiment of a neural network-based anomaly localization system.
In an embodiment, a computer storage medium is provided, on which a computer program is stored, which, when being executed by a processor, implements the steps of a neural network-based anomaly locating method in the above-described embodiments. Alternatively, the computer program may be adapted to perform the functions of the modules/units of the embodiment of a neural network based anomaly locating system described above when executed by a processor.
It will be understood by those skilled in the art that all or part of the processes of the methods of the embodiments described above can be implemented by hardware instructions of a computer program, which can be stored in a non-volatile computer-readable storage medium, and when executed, can include the processes of the embodiments of the methods described above. Any reference to memory, storage, database, or other medium used in the embodiments provided herein may include non-volatile and/or volatile memory, among others. Non-volatile memory can include read-only memory (ROM), Programmable ROM (PROM), Electrically Programmable ROM (EPROM), Electrically Erasable Programmable ROM (EEPROM), or flash memory. Volatile memory can include Random Access Memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in a variety of forms such as Static RAM (SRAM), Dynamic RAM (DRAM), Synchronous DRAM (SDRAM), Double Data Rate SDRAM (DDRSDRAM), Enhanced SDRAM (ESDRAM), Synchronous Link DRAM (SLDRAM), Rambus Direct RAM (RDRAM), direct bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM).
It will be apparent to those skilled in the art that, for convenience and brevity of description, only the above-mentioned division of the functional units and modules is illustrated, and in practical applications, the above-mentioned function distribution may be performed by different functional units and modules according to needs, that is, the internal structure of the apparatus is divided into different functional units or modules to perform all or part of the above-mentioned functions.
The above-mentioned embodiments are only used for illustrating the technical solutions of the present invention, and not for limiting the same; although the present invention has been described in detail with reference to the foregoing embodiments, it will be understood by those of ordinary skill in the art that: the technical solutions described in the foregoing embodiments may still be modified, or some technical features may be equivalently replaced; such modifications and substitutions do not substantially depart from the spirit and scope of the embodiments of the present invention, and are intended to be included within the scope of the present invention.

Claims (10)

1. An anomaly positioning method based on a neural network is characterized by comprising the following steps:
acquiring current fault alarm information preprocessed by a target service system;
acquiring current key characteristics according to the preprocessed current fault warning information and the trained restricted Boltzmann machine model;
acquiring risk prediction grade and abnormal position information according to the current key characteristics and the trained fault prediction neural network model;
and pre-alarming the target service system according to the risk prediction grade and the abnormal position information.
2. The neural network-based anomaly locating method according to claim 1, wherein the trained restricted boltzmann model is obtained by:
acquiring a preset number of historical fault alarm information training sets, wherein each historical fault alarm information training set comprises a plurality of historical fault alarm information corresponding to the same historical abnormal position information;
training each initial limited Boltzmann machine by using each historical fault warning information training set to obtain a preset number of trained initial limited Boltzmann machines;
and cascading a preset number of trained initial limited Boltzmann machines to obtain a trained limited Boltzmann machine model.
3. The neural network-based anomaly locating method according to claim 2, wherein the trained failure prediction neural network model is obtained by:
inputting each historical fault alarm information training set into the trained restricted Boltzmann machine model, and acquiring a historical key feature corresponding to each historical fault alarm information;
and training the initial fault prediction neural network model according to the historical key features corresponding to each historical fault alarm information, the preset historical risk level corresponding to each historical fault alarm information and the historical abnormal position information corresponding to each historical fault alarm information, and acquiring the trained fault prediction neural network model.
4. The neural network-based anomaly locating method according to claim 2, wherein said initial restricted boltzmann machine is a two-tier restricted boltzmann machine.
5. The method according to claim 4, wherein the current key features are obtained according to the preprocessed current fault warning information and the trained restricted boltzmann model, and a specific calculation formula is as follows:
ftr=[ftr1,…,ftrk,…,ftrn],
Figure FDA0003303574760000021
wherein ftr represents the current key feature, n represents the preset number, ftrkRepresenting the current key features extracted by the trained initial limited Boltzmann machine corresponding to the kth class,
Figure FDA0003303574760000022
for the interaction items between the hidden layer units and the visible units in the first-layer restricted boltzmann model corresponding to the kth category,
Figure FDA0003303574760000023
for the k-th class corresponding to the deviations of the visible layer in the first layer-constrained boltzmann model,
Figure FDA0003303574760000024
corresponding to the hidden layer deviation in the first layer restricted boltzmann model for the kth category,
Figure FDA0003303574760000025
for the interaction items between the hidden layer units and the visible units in the first-layer restricted boltzmann model corresponding to the kth category,
Figure FDA0003303574760000026
for the k-th class corresponding to the deviations of the visible layer in the second layer of the restricted boltzmann model,
Figure FDA0003303574760000027
and sigma (x) is a sigmoid activation function, and the k-th class corresponds to the deviation of the hidden layer in the second-layer restricted Boltzmann machine model.
6. The method for positioning an abnormality based on a neural network according to any one of claims 1 to 5, wherein the acquiring current fault warning information preprocessed by the target service system specifically includes:
acquiring monitoring index data of the target service system in real time, wherein the monitoring index data comprises basic resource monitoring data, application performance monitoring data and network safety detection data;
carrying out abnormal value processing, missing value processing, unified formatting and desensitization pretreatment on the monitoring index data to obtain pretreated monitoring index data;
and carrying out abnormal value detection on the preprocessed monitoring index data to acquire preprocessed current fault warning information.
7. The neural network-based anomaly locating method according to claim 6, wherein said outlier processing comprises one or more of fixed thresholding, dynamic thresholding and index data prediction.
8. An anomaly locating system based on a neural network, comprising:
the preprocessing module is used for acquiring current fault warning information preprocessed by the target service system;
the feature extraction module is used for acquiring current key features according to the preprocessed current fault warning information and the trained restricted Boltzmann machine model;
the prediction module is used for acquiring risk prediction grade and abnormal position information according to the current key characteristics and the trained fault prediction neural network model;
and the warning module is used for pre-warning the target service system according to the risk prediction grade and the abnormal position information.
9. A computer device comprising a memory, a processor and a computer program stored in the memory and executable on the processor, characterized in that the processor when executing the computer program implements the steps of the neural network based anomaly locating method according to any one of claims 1 to 7.
10. A computer storage medium storing a computer program, wherein the computer program, when executed by a processor, performs the steps of the neural network based anomaly locating method according to any one of claims 1 to 7.
CN202111197387.0A 2021-10-14 2021-10-14 Neural network-based anomaly locating method, system, equipment and storage medium Active CN113822421B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202111197387.0A CN113822421B (en) 2021-10-14 2021-10-14 Neural network-based anomaly locating method, system, equipment and storage medium

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202111197387.0A CN113822421B (en) 2021-10-14 2021-10-14 Neural network-based anomaly locating method, system, equipment and storage medium

Publications (2)

Publication Number Publication Date
CN113822421A true CN113822421A (en) 2021-12-21
CN113822421B CN113822421B (en) 2024-05-14

Family

ID=78916512

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202111197387.0A Active CN113822421B (en) 2021-10-14 2021-10-14 Neural network-based anomaly locating method, system, equipment and storage medium

Country Status (1)

Country Link
CN (1) CN113822421B (en)

Cited By (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN114363151A (en) * 2022-01-07 2022-04-15 北京金山云网络技术有限公司 Fault detection method and device, electronic equipment and storage medium
CN114374597A (en) * 2021-12-27 2022-04-19 浪潮通信信息系统有限公司 Fault processing method, device, equipment and product of network event
CN114928531A (en) * 2022-05-06 2022-08-19 广西电网有限责任公司 Network security integrated intelligent protection method and device, robot and medium
CN114936930A (en) * 2022-07-21 2022-08-23 平安银行股份有限公司 Method for managing abnormal timeliness service of network node, computer equipment and storage medium
CN115225460A (en) * 2022-07-15 2022-10-21 北京天融信网络安全技术有限公司 Failure determination method, electronic device, and storage medium
CN116108371A (en) * 2023-04-13 2023-05-12 西华大学 Cloud service abnormity diagnosis method and system based on cascade abnormity generation network
CN116471198A (en) * 2023-06-19 2023-07-21 南京典格通信科技有限公司 Power amplifier fault prediction method based on limited Boltzmann machine

Citations (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN109270442A (en) * 2018-08-21 2019-01-25 西安工程大学 High-voltage circuitbreaker fault detection method based on DBN-GA neural network
CN109615109A (en) * 2018-12-29 2019-04-12 大唐河南清洁能源有限责任公司 Deep learning wind-powered electricity generation warning information analysis method based on Small Sample Database
CN109669087A (en) * 2019-01-31 2019-04-23 国网河南省电力公司 A kind of method for diagnosing fault of power transformer based on Multi-source Information Fusion
CN110298401A (en) * 2019-06-28 2019-10-01 上海电力学院 It is a kind of based on fuzzy and de-redundancy limited Boltzmann machine image classification method
CN110334756A (en) * 2019-06-26 2019-10-15 国网江苏省电力有限公司 Power system monitor alarm event knows method for distinguishing, terminal installation, equipment and medium
CN110728230A (en) * 2019-10-10 2020-01-24 江南大学 Signal modulation mode identification method based on convolution limited Boltzmann machine
US20200285900A1 (en) * 2019-03-06 2020-09-10 Wuhan University Power electronic circuit fault diagnosis method based on optimizing deep belief network
CN112202630A (en) * 2020-09-16 2021-01-08 中盈优创资讯科技有限公司 Network quality abnormity detection method and device based on unsupervised model

Patent Citations (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN109270442A (en) * 2018-08-21 2019-01-25 西安工程大学 High-voltage circuitbreaker fault detection method based on DBN-GA neural network
CN109615109A (en) * 2018-12-29 2019-04-12 大唐河南清洁能源有限责任公司 Deep learning wind-powered electricity generation warning information analysis method based on Small Sample Database
CN109669087A (en) * 2019-01-31 2019-04-23 国网河南省电力公司 A kind of method for diagnosing fault of power transformer based on Multi-source Information Fusion
US20200285900A1 (en) * 2019-03-06 2020-09-10 Wuhan University Power electronic circuit fault diagnosis method based on optimizing deep belief network
CN110334756A (en) * 2019-06-26 2019-10-15 国网江苏省电力有限公司 Power system monitor alarm event knows method for distinguishing, terminal installation, equipment and medium
CN110298401A (en) * 2019-06-28 2019-10-01 上海电力学院 It is a kind of based on fuzzy and de-redundancy limited Boltzmann machine image classification method
CN110728230A (en) * 2019-10-10 2020-01-24 江南大学 Signal modulation mode identification method based on convolution limited Boltzmann machine
CN112202630A (en) * 2020-09-16 2021-01-08 中盈优创资讯科技有限公司 Network quality abnormity detection method and device based on unsupervised model

Cited By (10)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN114374597A (en) * 2021-12-27 2022-04-19 浪潮通信信息系统有限公司 Fault processing method, device, equipment and product of network event
CN114363151A (en) * 2022-01-07 2022-04-15 北京金山云网络技术有限公司 Fault detection method and device, electronic equipment and storage medium
CN114928531A (en) * 2022-05-06 2022-08-19 广西电网有限责任公司 Network security integrated intelligent protection method and device, robot and medium
CN114928531B (en) * 2022-05-06 2023-09-05 广西电网有限责任公司 Network security integrated intelligent protection method, device, robot and medium
CN115225460A (en) * 2022-07-15 2022-10-21 北京天融信网络安全技术有限公司 Failure determination method, electronic device, and storage medium
CN115225460B (en) * 2022-07-15 2023-11-28 北京天融信网络安全技术有限公司 Fault determination method, electronic device, and storage medium
CN114936930A (en) * 2022-07-21 2022-08-23 平安银行股份有限公司 Method for managing abnormal timeliness service of network node, computer equipment and storage medium
CN116108371A (en) * 2023-04-13 2023-05-12 西华大学 Cloud service abnormity diagnosis method and system based on cascade abnormity generation network
CN116471198A (en) * 2023-06-19 2023-07-21 南京典格通信科技有限公司 Power amplifier fault prediction method based on limited Boltzmann machine
CN116471198B (en) * 2023-06-19 2023-10-03 南京典格通信科技有限公司 Power amplifier fault prediction method based on limited Boltzmann machine

Also Published As

Publication number Publication date
CN113822421B (en) 2024-05-14

Similar Documents

Publication Publication Date Title
CN113822421B (en) Neural network-based anomaly locating method, system, equipment and storage medium
JP7167009B2 (en) System and method for predicting automobile warranty fraud
CN112800116B (en) Method and device for detecting abnormity of service data
Celikmih et al. Failure prediction of aircraft equipment using machine learning with a hybrid data preparation method
CN117455417B (en) Automatic iterative optimization method and system for intelligent wind control approval strategy
CN114881349A (en) Big data analysis-based early warning method and storage medium
Dou et al. Pc 2 a: predicting collective contextual anomalies via lstm with deep generative model
Lima et al. Smart predictive maintenance for high-performance computing systems: a literature review
CN113687972A (en) Method, device and equipment for processing abnormal data of business system and storage medium
Mobtahej et al. An LSTM‐Autoencoder Architecture for Anomaly Detection Applied on Compressors Audio Data
Chowdhury et al. Internet of Things resource monitoring through proactive fault prediction
Ali et al. ICS-IDS: application of big data analysis in AI-based intrusion detection systems to identify cyberattacks in ICS networks
CN116893924B (en) Equipment fault processing method, device, electronic equipment and storage medium
Vargas et al. A hybrid feature learning approach based on convolutional kernels for ATM fault prediction using event-log data
Gęca Performance comparison of machine learning algotihms for predictive maintenance
CN117579332A (en) Network threat detection method and device
CN112418460A (en) Fault diagnosis method and fault diagnosis device for engineering vehicle
CN116991615A (en) Cloud primary system fault self-healing method and device based on online learning
CN116702839A (en) Model training method and application system based on convolutional neural network
CN116882756A (en) Power safety control method based on block chain
CN113076217B (en) Disk fault prediction method based on domestic platform
Zoppi et al. Anomaly-based error and intrusion detection in tabular data: No DNN outperforms tree-based classifiers
Gavrylenko et al. Study of Methods for Improving the Meta-Algorithm of the Bagging Classifier
CN117807590B (en) Information security prediction and monitoring system and method based on artificial intelligence
CN115831339B (en) Medical system risk management and control pre-prediction method and system based on deep learning

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant