CN113822421B - Neural network-based anomaly locating method, system, equipment and storage medium - Google Patents

Neural network-based anomaly locating method, system, equipment and storage medium Download PDF

Info

Publication number
CN113822421B
CN113822421B CN202111197387.0A CN202111197387A CN113822421B CN 113822421 B CN113822421 B CN 113822421B CN 202111197387 A CN202111197387 A CN 202111197387A CN 113822421 B CN113822421 B CN 113822421B
Authority
CN
China
Prior art keywords
boltzmann machine
limited boltzmann
alarm information
fault alarm
target service
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
CN202111197387.0A
Other languages
Chinese (zh)
Other versions
CN113822421A (en
Inventor
吴毓霖
代本辉
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Ping An Technology Shenzhen Co Ltd
Original Assignee
Ping An Technology Shenzhen Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Ping An Technology Shenzhen Co Ltd filed Critical Ping An Technology Shenzhen Co Ltd
Priority to CN202111197387.0A priority Critical patent/CN113822421B/en
Publication of CN113822421A publication Critical patent/CN113822421A/en
Application granted granted Critical
Publication of CN113822421B publication Critical patent/CN113822421B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Classifications

    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06NCOMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00Computing arrangements based on biological models
    • G06N3/02Neural networks
    • G06N3/04Architecture, e.g. interconnection topology
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00Pattern recognition
    • G06F18/20Analysing
    • G06F18/21Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
    • G06F18/214Generating training patterns; Bootstrap methods, e.g. bagging or boosting
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06NCOMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00Computing arrangements based on biological models
    • G06N3/02Neural networks
    • G06N3/08Learning methods
    • G06N3/084Backpropagation, e.g. using gradient descent

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • Data Mining & Analysis (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Artificial Intelligence (AREA)
  • General Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • Evolutionary Computation (AREA)
  • Biophysics (AREA)
  • Computational Linguistics (AREA)
  • Software Systems (AREA)
  • Mathematical Physics (AREA)
  • Health & Medical Sciences (AREA)
  • Biomedical Technology (AREA)
  • Computing Systems (AREA)
  • Molecular Biology (AREA)
  • General Health & Medical Sciences (AREA)
  • Evolutionary Biology (AREA)
  • Bioinformatics & Cheminformatics (AREA)
  • Bioinformatics & Computational Biology (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Testing And Monitoring For Control Systems (AREA)

Abstract

The invention relates to the technical field of artificial intelligence, and provides an anomaly positioning method, system, equipment and storage medium based on a neural network, wherein the method comprises the following steps: acquiring current fault alarm information after preprocessing of a target service system; acquiring current key features according to the preprocessed current fault warning information and the trained limited Boltzmann machine model; acquiring risk prediction grades and abnormal position information according to the current key characteristics and the trained fault prediction neural network model; and pre-alarming the target service system according to the risk prediction grade and the abnormal position information. In the embodiment of the invention, the key features in the fault alarm information are extracted through the limited Boltzmann machine, and the key factors influencing the abnormality of the target service system can be obtained through probability processing of the fault alarm factors, so that the subsequent abnormality positioning precision of the target service system is improved.

Description

Neural network-based anomaly locating method, system, equipment and storage medium
Technical Field
The invention relates to the technical field of artificial intelligence, in particular to an anomaly locating method, system, equipment and storage medium based on a neural network.
Background
In the big data fields of banking industry, insurance industry, electronic commerce and the like, various application systems generate massive transaction data every day, high reliability is maintained for important business systems, and rapid positioning and repair are required for abnormal conditions of the systems, so that serious losses caused by failure of the business systems are avoided.
The traditional fault warning or risk early warning mechanism is a method for realizing system early warning by using methods such as an artificial neural network, a fuzzy algorithm and the like, and timely response and processing of the abnormality are realized by judging a custom rule or strategy and feeding back the abnormality to operation and maintenance personnel through manual judgment. Although these methods can help to diagnose the uncertainty fault information of the application service, because the service system is usually large in scale and has a large number of factors affecting the stability of the system, the positioning accuracy is not high, and a situation of false alarm often occurs, so that operation and maintenance personnel sometimes have to face the situation of false alarm.
Therefore, how to realize effective early warning of service system faults is one of the main problems faced in the operation and maintenance process.
Disclosure of Invention
The invention provides an anomaly positioning method based on a neural network, which mainly aims to carry out risk classification on fault alarms and effectively improve the fault pre-alarm accuracy.
In a first aspect, an embodiment of the present invention provides an anomaly positioning method based on a neural network, including:
Acquiring current fault alarm information after preprocessing of a target service system;
Acquiring current key features according to the preprocessed current fault warning information and the trained limited Boltzmann machine model;
Acquiring a risk prediction grade and abnormal position information according to the current key characteristics and the trained fault prediction neural network model;
And pre-alarming the target service system according to the risk prediction grade and the abnormal position information.
Preferably, the trained limited boltzmann machine model is obtained by:
Acquiring a preset number of historical fault alarm information training sets, wherein each historical fault alarm information training set comprises a plurality of historical fault alarm information corresponding to the same historical abnormal position information;
Training each initial limited Boltzmann machine by utilizing each historical fault alarm information training set to obtain a preset number of trained initial limited Boltzmann machines;
The initial limited boltzmann machines after the preset number of training are cascaded as a trained limited boltzmann machine model.
Preferably, the trained fault prediction neural network model is obtained by the following steps:
inputting each history fault alarm information training set into a trained limited Boltzmann machine model, and acquiring history key features corresponding to each history fault alarm information;
Training the initial fault prediction neural network model according to the historical key characteristics corresponding to each historical fault alarm information, the preset historical risk level corresponding to each historical fault alarm information and the historical abnormal position information corresponding to each historical fault alarm information, and obtaining a trained fault prediction neural network model.
Preferably, the initial confined boltzmann machine is a two-layered confined boltzmann machine.
Preferably, the current key features are obtained according to the preprocessed current fault warning information and the trained limited boltzmann machine model, and a specific calculation formula is as follows:
ftr=[ftr1,…,ftrk,…,ftrn],
wherein ftr represents the current key features, n represents the preset number, ftr k represents the current key features extracted by the initial limited boltzmann machine after training corresponding to the kth category, For the interaction term between hidden layer units and visible units in the k-th class corresponding first layer limited boltzmann machine model,/>For the k-th class corresponds to the bias of the visible layers in the first layer limited boltzmann machine model,/>For the k-th class to correspond to the bias of the hidden layer in the first layer limited boltzmann machine model,For the interaction term between hidden layer units and visible units in the k-th class corresponding first layer limited boltzmann machine model,/>For the k-th class corresponds to the bias of the visible layers in the second layer-restricted boltzmann machine model,/>For the bias of the k-th class corresponding to the hidden layer in the second layer restricted boltzmann machine model, σ (x) is the sigmoid activation function.
Preferably, the obtaining the current fault alarm information after the target service system is preprocessed specifically includes:
Collecting monitoring index data of the target service system in real time, wherein the monitoring index data comprises basic resource monitoring data, application performance monitoring data and network security detection data;
performing outlier processing, missing value processing, unified formatting and desensitization preprocessing on the monitoring index data to obtain preprocessed monitoring index data;
And detecting abnormal values of the preprocessed monitoring index data to acquire preprocessed current fault alarm information.
Preferably, the outlier processing includes one or more of a fixed thresholding method, a dynamic thresholding method and an index data prediction method.
In a second aspect, an embodiment of the present invention provides an anomaly locating system based on a neural network, including:
The preprocessing module is used for acquiring current fault alarm information after the preprocessing of the target service system;
the feature extraction module is used for acquiring current key features according to the preprocessed current fault alarm information and the trained limited Boltzmann machine model;
The prediction module is used for obtaining risk prediction grade and abnormal position information according to the current key characteristics and the trained fault prediction neural network model;
And the alarm module is used for pre-alarming the target service system according to the risk prediction grade and the abnormal position information.
In a third aspect, an embodiment of the present invention provides a computer device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, where the processor implements the steps of the neural network-based anomaly localization method described above when the processor executes the computer program.
In a fourth aspect, an embodiment of the present invention provides a computer storage medium storing a computer program, which when executed by a processor, implements the steps of the neural network-based anomaly localization method described above.
According to the abnormal positioning method, system, equipment and storage medium based on the neural network, due to the fact that the factors influencing the abnormality of the target service system are many, the abnormal positioning accuracy according to the monitored current fault alarm information in the subsequent positioning process is low.
According to the extracted key characteristics, the risk prediction grade of the current fault alarm information is obtained by combining the fault prediction neural network model, so that the risk grade of the fault of the target service system is divided, the abnormal positioning is carried out, the fault prediction accuracy is improved by dividing the risk probability, a reference basis is provided for the uncertain diagnosis of the current fault alarm information, so that an operation and maintenance person can determine the abnormal emergency degree of the target service system according to the risk grade, and reasonable time and manpower can be conveniently allocated for the subsequent fault treatment.
Drawings
Fig. 1 is an application scenario diagram of an anomaly locating method based on a neural network according to an embodiment of the present invention;
FIG. 2 is a flowchart of an anomaly locating method based on a neural network according to an embodiment of the present invention;
FIG. 3 is a schematic structural diagram of an anomaly locating system based on a neural network according to an embodiment of the present invention;
Fig. 4 is a schematic structural diagram of a computer device according to the present embodiment.
The achievement of the objects, functional features and advantages of the present invention will be further described with reference to the accompanying drawings, in conjunction with the embodiments.
Detailed Description
It should be understood that the specific embodiments described herein are for purposes of illustration only and are not intended to limit the scope of the invention.
Fig. 1 is an application scenario diagram of an anomaly positioning method based on a neural network, as shown in fig. 1, in which a target service system is monitored in real time by corresponding monitoring software in a client, monitoring information is extracted, if the target service system fails, current failure alarm information of the target service system is collected, the current failure alarm information is sent to a server, and after the server receives the current failure alarm information, the anomaly positioning method based on the neural network is executed to realize a pre-alarm for the target service system.
It should be noted that, the server may be implemented by an independent server or a server cluster formed by a plurality of servers, or may be a cloud server that provides cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, a content distribution network, and basic cloud computing services such as big data and an artificial intelligent platform. The client may be, but is not limited to, a smart phone, a tablet computer, a notebook computer, a desktop computer, etc. The client and the server may be connected by bluetooth, USB (Universal Serial Bus ) or other communication connection methods, which is not limited in this embodiment of the present invention.
Fig. 2 is a flowchart of an anomaly positioning method based on a neural network according to an embodiment of the present invention, as shown in fig. 2, where the method includes:
s210, acquiring current fault alarm information after preprocessing of a target service system;
in the embodiment of the invention, the target service system is a system which needs to perform fault monitoring and abnormality positioning, and can be a banking system, an insurance system, an e-commerce system and the like, and the target service system is specifically selected according to actual conditions.
Firstly, monitoring information of a target service system is collected, the monitoring information comprises MVC monitoring information, data information of system operation log cloud monitoring and the like, the MVC monitoring information can record execution time of actions under each controller and view rendering completion time, and log4net is adopted to record execution time and view rendering completion time of each action of MVC and data sent or received when the actions are requested. The system log is information that records hardware, software and target business system problems in the target business system, and can also monitor events occurring in the system. Through which the user can check the cause of the error or look for traces left by an attacker when under attack. The system log includes a system log, an application log, and a security log.
Some abnormal indexes are filtered from the collected monitoring information, for example, normal ranges corresponding to all the encountered indexes in the target service system are generally stored, and after the real-time data of the indexes are collected, the real-time values of the indexes are compared with the normal ranges, so that whether the indexes are abnormal or not is judged. For an indicator, the normal operating range is generally within a value range, and if the real-time value of the indicator is greater than the maximum value of the value range or less than the minimum value of the value range, the indicator is abnormal. According to the method, all indexes of the target service system are judged, and all indexes in abnormal states are filtered out to be used as current fault alarm information of the target service system.
In the process of monitoring a target service system, some abnormal values, missing values and the like are inevitably existed in the monitored current fault alarm information due to the interference of external environment factors, the existence of noise, system jitter and the like, so that the current fault alarm information needs to be preprocessed, some abnormal data in the current fault alarm information are removed, and standardized sample processing is performed on the current fault alarm information to form a matrix, so that the processed current fault alarm information meets the subsequent processing requirements.
S220, acquiring current key features according to the preprocessed current fault warning information and the trained limited Boltzmann machine model;
And inputting the preprocessed current fault alarm information into a trained limited Boltzmann machine model, and extracting key indexes, namely current key features, from all abnormal indexes of the current fault alarm information after probability processing. In the embodiment of the invention, as a plurality of factors influencing the abnormality of the target service system exist, the abnormality positioning precision according to the monitored current fault alarm information in the subsequent positioning process is not high.
It should be noted that the limited Boltzman machine (RESTRICTED BOLTZMANN MACHINE, abbreviated as RBM) is a special topology of Boltzman Machine (BM). The BM is a symmetrically coupled random feedback type binary unit neural network, which consists of a visible layer and a plurality of hidden layers, wherein network nodes are divided into visible units (visible units) and hidden units (hidden units), the visible units and the hidden units are used for expressing a learning model of the random network and a random environment, and the correlation between the weight expression units is used for expressing the correlation between the units.
As the name suggests, the restricted boltzmann machine is a variation of boltzmann machine, but the defined model must be a bipartite graph. The model contains input (visible) units corresponding to input parameters and hidden units corresponding to training results, and each side must be connected with one visible unit and one hidden unit. This limitation makes training of the restricted boltzmann machine more efficient than a general boltzmann machine.
The limited Boltzmann machine is widely applied to dimension reduction, classification, collaborative filtering, feature learning and theme modeling, and can be trained by using a supervised learning or unsupervised learning method according to different tasks.
In the embodiment of the invention, the limited Boltzmann machine is trained in an unsupervised learning mode, and a trained limited Boltzmann machine model is obtained.
S230, acquiring a risk prediction grade and abnormal position information according to the current key characteristics and the trained fault prediction neural network model;
Then inputting the current key features into a trained fault prediction neural network model to obtain risk prediction grade and abnormal position information of the target service system, wherein the risk grade represents the probability of abnormality of the target service system, and the target service system is divided into different layers according to the probability, such as general, medium, serious and the like, and can be specifically divided according to actual needs; the abnormal location information may represent a specific location in the target business system where an abnormality occurs, such as a system, middleware, database, or the like.
The risk level is to grade the probability of occurrence of faults according to a preset rule, the higher the risk level is, the more dangerous the faults of the target service system are, the lower the risk level is, the less dangerous the faults of the target service system are, the degree of the risk level of the faults of the target service system is quantified through the risk level, the current emergency situation of the target service system can be conveniently obtained, and the subsequent operation and maintenance personnel can conveniently process the faults.
In addition, the positions of the faults of the corresponding target service systems are different from each other according to different abnormal indexes, and certain indexes can only be acquired at specific positions of the target service systems, so that the positions of the faults of the target service systems can be conveniently positioned according to the abnormal indexes, and the abnormal position information of the target service systems is obtained.
In the embodiment of the invention, the fault prediction neural network model belongs to one of neural networks, and before the fault prediction neural network is used, the fault prediction neural network model also needs to be trained, and the fault prediction neural network is trained through a sample and a label which are obtained in advance. The training process of the fault prediction neural network can be divided into three steps: defining the structure of a fault prediction neural network and the output result of forward propagation; defining a loss function and a back propagation optimization algorithm; finally, generating a session and repeatedly running a back propagation optimization algorithm on the training data.
The neurons are the minimum units forming the neural network, one neuron can have a plurality of inputs and one output, and the input of each neuron can be the output of other neurons or the input of the whole neural network. The output of the neural network is the input weighted sum of all the neurons, the weights of different inputs are the neuron parameters, and the optimization process of the neural network is the process of optimizing the values of the neuron parameters.
The effect and optimization objective of the neural network are defined by a loss function, the loss function gives a calculation formula of the difference between the output result of the neural network and the real label, and supervised learning is one way of training the neural network, the idea is that the result given by the neural network is as close to the real answer (i.e. label) as possible on the labeling data set of the known answer. The training data is fitted by adjusting parameters in the neural network so that the neural network provides predictive power for unknown samples.
The backward propagation algorithm realizes an iterative process, and when each iteration starts, a part of training data is firstly taken, and a prediction result of the neural network is obtained through the forward propagation algorithm. Because the training data has correct answers, the gap between the predicted result and the correct answer can be calculated. Based on the gap, the back propagation algorithm correspondingly updates the values of the neural network parameters so that the values are closer to the true answers.
After the training process is finished by the method, the fault prediction neural network after the training is finished can be utilized for application.
In the embodiment of the invention, the fault prediction neural network may be specifically a convolutional neural network (Convolutional Neural Networks, abbreviated as CNN), a BP (back propagation) neural network, and the like, and is specifically determined according to an actual situation.
S240, pre-alarming is carried out on the target business system according to the risk prediction grade and the abnormal position information.
And then pre-alarming the target service system according to the risk prediction grade and the abnormal position information, specifically, feeding back the risk prediction grade and the abnormal position information of the target service system to related operation and maintenance personnel so as to realize pre-alarming of the target service system.
In summary, according to the neural network-based anomaly positioning method provided by the embodiment of the invention, as a plurality of factors influencing the anomaly of the target service system are provided, the anomaly positioning accuracy according to the monitored current fault alarm information in the subsequent positioning process is not high.
According to the extracted key characteristics, the risk prediction grade of the current fault alarm information is obtained by combining the fault prediction neural network model, so that the risk grade of the fault of the target service system is divided, the abnormal positioning is carried out, the fault prediction accuracy is improved by dividing the risk probability, a reference basis is provided for the uncertain diagnosis of the current fault alarm information, so that an operation and maintenance person can determine the abnormal emergency degree of the target service system according to the risk grade, and reasonable time and manpower can be conveniently allocated for the subsequent fault treatment.
On the basis of the above embodiment, preferably, the trained limited boltzmann machine model is obtained by:
Acquiring a preset number of historical fault alarm information training sets, wherein each historical fault alarm information training set comprises a plurality of historical fault alarm information corresponding to the same historical abnormal position information;
Training each initial limited Boltzmann machine by utilizing each historical fault alarm information training set to obtain a preset number of trained initial limited Boltzmann machines;
The initial limited boltzmann machines after the preset number of training are cascaded as a trained limited boltzmann machine model.
Specifically, since the boltzmann machine model is also a neural network model in nature, and similarly, before the boltzmann machine model is used, the boltzmann machine model needs to be trained, and when training is performed, various historical fault alarm information of the target service system, which occurs before, needs to be collected in advance, and the collection method and the collection process thereof are the same as those of the current fault alarm information, please refer to the above process in detail, and the embodiments of the present invention are not described in detail herein.
After enough historical fault alarm information is collected, the historical fault alarm information is classified according to the occurrence positions of the historical fault alarm information, and the historical fault alarm information belonging to the same fault position is classified into a training set, so that multiple categories of historical fault alarm information training sets can be obtained, and the abnormal positions corresponding to each training set are different.
And if the total of C historical fault alarm information training sets exist, C identical initial limited Boltzmann machines are acquired first, each historical fault alarm information training set corresponds to one initial limited Boltzmann machine, the corresponding initial limited Boltzmann machine is trained by utilizing the historical fault alarm information training set, and the trained initial limited Boltzmann machine is obtained. And cascading the obtained C trained initial limited Boltzmann machines to obtain a trained limited Boltzmann machine model.
In the embodiment of the invention, the fault alarm information of different abnormal positions is classified, and the fault alarm information of different abnormal positions trains different initial limited boltzmann machines, so that the training speed can be increased by the separate training mode, the initial limited boltzmann machines can be converged more quickly in the training process, and the trained parameters are more accurate. When the method is specifically applied, all the trained initial limited Boltzmann machines are cascaded, so that the trained limited Boltzmann machine model can be obtained, and the training parameters of each trained initial limited Boltzmann machine are more specific, so that the feature extraction accuracy can be improved through the independent training and cascading mode.
On the basis of the above embodiment, preferably, the trained failure prediction neural network model is obtained by:
inputting each history fault alarm information training set into a trained limited Boltzmann machine model, and acquiring history key features corresponding to each history fault alarm information;
Training the initial fault prediction neural network model according to the historical key characteristics corresponding to each historical fault alarm information, the preset historical risk level corresponding to each historical fault alarm information and the historical abnormal position information corresponding to each historical fault alarm information, and obtaining a trained fault prediction neural network model.
Specifically, the failure prediction neural network model is also one of the neural network models, and training of the failure prediction neural network model is also required before the failure prediction neural network model is used. In addition to acquiring enough historical fault alarm information, a preset historical risk level corresponding to the historical fault alarm information and a historical abnormal position corresponding to the historical fault alarm information also need to be acquired.
And inputting the historical fault warning information into the trained limited Boltzmann machine model, extracting the historical key features corresponding to the historical fault warning information, taking the historical key features corresponding to the historical fault warning information as the input of an initial fault prediction neural network model, taking the preset historical risk level and the historical abnormal position corresponding to the historical fault warning information as the labels of the initial fault prediction neural network model, and training the initial fault prediction neural network model to obtain the trained fault prediction neural network model.
On the basis of the above embodiment, preferably, the initial limited boltzmann machine is a two-layer limited boltzmann machine.
Specifically, the number of layers of the initially constrained boltzmann machine is 2. Correspondingly, for the historical fault warning information of the C categories, a two-layer limited Boltzmann machine model is adopted for training, and the C two-layer limited Boltzmann machine models are obtained. Wherein the i-th type of historical fault alarm informationThe trained limited Boltzmann machine parameters areAnd/>Wherein/>Interactive item between hidden layer unit and visible unit in first layer limited Boltzmann machine model corresponding to i type history fault alarm information,/>For the bias of the visible layer in the first layer limited boltzmann machine model corresponding to the i-th type sample,/>For the bias of hidden layers in the first-layer limited Boltzmann machine model corresponding to the i-th sample,/>Interactive item between hidden layer unit and visible unit in second-layer limited Boltzmann machine model corresponding to i-th type historical fault alarm information,/>For the deviation of the visible layer in the second layer limited boltzmann machine model corresponding to the i-th type sample,/>The bias of the hidden layer in the second layer limited boltzmann machine model corresponding to the i-th sample.
Experiments prove that the extraction precision of key features can be improved by adopting a two-layer limited Boltzmann machine model, and the training efficiency of the model is compatible, so that the model is the most suitable model when the precision and the instantaneity are ensured.
On the basis of the foregoing embodiment, preferably, the current key feature is obtained according to the preprocessed current fault warning information and the trained limited boltzmann machine model, and a specific calculation formula is as follows:
ftr=[ftr1,…,ftrk,…,ftrn],
wherein ftr represents the current key features, n represents the preset number, ftr k represents the current key features extracted by the initial limited boltzmann machine after training corresponding to the kth category, For the interaction term between hidden layer units and visible units in the k-th class corresponding first layer limited boltzmann machine model,/>For the k-th class corresponds to the bias of the visible layers in the first layer limited boltzmann machine model,/>For the k-th class to correspond to the bias of the hidden layer in the first layer limited boltzmann machine model,For the interaction term between hidden layer units and visible units in the k-th class corresponding first layer limited boltzmann machine model,/>For the k-th class corresponds to the bias of the visible layers in the second layer-restricted boltzmann machine model,/>For the bias of the k-th class corresponding to the hidden layer in the second layer restricted boltzmann machine model, σ (x) is the sigmoid activation function.
On the basis of the foregoing embodiment, preferably, the obtaining the current fault alarm information after preprocessing of the target service system specifically includes:
Collecting monitoring index data of the target service system in real time, wherein the monitoring index data comprises basic resource monitoring data, application performance monitoring data and network security detection data;
performing outlier processing, missing value processing, unified formatting and desensitization preprocessing on the monitoring index data to obtain preprocessed monitoring index data;
And detecting abnormal values of the preprocessed monitoring index data to acquire preprocessed current fault alarm information.
Specifically, the monitoring index data of the target service system is collected in real time, the monitoring index data comprises basic resource monitoring data, application performance monitoring data and network security monitoring data, the basic resource monitoring data refers to related data of software and hardware basic parameters of the target service system, such as target service system temperature, target service system memory size, target service system processor running speed and the like, the application performance monitoring data refers to application running conditions of the target service system, e.g. throughput rate, abnormal burst frequency and the like, and the network security monitoring data refers to data related to network security of the target service system, such as attack frequency, success rate of resisting attack and the like.
The monitoring index data is subjected to abnormal value detection processing, missing value processing, unified formatting processing, desensitization processing and the like, and the abnormal value processing includes: detecting an abnormal value of the acquired data, removing the abnormal value, and replacing the abnormal value by using a data average value of two adjacent moments before and after the abnormal point. The missing value processing includes: and carrying out interpolation and deficiency on the deficiency value to complete the data set. The unified formatting includes: carrying out unified formatting treatment on the monitoring index data of different types and different dimensions; data desensitization: and encrypting the acquired data, and then carrying out network transmission and storage.
By preprocessing the current fault alarm information, the abnormal prediction precision of the subsequent target service system can be improved.
In summary, the embodiment of the invention provides an anomaly positioning method based on a neural network, in the method, because factors are subjected to probability processing, key factors influencing the anomaly of a target service system can be obtained, so that the subsequent anomaly positioning accuracy of the target service system is improved.
According to the extracted key characteristics, the fault prediction neural network model is combined, the risk level of the fault of the target service system is divided, the abnormality is positioned, the fault prediction warning precision is improved through the division of the risk probability, and a reference basis is provided for the uncertain current fault warning information diagnosis, so that the operation and maintenance personnel can determine the abnormal emergency degree of the target service system according to the risk level, and reasonable time and manpower can be conveniently allocated for the subsequent fault treatment.
In addition, in the training process of the initial limited Boltzmann machine, fault alarm information of different abnormal positions is classified, the fault alarm information of different abnormal positions trains different initial limited Boltzmann machines, the training speed can be increased by the aid of the separate training mode, the initial limited Boltzmann machines can be converged more quickly in the training process, and the trained parameters are more accurate. When the method is specifically applied, all the trained initial limited Boltzmann machines are cascaded, so that the trained limited Boltzmann machine model can be obtained, and the training parameters of each trained initial limited Boltzmann machine are more specific, so that the feature extraction accuracy can be improved through the independent training and cascading mode.
Finally, the abnormal data and noise in the monitoring data of the target service system are removed by preprocessing the monitoring data, so that the subsequent prediction accuracy is improved.
Fig. 3 is a schematic structural diagram of an anomaly locating system based on a neural network according to an embodiment of the present invention, as shown in fig. 3, the system includes a preprocessing module 310, a feature extraction module 320, a prediction module 330, and an alarm module 340, where:
The preprocessing module 310 is configured to obtain current fault alarm information after preprocessing of the target service system;
The feature extraction module 320 is configured to obtain current key features according to the preprocessed current fault warning information and the trained limited boltzmann machine model;
The prediction module 330 is configured to obtain a risk prediction level and abnormal location information according to the current key feature and the trained failure prediction neural network model;
The alarm module 340 is configured to pre-alarm the target service system according to the risk prediction level and the abnormal location information.
Specifically, in the feature extraction module 320, the trained limited boltzmann machine model is obtained by:
Acquiring a preset number of historical fault alarm information training sets, wherein each historical fault alarm information training set comprises a plurality of historical fault alarm information corresponding to the same historical abnormal position information;
Training each initial limited Boltzmann machine by utilizing each historical fault alarm information training set to obtain a preset number of trained initial limited Boltzmann machines;
The initial limited boltzmann machines after the preset number of training are cascaded as a trained limited boltzmann machine model.
Specifically, in the prediction module 330, the trained failure prediction neural network model is obtained by the following steps:
inputting each history fault alarm information training set into a trained limited Boltzmann machine model, and acquiring history key features corresponding to each history fault alarm information;
Training the initial fault prediction neural network model according to the historical key characteristics corresponding to each historical fault alarm information, the preset historical risk level corresponding to each historical fault alarm information and the historical abnormal position information corresponding to each historical fault alarm information, and obtaining a trained fault prediction neural network model.
Specifically, the initial restricted boltzmann machine is a two-layered restricted boltzmann machine.
Specifically, in the feature extraction module 320, the current key feature is obtained according to the preprocessed current fault alarm information and the trained limited boltzmann machine model, and a specific calculation formula is as follows:
ftr=[ftr1,…,ftrk,…,ftrn],
wherein ftr represents the current key features, n represents the preset number, ftr k represents the current key features extracted by the initial limited boltzmann machine after training corresponding to the kth category, For the interaction term between hidden layer units and visible units in the k-th class corresponding first layer limited boltzmann machine model,/>For the k-th class corresponds to the bias of the visible layers in the first layer limited boltzmann machine model,/>For the k-th class to correspond to the bias of the hidden layer in the first layer limited boltzmann machine model,For the interaction term between hidden layer units and visible units in the k-th class corresponding first layer limited boltzmann machine model,/>For the k-th class corresponds to the bias of the visible layers in the second layer-restricted boltzmann machine model,/>For the bias of the k-th class corresponding to the hidden layer in the second layer restricted boltzmann machine model, σ (x) is the sigmoid activation function.
Specifically, the preprocessing module comprises a monitoring unit, a preprocessing unit and a detection unit, wherein:
The monitoring unit is used for collecting monitoring index data of the target service system in real time, wherein the monitoring index data comprises basic resource monitoring data, application performance monitoring data and network security detection data;
the preprocessing unit is used for performing outlier processing, missing value processing, unified formatting and desensitization preprocessing on the monitoring index data to obtain preprocessed monitoring index data;
The detection unit is used for detecting abnormal values of the preprocessed monitoring index data and acquiring preprocessed current fault alarm information.
Specifically, the outlier processing includes one or more of a fixed threshold method, a dynamic threshold method, and an index data prediction method.
The various modules in the neural network-based anomaly localization system described above may be implemented in whole or in part by software, hardware, or a combination thereof. The above modules may be embedded in hardware or may be independent of a processor in the computer device, or may be stored in software in a memory in the computer device, so that the processor may call and execute operations corresponding to the above modules.
The specific implementation process of the system embodiment corresponding to the above method is the same as that of the above method embodiment, and reference is made to the above method embodiment for details, which are not repeated here.
Fig. 4 is a schematic structural diagram of a computer device provided in this embodiment, where the computer device may be a server, and an internal structure diagram thereof may be as shown in fig. 4. The computer device includes a processor, a memory, a network interface, and a database connected by a system bus. Wherein the processor of the computer device is configured to provide computing and control capabilities. The memory of the computer device includes a computer storage medium, an internal memory. The computer storage medium stores an operating system, computer programs, and a database. The internal memory provides an environment for the operation of the operating system and computer programs in the computer storage media. The database of the computer equipment is used for storing data generated or acquired in the process of executing an anomaly locating method based on the neural network, such as current fault alarm information, a trained limited Boltzmann machine model, a trained fault prediction neural network model and the like. The network interface of the computer device is used for communicating with an external terminal through a network connection. The computer program, when executed by a processor, is configured to implement a neural network-based anomaly localization method, and specifically includes:
Acquiring current fault alarm information after preprocessing of a target service system;
Acquiring current key features according to the preprocessed current fault warning information and the trained limited Boltzmann machine model;
Acquiring a risk prediction grade and abnormal position information according to the current key characteristics and the trained fault prediction neural network model;
And pre-alarming the target service system according to the risk prediction grade and the abnormal position information.
In one embodiment, a computer device is provided that includes a memory, a processor, and a computer program stored on the memory and executable on the processor, the processor implementing the steps of one of the neural network-based anomaly localization methods of the above embodiments when the computer program is executed by the processor. Or the processor, when executing the computer program, implements the functions of the modules/units in this embodiment of a neural network-based anomaly localization system.
In one embodiment, a computer storage medium is provided, and a computer program is stored on the computer storage medium, where the computer program is executed by a processor to implement the steps of a neural network-based anomaly localization method in the above embodiment. Or the computer program when executed by a processor, performs the functions of the modules/units in the embodiment of a neural network-based anomaly localization system.
Those skilled in the art will appreciate that implementing all or part of the above described methods may be accomplished by way of a computer program stored on a non-transitory computer readable storage medium, which when executed, may comprise the steps of the embodiments of the methods described above. Any reference to memory, storage, database, or other medium used in embodiments provided herein may include non-volatile and/or volatile memory. The nonvolatile memory can include Read Only Memory (ROM), programmable ROM (PROM), electrically Programmable ROM (EPROM), electrically Erasable Programmable ROM (EEPROM), or flash memory. Volatile memory can include Random Access Memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in a variety of forms such as Static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double Data Rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link (SYNCHLINK) DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM), among others.
It will be apparent to those skilled in the art that, for convenience and brevity of description, only the above-described division of the functional units and modules is illustrated, and in practical application, the above-described functional distribution may be performed by different functional units and modules according to needs, i.e. the internal structure of the apparatus is divided into different functional units or modules to perform all or part of the above-described functions.
The above embodiments are only for illustrating the technical solution of the present invention, and not for limiting the same; although the invention has been described in detail with reference to the foregoing embodiments, it will be understood by those of ordinary skill in the art that: the technical scheme described in the foregoing embodiments can be modified or some technical features thereof can be replaced by equivalents; such modifications and substitutions do not depart from the spirit and scope of the technical solutions of the embodiments of the present invention, and are intended to be included in the scope of the present invention.

Claims (6)

1. An anomaly locating method based on a neural network is characterized by comprising the following steps:
Acquiring current fault alarm information after preprocessing of a target service system, wherein the target service system is an e-commerce system;
Acquiring current key features according to the preprocessed current fault warning information and the trained limited Boltzmann machine model;
Acquiring a risk prediction grade and abnormal position information according to the current key characteristics and the trained fault prediction neural network model;
pre-alarming the target service system according to the risk prediction grade and the abnormal position information;
The obtaining the current fault alarm information after the target service system preprocessing specifically comprises the following steps:
The method comprises the steps of collecting monitoring index data of a target service system in real time, wherein the monitoring index data comprise basic resource monitoring data, application performance monitoring data and network security detection data, the basic resource monitoring data comprise the temperature of the target service system, the memory size of the target service system and the running speed of a processor of the target service system, the application performance monitoring data comprise throughput rate and abnormal burst frequency, and the network security detection data comprise the frequency of being attacked and the success rate of resisting attack;
performing outlier processing, missing value processing, unified formatting and desensitization preprocessing on the monitoring index data to obtain preprocessed monitoring index data;
Performing outlier detection on the preprocessed monitoring index data to obtain preprocessed current fault warning information;
the trained limited boltzmann machine model is obtained by:
Acquiring a preset number of historical fault alarm information training sets, wherein each historical fault alarm information training set comprises a plurality of historical fault alarm information corresponding to the same historical abnormal position information;
Training each initial limited Boltzmann machine by utilizing each historical fault alarm information training set to obtain a preset number of trained initial limited Boltzmann machines;
Cascading a preset number of trained initial limited Boltzmann machines to serve as a trained limited Boltzmann machine model;
the initial limited boltzmann machine is a two-layer limited boltzmann machine;
The current key characteristics are obtained according to the preprocessed current fault alarm information and the trained limited Boltzmann machine model, and the specific calculation formula is as follows:
ftr=[ftr1,…,ftrk,…,ftrn],
wherein ftr represents the current key features, n represents the preset number, ftr k represents the current key features extracted by the initial limited boltzmann machine after training corresponding to the kth category, For the interaction term between hidden layer units and visible units in the k-th class corresponding first layer limited boltzmann machine model,/>For the k-th class corresponds to the bias of the visible layers in the first layer limited boltzmann machine model,/>For the interaction term between hidden layer units and visible units in the k-th class corresponding first layer limited boltzmann machine model,/>For the k-th class to correspond to the bias of the visible layers in the second layer-constrained boltzmann machine model, σ () is a sigmoid activation function,/>And representing the current fault alarm information after the jth preprocessing of the ith category.
2. The neural network-based anomaly localization method of claim 1, wherein the trained fault-prediction neural network model is obtained by:
inputting each history fault alarm information training set into a trained limited Boltzmann machine model, and acquiring history key features corresponding to each history fault alarm information;
Training the initial fault prediction neural network model according to the historical key characteristics corresponding to each historical fault alarm information, the preset historical risk level corresponding to each historical fault alarm information and the historical abnormal position information corresponding to each historical fault alarm information, and obtaining a trained fault prediction neural network model.
3. The neural network-based anomaly localization method of claim 1, wherein the outlier processing comprises one or more of a fixed-threshold approach, a dynamic-threshold approach, and an index data prediction approach.
4. An anomaly localization system based on a neural network, comprising:
The preprocessing module is used for acquiring current fault alarm information preprocessed by a target service system, wherein the target service system is an e-commerce system;
the feature extraction module is used for acquiring current key features according to the preprocessed current fault alarm information and the trained limited Boltzmann machine model;
The prediction module is used for obtaining risk prediction grade and abnormal position information according to the current key characteristics and the trained fault prediction neural network model;
The alarm module is used for pre-alarming the target service system according to the risk prediction grade and the abnormal position information;
The obtaining the current fault alarm information after the target service system preprocessing specifically comprises the following steps:
The method comprises the steps of collecting monitoring index data of a target service system in real time, wherein the monitoring index data comprise basic resource monitoring data, application performance monitoring data and network security detection data, the basic resource monitoring data comprise the temperature of the target service system, the memory size of the target service system and the running speed of a processor of the target service system, the application performance monitoring data comprise throughput rate and abnormal burst frequency, and the network security detection data comprise the frequency of being attacked and the success rate of resisting attack;
performing outlier processing, missing value processing, unified formatting and desensitization preprocessing on the monitoring index data to obtain preprocessed monitoring index data;
Performing outlier detection on the preprocessed monitoring index data to obtain preprocessed current fault warning information;
the trained limited boltzmann machine model is obtained by:
Acquiring a preset number of historical fault alarm information training sets, wherein each historical fault alarm information training set comprises a plurality of historical fault alarm information corresponding to the same historical abnormal position information;
Training each initial limited Boltzmann machine by utilizing each historical fault alarm information training set to obtain a preset number of trained initial limited Boltzmann machines;
Cascading a preset number of trained initial limited Boltzmann machines to serve as a trained limited Boltzmann machine model;
the initial limited boltzmann machine is a two-layer limited boltzmann machine;
The current key characteristics are obtained according to the preprocessed current fault alarm information and the trained limited Boltzmann machine model, and the specific calculation formula is as follows:
ftr=[ftr1,…,ftrk,…,ftrn],
wherein ftr represents the current key features, n represents the preset number, ftr k represents the current key features extracted by the initial limited boltzmann machine after training corresponding to the kth category, For the interaction term between hidden layer units and visible units in the k-th class corresponding first layer limited boltzmann machine model,/>For the k-th class corresponds to the bias of the visible layers in the first layer limited boltzmann machine model,/>For the interaction term between hidden layer units and visible units in the k-th class corresponding first layer limited boltzmann machine model,/>For the k-th class to correspond to the bias of the visible layers in the second layer-constrained boltzmann machine model, σ () is a sigmoid activation function,/>And representing the current fault alarm information after the jth preprocessing of the ith category.
5. A computer device comprising a memory, a processor and a computer program stored in the memory and executable on the processor, characterized in that the processor implements the steps of the neural network-based anomaly localization method of any one of claims 1 to 3 when the computer program is executed.
6. A computer storage medium storing a computer program, characterized in that the computer program when executed by a processor implements the steps of the neural network-based anomaly localization method of any one of claims 1 to 3.
CN202111197387.0A 2021-10-14 2021-10-14 Neural network-based anomaly locating method, system, equipment and storage medium Active CN113822421B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202111197387.0A CN113822421B (en) 2021-10-14 2021-10-14 Neural network-based anomaly locating method, system, equipment and storage medium

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202111197387.0A CN113822421B (en) 2021-10-14 2021-10-14 Neural network-based anomaly locating method, system, equipment and storage medium

Publications (2)

Publication Number Publication Date
CN113822421A CN113822421A (en) 2021-12-21
CN113822421B true CN113822421B (en) 2024-05-14

Family

ID=78916512

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202111197387.0A Active CN113822421B (en) 2021-10-14 2021-10-14 Neural network-based anomaly locating method, system, equipment and storage medium

Country Status (1)

Country Link
CN (1) CN113822421B (en)

Families Citing this family (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN114374597A (en) * 2021-12-27 2022-04-19 浪潮通信信息系统有限公司 Fault processing method, device, equipment and product of network event
CN114363151A (en) * 2022-01-07 2022-04-15 北京金山云网络技术有限公司 Fault detection method and device, electronic equipment and storage medium
CN114928531B (en) * 2022-05-06 2023-09-05 广西电网有限责任公司 Network security integrated intelligent protection method, device, robot and medium
CN115225460B (en) * 2022-07-15 2023-11-28 北京天融信网络安全技术有限公司 Fault determination method, electronic device, and storage medium
CN114936930B (en) * 2022-07-21 2022-11-29 平安银行股份有限公司 Method for managing abnormal timeliness service of network node, computer equipment and storage medium
CN116108371B (en) * 2023-04-13 2023-07-04 西华大学 Cloud service abnormity diagnosis method and system based on cascade abnormity generation network
CN116471198B (en) * 2023-06-19 2023-10-03 南京典格通信科技有限公司 Power amplifier fault prediction method based on limited Boltzmann machine

Citations (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN109270442A (en) * 2018-08-21 2019-01-25 西安工程大学 High-voltage circuitbreaker fault detection method based on DBN-GA neural network
CN109615109A (en) * 2018-12-29 2019-04-12 大唐河南清洁能源有限责任公司 Deep learning wind-powered electricity generation warning information analysis method based on Small Sample Database
CN109669087A (en) * 2019-01-31 2019-04-23 国网河南省电力公司 A kind of method for diagnosing fault of power transformer based on Multi-source Information Fusion
CN110298401A (en) * 2019-06-28 2019-10-01 上海电力学院 It is a kind of based on fuzzy and de-redundancy limited Boltzmann machine image classification method
CN110334756A (en) * 2019-06-26 2019-10-15 国网江苏省电力有限公司 Power system monitor alarm event knows method for distinguishing, terminal installation, equipment and medium
CN110728230A (en) * 2019-10-10 2020-01-24 江南大学 Signal modulation mode identification method based on convolution limited Boltzmann machine
CN112202630A (en) * 2020-09-16 2021-01-08 中盈优创资讯科技有限公司 Network quality abnormity detection method and device based on unsupervised model

Family Cites Families (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN109933881A (en) * 2019-03-06 2019-06-25 武汉大学 A kind of Fault Diagnosis of Power Electronic Circuits method based on optimization deepness belief network

Patent Citations (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN109270442A (en) * 2018-08-21 2019-01-25 西安工程大学 High-voltage circuitbreaker fault detection method based on DBN-GA neural network
CN109615109A (en) * 2018-12-29 2019-04-12 大唐河南清洁能源有限责任公司 Deep learning wind-powered electricity generation warning information analysis method based on Small Sample Database
CN109669087A (en) * 2019-01-31 2019-04-23 国网河南省电力公司 A kind of method for diagnosing fault of power transformer based on Multi-source Information Fusion
CN110334756A (en) * 2019-06-26 2019-10-15 国网江苏省电力有限公司 Power system monitor alarm event knows method for distinguishing, terminal installation, equipment and medium
CN110298401A (en) * 2019-06-28 2019-10-01 上海电力学院 It is a kind of based on fuzzy and de-redundancy limited Boltzmann machine image classification method
CN110728230A (en) * 2019-10-10 2020-01-24 江南大学 Signal modulation mode identification method based on convolution limited Boltzmann machine
CN112202630A (en) * 2020-09-16 2021-01-08 中盈优创资讯科技有限公司 Network quality abnormity detection method and device based on unsupervised model

Also Published As

Publication number Publication date
CN113822421A (en) 2021-12-21

Similar Documents

Publication Publication Date Title
CN113822421B (en) Neural network-based anomaly locating method, system, equipment and storage medium
CN111178456B (en) Abnormal index detection method and device, computer equipment and storage medium
CN108427720B (en) System log classification method
Darban et al. Deep learning for time series anomaly detection: A survey
CN112800116B (en) Method and device for detecting abnormity of service data
CN111914873A (en) Two-stage cloud server unsupervised anomaly prediction method
WO2021213247A1 (en) Anomaly detection method and device
He et al. MTAD-TF: Multivariate time series anomaly detection using the combination of temporal pattern and feature pattern
CN112433896B (en) Method, device, equipment and storage medium for predicting server disk faults
CN115174231B (en) Network fraud analysis method and server based on AI Knowledge Base
Dou et al. Pc 2 a: predicting collective contextual anomalies via lstm with deep generative model
CN117455417B (en) Automatic iterative optimization method and system for intelligent wind control approval strategy
Bao et al. Effect improved for high‐dimensional and unbalanced data anomaly detection model based on KNN‐SMOTE‐LSTM
CN114881349A (en) Big data analysis-based early warning method and storage medium
CN114255121A (en) Credit risk prediction model training method and credit risk prediction method
CN115809596A (en) Digital twin fault diagnosis method and device
Chowdhury et al. Internet of Things resource monitoring through proactive fault prediction
CN112418460A (en) Fault diagnosis method and fault diagnosis device for engineering vehicle
Xu et al. Bearing fault diagnosis in the mixed domain based on crossover-mutation chaotic particle swarm
Mansouri et al. Explainable fault prediction using learning fuzzy cognitive maps
Vargas et al. A hybrid feature learning approach based on convolutional kernels for ATM fault prediction using event-log data
CN115329834A (en) Cable state determination method, device and system
Ali et al. ICS-IDS: application of big data analysis in AI-based intrusion detection systems to identify cyberattacks in ICS networks
Ren et al. Triple: The Interpretable Deep Learning Anomaly Detection Framework based on Trace-Metric-Log of Microservice
e Souza et al. Development of a CNN-based fault detection system for a real water injection centrifugal pump

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant