CN113515434B

CN113515434B - Abnormality classification method, abnormality classification device, abnormality classification apparatus, and storage medium

Info

Publication number: CN113515434B
Application number: CN202110004842.4A
Authority: CN
Inventors: 黄荣庚; 周峰; 吴懿伦; 张梦妮
Original assignee: Tencent Technology Shenzhen Co Ltd
Current assignee: Tencent Technology Shenzhen Co Ltd
Priority date: 2021-01-04
Filing date: 2021-01-04
Publication date: 2024-09-10
Anticipated expiration: 2041-01-04
Also published as: CN113515434A

Abstract

The application discloses an anomaly classification method, an anomaly classification device and a storage medium, wherein the method comprises the following steps: responding to a triggering operation for carrying out anomaly classification on the target server, and acquiring target log information generated by the target server when the anomaly is generated; acquiring at least two target classification models, wherein one target classification model adopts an abnormal classification algorithm to perform abnormal classification on a target server; invoking each target classification model to carry out abnormal classification on the target server according to the target log information to obtain an abnormal classification result of each target classification model on the target server; and determining the target abnormal type of the target server according to the abnormal classification result of each target classification model. By adopting a plurality of target classification models to analyze the target log information, the abnormality classification equipment is helped to accurately obtain the abnormality of the target server, so that the real reason for the abnormality of the target server can be accurately positioned.

Description

Abnormality classification method, abnormality classification device, abnormality classification apparatus, and storage medium

Technical Field

The present invention relates to the field of computer technologies, and in particular, to an anomaly classification method, an anomaly classification device, an anomaly classification apparatus, and a computer storage medium.

Background

As society continues to develop and mature, internet traffic requires more servers to provide computing or application services. However, the server may be abnormal during operation. Therefore, in order to ensure normal execution of the internet service, it is necessary to locate the cause of the abnormality of the server in order to perform operation maintenance on the server based on the abnormality cause.

In order to locate the cause of the abnormality of the server, it is necessary to classify the abnormality server according to log information. In the existing anomaly classification scheme, the anomaly type of the anomaly server can be determined by manually analyzing the log information of the anomaly server. However, due to factors such as huge data volume of log information corresponding to an abnormal server or limited analysis capability of an analyst, manual analysis may be difficult and inefficient, and the abnormal type of the abnormal server may not be determined quickly and accurately. Therefore, how to classify an anomaly server is an important study.

Disclosure of Invention

The embodiment of the invention provides an anomaly classification method, an anomaly classification device and a storage medium, which are beneficial to the anomaly classification device to accurately obtain the anomaly of a target server by adopting a plurality of target classification models to analyze target log information, so that the real cause of the anomaly of the target server can be accurately positioned.

In one aspect, an embodiment of the present invention provides an anomaly classification method, where the anomaly classification method includes:

Responding to a triggering operation for carrying out abnormality classification on a target server, and acquiring target log information generated by the target server when abnormality occurs;

Acquiring at least two target classification models, wherein one target classification model adopts an abnormal classification algorithm to perform abnormal classification on the target server;

Invoking each target classification model to carry out abnormal classification on the target server according to the target log information to obtain an abnormal classification result of each target classification model on the target server;

and determining the target abnormal type of the target server according to the abnormal classification result of each target classification model.

In another aspect, an embodiment of the present invention provides an abnormality classification device, including:

the acquisition unit is used for responding to the triggering operation of carrying out abnormal classification on the target server and acquiring target log information generated when the target server generates the abnormality;

The acquisition unit is also used for acquiring at least two target classification models, wherein one target classification model adopts an abnormal classification algorithm to carry out abnormal classification on the target server;

The abnormal classification unit is used for calling each target classification model to perform abnormal classification on the target server according to the target log information to obtain an abnormal classification result of each target classification model on the target server;

And the determining unit is used for determining the target abnormal type of the target server according to the abnormal classification result of each target classification model.

In still another aspect, an embodiment of the present invention provides an anomaly classification device, including an input interface, an output interface, the anomaly classification device further including:

a processor adapted to implement one or more instructions; and

A computer storage medium storing one or more instructions adapted to be loaded by the processor and to perform the steps of:

In yet another aspect, embodiments of the present invention provide a computer storage medium storing one or more instructions adapted to be loaded by a processor and to perform the steps of:

In the embodiment of the invention, an anomaly classification device responds to a triggering operation for carrying out anomaly classification on a target server, acquires target log information generated by the target server when the anomaly is generated and trains by adopting an anomaly classification algorithm to obtain a plurality of target classification models; and then, a plurality of target classification models are called to carry out abnormal classification on the target servers according to the target log information. Because the abnormal classification of the target server based on the log information is realized by the target classification model constructed based on the machine learning algorithm, no analysis personnel is required to analyze the target log information of the target server one by one, so that the manual operation can be reduced, the labor cost is greatly reduced, the time cost is effectively saved, the labor resource is effectively saved, and the abnormal classification efficiency is improved. In addition, the embodiment of the invention utilizes at least two target classification models to carry out abnormal classification on the target server according to the target log information, adopts a multi-model fusion strategy, determines the abnormal type of the target server from the abnormal classification results of a plurality of target classification models by a voting mechanism, avoids the defect of a single classification model, and can further ensure the accuracy of abnormal classification of the server.

Drawings

In order to more clearly illustrate the embodiments of the invention or the technical solutions in the prior art, the drawings that are required in the embodiments or the description of the prior art will be briefly described, it being obvious that the drawings in the following description are only some embodiments of the invention, and that other drawings may be obtained according to these drawings without inventive effort for a person skilled in the art.

FIG. 1 is a schematic diagram of an anomaly classification system according to an embodiment of the present application;

FIG. 2 is a schematic flow chart of an anomaly classification method according to an embodiment of the present application;

FIG. 3 is a schematic flow chart of a method for constructing a target classification model according to an embodiment of the present application;

FIG. 4 is a schematic flow chart of a preprocessing provided by an embodiment of the present application;

FIG. 5 is a schematic diagram of a decision tree provided by an embodiment of the present application;

FIG. 6 is a schematic diagram of one class provided by an embodiment of the present application;

FIG. 7 is a graph of test results of a target classification model provided by an embodiment of the present application;

Fig. 8 is a schematic structural diagram of an abnormality classification device according to an embodiment of the present application;

fig. 9 is a schematic structural diagram of an abnormality classification device according to an embodiment of the present application.

Detailed Description

The following description of the embodiments of the present invention will be made clearly and completely with reference to the accompanying drawings, in which it is apparent that the embodiments described are only some embodiments of the present invention, but not all embodiments. All other embodiments, which can be made by those skilled in the art based on the embodiments of the invention without making any inventive effort, are intended to be within the scope of the invention.

The artificial intelligence technology is a comprehensive subject, and relates to the technology with wide fields, namely the technology with a hardware level and the technology with a software level. Artificial intelligence infrastructure technologies generally include technologies such as sensors, dedicated artificial intelligence chips, cloud computing, distributed storage, big data processing technologies, operation/interaction systems, mechatronics, and the like. The artificial intelligence software technology mainly comprises a computer vision technology, a voice processing technology, a natural language processing technology, machine learning/deep learning and other directions. Among these, machine learning (MACHINE LEARNING, ML) is a multi-domain intersection discipline that involves multiple disciplines such as probability theory, statistics, approximation theory, convex analysis, algorithm complexity theory, and the like. It is specially studied how a computer simulates or implements learning behavior of a human to acquire new knowledge or skills, and reorganizes existing knowledge structures to continuously improve own performance.

As artificial intelligence technology research and advances, artificial intelligence technology expands research and applications in a variety of fields, such as common smart homes, smart wearable devices, virtual assistants, smart speakers, smart marketing, unmanned, autopilot, unmanned, robotic, smart medical, smart customer service, etc. In addition, the artificial intelligence technology can be applied in other fields, for example, the intelligent operation and maintenance of the server can be realized by adopting machine learning in the artificial intelligence technology. The embodiment of the invention provides an anomaly classification method based on machine learning, so that anomaly classification equipment can construct a plurality of classification models through machine learning, when a server generates anomalies, the anomaly classification equipment can call the constructed plurality of classification models to perform anomaly classification on log information when the server generates anomalies, then the anomaly type of the server can be determined according to the anomaly classification result of each classification model on the log information, the error rate when the anomaly type of the server is determined can be reduced based on the classification result of different classification models on the anomaly type of the server, thereby realizing the accuracy of positioning the anomalies of the server, and meanwhile, the mode that the classification models obtained through machine learning are adopted to perform anomaly classification on the server is adopted, so that the anomaly classification equipment is beneficial to realizing intelligent operation and maintenance (ARTIFICIAL INTELLIGENCE for IT Operations, AIOps) to improve the maintenance efficiency of the server, wherein the intelligent operation and maintenance are combined with big data and machine learning technology, developed into an intelligent strategy, and the intelligent operation and maintenance strategy is integrated into an operation and maintenance system.

In one embodiment, the anomaly classification method may be used to perform anomaly classification on a server or a terminal device, so as to determine anomalies of the corresponding server or terminal device, where the anomaly classification method is used to determine anomalies of the server, the anomaly classification method may be applied in an anomaly classification system as shown in fig. 1, and as shown in fig. 1, the anomaly classification system may at least include: an abnormality classification device 11 and a server 12, wherein the abnormality classification device 11 is a device running at least two classification models, the abnormality classification device 11 may be a terminal device as shown in fig. 1, wherein the terminal device may include, but is not limited to: smart phones, tablet computers, laptop computers, wearable devices, desktop computers, etc.; or the abnormality classification device 11 may be a server, and the embodiment of the present invention is not limited. In addition, the anomaly classification system may include one or more servers 12, where the server 12 may be one of servers 12a, 12b, 12c, etc. as shown in fig. 1, or a combination of multiple servers, where the server 12 may be an independent physical server, a server cluster or a distributed system formed by multiple physical servers, a cloud server that provides cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, content distribution network (Content Delivery Network, CDN), middleware services, domain name services, security services, and basic cloud computing services such as big data and artificial intelligence platforms, etc.

The anomaly classification method has wide application scenes, and can be applied to various service platforms recorded with log information, such as cloud dial testing of a communication cloud, cloud virtual machines (Cloud Virtual Machine, CVM), cloud monitoring products and the like. And may be applicable to various types of Log information including System Log (syslog), various types of software Log information, hardware Log information, and the like.

In a specific implementation, the abnormality classification device 11 may determine whether the server 12 is abnormal based on the log information of the server 12, and then, when the abnormality classification device 11 classifies the server 12 as abnormal, the abnormality classification device may first obtain target log information generated when the server 12 generates the abnormality, where the log information is used to record user operations, a system running state, and the like, and is an important component of the system. For a backend program to serve many products as a base platform, consideration must be given to how to rely on good logs to ensure reliable operation of the system. Such as System Log (syslog) is an industry standard protocol that can be used to record logs of devices, and in network devices such as UNIX systems (an operating System), routers, and switches, syslog can record events of any size occurring in the System, so that a manager can grasp the System status at any time by looking at the System records. The system log of UNIX records the system related event record through the syslog process and also records the application program operation event, so that by proper configuration, the inter-machine communication running the syslog protocol can be realized, and the network behavior log is analyzed to track and master the conditions related to equipment and network. The log information may be, for example:

[INFO]RequestID:b1946ac92492d2347c6235b4d2611184，auth failed due to token expiration；

[INFO]RequestID:b1946ac92492d2347c6235b4d2611185,content digest does not match,expect 7b3f050bfa060b86ba781151c563c953,actual f60645e7107917250a6408f2f302d048;

[INFO]RequestID:b1946ac92492d2347c6235b4d2611186，request ip(＝202.17.34.1)not in whitelist。

After the anomaly classification device 11 obtains the target log information generated when the server 11 generates the anomaly, at least two trained target classification models can be obtained, so that each target classification model can be called to perform anomaly classification on the target server by adopting a corresponding anomaly classification algorithm, specifically, the anomaly classification device 11 can classify the target log information by adopting a corresponding anomaly classification algorithm in each target classification model, so as to obtain a voting result of each target classification model on anomaly classification of the target server, and further, the target anomaly type of the target server can be determined based on the voting result of each target classification model on anomaly classification. If the anomaly classification device calls three target classification models to classify and vote on the collected target log information, if the three target classification models are a model A, a model B and a model C respectively, if the model A and the model B in the three target classification models determine that the target server belongs to the class 1 anomaly according to the target log information, and the model C determines that the target server belongs to the class 2 anomaly according to the target log information, the anomaly classification device determines that the target server belongs to the class 1 anomaly according to the results of the three target classification models. By adopting a plurality of target classification models to analyze target log information, the abnormality classification equipment is helped to accurately obtain the abnormality of the target server, so that the real reason for the abnormality of the target server can be further positioned, corresponding parts can be replaced according to the reason for the abnormality of the target server, and the abnormality rate of each part is counted.

Fig. 2 is a schematic flow chart of an anomaly classification method according to an embodiment of the present invention. As shown in fig. 2, the abnormality classification method includes steps S201 to S204:

s201, in response to triggering operation of carrying out abnormality classification on the target server, acquiring target log information generated by the target server when abnormality occurs.

In one embodiment, the target server is any server which establishes communication connection with the abnormality classification device, wherein when an abnormality occurs, the target server reports the abnormality to the abnormality classification device, specifically, the target server can send work order information to the abnormality classification device, so that the abnormality classification device determines target log information generated when the target server generates the abnormality based on the work order information, and determines a target abnormality type of the target server according to the target log information; correspondingly, when the abnormality classification device acquires the work order information from the target server, determining that the triggering operation for performing abnormality classification on the target server is detected, wherein the work order information carries a work order number (ticket_id), and the work order information is corresponding to the target server based on the work order number; the work order information records time information of an abnormality of the target server, and based on the time information, the abnormality classification device may acquire target log information generated when the abnormality is generated by the target server. In another embodiment, the abnormality classification device may also determine that a triggering operation for performing abnormality classification on the target server is detected when an alarm message from the target server is detected, where the target server will perform alarm reminding when the target server generates an abnormality, where the alarm message will be sent to a responsible person (i.e. a person maintaining the target server) when the target server performs alarm reminding, and then the abnormality classification device may determine that a triggering operation for performing abnormality classification on the target server is detected when the target server determines that the target server sends the alarm message to the responsible person, where the alarm message (may also be referred to as a notification message) may be sent when the target server generates a basic alarm (e.g. a central processing unit (Central Processing Unit, a CPU) has too high usage amount, etc.), or when the target server generates a performance index alarm (e.g. a user access success rate is low, etc.), and may specifically send the alarm message to the responsible person through software, such as a micro-letter, a small program, or a micro-letter, etc.

In one embodiment, the log information generated when the target server generates the abnormality includes multiple types of log information, and the obtained target log information generated when the target server is abnormal and provided by the embodiment of the present invention is log information that facilitates abnormality classification of the target server by the abnormality classification device, that is, the obtained target log information of the target server is of a specific type and facilitates abnormality positioning (or abnormality classification), where, for example, the target log information may be:

① Log information for recording kernel information (i.e., dmesg log information), wherein dmesg log information is used for recording the last screen before the target server is down, and the last screen contains netconsole data (data for indicating whether the kernel has a fault or not);

② Log information (i.e., mcelog log information) for recording hardware information, wherein mcelog log information is used for recording an abnormality caused by a hardware error;

③ A system event log (i.e., sel log information), wherein sel log information is a log generated when a target server sensor collects data and finds an abnormality.

After the anomaly classification device obtains the target log information generated when the target server is anomalous, the trained target classification model may be used to determine the category of the anomaly to which the target server belongs according to the target log information, that is, step S202 is performed instead.

S202, acquiring at least two target classification models, wherein one target classification model adopts an abnormal classification algorithm to perform abnormal classification on a target server.

S203, each target classification model is called to conduct abnormal classification on the target server according to the target log information, and an abnormal classification result of each target classification model on the target server is obtained.

In step S202 and step S203, the anomaly classification algorithm includes, but is not limited to, one or more of the following: decision Tree (DT) algorithm, rocchio algorithm, extreme gradient lifting (Xtreme Gradient Boosting, XGBooste) algorithm, naive Bayes (NB) algorithm, linear discriminant analysis (LINEAR DISCRIMINANT ANALYSIS, LDA), support vector machine (Support Vector Machine, SVM) algorithm, random Forest (RF) algorithm, and logistic regression (Logistic Regression, LR) algorithm. The DT algorithm is a machine learning algorithm for constructing a tree structure, each non-leaf node of the constructed decision tree is used for representing a test on a characteristic attribute, each branch of the constructed decision tree represents the output of the characteristic attribute, and each leaf node of the constructed decision tree is used for representing the result of abnormal classification. The Rocchio algorithm is a text classification algorithm that determines outlier classification results mainly from cosine similarity between vectors. The GBoost algorithm is an integrated machine learning algorithm, and mainly determines abnormal classification results through a plurality of mutually associated classification regression trees. The inter-related classification regression tree refers to that the generated (i+1) th classification regression tree is related to a training result and a testing result of the i th classification regression tree, wherein i is a positive integer greater than 0. The NB algorithm is a classification algorithm based on bayesian theorem, and determines an abnormal classification result mainly from probabilities obtained by the maximum likelihood estimation algorithm. The LDA algorithm mainly determines an abnormal classification result by the distance obtained by the distance algorithm. The SVM algorithm is a classification model algorithm, and an abnormal classification result is determined through an optimal hyperplane constructed by a kernel function. The RF algorithm is also an integrated machine learning algorithm that determines outlier classification results primarily through multiple independent decision trees. The LR algorithm is a conventional classification model algorithm that can also be applied to multi-classification tasks. The method mainly comprises the steps of dividing a multi-classification task into a plurality of classification tasks, training a model for each classification task, and determining an abnormal classification result according to classification results of a plurality of models.

In one embodiment, the anomaly classification device may convert the target log information into a vector expression via vector conversion. And then, each target classification model in the at least two target classification models is called to carry out abnormal classification on the vector expression corresponding to the target log information, so that the target class to which the vector expression belongs can be determined from a plurality of classes to be selected. Then, the abnormal type indicated by the target category is used as an abnormal classification result of each target classification model on the target server. Since text vectorization is performed on the target log information of the target server (i.e., the target log information is converted into a vector expression of the target log information), and the target classification model performs abnormal classification on the vector expression corresponding to the target log information, the simple one-to-one correspondence between the target log information and the abnormal type is no longer required, and the accuracy of the method for performing abnormal classification on the vector expression corresponding to the target log information by adopting the target classification model is higher.

The process of the anomaly classification device for carrying out anomaly classification on the target server according to the target log information is a process of the anomaly classification device for analyzing the target log information by adopting a corresponding anomaly classification algorithm so as to determine the anomaly class to which the target server belongs. The exception classification results obtained by each object classification model performing exception classification on the object server according to the object log information may be the same or different, and after the exception classification device obtains the exception classification result of each object classification model on the object server, the object exception type of the object score may be determined based on the exception classification result of each object classification model, that is, step S204 is performed instead.

In one embodiment, if the anomaly classification device performs anomaly classification on the target server by using the DT algorithm, the anomaly classification device may first convert the target log information into a vector expression corresponding to the target log information. And then testing the vector expression from the root node of the constructed decision tree, selecting an output branch according to the output of the vector expression until the output branch reaches the leaf node of the decision tree, and determining the class corresponding to the leaf node as the target class to which the vector expression belongs. And then, taking the abnormal type indicated by the target category as an abnormal classification result of the target server by a target classification model constructed by the DT algorithm.

In one embodiment, if the anomaly classification device uses the Rocchio algorithm to perform anomaly classification on the target server, the anomaly classification device may first convert the target log information into a vector expression corresponding to the target log information. And then, respectively calculating the cosine similarity of the vector expression and the sample vector corresponding to the log samples in each log sample subset, and determining the class corresponding to the sample vector with the highest cosine similarity as the target class to which the vector expression belongs. Then, the anomaly type indicated by the target category is used as an anomaly classification result of the target server by a target classification model constructed by the Rocchio algorithm. Or if the anomaly classification device adopts XGBoost algorithm to perform anomaly classification on the target server, the anomaly classification device can convert the target log information into a vector expression corresponding to the target log information. And then inputting the vector expression into an ith classification regression tree to obtain an ith numerical value, and inputting the ith numerical value into an (i+1) th classification regression tree to obtain an (i+1) th numerical value. After the values output by the multiple classification regression trees are obtained, the output values of the multiple classification regression trees which are mutually related can be operated by utilizing an optimal algorithm to obtain target values. And finally, determining the target category of the vector expression according to the value range of the target value. And taking the abnormal type indicated by the target category as an abnormal classification result of the target server by a target classification model constructed by XGBoost algorithm.

In one embodiment, if the anomaly classification device adopts the NB algorithm to perform the anomaly classification on the target server, the anomaly classification device may first convert the target log information into a vector expression corresponding to the target log information, then respectively calculate the bayesian probabilities of the vector expression and sample vectors corresponding to the log samples in each log sample subset using the maximum likelihood estimation algorithm, and determine the class to be selected corresponding to the sample vector with the highest probability as the target class to which the vector expression belongs. And taking the abnormal type indicated by the target category as an abnormal classification result of the target server by a target classification model constructed by an NB algorithm.

In one embodiment, if the anomaly classification device adopts the LDA algorithm to perform the anomaly classification on the target server, the anomaly classification device may first convert the target log information into a vector expression corresponding to the target log information. And then, respectively calculating the distance between the vector expression and the sample vector corresponding to the log sample in each log sample subset by using a distance calculation method, and determining the class to be selected corresponding to the sample vector closest to the sample vector as the target class to which the vector expression belongs. And taking the abnormal type indicated by the target category as an abnormal classification result of the target server by a target classification model constructed by an LDA algorithm. Methods of calculating distance may include, but are not limited to, mahalanobis distance.

In one embodiment, if the anomaly classification device performs anomaly classification on the target server by using an SVM algorithm, the anomaly classification device may first convert the target log information into a vector expression corresponding to the target log information, then determine a relative position of the vector expression and the optimal hyperplane in the high-dimensional space, and if the vector expression is located on the first side of the optimal hyperplane, determine a class to be selected indicated by the sample vector in the first side as a target class to which the vector expression belongs. If the vector expression is positioned at the other side of the optimal hyperplane, determining the class to be selected indicated by the sample vector in the other side as the target class to which the vector expression belongs. And taking the abnormal type indicated by the target category as an abnormal classification result of the target server by a target classification model constructed by an SVM algorithm.

In one embodiment, if the anomaly classification device uses an RF algorithm to classify the anomaly of the target server, the anomaly classification device may first convert the target log information into a vector expression corresponding to the target log information. And then, classifying the vector expression by adopting a plurality of decision trees to obtain a plurality of reference sample vectors, determining a sample vector from the plurality of reference sample vectors based on a voting mechanism, and determining the class to be selected corresponding to the sample vector as the target class to which the vector expression belongs. And taking the abnormal type indicated by the target category as an abnormal classification result of the target server by a target classification model constructed by an RF algorithm.

In one embodiment, if the anomaly classification device performs anomaly classification on the target server by using an LR algorithm, the anomaly classification device may first convert the target log information into a vector expression corresponding to the target log information. And then inputting the vector expression of the target log information into training models of the two classification tasks, and integrating classification results of the models to obtain the target class to which the vector expression belongs.

Each of the object classification models mentioned herein has a different effect on the anomaly classification of the object server. Some object classification models have high accuracy in classifying the abnormality of the object server, and some object classification models have low accuracy in classifying the abnormality of the object server. The recall rate of the abnormal classification of the target servers by some target classification models is high, and the recall rate of the abnormal classification of the target servers by some target classification models is low. Therefore, the embodiment of the invention adopts a plurality of target classification models to carry out abnormal classification on the target server according to the target log information, so that the defect of a single target classification model can be avoided, and the accuracy and recall rate of abnormal classification are further ensured.

S204, determining the target abnormal type of the target server according to the abnormal classification result of each target classification model.

The abnormal classification device calls at least two target classification models to perform abnormal classification on vector expressions corresponding to the target log information, and calls different target classification models to perform abnormal classification on the target log information to obtain abnormal classification results which may be the same or different, that is to say, the abnormal classification device obtains a plurality of abnormal classification results aiming at the target server from each target classification model, and after the abnormal classification device obtains corresponding abnormal classification results from each target classification model, most models can be utilized to determine the abnormal type of the target server from the abnormal classification results of the target server. In one embodiment, if the abnormal classification results of each target classification model on the target server are different, the abnormal classification device may select the abnormal classification result of any target classification model on the target server, and use the abnormal type indicated by the selected abnormal classification result as the target abnormal result of the target server. For example, if the four object classification models are model a, model B, model C, and model D, respectively, if model a of the four object classification models determines that the object server belongs to the class 1 anomaly based on the object log information, model B determines that the object server belongs to the class 2 anomaly based on the object log information, model C determines that the object server belongs to the class 3 anomaly based on the object log information, and model D determines that the object server belongs to the class 4 anomaly based on the object log information, the anomaly classification device determines that the object server belongs to the class 1 anomaly, the class 2 anomaly, the class 3 anomaly, or the class 4 anomaly based on the results of the four object classification models. Most of these may also be a proportional threshold, which may be set empirically or by business requirements, which may be 80%,90%, etc.

In another embodiment, if there are at least two target classification models with the same abnormal classification result for the target server, a maximum of the same number of reference classification results may be determined from the abnormal classification results for the target server by each target classification model; and taking the selected abnormal types indicated by the reference classification results with the maximum number of the same as the target abnormal results of the target server. For example, if the four object classification models are model a, model B, model C, and model D, respectively, if model a, model B, and model C of the four object classification models determine that the object server belongs to the class 1 anomaly based on the object log information, and model D determines that the object server belongs to the class 2 anomaly based on the object log information, the anomaly classification device determines that the object server belongs to the class 1 anomaly based on the results of the four object classification models.

Further, if it is determined from the abnormal classification results of each target classification model on the target server that the number of the reference classification results of the same number is a plurality of at most, the abnormal type indicated by any one of the reference classification results may be used as the target abnormal result of the target server. For example, if the four object classification models are model a, model B, model C, and model D, respectively, if model a, model B of the four object classification models determine that the object server belongs to the class 1 anomaly based on the object log information, and model C and model D determine that the object server belongs to the class 2 anomaly based on the object log information, the anomaly classification device determines that the object server belongs to the class 1 anomaly or the class 2 anomaly based on the results of the four object classification models.

In one embodiment, in order to more quickly determine the target anomaly type of the target server, the anomaly classification device may further determine an association relationship between the target log information and the target anomaly type by using an association rule mining algorithm after determining the target anomaly type of the target server, and store the association relationship. The association relationship indicates the probability that the target server is of the target abnormal type when the target server generates the target log information, so that when the target server generates the target log information again, the abnormal classification equipment can not call the target classification model to perform abnormal classification, and the target abnormal type of the target server can be determined directly according to the probability in the association relationship. Or the anomaly classification device may also determine the target anomaly type of the target server according to the anomaly classification result of each target classification model on the target server and the probability in the association relation. The rate and accuracy with which the anomaly classification device determines the target anomaly type for the target server can be improved.

In one embodiment, after determining the target exception type of the target server, the exception classification device may further determine whether an associated exception type related to the target exception type exists, and if so, take the associated exception type as the predicted exception type of the target server. The associated exception type is an exception type generated according to the target exception after the target exception of the target exception type is generated. Because there is a correlation between the associated anomaly type and the target anomaly type, when the anomaly classification device determines that the target server has generated an anomaly of which the anomaly type is the target anomaly type, the anomaly classification device may guess that the target server may generate an anomaly of which the anomaly type is the associated anomaly type in a subsequent time of generating the target anomaly type. The data of the target server can be processed in a real-time streaming manner, and the prediction of the associated abnormal type of the target abnormal type is realized, so that the associated abnormal type of the target server can be prevented, and the user experience is further improved.

Referring to the above description of the method embodiment shown in fig. 2, the anomaly classification method shown in fig. 2 can implement anomaly classification for the anomaly server by calling the objective classification model. Each target classification model is a classification function/classification model constructed based on the existing log information set, namely a decision surface. Referring to fig. 3, a schematic flow chart of constructing a target classification model is provided in an embodiment of the present invention. As shown in fig. 3, the construction process of the object classification model mainly includes a model training part shown in the left side diagram of fig. 3 and a model testing part shown in the right side diagram of fig. 3. As shown in fig. 3, the process of constructing the object classification model may include steps S301-S304:

S301, acquiring a log information set when a server generates an abnormality, and preprocessing the log information set to obtain a log sample set, wherein the preprocessing comprises one or more of the following steps: data screening, data cleaning and text segmentation.

When the server generates an abnormality, the server reports the abnormality to the abnormality classification device, and the server can send work order information to the abnormality classification device so that the abnormality classification device can determine log information generated when the server generates the abnormality based on the work order information, and the abnormality classification device can acquire the log information generated when the server generates the abnormality in a time period to obtain a log information set. The time period may be a variable time period, such as 10 months in 2020 to 11 months in 2020, five days nearest the current day, or one month nearest the current month, etc. The server may be one server or a combination of multiple servers, which is not limited in the embodiment of the present invention.

After the log information set is obtained, the abnormality classification device may divide the log information into a training log information set and a test log information set through a data division operation. Wherein the data partitioning operation may include a random hierarchical sampling approach. The training log information set is used for constructing a target classification model, and the test log information set is used for verifying the model effect of the target classification model. For example, 70% of the log information set may be used as the test training log information set and 30% of the log information set may be used as the test log information set.

In one embodiment, after obtaining the training log information set and the test log information set, the anomaly classification device may perform preprocessing on the training log information set to obtain a log sample set for training, and perform preprocessing on the test log information set to obtain a log sample set for testing.

The process of preprocessing the training log information set to obtain a log sample set for training will be described in detail below. Fig. 4 is a schematic flow chart of a pretreatment method according to an embodiment of the present invention. The method mainly comprises a data screening s11, a data cleaning s12 and a data balancing s13.

Specifically, the anomaly classification device performs data screening on the training log sample set, then performs data cleaning, and judges whether the cleaned data are balanced, if not, data balancing is needed. After data balancing, a set of log samples for training may be obtained, which may be used to construct a target classification model and to perform anomaly analysis on the target server based on the target classification model.

S11, data screening

Aiming at the log information in the training log information set, the log information generated when the server acquired by the embodiment of the invention is abnormal is the log information which is beneficial to the abnormality classification of the server by the abnormality classification equipment, namely the log information of the server acquired by the embodiment of the invention is of a specific type and is beneficial to the abnormality positioning (or abnormality classification). However, the log information obtained by the abnormality classification device at the time of server abnormality may include log information not belonging to the specific type described above. Such as a boot log of a server, etc. The anomaly classification device needs to filter out log information belonging to a specific type from the training log information set. Wherein the specific type of log information may be set according to experience or business requirements.

S12, data cleaning

Aiming at the log information in the training log information set, the data cleaning is mainly used for cleaning useless information contained in the log information. For example, the anomaly classification device may clear special symbols (e.g., ' # ', ' < ',' > ', ' and, ' @ ', ' - | ',' (', ') ', ' _ ', etc.) in the log information, and for example, the anomaly classification device may clear format information (e.g., digital format, english format, etc.) in the log information, and for example, the anomaly classification device may clear stop words (e.g., ' is ', ' not ', ' this ', ' the ', ' do ', ' in ', etc.) in the log information. It should be clear that if the formats of the log information are inconsistent, the anomaly classification device needs to separately clean the log information in different formats.

S13, data balancing s13

The abnormality classification device determines log information generated when the server generates an abnormality based on the work order information. When orders of work order information of different categories are inconsistent, namely, the quantity of various anomaly types corresponding to log information is large in difference, the anomaly classification result of the target classification model is biased to the anomaly type with the large quantity. Considering the problem of the number imbalance of various exception types, a smaller number of exception types may be classified into one class (e.g., other classes). As shown in fig. 5, the exception types of the server may include nine exception types, such as a switch failure, a burst failure, a motherboard failure, a network card failure, an artificial reason, a disk array card failure, a kernel-related failure, a processor failure, and a memory failure. The number of the switch faults and the burst faults is 1, the number of the main board faults and the network card faults is 2, the number of the human reasons and the number of the disk array card faults is 3, the number of the abnormal types is relatively small, and six abnormal types including the switch faults, the burst faults, the main board faults, the network card faults, the human reasons and the disk array card faults can be classified into one type. However, the number of the related faults of the core is 15, the number of the faults of the processor is 20, the number of the faults of the memory is 85, and the number of the abnormal types is relatively large, so that the related faults of the core can be respectively taken as two types, the faults of the processor can be respectively taken as three types, and the faults of the memory can be respectively taken as four types. The nine-classification problem is converted into a four-classification problem. Under the condition that the quantity of the work order information is unbalanced or the abnormal types are extremely large, the method for classifying the abnormal types with small quantity into one type can enable the target classification model to have strong randomness.

In some embodiments, the anomaly classification device also requires text word segmentation of the log information. The anomaly classification device may perform text segmentation on the log information based on a dictionary segmentation algorithm (such as a forward maximum matching method, a reverse maximum matching method, a bi-directional matching segmentation method, etc.), or may perform text segmentation on the log information based on a statistical machine learning algorithm (such as a hidden markov model, a conditional random field model, an SVM algorithm, a deep learning algorithm, etc.). The embodiments of the present invention are not limited in this regard.

It should be understood that, in order to ensure that the formats of the log sample set for training and the log sample set for testing are uniform, the exception classification device needs to preprocess the log information in the test log information set to obtain the log sample set for testing, and the method of preprocessing the log information in the test log information set by the exception classification device is the same as the method of preprocessing the log information in the training log information set by the exception classification device, and specific implementation of preprocessing the test log information by the exception classification device can refer to steps s11-s13. And will not be described in detail here.

S302, acquiring a log sample set, performing vector conversion on each log sample in the log sample set, and determining a sample vector corresponding to each log sample in the log sample set.

The log samples in the log sample set are usually text types, and when a target classification model is constructed, the text type log samples need to be converted into a sample vector of a numerical value type corresponding to each log sample. Wherein the log sample set comprises a log sample set for training or a log sample set for testing.

Alternatively, the sample vector of log samples may be obtained by means of manual labeling. Alternatively, the log samples may be converted into sample vectors by a weight calculation method. The weight calculation method may include any one or more of the following: boolean weight (Boolean vector) method, frequency weight (Term frequence) method, and Term Frequency-inverse file Frequency (Term Frequency-Inverse Document Frequency, tf-idf) method.

Among them, the boolean weight method is the simplest weight calculation method. If a feature word appears in the log sample, the weight of the feature word is 1; if a feature word does not appear in the log sample, its weight is 0. The method is easy to lose the internal information of the log sample, and the effect is slightly poor. But is applicable to some models that employ two classifications, such as decision trees or probabilistic classifiers.

The frequency weighting method is the most intuitive weight calculation method. The frequency of occurrence of a feature word in the log sample is the frequency weight. The idea of this approach is that the more feature words that occur, the greater their importance. Specifically, the frequency weighting method can be characterized by the following expression:

word frequency = number of occurrences of feature words in log samples

The word frequency-inverse file frequency method is the most widely applied weight calculation method. The more feature words appear in the log sample and the fewer the number of occurrences in the log sample set, the more feature words can represent the anomaly type. The main idea is that if the log sample number containing the characteristic word t is smaller, the frequency of the reverse file is larger, the characteristic word t is proved to have good type distinguishing capability. The term frequency-inverse document frequency method can be characterized by the following expression:

word frequency = number of occurrences of feature words in log samples

Word frequency-inverse frequency = word frequency-inverse frequency

In one embodiment, text vectorization may be performed using a word frequency-inverse document frequency method, regularization may be selected 12, feature selection may be performed in combination with document frequency (df) and maximum word frequency (tf), and feature words of the log sample may be selected.

It is to be understood that, for some types of anomalies with fewer numbers, log information corresponding to the existing work order information can be analyzed through expert knowledge, feature words are manually extracted from the log information, and corresponding modules are written to realize automatic classification.

S303, classifying the log sample set according to the sample vector corresponding to each log sample to obtain log sample subsets under different categories, wherein the sample vector corresponding to the log sample in the log sample subset under one category corresponds to one category to be selected.

And constructing a target classification model by adopting an abnormal classification algorithm, and continuously performing parameter tuning in the construction process to construct an optimal target classification model. Specifically, the abnormal classification device may classify the sample vector corresponding to each log sample in the log sample set by using the initial classification model to obtain the log sample subsets of different categories, and update the initial parameters of the initial classification model according to the classification result. The target classification model can be obtained through training after multiple updates. The sample vector corresponding to the log sample contained in the log sample subset of one category corresponds to one category to be selected.

The decision tree is described as an example. Training the log sample set by using a decision tree algorithm can obtain a decision tree as shown in fig. 6. As shown in fig. 6, the decision tree includes a total of 11 nodes, including 5 non-leaf nodes, 6 leaf nodes, as shown in fig. 6, non-leaf nodes, which may be represented by solid line boxes, leaf nodes, which may be represented by dashed line boxes. One sample vector is [32, 41, 33], and 6 types of log sample subsets can be obtained respectively through classification processing of the decision tree.

It should be appreciated that the constituent object classification model may be constructed by other machine learning algorithms in addition to decision tree algorithms. In some embodiments, machine learning algorithms may also be used in combination with expert rules to classify anomalies. Specifically, an expert rule policy layer can be added into the constructed model to optimize the classification model, so as to realize a better classification effect.

And repeating the step S303, and training the log sample set by adopting at least two abnormal classification algorithms to obtain at least two classification models.

S304, testing the at least two target classification models by using the log samples for testing, and analyzing the test results of the target classification models.

Referring to fig. 7, fig. 7 shows test results of the object classification model corresponding to each anomaly classification algorithm. As can be seen from fig. 7, the accuracy of the target classification model constructed by the DT algorithm in the log sample set for training can reach 98.94%, and the accuracy in the log sample set for testing can reach 90.24%, so that the target classification model has good interpretability and good classification effect. The target classification model constructed by the Rocchio algorithm is stable and has good classification effect, but can misclassify the processor fault into the memory fault. The object classification model constructed by XGBoost algorithm is racing level and classification is good, but the kernel correlation may be misclassified with other faults or the processor fault may be misclassified with memory fault. The target classification model constructed by the NB algorithm is probability-based, the classification effect is general, and the target classification model has a small applicable range. The target classification model constructed by the LDA algorithm has good classification effect, but is easy to fit and easy to misclassify the abnormal types. The target classification model constructed by the SVM algorithm has excellent performance in other classification schemes, but has poor abnormal classification effect. The target classification model constructed by the RF algorithm is relatively stable and has good classification effect. But easily misclassifies processor faults into memory faults. The target classification model constructed by the LR algorithm is the simplest and has good classification effect. But easily misclassifies processor faults into memory faults.

According to the embodiment of the invention, by means of a machine learning method, the log data causing server abnormality can be learned, useless information in the log information is filtered, and a machine learning classification model, namely a target classification model, is constructed through characteristic words of the log information. When the accuracy rate of the abnormal classification of the target classification model is accurate enough, the automatic statement of the work order information corresponding to the target server can be realized without manual participation, the abnormality of the target server is automatically analyzed, and the abnormal type of the target server is accurately positioned.

Based on the description of the above embodiment of the abnormality classification method, the embodiment of the present invention also discloses an abnormality classification device that may be a computer program (including program code) that is run in the above-mentioned abnormality classification apparatus. The abnormality classification device may perform the method shown in fig. 2 or fig. 3. Referring to fig. 8, the abnormality classification device may operate as follows:

an obtaining unit 801, configured to obtain target log information generated when an abnormality occurs in a target server in response to a trigger operation for performing an abnormality classification on the target server;

The obtaining unit 801 is further configured to obtain at least two target classification models, where one target classification model uses an anomaly classification algorithm to perform anomaly classification on the target server;

The anomaly classification unit 802 is configured to invoke each target classification model to perform anomaly classification on the target server according to the target log information, so as to obtain an anomaly classification result of each target classification model on the target server;

a determining unit 803, configured to determine a target anomaly type of the target server according to an anomaly classification result of each target classification model.

In one embodiment, the anomaly classification unit 802 invokes each target classification model to perform anomaly classification on the target server according to the target log information, so as to obtain an anomaly classification result of each target classification model on the target server, including:

Determining a vector expression corresponding to the target log information, calling each target classification model to carry out abnormal classification on the vector expression corresponding to the target log information, and determining a target class to which the vector expression belongs;

and taking the abnormal type indicated by the target category as an abnormal classification result of each target classification model on the target server.

In yet another embodiment, the vector expression includes a plurality of categories to be selected, and the determining unit 803 is further configured to determine the plurality of categories to be selected, including:

acquiring a log sample set, performing vector conversion on each log sample in the log sample set, and determining a sample vector corresponding to each log sample in the log sample set; the log sample set comprises log samples which are obtained when one server generates an exception;

And classifying the log sample set according to the sample vector corresponding to each log sample to obtain log sample subsets under different categories, wherein the sample vector corresponding to the log sample in the log sample subset under one category corresponds to one category to be selected.

In another embodiment, if each object classification model is different from the abnormal classification result of the object server; the determining unit 803 determines a target anomaly type of the target server according to an anomaly classification result of each target classification model, including:

And selecting an abnormal classification result of any target classification model on the target server, and taking the abnormal type indicated by the selected abnormal classification result as a target abnormal result of the target server.

In another embodiment, if at least two target classification models exist, the abnormal classification results of the target servers are the same; the determining unit 803 determines a target anomaly type of the target server according to an anomaly classification result of each target classification model, including:

Determining at most the same number of reference classification results from the abnormal classification results of each target classification model on the target server;

And taking the selected abnormal types indicated by the reference classification results with the same number at most as the target abnormal results of the target server.

In yet another embodiment, after determining the target anomaly type of the target server, the determining unit 803 is further configured to:

Adopting an association rule mining algorithm to determine an association relationship between the target log information and the target abnormal type, and storing the association relationship;

The association relationship is used for indicating the probability that the target server is of a target abnormal type when the target server generates the target log information.

In yet another embodiment, the determining unit 803 is further configured to:

Judging whether an associated exception type related to the target exception type exists or not, wherein the associated exception type is an exception type generated according to the target exception after the target exception of the target exception type is generated;

If yes, the associated exception type is used as a predicted exception type for the target server.

According to one embodiment of the present invention, the steps involved in the method of fig. 2 or 3 may be performed by the units of the anomaly classification device of fig. 8. For example, steps S201 and S202 shown in fig. 2 are performed by the acquisition unit 801 shown in fig. 8, step S203 is performed by the abnormality classification unit 802 shown in fig. 8, and step S204 is performed by the determination unit 803 shown in fig. 8. As another example, S301, S302, S303, and S304 in the step of fig. 3 are performed by the abnormality classification unit 802 shown in fig. 8.

According to another embodiment of the present invention, each unit in the abnormality classification device shown in fig. 8 may be separately or completely combined into one or several other units, or some unit(s) thereof may be further split into a plurality of units with smaller functions, which may achieve the same operation without affecting the implementation of the technical effects of the embodiments of the present invention. The above units are divided based on logic functions, and in practical applications, the functions of one unit may be implemented by a plurality of units, or the functions of a plurality of units may be implemented by one unit. In other embodiments of the present invention, the anomaly-based classification device may also include other units, and in actual applications, these functions may also be implemented with assistance from other units, and may be implemented by cooperation of multiple units.

According to another embodiment of the present invention, the processing elements and the storage elements may be implemented by including a central processing unit (CentralProcessing Unit, CPU), a random access storage medium (RAM), a read only storage medium (ROM), or the like. A general-purpose computing device such as a computer, runs a computer program (including program code) capable of executing steps involved in the respective methods as shown in fig. 2 or 3 to construct an abnormality classification device as shown in fig. 8, and to implement the abnormality classification method of the embodiment of the invention. The computer program may be recorded on, for example, a computer-readable recording medium, and loaded into and run in the abnormality classification device described above via the computer-readable recording medium.

In the embodiment of the present invention, in response to a triggering operation for performing an anomaly classification on a target server, the obtaining unit 801 may obtain target log information generated when the anomaly is generated by the target server, and train to obtain a plurality of target classification models by using an anomaly classification algorithm; the anomaly classification unit 802 may then invoke a plurality of object classification models to perform anomaly classification on the object server according to the object log information, and the determination unit 803 may determine the anomaly type of the object server according to the anomaly classification result of each object classification model. Because the abnormal classification of the target server based on the log information is realized by the target classification model constructed based on the machine learning algorithm, no analysis personnel is required to analyze the target log information of the target server one by one, so that the manual operation can be reduced, the labor cost is greatly reduced, the time cost is effectively saved, the labor resource is effectively saved, and the abnormal classification efficiency is improved. In addition, the embodiment of the invention utilizes at least two target classification models to carry out abnormal classification on the target server according to the target log information, adopts a multi-model fusion strategy, determines the abnormal type of the target server from the abnormal classification results of a plurality of target classification models by a voting mechanism, avoids the defect of a single classification model, and can further ensure the accuracy of abnormal classification of the server.

Based on the description of the embodiment of the anomaly classification method, the embodiment of the invention also discloses anomaly classification equipment. Referring to fig. 9, the abnormality classification device includes at least a processor 901, an input interface 902, an output interface 903, and a computer storage medium 904, which may be connected by a bus or other means.

The computer storage medium 904 is a memory device in the abnormality classification device for storing programs and data. It will be appreciated that the computer storage medium 904 herein may include a built-in storage medium of the abnormality classification device, or may include an extended storage medium supported by the abnormality classification device. The computer storage medium 904 provides a storage space that stores the operating system of the abnormality classification device. Also stored in this memory space are one or more instructions, which may be one or more computer programs (including program code), adapted to be loaded and executed by the processor 901. Note that the computer storage medium herein may be a high-speed RAM memory; optionally, the processor may be referred to as a central processing unit (CentralProcessing Unit, CPU), which is a core of the anomaly classification device and a control center, and may be at least one computer storage medium remote from the foregoing processor, and is adapted to be implemented with one or more instructions, specifically loaded and executed, so as to implement a corresponding method flow or function.

In one embodiment, one or more instructions stored in the computer storage medium 904 may be loaded and executed by the processor 901 to implement the steps involved in performing the corresponding method as shown in fig. 2 or 3, in a specific implementation, the one or more instructions in the computer storage medium 904 are loaded and executed by the processor 901 to:

In still another embodiment, the processor 901 invokes each object classification model to perform abnormal classification on the object server according to the object log information, to obtain an abnormal classification result of each object classification model on the object server, including:

In yet another embodiment, the vector expression includes a plurality of categories to be selected; the processor 901 determining a plurality of categories to be selected includes:

In another embodiment, if each object classification model is different from the abnormal classification result of the object server; processor 901 determines a target anomaly type for the target server based on anomaly classification results for each target classification model, comprising:

In another embodiment, if at least two target classification models exist, the abnormal classification results of the target servers are the same; processor 901 determines a target anomaly type for the target server based on anomaly classification results for each target classification model, comprising:

In yet another embodiment, the processor 901 is further configured to:

It should be noted that the embodiments of the present invention also provide a computer program product or a computer program, which includes computer instructions stored in a computer-readable storage medium. The processor of the anomaly classification device reads the computer instructions from the computer readable storage medium and executes the computer instructions to cause the anomaly classification device to perform the steps performed in fig. 2 or 3 of the anomaly classification method embodiment described above.

The above disclosure is only a preferred embodiment of the present invention, and it should be understood that the scope of the invention is not limited thereto, and those skilled in the art will appreciate that all or part of the procedures described above can be performed according to the equivalent changes of the claims, and still fall within the scope of the present invention.

Claims

1. An anomaly classification method, comprising:

determining the target abnormal type of the target server according to the abnormal classification result of each target classification model;

Wherein after the determining the target anomaly type of the target server, the method further comprises:

2. The method of claim 1, wherein the invoking each object classification model to perform an anomaly classification on the object server according to the object log information to obtain an anomaly classification result of each object classification model on the object server comprises:

3. The method of claim 2, wherein the vector expression comprises a plurality of categories to be selected; the determining manner of the plurality of categories to be selected comprises the following steps:

4. The method of claim 1, wherein if each object classification model differs from the object server's anomaly classification result; the determining the target abnormal type of the target server according to the abnormal classification result of each target classification model comprises the following steps:

5. The method of claim 1, wherein if there are at least two object classification models that have the same abnormal classification result for the object server; the determining the target abnormal type of the target server according to the abnormal classification result of each target classification model comprises the following steps:

6. The method of claim 1, wherein after the determining the target anomaly type for the target server, the method further comprises:

7. An anomaly classification device, the device comprising:

The determining unit is used for determining the target abnormal type of the target server according to the abnormal classification result of each target classification model;

The determining unit is further used for judging whether an associated exception type related to the target exception type exists or not after determining the target exception type of the target server, wherein the associated exception type is an exception type generated according to the target exception after generating the target exception of the target exception type; if yes, the associated exception type is used as a predicted exception type for the target server.

8. An anomaly classification device, comprising an input interface, an output interface, characterized in that it further comprises:

a processor adapted to implement one or more instructions; and

Computer storage medium storing one or more instructions adapted to be loaded by the processor and to perform the anomaly classification method of any one of claims 1-6.

9. A computer storage medium storing one or more instructions adapted to be loaded by a processor and to perform the anomaly classification method of any one of claims 1-6.

10. A computer program product comprising computer programs/instructions which, when executed by a processor, implement the steps of the anomaly classification method of any one of claims 1 to 6.