CN116304910A

CN116304910A - Anomaly detection method, device, equipment and storage medium for operation and maintenance data

Info

Publication number: CN116304910A
Application number: CN202310267918.1A
Authority: CN
Inventors: 李兰彬; 王泽洋; 陈巧燕; 邵飞飞
Original assignee: Industrial and Commercial Bank of China Ltd ICBC
Current assignee: Industrial and Commercial Bank of China Ltd ICBC
Priority date: 2023-03-15
Filing date: 2023-03-15
Publication date: 2023-06-23

Abstract

The disclosure provides an anomaly detection method, an anomaly detection device and a storage medium for operation and maintenance data, which can be applied to the technical field of anomaly detection and the technical field of finance. The method comprises the following steps: acquiring operation and maintenance data of multiple dimensions from an operation and maintenance data monitoring platform; carrying out variance alignment checking treatment on the operation and maintenance data of each dimension to obtain target operation and maintenance data of a plurality of dimensions; performing correlation analysis on the target operation and maintenance data of multiple dimensions to obtain the operation and maintenance data of the target dimensions from the target operation and maintenance data of the multiple dimensions; carrying out aggregation treatment on the operation and maintenance data of the target dimension to obtain operation and maintenance data to be detected; and processing the operation and maintenance data to be detected to obtain an abnormal detection result.

Description

Anomaly detection method, device, equipment and storage medium for operation and maintenance data

Technical Field

The present disclosure relates to the field of anomaly detection technology and the field of financial technology, and in particular, to a method, apparatus, device, medium, and program product for anomaly detection of operation and maintenance data.

Background

With the wide application of computer technology in various fields, the number of dimensions of operation and maintenance data obtained from an operation and maintenance data monitoring platform is gradually increased. Because the traditional pre-warning method based on the preset rules is generally set based on data with a single dimension, the number of the preset rules is increased along with the increase of the number of dimensions of operation and maintenance data, so that the problem of high frequent alarm or false alarm rate of the operation and maintenance data monitoring platform is caused.

Disclosure of Invention

In view of the foregoing, the present disclosure provides an anomaly detection method, apparatus, device, medium, and program product for operation and maintenance data.

According to one aspect of the present disclosure, there is provided an anomaly detection method of operation and maintenance data, including: acquiring operation and maintenance data of multiple dimensions from an operation and maintenance data monitoring platform; carrying out variance alignment checking treatment on the operation and maintenance data of each dimension to obtain target operation and maintenance data of a plurality of dimensions; performing correlation analysis on the target operation and maintenance data of multiple dimensions to obtain the operation and maintenance data of the target dimensions from the target operation and maintenance data of the multiple dimensions; carrying out aggregation treatment on the operation and maintenance data of the target dimension to obtain operation and maintenance data to be detected; and processing the operation and maintenance data to be detected to obtain an abnormal detection result.

According to an embodiment of the present disclosure, the plurality of dimensions includes M, where M is an integer greater than 1, and the obtaining the target operation data of the plurality of dimensions by performing a variance alignment check process on the operation data of each dimension includes:

the following operations are sequentially carried out on the operation data of each dimension to obtain target operation data of multiple dimensions:

carrying out variance alignment checking treatment on the operation and maintenance data of the M-th dimension to obtain a checking result of the operation and maintenance data of the M-th dimension, wherein M is an integer greater than or equal to 1 and less than or equal to M; and obtaining the target operation and maintenance data of the m dimension from the operation and maintenance data of the m dimension based on the inspection result of the operation and maintenance data of the m dimension.

According to an embodiment of the present disclosure, based on a result of the inspection of the operation and maintenance data of the mth dimension, obtaining target operation and maintenance data of the mth dimension from the operation and maintenance data of the mth dimension includes:

sorting the operation and maintenance data of the m dimension based on the inspection result of the operation and maintenance data of the m dimension to obtain a sorting result; and according to the sorting result, performing data filtering operation on the operation and maintenance data of the m dimension to obtain the target operation and maintenance data of the m dimension.

According to an embodiment of the present disclosure, by performing correlation analysis on target operation and maintenance data of multiple dimensions, operation and maintenance data of the target dimensions are obtained from the target operation and maintenance data of multiple dimensions, including:

performing correlation analysis on the target operation and maintenance data of multiple dimensions based on a Pearson correlation coefficient method to obtain the correlation degree and the correlation direction of the target operation and maintenance data of multiple dimensions; and according to the correlation degree and the correlation direction, acquiring the operation and maintenance data of the target dimension from the target operation and maintenance data of a plurality of dimensions.

According to an embodiment of the present disclosure, according to a correlation degree and a correlation direction, obtaining operation and maintenance data of a target dimension from target operation and maintenance data of a plurality of dimensions includes:

classifying the target operation and maintenance data with multiple dimensions according to the correlation direction to obtain a first data set and a second data set, wherein the first data set represents the first target operation and maintenance data with multiple dimensions with positive correlation, and the second data set represents the second target operation and maintenance data with multiple dimensions with negative correlation; determining first target operation and maintenance data with the correlation degree larger than a first preset threshold value in the first data set as operation and maintenance data of the first target dimension; and determining the second target operation and maintenance data with the correlation degree greater than the first preset threshold value in the second data set as operation and maintenance data of the second target dimension.

According to an embodiment of the present disclosure, the operation and maintenance data of the target dimension includes operation and maintenance data of a first target dimension positively correlated with each other and operation and maintenance data of a second target dimension negatively correlated with each other, and the aggregation processing is performed on the operation and maintenance data of the target dimension to obtain operation and maintenance data to be detected, including:

carrying out aggregation treatment on the operation and maintenance data of the first target dimension by using a principal component analysis method to obtain first operation and maintenance data to be detected; and carrying out aggregation treatment on the operation and maintenance data of the second target dimension by using a principal component analysis method to obtain second operation and maintenance data to be detected.

According to an embodiment of the present disclosure, processing a fortune dimension to be detected to obtain an anomaly detection result includes: .

Processing the operation and maintenance data to be detected based on a preset rule to obtain a first detection result; processing the operation and maintenance data to be detected based on the anomaly detection model to obtain a second detection result; and obtaining an abnormal detection result according to the first detection result and the second detection result.

According to an embodiment of the present disclosure, a training method of an anomaly detection model includes:

inputting the historical operation and maintenance data into an initial anomaly detection model, and outputting a detection result; obtaining a loss value according to the detection result and the label of the historical operation and maintenance data based on the loss function; based on the loss value, model parameters of the initial anomaly detection model are adjusted to obtain the anomaly detection model.

According to an embodiment of the present disclosure, obtaining an anomaly detection result according to a first detection weight and a second detection weight includes:

determining the weight of the first detection result and the weight of the second detection result; and obtaining an abnormal detection result according to the first detection weight, the second detection weight, the weight of the first detection result and the weight of the second detection result.

Another aspect of the present disclosure provides an anomaly detection device for operation and maintenance data, including: the system comprises an acquisition module, a detection module, an analysis module, an aggregation module and a processing module. The acquisition module is used for acquiring the operation and maintenance data of multiple dimensions from the operation and maintenance data monitoring platform. And the checking module is used for obtaining target operation and maintenance data of multiple dimensions by carrying out variance alignment checking processing on the operation and maintenance data of each dimension. And the analysis module is used for obtaining the operation and maintenance data of the target dimension from the target operation and maintenance data of the multiple dimensions by carrying out correlation analysis on the target operation and maintenance data of the multiple dimensions. And the aggregation module is used for carrying out aggregation processing on the operation and maintenance data of the target dimension to obtain operation and maintenance data to be detected. And the processing module is used for processing the operation and maintenance data to be detected to obtain an abnormal detection result.

According to an embodiment of the present disclosure, the plurality of dimensions includes M, where M is an integer greater than 1, the inspection module includes: a verification unit and a first obtaining unit. The testing unit is used for carrying out variance alignment testing on the operation and maintenance data of the M dimension to obtain a testing result of the operation and maintenance data of the M dimension, wherein M is an integer greater than or equal to 1 and less than or equal to M. The first obtaining unit is used for obtaining the target operation and maintenance data of the m dimension from the operation and maintenance data of the m dimension based on the inspection result of the operation and maintenance data of the m dimension.

According to an embodiment of the present disclosure, the first obtaining unit includes an ordering subunit and a first obtaining subunit. The sorting subunit is configured to sort the operation and maintenance data of the m-th dimension based on the inspection result of the operation and maintenance data of the m-th dimension, so as to obtain a sorting result. And the first obtaining subunit is used for performing data filtering operation on the operation and maintenance data of the mth dimension according to the sequencing result to obtain the target operation and maintenance data of the mth dimension.

According to an embodiment of the present disclosure, an analysis module includes: an analysis unit and a second obtaining unit. And the analysis unit is used for carrying out correlation analysis on the target motion data of the multiple dimensions based on the Pearson correlation coefficient method to obtain the correlation degree and the correlation direction of the target motion data of the multiple dimensions. And the second obtaining unit is used for obtaining the operation and maintenance data of the target dimension from the target operation and maintenance data of a plurality of dimensions according to the correlation degree and the correlation direction.

According to an embodiment of the present disclosure, the second obtaining unit includes: a classification subunit, a first determination subunit, and a second determination subunit. The classifying subunit is configured to classify the target operation and maintenance data with multiple dimensions according to the correlation direction, so as to obtain a first data set and a second data set, where the first data set represents the first target operation and maintenance data with multiple dimensions with positive correlation, and the second data set represents the second target operation and maintenance data with multiple dimensions with negative correlation. And the first determining subunit is used for determining the first target operation dimension data with the correlation degree larger than a first preset threshold value in the first data set as the operation dimension data of the first target dimension. And the second determining subunit is used for determining the second target operation dimension data with the correlation degree larger than the first preset threshold value in the second data set as operation dimension data of the second target dimension.

According to an embodiment of the present disclosure, the operation and maintenance data of the target dimensions includes operation and maintenance data of a first target dimension positively correlated with each other and operation and maintenance data of a second target dimension negatively correlated with each other, and the aggregation module includes: a first polymeric unit and a second polymeric unit. The first aggregation unit is used for performing aggregation processing on the operation and maintenance data of the first target dimension by using a principal component analysis method to obtain first operation and maintenance data to be detected. And the second aggregation unit is used for carrying out aggregation treatment on the operation and maintenance data of the second target dimension by utilizing a principal component analysis method to obtain second operation and maintenance data to be detected.

According to an embodiment of the present disclosure, the processing module includes a first processing unit, a second processing unit, and a third obtaining unit. The first processing unit is used for processing the operation and maintenance data to be detected based on a preset rule to obtain a first detection result. And the second processing unit is used for processing the operation and maintenance data to be detected based on the anomaly detection model to obtain a second detection result. And the third obtaining unit is used for obtaining an abnormal detection result according to the first detection result and the second detection result.

According to an embodiment of the disclosure, the apparatus further includes a training module, where the training module includes a detection unit, a fourth obtaining unit, and an adjustment unit. Wherein, the detecting element is used for: inputting the historical operation and maintenance data into an initial anomaly detection model, and outputting a detection result; the fourth obtaining unit is used for obtaining a loss value according to the detection result and the label of the historical operation and maintenance data based on the loss function; the adjusting unit is used for adjusting model parameters of the initial anomaly detection model based on the loss value to obtain the anomaly detection model.

According to an embodiment of the present disclosure, the adjustment unit comprises a third determination subunit and a second obtaining subunit. And the third determining subunit is used for determining the weight of the first detection result and the weight of the second detection result. The second obtaining subunit is configured to obtain an abnormal detection result according to the first detection weight, the second detection weight, the weight of the first detection result, and the weight of the second detection result.

Another aspect of the present disclosure provides an electronic device, comprising: one or more processors; and a memory for storing one or more programs, wherein the one or more programs, when executed by the one or more processors, cause the one or more processors to perform the method described above.

Another aspect of the present disclosure also provides a computer-readable storage medium having stored thereon executable instructions that, when executed by a processor, cause the processor to perform the above-described method.

Another aspect of the present disclosure also provides a computer program product comprising a computer program which, when executed by a processor, implements the above method.

According to the embodiment of the disclosure, the operation and maintenance data to be detected with obvious characteristics can be obtained by sequentially carrying out variance alignment test, correlation analysis and aggregation treatment on the operation and maintenance data with multiple dimensions, so that the characteristic significance degree of the operation and maintenance data is improved. And then the operation and maintenance data to be detected are detected, so that the accuracy of an abnormal detection result can be effectively improved.

Drawings

The foregoing and other objects, features and advantages of the disclosure will be more apparent from the following description of embodiments of the disclosure with reference to the accompanying drawings, in which:

FIG. 1 schematically illustrates an application scenario diagram of an anomaly detection method, apparatus, device, medium and program product for operation and maintenance data according to an embodiment of the present disclosure;

FIG. 2 schematically illustrates a flow chart of an anomaly detection method of operation and maintenance data according to an embodiment of the present disclosure;

FIG. 3 schematically illustrates a schematic diagram of obtaining operational dimensional data to be detected, according to an embodiment of the present disclosure;

fig. 4 schematically illustrates a schematic diagram of processing a dimension of a ship to be detected to obtain an anomaly detection result according to an embodiment of the present disclosure.

Fig. 5 schematically illustrates a block diagram of a configuration of an abnormality detection apparatus of operation and maintenance data according to an embodiment of the present disclosure; and

fig. 6 schematically illustrates a block diagram of an electronic device adapted to implement an anomaly detection method of operation and maintenance data according to an embodiment of the present disclosure.

Detailed Description

Hereinafter, embodiments of the present disclosure will be described with reference to the accompanying drawings. It should be understood that the description is only exemplary and is not intended to limit the scope of the present disclosure. In the following detailed description, for purposes of explanation, numerous specific details are set forth in order to provide a thorough understanding of the embodiments of the present disclosure. It may be evident, however, that one or more embodiments may be practiced without these specific details. In addition, in the following description, descriptions of well-known structures and techniques are omitted so as not to unnecessarily obscure the concepts of the present disclosure.

The terminology used herein is for the purpose of describing particular embodiments only and is not intended to be limiting of the disclosure. The terms "comprises," "comprising," and/or the like, as used herein, specify the presence of stated features, steps, operations, and/or components, but do not preclude the presence or addition of one or more other features, steps, operations, or components.

All terms (including technical and scientific terms) used herein have the same meaning as commonly understood by one of ordinary skill in the art unless otherwise defined. It should be noted that the terms used herein should be construed to have meanings consistent with the context of the present specification and should not be construed in an idealized or overly formal manner.

Where expressions like at least one of "A, B and C, etc. are used, the expressions should generally be interpreted in accordance with the meaning as commonly understood by those skilled in the art (e.g.," a system having at least one of A, B and C "shall include, but not be limited to, a system having a alone, B alone, C alone, a and B together, a and C together, B and C together, and/or A, B, C together, etc.).

In the technical scheme of the disclosure, the related data (such as including but not limited to personal information of a user) are collected, stored, used, processed, transmitted, provided, disclosed, applied and the like, all conform to the regulations of related laws and regulations, necessary security measures are adopted, and the public welcome is not violated.

The embodiment of the disclosure provides an anomaly detection method of operation and maintenance data, which comprises the following steps: acquiring operation and maintenance data of multiple dimensions from an operation and maintenance data monitoring platform; carrying out variance alignment checking treatment on the operation and maintenance data of each dimension to obtain target operation and maintenance data of a plurality of dimensions; performing correlation analysis on the target operation and maintenance data of multiple dimensions to obtain the operation and maintenance data of the target dimensions from the target operation and maintenance data of the multiple dimensions; carrying out aggregation treatment on the operation and maintenance data of the target dimension to obtain operation and maintenance data to be detected; and processing the operation and maintenance data to be detected to obtain an abnormal detection result.

Fig. 1 schematically illustrates an application scenario diagram of an anomaly detection method of operation and maintenance data according to an embodiment of the present disclosure.

As shown in fig. 1, an application scenario 100 according to this embodiment may include a first terminal device 101, a second terminal device 102, a third terminal device 103, a network 104, and a server 105. The network 104 is a medium used to provide a communication link between the first terminal device 101, the second terminal device 102, the third terminal device 103, and the server 105. The network 104 may include various connection types, such as wired, wireless communication links, or fiber optic cables, among others.

The user may interact with the server 105 through the network 104 using at least one of the first terminal device 101, the second terminal device 102, the third terminal device 103, to receive or send messages, etc. Various communication client applications, such as a shopping class application, a web browser application, a search class application, an instant messaging tool, a mailbox client, social platform software, etc. (by way of example only) may be installed on the first terminal device 101, the second terminal device 102, and the third terminal device 103.

The first terminal device 101, the second terminal device 102, the third terminal device 103 may be various electronic devices having a display screen and supporting web browsing, including but not limited to smartphones, tablets, laptop and desktop computers, and the like.

The server 105 may be a server providing various services, such as a background management server (by way of example only) providing support for websites browsed by the user using the first terminal device 101, the second terminal device 102, and the third terminal device 103. The background management server may analyze and process the received data such as the user request, and feed back the processing result (e.g., the web page, information, or data obtained or generated according to the user request) to the terminal device.

It should be noted that, the method for detecting the anomaly of the operation and maintenance data provided by the embodiment of the disclosure may be generally performed by the server 105. Accordingly, the abnormality detection device for operation and maintenance data provided by the embodiments of the present disclosure may be generally provided in the server 105. The anomaly detection method of the operation and maintenance data provided by the embodiment of the present disclosure may also be performed by a server or a server cluster that is different from the server 105 and is capable of communicating with the first terminal device 101, the second terminal device 102, the third terminal device 103, and/or the server 105. Accordingly, the abnormality detection apparatus for operation and maintenance data provided by the embodiments of the present disclosure may also be provided in a server or a server cluster that is different from the server 105 and is capable of communicating with the first terminal device 101, the second terminal device 102, the third terminal device 103, and/or the server 105.

It should be understood that the number of terminal devices, networks and servers in fig. 1 is merely illustrative. There may be any number of terminal devices, networks, and servers, as desired for implementation.

The abnormality detection method of the operation and maintenance data of the disclosed embodiment will be described in detail below with reference to the scenario described in fig. 1 through fig. 2 to 4.

Fig. 2 schematically illustrates a flowchart of an anomaly detection method of operation and maintenance data according to an embodiment of the present disclosure.

As shown in fig. 2, the anomaly detection method of the operation and maintenance data of this embodiment includes operations S210 to S250.

In operation S210, operation and maintenance data of a plurality of dimensions is acquired from the operation and maintenance data monitoring platform.

In operation S220, the target operation data of multiple dimensions is obtained by performing a variance alignment check process on the operation data of each dimension.

In operation S230, operation dimensional data of a target dimension is obtained from the target operation dimensional data of a plurality of dimensions by performing correlation analysis on the target operation dimensional data of the plurality of dimensions.

In operation S240, the operation and maintenance data of the target dimension are aggregated to obtain operation and maintenance data to be detected.

In operation S250, the operation and maintenance data to be detected is processed to obtain an anomaly detection result.

According to embodiments of the present disclosure, the operation and maintenance data may include operation and maintenance data related to the CPU, for example: the utilization rate of the CPU; may also include memory-related operational data such as: disk occupancy, etc.

According to an embodiment of the present disclosure, before performing the variance alignment checking process on the operation data of each dimension, a cleaning operation and a morphing operation may be performed on the operation data. The cleaning operation can remove noise and redundant data caused by differences among various monitoring platforms. The deformation operation can normalize the operation data of various monitoring platforms with different units or different granularities.

According to the embodiment of the disclosure, for the operation and maintenance data of each dimension, the data which can embody the significant characteristics of the operation and maintenance data of the dimension can be screened from the operation and maintenance data of the dimension through a variance alignment test method, so that the target operation and maintenance data can be obtained.

According to the embodiment of the disclosure, correlation analysis is performed on the target operation data of multiple dimensions, and the target operation data of the dimension with higher correlation is determined as the target operation data of the target dimension according to the correlation between the target operation data of each dimension.

According to the embodiment of the disclosure, even though the variance alignment test and the correlation analysis are performed, the number of the target dimensions may still be more, so that the number of target targeting degrees can be reduced in an aggregation manner, so as to improve the significance degree of the target operation and maintenance data, and obtain the operation and maintenance data to be detected with obvious significance features.

According to the embodiment of the disclosure, the operation and maintenance data to be detected with obvious characteristics is detected, and an abnormal detection result can be obtained.

For example: when the operation and maintenance data to be detected accords with the preset rule, the obtained detection result can be that the operation and maintenance data are normal. When the operation and maintenance data to be detected does not accord with the preset rule, the obtained detection result can be abnormal operation and maintenance data.

Fig. 3 schematically illustrates a schematic diagram of obtaining operational data to be detected according to an embodiment of the present disclosure.

As shown in fig. 3, the method of this embodiment 300 includes operations S310-S320.

in operation S310, the variance-alignment test is performed on the operation and maintenance data of the M-th dimension to obtain a test result of the operation and maintenance data of the M-th dimension, where M is an integer greater than or equal to 1 and less than or equal to M.

In operation S320, the target operation and maintenance data of the mth dimension is obtained from the operation and maintenance data of the mth dimension based on the inspection result of the operation and maintenance data of the mth dimension.

For example: the operation and maintenance data of the m-th dimension may include 100, and the 100 operation and maintenance data may be randomly divided into a plurality of groups, and squares of standard deviations are calculated for each group of data, and a variance ratio is obtained according to square ratios of standard deviations of the two groups of data. And determining whether a significant difference exists between the two groups of data according to the variance ratio, and so on, and screening the operation and maintenance data with the significant difference from the multiple groups of operation and maintenance data based on the variance ratio result.

According to an embodiment of the present disclosure, based on a result of the inspection of the operation and maintenance data of the mth dimension, obtaining the target operation and maintenance data of the mth dimension from the operation and maintenance data of the mth dimension may include the following operations:

and ordering the operation and maintenance data of the m dimension based on the inspection result of the operation and maintenance data of the m dimension to obtain an ordering result. And according to the sorting result, performing data filtering operation on the operation and maintenance data of the m dimension to obtain the target operation and maintenance data of the m dimension.

For example: the operation and maintenance data of the m-th dimension may be divided into 3 groups, a variance ratio between the 1 st group operation and maintenance data and the 2 nd group operation and maintenance data may be 11, a variance ratio between the 1 st group operation and maintenance data and the 3 rd group operation and maintenance data may be 1, and a variance ratio between the 2 nd group operation and maintenance data and the 3 rd group operation and maintenance data may be 12. The degree of the significant difference between the 1 st group operation and maintenance data and the 2 nd group operation and maintenance data is 11, the degree of the significant difference between the 2 nd group operation and maintenance data and the 3 rd group operation and maintenance data is 12, and the degree of the significant difference between the 1 st group operation and maintenance data and the 3 rd group operation and maintenance data is 3.

According to embodiments of the present disclosure, the target dimension data of the m-th dimension may be the 2 nd set of dimension data, with respect to the same set of dimension data, e.g., the 3 rd set of dimension data, the 2 nd set of dimension data differs significantly from the 3 rd set of dimension data more than the 1 st set of dimension data differs significantly from the 3 rd set of dimension data.

According to the embodiment of the disclosure, the accuracy of anomaly detection can be improved by screening the target motion data with the characteristic of significant difference through the variance alignment test.

According to an embodiment of the present disclosure, by performing correlation analysis on target operation data of multiple dimensions, obtaining operation data of a target dimension from the target operation data of multiple dimensions may include the following operations:

and carrying out correlation analysis on the target operation and maintenance data of multiple dimensions based on a Pearson correlation coefficient method to obtain the correlation degree and the correlation direction of the target operation and maintenance data of multiple dimensions. And according to the correlation degree and the correlation direction, acquiring the operation and maintenance data of the target dimension from the target operation and maintenance data of a plurality of dimensions.

For example: based on the Pearson correlation coefficient method, correlation analysis is carried out on the A-dimension target motion data and the B-dimension target motion data, so that the forward correlation of the A-dimension target motion data and the B-dimension target motion data can be obtained, and the correlation degree is 60%. And performing correlation analysis on the target motion data of the A dimension and the target motion data of the C dimension to obtain that the target motion data of the A dimension and the target motion data of the C dimension are in forward correlation with each other, and the correlation degree is 30%. And performing correlation analysis on the C-dimension target motion data and the B-dimension target motion data to obtain that the C-dimension target motion data and the B-dimension target motion data are in forward correlation with each other, and the correlation degree is 20%. Thus, the operational dimension data for the target dimension may be the target operational dimension data for the A dimension and the target operational dimension data for the B dimension.

According to an embodiment of the present disclosure, obtaining operation and maintenance data of a target dimension from target operation and maintenance data of a plurality of dimensions according to a correlation degree and a correlation direction may include the following operations:

and classifying the target operation and maintenance data with multiple dimensions according to the correlation direction to obtain a first data set and a second data set, wherein the first data set represents the first target operation and maintenance data with multiple dimensions with positive correlation, and the second data set represents the second target operation and maintenance data with multiple dimensions with negative correlation. And determining the first target operation and maintenance data with the correlation degree larger than a first preset threshold value in the first data set as operation and maintenance data of the first target dimension. And determining the second target operation and maintenance data with the correlation degree greater than the first preset threshold value in the second data set as operation and maintenance data of the second target dimension.

For example: the correlation of positive correlation exists between the target motion data of the A dimension and the target motion data of the B dimension, and the correlation of negative correlation exists between the target motion data of the C dimension and the target motion data of the D dimension. Thus, the target dimension data of the A dimension and the target dimension data of the B dimension belong to the first data set. The target dimension data of the C dimension and the target dimension data of the D dimension belong to a second dataset.

For example: in the first data set, a degree of correlation of forward correlation between the target motion data of the a-dimension and the target motion data of the B-dimension may be 60%, and a degree of correlation of forward correlation between the target motion data of the E-dimension and the target motion data of the F-dimension may be 30%. The first preset threshold may be 50%, and the operation and maintenance data of the first target dimension may be the target operation and maintenance data of the a dimension and the target operation and maintenance data of the B dimension.

According to the embodiments of the present disclosure, the data processing method for the target operation and maintenance data in the second data set is the same as the data processing method for the target operation and maintenance data in the first data set, and will not be described herein.

According to the embodiment of the disclosure, the target operation and maintenance data with the same correlation direction and higher correlation degree are screened out through the pearson correlation coefficient, so that the obvious characteristics of the target operation and maintenance data are improved, and the accuracy of anomaly detection is improved.

According to an embodiment of the present disclosure, the operation and maintenance data of the target dimension includes operation and maintenance data of a first target dimension positively correlated with each other and operation and maintenance data of a second target dimension negatively correlated with each other, and the aggregation processing is performed on the operation and maintenance data of the target dimension to obtain operation and maintenance data to be detected, which may include the following operations:

And carrying out aggregation treatment on the operation and maintenance data of the first target dimension by using a principal component analysis method to obtain first operation and maintenance data to be detected. And carrying out aggregation treatment on the operation and maintenance data of the second target dimension by using a principal component analysis method to obtain second operation and maintenance data to be detected.

According to the embodiment of the disclosure, since the dimension of the operation and maintenance data of the target dimension may still be higher, the main component analysis method may be used for performing aggregation processing to reduce the dimension, so as to obtain the operation and maintenance data to be detected with lower dimension and higher significance.

According to the embodiment of the disclosure, the first operation data to be detected, which have forward correlation with each other, can be selected for detection according to the types of different operation and maintenance data and detection requirements. The band detection second operation data, which are negatively correlated with each other, may also be selected for detection. The present invention is not particularly limited herein.

According to the embodiment of the disclosure, the main component analysis method is utilized to carry out polymerization treatment so as to reduce the dimension and obtain the operation and maintenance data to be detected with lower dimension and higher significance.

According to an embodiment of the present disclosure, processing a motion data to be detected to obtain an anomaly detection result may include the following operations:

And processing the operation and maintenance data to be detected based on a preset rule to obtain a first detection result. And processing the operation and maintenance data to be detected based on the anomaly detection model to obtain a second detection result. And obtaining an abnormal detection result according to the first detection result and the second detection result.

According to an embodiment of the disclosure, the preset rule may be formulated based on a priori experience, and when the operation and maintenance data to be detected accords with the preset rule, the obtained first detection result may be 0.8, which indicates that the operation and maintenance data to be detected is normal operation and maintenance data. When the operation and maintenance data to be detected does not accord with the preset rule, the obtained second detection result can be 0.2, which indicates that the operation and maintenance data to be detected is abnormal operation and maintenance data.

According to embodiments of the present disclosure, the anomaly detection model may be trained based on historical running data. When the change of the operation and maintenance data to be detected accords with the change rule of the historical operation and maintenance data, the obtained second detection result can be 0.7, which indicates that the operation and maintenance data to be detected is normal operation and maintenance data. When the change of the operation and maintenance data to be detected does not accord with the change rule of the historical operation and maintenance data, the obtained second detection result can be 0.3, which indicates that the operation and maintenance data to be detected is abnormal operation and maintenance data.

According to an embodiment of the present disclosure, the weight of the first detection result and the weight of the second detection result may be preset configured, for example: the weight of the first detection result may be 0.4 and the weight of the second detection result may be 0.6. In the case where the first detection result is 0.8 and the second detection result is 0.7, the obtained detection result may be 0.74. Indicating that the operation and maintenance data is normal operation and maintenance data.

As shown in fig. 4, in an embodiment 400, a first detection result 402 is obtained based on operation and maintenance data 401 to be detected according to a preset rule. Based on the anomaly detection model, the operation and maintenance data 401 to be detected are processed, and a second detection result 403 is obtained. An anomaly detection result 406 is obtained from the first detection result 402, the weight 404 of the first detection result, the second detection result 403, and the weight 405 of the second detection result.

According to the embodiment of the disclosure, weighted summation is respectively carried out on the results obtained based on the preset rule and the abnormality detection model, so that the advantage complementation of the two alarm modes is realized, and the false alarm rate is reduced.

According to an embodiment of the present disclosure, a training method of an anomaly detection model may include the operations of:

and inputting the historical operation and maintenance data into an initial anomaly detection model, and outputting a detection result. And obtaining a loss value according to the detection result and the label of the historical operation data based on the loss function. Based on the loss value, model parameters of the initial anomaly detection model are adjusted to obtain the anomaly detection model.

According to the embodiment of the disclosure, the loss function may be a cross entropy loss function, a cross entropy loss value is obtained according to a detection result and a label of historical operation and maintenance data, and the anomaly detection model is obtained by adjusting model parameters of an initial anomaly detection model until the cross entropy loss value converges.

According to the embodiment of the disclosure, the difference between the current operation and maintenance data and the historical operation and maintenance data can be detected more accurately based on the anomaly detection model obtained by training the historical operation and maintenance data, and the alarm accuracy is improved.

Based on the anomaly detection method of the operation and maintenance data, the disclosure also provides an anomaly detection device of the operation and maintenance data. The device will be described in detail below in connection with fig. 5.

Fig. 5 schematically shows a block diagram of a configuration of an abnormality detection apparatus of operation and maintenance data according to an embodiment of the present disclosure.

As shown in fig. 5, the abnormality detection apparatus 500 of the operation and maintenance data of this embodiment acquires a module 510, a verification module 520, an analysis module 530, an aggregation module 540, and a processing module 550.

And the obtaining module 510 is configured to obtain the operation and maintenance data of multiple dimensions from the operation and maintenance data monitoring platform. The obtaining module 510 is configured to perform the aforementioned operation S210, which is not described herein.

And the checking module 520 is configured to obtain target operation and maintenance data of multiple dimensions by performing a variance alignment checking process on the operation and maintenance data of each dimension. The checking module 520 is configured to perform the aforementioned operation S220, which is not described herein.

The analysis module 530 is configured to obtain the operation and maintenance data of the target dimension from the target operation and maintenance data of the multiple dimensions by performing correlation analysis on the target operation and maintenance data of the multiple dimensions. The analysis module 530 is configured to perform the aforementioned operation S220, which is not described herein.

And the aggregation module 540 is used for performing aggregation processing on the operation and maintenance data of the target dimension to obtain operation and maintenance data to be detected. The aggregation module 540 is configured to perform the aforementioned operation S240, which is not described herein.

And the processing module 550 is used for processing the operation and maintenance data to be detected to obtain an abnormal detection result. The processing module 550 is configured to perform the aforementioned operation S250, which is not described herein.

Any of the acquisition module 510, the inspection module 520, the analysis module 530, the aggregation module 540, and the processing module 550 may be combined in one module to be implemented, or any of the modules may be split into a plurality of modules, according to embodiments of the present disclosure. Alternatively, at least some of the functionality of one or more of the modules may be combined with at least some of the functionality of other modules and implemented in one module. According to embodiments of the present disclosure, at least one of the acquisition module 510, the verification module 520, the analysis module 530, the aggregation module 540, and the processing module 550 may be implemented at least in part as hardware circuitry, such as a Field Programmable Gate Array (FPGA), a Programmable Logic Array (PLA), a system on a chip, a system on a substrate, a system on a package, an Application Specific Integrated Circuit (ASIC), or may be implemented in hardware or firmware in any other reasonable way of integrating or packaging the circuitry, or in any one of or a suitable combination of three of software, hardware, and firmware. Alternatively, at least one of the acquisition module 510, the verification module 520, the analysis module 530, the aggregation module 540, and the processing module 550 may be at least partially implemented as a computer program module, which when executed may perform the respective functions.

As shown in fig. 6, an electronic device 600 according to an embodiment of the present disclosure includes a processor 601 that can perform various appropriate actions and processes according to a program stored in a Read Only Memory (ROM) 602 or a program loaded from a storage section 608 into a Random Access Memory (RAM) 603. The processor 601 may include, for example, a general purpose microprocessor (e.g., a CPU), an instruction set processor and/or an associated chipset and/or a special purpose microprocessor (e.g., an Application Specific Integrated Circuit (ASIC)), or the like. Processor 601 may also include on-board memory for caching purposes. The processor 601 may comprise a single processing unit or a plurality of processing units for performing different actions of the method flows according to embodiments of the disclosure.

In the RAM 603, various programs and data necessary for the operation of the electronic apparatus 600 are stored. The processor 601, the ROM602, and the RAM 603 are connected to each other through a bus 604. The processor 601 performs various operations of the method flow according to the embodiments of the present disclosure by executing programs in the ROM602 and/or the RAM 603. Note that the program may be stored in one or more memories other than the ROM602 and the RAM 603. The processor 601 may also perform various operations of the method flow according to embodiments of the present disclosure by executing programs stored in the one or more memories.

According to an embodiment of the present disclosure, the electronic device 600 may also include an input/output (I/O) interface 605, the input/output (I/O) interface 605 also being connected to the bus 604. The electronic device 600 may also include one or more of the following components connected to an input/output (I/O) interface 605: an input portion 606 including a keyboard, mouse, etc.; an output portion 607 including a Cathode Ray Tube (CRT), a Liquid Crystal Display (LCD), and the like, a speaker, and the like; a storage section 608 including a hard disk and the like; and a communication section 609 including a network interface card such as a LAN card, a modem, or the like. The communication section 609 performs communication processing via a network such as the internet. The drive 610 is also connected to an input/output (I/O) interface 605 as needed. Removable media 611 such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, or the like is installed as needed on drive 610 so that a computer program read therefrom is installed as needed into storage section 608.

The present disclosure also provides a computer-readable storage medium that may be embodied in the apparatus/device/system described in the above embodiments; or may exist alone without being assembled into the apparatus/device/system. The computer-readable storage medium carries one or more programs which, when executed, implement methods in accordance with embodiments of the present disclosure.

According to embodiments of the present disclosure, the computer-readable storage medium may be a non-volatile computer-readable storage medium, which may include, for example, but is not limited to: a portable computer diskette, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing. In the context of this disclosure, a computer-readable storage medium may be any tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, apparatus, or device. For example, according to embodiments of the present disclosure, the computer-readable storage medium may include ROM602 and/or RAM603 and/or one or more memories other than ROM602 and RAM603 described above.

Embodiments of the present disclosure also include a computer program product comprising a computer program containing program code for performing the methods shown in the flowcharts. The program code, when executed in a computer system, causes the computer system to implement the item recommendation method provided by embodiments of the present disclosure.

The above-described functions defined in the system/apparatus of the embodiments of the present disclosure are performed when the computer program is executed by the processor 601. The systems, apparatus, modules, units, etc. described above may be implemented by computer program modules according to embodiments of the disclosure.

In one embodiment, the computer program may be based on a tangible storage medium such as an optical storage device, a magnetic storage device, or the like. In another embodiment, the computer program may also be transmitted, distributed in the form of signals over a network medium, and downloaded and installed via the communication section 609, and/or installed from the removable medium 611. The computer program may include program code that may be transmitted using any appropriate network medium, including but not limited to: wireless, wired, etc., or any suitable combination of the foregoing.

In such an embodiment, the computer program may be downloaded and installed from a network through the communication portion 609, and/or installed from the removable medium 611. The above-described functions defined in the system of the embodiments of the present disclosure are performed when the computer program is executed by the processor 601. The systems, devices, apparatus, modules, units, etc. described above may be implemented by computer program modules according to embodiments of the disclosure.

According to embodiments of the present disclosure, program code for performing computer programs provided by embodiments of the present disclosure may be written in any combination of one or more programming languages, and in particular, such computer programs may be implemented in high-level procedural and/or object-oriented programming languages, and/or assembly/machine languages. Programming languages include, but are not limited to, such as Java, c++, python, "C" or similar programming languages. The program code may execute entirely on the user's computing device, partly on the user's device, partly on a remote computing device, or entirely on the remote computing device or server. In the case of remote computing devices, the remote computing device may be connected to the user computing device through any kind of network, including a Local Area Network (LAN) or a Wide Area Network (WAN), or may be connected to an external computing device (e.g., connected via the Internet using an Internet service provider).

The flowcharts and block diagrams in the figures illustrate the architecture, functionality, and operation of possible implementations of systems, methods and computer program products according to various embodiments of the present disclosure. In this regard, each block in the flowchart or block diagrams may represent a module, segment, or portion of code, which comprises one or more executable instructions for implementing the specified logical function(s). It should also be noted that, in some alternative implementations, the functions noted in the block may occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently, or the blocks may sometimes be executed in the reverse order, depending upon the functionality involved. It will also be noted that each block of the block diagrams or flowchart illustration, and combinations of blocks in the block diagrams or flowchart illustration, can be implemented by special purpose hardware-based systems which perform the specified functions or acts, or combinations of special purpose hardware and computer instructions.

Those skilled in the art will appreciate that the features recited in the various embodiments of the disclosure and/or in the claims may be provided in a variety of combinations and/or combinations, even if such combinations or combinations are not explicitly recited in the disclosure. In particular, the features recited in the various embodiments of the present disclosure and/or the claims may be variously combined and/or combined without departing from the spirit and teachings of the present disclosure. All such combinations and/or combinations fall within the scope of the present disclosure.

The embodiments of the present disclosure are described above. However, these examples are for illustrative purposes only and are not intended to limit the scope of the present disclosure. Although the embodiments are described above separately, this does not mean that the measures in the embodiments cannot be used advantageously in combination. The scope of the disclosure is defined by the appended claims and equivalents thereof. Various alternatives and modifications can be made by those skilled in the art without departing from the scope of the disclosure, and such alternatives and modifications are intended to fall within the scope of the disclosure.

Claims

1. An anomaly detection method of operation and maintenance data, comprising:

acquiring operation and maintenance data of multiple dimensions from an operation and maintenance data monitoring platform;

Carrying out variance alignment checking treatment on the operation and maintenance data of each dimension to obtain target operation and maintenance data of a plurality of dimensions;

performing correlation analysis on the target operation and maintenance data of the multiple dimensions to obtain operation and maintenance data of the target dimensions from the target operation and maintenance data of the multiple dimensions;

carrying out aggregation treatment on the operation and maintenance data of the target dimension to obtain operation and maintenance data to be detected; and

and processing the operation and maintenance data to be detected to obtain an abnormal detection result.

2. The method of claim 1, wherein the plurality of dimensions includes M, where M is an integer greater than 1, and the obtaining the target dimension data of the plurality of dimensions by performing a variance-alignment test on the dimension data of each dimension includes:

the following operations are sequentially carried out on the operation data of each dimension to obtain the target operation data of the plurality of dimensions:

carrying out variance alignment checking treatment on the operation and maintenance data of the M-th dimension to obtain a checking result of the operation and maintenance data of the M-th dimension, wherein M is an integer greater than or equal to 1 and less than or equal to M; and

and obtaining the target operation and maintenance data of the m dimension from the operation and maintenance data of the m dimension based on the inspection result of the operation and maintenance data of the m dimension.

3. The method of claim 2, wherein the obtaining the target operation data of the m-th dimension from the operation data of the m-th dimension based on the inspection result of the operation data of the m-th dimension includes:

sorting the operation and maintenance data of the m dimension based on the inspection result of the operation and maintenance data of the m dimension to obtain a sorting result; and

and according to the sorting result, performing data filtering operation on the operation and maintenance data of the m dimension to obtain the target operation and maintenance data of the m dimension.

4. The method of claim 1, wherein the obtaining the operation and maintenance data of the target dimension from the target operation and maintenance data of the plurality of dimensions by performing correlation analysis on the target operation and maintenance data of the plurality of dimensions comprises:

performing correlation analysis on the target operation and maintenance data of the multiple dimensions based on a Pearson correlation coefficient method to obtain the correlation degree and the correlation direction of the target operation and maintenance data of the multiple dimensions; and

and according to the correlation degree and the correlation direction, acquiring the operation and maintenance data of the target dimension from the target operation and maintenance data of the multiple dimensions.

5. The method of claim 4, wherein the obtaining the operation and maintenance data of the target dimension from the target operation and maintenance data of the plurality of dimensions according to the degree of correlation and the direction of correlation comprises:

Classifying the target operation and maintenance data with multiple dimensions according to the correlation direction to obtain a first data set and a second data set, wherein the first data set represents the first target operation and maintenance data with multiple dimensions with positive correlation, and the second data set represents the second target operation and maintenance data with multiple dimensions with negative correlation;

determining a first target operation and maintenance data with the correlation degree larger than a first preset threshold value in the first data set as the operation and maintenance data of the first target dimension; and

and determining second target operation and maintenance data with the correlation degree larger than a first preset threshold value in the second data set as the operation and maintenance data of the second target dimension.

6. The method of claim 1, wherein the operation and maintenance data of the target dimension includes operation and maintenance data of a first target dimension positively correlated with each other and operation and maintenance data of a second target dimension negatively correlated with each other, and the aggregating the operation and maintenance data of the target dimension to obtain operation and maintenance data to be detected includes:

performing aggregation treatment on the operation and maintenance data of the first target dimension by using a principal component analysis method to obtain first operation and maintenance data to be detected; and

And carrying out aggregation treatment on the operation and maintenance data of the second target dimension by using a principal component analysis method to obtain second operation and maintenance data to be detected.

7. The method of claim 1, wherein the processing the operation data to be detected to obtain an anomaly detection result includes:

processing the operation and maintenance data to be detected based on a preset rule to obtain a first detection result;

processing the operation and maintenance data to be detected based on an anomaly detection model to obtain a second detection result; and

and obtaining the abnormal detection result according to the first detection result and the second detection result.

8. The method of claim 7, wherein the training method of the anomaly detection model comprises:

inputting the historical operation and maintenance data into an initial anomaly detection model, and outputting a detection result;

obtaining a loss value according to the detection result and the label of the historical operation and maintenance data based on a loss function; and

and adjusting model parameters of the initial anomaly detection model based on the loss value to obtain the anomaly detection model.

9. The method of claim 7, wherein the obtaining the anomaly detection result from the first detection weight and the second detection weight comprises:

Determining the weight of the first detection result and the weight of the second detection result; and

and obtaining the abnormal detection result according to the first detection weight, the second detection weight, the weight of the first detection result and the weight of the second detection result.

10. An anomaly detection device for operation and maintenance data, comprising:

the acquisition module is used for acquiring the operation and maintenance data of a plurality of dimensions from the operation and maintenance data monitoring platform;

the processing module is used for obtaining target operation and maintenance data of multiple dimensions by carrying out variance alignment checking processing on the operation and maintenance data of each dimension;

the analysis module is used for obtaining the operation and maintenance data of the target dimension from the target operation and maintenance data of the multiple dimensions by carrying out correlation analysis on the target operation and maintenance data of the multiple dimensions;

the aggregation module is used for carrying out aggregation processing on the operation and maintenance data of the target dimension to obtain operation and maintenance data to be detected; and

and the detection module is used for processing the operation and maintenance data to be detected to obtain an abnormal detection result.

11. An electronic device, comprising:

one or more processors;

storage means for storing one or more programs,

Wherein the one or more programs, when executed by the one or more processors, cause the one or more processors to perform the method of any of claims 1-9.

12. A computer readable storage medium having stored thereon executable instructions which, when executed by a processor, cause the processor to perform the method according to any of claims 1 to 9.

13. A computer program product comprising a computer program which, when executed by a processor, implements the method according to any one of claims 1 to 9.