CN116933121A - Data anomaly detection method and device - Google Patents
Data anomaly detection method and device Download PDFInfo
- Publication number
- CN116933121A CN116933121A CN202211408252.9A CN202211408252A CN116933121A CN 116933121 A CN116933121 A CN 116933121A CN 202211408252 A CN202211408252 A CN 202211408252A CN 116933121 A CN116933121 A CN 116933121A
- Authority
- CN
- China
- Prior art keywords
- value
- resource index
- values
- resource
- data
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000001514 detection method Methods 0.000 title claims abstract description 60
- 238000005457 optimization Methods 0.000 claims abstract description 125
- 230000002159 abnormal effect Effects 0.000 claims abstract description 70
- 238000012417 linear regression Methods 0.000 claims abstract description 54
- 238000000034 method Methods 0.000 claims abstract description 28
- 238000004590 computer program Methods 0.000 claims description 15
- 238000010276 construction Methods 0.000 claims description 9
- 230000005856 abnormality Effects 0.000 claims description 8
- 230000001419 dependent effect Effects 0.000 claims description 4
- 238000004422 calculation algorithm Methods 0.000 description 8
- 238000013459 approach Methods 0.000 description 6
- 238000004891 communication Methods 0.000 description 6
- 238000010586 diagram Methods 0.000 description 6
- 238000004364 calculation method Methods 0.000 description 4
- 230000002547 anomalous effect Effects 0.000 description 3
- 230000003287 optical effect Effects 0.000 description 3
- 238000011835 investigation Methods 0.000 description 2
- 238000002955 isolation Methods 0.000 description 2
- 238000012423 maintenance Methods 0.000 description 2
- 238000005192 partition Methods 0.000 description 2
- 230000004931 aggregating effect Effects 0.000 description 1
- 238000013500 data storage Methods 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000012545 processing Methods 0.000 description 1
- 239000004065 semiconductor Substances 0.000 description 1
- 239000007787 solid Substances 0.000 description 1
- 238000007619 statistical method Methods 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
- G06Q10/00—Administration; Management
- G06Q10/06—Resources, workflows, human or project management; Enterprise or organisation planning; Enterprise or organisation modelling
- G06Q10/063—Operations research, analysis or management
- G06Q10/0639—Performance analysis of employees; Performance analysis of enterprise or organisation operations
- G06Q10/06393—Score-carding, benchmarking or key performance indicator [KPI] analysis
Landscapes
- Business, Economics & Management (AREA)
- Human Resources & Organizations (AREA)
- Engineering & Computer Science (AREA)
- Strategic Management (AREA)
- Development Economics (AREA)
- Economics (AREA)
- Entrepreneurship & Innovation (AREA)
- Educational Administration (AREA)
- Operations Research (AREA)
- Marketing (AREA)
- Game Theory and Decision Science (AREA)
- Quality & Reliability (AREA)
- Tourism & Hospitality (AREA)
- Physics & Mathematics (AREA)
- General Business, Economics & Management (AREA)
- General Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- Management, Administration, Business Operations System, And Electronic Commerce (AREA)
Abstract
The application relates to the field of computers and provides a data anomaly detection method and device. The method comprises the following steps: inputting the service request data value into a linear regression model to obtain a plurality of resource index predicted values; obtaining a plurality of resource index optimization values according to the absolute error values of the plurality of resource index prediction values; constructing an isolated tree according to the optimized values of the plurality of resource indexes and the weights of the plurality of resource indexes; obtaining abnormal scores of the resource index optimization values according to a plurality of isolated trees; if the anomaly score is greater than or equal to the score threshold, determining that the resource index predicted value corresponding to the resource index optimized value is abnormal. The data anomaly detection method and device provided by the embodiment of the application can correct and optimize the resource index data with volatility by utilizing the linear regression model and the absolute value of the error of the resource index predicted value, and construct an isolated tree by utilizing the weight of the resource index, so that the preference of system service to specific resources is highlighted, and the efficiency and the accuracy of mass data anomaly detection are improved.
Description
Technical Field
The application relates to the technical field of computers, in particular to a data anomaly detection method and device.
Background
The key performance index is an important reference for measuring the performance of the system, whether the system is abnormal in a certain aspect can be analyzed through the appearance of the key performance index, and the traditional method for judging whether the key performance index is abnormal comprises fixed configuration-based abnormality detection and statistic-based abnormality detection.
The abnormality detection based on fixed configuration requires enterprise arrangement operation staff to perform threshold configuration on each key performance index, and the detection efficiency and accuracy of the method are low under the conditions of multiple types, multiple numbers and complex rules of the key performance indexes; statistical-based anomaly detection requires that the key performance indicators follow a certain distribution, however, in an actual operation and maintenance scenario, the key performance indicators are distributed differently according to different service scenarios, and such assumption also results in lower detection efficiency and accuracy.
Disclosure of Invention
The embodiment of the application provides a data anomaly detection method and device, which are used for solving the technical problems of lower detection efficiency and lower accuracy of the traditional detection method.
In a first aspect, an embodiment of the present application provides a method for detecting data anomalies, including:
inputting the service request data value into a linear regression model to obtain a plurality of resource index predicted values;
Obtaining a plurality of resource index optimization values according to the absolute error values of the plurality of resource index prediction values;
constructing an isolated tree according to the optimized values of the plurality of resource indexes and the weights of the plurality of resource indexes;
obtaining abnormal scores of the resource index optimization values according to a plurality of the isolated trees;
if the abnormal score is greater than or equal to a score threshold, determining that the resource index predicted value corresponding to the resource index optimized value is abnormal;
the linear regression model is a linear regression model between the service request data values and the resource indicator predictive values.
In one embodiment, the linear regression model is constructed based on the following steps:
according to a polynomial regression method, constructing a linear regression model which takes a service request data value at a first moment as an independent variable and takes a resource index predicted value at a second moment as a dependent variable;
the second time is the next time to the first time.
In one embodiment, the inputting the service request data value into the linear regression model to obtain a plurality of resource indicator predictors includes:
and inputting the service request data values at a plurality of moments into the linear regression model to obtain the predicted values of the resource indexes of a plurality of types at the plurality of moments.
In one embodiment, before the constructing the orphan tree according to the optimized values of the plurality of resource indexes and the weights of the plurality of resource indexes, the method includes:
if the sum of the resource index predicted values of any type of n moments is zero, determining that the weight of the resource index corresponding to the resource index predicted value of the type is zero;
if the sum of the resource index predicted values of any type of n moments is not zero, obtaining the weight of the resource index corresponding to the resource index predicted value of the type according to the change rate of the resource index predicted value between the ith moment and the (i+5) th moment and the change rate of the service request data value between the ith moment and the (i+5) th moment;
wherein i is an integer of 1 or more and n or less, and n is the total number of the moments.
In one embodiment, the constructing an orphan tree according to the plurality of resource index optimization values and the weights of the plurality of resource indexes includes:
constructing a plurality of data points according to the plurality of resource index optimization values; any one data point comprises at least one type of resource index optimization value, and the types of the resource index optimization values included in any two data points are the same;
Placing the plurality of data points into a root node of the isolated tree, and taking the root node as a current node;
adding the weights of the resource indexes corresponding to the resource index predicted values of all types to obtain a total weight value;
randomly selecting any numerical value from zero to the total weight value to obtain a type judgment value;
determining the type to be divided according to the size relation between the type judgment value and the weight threshold value; the type to be divided is one of the types of the resource index optimization values included in the data points;
randomly selecting any value between a specific maximum value and a specific minimum value to obtain a dividing threshold value; the specific maximum value is the maximum value of the resource index optimization values belonging to the type to be divided in the plurality of data points, and the specific minimum value is the minimum value of the resource index optimization values belonging to the type to be divided in the plurality of data points;
dividing the data point of the current node into a first data point set and a second data point set according to the type to be divided and the dividing threshold value, placing the first data point set into a first child node of the isolated tree, placing the second data point set into a second child node of the isolated tree, taking the first child node and the second child node as the current node, returning to randomly select any value from zero to the total weight value, and obtaining a type judgment value until only one data point exists in the current node of the isolated tree or the height of the isolated tree reaches the height threshold value, and completing the construction of the isolated tree.
In one embodiment, the dividing the data point of the current node into the first data point set and the second data point set according to the type to be divided and the dividing threshold value includes:
if the resource index optimization value belonging to the type to be divided in the data points of the current node is larger than the dividing threshold value, dividing the data points corresponding to the resource index optimization value into a first data point set;
and if the resource index optimization value belonging to the type to be divided in the data points of the current node is smaller than the division threshold value, dividing the data points corresponding to the resource index optimization value into a second data point set.
In one embodiment, the obtaining the anomaly score of the resource indicator optimization value according to a plurality of the isolated trees includes:
and obtaining the anomaly score of the resource index optimization value according to the average height of the plurality of the isolated trees and the average path height of the resource index optimization value in the plurality of the isolated trees.
In a second aspect, an embodiment of the present application provides a data anomaly detection apparatus, including:
the resource index prediction value acquisition module is used for: inputting the service request data value into a linear regression model to obtain a plurality of resource index predicted values;
The resource index optimization value acquisition module is used for: obtaining a plurality of resource index optimization values according to the absolute error values of the plurality of resource index prediction values;
an isolated tree construction module for: constructing an isolated tree according to the optimized values of the plurality of resource indexes and the weights of the plurality of resource indexes;
the anomaly score acquisition module is used for: obtaining abnormal scores of the resource index optimization values according to a plurality of the isolated trees;
the abnormality judgment module is used for: if the abnormal score is greater than or equal to a score threshold, determining that the resource index predicted value corresponding to the resource index optimized value is abnormal;
the linear regression model is a linear regression model between the service request data values and the resource indicator predictive values.
In a third aspect, an embodiment of the present application provides an electronic device, including a processor and a memory storing a computer program, where the processor implements the steps of the data anomaly detection method according to the first aspect when executing the program.
In a fourth aspect, an embodiment of the present application provides a computer program product, including a computer program, which when executed by a processor implements the steps of the data anomaly detection method described in the first aspect.
According to the data anomaly detection method and device, a service request data value is input into a linear regression model to obtain a plurality of resource index predicted values, a plurality of resource index optimized values are obtained according to the absolute error values of the plurality of resource index predicted values, an isolated tree is constructed according to the plurality of resource index optimized values and the weights of the plurality of resource indexes, and finally an anomaly score of the resource index optimized values is obtained according to the plurality of isolated trees, and if the anomaly score is greater than or equal to a score threshold value, the resource index predicted value corresponding to the resource index optimized value is determined to be anomalous. The application adopts an improved isolated forest algorithm, utilizes a linear regression model to represent the relation between a service request data value and a resource index predicted value, utilizes the absolute value of the error of the resource index predicted value to obtain a plurality of resource index optimized values, can correct and optimize resource index data with volatility, so that the resource index data becomes more stable, thereby better distinguishing normal data and abnormal data in massive resource index data, avoiding missed detection and improving the efficiency and accuracy of abnormal detection.
Drawings
In order to more clearly illustrate the application or the technical solutions of the prior art, the following description will briefly explain the drawings used in the embodiments or the description of the prior art, and it is obvious that the drawings in the following description are some embodiments of the application, and other drawings can be obtained according to the drawings without inventive effort for a person skilled in the art.
FIG. 1 is a schematic flow chart of a data anomaly detection method according to an embodiment of the present application;
FIG. 2 is a second flowchart of a data anomaly detection method according to an embodiment of the present application;
FIG. 3 is a schematic diagram of an isolated tree structure of a data anomaly detection method according to an embodiment of the present application;
fig. 4 is a schematic structural diagram of a data anomaly detection device according to an embodiment of the present application;
fig. 5 is a schematic structural diagram of an electronic device according to an embodiment of the present application.
Detailed Description
For the purpose of making the objects, technical solutions and advantages of the present application more apparent, the technical solutions of the present application will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present application, and it is apparent that the described embodiments are some embodiments of the present application, but not all embodiments. All other embodiments, which can be made by those skilled in the art based on the embodiments of the application without making any inventive effort, are intended to be within the scope of the application.
Fig. 1 is a schematic flow chart of a data anomaly detection method according to an embodiment of the present application. Referring to fig. 1, an embodiment of the present application provides a data anomaly detection method, which may include:
101. inputting the service request data value into a linear regression model to obtain a plurality of resource index predicted values;
102. obtaining a plurality of resource index optimization values according to the absolute error values of the plurality of resource index prediction values;
103. constructing an isolated tree according to the optimized values of the plurality of resource indexes and the weights of the plurality of resource indexes;
104. obtaining abnormal scores of the resource index optimization values according to a plurality of isolated trees;
105. if the abnormal score is greater than or equal to the score threshold, determining that the resource index predicted value corresponding to the resource index optimized value is abnormal;
the linear regression model is a linear regression model between the service request data values and the resource indicator predictive values.
In step 101, the service request data may be the number of times of requesting the host device for service, and the resource index may be a CPU usage amount, a memory usage amount, a network in bandwidth, a network out bandwidth, a block device read bandwidth, and a block device write bandwidth, where each resource index is of one type, for example, the CPU usage amount is of a first type, the memory usage amount is of a second type, the network in bandwidth is of a third type, the network out bandwidth is of a fourth type, the block device read bandwidth is of a fifth type, and the block device write bandwidth is of a sixth type. It should be noted that the resource index may be other types of indexes, which are not limited herein.
In step 102, the absolute value of the difference between each predicted value of the resource index and the actual value thereof is calculated, and the absolute value of the difference is used as the optimized value of the resource index of the predicted value of the resource index, so as to obtain a plurality of optimized values of the resource index.
In step 104, a plurality of isolated trees are obtained by repeating some of the steps 103, the plurality of isolated trees forming an isolated forest.
In step 105, if it is determined that the predicted value of a certain resource index is abnormal, an alarm can be given to the resource index, and because the efficiency and accuracy of the abnormality detection method of the present application are high, the alarm efficiency and accuracy are also improved, so that the fault problem can be rapidly located, and the fault detection efficiency is greatly improved.
According to the data anomaly detection method provided by the embodiment, a service request data value is input into a linear regression model to obtain a plurality of resource index predicted values, a plurality of resource index optimized values are obtained according to the absolute values of errors of the plurality of resource index predicted values, an isolated tree is constructed according to the plurality of resource index optimized values and the weights of the plurality of resource indexes, finally an anomaly score of the resource index optimized values is obtained according to the plurality of isolated trees, and if the anomaly score is greater than or equal to a score threshold value, the resource index predicted value corresponding to the resource index optimized value is determined to be anomalous. According to the embodiment, an improved isolated forest algorithm is adopted, a linear regression model is utilized to represent the relation between a service request data value and a resource index predicted value, and the absolute value of the error of the resource index predicted value is utilized to obtain a plurality of resource index optimized values, so that resource index data with volatility can be corrected and optimized, the resource index data become more stable, normal data and abnormal data in massive resource index data can be distinguished well, missed detection is avoided, the efficiency and the accuracy of abnormal detection are improved, meanwhile, the weight of the resource index is introduced to construct an isolated tree, the preference of the system service to specific resources can be highlighted by the weight, and the abnormal situation is easy to occur due to the fact that the system service is more preferred to the resource index, so that the isolated tree is constructed by the weight of the resource index, the specific resources can be highlighted in the construction process of the isolated tree, and the efficiency and the accuracy of abnormal detection of massive data are further improved.
In one embodiment, the linear regression model in the data anomaly detection method is constructed based on the following steps:
according to a polynomial regression method, constructing a linear regression model which takes a service request data value at a first moment as an independent variable and takes a resource index predicted value at a second moment as a dependent variable;
the second time is the next time to the first time.
In the field of operation and maintenance, the application is wider than the key performance indexes of conventional resource types such as CPU, memory, network, disk and the like. These metrics, while accurately describing the operational state of the online service, cannot be directly applied in an online service operating environment with volatility. The traditional isolated forest algorithm utilizes two distinct features of anomaly data:
1. the abnormal data occupies smaller total data set;
2. there is a clear difference between the abnormal data and the normal data.
By analyzing the online service, a conclusion that the online service resource index data has volatility is obtained, so that the problem that part of normal data points and abnormal data points are mixed together may exist. In this case, if the isolated tree and the isolated forest are still constructed using the unmodified resource index data, irrespective of the real running state of the online service, an abnormal data miss event may be caused, so that the recall rate of the algorithm is affected. Therefore, the present embodiment provides a data optimization method specifically, based on a historical service request data value and a historical resource index predicted value, a polynomial regression method is used to calculate a linear relationship between the historical service request data value and the historical resource index predicted value, then before each anomaly detection is performed, the service request data value is input into a linear relationship equation, the resource index predicted value is calculated, then the absolute value of the difference between the resource index predicted value and the actual value is calculated, and the absolute value of the difference is used as an optimized value of the resource index predicted value, so that a group of stable data which can still embody the actual running state of the service under the condition of continuous fluctuation of the load is obtained. Specifically, the linear regression model in the present embodiment may be as follows:
(RES k ) predict ＝a*QPS k-1 2 +b*QPS k-1 +c (1-1)
Wherein, (RES) k ) predict QPS is the predicted value of the resource index at the kth moment k-1 A data value for the service request at time k-1; a. b and c are parameters of the formula (1-1), and are obtained by fitting historical service request data values and historical resource index predicted values.
Since the resource index data of the online service has a certain hysteresis compared with the service request data, the service request data value at the first time is used to predict the resource index predicted value at the second time, which is the next time to the first time.
Further, the resource indicator optimization value may be calculated according to the following formula:
X K ＝|(RES k ) predict -(RES k ) true | (1-2)
wherein X is K Optimizing a value for a resource index at a kth time, (RES) k ) true Is the true value of the resource index at the kth moment.
It should be noted that, the linear regression model in this embodiment is a model between a service request data value and a single type of resource index predicted value, for example, fitting is performed according to a historical service request data value and a historical CPU usage amount (the data to be fitted is normal data), so as to obtain a linear regression model between the service request data value and the CPU usage amount predicted value, fitting is performed according to a historical service request data value and a historical memory usage amount, so as to obtain a linear regression model between the service request data value and the memory usage amount predicted value, and so on, so that service request data values at multiple moments can be input into different linear regression models, so as to obtain multiple types of resource index predicted values at multiple moments.
According to the method, the device and the system, the resource index predicted value is obtained by constructing the linear regression model, and the absolute value of the error of the resource index predicted value is used as the resource index optimized value, so that the resource index predicted value in a fluctuation state is more stable, and the normal data and the abnormal data in the resource index predicted value can be differentiated.
In one embodiment, before constructing the orphan tree from the plurality of resource-indicator-optimized values and the plurality of resource-indicator weights, the method comprises:
if the sum of the resource index predicted values of any type at a plurality of moments is zero, determining that the weight of the resource index corresponding to the resource index predicted value of the type is zero;
if the sum of the resource index predicted values of any type at a plurality of moments is not zero, obtaining the weight of the resource index corresponding to the resource index predicted value of the type according to the change rate of the resource index predicted value between the ith moment and the (i+5) th moment and the change rate of the service request data value between the ith moment and the (i+5) th moment;
where i is an integer of 1 or more and n or less, and n is the total number of times.
In the traditional isolated forest algorithm, the probability that each type of resource index is selected as the type to be divided is the same, but in practice, each online service has the preferred resource, in general, the more the service is preferred to a certain type of resource, the higher the probability that the abnormal occurrence of the type of resource occurs, and the greater the negative influence is caused when the problem really occurs. Therefore, the method of randomly selecting the resource type is still adopted to construct the isolated tree and the isolated forest to search the abnormal data of the online service, and the accuracy is poor.
Therefore, the embodiment provides an improved method for pertinently, a weight value is calculated for each type of resource index based on the preference of the requested service to the resource, and the random selection of the type to be divided of the isolated forest algorithm is changed into weighted selection on the basis. Therefore, the probability that the resource with the service preference is selected as the type to be divided is improved, so that the abnormal service state caused by the problem of the resource with the preference can be found more quickly, the processing is carried out in advance, and the negative influence is reduced.
Specifically, the weight of the resource index corresponding to any type of resource index predictor may be calculated according to the following formula:
wherein N is the weight of the resource index corresponding to the predicted value of the resource index of any type, RES i Is the predicted value of the resource index at the i-th time, Δ (RES i ) QPS for the rate of change of resource index predictor of that type between the i-th time and the i+5-th time i For the service request data value at the i-th time, Δ (QPS i ) For the rate of change of the service request data value between the i-th time and the i+5-th time, N 0 For the initial weight value, the fixed setting is 2, n is the total number of time.
The problem that the change rate of the service request data value and the change rate of the resource index predicted value are smaller may occur due to the too short statistical time, resulting in delta (RES i )/Δ(QPS i ) The calculation is inaccurate, and therefore, the present embodiment calculates Δ (RES i )/Δ(QPS i ). If the usage of a certain type of resource index is always zero in n times, which means that the requested service does not use the resource, the weight value of the resource index of the certain type is set to zero, and the resource index of the certain type is not considered in abnormality detection. Due to delta (QPS during service operation i ) Is always in dynamic change and the requested service preferenceThe amount of resources used will vary with the service request data value, delta (RES i )/Δ(QPS i ) The larger the weight value N, the more preferred the service that is specifying the request is for that type of resource.
The following describes the calculation method of the weight value of the present embodiment by way of an example:
assuming that n=10, i.e., the CPU usage prediction value of the adjacent 10 times and the service request data value of the same 10 times are selected, the weight value of the CPU usage is calculated.
If the sum of the predicted values of the CPU usage at the 10 moments is zero, determining that the weight value of the CPU usage is zero;
if the sum of the predicted values of the CPU usage at the 10 times is not zero, calculating the change rate of the predicted value of the CPU usage at the 1 st time and the 6 th time and the change rate of the service request data value at the 1 st time and the 6 th time, calculating the ratio of the two change rates as a first ratio, calculating the change rate of the predicted value of the CPU usage at the 6 th time and the 11 th time and the change rate of the service request data value at the 6 th time and the 11 th time, calculating the ratio of the two change rates as a second ratio, dividing the first ratio by 2 (namely 10 divided by 5), and finally adding 2 (namely N 0 ) And obtaining the weight of the CPU usage amount, and the weight calculation method of other types of resource indexes is the same.
Note that, in this embodiment, the data for weight calculation is normal data.
According to the embodiment, the weights of various types of resource indexes are calculated, the preference of the requested service to the resource can be represented through the weights, so that the abnormality is found in the resource with the preference of the service in time, and an alarm is sent out for the abnormal resource index in time.
FIG. 2 is a second flowchart of a data anomaly detection method according to an embodiment of the present application;
FIG. 3 is a schematic diagram of an isolated tree structure of a data anomaly detection method according to an embodiment of the present application;
referring to fig. 2-3, in one embodiment, constructing an orphan tree from a plurality of resource-indicator-optimized values and weights of a plurality of resource indicators may include:
201. constructing a plurality of data points according to the plurality of resource index optimization values;
any one data point comprises at least one type of resource index optimization value, and the types of the resource index optimization values included in any two data points are the same;
202. placing a plurality of data points into a root node of an isolated tree, and taking the root node as a current node;
203. adding the weights of the resource indexes corresponding to the resource index predicted values of all types to obtain a total weight value;
204. Randomly selecting any numerical value from zero to the total weight value to obtain a type judgment value;
205. determining the type to be divided according to the size relation between the type judgment value and the weight threshold value;
the type to be divided is one of the types of the resource index optimization values included in the data points;
206. randomly selecting any value between a specific maximum value and a specific minimum value to obtain a dividing threshold value;
the specific maximum value is the maximum value of the resource index optimization values belonging to the type to be divided in the plurality of data points, and the specific minimum value is the minimum value of the resource index optimization values belonging to the type to be divided in the plurality of data points;
207. dividing the data point of the current node into a first data point set and a second data point set according to the type to be divided and the dividing threshold value, placing the first data point set into a first child node of the isolated tree, placing the second data point set into a second child node of the isolated tree, taking the first child node and the second child node as the current node, and returning to the step 204;
208. if only one data point exists in the current node of the isolated tree or the height of the isolated tree reaches the height threshold value, the construction of the isolated tree is completed.
In step 201, the plurality of resource indicator optimization values are the resource indicator optimization values to be detected. Assuming that each data point includes 6 types of resource index optimization values, i.e., a CPU usage amount, a memory usage amount, a network in bandwidth, a network out bandwidth, a block device read bandwidth, and a block device write bandwidth, different data points represent resource index optimization values at different times, and the obtained multiple data points may be as shown in the following table:
Table 1 isolated tree data point case table
| CPU usage | Memory usage | Network ingress bandwidth | Network out-of-band bandwidth | Block device read bandwidth | Block device write bandwidth | |
| A | CPU optimum value 1 | Memory optimized value 1 | Into bandwidth optimization value 1 | Out bandwidth optimized value 1 | Read bandwidth optimization value 1 | Write bandwidth optimization value 1 |
| B | CPU optimum value 2 | Memory optimized value 2 | Into bandwidth optimization value 2 | Out bandwidth optimized value 2 | Read bandwidth optimization value 2 | Write bandwidth optimization value 2 |
| C | CPU optimum 3 | Memory optimized value 3 | Into bandwidth optimization value 3 | Out bandwidth optimized value 3 | Read bandwidth optimization value 3 | Write bandwidth optimization value 3 |
| D | CPU optimum 4 | Memory optimized value 4 | Into bandwidth optimization value 4 | Out bandwidth optimized value 4 | Read bandwidth optimization value 4 | Write Bandwidth optimization value 4 |
| E | CPU optimum 5 | Memory optimized value 5 | Into bandwidth optimization value 5 | Out bandwidth optimization value 5 | Read bandwidth optimization value 5 | Write Bandwidth optimization value 5 |
| F | CPU optimum 6 | Memory optimized value 6 | Into bandwidth optimization value 6 | Out bandwidth optimized value 6 | Read bandwidth optimization value 6 | Write Bandwidth optimization 6 |
| G | CPU optimum 7 | Memory optimized value 7 | Into bandwidth optimization value 7 | Out bandwidth optimized value 7 | Read bandwidth optimization value 7 | Write Bandwidth optimization value 7 |
| H | CPU optimum 8 | Memory optimized value 8 | Into bandwidth optimization value 8 | Out bandwidth optimized value 8 | Read bandwidth optimization value 8 | Write bandwidth optimization value 8 |
As shown in the table above, these 8 data points A, B, C, D, E, F, G, H were constructed from 48 resource index optimization values.
In step 202, the 8 data points A, B, C, D, E, F, G, H are placed into the root node of the orphan tree shown in fig. 3 and the root node is taken as the current node.
In step 203, the weights of the 6 types of resource indexes including the CPU usage, the memory usage, the network in bandwidth, the network out bandwidth, the block device read bandwidth and the block device write bandwidth are summed to obtain a summed weight value.
In step 206, assuming that the type to be divided determined in step 205 is the amount of CPU usage, and the maximum value from the CPU optimized value 1 to the CPU optimized value 8 is the CPU optimized value 2, and the minimum value is the CPU optimized value 6, any value between the CPU optimized value 2 and the CPU optimized value 6 is selected as the dividing threshold.
In step 207, if any one of the CPU optimization values 1 to 8 is greater than the dividing threshold, dividing the data point corresponding to the CPU optimization value into a first data point set; if any one of the CPU optimization values 1 to 8 is smaller than the division threshold, the data point corresponding to the CPU optimization value is divided into a second data point set, as shown in fig. 3, assuming that the CPU optimization value 1 is greater than the division threshold and the CPU optimization value 2 to 8 is smaller than the division threshold, the data point a is divided into a first data point set, the data point B to the data point H are divided into a second data point set, the data point a is placed into a first child node of the isolation tree, and the data point B to the data point H are placed into a second child node of the isolation tree.
The first sub-node and the second sub-node are used as current nodes, and return to step 204 to reselect a type determination value, and again determine a type to be divided and a dividing threshold, and since the first sub-node has only one data point, there is no need to divide again, so the data point of the second sub-node is divided equally according to the new type to be divided and the new dividing threshold, as shown in fig. 3, assuming that data point B and data point C are placed in one sub-node of the second sub-node, data point D to data point H are placed in another sub-node of the second sub-node, and then return to step 204 again with two sub-nodes of the second sub-node as current nodes, and so on.
It should be noted that the type to be divided selected for dividing the data point of the current node may be the same or different, but is not limited herein, and when the type to be divided is the same, the dividing threshold value must be different, so as to achieve effective division of the data point.
The same type to be divided is selected when dividing the data point of the current node each time, and the detection of single type resource index data is generally aimed at, for example, only detecting whether the CPU usage amount is abnormal or not, and then only selecting different dividing thresholds according to the CPU optimization value in each data point so as to divide each data point continuously.
The different types to be divided are selected when dividing the data point of the current node each time, and generally, detection is generally performed on multiple types of resource index data, for example, whether 6 types of resource index data, such as a CPU usage amount, a memory usage amount, a network input bandwidth, a network output bandwidth, a block device read bandwidth and a block device write bandwidth, are abnormal needs to be detected, a division threshold is selected according to multiple types of resource index optimization values in each data point, and each division is performed on the data point with different types to be divided, in this embodiment, detection is performed on multiple types of resource index data.
In step 208, assuming that the height threshold of the orphan tree is 4, the orphan tree constructed according to step 207 is shown in fig. 3, and if there is only one data point in each node of the orphan tree in fig. 3, it is indicated that the data point has a large difference from other data points, and is a suspected abnormal data point, and the resource index optimization value belonging to the corresponding type to be partitioned in the suspected abnormal data point is suspected abnormal data, for example, the data point a in fig. 3 is partitioned by the CPU usage amount, and the CPU optimization value 1 in the data point a is suspected abnormal data.
According to the embodiment, the isolated tree is constructed through the improved isolated forest algorithm, and the suspected abnormal data points can be accurately distinguished from the normal data points, so that the suspected abnormal data in the suspected abnormal data points can be found out.
In one embodiment, obtaining the anomaly score for the resource indicator optimization value from a plurality of orphaned trees may include:
and obtaining the abnormal score of the resource index optimization value according to the average height of the plurality of isolated trees and the average path height of the resource index optimization value in the plurality of isolated trees.
Constructing a plurality of isolated trees according to the steps, wherein the type judgment value and the partition threshold value are selected randomly, and the selected type to be partitioned and the partition threshold value are different when the isolated trees are constructed each time, so that the heights of the isolated trees may be different, and the path heights of any resource index optimization value in the plurality of isolated trees may be different.
The height of each isolated tree is the maximum node height, the path height of any resource index optimization value in a single isolated tree is the number of nodes through which the resource index optimization value is found from the root node of the single isolated tree downwards, for example, in fig. 3, the height of the isolated tree is 4, the path height of the cpu optimization value 1 is 2 (the path height of the cpu optimization value passes through the root node and the first node in the searching process), and it is required to be explained that when the child node of the resource index optimization value existing independently is found, the resource index optimization value is found.
The anomaly score for a resource indicator optimization value can be optimized according to the following formula:
s(x，m)＝2 (-I(h(x))/d(m)) ，m>2 (1-4)
L(y)＝ln(y)+ξ (1-6)
wherein x is the optimized value of the resource index to be scored, m is the number of all the optimized values of the resource index for constructing the isolated forest, as in table 1, m is 48, s (x, m) is the optimized value of the resource index to be scored is x, when the number of the optimized values of the resource index is m, h (x) is the abnormal score of the optimized value of the resource index x to be scored, h (x) is the path height of the optimized value of the resource index x to be scored in a single isolated tree, I (h (x)) is the average path height of the optimized value of the resource index x to be scored in all the isolated trees, d (m) is the average height of all the isolated trees, and ζ is the euler coefficient.
In the formulas (1-4), d (m) of each isolated forest is fixed, so that the anomaly score s (x, m) is inversely proportional to the average path height I (h (x)) of the resource index optimal value x to be scored, the smaller I (h (x)) is, the larger s (x, m) is, and the higher the anomaly score of the resource index optimal value to be scored is. The value range of the anomaly score s (x, m) is [0,1], and is generally about 0.5, and the following relation is satisfied between I (h (x)) and s (x, m):
when I (h (x)) approaches 0, s (x, m) approaches 1;
when I (h (x)) approaches m-1, s (x, m) approaches 0;
When I (h (x)) approaches d (m), s (x, m) approaches 0.5.
According to the embodiment, the abnormal scores of the resource index optimization values are calculated rapidly and accurately by combining a plurality of isolated trees, and the abnormal resource index optimization values can be found rapidly in the subsequent comparison with the score threshold values, so that the corresponding abnormal resource index prediction values are found according to the abnormal resource index optimization values.
In one embodiment, the data anomaly detection method can generate various reports or reports according to the data anomaly detection result, and specifically comprises the following steps:
1. aiming at single type resource index data, the dividing threshold value can be dynamically adjusted according to the abnormal data found in the mass data to generate an abnormal detection report;
2. aiming at various types of resource index data, big data statistical analysis can be performed, and the influence duty ratio of various types of resource index data when faults occur is analyzed and an abnormal analysis report is generated by aggregating various types of resource index data;
3. aiming at navigation type fault investigation, the existing fault investigation flow can be summarized and solidified into a workflow, the workflow is combined with single type resource index data anomaly detection, a division threshold value is dynamically adjusted, abnormal resource index data are identified again, and a fault analysis report is generated in real time.
The data anomaly detection device provided in the embodiment of the present application is described below, and the data anomaly detection device described below and the data anomaly detection method described above may be referred to correspondingly to each other.
Fig. 4 is a schematic structural diagram of a data anomaly detection device according to an embodiment of the present application. Referring to fig. 4, an embodiment of the present application provides a data anomaly detection apparatus, which may include:
the resource index prediction value obtaining module 401 is configured to: inputting the service request data value into a linear regression model to obtain a plurality of resource index predicted values;
a resource index optimization value acquisition module 402, configured to: obtaining a plurality of resource index optimization values according to the absolute error values of the plurality of resource index prediction values;
an orphan tree construction module 403 for: constructing an isolated tree according to the optimized values of the plurality of resource indexes and the weights of the plurality of resource indexes;
an anomaly score acquisition module 404 configured to: obtaining abnormal scores of the resource index optimization values according to a plurality of the isolated trees;
an anomaly determination module 405, configured to: if the abnormal score is greater than or equal to a score threshold, determining that the resource index predicted value corresponding to the resource index optimized value is abnormal;
The linear regression model is a linear regression model between the service request data values and the resource indicator predictive values.
According to the data anomaly detection device provided by the embodiment, a service request data value is input into a linear regression model to obtain a plurality of resource index predicted values, a plurality of resource index optimized values are obtained according to the absolute values of errors of the plurality of resource index predicted values, an isolated tree is constructed according to the plurality of resource index optimized values and the weights of the plurality of resource indexes, finally anomaly scores of the resource index optimized values are obtained according to the plurality of isolated trees, and if the anomaly scores are greater than or equal to a score threshold value, the resource index predicted values corresponding to the resource index optimized values are determined to be anomalous. According to the embodiment, an improved isolated forest algorithm is adopted, a linear regression model is utilized to represent the relation between a service request data value and a resource index predicted value, and the absolute value of the error of the resource index predicted value is utilized to obtain a plurality of resource index optimized values, so that resource index data with volatility can be corrected and optimized, the resource index data become more stable, normal data and abnormal data in massive resource index data can be distinguished well, missed detection is avoided, the efficiency and the accuracy of abnormal detection are improved, meanwhile, the weight of the resource index is introduced to construct an isolated tree, the preference of the system service to specific resources can be highlighted by the weight, and the abnormal situation is easy to occur due to the fact that the system service is more preferred to the resource index, so that the isolated tree is constructed by the weight of the resource index, the specific resources can be highlighted in the construction process of the isolated tree, and the efficiency and the accuracy of abnormal detection of massive data are further improved.
In one embodiment, the method further comprises a linear regression model building module (not shown in the figure) for:
according to a polynomial regression method, constructing a linear regression model which takes a service request data value at a first moment as an independent variable and takes a resource index predicted value at a second moment as a dependent variable;
the second time is the next time to the first time.
In one embodiment, the resource indicator predictor obtaining module 401 is specifically configured to:
and inputting the service request data values at a plurality of moments into the linear regression model to obtain the predicted values of the resource indexes of a plurality of types at the plurality of moments.
In one embodiment, the system further comprises a weight acquisition module (not shown in the figure) for:
if the sum of the resource index predicted values of any type of n moments is zero, determining that the weight of the resource index corresponding to the resource index predicted value of the type is zero;
if the sum of the resource index predicted values of any type of n moments is not zero, obtaining the weight of the resource index corresponding to the resource index predicted value of the type according to the change rate of the resource index predicted value between the ith moment and the (i+5) th moment and the change rate of the service request data value between the ith moment and the (i+5) th moment;
Wherein i is an integer of 1 or more and n or less, and n is the total number of the moments.
In one embodiment, the orphan tree building module 403 is specifically configured to:
constructing a plurality of data points according to the plurality of resource index optimization values; any one data point comprises at least one type of resource index optimization value, and the types of the resource index optimization values included in any two data points are the same;
placing the plurality of data points into a root node of the isolated tree, and taking the root node as a current node;
adding the weights of the resource indexes corresponding to the resource index predicted values of all types to obtain a total weight value;
randomly selecting any numerical value from zero to the total weight value to obtain a type judgment value;
determining the type to be divided according to the size relation between the type judgment value and the weight threshold value; the type to be divided is one of the types of the resource index optimization values included in the data points;
randomly selecting any value between a specific maximum value and a specific minimum value to obtain a dividing threshold value; the specific maximum value is the maximum value of the resource index optimization values belonging to the type to be divided in the plurality of data points, and the specific minimum value is the minimum value of the resource index optimization values belonging to the type to be divided in the plurality of data points;
Dividing the data point of the current node into a first data point set and a second data point set according to the type to be divided and the dividing threshold value, placing the first data point set into a first child node of the isolated tree, placing the second data point set into a second child node of the isolated tree, taking the first child node and the second child node as the current node, returning to randomly select any value from zero to the total weight value, and obtaining a type judgment value until only one data point exists in the current node of the isolated tree or the height of the isolated tree reaches the height threshold value, and completing the construction of the isolated tree.
In one embodiment, the orphan tree building module 403 is specifically configured to:
if the resource index optimization value belonging to the type to be divided in the data points of the current node is larger than the dividing threshold value, dividing the data points corresponding to the resource index optimization value into a first data point set;
and if the resource index optimization value belonging to the type to be divided in the data points of the current node is smaller than the division threshold value, dividing the data points corresponding to the resource index optimization value into a second data point set.
In one embodiment, the anomaly score acquisition module 404 is specifically configured to:
and obtaining the anomaly score of the resource index optimization value according to the average height of the plurality of the isolated trees and the average path height of the resource index optimization value in the plurality of the isolated trees.
Fig. 5 illustrates a physical schematic diagram of an electronic device, as shown in fig. 5, which may include: processor 510, communication interface (Communication Interface) 520, memory 530, and communication bus 540, wherein processor 510, communication interface 520, memory 530 complete communication with each other through communication bus 540. Processor 510 may invoke a computer program in memory 530 to perform the steps of a data anomaly detection method, including, for example:
inputting the service request data value into a linear regression model to obtain a plurality of resource index predicted values;
obtaining a plurality of resource index optimization values according to the absolute error values of the plurality of resource index prediction values;
constructing an isolated tree according to the optimized values of the plurality of resource indexes and the weights of the plurality of resource indexes;
obtaining abnormal scores of the resource index optimization values according to a plurality of the isolated trees;
If the abnormal score is greater than or equal to a score threshold, determining that the resource index predicted value corresponding to the resource index optimized value is abnormal;
the linear regression model is a linear regression model between the service request data values and the resource indicator predictive values.
Further, the logic instructions in the memory 530 described above may be implemented in the form of software functional units and may be stored in a computer-readable storage medium when sold or used as a stand-alone product. Based on this understanding, the technical solution of the present application may be embodied essentially or in a part contributing to the prior art or in a part of the technical solution, in the form of a software product stored in a storage medium, comprising several instructions for causing a computer device (which may be a personal computer, a server, a network device, etc.) to perform all or part of the steps of the method according to the embodiments of the present application. And the aforementioned storage medium includes: a U-disk, a removable hard disk, a Read-Only Memory (ROM), a random access Memory (RAM, random Access Memory), a magnetic disk, or an optical disk, or other various media capable of storing program codes.
In another aspect, embodiments of the present application further provide a computer program product, where the computer program product includes a computer program, where the computer program may be stored on a non-transitory computer readable storage medium, where the computer program when executed by a processor is capable of executing the steps of the data anomaly detection method provided in the foregoing embodiments, where the steps include:
inputting the service request data value into a linear regression model to obtain a plurality of resource index predicted values;
obtaining a plurality of resource index optimization values according to the absolute error values of the plurality of resource index prediction values;
constructing an isolated tree according to the optimized values of the plurality of resource indexes and the weights of the plurality of resource indexes;
obtaining abnormal scores of the resource index optimization values according to a plurality of the isolated trees;
if the abnormal score is greater than or equal to a score threshold, determining that the resource index predicted value corresponding to the resource index optimized value is abnormal;
the linear regression model is a linear regression model between the service request data values and the resource indicator predictive values.
In another aspect, embodiments of the present application further provide a processor-readable storage medium storing a computer program for causing a processor to execute the steps of the method provided in the above embodiments, for example, including:
Inputting the service request data value into a linear regression model to obtain a plurality of resource index predicted values;
obtaining a plurality of resource index optimization values according to the absolute error values of the plurality of resource index prediction values;
constructing an isolated tree according to the optimized values of the plurality of resource indexes and the weights of the plurality of resource indexes;
obtaining abnormal scores of the resource index optimization values according to a plurality of the isolated trees;
if the abnormal score is greater than or equal to a score threshold, determining that the resource index predicted value corresponding to the resource index optimized value is abnormal;
the linear regression model is a linear regression model between the service request data values and the resource indicator predictive values.
The processor-readable storage medium may be any available medium or data storage device that can be accessed by a processor, including, but not limited to, magnetic storage (e.g., floppy disks, hard disks, magnetic tape, magneto-optical disks (MOs), etc.), optical storage (e.g., CD, DVD, BD, HVD, etc.), semiconductor storage (e.g., ROM, EPROM, EEPROM, nonvolatile storage (NAND FLASH), solid State Disk (SSD)), and the like.
The apparatus embodiments described above are merely illustrative, wherein the elements illustrated as separate elements may or may not be physically separate, and the elements shown as elements may or may not be physical elements, may be located in one place, or may be distributed over a plurality of network elements. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of this embodiment. Those of ordinary skill in the art will understand and implement the present invention without undue burden.
From the above description of the embodiments, it will be apparent to those skilled in the art that the embodiments may be implemented by means of software plus necessary general hardware platforms, or of course may be implemented by means of hardware. Based on this understanding, the foregoing technical solution may be embodied essentially or in a part contributing to the prior art in the form of a software product, which may be stored in a computer readable storage medium, such as ROM/RAM, a magnetic disk, an optical disk, etc., including several instructions for causing a computer device (which may be a personal computer, a server, or a network device, etc.) to execute the method described in the respective embodiments or some parts of the embodiments.
Finally, it should be noted that: the above embodiments are only for illustrating the technical solution of the present application, and are not limiting; although the application has been described in detail with reference to the foregoing embodiments, it will be understood by those of ordinary skill in the art that: the technical scheme described in the foregoing embodiments can be modified or some technical features thereof can be replaced by equivalents; such modifications and substitutions do not depart from the spirit and scope of the technical solutions of the embodiments of the present application.
Claims (10)
1. A data anomaly detection method, comprising:
inputting the service request data value into a linear regression model to obtain a plurality of resource index predicted values;
obtaining a plurality of resource index optimization values according to the absolute error values of the plurality of resource index prediction values;
constructing an isolated tree according to the optimized values of the plurality of resource indexes and the weights of the plurality of resource indexes;
obtaining abnormal scores of the resource index optimization values according to a plurality of the isolated trees;
if the abnormal score is greater than or equal to a score threshold, determining that the resource index predicted value corresponding to the resource index optimized value is abnormal;
the linear regression model is a linear regression model between the service request data values and the resource indicator predictive values.
2. The method of claim 1, wherein the linear regression model is constructed based on the steps of:
according to a polynomial regression method, constructing a linear regression model which takes a service request data value at a first moment as an independent variable and takes a resource index predicted value at a second moment as a dependent variable;
the second time is the next time to the first time.
3. The method for detecting data anomalies according to claim 2, wherein inputting the service request data values into a linear regression model results in a plurality of resource indicator predictors, including:
And inputting the service request data values at a plurality of moments into the linear regression model to obtain the predicted values of the resource indexes of a plurality of types at the plurality of moments.
4. The method for detecting data anomalies according to claim 1, wherein before constructing an orphan tree from the plurality of resource-indicator-optimized values and the weights of the plurality of resource indicators, comprising:
if the sum of the resource index predicted values of any type of n moments is zero, determining that the weight of the resource index corresponding to the resource index predicted value of the type is zero;
if the sum of the resource index predicted values of any type of n moments is not zero, obtaining the weight of the resource index corresponding to the resource index predicted value of the type according to the change rate of the resource index predicted value between the ith moment and the (i+5) th moment and the change rate of the service request data value between the ith moment and the (i+5) th moment;
wherein i is an integer of 1 or more and n or less, and n is the total number of the moments.
5. The method of claim 4, wherein constructing an orphan tree based on the plurality of resource indicator optimization values and the plurality of resource indicator weights comprises:
Constructing a plurality of data points according to the plurality of resource index optimization values; any one data point comprises at least one type of resource index optimization value, and the types of the resource index optimization values included in any two data points are the same;
placing the plurality of data points into a root node of the isolated tree, and taking the root node as a current node;
adding the weights of the resource indexes corresponding to the resource index predicted values of all types to obtain a total weight value;
randomly selecting any numerical value from zero to the total weight value to obtain a type judgment value;
determining the type to be divided according to the size relation between the type judgment value and the weight threshold value; the type to be divided is one of the types of the resource index optimization values included in the data points;
randomly selecting any value between a specific maximum value and a specific minimum value to obtain a dividing threshold value; the specific maximum value is the maximum value of the resource index optimization values belonging to the type to be divided in the plurality of data points, and the specific minimum value is the minimum value of the resource index optimization values belonging to the type to be divided in the plurality of data points;
dividing the data point of the current node into a first data point set and a second data point set according to the type to be divided and the dividing threshold value, placing the first data point set into a first child node of the isolated tree, placing the second data point set into a second child node of the isolated tree, taking the first child node and the second child node as the current node, returning to randomly select any value from zero to the total weight value, and obtaining a type judgment value until only one data point exists in the current node of the isolated tree or the height of the isolated tree reaches the height threshold value, and completing the construction of the isolated tree.
6. The method of claim 5, wherein the dividing the data point of the current node into the first data point set and the second data point set according to the type to be divided and the division threshold value comprises:
if the resource index optimization value belonging to the type to be divided in the data points of the current node is larger than the dividing threshold value, dividing the data points corresponding to the resource index optimization value into a first data point set;
and if the resource index optimization value belonging to the type to be divided in the data points of the current node is smaller than the division threshold value, dividing the data points corresponding to the resource index optimization value into a second data point set.
7. The method for detecting data anomalies according to claim 1, wherein said obtaining anomaly scores for the resource-indicator optimization values from a plurality of said orphaned trees includes:
and obtaining the anomaly score of the resource index optimization value according to the average height of the plurality of the isolated trees and the average path height of the resource index optimization value in the plurality of the isolated trees.
8. A data anomaly detection device, comprising:
the resource index prediction value acquisition module is used for: inputting the service request data value into a linear regression model to obtain a plurality of resource index predicted values;
The resource index optimization value acquisition module is used for: obtaining a plurality of resource index optimization values according to the absolute error values of the plurality of resource index prediction values;
an isolated tree construction module for: constructing an isolated tree according to the optimized values of the plurality of resource indexes and the weights of the plurality of resource indexes;
the anomaly score acquisition module is used for: obtaining abnormal scores of the resource index optimization values according to a plurality of the isolated trees;
the abnormality judgment module is used for: if the abnormal score is greater than or equal to a score threshold, determining that the resource index predicted value corresponding to the resource index optimized value is abnormal;
the linear regression model is a linear regression model between the service request data values and the resource indicator predictive values.
9. An electronic device comprising a processor and a memory storing a computer program, characterized in that the processor implements the steps of the data anomaly detection method of any one of claims 1 to 7 when the computer program is executed.
10. A computer program product comprising a computer program, characterized in that the computer program when executed by a processor implements the steps of the data anomaly detection method of any one of claims 1 to 7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202211408252.9A CN116933121A (en) | 2022-11-10 | 2022-11-10 | Data anomaly detection method and device |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202211408252.9A CN116933121A (en) | 2022-11-10 | 2022-11-10 | Data anomaly detection method and device |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN116933121A true CN116933121A (en) | 2023-10-24 |
Family
ID=88378036
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202211408252.9A Pending CN116933121A (en) | 2022-11-10 | 2022-11-10 | Data anomaly detection method and device |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN116933121A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN118071386A (en) * | 2024-04-19 | 2024-05-24 | 海门裕隆光电科技有限公司 | Electronic cigarette big data processing method and system |
-
2022
- 2022-11-10 CN CN202211408252.9A patent/CN116933121A/en active Pending
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN118071386A (en) * | 2024-04-19 | 2024-05-24 | 海门裕隆光电科技有限公司 | Electronic cigarette big data processing method and system |
| CN118071386B (en) * | 2024-04-19 | 2024-08-20 | 海门裕隆光电科技有限公司 | Electronic cigarette big data processing method and system |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN111339129B (en) | Remote meter reading abnormity monitoring method and device, gas meter system and cloud server | |
| CN107168995B (en) | Data processing method and server | |
| CN114143326B (en) | Load adjustment method, management node, and storage medium | |
| CN104462808A (en) | Method for fitting safe horizontal displacement and dynamic data of variable sliding window of water level | |
| CN116933121A (en) | Data anomaly detection method and device | |
| CN110932901A (en) | Alarm level adjusting method and system | |
| CN114116828A (en) | Association rule analysis method, device and storage medium for multidimensional network index | |
| CN111598457B (en) | Method and device for determining quality of power wireless network | |
| CN116663747A (en) | Intelligent early warning method and system based on data center infrastructure | |
| CN113300373B (en) | Stability margin value prediction method and device based on PRMSE evaluation index | |
| CN110619413A (en) | Method and device for evaluating health degree of wind generating set | |
| CN112836843B (en) | Base station out-of-service alarm prediction method and device | |
| CN108197498A (en) | Obtain the method and device of data | |
| CN116319255A (en) | Root cause positioning method, device, equipment and storage medium based on KPI | |
| CN111783883A (en) | Abnormal data detection method and device | |
| CN115690681A (en) | Processing method of abnormity judgment basis, abnormity judgment method and device | |
| CN117135037A (en) | Method and device for defining network traffic performance abnormality | |
| CN112882854B (en) | Method and device for processing request exception | |
| CN112015619A (en) | Method for optimizing and screening core key indexes of system through parameters | |
| CN112653129B (en) | Transient power angle stability margin estimation method, device and system | |
| CN116069595B (en) | Operation and maintenance monitoring method based on log | |
| US12019869B2 (en) | Method, electronic device, and computer program product for scheduling data collection | |
| CN117745080B (en) | Multi-factor authentication-based data access control and security supervision method and system | |
| CN116610535B (en) | Method and system for processing operation and maintenance monitoring data of machine room | |
| CN117911010B (en) | Water turbine speed regulator overhaul evaluation equipment, method, device and medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |