CN111984503B - Method and device for identifying abnormal data of monitoring index data - Google Patents
Method and device for identifying abnormal data of monitoring index data Download PDFInfo
- Publication number
- CN111984503B CN111984503B CN202010824430.0A CN202010824430A CN111984503B CN 111984503 B CN111984503 B CN 111984503B CN 202010824430 A CN202010824430 A CN 202010824430A CN 111984503 B CN111984503 B CN 111984503B
- Authority
- CN
- China
- Prior art keywords
- index data
- data
- value
- determining
- current moment
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 230000002159 abnormal effect Effects 0.000 title claims abstract description 70
- 238000000034 method Methods 0.000 title claims abstract description 38
- 238000012544 monitoring process Methods 0.000 title claims abstract description 31
- 230000003321 amplification Effects 0.000 claims description 57
- 238000003199 nucleic acid amplification method Methods 0.000 claims description 57
- 230000009467 reduction Effects 0.000 claims description 55
- 238000004422 calculation algorithm Methods 0.000 claims description 30
- 238000012545 processing Methods 0.000 claims description 22
- 238000004364 calculation method Methods 0.000 claims description 21
- 230000006870 function Effects 0.000 claims description 21
- 238000001514 detection method Methods 0.000 claims description 11
- 238000003860 storage Methods 0.000 claims description 11
- 238000003062 neural network model Methods 0.000 claims description 10
- 238000012217 deletion Methods 0.000 claims description 3
- 230000037430 deletion Effects 0.000 claims description 3
- 230000002547 anomalous effect Effects 0.000 claims 1
- 238000005520 cutting process Methods 0.000 abstract description 3
- 238000010586 diagram Methods 0.000 description 10
- 238000004590 computer program Methods 0.000 description 7
- 230000003247 decreasing effect Effects 0.000 description 7
- 239000011159 matrix material Substances 0.000 description 5
- 230000008569 process Effects 0.000 description 4
- 230000008859 change Effects 0.000 description 3
- 238000012423 maintenance Methods 0.000 description 3
- 238000012935 Averaging Methods 0.000 description 2
- 238000004891 communication Methods 0.000 description 2
- 238000012986 modification Methods 0.000 description 2
- 230000004048 modification Effects 0.000 description 2
- 230000000737 periodic effect Effects 0.000 description 2
- 238000013138 pruning Methods 0.000 description 2
- 238000012549 training Methods 0.000 description 2
- 230000005856 abnormality Effects 0.000 description 1
- 230000004075 alteration Effects 0.000 description 1
- 238000009826 distribution Methods 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 238000001914 filtration Methods 0.000 description 1
- 238000004519 manufacturing process Methods 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 238000012795 verification Methods 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F11/00—Error detection; Error correction; Monitoring
- G06F11/30—Monitoring
- G06F11/3003—Monitoring arrangements specially adapted to the computing system or computing system component being monitored
- G06F11/3037—Monitoring arrangements specially adapted to the computing system or computing system component being monitored where the computing system component is a memory, e.g. virtual memory, cache
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F11/00—Error detection; Error correction; Monitoring
- G06F11/30—Monitoring
- G06F11/3051—Monitoring arrangements for monitoring the configuration of the computing system or of the computing system component, e.g. monitoring the presence of processing resources, peripherals, I/O links, software programs
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F11/00—Error detection; Error correction; Monitoring
- G06F11/30—Monitoring
- G06F11/3065—Monitoring arrangements determined by the means or processing involved in reporting the monitored data
- G06F11/3072—Monitoring arrangements determined by the means or processing involved in reporting the monitored data where the reporting involves data filtering, e.g. pattern matching, time or event triggered, adaptive or policy-based reporting
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/049—Temporal neural networks, e.g. delay elements, oscillating neurons or pulsed inputs
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
Abstract
The invention discloses a method and a device for identifying abnormal data of monitoring index data, comprising the following steps: acquiring index data at the current moment, determining a prediction interval of the index data according to historical index data in a preset period, and determining a fluctuation value of the index data according to the prediction interval and the index data at the current moment. And when the fluctuation value does not accord with the fluctuation threshold value, determining the index data as problem data, wherein the fluctuation threshold value is obtained by comparing the index data with historical index data in a first period before the current moment. If the index data at the current moment is stable, the problem data is determined to be abnormal data. The method and the device realize the identification of the index data at the current moment according to the historical index data, improve the alarm accuracy, and determine the problem data as the abnormal data according to the stability of the index data at the current moment so as to improve the accuracy of identifying the abnormal data and reduce the false alarm caused by the increment of data or the cutting quantity of data.
Description
Technical Field
The invention relates to the field of operation and maintenance, in particular to a method and a device for identifying abnormal data of monitoring index data.
Background
In the prior art, the operation and maintenance data has large scale, burst growth and increased monitoring index categories, such as: system index (such as overload of memory use), service index, however, the index features are many and complex, and the content included in the index features is more and more abundant, such as: stable trend, periodical change, large fluctuation and the like.
In the technology of monitoring indexes, the prior art realizes the monitoring of index data by setting a threshold value through traditional manual experience, and informs a user by generating an alarm when the threshold value is exceeded.
However, the method has long verification period, high maintenance cost, large alarm quantity and false alarm missing, so that the monitoring early warning accuracy and coverage rate are low, faults cannot be timely and accurately found and solved, and complaints are easy to generate, and therefore, a method for identifying abnormal data of monitoring index data is needed, the alarm accuracy is improved, and the alarm quantity is reduced.
Disclosure of Invention
The embodiment of the invention provides a method and a device for identifying abnormal data of monitoring index data, which are used for identifying the abnormal data in the index data after setting the threshold value of the index data, the threshold value is not required to be set manually, the accuracy of identifying the abnormal data is improved, and false alarm caused by data increment or data cutting quantity is reduced.
In a first aspect, an embodiment of the present invention provides a method including:
acquiring index data at the current moment;
according to historical index data in a preset period, determining a prediction interval of the index data;
determining the fluctuation value of the index data according to the prediction interval and the index data at the current moment;
when the fluctuation value does not accord with the fluctuation threshold value, determining the index data as problem data; the fluctuation threshold is obtained by comparing the index data with historical index data in a first period before the current moment;
if the index data at the current moment is stable, determining the problem data as abnormal data; the stability of the index data is determined from historical index data during a second period preceding the current time of the index data.
According to the technical scheme, the prediction interval of the index data at the current moment is obtained according to the determined historical index data in the preset period, then the fluctuation value of the first index and the corresponding fluctuation threshold value are obtained according to the prediction interval, whether the index data at the current moment is problem data or not is determined, the index data at the current moment is determined according to the historical index data, therefore index data identification can be carried out on different types of monitoring indexes, the fluctuation threshold value is obtained according to comparison of the index data at the current moment and the historical index data in the first period before the current moment, the fluctuation threshold value corresponding to the index data at the current moment can be obtained in real time, the alarm accuracy is improved, the problem data is determined to be abnormal data according to the stability of the index data at the current moment, the accuracy of identifying the abnormal data is improved, and false alarm caused by data increment or data cutting quantity is reduced.
Optionally, the determining the prediction interval of the index data according to the historical index data in the preset period includes:
calculating and deleting abnormal values in the historical index data according to a Gaussian function to obtain a mean value of the historical index data; determining a first sample set and a second sample set from the average value of the historical index data;
calculating the similarity of the first sample set and the second sample set according to a DTW algorithm, and if the similarity is smaller than a first threshold value, inputting the average value of the historical index data into an LSTM neural network model for prediction to obtain a predicted value; obtaining the predicted interval according to an interval estimation algorithm by the predicted value;
if the similarity is not smaller than a first threshold value, performing differential detection calculation on the historical index data to obtain a third sample set; carrying out Gaussian function calculation on the third sample set, deleting abnormal values in the third sample set, and obtaining a fourth sample set; and determining the mean value and standard deviation of the fourth sample set, and determining the prediction interval according to an interval estimation algorithm.
According to the technical scheme, abnormal values are removed according to the historical index data, so that a first sample set and a second sample set in the average value of the historical index data are obtained, whether the average value of the historical index data has periodicity or not is determined according to a mode of judging the periodicity of the first sample set and the second sample set, if yes, a prediction interval corresponding to the current index data is determined according to an LSTM neural network model, otherwise, the periodical data in the average value of the historical index data are identified according to differential change point detection, the prediction interval is further obtained, stability of the index data is improved, data errors caused by the abnormal data in the historical index data are reduced, and accuracy of monitoring the abnormal data identification of the index data is improved.
Optionally, the fluctuation value includes an amplitude increasing value and an amplitude decreasing value;
the determining the fluctuation value of the index data according to the prediction interval and the index data of the current moment comprises the following steps:
if the index data is larger than the upper limit value of the prediction interval, determining an amplification value of the index data according to the index data, the upper limit value and the average value of the prediction interval;
and if the index data is smaller than the lower limit value of the prediction interval, determining the amplitude reduction value of the index data according to the index data, the lower limit value and the average value of the prediction interval.
In the above technical solution, the amplitude value includes a first amplitude value and a second amplitude value, and the amplitude value includes a first amplitude value and a second amplitude value. If the index data is larger than the upper limit value of the prediction interval, determining the ratio of the difference value between the index data and the upper limit value of the prediction interval to the upper limit value of the prediction interval as a first increment value of the index data. And determining the ratio of the difference value between the index data and the average value of the prediction interval to the average value of the prediction interval as a second amplification of the index data.
If the index data is smaller than the lower limit value of the prediction interval, determining the ratio of the difference value between the index data and the lower limit value of the prediction interval to the lower limit value of the prediction interval as a first amplitude reduction value of the index data; and determining the ratio of the difference value between the index data and the average value of the prediction interval to the average value of the prediction interval as a second falling amplitude value of the index data. According to the obtained prediction interval, calculating the fluctuation value of the index data at the current moment, so as to reduce the error of the fluctuation value of the index data and improve the accuracy of monitoring the abnormal data identification of the index data.
Optionally, the fluctuation threshold includes an amplification section and a reduction section;
the fluctuation threshold is obtained according to the comparison of the index data and the index historical data in the first period before the current moment, and the method comprises the following steps:
determining a difference value which is larger than 0 between each data in the historical index data in the first period before the current moment and the average value and the maximum value of the prediction interval as an amplification set; performing box type operation on the amplification set to determine the amplification section;
determining a difference value smaller than 0 between each data in the historical index data in the first period before the current moment and the average value and the minimum value of the prediction interval as a reduced amplitude set; and performing box type operation on the amplitude reduction set to determine the amplitude reduction interval.
In the above technical solution, the amplifying section includes a first amplifying section and a second amplifying section, and the reducing section includes a first reducing section and a second reducing section;
and determining a difference value between each data in the historical index data in the first period before the current moment and the maximum value of the prediction interval to be a first amplification set, and performing box operation on the first amplification set to determine a first amplification interval.
And determining a difference value which is larger than 0 between each data in the historical index data in the first period before the current moment and the average value of the prediction interval as a second amplification set, and performing box operation on the second amplification set to determine a second amplification interval.
And determining a difference value smaller than 0 between each data in the historical index data in the first period before the current moment and the minimum value of the prediction interval as a first amplitude reduction set, and performing box operation on the first amplitude reduction set to determine a first amplitude reduction interval.
And determining a difference value smaller than 0 between each data in the historical index data in the first period before the current moment and the average value of the prediction interval as a second amplitude reduction set, and performing box operation on the second amplitude reduction set to determine a second amplitude reduction interval.
According to the prediction interval corresponding to the index data at the current moment and the historical data in the preset period before the current moment, the fluctuation threshold value is obtained, so that the instantaneity of the index data at the current moment is improved, the accuracy of the fluctuation value and the fluctuation threshold value of the index data at the current moment is improved, and the accuracy of monitoring the abnormal data identification of the index data is further improved.
Optionally, when the fluctuation value does not meet the fluctuation threshold, determining that the index data is problem data includes:
if the amplification value of the index data is larger than the upper limit value of the amplification section, determining that the index data is the problem data; or (b)
And if the amplitude reduction value of the index data is smaller than the lower limit value of the amplitude reduction section, determining that the index data is the problem data.
In the above technical solution, if the first increment value of the index data is greater than the upper limit value of the first increment interval and the second increment value is greater than the upper limit value of the second increment interval, determining that the index data is problem data; or (b)
And if the first amplitude reduction value of the index data is smaller than the lower limit value of the first amplitude reduction interval and the second amplitude reduction value is smaller than the lower limit value of the second amplitude reduction interval, determining the index data as problem data. And the index data at the current moment is judged through the threshold values of the two sections, so that the accuracy of monitoring the abnormal data identification of the index data is improved.
Optionally, determining the stability of the index data according to the historical index data in the second period before the current moment of the index data includes:
determining historical index data of which the index data is positioned in a second period before the current moment;
determining a range ratio and a variance according to historical index data in a second period before the current moment; the maximum difference ratio is determined according to the ratio of the difference value between the average value and the minimum value of the historical index data in the second period before the current moment and the difference value between the maximum value and the minimum value of the historical index data in the second period before the current moment;
If the range ratio is smaller than a second threshold value and the variance is smaller than a third threshold value, determining that the index data at the current moment is stable;
and if the range ratio is not smaller than a second threshold value and/or the variance is not smaller than a third threshold value, determining that the index data at the current moment is unstable.
According to the technical scheme, whether the data at the current moment is stable or not is determined, whether the index data at the current moment is in a state of stable data or not is determined, if yes, the fluctuation of the current problem data is judged to be abnormal and belongs to abnormal data, otherwise, the current problem data is judged to be in a state of unstable fluctuation and not to belong to abnormal data, so that abnormal data identification errors are reduced, and the accuracy of monitoring the abnormal data identification of the index data is improved.
Optionally, after determining the abnormal data, the method further includes:
and if the number of the abnormal data reaches the fourth threshold value within the preset period, sending alarm information to the terminal so that the terminal alarms according to the alarm information.
According to the technical scheme, the alarm is carried out according to the mode of setting the threshold value, the situation that the data fluctuation occurs for many times in the preset period is used for alarming the user, the alarm is carried out without once fluctuation, unnecessary trouble caused to the user can be avoided, and the alarm quantity is reduced.
In a second aspect, an embodiment of the present invention provides an apparatus for monitoring abnormal data identification of index data, including:
the acquisition module is used for acquiring index data at the current moment;
the processing module is used for determining a prediction interval of the index data according to the historical index data in a preset period;
determining the fluctuation value of the index data according to the prediction interval and the index data at the current moment;
when the fluctuation value does not accord with the fluctuation threshold value, determining the index data as problem data; the fluctuation threshold is obtained by comparing the index data with historical index data in a first period before the current moment;
if the index data at the current moment is stable, determining the problem data as abnormal data; the stability of the index data is determined from historical index data during a second period preceding the current time of the index data.
Optionally, the processing module is specifically configured to:
calculating and deleting abnormal values in the historical index data according to a Gaussian function to obtain a mean value of the historical index data; determining a first sample set and a second sample set from the average value of the historical index data;
Calculating the similarity of the first sample set and the second sample set according to a DTW algorithm, and if the similarity is smaller than a first threshold value, inputting the average value of the historical index data into an LSTM neural network model for prediction to obtain a predicted value; obtaining the predicted interval according to an interval estimation algorithm by the predicted value;
if the similarity is not smaller than a first threshold value, performing differential detection calculation on the historical index data to obtain a third sample set; carrying out Gaussian function calculation on the third sample set, deleting abnormal values in the third sample set, and obtaining a fourth sample set; and determining the mean value and standard deviation of the fourth sample set, and determining the prediction interval according to an interval estimation algorithm.
Optionally, the fluctuation value includes an amplitude increasing value and an amplitude decreasing value;
the processing module is specifically configured to:
if the index data is larger than the upper limit value of the prediction interval, determining an amplification value of the index data according to the index data, the upper limit value and the average value of the prediction interval;
and if the index data is smaller than the lower limit value of the prediction interval, determining the amplitude reduction value of the index data according to the index data, the lower limit value and the average value of the prediction interval.
Optionally, the fluctuation threshold includes an amplification section and a reduction section;
the processing module is specifically configured to:
determining a difference value which is larger than 0 between each data in the historical index data in the first period before the current moment and the average value and the maximum value of the prediction interval as an amplification set; performing box type operation on the amplification set to determine the amplification section;
determining a difference value smaller than 0 between each data in the historical index data in the first period before the current moment and the average value and the minimum value of the prediction interval as a reduced amplitude set; and performing box type operation on the amplitude reduction set to determine the amplitude reduction interval.
Optionally, the processing module is specifically configured to:
if the amplification value of the index data is larger than the upper limit value of the amplification section, determining that the index data is the problem data; or (b)
And if the amplitude reduction value of the index data is smaller than the lower limit value of the amplitude reduction section, determining that the index data is the problem data.
Optionally, the processing module is specifically configured to:
determining historical index data of which the index data is positioned in a second period before the current moment;
determining a range ratio and a variance according to historical index data in a second period before the current moment; the maximum difference ratio is determined according to the ratio of the difference value between the average value and the minimum value of the historical index data in the second period before the current moment and the difference value between the maximum value and the minimum value of the historical index data in the second period before the current moment;
If the range ratio is smaller than a second threshold value and the variance is smaller than a third threshold value, determining that the index data at the current moment is stable;
and if the range ratio is not smaller than a second threshold value and/or the variance is not smaller than a third threshold value, determining that the index data at the current moment is unstable.
Optionally, the processing module is further configured to:
and if the number of the abnormal data reaches the fourth threshold value within the preset period, sending alarm information to the terminal so that the terminal alarms according to the alarm information.
In a third aspect, embodiments of the present invention also provide a computing device, comprising:
a memory for storing program instructions;
and the processor is used for calling the program instructions stored in the memory and executing the method for identifying the abnormal data of the monitoring index data according to the obtained program.
In a fourth aspect, an embodiment of the present invention further provides a computer-readable storage medium storing computer-executable instructions for causing a computer to perform the above method for identifying abnormal data of monitor index data.
Drawings
In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings that are needed in the description of the embodiments will be briefly described below, and it is obvious that the drawings in the following description are only some embodiments of the present invention, and other drawings may be obtained according to these drawings without inventive effort for a person skilled in the art.
FIG. 1 is a schematic diagram of a system architecture according to an embodiment of the present invention;
FIG. 2 is a flowchart of a method for monitoring abnormal data identification of index data according to an embodiment of the present invention;
FIG. 3 is a schematic diagram of a method for monitoring abnormal data identification of index data according to an embodiment of the present invention;
fig. 4 is a schematic structural diagram of a device for identifying abnormal data of monitoring index data according to an embodiment of the present invention.
Detailed Description
In order to make the objects, technical solutions and advantages of the present invention more apparent, the present invention will be described in further detail below with reference to the accompanying drawings, and it is apparent that the described embodiments are only some embodiments of the present invention, not all embodiments. All other embodiments, which can be made by those skilled in the art based on the embodiments of the invention without making any inventive effort, are intended to be within the scope of the invention.
Fig. 1 illustrates a system architecture to which embodiments of the present invention are applicable, the system architecture including a server 100, the server 100 may include a processor 110, a communication interface 120, and a memory 130.
The communication interface 120 is used for transmitting the historical index data.
The processor 110 is a control center of the server 100, connects various parts of the entire server 100 using various interfaces and routes, and performs various functions of the server 100 and processes data by running or executing software programs and/or modules stored in the memory 130, and calling data stored in the memory 130. Optionally, the processor 110 may include one or more processing units.
The memory 130 may be used to store software programs and modules, and the processor 110 performs various functional applications and data processing by executing the software programs and modules stored in the memory 130. The memory 130 may mainly include a storage program area and a storage data area, wherein the storage program area may store an operating system, application programs required for at least one function, and the like; the storage data area may store data created according to business processes, etc. In addition, memory 130 may include high-speed random access memory, and may also include non-volatile memory, such as at least one magnetic disk storage device, flash memory device, or other volatile solid-state storage device.
It should be noted that the structure shown in fig. 1 is merely an example, and the embodiment of the present invention is not limited thereto.
Based on the above description, fig. 2 illustrates an exemplary flow of a method for monitoring abnormal data identification of index data according to an embodiment of the present invention, where the flow may be executed by an apparatus for monitoring abnormal data identification of index data.
As shown in fig. 2, the process specifically includes:
step 201, index data of the current time is acquired.
According to the embodiment of the invention, the acquired index data at the current moment is identified, the types of the index data are not distinguished, and the method is suitable for identifying index data with different orders, such as the ratio (between 0 and 1), the bandwidth (the power n of 10) and the like.
Step 202, determining a prediction interval of the index data according to the historical index data in a preset period.
According to the embodiment of the invention, the prediction interval corresponding to the index data at the current moment can be obtained according to the historical index data at the current moment, for example, the current moment is 19 points and 27 minutes, and then 19 points 00 to 27 points per day in 30 days in a preset period are obtained, and the prediction interval of the index data at the current moment is obtained according to the historical index data from 19 points 00 to 27 points per day in the first 30 days of the current moment, wherein the preset period is a value which can be set according to experience, for example, the preset period can be 45 days, 60 days and the like.
Further, outliers in the pruned historical index data are calculated according to the Gaussian function to obtain a mean value of the historical index data, and the first sample set and the second sample set are determined from the mean value of the historical index data. Then calculating the similarity of the first sample set and the second sample set according to a DTW algorithm, and if the similarity is smaller than a first threshold value, inputting the average value of the historical index data into an LSTM neural network model for prediction to obtain a predicted value; and obtaining a predicted interval according to the interval estimation algorithm by the predicted value. If the similarity is not smaller than the first threshold, performing differential detection calculation on the historical index data to obtain a third sample set, performing Gaussian function calculation on the third sample set, deleting abnormal values in the third sample set to obtain a fourth sample set, determining the mean value and standard deviation of the fourth sample set, and determining a prediction interval according to an interval estimation algorithm.
According to the embodiment of the invention, through carrying out outlier deletion on historical index data of each day and averaging to obtain sample data of each day (namely, average value of the historical index data with the outlier deleted in each day), obtaining two sample sets (namely, a first sample set and a second sample set) according to all sample data in a set period, determining similarity of the two sample sets, determining whether all sample data have periodicity, if so, directly obtaining a prediction interval of index data at the current moment according to all sample data through an LSTM neural network model, if not, calculating periodic data in all sample data according to differential variable point detection, further obtaining a third sample set with periodicity, deleting outlier in the third sample set, obtaining a fourth sample set, and obtaining the prediction interval of the index data at the current moment according to the fourth sample set through a Gaussian distribution interval estimation algorithm. The LSTM neural network model is obtained by training and learning according to preset historical index data before the current moment.
For example, the historical index data of 15 points 1 minute to 15 points 10 minutes per day in 30 days before the current time is determined, wherein the time series is 1 minute, that is, 10 historical index data are total from 15 points 1 minute to 15 points 10 minutes per day, and 300 historical index data are total in 30 days. And then carrying out Gaussian function calculation by taking 10 pieces of historical index data of each day as a unit, deleting abnormal values in the 10 pieces of historical index data, averaging the rest historical index data to serve as sample data of the day, and finally generating a sample set A containing 30 pieces of sample data. Then dividing the sample set A into a first sample set B and a second sample set C, performing a DTW algorithm on the first sample set B and the second sample set C, and calculating the similarity of the first sample set B and the second sample set C, wherein if the similarity is smaller than a first threshold value, the sample set A has periodicity, otherwise, the sample set A has no periodicity.
When the sample set A has periodicity, inputting the sample set A into the LSTM neural network model to generate a predicted value, taking the predicted value as the average value of the sample set A, and performing an interval estimation algorithm to obtain a predicted interval corresponding to the index data at the current moment.
When the sample set A does not have periodicity, carrying out differential variable point detection calculation on the sample set A, calculating the slope between every two index data in the sample set A, when the absolute value of the slope is higher than a threshold value c and is continuous for a plurality of times, determining that the index data in the sample set A generate incremental data or cut data, filtering the index data to obtain a third sample set E, carrying out Gaussian function calculation on the third sample set E, deleting abnormal values to obtain a fourth sample set F, calculating the mean value and the variance of the fourth sample set F, and determining a prediction interval corresponding to the index data at the current moment according to an interval estimation algorithm.
The gaussian function pruning outliers means pruning values that are too large or too small in the sample set, and if the data does not satisfy the following formula, the outliers are pruned if a samples are included.
μ-ε2*σ<x i <μ-ε1*σ；
Where σ is the standard deviation of the samples, μ is the mean of the sample set, ε is the empirically set weight.
The DTW algorithm is used to measure the similarity of two time series of inconsistent length. For example, two time sequences g= { a1, a2, a3, …, an } and h= { b1, b2, b3, …, bm } are respectively n and m in length, a matrix I with a size of n×m is constructed, a matrix element dij=dist (ai, bj) is searched for a shortest path from d11 to dn in the matrix I at the dij position, and the shortest distance from d11 to dn in the matrix I is regarded as the similarity of the matrix I and the time sequence G, and the shorter the distance is, the more similar, i.e., the smaller the similarity is, the more similar.
The differential variable point detection algorithm is to successively calculate the ratio between every two data for the values in the time sequence, and if the absolute value of the ratio is greater than a threshold value, the data magnitude of the index data is determined to change.
Wherein the interval estimation algorithm needs to satisfy the following formula.
Wherein μ is the mean, σ is the standard deviation, and the confidence is 95%. Confidence is a value that can be empirically set, for example, the value 90%, 85%, etc. can be taken.
In order to better describe the above technical solution, a specific example will be described below.
Example 1
As shown in fig. 3, fig. 3 is a schematic diagram schematically showing a method for monitoring abnormal data identification of index data, which specifically includes the following steps:
step 301, outliers are pruned.
Historical index data of 15 points 1 minute to 15 points 10 minutes per day in 28 days before the current moment is obtained, and 280 samples are taken in total. Outliers in the 10 sample data per day were then subtracted by a gaussian function for the 10 historical index data per day.
In step 302, a sample set is determined.
The historical index data of each day after the outlier is subtracted is averaged to obtain the sample data of each day, and then the sample data containing 28 sample sets J are determined.
Step 303, determining whether periodicity exists, if yes, executing step 304, otherwise executing step 305.
Step 304, a predicted value is generated.
And for a periodic sample set J, inputting the sample set J into an LSTM training model to obtain a predicted value, substituting the predicted value as the average value of the sample set J into an interval estimation algorithm, and generating a predicted interval [ Y1, Y2].
In step 305, a predicted value is calculated.
And for the sample set F without periodicity, a third sample set K with periodicity is obtained through differential variable point detection calculation, then abnormal values in the third sample set K are deleted according to Gaussian function calculation, a fourth sample set L is obtained, the average value of the fourth sample set L is obtained, and a prediction interval [ Y3, Y4] is generated through an interval estimation algorithm.
And 203, determining the fluctuation value of the index data according to the prediction interval and the index data of the current moment.
In the embodiment of the invention, the fluctuation value comprises an amplitude increasing value and a amplitude decreasing value.
Further, if the index data is larger than the upper limit value of the prediction interval, determining the amplification value of the index data according to the index data, the upper limit value and the average value of the prediction interval.
If the index data is smaller than the lower limit value of the prediction interval, determining the amplitude reduction value of the index data according to the index data, the lower limit value and the average value of the prediction interval.
In the embodiment of the invention, the amplitude value comprises a first amplitude value and a second amplitude value, and the amplitude value comprises a first amplitude value and a second amplitude value.
If the index data is larger than the upper limit value of the prediction interval, determining the ratio of the difference value between the index data and the upper limit value of the prediction interval to the upper limit value of the prediction interval as a first increment value of the index data. And determining the ratio of the difference value between the index data and the average value of the prediction interval to the average value of the prediction interval as a second amplification of the index data.
If the index data is smaller than the lower limit value of the prediction interval, determining the ratio of the difference value between the index data and the lower limit value of the prediction interval to the lower limit value of the prediction interval as a first amplitude reduction value of the index data; and determining the ratio of the difference value between the index data and the average value of the prediction interval to the average value of the prediction interval as a second falling amplitude value of the index data.
And determining a first increment value, a first decrement value, a second increment value and a second decrement value of the index data at the current moment according to the upper limit value and the lower limit value of the prediction interval of the index data at the current moment.
For example, if the index data at the current time is 25 and the prediction interval is 10 to 20, the index data at the current time is 20 and is larger than the upper limit value of the prediction interval, and the first increment value of the index data is (25 to 20)/20=25% and the second increment value is (25 to 15)/15=66.6% are determined.
For example, if the index data at the current time exists in the corresponding prediction interval, the index data is considered to be abnormal-free data, and the amplitude calculation is not performed any more. For example, if the prediction interval corresponding to the index data T is [ Y5, Y6], and the minimum value and the maximum value of the prediction interval are simultaneously enlarged by 10, the second prediction interval corresponding to the index data T is [ Y5-10, y6+10], or the prediction interval is enlarged by 1.2 times, the second prediction interval corresponding to the index data T is [ Y5/1.2, y6×1.2], and if the size T of the index data is larger than the minimum value and smaller than the maximum value of the corresponding second prediction interval, the index data T is determined to be data without abnormality.
And 204, determining the index data as problem data when the fluctuation value does not accord with a fluctuation threshold value.
In the embodiment of the invention, the fluctuation threshold is obtained by comparing the index data with the historical index data in the first period before the current moment, and the fluctuation threshold comprises an amplification section and a reduction section.
Specifically, a difference value between each data in the historical index data in the first period before the current moment and the average value and the maximum value of the prediction interval is determined to be more than 0 as an amplification set, and box operation is performed on the amplification set to determine the amplification interval.
And determining a difference value smaller than 0 between the average value and the minimum value of each data in the historical index data in the first period before the current moment and the prediction interval as a reduced amplitude set, and performing box operation on the reduced amplitude set to determine a reduced amplitude interval.
Further, the amplifying section includes a first amplifying section and a second amplifying section, and the decreasing section includes a first decreasing section and a second decreasing section.
And determining a difference value between each data in the historical index data in the first period before the current moment and the maximum value of the prediction interval to be a first amplification set, and performing box operation on the first amplification set to determine a first amplification interval.
And determining a difference value which is larger than 0 between each data in the historical index data in the first period before the current moment and the average value of the prediction interval as a second amplification set, and performing box operation on the second amplification set to determine a second amplification interval.
And determining a difference value smaller than 0 between each data in the historical index data in the first period before the current moment and the minimum value of the prediction interval as a first amplitude reduction set, and performing box operation on the first amplitude reduction set to determine a first amplitude reduction interval.
And determining a difference value smaller than 0 between each data in the historical index data in the first period before the current moment and the average value of the prediction interval as a second amplitude reduction set, and performing box operation on the second amplitude reduction set to determine a second amplitude reduction interval.
According to the embodiment of the invention, the fluctuation threshold of the index data at the current moment can be determined according to the upper limit value and the lower limit value of each data and the prediction interval in the history index data in the first period before the current moment. For example, 30 pieces of history index data 30 minutes before the current time are determined, 30 pieces of history index data are calculated in total, wherein 10 pieces of history index data larger than the upper limit value of the prediction interval are calculated, corresponding difference values between the 10 pieces of history index data and the upper limit value of the prediction interval are determined, box operation is performed on the 10 corresponding difference values, and a first amplification interval of the index data at the current time is determined. Calculating the average value of 30 data and the prediction interval, wherein the number of the historical index data larger than the average value of the prediction interval is 12, determining the corresponding difference value between the 12 historical index data and the average value of the prediction interval, performing box operation on the 12 corresponding difference values, and determining the second amplification interval of the index data at the current moment. Similarly, a first amplitude reduction interval and a second amplitude reduction interval of index data at the current moment can be determined.
It should be noted that, the bin algorithm is one of the quantiles in the quartile statistics, for example, for the data sample R, the difference between the second quantile R25 and the third quantile R75 is identified as IQR, and the reasonable interval of the data is [ R25-n×iqr, r75+n×iqr ], where n is empirically set.
After the index data at the current moment is determined, if the amplification value of the index data is larger than the upper limit value of the amplification section, determining that the index data is problem data. Or (b)
And if the amplitude reduction value of the index data is smaller than the lower limit value of the amplitude reduction section, determining the index data as problem data.
Further, if the first increment value of the index data is greater than the upper limit value of the first increment interval and the second increment value is greater than the upper limit value of the second increment interval, determining the index data as problem data. Or (b)
And if the first amplitude reduction value of the index data is smaller than the lower limit value of the first amplitude reduction interval and the second amplitude reduction value is smaller than the lower limit value of the second amplitude reduction interval, determining the index data as problem data.
In the embodiment of the invention, the index data is determined as the problem data only when the conditions of the first amplification section and the second amplification section or the conditions of the first amplification section and the second amplification section are simultaneously met, otherwise, the index data is determined as the abnormal-free data.
And step 205, if the index data at the current moment is stable, determining the problem data as abnormal data.
According to the embodiment of the invention, the stability of the index data is determined according to the historical index data in the second period before the current moment of the index data.
Specifically, determining historical index data of which the index data is positioned in a second period before the current moment;
and determining a maximum difference ratio and a variance according to the historical index data in the second period before the current moment, wherein the maximum difference ratio is determined according to the ratio of the difference value between the average value and the minimum value of the historical index data in the second period before the current moment and the difference value between the maximum value and the minimum value of the historical index data in the second period before the current moment. And if the range ratio is smaller than the second threshold and the variance is smaller than the third threshold, determining that the index data at the current moment is stable. And if the range ratio is not smaller than the second threshold value and/or the variance is not smaller than the third threshold value, determining that the index data at the current moment is unstable.
According to the embodiment of the invention, the stability of the index data at the current moment is determined through the history index data before the current moment, when the index data at the current moment is determined to be in a stable state, the fluctuation of the current index data is determined to be abnormal, and when the current index data is problem data and is in a stable state, the current index data is determined to belong to abnormal data. For example, a maximum value, a minimum value, and a mean value of the history index data before the current time are determined. When (average value-minimum value)/(maximum value-minimum value) < 0.1 (second threshold value) and variance of the history index data before the current time is < 0.3 (third threshold value), the index data at the previous time is determined to be in a stable state.
After the abnormal data are determined, if the quantity of the abnormal data reaches a fourth threshold value within a preset period, sending alarm information to the terminal so that the terminal alarms according to the alarm information.
According to the embodiment of the invention, when the abnormal data are identified for a plurality of times within a period of time, the data are determined to be abnormal, the alarm prompt is needed, and the alarm is not immediately given after a time point is identified, so that the alarm quantity is reduced. For example, the preset period is set to be half an hour, the fourth threshold is set to be 10, if 10 abnormal data are identified within half an hour, the alarm information is sent to the terminal, and the terminal alarms according to the alarm information to prompt the user.
According to the embodiment of the invention, aiming at monitoring index identification, by combining historical data before the current moment, a plurality of algorithms are adopted to identify whether index data are abnormal data, and corresponding algorithm adjustment is made aiming at two conditions of sudden increase and sudden decrease. The method can predict according to the index data of different periods, and avoid error calculation caused by incremental data and cut data. Meanwhile, the method is suitable for different types of time sequence indexes, and the manual setting of threshold adjustment model parameters is not needed. The identification accuracy is improved, and the alarm quantity is reduced.
Based on the same technical concept, fig. 4 illustrates an exemplary structure of an apparatus for monitoring abnormal data identification of index data, which is provided in an embodiment of the present invention, and the apparatus may perform a flow of a method for monitoring abnormal data identification of index data.
As shown in fig. 4, the apparatus specifically includes:
an obtaining module 401, configured to obtain index data at a current time;
a processing module 402, configured to determine a prediction interval of the index data according to historical index data in a preset period;
determining the fluctuation value of the index data according to the prediction interval and the index data at the current moment;
when the fluctuation value does not accord with the fluctuation threshold value, determining the index data as problem data; the fluctuation threshold is obtained by comparing the index data with historical index data in a first period before the current moment;
if the index data at the current moment is stable, determining the problem data as abnormal data; the stability of the index data is determined from historical index data during a second period preceding the current time of the index data.
Optionally, the processing module 402 is specifically configured to:
Calculating and deleting abnormal values in the historical index data according to a Gaussian function to obtain a mean value of the historical index data; determining a first sample set and a second sample set from the average value of the historical index data;
calculating the similarity of the first sample set and the second sample set according to a DTW algorithm, and if the similarity is smaller than a first threshold value, inputting the average value of the historical index data into an LSTM neural network model for prediction to obtain a predicted value; obtaining the predicted interval according to an interval estimation algorithm by the predicted value;
if the similarity is not smaller than a first threshold value, performing differential detection calculation on the historical index data to obtain a third sample set; carrying out Gaussian function calculation on the third sample set, deleting abnormal values in the third sample set, and obtaining a fourth sample set; and determining the mean value and standard deviation of the fourth sample set, and determining the prediction interval according to an interval estimation algorithm.
Optionally, the fluctuation value includes an amplitude increasing value and an amplitude decreasing value;
the processing module 402 is specifically configured to:
if the index data is larger than the upper limit value of the prediction interval, determining an amplification value of the index data according to the index data, the upper limit value and the average value of the prediction interval;
And if the index data is smaller than the lower limit value of the prediction interval, determining the amplitude reduction value of the index data according to the index data, the lower limit value and the average value of the prediction interval.
Optionally, the fluctuation threshold includes an amplification section and a reduction section;
the processing module 402 is specifically configured to:
determining a difference value which is larger than 0 between each data in the historical index data in the first period before the current moment and the average value and the maximum value of the prediction interval as an amplification set; performing box type operation on the amplification set to determine the amplification section;
determining a difference value smaller than 0 between each data in the historical index data in the first period before the current moment and the average value and the minimum value of the prediction interval as a reduced amplitude set; and performing box type operation on the amplitude reduction set to determine the amplitude reduction interval.
Optionally, the processing module 402 is specifically configured to:
if the amplification value of the index data is larger than the upper limit value of the amplification section, determining that the index data is the problem data; or (b)
And if the amplitude reduction value of the index data is smaller than the lower limit value of the amplitude reduction section, determining that the index data is the problem data.
Optionally, the processing module 402 is specifically configured to:
determining historical index data of which the index data is positioned in a second period before the current moment;
determining a range ratio and a variance according to historical index data in a second period before the current moment; the maximum difference ratio is determined according to the ratio of the difference value between the average value and the minimum value of the historical index data in the second period before the current moment and the difference value between the maximum value and the minimum value of the historical index data in the second period before the current moment;
if the range ratio is smaller than a second threshold value and the variance is smaller than a third threshold value, determining that the index data at the current moment is stable;
and if the range ratio is not smaller than a second threshold value and/or the variance is not smaller than a third threshold value, determining that the index data at the current moment is unstable.
Optionally, the processing module 402 is further configured to:
after the abnormal data are determined, if the quantity of the abnormal data reaches a fourth threshold value within a preset period, sending alarm information to a terminal so that the terminal alarms according to the alarm information.
Based on the same technical concept, the embodiment of the invention further provides a computing device, including:
A memory for storing program instructions;
and the processor is used for calling the program instructions stored in the memory and executing the method for identifying the abnormal data of the monitoring index data according to the obtained program.
Based on the same technical concept, the embodiment of the application also provides a computer readable storage medium, wherein the computer readable storage medium stores computer executable instructions for causing a computer to execute the method for identifying abnormal data of monitoring index data.
It will be appreciated by those skilled in the art that embodiments of the present application may be provided as a method, system, or computer program product. Accordingly, the present application may take the form of an entirely hardware embodiment, an entirely software embodiment or an embodiment combining software and hardware aspects. Furthermore, the present application may take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, and the like) having computer-usable program code embodied therein.
The present application is described with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems) and computer program products according to the application. It will be understood that each flow and/or block of the flowchart illustrations and/or block diagrams, and combinations of flows and/or blocks in the flowchart illustrations and/or block diagrams, can be implemented by computer program instructions. These computer program instructions may be provided to a processor of a general purpose computer, special purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instruction means which implement the function specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
It will be apparent to those skilled in the art that various modifications and variations can be made to the present application without departing from the spirit or scope of the application. Thus, it is intended that the present application also include such modifications and alterations insofar as they come within the scope of the appended claims or the equivalents thereof.
Claims (9)
1. A method of monitoring for identification of anomalous data of index data, comprising:
acquiring index data at the current moment;
according to historical index data in a preset period, determining a prediction interval of the index data;
determining the fluctuation value of the index data according to the prediction interval and the index data at the current moment;
when the fluctuation value does not accord with the fluctuation threshold value, determining the index data as problem data; the fluctuation threshold is obtained by comparing the predicted interval of the index data with historical index data in a first period before the current moment;
if the index data at the current moment is stable, determining the problem data as abnormal data; the stability of the index data is determined according to historical index data in a second period before the current moment of the index data;
the determining the prediction interval of the index data according to the historical index data in the preset period comprises the following steps:
performing outlier deletion and average value calculation on the historical index data of each day to obtain sample data of each day;
determining a first sample set and a second sample set according to all sample data in a preset period;
Calculating the similarity of the first sample set and the second sample set according to a DTW algorithm, and if the similarity is smaller than a first threshold value, inputting the average value of the historical index data into an LSTM neural network model for prediction to obtain a predicted value; obtaining the predicted interval according to an interval estimation algorithm by the predicted value;
if the similarity is not smaller than a first threshold, performing differential detection calculation on the historical index data, determining index data with changed data magnitude in the historical index data, and deleting the index data with changed data magnitude to obtain a third sample set;
carrying out Gaussian function calculation on the third sample set, deleting abnormal values in the third sample set, and obtaining a fourth sample set; and determining the mean value and standard deviation of the fourth sample set, and determining the prediction interval according to an interval estimation algorithm.
2. The method of claim 1, wherein the fluctuation value comprises an amplitude value and a amplitude value; the amplitude value comprises a first amplitude value and a second amplitude value, and the amplitude value comprises a first amplitude value and a second amplitude value;
the determining the fluctuation value of the index data according to the prediction interval and the index data of the current moment comprises the following steps:
If the index data is larger than the upper limit value of the prediction interval, determining a first increment value of the index data according to the index data and the upper limit value of the prediction interval, and determining a second increment value of the index data according to the average value of the index data and the prediction interval;
and if the index data is smaller than the lower limit value of the prediction interval, determining a first drop value of the index data according to the index data and the lower limit value of the prediction interval, and determining a second drop value of the index data according to the average value of the index data and the prediction interval.
3. The method of claim 2, wherein the fluctuation threshold includes an amplification interval and a reduction interval; the amplifying section comprises a first amplifying section and a second amplifying section, and the reducing section comprises a first reducing section and a second reducing section;
the fluctuation threshold is obtained according to the comparison of the predicted interval of the index data and the index history data in the first period before the current moment, and the method comprises the following steps:
determining a difference value which is larger than 0 between each data in the historical index data in the first period before the current moment and the maximum value of the prediction interval as a first amplification set, performing box operation on the first amplification set, and determining the first amplification interval;
Determining a difference value which is larger than 0 between each data in the historical index data in the first period before the current moment and the average value of the prediction interval as a second amplification set, performing box operation on the second amplification set, and determining the second amplification interval;
determining a difference value smaller than 0 between each data in the historical index data in the first period before the current moment and the minimum value of the prediction interval as a first amplitude reduction set, performing box operation on the first amplitude reduction set, and determining the first amplitude reduction interval;
and determining a difference value smaller than 0 between each data in the historical index data in the first period before the current moment and the average value of the prediction interval as a second amplitude reduction set, performing box operation on the second amplitude reduction set, and determining the second amplitude reduction interval.
4. The method of claim 3, wherein the determining the indicator data as problem data when the fluctuation value does not meet a fluctuation threshold value comprises:
if the amplification value of the index data is larger than the upper limit value of the amplification section, determining that the index data is the problem data; or (b)
And if the amplitude reduction value of the index data is smaller than the lower limit value of the amplitude reduction section, determining that the index data is the problem data.
5. The method of claim 1, wherein determining the stability of the metric data based on historical metric data for a second period of time prior to the current time of the metric data comprises:
determining historical index data of which the index data is positioned in a second period before the current moment;
determining a range ratio and a variance according to historical index data in a second period before the current moment; the maximum difference ratio is determined according to the ratio of the difference value between the average value and the minimum value of the historical index data in the second period before the current moment and the difference value between the maximum value and the minimum value of the historical index data in the second period before the current moment;
if the range ratio is smaller than a second threshold value and the variance is smaller than a third threshold value, determining that the index data at the current moment is stable;
and if the range ratio is not smaller than a second threshold value and/or the variance is not smaller than a third threshold value, determining that the index data at the current moment is unstable.
6. The method of claim 1, wherein after determining the anomaly data, further comprising:
and if the number of the abnormal data reaches the fourth threshold value within the preset period, sending alarm information to the terminal so that the terminal alarms according to the alarm information.
7. An apparatus for monitoring identification of abnormal data of index data, comprising:
the acquisition module is used for acquiring index data at the current moment;
the processing module is used for determining a prediction interval of the index data according to the historical index data in a preset period;
determining the fluctuation value of the index data according to the prediction interval and the index data at the current moment;
when the fluctuation value does not accord with the fluctuation threshold value, determining the index data as problem data; the fluctuation threshold is obtained by comparing the predicted interval of the index data with historical index data in a first period before the current moment;
if the index data at the current moment is stable, determining the problem data as abnormal data; the stability of the index data is determined according to historical index data in a second period before the current moment of the index data;
the processing module is specifically configured to:
performing outlier deletion and average value calculation on the historical index data of each day to obtain sample data of each day;
determining a first sample set and a second sample set according to all sample data in a preset period;
Calculating the similarity of the first sample set and the second sample set according to a DTW algorithm, and if the similarity is smaller than a first threshold value, inputting the average value of the historical index data into an LSTM neural network model for prediction to obtain a predicted value; obtaining the predicted interval according to an interval estimation algorithm by the predicted value;
if the similarity is not smaller than a first threshold, performing differential detection calculation on the historical index data, determining index data with changed data magnitude in the historical index data, and deleting the index data with changed data magnitude to obtain a third sample set;
carrying out Gaussian function calculation on the third sample set, deleting abnormal values in the third sample set, and obtaining a fourth sample set; and determining the mean value and standard deviation of the fourth sample set, and determining the prediction interval according to an interval estimation algorithm.
8. A computing device, comprising:
a memory for storing program instructions;
a processor for invoking program instructions stored in said memory to perform the method of any of claims 1 to 6 in accordance with the obtained program.
9. A computer-readable storage medium storing computer-executable instructions for causing a computer to perform the method of any one of claims 1 to 6.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010824430.0A CN111984503B (en) | 2020-08-17 | 2020-08-17 | Method and device for identifying abnormal data of monitoring index data |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010824430.0A CN111984503B (en) | 2020-08-17 | 2020-08-17 | Method and device for identifying abnormal data of monitoring index data |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN111984503A CN111984503A (en) | 2020-11-24 |
| CN111984503B true CN111984503B (en) | 2023-12-01 |
Family
ID=73435282
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202010824430.0A Active CN111984503B (en) | 2020-08-17 | 2020-08-17 | Method and device for identifying abnormal data of monitoring index data |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111984503B (en) |
Families Citing this family (15)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112232719A (en) * | 2020-12-11 | 2021-01-15 | 北京基调网络股份有限公司 | Index quantitative scoring method, computer equipment and storage medium |
| CN112801345A (en) * | 2021-01-07 | 2021-05-14 | 山东润一智能科技有限公司 | Equipment measuring point time interval early warning method and system based on expectation and fluctuation |
| CN112783727A (en) * | 2021-01-28 | 2021-05-11 | 北京百度网讯科技有限公司 | Work amount monitoring method and device, electronic equipment and computer readable medium |
| CN113010805B (en) * | 2021-02-23 | 2023-09-01 | 腾讯科技（深圳）有限公司 | Index data processing method, device, equipment and storage medium |
| CN112948210A (en) * | 2021-02-26 | 2021-06-11 | 深圳依时货拉拉科技有限公司 | Method and device for detecting QPS index abnormity and computer equipment |
| CN113064796B (en) * | 2021-04-13 | 2022-08-12 | 上海浦东发展银行股份有限公司 | Unsupervised index abnormality detection method |
| CN113190798A (en) * | 2021-04-23 | 2021-07-30 | 通号（长沙）轨道交通控制技术有限公司 | Harmonic transmission coefficient estimation method from low-voltage side to high-voltage side of traction transformer |
| CN113556253B (en) * | 2021-07-30 | 2023-05-26 | 济南浪潮数据技术有限公司 | Method, system, equipment and storage medium for predicting real-time traffic of switch port |
| CN114741377B (en) * | 2022-04-01 | 2023-07-21 | 深圳市爱路恩济能源技术有限公司 | Method and device for identifying and processing natural gas abnormal data |
| CN115174354A (en) * | 2022-07-22 | 2022-10-11 | 科来网络技术股份有限公司 | Platform side data alarm method and device, monitoring equipment and readable storage medium |
| CN115408449B (en) * | 2022-10-28 | 2023-03-07 | 支付宝(杭州)信息技术有限公司 | User behavior processing method, device and equipment |
| CN115392812B (en) * | 2022-10-31 | 2023-03-24 | 成都飞机工业（集团）有限责任公司 | Abnormal root cause positioning method, device, equipment and medium |
| CN116738351B (en) * | 2023-08-10 | 2023-11-21 | 重庆华悦生态环境工程研究院有限公司深圳分公司 | Drainage facility detection method and device |
| CN116884517B (en) * | 2023-09-06 | 2023-12-22 | 合肥金星智控科技股份有限公司 | Online detection equipment, display method, device and medium thereof |
| CN116975768B (en) * | 2023-09-22 | 2023-12-19 | 山东爱福地生物股份有限公司 | Data anomaly detection method for fertilizer safety production |
Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN105373857A (en) * | 2015-11-27 | 2016-03-02 | 许昌许继软件技术有限公司 | Photovoltaic power station irradiance prediction method |
| CN108089962A (en) * | 2017-11-13 | 2018-05-29 | 北京奇艺世纪科技有限公司 | A kind of method for detecting abnormality, device and electronic equipment |
| CN110334843A (en) * | 2019-04-22 | 2019-10-15 | 山东大学 | A kind of time-varying attention improves be hospitalized medial demand prediction technique and the device of Bi-LSTM |
| CN110414788A (en) * | 2019-06-25 | 2019-11-05 | 国网上海市电力公司 | A kind of power quality prediction technique based on similar day and improvement LSTM |
| CN111260136A (en) * | 2020-01-17 | 2020-06-09 | 上海电力大学 | Building short-term load prediction method based on ARIMA-LSTM combined model |
| WO2020134867A1 (en) * | 2018-12-29 | 2020-07-02 | 中国电力科学研究院有限公司 | Method and device for detecting abnormal data of power terminal |
-
2020
- 2020-08-17 CN CN202010824430.0A patent/CN111984503B/en active Active
Patent Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN105373857A (en) * | 2015-11-27 | 2016-03-02 | 许昌许继软件技术有限公司 | Photovoltaic power station irradiance prediction method |
| CN108089962A (en) * | 2017-11-13 | 2018-05-29 | 北京奇艺世纪科技有限公司 | A kind of method for detecting abnormality, device and electronic equipment |
| WO2020134867A1 (en) * | 2018-12-29 | 2020-07-02 | 中国电力科学研究院有限公司 | Method and device for detecting abnormal data of power terminal |
| CN110334843A (en) * | 2019-04-22 | 2019-10-15 | 山东大学 | A kind of time-varying attention improves be hospitalized medial demand prediction technique and the device of Bi-LSTM |
| CN110414788A (en) * | 2019-06-25 | 2019-11-05 | 国网上海市电力公司 | A kind of power quality prediction technique based on similar day and improvement LSTM |
| CN111260136A (en) * | 2020-01-17 | 2020-06-09 | 上海电力大学 | Building short-term load prediction method based on ARIMA-LSTM combined model |
Also Published As
| Publication number | Publication date |
|---|---|
| CN111984503A (en) | 2020-11-24 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN111984503B (en) | Method and device for identifying abnormal data of monitoring index data | |
| CN106991145B (en) | Data monitoring method and device | |
| CN111324639B (en) | Data monitoring method, device and computer readable storage medium | |
| CN113518011B (en) | Abnormality detection method and apparatus, electronic device, and computer-readable storage medium | |
| CN113556258B (en) | Anomaly detection method and device | |
| CN107766533B (en) | Automatic detection method and system for telephone traffic abnormality, storage medium and electronic equipment | |
| CN105406991A (en) | Method and system for generating service threshold by historical data based on network monitoring indexes | |
| CN109035021B (en) | Method, device and equipment for monitoring transaction index | |
| CN110874674A (en) | Anomaly detection method, device and equipment | |
| CN112800116A (en) | Method and device for detecting abnormity of service data | |
| CN110019124B (en) | Method and system for revising aggregated weather forecast | |
| CN111179591B (en) | Road network traffic time sequence characteristic data quality diagnosis and restoration method | |
| CN111176953B (en) | Abnormality detection and model training method, computer equipment and storage medium | |
| US9600391B2 (en) | Operations management apparatus, operations management method and program | |
| CN115796610A (en) | Comprehensive monitoring method and system for operation of branch pipe forming system and storage medium | |
| CN110635947A (en) | Abnormal access monitoring method and device | |
| CN114138601A (en) | Service alarm method, device, equipment and storage medium | |
| EP3187950A1 (en) | A method for managing alarms in a control system | |
| CN114338348A (en) | Intelligent alarm method, device, equipment and readable storage medium | |
| CN113342625A (en) | Data monitoring method and system | |
| CN110517731A (en) | Genetic test quality monitoring data processing method and system | |
| CN116108991A (en) | Data processing method, device, equipment and storage medium | |
| CN111459996B (en) | Method and device for detecting working state of oil gun in specified time period | |
| CN114091357A (en) | Traffic flow prediction method, traffic flow prediction device, electronic device, and storage medium | |
| CN113591266A (en) | Method and system for analyzing fault probability of electric energy meter |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |