CN115409066A - Method and device for detecting abnormality of time series data, and computer storage medium - Google Patents
Method and device for detecting abnormality of time series data, and computer storage medium Download PDFInfo
- Publication number
- CN115409066A CN115409066A CN202211060141.3A CN202211060141A CN115409066A CN 115409066 A CN115409066 A CN 115409066A CN 202211060141 A CN202211060141 A CN 202211060141A CN 115409066 A CN115409066 A CN 115409066A
- Authority
- CN
- China
- Prior art keywords
- data
- frequency
- component
- low
- prediction
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 47
- 230000005856 abnormality Effects 0.000 title claims description 17
- 238000003860 storage Methods 0.000 title abstract description 21
- 238000012544 monitoring process Methods 0.000 claims abstract description 61
- 230000002159 abnormal effect Effects 0.000 claims abstract description 43
- 238000012545 processing Methods 0.000 claims description 66
- 230000015654 memory Effects 0.000 claims description 65
- 238000000354 decomposition reaction Methods 0.000 claims description 47
- 238000001514 detection method Methods 0.000 claims description 36
- 230000000737 periodic effect Effects 0.000 claims description 31
- 238000004590 computer program Methods 0.000 claims description 17
- 238000004458 analytical method Methods 0.000 claims description 16
- 230000008569 process Effects 0.000 claims description 16
- 230000008859 change Effects 0.000 claims description 13
- 238000005259 measurement Methods 0.000 claims description 13
- 238000007499 fusion processing Methods 0.000 claims description 9
- 238000004140 cleaning Methods 0.000 claims description 6
- 230000010354 integration Effects 0.000 claims description 6
- 238000000605 extraction Methods 0.000 claims description 4
- 238000011897 real-time detection Methods 0.000 abstract description 4
- 238000012423 maintenance Methods 0.000 abstract description 3
- 210000004027 cell Anatomy 0.000 description 16
- 230000006870 function Effects 0.000 description 7
- 230000002547 anomalous effect Effects 0.000 description 6
- 230000003287 optical effect Effects 0.000 description 6
- 238000007418 data mining Methods 0.000 description 4
- 238000011161 development Methods 0.000 description 4
- 230000018109 developmental process Effects 0.000 description 4
- 238000010586 diagram Methods 0.000 description 4
- 238000009826 distribution Methods 0.000 description 4
- 238000005516 engineering process Methods 0.000 description 4
- 238000011160 research Methods 0.000 description 4
- 238000013528 artificial neural network Methods 0.000 description 3
- 230000008901 benefit Effects 0.000 description 3
- 238000004364 calculation method Methods 0.000 description 3
- 230000007787 long-term memory Effects 0.000 description 3
- 230000006403 short-term memory Effects 0.000 description 3
- 238000012549 training Methods 0.000 description 3
- 238000007405 data analysis Methods 0.000 description 2
- 230000000694 effects Effects 0.000 description 2
- 238000013178 mathematical model Methods 0.000 description 2
- 239000013307 optical fiber Substances 0.000 description 2
- 230000000644 propagated effect Effects 0.000 description 2
- 239000004065 semiconductor Substances 0.000 description 2
- 238000012360 testing method Methods 0.000 description 2
- 238000004148 unit process Methods 0.000 description 2
- 238000007476 Maximum Likelihood Methods 0.000 description 1
- 230000009471 action Effects 0.000 description 1
- 230000032683 aging Effects 0.000 description 1
- 238000013473 artificial intelligence Methods 0.000 description 1
- 230000006399 behavior Effects 0.000 description 1
- 210000003850 cellular structure Anatomy 0.000 description 1
- 230000001427 coherent effect Effects 0.000 description 1
- 238000004891 communication Methods 0.000 description 1
- 238000010276 construction Methods 0.000 description 1
- 238000012937 correction Methods 0.000 description 1
- 230000007547 defect Effects 0.000 description 1
- 238000012217 deletion Methods 0.000 description 1
- 230000037430 deletion Effects 0.000 description 1
- 238000004880 explosion Methods 0.000 description 1
- 238000009776 industrial production Methods 0.000 description 1
- 230000004807 localization Effects 0.000 description 1
- 230000007774 longterm Effects 0.000 description 1
- 238000004519 manufacturing process Methods 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 230000000306 recurrent effect Effects 0.000 description 1
- 230000004044 response Effects 0.000 description 1
- 230000000717 retained effect Effects 0.000 description 1
- 238000012795 verification Methods 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- Data Mining & Analysis (AREA)
- General Health & Medical Sciences (AREA)
- Biomedical Technology (AREA)
- Biophysics (AREA)
- Computational Linguistics (AREA)
- Life Sciences & Earth Sciences (AREA)
- Evolutionary Computation (AREA)
- Artificial Intelligence (AREA)
- Molecular Biology (AREA)
- Computing Systems (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Mathematical Physics (AREA)
- Software Systems (AREA)
- Health & Medical Sciences (AREA)
- Management, Administration, Business Operations System, And Electronic Commerce (AREA)
Abstract
The method, the device and the computer storage medium decompose time sequence monitoring data, extract high-frequency components and low-frequency components in the time sequence monitoring data, analyze and predict the high-frequency components and the low-frequency components by adopting corresponding models according to the characteristics of the high-frequency components and the low-frequency components respectively, superpose prediction results corresponding to the high-frequency components and the low-frequency components to obtain a prediction value of the data under a future time sequence, and further identify abnormal point data in real data under the future time sequence by taking the prediction value as a basis, so that the real-time detection of the time sequence data can be realized, the abnormal point data in the time sequence data can be found in time, maintenance personnel can be reminded to take corresponding measures as early as possible, and the loss which cannot be compensated is avoided.
Description
Technical Field
The application belongs to the technical field of big data processing, and particularly relates to a method and a device for detecting time series data abnormity and a computer storage medium.
Background
With the continuous development of the internet and artificial intelligence, the world is in an era of big explosion of information, big data is more and more emphasized by people, and in the development process of informatization, data mining and analysis are helpful for people to understand the value and the law contained behind the data, so that information technologies such as big data and data mining are developed in many fields, and the development of related fields is promoted by exploring important information hidden behind the data.
The time series data is one of various big data, the time series is widely applied in various fields, such as medical treatment, finance, industry and the like, and how to extract valuable and regular information from the time series data becomes a current research hotspot. The abnormal detection of the time sequence data is contained in the time sequence data mining field, the abnormal data in the time sequence data is identified, the monitoring problems of equipment failure, medical conditions, illegal intrusion of network security and the like are solved in time through finding the abnormal data, along with the gradual enlargement of a service system, the service combination is more complicated, the scale of the time sequence data is larger and larger, the data quantity and the data dimension of the data are larger and larger, and the increasing monitoring requirements cannot be met by simply depending on human detection.
Disclosure of Invention
In view of this, the present application provides a method and an apparatus for detecting time series data anomalies, and a computer storage medium, so as to efficiently and accurately implement the time series data anomalies detection, and solve the problems of time consuming, labor consuming, and error prone in the manual detection method.
The specific scheme is as follows:
an abnormality detection method for time-series data, comprising:
acquiring time series monitoring data;
extracting high-frequency components and low-frequency components of the time series monitoring data;
performing prediction processing on the basis of the high-frequency component by using a first prediction model to obtain a first prediction result of data in a future time sequence;
performing prediction processing based on the low-frequency component by using a second prediction model to obtain a second prediction result of the data in the future time sequence;
superposing the first prediction result and the second prediction result to obtain a target prediction result of the data in the future time sequence;
and identifying abnormal point data in the real data under the future time sequence according to the target prediction result.
Optionally, the extracting the high frequency component and the low frequency component of the time-series monitoring data includes:
performing multi-scale decomposition reconstruction on the time series monitoring data by using a wavelet analysis theory, and extracting a high-frequency periodic component, a high-frequency random component and a low-frequency trend component of the time series monitoring data;
wherein the high frequency component includes the high frequency periodic component and the high frequency stochastic component, and the low frequency component includes the low frequency trend component.
Optionally, the performing multi-scale decomposition and reconstruction on the time series monitoring data by using a wavelet analysis theory, and extracting a high-frequency periodic component, a high-frequency stochastic component, and a low-frequency trend component of the time series monitoring data includes:
acquiring deformation actual measurement data of the time series monitoring data, wherein the deformation actual measurement data is obtained by cleaning abnormal values of the time series monitoring data;
taking the deformation actual measurement data as current data to be decomposed;
decomposing the current data to be decomposed into a low-frequency part and a high-frequency part based on wavelet decomposition to obtain a low-frequency component and a high-frequency component corresponding to the current data to be decomposed;
determining whether a low-frequency subsequence contained in a low-frequency component corresponding to the current data to be decomposed has a preset variation trend characteristic or not;
if so, ending the decomposition processing of the data;
if not, updating the data to be decomposed into a low-frequency component corresponding to the current data to be decomposed, and circulating to the step of decomposing the current data to be decomposed into a low-frequency part and a high-frequency part until a low-frequency subsequence contained in the low-frequency component corresponding to the current data to be decomposed has the change trend characteristic, and ending the decomposition processing of the data; in the decomposition process, each layer only decomposes the low-frequency component, and the high-frequency component is not processed;
and extracting the low-frequency component of the last layer in the decomposition result as the low-frequency tendency component, and extracting a high-frequency periodic component and a high-frequency random component from the high-frequency components of the layers.
Optionally, the performing, by using the first prediction model, prediction processing based on the high-frequency component to obtain a first prediction result of data in a future time sequence includes:
denoising the high-frequency stochastic component;
and inputting the high-frequency periodic component and the denoised high-frequency stochastic component into a long-short term memory network model, and performing prediction processing on the long-short term memory network model based on the high-frequency periodic component and the denoised high-frequency stochastic component to obtain a first prediction result of the data in the future time sequence.
Optionally, the denoising processing on the high-frequency stochastic component includes:
denoising the high-frequency stochastic component based on a preset threshold;
wherein the threshold is
σ 1 ＝MAD/0.674And 5, MAD represents the middle value of the absolute value of the wavelet decomposition coefficient of the first layer, 0.6745 is an adjusting coefficient of the standard deviation of Gaussian noise, and N1 represents the size or the length of a high-frequency random component signal.
Optionally, the long-term and short-term memory network model includes a plurality of memory units connected in sequence, where the memory unit includes a forgetting gate, an input gate, and an output gate;
the current memory unit obtains the output information of the current memory unit by carrying out fusion processing on the feature information transmitted by the previous memory unit and the high-frequency component input data of the current memory unit, and processes the feature information transmitted by the previous memory unit by selectively memorizing and forgetting the input high-frequency component when carrying out fusion processing;
respectively processing the high-frequency components input by the memory units and the characteristic information transmitted by the previous memory unit through the memory units connected in sequence to realize the prediction processing of the model based on the high-frequency components;
the characteristic information transmitted by the previous memory unit comprises a state value and an output value of the previous memory unit.
Optionally, the performing, by using a second prediction model, prediction processing based on the low-frequency component to obtain a second prediction result of the data in the future time sequence includes:
and inputting the low-frequency tendency component into a differential integration moving average autoregressive model, and performing prediction processing on the low-frequency tendency component by the differential integration moving average autoregressive model to obtain a second prediction result of the data in the future time sequence.
Optionally, the identifying, according to the target prediction result, abnormal point data in the real data in the future time sequence includes:
determining deviation information between real data at the future time sequence and a target prediction result of the data at the future time sequence;
and identifying abnormal point data in the real data under the future time sequence according to the deviation information.
An abnormality detection device for time-series data, comprising:
the acquisition module is used for acquiring time series monitoring data;
the extraction module is used for extracting a high-frequency component and a low-frequency component of the time series monitoring data;
the first prediction processing module is used for performing prediction processing on the basis of the high-frequency component by using a first prediction model to obtain a first prediction result of data in a future time sequence;
the second prediction processing module is used for performing prediction processing on the basis of the low-frequency component by using a second prediction model to obtain a second prediction result of the data in the future time sequence;
the superposition module is used for carrying out superposition processing on the first prediction result and the second prediction result to obtain a target prediction result of the data under the future time sequence;
and the abnormal point identification module is used for identifying abnormal point data in the real data under the future time sequence according to the target prediction result.
A computer-readable medium having stored thereon a computer program comprising program code for executing the method for anomaly detection of time-series data as described in any one of the above.
A computer program product comprising a computer program carried on a non-transitory computer readable medium, the computer program comprising program code for performing an anomaly detection method for time-series data as any one of the above.
In summary, according to the method, the device and the computer storage medium for detecting the time series data anomaly provided by the application, the time series monitoring data is decomposed and the high frequency component and the low frequency component in the time series monitoring data are extracted, the high frequency component and the low frequency component are analyzed and predicted respectively according to the characteristics of the high frequency component and the low frequency component by using the corresponding models, the prediction results corresponding to the high frequency component and the low frequency component are superposed to obtain the predicted value of the data in the future time series, and then the predicted value is used as the basis to identify the anomalous data in the real data in the future time series, so that the real-time detection of the time series data can be realized, the anomalous data in the time series data can be found out, and the maintainers can be reminded to take countermeasures as early as possible, and the irreparable loss can be avoided.
Drawings
The above and other features, advantages and aspects of various embodiments of the present application will become more apparent from the following detailed description when taken in conjunction with the accompanying drawings. Throughout the drawings, the same or similar reference numbers refer to the same or similar elements. It should be understood that the drawings are schematic and that elements and features are not necessarily drawn to scale.
FIG. 1 is a flow chart of a method for anomaly detection of time series data provided herein;
FIG. 2 is a diagram of a wavelet decomposition structure provided in the present application;
FIG. 3 is a diagram of the cellular structure of the LSTM model provided herein;
FIG. 4 is a flowchart of WA-LSTM-ARIMA model-based time series data anomaly detection provided herein;
FIG. 5 is a schematic diagram of a raw data set and its data distribution upon which the model training provided herein is based;
FIG. 6 is an exemplary residual plot of true values versus predicted values provided herein;
fig. 7 is a configuration diagram of an abnormality detection device for time-series data according to the present application.
Detailed Description
Embodiments of the present application will be described in more detail below with reference to the accompanying drawings. While certain embodiments of the present application are shown in the drawings, it should be understood that the present application may be embodied in various forms and should not be construed as limited to the embodiments set forth herein, but rather are provided for a more thorough and complete understanding of the present application. It should be understood that the drawings and embodiments of the present application are for illustration purposes only and are not intended to limit the scope of the present application.
The term "include" and variations thereof as used herein are open-ended, i.e., "including but not limited to". The term "based on" is "based at least in part on". The term "one embodiment" means "at least one embodiment"; the term "another embodiment" means "at least one additional embodiment"; the term "some embodiments" means "at least some embodiments". Relevant definitions for other terms will be given in the following description.
It should be noted that the terms "first", "second", and the like in the present application are only used for distinguishing different devices, modules or units, and are not used for limiting the order or interdependence of the functions performed by the devices, modules or units.
It is noted that references to "a" or "an" modification in this application are intended to be illustrative rather than limiting, and those skilled in the art will appreciate that references to "one or more" are intended to be exemplary unless the context clearly indicates otherwise.
The time-series data is one of various types of big data, and is defined as a series of statistics or observations according to time. Time series is widely applied to various fields such as medical, financial, industrial and the like, the data quantity and data dimension of the time series data are increasing, and how to extract valuable and regular information from the time series data becomes a current research hotspot. Time series anomaly detection is included in the field of time series data mining and is defined as the process of identifying abnormal events or behaviors from a normal time series, which serves to identify abnormal data in the time series. Sometimes, the abnormal data has more important value, can provide a lot of useful information, and effective abnormal detection is widely applied to many fields of the real world, such as quantitative transactions, network security detection, daily maintenance of automatic driving automobiles and large-scale industrial equipment, and the like.
In order to solve the problems of time consumption, labor consumption, high error possibility and the like of the conventional manual detection mode, the application provides the time sequence data abnormity detection method, the time sequence data abnormity detection device, the computer storage medium and the computer program product, so that data which do not accord with a normal development rule in the time sequence data can be efficiently and accurately analyzed and found, equipment faults can be timely found through finding of abnormal data, major disaster accidents can be effectively avoided, the safety of daily production and life can be ensured, and unnecessary loss of manpower and financial resources can be reduced.
Referring to a flow chart of the method for detecting the time-series data abnormality shown in fig. 1, the method for detecting the time-series data abnormality provided by the present application includes the following processing flows:
The method of the application can be applied to various fields, such as illegal intrusion detection of network security, financial fraud detection in financial transactions, equipment failure detection in industrial production, medical condition detection and the like. Taking the industrial field as an example, through record and analysis to equipment action data, when data take place unusually at a certain moment, suggestion equipment probably receives the damage because of long-time continuous operation or wearing and tearing, needs in time to overhaul to avoid not in time finding the problem and causing more serious loss, these unusual data contain more valuable information than normal data, find these unusual data in time effectively, very important to many fields.
The time-series monitoring data obtained in this step may be, but is not limited to, time-series data in aspects of network security monitoring, financial transactions, equipment status monitoring, medical condition monitoring, and the like, and may be determined according to actual needs.
And 102, extracting high-frequency components and low-frequency components of the time series monitoring data.
According to the method, time series monitoring data (short for 'time series data') are subjected to multi-scale decomposition and reconstruction, and high-frequency components and low-frequency components in the time series monitoring data are extracted, wherein the high-frequency components comprise high-frequency periodic components and high-frequency random components, and the low-frequency components comprise low-frequency trend components.
Further, optionally, the time series monitoring data is decomposed by using a WA technique, where WA refers to wavelet transform, inherits and develops the idea of short-time fourier transform localization, and overcomes the disadvantages that the window size does not change with frequency, and the like.
Correspondingly, the multi-scale decomposition reconstruction is carried out on the time series monitoring data by applying the wavelet analysis theory, the high-frequency periodic component, the high-frequency random component and the low-frequency trend component are extracted by using the multi-scale decomposition reconstruction of the wavelet analysis theory on the time series monitoring data, and the process can be specifically realized as follows:
11 Obtaining deformation actual measurement data of the time series monitoring data, wherein the deformation actual measurement data is obtained by cleaning abnormal values of the time series monitoring data;
12 Taking the deformation measured data as current data to be decomposed;
13 Based on wavelet decomposition, decomposing the current data to be decomposed into a low-frequency part and a high-frequency part to obtain a low-frequency component and a high-frequency component corresponding to the current data to be decomposed;
14 Determining whether a low-frequency subsequence contained in a low-frequency component corresponding to the current data to be decomposed has a preset variation trend characteristic or not;
optionally, the preset variation trend characteristic may be set as: the low-frequency time sequence data waveform corresponding to the current decomposition layer has obvious fluctuation change and no periodical change. The obvious fluctuation change can be measured based on a set waveform change threshold, and the fluctuation of the low-frequency time sequence data waveform reaches the threshold, namely the obvious fluctuation change is considered to be present.
15 If yes, the decomposition processing of the data is finished;
16 If not, updating the data to be decomposed into a low-frequency component corresponding to the current data to be decomposed, and circulating to the step 13) for decomposing the current data to be decomposed into a low-frequency part and a high-frequency part until a low-frequency subsequence contained in the low-frequency component corresponding to the current data to be decomposed has the change trend characteristic, and ending the decomposition processing of the data;
wherein, each layer only decomposes the low-frequency component in the decomposition process, and the high-frequency component is not processed.
17 Extracting a low-frequency component of the last layer in the decomposition result as the low-frequency tendency component, and extracting a high-frequency periodic component and a high-frequency stochastic component from the high-frequency components of the respective layers.
For ease of understanding, the following is described in further detail.
The applicant researches and discovers that influence information of environment, an acquisition process and the like on time series data is contained in a data sequence of the time series data, wherein high-frequency data mainly shows the characteristics of high frequency and periodicity and may be accompanied by some noises, and low-frequency data shows the characteristics of relatively obvious low frequency and tendency, so that the sequence is subjected to multi-scale decomposition based on WA technology to extract a high-frequency periodic component, a low-frequency tendency component and a high-frequency random component.
Considering the discreteness of the acquired data, it is preferable to use a discrete wavelet transform W f And (3) carrying out data analysis:
in the formula, a 0 、b 0 Are all real constants, j and k are integers,
the complex number of wavelet basis functions is expressed, f (t) represents the deformation time sequence, and t is time.
The structure of wavelet decomposition is shown in FIG. 2, wherein f in FIG. 2 0 The deformation actual measurement data represents time series monitoring data, the deformation actual measurement data refers to data required for detection selected/extracted from source data, namely the time series monitoring data according to actual needs, specifically can be data obtained after cleaning abnormal values of the time series monitoring data, and therefore in a training stage or a prediction stage, the data after cleaning the abnormal values is selected for training or prediction; f. of 1 ,f 2 ,…,f N1 A low frequency part, d 1 ,d 2 ,…,d N1 Is a high frequency part.
And if the low-frequency time sequence data waveform of the current decomposition layer has obvious variation trend, namely the low-frequency time sequence data waveform of the current decomposition layer has obvious fluctuation change and no periodical change, judging that the low-frequency time sequence data waveform meets the preset variation trend characteristic, stopping decomposition, and otherwise, continuing to decompose until the trend is obvious if the trend is not obvious. Assuming that the number of decomposition layers is N1, N1 high-frequency components and 1 low-frequency component are obtained, and after superposition, the following results are obtained:
f 0 ＝d 1 +d 2 +…+d N1 +f N1
that is, the low frequency component (f) of the last layer in the decomposition result is decomposed N1 ) Extracting low frequency trend components of the time series monitoring data, and dividing high frequency components (d) of each layer 1 ,d 2 ,…,d N1 ) And extracting a high-frequency component of the monitoring data which is a time series. The applicant researches and discovers that high-frequency data mainly show the characteristics of high frequency and periodicity, and are interfered by factors such as working conditions, instrument errors and the like, a data sequence often contains noise, and accordingly, high-frequency periodic components and high-frequency random components can be further extracted from high-frequency components.
The high-frequency periodic component mainly reflects high-frequency data part information of the time sequence monitoring data, the low-frequency trend component mainly reflects low-frequency data part information of the time sequence monitoring data, and the high-frequency random component mainly reflects the influence of noise.
And 103, performing prediction processing on the basis of the high-frequency component by using a first prediction model to obtain a first prediction result of the data in the future time sequence.
Preferably, for the characteristics of the high frequency component, the embodiment of the present application uses an LSTM (Long Short-Term Memory network) as an analysis and prediction processing model of the high frequency component, that is, the first prediction model is an LSTM model. The LSTM is a time-cycle neural network and is specially designed for solving the long-term dependence problem of the general cycle neural network.
The data sequence of the time series monitoring data is interfered by factors such as working conditions and instrument errors, noise is often contained in the data sequence of the time series monitoring data, and the accuracy of forecasting the interference is interfered, so that the monitoring data sequence needs to be denoised. In the embodiment of the present application, since the influence of noise is mainly embodied by the high-frequency stochastic component, the extracted high-frequency stochastic component can be denoised correspondingly and specifically, wherein the key of denoising is to determine the threshold λ, and the denoising effect is affected when the threshold is selected to be too large or too small, and the value of the threshold λ is preferably determined by using the following calculation formula:
σ 1 ＝MAD/0.6745
wherein, MAD represents the median of the absolute value of the wavelet decomposition coefficient of the top layer, 0.6745 is the adjustment coefficient of the standard deviation of gaussian noise, and N1 represents the size or length of the high-frequency stochastic component signal (i.e. the number of decomposition layers).
Through verification, the threshold determined based on the above calculation formula is used for denoising, and a good denoising effect can be achieved on the high-frequency stochastic component.
And after denoising the high-frequency stochastic component, merging the high-frequency stochastic component into a high-frequency sequence for uniform processing. The high-frequency stochastic component and the high-frequency periodic component after denoising are input into the LSTM model, and the LSTM model carries out prediction processing on the basis of the high-frequency stochastic component and the high-frequency periodic component after denoising to obtain a first prediction result of data in a future time sequence.
The long-term and short-term memory network model comprises a plurality of memory units which are sequentially connected, wherein each memory unit comprises a forgetting gate, an input gate and an output gate.
The current memory unit obtains the output information of the current memory unit by carrying out fusion processing on the feature information transmitted by the previous memory unit and the high-frequency component input data of the current memory unit, and when the fusion processing is carried out, the current memory unit processes the feature information transmitted by the previous memory unit by selectively memorizing and forgetting the input high-frequency component;
respectively processing the high-frequency component input by each memory unit and the characteristic information transmitted by the previous memory unit through a plurality of memory units connected in sequence, and realizing the prediction processing based on the high-frequency component in the model;
the characteristic information passed by the previous cell includes the state value and output value of the previous cell as described in further detail below.
The LSTM model is a special form of RNN (Recurrent Neural Network), and the conventional RNN model generates a forgetting phenomenon for information at a longer distance, is difficult to store information of a past longer time series, performs future prediction only by using information of several steps nearby, and is difficult to recover once an error occurs. The LSTM model controls the reservation, deletion and updating of information through an input gate, a forgetting gate and an output gate, effectively overcomes the defects of the input gate, the forgetting gate and the output gate, and greatly improves the prediction precision.
The LSTM model is composed of a plurality of memory cells, which are referred to as cells in this embodiment, the cells s of the model participate in the prediction process based on the input data, and the cell structure of the model can be specifically shown in fig. 3, where f, i, o respectively represent a forgetting gate, an input gate, an output gate, and x t 、h t Representing input and output, respectively, sigma represents sigmoid function, C t A state value representing the time t, i.e. the characteristic information transmitted by the cell unit before this time by selectively memorizing and forgetting the input high-frequency component, C t Specifically, the fusion processing is performed on the state value and the output value of the previous cell and the input data of the current cell.
Referring to fig. 3 in combination, the process of performing prediction processing on the high-frequency components (the high-frequency periodic component and the denoised high-frequency stochastic component) by using the LSTM model is as follows:
21 Using forgetting gate to control the trade-off of high frequency component information obtained from wavelet decomposition data, determine how much old information of the cell in the previous state is retained:
f t ＝σ(W hf h t-1 +W xf x t +b f )
in the formula, f t Output value for forgetting gate, i.e. a certain value from 0 to 1；h t-1 ,W hf Respectively the output and weight of the previous moment; x is the number of t ,W xf Respectively the input and the weight of the current moment; b f Is a bias term.
22 Solving for a candidate value of update information
And an output value i of the update information t Determining the information that the cell needs to be updated, namely the data information of the previous cell output value and the current cell input value which are selectively memorized through the input gate operation:
i t ＝σ(W hi h t-1 +W xi x i +b i )
in the formula, W hc ,W xc Respectively to solve for
Time h t-1 And x t Corresponding update information weight; w hi ,W xi Are respectively solved for i t Time h t-1 And x i A corresponding entry gate weight; b c ,b i Is a bias term.
23 Output value f) t And the previous state C of the cell t-1 Multiplying, to obtain candidate values
And the output value i t Multiplying and overlapping the two products, thereby achieving the purpose of forgetting part of old information and remembering part of new information, and obtaining the new state C of the cell t ：
24 Using the output gate to determine the information o that the new cell needs to be output t And with new cellsC t Multiplying the results after the tanh layer processing to obtain a final output result h t ：
o t ＝σ(W ho h t-1 +W xo x i +b o )
h t ＝o t tanh(C t )
In the formula, W ho ,W xo Respectively to solve for o t Time h t-1 And x i A corresponding output gate weight; b o Is the bias term.
And 104, performing prediction processing on the basis of the low-frequency component by using a second prediction model to obtain a second prediction result of the data in the future time sequence.
Preferably, for the characteristics of the low-frequency component, an ARIMA (automated Integrated Moving Average Model) is used as the analysis and prediction processing Model of the low-frequency component in the embodiments of the present application, that is, the second prediction Model is an ARIMA Model. ARIMA is a differential integrated moving average autoregressive model, also called integrated moving average autoregressive model (moving can also be called sliding), which is one of the time series prediction analysis methods.
The ARIMA model combines differential operation with an autoregressive moving average model (ARMA), and the ARIMA model predicts a non-stationary sequence by differentiating the sequence to make the sequence tend to be stationary. ARIMA has the advantage of describing time-aged sequences with a mathematical model and predicting future trends from past and present values of the sequence.
The ARIMA model is represented as follows:
wherein p is the autoregressive order;
is an autoregressive coefficient; l is a lag operator; d is the difference order; y is t Is a time series of inputs; q is the moving average order; theta.theta. i Is a moving average coefficient; epsilon t Is a residual sequence.
In the embodiment of the application, the construction process of the ARIMA model includes:
31 Differential processing is carried out on the time series data, when the processed series becomes a stable series, the difference is stopped, and the differential order d at the moment is determined and is used as the differential order of the model;
32 P, q values are determined by SBC criteria on the basis of differential processing.
The SBC criterion is a coherent estimate of the true order of the optimal model. The expression is as follows:
SBC＝-2lnM l +lnN·N
in the formula, M l And L is a model maximum likelihood function value.
The criterion aims to convert the penalty weight of the number of unknown parameters needing fitting in the model from a fixed value to a dynamic value related to the sample capacity, so that the criterion can adapt to the change of the time sequence length and the judgment result is more reliable. In practical application of the criterion, the present embodiment compares SBC indexes with p and q values within a certain range, and selects the p and q values corresponding to the minimum value of the SBC index as optimal parameters.
And 105, overlapping the first prediction result and the second prediction result to obtain a target prediction result of the data in the future time sequence.
And then, superposing the prediction results of the first prediction model and the second prediction model to obtain a final prediction result of the data under the future time sequence, namely a target prediction result.
And 106, identifying abnormal point data in the real data under the future time sequence according to the target prediction result.
After the final prediction result, namely the target prediction result, of the data under the future time sequence is obtained according to the superposition processing of the prediction results of different models, the deviation information between the real data under the future time sequence and the target prediction result of the data under the future time sequence can be specifically determined, and the abnormal point data in the real data under the future time sequence can be identified according to the deviation information.
Further, in practical application, after the prediction results of the two models are superimposed to obtain a final target prediction result, a residual map corresponding to a residual between the real value and the predicted value can be further analyzed and drawn, and an abnormal point is identified based on the residual map, so as to determine whether the application/system has a fault/abnormality based on the abnormal point identification, such as determining whether network illegal intrusion, financial fraud, equipment fault or medical accident condition occurs.
In summary, according to the method, the device and the computer storage medium for detecting the time series data anomaly provided by the application, the time series monitoring data is decomposed and the high frequency component and the low frequency component in the time series monitoring data are extracted, the high frequency component and the low frequency component are analyzed and predicted respectively according to the characteristics of the high frequency component and the low frequency component by using the corresponding models, the prediction results corresponding to the high frequency component and the low frequency component are superposed to obtain the predicted value of the data in the future time series, and then the predicted value is used as the basis to identify the anomalous data in the real data in the future time series, so that the real-time detection of the time series data can be realized, the anomalous data in the time series data can be found out, and the maintainers can be reminded to take countermeasures as early as possible, and the irreparable loss can be avoided.
Next, an application example of the method of the present application is provided.
In the example, a WA-LSTM-ARIMA-based time series anomaly detection model is established in advance, the model is used for carrying out multi-scale decomposition and reconstruction on time series data by applying a wavelet analysis theory, the LSTM model and the ARIMA model are respectively adopted for analyzing high-frequency components and low-frequency components extracted after decomposition, then the prediction results of the two models are superposed to obtain a final prediction result, and finally anomaly points are identified based on the superposition value of the prediction results of the two models.
In the WA-LSTM-ARIMA-based time series anomaly detection model, a detailed anomaly detection flow for time series data is shown in fig. 4, and includes: performing multi-scale decomposition and reconstruction on the time series data by using a wavelet analysis theory, and extracting a high-frequency periodic component, a low-frequency trend component and a high-frequency random component; the high-frequency periodic component mainly reflects partial information of high-frequency data, and an LSTM model is adopted for analysis and prediction processing; the low-frequency trend component reflects the information of the low-frequency data part, and an ARIMA model is adopted for analysis and prediction processing; the high-frequency random component mainly reflects the influence of noise, and is merged into a high-frequency sequence for uniform processing after being subjected to denoising processing. And superposing the prediction results of the models to obtain the final prediction result of the data in the future time sequence, analyzing and drawing a residual error map based on the prediction value and the true value (observed value) of the data in the future time sequence, and identifying abnormal points.
The example pre-selects a dataset to train, test and evaluate the model.
Considering that data such as the trading price and the trading volume of stocks in the stock market form a continuous time sequence and contain useful information related to time, the present example adopts related data in terms of stock trading as an experimental data set, the data set comprises information such as date and opening price, the original data set and data distribution thereof are shown in fig. 5, two columns of data such as data and volume (referred to as volume in the data set) in the data set are obtained because the trend of a predicted value in a time dimension is observed, wherein the first 60% of data in the data set is used as learning data of a model, the second 40% of data is used for prediction, and the first 4% of the maximum deviation between the real value and the predicted value is used as an anomaly point based on a residual map which is shown in fig. 6.
When evaluating the performance of the time series anomaly detection model, the example adopts indexes such as accuracy (P), recall rate (R) and F1 value to measure, wherein the accuracy represents the proportion of the number of samples which are both abnormal in prediction and actual in the total number of the abnormal samples in prediction, and the larger the value is, the better the performance is; the recall rate represents the proportion of the number of samples which are abnormal in prediction and reality to the actual total abnormal constant, and the larger the value is, the better the performance is; the value of F1 represents the weighted harmonic mean of accuracy and recall, with higher values yielding better performance. The calculation formulas of the accuracy rate, the recall rate and the F1 value are respectively as follows:
where TP represents the number of samples for which the prediction is abnormal and abnormal, FP represents the number of samples for which the prediction is abnormal and normal, and FN represents the number of samples for which the prediction is normal and abnormal. Thus, TP + FP represents the total number of samples predicted to be anomalous and TP + FN represents the total number of samples actually anomalous.
Since the model of the present example is an abnormality detection model, the positive example herein refers to a sample of an abnormality. The total number of samples used in the test of this example was 730, where the distribution of the number of samples of positive and negative examples is shown in table 1.
TABLE 1 Positive and negative sample number distribution
| Practical correction | Practical negative example | |
| Example of prediction | 25 | 4 |
| Negative example of prediction | 3 | 698 |
The accuracy, recall, and F1 values of the model can be calculated from table 1, as shown in table 2:
TABLE 2 accuracy, recall, and F1 values of the model
| Accuracy (P) | 0.862 |
| Recall rate (R) | 0.892 |
| F1 | 0.877 |
The model of the example predicts the subsequent data based on learning the historical data of a certain system, judges whether the system is abnormal or not by comparing the deviation between the real value and the predicted value, can realize real-time detection of time series, accurately and timely find abnormal point data, and remind maintenance personnel to take corresponding measures as early as possible, avoids causing irreparable loss, and can thoroughly solve the problems of time consumption, labor consumption, high error probability and the like existing in the conventional manual detection mode.
In response to the above-described method for detecting an abnormality in time-series data, the present application also provides an apparatus for detecting an abnormality in time-series data, the apparatus having a configuration as shown in fig. 7, and including:
an obtaining module 10, configured to obtain time series monitoring data;
an extraction module 20, configured to extract a high-frequency component and a low-frequency component of the time-series monitoring data;
a first prediction processing module 30, configured to perform prediction processing based on the high-frequency component by using a first prediction model, so as to obtain a first prediction result of data in a future time sequence;
a second prediction processing module 40, configured to perform prediction processing based on the low-frequency component by using a second prediction model to obtain a second prediction result of the data in the future time sequence;
the superposition module 50 is configured to perform superposition processing on the first prediction result and the second prediction result to obtain a target prediction result of the data in the future time sequence;
and an abnormal point identification module 60, configured to identify abnormal point data in the real data in the future time sequence according to the target prediction result.
In an embodiment, the extraction module 20 is specifically configured to:
performing multi-scale decomposition reconstruction on the time series monitoring data by using a wavelet analysis theory, and extracting a high-frequency periodic component, a high-frequency random component and a low-frequency trend component of the time series monitoring data;
wherein the high frequency component includes the high frequency periodic component and the high frequency stochastic component, and the low frequency component includes the low frequency trend component.
In one embodiment, the extracting module 20, when performing multi-scale decomposition and reconstruction on the time-series monitoring data by using wavelet analysis theory to extract a high-frequency periodic component, a high-frequency stochastic component, and a low-frequency tendency component of the time-series monitoring data, is specifically configured to:
acquiring deformation actual measurement data of the time series monitoring data, wherein the deformation actual measurement data is obtained by cleaning abnormal values of the time series monitoring data;
taking the deformation actual measurement data as current data to be decomposed;
decomposing the current data to be decomposed into a low-frequency part and a high-frequency part based on wavelet decomposition to obtain a low-frequency component and a high-frequency component corresponding to the current data to be decomposed;
determining whether a low-frequency subsequence contained in a low-frequency component corresponding to current data to be decomposed has a preset variation trend characteristic or not;
if so, ending the decomposition processing of the data;
if not, updating the data to be decomposed into a low-frequency component corresponding to the current data to be decomposed, and circulating to the step of decomposing the current data to be decomposed into a low-frequency part and a high-frequency part until a low-frequency subsequence contained in the low-frequency component corresponding to the current data to be decomposed has the change trend characteristic, and ending the decomposition processing of the data; in the decomposition process, each layer only decomposes the low-frequency component, and the high-frequency component is not processed;
and extracting the low-frequency component of the last layer in the decomposition result as the low-frequency tendency component, and extracting a high-frequency periodic component and a high-frequency random component from the high-frequency components of the layers.
In an embodiment, the first prediction processing module 30 is specifically configured to:
denoising the high-frequency stochastic component;
and inputting the high-frequency periodic component and the denoised high-frequency stochastic component into a long-short term memory network model, and performing prediction processing on the long-short term memory network model based on the high-frequency periodic component and the denoised high-frequency stochastic component to obtain a first prediction result of the data in the future time sequence.
In an embodiment, the first prediction processing module 30, when performing denoising processing on the high-frequency stochastic component, is specifically configured to:
denoising the high-frequency stochastic component based on a preset threshold;
wherein the threshold is
σ 1 And the ratio of the maximum and minimum coefficients is = MAD/0.6745, MAD represents the middle value of the absolute value of the wavelet decomposition coefficient of the top layer, 0.6745 is the adjustment coefficient of the standard deviation of Gaussian noise, and N1 represents the size or length of the high-frequency random component signal.
In one embodiment, the long-short term memory network model comprises a plurality of memory units which are connected in sequence, wherein each memory unit comprises a forgetting gate, an input gate and an output gate;
the current memory unit obtains the output information of the current memory unit by carrying out fusion processing on the feature information transmitted by the previous memory unit and the high-frequency component input data of the current memory unit, and when the fusion processing is carried out, the current memory unit processes the feature information transmitted by the previous memory unit by selectively memorizing and forgetting the input high-frequency component;
respectively processing the high-frequency component input by each memory unit and the characteristic information transmitted by the previous memory unit through a plurality of memory units connected in sequence, and realizing the prediction processing based on the high-frequency component in the model;
the characteristic information transmitted by the previous memory unit comprises a state value and an output value of the previous memory unit.
In an embodiment, the second prediction processing module 40 is specifically configured to:
and inputting the low-frequency tendency component into a differential integration moving average autoregressive model, and performing prediction processing on the low-frequency tendency component by the differential integration moving average autoregressive model to obtain a second prediction result of the data in the future time sequence.
In an embodiment, the outlier identifying module 60 is specifically configured to:
determining deviation information between real data at the future time sequence and a target prediction result of the data at the future time sequence;
and identifying abnormal point data in the real data under the future time sequence according to the deviation information.
The time-series data abnormality detection device applied in the embodiment of the present application is relatively simple in description because it corresponds to the time-series data abnormality detection method applied in the above method embodiment, and for the relevant similarities, please refer to the description of the above method embodiment, and the details are not described here.
The present application also provides a computer-readable medium having stored thereon a computer program comprising program code for executing the method for anomaly detection of time-series data as claimed in the above method embodiments.
In the context of this application, a computer-readable medium (machine-readable medium) may be a tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, apparatus, or device. The machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any suitable combination of the foregoing. More specific examples of a machine-readable storage medium would include an electrical connection based on one or more wires, a portable computer diskette, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.
It should be noted that the computer readable medium mentioned above in the present application may be a computer readable signal medium or a computer readable storage medium or any combination of the two. A computer readable storage medium may be, for example, but not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination of the foregoing. More specific examples of the computer readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer diskette, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing. In the present application, a computer readable storage medium may be any tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, apparatus, or device. In this application, however, a computer readable signal medium may include a propagated data signal with computer readable program code embodied therein, for example, in baseband or as part of a carrier wave. Such a propagated data signal may take any of a variety of forms, including, but not limited to, electro-magnetic, optical, or any suitable combination thereof. A computer readable signal medium may be any computer readable medium that is not a computer readable storage medium and that can communicate, propagate, or transport a program for use by or in connection with an instruction execution system, apparatus, or device. Program code embodied on a computer readable medium may be transmitted using any appropriate medium, including but not limited to: electrical wires, optical cables, RF (radio frequency), etc., or any suitable combination of the foregoing.
The computer readable medium may be embodied in an electronic device; or may be separate and not incorporated into the electronic device.
The present application also provides a computer program product comprising a computer program carried on a non-transitory computer readable medium, the computer program comprising program code for performing an anomaly detection method for time-series data as claimed in the above method embodiments.
In particular, according to embodiments of the present application, the processes described above with reference to the flowcharts may be implemented as computer software programs. In such an embodiment, the computer program may be downloaded and installed from a network via the communication means, or installed from a storage means, or installed from a ROM. The computer program, when executed by a processing device, performs the above-described functions defined in the method of the embodiments of the present application.
In summary, the anomaly detection method, apparatus, computer storage medium and computer program product for time series data provided by the present application have at least the following technical advantages:
a) Time series data are decomposed based on WA technology, the time series data can be decomposed into low-frequency components and high-frequency components, noise in the high-frequency components can be removed, and prediction accuracy of the model is improved;
b) Aiming at the characteristics of low frequency, tendency and the like of low-frequency components, an ARIMA model is adopted to analyze the low-frequency components, so that an aging sequence can be described by using a mathematical model based on the ARIMA, and the future trend can be predicted through the past value and the present value of the sequence;
c) For the high-frequency periodic component, the LSTM model with the long-term memory function is used for analyzing the high-frequency periodic component, so that the result can be predicted more accurately, and the LSTM can be better represented in a longer sequence.
It is noted that, although the subject matter has been described in language specific to structural features and/or methodological acts, it is to be understood that the subject matter defined in the appended claims is not necessarily limited to the specific features or acts described above. Rather, the specific features and acts described above are disclosed as example forms of implementing the claims.
While several specific implementation details are included in the above discussion, these should not be construed as limitations on the scope of the application. Certain features that are described in the context of separate embodiments can also be implemented in combination in a single embodiment. Conversely, various features that are described in the context of a single embodiment can also be implemented in multiple embodiments separately or in any suitable subcombination.
The above description is only a preferred embodiment of the application and is illustrative of the principles of the technology employed. It will be appreciated by those skilled in the art that the scope of the application referred to in the present application is not limited to the embodiments with a particular combination of the above-mentioned features, but also encompasses other embodiments with any combination of the above-mentioned features or their equivalents without departing from the scope of the application. For example, the above features may be replaced with (but not limited to) features having similar functions as those described in this application.
Claims (11)
1. A method for detecting an abnormality in time-series data, comprising:
acquiring time series monitoring data;
extracting high-frequency components and low-frequency components of the time series monitoring data;
performing prediction processing on the basis of the high-frequency component by using a first prediction model to obtain a first prediction result of data in a future time sequence;
performing prediction processing based on the low-frequency component by using a second prediction model to obtain a second prediction result of the data in the future time sequence;
superposing the first prediction result and the second prediction result to obtain a target prediction result of the data in the future time sequence;
and identifying abnormal point data in the real data under the future time sequence according to the target prediction result.
2. The method of claim 1, wherein extracting the high frequency component and the low frequency component of the time series monitoring data comprises:
performing multi-scale decomposition reconstruction on the time series monitoring data by using a wavelet analysis theory, and extracting a high-frequency periodic component, a high-frequency random component and a low-frequency trend component of the time series monitoring data;
wherein the high frequency component includes the high frequency periodic component and the high frequency stochastic component, and the low frequency component includes the low frequency trending component.
3. The method according to claim 2, wherein the extracting the high-frequency periodic component, the high-frequency stochastic component and the low-frequency tendency component of the time-series monitoring data by performing multi-scale decomposition reconstruction on the time-series monitoring data by using wavelet analysis theory comprises:
acquiring deformation actual measurement data of the time series monitoring data, wherein the deformation actual measurement data is obtained by cleaning abnormal values of the time series monitoring data;
taking the deformation actual measurement data as current data to be decomposed;
decomposing the current data to be decomposed into a low-frequency part and a high-frequency part based on wavelet decomposition to obtain a low-frequency component and a high-frequency component corresponding to the current data to be decomposed;
determining whether a low-frequency subsequence contained in a low-frequency component corresponding to current data to be decomposed has a preset variation trend characteristic or not;
if so, ending the decomposition processing of the data;
if not, updating the data to be decomposed into a low-frequency component corresponding to the current data to be decomposed, and circulating to the step of decomposing the current data to be decomposed into a low-frequency part and a high-frequency part until a low-frequency subsequence contained in the low-frequency component corresponding to the current data to be decomposed has the change trend characteristic, and ending the decomposition processing of the data; in the decomposition process, each layer only decomposes low-frequency components, and high-frequency components are not processed;
and extracting the low-frequency component of the last layer in the decomposition result as the low-frequency tendency component, and extracting a high-frequency periodic component and a high-frequency random component from the high-frequency components of the layers.
4. The method according to claim 2, wherein the performing a prediction process based on the high frequency component by using a first prediction model to obtain a first prediction result of data at a future time sequence comprises:
denoising the high-frequency stochastic component;
and inputting the high-frequency periodic component and the denoised high-frequency stochastic component into a long-short term memory network model, and performing prediction processing on the long-short term memory network model based on the high-frequency periodic component and the denoised high-frequency stochastic component to obtain a first prediction result of the data in the future time sequence.
5. The method of claim 4, wherein the denoising the high-frequency stochastic component comprises:
denoising the high-frequency stochastic component based on a preset threshold;
wherein the threshold is
σ 1 = MAD/0.6745, MAD represents the middle value of the absolute value of the wavelet decomposition coefficient of the first layer,0.6745 is the adjustment coefficient of the standard deviation of gaussian noise, and N1 represents the size or length of the high frequency stochastic component signal.
6. The method according to claim 4, wherein the long-short term memory network model comprises a plurality of memory units which are connected in sequence, wherein the memory units comprise a forgetting gate, an input gate and an output gate;
the current memory unit obtains the output information of the current memory unit by carrying out fusion processing on the feature information transmitted by the previous memory unit and the high-frequency component input data of the current memory unit, and processes the feature information transmitted by the previous memory unit by selectively memorizing and forgetting the input high-frequency component when carrying out fusion processing;
respectively processing the high-frequency components input by the memory units and the characteristic information transmitted by the previous memory unit through the memory units connected in sequence to realize the prediction processing of the model based on the high-frequency components;
the characteristic information transmitted by the previous memory unit comprises a state value and an output value of the previous memory unit.
7. The method of claim 2, wherein the performing a prediction process based on the low frequency component using a second prediction model to obtain a second prediction result of the data at the future time sequence comprises:
and inputting the low-frequency tendency component into a differential integration moving average autoregressive model, and performing prediction processing on the low-frequency tendency component by the differential integration moving average autoregressive model to obtain a second prediction result of the data in the future time sequence.
8. The method of claim 1, wherein identifying outlier data in real data at the future time sequence based on the target prediction result comprises:
determining deviation information between real data at the future time sequence and a target prediction result of the data at the future time sequence;
and identifying abnormal point data in the real data under the future time sequence according to the deviation information.
9. An abnormality detection device for time-series data, comprising:
the acquisition module is used for acquiring time series monitoring data;
the extraction module is used for extracting a high-frequency component and a low-frequency component of the time series monitoring data;
the first prediction processing module is used for performing prediction processing on the basis of the high-frequency component by using a first prediction model to obtain a first prediction result of data in a future time sequence;
the second prediction processing module is used for performing prediction processing on the basis of the low-frequency component by using a second prediction model to obtain a second prediction result of the data in the future time sequence;
the superposition module is used for carrying out superposition processing on the first prediction result and the second prediction result to obtain a target prediction result of the data under the future time sequence;
and the abnormal point identification module is used for identifying abnormal point data in the real data under the future time sequence according to the target prediction result.
10. A computer-readable medium, characterized in that a computer program is stored thereon, the computer program comprising program code for executing the abnormality detection method for time-series data according to any one of claims 1 to 8.
11. A computer program product, characterized in that it comprises a computer program carried on a non-transitory computer-readable medium, the computer program comprising program code for executing the method of anomaly detection of time-series data according to any of claims 1-8.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202211060141.3A CN115409066A (en) | 2022-08-31 | 2022-08-31 | Method and device for detecting abnormality of time series data, and computer storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202211060141.3A CN115409066A (en) | 2022-08-31 | 2022-08-31 | Method and device for detecting abnormality of time series data, and computer storage medium |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN115409066A true CN115409066A (en) | 2022-11-29 |
Family
ID=84162715
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202211060141.3A Pending CN115409066A (en) | 2022-08-31 | 2022-08-31 | Method and device for detecting abnormality of time series data, and computer storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN115409066A (en) |
Cited By (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN115827384A (en) * | 2022-12-08 | 2023-03-21 | 国网湖北省电力有限公司信息通信公司 | Optimized real-time data center network equipment monitoring method |
| CN115994833A (en) * | 2023-03-14 | 2023-04-21 | 中化现代农业有限公司 | Crop growth time sequence prediction method, device and storage medium |
| CN116340875A (en) * | 2023-05-30 | 2023-06-27 | 单县鑫和纺织有限公司 | Roving frame operation fault prediction system based on data analysis |
| CN118394970A (en) * | 2024-07-01 | 2024-07-26 | 中数通信息有限公司 | Data supervision method for data management process management |
-
2022
- 2022-08-31 CN CN202211060141.3A patent/CN115409066A/en active Pending
Cited By (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN115827384A (en) * | 2022-12-08 | 2023-03-21 | 国网湖北省电力有限公司信息通信公司 | Optimized real-time data center network equipment monitoring method |
| CN115827384B (en) * | 2022-12-08 | 2024-02-02 | 国网湖北省电力有限公司信息通信公司 | Monitoring method for network equipment of optimized real-time data center |
| CN115994833A (en) * | 2023-03-14 | 2023-04-21 | 中化现代农业有限公司 | Crop growth time sequence prediction method, device and storage medium |
| CN116340875A (en) * | 2023-05-30 | 2023-06-27 | 单县鑫和纺织有限公司 | Roving frame operation fault prediction system based on data analysis |
| CN116340875B (en) * | 2023-05-30 | 2023-10-17 | 单县鑫和纺织有限公司 | Roving frame operation fault prediction system based on data analysis |
| CN118394970A (en) * | 2024-07-01 | 2024-07-26 | 中数通信息有限公司 | Data supervision method for data management process management |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN115409066A (en) | Method and device for detecting abnormality of time series data, and computer storage medium | |
| Ren et al. | Multi-bearing remaining useful life collaborative prediction: A deep learning approach | |
| Pan et al. | A two-stage method based on extreme learning machine for predicting the remaining useful life of rolling-element bearings | |
| Zhao et al. | A recurrent neural network approach for remaining useful life prediction utilizing a novel trend features construction method | |
| Ali et al. | Linear feature selection and classification using PNN and SFAM neural networks for a nearly online diagnosis of bearing naturally progressing degradations | |
| JP6740247B2 (en) | Anomaly detection system, anomaly detection method, anomaly detection program and learned model generation method | |
| Li et al. | Wiener-based remaining useful life prediction of rolling bearings using improved Kalman filtering and adaptive modification | |
| Dai et al. | A reliability evaluation model of rolling bearings based on WKN-BiGRU and Wiener process | |
| CN108763729B (en) | Process industry electromechanical system coupling state evaluation method based on network structure entropy | |
| Yan et al. | Fisher’s discriminant ratio based health indicator for locating informative frequency bands for machine performance degradation assessment | |
| Wang et al. | A hybrid prognostics approach for estimating remaining useful life of wind turbine bearings | |
| Qiu et al. | A piecewise method for bearing remaining useful life estimation using temporal convolutional networks | |
| CN113642754B (en) | Complex industrial process fault prediction method based on RF noise reduction self-coding information reconstruction and time convolution network | |
| Nguyen et al. | Probabilistic deep learning methodology for uncertainty quantification of remaining useful lifetime of multi-component systems | |
| CN105205288A (en) | Mode evolution-based forecasting method for satellite long-term on-orbit running state | |
| US20240085274A1 (en) | Hybrid bearing fault prognosis with fault detection and multiple model fusion | |
| CN118154174B (en) | Intelligent operation and maintenance cloud platform for industrial equipment | |
| Liu et al. | Complex engineered system health indexes extraction using low frequency raw time-series data based on deep learning methods | |
| CN112529234A (en) | Surface water quality prediction method based on deep learning | |
| CN112288021A (en) | Medical wastewater monitoring data quality control method, device and system | |
| CN110757510A (en) | Method and system for predicting remaining life of robot | |
| Zhang et al. | Health indicator based on signal probability distribution measures for machinery condition monitoring | |
| Xu et al. | An adaptive gamma process based model for residual useful life prediction | |
| CN116894153A (en) | Landslide displacement prediction method based on singular spectrum analysis and ARIMA-LSTM combined model | |
| CN116597939A (en) | Medicine quality control management analysis system and method based on big data |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| TA01 | Transfer of patent application right | ||
| TA01 | Transfer of patent application right |
Effective date of registration: 20231227 Address after: 102299 Jiaming Aosen Industrial Park, Changping District, Beijing Applicant after: BEIJING CHINA POWER INFORMATION TECHNOLOGY Co.,Ltd. Applicant after: STATE GRID INFORMATION & TELECOMMUNICATION GROUP Co.,Ltd. Address before: 102299 Jiaming Aosen Industrial Park, Changping District, Beijing Applicant before: BEIJING CHINA POWER INFORMATION TECHNOLOGY Co.,Ltd. |