CN114048771A - Time sequence data abnormal value processing method based on adaptive threshold stationary wavelet transformation - Google Patents

Time sequence data abnormal value processing method based on adaptive threshold stationary wavelet transformation Download PDF

Info

Publication number
CN114048771A
CN114048771A CN202111317242.XA CN202111317242A CN114048771A CN 114048771 A CN114048771 A CN 114048771A CN 202111317242 A CN202111317242 A CN 202111317242A CN 114048771 A CN114048771 A CN 114048771A
Authority
CN
China
Prior art keywords
data
sequence
reconstruction
abnormal
layer
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Granted
Application number
CN202111317242.XA
Other languages
Chinese (zh)
Other versions
CN114048771B (en
Inventor
左磊
徐竟翔
李亚超
李明
高永婵
禄晓飞
赵正
李治国
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Xidian University
Original Assignee
Xidian University
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Xidian University filed Critical Xidian University
Priority to CN202111317242.XA priority Critical patent/CN114048771B/en
Publication of CN114048771A publication Critical patent/CN114048771A/en
Application granted granted Critical
Publication of CN114048771B publication Critical patent/CN114048771B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F2218/00—Aspects of pattern recognition specially adapted for signal processing
    • G06F2218/02—Preprocessing
    • G06F2218/04—Denoising
    • G06F2218/06—Denoising by applying a scale-space analysis, e.g. using wavelet analysis
    • Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
    • Y04—INFORMATION OR COMMUNICATION TECHNOLOGIES HAVING AN IMPACT ON OTHER TECHNOLOGY AREAS
    • Y04S—SYSTEMS INTEGRATING TECHNOLOGIES RELATED TO POWER NETWORK OPERATION, COMMUNICATION OR INFORMATION TECHNOLOGIES FOR IMPROVING THE ELECTRICAL POWER GENERATION, TRANSMISSION, DISTRIBUTION, MANAGEMENT OR USAGE, i.e. SMART GRIDS
    • Y04S10/00—Systems supporting electrical power generation, transmission or distribution
    • Y04S10/50—Systems or methods supporting the power network operation or management, involving a certain degree of interaction with the load-side end user applications

Landscapes

  • Complex Calculations (AREA)

Abstract

The invention discloses a time sequence data abnormal value processing method based on adaptive threshold stationary wavelet transform. The method mainly solves the problems that the probability of false detection of the abnormal value is high and the efficiency is low in the prior art. The scheme is as follows: acquiring time sequence data f (n) containing abnormal values to be processed from the time sequence data of the abnormal values to be processed, and performing m-layer stationary wavelet transform to obtain wavelet reconstruction detail coefficients D of each layerj(n) and m-th layer reconstruction approximation coefficient sequence Am(n); summing the reconstruction detail coefficients of each layer to obtain a reconstruction detail coefficient and a sequence D (n); calculating an abnormal value detection threshold corresponding to each element in D (n); judging each element in D (n) according to the threshold to obtain a detected reconstruction detail coefficient and a sequence D' (n); d' (n) is compared with AmAnd (n) adding to obtain data f' (n) processed by abnormal values. The invention can accurately detect and process abnormal values in a large amount of time series data and can be used for cleaning the abnormal data in the time series data.

Description

Time sequence data abnormal value processing method based on adaptive threshold stationary wavelet transformation
Technical Field
The invention belongs to the technical field of signal processing, and particularly relates to a time series data abnormal value processing method which can be used for cleaning abnormal data in large-data-volume time series data.
Background
With the advent of the big data era, there are often some data deviating from the vast majority of data in massive data of each application field due to the influence of accidental or inevitable factors. The mechanism by which these data are generated is different from most other data, and is often referred to as outliers or outliers. If the abnormal data is mixed in the subsequent data analysis processing, the subsequent data analysis results are very adversely affected, and even wrong data analysis results are generated. Therefore, it is necessary to detect abnormal data from a large number of data samples and perform cleaning and repair.
In recent years, an abnormal data detection method based on wavelet transform is widely used in the field of data processing. Due to the flexible calculation mode of the wavelet transform, the high-order change of the time sequence data in a certain local range, namely the change inconsistent with the overall change, can be reflected by the calculated wavelet coefficient. When a wavelet function similar to the data mutation waveform and a proper scale parameter are selected, the wavelet coefficient of the time sequence data obtained by calculation is approximately zero at the stationary part of the time sequence data sample, and the wavelet coefficient at the mutation part is an extreme value, so that the wavelet analysis can effectively detect abnormal data in the time sequence data sample. Unlike the conventional wavelet transform, the stationary wavelet transform is a non-orthogonal wavelet transform that does not downsample after each decomposition, but instead interpolates 2 between every two coefficients of the low-pass and high-pass filters, respectivelyj-1The advantage of the stationary wavelet transform compared with the traditional discrete wavelet transform is that the transformed approximate signal and detail signal have the same length as the original signal, so that the low-frequency coefficient and high-frequency coefficient on each layer are subjected to reconstructionThe method is used for directly adding after reconstructing the filter to obtain the low-frequency coefficient of the upper layer, can effectively avoid the problem of pseudo Gibbs oscillation generated when reconstructing the signal due to the fact that the signal is sampled under the condition that the wavelet base does not have translation invariance, and makes up the defects of the traditional discrete wavelet transform.
The existing mature abnormal data detection method based on the traditional wavelet transform usually detects abnormal coefficient points in detail coefficients of each layer based on a global threshold value obtained by calculating the detail wavelet coefficients of each layer. The specific threshold calculation steps are as follows: firstly, performing m-layer discrete wavelet transform on a time sequence data sample to obtain each layer of wavelet detail coefficients; then, calculating the global mean value and the standard deviation of each layer of wavelet coefficient; then, according to the 3 sigma rule, detecting the wavelet detail coefficients of each layer in sequence, comparing the absolute value of the difference value of each point wavelet detail coefficient and the global mean value of the layer wavelet coefficient with three times of the standard deviation of the layer wavelet coefficient, and if the absolute value is more than three times of the standard deviation, indicating that the point wavelet detail coefficient is abnormal, namely the point data in the original time sequence data sample is abnormal data.
The method has the advantages that the abnormal data detection of the original data sample can be realized by effectively and intuitively detecting the size of the wavelet detail coefficient of each layer through the wavelet transformation, the calculation process is very simple, and meanwhile, the method can accurately detect the abnormal data in the time sequence data sample with high detection probability. The deficiencies are shown as follows:
(1) the method constructs an abnormal data detection threshold based on the global statistical characteristics of wavelet detail coefficients, when time series data with complex stationarity is processed, the abnormal data detection threshold is influenced by a non-stationary data segment and a stationary data segment, when the stationary data segment is processed, part of abnormal data is smaller than the detection threshold, and the missing detection phenomenon is generated; when the non-stationary data segment is processed, part of normal data is larger than the detection threshold, and the phenomenon of false detection occurs, and the two phenomena finally cause the detection performance to be reduced.
(2) Because the method utilizes the traditional discrete wavelet transform, when abnormal data in original data is detected based on wavelet detail coefficients of all layers, the wavelet detail coefficients of a plurality of data points around the abnormal data are all detected as abnormal wavelet coefficients, and the specific unit data cannot be accurately determined as the abnormal data simply through the values of the wavelet detail coefficients, so that the judgment is usually carried out by means of other information in the original data, and the efficiency of detecting the abnormal data is greatly reduced;
(3) due to the limitation of the traditional discrete wavelet transform, the method can only detect abnormal data in the data according to the reconstructed wavelet detail coefficients, and cannot reconstruct original time sequence data by processing the detected abnormal wavelet detail coefficients. Therefore, only abnormal data can be detected, and other filling methods are needed for processing the detected abnormal data, so that the integration of detection and processing of the abnormal data cannot be realized.
Disclosure of Invention
The invention aims to overcome the defects of the prior art and provides a time sequence data abnormal value processing method based on adaptive threshold stationary wavelet transform, so as to improve the detection performance and detection efficiency of abnormal data in a complex time sequence data sample and realize the detection and processing integration of the abnormal data.
In order to achieve the technical purpose, the technical scheme of the invention comprises the following steps:
(1) acquiring time sequence data f (N) to be processed with the length of N, performing m-layer stationary wavelet transform on the time sequence data f (N), and calculating a wavelet reconstruction detail coefficient sequence D of each layerj(n), and reconstructing an approximation coefficient sequence Am(n)，j＝1,2...m；
(2) Reconstructing detail coefficient sequence D for m layers of waveletsj(n) summing to obtain reconstructed detail coefficients and a sequence D (n);
(3) calculating the upper limit P of the abnormal value detection threshold corresponding to each element in the D (n) according to a box type graph method by using the reconstruction detail coefficient and the L nearest elements around each element in the sequence D (n)1(n) and a lower limit P2(n)；
(4) Comparing the reconstruction detail coefficient with each element in the sequence D (n) and the corresponding upper limit and lower limit thereof, and judging whether the element is abnormal;
(4a) let d (q) be the qth element of d (N), q ═ 1, 2.. N;
if P2(q)＜D(q)＜P1(q), if D (q) is normal, i.e. f (q) in the time sequence data is correct, do not process D (q), wherein P1(q) is D (q) the upper limit of the abnormal value detection threshold corresponding to P2(q) is D (q) the lower limit of the abnormal value detection threshold corresponding to D (q);
if D (q) < P2(q) or D (q) > P1(q), d (q) is judged to be abnormal, namely f (q) in the time sequence data is abnormal data, and d (q) is made to be 0;
(4b) according to the operation process of (4a), completing the detection of all elements in D (n), and obtaining a detected reconstruction detail coefficient and a sequence D' (n);
(5) the detected reconstruction detail coefficient and sequence D' (n) and the m-th layer reconstruction approximate coefficient A are comparedmAnd (n) adding to obtain data f' (n) processed by abnormal values.
The invention has the following beneficial effects:
in the time series data abnormal value processing, the invention adopts the technical scheme that each layer of reconstructed wavelet detail coefficients after the stable wavelet transformation of the original time series data is used as the detection characteristic quantity for the first time, then the local self-adaptive detection threshold is constructed based on the box graph method, the abnormal data in the original time series data is detected according to the reconstructed detail coefficients and the sequence, and finally the original data is reconstructed and calculated based on the processed reconstructed detail coefficients and the sequence to realize the abnormal data processing.
The experimental result shows that when the method is used for processing time series data under the conditions of different signal-to-noise ratios or complex signal-to-noise ratios, the abnormal value false detection probability is lower than that of the traditional abnormal data detection method based on wavelet transformation, and meanwhile, the abnormal value detection probability is higher under the condition of low signal-to-noise ratio.
Drawings
FIG. 1 is a flow chart of an implementation of the present invention;
FIG. 2 is a graph of sequence data of an outlier-containing sine wave to be processed in an experiment according to the present invention;
FIG. 3 is a graph illustrating the abnormal value processing result of the sinusoidal sequence data in FIG. 2 according to the present invention;
FIG. 4 is a comparison graph of the average detection probability of 10000 Monte Carlo tests performed under different SNR conditions using the wavelet transform abnormal data detection method based on the global threshold of the present invention and the conventional wavelet transform abnormal data detection method;
FIG. 5 is a comparison graph of the average false detection probability of 10000 Monte Carlo tests performed under different SNR conditions using the wavelet transform abnormal data detection method based on the global threshold of the present invention and the conventional wavelet transform abnormal data detection method;
FIG. 6 is a graph of complex SNR outlier data for processing used in experiments with the present invention;
FIG. 7 is a graph of adaptive threshold results from the detection of outliers in the data of FIG. 6 using the present invention;
FIG. 8 is a graph illustrating the results of processing the outliers of the data of FIG. 6 according to the present invention;
fig. 9 is a comparison graph of false detection probabilities of 1000 monte carlo tests performed on the data in fig. 6 by using the wavelet transform abnormal data detection method based on the global threshold according to the present invention and the conventional method.
Detailed Description
Embodiments and effects of the present invention will be described in further detail below with reference to the accompanying drawings.
Referring to fig. 1, the implementation steps of the invention are as follows:
step 1, acquiring time sequence data f (n) to be processed, and setting a wavelet transformation basis function and a wavelet transformation layer number m.
Acquiring time series data f (N) containing abnormal values to be processed from the time series data needing to process the abnormal values, and calculating the data length of the time series data, wherein the assumed data length is N;
in wavelet transformation, because wavelet transformation basis functions have characteristics when processing data, when abnormal value detection is carried out by utilizing wavelet transformation, wavelet functions similar to a mutation waveform are required to be selected as the wavelet transformation basis functions, the wavelet transformation basis functions commonly used for abnormal value detection at present comprise dbN series wavelet functions, symN series wavelet functions, haar wavelets and the like, and the db8 wavelet function is selected as the wavelet transformation basis function in the embodiment without limitation;
according to the characteristic that the larger the wavelet transform layer number is, the more obvious the different characteristics of the abnormal value and the normal data are, the more favorable the detection of the abnormal value is, the larger the decomposition layer number is, the larger the reconstructed signal distortion is, and the contradiction that the final data reconstruction calculation effect is influenced to a certain extent is considered, the wavelet transform layer number m is usually 3-8, and in the example, the wavelet transform layer number is selected from, but not limited to, m-5.
Step 2, performing m-layer stationary wavelet transform on the time series data f (n) containing the abnormal values to be processed, and calculating wavelet reconstruction detail coefficient sequences D of each layerj(n) and m-th layer reconstruction approximation coefficient sequence Am(n)。
The stationary wavelet transform is a non-orthogonal wavelet transform that does not downsample after each decomposition, but rather interpolates 2 between every two coefficients of the low-pass and high-pass decomposition filters, respectivelyj-1The smooth wavelet transform has the advantages that the transformed approximate coefficient and detail coefficient have the same length as the original signal, so that the approximate coefficient and detail coefficient on each layer can be directly added after acting on the reconstruction filter during reconstruction to obtain the approximate coefficient on the previous layer, and signal information loss caused by sampling under the signal can be effectively avoided. Therefore, in this example, the stationary wavelet transform is selected, m layers of stationary wavelet transforms are performed on f (n), and the reconstructed detail coefficients D of each layer are calculatedj(n) and m-th layer reconstruction approximation coefficient Am(n) which is implemented as follows:
2a) performing m-layer stationary wavelet transform on the time sequence data f (n), and calculating wavelet detail coefficient sequences d of each layerj(n) and an approximation coefficient sequence aj(n)，j＝1,2...m；
Figure BDA0003344135780000051
Figure BDA0003344135780000052
Wherein h isj(n) is a jth layer low-pass decomposition filter, gj(n) is a jth layer high-pass decomposition filter determined by wavelet transform basis functions,
Figure BDA0003344135780000053
represents a convolution operation;
2b) calculating wavelet reconstruction detail coefficient sequence D of each layerj(n) and m-th layer reconstruction approximation coefficient sequence Am(n)：
Figure BDA0003344135780000054
Figure BDA0003344135780000055
Where H (n) is a low-pass reconstruction filter, G (n) is a high-pass reconstruction filter, determined by wavelet transform basis functions, am(n) is the mth layer approximation coefficient.
Step 3, reconstructing a detail coefficient sequence D according to the waveletj(n) obtaining an abnormal value detection feature quantity.
Reconstructing detail coefficient D for each layer of waveletj(n) summing to obtain the reconstructed detail coefficients and the sequence d (n):
Figure BDA0003344135780000056
and selecting the reconstruction detail coefficient and the sequence D (n) as the abnormal value detection characteristic quantity.
Compared with the traditional method for selecting the detail coefficient D of wavelet reconstruction of each layer by using the detail coefficient D (n) of reconstruction and the sequence D (n) as the abnormal value detection characteristic quantity in the examplej(n) the method of detecting a feature amount as an abnormal value has the following two advantages: firstly, the reconstruction detail coefficient and the sequence D (n) are more accurate in detection of the abnormal value, and the surrounding of the abnormal value cannot be causedFalse detection of data; and secondly, the reconstructed detail coefficient and the sequence D (n) can be used for detection, direct D (n) can be processed according to the detection result, and then wavelet reconstruction calculation is carried out on the original data to realize abnormal value processing in the time sequence data.
And 4, constructing a self-adaptive detection threshold according to a box graph method, and sequentially detecting the detail coefficient and each element in the sequence D (n).
The box-type graph method is a standardized detection method for displaying data distribution based on statistical measurement, and is different from other statistical analysis methods such as a 3 sigma method in that data samples are supposed to be normally distributed, the box-type graph method has better robustness for actual conditions with more complex data distribution, and abnormal value identification results are more objective. Therefore, in this example, a box-type graph method is selected to construct the adaptive detection threshold, and the detail coefficient and each element in the sequence d (n) are sequentially detected, which is implemented as follows:
4.1) obtaining the nearest L elements of each element in d (n), where L is set according to the data length, and is selected from, but not limited to, L being 50 in this example;
4.2) median Q according to the nearest L elements of each element0(n) lower quartile QL(n) and upper quartile QU(n) calculating the quartile range IQR (n) of the nearest L elements of each element
IQR(n)＝QU(n)-QL(n)；
4.3) median Q from L elements0(n) and the quartile interval IQR (n), calculating the abnormal value detection threshold upper limit P corresponding to each element1(n) and lower limit P2(n)：
P1(n)＝Q0(n)+1.5IQR(n)
P2(n)＝Q0(n)-1.5IQR(n)；
4d) Comparing the reconstruction detail coefficient with each element in the sequence D (n) and the corresponding upper limit and lower limit thereof, and judging whether the reconstruction detail coefficient is abnormal:
let d (q) be the qth element of d (N), q ═ 1, 2.. N;
if P2(q)＜D(q)＜P1(q), then D (q) is normal, i.e., time sequenceF (q) in the data is correct data, and D (q) is not processed;
if D (q) < P2(q) or D (q) > P1(q), d (q) is judged to be abnormal, namely f (q) in the time sequence data is abnormal data, and d (q) is made to be 0, wherein P1(q) is D (q) the upper limit of the abnormal value detection threshold corresponding to P2And (q) is the lower limit of the abnormal value detection threshold corresponding to D (q), and the detection of all elements in D (n) is completed to obtain the reconstruction detail coefficient and the sequence D' (n) after the detection.
Step 5, the detected reconstruction detail coefficient and sequence D' (n) and the m-th layer reconstruction approximate coefficient AmAnd (n) adding to obtain data f' (n) processed by abnormal values.
Figure BDA0003344135780000071
Wherein D' (n) reconstructed detail coefficients and sequences after detection, Am(n) is the mth layer reconstruction approximation coefficient.
The effect of the invention can be further illustrated by the following simulation experiment:
1 experimental environment: matlab R2018a, Window 10
2, experimental conditions:
the experiment is based on sine sequence data generated by simulation, and the data length is 5120;
experiment 1: the method of the invention is adopted to process the abnormal value of the sine data containing the abnormal value in the figure 2, and the processed result is shown in figure 3.
As can be seen from FIG. 3, the abnormal values in the time series data can be accurately detected and processed, compared with the traditional wavelet transform abnormal value detection method based on the global threshold, the abnormal value detection method only can detect the abnormal values, and the replacement of the abnormal values is realized by other methods, so that the method has higher data processing efficiency.
Experiment 2: the invention and the traditional wavelet transform abnormal value detection method based on the global threshold are used for respectively detecting abnormal values in sine sequences containing random abnormal values under the conditions of different signal-to-noise ratios and carrying out 10000 Monte Carlo tests to obtain various informationAverage abnormal value detection probability P of two methods under noise ratio conditiondThe comparison result is shown in FIG. 4, where the average abnormal value false detection probability PwThe comparative results are shown in FIG. 5.
As can be seen from fig. 4 and 5, compared with the conventional method, under the condition of different signal-to-noise ratios, the method of the present invention has not only a very high abnormal data detection probability but also a lower false detection probability, which indicates that the method of the present invention has a better abnormal data detection performance than the conventional detection method.
Experiment 3: the complex signal-to-noise ratio sinusoidal data containing the abnormal value in the figure 6 is detected and processed by the method of the invention, the detection threshold obtained based on the boxplot method is shown in figure 7, and the processed result is shown in figure 8.
As can be seen from fig. 7 and 8, the abnormal value detection threshold constructed based on the boxed graph method in the invention can adaptively adjust the threshold value when processing the complex time series data, thereby realizing stable detection and processing of the abnormal value in the complex signal-to-noise ratio time series data.
Experiment 4: by using the method and the traditional wavelet transform abnormal value detection method based on the global threshold, abnormal values in the sinusoidal data containing random abnormal values of the complex signal-to-noise ratio in the graph 6 are respectively detected, and 1000 Monte Carlo tests are carried out, so that the comparison result of the false detection probability of the abnormal values in each Monte Carlo test is obtained and is shown in the graph 9.
As can be seen from FIG. 9, when processing time series data under the condition of complex signal-to-noise ratio, the invention has lower false detection probability than the traditional wavelet transform outlier detection method based on the global threshold.
The foregoing description is only an example of the present invention and is not intended to limit the invention, so that it will be apparent to those skilled in the art that various changes and modifications in form and detail may be made therein without departing from the spirit and scope of the invention.

Claims (5)

1. A time series data abnormal value processing method based on adaptive threshold stationary wavelet transform is characterized by comprising the following steps:
(1) acquiring time series data f (N) containing abnormal values to be processed from the time series data needing to process the abnormal values, calculating the data length of the time series data, supposing N, performing m layers of stationary wavelet transform on the time series data, and calculating the wavelet reconstruction detail coefficient sequence D of each layerj(n), and reconstructing an approximation coefficient sequence Am(n)，j＝1,2...m；
(2) Reconstructing detail coefficient sequence D for m layers of waveletsj(n) summing to obtain reconstructed detail coefficients and a sequence D (n);
(3) calculating the upper limit P of the abnormal value detection threshold corresponding to each element in the D (n) according to a box type graph method by using the reconstruction detail coefficient and the L nearest elements around each element in the sequence D (n)1(n) and a lower limit P2(n)；
(4) Comparing the reconstruction detail coefficient with each element in the sequence D (n) and the corresponding upper limit and lower limit thereof, and judging whether the element is abnormal;
(4a) let d (q) be the qth element of d (N), q ═ 1, 2.. N;
if P2(q)＜D(q)＜P1(q), if D (q) is normal, i.e. f (q) in the time sequence data is correct data, and D (q) is not processed;
if D (q) < P2(q) or D (q) > P1(q), d (q) is judged to be abnormal, namely f (q) in the time sequence data is abnormal data, and d (q) is made to be 0;
wherein P is1(q) is D (q) the upper limit of the abnormal value detection threshold corresponding to P2(q) is D (q) the lower limit of the abnormal value detection threshold corresponding to D (q);
(4b) according to the operation process of (4a), completing the detection of all elements in D (n), and obtaining a detected reconstruction detail coefficient and a sequence D' (n);
(5) the detected reconstruction detail coefficient and sequence D' (n) and the m-th layer reconstruction approximate coefficient A are comparedmAnd (n) adding to obtain data f' (n) processed by abnormal values.
2. The method of claim 1, wherein wavelet reconstruction details for each layer are calculated in (1)Pitch coefficient sequence Dj(n) and m-th layer reconstruction approximation coefficient sequence Am(n), expressed as follows:
1a) calculating wavelet detail coefficient sequence d of each layerj(n) and wavelet approximation coefficient sequence aj(n)：
Figure FDA0003344135770000021
Figure FDA0003344135770000022
Wherein h isj(n) is a jth layer low-pass decomposition filter, gj(n) is a jth layer high-pass decomposition filter determined by wavelet transform basis functions,
Figure FDA0003344135770000023
represents a convolution operation;
1b) calculating wavelet reconstruction detail coefficient sequence D of each layerj(n) and m-th layer reconstruction approximation coefficient sequence Am(n)：
Figure FDA0003344135770000024
Figure FDA0003344135770000025
Where H (n) is a low-pass reconstruction filter, G (n) is a high-pass reconstruction filter, determined by wavelet transform basis functions, am(n) is the mth layer approximation coefficient.
3. The method as claimed in claim 1, wherein in (2), the detail coefficient sequence D is reconstructed for m layers of waveletsj(n) summing to obtain the reconstructed detail coefficients and the sequence d (n), as follows:
Figure FDA0003344135770000026
4. the method as claimed in claim 1, wherein in (3), the reconstruction detail coefficient and the abnormal value detection upper threshold limit P corresponding to each element in the sequence D (n) are calculated according to a box plot method1(n) and a lower limit P2(n) which is implemented as follows:
3a) obtaining the L nearest elements of each element in D (n);
3b) median Q according to the nearest L elements of each element0(n) lower quartile QL(n) and upper quartile QU(n) calculating the quartile range IQR (n) of the nearest L elements of each element
IQR(n)＝QU(n)-QL(n)
3c) Median Q from L elements0(n) and the quartile interval IQR (n), calculating the abnormal value detection threshold upper limit P corresponding to each element1(n) and lower limit P2(n)：
Figure FDA0003344135770000031
5. The method according to claim 1, wherein the outlier processed data f' (n) obtained in (5) is represented as follows:
Figure FDA0003344135770000032
wherein D' (n) reconstructed detail coefficients and sequences after detection, Am(n) is the mth layer reconstruction approximation coefficient.
CN202111317242.XA 2021-11-09 2021-11-09 Time sequence data outlier processing method based on adaptive threshold stable wavelet transformation Active CN114048771B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202111317242.XA CN114048771B (en) 2021-11-09 2021-11-09 Time sequence data outlier processing method based on adaptive threshold stable wavelet transformation

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202111317242.XA CN114048771B (en) 2021-11-09 2021-11-09 Time sequence data outlier processing method based on adaptive threshold stable wavelet transformation

Publications (2)

Publication Number Publication Date
CN114048771A true CN114048771A (en) 2022-02-15
CN114048771B CN114048771B (en) 2023-05-30

Family

ID=80207578

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202111317242.XA Active CN114048771B (en) 2021-11-09 2021-11-09 Time sequence data outlier processing method based on adaptive threshold stable wavelet transformation

Country Status (1)

Country Link
CN (1) CN114048771B (en)

Cited By (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN116074876A (en) * 2023-03-07 2023-05-05 南京邮电大学 Communication base station abnormality detection method based on wavelet transformation
CN116740053A (en) * 2023-08-08 2023-09-12 山东顺发重工有限公司 Management system of intelligent forging processing production line
CN116821836A (en) * 2023-08-31 2023-09-29 深圳特力自动化工程有限公司 Multi-sensor-based axle bush abnormal state monitoring method and system

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN105701456A (en) * 2016-01-05 2016-06-22 北京理工大学 Angular accelerometer signal adaptive denoising method based on wavelet analysis
CN106055919A (en) * 2016-08-09 2016-10-26 航天东方红卫星有限公司 Satellite abnormity detection method based on telemetry data wavelet transformation
CN109635781A (en) * 2018-12-29 2019-04-16 国网陕西省电力公司电力科学研究院 The coarse Data Detection modification method of digital signal and system based on wavelet transformation
CN110839016A (en) * 2019-10-18 2020-02-25 平安科技（深圳）有限公司 Abnormal flow monitoring method, device, equipment and storage medium
US20210333237A1 (en) * 2020-04-27 2021-10-28 Harbin Engineering University Distortion-free boundary extension method for online wavelet denoising

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN105701456A (en) * 2016-01-05 2016-06-22 北京理工大学 Angular accelerometer signal adaptive denoising method based on wavelet analysis
CN106055919A (en) * 2016-08-09 2016-10-26 航天东方红卫星有限公司 Satellite abnormity detection method based on telemetry data wavelet transformation
CN109635781A (en) * 2018-12-29 2019-04-16 国网陕西省电力公司电力科学研究院 The coarse Data Detection modification method of digital signal and system based on wavelet transformation
CN110839016A (en) * 2019-10-18 2020-02-25 平安科技（深圳）有限公司 Abnormal flow monitoring method, device, equipment and storage medium
US20210333237A1 (en) * 2020-04-27 2021-10-28 Harbin Engineering University Distortion-free boundary extension method for online wavelet denoising

Non-Patent Citations (3)

* Cited by examiner, † Cited by third party
Title
卓宁: "基于多小波基多信源融合异常值剔除方法研究", 《宇航计测技术》 *
臧玉萍 等: "基于小波变换技术的发动机异响故障诊断", 《机械工程学报》 *
董泽 等: "基于EWT-LOF的热工过程数据异常值检测方法", 《仪器仪表学报》 *

Cited By (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN116074876A (en) * 2023-03-07 2023-05-05 南京邮电大学 Communication base station abnormality detection method based on wavelet transformation
CN116740053A (en) * 2023-08-08 2023-09-12 山东顺发重工有限公司 Management system of intelligent forging processing production line
CN116740053B (en) * 2023-08-08 2023-11-07 山东顺发重工有限公司 Management system of intelligent forging processing production line
CN116821836A (en) * 2023-08-31 2023-09-29 深圳特力自动化工程有限公司 Multi-sensor-based axle bush abnormal state monitoring method and system
CN116821836B (en) * 2023-08-31 2023-10-27 深圳特力自动化工程有限公司 Multi-sensor-based axle bush abnormal state monitoring method and system

Also Published As

Publication number Publication date
CN114048771B (en) 2023-05-30

Similar Documents

Publication Publication Date Title
CN114048771B (en) Time sequence data outlier processing method based on adaptive threshold stable wavelet transformation
CN110361778B (en) Seismic data reconstruction method based on generation countermeasure network
US20210333237A1 (en) Distortion-free boundary extension method for online wavelet denoising
CN112307619B (en) Construction method of early warning model, equipment fault early warning method and device
CN110717472B (en) Fault diagnosis method and system based on improved wavelet threshold denoising
CN109992608B (en) Multi-model fusion prediction method and system based on frequency domain
CN113670616B (en) Bearing performance degradation state detection method and system
CN113486750A (en) Oil-immersed transformer partial discharge signal denoising method based on improved VMD algorithm and wavelet packet denoising algorithm
CN109724693B (en) Fusion spectrum denoising method based on stationary wavelet
CN113670186A (en) GNSS deformation monitoring denoising method combining variational modal decomposition and permutation entropy
CN109557434B (en) Partial discharge signal identification method under strong background noise based on composite dictionary sparse representation classification
TWI428581B (en) Method for identifying spectrum
CN113589253A (en) Method for detecting weak echo signal based on wavelet transform algorithm of pseudo time domain
Thibault et al. Comparison of signal processing methods considering their optimal parameters using synthetic signals in a heat exchanger network simulation
CN116150594B (en) Method for identifying switch element characteristics in spectrum test data
CN113033323A (en) Gear fault identification method based on filter decomposition combination index
CN110112757B (en) Low-frequency oscillation analysis method based on SURE wavelet denoising and improved HHT
CN111582205A (en) Noise reduction method based on multi-resolution singular value decomposition model
CN116167007A (en) Analog circuit detection method based on gating recursion unit self-coding neural network
CN116522080A (en) Partial discharge signal noise reduction method
CN111144230A (en) Time domain load signal denoising method based on VMD
CN116046968A (en) Liquid chromatography workstation data processing method, system and storable medium
CN116667920A (en) Transient signal detection method combining principal component analysis and Hilbert-Huang transform
CN114612434B (en) Corrugated pipe surface defect detection method and system
CN115563476A (en) Bayes sparse representation method for mechanical fault redundancy lifting wavelet dictionary

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant