CN108898117A - A kind of self-adapting random abnormal signal extracting method for sliding threshold value - Google Patents
A kind of self-adapting random abnormal signal extracting method for sliding threshold value Download PDFInfo
- Publication number
- CN108898117A CN108898117A CN201810719528.2A CN201810719528A CN108898117A CN 108898117 A CN108898117 A CN 108898117A CN 201810719528 A CN201810719528 A CN 201810719528A CN 108898117 A CN108898117 A CN 108898117A
- Authority
- CN
- China
- Prior art keywords
- threshold value
- data
- sliding
- random signal
- signal
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F2218/00—Aspects of pattern recognition specially adapted for signal processing
- G06F2218/02—Preprocessing
- G06F2218/04—Denoising
- G06F2218/06—Denoising by applying a scale-space analysis, e.g. using wavelet analysis
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F2218/00—Aspects of pattern recognition specially adapted for signal processing
- G06F2218/08—Feature extraction
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Artificial Intelligence (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Signal Processing (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Geophysics And Detection Of Objects (AREA)
Abstract
The invention discloses a kind of self-adapting random abnormal signal extracting methods for sliding threshold value, include the following steps：The first step：It needs to calculate sliding average；Second step：It needs to calculate moving standard deviation；Third step：Calculate sliding threshold value；4th step：Determine the random signal abnormal data present invention in face of multiresolution, survey item to random signal data, exception is extracted as threshold value using sliding 2 times of mean square deviations of mean value ±, it is reasonable, precursor data abnormal signal rapidly can be quantitatively extracted, the natural calamities such as research anomalous weather, earthquake prediction, tsunami early warning are occurred significant.
Description
Technical field
The present invention relates to it is a kind of extract random signal exception method, specifically, be related to it is a kind of slide threshold value it is adaptive
Answer random signal abnormal extraction method.
Background technique
In actual life and scientific research, random signal is a kind of very common Serial No., at present information
It is widely used in reason technology, such as weather data analysis, Earthquake Precursor Anomalies extract, water table measure processing field makes extensively
With.Random signal geochemical anomalies studying is there are many method at present, including given threshold method, 2 σ methods, can open up method and wavelet analysis
Method etc..
The first threshold method.After this method seeks the mean value that random signal acquires data by segmentation, artificially increase by one
A threshold value, calculation formula are as follows：
Mean value is calculated first：
Wherein x (i) is the random data sequence that one group of length is n.
Secondly a threshold value T given by man is added on the basis of mean value, it is as follows：
μ±T
Data more than up-and-down boundary are considered as abnormal data.
Second of 2 σ methods.Here σ is mean square deviation, and formula is as follows：
Wherein x (i) is the random data sequence that one group of length is n.
Meansquaredeviationσ is the arithmetic square root of variance, reflects the dispersion degree of a data set.
2 σ methods are the threshold value T that mean square deviation is replaced to first method, as follows：
μ±2σ
Threshold value can be obtained and seeking mean square deviation in this way.
The third can open up method.Can the quantization method that solves the problems, such as of the method for opening up be to establish correlation function, according to related extension science
Theory and method, simple correlation function are defined as follows：
Wherein X=<A, b>Referred to as matter-element Classical field, MIX.
In actual operation, the determination of Classical field generally uses the data area in standard or specification, but is not advising
In the case where model, it can be determined using " 2-8 principle ".As shown in Figure 1.
4th kind is wavelet analysis method.Wavelet transformation is most common method in signal analysis, can be in time-domain and frequency
Signal is analysed in depth on domain, is a kind of effective ways for analyzing non-stationary signal.Signal decomposition can be by wavelet decomposition
Low-frequency component (proximate component) and radio-frequency component (details ingredient) under different resolution, low-frequency component can show the big of signal
Cause trend, radio-frequency component illustrate the detailed information of different levels, and radio-frequency component includes noise information, are that abnormal signal exists
Area.Such as seismic precursor observation data belongs to non-stationary signal, by wavelet decomposition, it can be achieved that SNR estimation and compensation, so as to more
Add and effectively extracts exception information.
Wavelet transformation under different scale by stretching to morther wavelet and being translated and make inner product fortune with signal to be analyzed
Calculation obtains wavelet sequence, realizes the wavelet transformation to non-stationary signal, to realize the multi-resolution decomposition to signal.Wavelet basis can
To choose the Daubechies race small echo (dbN) with compactly supported and nearly symmetry, order N.The wavelet decomposition number of plies
Selection will comprehensively consider the smoothness and operation power size of wavelet function, and Decomposition order and wavelet smoothing degree and operation size are at just
Than the general wavelet decomposition number of plies is unsuitable excessively high.Fig. 2 is that db4 small echo is selected to carry out two layers of wavelet decomposition to seismic precursor signal,
Middle Fig. 2A is scaling function, and Fig. 2 B is wavelet function.
In above-mentioned four kinds of abnormal data extracting methods, first two has very big uncertainty, and whether threshold value rationally exists
There are larger disagreements for educational circles；The third Method of extenics needs to be ranked up data that " 2-8 principle " is utilized to seek classics
Domain, but there is uncertainty in this way for mass data or dynamic data, it is difficult to it promotes.Wavelet decomposition method is when to long
Between section random signal data research when, apparent kick, rank change, up and down etc. can only be often found out from initial data
Anomalous variation, and small echo radio-frequency component highlights the exception information of signal while eliminating signal Long-term change trend, is conducive to
It is abnormal that quantitative extraction is carried out to signal using the method for threshold value.But and not all projecting point is all abnormal point, need to comprising
Abnormal radio-frequency component carries out anomaly extracting.
In short, existing random signal abnormal data extracting method the disadvantages of that there are precision is not high, uncertain high.More than
It states and mentions threshold method and 2 σ methods, it is evident that too arbitrarily, cause precision not good enough；In Method of extenics, it is associated with letter despite the use of
The degree of association that number calculates, but still there is precision problem, even usually judge by accident.And Method of extenics, need logarithm
According to being ranked up to utilizing " 2-8 principle " to seek Classical field, but have for mass data or dynamic data in this way
It is uncertain, it is difficult to promote；Wavelet analysis method is in the random signal data research to long period, from initial data often
It can only find out the anomalous variations such as apparent kick, rank change, up and down, and small echo radio-frequency component is eliminating signal Long-term change trend
While highlight the exception information of signal again, although this be conducive to the method using threshold value to signal carry out it is quantitative extract it is different
Often, but and not all projecting point be all abnormal point, there have been paradoxes for this, it is therefore desirable to including abnormal radio-frequency component
Carry out anomaly extracting.
Summary of the invention
It is an object of the invention to overcome defect existing in the prior art, propose it is a kind of sliding threshold value it is adaptive with
On the one hand machine abnormal signal extracting method solves threshold value by subjectivity and determines to cause not objective problem, it is different personal because undergo,
Several aspect factors such as experience, experience, knowledge cause threshold value to determine inconsistent；On the other hand, " 2-8 principle " is also overcomed
It calculates threshold value and needs Small Sample Database or static data, the problem of adapting to is difficult to for dynamic data and big data problem；The
Three, it solves and usually occurs judging by accident when extracting abnormal signal in wavelet analysis method and redundancy issue, especially improve calculating
Efficiency and precision.
Its technical solution is as follows：
A kind of self-adapting random abnormal signal extracting method for sliding threshold value, includes the following steps：
The first step：It needs to calculate sliding average, withIt indicates, then the formula calculated is as follows：
(i=1,2 ..., n-s+1；J=i ..., i+s-1)
In formula,The sliding for observing i-th to (i+s-1) number formation in data sequence for random signal is flat
Mean value, n are the sequence length that random signal observes data, and s is sliding window size, and i is cyclic variable, xjFor j-th of data,
∑ is summation symbol.
Second step：It needs to calculate moving standard deviation, be indicated with σ, formula is as follows：
(i=1,2 ..., n-s+1；J=i ..., i+s-1)
In formula, σ(i:i+s-1)The sliding for observing i-th to (i+s-1) number formation in data sequence for random signal is equal
Variance,For sliding average, n is the sequence length that random signal observes data, and s is sliding window size, and i is to follow
Ring variable, xjFor j-th of data, ∑ is summation symbol.
Third step：Calculate sliding threshold value
It is all common some algorithms in the above-mentioned first step, second step, it is also fairly simple.In this step, it needs to calculate
The supremum and infimum of threshold value, specific calculating formula are as follows：
(i=1,2 ..., n-s+1；J=i ..., i+s-1)
TU(i:i+s-1)For the threshold value upper bound；TL(i:i+s-1)For threshold value lower bound；N positive integer can need root with value 2,3,4 etc.
It is adjusted according to sample frequency, n is the sequence length that random signal observes data, and s is sliding window size, and i is cyclic variable, xjFor
J-th of data.In the random signal anomaly extracting algorithm of forefathers, the determination of threshold value is a fixed value, is not adapted to random
The normal amplitude of signal changes.And numerical value of this method using the multiple of mean square deviation as automatic adaptation signal amplitude variation, energy
It is enough preferably prominent abnormal.
4th step：Determine random signal abnormal data
The threshold value true boundary above and below calculated using third step, determines which belongs to random signal abnormal data by following formula：
xj> TL(i:i+s-1)Or xj< TU(i:i+s-1)For exception
TL(i:i+s-1)≤xj≤TU(i:i+s-1)It is normal
TU(i:i+s-1)For the threshold value upper bound；TL(i:i+s-1)For threshold value lower bound；N positive integer can need root with value 2,3,4 etc.
It is adjusted according to sample frequency, s is sliding window size, and i is cyclic variable, xjFor j-th of data.
Beneficial effects of the present invention are：
The present invention in face of multiresolution, survey item to random signal data, using sliding 2 times of mean square deviation conducts of mean value ±
Threshold value extracts exception, is that reasonably, rapidly can quantitatively extract precursor data abnormal signal, to research anomalous weather, earthquake
The natural calamities such as prediction, tsunami early warning occur significant.
Detailed description of the invention
Fig. 1 is the determination of Classical field；
Fig. 2 is db4 wavelet decomposition, wherein Fig. 2A is scaling function, and Fig. 2 B is wavelet function；
Fig. 3 is the sliding threshold adaptive method calculated result of monitoring of earthquake precursors data in embodiment；
Fig. 4 is the enlarged drawing in Fig. 3 in box.
Specific embodiment
Technical solution of the present invention is described in more detail with reference to the accompanying drawings and detailed description.
Concrete principle of the invention is as follows：
Since n times of mean square deviation of mean value ± is frequently as the decision threshold of anomaly extracting, sliding window thought has data processing
Good adaptivity, the two is combined, formed sliding n times of mean square deviation exception method of mean value ±, can preferably discriminant information it is different
Often.Concrete principle is as follows：
1) sliding average is calculated
(i=1,2 ..., n-s+1；J=i ..., i+s-1)
In formula,The sliding for observing i-th to (i+s-1) number formation in data sequence for random signal is flat
Mean value, n are the sequence length that random signal observes data, and s is sliding window size, and i is cyclic variable, xjFor j-th of data,
∑ is summation symbol.
2) moving standard deviation (σ) is calculated
(i=1,2 ..., n-s+1；J=i ..., i+s-1)
In formula, σ(i:i+s-1)The sliding for observing i-th to (i+s-1) number formation in data sequence for random signal is equal
Variance,For sliding average, n is the sequence length that random signal observes data, and s is sliding window size, and i is to follow
Ring variable, xjFor j-th of data, ∑ is summation symbol.
3) sliding threshold value is calculated
(i=1,2 ..., n-s+1；J=i ..., i+s-1)
TU(i:i+s-1)For the threshold value upper bound；TL(i:i+s-1)For threshold value lower bound；N positive integer can need root with value 2,3,4 etc.
It is adjusted according to sample frequency, n is the sequence length that random signal observes data, and s is sliding window size, and i is cyclic variable, xjFor
J-th of data.
4) random signal data exception judges
xj> TL(i:i+s-1)Or xj< TU(i:i+s-1)For exception (5)
TL(i:i+s-1)< < xj< < TU(i:i+s-1)For normal (6)
TU(i:i+s-1)For the threshold value upper bound；TL(i:i+s-1)For threshold value lower bound；N positive integer can need root with value 2,3,4 etc.
It is adjusted according to sample frequency, s is sliding window size, and i is cyclic variable, xjFor j-th of data.
Embodiment
Below using the acquisition data in certain secondary earthquake for the previous period as sample, the precursory anomaly of the method for the present invention is shown
Data extract result.
Fig. 4 is the enlarged drawing in Fig. 3 in box, there it can be seen that using the threshold value side of sliding 2 times of mean square deviations of mean value ±
Method can detecte out abnormal point.
The foregoing is only a preferred embodiment of the present invention, the scope of protection of the present invention is not limited to this, it is any ripe
Know those skilled in the art within the technical scope of the present disclosure, the letter for the technical solution that can be become apparent to
Altered or equivalence replacement are fallen within the protection scope of the present invention.
Claims (2)
1. a kind of self-adapting random abnormal signal extracting method for sliding threshold value, which is characterized in that include the following steps：
The first step：It needs to calculate sliding average, withIt indicates, then the formula calculated is as follows：
In formula,The i-th sliding average formed to (i+s-1) number, n in data sequence are observed for random signal
The sequence length of data is observed for random signal, s is sliding window size, and i is cyclic variable, xjFor j-th of data, ∑ is to ask
And symbol；
Second step：It needs to calculate moving standard deviation, be indicated with σ, formula is as follows：
In formula, σ(i:i+s-1)The i-th moving standard deviation formed to (i+s-1) number in data sequence is observed for random signal,For sliding average, n is the sequence length that random signal observes data, and s is sliding window size, and i is that circulation becomes
Amount, xjFor j-th of data, ∑ is summation symbol；
Third step：Calculate sliding threshold value
Need to calculate the supremum and infimum of threshold value；
4th step：Determine random signal abnormal data
The threshold value true boundary above and below calculated using third step, determines which belongs to random signal abnormal data by following formula：
xj> TL(i:i+s-1)Or xj< TU(i:i+s-1)For exception
TL(i:i+s-1)≤xj≤TU(i:i+s-1)It is normal
TU(i:i+s-1)For the threshold value upper bound；TL(i:i+s-1)For threshold value lower bound；N positive integer, value 2,3,4, needs according to sample frequency
Adjustment, s are sliding window size, and i is cyclic variable, xjFor j-th of data.
2. the self-adapting random abnormal signal extracting method of sliding threshold value according to claim 1, which is characterized in that third
Step, the specific calculating formula for calculating the supremum and infimum of threshold value are as follows：
TU(i:i+s-1)For the threshold value upper bound；TL(i:i+s-1)For threshold value lower bound；N positive integer, value 2,3,4, needs according to sample frequency
Adjustment, n are the sequence length that random signal observes data, and s is sliding window size, and i is cyclic variable, xjFor j-th of data.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201810719528.2A CN108898117A (en) | 2018-06-30 | 2018-06-30 | A kind of self-adapting random abnormal signal extracting method for sliding threshold value |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201810719528.2A CN108898117A (en) | 2018-06-30 | 2018-06-30 | A kind of self-adapting random abnormal signal extracting method for sliding threshold value |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN108898117A true CN108898117A (en) | 2018-11-27 |
Family
ID=64347823
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201810719528.2A Pending CN108898117A (en) | 2018-06-30 | 2018-06-30 | A kind of self-adapting random abnormal signal extracting method for sliding threshold value |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN108898117A (en) |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110008096A (en) * | 2018-11-29 | 2019-07-12 | 阿里巴巴集团控股有限公司 | Data monitoring method, device, electronic equipment and computer readable storage medium |
| CN114492529A (en) * | 2022-01-27 | 2022-05-13 | 中国汽车工程研究院股份有限公司 | Power battery system connection abnormity fault safety early warning method |
Citations (10)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20070183305A1 (en) * | 2006-02-06 | 2007-08-09 | Whisnant Keith A | High-sensitivity detection of an anomaly in a quantized signal |
| CN103291279A (en) * | 2012-03-02 | 2013-09-11 | 中国石油化工股份有限公司 | Method for optimizing micro abnormal signal of gas logging value |
| CN103543469A (en) * | 2012-07-17 | 2014-01-29 | 中国石油化工股份有限公司 | Small-scale threshold denoising method based on wavelet transform |
| CN104008294A (en) * | 2014-05-30 | 2014-08-27 | 东南大学 | System and method for detecting abnormality of bearing |
| CN105160181A (en) * | 2015-09-02 | 2015-12-16 | 华中科技大学 | Detection method of abnormal data of numerical control system instruction field sequence |
| CN105701456A (en) * | 2016-01-05 | 2016-06-22 | 北京理工大学 | Angular accelerometer signal adaptive denoising method based on wavelet analysis |
| CN106055919A (en) * | 2016-08-09 | 2016-10-26 | 航天东方红卫星有限公司 | Satellite abnormity detection method based on telemetry data wavelet transformation |
| CN106932686A (en) * | 2017-03-21 | 2017-07-07 | 国网上海市电力公司 | Power network exception method for quick based on minimax method of characteristic |
| CN107317701A (en) * | 2017-06-13 | 2017-11-03 | 电子科技大学 | A kind of network flow abnormal detecting method based on empirical mode decomposition |
| CN107356979A (en) * | 2017-05-27 | 2017-11-17 | 淮海工学院 | A kind of method of ionized layer TEC exception detection |
-
2018
- 2018-06-30 CN CN201810719528.2A patent/CN108898117A/en active Pending
Patent Citations (10)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20070183305A1 (en) * | 2006-02-06 | 2007-08-09 | Whisnant Keith A | High-sensitivity detection of an anomaly in a quantized signal |
| CN103291279A (en) * | 2012-03-02 | 2013-09-11 | 中国石油化工股份有限公司 | Method for optimizing micro abnormal signal of gas logging value |
| CN103543469A (en) * | 2012-07-17 | 2014-01-29 | 中国石油化工股份有限公司 | Small-scale threshold denoising method based on wavelet transform |
| CN104008294A (en) * | 2014-05-30 | 2014-08-27 | 东南大学 | System and method for detecting abnormality of bearing |
| CN105160181A (en) * | 2015-09-02 | 2015-12-16 | 华中科技大学 | Detection method of abnormal data of numerical control system instruction field sequence |
| CN105701456A (en) * | 2016-01-05 | 2016-06-22 | 北京理工大学 | Angular accelerometer signal adaptive denoising method based on wavelet analysis |
| CN106055919A (en) * | 2016-08-09 | 2016-10-26 | 航天东方红卫星有限公司 | Satellite abnormity detection method based on telemetry data wavelet transformation |
| CN106932686A (en) * | 2017-03-21 | 2017-07-07 | 国网上海市电力公司 | Power network exception method for quick based on minimax method of characteristic |
| CN107356979A (en) * | 2017-05-27 | 2017-11-17 | 淮海工学院 | A kind of method of ionized layer TEC exception detection |
| CN107317701A (en) * | 2017-06-13 | 2017-11-03 | 电子科技大学 | A kind of network flow abnormal detecting method based on empirical mode decomposition |
Non-Patent Citations (3)
| Title |
|---|
| GU Y等: "A selective KPCA algorithm based on high-order statistics for anomaly detection in hyperspectral imagery", 《IEEE GEOSCIENCE AND REMOTE SENSING LETTERS》 * |
| 安建琴: "关联分析方法在地震前兆数据中的应用研究", 《中国优秀硕士学位论文全文数据库 (基础科学辑)》 * |
| 邹京发: "运用平均值和均方差判断指标的异常波动", 《冶金动力》 * |
Cited By (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110008096A (en) * | 2018-11-29 | 2019-07-12 | 阿里巴巴集团控股有限公司 | Data monitoring method, device, electronic equipment and computer readable storage medium |
| CN110008096B (en) * | 2018-11-29 | 2024-02-06 | 创新先进技术有限公司 | Data monitoring method, device, electronic equipment and computer readable storage medium |
| CN114492529A (en) * | 2022-01-27 | 2022-05-13 | 中国汽车工程研究院股份有限公司 | Power battery system connection abnormity fault safety early warning method |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| Yu et al. | A bearing fault diagnosis technique based on singular values of EEMD spatial condition matrix and Gath-Geva clustering | |
| CN108600135B (en) | Method for identifying signal modulation mode | |
| CN109916628B (en) | Rolling bearing fault diagnosis method based on improved multi-scale amplitude perception permutation entropy | |
| Shahi et al. | An efficient algorithm to identify strong‐velocity pulses in multicomponent ground motions | |
| Gauri et al. | Recognition of control chart patterns using improved selection of features | |
| Gao et al. | Total variation on horizontal visibility graph and its application to rolling bearing fault diagnosis | |
| Cui et al. | An investigation of rolling bearing early diagnosis based on high-frequency characteristics and self-adaptive wavelet de-noising | |
| CN206504869U (en) | A kind of rolling bearing fault diagnosis device | |
| CN117174313B (en) | Method and system for establishing cerebral hemorrhage patient neural function prognosis prediction model | |
| CN107247968A (en) | Based on logistics equipment method for detecting abnormality under nuclear entropy constituent analysis imbalance data | |
| Zhang et al. | Bearing performance degradation assessment based on time-frequency code features and SOM network | |
| CN112213687B (en) | Gateway electric energy meter data anomaly detection method and system based on pseudo-anomaly point identification | |
| WO2017058688A1 (en) | Downhole tool analysis using anomaly detection of measurement data | |
| CN108898117A (en) | A kind of self-adapting random abnormal signal extracting method for sliding threshold value | |
| CN109598175A (en) | It is a kind of based on before multi-wavelet bases function and transothogonal to the Time-Frequency Analysis Method of recurrence | |
| CN106770861B (en) | The evaluation method of oil-filled transformer on-line monitoring availability of data | |
| CN116402825A (en) | Bearing fault infrared diagnosis method, system, electronic equipment and storage medium | |
| Winston et al. | Single parameter fault identification technique for DC motor through wavelet analysis and fuzzy logic | |
| Lan et al. | Improved wavelet packet noise reduction for microseismic data via fuzzy partition | |
| Yan et al. | Mutual Information‐Assisted Wavelet Function Selection for Enhanced Rolling Bearing Fault Diagnosis | |
| Hu et al. | Incipient mechanical fault detection based on multifractal and MTS methods | |
| Cuoco et al. | Wavelet-based classification of transient signals for gravitational wave detectors | |
| CN115860708A (en) | Ship equipment state monitoring and fault diagnosis method | |
| Choong et al. | Advances in signal processing and artificial intelligence technologies in the classification of power quality events: a survey | |
| Milo et al. | Anomaly detection in rolling element bearings via hierarchical transition matrices |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20181127 |