CN104156473B - LS-SVM-based method for detecting anomaly slot of sensor detection data - Google Patents

LS-SVM-based method for detecting anomaly slot of sensor detection data Download PDF

Info

Publication number
CN104156473B
CN104156473B CN201410422006.8A CN201410422006A CN104156473B CN 104156473 B CN104156473 B CN 104156473B CN 201410422006 A CN201410422006 A CN 201410422006A CN 104156473 B CN104156473 B CN 104156473B
Authority
CN
China
Prior art keywords
data
svm
length
fragment
detection
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
CN201410422006.8A
Other languages
Chinese (zh)
Other versions
CN104156473A (en
Inventor
刘大同
彭宇
宋歌
庞景月
彭喜元
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Harbin Institute of Technology
Original Assignee
Harbin Institute of Technology
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Harbin Institute of Technology filed Critical Harbin Institute of Technology
Priority to CN201410422006.8A priority Critical patent/CN104156473B/en
Publication of CN104156473A publication Critical patent/CN104156473A/en
Application granted granted Critical
Publication of CN104156473B publication Critical patent/CN104156473B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Classifications

    • G—PHYSICS
    • G05—CONTROLLING; REGULATING
    • G05B—CONTROL OR REGULATING SYSTEMS IN GENERAL; FUNCTIONAL ELEMENTS OF SUCH SYSTEMS; MONITORING OR TESTING ARRANGEMENTS FOR SUCH SYSTEMS OR ELEMENTS
    • G05B23/00—Testing or monitoring of control systems or parts thereof
    • G05B23/02—Electric testing or monitoring
    • G05B23/0205—Electric testing or monitoring by means of a monitoring system capable of detecting and responding to faults
    • G05B23/0218—Electric testing or monitoring by means of a monitoring system capable of detecting and responding to faults characterised by the fault detection method dealing with either existing or incipient faults
    • G05B23/0243—Electric testing or monitoring by means of a monitoring system capable of detecting and responding to faults characterised by the fault detection method dealing with either existing or incipient faults model based detection method, e.g. first-principles knowledge model
    • G05B23/0254—Electric testing or monitoring by means of a monitoring system capable of detecting and responding to faults characterised by the fault detection method dealing with either existing or incipient faults model based detection method, e.g. first-principles knowledge model based on a quantitative model, e.g. mathematical relationships between inputs and outputs; functions: observer, Kalman filter, residual calculation, Neural Networks

Abstract

The invention discloses an LS-SVM-based method for detecting an anomaly slot of sensor detection data, and relates to the field of anomaly detection of spacecraft monitoring data. The LS-SVM-based method aims at solving the problem that a short-time trend appearing in a time series or changes or anomalies appearing in a mode can not be easily judged in an existing single-detecting-point anomaly detection mode, and includes the steps of (1) setting the needed detection fiducial probability p, the detection slot length n and the minimum number m of abnormal points in the slot; (2) carrying out LS-SVM point anomaly detection with the point anomaly detection fiducial probability p on data in the time series length n from the moment t＜0＞, and obtaining predication residual errors and the number of the data anomaly points in the time series length n; (3) determining the positions of the anomaly points according to the residual errors and the number of the data anomaly points. The LS-SVM-based method can be applied in the aerospace flight vehicle monitoring field.

Description

A kind of sensor detection data exception fragment detection method based on LS-SVM
Technical field
The present invention relates to a kind of sensor detection data exception fragment detection method based on LS-SVM.Belong to spacecraft prison Survey data exception detection field.
Background technology
With the modernization of national defense fast development and national security in the urgent need to need of the China to all kinds of spacecrafts Constantly growth, and the completeness and reliability to spacecraft function is asked to put forward higher requirement.In order to ensure such space flight The highly reliable and long-life of equipment, during design, development, production, use, maintenance, substantial amounts of test is be unable to do without all the time Work.By taking satellite as an example, as a class large-sized multifunction complication system, be born in a satellite, transmitting, the full longevity of in-orbit maintenance In the life cycle, it will have substantial amounts of test data to be recorded, when these data are often with time serieses, particularly multidimensional Between in the form of sequence.If these time series datas can be carried out with science, effectively analyzed and processed, in finding data Exception, can be just that the status real time monitor of satellite and health maintenance provide foundation.With spacecraft quantity increase and set Gradually stepping up for complexity is counted, the quantity and fault rate of event of failure also have obvious amplification.The time serieses number of test gained Hidden failure develops or incipient fault has close ties for the change of exception according in and spacecraft mode of operation, only Only by convectional reliability engineering method means, the manual analyses method of expert personal experience is relied on to be difficult to carry out data Fully process, this is not only the waste of information, while being also difficult to meet spacecraft safe and reliable operation to data mining and analysis The demand of process.Therefore, abnormality detection analysis is carried out to time series data effectively how in real time, to spacecraft Working mould The judgement of formula, operation conditions and health degree has vital effect.This problem is from all kinds of space flight with satellite as representative The application demand of the abnormality detection of device test data is set out, for the seasonal effect in time series abnormality detection problem exhibition that numerous areas occur Open research.
Least square method supporting vector machine (Least Square Support Vector Machine, LS-SVM) conduct A kind of innovatory algorithm of vector machine is held, has the advantages that model training efficiency high, learning performance are good, be widely used in solving Classification and regression problem.LS-SVM is a kind of machine learning algorithm of employing empirical risk minimization, in the bar of small sample Training can be just completed under part, generalization ability is strong, training effectiveness is high.Moreover, LS-SVM model structures are simple, adopt equation Constraints Optimization Solution, it is easy to realize under the conditions of Embedded Application, the hardware platform based on LS-SVM can be developed.It is based on The above advantage and laboratory of LS-SVM algorithms is accumulated to the early stage of this algorithm, and decision sets time a sequence using this algorithm The research of row method for detecting abnormality.
To sum up, the time serieses method for detecting abnormality based on LS-SVM studies the status monitoring to complication systems such as spacecrafts There is important reference value and practical significance with abnormality detection application.
The content of the invention
The present invention is to be difficult to judge occur in time serieses for the abnormality detection mode for solving existing single test point Short-term trend or the change that occurs of pattern or exception problem.A kind of sensor detection data based on LS-SVM is provided now different Normal fragment detection method.
A kind of sensor detection data exception fragment detection method based on LS-SVM, it comprises the steps：
There is number m, and n in the confidence level c of detection needed for step one, setting, length of time series n and minimum abnormity point Setting with m meets the restriction of confidence level c, and n and m is positive integer；
Step 2, from t0From moment the data in length of time series n are carried out with an abnormality detection confidence probability for p's LS-SVM point abnormality detections, the number of the abnormity point of prediction residual and data in acquisition length of time series n；
Whether step 3, the number for judging abnormity point during length is for the time serieses of n in step 2 are at least m abnormal Point, i.e., | En(t0) | >=m, m are positive integer, | En(t0) | express time sequence fragment, if it is, the length of time series is n Time serieses in there is abnormal data, execution step four, if it is not, then to t0The data execution step two at+1 moment；
The time range that step 4, abnormal data are present is [t0-n+1,t0]；
Step 5, judge step 2 in whether have from t0From moment and the prediction residual at least the first six moment that is connected goes out Now persistently rise or continuous decrease, if it is, in length of time series n, there is abnormal data, execution step six, if It is no, then to t0The data execution step two at+1 moment；
The time range that step 6, abnormal data are present is [t0-h,t0], wherein h>=6；
Step 7, step 4 and step 6 are obtained the time range that abnormal data is present take union, determine data exception The time range that fragment is present is [t0-n+1,t0]∪[t0-h,t0], judge whether that all detection terminates, if it is, performing step Rapid eight, if it is not, then to t0The data execution step two at+1 moment；
Step 8, end.
Beneficial effects of the present invention are：The present invention is by carrying out entirety with the time serieses fragment in length of time series n Abnormal estimation, considers the overall abnormal conditions in this time, realizes the detection of abnormal fragment, and give exception The confidence probability that fragment occurs, realizes the short term patterns exception occurred in time serieses with this.It can be applicable to Aero-Space and flies Row device monitoring field.
Description of the drawings
Fig. 1 is a kind of sensor detection data exception fragment detection side based on LS-SVM described in specific embodiment one The flow chart of method；
Fig. 2 is the position view in abnormity point existence time sequence fragment in specific embodiment, wherein, ● represent abnormal Point, zero represents normal point.
Specific embodiment
Specific embodiment one：Present embodiment is illustrated referring to Fig. 1, it is a kind of based on LS-SVM's described in present embodiment Sensor detection data exception fragment detection method, it comprises the steps：
There is number m, and n in the confidence level c of detection needed for step one, setting, length of time series n and minimum abnormity point Setting with m meets the restriction of confidence level c, and n and m is positive integer；
Step 2, from t0From moment the data in length of time series n are carried out with an abnormality detection confidence probability for p's LS-SVM point abnormality detections, the number of the abnormity point of prediction residual and data in acquisition length of time series n；
Whether step 3, the number for judging abnormity point during length is for the time serieses of n in step 2 are at least m abnormal Point, i.e., | En(t0) | >=m, m are positive integer, | En(t0) | express time sequence fragment, if it is, the length of time series is n Time serieses in there is abnormal data, execution step four, if it is not, then to t0The data execution step two at+1 moment；
The time range that step 4, abnormal data are present is [t0-n+1,t0]；
Step 5, judge step 2 in whether have from t0From moment and the prediction residual at least the first six moment that is connected goes out Now persistently rise or continuous decrease, if it is, in length of time series n, there is abnormal data, execution step six, if It is no, then to t0The data execution step two at+1 moment；
The time range that step 6, abnormal data are present is [t0-h,t0], wherein h>=6；
Step 7, step 4 and step 6 are obtained the time range that abnormal data is present take union, determine data exception The time range that fragment is present is [t0-n+1,t0]∪[t0-h,t0], judge whether that all detection terminates, if it is, performing step Rapid eight, if it is not, then to t0The data execution step two at+1 moment；
Step 8, end.
In present embodiment, when persistently rising or continuous decrease occurs in residual error, then there is higher probability to occur in that abnormal feelings Condition.There is the probability of the situation for rising successively or declining successively in continuous h residual error：
H=7 is typically chosen, thenNow judge that the time serieses fragment in this time occurs abnormal Probability be at least 1-0.0004=99.96%.
Specific embodiment two：Present embodiment is to a kind of sensing based on LS-SVM described in specific embodiment one Device detection data exception fragment detection method is described further, in present embodiment, length of time series n in step one, most There is number m and confidence level c in little abnormal point, and triadic relation meets below equation：
P(|En(t0)|≥m)>C,
Wherein, length for n time serieses fragment in, at least occur m abnormity point probability P (| En(t0) | >=m) table Be shown as P (| En(t0) | >=m)=P (m)+P (m+1)+...+P (n), P (m) they are that length is individual different to occur m in the length of time series of n The probability of constant value, P (| En(t0) |) be that length is appearance in the time serieses fragment of n | En(t0) | the probability of individual abnormity point,This fragment is that the probability of abnormal fragment is at least 1-P, E (t0) =[O (t0-n+1),O(t0-n+2),…,O(t0)]T, O (t0) represent t0Whether the observation at moment is abnormity point, O (t0)=1 For abnormity point, O (t0)=0 is normal point, using En(t0) two norms as fragment abnormity point occur number, be expressed as：
In formula, i is positive integer.
Specific embodiment three, present embodiment is to a kind of sensing based on LS-SVM described in specific embodiment one Device detection data exception fragment detection method is described further, in present embodiment, from t in step 20To the time from moment Data in sequence length n carry out the LS-SVM point abnormality detections that an abnormality detection confidence probability is p, obtain length of time series The method of the number of the abnormity point of the data in n is：
Step 2 one：The data that training data is concentrated are carried out phase space reconfiguration by setting training dataset, obtain input to Amount and output vector；
Step 2 two：The input vector for obtaining is normalized with output vector using Z-zeros methods, by institute State input vector to be normalized in the range of [- 1,1] with output vector；
Step 2 three：The kernel function of selection LS-SVM algorithms, and the parameter of LS-SVM forecast models is set, according to step 2 Input vector after two normalizeds trains LS-SVM regression models with output vector, so as to obtain LS-SVM forecast models；
Step 2 four：Prediction output is obtained to forecast model input prediction vector, when subsequent time predictive value arrives, is removed One moment predictive value and the residual error of the prediction output valve at this moment, judge whether residual error exceeds the model of the normal forecast error of gained Enclose, if it is, this moment data markers is abnormal, if it is not, then this moment data markers is normal.
Specific embodiment four, present embodiment is to a kind of sensing based on LS-SVM described in specific embodiment three Device detection data exception fragment detection method is described further, in present embodiment,
The LS-SVM forecast models are：
Wherein, K (x, xi) for Radial basis kernel function；B is departure, αiIt is the array element of Lagrange multiplier α, y is The predictive value of the output of LS-SVM forecast models, observation x of the data for newly observingi∈Rn。
Specific embodiment：
In n=6, m=3, p (t0During)=1%, length be 6 fragment at least there is the probability of 3 exceptional values and be：
P(|E6(t0) | >=3)=P (3)+P (4)+...+P (6)=0.02%,
Therefore, once the abnormity point of 3 and the above is occurred in that in the fragment for being detected, then there is the general of exception in this fragment Rate 1-0.02%=99.8%.It is in Fig. 2 under such abnormal conditions, an example of fragment abnormality detection, wherein two lines Region representation institute detection time sequence fragment [0,10] between section, the probability for having 99.98% occur in that exception.
1st, the emulation data set proposed using Ma et al., quoting this data set carries out the experimental verification of fragment abnormality detection.
When being not added with abnormal, sequence x0It is expressed as：
Wherein, N=1200, n0For the Gaussian noise that average is 0, standard deviation 0.1.
Add abnormal fragment e in former sequence1Obtain sequence x1, add abnormal fragment e in former sequence1With e2Obtain sequence Row x2。e1It is 0 for average, standard deviation is 0.5 Gaussian noise, is added at 600～620；
e2ForIt is added at 820～870.
Using Ma simulation sequences, abnormal fragment addition scope is [600,620] ∪ [820,870].Confirmatory experiment is flat in PC Complete under platform, the experiment simulation environment for adopting is for Matlab2013a.Using first 400 points in data set 2 as training set, afterwards 800 Point is used as test set.Input vector takes front Embedded dimensions 20 when constructing, using RBF kernel functions.Using tool kit LS-SVM1.8 certainly Band optimizing function select hyper parameter, LS-SVM parameter optimizations result be C=1.57532, σ=10.3102.
In order to verify impact of the exceptional value replacement policy to abnormality detection result, we are adopted respectively to the abnormity point for detecting With do not process and with predictive value replace exceptional value mode tested.Hereinafter referred to as without replacement policy and having replacement plan Slightly.
(1) without replacement policy
After detecting abnormity point, when next step prediction is carried out, do not use predictive value to replace and abnormal actual value occur.
Strategy 1：
Sequential value of the actual value with predictive value residual error beyond error threshold has 61, that is, detect 61 abnormity points, sends out The raw time is as follows：
T=471,500,570,601,605,606,609,610,613,614,616,617,618,619,620,622, 623,624,649,650,715,725,811,820,821,822,823,824,825,826,827,828,829,830,831, 832,833,834,845,846,847,849,850,851,852,853,854,855,856,857,858,859,860,861, 862,863,864,865,934,1051,1056
In length is 6 time serieses fragments, 3 and more than 3 abnormity points, abnormal fragment testing result are occurred in that：t =[601,627] ∪ [820,837] ∪ [845,868]
Fragment abnormal patterns 2 are the occurrence of residual values appearance persistently rises for more than 7 time：
T=837,838,839,840,841,842,843,844,845,846,847
Comprehensive both the above abnormal conditions, the abnormal segment ranges for finally detecting are：T=[601,627] ∪ [820, 868]。
Can correspond to very well with real abnormal ranges [600,620] ∪ [820,870], show that this method can compare into The time for detecting abnormal fragment generation of work(.
The aberrant continuation length of testing result has more more than real exception segment ranges.There is such case in analysis Reason mainly has two kinds.The first is possibly due to the input vector after constructing using exceptional value, affects forecasting accuracy. Exceptional value replacement policy may be adopted for this kind of, the exceptional value for detecting is replaced with predictive value, is used again afterwards With the construction of new input vector.The false drop rate of the detection algorithm of second possibly used abnormity point is higher, will just Constant value has been judged into exceptional value.Then need necessarily to optimize an Outlier Detection Algorithm for this kind of situation.
After detecting abnormity point, when next step prediction is carried out, predictive value is not made to replace actual value.
Strategy 2：
Sequential value of the actual value with predictive value residual error beyond error threshold has 41, that is, detect 41 abnormity points, sends out The raw time is as follows：
T=500,570,601,605,606,609,610,613,614,616,617,618,620,649,725,811, 820,822,824,825,826,827,832,833,850,853,854,855,856,857,858,860,861,862,864, 934
In length is 6 time serieses fragments, 3 and more than 3 abnormity points, abnormal fragment testing result are occurred in that：t =[601,622] ∪ [820,837] ∪ [846,868].
The occurrence of residual values appearance persistently rises for more than 7 time：
T=837,838,839,840,841,842,843,844,845,846,847
As can be seen that within t=[837, the 847] times, actual value occurs persistently rising situation with predictive value residual error, judges This time occurs in that fragment exception.
Comprehensive both the above abnormal conditions, the abnormal segment ranges for finally detecting are：T=[601,622] ∪ [820, 868]。
Very little is differed with real abnormal ranges [600,620] ∪ [820,870], almost accurately detection is different except assembling Normal time of occurrence and persistence length.
The experimental result that contrast is drawn using outlier detection strategy 1, the result drawn using strategy 2 are more accurate, say The Detection results of clear fragment method for detecting abnormality are affected by abnormal point detecting method, and abnormal point detecting method is more excellent, equally Under the conditions of fragment detection effect it is better.
(2) there is replacement policy
As the input vector of the Outlier Detection Algorithm predicted based on LS-SVM is constructed using history value, if history value There is exception to impact to predicting the outcome afterwards unavoidably, so that being forbidden occurs in testing result.Based on considerations above, we It is used further to construct input vector after the exceptional value predictive value for having detected is replaced.
Abnormal fragment detection based on LS-SVM is that strategy 1 is used when outlier detection is carried out,
There is the occurrence of persistently not rising for more than 7 time in residual values.
Sequential value of the actual value with predictive value residual error beyond error threshold has 54, that is, detect 54 abnormity points, sends out The raw time is as follows：
T=471,500,570,601,605,606,609,610,613,614,615,616,617,618,649,650, 715,725,811,820,821,822,823,824,825,826,827,828,829,830,831,832,833,834,845, 846,847,850,851,852,853,854,855,856,857,858,859,860,861,862,863,864,865,934
The testing result of fragment abnormal patterns 1：T=[601,621] ∪ [820,837] ∪ [845,868].
An abnormality detection strategy 2 is applied to into abnormal fragment detection, the exceptional value for replacing detecting using predictive value.
It is the strategy 2 used when outlier detection is carried out based on the abnormal fragment detection of LS-SVM,
There is the occurrence of persistently not rising for more than 7 time in residual values, i.e., second abnormal conditions do not occur.
Sequential value of the actual value with predictive value residual error beyond error threshold has 34, that is, detect 34 abnormity points, sends out The raw time is as follows：
T=601,605,606,609,610,613,614,616,617,618,811,820,822,824,825,826, 827,831,832,833,850,851,853,854,855,856,857,858,860,861,862,863,864,865
1 testing result of fragment abnormal patterns：T=[601,621] ∪ [820,836] ∪ [850,868].
Using the testing result after replacement policy compared with not replacing, the abnormal segment ranges for detecting are less, and true As a result compare the effect that degree of agreement do not replace good.
2nd, this carries out switch experiment for certain electromagnetic valve in coming from space shuttle disclosed in NASA (NASA) When status data.The former packet in each cycle contains 1000 sampled points, and which is normal and abnormal by NASA expert engineer Have been carried out normal and abnormal mark.In order to improve efficiency of algorithm, in experiment, carried out uniform resampling, each cycle only with 200 points representing.
It is that space shuttle Marotta sequences carry out abnormal fragment detection to truthful data collection 3.Using three in data set 3 Normal cycle is used as training set.Input vector takes front Embedded dimensions 20 when constructing, using RBF kernel functions.Using tool kit LS- SVM1.8 carries optimizing function and selects hyper parameter, and LS-SVM parameter optimizations result is C=1.57532, σ2=10.3102.With it is right The experimental design of Ma data sets is similar to, to truthful data collection 3, the exception fragment inspection of space shuttle Marotta values seasonal effect in time series Survey also respectively using being tested based on two kinds of difference abnormality detection strategies, and carry out exceptional value and replace and not replacement policy Contrasted.
(1) exceptional value replacement policy is not adopted
After detecting abnormity point, when next step prediction is carried out, do not use predictive value to replace and abnormal actual value occur.Point Method for detecting abnormality is using an abnormality detection strategy 1.
During length is 6 time serieses fragments, 3 and more than 3 abnormity points, abnormal fragment testing result are occurred in that：T= [257,280]∪[308,334]∪[507,525]∪[527,535]。
The occurrence of residual values appearance persistently rises for more than 7 time：T=[296,303] ∪ [520,527]；Residual error The occurrence of value appearance persistently rises for more than 7 time：T=[266,280] ∪ [305,316].
Comprehensive both the above abnormal conditions, the abnormal segment ranges for finally detecting are：T=[257,280] ∪ [296, 303]∪[305,334]∪[507,535]。
In length is 6 time serieses fragments, 3 and more than 3 abnormity points, abnormal fragment testing result are occurred in that：t =[257,279] ∪ [309,325] ∪ [507,524]
The occurrence of residual values appearance persistently rises for more than 7 time is identical with using strategy 1.Residual values occur 7 Below the occurrence of persistently the rising time：T=[296,303] ∪ [520,527]；Persistently rising occur more than 7 in residual values The occurrence of the time：T=[266,280] ∪ [305,316].
Comprehensive both the above abnormal conditions, the abnormal segment ranges for finally detecting are：T=[257,280] ∪ [296, 303]∪[305,325]∪[507,527]。
Compared with the testing result based on point of use abnormality detection strategy 1, scope slightly reduces, with true abnormal conditions More it coincide, substantially completely detects the position that all abnormal fragments occur.
(2) using exceptional value replacement policy replacement policy
After strategy is replaced, predicts the outcome and occur in that serious distortion, the result of abnormality detection loses meaning.Illustrate right In this data set, this kind of strategy is not simultaneously applied to.
In actual applications, actual value is replaced to have greater risk with predictive value, because the error of prediction can be accumulated In new value prediction afterwards, it is that predicted distortion further makes Outlier Detection Algorithm fail.
3rd, using truthful data collection 4, Heilungkiang Harbin City 4 days 1 month to 28 days 2 months January in 2010, totally 56 days, 8 weeks Traffic data is tested as detection object.
(1) without replacement policy
After detecting abnormity point, when next step prediction is carried out, do not use predictive value to replace and abnormal actual value occur.Point Method for detecting abnormality is using strategy 1.
In length is 6 time serieses fragments, 3 and more than 3 abnormity points, abnormal fragment testing result are occurred in that：t =[139,150] ∪ [152,173] ∪ [176,196] ∪ [207,214] ∪ [231,236] ∪ [498,503].
Above-mentioned testing result is transformed into actual dates finds the result for detecting as shown in table 3-1：
Table 3-1 abnormality detection result times correspondence table (is not replaced, strategy is 1)
Point method for detecting abnormality is using point abnormality detection strategy 2.
This detection do not find residual error continuously rise or it is continuous decline 7 and more than 7 fragment.
Sequential value of the actual value with predictive value residual error beyond error threshold has 60, that is, detect 60 abnormity points, sends out The raw time is as follows：
In length is 6 time serieses fragments, 3 and more than 3 abnormity points, abnormal fragment testing result are occurred in that：t =[139,150] ∪ [156,174] ∪ [176,197] ∪ [207,213].
Above-mentioned testing result is transformed into actual dates finds the result for detecting as shown in table 3-2：
Table 3-2 abnormality detection result times correspondence table (is not replaced, strategy is 2)
As Spring Festival holiday belongs to the national legal festivals and holidays, and this special red-letter day has more special to Chinese Meaning.As a result as can be seen that in morning on New Year's Eve to the lunar New Year's Day, true telephone traffic has been higher by much than predictive value, has occurred in Custom time for making a phone call mutually to pay a New Year call with compatriots time almost coincide.During festivals or holidays, telephone traffic will be significantly less than work Make day, the abnormal fragment for detecting is relatively reasonable.
The experimental result that contrast is drawn using outlier detection strategy 1, the result drawn using strategy 2 are more accurate, say The Detection results of clear fragment method for detecting abnormality are affected by abnormal point detecting method, and abnormal point detecting method is more excellent, equally Under the conditions of fragment detection effect it is better.The experimental result of this chapter also demonstrate again and outlier detection is proposed in chapter 2 The superiority of tactful 2 algorithms.
(2) there is replacement policy
As the input vector of the Outlier Detection Algorithm predicted based on LS-SVM is constructed using history value, if history value There is exception to impact to predicting the outcome afterwards unavoidably, so that being forbidden occurs in testing result.Based on considerations above, we It is used further to construct input vector after the exceptional value predictive value for having detected is replaced.
Abnormal fragment detection based on LS-SVM is the point of use abnormality detection strategy 1 when outlier detection is carried out.
In length is 6 time serieses fragments, 3 and more than 3 abnormity points, abnormal fragment testing result are occurred in that：t =[58,63] ∪ [82,87] ∪ [107,119] ∪ [129,134] ∪ [138,149] ∪ [152,168] ∪ [176,192] ∪ [200,216]∪[224,240]∪[249,264]∪[275,280]∪[499,506]。
The occurrence of residual values appearance persistently rises for more than 7 time：T=[113,121] ∪ [145,152].
The testing result of comprehensive both the above abnormal conditions, the abnormal segment ranges for finally detecting are：T=[58,63] ∪[82,87]∪[107,121]∪[129,134]∪[138,168]∪[176,192]∪[200,216]∪[224,240]∪ [249,264]∪[275,280]∪[499,506].The corresponding date and time of final testing result is shown in Table 3-2.
Table 3-2 abnormality detection result times correspondence table (is replaced, strategy is 1)
Table 3-2 (continued)
According to experimental result as can be seen that testing result expanded range many after replacing, New Year's Eve is not only included at the beginning of Six totally seven day legal festivals and holidays, the Lantern Festival is also had more.
An abnormality detection strategy 2 is applied to into abnormal fragment detection,
In length is 6 time serieses fragments, 3 and more than 3 abnormity points, abnormal fragment testing result are occurred in that：t =[58,63] ∪ [82,87] ∪ [136,150] ∪ [152,168] ∪ [176,192] ∪ [201,216] ∪ [224,240] ∪ [250,260]∪[498,506]。
The occurrence of residual values appearance persistently rises for more than 7 time：T=[145,154].
The testing result of comprehensive both the above abnormal conditions, the abnormal segment ranges for finally detecting are：T=[58,63] ∪ [82,87] ∪ [136,168] ∪ [176,192] ∪ [201,216] ∪ [224,240] ∪ [250,260] ∪ [498,506]
The corresponding date and time of final testing result is shown in Table 3-3.
Table 3-3 abnormality detection result times correspondence table (is not replaced, strategy is 2)
Replaced after the exceptional value for detecting using predictive value, two kinds of tactful abnormal fragments for detecting become many.Make Replacement policy scope is taken to have more the fifth day of a lunar month, the sixth day of lunar month with strategy 1, more than 2 detection of strategy is except the forth day of a lunar month the fifth day of a lunar month.According to general knowledge, the Spring Festival Legal festivals and holidays period are New Year's Eve to the sixth day of lunar month, and finding may be more accurate using testing result after replacement policy.
When the proposed method for detecting abnormality of description of test that three above data set is carried out preferably can be detected Between abnormal fragment in sequence appearance.There is risk in the use of exceptional value replacement policy, in some application scenarios, using replacement Detection results can be improved, but makes whole algorithm failure sometimes.The Hazard ratio that replacement policy brings plays effect promoting degree It is big many, can not use easily in actual applications.

Claims (4)

1. a kind of sensor detection data exception fragment detection method based on LS-SVM, it is characterised in that it includes following step Suddenly：
There is number m, and n and m in the confidence level c of detection needed for step one, setting, length of time series n and minimum abnormity point Setting meet the restriction of confidence level c, n and m is positive integer；
Step 2, from t0From moment the data in length of time series n are carried out with the LS-SVM that an abnormality detection confidence probability is p Point abnormality detection, the number of the abnormity point of prediction residual and data in acquisition length of time series n；
Whether step 3, the number for judging abnormity point during length is for the time serieses of n in step 2 are at least m abnormity point, i.e., | En(t0) | >=m, m are positive integer, | En(t0) | the number of abnormity point in the time serieses fragment that length is n is represented, if it is, The length of time series for n time serieses in there is abnormal data, execution step four, if it is not, then to t0The number at+1 moment According to execution step two；
The time range that step 4, abnormal data are present is [t0-n+1,t0]；
Step 5, judge step 2 in whether have from t0From moment and holding occurs in the prediction residual at least the first six moment that is connected It is continuous to rise or continuous decrease, if it is, in length of time series n, there is abnormal data, execution step six, if it is not, then To t0The data execution step two at+1 moment；
The time range that step 6, abnormal data are present is [t0-h,t0], wherein h>=6；
Step 7, step 4 and step 6 are obtained the time range that abnormal data is present take union, determine data exception fragment The time range of presence is [t0-n+1,t0]∪[t0-h,t0], judge whether that all detection terminates, if it is, execution step eight, If it is not, then to t0The data execution step two at+1 moment；
Step 8, end.
2. the sensor detection data exception fragment detection method based on LS-SVM according to claim 1, its feature exist In in step one, number m and confidence level c occur in length of time series n, minimum abnormity point, and triadic relation meets below equation：
P(|En(t0)|≥m)>C,
Wherein, length for n time serieses fragment in, at least occur m abnormity point probability P (| En(t0) | >=m) it is expressed as P(|En(t0) | >=m)=P (m)+P (m+1)+...+P (n), P (m) be length for n length of time series in there is m exceptional value Probability, P (| En(t0) |) be that length is appearance in the time serieses fragment of n | En(t0) | the probability of individual abnormity point,This fragment is that the probability of abnormal fragment is at least 1-P, E (t0)=[O (t0-n+1),O(t0-n+2),…,O(t0)]T, O (t0) represent t0Whether the observation at moment is abnormity point, O (t0)=1 is different Chang Dian, O (t0)=0 is normal point, using En(t0) two norms as fragment abnormity point occur number, be expressed as：
| E n ( t 0 ) | = Σ i = 0 n - 1 O ( t 0 - i )
In formula, i is positive integer.
3. the sensor detection data exception fragment detection method based on LS-SVM according to claim 1, its feature exist In from t in step 20From moment the data in length of time series n are carried out with the LS-SVM that an abnormality detection confidence probability is p Abnormality detection is put, the method for obtaining the number of the abnormity point of the data in length of time series n is：
Step 2 one：The observation data in length of time series n are obtained, and phase space reconfiguration are carried out to observing data, are input into Vector and output vector；
Step 2 two：N is normalized with output vector to input vector using Z-zeros methods, by the input Vector is normalized in the range of [- 1,1] with output vector, chooses front N to input vector and output vector as training data, its It is remaining as test data, the N is positive integer；
Step 2 three：The kernel function of selection LS-SVM algorithms, and the parameter of LS-SVM forecast models is set, returned according to step 2 two Input vector after one change is processed trains LS-SVM regression models with output vector, so as to obtain LS-SVM forecast models；
Step 2 four：Prediction output is obtained to forecast model input prediction vector, when subsequent time predictive value arrives, is removed for the moment The residual error of predictive value and the prediction output valve at this moment is carved, judges residual error whether beyond the institute under an abnormality detection confidence probability p The scope of the normal forecast error for obtaining, if it is, this moment data markers is abnormal, if it is not, then this moment data markers For normal.
4. the sensor detection data exception fragment detection method based on LS-SVM according to claim 3, its feature exist In,
The LS-SVM forecast models are：
y = Σ i = 1 N α i K ( x , x i ) + b
Wherein, K (x, xi) for Radial basis kernel function；B is departure, αiIt is the array element of Lagrange multiplier α, y is LS-SVM The predictive value of the output of forecast model, observation x of the data for newly observingi∈Rn。
CN201410422006.8A 2014-08-25 2014-08-25 LS-SVM-based method for detecting anomaly slot of sensor detection data Active CN104156473B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN201410422006.8A CN104156473B (en) 2014-08-25 2014-08-25 LS-SVM-based method for detecting anomaly slot of sensor detection data

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN201410422006.8A CN104156473B (en) 2014-08-25 2014-08-25 LS-SVM-based method for detecting anomaly slot of sensor detection data

Publications (2)

Publication Number Publication Date
CN104156473A CN104156473A (en) 2014-11-19
CN104156473B true CN104156473B (en) 2017-05-03

Family

ID=51881971

Family Applications (1)

Application Number Title Priority Date Filing Date
CN201410422006.8A Active CN104156473B (en) 2014-08-25 2014-08-25 LS-SVM-based method for detecting anomaly slot of sensor detection data

Country Status (1)

Country Link
CN (1) CN104156473B (en)

Families Citing this family (10)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN106125643A (en) * 2016-06-22 2016-11-16 华东师范大学 A kind of industry control safety protection method based on machine learning techniques
CN107092582B (en) * 2017-03-31 2021-04-27 江苏方天电力技术有限公司 Online abnormal value detection and confidence evaluation method based on residual posterior
CN108801322A (en) * 2018-07-06 2018-11-13 哈尔滨工业大学 Reliability estimation method for UAV Flight Control System MEMS sensor
CN109408772A (en) * 2018-10-11 2019-03-01 四川长虹电器股份有限公司 To the restoration methods of the abnormal data in continuity data
CN109697207B (en) * 2018-12-25 2020-08-28 苏州思必驰信息科技有限公司 Method and system for monitoring abnormity of time sequence data
CN110071913B (en) * 2019-03-26 2020-10-02 同济大学 Unsupervised learning-based time series anomaly detection method
CN111612082B (en) * 2020-05-26 2023-06-23 河北小企鹅医疗科技有限公司 Method and device for detecting abnormal subsequence in time sequence
CN111783356B (en) * 2020-06-29 2024-03-29 清华大学深圳国际研究生院 Oil yield prediction method and device based on artificial intelligence
CN111967508A (en) * 2020-07-31 2020-11-20 复旦大学 Time series abnormal point detection method based on saliency map
CN113051825B (en) * 2021-03-29 2022-08-05 中国人民解放军92859部队 Method for constructing dynamic LS-SVM (least squares-support vector machine) of submarine trend surface

Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US7853072B2 (en) * 2006-07-20 2010-12-14 Sarnoff Corporation System and method for detecting still objects in images
CN102288398A (en) * 2011-05-16 2011-12-21 南京航空航天大学 Momentum wheel fault detection device and method based on support vector machine

Patent Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US7853072B2 (en) * 2006-07-20 2010-12-14 Sarnoff Corporation System and method for detecting still objects in images
CN102288398A (en) * 2011-05-16 2011-12-21 南京航空航天大学 Momentum wheel fault detection device and method based on support vector machine

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
移动通信话务量多步预测的LS_SVM方法研究;王少军;《仪器仪表学报》;20110630;第32卷(第6期);第1258-1264页 *

Also Published As

Publication number Publication date
CN104156473A (en) 2014-11-19

Similar Documents

Publication Publication Date Title
CN104156473B (en) LS-SVM-based method for detecting anomaly slot of sensor detection data
Wang et al. Burst detection in district metering areas using deep learning method
US9441988B2 (en) System and method for identifying likely geographical locations of anomalies in a water utility network
US8988236B2 (en) System and method for failure prediction for rod pump artificial lift systems
CN107533560A (en) For inferring the data scrubbing system and method for material composition
CN104156615A (en) Sensor test data point anomaly detection method based on LS-SVM
AU2016287383A1 (en) Method for detecting anomalies in a water distribution system
CN107430706A (en) High-level data clears up system and method
CN112083244B (en) Integrated intelligent diagnosis system for faults of avionic equipment
CN102789545A (en) Method for predicating remaining life of turbine engine based on degradation model matching
Kakouei et al. Quantitative hydrological preferences of benthic stream invertebrates in Germany
CN103345593A (en) Gathering abnormity detection method for single sensor data flow
CN110348150A (en) A kind of fault detection method based on dependent probability model
Giordano et al. The value of monitoring a structural health monitoring system
CN114925536A (en) Airborne system PHM testability modeling and diagnosis strategy optimization method and device
Tamilselvan et al. Optimization of wind turbines operation and maintenance using failure prognosis
CN114169254A (en) Abnormal energy consumption diagnosis method and system based on short-term building energy consumption prediction model
KR20140137210A (en) computing system for well placement optimization developed by SA/ANN and well placement optimization method using Thereof
CN101382474A (en) Multi-purpose evaluating method for bridge structure safety
RU2011143385A (en) METHOD AND SYSTEM FOR ASSESSING DATA OF SENSORS FROM INSTALLATION FOR REPAIR OF WELLS
Li et al. Meteorological radar fault diagnosis based on deep learning
CN103337000B (en) A kind of safe monitoring and pre-alarming method of oil-gas gathering and transportation system
CN116189802A (en) Transformer fault early warning method based on gas concentration time sequence data
EP2956751B1 (en) Method and monitoring device for monitoring a structure
Wang et al. Prognosis-informed wind farm operation and maintenance for concurrent economic and environmental benefits

Legal Events

Date Code Title Description
C06 Publication
PB01 Publication
C10 Entry into substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant