CN105653129A - Classic algorithm based real-time signal discrimination and correction method - Google Patents

Classic algorithm based real-time signal discrimination and correction method Download PDF

Info

Publication number
CN105653129A
CN105653129A CN201511010563.XA CN201511010563A CN105653129A CN 105653129 A CN105653129 A CN 105653129A CN 201511010563 A CN201511010563 A CN 201511010563A CN 105653129 A CN105653129 A CN 105653129A
Authority
CN
China
Prior art keywords
data
window
abnormal
flow process
judges
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN201511010563.XA
Other languages
Chinese (zh)
Inventor
许凤琴
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
JIANGSU FEISHANG SAFETY MONITORING CONSULTING Co Ltd
Original Assignee
JIANGSU FEISHANG SAFETY MONITORING CONSULTING Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by JIANGSU FEISHANG SAFETY MONITORING CONSULTING Co Ltd filed Critical JIANGSU FEISHANG SAFETY MONITORING CONSULTING Co Ltd
Priority to CN201511010563.XA priority Critical patent/CN105653129A/en
Publication of CN105653129A publication Critical patent/CN105653129A/en
Pending legal-status Critical Current

Links

Classifications

    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/048Interaction techniques based on graphical user interfaces [GUI]
    • G06F3/0481Interaction techniques based on graphical user interfaces [GUI] based on specific properties of the displayed interaction object or a metaphor-based environment, e.g. interaction with desktop elements like windows or icons, or assisted by a cursor's changing behaviour or appearance

Landscapes

  • Engineering & Computer Science (AREA)
  • General Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Human Computer Interaction (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Complex Calculations (AREA)

Abstract

The present invention relates to a classic algorithm based real-time signal discrimination and correction method, and in particular, to a method for performing data discrimination and correction based on different scenarios and performing abnormal data determination by selecting a different algorithm according to the different scenarios. The method comprises the following steps: (1) providing a constrained man-machine interaction interface; (2) calling different algorithms in need according to feedback information to process data; (3) determining whether a requirement is met; (4) if the requirement is not met, enabling a system to automatically correct a parameter and returning to (2), or otherwise proceeding to (5); and (5) returning final configuration information and a result of data processing on the interactive interface.

Description

A kind of method that live signal based on classic algorithm is screened and revised
Technical field
The present invention relates to a kind of method that live signal based on classic algorithm is screened and revised, particularly relate to one and carry out data examination and correction based on different scenes, carry out the judgement of abnormal data according to the algorithm that the different choice of scene is different.
Background technology
Along with the arrival of big data age, data are increasingly used frequently in daily life and practical application. Along with the popularization of works safety monitoring, due to the interference of presence sensor self unstability and external environment, outwardness error in data acquisition. A job that is very meaningful and that challenge is become for improving the accuracy gathering data and reliability, and only use a kind of simple rejecting abnormalities data of algorithm can not fully meet practical application under different scenes, so under different scene, abnormal data examination and correction become key technology therein. Document 1 is for the invention provides strong theory support. For the result that display data vivid, effective is screened and revised, man-machine interaction just seems essential.
Summary of the invention
The present invention is in order to overcome drawbacks described above, a kind of method that purpose is in that to provide live signal based on classic algorithm to screen and revise, utilize man-machine interaction to allow user participate in the processing procedure of data, according to the application scenarios of algorithms of different, configurableization interface is carried out certain constraint;
Abnormal data according to some classics screens algorithm and weighted average obtains one group of normal data source, judges the abnormal conditions of follow-up data stream being identified and revising abnormal data based on this group normal data;
The parameter of configuration is revised by thought automatically that utilize iteration, makes the result that data process reach requirement.
The present invention to achieve these goals, adopts the following technical scheme that
A kind of method that live signal based on classic algorithm is screened and revised, the method comprises the steps:
(1) interface of constrained man-machine interaction is provided;
(2) different algorithm process data are called according to feedback information is on-demand;
(3) judge whether to reach requirement;
(4) without reaching automatic corrected parameter return (2) of requirement system, (5) are otherwise arrived; (5) return on interactive interface
Return final configuration information and the result that data process;
Data stream is exactly data acquisition system sequentially in time, and human-computer interaction interface provides the parameter information that abnormal data is screened, the result requiring information display data process simultaneously:
The size (positive integer: [5,100]) of n---window;
A---risk (value: 0.1,0.05,0.01);
P---quantile (span: [0,1]);
Request---requires information (coefficient of variation scope cv [c1, c2]);
According to the parameter information provided, require information, carry out the filtration of abnormal data; If the abnormal result filtered reaches requirement, then showing the result that final configuration information and data process on interactive interface, otherwise automatically being revised by system provides parameter information again, hence into the calculating of a new round;
Specifically comprise the following steps that
By step 1, step 2 obtains the individual data data gathered, step 3 judges data amount check WS in window>=n, step 4 is in the time execution of WS<n, step 5 judges whether one group of normal data, step 6 performs when not having one group of normal data, construct one group of normal data, perform when step 7 has one group of normal data, step 8 judges the exceptional value mark of data, step 9 performs when the exceptional value of data is designated true, and step 10 performs when the exceptional value of data is designated false or after step 8, until step 11 process ends.
Further,
Construct one group of normal data flow process: step 12 is beginning action, step 13 obtains the individual data data gathered, step 14 data are screened, step 15 judges that in window, whether the number Count1 of abnormal data is equal to 0, step 16 is at Count1 > 0 time perform, it is incorporated into inside available data sources and abnormal is designated false or correction is designated the data of true, step 17 utilizes weighted average to process abnormal data, step 18 judges in window, whether uncorrected abnormal data number Count2 is 0, step 19 performs when Count1=0 or Count2=0, data in output window, construct one group of normal data flow process to step 20 to terminate.
Further,
On-demand selection algorithm flow process: step 21 is beginning action, step 22 judges the relation between n and n1, n2, step 23 is at n>n2 time perform, the relation that judges between n and n2, n3, step 24 performs when n1��n��n2, utilizes Grubbs test method, step 25<performs during n��n3 at n2, utilizing sample fractiles algorithm, step 26 is at n>n3 time perform, utilize Rhein to reach criterion, the result that step 27 output judges, terminates to this flow process of step 28.
Further,
Grubbs test method flow process: step 29 is beginning action, step 30 inputs data, the arithmetical average U of data, residual error R, standard deviation sigma in step 31 calculation window, step 32 calculates T value, step 33 finds the T (n of correspondence according to risk a, window number n, a) corresponding boundary value, step 34 compares T and T (n, a) relation judges the abnormal conditions of input data, step 35 performs when data are abnormal, it is identified to corresponding abnormal data, terminates this flow process to step 36.
Further,
Sample fractiles algorithm flow: step 37 is beginning action, step 38 inputs data, step 39 constructs statistic of test S1, Sn, step 40 calculates S1, Sn according to the statistic formula of quantile p and structure, step 41 finds the S (n of correspondence according to risk a, window number n, a) corresponding boundary value, step 42 compares S1, Sn and S (n, a) relation judges the abnormal conditions of input data, step 43 performs when data are abnormal, it is identified to corresponding abnormal data, terminates this flow process to step 44.
Further,
Rhein reaches criterion flow process: from step 45, step 46 inputs data, step 47 calculates the arithmetical average U of data, step 48 calculates maximum absolute value residual error R, standard deviation sigma, step 49 judges the abnormal conditions of input data, step 50 performs when data are abnormal, is identified to corresponding abnormal data, terminates this flow process to step 51.
Further,
Weighted average flow process: from step 52, step 53 generates the data between N number of (0,1) immediately, and step 54 is ranked up facing to N number of data, and step 55 calculates weighted mean Mean, and step 56 exports Mean, terminates to this flow process of step 57.
Further,
Filled window flow: 58 start in steps, and step 59 judges whether window exists, and step 60 window is absent from being set up window, and step 61 is directly filled into inside window data, and step 60 terminates once to fill the flow process of window.
Further,
Sliding window flow: step 63 is beginning action, step 64 is filled into data in window, and step 65 removes first data in window, terminates the flow process of a window sliding to step 66;
Window data structure: comprise four groups of data inside window, first group is initial data, and second group is value after the verification that initial data is corresponding, and the 3rd group is whether bool value mark data are abnormal, and the 4th group is that bool value identifies whether data have modified. The meaning of window here is just by the container of data storage.
Further,
Judge whether data result reaches the flow process of requirement: from step 68, step 69 calculates the coefficient of variation CV of data, and step 70 compares the relation of CV and c1, c2, and step 73 performs when c1��CV��c2, step 74 performs when CV<c1 or CV>c2, terminates this flow process to step 73.
Beneficial effects of the present invention:
The present invention utilizes man-machine interaction to allow user participate in the processing procedure of data, according to the application scenarios of algorithms of different, configurableization interface has been carried out certain constraint; Abnormal data according to some classics screens algorithm and weighted average obtains one group of normal data source, judges the abnormal conditions of follow-up data stream being identified and revising abnormal data based on this group normal data; The parameter of configuration is revised by thought automatically that utilize iteration, makes the result that data process reach requirement.
Accompanying drawing explanation
Fig. 1 is that data of the present invention are screened and correcting device workflow diagram;
Fig. 2 be the present invention mechanism flow chart based on classic algorithm exceptional value judge and revise flow chart;
Fig. 3 is that the present invention constructs one group of normal data flow process figure;
Fig. 4 is the on-demand selection algorithm flow chart of the present invention;
Fig. 5 is Grubbs test method flow chart of the present invention;
Fig. 6 is sample fractiles algorithm flow chart of the present invention;
Fig. 7 is that Rhein of the present invention reaches criterion flow chart;
Fig. 8 is weighted average flow chart of the present invention;
Fig. 9 is that the present invention fills window flow figure;
Figure 10 is sliding window flow figure of the present invention;
Figure 11 is window data structure chart of the present invention;
Figure 12 is that the present invention judges whether data result reaches the flow chart of requirement.
Detailed description of the invention
Describe the present invention below in conjunction with accompanying drawing:
Fig. 1 is that data are screened and correcting device workflow diagram. Data stream is exactly data acquisition system sequentially in time, and human-computer interaction interface provides the parameter information that abnormal data is screened, the result requiring information display data process simultaneously:
The size (positive integer: [5,100]) of n---window;
A---risk (value: 0.1,0.05,0.01);
P---quantile (span: [0,1]);
Request---requires information (coefficient of variation scope cv [c1, c2]);
According to the parameter information provided, require information, carry out the filtration of abnormal data. If the abnormal result filtered reaches requirement, then showing the result that final configuration information and data process on interactive interface, otherwise automatically being revised by system provides parameter information again, hence into the calculating of a new round.The detailed process of step 67 is shown in the explanation of Figure 12. Special circumstances, if automatically corrected parameter has all tried all of parameter situation or can not reach requirement, export process data failure on request and please reselect and require information.
Fig. 2 be the present invention mechanism flow chart based on classic algorithm exceptional value judge and revise flow chart. by step 1, step 2 obtains the individual data data gathered, step 3 judges data amount check WS in window>=n, step 4 is in the time execution of WS<n, step 5 judges whether one group of normal data, step 6 performs when not having one group of normal data, the detailed process constructing one group of normal data is shown in the explanation of Fig. 3, perform when step 7 has one group of normal data, the detailed process that data are screened is shown in the explanation of Fig. 4, step 8 judges the exceptional value mark of data, step 9 performs concrete formula when the exceptional value of data is designated true and sees formula (1), step 10 performs when the exceptional value of data is designated false or after step 8, the detailed process of sliding window is shown in the explanation of Figure 10, until step 11 process ends.
Fig. 3 is one group of normal data flow process figure of structure. step 12 is beginning action, step 13 obtains the individual data data gathered, step 14 data are screened, detailed process is shown in the explanation of Fig. 4, step 15 judges that in window, whether the number Count1 of abnormal data is equal to 0, step 16 is at Count1 > 0 time perform, it is incorporated into inside available data sources and abnormal is designated false or correction is designated the data of true, step 17 utilizes weighted average to process abnormal data, concrete formula is shown in formula (8), step 18 judges in window, whether uncorrected abnormal data number Count2 is 0, step 19 performs when Count1=0 or Count2=0, data in output window, construct one group of normal data flow process to step 20 to terminate.
Fig. 4 is on-demand selection algorithm flow chart. Step 21 is beginning action, step 22 judges the relation between n and n1, n2, step 23 is at n>n2 time perform, the relation that judges between n and n2, n3, at n1��n,<during n2, execution, utilizes the detailed process of Grubbs test method to see Fig. 5 to step 24, step 25<performs during n��n3 at n2, the detailed process utilizing sample fractiles algorithm is shown in Fig. 6, and step 26 is at n>n3 time perform, utilize Rhein to reach criterion detailed process and see Fig. 7, the result that step 27 output judges, terminates to this flow process of step 28. (note: n1=5, n2=20, n3=50 here)
Fig. 5 is Grubbs test method flow chart. step 29 is beginning action, step 30 inputs data, the arithmetical average U (see formula (2)) of data in step 31 calculation window, residual error R (see formula (3)), standard deviation sigma (see formula (4)), step 32 calculates T value, see formula (5), step 33 is according to risk a, window number n finds the T (n of correspondence, a) corresponding boundary value, step 34 compares T and T (n, a) relation judges the abnormal conditions of input data, step 35 performs when data are abnormal, it is identified to corresponding abnormal data, this flow process is terminated to step 36.
Fig. 6 is sample fractiles algorithm flow chart. Step 37 is beginning action, step 38 inputs data, step 39 constructs statistic of test S1 (see formula (6)), Sn (see formula (7)), step 40 calculates S1, Sn according to the statistic formula of quantile p and structure, step 41 finds the S (n of correspondence according to risk a, window number n, a) corresponding boundary value, step 42 compares S1, Sn and S (n, a) relation judges the abnormal conditions of input data, step 43 performs when data are abnormal, it is identified to corresponding abnormal data, terminates this flow process to step 44.
Illustrate this algorithm:
If data source is x1, x2����xn;
Calculate average U = &Sigma; i = 0 n x i n ;
Above-mentioned data source is asked residual error and Vi=xi-U, i=1,2 ... n, order carries out arrangement and V by size simultaneously(1)��V(2)�ܡ���V(n);
Step 39 constructs statistic
S 1 = 1 2 { V ( n 3 ) + V ( n 4 ) } - V ( 1 ) V ( n 3 ) - V ( n 4 ) - - - ( 6 )
S n = V ( n ) + 1 2 { V ( n 3 ) + V ( n 4 ) } V ( n 3 ) - V ( n 4 ) - - - ( 7 )
Here:P is quantile, n4=n-n3+ 1;
Step 40, according to p value, calculates S1, Sn, according to the adnexa of n, a list of references 1 find S (n, a);
If step 41 S1 > S, then V is described(1)It is the exceptional value i.e. former data of correspondence it is assumed herein that be xlAbnormal data, in like manner can be determined that V(n)Whether abnormal;
Remove in data source abnormal data, repeat 2), 3), 4), 5), 6) until there is no abnormal data.
Fig. 7 is that Rhein reaches criterion flow chart. From step 45, step 46 inputs data, step 47 calculates the arithmetical average U (see formula (2)) of data, step 48 calculates maximum absolute value residual error R (see formula (8)), standard deviation sigma (see formula (9)), step 49 judges the abnormal conditions of input data, step 50 performs when data are abnormal, is identified to corresponding abnormal data, terminates this flow process to step 51.
Illustrate this algorithm:
If data source is x1, x2����xn;
Step 47 calculates average
Above-mentioned data source is asked residual error and V by step 48i=xi-U, i=1,2 ... n, it is thus achieved that maximum absolute value residual error R=max (| Vi|, i=1,2 ... n); (8)
&sigma; = &Sigma; i = 1 n ( x i - U ) 2 n - 1 - - - ( 9 )
If step 49 R > 3 is ��, then illustrate that R is the exceptional value i.e. former data of correspondence it is assumed herein that be xlFor abnormal data;
Remove in data source abnormal data, repeat 2), 3), 4) until there is no abnormal data.
Fig. 8 is weighted average flow chart. From step 52, step 53 generates the data between N number of (0,1) immediately, step 54 is ranked up facing to N number of data, step 55 calculates weighted mean Mean (see formula (1)), and step 56 exports Mean, terminates to this flow process of step 57
Fig. 9 is for filling window flow figure. 58 start in steps, and step 59 judges whether window exists, and step 60 window is absent from being set up window, and step 61 is directly filled into data inside window, and step 60 terminates once to fill the flow process of window.
Figure 10 is sliding window flow figure. Step 63 is beginning action, and step 64 is filled into data in window, and step 65 removes first data in window, terminates the flow process of a window sliding to step 66.
Figure 11 is window data structure chart. Four groups of data are comprised inside window, first group is initial data, second group is value after the verification that initial data is corresponding (value after namely filtering), and the 3rd group is whether bool value mark data are abnormal, and the 4th group is that bool value identifies whether data have modified. The meaning of window here is just by the container of data storage.
Figure 12 judges the flow chart whether data result reaches requirement. From step 68, step 69 calculates the coefficient of variation CV (see formula (11)) of data, and step 70 compares the relation of CV and c1, c2, and step 71 performs when c1��CV��c2, step 72 performs when CV<c1 or CV>c2, terminates this flow process to step 73.
The computing formula of CV is given below:
If data source is x1, x2����xn;
First step 69 calculates average U = &Sigma; i = 0 n x i n , Standard deviation &sigma; = &Sigma; i = 1 n ( x i - U ) 2 n
CV=��/| U | (11)
If U=0, then CV=��
Concrete example
Some above-mentioned steps and formula is resolved below by a concrete example:
Assume that configuration information is respectively as follows: n=m1, a=a1, p=p1, cv=[c1, c2];
Assume that in window, existing data have m1-1, i.e. WindowDatas={object1..., objectm1-1},
Here objectiForm be { xi, yi, false, false}, correspond to the initial data inside Figure 12, verification after data, exceptional value mark and revise mark, window less than time suppose yi=xi, i=1 ... m1-1;
Now to data x incoming in windowm1WindowSize=m1-1 < m1 needed to carry out step 4 and filled window this time, object objectm1={ xm1, ym1, fals, false} is filled into window, here ym1=xm1, then arrive step 5;
Now to data x incoming in windowm1+1At this moment need to carry out step 5 and judge that IsHasNormalDatas=true (so far has be carried out the abnormal normal data source screened and process but without one group, IsHasNormalDatas=false), enter step 6 and utilize the one group of normal data of data configuration also having in window;
Data source is: x1����xm1
Step 22 judges m1 and n1, relation between n2, n3, it is assumed here that n1��m1��n2
Step 31 calculates the arithmetical average U of data source, residual error R, standard deviation sigma:
U = &Sigma; i = 1 m 1 x i m 1 - - - ( 2 )
Calculate residual error R:Ri=xi-U, i=1,2 ... m1, (3)
Here R is the vector of 1 row m1 row and a R=[R1,������Rm1]
&sigma; = &Sigma; i = 1 m 1 R i 2 m 1 - 1 - - - ( 4 )
Step 32 to the data inside R according to being ranked up from small to large: R=[R1' ... Rm1' calculateNamely the T value (5) that calculated minimum is corresponding with maximum
The step 33 subordinate list according to a, n list of references 1, find correspondence T (m1, a);
If step 34Then find R1' corresponding data are assumed to xm, then x1It is the object that abnormal data is namely correspondingm={ xm, ym, true, false} judges equallyWith T (m1, relation a), ifOtherwise then corresponding data are normal is abnormal. Remove first time judge in abnormal data, repeat 3), 4), 5), 6) until can not find abnormal data.
It is assumed by x herein for facilitating description belowm, xl, m < l is abnormal data;
Step 15, by judging the exception mark=true of data objecti in window, obtains Count1=2;
Step 16 this be normal data be xi, i=1 ... m1 and i �� m, l, i.e. zi, i=1 ... m1-2
Step 17 first revises x according to the time sequencing of datamAt this moment stochastic generation m1-2 (0-1) weight dj, j=1 ... m1-2, to weight djCarry out sequence from big to small and obtain dj��. From zmTime more near weight is more big, then objectm={ xm, ym, true, true}.
At this time y m = &Sigma; i = 1 m 1 - 2 z i d i &prime; &Sigma; i = 1 m 1 - 2 d i &prime; ; - - - ( 1 )
Step 18 is Count2=1 at this time. XmRevised value ymIt is filled into inside normal data source at this moment xi, i=1 ... m1 and i �� l, continues step 17 and obtains xlRevised data
Data after step 19 output processing, at this moment IsHasNormalDatas=true;
Step 7 at this time have one group of normal data source y1����ym1
U, R, �� is calculated according to step 31;
If R1' < ym1+1-U < Rm1+1', then objectm1+1={ xm1+1, ym1+1, false, false}, ym1+1=xm1+1To step 10;
If step 34 R1'=ym1+1-U or ym1+1-U=Rm1+1', calculateJudgeWith T (m1+1, relation a), ifThen objectm1+1={ xm1+1, ym1+1, true, false} to step 9 otherwise objectm1+1={ xm1+1, ym1+1, false, false} is to step 10;
Step 9 stochastic generation m1 (0-1) weight dj, j=1 ... m1, to weight djCarry out sequence from big to small and obtain dj��.From xm1+1Time more near weight is more big, then objectm1+1={ xm1+1, ym1+1, true, true}. At this time
The step 10 order entering window according to data, removes object1, add objectm1+1;
Terminate to step 11.
Application scenarios
According to initiation parameter and target call, data are screened. If data are abnormal, utilize weighted average that abnormal data is processed;
If target call is configured without, then only according to initiation parameter, data are processed. If abnormal data then carries out reporting to the police processing simultaneously.

Claims (10)

1. the method that the live signal based on classic algorithm is screened and revised, it is characterised in that the method comprises the steps:
(1) interface of constrained man-machine interaction is provided;
(2) different algorithm process data are called according to feedback information is on-demand;
(3) judge whether to reach requirement;
(4) without reaching automatic corrected parameter return (2) of requirement system, (5) are otherwise arrived; (5) on interactive interface, final configuration information is returned, and the result that data process;
Data stream is exactly data acquisition system sequentially in time, and human-computer interaction interface provides the parameter information that abnormal data is screened, the result requiring information display data process simultaneously:
The size (positive integer: [5,100]) of n---window;
A---risk (value: 0.1,0.05,0.01);
P---quantile (span: [0,1]);
Request---requires information (coefficient of variation scope cv [c1, c2]);
According to the parameter information provided, require information, carry out the filtration of abnormal data; If the abnormal result filtered reaches requirement, then showing the result that final configuration information and data process on interactive interface, otherwise automatically being revised by system provides parameter information again, hence into the calculating of a new round;
Specifically comprise the following steps that
By step 1, step 2 obtains the individual data data gathered, step 3 judges data amount check WS in window>=n, step 4 is in the time execution of WS<n, step 5 judges whether one group of normal data, step 6 performs when not having one group of normal data, construct one group of normal data, perform when step 7 has one group of normal data, step 8 judges the exceptional value mark of data, step 9 performs when the exceptional value of data is designated true, and step 10 performs when the exceptional value of data is designated false or after step 8, until step 11 process ends.
2. the method that the live signal based on classic algorithm according to claim 1 is screened and revised, it is characterised in that:
Construct one group of normal data flow process: step 12 is beginning action, step 13 obtains the individual data data gathered, step 14 data are screened, step 15 judges that in window, whether the number Count1 of abnormal data is equal to 0, step 16 is at Count1 > 0 time perform, it is incorporated into inside available data sources and abnormal is designated false or correction is designated the data of true, step 17 utilizes weighted average to process abnormal data, step 18 judges in window, whether uncorrected abnormal data number Count2 is 0, step 19 performs when Count1=0 or Count2=0, data in output window, construct one group of normal data flow process to step 20 to terminate.
3. the method that the live signal based on classic algorithm according to claim 1 is screened and revised, it is characterised in that:
On-demand selection algorithm flow process: step 21 is beginning action, step 22 judges the relation between n and n1, n2, step 23 is at n>n2 time perform, the relation that judges between n and n2, n3, step 24 performs when n1��n��n2, utilizes Grubbs test method, step 25<performs during n��n3 at n2, utilizing sample fractiles algorithm, step 26 is at n>n3 time perform, utilize Rhein to reach criterion, the result that step 27 output judges, terminates to this flow process of step 28.
4. the method that the live signal based on classic algorithm according to claim 1 is screened and revised, it is characterised in that:
Grubbs test method flow process: step 29 is beginning action, step 30 inputs data, the arithmetical average U of data, residual error R, standard deviation sigma in step 31 calculation window, step 32 calculates T value, step 33 finds the T (n of correspondence according to risk a, window number n, a) corresponding boundary value, step 34 compares T and T (n, a) relation judges the abnormal conditions of input data, step 35 performs when data are abnormal, it is identified to corresponding abnormal data, terminates this flow process to step 36.
5. the method that the live signal based on classic algorithm according to claim 1 is screened and revised, it is characterised in that:
Sample fractiles algorithm flow: step 37 is beginning action, step 38 inputs data, step 39 constructs statistic of test S1, Sn, step 40 calculates S1, Sn according to the statistic formula of quantile p and structure, step 41 finds the S (n of correspondence according to risk a, window number n, a) corresponding boundary value, step 42 compares S1, Sn and S (n, a) relation judges the abnormal conditions of input data, step 43 performs when data are abnormal, it is identified to corresponding abnormal data, terminates this flow process to step 44.
6. the method that the live signal based on classic algorithm according to claim 1 is screened and revised, it is characterised in that:
Rhein reaches criterion flow process: from step 45, step 46 inputs data, step 47 calculates the arithmetical average U of data, step 48 calculates maximum absolute value residual error R, standard deviation sigma, step 49 judges the abnormal conditions of input data, step 50 performs when data are abnormal, is identified to corresponding abnormal data, terminates this flow process to step 51.
7. the method that the live signal based on classic algorithm according to claim 1 is screened and revised, it is characterised in that:
Weighted average flow process: from step 52, step 53 generates the data between N number of (0,1) immediately, and step 54 is ranked up facing to N number of data, and step 55 calculates weighted mean Mean, and step 56 exports Mean, terminates to this flow process of step 57.
8. the method that the live signal based on classic algorithm according to claim 1 is screened and revised, it is characterised in that:
Filled window flow: 58 start in steps, and step 59 judges whether window exists, and step 60 window is absent from being set up window, and step 61 is directly filled into inside window data, and step 60 terminates once to fill the flow process of window.
9. the method that the live signal based on classic algorithm according to claim 1 is screened and revised, it is characterised in that:
Sliding window flow: step 63 is beginning action, step 64 is filled into data in window, and step 65 removes first data in window, terminates the flow process of a window sliding to step 66;
Window data structure: comprise four groups of data inside window, first group is initial data, and second group is value after the verification that initial data is corresponding, and the 3rd group is whether bool value mark data are abnormal, and the 4th group is that bool value identifies whether data have modified; The meaning of window here is just by the container of data storage.
10. the method that the live signal based on classic algorithm according to claim 1 is screened and revised, it is characterised in that:
Judge whether data result reaches the flow process of requirement: from step 68, step 69 calculates the coefficient of variation CV of data, and step 70 compares the relation of CV and c1, c2, and step 71 performs when c1��CV��c2, step 72 performs when CV<c1 or CV>c2, terminates this flow process to step 73.
CN201511010563.XA 2015-12-29 2015-12-29 Classic algorithm based real-time signal discrimination and correction method Pending CN105653129A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN201511010563.XA CN105653129A (en) 2015-12-29 2015-12-29 Classic algorithm based real-time signal discrimination and correction method

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN201511010563.XA CN105653129A (en) 2015-12-29 2015-12-29 Classic algorithm based real-time signal discrimination and correction method

Publications (1)

Publication Number Publication Date
CN105653129A true CN105653129A (en) 2016-06-08

Family

ID=56478249

Family Applications (1)

Application Number Title Priority Date Filing Date
CN201511010563.XA Pending CN105653129A (en) 2015-12-29 2015-12-29 Classic algorithm based real-time signal discrimination and correction method

Country Status (1)

Country Link
CN (1) CN105653129A (en)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN106324583A (en) * 2016-08-25 2017-01-11 哈尔滨工程大学 Vector array passive sonar abnormal data online eliminating method

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN102945320A (en) * 2012-10-29 2013-02-27 河海大学 Time series data abnormity detection method and device
CN103294768A (en) * 2013-04-23 2013-09-11 税友软件集团股份有限公司 Method for removing exceptional data
CN104598251A (en) * 2015-02-12 2015-05-06 江苏飞尚安全监测咨询有限公司 Real-time signal processing method of data acquisition instrument
CN104915846A (en) * 2015-06-18 2015-09-16 北京京东尚科信息技术有限公司 Electronic commerce time sequence data anomaly detection method and system
CN105095482A (en) * 2015-08-13 2015-11-25 浪潮(北京)电子信息产业有限公司 Data mining method and system for detecting abnormal data interval

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN102945320A (en) * 2012-10-29 2013-02-27 河海大学 Time series data abnormity detection method and device
CN103294768A (en) * 2013-04-23 2013-09-11 税友软件集团股份有限公司 Method for removing exceptional data
CN104598251A (en) * 2015-02-12 2015-05-06 江苏飞尚安全监测咨询有限公司 Real-time signal processing method of data acquisition instrument
CN104915846A (en) * 2015-06-18 2015-09-16 北京京东尚科信息技术有限公司 Electronic commerce time sequence data anomaly detection method and system
CN105095482A (en) * 2015-08-13 2015-11-25 浪潮(北京)电子信息产业有限公司 Data mining method and system for detecting abnormal data interval

Non-Patent Citations (2)

* Cited by examiner, † Cited by third party
Title
张璇 等: "利用Origin对数据异常值的剔除方法进行比较", 《实验科学与技术》 *
陈刚: "车联网条件下的混合动力客车车载传感器实时数据预处理研究", 《中国优秀硕士学位论文全文数据库(信息科技辑)》 *

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN106324583A (en) * 2016-08-25 2017-01-11 哈尔滨工程大学 Vector array passive sonar abnormal data online eliminating method

Similar Documents

Publication Publication Date Title
CN113037577B (en) Network traffic prediction method, device and computer readable storage medium
CN107301570A (en) Traffic prediction method, abnormal traffic quantity measuring method, device and electronic equipment
US20240037677A1 (en) Method for power consumption analysis and troubleshooting of production line based on industrial internet of things, system and storage medium thereof
CN107426019A (en) Network failure determines method, computer equipment and computer-readable recording medium
CN112787878B (en) Network index prediction method and electronic equipment
CN117574244B (en) Ultrasonic water meter fault prediction method, device and equipment based on Internet of things
CN105653129A (en) Classic algorithm based real-time signal discrimination and correction method
CN117227177B (en) Multitasking printing control method and system based on equipment supervision
KR20200039877A (en) Apparatus for detecting change of electric load and method thereof
CN111626497A (en) People flow prediction method, device, equipment and storage medium
CN109472289B (en) Key point detection method and device
CN108429771B (en) Mutation theory-based software defined network security state evaluation method and device
CN116383048A (en) Software quality information processing method and device
CN116435995A (en) Time series processing method, computer readable storage medium and electronic device
CN109919744A (en) Detection method neural network based and device
CN110636246B (en) Maintenance method of video communication cloud hardware terminal
CN108961071A (en) The method and terminal device of automatic Prediction composite service income
CN110866652B (en) Online PMU data error correction method and system based on LSTM model
CN114067442A (en) Hand washing action detection method, model training method and device and electronic equipment
CN112152968A (en) Network threat detection method and device
CN113986700A (en) Data acquisition frequency optimization method, system, device and storage medium
CN113379279A (en) Deep reinforcement learning short-term voltage stability evaluation method based on incomplete data
CN113205215A (en) Knowledge-based battlefield situation prediction method
RU2450336C1 (en) Modified intelligent controller with adaptive critic
CN112529739A (en) Building quality global detection method and system

Legal Events

Date Code Title Description
C06 Publication
PB01 Publication
C10 Entry into substantive examination
SE01 Entry into force of request for substantive examination
WD01 Invention patent application deemed withdrawn after publication

Application publication date: 20160608

WD01 Invention patent application deemed withdrawn after publication