CN115600057A - Robust estimation method for sequence truncation processing, electronic device and readable medium - Google Patents
Robust estimation method for sequence truncation processing, electronic device and readable medium Download PDFInfo
- Publication number
- CN115600057A CN115600057A CN202211294956.8A CN202211294956A CN115600057A CN 115600057 A CN115600057 A CN 115600057A CN 202211294956 A CN202211294956 A CN 202211294956A CN 115600057 A CN115600057 A CN 115600057A
- Authority
- CN
- China
- Prior art keywords
- error
- residual
- observation
- vector
- estimation
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F17/00—Digital computing or data processing equipment or methods, specially adapted for specific functions
- G06F17/10—Complex mathematical operations
- G06F17/15—Correlation function computation including computation of convolution operations
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F17/00—Digital computing or data processing equipment or methods, specially adapted for specific functions
- G06F17/10—Complex mathematical operations
- G06F17/16—Matrix or vector computation, e.g. matrix-matrix or matrix-vector multiplication, matrix factorization
Abstract
The invention discloses a robust estimation method for sequencing truncation processing, electronic equipment and a readable medium, and relates to the field of data processing. The method comprises the following steps: calculating an observation value residual vector, and sequencing the absolute values of the observation value residual vectors from small to large according to the condition that the gross error absolute value is greater than the random error absolute value to obtain a residual sequence; based on the sum of the minimum value rn of the number of the normal redundant observed values and the number t of the necessary observed values, performing truncation processing on the residual sequence to obtain a truncated observed residual sequence; calculating a mean error after the test based on the truncated observation residual sequence; updating a configuration weight matrix based on the error in the check; and if the iteration termination condition is met, finishing the iterative robust estimation. The technical scheme of the invention accurately identifies the gross error and reduces the weight, accelerates the iterative convergence speed, reduces the iteration times, improves the tolerance capability of the estimation method and ensures the reliability of the parameter estimation result.
Description
Technical Field
The present invention relates to the field of data processing, and in particular, to a robust estimation method, an electronic device, and a readable medium for sequence truncation processing.
Background
Influenced by observation equipment and observation environment, the observation data has various errors including random errors, system errors, gross errors and the like. When the observation equipment runs suddenly and abnormally, or unpredictable sudden changes occur in the observation environment, gross errors occur in corresponding observed values. Generally, the traditional robust estimation method uses the median error as a reference standard to identify and process the gross error, specifically: estimating the residual error of the observed value, calculating the median error of the observed value, judging the residual error of the observed value according to the median error, and setting a weight function to detect and process the observed value containing the gross error.
The traditional robust estimation method mainly uses a median robust method and a minimum cut-off sum of squares method to obtain parameter estimation initial values, calculates the errors in the process of verification, namely extracts t necessary observed values from n existing observed values, and carries out parameter estimation, residual observation and error calculation in the process of verification. The method specifically comprises the following steps: first, it needs to be extracted
(ii) essential observation data with no duplication of groups; then, parameter estimation, residual error and error calculation in the test are carried out on each necessary observation data set; finally, after checkAnd sequencing the medium errors from small to large, and acquiring the medium errors after final verification according to the median position. However, as the number of observations increases, the number of necessary observation data sets to be extracted significantly increases, and the workload of calculating the corresponding parameter estimation and the median error increases sharply, resulting in low calculation efficiency.
The traditional robust estimation method usually adopts a weight selection iterative method to obtain the optimal estimation value. The weight function is applied to adjust the weight matrix, and the weight selection iterative robust estimation method provides simple, convenient, effective and practical robust estimation for measurement data processing. Here, the common weight function includes IGG series weight functions and the like. However, the robust effect of the existing weighted iteration method depends on the accuracy and reasonableness of the error in the standard. If the error in the first iteration estimation after the test is polluted by the gross error, the error in the first iteration estimation after the test polluted by the gross error is continuously used as a reference to judge whether the observed data contains the gross error, so that the problems of inaccurate identification, poor anti-error effect and the like are caused.
In summary, the conventional robust estimation method has the following problems: when the observation data amount is large, the parameter estimation and medium error calculation workload of the minimum truncated sum-of-squares method and the medium bit error robust estimation method is increased sharply, and the calculation efficiency is low; errors in the process of the selection iteration estimation are easily polluted by gross errors, so that the problems of inaccurate gross error identification and poor anti-poor effect are caused.
Disclosure of Invention
The invention aims to provide a robust estimation method, electronic equipment and readable medium for sequencing truncation processing, which solve the problems of inaccurate coarse difference identification and poor robust effect caused by the fact that errors in the process of verification are easily polluted by coarse differences, and solve the problem of low efficiency of selecting necessary observation data sets to calculate errors in the process of verification when the amount of observation data is large.
The first aspect of the present invention provides a robust estimation method for ordered truncation processing, the method comprising:
s1, calculating an observed value residual vector, and sequencing absolute values of the observed value residual vector from small to large according to the condition that a gross error absolute value is larger than a random error absolute value to obtain a residual sequence;
s2, based on the sum of the minimum value rn of the number of the normal redundant observed values and the number t of the necessary observed values, carrying out truncation processing on the residual error sequence to obtain a truncated observed residual error sequence with the sequence number positioned in the interval [1, t + rn ];
s3, calculating a mean error after test based on the truncated observation residual sequence;
s4, updating and configuring a weight matrix of all the observed values based on the errors in the check;
s5, if the iteration termination condition is met, the iteration robust estimation is completed;
at least before S2, the method also comprises the following steps: and dividing the redundant observed values into normal redundant observed values and abnormal redundant observed values, and setting a minimum value rn of the number of the normal redundant observed values.
Further, in step a, the minimum value rn of the number of the normal redundant observation values is calculated according to formula (1):
rn＝int((r+1)/2) (1)
wherein r = n-t, r represents the number of redundant observation values, n represents the total number of observation values, and t represents the number of parameters to be estimated.
Further, at least before step S1, the following steps are included:
s001, calculating an observation value residual vector by using a formula (2) based on the observation value vector L, the n observation values and the t parameters to be estimated,
V＝Bx-l (2)
v represents an n multiplied by 1 order observation value residual vector, B represents an n multiplied by t order design matrix, x represents a t multiplied by 1 order parameter vector to be estimated, and l represents a difference value between an observation vector and an approximate calculation observation vector;
s002, solving error equation based on least square, calculating parameter estimation vector x to be estimated by formula (3),
x＝(B T PB T ) -1 B T Pl (3)
the weight matrix P is the inverse of Q, Q is the variance covariance matrix of the observation vector L, and T is the matrix transpose symbol.
Further, S3, calculating an error in the experiment based on the truncated observation residual sequence, wherein the calculation formula is formula (4):
wherein σ s Representing the error in the experiment, and sequencing the absolute values of the observation residual errors estimated by the kth iteration to obtain a residual sequence
i denotes the position index number of the sorted observation residual vector elements, i =1,2,3 \8230;, t + rn.
Further, in S4, configuring a weight matrix based on the error in the posterior, specifically:
s401, based on the error sigma in the experiment s Standardizing all the observed value residual vectors by using a formula (5) to obtain a residual standardized value v i ：
V k Representing a parameter estimation vector x to be estimated based on a kth iteration estimation k Calculating an observed value residual vector, wherein i represents a position index number of an observed residual vector element, and i =1,2,3 \ 8230 \8230;, n;
s402, based on the residual error standardized value v i Updating diagonal elements of the weight matrix configuring all the observed values by using formula (6):
wherein, c 1 Is 2.5,c 2 Is 3.5,P k Weight matrix representing the kth time, P k+1 The weight matrix of the (k + 1) th order is shown.
Further, at least before S5, constructing an end condition (7) for the iterative estimation:
wherein σ s (k + 1) represents the error in the posteriori, σ, of the (k + 1) th iterative estimate s (k) Represents the error in the posterior error of the k-th iterative estimate, C =0.01, k represents the iterative estimate number, k>1；
In step S5, if the termination condition of iterative estimation is satisfied, the updating of the configuration weight matrix is stopped, and the parameter estimation vector x to be estimated is output k+1 As the final estimate; and if the termination condition of the iterative estimation is not met, returning to S1 until the termination condition of the iterative estimation is met.
A second aspect of the present invention provides an electronic device comprising: at least one processor; and a memory communicatively coupled to the at least one processor; wherein the memory stores instructions executable by the at least one processor and arranged to perform a robust estimation method of the ordered truncation process.
A third aspect of the present invention provides a computer-readable medium having a computer program stored thereon, wherein the computer program, when executed by a processor, implements a robust estimation method of the order truncation process.
Compared with the traditional weight selection iteration method, the technical scheme of the invention avoids the abnormal observation value residual error from participating in the error calculation in the posteriori through the residual sorting and reasonable truncation processing, reduces the pollution of the abnormal observation value gross error to the error in the posteriori, effectively controls the adverse effect of the gross error to the error calculation in the posteriori, is beneficial to accurately identifying the gross error and reducing the weight of the gross error, accelerates the iteration convergence speed, reduces the iteration times, improves the tolerance capability of the estimation method, and ensures the reliability of the parameter estimation result.
Compared with a median method and a least square sum method (LTS), the technical scheme breaks through the limitation of low robust estimation efficiency, avoids the burden of extracting t observed values from all observed values on a large scale to perform adjustment calculation and sequencing, reduces the calculation workload, and improves the calculation efficiency.
Drawings
Fig. 1 is a flowchart illustrating a robust estimation method of the order truncation process.
Detailed Description
In order to make the objects, technical solutions and advantages of the present invention more apparent, the present invention is further described in detail below with reference to the accompanying drawings. It should be understood that the detailed description and specific examples, while indicating the invention, are intended for purposes of illustration only and are not intended to limit the scope of the invention.
Examples
Referring to fig. 1, in the robust estimation method for sequencing truncation processing in the technical scheme of this embodiment, on the basis of collecting observation data, configuring an initial weight matrix, and setting iteration number k =0, a thought of redundant observation clustering is first designed, and redundant observations are divided into normal redundant observations and abnormal redundant observations affected by gross errors. And setting the minimum value of the number of normal redundant observations according to the requirement of correct convergence of estimation. And secondly, sequencing the absolute values of the observation residuals from small to large. And according to the minimum value of the number of normal redundant observations, truncating the sequenced residual sequence, eliminating possible abnormal observation residual, and not participating in error calculation after the experiment. And finally, adjusting the weight matrix according to the error in the test updated by the cut residual sequence to finish iterative robust estimation. The method and the device realize reduction of pollution of the gross errors to error calculation after the verification and effectively control adverse effects of the gross errors on the error calculation after the verification.
1. Initialization processing
For a group of observation vectors L, the number of observation values is n, and t parameters to be estimated are provided. L is an n × 1 order observation vector. The error equation for the observation vector is:
V＝Bx-l (1)
wherein, V is an observation value residual vector of order nx1, B is a design matrix of order nxt, x is a parameter vector to be estimated of order tx1, and l is a difference value between the observation vector and the approximate calculation observation vector.
The variance covariance matrix of observation vector L is Q, and the weight matrix P is the inverse of Q. Typically, the weight matrix is determined by the ratio of the standard deviation and the actual deviation of the observed values. If the actual variance of the observed value is unknown, the weight matrix can be set as an identity matrix. And solving an error equation by applying least square to obtain a parameter estimation vector x to be estimated:
x＝(B T PB T ) -1 B T Pl (2)
2. the redundant observations are grouped into groups, and the groups are,
for n observation values, t parameters to be estimated are estimated, any two parameters to be estimated are independent, and the number of redundant observation values is as follows:
r＝n-t (3)
and dividing the redundant observed value into a normal redundant observed value and an abnormal observed value influenced by the gross error. The number of normal redundant observation values is rn, and the number of abnormal redundant observation values is ra. Taking a single-parameter observation model as an example, in order to ensure that the parameters of iterative least square estimation can be correctly converged, the normal redundant observation number rn should satisfy:
rn＞r/2 (4)
since rn is an integer, the minimum value of rn is:
rn＝int((r+1)/2) (5)
wherein the int function is a direct rounding processing function, and the decimal is removed to obtain an integer part value.
3. Error and weight matrix update in a posteriori
According to the formula (1), estimating a vector x by using the parameter to be estimated of the k iteration estimation k Calculating residual vector V of observed value k Configuring a weight matrix P of the (k + 1) th iteration estimation k+1 . Sequencing the absolute values of the observation residual errors of the kth iterative estimation to obtain
. According to the formula (5), a normal redundant observation number minimum value rn is obtained. Truncating the sorted residual vectors
The first t + rn residual values, calculating the error in the test:
post-test error sigma calculated after sorting truncation s For the judgment reference, all the observation residual values are standardized to obtain:
in formula 7, i denotes a position index number of an observation residual vector element, i =1,2,3 \8230;, n;
updating diagonal elements of a weight matrix configuring all the observed values according to the residual normalized values:
wherein c is 1 Is 2.5,c 2 Is 3.5.
4 iterative estimation
Using the updated weight matrix P k+1 On the basis of the formula (1) and the formula (2), the parameter estimation vector x to be estimated is obtained by recalculation k+1 Sum observed residual vector V k+1 . Obtaining the error sigma in the posterior (k + 1) th iteration estimation according to the formula (6) s (k + 1), constructing an iteration termination condition:
where C is the limit, generally 0.01. If the termination condition is met, the updating is stopped and x is output k+1 Is the final estimation value of the parameter to be estimated. Otherwise, the weight matrix is continuously updated according to the formula (8), the parameter estimation vector to be estimated is re-estimated, and the observed value residual error is calculated. And (5) obtaining the error in the posterior process through sequencing truncation processing, and substituting the updated error in the posterior process into the formula (9). And (4) judging the difference value of the errors in the current test and the previous test, and iterating until the termination condition of the formula (9) is met.
By adopting the technical scheme disclosed by the invention, the following beneficial effects are obtained: compared with the traditional weight selection iteration method, the technical scheme of the invention reduces the risk that the residual error of the abnormal observation value participates in the error calculation in the posteriori through residual sorting and reasonable truncation processing, reduces the pollution of the coarse error of the abnormal observation value to the error in the posteriori, effectively controls the adverse effect of the coarse error to the error calculation in the posteriori, is beneficial to accurately identifying the coarse error and reducing the weight of the coarse error, accelerates the iteration convergence speed, reduces the iteration times, improves the tolerance capability of the estimation method, and ensures the reliability of the parameter estimation result.
Compared with a median method and a least square sum (LTS) method, the technical scheme of the invention breaks through the limitation of low robust processing efficiency, avoids the burden of extracting t observed values from all observed values on a large scale to perform adjustment calculation and sequencing, reduces the calculation workload and improves the calculation efficiency.
The foregoing is only a preferred embodiment of the present invention, and it should be noted that, for those skilled in the art, many modifications and adaptations can be made without departing from the principle of the present invention, and such modifications and adaptations should also be considered to be within the scope of the present invention.
Claims (8)
1. A method of robust estimation for ordered truncation, the method comprising:
s1, calculating an observed value residual vector, and sequencing absolute values of the observed value residual vector from small to large according to the condition that a gross error absolute value is larger than a random error absolute value to obtain a residual sequence;
s2, based on the sum of the minimum value rn of the number of the normal redundant observed values and the number t of the necessary observed values, carrying out truncation processing on the residual error sequence to obtain a truncated observed residual error sequence with the sequence number positioned in the interval [1, t + rn ];
s3, calculating a mean error after test based on the truncated observation residual sequence;
s4, updating and configuring a weight matrix of all the observed values based on the errors in the check;
s5, if the iteration termination condition is met, the iteration robust estimation is completed;
at least before S2, the method also comprises the following steps: and dividing the redundant observed values into normal redundant observed values and abnormal redundant observed values, and setting a minimum value rn of the number of the normal redundant observed values.
2. The robust estimation method for the ordered truncation process according to claim 1, wherein in step a, the minimum value rn of the number of the normal redundant observations is calculated according to formula (1):
rn＝int((r+1)/2) (1)
wherein r = n-t, r represents the number of redundant observation values, n represents the total number of observation values, and t represents the number of parameters to be estimated.
3. The method of ordered truncation processed robust estimation according to claim 1, further comprising the steps of, at least before step S1:
s001, calculating an observation value residual vector by using a formula (2) based on the observation value vector L, the n observation values and the t parameters to be estimated,
V＝Bx-l (2)
v represents an nxt order design matrix, x represents a tx1 order parameter vector to be estimated, and l represents a difference value between an observation vector and an approximate calculation observation vector;
s002, solving error equation based on least square, calculating parameter estimation vector x to be estimated by formula (3),
x＝(B T PB T ) -1 B T Pl (3)
the weight matrix P is the inverse of Q, Q is the variance covariance matrix of the observation vector L, and T is the matrix transpose symbol.
4. The robust estimation method for sequential truncation processing according to claim 1, wherein S3, based on the truncated observation residual sequence, calculates a mid-test error as formula (4):
wherein σ s Representing errors in the experiment, and sequencing absolute values of observation residuals of the kth iterative estimation to obtain a residual sequence
i denotes the position index number of the sorted observation residual vector elements, i =1,2,3 \8230;, t + rn.
5. The robust estimation method for the sequential truncation process according to claim 1, wherein in S4, the configuring weight matrix based on the error in the posterior is specifically:
s401, based on the error sigma in the experiment s Standardizing all observed value residual vectors by using a formula (5) to obtain a residual standardized value v i ：
V k Representing a parameter estimation vector x to be estimated based on a kth iterative estimation k Calculating an observed value residual vector, wherein i represents a position index number of an observed residual vector element, and i =1,2,3 \ 8230 \8230;, n;
s402, based on the residual normalized value v i Updating diagonal elements of the weight matrix configuring all the observed values by using formula (6):
wherein, c 1 Is 2.5,c 2 Is 3.5,P k Weight matrix representing the k-th order, P k+1 The weight matrix at the k +1 th order is shown.
6. The method of ordered truncation processed robust estimation according to claim 1, comprising, at least before S5, constructing an iterative estimation termination condition (7):
wherein σ s (k + 1) denotes the error in the posteriori, σ, of the (k + 1) th iterative estimate s (k) Represents the error in the posterior error of the k-th iterative estimate, C =0.01, k represents the iterative estimate number, k>1；
In step S5, if the termination condition of iterative estimation is satisfied, the updating of the configuration weight matrix is stopped, and the parameter estimation vector x to be estimated is output k+1 As the final estimate; and if the termination condition of the iterative estimation is not met, returning to S1 until the termination condition of the iterative estimation is met.
7. An electronic device, comprising: at least one processor; and a memory communicatively coupled to the at least one processor; wherein the memory stores instructions executable by the at least one processor and configured to perform a robust estimation method of the order truncation process of any of claims 1 to 6.
8. A computer-readable medium, on which a computer program is stored, which, when being executed by a processor, carries out a robust estimation method of a rank truncation process according to any one of claims 1 to 6.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202211294956.8A CN115600057B (en) | 2022-10-21 | 2022-10-21 | Robust estimation method for sorting truncation processing, electronic equipment and readable medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202211294956.8A CN115600057B (en) | 2022-10-21 | 2022-10-21 | Robust estimation method for sorting truncation processing, electronic equipment and readable medium |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN115600057A true CN115600057A (en) | 2023-01-13 |
| CN115600057B CN115600057B (en) | 2023-05-02 |
Family
ID=84849290
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202211294956.8A Active CN115600057B (en) | 2022-10-21 | 2022-10-21 | Robust estimation method for sorting truncation processing, electronic equipment and readable medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN115600057B (en) |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20100074358A1 (en) * | 2008-09-25 | 2010-03-25 | Nec Laboratories America, Inc. | Sparse Channel Estimation for MIMO OFDM Systems |
| CN103869344A (en) * | 2012-12-13 | 2014-06-18 | 东莞市泰斗微电子科技有限公司 | Robust estimation method |
| CN110515097A (en) * | 2019-09-02 | 2019-11-29 | 江苏省测绘工程院 | GNSS satellite applied to base station observes elimination of rough difference method and apparatus |
| CN111405484A (en) * | 2020-03-13 | 2020-07-10 | 腾讯科技（深圳）有限公司 | Network position mining method, device, equipment and storage medium |
-
2022
- 2022-10-21 CN CN202211294956.8A patent/CN115600057B/en active Active
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20100074358A1 (en) * | 2008-09-25 | 2010-03-25 | Nec Laboratories America, Inc. | Sparse Channel Estimation for MIMO OFDM Systems |
| CN103869344A (en) * | 2012-12-13 | 2014-06-18 | 东莞市泰斗微电子科技有限公司 | Robust estimation method |
| CN110515097A (en) * | 2019-09-02 | 2019-11-29 | 江苏省测绘工程院 | GNSS satellite applied to base station observes elimination of rough difference method and apparatus |
| CN111405484A (en) * | 2020-03-13 | 2020-07-10 | 腾讯科技（深圳）有限公司 | Network position mining method, device, equipment and storage medium |
Also Published As
| Publication number | Publication date |
|---|---|
| CN115600057B (en) | 2023-05-02 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN109521443B (en) | Method for detecting ephemeris abnormity | |
| KR100817092B1 (en) | Measuring system for correcting overlay measurement error and method of measurement using the system | |
| Maydeu-Olivares | Evaluating the fit of IRT models | |
| CN104102836A (en) | Method for quickly estimating robust state of power system | |
| CN106529080A (en) | Establishment method for square resistor SPICE (Simulation Program with Integrated Circuit Emphasis) model | |
| CN104881867A (en) | Method for evaluating quality of remote sensing image based on character distribution | |
| CN110059705B (en) | OCR recognition result judgment method and device based on modeling | |
| CN116578870A (en) | Distribution network voltage abnormal data filling method based on fluctuation cross-correlation analysis | |
| CN112734310B (en) | Fault correlation analysis system and method | |
| CN115600057A (en) | Robust estimation method for sequence truncation processing, electronic device and readable medium | |
| CN111027190A (en) | Evaluation method and device for numerical similarity of equipment model | |
| CN112083463A (en) | Method and device for detecting whether ambiguity is fixed correctly or not and positioning terminal | |
| CN106874531B (en) | Method for automatically recovering abnormal measurement value data of atmospheric data system in case of failure | |
| CN102385642B (en) | Correction method for device dismatch of resistor | |
| CN112651500B (en) | Method for generating quantization model and terminal | |
| CN102262188B (en) | Sampling inspection method for workpieces | |
| CN110991001B (en) | Spring coiling service life evaluation method based on monotonic regression theory | |
| Spanos et al. | Heteroskedasticity/autocorrelation consistent standard errors and the reliability of inference | |
| Murray | One-way analysis of variance (ANOVA) | |
| CN111275328A (en) | RNGRU (radio network unit) position marker part matching method based on comprehensive grey correlation sequence | |
| Wilcox | Detecting nonlinear associations, plus comments on testing hypotheses about the correlation coefficient | |
| CN112580714B (en) | Article identification method for dynamically optimizing loss function in error-cause reinforcement mode | |
| Canepa | A note on Bartlett correction factor for tests on cointegrating relations | |
| CN108614903A (en) | Integrated circuit simulating data dependence modeling method and device | |
| CN116184216B (en) | Battery state detection method and device |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |