CN115632633A - Minimum error entropy self-adaptive filtering method based on robust M estimation - Google Patents

Abstract

The invention discloses a minimum error entropy self-adaptive filtering method based on robust M estimation, which comprises the following steps: s1, constructing a self-adaptive filtering system and acquiring a prediction error; s2, according to the prediction error, constructing a target function and an optimization target based on a new steady M estimation minimum error entropy criterion, and calculating to obtain an MMEE optimization loss criterion corresponding to the prediction error sample; s3, performing partial differentiation on the error function, and updating the weight of the adaptive filtering system; and S4, iterating the steps S1 to S3 to enable the error value of the adaptive filtering system to be converged, and taking the weight of the adaptive filtering system at the moment as a final weight for filtering the input signal. The adaptive filter has extremely high convergence rate and low steady-state error, can effectively improve the robustness of the adaptive filter under various non-Gaussian noises, has good environment adaptability, and can stably process the sudden change of the noise type in the environment.

Description

A Minimum Error Entropy Adaptive Filtering Method Based on Robust M Estimation

technical field

The invention relates to technical research in the field of digital signal processing, in particular to a minimum error entropy self-adaptive filtering method based on robust M estimation.

Background technique

In recent years, adaptive filtering algorithms have been gradually developed on the basis of Wiener filtering (Wiener) and Kalman filtering (Kalman) algorithms. The adaptive filtering algorithm is considered to be the best filtering technology, its outstanding advantages lie in its strong adaptability and improvement, and its performance is stronger in reducing the impact of noise on the overall signal system, so it is practical in engineering , that is, some practical signal processing tasks, such as echo cancellation, system identification, etc., have been widely used. The research object of adaptive filtering is an uncertain system or information process. The "uncertainty" here means that the mathematical model of the studied information processing and its environment is not completely deterministic. There are some unknowns and random factors involved.

Traditional adaptive filtering techniques are based on the minimum mean square error criterion (MMSE), which is the best criterion for linear filters under the Gaussian assumption. However, in various practical application scenarios, the signal may be affected by some non-Gaussian noise. pollution, when the system or information process is non-Gaussian, the second-order statistics of the minimum error criterion will not be enough to fully extract the effective information from the data sample. In this case, the method based on the minimum mean square error criterion Filtering performance will be severely degraded. In this context, related algorithm guidelines based on information theory learning (ITL) have been widely studied. Among them, the minimum error entropy (MEE) algorithm and its extension can capture the high-order statistics of signal error samples, and the minimum contrast The mean square error criterion and the minimum error entropy criterion have better effects when analyzing signal systems polluted by non-Gaussian noise. Usually, the minimum error entropy criterion uses the non-parametric Gaussian kernel estimation method based on the Parzen window to obtain the estimated value of the second-order Rayleigh entropy of the processed data samples, so as to provide a more robust optimization for the non-Gaussian signal process guidelines. The minimum error entropy can be used in the iterative update process of adaptive system identification, and is considered to be an effective method for dealing with non-Gaussian noise.

However, the above criterion still has certain limitations. When the desired signal is interfered by heavy-tailed or multi-peak impulse noise (a large outlier will appear in the data sample), directly using the error sample containing the outlier will reduce the It is difficult to achieve a highly accurate estimation of the Rayleigh entropy, which may limit the performance improvement of the algorithm; in addition, once the iteration parameters (such as kernel width and window length) are not selected properly, the Gaussian kernel probability density estimation method used in the criterion will not affect the parameters. High sensitivity will further lead to the degradation of filtering performance.

Rayleigh (Renyi's) entropy, as a very important nonlinear information similarity measure, is the core element of the minimum error entropy (MEE) criterion, because the traditional minimum error entropy (MEE) criterion cannot achieve accurate estimation of Rayleigh entropy , especially when the error samples contain a large number of outliers, so there is a huge room for algorithm performance improvement.

Contents of the invention

In order to overcome the above-mentioned deficiencies in the prior art, the present invention provides a minimum error entropy adaptive filtering method based on robust M estimation. The filtering performance is better when working in a non-Gaussian environment while converging faster.

The present invention is achieved through the following technical solutions: a minimum error entropy adaptive filtering method based on robust M estimation, characterized in that: comprising the following steps:

S1. Build an adaptive filtering system. At each moment, the input signal passes through the adaptive filtering system to obtain the output signal as the predicted value, and make a difference with the expected output signal to obtain the prediction error;

Include in the described step S1:

S101. Build an adaptive filtering system: the weight of the adaptive filtering system is w _n ; assuming that the input signal u _{n at time n} is Gaussian white noise with a mean value of 0 and a variance of 1, the optimal weight of the adaptive filtering system is preset for w _o ;

S102. Multiply the input signal u _n with the optimal weight w ^o expected by the filter, and add the noise signal v(n) to obtain the desired output signal d(n):

Where v _n is mixed Gaussian noise, and v _n ～0.95N(0,0.01)+0.05N(0,10),

where the input signal u _n is uncorrelated with the noise signal v _n ;

S102. Multiply the input signal u _n with the weight w _n of the adaptive filter system to obtain the predicted output signal y _n :

S103. Calculate the prediction error between the expected output signal d(n) and the predicted output signal yn, denoted as:

e _n =d _n -y _n .

S2. According to the prediction error, construct the objective function and optimization objective based on the new robust M estimation minimum error entropy criterion, and calculate the MMEE optimization loss criterion corresponding to the prediction error sample;

Described step S2 comprises:

S201. Before each round of iteration, calculate the error in the first L window lengths including time n according to step S1: e _n-L+1 to e _n , as the error sample vector required for the Parzen window to slide once, where n is an integer not less than L;

S202. Construct the loss function and optimization objective based on the new robust M-estimation minimum error entropy criterion: use the statistical characteristics of M estimation, on the basis of the traditional minimum error entropy criterion, set the M estimation weight factor to weight or truncate the error , by resetting the distribution of error samples, reducing the impact of outliers on adaptive system estimation, and then obtaining a minimum error entropy loss function based on robust M estimation, specifically:

Where ε _n =[e _n ,e _n-1 ,...,e _n-L+1 ] ^T represents the error sample vector;

表示基于窗长内L个误差样本计算所得到的M估计权重因子，取值范围限制在(0,1)之间，其中Δ₁、Δ₂、Δ₃是分界参数，均为预先设置的常数，所述M估计权函数包括Hampel权函数，记为：The minimum error entropy criterion for robust M estimation aims to maximize the above loss function, thereby minimizing the gap between the predicted output and the expected output of the adaptive filtering system, where

Indicates the M estimated weight factor calculated based on L error samples in the window length, and the value range is limited between (0,1), where Δ ₁ , Δ ₂ , and Δ ₃ are boundary parameters, all of which are preset constants , the M estimated weight function includes the Hampel weight function, denoted as:

S3. Use the stochastic gradient ascent method to update the weight of the adaptive filtering system: first perform partial differentiation on the error function, and then update the weight of the adaptive filtering system;

In the step S3, after obtaining the objective function, the stochastic gradient ascending method is used to update the weight parameters of the adaptive filtering, specifically including:

Partially differentiate the objective first:

in

Then perform a gradient ascent update algorithm on the weight parameter w _n of the filter area, w _n represents the weight parameter at the nth moment, and the subscript n+1 represents the weight parameter at the n+1th moment of the adaptive filter. The update formula is as follows:

S4. Iterative steps S1 to S3 make the error value of the adaptive filtering system converge, that is, the gap between the predicted output of the adaptive filtering system and the expected output is smaller than the set threshold, and the weight of the adaptive filtering system at this time is used as the final weight. It is used for filtering processing of input signal.

The beneficial effects of the present invention are as follows: the present invention utilizes the robust statistical characteristics of M-estimation, combines the minimum error entropy algorithm and M-estimation numerical statistics theory, introduces some M-estimation weight factors to recalculate the statistical distribution of error samples, aiming at Minimize the influence of large outliers, while maintaining a high convergence speed, the filtering performance is better in non-Gaussian environments, and it is highly insensitive to parameter changes. The algorithm is flexible and easy to adjust, and adapts to a variety of non-Gaussian environments. .

Description of drawings

Fig. 1 is the basic principle block diagram of general adaptive filtering algorithm;

Fig. 2 is the functional block diagram of the minimum error entropy adaptive filtering algorithm based on robust M estimation of the present invention;

Fig. 3 is a schematic flow sheet of the present invention;

Fig. 4 is the theoretical analysis and experimental result comparison schematic diagram of the steady-state performance of the minimum error entropy adaptive filtering method based on the robust M estimation of the present invention as the step size changes;

Fig. 5 is the steady-state mean square error effect comparison figure of this method and other relevant robust adaptive filtering methods;

Fig. 6 is a schematic diagram of the comparison of the system convergence curves of the sudden change of the noise type in the adaptive system at 1000 points and 2000 points respectively in the case of a sudden change in the environmental noise.

Detailed ways

The technical solution of the present invention will be further described in detail below in conjunction with the accompanying drawings and examples, but the protection scope of the present invention is not limited to the following description.

The basic block diagram of the basic adaptive filter is shown in Figure 1. The input signal u _n passes through the parameter-adjustable adaptive digital filter to generate an output signal y _n , which is compared with the expected signal d _n to form an error signal e _n , adjust the filter parameters through an adaptive algorithm, and finally minimize the loss function. Adaptive filtering can use the results of the filter parameters obtained at the previous moment to automatically adjust the filter parameters at the current moment to adapt to the unknown or time-varying statistical characteristics of the signal and noise, thereby achieving optimal filtering. Due to the different objective functions, the adaptive filtering algorithms under different criteria also have different effects.

通过自适应数字滤波器系统后卷积运算产生输出信号

两者相减形成误差信号e_n＝d_n-y_n，迭代运算得到数个误差样本，通过M估计方法，重新计算误差样本分布，对大型误差进行归0，对中型误差进行截短，对小型误差离群值不进行操作，最终得到MMEE误差熵损失函数。The principle block diagram of the minimum error entropy adaptive filter based on robust M estimation is shown in Figure 2. The input signal u _n passes through the unknown system (the unknown system has the best weight), and the mixed Gaussian noise v _n ～0.95N is used (0,0.01)+0.05N(0,10) as the noise signal to generate the actual desired output signal

The output signal is generated by the post-convolution operation of the adaptive digital filter system

The two are subtracted to form an error signal e _n =d _n -y _n , and several error samples are obtained through iterative operation, and the error sample distribution is recalculated through the M estimation method, and the large error is returned to 0, and the medium error is truncated. Small error outliers are not manipulated, and the MMEE error entropy loss function is finally obtained.

The present invention is based on the adaptive filtering method of M estimation and minimum error entropy, is the improvement on the adaptive filtering algorithm under the minimum error entropy criterion, the overall flowchart of this method of Fig. 3 (a), Fig. 3 (b) Then, the first, second and third steps of this method are introduced in detail:

1. The input signal u _n vector obeys a Gaussian distribution with a mean value of 0 and a variance of 1, and the instantaneous measurement noise signal obeys a mixed Gaussian distribution: v _n ～0.95N(0,0.01)+0.05N(0,10);

其中T为转置算子，同时可得到滤波器实际输出

其中w₀为被估计的系统权重向量，它是一个未知但实际存在的值，可以给其赋予任何初值，但这个初值却是我们假设未知的，自适应滤波的任务便是通过w_n的迭代更新去逼近真实值w_o。2. Ideal regression output signal

Where T is the transpose operator, and the actual output of the filter can be obtained at the same time

Among them, w ₀ is the estimated system weight vector, which is an unknown but actually existing value, and any initial value can be assigned to it, but this initial value is assumed to be unknown. The task of adaptive filtering is to pass w _n Iterative update to approximate the true value w _o .

3. Calculate the difference between the estimated output value and the actual value, that is, the prediction error e _n =d _n −y _n .

4. Establish the minimum error entropy criterion (MMEE) based on robust M estimation, determine the optimization object, consider the error e _n as a random variable, and its probability density function (PDF) is p _e , we can know the two The sub-Rayli entropy can be written as:

Among them, V ₂ (e) is the secondary information potential. Obviously, minimizing the secondary Rayleigh entropy is equivalent to maximizing the secondary information potential. The error distribution is recalculated by the M estimation method, and the objective function is constructed at the same time, in order to approach the theoretical quadratic information potential and quadratic Rayleigh entropy to the greatest extent through the practical and feasible non-parametric estimation formula.

为M估计权函数Among them, L is the number of samples of the window length, κ _σ () is the Gaussian kernel function,

Estimate the weight function for M

The above-mentioned Hampel function is the most suitable estimation weight function, but other M estimation weight functions can also be selected according to the situation being dealt with:

由于需要最大化目标函数，在基于稳健M估计的最小误差熵的准则下理想的权重向量可以由此获得

根据梯度上升法得到滤波器权重向量的迭代更新公式：5. Find the partial derivative of the objective function with respect to w _n

Since the objective function needs to be maximized, the ideal weight vector can be obtained under the criterion of minimum error entropy based on robust M estimation

According to the gradient ascent method, the iterative update formula of the filter weight vector is obtained:

n表示第n步迭代。where η is the step size parameter,

n represents the nth iteration.

In summary, the method and innovation provided by the present invention extend and improve the existing minimum error entropy adaptive filtering, and reduce the influence of outliers on Rayleigh entropy estimation by recalculating the statistical distribution of error samples. The utilization of probability information by the error criterion is improved, and the robustness of the algorithm in dealing with various non-Gaussian noises and the flexibility for practical situations are improved by using the robust M estimation.

In addition, the present application has also carried out corresponding theoretical analysis and verification of the invented method, including mean value stability and mean square steady-state performance, which greatly ensured the theoretical basis of the invention. Compared with some existing robust filters for non-Gaussian noise, this filter is extremely superior and effective, and can be applied to various types of non-Gaussian systems or signal processes, while satisfying fast convergence and low The performance index of steady-state error; this method has good universality and flexible adjustment space, and can be applied to different practical application scenarios through flexible adjustment of parameters, which has relatively important research significance and extensive engineering application value. Verification simulation and comparative experiments further illustrate this method:

As shown in FIG. 4 , the theoretical analysis of the ultimate steady-state performance of the present invention is based on experimental verification, which proves that the simulation experimental results are in good agreement with the theoretical analysis results. Among them, MSD means the steady-state mean square error, η represents the step size, theory represents the theoretical value, and simulation represents the experimental value.

横坐标代表迭代次数：每次实验迭代运行了1500次，噪声采用的混合高斯噪声。我们为每个算法选择了不同的参数，以确保具有相同初始收敛速度以及理想的性能。从仿真结果中可以看出，与其他算法相比，本发明提出的方法综合性能最佳，收敛速度快，稳态误差低，其中采用Hampel估计权函数的MMEE方法表现最为突出。Figure 5 is a comparison chart of the effect of this method and other related robust adaptive filtering methods in the literature, and the ordinate represents the steady-state error obtained by averaging 100 experiments

The abscissa represents the number of iterations: each experimental iteration runs 1500 times, and the noise uses a mixed Gaussian noise. We chose different parameters for each algorithm to ensure the same initial convergence rate and ideal performance. It can be seen from the simulation results that compared with other algorithms, the method proposed by the present invention has the best comprehensive performance, fast convergence speed and low steady-state error, among which the MMEE method using Hampel estimation weight function is the most outstanding.

The MMEE algorithm proposed in this method performs well in non-Gaussian noise, and, in general, is insensitive to changes in kernel width. Therefore, we can take advantage of this feature of the MMEE algorithm to further explore its performance under unsteady conditions. Figure 6 shows that the adaptive system changes time at the 1000th point and the 2000th point, that is, the noise type of the unknown system suddenly changes from Gaussian noise to non-Gaussian noise, using the robust M estimation method in this method and other traditional least mean Square error and minimum error entropy methods, running 3000 times to obtain the comparison chart of the average value of the mean square error. It can be seen from Figure 6 that after time-varying at 1000 and 2000 points: interference by heavy-tail impulse noise, which is actually very common in reality, the traditional fixed-step LMS method and the MEE method at the 1000th point -2000 sampling points, the performance of suppressing noise is still severely degraded, while the MMEE-Hampel method can still maintain consistent filtering capabilities, and is not affected by changes in noise types in the environment. The method proposed by the present invention has noise jumps in the system environment When , the speed of returning to the steady state is faster, the tracking performance is stronger, and the filtering performance is better. This further proves the excellent robustness of the MMEE algorithm proposed by this method in the case of sudden changes in the system environment.

The above description shows and describes a preferred embodiment of the present invention, but as mentioned above, it should be understood that the present invention is not limited to the form disclosed herein, and should not be regarded as excluding other embodiments, but can be used in various Various other combinations, modifications, and environments can be made within the scope of the inventive concept described herein, by the above teachings or by skill or knowledge in the relevant field. However, changes and changes made by those skilled in the art do not depart from the spirit and scope of the present invention, and should all be within the protection scope of the appended claims of the present invention.

Claims (4)

1. A minimum error entropy self-adaptive filtering method based on robust M estimation is characterized in that: the method comprises the following steps:

s1, constructing a self-adaptive filtering system, and obtaining an output signal as a predicted value by an input signal through the self-adaptive filtering system at each moment, and subtracting an expected output signal to obtain a prediction error;

s2, according to the prediction error, constructing a target function and an optimization target based on a new steady M estimation minimum error entropy criterion, and calculating to obtain an MMEE optimization loss criterion corresponding to the prediction error sample;

s3, updating the weight of the self-adaptive filtering system by adopting a random gradient ascending method: firstly, partial differentiation is carried out on an error function, and then the weight of the self-adaptive filtering system is updated;

and S4, iterating the steps S1 to S3 to enable the error value of the adaptive filtering system to be converged, namely the difference between the predicted output and the expected output of the adaptive filtering system is smaller than a set threshold value, and taking the weight of the adaptive filtering system at the moment as a final weight for filtering the input signal.

2. The robust M-estimation based minimum error entropy adaptive filtering method of claim 1, wherein: the step S1 comprises the following steps:

s101, constructing a self-adaptive filtering system: the weight of the adaptive filter system is w _n (ii) a Let n time input signal u _n Is white Gaussian noise with mean value of 0 and variance of 1, and the optimal weight of the self-adaptive filtering system is preset to be w _o ；

S102, inputting a signal u _n And the optimal weight w expected by the filter ^o Multiplying, and adding the noise signal v (n) to obtain the desired output signal d (n):

wherein v is _n Is a mixture of Gaussian noise, and v _n ～0.95N(0,0.01)+0.05N(0,10)，

Wherein the input signal u _n And a noise signal v _n Not related;

s102, inputting a signal u _n And adaptive filter systemWeight w of _n Multiplying to obtain a predicted output signal y _n ：

S103, calculating an expected output signal d (n) and a prediction output signal y _n The prediction error between, noted as:

e _n ＝d _n -y _n 。

3. the robust M-estimation based minimum error entropy adaptive filtering method of claim 2, wherein: the step S2 includes:

s201, before each iteration, calculating errors in the first L window lengths including the time n according to the step S1: e.g. of the type _n-L+1 To e _n As an error sample vector required for sliding a Parzen window once, where n is an integer not less than L;

s202, constructing a loss function and an optimization target based on the new robust M estimation minimum error entropy criterion: by utilizing the statistical characteristic of M estimation, on the basis of the traditional minimum error entropy criterion, M estimation weight factors are set to weight or truncate errors, and the influence of outliers on the estimation of the self-adaptive system is reduced by resetting the distribution of error samples, so that a minimum error entropy loss function based on the steady M estimation is obtained, specifically:

wherein epsilon _n ＝[e _n ,e _n-1 ,...,e _n-L+1 ] ^T Representing an error sample vector;

the criterion of the minimum error entropy of the robust M estimation aims to maximize the loss function and further minimize the difference between the prediction output and the expected output of the adaptive filtering system, wherein

Represents M estimated weight factors obtained by calculation based on L error samples in the window length, and the value range is limited between (0, 1), wherein, delta ₁ 、Δ ₂ 、Δ ₃ Is a boundary parameter which is a preset constant, the weight factor is calculated, and a Hampel weight function in an M estimation theory is generally adopted, namely, the order is given

The expression is noted as:

wherein r is _i Is a "normalized" residual indicator,

med () is the median, s is e _i The corresponding residual measure.

4. A method for minimum error entropy adaptive filtering based on robust M estimation according to claim 3, characterized by: in step S3, after obtaining the objective function, the adaptive filtering weight parameter is updated by using a random gradient ascent method, which specifically includes:

firstly, partial differentiation is carried out on a target:

wherein

Re-pair the filter region weight parameter w _n Performing a gradient ascent update algorithm, w _n Represents the weighting parameter at the nth time, the subscript n +1 represents the weighting parameter at the nth +1 th time of the adaptive filter, and the updating formula is as follows：

Images