CN117056671A - EMD-based Raman spectrum noise reduction method - Google Patents

Abstract

The application discloses a Raman spectrum noise reduction method based on EMD, which comprises the following steps: step S1: acquiring Raman spectrum data containing noise; step S2: carrying out empirical mode decomposition on the noisy Raman spectrum data; step S3: judging the IMF noise content decomposed by EMD by using the arrangement entropy of IMF; step S4: detecting an n-dimensional eigenmode component Raman peak signal; step S5: soft threshold filtering denoising and peak decomposition boundary processing; step S6: the Savi tzky-Golay filtering treatment; step S7: and (5) carrying out signal recombination to obtain the Raman spectrum data after noise reduction. The Raman spectrum noise reduction method based on the EMD has the advantages of simplicity in operation, high fidelity of Raman characteristic peak information and the like.

Description

EMD-based Raman spectrum noise reduction method

Technical Field

The application relates to the technical field of Raman spectrum noise reduction, in particular to a Raman spectrum noise reduction method based on EMD (Empirical Mode Decomposition ).

Background

There are two noise reduction approaches to the existing raman spectroscopy noise reduction methods.

The first is signal fitting noise reduction. Savitzky-Golay filtering and Whittaker filtering are noise reduction methods that apply a broader class of signal fits over raman spectral noise reduction.

Savitzky-Golay filtering smoothes spectral data by fitting a polynomial, and by this smoothing, high-frequency noise and abrupt changes in the spectrum can be removed, making the data smoother.

Whittaker filtering is based on the idea of least squares curve fitting, and the smoothing effect is achieved by balancing smoothness and data fitting during filtering.

The second is signal decomposition noise reduction. Signal decomposition noise reduction methods attempt to separate the effective signal from the noise signal in the spectrum. Fourier transform (Fourier Transformation) threshold filtering, wavelet transform (Wavelet Transform) filtering, and empirical mode decomposition (Empirical Mode Decomposition) all pertain to signal decomposition noise reduction.

Fourier transform threshold filtering is the decomposition of the signal from the frequency domain perspective. Through the frequency domain characteristics of noise, a specific filter (usually a low-pass or band-pass filter) is constructed to carry out filtering operation on the frequency domain signals, the effect of selectively enhancing or inhibiting the signals is achieved, and finally, the filtered frequency domain signals are subjected to inverse Fourier transform to obtain Raman spectrum data after noise reduction.

The wavelet transformation threshold filtering is to decompose the signal into linear combinations of wavelet basis functions of different scales and frequencies by using selected wavelet basis functions to obtain wavelet coefficients at different scales. The low frequency wavelet coefficients represent smooth components of the signal and the high frequency wavelet coefficients represent detailed components of the signal. And selecting a proper threshold value according to the selected wavelet basis function and the decomposed wavelet coefficients to carry out filtering treatment, and finally reconstructing the filtered wavelet coefficients to obtain the Raman spectrum data after noise reduction.

The empirical mode decomposition is to construct an eigenmode component (Intrinsic Mode Functions) decomposed signal satisfying a specific condition from extrema of the signal. EMD (empirical mode decomposition) is carried out on the noise-containing spectrum data to obtain n intrinsic mode components (IMF) and a residual component, a certain screening rule is constructed and applied, IMFs with high noise duty ratio are found and screened out, and the screened reconstructed signals of all components are utilized to obtain the Raman spectrum data after noise reduction.

Drawbacks/deficiencies of the prior art:

although savitzky-Golay filtering is a common method of noise reduction in raman spectroscopy, SG filtering still suffers from some drawbacks. Smoothing parameter selection problem: for different raman spectrum signals, to obtain more ideal noise reduction data, different noise reduction parameters, such as a filtering window and a fitting order, are generally required to be selected according to signal characteristics, and unsuitable parameter selection can cause 'over-smoothing' and 'under-smoothing'. The 'overcomplete' is caused by excessive smoothing, and although random noise such as environment and dark current is greatly reduced, the raman peak shape deformation is caused, so that peak value and detail information in a signal are affected, which is equivalent to the additional introduction of noise of another form; the 'under-smoothing' is caused by poor smoothing effect, and the filtered signal retains excessive noise to influence the subsequent use of the Raman signal; boundary processing problem: since SG filtering is based on sliding windows, the same processing as in other ranges cannot be done in the range where the two-end boundary is smaller than the selected window.

Whittaker filtering belongs to the class of signal fitting noise reduction as does SG filtering, and similar problems exist in SG filtering. Smoothing parameter selection problem: smoothness is an important parameter affecting noise reduction and controls the trade-off between smoothness and data fit. Too large or too small a smoothness parameter may result in an undesirable filtering effect. A larger smoothness parameter may excessively smooth the signal, lose detail information, and a smaller smoothness parameter may not effectively reduce noise; selection of fitting order: the higher the fitting order is typically chosen, the more signal detail remains, but the detail information that remains may be a valid signal or a noisy signal. Fitting error: the Whi filter achieves the filtering effect by minimizing the data fit term and the smoothness constraint term. However, such a fitting process may introduce certain fitting errors, especially when complex noise or nonlinear components are present in the original signal. Outlier handling problem: the Whi filter is sensitive to outliers. When extreme values or outliers exist, the filtering result may be disturbed by outliers, resulting in an undesirable filtering effect.

The Fourier transform threshold filtering and the wavelet transform filtering are both to try to use the basis function to take the decomposed signal, and the difference is that the Fourier transform uses a sine function, while the wavelet transform can freely select the basis function to be used, such as Daubechies wavelet, haar wavelet, morlet wavelet and the like, and can make the basis function decomposed in different scales and peaceful removal modes, so that the method has higher freedom, and simultaneously brings higher using threshold and a special processing method of a specific signal, and the model has poorer universality; the fourier transform threshold filtering assumes that the signal is periodic. When the signal does not meet this assumption, the fourier transform may cause spectral leakage, i.e. the spectral components are erroneously distributed over other frequencies in the frequency domain. Spectral leakage may lead to errors and artifacts in the filtering result.

The existing empirical mode decomposition noise reduction has large influence on effective Raman peak signals of the Raman spectrum, causes distortion of half-peak width, peak intensity, peak center wavelength and the like, seriously reduces the content of effective information in the Raman spectrum, and influences the subsequent data analysis work of the Raman spectrum. In the existing EMD noise reduction, the first k (k < n) IMF components with high noise ratio, which are decomposed by the EMD, are directly discarded, which can filter most high-frequency noise, but in the case of dense multiple raman peaks, the first k IMF components also contain a plurality of raman peak signals, which can be filtered together, so that the signals are missing. Since the EMD decomposition process needs to ensure that each IMF meets the zero-mean condition, which is one of the reasons for generating modal aliasing, this condition causes one raman peak signal to be decomposed into multiple IMFs, which also contains the first k IMF components, and direct filtering also affects the peak intensity of the raman peak. At the same time, in the case of multiple IMF filtering, half-width distortion and peak center wavelength shift may result.

Disclosure of Invention

The application aims to provide a Raman spectrum noise reduction method based on EMD, which aims to solve the problems in the prior art.

In order to achieve the above purpose, the present application provides the following technical solutions: an EMD-based Raman spectrum noise reduction method comprises the following steps: step S1: acquiring Raman spectrum data containing noise; step S2: carrying out empirical mode decomposition on the noisy Raman spectrum data; step S3: judging the IMF noise content decomposed by EMD by using the arrangement entropy of IMF; step S4: detecting an n-dimensional eigenmode component Raman peak signal; step S5: soft threshold filtering denoising and peak decomposition boundary processing; step S6: savitzky-Golay filtering; step S7: and (5) carrying out signal recombination to obtain the Raman spectrum data after noise reduction.

Further, the step S2 carries out EMD decomposition on the noisy Raman spectrum data to obtain corresponding n-dimensional eigenmode components and one-dimensional decomposition residual components, and each one-dimensional eigenmode component is recorded as IMF _i Where i=1, 2,3 … n, the one-dimensional decomposition residual component is denoted IMF _n+1 。

Further, the step S3 calculates the permutation entropy P of each one-dimensional eigenmode component and one-dimensional decomposition residual component by using the method 1 _i Calculating window P by combining specific permutation entropy _i Setting a proper T _P Screening and calculating to obtain each P _i If P _i ＞T _P Step S4, otherwise, IMF _i ”＝IMF _i Continuing to process the next IMF component;

further, the step S4 includes: searching for IMF _i All peak information P in _i,j Where j < L, combined with permutation entropy P of the current component _i Maximum peak value M from current component _i ＝max(P _i,j ) Calculating an effective Raman peak-to-peak value judgment threshold R of the current component according to the method of 2 _i ；

Further, the step S5 includes: sequentially taking adjacent peak information P _i,j And P _i,j+1 And R is as follows _i Make a comparison judgment and record P _i,j Corresponding signal subscript I _j And P _i,j+1 The method comprises the steps of carrying out a first treatment on the surface of the And the processed IMF _i Recombined into IMF _i '。

Further, the step S6 includes: for IMF _i ' fitting window is 5, fitting orderThe Savitzky-Golay filter processing of specific smoothing parameter with number of 2 to obtain IMF _i ”。

Further, the step S7 is to divide all IMFs _i "performing recombination according to formula 3 to obtain noise-reduced raman spectrum data S';

further, the signal index I corresponding to the step S5 _j+1 Three kinds of cases are treated:

a.|P _i,j |＞|P _i,j+1 |＞R _i or |P _i,j+1 |＞|P _i,j |＞R _i : at this time IMF _i From I _j To I _j+1 Directly reserving the range signal for the decomposition information of the effective raman peak;

b.|P _i,j |＞R _i ＞|P _i,j+1 i or |P _i,j+1 |＞R _i ＞|P _i,j I (L): at this time IMF _i From I _j To I _j+1 The decomposition information of partial effective Raman peak exists in the range of the (I) and the zero point subscript is found in the range by combining IMF structural characteristics _z ；|P _i,j |＞R _i ＞|P _i,j+1 When I, IMF is to be performed _i Middle I _j To I _z Setting a signal of a range to 0; p (P) _i,j+1 |＞R _i ＞|P _i,j When I, IMF is to be performed _i Middle I _z To I _j+1 Setting a signal of a range to 0;

c.R _i ＞|P _i,j |＞|P _i,j+1 i or R _i ＞|P _i,j+1 |＞|P _i,j I (L): at this time IMF _i From I _j To I _j+1 No decomposition information of effective raman peak exists in the range of (2), and all the decomposition information is subjected to 0 setting treatment.

Compared with the prior art, the application has the beneficial effects that:

1. the application has simple use and does not need to carry out different noise reduction treatments on different Raman spectrums;

2. the problem of loss of effective Raman peak information caused by modal aliasing is avoided;

3. the Raman peak detail information including peak value, half-peak width, peak center wavelength and the like is well maintained while noise is reduced.

Drawings

The accompanying drawings are included to provide a further understanding of the application and are incorporated in and constitute a part of this specification, illustrate the application and together with the embodiments of the application, serve to explain the application. In the drawings:

FIG. 1 is a Raman spectrum noise reduction flow chart of the present application;

FIG. 2 is a graph comparing the noise reduction of the existing EMD decomposition screening noise reduction, savitzky-Golay filtering noise reduction and the application method noise reduction;

FIG. 3 is a graph of acquired noisy acetaminophen raman spectral data S in accordance with the present application;

FIG. 4 shows RES as IMF according to the present application _n+1 An exploded signal diagram;

FIG. 5 is an IMF of the present application ₁ With IMF ₁ ' a comparison graph;

FIG. 6 is an IMF of the present application ₁ With IMF ₁ "comparative graph;

fig. 7 is a graph of the raman spectral data S' after noise reduction according to the present application.

Detailed Description

For the purpose of making the objects, technical solutions and advantages of the embodiments of the present application more apparent, the technical solutions of the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present application, and it is apparent that the described embodiments are some embodiments of the present application, but not all embodiments. All other embodiments, based on the embodiments of the application, which are apparent to those of ordinary skill in the art without inventive faculty, are intended to be within the scope of the application. Thus, the following detailed description of the embodiments of the application, as presented in the figures, is not intended to limit the scope of the application, as claimed, but is merely representative of selected embodiments of the application.

Referring to fig. 1-7, in an embodiment of the present application, a method for reducing noise of raman spectrum based on EMD is provided, including: s1: acquiring Raman spectrum data containing noise; s2: empirical mode decomposition; s3: judging the noise content; s4: detecting an n-dimensional intrinsic mode component (IMF) Raman peak signal; s5: IMF soft threshold filtering denoising and peak decomposition boundary processing; s6: IMF Savitzky-Golay filtering treatment; s7: and (5) carrying out signal recombination to obtain the Raman spectrum data after noise reduction.

The method comprises the following steps:

step S1: acquiring noisy acetaminophen raman spectrum data S, the signal length of which is L;

step S2: EMD-decomposing the noisy Raman spectrum data to obtain corresponding n-dimensional eigenmode components (IMFs) and one-dimensional decomposed residual components (RES), and marking each one-dimensional IMF as IMF _i Where i=1, 2,3 … n, RES is denoted IMF _n+1 ；

Step S3: calculating the permutation entropy P of each IMF and RES component using 1 _i Calculating window P by combining specific permutation entropy _i Setting a proper T _P Screening each calculated P _i When the permutation entropy is calculated, a parameter of a calculation window exists, and different calculation windows can obtain different permutation entropy value ranges, specifically PE epsilon [0, log ₂ D！]Where D is the window size. Since the permutation entropy range varies with the change of the calculation window of the permutation entropy, the decision threshold for the noisy component should also vary accordingly, and in this algorithm, this calculation window and permutation entropy threshold are fixed. Optionally, the window value is 5, and the permutation entropy threshold value is 1.3. If P _i ＞T _P Step S4, otherwise, IMF _i ”＝IMF _i Continuing to process the next IMF component;

step S4: searching for IMF _i All peak information P in _i,j Where j < L, combined with permutation entropy P of the current component _i Maximum peak value M from current component _i ＝max(P _i,j ) Calculated according to the method of 2Effective raman peak-to-peak value determination threshold R of current component _i ；

Step S5: the IMF soft threshold filtering denoising is used for eliminating high-frequency low-amplitude noise, which means that the filtered signal is meaningless noise signal with great grasp, meanwhile, the threshold calculation is relatively conservative because the part of the signal is directly eliminated, the effective Raman signal is prevented from being misjudged to be noise, and the following Savitzky-Golay filtering is used for compensating for the conservation strategy and inhibiting the noise signal with relatively high amplitude which possibly is released. The decomposition peak boundary processing specifically includes sequentially taking the adjacent peak information P _i,j And P _i,j+1 And R is as follows _i Make a comparison judgment and record P _i,j Corresponding signal subscript I _j And P _i,j+1 The method comprises the steps of carrying out a first treatment on the surface of the Corresponding signal subscript I _j+1 Three kinds of cases are treated:

a.|P _i,j |＞|P _i,j+1 |＞R _i or |P _i,j+1 |＞|P _i,j |＞R _i : at this time IMF _i From I _j To I _j+1 Directly reserving the range signal for the decomposition information of the effective raman peak;

b.|P _i,j |＞R _i ＞|P _i,j+1 i or |P _i,j+1 |＞R _i ＞|P _i,j I (L): at this time IMF _i From I _j To I _j+1 The decomposition information of partial effective Raman peak exists in the range of the (I) and the zero point subscript is found in the range by combining IMF structural characteristics _z 。|P _i,j |＞R _i ＞|P _i,j+1 When I, IMF is to be performed _i Middle I _j To I _z Setting a signal of a range to 0; p (P) _i,j+1 |＞R _i ＞|P _i,j When I, IMF is to be performed _i Middle I _z To I _j+1 Setting a signal of a range to 0;

c.R _i ＞|P _i,j |＞|P _i,j+1 i or R _i ＞|P _i,j+1 |＞|P _i,j |：At this time IMF _i From I _j To I _j+1 No decomposition information of effective Raman peak exists in the range of (1), and all the decomposition information is subjected to 0 setting treatment;

the processed IMF _i Recombined into IMF _i ' IMF of ₁ For example, IMF ₁ With IMF ₁ ' in contrast to FIG. 5, in removing IMF ₁ While the noise signals with high frequency and lower than the threshold value in the components, the boundary signals of the characteristic peaks are reserved;

step S6: for IMF _i Carrying out Savitzky-Golay filtering treatment of specific smoothing parameters with fitting window of 5 and fitting order of 2 to obtain IMF _i ", in IMF ₁ For example, IMF ₁ With IMF ₁ "compared with FIG. 6, the method shows the data change before and after single IMF component processing, introduces effective Raman characteristic peaks after Savitzky-Golay filter fitting decomposition, can effectively inhibit high-frequency but higher-amplitude noise signals released in step S5, and improves the signal smoothness of the characteristic peaks and non-effective signal transition areas;

step S7: all IMFs are processed _i "performing recombination according to formula 3 to obtain noise-reduced raman spectrum data S';

the IMF noise content judgment after EMD decomposition is divided by using the arrangement entropy of IMFs.

The effective Raman peak signal judgment of the noise-containing IMF is calculated and judged by integrating the arrangement entropy of the IMF and the peak value of the IMF.

And removing non-effective Raman characteristic peak signals in the IMF, and fitting and retaining the integrated signal decomposition filtering noise and the effective Raman characteristic peak signals.

The method comprises the steps of judging effective Raman characteristic peak signals of decomposed signals and adopting corresponding processing methods for different signals so as to realize the noise reduction of the parameter-free and high-characteristic peak information of Raman spectrum; no reference means that no additional setting of any parameters is required when the algorithm is used; the high characteristic peak information means that the influence on the effective Raman characteristic peak is small while noise is reduced, and the characteristic peak information can be effectively reserved.

Empirical Mode Decomposition (EMD): is a signal processing technique for decomposing nonlinear and non-stationary signals into a series of vibration modes called eigenmode components (Intrins ic Mode Funct ions, IMF for short). The EMD method does not need prior assumption or model, and can adapt to noise reduction requirements of different types and nonlinear and non-stationary signals. The EMD decomposes the signal into local vibration modes, which is helpful for processing and analyzing the local characteristics, and the decomposed components can better retain the time domain and frequency domain characteristics of the signal, thereby avoiding artifacts and distortion possibly introduced in the traditional method. However, in some cases, the EMD may have a modal aliasing phenomenon, that is, a specific signal is decomposed into two or more IMFs, or one IMF shows a plurality of different time-domain and frequency-domain characteristics, which are difficult to clearly distinguish and process.

Permutation entropy (Permutat ion Entropy): is a method for analyzing nonlinear dynamics of a time series. By comparing all D ≡ for each successive value of D in the sequential sequence! The seed arrangement is calculated and can be used to describe the complexity in the sequential sequence, especially in the case of dynamics or observation noise.

Savi tzky-Golay filtering: the main idea is to implement smooth filtering with least squares polynomial fitting. The principle is that polynomial fitting is performed in sliding windows, and a smoothing effect is achieved by calculating polynomial fitting coefficients in each window. These coefficients are determined by the least squares method such that the square error of the fitting polynomial from the original data within the window is minimized. The data within the window is then weighted averaged using the coefficients to obtain a filtered value.

The working principle of the application is as follows: different noise reduction parameters are required to be set for different Raman spectra; the raman peak caused by noise reduction is distorted, so that effective information is lost or wrong; EMD decomposition noise reduction Raman peak decomposition information loss due to modal aliasing on the Raman spectrum. The application has simple use and does not need to carry out different noise reduction treatments on different Raman spectrums; the problem of loss of effective Raman peak information caused by modal aliasing is avoided, and the advantages are shown by comparing the existing EMD decomposition screening noise reduction with the noise reduction result of the method of the application in FIG. 2; the method has the advantages that the details of the Raman peak, including peak value, half-peak width, peak center wavelength and the like, are well maintained while noise is reduced, and the Savi tzky-Golay filtering noise reduction and the noise reduction result of the method of the application are compared in FIG. 2.

Finally, it should be noted that: the foregoing description is only a preferred embodiment of the present application, and the present application is not limited thereto, but it is to be understood that modifications and equivalents of some of the technical features described in the foregoing embodiments may be made by those skilled in the art, although the present application has been described in detail with reference to the foregoing embodiments. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application should be included in the protection scope of the present application.

Claims (8)

1. The Raman spectrum noise reduction method based on EMD is characterized by comprising the following steps of:

step S1: acquiring Raman spectrum data containing noise;

step S2: carrying out empirical mode decomposition on the noisy Raman spectrum data;

step S3: judging the IMF noise content decomposed by EMD by using the arrangement entropy of IMF;

step S4: detecting an n-dimensional eigenmode component Raman peak signal;

step S5: soft threshold filtering denoising and peak decomposition boundary processing;

step S6: savitzky-Golay filtering;

step S7: and (5) carrying out signal recombination to obtain the Raman spectrum data after noise reduction.

2. The EMD-based raman spectrum noise reduction method according to claim 1, wherein the step S2 performs EMD decomposition on the noisy raman spectrum data to obtain corresponding n-dimensional eigenmode components and one-dimensional decomposed residual components, and records each one-dimensional eigenmode componentIs IMF _i Where i=1, 2,3 … n, the one-dimensional decomposition residual component is denoted IMF _n+1 。

3. The EMD-based raman spectral noise reduction method according to claim 2, wherein the step S3 calculates an arrangement entropy P of each one-dimensional eigenmode component and one-dimensional decomposition residual component using equation 1 _i Calculating window P by combining specific permutation entropy _i Setting a proper T _P Screening and calculating to obtain each P _i If P _i ＞T _P Step S4, otherwise, IMF _i ”＝IMF _i Continuing to process the next IMF component;

4. the EMD-based raman spectrum noise reduction method according to claim 2, wherein the step S4 comprises: searching for IMF _i All peak information P in _i,j Where j < L, combined with permutation entropy P of the current component _i Maximum peak value M from current component _i ＝max(P _i,j ) Calculating an effective Raman peak-to-peak value judgment threshold R of the current component according to the method of 2 _i ；

5. The EMD-based raman spectrum noise reduction method according to claim 2, wherein the step S5 comprises: sequentially taking adjacent peak information P _i,j And P _i,j+1 And R is as follows _i Make a comparison judgment and record P _i,j Corresponding signal subscript I _j And P _i,j+1 The method comprises the steps of carrying out a first treatment on the surface of the And the processed IMF _i Recombined into IMF _i '。

6. The EMD-based raman spectrum noise reduction method according to claim 2, wherein the step S6 comprises: for IMF _i Carrying out Savitzky-Golay filtering treatment of specific smoothing parameters with fitting window of 5 and fitting order of 2 to obtain IMF _i ”。

7. A method of EMD-based raman spectroscopy noise reduction according to claim 3, wherein step S7 is performed by combining all IMFs _i "performing recombination according to formula 3 to obtain noise-reduced raman spectrum data S';

8. the EMD-based raman spectrum noise reduction method of claim 5, wherein: the signal index I corresponding to the step S5 _j+1 Three kinds of cases are treated:

a.|P _i,j |＞|P _i,j+1 |＞R _i or |P _i,j+1 |＞|P _i,j |＞R _i : at this time IMF _i From I _j To I _j+1 Directly reserving the range signal for the decomposition information of the effective raman peak;

b.|P _i,j |＞R _i ＞|P _i,j+1 i or |P _i,j+1 |＞R _i ＞|P _i,j I (L): at this time IMF _i From I _j To I _j+1 The decomposition information of partial effective Raman peak exists in the range of the (I) and the zero point subscript is found in the range by combining IMF structural characteristics _z ；|P _i,j |＞R _i ＞|P _i,j+1 When I, IMF is to be performed _i Middle I _j To I _z Setting a signal of a range to 0; p _i,j+1 |＞R _i ＞|P _i,j When I, IMF is to be performed _i Middle I _z To I _j+1 Setting a signal of a range to 0;

c.R _i ＞|P _i,j |＞|P _i,j+1 i or R _i ＞|P _i,j+1 |＞|P _i,j I (L): at this time IMF _i From I _j To I _j+1 No decomposition information of effective raman peak exists in the range of (2), and all the decomposition information is subjected to 0 setting treatment.