WO2020232959A1

WO2020232959A1 - Near infrared spectral feature extraction method and system based on functional principal component analysis

Info

Publication number: WO2020232959A1
Application number: PCT/CN2019/111602
Authority: WO
Inventors: 潘天红; 李浩然; 陈山; 邹小波
Original assignee: 安徽大学
Priority date: 2019-05-22
Filing date: 2019-10-17
Publication date: 2020-11-26
Also published as: CN110006844A

Abstract

A near infrared spectral feature extraction method based on functional principal component analysis, and an extraction system thereof. The extraction method comprises steps of: S1, acquiring near infrared spectrum data in multiple samples; S2, preprocessing the near infrared spectrum data using standard normal transformation; S3, obtaining a spline function of the processed near infrared spectrum data; S4, centralizing the spline function; S5, calculating the covariance of the centralized spline function in different waveband functions; S6, calculating the j-th feature value of the covariance; S7, calculating cumulative contribution; and S8, calculating scores of functional principal components in equations of different wavebands.

Description

Near-infrared spectrum feature extraction method and system based on functional principal component analysis

Technical field

The invention relates to the technical field of near-infrared spectroscopy non-destructive analysis, and more specifically to a near-infrared spectroscopy feature extraction method and system based on functional principal component analysis.

Background technique

Because near-infrared light has good transmission characteristics in conventional optical fibers, and its instruments are simple, fast analysis, non-destructive and small sample preparation, it is almost suitable for all kinds of samples (liquid, viscous, coating, powder and Solid) analysis, multi-component multi-channel simultaneous determination, etc., have been widely used in many fields including agriculture and animal husbandry, food, chemical industry, petrochemical, pharmaceutical, tobacco, etc., providing a great opportunity for scientific research, teaching and production process control. Broad use space.

Near-infrared spectroscopy is mainly due to the non-resonance of molecular vibration when the molecular vibration transitions from the ground state to the higher energy level, and the recording is mainly the frequency double and combined frequency absorption of the hydrogen-containing group XH (X=C, N, O) vibration . Different groups (such as methyl, methylene, benzene ring, etc.) or the same group have obvious differences in the near-infrared absorption wavelength and intensity in different chemical environments. Near-infrared spectroscopy has rich structure and composition information, which is very suitable It is used to measure the composition and properties of hydrocarbon organic substances. Material quality parameters (such as component content) are also related to their composition and structure information. The application of chemometric methods to correlate the two can determine the qualitative or The quantitative relationship is: the calibration model. After the calibration model is established, as long as the near-infrared spectrum of the unknown sample is measured, the quality parameters of the sample can be predicted based on the calibration model.

However, the near-infrared spectroscopy data has the characteristics of high dimensionality and band overlap, which brings a certain degree of difficulty and challenge to extracting the key principal component information of the sample. How to realize the feature mapping relationship from high-dimensional space to low-dimensional space so as to facilitate the extraction of the key principal component information of sample spectral data is a technical problem to be solved urgently. In recent years, in order to solve the problem of high-dimensional spectral data dimensionality reduction, a large number of dimensionality reduction algorithms have emerged at home and abroad, such as: principal component analysis (PCA), linear discriminant analysis (LDA), genetics Algorithm (Genetic Algorithm, GA), Uniformative Variable Elimination (UVE), Interval Partial Least Squares (iPLS), Successive Projections Algorithm (SPA), etc. . The above methods have their own characteristics, but they also have their own shortcomings. For example, the principal component analysis is based on the linear statistical method. When solving the problems of nonlinear correlation and correcting uneven sample distribution, the results are often unreliable; genetic algorithms use random In the evolutionary method, the selection, crossover and mutation operators are often based on experience, and the parameter tuning process is relatively cumbersome. In addition, the selection of its fitness function is also very important. Different fitness functions will have quite different results.

However, the dimensionality reduction algorithm commonly used in the prior art only starts from the spectral data itself, that is, the discrete points of the spectral data, to realize feature mapping from a high-dimensional space to a low-dimensional space. In fact, the internal structure of spectral data presents a "functional type", which is continuous. However, the dimensionality reduction algorithm in the prior art will result in a lot of potential feature information, such as derivative, order and other feature information, which cannot be mined.

Summary of the invention

The technical problem to be solved by the present invention is to provide a near-infrared spectrum feature extraction method and system based on functional principal component analysis to solve the problem that the prior art dimensionality reduction algorithm in the background art cannot obtain feature information such as derivative and order.

In order to solve the above technical problems, the present invention provides the following technical solutions:

A feature extraction method of near infrared spectroscopy based on functional principal component analysis, including the following steps:

S1. Collect near-infrared spectroscopy data in various samples;

S2. Preprocessing the near-infrared spectroscopy data by using standard normal transformation;

S3. Obtain the spline function of the processed near-infrared spectrum data;

S4. Perform centralization processing on the spline function;

S5. Calculate the covariance of the centered spline function between different waveband functions;

S6. Calculate the j-th eigenvalue of the covariance;

S7. Calculate the cumulative contribution degree through the characteristic value, and the principal element whose contribution degree exceeds the threshold is used as the characteristic value of the near-infrared spectrum;

S8. Calculate the scores of the functional principal components in the equations of different bands according to the eigenvalues.

Through this method, the feature mapping from high dimensionality to low latitude is completed, and at the same time, further information such as the order and derivative of the intrinsic function of the spectral data can be mined.

As a further solution of the present invention: the step S1 includes:

Collect near-infrared spectroscopy data of the sample to be tested, and determine the content of the main nutrients through physical and chemical tests; the nutrients include: protein, fat and a variety of amino acids.

The spectrometer adopts the NIRQuest512 near-infrared spectrometer produced by Ocean Optics in the United States. It is equipped with HL-2000 series halogen lamp light source with a wavelength range of 360nm-2000nm. The resolution of the spectrometer is 3cm ^-1 , the integration time is 45s, and the scanning wavelength range is 900 -1700nm, built-in Hamamatsu indium gallium arsenide (InGaAs) array detector with 512 pixels, high stability, 32 scans;

The collected samples to be tested are several dry slices of wild matsutake, Agaricus blazei, old man's head, and Pleurotus eryngii, and spectral sampling is performed on the dry slices.

As a further solution of the present invention: the step S3 includes:

S31. Obtain the B-spline function of the near-infrared spectrum data of each sample by using a formula, the formula is as follows:

Among them, φ _k (t) is the k-th B-spline basis function of the near-infrared spectrum band, k is less than or equal to m, m represents the number of B-spline basis functions, C is the coefficient matrix, and X(t) is the near-infrared spectrum The function form of the data, t is the band of the near-infrared spectrum, and ∑ represents the summation function;

S32. Use a formula to smooth the X(t) function. The formula is as follows:

PEN ₂ (X)＝∫[DDX(t)] ² dt (2)

Among them, PEN ₂ (X) represents the rough penalty, and DDX(t) represents the second derivative of the function X(t);

S33. Calculate the coefficient matrix C of the near-infrared spectrum data function using a formula; the formula is as follows:

PENSSE _λ =SMSSE(x|c)+γPEN ₂ (X); (3)

Among them, x is the observation data of the j-th near-infrared spectrum, x _j is the observation data of the j-th near-infrared spectrum, j is a positive integer less than m, and SMESS(x|c) represents the sum of squared residuals to minimize; t _j represents the j-th near-infrared spectrum band, k≤K≤j, φ _k (t _j ) represents the B-spline basis function of the j-th near-infrared spectrum band;

Through this method, the coefficient matrix C is estimated, and the rough penalty is used to smooth the function, which effectively avoids the phenomenon of over-fitting.

As a further solution of the present invention:

The step S4 includes:

Use formulas to centralize the sample data. The centralization formula is as follows:

In the formula, i is the sample number, n is the total number of samples,

Is the mean value of the NIR spectral band function of n samples, X _i (t) is the function of the i-th NIR spectral band t,

Is a function of the near-infrared spectrum band t of the i-th sample after the centralization process, c represents centralization, and st represents the condition function;

Through centralized processing, the difference between the near-infrared spectrum data of each sample is eliminated, and the accuracy of the method is improved.

As a further solution of the present invention: the step S5 includes:

Use the formula to calculate the covariance, the covariance calculation formula is as follows:

Choose a band different from t arbitrarily and record it as s; V(s, t) represents the covariance of two different bands of s and t.

As a further solution of the present invention: the step S6 includes:

Use the formula to calculate the j-th eigenvalue of the covariance; the calculation formula of the eigenvalue is as follows:

Where ξ _j (t) is the pivot weight function of the j-th band t, ξ _j (s) represents the pivot weight function of the j-th band s, j is a positive integer, ρ _j is the eigenvalue, and st is the condition Function, from equation (7) we can see that the principal component functions ξ ₁ (t), ξ ₂ (t),..., ξ _j (t) are not related to each other.

As a further solution of the present invention: the step S7 includes:

Calculate cumulative contribution

Select the M principal elements whose contribution degree exceeds the threshold value as the characteristic values of the near-infrared spectrum band, construct a quantitative model, and complete the qualitative/quantitative analysis of the sample to be tested;

Among them, M represents the number of pivots,

The threshold is set to 90%.

As a further solution of the present invention: the step S8 includes:

Use formulas to calculate the function-shaped principal component scores in the different near-infrared spectrum band equations after the centralization process, and the formula for the function-shaped principal component scores is as follows:

Where f _j is a function

The j-th pivot.

The present invention also provides a system adopting the near-infrared spectrum feature extraction method based on functional principal component analysis according to any one of the above solutions, including:

Acquisition module, used to collect near-infrared spectrum data of various samples;

A preprocessing module for preprocessing the near-infrared spectrum data using standard normal transformation;

The acquisition module is used to acquire the spline function of the processed near-infrared spectrum data;

Centralized processing module, used to centralize the spline function;

The covariance module is used to calculate the covariance of the centered spline function between different band functions;

The eigenvalue module is used to calculate the jth eigenvalue of the covariance;

The contribution degree module is used to calculate the cumulative contribution degree through the characteristic value of the covariance, and the principal element whose contribution degree exceeds the threshold is used as the characteristic value of the near-infrared spectrum band;

The principal component score module is used to use the characteristic values of the near-infrared spectrum bands to calculate the functional principal component scores in the equations of different bands.

Compared with the prior art, the beneficial effects of the present invention are:

1. The present invention treats near-infrared spectroscopy data as a continuous function and utilizes full-band information. While accurately extracting band features with effective information, the feature values are obtained through the covariance of different band functions, and through the covariance features Calculate the contribution degree of the value to obtain the characteristic value of the near-infrared spectrum band, thereby obtaining the function-shaped principal component scores in the equations of different bands, and realize further mining of the order and derivative (rate of change, slope, curvature, etc.) information of the intrinsic function of the spectrum data;

2. The present invention obtains the function-shaped principal component scores in the equations of different bands, and at the same time obtains the rate of change, slope, curvature and other information, thereby enhancing the robustness of the calibration model and improving the predictive ability of the calibration model. Infrared spectrum data provides a new feature extraction method, which has high practical value.

Description of the drawings

In order to explain the technical solutions in the embodiments of the present invention more clearly, the following will briefly introduce the drawings that need to be used in the description of the embodiments or the prior art. Obviously, the drawings in the following description are only of the present invention. Some examples.

FIG. 1 is a schematic flowchart of a near-infrared spectrum feature extraction method based on functional principal component analysis according to Embodiment 1 of the present invention.

2 is a schematic diagram of B-spline basis functions in a near-infrared spectrum feature extraction method based on functional principal component analysis according to Embodiment 1 of the present invention;

3 is a schematic diagram of the functional description of the near-infrared spectra of edible fungi in a near-infrared spectrum feature extraction method based on functional principal component analysis according to Embodiment 1 of the present invention;

4 is a load distribution diagram of functional principal component analysis of edible fungi in a near-infrared spectral feature extraction method based on functional principal component analysis according to Embodiment 1 of the present invention;

FIG. 5 is a diagram of two principal component analysis results of four kinds of edible fungi spectral data in a near-infrared spectral feature extraction method based on functional principal component analysis according to Embodiment 1 of the present invention.

Detailed ways

In order to make the technical problems, technical solutions and beneficial effects to be solved by the present invention clearer, the following further describes the present invention in detail with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described here are only used to explain the present invention, but not to limit the present invention.

A feature extraction method of near-infrared spectroscopy based on functional principal component analysis. This embodiment uses near-infrared spectroscopy analysis to distinguish different types of edible fungi, and obtain orders and derivatives (rate of change, slope, curvature, etc.) Information; Of course, the present invention is not limited to screening the types of edible fungi. The steps are as follows:

S1. Near-infrared spectrum data of several samples;

Collect the near-infrared spectroscopy data of the sample to be tested, and determine the content of the main nutrients through physical and chemical tests. The nutrient ingredients include: protein, fat and a variety of amino acids; the spectrometer adopts the NIRQuest512 near-infrared spectrometer produced by Ocean Optics in the United States, with wavelength The range of HL-2000 series halogen tungsten light source is 360nm-2000nm, the resolution of the spectrometer is 3cm ^-1 , the integration time is 45s, the scanning wavelength range is: 900-1700nm, built-in Hamamatsu with 512 pixels and high stability Indium gallium arsenide (InGaAs) array detector with 32 scans;

Among them, the collected samples to be tested are wild matsutake, Agaricus blazei, old man's head, and Pleurotus eryngii, a total of 166 dry slices, and spectrum sampling is performed on 166 dry slices;

S2, preprocessing the near-infrared spectrum data of several samples;

Preprocess the collected spectrum data using standard normal transformation to eliminate the influence of noise and baseband drift;

S3. Perform functional description of the preprocessed near-infrared spectrum data; including:

S31. As shown in Fig. 2, Fig. 2 is a schematic diagram of the B-spline basis function in a near-infrared spectrum feature extraction method based on functional principal component analysis according to an embodiment of the present invention. The B-spline function is a seventh-order spline function. 21 basis functions, using seven B-spline functions, and using formulas to describe the near-infrared spectrum data of each sample as a function, the formula is as follows:

In the formula, φ _k (t) is the k-th B-spline basis function of the near-infrared spectral band, k is less than or equal to m, and m represents the number of corresponding B-spline basis functions, C is the coefficient matrix, and X(t) is the near The function form of the infrared spectrum data, t is the band of the near-infrared spectrum, ∑ represents the summation function, so that the near-infrared spectrum data can use a set of selected basis functions φ _k (t), using the basis function φ _k (t ) Linear combination description;

S32. Use the formula to smooth the X(t) function. Mathematically, the smoothness of the spline function can be described by the second derivative of the function. In this step, DDX(t) is used to represent the second derivative of the function X(t). The rough penalty formula is as follows:

PEN ₂ (X)＝∫[DDX(t)] ² dt (2)

PENSSE _λ =SMSSE(x|c)+γPEN ₂ (X) (3)

Among them, x _j is the observation data of the j-th near-infrared spectrum, SMESS(x|c) represents the sum of squared residuals of the minimization; t _j represents the band of the j-th near-infrared spectrum, k≤K≤j, φ _k (t _j ) is expressed as the B-spline basis function of the j-th near-infrared spectral band;

S33. Equation (4) can be estimated by the least square method, but the least square method estimation is susceptible to noise and over-fitting. Therefore, the rough penalty is used to smooth the function, that is, the integral of the second derivative square is used to control the function The smoothness of the curve, then, the residual sum of squares and the roughness penalty are combined to estimate the coefficient matrix C, where γ is the smoothness coefficient, the larger the γ, the closer the fitting is to the straight line, and the fitting is given For some points in space, find a continuous surface with known form and unknown parameters to approximate these points as much as possible;

S4. As shown in FIG. 3, FIG. 3 is a schematic diagram of functional description of the near-infrared spectrum of edible fungi in a method for extracting near-infrared spectrum features based on functional principal component analysis according to an embodiment of the present invention. The difference between near-infrared spectroscopy data requires centralized processing of the functions of near-infrared spectroscopy data. The formula for centralized processing is as follows:

In the formula, i is the sample number, n is the total number of samples,

S5. Find the covariance between the different band functions in all samples, choose a band different from t arbitrarily, and record it as s, as follows:

V(s,t) represents the covariance between two different band functions,

Represents the function of the near-infrared spectrum band s of the i-th sample after the centralized processing;

S6. Calculate the j-th eigenvalue of the covariance, and the calculation formula is as follows:

Where ξ _j (t) is the j-th pivot weight function of band t, ξ _j (s) is the j-th pivot weight function of band s, j is a positive integer, ρ _j is the eigenvalue, and st is the condition Function, from equation (7) we can see that the principal component functions ξ ₁ (t), ξ ₂ (t),..., ξ _j (t) are not correlated with each other;

S7. Calculate the cumulative contribution from the eigenvalue ρ _j corresponding to the pivot weight function

The M principal components whose contribution degree exceeds the threshold are selected as the characteristic values of the near-infrared spectrum band, and M represents the number of principal components; as shown in FIG. 4, FIG. 4 is a functional principal component analysis based on the embodiment of the present invention. The load distribution diagram of the functional principal component analysis of edible fungi in the near-infrared spectroscopy feature extraction method. In this embodiment, M=2, that is, 2 principal components can display all the contributions, and then construct a quantitative model to realize the test Qualitative/quantitative analysis of samples;

Wherein, the threshold in this embodiment is set to 90%;

S8. According to the characteristic value, calculate the function-shaped principal component scores in the equations after the centralization processing corresponding to different near-infrared spectral bands to realize the analysis of the spectral data; the function-shaped principal component score formula is as follows:

Where f _j is a function

According to the calculation process of formula (7), the variance of each weight function satisfies var(f ₁ )＞var(f ₂ )＞…＞var(f _j ), var represents the variance of random variables, c Represents centralization;

As shown in FIG. 5, FIG. 5 is a diagram of two principal component analysis results of four kinds of edible fungi spectral data in a near-infrared spectrum feature extraction method based on functional principal component analysis provided by an embodiment of the present invention. , Use 2 principal elements to classify the four edible fungi, and obtain the order and derivative of the intrinsic function of the spectral data. The derivative information includes the rate of change, slope, curve, etc. The four edible fungi are wild Matsutake, Ji For matsutake, old man's head, and Pleurotus eryngii, the abscissa in Figure 5 represents the first pivot and the ordinate represents the second pivot.

Example 2

A near-infrared spectrum feature extraction system based on functional principal component analysis, including:

Centralized processing module, used to centralize the spline function;

The principal component score module is used to use the characteristic values of the near-infrared spectrum bands to calculate the functional principal component scores in the equations of different bands. In the description of the present invention, unless otherwise clearly specified and limited, the terms "installed", "connected", and "connected" should be interpreted broadly. For example, they may be fixedly connected, detachably connected, or integrally connected. Connection; it can be a mechanical connection or an electrical connection; it can be directly connected, or indirectly connected through an intermediate medium, and it can be the internal communication between two components. For those of ordinary skill in the art, the specific meaning of the above-mentioned terms in the present invention can be understood in specific situations.

The above descriptions are only preferred embodiments of the present invention and are not intended to limit the present invention. Any modification, equivalent replacement and improvement made within the spirit and principle of the present invention shall be included in the protection of the present invention. Within range.

Claims

A feature extraction method for near infrared spectroscopy based on functional principal component analysis, which is characterized in that the steps are as follows:

S1. Collect near-infrared spectroscopy data in various samples;

S2. Preprocessing the near-infrared spectroscopy data by using standard normal transformation;

S3. Obtain the spline function of the processed near-infrared spectrum data;

S4. Perform centralization processing on the spline function;

S5. Calculate the covariance of the centered spline function between different waveband functions;

S6. Calculate the j-th eigenvalue of the covariance;

S7. Calculate the cumulative contribution degree through the characteristic value of the covariance, and the principal element whose contribution degree exceeds the threshold is used as the characteristic value of the near infrared spectrum band;

S8. Using the characteristic values of the near-infrared spectrum bands, calculate the functional principal component scores in the equations of different bands.
The method for extracting near-infrared spectroscopy features based on functional principal component analysis according to claim 1, characterized in that, in the step S1, near-infrared spectroscopy data of the sample to be tested is collected, and the content of nutrients is determined through physical and chemical tests ; Nutritional ingredients include: protein, fat and a variety of amino acids.
The near-infrared spectroscopy feature extraction method based on functional principal component analysis according to claim 2, characterized in that the collected samples to be tested are respectively several dried slices of wild matsutake, Agaricus blazei, old man's head, and Pleurotus eryngii, And perform spectral sampling on the dry slice sample.
The near-infrared spectrum feature extraction method based on functional principal component analysis according to claim 1, wherein the step S3 comprises:

S31. Obtain the B-spline function of the near-infrared spectrum data of each sample by using a formula, the formula is as follows:

Among them, φ k (t) is the k-th B-spline basis function of the near-infrared spectrum band, k is less than or equal to m, m represents the number of B-spline basis functions, C is the coefficient matrix, and X(t) is the near-infrared spectrum The function form of the data, t is the band of the near-infrared spectrum, and ∑ represents the summation function;

S32. Use a formula to smooth the X(t) function. The formula is as follows:

PEN 2 (X)＝∫[DDX(t)] 2 dt (2)

Among them, PEN 2 (X) represents the rough penalty, and DDX(t) represents the second derivative of the function X(t);

S33. Calculate the coefficient matrix C of the near-infrared spectrum data function using a formula; the formula is as follows:

PENSSE λ =SMSSE(x|c)+γPEN 2 (X); (3)

Among them, PENSSE λ represents the sum of the residual sum of squares and the rough penalty, and γ is the smoothing coefficient;

Among them, x is the observation data of the near-infrared spectrum, x j is the observation data of the j-th near-infrared spectrum, j is a positive integer less than m, SMESS(x|c) represents the sum of squares of the minimized residuals; t j represents The j-th near-infrared spectrum band, k≤K≤j, φ k (t j ) is expressed as the B-spline basis function of the j-th near-infrared spectrum band.
The near-infrared spectrum feature extraction method based on functional principal component analysis according to claim 1, wherein said step S4 comprises:

Use the formula to centralize the sample data, the formula is as follows:

In the formula, i is the sample number, n is the total number of samples,
Is the mean value of the NIR spectral band function of n samples, X i (t) is the function of the i-th NIR spectral band t,
It is a function of the near-infrared spectrum band t of the i-th sample after the centering process, c means centering, and st means conditional function.
The near-infrared spectrum feature extraction method based on functional principal component analysis according to claim 5, wherein said step S5 comprises:

The calculation formula of the covariance is as follows:

Among them, s represents the band of the near-infrared spectrum that is different from t; X i c (s) represents the function of the near-infrared spectrum band s of the i-th sample after the centering process, and V(s, t) represents s, t two The covariance of two bands.
The near-infrared spectrum feature extraction method based on functional principal component analysis according to claim 6, wherein the step S6 comprises:

Use the formula to calculate the j-th eigenvalue of the covariance; the formula is as follows:

Among them, ξ j (t) is the pivot weight function of the j-th band t, ξ j (s) represents the pivot weight function of the j-th band s, j is a positive integer, ρ j is the eigenvalue, and st is the condition function .
The near-infrared spectroscopy feature extraction method based on functional principal component analysis according to claim 7, wherein said step S7 comprises:

The formula for cumulative contribution is
Select the M principal elements whose contribution degree exceeds the threshold value as the characteristic value of the near-infrared spectrum band, construct a quantitative model, and complete the qualitative or quantitative analysis of the sample to be tested;

Among them, M represents the number of pivots.
The near-infrared spectrum feature extraction method based on functional principal component analysis according to claim 7, characterized in that,

The step S8 includes:

Use the formula to calculate the function-shaped principal component scores in the different near-infrared spectrum band equations after the centralization process, the formula is as follows:

Where f j is a function
The j-th pivot.
An extraction system using the near-infrared spectral feature extraction method based on functional principal component analysis according to any one of claims 1-9, characterized in that it comprises:

Acquisition module, used to collect near-infrared spectrum data of various samples;

A preprocessing module for preprocessing the near-infrared spectrum data using standard normal transformation;

The acquisition module is used to acquire the spline function of the processed near-infrared spectrum data;

Centralized processing module, used to centralize the spline function;

The covariance module is used to calculate the covariance of the centered spline function between different band functions;

The eigenvalue module is used to calculate the jth eigenvalue of the covariance;

The contribution degree module is used to calculate the cumulative contribution degree through the characteristic value of the covariance, and the principal element whose contribution degree exceeds the threshold is used as the characteristic value of the near-infrared spectrum band;

The principal component score module is used to use the characteristic values of the near-infrared spectrum bands to calculate the functional principal component scores in the equations of different bands.