CN112966735B - A supervised multi-set correlation feature fusion method based on spectral reconstruction - Google Patents

Abstract

The invention discloses a method for fusing relevant features of a supervision multi-set based on spectrum reconstruction, which comprises the following steps of 1) defining the projection direction of a training sample set; 2) Calculating inter-group intra-class correlation matrix and auto-covariance matrix of the training samples; 3) Singular value decomposition is carried out on the inter-group intra-class correlation matrix, and eigenvalue decomposition is carried out on the auto-covariance matrix; 4) Reconstructing a fractional inter-class correlation matrix and a fractional auto-covariance matrix; 5) Constructing an optimization model of FDMCCA; 6) Solving a feature vector matrix to form a projection matrix; 7) Fusing the feature after dimension reduction; 8) And selecting different numbers of images to respectively serve as a training set and a testing set, and calculating the recognition rate. The invention can effectively solve the information fusion problem of multiple view data, and meanwhile, the introduction of fractional order parameters weakens the influence caused by noise interference and limited training samples, and improves the accuracy of system identification.

Description

A supervised multi-set correlation feature fusion method based on spectral reconstruction

Technical field

The invention relates to the field of pattern recognition, and in particular to a supervised multi-set related feature fusion method based on spectral reconstruction.

Background technique

Canonical correlation analysis (CCA) studies the linear correlation between two sets of data. CCA can linearly project two sets of random variables into a low-dimensional subspace with maximum correlation. Researchers use CCA to simultaneously reduce the dimensions of two sets of feature vectors (i.e., two views) to obtain two low-dimensional feature representations, and then effectively fuse these two low-dimensional feature representations to form discriminative features, thereby improving the model classification accuracy. Because the CCA method is simple and effective, it has been widely used in blind source separation, computer vision and speech recognition.

Canonical correlation analysis is an unsupervised linear learning method. However, there are situations in real life where the dependency between two views cannot be simply expressed linearly. If there is a non-linear relationship between the two views, it is inappropriate to still use the CCA method in this case. The proposal of Kernel Canonical Correlation Analysis (KCCA) effectively solves nonlinear problems. KCCA is a nonlinear extension of CCA and has better results when dealing with simple nonlinear problems. When encountering more complex nonlinear problems, deep canonical correlation analysis (Deep CCA) can better solve such problems. Deep CCA combines deep neural networks with CCA to learn complex nonlinear relationships between two views of data. From another perspective of nonlinear expansion, the idea of locality can be incorporated into CCA, and the locality-preserving canonical correlation analysis (LPCCA) method came into being. LPCCA can discover the local manifold structure of each view data to visualize the data.

Although CCA has good recognition results in some pattern recognition problems, it is an unsupervised learning method and does not make full use of class label information. This not only causes a waste of resources, but also greatly reduces the recognition effect. To solve this problem, researchers considered the inter-class and intra-class information of samples and proposed discriminant canonical correlation analysis (DCCA). The DCCA method maximizes the correlation between features of samples of the same category and minimizes the correlation between features of samples of different categories, which can improve the accuracy of pattern classification.

The above methods are all suitable for analyzing the relationship between two views. When there are three or more views, the application of the above methods has limitations. The multiset canonical correlation analysis (MCCA) method is a multi-view extension of the CCA method. MCCA not only retains the characteristic of maximum correlation between views of CCA, but also makes up for the shortcoming of CCA that cannot be applied to multiple views, improving the recognition performance of the CCA method. Researchers have combined MCCA with DCCA and have proposed discriminant multiset canonical correlation analysis (DMCCA). Experiments have proven that this method has good recognition performance in face recognition, handwritten digit recognition, emotion recognition, etc.

When there is noise interference or there are few training samples, the auto-covariance matrix and cross-covariance matrix in CCA will deviate from the true value, resulting in poor final recognition effect. In order to solve this problem, the researchers combined fractional-order ideas with CCA, reconstructed the auto-covariance matrix and cross-covariance matrix by introducing fractional-order parameters, and proposed a canonical correlation analysis of fractional-order embedding, thereby weakening this deviation band. The influence of this method improves the recognition performance of the method.

Traditional canonical correlation analysis mainly studies the correlation between two views. It is an unsupervised learning method that does not consider class label information and cannot directly handle high-dimensional data with more than two views.

Contents of the invention

The purpose of this invention is to overcome the shortcomings of the existing technology and provide a supervised multi-set correlation feature fusion method (FDMCCA) based on spectral reconstruction, which can effectively handle the multi-view feature fusion problem. At the same time, the introduction of fractional-order parameters weakens the problem of noise interference and The impact of limited training samples improves the accuracy of system recognition.

The purpose of the present invention is achieved as follows: a supervised multi-set correlation feature fusion method based on spectral reconstruction, including the following steps:

Step 1) Assume that there are P groups of training samples, the mean of each group of samples is 0 and the number of categories is c, as follows:

in represents the k-th sample of the j-th category in the i-th group, m _i represents the feature dimension of the i-th group data set, n _j represents the number of j-th category samples, and defines the projection direction of the training sample set as/>

Step 2) Calculate the intra-class correlation matrix of training samples between groups and autocovariance matrix Among them/> Represents a matrix in which each element is 1;

Step 3) For the inter-group intra-class correlation matrix obtained in step 2) Perform singular value decomposition to obtain the left and right singular vector matrices and singular value matrices. Perform eigenvalue decomposition of the autocovariance matrix C _ii to obtain the eigenvector matrix and eigenvalue matrix;

Step 4) Select appropriate fractional order parameters α and β, reassign the singular value matrix and eigenvalue matrix obtained in step 3), and construct the intra-class correlation matrix between fractional order groups. and fractional-order autocovariance matrix/>

Step 5) Construct the optimization model of FDMCCA as in Introducing the Lagrange multiplier method, we obtain the generalized eigenvalue problem Eω = μFω, and find the projection direction ω, where μ is the eigenvalue,

Step 6) Considering that the autocovariance matrix may be a singular matrix, introduce the regularization parameter η based on step 5) and establish the optimization model under regularization as Introducing the Lagrange multiplier method, the following generalized eigenvalue problem is obtained:

in is the identity matrix of size m _i × m _i , i=1,2,…,P;

Step 7) Solve the eigenvectors corresponding to the first d largest eigenvalues according to the generalized eigenvalue problem in step 6), thereby forming the projection matrix _Wi for each set of data = [ω _i1 ,ω _i2 ,...,ω _id ], i =1,2,…,P, d≤min{m ₁ ,…,m _P };

Step 8) Use the projection matrix _Wi of each group of data to calculate the low-dimensional projection of each group of training samples and test samples respectively, and then use a serial feature fusion strategy to form the final fusion features for classification; and calculate the recognition rate.

Further, the intra-class correlation matrix between pairs of groups described in step 3) Doing singular value decomposition and autocovariance matrix C _ii to do eigenvalue decomposition includes the following steps:

Step 3-1) Compute the intra-class correlation matrix between groups Do singular value decomposition:

in and/> They are/> The left and right singular vector matrices of ,/> Yes/> A diagonal matrix composed of singular values, and/>

Step 3-2) Perform eigenvalue decomposition of the autocovariance matrix C _ii :

in is the eigenvector matrix of C _ii ,/> is a diagonal matrix composed of eigenvalues of C _ii , and r _i =rank(C _ii ).

Further, constructing the fractional inter-group intra-class correlation matrix as described in step 4) and fractional-order autocovariance matrix/> Contains the following steps:

Step 4-1) Assume α is a fraction and satisfies 0≤α≤1, define the intra-class correlation matrix between fractional-order groups for:

in U _ij and V _ij and r _ij are defined in step 3-1).

Step 4-2) Assume that β is a fraction and satisfies 0≤β≤1, define the fractional autocovariance matrix for:

in The definitions of Q _i and r _i are given in step 3-2).

Compared with the existing technology, the beneficial effect of the present invention is that based on canonical correlation analysis, it combines fractional embedded canonical correlation analysis (FECCA) and discriminative multiset canonical correlation analysis (DMCCA), making full use of class labels information, can handle information fusion problems greater than two views, and can be applied to multi-view feature fusion. The introduction of fractional-order parameters weakens the impact of noise interference and limited training samples, improving the accuracy of face recognition; When the number of training samples is small, the present invention has a better recognition effect; it is suitable for dimensionality reduction and feature fusion of multiple views; because it carries class label information, among similar methods, the recognition effect of the present invention is better than other methods .

Description of drawings

Figure 1 is a flow chart of the present invention.

Figure 2 is a line chart of the present invention and other methods as the dimensions change.

Figure 3 is a diagram of the recognition rate of the present invention under different numbers of training samples.

Detailed ways

As shown in Figure 1, a supervised multi-set correlation feature fusion method based on spectral reconstruction is characterized by including the following steps:

Step 1) Assume that there are P groups of training samples, the mean of each group of samples is 0 and the number of categories is c, as follows

in represents the k-th sample of the j-th category in the i-th group, m _i represents the feature dimension of the i-th group data set, n _j represents the number of j-th category samples, and defines the projection direction of the training sample set as/>

Step 2) Calculate the intra-class correlation matrix of training samples between groups and autocovariance matrix Among them/> Represents a matrix in which each element is 1;

Step 3) For the inter-group intra-class correlation matrix obtained in step 2) Perform singular value decomposition to obtain the left and right singular vector matrices and singular value matrices. Perform eigenvalue decomposition of the autocovariance matrix C _ii to obtain the eigenvector matrix and eigenvalue matrix;

Step 3-1) Compute the intra-class correlation matrix between groups Do singular value decomposition:

in and/> They are/> The left and right singular vector matrices of ,/> Yes/> A diagonal matrix composed of singular values, and/>

Step 3-2) Perform eigenvalue decomposition of the autocovariance matrix C _ii :

in is the eigenvector matrix of C _ii ,/> is a diagonal matrix composed of eigenvalues of C _ii , and r _i =rank(C _ii ).

Step 4) Select appropriate fractional order parameters α and β, reassign the singular value matrix and eigenvalue matrix obtained in step 3), and construct the intra-class correlation matrix between fractional order groups. and fractional-order autocovariance matrix/>

Step 4-1) Assume α is a fraction and satisfies 0≤α≤1, define the intra-class correlation matrix between fractional-order groups for:

in U _ij and V _ij and r _ij are defined in step 3-1);

Step 4-2) Assume that β is a fraction and satisfies 0≤β≤1, define the fractional autocovariance matrix for:

in The definitions of Q _i and r _i are given in step 3-2).

Step 5) Construct the optimization model of FDMCCA as in Introducing the Lagrange multiplier method, we obtain the generalized eigenvalue problem Eω = μFω, and find the projection direction ω, where μ is the eigenvalue,

Step 6) Considering that the autocovariance matrix may be a singular matrix, introduce the regularization parameter η based on step 5) and establish the optimization model under regularization as Introducing the Lagrange multiplier method, the following generalized eigenvalue problem is obtained:

in is the identity matrix of size m _i × m _i , i=1,2,…,P;

Step 7) Solve the eigenvectors corresponding to the first d largest eigenvalues according to the generalized eigenvalue problem in step 6), thereby forming the projection matrix _Wi for each set of data = [ω _i1 ,ω _i2 ,...,ω _id ], i =1,2,…,P, d≤min{m ₁ ,…,m _P };

Step 8) Use the projection matrix _Wi of each group of data to calculate the low-dimensional projection of each group of training samples and test samples respectively, and then use a serial feature fusion strategy to form the final fusion features for classification; and calculate the recognition rate.

The present invention can be further explained through the following embodiments: taking the CMU-PIE face database as an example, the CMU-PIE face database contains face images of 68 people, and the size of each image is 64×64. In this experiment, the first 10 images of each person were used as the training set, and the last 14 images were used as the test set. Read the input face image data and form three different features, namely: feature 1 is the original image data, feature 2 is the image data after median filtering, and feature 3 is the image data after mean filtering. Use principal component analysis to reduce the dimension of each feature to form the final three sets of feature data.

Step 1) Construct three groups of data Xi with mean value 0 _, i=1, 2, 3, and define the projection direction of the training sample set as

Step 2) The goal of FDMCCA is to maximize the correlation of samples within a class while minimizing the correlation of samples between classes. Calculate the intra-class correlation matrix of training samples between groups and autocovariance matrix Among them/> Represents a matrix in which each element is 1;

Step 3) For the inter-group intra-class correlation matrix obtained in step 2) Perform singular value decomposition to obtain the left and right singular vector matrices and singular value matrices. Perform eigenvalue decomposition of the autocovariance matrix C _ii to obtain the eigenvector matrix and eigenvalue matrix;

Step 3-1) Compute the intra-class correlation matrix between groups Do singular value decomposition:

in and/> They are/> The left and right singular vector matrices of Yes/> A diagonal matrix composed of singular values, and/>

Step 3-2) Perform eigenvalue decomposition of the autocovariance matrix C _ii :

in is the eigenvector matrix of C _ii ,/> is a diagonal matrix composed of eigenvalues of C _ii , and r _i =rank(C _ii ).

Step 4) Define the value range of fractional order parameters α and β to be {0.1, 0.2,...,1}, select appropriate fractional order parameters α and β, and reassign the singular value matrix and eigenvalue matrix obtained in step 3). Constructing a fractional-order intra-class correlation matrix between groups and fractional-order autocovariance matrix/>

Step 4-1) Assume α is a fraction and satisfies 0≤α≤1, define the intra-class correlation matrix between fractional-order groups for:

in U _ij and V _ij and r _ij are defined in step 3-1).

Step 4-2) Assume that β is a fraction and satisfies 0≤β≤1, define the fractional autocovariance matrix for:

in The definitions of Q _i and r _i are given in step 3-2).

Step 5) Construct the optimization model of FDMCCA as in By introducing the Lagrange multiplier method, the generalized eigenvalue problem Eω = μFω can be obtained, and then the projection direction ω can be obtained, where

Step 6) Considering that the autocovariance matrix may be a singular matrix, introduce the regularization parameter η on the basis of step 5), where the value range of eta is {10 ^-5 , 10 ^-4 ,...,10}, and establish The optimization model under regularization is Introducing the Lagrange multiplier method, the following generalized eigenvalue problem can be obtained:

Step 7) Obtain the projection direction ω according to the generalized eigenvalue problem in step 6), calculate the projection of the test sample in the projection direction, adopt a serial feature fusion strategy, use the nearest neighbor classifier for classification, and calculate the recognition rate. Solve the eigenvectors corresponding to the first d largest eigenvalues according to the generalized eigenvalue problem in step 6), thereby forming the projection matrix _Wi = [ω _i1 ,ω _i2 ,...,ω _id ] for each set of data, i=1, 2,3,d≤min{m ₁ ,m ₂ ,m ₃ };

Step 8) Using the projection matrix _Wi of each group of data, calculate the low-dimensional projection of each group of training samples and test samples respectively, and then use a serial feature fusion strategy to form the final fusion feature for classification; and use the nearest neighbor classifier for Classify and calculate the recognition rate. The recognition rate results are shown in Table 1 and Figure 2 (BASELINE refers to the classification result after concatenating three features). It can be seen from Table 1 and Figure 2 that compared with other methods, the FDMCCA method proposed by the present invention is more effective. This is because: Compared with MCCA, CCA, and BASELINE, FDMCCA is a supervised learning method with prior information and can achieve better recognition results. Compared with DMCCA, FDMCCA introduces the fractional order idea to correct the covariance deviation caused by factors such as noise interference and improve the accuracy of identification.

Table 1 Recognition rate on CMU-PIE data set

method Recognition rate(%) MCCA 84.09 CCA (feature 1 + feature 2) 71.43 CCA (feature 1 + feature 3) 74.03 CCA (feature 2 + feature 3) 76.30 BASELINE 48.05 DMCCA 79.22 FDMCCA 86.04

In order to test the impact of the number of training samples on the recognition rate, the present invention fixes the fractional parameters α, β and the regularization parameter η, and selects different numbers of images as training sets and test sets respectively. The recognition rate is shown in Figure 3. As can be seen from Figure 3, FDMCCA has better results when there are fewer training samples.

In summary, the present invention is based on the CCA method and introduces the idea of fractional embedding to propose a supervised multi-set correlation feature fusion method (FDMCCA) based on spectral reconstruction. This method can correct the deviation of the intra-class correlation matrix and autocovariance matrix caused by noise interference and limited training samples by introducing fractional-order parameters. At the same time, this method makes full use of class label information and can handle information fusion problems larger than two views, with a wider application range and better recognition performance.

The present invention is not limited to the above embodiments. On the basis of the technical solutions disclosed in the present invention, those skilled in the art can make some substitutions and modifications to some of the technical features without any creative work according to the disclosed technical contents. Modifications, these substitutions and modifications are within the protection scope of the present invention.

Claims (3)

1. A supervised multi-set related feature fusion method based on spectral reconstruction, which is characterized by including the following steps: Step 1) Assume that there are P groups of training samples, the training samples are face images, the mean value of each group of samples is 0 and the number of categories is c, as follows: in represents the k-th sample of the j-th category in the i-th group, m _i represents the feature dimension of the i-th group data set, n _j represents the number of j-th category samples, and defines the projection direction of the training sample set as/> Step 2) Calculate the inter-group intra-class correlation matrix of the training samples between groups and autocovariance matrix Among them/> Represents a matrix in which each element is 1; Step 3) For the inter-group intra-class correlation matrix obtained in step 2) Perform singular value decomposition to obtain the left and right singular vector matrices and singular value matrices. Perform eigenvalue decomposition of the autocovariance matrix C _ii to obtain the eigenvector matrix and eigenvalue matrix; Step 4) Select appropriate fractional order parameters α and β, reassign the singular value matrix and eigenvalue matrix obtained in step 3), and construct the intra-class correlation matrix between fractional order groups. and fractional-order autocovariance matrix/> Step 5) Construct the optimization of FDMCCA The model is Among them/> Introducing the Lagrange multiplier method, we obtain the generalized eigenvalue problem Eω = μFω, and find the projection direction ω, where μ is the eigenvalue, Step 6) Considering that the autocovariance matrix may be a singular matrix, introduce the regularization parameter η based on step 5) and establish the optimization model under regularization as Introducing the Lagrange multiplier method, the following generalized eigenvalue problem is obtained: in is the identity matrix of size m _i × m _i , i=1,2,…,P; Step 7) Solve the eigenvectors corresponding to the first d largest eigenvalues according to the generalized eigenvalue problem in step 6), thereby forming the projection matrix _Wi for each set of data = [ω _i1 ,ω _i2 ,...,ω _id ], i =1,2,…,P, d≤min{m ₁ ,…,m _P }; Step 8) Use the projection matrix _Wi of each group of data to calculate the low-dimensional projection of each group of training samples and test samples respectively, and then use a serial feature fusion strategy to form the final fusion features for classification; and calculate the recognition rate. 2. A supervised multi-set correlation feature fusion method based on spectral reconstruction according to claim 1, characterized in that the inter-group intra-class correlation matrix described in step 3) Doing singular value decomposition and autocovariance matrix C _ii to do eigenvalue decomposition includes the following steps: Step 3-1) Compute the intra-class correlation matrix between groups Do singular value decomposition: in and/> They are/> The left and right singular vector matrices of Yes/> A diagonal matrix composed of singular values, and/> Step 3-2) Perform eigenvalue decomposition of the autocovariance matrix C _ii : in is the eigenvector matrix of C _ii ,/> is a diagonal matrix composed of eigenvalues of C _ii , and r _i =rank(C _ii ). 3. A supervised multi-set correlation feature fusion method based on spectral reconstruction according to claim 1 or 2, characterized in that step 4) constructs a fractional inter-group intra-class correlation matrix and fractional-order autocovariance matrix/> Contains the following steps: Step 4-1) Assume α is a fraction and satisfies 0≤α≤1, define the intra-class correlation matrix between fractional-order groups for: in U _ij and V _ij and r _ij are defined in step 3-1); Step 4-2) Assume that β is a fraction and satisfies 0≤β≤1, define the fractional autocovariance matrix for: in The definitions of Q _i and r _i are given in step 3-2).