CN104462818B - A kind of insertion manifold regression model based on Fisher criterions - Google Patents

Abstract

A kind of insertion manifold regression model based on Fisher criterions, including：Initialization, by training sample matrixRepresent, x_mCorresponding class label is l (x_m)∈{1,2,…,c}；Training sample is pre-processed：Training sample is mapped to principal component analysis subspace；Similar matrix is set up, using Fisher criterions, by sample is separately handled between sample and class in class；Calculate embedded subspace：Define D × d mapping matrix W=[ω₁…ω_d], d is the dimension of sample after Feature Conversion, passes through solution matrixCharacteristic vectorTo find mapping subspace；Sample is y from original higher dimensional space to the transition form in low dimensional manifold space_i=W^Tx_i=F (x_i), it is expressed in matrix as：Y=W^TX=F (X), Y=[y₁,…,y_M].The present invention is on the premise of the label information of sample is made full use of, and similar sample keeps local geometry before and after making dimensionality reduction, and makes the similarity reduction after dimensionality reduction in the sample that luv space is different classes of but similarity is high.

Description

A kind of insertion manifold regression model based on Fisher criterions

Technical field

The present invention relates to a kind of embedded manifold regression model.It is more particularly to a kind of to utilize markup information and linear regression mould Type, the insertion manifold regression model based on Fisher criterions being improved to traditional manifold learning and linear discriminant analysis.

Background technology

Sub-space learning and feature extraction are machine vision and an important research direction of area of pattern recognition, current general Time method be to find a mapping matrix by the subspace of the Feature Conversion in original input space to low-dimensional, traditional side Method has principal component analysis (Principal component analysis, PCA), independent component analysis (Independent Component analysis, ICA) etc. do not utilize label information unsupervised learning method；And linear discriminant analysis (Linear discriminative analysis, LDA) etc. utilizes the supervised learning method of training sample label information, and The core mutation of these learning methods, these methods weigh the similarity between sample using Euclidean distance mostly, but geodetic Linear distance can more truly reflect the geometry implied in data on a macroscopic level, so will using manifold learning Original sample Projection Character can often obtain more preferable classifying quality to manifold space, the current algorithm master using manifold learning Linear approximation is carried out to non-linear method using the embedded mode of figure, it is such as local to keep mapping (Locality Preserving projection, LPP), locally embedding analysis (Locally Embedded Analysis, LEA), Jin Linbao Hold embedded (Neighborhood Preserving Embedding, NPE) etc., the basic thought of these algorithms is using affine Figure keep model local manifolds structure, but these methods only focus on local holding structure ignore between classification differentiation letter Breath.

In addition, linear regression disaggregated model can be used for feature extraction.The skill for example, the application linear regression such as Chai is classified Art predicts its corresponding positive face sample using the face sample of side face；Naseem etc. is based on congener's face image positioned at same The hypothesis of linear subspaces proposes the linear regression disaggregated model for recognition of face, is returned by least squares estimate Return coefficient, and judge according to the Euclidean distance between original vector and map vector the ownership of classification.Huang etc. proposes one The enhancing principal component analysis model for recognition of face is planted, this method can solve multicollinearity in linear regression model (LRM) Problem., a kind of thought of the embedded linear discriminant analysis on linear regression model (LRM) such as Huang, it is proposed that linear discriminant in 2013 Return disaggregated model, this method finds an optimal mapping using the ratio for maximizing reconstructed error in reconstructed error and class between class Matrix, by optimal mapping matrix by original sample Feature Mapping to subspace, utilizes linear regression model (LRM) on this subspace To realize more preferable identification and classification.

The content of the invention

The technical problems to be solved by the invention are to provide a kind of insertion manifold regression model based on Fisher criterions, Make full use of label information, to belong to similar sample between and belong between inhomogeneous sample and build an Affinity diagram respectively To keep the local manifolds structure between similar sample, the connection while cut-out belongs to a different category between the high sample of similarity System, a subspace that can be kept local manifolds structure and have discriminating power is mapped to by the sample of luv space.

The technical solution adopted in the present invention is：A kind of insertion manifold regression model based on Fisher criterions, including such as Lower step：

1) initialize, M training image sample is had provided with c class, each image size is a × b, Ke Yiyong MatrixTo represent, each image array is converted into column vector by wherein m=1,2 ..., M, is usedTable Show, D=a × b, so training sample can use matrixx_mCorresponding class label is l (x_m)∈{1,2,…,c}；

2) training sample is pre-processed：Training sample is mapped to principal component analysis subspace.

3) similar matrix is set up, using Fisher criterions, by sample is separately handled between sample and class in class, definition is both It is Affinity diagram G in the class of M node_BThe Affinity diagram G between class_W, each one sample of node on behalf, for Affinity diagram G in class_B, only Consider to belong to similar sample data to x_i, x_j, l (x_i)=l (x_j), i, j ∈ { 1 ..., M }；For Affinity diagram G between class_W, only examine Sample data between worry is different classes of is to x_i, x_j, l (x_i)≠l(x_j)；

4) embedded subspace is calculated：Define D × d mapping matrix W=[ω₁…ω_d], d is sample after Feature Conversion This dimension, passes through solution matrixCharacteristic vectorTo find mapping subspace, wherein, For corresponding preceding d most Big characteristic vector,It is linear regression image of j-th of sample in kth class；

5) sample is y from original higher dimensional space to the transition form in low dimensional manifold space_i=W^Tx_i=F (x_i), use matrix table It is shown as：Y=W^TX=F (X), Y=[y₁,…,y_M]。

Step 3) in, for Affinity diagram G between class_WWith Affinity diagram G in class_BSimilarity between interior joint, defines following two sides Formula：

(1) soft weight, if node i is connected with j, corresponding sample x_iAnd x_jBetween similarity be：s_ij=exp (- ||x_i-x_j||²/ σ), whereinx_ipRepresent x_iThe pth neighbour of sample, if node i and j are not attached to, Then corresponding sample x_iAnd x_jBetween similarity be s_ij=0；

(2) hard weight, if node i is connected with j, corresponding sample x_iAnd x_jBetween similarity be：s_ij=1, such as Fruit node i and j are not attached to, then corresponding sample x_iAnd x_jBetween similarity be s_ij=0.

Step 3) in, for Affinity diagram G in class_W, following two modes are defined to connect the node corresponding to sample：

(1) it is flexible coupling, if sample x_iWith sample x_jClassification is identical, and sample x_iBelong to sample x_jK_WNeighbour, or Sample x_jBelong to sample x_iK_WNeighbour, connecting node i and node j.

(2) Hard link, if sample x_iWith sample x_jClassification is identical, will two-by-two be connected between the node corresponding to sample.

Step 3) in for Affinity diagram G between class_B, following two modes are defined to connect the node corresponding to sample：

(1) it is flexible coupling, if sample x_iWith sample x_jClassification is differed, and sample x_iBelong to sample x_jK_BNeighbour, or Person's sample x_jBelong to sample x_iK_BNeighbour, connecting node i and node j.

(2) Hard link, if sample x_iWith sample x_jClassification is differed, and will be two-by-two connected between the node corresponding to sample.

k_BAnd k_WIt is identical or differ.

Step 4) described inIt is to be obtained by following formula：

Wherein,

K_kThe characteristic vector of the mark sample of k-th of class is represented,

The present invention a kind of insertion manifold regression model based on Fisher criterions, apply in multimedia retrieval with row The dimension of sequence association area about subtracts method, on the premise of the label information of sample is made full use of, and makes similar sample before and after dimensionality reduction Local geometry is kept, and makes the similarity reduction after dimensionality reduction in the sample that luv space is different classes of but similarity is high.This Invention has following feature：

1st, using linear regression model (LRM), it is embedded into one from original higher dimensional space by building the form of Affinity diagram by sample With the low dimensional manifold space for differentiating characteristic, not only consider to keep the geometry between similar sample, while utilizing Fisher Criterion separates the sample between inhomogeneity, it is proposed that suitable for the sub-space learning algorithm of area of pattern recognition.

2nd, it has been experimentally confirmed and principal component analysis, linear discriminant analysis, it is local to keep traditional subspaces such as mapping Learning method compares, property of the embedded type manifold regression model in classification experiments based on Fisher criterions that the present invention is designed It can substantially be dominant, effectively can be classified using supervision message, therefore be more appropriately applied in classification problem.

3rd, simple possible, processing speed is fast, can be used in the related field of the classification and identification of pattern-recognition.

Brief description of the drawings

Fig. 1 is the flow chart of the inventive method；

Fig. 2 is similar degree in the class matrix S_WThe similarity matrix S between class_BStructure.

Embodiment

A kind of insertion manifold regression model based on Fisher criterions of the present invention is done with reference to embodiment and accompanying drawing Go out to describe in detail.

In view of the current algorithm based on manifold learning more they tends to pay close attention to local manifolds structure, and ignore and sentence between classification The characteristics of other information, the present invention proposes a kind of insertion manifold regression model based on Fisher criterions, its main thought be On the basis of linear regression model (LRM), make full use of label information, to belong to similar sample between and belong to inhomogeneous sample Between build an Affinity diagram respectively to keep the local manifolds structure between similar sample, while cut-out belongs to a different category phase The contact seemingly spent between high sample, all samples are mapped on the low-dimensional submanifold of embedded high-dimensional feature space, i.e. by original Beginning space sample be mapped to one can keep local manifolds structure and with discriminating power subspace.Learned by shifting Acquistion is to this low dimensional manifold, therefore the submanifold learnt had both remained the geometry between similar sample, while fully Different classes of sample is differentiated using the discriminant information between classification.

The present invention takes into full account the discriminant information between classification and learns a submanifold on the basis of linear regression model (LRM) To describe the feature after dimensionality reduction, the submanifold is embedded in original feature space to keep the local manifolds knot between similar sample Structure, the contact while cut-out belongs to a different category between the high sample of similarity causes dimensionality reduction while sample dimension is reduced Feature afterwards is more beneficial for classification.

As shown in figure 1, a kind of insertion manifold regression model based on Fisher criterions of the present invention, comprises the following steps：

1) initialize, M training image sample is had provided with c class, each image size is a × b, Ke Yiyong MatrixTo represent, each image array is converted into column vector by wherein m=1,2 ..., M, is usedTable Show, D=a × b, so training sample can use matrixx_mCorresponding class label is l (x_m)∈{1,2,…,c}；

2) training sample is pre-processed：In order to reduce amount of calculation, training sample is mapped to principal component by us first (PCA) subspace is analyzed, the effect of noise reduction can also be played by carrying out pretreatment to image using PCA.

3) set up similar matrix, in order to make full use of label information, the present invention utilizes Fisher criterions, by sample in class and Relation between class between sample is separately handled, as shown in Fig. 2 definition is both Affinity diagram G in the class of M node_BBetween class Affinity diagram G_W, each one sample of node on behalf, for Affinity diagram G in class_B, only consider to belong to similar sample data to x_i, x_j, l (x_i)=l (x_j), i, j ∈ { 1 ..., M }；For Affinity diagram G between class_W, only consider it is different classes of between sample data pair x_i, x_j, l (x_i)≠l(x_j)。

For Affinity diagram G between class_WWith Affinity diagram G in class_BSimilarity between interior joint, defines following two modes：

(1) soft weight, if node i is connected with j, corresponding sample x_iAnd x_jBetween similarity be：s_ij=exp (- ||x_i-x_j||²/ σ), whereinx_ipRepresent x_iThe pth neighbour of sample, if node i and j are not attached to, Then corresponding sample x_iAnd x_jBetween similarity be s_ij=0；

(2) hard weight, if node i is connected with j, corresponding sample x_iAnd x_jBetween similarity be：s_ij=1, such as Fruit node i and j are not attached to, then corresponding sample x_iAnd x_jBetween similarity be s_ij=0.

For Affinity diagram G in class_W, following two modes are defined to connect the node corresponding to sample：

(1) it is flexible coupling, if sample x_iWith sample x_jClassification is identical, and sample x_iBelong to sample x_jK_WNeighbour, or Sample x_jBelong to sample x_iK_WNeighbour, connecting node i and node j.

(2) Hard link, if sample x_iWith sample x_jClassification is identical, will two-by-two be connected between the node corresponding to sample.

For Affinity diagram G between class_B, following two modes are defined to connect the node corresponding to sample：

(1) it is flexible coupling, if sample x_iWith sample x_jClassification is differed, and sample x_iBelong to sample x_jK_BNeighbour, or Person's sample x_jBelong to sample x_iK_BNeighbour, connecting node i and node j.

(2) Hard link, if sample x_iWith sample x_jClassification is differed, and will be two-by-two connected between the node corresponding to sample.

Above-mentioned k_BAnd k_WIt is identical or differ, selected using empirical value.

4) embedded subspace is calculated：Define D × d mapping matrix W=[ω₁…ω_d], d is the dimension of sample after Feature Conversion Number, passes through solution matrixCharacteristic vectorTo find mapping subspace, wherein, For corresponding preceding d most Big characteristic vector,It is linear regression image of j-th of sample in k-th of class；DescribedIt is to be obtained by following formula Arrive：

Wherein,

Kk represents the characteristic vector of the mark sample of k-th of class,

According to the theory of linear regression, if sample x belongs to k-th of class, the sample can use k-th of class training sample Linear combination represent, i.e.,：

X=K_kβ_k+ e k=1,2 ..., c (2)

WhereinIt is regression parameter vector, e is residual error, wherein e obeys (0, σ²) Gaussian Profile.Linearly The target of regression model is to findSo that residual error e is minimum, i.e.,：

Regression parameter vector can be solved by least-squares estimation, and its matrix form is：

By estimating parameterWith the sub- K of prediction_kTo predict the corresponding vector of kth class

(4) are updated in (5),

Matrix A is obtained by abbreviation object function.The object function of the present invention can be write as：

In Affinity diagram between building class, for the sample y between similar_iAnd y_jSimilarityBuilding class in-laws During with figure, for the sample y between inhomogeneity_iAnd y_jSimilaritySo object function can be equivalent to：

Object function can be equivalent to：

(14)

Obtained by abbreviation：

So object function can using abbreviation as：

(16)

Wherein：

The vector of transition matrix W each rowIt can be obtained by solving following characteristics value problem：

E_Bω_t=λ_tE_Wω_tT=1 ..., d (17)

Wherein λ₁≥…≥λ_t≥…≥λ_d。

Then transition matrix is W=[ω₁,…,ω_t,…,ω_d] (18)

5) sample is y from original higher dimensional space to the transition form in low dimensional manifold space_i=W^Tx_i=F (x_i), use matrix table It is shown as：

Y=W^TX=F (X), Y=[y₁,…,y_M]。

Claims (5)

1. a kind of method for setting up the insertion manifold regression model based on Fisher criterions, it is characterised in that comprise the following steps：

1) initialize, M training image sample is had provided with c class, each image size is a × b, uses matrixTo represent, each image array is converted into column vector by wherein m=1,2 ..., M, is usedRepresent, D= A × b, so training sample matrixRepresent, x_mCorresponding class label is l (x_m)∈ {1,2,…,c}；

2) training sample is pre-processed：Training sample is mapped to principal component analysis subspace；

3) similar matrix is set up, using Fisher criterions, by sample is separately handled between sample and class in class, definition is both M Affinity diagram G in the class of individual node_WThe Affinity diagram G between class_B, each one sample of node on behalf, for Affinity diagram G in class_W, only examine The sample data for belonging to similar is considered to x_i, x_j, l (x_i)=l (x_j), i, j ∈ { 1 ..., M }；For Affinity diagram G between class_B, only consider Sample data between different classes of is to x_i, x_j, l (x_i)≠l(x_j)；

4) embedded subspace is calculated：Define D × d mapping matrix W=[ω₁…ω_d], d is sample after Feature Conversion Dimension, pass through solution matrixCharacteristic vectorTo find mapping subspace, wherein, For corresponding preceding d most Big characteristic vector,It is linear regression image of j-th of sample in kth class；

5) sample is from original higher dimensional space to the transition form in low dimensional manifold spaceRepresented with matrix For：Y=W^TX=F (X), Y=[y₁,…,y_M]。

2. a kind of method for setting up the insertion manifold regression model based on Fisher criterions according to claim 1, it is special Levy and be, step 3) in, for Affinity diagram G between class_BWith Affinity diagram G in class_WSimilarity between interior joint, defines following two sides Formula：

(1) soft weight, if node i is connected with j, corresponding sample x_iAnd x_jBetween similarity be：x_ipRepresent x_iThe pth neighbour of sample, if node i and j It is not attached to, then corresponding sample x_iAnd x_jBetween similarity be s_ij=0；

(2) hard weight, if node i is connected with j, corresponding sample x_iAnd x_jBetween similarity be：s_ij=1, if node I and j are not attached to, then corresponding sample x_iAnd x_jBetween similarity be s_ij=0.

3. a kind of method for setting up the insertion manifold regression model based on Fisher criterions according to claim 1, it is special Levy and be, step 3) in, for Affinity diagram G in class_W, following two modes are defined to connect the node corresponding to sample：

(1) it is flexible coupling, if sample x_iWith sample x_jClassification is identical, and sample x_iBelong to sample x_jK_WNeighbour, or sample x_jBelong to sample x_iK_WNeighbour, connecting node i and node j；

(2) Hard link, if sample x_iWith sample x_jClassification is identical, will two-by-two be connected between the node corresponding to sample.

4. a kind of method for setting up the insertion manifold regression model based on Fisher criterions according to claim 1, it is special Levy and be, step 3) in for Affinity diagram G between class_B, following two modes are defined to connect the node corresponding to sample：

(1) it is flexible coupling, if sample x_iWith sample x_jClassification is differed, and sample x_iBelong to sample x_jK_BNeighbour, or sample This x_jBelong to sample x_iK_BNeighbour, connecting node i and node j；

(2) Hard link, if sample x_iWith sample x_jClassification is differed, and will be two-by-two connected between the node corresponding to sample.

5. a kind of method for setting up the insertion manifold regression model based on Fisher criterions according to claim 1, it is special Levy and be, step 4) described inIt is to be obtained by following formula：

Wherein,

K_kThe characteristic vector of the mark sample of k-th of class is represented,