CN108573263A - A Dictionary Learning Approach to Jointly Structured Sparse Representations and Low-Dimensional Embeddings - Google Patents

Abstract

The invention discloses the dictionary learning methods of a kind of co-ordinative construction rarefaction representation and low-dimensional insertion, dictionary is constructed to carry out with dimensionality reduction projection matrix time-interleaved, pass through incoherence between forced class of the rarefaction representation coefficient matrix with block diagonalization structure to enhance dictionary in low dimension projective space, at the same time, correlation in class using the low-rank of the expression coefficient on class small pin for the case dictionary to keep dictionary, dictionary is constructed can promote mutually with projection study, fully to keep the sparsity structure of data, to encode out the code coefficient with more classification judgement index, the expression coefficient class discriminating power that the present invention solves higher-dimension characteristic in dictionary learning method existing in the prior art due to training sample data and lacks the dictionary of stringent block diagonalization structural constraint and go out coded by making is weak, the not strong problem of distinction.

Description

A Dictionary Learning Approach to Jointly Structured Sparse Representations and Low-Dimensional Embeddings

technical field

The invention belongs to the technical field of digital image processing, and in particular relates to a dictionary learning method combining structured sparse representation and low-dimensional embedding.

Background technique

The core idea of sparse representation is mainly based on the following objective fact: many signals in nature can be represented or encoded by linear combination with only a few dictionary items in an oversaturated dictionary. The most critical issue in sparse representation research is the construction of dictionaries with strong representation capabilities. At present, sparse representation technology is widely used in many application fields, such as image classification, face recognition and human action recognition.

Dictionary learning is dedicated to learning an optimal dictionary from training samples in order to better represent or encode a given signal or feature. For classification recognition based on sparse representation, the ideal sparse matrix of the training sample under the dictionary should be block diagonal, that is, the coefficient of the sample on the same sub-dictionary where it is located is non-zero, while the coefficient on the heterogeneous sub-dictionary is to zero. Such a structured coefficient matrix will have the best class discrimination ability. In addition, due to the high-dimensional nature of the training data and the lack of training samples, the dictionary learning method still faces great challenges. Therefore, people naturally introduce data dimensionality reduction processing in the dictionary learning process. But unfortunately, these dictionary learning methods often treat data dimensionality reduction and dictionary learning as two independent processing steps to study separately, that is, first perform dimensionality reduction processing on training data, and then perform dictionary learning in low-dimensional feature space. This serial cascading approach is likely to make the pre-learned low-dimensional projections unable to maintain and improve the underlying sparse structure of the data well, which is not conducive to dictionary learning with strong discriminative properties.

Contents of the invention

The purpose of the present invention is to provide a dictionary learning method that combines structured sparse representation and low-dimensional embedding, which solves the problem of the high-dimensional characteristics of training sample data and the lack of strict block diagonalization structure in the dictionary learning method existing in the prior art. Due to the restricted dictionary, the coded representation coefficients have weak discrimination ability and poor discrimination.

The technical solution adopted in the present invention is a dictionary learning method combining structured sparse representation and low-dimensional embedding, which is specifically implemented according to the following steps:

Step 1. Read in the feature data set of the training sample Where C is the number of categories, n is the dimension of the feature, is a feature subset composed of N _i samples of the i-th class, i=1,2,...,C,

Step 2. Solve the optimization problem by using the alternate direction Lagrange multiplier method Obtain the coding dictionary D, dimensionality reduction projection matrix P and coding coefficient matrix X;

Step 3. Read in the test sample characteristic data The test samples are obtained from the encoding dictionary D and the dimensionality reduction projection matrix P by solving the following optimization problem The sparse representation coefficient of

Step 4. Calculate the sparse representation coefficient of the test sample The reconstruction error e _i on each category sub-dictionary D _i : in For the coding coefficient D=[D ₁ , D ₂ ,...,D _C ] corresponding to the i-th sub-dictionary D _i , i=1, 2,..., C;

Step 5, according to the minimum reconstruction error criterion for the test sample Classify, its category label for:

The present invention is also characterized in that,

in step 2

stX=diag(X ₁₁ , X ₂₂ , . . . , X _CC ),PP ^T =I,

Wherein, parameters λ ₁ , λ ₂ , λ ₃ >0; It is a low-dimensional projection transformation matrix, m<<n; the training sample Y is in the dictionary The following representation coefficient matrix is X:

For the j-th class training sample Y _j the i-th class sub-dictionary On the expression coefficient, i,j∈{1,2,…,C};

Let X satisfy the following block diagonalization structural constraints:

Step 2 is specifically implemented according to the following steps:

Step 2.1, introduce auxiliary variable set And let Zi _ii =X _ii , the optimization problem transform into:

stX＝diag(X ₁₁ ,X ₂₂ ,...,X _CC ),PP ^T ＝I,

Z _ii =X _ii , i=1,2,...,C,

Its augmented Lagrange function expression is:

stPP ^T = I,

Among them, F _ii is the Lagrange multiplier, and γ>0 is the penalty parameter;

Step 2.2. Iteratively update the matrices P, D, X and Z _ii alternately until P, D, X and Z _ii converge.

Step 2.2 is specifically implemented according to the following steps:

Step 2.2.1, fix other variables, and update the matrix X by the following formula:

Among them, sgn(x) is defined as:

Step 2.2.2, fix other variables, and update the matrix Z _ii by the following formula:

Among them, UΛV ^T is the matrix The singular value decomposition of is the soft threshold operator, It is defined as follows:

Step 2.2.3, fix other variables, and update the matrix D through the following formula:

After updating the dictionary D column by column according to the above formula, the updated value of the entire dictionary is obtained:

Step 2.2.4, fix other variables, and update the matrix P by the following formula:

First, perform eigenvalue decomposition on the matrix (φ(P ^(t-1) )-λ ₁ S):

[U,Λ,V]=EVD(φ(P ^(t-1) )-λ ₁ S),

Among them, φ(P)=(YP ^T Δ)(YP ^T Δ) ^T , Δ=DX, S=YY ^T , Λ is defined by the eigenvalues of the matrix (φ(P ^(t-1) )-λ ₁ S) The diagonal matrix formed by , so that the update of the projection matrix P is the eigenvector U(1:m,:) corresponding to the first m eigenvalues of the matrix (φ(P ^(t-1) )-λ ₁ S), which is:

P ^(t) = U(1:m,:);

Step 2.2.5, update the multiplier F _ii and parameter γ by the following formula:

γ ^(t) = min{ργ ^(t-1) ,γ ^max }.

Wherein, ρ=1.1, γ ^max =10 ⁶ ,

After the above updates, the encoding dictionary D and the dimensionality reduction projection matrix P are obtained.

The beneficial effect of the present invention is a dictionary learning method that combines structured sparse representation and low-dimensional embedding to obtain coding coefficients with better discriminative performance by eliminating inter-class correlation; enhance intra-class sparsity through low-rank constraints Represent the coherence between the coefficients to further improve the clustering of the representation coefficients on the category sub-dictionary; at the same time, through the learning of the projection matrix, the representation ability of the dictionary is enhanced and the robustness of the sparse representation model is improved.

Detailed ways

The present invention will be described in detail below in combination with specific embodiments.

The present invention is a dictionary learning method that combines structured sparse representation and low-dimensional embedding. The dictionary construction and the dimensionality reduction projection matrix are carried out in parallel and alternately, and the sparse representation coefficient matrix with a block diagonal structure is forced to be enhanced in the low-dimensional projection space. The inter-class incoherence of dictionaries, while maintaining the intra-class correlation of dictionaries by exploiting the low-rank property of the representation coefficients on the class sub-dictionary. Dictionary construction and projection learning can promote each other to fully maintain the sparse structure of the data, so as to encode more class-discriminative coding coefficients. Specifically, follow the steps below:

Step 1. Read in the feature data set of the training sample Where C is the number of categories, n is the dimension of the feature, is a feature subset composed of N _i samples of the i-th class, i=1,2,...,C,

Step 2. Solve the optimization problem by using the alternate direction Lagrange multiplier method Obtain the coding dictionary D, the dimensionality reduction projection matrix P and the coding coefficient matrix X, where,

stX=diag(X ₁₁ , X ₂₂ , . . . , X _CC ),PP ^T =I,

Wherein, parameters λ ₁ , λ ₂ , λ ₃ >0; It is a low-dimensional projection transformation matrix, m<<n; the training sample Y is in the dictionary The following representation coefficient matrix is X:

For the j-th class training sample Y _j the i-th class sub-dictionary On the expression coefficient, i,j∈{1,2,…,C};

Let X satisfy the following block diagonalization structural constraints:

Step 2 is specifically implemented according to the following steps:

Step 2.1, introduce auxiliary variable set And let Zi _ii =X _ii , the optimization problem transform into:

stX＝diag(X ₁₁ ,X ₂₂ ,...,X _CC ),PP ^T ＝I,

Z _ii =X _ii , i=1,2,...,C,

Its augmented Lagrange function expression is:

stPP ^T = I,

Among them, F _ii is the Lagrange multiplier, and γ>0 is the penalty parameter;

Step 2.2, iteratively update the matrices P, D, X, and Z _ii alternately until P, D, X, and Z _ii converge. Specifically, follow the steps below:

Step 2.2.1, fix other variables, and update the matrix X by the following formula:

Among them, sgn(x) is defined as:

Step 2.2.2, fix other variables, and update the matrix Z _ii by the following formula:

Among them, UΛV ^T is the matrix The singular value decomposition of is the soft threshold operator, It is defined as follows:

Step 2.2.3, fix other variables, and update the matrix D through the following formula:

After updating the dictionary D column by column according to the above formula, the updated value of the entire dictionary is obtained:

Step 2.2.4, fix other variables, and update the matrix P by the following formula:

First, perform eigenvalue decomposition on the matrix (φ(P ^(t-1) )-λ ₁ S):

[U,Λ,V]=EVD(φ(P ^(t-1) )-λ ₁ S),

Among them, φ(P)=(YP ^T Δ)(YP ^T Δ) ^T , Δ=DX, S=YY ^T , Λ is defined by the eigenvalues of the matrix (φ(P ^(t-1) )-λ ₁ S) The diagonal matrix formed by , so that the update of the projection matrix P is the eigenvector U(1:m,:) corresponding to the first m eigenvalues of the matrix (φ(P ^(t-1) )-λ ₁ S), which is:

P ^(t) = U(1:m,:);

Step 2.2.5, update the multiplier F _ii and parameter γ by the following formula:

γ ^(t) = min{ργ ^(t-1) ,γ ^max }.

Wherein, ρ=1.1, γ ^max =10 ⁶ ,

After the above update, the encoding dictionary D and the dimensionality reduction projection matrix P are obtained;

Step 3. Read in the test sample characteristic data The test samples are obtained from the encoding dictionary D and the dimensionality reduction projection matrix P by solving the following optimization problem The sparse representation coefficient of

Step 4. Calculate the sparse representation coefficient of the test sample The reconstruction error e _i on each category sub-dictionary D _i : in For the coding coefficient D=[D ₁ , D ₂ ,...,D _C ] corresponding to the i-th sub-dictionary D _i , i=1, 2,..., C;

Step 5, according to the minimum reconstruction error criterion for the test sample Classify, its category label for:

A dictionary learning method of the present invention that combines structured sparse representation and low-dimensional embedding makes full use of category prior information in the low-dimensional space to perform block-structured sparse coding on learning samples in the process of dictionary construction. The coded coefficients have stronger representation ability and category discrimination ability, which significantly improves the accuracy of the classification problem.

Claims (4)

1. the dictionary learning method of a kind of co-ordinative construction rarefaction representation and low-dimensional insertion, which is characterized in that specifically according to following Step is implemented：

Step 1, the characteristic data set for reading in training sampleWherein C is classification number, and n is characterized Dimension,For the N of the i-th class_iThe character subset that a sample is constituted, i=1,2 ..., C,

Step 2, using alternating direction Lagrange multiplier method solving-optimizing problemsObtain encoder dictionary D, dimensionality reduction projection matrix P and code coefficient matrix X；

Step 3 reads in test sample characteristicBy encoder dictionary D and dimensionality reduction projection matrix P and by solving following optimization Problem obtains test sampleRarefaction representation coefficient

Step 4, the rarefaction representation coefficient for calculating test sampleIn all kinds of small pin for the case dictionary D_iOn reconstructed error e_i：WhereinTo correspond in i-th of sub- dictionary D_iOn code coefficient D=[D₁,D₂,…,D_C], i=1, 2,…,C；

Step 5, according to minimal reconstruction error criterion to test sampleClassify, category labelFor：

2. the dictionary learning method of a kind of co-ordinative construction rarefaction representation according to claim 1 and low-dimensional insertion, special Sign is, in the step 2

S.t.X=diag (X₁₁,X₂₂,…,X_CC),PP^T=I,

Wherein, parameter lambda₁,λ₂,λ₃＞ 0；For low dimension projective transformation matrix, m ＜ ＜ n；Training sample

This Y is in dictionaryUnder expression coefficient matrix be X：

For jth class training sample Y_jIn the sub- dictionary of the i-th classOn expression coefficient, i, j ∈ 1,2 ..., C}；

X is enabled to meet following block diagonalization structural constraint：

3. the dictionary learning method of a kind of co-ordinative construction rarefaction representation according to claim 2 and low-dimensional insertion, special Sign is that the step 2 is specifically implemented according to the following steps：

Step 2.1 introduces auxiliary variable collectionAnd enable Z_ii=X_ii, optimization problemIt is converted into：

S.t.X=diag (X₁₁,X₂₂,…,X_CC),PP^T=I,

Z_ii=X_ii, the Lagrange function expressions of i=1,2 ..., C, augmentation are：

s.t.PP^T=I,

Wherein, F_iiFor Lagrange multipliers, γ ＞ 0 are punishment parameter；

Step 2.2, alternating iteration update matrix P, D, X and Z_ii, until P, D, X and Z_iiConvergence.

4. the dictionary learning method of a kind of co-ordinative construction rarefaction representation according to claim 3 and low-dimensional insertion, special Sign is that the step 2.2 is specifically implemented according to the following steps：

Step 2.2.1, fixed other variables, update matrix X by following formula：

Wherein, sgn (x) is defined as：

Step 2.2.2, fixed other variables, update matrix Z by following formula_ii：

Wherein, U Λ V^TFor matrixSingular value decomposition,For soft-threshold operator,Definition is such as Under：

Step 2.2.3, fixed other variables, update matrix D by following formula：

After having been updated by column by above formula to dictionary D, that is, obtain the value after entire dictionary updating：

Step 2.2.4, fixed other variables, update matrix P by following formula：

First, to matrix (φ (P^(t-1))-λ₁S Eigenvalues Decomposition) is carried out：

[U, Λ, V]=EVD (φ (P^(t-1))-λ₁S),

Wherein, φ (P)=(Y-P^TΔ)(Y-P^TΔ)^T, Δ=DX, S=YY^T, Λ is matrix (φ (P^(t-1))-λ₁S characteristic value) The diagonal matrix constituted is matrix (φ (P to the update of projection matrix P^(t-1))-λ₁S corresponding to preceding m characteristic value) Feature vector U (1:m,:), i.e.,：

P^(t)=U (1:m,:)；

Step 2.2.5, multiplier F is updated by following formula_iiAnd parameter γ：

γ^(t)=min { ρ γ^(t-1),γ^max}。

Wherein, ρ=1.1, γ^max=10⁶,

Encoder dictionary D and dimensionality reduction projection matrix P are obtained after updating above.