CN110866560A - A subspace clustering method for symmetric low-rank representations based on structural constraints - Google Patents

Abstract

The subspace clustering method based on the symmetric low-rank representation provided by the present invention includes: S1: obtaining a data matrix X' of the original image; S2: performing PCA dimension reduction on the data matrix X' obtained by S1 to obtain X; S3: Use the angle to determine the weight to construct the information error correction matrix R of X; S4: Input the data matrix X obtained by S2 and the error correction matrix R obtained by S3 into the symmetric low-rank representation model with structural constraints, and the optimized output representation matrix Z; S5 : use the angle information representing the main direction of the matrix Z output by S4 to obtain the weight matrix L for spectral clustering; S6 : use the weight matrix L obtained in S5 for spectral clustering to obtain the clustering result. Extensive experimental results on both datasets demonstrate that the proposed method can well reveal the structure of complex subspaces and yield state-of-the-art clustering performance compared to several state-of-the-art methods.

Description

A subspace clustering method for symmetric low-rank representations based on structural constraints

technical field

The invention belongs to the field of subspace clustering, and in particular relates to a subspace clustering method based on structural constraint symmetric low-rank representation.

Background technique

Recently, clustering algorithms for large-scale and high-dimensional data are one of the hotspots and difficulties in the field of clustering analysis. Due to the sparsity of high-dimensional data, traditional clustering algorithms cannot obtain satisfactory results when clustering high-dimensional data. The subspace clustering algorithm is dedicated to solving the problems encountered by traditional clustering algorithms in high-dimensional data clustering, and is a new branch of clustering algorithms. Subspace clustering methods can reveal the underlying subspace structure of high-dimensional data by partitioning the data into the corresponding subspaces from which they are derived, and are widely used in many computer vision and machine learning applications, such as saliency detection , motion segmentation, face clustering, image segmentation, etc. In fact, high-dimensional data has such a remarkable feature: high-dimensional data is not without structure, but is shown to lie in the union of multiple low-dimensional subspaces. Based on this finding, subspace clustering methods have been studied to cluster high-dimensional data. Combining representation methods and spectral clustering algorithms is the most representative method.

The most representative methods in the past are the low-rank representation (LRR) proposed by Liu et al. and the sparse representation (SSC) proposed by Elhamifar et al. On the basis of these two methods, some scholars have added some constraints to achieve better performance. Ni et al. propose a low-rank representation with positive semi-definite constraints (LRR-PSD), and Vidal et al. decompose the corrupted data matrix into a clean, self-expressive dictionary plus the sum of noise and/or severe error matrices, A low-rank subspace clustering (LRSC) non-convex model is proposed. Chen et al. propose a low-rank representation with symmetry constraints (LRRSC) method, which extends the original low-rank representation algorithm by incorporating symmetry constraints into the low-rank properties of high-dimensional data representations. However, these methods only consider low rank or sparsity, which will cause the problem that the coefficients are too sparse or too low.

The present invention proposes a subspace clustering method based on symmetric low-rank representation based on structural constraints, we apply symmetric constraints and structural constraints to the low-rank representation coefficients, which can not only capture the global and local structure of the data, but also preserve the representation Symmetrical consistency of matrices. The model well balances the inter-class sparsity and intra-class clustering of the coefficient matrix, which can better reveal the subspace structure of the data.

SUMMARY OF THE INVENTION

The subspace clustering method of symmetric low-rank representation based on structural constraints proposed by the present invention imposes both structural constraints and symmetry constraints on the solution model of low-rank representation, uses angles to determine weights to construct an information error correction matrix R, and adopts interleaved directions. Then use the main direction information of the coefficient to calculate the weight matrix W, and finally, use W for clustering to obtain the clustering result.

The technical scheme of the inventive method is as follows:

Step S1, obtain the data matrix X' of the original image:

获取的，每个子空间i包含n_i个数据样本

任务是将数据点x_i聚类到从其提取的子空间S_i中，实际的，通过对原始数据192*168尺寸的人脸图像调整尺寸为48*42像素进行预处理，即数据矩阵的原始维数为48*42，2016维。Given a data matrix X′=[x ₁ ,x ₂ ,…x _n ]∈R ^d×n is from k subspaces

obtained, each subspace i contains n _i data samples

The task is to cluster the data points x _i into the subspace S _i extracted from it. Actually, preprocessing is performed by resizing the face image of the original data 192*168 size to 48*42 pixels, that is, the data matrix of The original dimension is 48*42, 2016 dimensions.

Step S2, using PCA to reduce the dimension of the data matrix XX obtained in S1 to obtain a matrix data matrix X: using PCA to reduce the data dimension to obtain original pixels (2016-dimensional), 500-dimensional data, 300-dimensional data, and 100-dimensional data respectively.

Step S3, use the angle to determine the weight to construct the information error correction matrix R of the data matrix X obtained by S2:

The weight between data points can be determined by the angle between the data points. If the angle between the data points is small, they are likely to belong to the same class, and it can be inferred that the samples within the class have less weight, and then the data is classified. Normalize and calculate the absolute value between them, so the ideal R is as follows:

和

是x_i和x_j数据点的标准化数据，并且根据经验将σ设置

元素的均值。in

and

is the normalized data for the x _i and x _j data points and empirically sets σ

mean of the elements.

In step S4, the data matrix X obtained in step S2 and the error correction matrix R obtained in step S3 are input into the constructed symmetric low-rank model based on structural constraints, and the optimized output represents matrix Z;

In order to make it a natural idea to impose both symmetry and structural constraints on the low-rank representation to impose constraints on the structure of the solution of the LRR model, by adding ∑ _i,j R _ij |Z _ij | and Z _ij =Z to the objective function _ji to constrain the structure of the solution, meanwhile, to make the obtained Z more robust to noise and avoid NP problems, the symmetric low-rank representation model with structural constraints is as follows:

stX=AZ+E, Z=Z ^T

By introducing two auxiliary variables J and L, the ADM method is used to perform Laplace alternating optimization on the model, and the augmented Lagrange function of the above formula is obtained, as follows:

where Y ₁ , Y ₂ and Y ₃ are Lagrange multipliers and μ is the penalty coefficient. The augmented Lagrange function is optimized by fixing the other variables each time and updating only one variable.

Optimize step 1, update the J matrix, and obtain the following formula;

Optimize step 2, update the L matrix to obtain the following formula;

Optimize step 3, update the Z matrix to obtain the following formula;

Optimize step 4, update E matrix, obtain following formula;

In optimization step 5, after updating J, L, Z, and E in sequence, update ₁ , Y ₂ , Y ₃ and β by the following formula:

Iteratively runs until the update optimization stops when it converges, and the output representation matrix Z is obtained.

Step S5: use the angle information representing the main direction of the matrix Z output in step S4 to obtain a weight matrix L for spectral clustering; specifically, let the SVD of Z be U ^* Σ ^* (V ^* ) ^T , the parameter U ^* and V ^* is an orthonormal basis for Z, we multiply each column weight of U ^* by (Σ ^* ) ^1/2 and each row weight of V ^* by (Σ ^* ) ^1/2 , then we define M = U ^* (Σ ^* ) ^1/2 , N = (Σ ^* ) ^1/2 (V ^* ) ^T . Then there is Z ^* = MN, and finally by using all row vectors from matrix M or all column vectors from matrix N Angle information to define the affinity graph matrix L, there are:

Among them, m _i , m _j are the rows and columns of M, respectively, and n _i , n _j are the rows and columns of N, respectively.

Step S6: The weight matrix L obtained in S5 is used for spectral clustering Ncuts to obtain clustering results. Compared with the prior art, the present invention adopts the above technical solution and has the following technical effects: constructing a symmetric low-rank representation model with structural constraints, The low-rank representation imposes both symmetric constraints and structural constraints to obtain a more block-diagonal representation coefficient Z, which can also better reveal the natural structure of the subspace. Whether it is from global symmetric low-rank or weighted local linearity, it has more favorable The advantages. The present invention proves to have better performance in subspace clustering by clustering images.

Description of drawings

Fig. 1 is a flow chart of the work of a subspace clustering method based on a symmetric low-rank representation of structural constraints of the present invention.

Figure 2 Example graphs under both Yale and Hopkins datasets.

Figure 3. Clustering accuracy of SCSLR under yale and Hopkins datasets.

Fig. 4 Convergence curve of SCSLR optimization algorithm on yale and Hopkins datasets.

Table 1. The performance of each algorithm on the Yale dataset.

Table 2. The performance of each algorithm on the Hopkins dataset.

Detailed ways

The subspace clustering method based on symmetric low-rank representation based on structural constraints, the flow chart is shown in Figure 1: It consists of six steps to complete, firstly, resizing the data to make the experiment faster, secondly using PCA to reduce dimensionality, and then Calculate the required information error correction matrix R, then build our main model, use ADM to alternately optimize, and use the obtained representation matrix Z to calculate the angle information of the main direction to obtain the weight matrix L for spectral clustering, and finally to the obtained clustering. The results are compared and analyzed.

The present invention will be further described below through an embodiment, the purpose of which is only to better understand the research content of the present invention and not to limit the protection scope of the present invention. The specific technical steps are as follows:

Step S1, the present embodiment adopts the face and motion data set given by Yale and Hopkins. The Yale data set contains 10 people, and each person has 64 pieces of photo data with different information, a total of 640 pieces of face data, Figure 2 (top) It is the photo data of a single person in the Yale dataset. We resize the pixels per person in the Yale dataset from 192*168 to 48*42 pixels.

The Hopkins dataset contains 155 video sequences, each matched to a low-dimensional subspace, and contains 39 to 550 data vectors extracted from two or three motions, Figure 2 (bottom) for two video sequences Some frames of tracked feature points.

This embodiment is tested on two data sets, each is run once, and the standard deviation of the accuracy of each clustering is calculated.

Step S2: After standardizing the samples of the two datasets in this embodiment, the face dataset is reduced to 2016 dimensions, 500 dimensions, 300 dimensions, and 100 dimensions by PCA, and Hopkins is reduced to a 4n-dimensional subspace (n is the number of subspaces).

Step S3, after the two data sets are reduced in dimension in this embodiment, the data matrix is normalized and the information error correction matrix R is calculated.

Step S4, input the data matrix X and the error correction matrix R into the constructed model, determine that the dimensions in the yale data set are 2016 dimensions, 500 dimensions, 300 dimensions, and 100 dimensions, respectively, and determine that the dimensions in the Hopkins data set are 4n of the subspace. dimension.

Determine the penalty parameters λ, β.

According to the Lagrangian augmentation formula of the model, the parameters are initialized in this example, and the parameter list used in this example is:

parameter J L Z E λ beta Y₁ Y₂ Y₃ μ_max ρ μ Numerical value 0 0 0 0 2.5 0.03 0 0 0 105 1.01 0.1

Start training the model, use the staggered direction method, update and optimize the J, L, Z, E matrices in turn, and then update Y ₁ , Y ₂ , Y ₃ and β, through the following formula:

The loop runs until convergence. Get Z and E matrices.

At this point, the SCSLR model has been set up. Contains two input channels and two output channels.

Step S5, this example determines the penalty parameter α, and uses the representation matrix Z output by the model to construct the affinity matrix L,.

Step S6. In this example, the L input spectrum is clustered by the Ncuts algorithm to calculate the clustering result, and experiments are performed on two data sets. The clustering error will also change with the adjustment of the parameters. As the parameters are adjusted, the clustering error will also change accordingly. It can be seen from Figure 3 that the clustering error of this model is very low, indicating that the clustering accuracy is very good

In this example, SCSLR is compared with the correlation algorithm based on low rank or sparseness. In this example, the clustering error is taken after the experiment is run once. The experimental results are shown in Table 1 and Table 2.

Observing the above chart, we can see that in terms of clustering error rate, the clustering error rate of the SCSLR algorithm is much lower than that of the previous low-rank sparse algorithm, and in most cases, it is much lower than the latest symmetric low-rank algorithm;

We employ an optimization algorithm based on the staggered orientation method to solve the SCSLR model. We can get the convergence of the optimization algorithm on different datasets. Convergence is expressed using the value of the objective function and the number of iterations. We can see that the curve generally decreases as the number of iterations increases. The convergence curve is shown in Figure 4. It shows that the algorithm based on ADM has good convergence.

Therefore, it can be shown that the SCSLR algorithm is superior to the previous low-rank or sparse-based methods in terms of accuracy and stability; and the SCSLR algorithm also appears to be more excellent in clustering performance.

Table 1: The performance of each algorithm on the Yale dataset

Table 2: The performance of each algorithm on the Hopkins dataset

Claims (7)

1. A subspace clustering method based on symmetric low-rank representation of structural constraint is characterized by comprising the following steps:

s1: acquiring a data matrix X' of an original image;

s2: carrying out PCA (principal component analysis) dimensionality reduction on the data matrix X' obtained in the step S1 to obtain X;

s3: constructing an information error correction matrix R of the data matrix X obtained in the step S2 by using the angle decision weight;

s4: inputting the data matrix X obtained in the step S2 and the error correction matrix R obtained in the step S3 into a constructed symmetrical low-rank model based on structural constraint, and optimizing an output representation matrix Z;

s5: obtaining a weight matrix L for spectral clustering using the angle information indicating the principal direction of the matrix Z output at S4;

s6: the weight matrix L obtained in S5 is used for spectral clustering to obtain a clustering result.

2. A subspace clustering method based on symmetric low rank representation of structural constraint, wherein in step S1: the obtained data matrix X' for the original image is [ X ]₁，x₂，…x_n]∈R^d×nIn expression, let X' be from k subspaces

Obtained, each sonSpace i contains n_iIndividual data sample

The task is to combine the data points x_iClustering into subspaces S extracted therefrom_iIn (1).

3. A subspace clustering method based on symmetric low rank representation of structural constraint, wherein in step S2: and (5) carrying out dimensionality reduction on the data matrix X' obtained in the step (S1) by utilizing PCA to obtain a matrix data matrix X.

4. A subspace clustering method based on symmetric low rank representation of structural constraint, wherein in step S3: and (3) constructing an information error correction matrix R by using an angle decision weight method:

the weight between data points can be determined by the angle between data points, and if the angle between data points is small, they are likely to belong to the same class, it can be inferred that the weight of the samples within the class is small, then the data is normalized and the absolute value between them is calculated, so the ideal R is as follows:

wherein

And

is x_iAnd x_jNormalizing the data points and empirically setting σ

The mean of the elements.

5. A subspace clustering method based on symmetric low rank representation of structural constraint, wherein in step S4: constructing a structural constrained symmetric low-rank representation model:

it is a natural idea to put constraints on the structure of solutions for LRR models by adding Σ in the objective function in order to have both symmetric and structural constraints on low rank representations_i，jR_ij|Z_ijI and Z_ij＝Z_jiTo limit the structure of the solution, while, in order to make the obtained Z more robust to noise and avoid NP problems, a symmetric low rank representation model with structural constraints is as follows:

s.t.X＝AZ+E，Z＝Z^T

a symmetrical low-rank representation model with structural constraint is constructed, the low-rank representation is simultaneously subjected to symmetrical constraint and structural constraint to obtain a representation coefficient Z with more block diagonal, and the solution is to update and optimize a representation matrix Z and an error matrix E by introducing two auxiliary variables J and L and using an ADM method to the model.

6. A subspace clustering method based on symmetric low rank representation of structural constraint, wherein in step S5: obtaining a weight matrix L for spectral clustering using the angle information indicating the principal direction of the matrix Z output at S5, specifically, Z ═ U^*∑^*(V^*)^TThen define M ═ U^*(∑^*)^1/2，N＝(∑^*)^1/2(V^*)^TThe affinity graph matrix L is defined by using angle information from all row vectors of matrix M or all column vectors of matrix N.

7. A subspace clustering method based on symmetric low rank representation of structural constraint, wherein in step S6: the weight matrix L obtained in S6 is used for spectral clustering to obtain a clustering result.

Images