CN109840545A - A kind of robustness structure Non-negative Matrix Factorization clustering method based on figure regularization - Google Patents

Abstract

The present invention provides a kind of robustness structure Non-negative Matrix Factorization clustering method based on figure regularization, comprising: S10 obtains m images to be clustered, and according to image configuration k to be clustered closest figures；S20 obtains corresponding data matrix Y for each closest figure, includes n data point in data matrix Y, is decomposed to obtain eigenmatrix W and coefficient matrix H to data matrix Y using non-negative matrix factorization method；S30 is based on l_2,pThe objective function O of robustness structure Non-negative Matrix Factorization of the norm foundation based on figure regularization；For S40 according to objective function O, the method iteration preset times weighted using iteration are updated eigenmatrix W, coefficient entry and figure regular terms；Using k-means clustering algorithm, eigenmatrix W obtained to each arest neighbors figure is analyzed and is clustered S50 respectively.It uses robust loss function to measure reconstructed error therein, flag data is not used to be differentiated in the robust loss function, after introducing the semi-supervised method of Non-negative Matrix Factorization, can effectively improve efficiency and accurate rate.

Description

A robust structural non-negative matrix factorization clustering method based on graph regularization

technical field

The invention relates to the technical field of image processing, in particular to a non-negative matrix decomposition clustering method based on graph regularization.

Background technique

In recent years, high-dimensional data has appeared in many fields, such as multimedia analysis, computer vision, pattern recognition, etc., how to perform dimensionality reduction operations has become a research topic. As a commonly used dimensionality reduction method, non-negative matrix factorization aims to learn local-based feature representations, although it provides a large number of problem formation techniques and algorithmic methods for clustering problems and is widely used in various applications. However, this dimensionality reduction method cannot guarantee good clustering performance, therefore, non-negative matrix variables are proposed. The basic idea of non-negative matrix factorization is to learn two non-negative matrices that are closest to the original matrix, that is, the original data matrix is reconstructed using only addition operations, without any subtractive combination of learning factors, and then a part-based data representation is obtained.

In semi-supervised learning, the whole composed of labeled data and unlabeled data in the same class often comes from the same subspace, that is, the intrinsic representations of labeled data and unlabeled data are consistent, and the representation of unlabeled data also has a block-diagonal structure . In practical applications, the data is always polluted by noise and outliers, which may greatly affect the discovery of block diagonal structure and reduce its performance.

SUMMARY OF THE INVENTION

In view of the above problems, the present invention provides a robust structural non-negative matrix factorization clustering method based on graph regularization, which effectively solves the problem that in the prior art, after the data is polluted by noise and outliers, the block diagonal may be affected. The discovery of the structure and the technical problems of reducing its performance.

The technical scheme provided by the present invention is as follows:

A robust structural non-negative matrix factorization clustering method based on graph regularization, including:

S10 obtains m images to be clustered, and constructs k nearest neighbor graphs according to the images to be clustered;

S20 obtains a corresponding data matrix Y for each nearest neighbor graph, the data matrix Y includes n data points, and uses a non-negative matrix decomposition method to decompose the data matrix Y to obtain a feature matrix W and a coefficient matrix H;

The S30 is based on The norm establishes the objective function O for robust structural non-negative matrix factorization based on graph regularization;

in, represents the regularization term, μ represents the base coefficient, Z represents the index matrix in the regularization term, represents a matrix of labeled data points, a matrix of zeros representing unlabeled data points; represents the coefficient term, β represents the sparse coefficient; λTr(W ^T L _W W) and μTr(H ^T L _H H) represent the graph regularization term, λ represents the low-rank coefficient, L ^W =E ^W -V ^W and L ^H =E ^H -V ^H is the Laplace equation, V is the weight matrix of the constructed nearest neighbor graph, and E is the diagonal matrix;

S40, according to the objective function O, use the iterative weighting method to iterate a preset number of times, and perform the iteration on the feature matrix W, the coefficient term And the graph regular terms λTr(W ^T L _W W) and μTr(H ^T L _H H) are updated;

S50 uses the k-means clustering algorithm to analyze and cluster the feature matrix W obtained by each nearest neighbor graph respectively.

The robust structural non-negative matrix factorization clustering method based on graph regularization provided by the present invention, as an unsupervised learning mechanism, evaluates through a specific index (pre-set index matrix) to seek for those with similar Appropriate partitioning of unlabeled data points, the method encodes the geometric information of the data space, not only preserving the inherent geometric structure, but also obtaining its hidden compact representation. For the problem that noise and outliers in the prior art may affect the discovery of block-diagonal structures, a robust loss function is used to measure the reconstruction error. In this robust loss function, labeled data is not used for discrimination, and the introduction of The semi-supervised method of non-negative matrix factorization can effectively improve the efficiency and accuracy.

Description of drawings

The preferred embodiments will be described below in a clear and easy-to-understand manner with reference to the accompanying drawings, and the above-mentioned characteristics, technical features, advantages and implementations thereof will be further described.

FIG. 1 is a schematic flowchart of the robust structural non-negative matrix factorization clustering method based on graph regularization in the present invention.

Detailed ways

In order to more clearly describe the embodiments of the present invention or the technical solutions in the prior art, the specific embodiments of the present invention will be described below with reference to the accompanying drawings. Obviously, the accompanying drawings in the following description are only some embodiments of the present invention. For those of ordinary skill in the art, other drawings can also be obtained from these drawings without creative efforts, and obtain other implementations.

For any matrix B, b _i represents the ith row vector of matrix B, and b ⁱ represents the ith column vector of matrix B. If matrix B is a square matrix, use Tr[B] to represent the trajectory of matrix B, and the transpose matrix of matrix B is represented as B ^T . When p>0, define the vector ^b∈Rm norm is The Frobenius norm of a matrix B ∈ R ^m×n is matrix B norm is

When 0<p<1, mix The (q=2) norm is not a valid matrix norm, so it does not recognize trigonometric inequalities.

Consider a dataset with n data points Data matrix Y=[y ₁ ,...,y _n ], y _i ∈R ^d represents the i-th feature description sample, the purpose of non-negative matrix decomposition is to find two non-negative matrices and The best approximate original data matrix Y has many divergence functions to measure the reconstruction error, then the objective function O ₁ established based on NMF (Non-negative matrix factorization, non-negative matrix factorization) and Frobenius norm is as formula (1):

In semi-supervised learning, if n data points contain L (L<<n) labeled samples and U unlabeled samples, then in, represents a matrix of labeled data points, Represents an unlabeled point data matrix. Assuming that the data points are all sampled from class C, Each labeled data point in is labeled with a class label, then in, With the feature matrix of n _i data points of the C-th class, we can get, n=L+U. Let the dimension of each subspace be m, and define r=mC, represents the n _c data points of the C-th subspace. To explicitly pursue block diagonalization, a regularization term is implemented based on non-negative matrix factorization The objective function O ₂ of formula (2) is obtained:

Among them, loss() represents the loss function, and μ represents the base coefficient.

In practical applications, data are often polluted by noise and outliers, which in turn affect the discovery of diagonal structures and reduce their performance. Based on this, the present invention is based on (p>0) norm establishes a robust loss function, and on the basis of the objective function O ₂ , the objective function O ₃ as shown in formula (3) is obtained:

Since the purpose of the present invention is to find the diagonal structure, therefore, a simple function is used to achieve the goal, for the regularization term define an indicator matrix in, a matrix of zeros representing unlabeled data points, Represents the matrix of labeled data points, and obtains the regularization term as in Eq. (4):

According to the regularization term, the objective function of equation (5) is obtained:

In fact, the block-shaped diagonal structure can be regarded as a structured overall form, which can affect the representation of all classes. Therefore, the present invention further introduces sparse constraints on the non-negative matrix factorization objective function coefficient matrix factor constraints to control its coefficients degree. In a nonnegative matrix framework based on alternating nonnegative least squares, use Norm regularization controls the sparsity of equation (5). After introducing the coefficient term, the objective function of equation (6) is obtained:

where β is the sparse coefficient.

Considering the geometric structure of the data manifold and the functional manifold, two graphs are created to reflect the structural distribution of the manifold. Therefore, on the basis of Equation (6), a graph regular term is introduced to obtain Equation (7):

Among them, λ represents a low-rank coefficient, L ^W =E ^W -V ^W and L ^H =E ^H -V ^H are Laplace equations, V is the weight matrix of the constructed nearest neighbor graph, and E is a diagonal matrix.

In equation (7), the matrices H and W are not convex or smooth, so an iterative multiplication algorithm is used to optimize them. Since the loss function is robust to noise and outliers, and (0<p<1) norm ratio The norm is more sparse, so (0<p≤1) is limited by the mixed function. For the objective function of formula (7), let I=Y-HW, and the objective function O of formula (8) can be obtained:

Since H, W≥0, the constrained Lagrangian multipliers ψ and φ are introduced, so that the coefficient matrix H>0 and the feature matrix W>0. Let Ψ=[ψ], Φ=[φ], the Lagrangian function L of the objective function O is as formula (9):

According to the properties of the matrix The partial derivative of the Lagrangian function L with respect to the coefficient matrix H is shown in formula (10):

Among them, the diagonal matrix i _k is in the diagonal matrix D, and its value is close to 0, we can get

The partial derivative of the Lagrangian function L with respect to the eigenmatrix W is shown in formula (11):

Setting equations (10) and (11) equal to 0, and setting ψH = 0 and φW = 0, yields equations (12) and (13):

(HWDW ^T )H-(YDW ^T )H=0(12)

(H ^T HWD-H ^T YD)W+μ(Z⊙W)W=0(13)

Then formulas (14) and (15) are obtained:

In this way, the coefficient matrix H and feature matrix W update rules are obtained, such as formulas (16) and (17):

Based on this, the update rules for coefficient terms and graph regular terms are as follows (18) and (19):

In the objective function O, L ^H =E ^H -V ^H , L ^W =E ^W -V ^W is the Laplace equation, V is the weight matrix, E is the diagonal matrix, the parameters μ and λ are both greater than 0, and the coefficient The update rules of terms and graph regular terms are as formulas (20) and (21):

It should be noted that h and w in formulas (20) and (21) are essentially the coefficient matrix H and the feature matrix W, which are denoted by lowercase here in order to express distinction.

Based on the above-mentioned decomposition of the robust structural non-negative matrix based on graph regularization, when applied to image clustering, it includes the following steps:

S10 obtains m images to be clustered, and constructs k nearest neighbor graphs according to the images to be clustered;

S20 obtains the corresponding data matrix Y for each nearest neighbor graph, includes n data points in the data matrix Y, and uses the non-negative matrix decomposition method to decompose the data matrix Y to obtain the characteristic matrix W and the coefficient matrix H;

The S30 is based on The norm establishes the objective function O for robust structural non-negative matrix factorization based on graph regularization;

in, represents the regularization term, μ represents the base coefficient, Z represents the index matrix in the regularization term, represents a matrix of labeled data points, a matrix of zeros representing unlabeled data points; represents the coefficient term, β represents the sparse coefficient; λTr(W ^T L _W W) and μTr(H ^T L _H H) represent the graph regularization term, λ represents the low-rank coefficient, L ^W =E ^W -V ^W and L ^H =E ^H -V ^H is the Laplace equation, V is the weight matrix of the constructed nearest neighbor graph, and E is the diagonal matrix;

S40, according to the objective function O, use the iterative weighting method to iterate a preset number of times, and perform the iteration on the feature matrix W, the coefficient term And the graph regular terms λTr(W ^T L _W W) and μTr(H ^T L _H H) are updated;

S50 uses the k-means clustering algorithm to analyze and cluster the feature matrix W obtained by each nearest neighbor graph respectively.

It should be noted that the above embodiments can be freely combined as required. The above are only the preferred embodiments of the present invention. It should be pointed out that, for those of ordinary skill in the art, several improvements and modifications can be made without departing from the principles of the present invention, and these improvements and modifications should also be regarded as It is the protection scope of the present invention.

Claims (3)

1. A robustness structure non-negative matrix factorization clustering method based on graph regularization is characterized by comprising the following steps:

s10, acquiring m images to be clustered, and constructing k nearest graphs according to the images to be clustered;

s20, obtaining a corresponding data matrix Y for each nearest graph, wherein the data matrix Y comprises n data points, and decomposing the data matrix Y by using a non-negative matrix decomposition method to obtain a feature matrix W and a coefficient matrix H;

s30 is based on l_2,pNorm establishment graph regularization-based robustAn objective function O of non-negative matrix factorization of the sexual structure;

wherein,represents a regularization term, mu represents a base coefficient, Z represents an index matrix in the regularization term, a matrix of marker data points is represented,a zero matrix representing unlabeled data points;representing coefficient terms, β representing sparse coefficients,. lambda.Tr (W)^TL_WW) and μ Tr (H)^TL_HH) Denotes the graph regularization term, λ denotes the low rank coefficient, L^W＝E^W-V^WAnd L^H＝E^H-V^HThe method comprises the following steps of (1) taking Laplace equation as a basis, wherein V is a weight matrix of a constructed nearest neighbor graph, and E is a diagonal matrix;

s40, according to the objective function O, using the iterative weighting method to iterate the preset times, and carrying out the iteration on the characteristic matrix W and the coefficient itemsAnd graph regularization term λ Tr (W)^TL_WW) and μ Tr (H)^TL_HH) Updating is carried out;

s50, analyzing and clustering the feature matrix W obtained by each nearest neighbor graph by adopting a k-means clustering algorithm.

2. The clustering method according to claim 1, wherein in step S40, the update rule of the feature matrix W is:

wherein,i_kin the diagonal matrix D, the value thereof approaches 0.

3. The clustering method according to claim 1, wherein in step S40, coefficient termsAnd graph regularization term λ Tr (W)^TL_WW) and μ Tr (H)^TL_HH) The update rule of (1) is:

wherein L is^W＝E^W-V^WAnd L^H＝E^H-V^HIn Laplace's equation, V is the weight matrix of the constructed nearest neighbor graph, and E is the diagonal matrix.