CN109840545A - A kind of robustness structure Non-negative Matrix Factorization clustering method based on figure regularization - Google Patents
A kind of robustness structure Non-negative Matrix Factorization clustering method based on figure regularization Download PDFInfo
- Publication number
- CN109840545A CN109840545A CN201811597620.2A CN201811597620A CN109840545A CN 109840545 A CN109840545 A CN 109840545A CN 201811597620 A CN201811597620 A CN 201811597620A CN 109840545 A CN109840545 A CN 109840545A
- Authority
- CN
- China
- Prior art keywords
- matrix
- graph
- coefficient
- regularization
- data
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 239000011159 matrix material Substances 0.000 title claims abstract description 109
- 238000000034 method Methods 0.000 title claims abstract description 28
- 238000004422 calculation algorithm Methods 0.000 claims abstract description 5
- 238000003064 k means clustering Methods 0.000 claims abstract description 4
- 238000000354 decomposition reaction Methods 0.000 claims description 5
- 239000003550 marker Substances 0.000 claims description 4
- 238000013459 approach Methods 0.000 claims description 2
- 230000001568 sexual effect Effects 0.000 claims 1
- HUTDUHSNJYTCAR-UHFFFAOYSA-N ancymidol Chemical compound C1=CC(OC)=CC=C1C(O)(C=1C=NC=NC=1)C1CC1 HUTDUHSNJYTCAR-UHFFFAOYSA-N 0.000 abstract 1
- 230000006870 function Effects 0.000 description 27
- 230000009467 reduction Effects 0.000 description 3
- 230000006978 adaptation Effects 0.000 description 2
- 238000012986 modification Methods 0.000 description 2
- 230000004048 modification Effects 0.000 description 2
- 238000012152 algorithmic method Methods 0.000 description 1
- 230000015572 biosynthetic process Effects 0.000 description 1
- 238000005259 measurement Methods 0.000 description 1
- 230000007246 mechanism Effects 0.000 description 1
- 238000003909 pattern recognition Methods 0.000 description 1
- 238000012545 processing Methods 0.000 description 1
- 238000011160 research Methods 0.000 description 1
- 238000000638 solvent extraction Methods 0.000 description 1
Landscapes
- Image Analysis (AREA)
Abstract
The present invention provides a kind of robustness structure Non-negative Matrix Factorization clustering method based on figure regularization, comprising: S10 obtains m images to be clustered, and according to image configuration k to be clustered closest figures；S20 obtains corresponding data matrix Y for each closest figure, includes n data point in data matrix Y, is decomposed to obtain eigenmatrix W and coefficient matrix H to data matrix Y using non-negative matrix factorization method；S30 is based on l2,pThe objective function O of robustness structure Non-negative Matrix Factorization of the norm foundation based on figure regularization；For S40 according to objective function O, the method iteration preset times weighted using iteration are updated eigenmatrix W, coefficient entry and figure regular terms；Using k-means clustering algorithm, eigenmatrix W obtained to each arest neighbors figure is analyzed and is clustered S50 respectively.It uses robust loss function to measure reconstructed error therein, flag data is not used to be differentiated in the robust loss function, after introducing the semi-supervised method of Non-negative Matrix Factorization, can effectively improve efficiency and accurate rate.
Description
Technical Field
The invention relates to the technical field of image processing, in particular to a non-negative matrix factorization clustering method based on graph regularization.
Background
In recent years, high-dimensional data appears in many fields, such as multimedia analysis, computer vision, pattern recognition, etc., and how to perform dimension reduction operation becomes a research topic. Non-negative matrix factorization, a commonly used dimension reduction method, aims at learning a feature representation based on locality, although it provides a large number of problem formation techniques and algorithmic methods for clustering problems, and is widely used in various applications. However, this dimensionality reduction method cannot guarantee good clustering performance, and therefore, a non-negative matrix variable is proposed. The basic idea of non-negative matrix factorization is to learn two non-negative matrices closest to the original matrix, i.e. the original data matrix is reconstructed using only addition operations, without any subtraction combination of learning factors, and thus a partial-based data representation is obtained.
In semi-supervised learning, the whole body composed of labeled data and unlabeled data in the same class tends to come from the same subspace, i.e., the intrinsic representations of labeled data and unlabeled data are consistent, and the representations of unlabeled data also have a block diagonal structure. In practical applications, data is always polluted by noise and outliers, which may greatly affect the finding of the diagonal structure of a block and reduce the performance of the block.
Disclosure of Invention
Aiming at the problems, the invention provides a graph regularization-based non-negative matrix factorization clustering method for a robust structure, which effectively solves the technical problems that the discovery of a block diagonal structure is possibly influenced and the performance of the block diagonal structure is possibly reduced after data is polluted by noise and outliers in the prior art.
The technical scheme provided by the invention is as follows:
a robustness structure non-negative matrix factorization clustering method based on graph regularization comprises the following steps:
s10, acquiring m images to be clustered, and constructing k nearest graphs according to the images to be clustered;
s20, obtaining a corresponding data matrix Y for each nearest graph, wherein the data matrix Y comprises n data points, and decomposing the data matrix Y by using a non-negative matrix decomposition method to obtain a feature matrix W and a coefficient matrix H;
s30 is based onThe norm establishes a target function O based on the graph regularization robust structure non-negative matrix factorization;
wherein,represents a regularization term, mu represents a base coefficient, Z represents an index matrix in the regularization term, a matrix of marker data points is represented,a zero matrix representing unlabeled data points;representing coefficient terms, β representing sparse coefficients,. lambda.Tr (W)TLWW) and μ Tr (H)TLHH) Denotes the graph regularization term, λ denotes the low rank coefficient, LW＝EW-VWAnd LH＝EH-VHThe method comprises the following steps of (1) taking Laplace equation as a basis, wherein V is a weight matrix of a constructed nearest neighbor graph, and E is a diagonal matrix;
s40, according to the objective function O, using the iterative weighting method to iterate the preset times, and carrying out the iteration on the characteristic matrix W and the coefficient itemsAnd graph regularization term λ Tr (W)TLWW) and μ Tr (H)TLHH) Updating is carried out;
s50, analyzing and clustering the feature matrix W obtained by each nearest neighbor graph by adopting a k-means clustering algorithm.
The non-negative matrix factorization clustering method of the robustness structure based on graph regularization, provided by the invention, is used as an unsupervised learning mechanism, and is used for evaluating through a specific index (a preset index matrix) to seek to carry out proper partitioning on similar unmarked data points. For the problem that noise and outlier possibly influence the discovery of a block diagonal structure in the prior art, a robust loss function is adopted to measure reconstruction errors in the robust loss function, marking data are not used in the robust loss function for distinguishing, and after a semi-supervision method of nonnegative matrix decomposition is introduced, the efficiency and the accuracy rate can be effectively improved.
Drawings
The foregoing features, technical features, advantages and embodiments are further described in the following detailed description of the preferred embodiments, which is to be read in connection with the accompanying drawings.
FIG. 1 is a schematic flow chart of a robust structure non-negative matrix factorization clustering method based on graph regularization.
Detailed Description
In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following description will be made with reference to the accompanying drawings. It is obvious that the drawings in the following description are only some examples of the invention, and that for a person skilled in the art, other drawings and embodiments can be derived from them without inventive effort.
For an arbitrary matrix B, BiThe ith row vector, B, representing matrix BiDenoted as the ith column vector of matrix B. If matrix B is a square matrix, Tr [ B ] is used]Representing the locus of the matrix B, the transpose of the matrix B being denoted BT. When p is>At 0, define vector b ∈ RmIs/are as followsNorm ofThe matrix B belongs to Rm×nThe Frobenius norm ofOf matrix BNorm of
When 0 is present<p<At 1 time, mixingThe (q ═ 2) norm is not a valid matrix norm and therefore it does not admit the triangle inequality.
Consider a data set containing n data pointsData matrix Y ═ Y1,...,yn]，yi∈RdThe non-negative matrix factorization aims at finding two non-negative matricesAndthe best approximate original data matrix Y has a plurality of divergence function measurement reconstruction errors, and an objective function O is established based on a Non-Negative Matrix Factorization (NMF) and a Frobenius norm1As shown in formula (1):
in semi-supervised learning, if L (L < n) labeled samples and U unlabeled samples are included in n data points, thenWherein,a matrix of marker data points is represented,representing an unmarked dot data matrix. Assuming that the data points are all sampled from class C,wherein each labeled data point is labeled with a class label, thenWherein,with n of the class CiA feature matrix of the data points, which may be obtained,n is L + U. The dimension of each subspace is set to m, and r ═ mC is defined,n representing the C-th layer subspacecData points. For the display pursuit of block diagonalization, regularization terms are implemented on the basis of non-negative matrix factorizationObtaining an objective function O as shown in formula (2)2：
Where loss () represents a loss function and μ represents a base coefficient.
In practical application, the data is often polluted by noise and outliers, which further affects the discovery of diagonal structures and reduces the performance thereof, and the invention is based on the following(p>0) The norm establishes a robust loss function in an objective function O2Obtaining an objective function O as shown in formula (3)3：
It is an object of the present invention to find the diagonal structure, and therefore, to achieve the goal using a simple function, for the regularization termDefining an index matrixWherein,a zero matrix representing the unmarked data points,and (3) representing the marked data point matrix to obtain a regularization term as shown in a formula (4):
according to the regularization term, an objective function is obtained as shown in formula (5):
in fact, the block diagonal structure can be regarded as a structural integral form, and can affect the representation of all classes, so that the invention further introduces the constraint of sparse constraint on the coefficient matrix factors of the nonnegative matrix factorization objective function to control the coefficient degree of the block diagonal structure. In a non-negative matrix framework based on alternating non-negative least squares, usingAnd (3) the norm regularization controls the sparsity of the system (5), and a target function as shown in the formula (6) is obtained after a coefficient term is introduced:
where β is a sparse coefficient.
Considering the geometric structure of the data manifold and the functional manifold, two charts are respectively created to reflect the structural distribution of the manifold, so a graph regular term is introduced on the basis of the formula (6) to obtain a formula (7):
wherein λ represents a low rank coefficient, LW＝EW-VWAnd LH＝EH-VHIn Laplace's equation, V is the weight matrix of the constructed nearest neighbor graph, and E is the diagonal matrix.
In equation (7), neither of the matrices H and W is convex or smooth, so it is optimized using an iterative multiplication algorithm. Since the loss function is robust to noise and outliers, and(0<p<1) ratio of normNorm is more sparse, so it is right(0<p ≦ 1) and an objective function O of equation (8) can be obtained by limiting the mixing function, and for the objective function of equation (7), let I ═ Y-HW:
because H and W are more than or equal to 0, restricted Lagrange multipliers psi and phi are introduced, so that the coefficient matrix H is more than 0, and the characteristic matrix W is more than 0. Let Ψ ═ ψ ], Φ ═ Φ ], and the lagrangian function L of the objective function O is as in equation (9):
according to matrix propertiesThe partial derivative of the lagrange function L on the coefficient matrix H is as follows (10):
wherein, the diagonal matrixikIn the diagonal matrix D, the value approaches 0, and the value can be obtained
The partial derivative of the lagrange function L on the characteristic matrix W is as follows (11):
let equations (10) and (11) be equal to 0, and let ψ H be 0 and Φ W be 0, to obtain equations (12) and (13):
(HWDWT)H-(YDWT)H＝0(12)
(HTHWD-HTYD)W+μ(Z⊙W)W＝0(13)
to obtain the following formulae (14) and (15):
therefore, a coefficient matrix H and a feature matrix W update rule are obtained, and the formula is (16) and (17):
based on this, the update rules of the coefficient term and the graph regularization term are as formulas (18) and (19):
in the objective function O, LH＝EH-VH，LW＝EW-VWIs Laplace equation, V is a weight matrix, E is a diagonal matrix, parameters mu and lambda are both larger than 0, and the updating rules of the coefficient term and the graph regular term are as shown in formulas (20) and (21):
note that H and W in the formulas (20) and (21) are substantially the coefficient matrix H and the feature matrix W, and are represented in lower case here for the sake of distinction.
When the decomposition of the robustness structure non-negative matrix based on graph regularization is applied to image clustering, the method comprises the following steps:
s10, acquiring m images to be clustered, and constructing k nearest graphs according to the images to be clustered;
s20, obtaining a corresponding data matrix Y for each nearest graph, wherein the data matrix Y comprises n data points, and decomposing the data matrix Y by using a non-negative matrix decomposition method to obtain a feature matrix W and a coefficient matrix H;
s30 is based onThe norm establishes a target function O based on the graph regularization robust structure non-negative matrix factorization;
wherein,represents a regularization term, mu represents a base coefficient, Z represents an index matrix in the regularization term, a matrix of marker data points is represented,a zero matrix representing unlabeled data points;representing coefficient terms, β representing sparse coefficients,. lambda.Tr (W)TLWW) and μ Tr (H)TLHH) Denotes the graph regularization term, λ denotes the low rank coefficient, LW＝EW-VWAnd LH＝EH-VHThe method comprises the following steps of (1) taking Laplace equation as a basis, wherein V is a weight matrix of a constructed nearest neighbor graph, and E is a diagonal matrix;
s40, according to the objective function O, using the iterative weighting method to iterate the preset times, and carrying out the iteration on the characteristic matrix W and the coefficient itemsAnd graph regularization term λ Tr (W)TLWW) and μ Tr (H)TLHH) Updating is carried out;
s50, analyzing and clustering the feature matrix W obtained by each nearest neighbor graph by adopting a k-means clustering algorithm.
It should be noted that the above embodiments can be freely combined as necessary. The foregoing is only a preferred embodiment of the present invention, and it should be noted that, for persons skilled in the art, numerous modifications and adaptations can be made without departing from the principle of the present invention, and such modifications and adaptations should be considered as within the scope of the present invention.
Claims (3)
1. A robustness structure non-negative matrix factorization clustering method based on graph regularization is characterized by comprising the following steps:
s10, acquiring m images to be clustered, and constructing k nearest graphs according to the images to be clustered;
s20, obtaining a corresponding data matrix Y for each nearest graph, wherein the data matrix Y comprises n data points, and decomposing the data matrix Y by using a non-negative matrix decomposition method to obtain a feature matrix W and a coefficient matrix H;
s30 is based on l2,pNorm establishment graph regularization-based robustAn objective function O of non-negative matrix factorization of the sexual structure;
wherein,represents a regularization term, mu represents a base coefficient, Z represents an index matrix in the regularization term, a matrix of marker data points is represented,a zero matrix representing unlabeled data points;representing coefficient terms, β representing sparse coefficients,. lambda.Tr (W)TLWW) and μ Tr (H)TLHH) Denotes the graph regularization term, λ denotes the low rank coefficient, LW＝EW-VWAnd LH＝EH-VHThe method comprises the following steps of (1) taking Laplace equation as a basis, wherein V is a weight matrix of a constructed nearest neighbor graph, and E is a diagonal matrix;
s40, according to the objective function O, using the iterative weighting method to iterate the preset times, and carrying out the iteration on the characteristic matrix W and the coefficient itemsAnd graph regularization term λ Tr (W)TLWW) and μ Tr (H)TLHH) Updating is carried out;
s50, analyzing and clustering the feature matrix W obtained by each nearest neighbor graph by adopting a k-means clustering algorithm.
2. The clustering method according to claim 1, wherein in step S40, the update rule of the feature matrix W is:
wherein,ikin the diagonal matrix D, the value thereof approaches 0.
3. The clustering method according to claim 1, wherein in step S40, coefficient termsAnd graph regularization term λ Tr (W)TLWW) and μ Tr (H)TLHH) The update rule of (1) is:
wherein L isW＝EW-VWAnd LH＝EH-VHIn Laplace's equation, V is the weight matrix of the constructed nearest neighbor graph, and E is the diagonal matrix.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201811597620.2A CN109840545A (en) | 2018-12-26 | 2018-12-26 | A kind of robustness structure Non-negative Matrix Factorization clustering method based on figure regularization |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201811597620.2A CN109840545A (en) | 2018-12-26 | 2018-12-26 | A kind of robustness structure Non-negative Matrix Factorization clustering method based on figure regularization |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN109840545A true CN109840545A (en) | 2019-06-04 |
Family
ID=66883376
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201811597620.2A Pending CN109840545A (en) | 2018-12-26 | 2018-12-26 | A kind of robustness structure Non-negative Matrix Factorization clustering method based on figure regularization |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN109840545A (en) |
Cited By (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110717538A (en) * | 2019-10-08 | 2020-01-21 | 广东工业大学 | Color picture clustering method based on non-negative tensor ring |
| CN110990775A (en) * | 2019-11-28 | 2020-04-10 | 江苏理工学院 | Multi-view clustering method based on multi-manifold-form dual graph regularization non-negative matrix factorization |
| CN111191719A (en) * | 2019-12-27 | 2020-05-22 | 北京工业大学 | Image clustering method based on self-expression and atlas constraint non-negative matrix factorization |
| CN112667863B (en) * | 2021-01-16 | 2024-02-02 | 北京工业大学 | Financial fraud group identification method based on hypergraph segmentation |
-
2018
- 2018-12-26 CN CN201811597620.2A patent/CN109840545A/en active Pending
Cited By (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110717538A (en) * | 2019-10-08 | 2020-01-21 | 广东工业大学 | Color picture clustering method based on non-negative tensor ring |
| CN110717538B (en) * | 2019-10-08 | 2022-06-24 | 广东工业大学 | Color picture clustering method based on non-negative tensor ring |
| CN110990775A (en) * | 2019-11-28 | 2020-04-10 | 江苏理工学院 | Multi-view clustering method based on multi-manifold-form dual graph regularization non-negative matrix factorization |
| CN110990775B (en) * | 2019-11-28 | 2023-11-07 | 江苏理工学院 | Multi-view clustering method based on multi-manifold dual graph regularized non-negative matrix factorization |
| CN111191719A (en) * | 2019-12-27 | 2020-05-22 | 北京工业大学 | Image clustering method based on self-expression and atlas constraint non-negative matrix factorization |
| CN111191719B (en) * | 2019-12-27 | 2023-09-05 | 北京工业大学 | Image clustering method based on self-representation and map constraint and adopting non-negative matrix factorization |
| CN112667863B (en) * | 2021-01-16 | 2024-02-02 | 北京工业大学 | Financial fraud group identification method based on hypergraph segmentation |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN109840545A (en) | A kind of robustness structure Non-negative Matrix Factorization clustering method based on figure regularization | |
| Deng et al. | On-line pattern analysis by evolving self-organizing maps | |
| Hoyer | Non-negative matrix factorization with sparseness constraints. | |
| JP6192010B2 (en) | Weight setting apparatus and method | |
| CN108776812A (en) | Multiple view clustering method based on Non-negative Matrix Factorization and various-consistency | |
| Lam et al. | Clustering | |
| CN110889865B (en) | Video target tracking method based on local weighted sparse feature selection | |
| CN109657611B (en) | Adaptive image regularization non-negative matrix decomposition method for face recognition | |
| Peng et al. | Correntropy based graph regularized concept factorization for clustering | |
| CN108415883A (en) | Convex non-negative matrix factorization method based on subspace clustering | |
| Wei et al. | Compact MQDF classifiers using sparse coding for handwritten Chinese character recognition | |
| Zhang et al. | An efficient framework for unsupervised feature selection | |
| Peng et al. | Hyperplane-based nonnegative matrix factorization with label information | |
| CN114254703A (en) | Robust local and global regularization non-negative matrix factorization clustering method | |
| Chen et al. | Variational Graph Embedding and Clustering with Laplacian Eigenmaps. | |
| Mahapatra et al. | S-isomap++: Multi manifold learning from streaming data | |
| CN110276049A (en) | A kind of semi-supervised adaptive figure regularized discriminant non-negative matrix factorization method | |
| Li et al. | Dimensionality reduction with sparse locality for principal component analysis | |
| CN109615026B (en) | Discrimination projection method based on sparse regularization and image recognition device | |
| CN114037931B (en) | Multi-view discriminating method of self-adaptive weight | |
| Ma et al. | BYY harmony learning on finite mixture: adaptive gradient implementation and a floating RPCL mechanism | |
| CN109614581A (en) | The Non-negative Matrix Factorization clustering method locally learnt based on antithesis | |
| Zhang et al. | Adaptive graph-based discriminative nonnegative matrix factorization for image clustering | |
| Xie et al. | Multiscale online tracking of manifolds | |
| Bersson et al. | Bayesian covariance estimation for multi-group matrix-variate data |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20190604 |