CN116188829A - Image clustering method based on hypergraph regular depth non-negative matrix factorization - Google Patents
Image clustering method based on hypergraph regular depth non-negative matrix factorization Download PDFInfo
- Publication number
- CN116188829A CN116188829A CN202211588025.9A CN202211588025A CN116188829A CN 116188829 A CN116188829 A CN 116188829A CN 202211588025 A CN202211588025 A CN 202211588025A CN 116188829 A CN116188829 A CN 116188829A
- Authority
- CN
- China
- Prior art keywords
- matrix
- hypergraph
- representation
- layer
- base
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 239000011159 matrix material Substances 0.000 title claims abstract description 186
- 238000000034 method Methods 0.000 title claims abstract description 19
- 238000000354 decomposition reaction Methods 0.000 claims abstract description 16
- 230000014509 gene expression Effects 0.000 claims description 27
- 238000004364 calculation method Methods 0.000 claims description 7
- 238000003064 k means clustering Methods 0.000 claims description 5
- 230000017105 transposition Effects 0.000 claims description 3
- 230000000977 initiatory effect Effects 0.000 claims description 2
- 230000006870 function Effects 0.000 description 5
- 238000005516 engineering process Methods 0.000 description 3
- 238000000605 extraction Methods 0.000 description 2
- 238000010801 machine learning Methods 0.000 description 2
- 230000009286 beneficial effect Effects 0.000 description 1
- 238000010586 diagram Methods 0.000 description 1
- 238000005286 illumination Methods 0.000 description 1
- 238000005457 optimization Methods 0.000 description 1
- 230000003595 spectral effect Effects 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
- G06V10/762—Arrangements for image or video recognition or understanding using pattern recognition or machine learning using clustering, e.g. of similar faces in social networks
- G06V10/763—Non-hierarchical techniques, e.g. based on statistics of modelling distributions
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F17/00—Digital computing or data processing equipment or methods, specially adapted for specific functions
- G06F17/10—Complex mathematical operations
- G06F17/16—Matrix or vector computation, e.g. matrix-matrix or matrix-vector multiplication, matrix factorization
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Mathematical Physics (AREA)
- Software Systems (AREA)
- Mathematical Optimization (AREA)
- Pure & Applied Mathematics (AREA)
- Computational Mathematics (AREA)
- Mathematical Analysis (AREA)
- Data Mining & Analysis (AREA)
- Computing Systems (AREA)
- Databases & Information Systems (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Image Analysis (AREA)
- Health & Medical Sciences (AREA)
- Evolutionary Computation (AREA)
- Artificial Intelligence (AREA)
- Probability & Statistics with Applications (AREA)
- Medical Informatics (AREA)
- Algebra (AREA)
- General Health & Medical Sciences (AREA)
- General Engineering & Computer Science (AREA)
- Multimedia (AREA)
Abstract
The invention provides an image clustering method based on hypergraph regular depth non-negative matrix factorization, which is used for solving the technical problem of low accuracy of the existing clustering algorithm. The implementation steps are as follows: inputting data to be clustered; calculating and constructing a hypergraph matrix; defining and initializing base matrixes and representation matrixes of each layer of hypergraph regular depth non-negative matrix factorization; setting iteration times; acquiring each layer of base matrix of the hypergraph regular depth non-negative matrix decomposition and an updating formula of a representation matrix; updating each layer of base matrix and representing matrix of the regular depth non-negative matrix decomposition of the hypergraph; defining and calculating a low-dimensional data representation matrix; and clustering and outputting the images. The method can be used for practical applications such as image, text clustering, face recognition and the like.
Description
Technical Field
The method is used for the technical field of data clustering, and mainly relates to hypergraph regularization, a depth matrix decomposition method and a k-means clustering technology for clustering data.
Background
With the continuous development of machine learning technology, a plurality of efficient clustering algorithms are presented to mine data potential information. Wherein more extensive techniques such as non-Negative Matrix Factorization (NMF), K-means, spectral clustering are used.
However, the classical non-negative matrix factorization only performs single factorization on the data matrix, multi-angle feature expression cannot be performed on the data, and the standard graph regular term only can capture geometric structure information of low-order pairs between the data, so that hierarchical feature extraction is difficult to perform on complex data, and therefore clustering accuracy is poor.
NMF decomposes the input data matrix into a base matrix and a representation matrix, has wide application in the aspects of voice, image recognition, data dimension reduction, data representation and the like, and is also a common clustering technology in machine learning and data processing tasks.
Noun interpretation:
Karush-Kuhn-Tucker condition Karush-Coulomb-Tacke condition (English name: karush-Kuhn-Tucker Conditions common alias: kuhn-Tucker, KKT condition, karush-Kuhn-Tucker optimization condition, kuhn-Tucker condition) is a necessary and sufficient condition for a nonlinear programming (Nonlinear Programming) problem to optimize the solution under certain regular conditions.
Disclosure of Invention
In order to solve the problems, the invention discloses an image clustering method based on hypergraph regular depth non-negative matrix factorization. Thus, a more semantically rich low-dimensional subspace representation is obtained.
In order to achieve the above object, the present invention is realized by the following technical scheme:
an image clustering method based on hypergraph regular depth non-negative matrix factorization mainly comprises the following steps:
(1) Inputting image data to be clustered; graying the images in the image data, and then taking the position and gray value of each pixel point to obtain an image data matrix;
(2) Calculating a node degree matrix of the image data matrix;
(3) Calculating a supermargin matrix of the image data matrix;
(4) Calculating a hypergraph matrix of the image data matrix;
(5) Defining and initiating a hypergraph regularization depth matrix to decompose a base matrix and a representation matrix of each layer:
the base matrix W of each layer i I=1, … h and the representation matrix V of each layer i I=1 and … h, respectively performing initialization definition, wherein h is the maximum decomposition layer number;
(6) Setting the maximum iteration times t;
(7) The method for obtaining the updated formula of decomposing the base matrix and the representation matrix of each layer of the hypergraph regularization depth matrix comprises the following implementation steps:
(7.1) defining a target equation of the hypergraph regularization depth matrix decomposition, wherein the expression is as follows:
s.t H m ≥0,Z 1 ,...,Z m ≥0
wherein OHDNMF Regularizing a depth matrix-decomposed target equation for a hypergraph, wherein X represents a data matrix and Z 1 ,…,Z m Base matrices of 1 st layer to m th layer respectively, H m The m is the number of layers of depth non-negative matrix decomposition, lh is the hyper-graph Laplacian matrix, and T is the matrix transposition;
(7.2) solving Lagrangian function according to objective function of hypergraph regularized depth matrix decomposition
The expression is as follows:
wherein ,
phi is the Lagrangian multiplier; zj is the j-th layer base matrix;
(7.3) Using Lagrangian function
Respectively solving bias derivatives for the base matrix and the representation matrix, and acquiring update formulas of the base matrix and the representation matrix by using Karush-Kuhn-Tucker conditions; updating the base matrix and the representation matrix through an updating formula of the base matrix and the representation matrix;
(8) Defining and calculating a low-dimensional data representation matrix:
repeating the steps (7.2) and (7.3) until the set maximum iteration times t are reached, and obtaining a final low-dimensional data representation matrix which is H m ；
(9) Clustering and outputting images:
the final low-dimensional data representation matrix is H by using a k-means clustering algorithm m Clustering is carried out, and a clustered image is obtained and output.
Further improvement, the node degree matrix in the step (2) has a calculation formula as follows:
where V represents a set of nodes in hypergraph g= (V, E, W), E represents a set of hyperedges, W represents a set of weights for the hyperedges, if V i ∈e i Then r (v) i ,e j ) =1, otherwise r (v i ,e j )＝0。
Further improvement, the calculation formula of the superlimit matrix in the step (3) is as follows:
further improvement, the hypergraph matrix in the step (4) has a calculation formula as follows:
wherein DV ,D E Are diagonal matrices, and the element values are respectively represented by d (v i) and d(ej ) And (5) determining.
In a further improvement, in the step (7), the expressions of the base matrix and the expression matrix are respectively:
the expression of the base matrix update formula is:
the expression representing the matrix update formula is:
wherein [Q]- All positive elements in the representation matrix are replaced with 0, [ Q ]] + All negative elements in the representation matrix are replaced with 0's, i.e.:
compared with the prior art, the method has the following advantages:
firstly, carrying out deep non-negative matrix factorization on a data matrix, and carrying out hierarchical feature extraction on complex data by utilizing a deep network structure; the data matrix is decomposed only once by classical non-negative matrix decomposition, and the data cannot be subjected to multi-angle characteristic expression; thus, in comparison, deep non-negative matrix factorization can yield a better representation of data than non-negative matrix factorization.
Second, the present invention introduces hypergraph regularization terms. Standard graph regularization terms can only capture low-order pairs of geometric information between data, while hypergraphs can capture high-order geometric information between data. The hypergraph regularization term can therefore obtain a better representation of data than the standard graph regularization term.
Thirdly, on the basis of deep non-negative matrix factorization, the hypergraph regularization term is introduced, the deep network structure is utilized to extract layered characteristics of complex data, and high-order geometric structure information among the data is reserved, so that potential manifold information of the data can be fully mined, and the accuracy of image clustering is effectively improved.
Drawings
FIG. 1 is a schematic flow chart of the present invention;
FIG. 2 is a schematic diagram of an algorithm for hypergraph canonical depth matrix decomposition.
Detailed Description
The invention will now be described in further detail with reference to the drawings and to specific embodiments.
Referring to fig. 1, an image clustering method based on hypergraph regular depth non-negative matrix factorization comprises the following steps:
step 1) inputting image data of images to be clustered:
the image data set PIE contains 2856 images, 68 people, and each person has 42 face images with 4 expressions under different lighting and illumination conditions. Each image contains 32 x 32 pixels/dimension. The image data input in this embodiment is 1050 images randomly selected from the image data set PIE, and there are 25 categories, each category having 42 different images.
Step 2) computing a hypergraph matrix:
(2a) Respectively calculating a node degree matrix and a superside degree matrix, wherein the expressions are as follows:
the expression of the node degree matrix is:
the expression of the superside matrix is:
calculating to obtain a hypergraph matrix:
wherein Lh Is a hypergraph matrix, D V ,D E Respectively a node degree matrix and a superside degree matrix, wherein the element values of the node degree matrix and the superside degree matrix are respectively represented by d (v i) and d(ej ) And (5) determining. W represents the set of weights for the hyperedge. R represents the association matrix of the node and the superside, if V i ∈e i Then r (v) i ,e j ) =1, otherwise r (v i ,e j )＝0。
Step 3) defining and initially regularizing a depth non-negative matrix to decompose a base matrix and a representation matrix of each layer of the hypergraph:
step 4) setting iteration times t;
setting the initial value of the iteration number T to be 1, setting the maximum value to be T, and setting the T to be 100 in the embodiment;
step 5) obtaining an update formula of a base matrix and a representation matrix in the hypergraph regularization depth non-negative matrix factorization, wherein the implementation steps are as follows:
5 (a) defining an objective function formula of hypergraph regularization depth non-negative matrix factorization, wherein the expression is as follows:
s.t H m ≥0,Z 1 ,...,Z m ≥0
wherein OHDNMF Regularizing a depth matrix-decomposed target equation for a hypergraph, wherein X represents a data matrix and Z 1 ,…,Z m Base matrices of 1 st layer to m th layer respectively, H m Is the representation matrix of the m-th layer, m is the layer number of the depth non-negative matrix factorization, L h The matrix is a hypergraph Laplace matrix, T represents matrix transposition, and beta is a hyperparameter;
5 (b) solving Lagrangian function according to objective function of non-negative matrix factorization of hypergraph regularized depth
The expression is as follows:
wherein ,
is a lagrange multiplier.
5 (c) utilizing Lagrangian function
Respectively solving bias derivatives for the base matrix and the expression matrix, and acquiring update formulas of the base matrix and the expression matrix by using Karush-Kuhn-Tucker conditions, wherein the expressions are respectively as follows:
the expression of the base matrix update formula is:
wherein 
Representing Moore-Penrose pseudo-inverts.
The expression representing the matrix update formula is:
wherein [Q]- All positive elements in the representation matrix are replaced with 0, [ Q ]] + All negative elements in the representation matrix are replaced with 0's, i.e.:
step 6) updating the base matrix and the representation matrix of the hypergraph regularization depth non-negative matrix factorization under the t-th iteration:
(6a) Updating the base matrix by using a base matrix updating formula to obtain an updated base matrix;
(6b) Updating the representation matrix by using a representation matrix updating formula to obtain an updated representation matrix;
(6b) Updating the representation matrix by using a representation matrix updating formula to obtain an updated representation matrix;
the invention adopts a depth non-negative matrix factorization algorithm to decompose the original high-dimensional image data into a plurality of base matrixes and a representation matrix, can effectively reduce the dimension of the high-dimensional image data, and is beneficial to image clustering. The hypergraph matrix constructed in the invention aims to preserve the high-order geometric structure information among data. Therefore, the hypergraph regular non-negative matrix factorization model combining the two can learn the hidden layer characteristics inside complex data and can also reserve the high-order geometric structure information among the data, so that the potential structure information of the data is fully mined, and the accuracy of image clustering is improved.
Step 7) judging whether the iteration times T reach the maximum value T, if so, executing the step (8), otherwise, adding 1 to the iteration times T, and executing the step (6);
step 8) defining and calculating a low-dimensional data representation matrix;
H m the matrix is a low-dimensional data representation matrix;
the defined low-dimensional data represents a matrix calculation formula, the expression of which is:
step 9) image clustering and output:
and clustering the low-dimensional data representation matrix by using a k-means clustering algorithm to obtain a clustered image and outputting the clustered image. The method is mainly focused on the application in the field of image clustering, the depth matrix decomposition can obtain more abundant semantics than standard non-negative matrix decomposition, and the hypergraph regular term can describe the high-order geometric structure among data. The invention provides a hyperspectral regular depth non-negative matrix factorization algorithm which obtains subspaces of rich-level semantics, reserves high-order geometric structures among data and obtains low-dimensional representation with rich semantics. And the standard deep non-negative matrix factorization and the graph regularization deep non-negative matrix factorization only obtain a hierarchical semantic subspace and a hierarchical semantic subspace which reserves a low-order geometric structure, and the expression capability of the obtained semantic subspace is limited to a great extent. In the prior art, depth matrix decomposition is commonly used for data clustering, and a plurality of base matrixes and a representation matrix are obtained by carrying out multi-layer decomposition on an original data feature matrix, and the representation matrix is a low-dimensional semantic representation of the original feature matrix, so that a final clustering result can be obtained by carrying out K-means clustering on the representation matrix.
Claims (5)
1. An image clustering method based on hypergraph regular depth non-negative matrix factorization mainly comprises the following steps:
(1) Inputting image data to be clustered; graying the images in the image data, and then taking the position and gray value of each pixel point to obtain an image data matrix;
(2) Calculating a node degree matrix of the image data matrix;
(3) Calculating a supermargin matrix of the image data matrix;
(4) Calculating a hypergraph matrix of the image data matrix;
(5) Defining and initiating a hypergraph regularization depth matrix to decompose a base matrix and a representation matrix of each layer: the base matrix W of each layer i I=1, … h and the representation matrix V of each layer i I=1 and … h, respectively performing initialization definition, wherein h is the maximum decomposition layer number;
(6) Setting the maximum iteration times t;
(7) The method for obtaining the updated formula of decomposing the base matrix and the representation matrix of each layer of the hypergraph regularization depth matrix comprises the following implementation steps:
(7.1) defining a target equation of the hypergraph regularization depth matrix decomposition, wherein the expression is as follows:
s.t H m ≥0,Z 1 ,...,Z m ≥0
wherein OHDNMF Regularizing a depth matrix-decomposed target equation for a hypergraph, wherein X represents a data matrix and Z 1 ,…,Z m Base matrices of 1 st layer to m th layer respectively, H m Is the representation matrix of the m-th layer, m is the layer number of the depth non-negative matrix factorization, L h Is a hypergraph Laplace matrix, and T represents matrix transposition;
(7.2) solving Lagrangian function according to objective function of hypergraph regularized depth matrix decomposition
The expression is as follows:
wherein ,
phi is the Lagrangian multiplier; zj is the j-th layer base matrix;
(7.3) Using Lagrangian function
Respectively solving bias derivatives for the base matrix and the representation matrix, and acquiring update formulas of the base matrix and the representation matrix by using Karush-Kuhn-Tucker conditions; updating the base matrix and the representation matrix through an updating formula of the base matrix and the representation matrix;
(8) Defining and calculating a low-dimensional data representation matrix:
repeating the steps (7.2) and (7.3) until the set maximum iteration times t are reached, and obtaining a final low-dimensional data representation matrix which is H m ；
(9) Clustering and outputting images:
the final low-dimensional data representation matrix is H by using a k-means clustering algorithm m Clustering is carried out, and a clustered image is obtained and output.
2. The image clustering method based on hypergraph regular depth non-negative matrix factorization of claim 1, wherein the node degree matrix in the step (2) has a calculation formula as follows:
where V represents a set of nodes in hypergraph g= (V, E, W), E represents a set of hyperedges, W represents a set of weights for the hyperedges, if V i ∈e i Then r (v) i ,e j ) =1, otherwise r (v i ,e j )＝0。
3. The image clustering method based on hypergraph regular depth non-negative matrix factorization of claim 1, wherein the hyperedge matrix in the step (3) has a calculation formula as follows:
4. the image clustering method based on hypergraph regular depth non-negative matrix factorization of claim 1, wherein the hypergraph matrix in the step (4) has a calculation formula as follows:
wherein DV ,D E Are diagonal matrices, and the element values are respectively represented by d (v i) and d(ej ) And (5) determining.
5. The image clustering method based on hypergraph regular depth non-negative matrix factorization of claim 1, wherein in step (7), the expressions of the base matrix and the expression matrix are respectively:
the expression of the base matrix update formula is:
the expression representing the matrix update formula is:
wherein [Q]- All positive elements in the representation matrix are replaced with 0, [ Q ]] + All negative elements in the representation matrix are replaced with 0's, i.e.:
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202211588025.9A CN116188829B (en) | 2022-12-12 | 2022-12-12 | Image clustering method based on hypergraph regular depth non-negative matrix factorization |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202211588025.9A CN116188829B (en) | 2022-12-12 | 2022-12-12 | Image clustering method based on hypergraph regular depth non-negative matrix factorization |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN116188829A true CN116188829A (en) | 2023-05-30 |
| CN116188829B CN116188829B (en) | 2024-01-05 |
Family
ID=86443178
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202211588025.9A Active CN116188829B (en) | 2022-12-12 | 2022-12-12 | Image clustering method based on hypergraph regular depth non-negative matrix factorization |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN116188829B (en) |
Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20200057809A1 (en) * | 2018-08-14 | 2020-02-20 | Hrl Laboratories, Llc | Hypergraph-based method for segmenting and clustering customer observables for vehicles |
| CN115331063A (en) * | 2022-09-02 | 2022-11-11 | 安徽大学 | Hyperspectral image classification method and system based on dynamic hypergraph convolution network |
-
2022
- 2022-12-12 CN CN202211588025.9A patent/CN116188829B/en active Active
Patent Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20200057809A1 (en) * | 2018-08-14 | 2020-02-20 | Hrl Laboratories, Llc | Hypergraph-based method for segmenting and clustering customer observables for vehicles |
| CN115331063A (en) * | 2022-09-02 | 2022-11-11 | 安徽大学 | Hyperspectral image classification method and system based on dynamic hypergraph convolution network |
Non-Patent Citations (2)
| Title |
|---|
| WENHUI WU: "Nonnegative matrix factorization with mixed hypergraph regularization for community detection", 《INFORMATION SCIENCES》, pages 263 - 281 * |
| 张鼎: "基于拓展约束投影的加权半监督聚类集成算法", 《南京大学学报》, pages 570 - 583 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN116188829B (en) | 2024-01-05 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| Liu et al. | CNN and RNN based payload classification methods for attack detection | |
| CN108010514B (en) | Voice classification method based on deep neural network | |
| US11631007B2 (en) | Method and device for text-enhanced knowledge graph joint representation learning | |
| CN107832458B (en) | Character-level text classification method based on nested deep network | |
| CN107273897A (en) | A kind of character recognition method based on deep learning | |
| Cherny et al. | The recognition of handwritten digits using neural network technology | |
| CN106959946B (en) | Text semantic feature generation optimization method based on deep learning | |
| CN112800344B (en) | Deep neural network-based movie recommendation method | |
| CN108536844B (en) | Text-enhanced network representation learning method | |
| Rao et al. | Exploring deep learning techniques for Kannada handwritten character recognition: a boon for digitization | |
| CN112860904A (en) | External knowledge-integrated biomedical relation extraction method | |
| Zhiyan et al. | Speech emotion recognition based on deep learning and kernel nonlinear PSVM | |
| Paul et al. | A modern approach for sign language interpretation using convolutional neural network | |
| CN115062727A (en) | Graph node classification method and system based on multi-order hypergraph convolutional network | |
| CN109447147B (en) | Image clustering method based on depth matrix decomposition of double-image sparsity | |
| Singh et al. | Exploratory data analysis and machine learning on titanic disaster dataset | |
| CN109033304B (en) | Multi-modal retrieval method based on online deep topic model | |
| WO2022052468A1 (en) | Methods and systems for product quantization-based compression of matrix | |
| CN112541541B (en) | Lightweight multi-modal emotion analysis method based on multi-element layering depth fusion | |
| CN108460412B (en) | Image classification method based on subspace joint sparse low-rank structure learning | |
| CN114186062A (en) | Text classification method based on graph neural network topic model | |
| CN116188829B (en) | Image clustering method based on hypergraph regular depth non-negative matrix factorization | |
| Jiang et al. | A massive multi-modal perception data classification method using deep learning based on internet of things | |
| Luo | Analysis of artistic modeling of opera stage clothing based on big data clustering algorithm | |
| CN115631361A (en) | Image clustering method fusing low-rank kernel learning and self-adaptive hypergraph |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |