CN113591879A - Deep multi-view clustering method, network, device and storage medium based on self-supervision learning - Google Patents

Deep multi-view clustering method, network, device and storage medium based on self-supervision learning Download PDF

Info

Publication number
CN113591879A
CN113591879A CN202110831409.8A CN202110831409A CN113591879A CN 113591879 A CN113591879 A CN 113591879A CN 202110831409 A CN202110831409 A CN 202110831409A CN 113591879 A CN113591879 A CN 113591879A
Authority
CN
China
Prior art keywords
view
matrix
constraint
shared
views
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Granted
Application number
CN202110831409.8A
Other languages
Chinese (zh)
Other versions
CN113591879B (en
Inventor
宗林林
苗发强
徐博
梁文新
张宪超
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Dalian University of Technology
Original Assignee
Dalian University of Technology
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Dalian University of Technology filed Critical Dalian University of Technology
Priority to CN202110831409.8A priority Critical patent/CN113591879B/en
Priority claimed from CN202110831409.8A external-priority patent/CN113591879B/en
Publication of CN113591879A publication Critical patent/CN113591879A/en
Application granted granted Critical
Publication of CN113591879B publication Critical patent/CN113591879B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
    • G06F18/213—Feature extraction, e.g. by transforming the feature space; Summarisation; Mappings, e.g. subspace methods
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/22—Matching criteria, e.g. proximity measures
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/23—Clustering techniques
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Data Mining & Analysis (AREA)
  • Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Artificial Intelligence (AREA)
  • General Physics & Mathematics (AREA)
  • Evolutionary Computation (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Bioinformatics & Cheminformatics (AREA)
  • Evolutionary Biology (AREA)
  • Bioinformatics & Computational Biology (AREA)
  • Health & Medical Sciences (AREA)
  • Biomedical Technology (AREA)
  • Biophysics (AREA)
  • Computational Linguistics (AREA)
  • General Health & Medical Sciences (AREA)
  • Molecular Biology (AREA)
  • Computing Systems (AREA)
  • Mathematical Physics (AREA)
  • Software Systems (AREA)
  • Information Retrieval, Db Structures And Fs Structures Therefor (AREA)

Abstract

The invention discloses a depth multi-view clustering method, a network, a device and a storage medium based on self-supervision learning, belongs to the field of depth multi-view clustering, and is applied to multiple application scenes such as abnormal point detection, value combination of multiple products, group division of target users and the like.

Description

Deep multi-view clustering method, network, device and storage medium based on self-supervision learning
Technical Field
The invention belongs to the field of image processing, relates to a deep multi-view clustering method, and particularly relates to a deep multi-view clustering method based on self-supervision learning.
Background
With the rapid development of big data and the internet of things, data collection from different sources and all-around and multi-angle is more convenient, multi-view data is more and more, and deep multi-view clustering also becomes a hotspot of current research. Deep multi-view clustering refers to the task of clustering using representation data collected from multiple ways for the same object. The multi-view clustering utilizes the information collected from a wider layer to mine the complementarity and consistency relationship between views, explore the internal relation between data more deeply and mine the potential value. For example, in the aspect of advertisement push, information can be mined from multiple angles such as attributes, behaviors and attention relations of users, so that a more reasonable user partition is obtained, and advertisements are accurately put. The method also has important significance in the fields of financial anti-fraud, medical data abnormity reminding and the like. Most existing multi-view clustering algorithms explore and represent information recognition clustering in an unsupervised mode, but the development of multi-view clustering is restricted to a certain extent by an unsupervised learning mode. The constraint clustering method breaks through the limitation that the traditional clustering relies on an unsupervised model, but the manual marking constraint increases additional labor cost. The recently proposed self-supervision clustering does not need manual labeling constraint and further reduces the consumption of manual resources.
Disclosure of Invention
In order to solve the problem of consumption of artificial resources caused by artificial marking in deep multi-view clustering, the invention introduces an automatic supervision learning mechanism to enable multi-view data to be suitable for an automatic supervision learning algorithm, thereby improving the clustering processing capability of the multi-view data.
In order to achieve the purpose, the invention provides a deep multi-view clustering method, a network, a device and a storage medium based on self-supervision learning, which guides and optimizes a multi-view clustering task by extracting paired constraint information and introducing a self-supervision model.
A depth multi-view clustering method based on self-supervision learning comprises the following steps:
compressing original multi-view data to a low-dimensional potential space to obtain potential feature representation of a view;
performing feature fusion on potential features of each view to obtain potential feature representation shared by multiple views;
adding orthogonal constraint to the potential feature representation shared by the views to obtain the orthogonal potential feature representation shared by the views, and calculating and constructing a similarity matrix shared by the views through a Gaussian kernel function;
acquiring potential feature representation of each view, constructing a corresponding similarity matrix, extracting must-be-trusted links and must-not-be-trusted links by setting a threshold value to form a pair-wise constraint matrix of each view, and selecting the pair-wise constraint matrix with extremely high reliability to form the view-shared pair-wise constraint matrix by solving an intersection;
carrying out diffusion propagation on the paired constraint matrixes shared by the views on the initial matrix through a propagation network to obtain the paired constraint matrixes shared by the views which are completely propagated, and adjusting the similarity matrix shared by the views through the paired constraint matrixes to ensure that the similarity matrix shared by the views is fused with constraint information to represent the similarity between the instances and simultaneously meet the nonnegativity and symmetry to obtain the adjusted similarity matrix shared by the views;
and inputting the similarity matrix shared by the adjusted views as a similarity matrix of the spectral clustering model for clustering to obtain a cluster division result of the multi-view data.
Further, the method for compressing the original multi-view data into the low-dimensional potential space is as follows:
for multi-view data X ═ X1,X2,…,Xm}，
Using an encoder Ev(Xv)＝ZvExtracting view latent features Z of multiple viewsv，
Using a decoder
Figure BDA0003175581030000021
Decoding view latent features ZvReconstructing view raw data Xv。
The loss function of the autoencoder model is:
Figure BDA0003175581030000031
where v ∈ {1,2, …, m }, denotes the vth view, and m denotes the number of views.
Further, in feature Fusion, Fusion (. circle.) denotes a Fusion network, Z*Potential features for multi-view sharing:
Z*＝Fusion(Z1,Z2,…,Zm)
defining fusion losses
Figure BDA0003175581030000032
Optimizing parameters of the converged network:
Figure BDA0003175581030000033
the orthogonal underlying features of view sharing are represented as:
Figure BDA0003175581030000034
the similarity matrix shared by the views is:
Figure BDA0003175581030000035
Figure BDA0003175581030000036
for orthogonal latent feature representation of view sharing, d (·,) represents the distance between the two data, σ is the bandwidth parameter of the gaussian kernel.
Furthermore, a threshold value is set to extract the must-credible link and the must-incredible link, paired constraint matrixes of the views are formed, and paired constraint matrixes with extremely high credibility are selected in an intersection solving mode to form the paired constraint matrixes shared by the views, wherein the method comprises the following steps:
calculating potential feature representation Z of each view respectivelyvSimilarity matrix Wv；
Setting a confidence threshold δmAnd must not be trusted threshold δcA set of linked-by pairwise constraints of
Figure BDA0003175581030000037
Figure BDA0003175581030000038
And a set of pairwise constraints that are not necessarily linked as
Figure BDA0003175581030000039
Wherein
Figure BDA00031755810300000310
Is data
Figure BDA00031755810300000311
A category label of (1);
definition of
Figure BDA0003175581030000041
Expressed as the degree of similarity of the ith data and the jth data in the vth view,
Figure BDA0003175581030000042
Figure BDA0003175581030000043
stipulate if
Figure BDA0003175581030000044
Then define
Figure BDA0003175581030000045
If it is not
Figure BDA0003175581030000046
Then define
Figure BDA0003175581030000047
Will be provided with
Figure BDA0003175581030000048
The corresponding element within is defined as 1,
Figure BDA0003175581030000049
setting the corresponding element in the view to be-1 and setting the other uncertain elements to be 0 to obtain a pair-wise constraint matrix L corresponding to each viewv；
Figure BDA00031755810300000410
The bound credibility and bound incredibility threshold values are indirectly obtained by a constraint selectivity delta, n is defined as the number of data samples, and the specification is provided
Figure BDA00031755810300000411
Similarity matrix W from view potential featuresvBefore n is selected1The larger value as the must-be-trusted instance pair, then the nth value1The larger must-be-trusted instance corresponds to a value of δm(ii) a Provision for
Figure BDA00031755810300000412
Similarity matrix W from view potential featuresvBefore n is selected2The smaller value as the must not be trusted instance pair, n2The smaller must-be-untrusted example corresponds to a value of δc。
Further, the matrix L is constrained in pairs from the view by intersectionvMiddle screening out paired constraint forming view shared constraint matrix L with extremely high credibility*Expressed as:
Figure BDA00031755810300000413
further, the propagation network is composed of a plurality of node layers similar to neurons, the output of the upper layer is the input of the next layer, F (t-1) is designated as the output of the t-1 layer, and the single-layer propagation network works in the following mode:
Figure BDA00031755810300000414
where U is a parameter of the constraint handling network, V is a parameter of the constraint aggregation network, b is a bias,
Figure BDA00031755810300000415
is an activation function;
the input of the propagation network is a pair-wise constraint matrix L*And an initial matrix F0The output is a constraint matrix F with full propagation*；
The loss function of the training propagation network is
Figure BDA0003175581030000051
Wherein gamma is>0 is a hyper-parameter controlling network training; tr (-) is the trace of the matrix,
Figure BDA0003175581030000052
is a normalized laplacian matrix; d is a diagonal matrix with diagonal elements having values of W*Adding corresponding row elements; and I is an identity matrix.
Further, a fully propagated constraint matrix F*Each item in (1)
Figure BDA0003175581030000053
Representing confidence values of pairwise constraints, and
Figure BDA0003175581030000054
through a fully propagated constraint matrix F*Adjusting similarity matrix W of view potential features*Expressed as:
Figure BDA0003175581030000055
wherein
Figure BDA0003175581030000056
Indicating how similar the ith instance is to the jth instance.
A deep multi-view clustering network based on self-supervised learning, comprising:
and the automatic encoder compresses the original multi-view data to a low-dimensional potential space and acquires the potential characteristic representation of the view.
The view fusion module comprises a fusion network, and is used for performing feature fusion on potential features of all views to acquire potential features shared by the views; and the orthogonal layer is used for adding orthogonal constraint to the view sharing potential feature to obtain the view sharing orthogonal potential feature representation, and responding to the orthogonal potential feature and constructing a view sharing similarity matrix through Gaussian kernel function calculation.
The self-supervision module comprises a pair-constrained module, a pair-constrained matrix and a pair-constrained matrix, wherein the pair-constrained matrix comprises a pair-constrained matrix, the potential feature representation of each view is obtained, a corresponding similarity matrix is constructed, a necessary credible link and a necessary incredible link are extracted by setting a threshold value to form the pair-constrained matrix of each view, and the pair-constrained matrix with extremely high credibility is selected by solving an intersection to form the pair-constrained matrix shared by the views;
and the propagation network is used for performing diffusion propagation on the view-shared paired constraint matrix on the initial matrix to obtain the completely propagated view-shared paired constraint matrix, adjusting the view-shared similarity matrix through the propagation network to ensure that the view-shared similarity matrix integrates constraint information to represent the similarity between the instances and simultaneously meet the nonnegativity and symmetry, and obtaining the adjusted view-shared similarity matrix.
And the spectral clustering model is used for clustering by taking the adjusted similarity matrix shared by the views as the similarity matrix input of the spectral clustering model, and obtaining the cluster division result of the multi-view data.
An apparatus for deep multi-view clustering based on self-supervised learning, comprising a processor and a memory, the processor executing code in the memory to implement any of the methods.
A computer storage medium storing a computer program for execution by hardware to implement any of the methods.
The invention has the beneficial effect of providing an automatic supervision depth multi-view clustering method. The method has the advantages that constraint information is efficiently acquired from multi-view data, labor cost is reduced, potential feature representation is continuously optimized through a self-supervision learning mechanism, and clustering capability is further improved. The method can be applied to various application scenes such as abnormal point detection, value combination of various products, group division of target users and the like.
Drawings
FIG. 1 is a general flow diagram of the process of the present invention.
Fig. 2 is a diagram of a network architecture model of the present invention.
Fig. 3 is a diagram of a propagation network model of the present invention.
FIG. 4 is a graph of the effect of changing the constraint selection rate on the accuracy of the constraint pair selected, where (a) is represented on the ALOI dataset and (b) is represented on the LUse-21 dataset.
FIG. 5 is a graph of the impact of varying constraint selectivity on clustering performance, where (a) is represented on the ALOI dataset and (b) is represented on the LUse-21 dataset.
FIG. 6 is a graph of the impact of varying the hyperparameters β and γ on clustering performance, where (a) is shown on the ALOI dataset and (b) is shown on the LUse-21 dataset.
Detailed Description
The following detailed description of the invention refers to the accompanying drawings that illustrate specific embodiments of the invention.
As shown in fig. 1, the present invention provides a deep multi-view clustering method based on an auto-supervised learning mechanism. By extracting the paired constraint information of the multi-view data instances, the self-monitoring model is combined with the constraint clustering, and the multi-view clustering model is optimized, so that the clustering accuracy of the algorithm is improved.
In one scheme, the self-supervised learning based deep multi-view clustering method SDMvSC comprises the following steps of deep potential feature extraction, similarity matrix extraction, pairwise constraint propagation, similarity matrix adjustment of view sharing and clustering result acquisition, and the steps are summarized as follows:
1. deep latent feature extraction: obtaining potential feature representation Z of each view using an auto-encoder network to compress raw view data into a low-dimensional potential spacevAnd fusing them to obtain a potential feature representation Z shared by the views*。
2. Extracting a similarity matrix: obtaining potential feature representation Z of view sharing*Adding orthogonal constrained view-shared orthogonal latent feature representations
Figure BDA0003175581030000071
Similarity matrix W for view sharing construction through Gaussian kernel function calculation*。
3. And (3) pair-wise constraint extraction: obtaining potential feature representations Z for viewsvAnd (3) constructing a similarity matrix W of potential features of each view through Gaussian kernel function calculationvThen setting a threshold value to select a similarity matrix W of potential features of each viewvAnd the must-link and must-not-link form a pair-wise constraint matrix L of each viewvSelecting the paired constraint matrix L shared by the paired constraint component views with extremely high credibility by solving the intersection*。
4. Propagation of pairwise constraints: pair-wise constraint matrix L for sharing views through constraint propagation network*Paired constraint matrix F for obtaining view sharing of full propagation by diffusion propagation on initial matrix as much as possible*。
5. Adjusting a similarity matrix shared by the views: pair-wise constraint matrix F shared by fully propagated views*Adjusting a similarity matrix W for view sharing*Obtaining an adjusted view-shared similarity matrix
Figure BDA0003175581030000072
6. Obtaining a clustering result: similarity matrix for sharing adjusted views
Figure BDA0003175581030000087
And inputting the similarity matrix serving as a spectral clustering model for clustering to obtain a cluster division result of the multi-view data.
In a specific embodiment, the method is specifically described as follows:
1. definition of
1.1. Paired constraints
Paired constraints are divided into two categories, namely, must-join constraints and must-don constraints. A pair-wise constraint representing a set of must-link links as
Figure BDA0003175581030000081
And a set of pairwise constraints that are not necessarily linked as
Figure BDA0003175581030000082
Figure BDA0003175581030000083
Wherein
Figure BDA0003175581030000084
Is data
Figure BDA0003175581030000085
The category label of (1).
1.2. Multi-view data
The multiview data is represented as X ═ X1,X2,…,XmWhere m denotes the number of views of the multi-view data. Usually using XvData representing the v-th view and v ∈ {1,2, …, m }.
2. Clustering process
The method of the invention aims to accurately obtain clustering results by using data of a plurality of views, and firstly uses a self-encoder to obtain clustering results from a plurality of viewsExtracting potential feature representation Z of each viewvThereafter, potential feature representation Z of multi-view sharing is extracted through converged network*(ii) a Constraint matrix L shared by views formed by extracting paired constraint information by utilizing common features among multi-view data*After propagating pairwise constrained information through constraint propagation network with history memory, a pairwise constraint matrix F shared by propagated views is utilized*Similarity matrix W for optimized view sharing*And finally, obtaining a clustering result through spectral clustering.
2.1. Deep latent feature extraction
Since the deep neural network can more efficiently mine the latent features of the data, the present invention learns the low-dimensional latent feature representation of each view using a deep self-coder. First of all, the invention uses an encoder Ev(Xv)＝ZvExtracting potential feature representation Z of each viewvThen using a decoder
Figure BDA0003175581030000086
Decoding ZvTo reconstruct the data Xv. For the depth autoencoder model, the loss function is:
Figure BDA0003175581030000091
in order to further extract potential features shared by multiple views, in a depth feature fusion layer, the invention mines consistency information and complementarity information between multi-view data by fusing features of all views. Fusion (. smallcircle.) denotes a converged network, Z*Is a potential feature representation shared for multiple views. Thus, the present invention has Z*＝Fusion(Z1,Z2,…,Zm). The invention defines the fusion loss
Figure BDA0003175581030000092
To optimize parameters of the converged network.
Figure BDA0003175581030000093
Extracting potential feature representations Z for multi-view sharing*Then, the invention passes through the orthogonal layer as Z*An orthogonal constraint is added. The orthogonal layers are realized by Cholesky decomposition, i.e.
Figure BDA0003175581030000094
2.2. Similarity matrix extraction
Acquisition of orthogonal latent feature representations for multi-view sharing
Figure BDA0003175581030000095
Then, the similarity matrix W shared by multiple views is calculated and constructed through the Gaussian kernel function*。
Figure BDA0003175581030000096
Where d (·,) denotes the distance between the two data, σ is the bandwidth parameter of the gaussian kernel, which controls the radial range of action. The method sets σ as the median of the euclidean distances between the data points.
2.3. Pairwise constrained extraction
The paired constraints describe weak supervision information of whether two instances belong to the same class, so that the extraction of the paired constraints can improve the accuracy of data feature mining. By using the method for constructing the similarity matrix in the formula (4), the invention respectively calculates the potential feature representation Z of each viewvI.e. the similarity matrix W of the potential features of the viewv. Then, the invention sets the necessary confidence threshold deltamAnd must not be trusted threshold δcTo look for WvThe pair-wise constraint information of (1). Definition of
Figure BDA0003175581030000097
Expressed as the degree of similarity of the ith data and the jth data in the vth view,
Figure BDA0003175581030000101
stipulate if
Figure BDA0003175581030000102
Then define
Figure BDA0003175581030000103
If it is not
Figure BDA0003175581030000104
Then define
Figure BDA0003175581030000105
Then will be
Figure BDA0003175581030000106
The corresponding element within is defined as 1,
Figure BDA0003175581030000107
setting the corresponding element in the view to be-1 and setting the other uncertain elements to be 0 to obtain a pair-wise constraint matrix L corresponding to each viewv。
Figure BDA0003175581030000108
In the experiment, the bound credibility threshold and the bound incredibility threshold are indirectly obtained by restricting the selectivity delta. Defining n as the number of data samples, specifying
Figure BDA0003175581030000109
Similarity matrix W from view potential featuresvBefore n is selected1The larger value as the must-be-trusted instance pair, then the nth value1The larger must-be-trusted instance corresponds to a value of δm(ii) a Provision for
Figure BDA00031755810300001010
Similarity matrix W from view potential featuresvBefore n is selected2The smaller value as the must not be trusted instance pair, n2The smaller must-be-untrusted example corresponds to a value of δc。
Because the extracted pair-wise constraint is not completely correct and because the multi-view data has consistency and complementarity information, the invention solves the problem that the pair-wise constraint matrix L is extracted from the views in an intersection mannervMiddle screening out paired constraint forming view shared constraint matrix L with extremely high credibility*。
Figure BDA00031755810300001011
2.4. Pairwise constrained propagation
In general, the greater the number of active pairwise constraints, the better the constraint performance. However, the number of reliable paired constraints obtained in equation (6) tends to be small. In order to spread the extracted pair-wise constraint information as much as possible throughout the constraint matrix, the present invention constructs a propagation network to propagate the pair-wise constraints.
The propagation network is composed of several layers of nodes like neurons. It is specified that the output of the upper layer is the input of the next layer. In order to make propagated constraint matrix and extracted paired constraint matrix L with higher reliability*Keeping consistency, and continuously using a constraint matrix L with higher credibility in the propagation process*The input propagates the network. F (t-1) is designated as the output of layer t-1, U and V are parameters of the propagation network, b is the bias,
Figure BDA0003175581030000111
for the activation function, the single-layer propagation network works in the following way:
Figure BDA0003175581030000112
where U is a parameter of the constraint handling network, V is a parameter of the constraint aggregation network, b is a bias,
Figure BDA0003175581030000113
is an activation function.
Input of propagation network is a constraint matrix L shared by views*And an initial matrix F0I-S. Constraint matrix F whose output is fully propagated*. The loss function of the training propagation network is
Figure BDA0003175581030000114
Wherein gamma is>0 is a hyper-parameter controlling network training; tr (-) is the trace of the matrix,
Figure BDA0003175581030000115
is a normalized laplacian matrix; d is a diagonal matrix with diagonal elements having values of W*Adding corresponding row elements; and I is an identity matrix.
2.5. View-shared similarity matrix adjustment
Constraint matrix F of the invention for complete propagation*Each item in (1)
Figure BDA0003175581030000116
Representing confidence values of pairwise constraints, computing
Figure BDA0003175581030000117
Ensure F*With symmetry, using a fully propagated constraint matrix F*Adjusting a similarity matrix W for view sharing*Obtaining an optimized multi-view shared similarity matrix
Figure BDA0003175581030000118
Figure BDA0003175581030000119
Wherein
Figure BDA00031755810300001110
Indicating how similar the ith instance is to the jth instance,
Figure BDA00031755810300001111
constraint information is fused so that similarities between instances can be represented more accurately.
2.6. Clustering result acquisition
Adjusted view-shared similarity matrix
Figure BDA00031755810300001112
Satisfy the nonnegativity and symmetry, will
Figure BDA00031755810300001113
And inputting the similarity matrix serving as a spectral clustering model for clustering to obtain a cluster division result of the multi-view data.
3. Experiment of
3.1. Data set
The invention uses two multi-view data sets for experiments, and the information of the data sets is as follows:
ALOI image dataset: the dataset contains three views, 77-dimensional RBG color histogram, 13-dimensional HSV/HSB color histogram and 64-dimensional color similarity.
LUse-21 dataset: the dataset has three views, including 254-dimensional LBP, 512-dimensional GIST, and 256-dimensional CENTRIST.
Table 1 details of each data set
Figure BDA0003175581030000121
3.2. Evaluation index
The algorithm of the invention is evaluated by using three traditional clustering indexes of accuracy, mutual information and purity:
the accuracy is as follows:
Figure BDA0003175581030000122
where l is the actual label, c is the prediction is the clustering label, and pi is the set of all permutations of {1,2, …, k }. The optimal arrangement pi is calculated by the Cohn-Monkles algorithm.
Mutual information:
Figure BDA0003175581030000123
wherein I (l; c) represents mutual information between l and c, and H (-) represents entropy thereof.
Purity:
Figure BDA0003175581030000124
wherein Ω ═ { w ═ w1,w2,…,wkDenotes cluster division, C ═ C1,c2,…,cjDenotes true category classification.
The value ranges of the three clustering indexes are all between [0,1], and the closer to 1, the better the clustering effect is.
3.3. Comparison algorithm
In this experiment, 4 algorithms were mainly used for comparison with the present invention algorithm.
ECMSC: the method is called exclusive-structured Multi-view Subspace Clustering, and is a traditional Multi-view Clustering algorithm based on a Subspace model. It attempts to deal with complementary information between different view representations by introducing a new location-aware exclusivity term, and then using the correspondence term to further bring these complementary representations to a common clustering indication.
MvSCN: the method is called a Multi-view Spectral Clustering Network, and is a depth Multi-view Spectral Clustering algorithm. It combines the local invariance defined by the depth metric learning network in each view and the consistency of different views into one objective function to obtain the view-shared features.
S2 DMVSC: it is called Self-Supervised Deep Multi-View subframe Cluster in its entirety. The method integrates spectral clustering and similarity learning into a deep learning framework, fully utilizes clustering results to supervise potential representation learning and common potential subspace learning of each view, and automatically calculates a similarity matrix between data objects according to high-level and cluster-driven representations.
The incomplete version of SDMvSC algorithm: by sequentially deleting some parts of the algorithm, the importance of each part can be seen through comparison with the algorithm of the invention. The DMvSC algorithm does not comprise a self-supervision model part, and a clustering result is obtained through potential features extracted from an encoder. The PDMvSC algorithm uses a propagation calculation formula to optimize the pairwise constraint matrix.
3.4. Results of the experiment
The experimental results are shown in table 2 and evaluated using evaluation criteria such as ACC and NMI.
Table 2 experimental results (%)
Figure BDA0003175581030000131
Figure BDA0003175581030000141
Comparing an ECMSC algorithm: as can be seen from the table, the sdmvcs algorithm is superior to the ECMSC algorithm, mainly because the depth model can better explore the relationships between data and extract potential feature representations.
Comparing the MvSCN algorithm: the data in the table show that the SDMvSC algorithm is superior to the MVSCN algorithm. After the view features are extracted by the MVSCN algorithm, the fusion features are obtained by adopting a splicing mode, and the SDMvSC algorithm adopts a fusion network mode to fuse the features, so that the fusion view features are better mined by adopting a network mode.
Comparing the S2DMVSC algorithm: the S2DMVSC algorithm drives the self-supervision model by using the clustering label, the clustering label used for supervision is changed in the process of each iteration, and the uncertainty of the label has the possibility of misleading the optimization direction of the model. In contrast, the SDMvSC algorithm extracts the pair constraint of a plurality of views, which is consistent, and the correctness of the supervision information is ensured to a great extent.
And fourthly, comparing the incomplete version of the SDMvSC algorithm. Compared with the DMvSC algorithm, the self-supervision module optimizes the fused view potential representation, and the clustering performance is improved; compared with the PDMvSC algorithm, the paired constraint propagation network can more effectively diffuse paired constraint information compared with the traditional propagation formula.
3.5. Analysis of experiments
3.5.1. Effect of constraint Selectivity δ on Pair-constrained accuracy
The choice of constraint selectivity δ affects the number of pairs of constraints that are selected. In this experiment, the constraint selection rate δ was sequentially changed in the range of [0.001-0.015], and the actual number of selected pairwise constraints and the selected pairwise constraint accuracy were observed. Fig. 3 illustrates the case of varying the constraint selection rate δ versus the actual number of selected pairwise constraints and the accuracy of the selected pairwise constraints. It is found that by extracting the operation of intersecting the pair-wise constraint matrixes of all views, the ratio of the actually obtained pair-wise constraints is far lower than the constraint selectivity delta, so the operation helps to screen out incorrect constraints and improve the accuracy of the selected constraint information.
3.5.2. Effect of constraint Selectivity δ on clustering Performance
The constraint selectivity δ determines the quality of the chosen pairwise constraints, and pairwise constraint information in turn affects the clustering performance. In the experiment, the constraint selectivity delta is changed in turn in the range of [0.001-0.015], and the influence of constraint information on the clustering result is observed. Fig. 4 shows the influence of the constraint selectivity δ on the clustering performance, and as the constraint selectivity δ increases, the constraint information increases continuously, but the clustering result tends to show a trend of increasing first and then gradually decreasing. The reason is that the paired constraint as constraint information can improve the performance of clustering, but as the delta is increased, the proportion of error constraint is increased, and the capability of a propagation network is inhibited. Therefore, the reasonable delta is set, and the performance of multi-view clustering can be improved.
3.6. Hyper-parameter settings analysis
In the network training phase, two hyper-parameters beta and gamma are set to adjust the network training rate. In this experiment, the effect of changes in β and γ on algorithm performance was analyzed. Fig. 5 shows the effect of β and γ on the clustering performance. The variation range of beta and gamma is [0.1,0.3,0.5,0.7,0.9], when beta belongs to [0.7,0.9] and gamma belongs to [0.5,0.7,0.9], the SDMvSC obtains stable and good performance.

Claims (10)

1. A deep multi-view clustering method based on self-supervision learning is characterized by comprising the following steps:
compressing original multi-view data to a low-dimensional potential space to obtain potential feature representation of a view;
performing feature fusion on potential features of each view to obtain potential feature representation shared by multiple views;
adding orthogonal constraint to the potential feature representation shared by the views to obtain the orthogonal potential feature representation shared by the views, and calculating and constructing a similarity matrix shared by the views through a Gaussian kernel function;
acquiring potential feature representation of each view, constructing a corresponding similarity matrix, extracting must-be-trusted links and must-not-be-trusted links by setting a threshold value to form a pair-wise constraint matrix of each view, and selecting the pair-wise constraint matrix with extremely high reliability to form the view-shared pair-wise constraint matrix by solving an intersection;
carrying out diffusion propagation on the paired constraint matrixes shared by the views on the initial matrix through a propagation network to obtain the paired constraint matrixes shared by the views which are completely propagated, and adjusting the similarity matrix shared by the views through the paired constraint matrixes to ensure that the similarity matrix shared by the views is fused with constraint information to represent the similarity between the instances and simultaneously meet the nonnegativity and symmetry to obtain the adjusted similarity matrix shared by the views;
and inputting the similarity matrix shared by the adjusted views as a similarity matrix of the spectral clustering model for clustering to obtain a cluster division result of the multi-view data.
2. The depth multi-view clustering method based on the self-supervised learning as recited in claim 1, wherein: the method of compressing the original multi-view data into a low-dimensional potential space is:
for multi-view data X ═ X1，X2，…，Xm}，
Using codesDevice Ev(Xv)＝ZvExtracting view latent features Z of multiple viewsv，
Using a decoder
Figure FDA0003175581020000011
Decoding view latent features ZvReconstructing view raw data Xv。
The loss function of the autoencoder model is:
Figure FDA0003175581020000012
where v ∈ {1,2, …, m }, denotes the vth view, and m denotes the number of views.
3. The self-supervised learning based deep multi-view clustering method according to claim 1 or 2, wherein:
in feature Fusion, Fusion (. circle.) denotes a Fusion network, Z*Potential features for multi-view sharing:
Z*＝Fusion(Z1，Z2，…，Zm)
defining fusion losses
Figure FDA0003175581020000021
Optimizing parameters of the converged network:
Figure FDA0003175581020000022
the orthogonal underlying features of view sharing are represented as:
Figure FDA0003175581020000023
the similarity matrix shared by the views is:
Figure FDA0003175581020000024
Figure FDA0003175581020000025
for orthogonal latent feature representation of view sharing, d (·,) represents the distance between the two data, σ is the bandwidth parameter of the gaussian kernel.
4. The deep multi-view clustering method based on the self-supervised learning as recited in claim 3, wherein paired constraint matrixes of the views are formed by setting a threshold value to extract a must-trusted link and a must-untrusted link, and paired constraint matrixes with extremely high credibility shared by the views are selected by intersection, and the method comprises the following steps:
calculating potential feature representation Z of each view respectivelyvSimilarity matrix Wv；
Setting a confidence threshold δmAnd must not be trusted threshold δcA set of linked-by pairwise constraints of
Figure FDA0003175581020000026
Figure FDA0003175581020000027
And a set of pairwise constraints that are not necessarily linked as
Figure FDA0003175581020000028
Wherein
Figure FDA0003175581020000029
Is data
Figure FDA00031755810200000210
A category label of (1);
definition of
Figure FDA00031755810200000211
Expressed as the degree of similarity of the ith data and the jth data in the vth view,
Figure FDA00031755810200000212
Figure FDA00031755810200000213
stipulate if
Figure FDA00031755810200000214
Then define
Figure FDA00031755810200000215
If it is not
Figure FDA00031755810200000216
Then define
Figure FDA00031755810200000217
Will be provided with
Figure FDA0003175581020000031
The corresponding element within is defined as 1,
Figure FDA0003175581020000032
setting the corresponding element in the view to be-1 and setting the other uncertain elements to be 0 to obtain a pair-wise constraint matrix L corresponding to each viewv；
Figure FDA0003175581020000033
The bound credibility and bound incredibility threshold values are indirectly obtained by a constraint selectivity delta, n is defined as the number of data samples, and the specification is provided
Figure FDA0003175581020000034
From the viewSimilarity matrix W of potential featuresvBefore n is selected1The larger value as the must-be-trusted instance pair, then the nth value1The larger must-be-trusted instance corresponds to a value of δm(ii) a Provision for
Figure FDA0003175581020000035
Similarity matrix W from view potential featuresvBefore n is selected2The smaller value as the must not be trusted instance pair, n2The smaller must-be-untrusted example corresponds to a value of δc。
5. The depth multi-view clustering method based on the self-supervised learning as recited in claim 4, wherein: from view pair-wise constraint matrix L by intersectionvMiddle screening out paired constraint forming view shared constraint matrix L with extremely high credibility*Expressed as:
Figure FDA0003175581020000036
6. the self-supervised learning based deep multi-view clustering method according to claim 4 or 5, wherein:
the propagation network is composed of a plurality of node layers similar to neurons, the output of the upper layer is the input of the next layer, F (t-1) is designated as the output of the t-1 layer, and the single-layer propagation network has the working mode that:
Figure FDA0003175581020000037
where U is a parameter of the constraint handling network, V is a parameter of the constraint aggregation network, b is a bias,
Figure FDA0003175581020000038
is an activation function;
of propagation networksThe input is a pairwise constraint matrix L*And an initial matrix F0The output is a constraint matrix F with full propagation*；
The loss function of the training propagation network is
Figure FDA0003175581020000041
Wherein gamma > 0 is a hyper-parameter controlling network training; tr (-) is the trace of the matrix,
Figure FDA0003175581020000042
is a normalized laplacian matrix; d is a diagonal matrix with diagonal elements having values of W*Adding corresponding row elements; and I is an identity matrix.
7. The self-supervised learning based deep multi-view clustering method of claim 5, wherein: fully propagated constraint matrix F*Each item in (1)
Figure FDA0003175581020000043
Representing confidence values of pairwise constraints, and
Figure FDA0003175581020000044
through a fully propagated constraint matrix F*Adjusting similarity matrix W of view potential features*Expressed as:
Figure FDA0003175581020000045
wherein
Figure FDA0003175581020000046
Indicating how similar the ith instance is to the jth instance.
8. A deep multi-view clustering network based on self-supervised learning, comprising:
and the automatic encoder compresses the original multi-view data to a low-dimensional potential space and acquires the potential characteristic representation of the view.
The view fusion module comprises a fusion network, and is used for performing feature fusion on potential features of all views to acquire potential features shared by the views; and the orthogonal layer is used for adding orthogonal constraint to the view sharing potential feature to obtain the view sharing orthogonal potential feature representation, and responding to the orthogonal potential feature and constructing a view sharing similarity matrix through Gaussian kernel function calculation.
The self-supervision module comprises a pair-constrained module, a pair-constrained matrix and a pair-constrained matrix, wherein the pair-constrained matrix comprises a pair-constrained matrix, the potential feature representation of each view is obtained, a corresponding similarity matrix is constructed, a necessary credible link and a necessary incredible link are extracted by setting a threshold value to form the pair-constrained matrix of each view, and the pair-constrained matrix with extremely high credibility is selected by solving an intersection to form the pair-constrained matrix shared by the views;
and the propagation network is used for performing diffusion propagation on the view-shared paired constraint matrix on the initial matrix to obtain the completely propagated view-shared paired constraint matrix, adjusting the view-shared similarity matrix through the propagation network to ensure that the view-shared similarity matrix integrates constraint information to represent the similarity between the instances and simultaneously meet the nonnegativity and symmetry, and obtaining the adjusted view-shared similarity matrix.
And the spectral clustering model is used for clustering by taking the adjusted similarity matrix shared by the views as the similarity matrix input of the spectral clustering model, and obtaining the cluster division result of the multi-view data.
9. An apparatus for deep multi-view clustering based on self-supervised learning, comprising a processor and a memory, wherein the processor executes code in the memory to implement the method of any one of claims 1 to 7.
10. A computer storage medium, in which a computer program is stored, the computer program being executable by hardware to implement the method of any one of claims 1 to 7.
CN202110831409.8A 2021-07-22 Depth multi-view clustering method, network, device and storage medium based on self-supervision learning Active CN113591879B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202110831409.8A CN113591879B (en) 2021-07-22 Depth multi-view clustering method, network, device and storage medium based on self-supervision learning

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202110831409.8A CN113591879B (en) 2021-07-22 Depth multi-view clustering method, network, device and storage medium based on self-supervision learning

Publications (2)

Publication Number Publication Date
CN113591879A true CN113591879A (en) 2021-11-02
CN113591879B CN113591879B (en) 2024-10-29

Family

ID=

Cited By (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN114201635A (en) * 2021-11-12 2022-03-18 东北大学 Scheme source cable classification method based on multi-view graph data feature learning
CN114742132A (en) * 2022-03-17 2022-07-12 湖南工商大学 Deep multi-view clustering method, system and equipment based on common difference learning
CN115098672A (en) * 2022-05-11 2022-09-23 合肥工业大学 User demand discovery method and system based on multi-view deep clustering
CN116861923A (en) * 2023-04-04 2023-10-10 大连理工大学 Multi-view unsupervised graph contrast learning model construction method, system, computer, storage medium and application
CN117009838A (en) * 2023-09-27 2023-11-07 江西师范大学 Multi-scale fusion contrast learning multi-view clustering method and system
CN117972497A (en) * 2024-04-01 2024-05-03 中国传媒大学 False information detection method and system based on multi-view feature decomposition
CN118312818A (en) * 2024-06-07 2024-07-09 中国人民解放军国防科技大学 Semi-supervised clustering method based on bipartite graph acceleration and fused into paired constraint

Citations (11)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN107292341A (en) * 2017-06-20 2017-10-24 西安电子科技大学 Adaptive multi views clustering method based on paired collaboration regularization and NMF
CN108776812A (en) * 2018-05-31 2018-11-09 西安电子科技大学 Multiple view clustering method based on Non-negative Matrix Factorization and various-consistency
CN109949314A (en) * 2019-02-23 2019-06-28 西安邮电大学 A kind of multiple target Fast Fuzzy Cluster color image segmentation method based on semi-supervised learning and statistics with histogram
CN109993214A (en) * 2019-03-08 2019-07-09 华南理工大学 Multiple view clustering method based on Laplace regularization and order constraint
CN110598740A (en) * 2019-08-08 2019-12-20 中国地质大学（武汉） Spectrum embedding multi-view clustering method based on diversity and consistency learning
US20200074220A1 (en) * 2018-09-04 2020-03-05 Inception Institute of Artificial Intelligence, Ltd. Multi-view image clustering techniques using binary compression
CN111259979A (en) * 2020-02-10 2020-06-09 大连理工大学 Deep semi-supervised image clustering method based on label self-adaptive strategy
CN112164067A (en) * 2020-10-12 2021-01-01 西南科技大学 Medical image segmentation method and device based on multi-mode subspace clustering
CN112270345A (en) * 2020-10-19 2021-01-26 西安工程大学 Clustering algorithm based on self-supervision dictionary learning
CN112990265A (en) * 2021-02-09 2021-06-18 浙江师范大学 Post-fusion multi-view clustering machine learning method and system based on bipartite graph
CN112990264A (en) * 2021-02-08 2021-06-18 浙江师范大学 Multi-view clustering method based on consistent graph learning

Patent Citations (11)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN107292341A (en) * 2017-06-20 2017-10-24 西安电子科技大学 Adaptive multi views clustering method based on paired collaboration regularization and NMF
CN108776812A (en) * 2018-05-31 2018-11-09 西安电子科技大学 Multiple view clustering method based on Non-negative Matrix Factorization and various-consistency
US20200074220A1 (en) * 2018-09-04 2020-03-05 Inception Institute of Artificial Intelligence, Ltd. Multi-view image clustering techniques using binary compression
CN109949314A (en) * 2019-02-23 2019-06-28 西安邮电大学 A kind of multiple target Fast Fuzzy Cluster color image segmentation method based on semi-supervised learning and statistics with histogram
CN109993214A (en) * 2019-03-08 2019-07-09 华南理工大学 Multiple view clustering method based on Laplace regularization and order constraint
CN110598740A (en) * 2019-08-08 2019-12-20 中国地质大学（武汉） Spectrum embedding multi-view clustering method based on diversity and consistency learning
CN111259979A (en) * 2020-02-10 2020-06-09 大连理工大学 Deep semi-supervised image clustering method based on label self-adaptive strategy
CN112164067A (en) * 2020-10-12 2021-01-01 西南科技大学 Medical image segmentation method and device based on multi-mode subspace clustering
CN112270345A (en) * 2020-10-19 2021-01-26 西安工程大学 Clustering algorithm based on self-supervision dictionary learning
CN112990264A (en) * 2021-02-08 2021-06-18 浙江师范大学 Multi-view clustering method based on consistent graph learning
CN112990265A (en) * 2021-02-09 2021-06-18 浙江师范大学 Post-fusion multi-view clustering machine learning method and system based on bipartite graph

Non-Patent Citations (2)

* Cited by examiner, † Cited by third party
Title
刘冶;朱蔚恒;潘炎;印鉴;: "基于低秩和稀疏矩阵分解的多源融合链接预测算法", 计算机研究与发展, no. 02, 15 February 2015 (2015-02-15) *
肖成龙;张重鹏;王珊珊;张睿;王万里;魏宪;: "基于流形正则化与成对约束的深度半监督谱聚类算法", 系统科学与数学, no. 08, 15 August 2020 (2020-08-15) *

Cited By (9)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN114201635A (en) * 2021-11-12 2022-03-18 东北大学 Scheme source cable classification method based on multi-view graph data feature learning
CN114742132A (en) * 2022-03-17 2022-07-12 湖南工商大学 Deep multi-view clustering method, system and equipment based on common difference learning
CN115098672A (en) * 2022-05-11 2022-09-23 合肥工业大学 User demand discovery method and system based on multi-view deep clustering
CN116861923A (en) * 2023-04-04 2023-10-10 大连理工大学 Multi-view unsupervised graph contrast learning model construction method, system, computer, storage medium and application
CN116861923B (en) * 2023-04-04 2024-07-09 大连理工大学 Implicit relation mining method, system, computer and storage medium based on multi-view unsupervised graph contrast learning
CN117009838A (en) * 2023-09-27 2023-11-07 江西师范大学 Multi-scale fusion contrast learning multi-view clustering method and system
CN117009838B (en) * 2023-09-27 2024-01-26 江西师范大学 Multi-scale fusion contrast learning multi-view clustering method and system
CN117972497A (en) * 2024-04-01 2024-05-03 中国传媒大学 False information detection method and system based on multi-view feature decomposition
CN118312818A (en) * 2024-06-07 2024-07-09 中国人民解放军国防科技大学 Semi-supervised clustering method based on bipartite graph acceleration and fused into paired constraint

Similar Documents

Publication Publication Date Title
Liu et al. Localized sparse incomplete multi-view clustering
Huang et al. Learning label specific features for multi-label classification
Yoshida et al. Bayesian learning in sparse graphical factor models via variational mean-field annealing
Wang et al. Hybrid feature selection using component co-occurrence based feature relevance measurement
Zhang et al. Non-negative multi-label feature selection with dynamic graph constraints
US11971892B2 (en) Methods for stratified sampling-based query execution
Qian et al. A survey on multi-label feature selection from perspectives of label fusion
Sang et al. An effective discretization method for disposing high-dimensional data
Qian et al. Disambiguation-based partial label feature selection via feature dependency and label consistency
Huang et al. Self-supervised graph attention networks for deep weighted multi-view clustering
Do et al. Energy-based anomaly detection for mixed data
Bonaccorso Hands-on unsupervised learning with Python: implement machine learning and deep learning models using Scikit-Learn, TensorFlow, and more
Zhang et al. Supervised feature selection algorithm via discriminative ridge regression
Liu et al. Self-supervised transformer-based pre-training method using latent semantic masking auto-encoder for pest and disease classification
Fang et al. Patent2Vec: Multi-view representation learning on patent-graphs for patent classification
Ahmed et al. Pattern Recognition: An Introduction
Li et al. Label correlations variation for robust multi-label feature selection
Zeng et al. Pyramid hybrid pooling quantization for efficient fine-grained image retrieval
Zhao et al. Classification and saliency detection by semi-supervised low-rank representation
Zhu et al. Learning relation-based features for fine-grained image retrieval
Wu et al. Multi-level correlation learning for multi-view unsupervised feature selection
CN113591879A (en) Deep multi-view clustering method, network, device and storage medium based on self-supervision learning
CN113591879B (en) Depth multi-view clustering method, network, device and storage medium based on self-supervision learning
Wang et al. Heterogeneous graph convolutional network for multi-view semi-supervised classification
Zhao et al. Robust graph convolutional clustering with adaptive graph learning

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant