CN113269231A - Local kernel-based optimal neighbor multi-core clustering method and system - Google Patents
Local kernel-based optimal neighbor multi-core clustering method and system Download PDFInfo
- Publication number
- CN113269231A CN113269231A CN202110447615.9A CN202110447615A CN113269231A CN 113269231 A CN113269231 A CN 113269231A CN 202110447615 A CN202110447615 A CN 202110447615A CN 113269231 A CN113269231 A CN 113269231A
- Authority
- CN
- China
- Prior art keywords
- matrix
- kernel
- representing
- core
- clustering
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/23—Clustering techniques
- G06F18/232—Non-hierarchical techniques
- G06F18/2321—Non-hierarchical techniques using statistics or function optimisation, e.g. modelling of probability density functions
- G06F18/23213—Non-hierarchical techniques using statistics or function optimisation, e.g. modelling of probability density functions with fixed number of clusters, e.g. K-means clustering
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/23—Clustering techniques
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F17/00—Digital computing or data processing equipment or methods, specially adapted for specific functions
- G06F17/10—Complex mathematical operations
- G06F17/16—Matrix or vector computation, e.g. matrix-matrix or matrix-vector multiplication, matrix factorization
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
- G06F18/213—Feature extraction, e.g. by transforming the feature space; Summarisation; Mappings, e.g. subspace methods
Abstract
The invention discloses an optimal neighbor multi-core clustering method and system based on local cores, wherein the related optimal neighbor multi-core clustering method based on local cores comprises the following steps: s11, acquiring clustering tasks and target data samples; s12, calculating a kernel matrix of each view corresponding to the target data sample, and performing centralization and normalization processing on the kernel matrix to obtain a processed kernel matrix; s13, establishing an optimal neighbor multi-core clustering target function based on a local core according to the obtained processed core matrix; s14, solving the established objective function in a circulation mode to obtain a division matrix after view fusion; and S15, carrying out k-means clustering on the obtained partition matrix to obtain a clustering result.
Description
Technical Field
The invention relates to the technical field of data analysis, in particular to an optimal neighbor multi-core clustering method and system based on local cores.
Background
Kernel clustering has been widely explored in current machine learning and data mining literature. It implicitly maps the original non-separable data to a high-dimensional hilbert space, with the corresponding vertices having explicit decision boundaries. Then, various clustering methods are applied, including k-means [ K.Krishna And N.M.Murty, "Genetic k-means algorithm," IEEE Transactions On Systems Man And Cybernetics-Part B: Cybernetics, vol.29, No.3, pp.433-439,1999 ], fuzzy c-means [ J.C.Bezdek, R.Ehrlich, And W.full, "Fcm: The fuzzy c-means clustering," Computers & Geosciences, vol.10, No.2-3, pp.191-203,1984 ], spectral clustering [ A.Y.Ng, M.I.Jordan, And Y.Weiss, "On spectral clustering: Analysis," in Advance in, And G.I.J.P.12, And Y.Weiss, "On spectral clustering" And "Analysis: Analysis," in addition, And "classification into high-density data (PR.12, P.31, P.V.S.V.V.S. transformation, P.V.V.V.V.V.V.S. 849, P.V.S.V.V.V.S. transformation). Although core clustering algorithms have met with great success in a large number of applications, they can only process data using a single core. Meanwhile, the kernel functions have different types, such as polynomial, gaussian, linear, etc., and need to be parameterized manually. How to select the correct kernel function and optimize the predefined parameters for a particular clustering task remains an open question. Nevertheless, in most practical environments sample features are collected from different sources. For example, news is reported by a plurality of news agencies; a person can be described by a fingerprint, palm vein, palm print, DNA, etc. The most common approach is to concatenate all features into one vector. But it ignores the fact that these functions may not be directly comparable.
The multi-core clustering (MKC) algorithm solves the above problems using complementary information of pre-specified cores, has been widely studied in the literature, and can be roughly classified into three categories. The method in the first class is to construct a cluster consensus kernel by integrating low rank optimization [ A.Trivedi, P.rai, H.Daume III, and S.L.DuVall, "Multi' clustering with completed views," in NIPS Workshop, vol.224,2010 ]. For example, Zhou et al first recover a shared low rank matrix from the transition probability matrices of the multi-core and then cluster it as input to the standard markov chain method [ p.zhou, l.du, l.shi, h.wang, and y. -d.shen "," Recovery of corrected multiple kernels for clustering ", in two-Fourth International Joint Conference on intelligent Intelligence 2015 ]. The second category of techniques uses a partitioning matrix generated from each core to compute their clustering results. Liu et al first performs a kernel k-means on each incomplete view, and then explores complementary information among all incomplete clustering results to obtain a final solution [ X.Liu, X.Zhu, M.Li, L.Wang, C.Tang, J.Yin, D.Shen, H.Wang, and W.Gao, "Late fusion in complete multi-view clustering," IEEE transactions on pattern analysis and machine interaction, 2018 ]. In contrast, the third class of algorithms establishes a consistent kernel during the clustering process. The basic assumption of most algorithms is that the optimal kernel can be represented as a weighted combination of pre-specified kernels. In addition to this, various regularization methods have been proposed to constrain the kernel weights and affinity matrices. For example, Du et al use the L21 norm in the original feature space to minimize reconstruction errors [ l.du, p.zhou, l.shi, h.wang, m.fan, w.wang, and y. -d.shen, "Robust multiple kernel k-means using 21-norm," in two ways source International Joint Conference on intellectual Intelligence,2015 ]. Liu et al processed the incomplete kernel and proposed a cross-kernel complete term to calculate the missing term in the kernel and also learned the weight of the kernel [ X.Liu, X.Zhu, M.Li, L.Wang, E.Zhu, T.Liu, M.Kloft, D.Shen, J.yin, and W.Gao, "Multiple kernel k-means with complete kernel," IEEE transactions on pattern analysis and machine interaction, 2019 ]. Some studies do not assume that samples are equal in a kernel, but perform clustering by assigning different weights to samples, such as [ p.zhang, y.yang, b.peng, and m.he ], "Multi-view clustering algorithm based on variable weight and mkl," in International Joint Conference on Rough sets, springer,2017, pp.599-610 ].
The kernel alignment is an effective regularization method in the multi-kernel k-means algorithm [ G.F.Tzortzis and A.C.Likas, "Multiple view clustering using a weighted combination of empirical-based textures," IEEE Transactions on neural networks, vol.21, No.12, pp.1925-1938,2010 ]. However, Li et al claim that collation forces all pairs of samples to align equally with the same ideal similarity, which is in conflict with the accepted concept that aligning two more distant samples with low similarity in high dimensional space is less reliable [ m.li, x.liu, w.lei, d.yong, j.yin, and e.zhu, 'Multiple kernel clustering with local kernel alignment knowledge,' in International Joint Conference on intellectual alignment, 2016 ]. Observing local kernel techniques in m.gonen and a.a.margolin, "Localized data fusion for kernel k-means clustering with application to cancer biology," in advance in Neural Information Processing Systems,2014, pp.1305-1313 ] can better capture sample-specific features in the data, they construct local kernels using the neighborhood of each sample and maximize their sum of alignments with the ideal similarity matrix. Furthermore, it was demonstrated that Local kernels can help clustering algorithms to better exploit the information provided by closer sample pairs [ q.wang, y.dou, x.liu, f.xia, q.lv, and k.yang, "Local kernel alignment based multi-view clustering using extreme learning machine," neuro clustering, vol.275, pp.1099-1111,2018 ].
The above-mentioned MKC algorithm has two problems: the local density around a single data sample and the expressive power of the over-constrained learning optimal kernel are not fully considered. Specifically, the local kernel in [ m.li, x.liu, w.lei, d.yong, j.yin, and e.zhu, "Multiple kernel clustering with local kernel alignment mapping," in International Joint conference on intelligent intellectual alignment, 2016] globally sets the neighborhood number of each sample to a constant, which does not guarantee that all pairs of samples in the local kernel are relatively close. It is well known that it is less reliable to compare to distant pairs of samples. Thus, such local kernels cannot minimize unreliability due to neglecting the local density around a single data sample. Meanwhile, most multi-core clustering algorithms assume that the optimal core is a weighted combination of pre-specified cores, and ignore some more robust cores in the complementary set of core combinations.
Disclosure of Invention
The invention aims to provide an optimal neighbor multi-core clustering method and system based on local cores, aiming at the defects of the prior art.
In order to achieve the purpose, the invention adopts the following technical scheme:
an optimal neighbor multi-core clustering method based on local cores comprises the following steps:
s1, acquiring a clustering task and a target data sample;
s2, calculating a kernel matrix of each view corresponding to the target data sample, and performing centralization and normalization processing on the kernel matrix to obtain a processed kernel matrix;
s3, establishing an optimal neighbor multi-core clustering target function based on a local core according to the obtained processed core matrix;
s4, solving the established objective function in a circulation mode to obtain a partition matrix after view fusion;
and S5, carrying out k-means clustering on the obtained partition matrix to obtain a clustering result.
Further, the step S2 is to calculate a kernel matrix of each view corresponding to the target data sample, specifically: for target data sampleThe p view in (1) is subjected to kernel function mapping to obtain a kernel matrix of the p view, which is expressed as:
wherein the content of the first and second substances,andrepresents the ith, j sample, and sigma represents the average of the distances between all the target data samples; e represents a natural constant; kp(i, j) represents the value of the ith row and j column in the kernel matrix of the pth view; m represents the number of views.
Further, in the step S3, an optimal neighbor multi-core clustering objective function based on the local core is established, and is represented as:
wherein H represents a partition matrix; β represents a combination coefficient; j represents an optimal neighbor core; n represents the number of all samples; μ represents an adaptive kernel similarity threshold; m represents a kernel relation matrix; h(i)Representing a partition matrix corresponding to the ith sample; h(i)TRepresents H(i)Transposing; beta is aTRepresenting a combined coefficient vector; m(i)A relation matrix representing an optimal neighbor kernel matrix of the ith sample; rho represents a hyper-parameter and needs to be set in advance; kβRepresenting the matrix obtained by combining the kernel matrices according to the beta coefficient,HT denotes the transpose of the partition matrix; i iskRepresenting a k-order identity matrix; beta is apA value representing the beta vector position p;indicates that for all p; j. the design is a square(i)=S(i)TJS(i),Represents μ of the ith sample(i)Nearest neighbor, S(i)TDenotes S(i)Transposing;represents μ(i)The identity matrix of (2).
Further, the step S4 is specifically:
s41, fixing J and beta, and optimizing H;
the objective function is converted into:
wherein A is(i)=S(i)S(i)T;A(i)Denotes an intermediate variable, A(i)=S(i)S(i)T;InRepresenting an n-order identity matrix;
s42, fixing H and beta, and optimizing J;
the objective function is converted into:
the matrix J obtains a solution to the problem by removing the negative characteristic values in the matrix B;
s43, fixing H and J, and optimizing beta;
the objective function is converted into:
wherein alpha isTRepresents a transposition of α;representing the relationship between the local kernel matrices p and q for the sample of the ith;a local kernel representing an ith sample in the pth kernel matrix;a local kernel representing an ith sample in the qth kernel matrix; mpqTo representThe relationship between the kernel matrices p and q; kp、KqAnd alpha each represents an intermediate variable; alpha is alphapRepresenting the value of position p in the alpha vector.
Further, the termination conditions in the steps S41, S42, S43 are expressed as:
(objt+1-objt)/objt≤σ
wherein objt+1And objtValues representing the objective function of the t +1 th and the t-th iterations, respectively; σ denotes the setting accuracy.
Correspondingly, an optimal neighbor multi-core clustering system based on local cores is also provided, which comprises:
the acquisition module is used for acquiring clustering tasks and target data samples;
the calculation module is used for calculating a kernel matrix of each view corresponding to the target data sample, and performing centralization and normalization processing on the kernel matrix to obtain a processed kernel matrix;
the establishing module is used for establishing an optimal neighbor multi-core clustering target function based on a local core according to the obtained processed core matrix;
the solving module is used for solving the established objective function in a circulating mode to obtain a division matrix after view fusion;
and the clustering module is used for carrying out k-means clustering on the obtained partition matrix to obtain a clustering result.
Further, the calculating, by the calculating module, the kernel matrix of each view corresponding to the target data sample specifically includes: for target data sampleThe p view in (1) is subjected to kernel function mapping to obtain a kernel matrix of the p view, which is expressed as:
wherein the content of the first and second substances,andrepresents the ith, j sample, and sigma represents the average of the distances between all the target data samples; e represents a natural constant; kp(i, j) represents the value of the ith row and j column in the kernel matrix of the pth view; m represents the number of views.
Further, the establishing module establishes an optimal neighbor multi-core clustering objective function based on a local core, which is expressed as:
wherein H represents a partition matrix; β represents a combination coefficient; j represents an optimal neighbor core; n represents the number of all samples; μ represents an adaptive kernel similarity threshold; m represents a kernel relation matrix; h(i)Representing a partition matrix corresponding to the ith sample; h(i)TRepresents H(i)Transposing; beta is aTRepresenting a combined coefficient vector; m(i)A relation matrix representing an optimal neighbor kernel matrix of the ith sample; rho represents a hyper-parameter and needs to be set in advance; kβRepresenting the matrix obtained by combining the kernel matrices according to the beta coefficient,HTrepresenting a transpose of a partition matrix; i iskRepresenting a k-order identity matrix; beta is apA value representing the beta vector position p;indicates that for all p; j. the design is a square(i)=S(i)TJS(i),Represents μ of the ith sample(i)Nearest neighbor, S(i)TDenotes S(i)Transposing;represents μ(i)The identity matrix of (2).
Further, the solving module specifically includes:
the first fixing module is used for fixing J and beta and optimizing H;
the objective function is converted into:
wherein A is(i)=S(i)S(i)T;A(i)Denotes an intermediate variable, A(i)=S(i)S(i)T;InRepresenting an n-order identity matrix;
the second fixing module is used for fixing H and beta and optimizing J;
the objective function is converted into:
the matrix J obtains a solution to the problem by removing the negative characteristic values in the matrix B;
the third fixing module is used for fixing H and J and optimizing beta;
the objective function is converted into:
wherein alpha isTRepresents a transposition of α;representing the relationship between the local kernel matrices p and q for the sample of the ith;a local kernel representing an ith sample in the pth kernel matrix;a local kernel representing an ith sample in the qth kernel matrix; mpqRepresenting the relationship between the kernel matrices p and q; kp、KqAnd alpha each represents an intermediate variable; alpha is alphapRepresenting the value of position p in the alpha vector.
Further, the termination conditions in the first, second and third fixed modules are expressed as:
(objt+1-objt)/objt≤σ
wherein objt+1And objtValues representing the objective function of the t +1 th and the t-th iterations, respectively; σ denotes the setting accuracy.
Compared with the prior art, the invention provides a novel local kernel-based optimal neighbor multi-kernel clustering method and system, which comprises three parts of constructing a self-adaptive local kernel matrix, searching and constructing an optimal neighbor kernel matrix and fusing the construction of the self-adaptive local kernel matrix, searching and clustering the optimal neighbor kernel matrix, and fusing the three parts in the same target formula for solving. The method greatly improves the performance of the multi-core clustering algorithm, and the experimental results on four public data sets prove that the performance of the method is superior to that of the existing algorithm.
Drawings
FIG. 1 is a flowchart of a local-core-based optimal neighbor multi-core clustering method according to an embodiment;
fig. 2 is a comparison diagram of the local kernel provided in the first embodiment and the second embodiment.
Detailed Description
The embodiments of the present invention are described below with reference to specific embodiments, and other advantages and effects of the present invention will be easily understood by those skilled in the art from the disclosure of the present specification. The invention is capable of other and different embodiments and of being practiced or of being carried out in various ways, and its several details are capable of modification in various respects, all without departing from the spirit and scope of the present invention. It is to be noted that the features in the following embodiments and examples may be combined with each other without conflict.
Aiming at the existing defects, the invention provides an optimal neighbor multi-core clustering method and system based on local cores.
Example one
The optimal neighbor multi-core clustering method based on local cores provided by this embodiment, as shown in fig. 1, includes:
s11, acquiring clustering tasks and target data samples;
s12, calculating a kernel matrix of each view corresponding to the target data sample, and performing centralization and normalization processing on the kernel matrix to obtain a processed kernel matrix;
s13, establishing an optimal neighbor multi-core clustering target function based on a local core according to the obtained processed core matrix;
s14, solving the established objective function in a circulation mode to obtain a division matrix after view fusion;
and S15, carrying out k-means clustering on the obtained partition matrix to obtain a clustering result.
Compared with the existing method, the new method for optimal neighbor multi-core clustering based on the local core comprises three parts of constructing an adaptive local core matrix, searching and constructing an optimal neighbor core matrix and fusing the construction of the adaptive local core matrix, searching and clustering the optimal neighbor core matrix, and fusing the three parts in the same target formula for solving, so that the clustering performance is greatly improved.
In step S12, a kernel matrix of each view corresponding to the target data sample is calculated, and the kernel matrix is subjected to centering and normalization processing to obtain a processed kernel matrix.
For target data sampleIn the present embodiment, the gaussian kernel function is taken as an example, and finally, a kernel matrix of the vth view is obtained, where the kernel matrix is expressed as:
wherein the content of the first and second substances,andrepresents the ith, j sample, and sigma represents the average of the distances between all the target data samples; e represents a natural constant; kp(i, j) represents the value of the ith row and j column in the kernel matrix of the pth view; m represents the number of views.
Thus, a kernel matrix of v views can be obtainedNext, each kernel matrix is centered and normalized, i.e., the mean is 0 and the variance is 1.
In step S13, an optimal neighbor multi-core clustering objective function based on the local core is established according to the obtained processed core matrix.
In this embodiment, an adaptive local kernel matrix based on adaptation is employed. The adaptive local kernel matrix is constructed as follows: for the kernel matrix J, the local kernel matrix corresponding to the ith sample consists of samples with the similarity greater than zeta, and can be formally expressed as J(i)=S(i)TJS(i),Represents μ of the ith sample(i)Nearest neighbor, S(i)TDenotes S(i)The transposing of (1). Drawing (A)2 is its visualization. Fig. 2(a) is a kernel matrix J, and the greater the inter-sample similarity, the higher the grayscale value, labeled 1, 0.75, 0.5, and 0.25 in order. When ζ is set to 0.75, subfigures 2(b.1) and 2(c.2) can be obtained. Fig. 2(b.1) shows an adaptive local kernel corresponding to the 1 st sample, and fig. 2(c.2) shows an adaptive local kernel corresponding to the 3 rd sample.
In this embodiment, an optimal neighbor multi-core clustering objective function based on a local core is established, and is expressed as:
wherein H represents a partition matrix; β represents a combination coefficient; j represents an optimal neighbor core; n represents the number of all samples; μ represents an adaptive kernel similarity threshold; m represents a kernel relation matrix; h(i)Representing a partition matrix corresponding to the ith sample; h(i)TRepresents H(i)Transposing; beta is aTRepresenting a combined coefficient vector; m(i)A relation matrix representing an optimal neighbor kernel matrix of the ith sample; rho represents a hyper-parameter and needs to be set in advance; kβRepresenting the matrix obtained by combining the kernel matrices according to the beta coefficient,HTrepresenting a transpose of a partition matrix; i iskRepresenting a k-order identity matrix; beta is apA value representing the beta vector position p;indicates that for all p; j. the design is a square(i)=S(i)TJS(i),Represents μ of the ith sample(i)Nearest neighbor, S(i)TDenotes S(i)Transposing;represents μ(i)The identity matrix of (2).
In step S14, the established objective function is solved in a cyclic manner to obtain a partition matrix after view fusion. The method specifically comprises the following steps:
s141, fixing J and beta, and optimizing H;
the objective function is converted into:
wherein A is(i)=S(i)S(i)T;A(i)Denotes an intermediate variable, A(i)=S(i)S(i)T;InRepresenting an n-order identity matrix;
s142, fixing H and beta, and optimizing J;
the objective function is converted into:
the matrix J obtains a solution to the problem by removing the negative characteristic values in the matrix B;
s143, fixing H and J, and optimizing beta;
the objective function is converted into:
wherein alpha isTRepresents a transposition of α;representing the relationship between the local kernel matrices p and q for the sample of the ith;a local kernel representing an ith sample in the pth kernel matrix;a local kernel representing an ith sample in the qth kernel matrix; mpqRepresenting the relationship between the kernel matrices p and q; kp、KqAnd alpha each represents an intermediate variable; alpha is alphapRepresenting the value of position p in the alpha vector.
This is a standard QP problem that can be solved by existing algorithmic packages.
In the prior art, the algorithm can be solved by a Lagrange multiplier method, and the algorithm is directly called in Matlab for solving.
In the present embodiment, steps S141, S142, and S143 need to be performed alternately by an alternating method until convergence, where the termination condition (i.e., convergence condition) is expressed as:
(objt+1-objt)/objt≤σ
wherein objt+1And objtValues representing the objective function of the t +1 th and the t-th iterations, respectively; σ denotes the setting accuracy.
In step S15, k-means clustering is performed on the obtained partition matrix to obtain a clustering result.
And performing k-means clustering on the obtained partition matrix to obtain a clustering result, namely performing standard k-means clustering on the matrix H to obtain a final clustering result.
Compared with the whole kernel matrix ONKC, the clustering performance obtained by the method is different by adopting the self-adaptive local kernel matrix of the embodiment. This example results in better performance (expressed using cluster accuracy ACC) as shown in table 1 below:
ONKC | method for producing a composite material |
41.56 | 45.44 |
91.00 | 96.30 |
35.91 | 38.04 |
39.19 | 40.63 |
TABLE 1
The traditional multi-kernel clustering algorithm does not fully consider the local density among samples, and severely limits the value range of the optimal kernel for final clustering, so that the obtained performance is not high. The embodiment aims to provide an optimal neighbor multi-core clustering method based on local cores. The method comprises the steps of constructing a self-adaptive local kernel matrix, searching an optimal neighbor kernel around a linear combination of a plurality of predefined kernels, and clustering by using the neighbor kernel. Meanwhile, the three processes are put in the same target formula for alternate optimization, and when the change of loss tends to be stable, the final clustering result is obtained.
Correspondingly, the present embodiment further provides an optimal neighbor multi-core clustering system based on local cores, including:
the acquisition module is used for acquiring clustering tasks and target data samples;
the calculation module is used for calculating a kernel matrix of each view corresponding to the target data sample, and performing centralization and normalization processing on the kernel matrix to obtain a processed kernel matrix;
the establishing module is used for establishing an optimal neighbor multi-core clustering target function based on a local core according to the obtained processed core matrix;
the solving module is used for solving the established objective function in a circulating mode to obtain a division matrix after view fusion;
and the clustering module is used for carrying out k-means clustering on the obtained partition matrix to obtain a clustering result.
Further, the calculating, by the calculating module, the kernel matrix of each view corresponding to the target data sample specifically includes: for target data sampleThe p view in (1) is subjected to kernel function mapping to obtain a kernel matrix of the p view, which is expressed as:
wherein the content of the first and second substances,andrepresents the ith, j sample, and sigma represents the average of the distances between all the target data samples; e represents a natural constant; kp(i, j) represents the value of the ith row and j column in the kernel matrix of the pth view; m represents the number of views.
Further, the establishing module establishes an optimal neighbor multi-core clustering objective function based on a local core, which is expressed as:
wherein H representsDividing a matrix; β represents a combination coefficient; j represents an optimal neighbor core; n represents the number of all samples; μ represents an adaptive kernel similarity threshold; m represents a kernel relation matrix; h(i)Representing a partition matrix corresponding to the ith sample; h(i)TRepresents H(i)Transposing; beta is aTRepresenting a combined coefficient vector; m(i)A relation matrix representing an optimal neighbor kernel matrix of the ith sample; rho represents a hyper-parameter and needs to be set in advance; kβRepresenting the matrix obtained by combining the kernel matrices according to the beta coefficient,HTrepresenting a transpose of a partition matrix; i iskRepresenting a k-order identity matrix; beta is apA value representing the beta vector position p;indicates that for all p; j. the design is a square(i)=S(i)TJS(i),Represents μ of the ith sample(i)Nearest neighbor, S(i)TDenotes S(i)Transposing;represents μ(i)The identity matrix of (2).
Further, the solving module specifically includes:
the first fixing module is used for fixing J and beta and optimizing H;
the objective function is converted into:
wherein A is(i)=S(i)S(i)T;A(i)Denotes an intermediate variable, A(i)=S(i)S(i)T;InRepresenting an n-order identity matrix;
the second fixing module is used for fixing H and beta and optimizing J;
the objective function is converted into:
the matrix J obtains a solution to the problem by removing the negative characteristic values in the matrix B;
the third fixing module is used for fixing H and J and optimizing beta;
the objective function is converted into:
wherein alpha isTRepresents a transposition of α;representing the relationship between the local kernel matrices p and q for the sample of the ith;a local kernel representing an ith sample in the pth kernel matrix;a local kernel representing an ith sample in the qth kernel matrix; mpqRepresenting the relationship between the kernel matrices p and q; kp、KqAnd alpha each represents an intermediate variable; alpha is alphapRepresenting the value of position p in the alpha vector.
Further, the termination conditions in the first, second and third fixed modules are expressed as:
(objt+1-objt)/objt≤σ
wherein objt+1And objtValues representing the objective function of the t +1 th and the t-th iterations, respectively; σ denotes the setting accuracy.
Compared with the prior art, the invention provides a novel local kernel-based optimal neighbor multi-core clustering system which comprises three parts of building a self-adaptive local kernel matrix, searching and building an optimal neighbor kernel matrix and fusing the building of the self-adaptive local kernel matrix, searching and clustering the optimal neighbor kernel matrix, and fusing the three parts in the same objective formula for solving. The method greatly improves the performance of the multi-core clustering algorithm.
Example two
The difference between the optimal neighbor multi-core clustering method based on the local core provided by the embodiment and the embodiment one is that:
the main content of the embodiment includes that aiming at two problems that the local density of a single data sample and the representation capability of an optimal kernel learned by over-limitation are not fully considered in the current multi-kernel clustering algorithm, a self-adaptive local kernel is designed, and the optimal kernel is positioned from a pre-specified linear combination neighborhood of kernels; utilizing both techniques into a single multi-kernel cluster framework; the popularization range of the optimal neighborhood multi-core clustering algorithm based on the self-adaptive local core is researched.
The adaptive local kernel is a sub-matrix of the kernel function, and the main function is to reflect the relationship between the sample and its neighborhood. First, a threshold ζ is defined, and the corresponding index set Ω of the ith sample(i)Can be written as omega(i){ j | K (i, j) ≧ ζ } then the corresponding index matrix Is defined as:
the ith adaptive local kernel of matrix K can be represented as:
in other words, the above equation selects μ with a kernel value greater than ζ corresponding to the ith sample(i)Adjacent samples and the other samples are discarded. Using the constructed local kernels in the multi-kernel k-means and setting the weight of matrix-induced regularization in to 1, we can rewrite the following form
WhereinH(i)=S(i)TH,Is in the size of mu(i)And mu(i)The identity matrix of (a) varies with the density around the sample.
The adaptive local kernel proposed in this example is extended from the local kernel in [ m.li, x.liu, w.lei, d.yong, j.yin, and e.zhu, "Multiple kernel clustering with local kernel alignment mapping," in International Joint conference on intelligent kernel, 2016], directly to make the size of the local kernel constant. However, this does not guarantee that all pairs of samples are in a local kernel of high similarity. In contrast, the present embodiment constructs the ith adaptive local kernel by selecting samples whose similarity to sample i is higher than the threshold ζ. Fig. 2 compares these two types of local kernels globally. It can be seen that the local kernels generated in [ m.li, x.liu, w.lei, d.yong, j.yin, and e.zhu, "Multiple kernel clustering with local kernel alignment mapping," in International Joint conference on scientific intellgence, 2016] have the same size, while the proposed adaptive local kernel is determined by the similarity of the sample pairs. Comparing b.1, b.2 and c.1, c.2 in fig. 2, it can be noted that the proposed adaptive local kernel is generally smaller than the local kernel in [ m.li, x.liu, w.lei, d.yong, j.yin, and e.zhu, "Multiple kernel clustering with local kernel alignment knowledge," in International Joint conference on scientific integration, 2016] ], thus ensuring relatively high similarity of all neighbors and reducing the unreliability of further comparison sample pairs.
Local kernel comparison as shown in fig. 2: the darkness of the boxes indicates the degree of similarity between the sample pairs. The darker the box, the more similar the corresponding sample pair. a is the original kernel matrix, b.1 and b.2 are the local kernels corresponding to the 1/3 samples generated in [ M.Li, X.Liu, W.Lei, D.Yong, J.Yin, and E.Zhu, "Multiple kernel clustering with local kernel alignment mapping," in International Joint conference on Intelligent Intelligence,2016 ]. The size mu is fixed to 3; c.1 and c.2 are adaptive local kernels corresponding to the 1/3 samples. The similarity to its neighbors is higher than ζ.
The optimal core (referred to as J) is assumed to reside in the neighborhood of the core combination, denoted as:
this assumption yields the target equation in the equation as follows:
the objectives in the objective equation above are difficult to optimize due to constraints on J. It was observed that KβProvides prior knowledge for clustering, J is more likely to be at KβThe optimum value is reached in the case where the difference between them is small. This example does notThe maximum gap theta is set explicitly, and the actual gap is learned in the clustering process, so that the final target formula is formed
Wherein the content of the first and second substances,K(i)=S(i)TKS(i),J(i)=S(i)TJS(i),represents μ of the ith sample(i)A nearest neighbor, where n is the number of all samples,is in the size of mu(i)The identity matrix of (2). The optimal kernel J serves as a linkage clustering process and a knowledge acquisition process. In this case, it uses complementary information in the pre-specified cores to assist in the clustering process, and information from the clusters to assist in the weight assignment of the pre-specified cores as feedback.
EXAMPLE III
The difference between the optimal neighbor multi-core clustering method based on the local core provided by the embodiment and the embodiment one is that:
this example compares the present invention across multiple data sets to verify the effectiveness of the invention.
Data set:
flower 102: the data set contains 8189 samples, evenly distributed in 102 classes, with 4 kernel matrices.
Digital: the data set consists of 2000 samples, evenly distributed in 10 classes, with 3 kernel matrices.
Caltech 101: the data set contains 1530 samples, evenly distributed in 102 classes, with 25 kernel matrices.
Protein Fold: the data set comprised 694 samples, evenly distributed in 27 classes, with 12 kernel matrices.
The statistical information of the above data sets is shown in table 2.
TABLE 2
Data preparation and parameter setting:
in the initialization phase, the core matrix is centered according to the method described in [ C.Cortes, M.Mohri, and A.Rosemizadeh, "Algorithms for Learning kernels based on centered alignment," Journal of Machine Learning Research, vol.13, No.2, pp.795-828,2012 ]. It is then normalized to better specify a range of similarity values between pairs of samples between-1 and 1.
The method has two superparameters, namely rho and xi. And p represents the relative importance degree of the construction of the self-adaptive local kernel matrix and the optimal neighbor kernel matrix. ξ represents the similarity threshold between the neighbor samples. A grid search technique is employed for selecting these two parameters, where p is at 2-15To 215And ξ varies from-0.5 to 0.5.
Evaluation indexes are as follows:
and evaluating by adopting a clustering algorithm general Accuracy (ACC) evaluation index.
This example compares the results of RMKC [ P.ZHOU, L.Du, L.Shi, H.Wang, and Y. -D.Shen, "Recovery of corrected Multiple keys for use in the clustering," in twin-Fourth International journal Conference architecture, 2015.], RMKKM [ L.Du, P.ZHOU, L.Shi, H.Wang, M.Fan, W.Wang, and Y. -D.Shen, "Robust Multiple keys k-means using 21-norm," in twin-Fourth International journal architecture, and III ] comparing the results with the results of the three multi-view clustering methods in the literature, as shown in the results of RMKC [ P.ZHOU, L.Du, L.Shi, W.Wang, and Y. -D.Shen, "Mr. K-media linkage architecture k-means using 21-norm," in twin-Fourth journal architecture, III.J., and M.J., while "III-III".
TABLE 3
The experimental results of the embodiment on four common data sets prove that the performance of the method is superior to that of the existing algorithm.
It is to be noted that the foregoing is only illustrative of the preferred embodiments of the present invention and the technical principles employed. It will be understood by those skilled in the art that the present invention is not limited to the particular embodiments described herein, but is capable of various obvious changes, rearrangements and substitutions as will now become apparent to those skilled in the art without departing from the scope of the invention. Therefore, although the present invention has been described in greater detail by the above embodiments, the present invention is not limited to the above embodiments, and may include other equivalent embodiments without departing from the spirit of the present invention, and the scope of the present invention is determined by the scope of the appended claims.
Claims (10)
1. An optimal neighbor multi-core clustering method based on local cores is characterized by comprising the following steps:
s1, acquiring a clustering task and a target data sample;
s2, calculating a kernel matrix of each view corresponding to the target data sample, and performing centralization and normalization processing on the kernel matrix to obtain a processed kernel matrix;
s3, establishing an optimal neighbor multi-core clustering target function based on a local core according to the obtained processed core matrix;
s4, solving the established objective function in a circulation mode to obtain a partition matrix after view fusion;
and S5, carrying out k-means clustering on the obtained partition matrix to obtain a clustering result.
2. The local-core-based optimal neighbor multi-core aggregation according to claim 1The class method is characterized in that the step S2 of calculating the kernel matrix of each view corresponding to the target data sample is specifically: for target data sampleThe p view in (1) is subjected to kernel function mapping to obtain a kernel matrix of the p view, which is expressed as:
wherein the content of the first and second substances,andrepresents the ith, j sample, and sigma represents the average of the distances between all the target data samples; e represents a natural constant; kp(i, j) represents the value of the ith row and j column in the kernel matrix of the pth view; m represents the number of views.
3. The method according to claim 2, wherein the step S3 is implemented by establishing an optimal neighbor multi-core clustering objective function based on the local core, which is expressed as:
wherein H represents a partition matrix; β represents a combination coefficient; j represents an optimal neighbor core; n represents the number of all samples; μ represents an adaptive kernel similarity threshold; m represents a kernel relation matrix; h(i)Represents the ith sampleThe corresponding division matrix;represents H(i)Transposing;representing a combined coefficient vector; m(i)A relation matrix representing an optimal neighbor kernel matrix of the ith sample; rho represents a hyper-parameter and needs to be set in advance; kβRepresenting the matrix obtained by combining the kernel matrices according to the beta coefficient, representing a transpose of a partition matrix; i iskRepresenting a k-order identity matrix; beta is apA value representing the beta vector position p;indicates that for all p; represents μ of the ith sample(i)The nearest neighbourhood of the nearest neighbourhood,denotes S(i)Transposing;represents μ(i)The identity matrix of (2).
4. The optimal neighbor multi-core clustering method based on local cores according to claim 3, wherein the step S4 specifically comprises:
s41, fixing J and beta, and optimizing H;
the objective function is converted into:
wherein the content of the first and second substances,A(i)the intermediate variable is represented by a number of variables,Inrepresenting an n-order identity matrix;
s42, fixing H and beta, and optimizing J;
the objective function is converted into:
the matrix J obtains a solution to the problem by removing the negative characteristic values in the matrix B;
s43, fixing H and J, and optimizing beta;
the objective function is converted into:
α=[α1,…,αm],αp=-ρTr(JKp)
wherein the content of the first and second substances,represents a transposition of α;representing the relationship between the local kernel matrices p and q for the sample of the ith;a local kernel representing an ith sample in the pth kernel matrix;a local kernel representing an ith sample in the qth kernel matrix; mpqRepresenting the relationship between the kernel matrices p and q; kp、KqAnd alpha each represents an intermediate variable; alpha is alphapRepresenting the value of position p in the alpha vector.
5. The local-kernel-based optimal neighbor multi-kernel clustering method according to claim 4, wherein the termination conditions in the steps S41, S42 and S43 are expressed as:
(objt+1-objt)/objt≤σ
wherein objt+1And objtIndividual watchShowing the values of the objective function of the t +1 th iteration and the t th iteration; σ denotes the setting accuracy.
6. An optimal neighbor multi-core clustering system based on local cores, comprising:
the acquisition module is used for acquiring clustering tasks and target data samples;
the calculation module is used for calculating a kernel matrix of each view corresponding to the target data sample, and performing centralization and normalization processing on the kernel matrix to obtain a processed kernel matrix;
the establishing module is used for establishing an optimal neighbor multi-core clustering target function based on a local core according to the obtained processed core matrix;
the solving module is used for solving the established objective function in a circulating mode to obtain a division matrix after view fusion;
and the clustering module is used for carrying out k-means clustering on the obtained partition matrix to obtain a clustering result.
7. The optimal neighbor multi-core clustering system based on local cores according to claim 6, wherein the calculating module calculates the core matrix of each view corresponding to the target data sample specifically as follows: for target data sampleThe p view in (1) is subjected to kernel function mapping to obtain a kernel matrix of the p view, which is expressed as:
wherein the content of the first and second substances,andrepresents the ith, j sample, and sigma represents the average of the distances between all the target data samples; e represents a natural constant; kp(i, j) represents the value of the ith row and j column in the kernel matrix of the pth view; m represents the number of views.
8. The system according to claim 7, wherein the establishing module establishes the optimal neighbor multi-core clustering objective function based on the local core, which is expressed as:
wherein H represents a partition matrix; β represents a combination coefficient; j represents an optimal neighbor core; n represents the number of all samples; μ represents an adaptive kernel similarity threshold; m represents a kernel relation matrix; h(i)Representing a partition matrix corresponding to the ith sample;represents H(i)Transposing;representing a combined coefficient vector; m(i)A relation matrix representing an optimal neighbor kernel matrix of the ith sample; rho represents a hyper-parameter and needs to be set in advance; kβRepresenting the matrix obtained by combining the kernel matrices according to the beta coefficient, representing a transpose of a partition matrix; i iskRepresenting units of order kA matrix; beta is apA value representing the beta vector position p;indicates that for all p; represents μ of the ith sample(i)The nearest neighbourhood of the nearest neighbourhood,denotes S(i)Transposing;represents μ(i)The identity matrix of (2).
9. The local-kernel-based optimal neighbor multi-kernel clustering system according to claim 8, wherein the solving module is specifically:
the first fixing module is used for fixing J and beta and optimizing H;
the objective function is converted into:
wherein the content of the first and second substances,A(i)the intermediate variable is represented by a number of variables,Inrepresenting an n-order identity matrix;
the second fixing module is used for fixing H and beta and optimizing J;
the objective function is converted into:
the matrix J obtains a solution to the problem by removing the negative characteristic values in the matrix B;
the third fixing module is used for fixing H and J and optimizing beta;
the objective function is converted into:
α=[α1,…,αm],αp=-ρTr(JKp)
wherein the content of the first and second substances,represents a transposition of α;representing the relationship between the local kernel matrices p and q for the sample of the ith;a local kernel representing an ith sample in the pth kernel matrix;a local kernel representing an ith sample in the qth kernel matrix; mpqRepresenting the relationship between the kernel matrices p and q; kp、KqAnd alpha each represents an intermediate variable; alpha is alphapRepresenting the value of position p in the alpha vector.
10. The local-core-based optimal neighbor multi-core clustering system according to claim 9, wherein the termination conditions in the first fixed module, the second fixed module and the third fixed module are expressed as:
(objt+1-objt)/objt≤σ
wherein objt+1And objtValues representing the objective function of the t +1 th and the t-th iterations, respectively; σ denotes the setting accuracy.
Priority Applications (4)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CN202110447615.9A CN113269231A (en) | 2021-04-25 | 2021-04-25 | Local kernel-based optimal neighbor multi-core clustering method and system |
PCT/CN2022/082643 WO2022227956A1 (en) | 2021-04-25 | 2022-03-24 | Optimal neighbor multi-kernel clustering method and system based on local kernel |
LU503092A LU503092B1 (en) | 2021-04-25 | 2022-03-24 | A method and system for optimal neighbor multicore clustering based on local kernels |
ZA2022/07734A ZA202207734B (en) | 2021-04-25 | 2022-07-12 | A method and system for optimal neighbor multicore clustering based on local kernels |
Applications Claiming Priority (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CN202110447615.9A CN113269231A (en) | 2021-04-25 | 2021-04-25 | Local kernel-based optimal neighbor multi-core clustering method and system |
Publications (1)
Publication Number | Publication Date |
---|---|
CN113269231A true CN113269231A (en) | 2021-08-17 |
Family
ID=77229306
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
CN202110447615.9A Pending CN113269231A (en) | 2021-04-25 | 2021-04-25 | Local kernel-based optimal neighbor multi-core clustering method and system |
Country Status (4)
Country | Link |
---|---|
CN (1) | CN113269231A (en) |
LU (1) | LU503092B1 (en) |
WO (1) | WO2022227956A1 (en) |
ZA (1) | ZA202207734B (en) |
Cited By (1)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
WO2022227956A1 (en) * | 2021-04-25 | 2022-11-03 | 浙江师范大学 | Optimal neighbor multi-kernel clustering method and system based on local kernel |
Families Citing this family (1)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN117292162B (en) * | 2023-11-27 | 2024-03-08 | 烟台大学 | Target tracking method, system, equipment and medium for multi-view image clustering |
Family Cites Families (3)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN109145976A (en) * | 2018-08-14 | 2019-01-04 | 聚时科技(上海)有限公司 | A kind of multiple view cluster machine learning method based on optimal neighbours' core |
CN110188812A (en) * | 2019-05-24 | 2019-08-30 | 长沙理工大学 | A kind of multicore clustering method of quick processing missing isomeric data |
CN113269231A (en) * | 2021-04-25 | 2021-08-17 | 浙江师范大学 | Local kernel-based optimal neighbor multi-core clustering method and system |
-
2021
- 2021-04-25 CN CN202110447615.9A patent/CN113269231A/en active Pending
-
2022
- 2022-03-24 WO PCT/CN2022/082643 patent/WO2022227956A1/en active Application Filing
- 2022-03-24 LU LU503092A patent/LU503092B1/en active IP Right Grant
- 2022-07-12 ZA ZA2022/07734A patent/ZA202207734B/en unknown
Cited By (1)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
WO2022227956A1 (en) * | 2021-04-25 | 2022-11-03 | 浙江师范大学 | Optimal neighbor multi-kernel clustering method and system based on local kernel |
Also Published As
Publication number | Publication date |
---|---|
ZA202207734B (en) | 2022-07-27 |
LU503092B1 (en) | 2023-03-22 |
WO2022227956A1 (en) | 2022-11-03 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
Zhou et al. | Self-paced clustering ensemble | |
Wang et al. | Feature selection and multi-kernel learning for adaptive graph regularized nonnegative matrix factorization | |
Wang et al. | Robust dimension reduction for clustering with local adaptive learning | |
CN113269231A (en) | Local kernel-based optimal neighbor multi-core clustering method and system | |
Zamiri et al. | MVDF-RSC: Multi-view data fusion via robust spectral clustering for geo-tagged image tagging | |
Mahmoud et al. | Pseudoinverse learning autoencoder with DCGAN for plant diseases classification | |
Liu et al. | Structured block diagonal representation for subspace clustering | |
CN113837218A (en) | Text clustering method and system based on one-step post-fusion multi-view | |
Liu et al. | Incomplete multi-view partial multi-label learning | |
Fang et al. | Learning decomposed spatial relations for multi-variate time-series modeling | |
Zhu et al. | Local information fusion network for 3D shape classification and retrieval | |
CN113435603A (en) | Agent graph improvement-based late-stage fusion multi-core clustering machine learning method and system | |
Zhou et al. | Semantic adaptation network for unsupervised domain adaptation | |
Chen et al. | Dual distance adaptive multiview clustering | |
CN117409456A (en) | Non-aligned multi-view multi-mark learning method based on graph matching mechanism | |
Wu et al. | Deep clustering and representation learning that preserves geometric structures | |
CN116304768A (en) | High-dimensional density peak clustering method based on improved equidistant mapping | |
Kerner et al. | Introduction to machine learning | |
CN115169436A (en) | Data dimension reduction method based on fuzzy local discriminant analysis | |
CN113095270B (en) | Unsupervised cross-library micro-expression identification method | |
CN113627237A (en) | Late-stage fusion face image clustering method and system based on local maximum alignment | |
Zhang et al. | Leverage triple relational structures via low-rank feature reduction for multi-output regression | |
Dornaika et al. | Simultaneous label inference and discriminant projection estimation through adaptive self-taught graphs | |
Shiao et al. | Implementation and comparison of SVM-based multi-task learning methods | |
Jun et al. | Two-view correspondence learning via complex information extraction |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
PB01 | Publication | ||
PB01 | Publication | ||
SE01 | Entry into force of request for substantive examination | ||
SE01 | Entry into force of request for substantive examination |