CN105844303A - Sampling type clustering integration method based on local and global information - Google Patents

Abstract

The invention discloses a sampling type clustering integration method based on the local and global information. Firstly, mixed sampling of a target data set is carried out, a learning sample is further generated, clustering analysis on the learning sample space is carried out, clustering division is further generated, quality evaluation on the clustering division is carried out, and a weight vector of the target data set is updated according to the evaluation result; the previous steps are repeated for multiple times, and multiple clustering divisions are generated; the multiple clustering divisions are fused to be one new characteristic expression, a traditional clustering algorithm is utilized to carry out clustering analysis on the characteristic expression, and the integration clustering result is generated. According to the method, integration learning has relatively strong noise immunity, and relatively high problem-data solving capability is further realized; new characteristics can effectively and comprehensively express the global and local cluster structure information, so the integration learning algorithm generates good effects on data sets having different characteristics.

Description

A kind of sampling type clustering ensemble method based on local with global information

Technical field

The invention belongs to machine learning field, particularly relate to a kind of sampling type clustering ensemble based on local with global information Method.

Background technology

The invention discloses a kind of sampling type clustering ensemble method based on local with global information, relate generally to learn sample This sampling mechanism and two aspects of clustering ensemble learning algorithm: (1) learning sample sampling mechanism data sampling techniques mainly includes Three kinds: lack sampling, over-sampling and mixing sampling.Stochastical sampling is one relatively simple in Undersampling technique, and it removes at random Most class samples in data set, reduce the calculating cost in learning process, when especially there is noise in data set, use Random Undersampling technique effect is preferable；Another is exactly weight sampling method, each in data set of this method of sampling Sample one weights of distribution, the probability of the learning sample extracted is determined by the size of its weights, and it makes learning process have There is directivity, focus on problem sample, but, this type of method is the most sensitive for noise and exceptional sample, easily causes That practises is inaccurate.The shortcoming of sampling technique is to easily cause the overfitting of grader, the most also increases and calculates cost.Right When most classes use Undersampling technique, some useful informations lost in most class data can be there are, and to minority class data Oversampling technique, the time of training can be the highest with complexity, sometimes results even in the over-fitting of grader.ALTERNATE SAMPLING method Mainly in supervised learning problem, its sampling mechanism is to be realized by the data set category information of priori, and this information is non-supervisory Problem concerning study does not provide.So for the stochastical sampling of clustering ensemble algorithm, only Bagging and the weighting of Boosting The method of sampling is the most feasible.But all there is respective defect in both approaches.(2) clustering ensemble algorithm, clustering ensemble study Purpose is multiple cluster analysis results of same target data set to be combined generation one have higher performance Whole clustering ensemble result.In general, the structure of Ensemble Learning Algorithms is made up of two parts, i.e. member cluster the generation of device with Merging, existing clustering ensemble learning algorithm mainly there are differences in these two aspects.Generate multiple members in the first step and cluster device Time, if can produce high-quality and the big member of difference cluster device set be determine integrated learning outcome quality important because of Element.For clustering ensemble learning algorithm, a lot of methods can be used to produce multiple member and cluster device, and conventional method can be divided into: Same data set uses different clustering algorithms, to produce different cluster results；Use same clustering algorithm, but combine Different initialization and parameter arrange and produce different cluster results；Phase is used in multiple feature spaces of same data set Same clustering algorithm, to produce different cluster results；Target data set is carried out learning sample sampling, at different study samples This space uses identical clustering algorithm, to produce different cluster results.

There is significant limitation in existing clustering ensemble learning algorithm: just for single feature data set effectively and logarithm Have stronger hypothetical according to the clustering architecture of collection.To this end, develop a kind of cluster set preconceived plan being generally applicable to different characteristics data set Method is the most urgent.

Summary of the invention

It is an object of the invention to provide a kind of sampling type clustering ensemble method based on local with global information, it is intended to solve Certainly existing clustering ensemble learning algorithm exists the data set just for single feature effectively and has stronger to the clustering architecture of data set Hypothetical problem.

The present invention is achieved in that a kind of sampling type clustering ensemble method based on local with global information, described Sampling type clustering ensemble method based on local and global information comprises the following steps:

First target data set is carried out mixing sampling and generates learning sample, learning sample space carries out cluster point Analyse and generate clustering, next clustering being carried out quality evaluation, and updates target data set according to assessment result Weight vectors, repeats many wheels, and then produces multiple clustering；

Then multiple clusterings are permeated a new character representation, and use clustering algorithm that this character representation is done Cluster analysis, and generate integrated cluster result.

Further, the clustering ensemble fusion function of described clustering algorithm according to the local of clustering architecture and global information by same Multiple divisions of data set are converted into a new character representation, in this feature space, use clustering algorithm to generate final Integrated division.

Further, described mixing sampling includes stochastical sampling and two kinds of method of samplings of weight sampling, method particularly includes:

Step one, the method for use stochastical sampling are concentrated from target data and are extracted initial learn sample；

Step 2, the method for use weight sampling extract final learning sample from initial learn sample.

Further, to target data setCarry out taking turns sampling more, and use K-means clustering algorithm to sample space Carry out cluster analysis, produce multiple initial clustering and divideK-means clustering algorithm is as follows:

Step one, arbitrarily select k object as initial cluster center from n data object；

Step 2, average according to each clustering object, calculate the distance of each object and center object；And according to minimum Distance carries out clustering to corresponding object again；

Step 3, recalculate the average of each cluster；

Step 4, circulation step two to step 3 are until each cluster no longer changes.

Further, described clustering ensemble fusion function integrates local and the global information of clustering architecture, by same data setMultiple divisionsIt is converted into a new character representation H={ α₁H¹.....α_tH^t, wherein,And α_t Respectively divide p_tCharacter representation and weights；

$h_{i,j}=\exp ({\left(\frac{1}{\left|NB\right|}\underset{\left|NB\left(x_i\right)\right|}{\Σ}d\left(NB\left(x_i\right),{\mathrm{\μ}}_j\right)\right)}^2/2{\mathrm{\σ}}_i{\mathrm{\σ}}_j)$

x_iFor the mark of sample i, | NB | is self-defining Size of Neighborhood,Point in neighborhood to bunch Characterize some μ_jAverage distance,WithIt is in x respectively_iPoint is characterized with bunch μ_jLocal scaling factor.

The mixing method of sampling that the present invention proposes has merged stochastical sampling and weight sampling in one, mixing sampling mechanism Stochastical sampling can reduce the noise data point in learning sample, the weight sampling in mixing sampling mechanism can select again simultaneously Take more difficult learning sample and carry out that there is cluster analysis targetedly, its combine two kinds of classical Integrated Algorithm Bagging with The advantage of Boosting；Clustering ensemble fusion function according to the local of clustering architecture and global information by multiple strokes of same data set Divide and be converted into a new character representation；In this feature space, it is possible to use any traditional clustering algorithm generates Whole integrated division, this character representation can effectively and all sidedly characterize the clustering architecture information of overall situation and partial situation so that integrated Practise the effect that algorithm produces on the data set of different characteristics.First the present invention carries out mixing sampling life to target data set Becoming learning sample, the stochastical sampling in mixing sampling mechanism can reduce the noise data point in learning sample, mixing simultaneously is adopted Weight sampling in model machine system can be chosen more difficult learning sample and carry out having cluster analysis targetedly, and generates cluster stroke Point；Next clustering is carried out quality evaluation, and according to the weight vector of assessment result renewal target data set, has relatively The weight of the data point of high clustering result quality reduces, and the weight of the data point with poor clustering result quality raises, then according to renewal Weight vector be weighted sampling, the data point of poor clustering result quality will be selected in next round, and have it Cluster analysis targetedly, above work repeats many wheels, and then produces multiple clustering；Then multiple clusterings are merged It is a new character representation, and uses traditional clustering algorithm that this character representation is done cluster analysis, and generate clustering ensemble As a result, this character representation can effectively and all sidedly characterize the clustering architecture information of overall situation and partial situation so that Ensemble Learning Algorithms exists The effect produced on the data set of different characteristics.

Accompanying drawing explanation

Fig. 1 is the sampling type clustering ensemble method flow diagram based on local with global information that the embodiment of the present invention provides.

Fig. 2 is the artificial data-method comparison of the different densities distribution clustering architecture that the embodiment of the present invention provides.

Fig. 3 is the artificial data-method comparison of the uneven clustering architecture that the embodiment of the present invention provides.

Fig. 4 is the artificial data-method comparison of the special clustering architecture that the embodiment of the present invention provides.

Detailed description of the invention

For the summary of the invention of the present invention, feature and effect can be further appreciated that, hereby enumerate following example, and coordinate accompanying drawing Describe in detail as follows.

Refer to Fig. 1:

A kind of sampling type clustering ensemble method based on local with global information, including:

S101: target data；

S102: initial clustering generation module based on blended learning sample collection mechanism；

S103: clustering ensemble fusion function module based on overall situation and partial situation's clustering architecture information；

S104: clustering ensemble result.

Further, described initial clustering generation module based on blended learning sample collection mechanism is random by optimum organization Sampling and two kinds of method of samplings of weight sampling, carry out hybrid sampling, method particularly includes:

Step one, the method for use stochastical sampling are concentrated from target data and are extracted initial learn sample；

Step 2, the method for use weight sampling extract final learning sample from initial learn sample.

This new learning sample acquisition method not only makes integrated study have the anti-noise that Bagging Integrated Algorithm is the same Property, also make it have the same specific aim of Boosting Integrated Algorithm simultaneously and solve the ability of problem data.From the angle of theory analysis Degree, can prove the reasonability of mixing sampling mechanism by the cost function that derivation clustering ensemble learns.Such as formula (eq.1) Shown in, it can be deduced that conclusion: the integrated study cost function of exponential form is the upper limit of actual cost function in fact.

$\begin{array}{c}{L}_e=\underset{n}{\Σ}p\left(x_n\right)\exp \left(\underset{t}{\Σ}{\mathrm{\α}}_t{l}_t\right(x_n\left)\right)\\ =\underset{n}{\Σ}p\left(x_n\right)\exp \left({\mathrm{\α}}_T{l}_T\right(x_n\left)\right)\underset{t=1}{\overset{T-1}{\Π}}\exp \left({\mathrm{\α}}_t{l}_t\right(x_n\left)\right)\\ =\underset{n}{\Σ}p\left(x_n\right)w_T\left(x_n\right)\exp \left({\mathrm{\α}}_T{l}_T\right(x_n\left)\right)\end{array}---\left(eq.1\right)$

Wherein,The weights of Pt, reaction is divided for its initial clustering Divide the clustering result quality of Pt；p(x_n) it is data setPrior probability, be unknown, be commonly defined as p (x_n)=1/N； l_t(x_n) it is that clustering Pt is in data point x_nCost value.Its specific formula for calculation is as follows:

$l_t\left(x_n\right)=1-max\left(h_n^t\right)+min\left(h_n^t\right)---\left(eq.2\right)$

For clustering Pt,WithRepresent data point x respectively_nBe assigned to from it nearest bunch with farthest Bunch Confidence, the computing formula of Confidence such as (eq.5).

It follows that the cost function by its exponential form of deriving further to formula (eq.1) is as follows:

$\begin{array}{c}L=\underset{t}{\&Sigma;}{\mathrm{\&alpha;}}_t\underset{n}{\&Sigma;}p\left(x_n\right)l_t\left(x_n\right)\\ =\underset{n}{\&Sigma;}p\left(x_n\right)\underset{t}{\&Sigma;}{\mathrm{\&alpha;}}_t{l}_t\left(x_n\right)\\ <\exp \left(\underset{n}{\&Sigma;}p\left(x_n\right)\underset{t}{\&Sigma;}{\mathrm{\&alpha;}}_t{l}_t\left(x_n\right)\right)\\ \&le;\underset{n}{\&Sigma;}p\left(x_n\right)\exp \left(\underset{t}{\&Sigma;}{\mathrm{\&alpha;}}_t{l}_t\left(x_n\right)\right)\end{array}---\left(eq.3\right)$

It can be appreciated thatActually react integrated Practising the cluster analysis quality of model, this point is consistent with Boosting algorithm.Therefore to take turns generation one initially poly-each After class divides Pt, in mixing sampling, weight sampling mechanism will be calculated as follows to each data point from one weights of new distribution:

$w_{t+1}\left(x_n\right)=\frac{w_t\left(x_n\right)\exp \left({\mathrm{\&alpha;}}_t{l}_t\right(x_n\left)\right)}{\underset{n}{\&Sigma;}{w}_t\left(x_n\right)\exp \left({\mathrm{\&alpha;}}_t{l}_t\right(x_n\left)\right)}---\left(eq.4\right)$

It addition, the p (x in formula (eq.3)_nThe stochastical sampling mechanism of)=1/N just corresponding Bagging algorithm.So, Mixing sampling mechanism proposed by the invention the most most preferably optimizes the exponential form cost function Le of clustering ensemble study.

Further, described initial clustering generation module based on blended learning sample collection mechanism is to target data setCarry out taking turns sampling more, and use K-means clustering algorithm that sample space is carried out cluster analysis, produce multiple initial poly- Class divides

Further, K-means clustering algorithm is described as follows:

Step one, arbitrarily select k object as initial cluster center from n data object；

Step 2, average (center object) according to each clustering object, calculate each object and these center object Distance；And again corresponding object is carried out clustering according to minimum range；

Step 3, recalculate the average (center object) that each (changing) clusters；

Step 4, circulation step two to step 3 are until each cluster no longer changes.

Further, in clustering ensemble fusion function module based on overall situation and partial situation's clustering architecture information, new fusion function Integrate local and the global information of clustering architecture, by same data setMultiple divisionsIt is converted into a new feature Represent H={ α₁H¹.....α_tH^t, wherein,And α_tRespectively divide p_tCharacter representation and weights；

$h_{i,j}=\exp ({\left(\frac{1}{\left|NB\right|}\underset{\left|NB\left(x_i\right)\right|}{\&Sigma;}d\left(NB\left(x_i\right),{\mathrm{\&mu;}}_j\right)\right)}^2/2{\mathrm{\&sigma;}}_i{\mathrm{\&sigma;}}_j)---\left(eq.5\right)$

In formula (eq.5), x_iFor the mark of sample i, | NB | is self-defining Size of Neighborhood,It is Point in neighborhood puts a μ to a bunch sign_jAverage distance,WithIt is respectively It is positioned at x_iPoint μ is characterized with bunch_jLocal scaling factor.In this feature space, it is possible to use any traditional cluster is calculated Method, such as K-means, generates final clustering ensemble result.

In order to verify that the performance of the present invention generates many sets and has the two-dimentional data set of complicated clustering architecture, and use this respectively The clustering ensemble algorithm of Invention Announce, K-means algorithm, Bagging Integrated Algorithm, Boosting Integrated Algorithm to overlapping two dimension more Data set carries out cluster analysis, and the classification accuracy rate of its performance unified standard is weighed, and computing formula is described as follows:

$CA(L,P)=\left(\underset{i=1}{\overset{K*}{\&Sigma;}}\underset{j=\{1,...,K)}{max}\right\{2\frac{\left|G_i\&cap;C_j\right|}{\left|G_i\right|+\left|C_j\right|}\left\}\right)/K*---\left(eq.6\right)$

Wherein, L={G₁.....G_K*Represent data set true class mark, P={C₁.....C_KRepresent cluster result.

Because the clustering ensemble algorithm of Invention Announce, Bagging Integrated Algorithm, Boosting Integrated Algorithm is based on adopting The learning model of sample, in order to test the fairness compared, three kinds of algorithms are arranged identical sample rate is 10%.According to research and development Mixing sampling mechanism, S=S_R×S_w, set stochastical sampling rate S further_R=50%, weight sampling rate S_R=20%；As Fig. 2- Shown in 4.a, being artificially generated three set two-dimentional data sets, different labellings represents different classes of data point；Wherein, Fig. 2 .a table Show is the artificial data collection with different densities distribution clustering architecture, and it has a data point of three class equal numbers, but each class Data point has different Density Distribution；What Fig. 3 .a represented is the artificial data collection with uneven clustering architecture, and it has four classes The data point of equal densities distribution, but the number of each class data point differs；What Fig. 4 .a represented has special clustering architecture Artificial data collection, it has the data point of two class varying numbers, and respective distribution has special clustering architecture；Two are overlapped three Our tested K-means algorithm on dimension data collection, Bagging Integrated Algorithm, Boosting Integrated Algorithm is poly-with Invention Announce Class Integrated Algorithm；As in Figure 2-4, the present invention is after the method using mixing sampling, in the artificial data to different clustering architectures Carry out (Fig. 2-4.e) in cluster analysis and be better than other algorithms compared, after adding new fusion function, its performance (Fig. 2- 4.f) obtain further lifting.

The above is only to presently preferred embodiments of the present invention, and the present invention not makees any pro forma restriction, Every technical spirit according to the present invention, to any simple modification made for any of the above embodiments, equivalent variations and modification, belongs to In the range of technical solution of the present invention.

Claims (5)

1. one kind based on local and the sampling type clustering ensemble method of global information, it is characterised in that described based on local and The sampling type clustering ensemble method of global information comprises the following steps:

First target data set carried out mixing sampling and generates learning sample, in learning sample space, carrying out cluster analysis also Generate clustering, next clustering is carried out quality evaluation, and updates the weight of target data set according to assessment result Vector, repeats many wheels, and then produces multiple clustering；

Then multiple clusterings are permeated a new character representation, and use clustering algorithm that this character representation is clustered Analyze, and generate integrated cluster result.

2. as claimed in claim 1 based on local and the sampling type clustering ensemble method of global information, it is characterised in that described Multiple divisions of same data set are turned by the clustering ensemble fusion function of clustering algorithm according to local and the global information of clustering architecture Turn to a new character representation, in this feature space, use clustering algorithm to generate final integrated division.

3. as claimed in claim 1 based on local and the sampling type clustering ensemble method of global information, it is characterised in that described Mixing sampling includes stochastical sampling and two kinds of method of samplings of weight sampling, method particularly includes:

Step one, the method for use stochastical sampling are concentrated from target data and are extracted initial learn sample；

Step 2, the method for use weight sampling extract final learning sample from initial learn sample.

4. as claimed in claim 1 based on local and the sampling type clustering ensemble method of global information, it is characterised in that to mesh Mark data setCarrying out taking turns sampling more, and use K-means clustering algorithm that sample space is carried out cluster analysis, generation is many Individual initial clustering dividesK-means clustering algorithm is as follows:

Step one, arbitrarily select k object as initial cluster center from n data object；

Step 2, average according to each clustering object, calculate the distance of each object and center object；And according to minimum range Again corresponding object is carried out clustering；

Step 3, recalculate the average of each cluster；

Step 4, circulation step two to step 3 are until each cluster no longer changes.

5. as claimed in claim 2 based on local and the sampling type clustering ensemble method of global information, it is characterised in that described Clustering ensemble fusion function integrates local and the global information of clustering architecture, by same data setMultiple divisionsTurn Turn to a new character representation H={ α₁H¹.....α_tH^t, wherein,And α_tRespectively divide p_tCharacter representation And weights；

$h_{i,j}=\exp \left({\left(\frac{1}{\left|NB\right|}\underset{\left|NB\left(x_i\right)\right|}{\&Sigma;}d\left(NB\left(x_i\right),{\mathrm{\&mu;}}_j\right)\right)}^2/2{\mathrm{\&sigma;}}_i{\mathrm{\&sigma;}}_j\right)$

x_iFor the mark of sample i, | NB | is self-defining Size of Neighborhood,It is that the point in neighborhood is to a bunch sign point μ_jAverage distance,WithIt is in x respectively_iPoint μ is characterized with bunch_jOffice Portion's scale factor.