CN104598565B - A kind of K mean value large-scale data clustering methods based on stochastic gradient descent algorithm - Google Patents

Abstract

The present invention provides a K-means large-scale data clustering method based on a stochastic gradient descent algorithm, comprising the following steps: randomly initializing K cluster centers; sampling data samples, and dividing the data samples into their types; Iteration; repeat steps 1-3 to make the cluster center converge. The K-means large-scale data clustering method based on the stochastic gradient descent algorithm provided by the present invention greatly improves the execution efficiency of the algorithm and achieves a better clustering effect. It can mine data more quickly and effectively, and the proposal of this method provides a possibility for dealing with power big data and other data problems.

Description

A K-means Large-Scale Data Clustering Method Based on Stochastic Gradient Descent Algorithm

technical field

The invention relates to a clustering method, in particular to a K-means large-scale data clustering method based on a stochastic gradient descent algorithm.

Background technique

In recent years, with the improvement of data collection methods and capabilities, the amount of data that individuals, especially enterprises, can obtain has increased dramatically. For example, after the completion of the SG186 project of the State Grid Corporation of China, the average daily increase of data records for the eight major business applications reached more than 50 million (144G); and with the construction of smart grid and SG-ERP, the company's data growth rate will double again Fan. Ultra-large-scale composite information storage, backup and disaster recovery will all become important technical fields, and the construction effect of the data center and disaster recovery center will directly affect the continuity of the overall business of the enterprise. How to make full use of historical data, real-time data, forecast data, and data of different geographical spaces and levels through powerful algorithms to complete the value "purification" of data more quickly is an urgent need for electric power big data solved puzzles.

Enterprise data comes from a wide variety of sources and is growing in scale. In a sense, the proportion of valuable information to companies is decreasing, and how to find useful information from massive amounts of information is becoming more and more difficult. Effectively and fully organize and analyze data, reduce or compress worthless data, increase the utilization value of effective data, reduce the scale of data storage, reduce the computing resources occupied by data analysis, and directly guide the optimization of enterprise information assets.

With the rapid development of computer technology and storage devices, people can easily obtain tens of thousands or even millions of data. How to analyze useful or interesting information from these data has become an urgent problem to be solved. The traditional K-means clustering algorithm is a relatively common method used in the field of data mining. First, K cluster centers are randomly initialized, and then all samples are divided into K different types according to the distance from each sample to the cluster center. Finally, The average value of all samples in each class is used to update the cluster centers, and the whole process is iterated until convergence. Obviously, it is necessary to calculate the distances from all samples to K cluster centers in each iteration. When faced with large-scale data, the calculation process takes a lot of time, which greatly reduces the execution efficiency of the algorithm.

At present, the processing flow of big data can generally be summarized into four steps: data collection, import and preprocessing, statistics and analysis, mining and decision support. Among them, mining and decision support are mainly based on calculations based on various algorithms on existing data, so as to achieve the effect of prediction and decision support, so as to meet the needs of some high-level data analysis, which is typically used for clustering The K-means clustering algorithm. However, the biggest problem that the traditional data mining technology faces is the poor real-time performance, and it takes a lot of time to process the data. For data that changes in real time, it is difficult to obtain useful information in a timely manner, thereby affecting corporate decision-making.

Contents of the invention

In order to overcome the shortcomings of the prior art above, the present invention provides a K-means large-scale data clustering method based on the stochastic gradient descent algorithm, which greatly improves the execution efficiency of the algorithm and achieves better clustering effects. It can mine data more quickly and effectively, and the proposal of this method provides a possibility for dealing with power big data and other data problems.

In order to realize the above-mentioned purpose of the invention, the present invention takes the following technical solutions:

The invention provides a K-means large-scale data clustering method based on stochastic gradient descent algorithm, said method comprising the following steps:

Step 1: Randomly initialize K cluster centers;

Step 2: Sampling data samples and classifying the data samples into their types;

Step 3: Iterate over the objective function;

Step 4: Repeat steps 1-3 until the cluster centers converge.

In the step 1, for the K-type data sets to be processed, K cluster centers w ₁ , w ₂ , ..., w _k , ..., w _K ∈ R ^d are randomly initialized, where R represents a real number and d represents a dimension , so R ^d represents a d-dimensional real number, and w _k represents the cluster center corresponding to the k-th type of data set.

In the step 1, the number n ₁ , n ₂ ,...,n _k ,...,n _K ∈ N of data samples in each cluster center is initialized to 0, where N represents an integer, and n _k represents the kth class The number of data samples corresponding to the data set.

In the step 2, the data sample z∈R ^d is randomly sampled, and the data sample z is classified into the type according to the cluster center corresponding to the minimum distance.

The code of the data set in the cluster center corresponding to the minimum distance is represented by k ^* , which is:

Among them, (zw _k ) ² represents the distance from data sample z to w _k .

Described step 3 specifically comprises the following steps:

Step 3-1: Let the objective function be Q _kmeans , there are:

Q _kmeans about The derivative of Indicates that there are:

in, is the cluster center corresponding to the k ^* th class data set;

Step 3-2: Set Indicates the number of data samples corresponding to the ^kth class data set, using Q _kmeans and Update separately and

In the step 4, repeat steps 1-3, if the cluster center distance threshold of the two previous iterations is less than 10 ^-6 , it indicates that the cluster centers w ₁ ,w ₂ ,...,w _k ,...,w _K convergence.

Compared with prior art, the beneficial effect of the present invention is:

The K-means large-scale data clustering method based on the stochastic gradient descent algorithm provided by the present invention greatly reduces the computational complexity of the algorithm, can achieve convergence more quickly, and can also obtain better clustering effects. Since the samples are randomly selected in each iteration without considering the situation of the previous samples, the stochastic gradient descent algorithm is essentially a process of minimizing the expected risk. The proposed method provides a possibility for dealing with power big data and other data problems.

Description of drawings

Fig. 1 is the schematic diagram of stochastic gradient descent algorithm in the embodiment of the present invention;

Fig. 2 is the distribution figure of raw data in the embodiment of the present invention;

Fig. 3 is the clustering result diagram of the K-means clustering method in the prior art;

Fig. 4 is a graph of the K-means clustering results based on the stochastic gradient descent algorithm in the embodiment of the present invention.

Detailed ways

The present invention will be described in further detail below in conjunction with the accompanying drawings.

Example

First, two "moon"-shaped sample families are randomly generated, represented by triangles and dots, as shown in Figure 2. The data consists of two-dimensional features. Each type of data contains 200,000 samples, and there are a total of 400,000 data. It belongs to the problem of big data processing. For the convenience of display, some data are selected for graphing. The computer configuration of the experiment done in this embodiment is: 64-bit operating system, 16GB memory, Intel processor, and the software operating environment is MATLAB R2012a version. The specific process is as follows:

a) Randomly initialize two cluster centers w ₁ , w ₂ ∈ R ² , and the number n ₁ , n ₂ ∈ N of each type of samples is initialized to 0;

b) Randomly sample a data sample z∈R ² , according to the formula Divide it into the corresponding type;

d) update and :

e) Steps b) to d) are repeated until the cluster centers w ₁ and w ₂ converge.

Figure 3 is the result graph obtained by the classic K-means clustering algorithm after 3 iterations, which takes a total of 32 seconds, and Figure 4 is the result obtained by the K-means clustering algorithm based on the gradient descent algorithm when it takes 17 seconds , after 500 iterations, the "x"-shaped circles represent two cluster centers. It can be seen from the figure that the cluster centers of the two figures are almost the same. Among the quantitative results, the classic K-means clustering takes 32 seconds, while the k-means clustering based on the stochastic gradient descent algorithm only takes 17 seconds, and the accuracy rate reaches 78.41%, which is slightly higher than that of the classic K-means clustering. 78.1%.

Finally, it should be noted that: the above embodiments are only used to illustrate the technical solutions of the present invention and not to limit them. Those of ordinary skill in the art can still modify or equivalently replace the specific implementation methods of the present invention with reference to the above embodiments. Any modifications or equivalent replacements departing from the spirit and scope of the present invention are within the protection scope of the claims of the pending application of the present invention.

Claims (1)

1. a K mean value large-scale data clustering method based on stochastic gradient descent algorithm, it is characterized in that: described method comprises the following steps: Step 1: Randomly initialize K cluster centers; Step 2: Sampling data samples and classifying the data samples into their types; Step 3: Iterate over the objective function; Step 4: Repeat steps 1-3 until the cluster centers converge; In the step 1, for the K-type data sets to be processed, K cluster centers w ₁ , w ₂ , ..., w _k , ..., w _K ∈ R ^d are randomly initialized, where R represents a real number and d represents a dimension , so R ^d represents a d-dimensional real number, and w _k represents the cluster center corresponding to the k-th type of data set; In the step 1, the number n ₁ , n ₂ ,...,n _k ,...,n _K ∈ N of data samples in each cluster center is initialized to 0, where N represents an integer, and n _k represents the kth class The number of data samples corresponding to the data set; In the step 2, the data sample z∈R ^d is randomly sampled, and the data sample z is divided into the type according to the cluster center corresponding to the minimum distance; The code of the data set in the cluster center corresponding to the minimum distance is represented by k ^* , which is: Among them, (zw _k ) ² represents the distance from data sample z to w _k ; Described step 3 specifically comprises the following steps: Step 3-1: Let the objective function be Q _kmeans , there are: Q _kmeans about The derivative of Indicates that there are: in, is the cluster center corresponding to the k ^* th class data set; Step 3-2: Set Indicates the number of data samples corresponding to the ^kth class data set, using and Update separately and In the step 4, repeat steps 1-3, if the cluster center distance threshold of the two previous iterations is less than 10 ^-6 , it indicates that the cluster centers w ₁ ,w ₂ ,...,w _k ,...,w _K convergence.