JP5410741B2 - Data processing system and data processing program - Google Patents

Description

  The present invention relates to clustering of data sets, and more particularly to a data processing system and data processing program for clustering large-scale data sets.

  The technique for classifying a given data set without external criteria such as the type and number of clusters is called clustering or cluster analysis. As a representative algorithm, k-means (also called c-means) or merged type is used. Hierarchical clustering (Non-Patent Document 1) is well known.

  On the other hand, the normal mixture distribution model (Gaussian Mixture Model) of Non-Patent Document 1 is a method for estimating the probability density function from the data set. However, since the cluster membership probability of each data can be calculated from the probability density function, It can be regarded as one of the clustering methods. As a clustering technique based on probability density estimation, an average moving method (Mean Shift: Non-Patent Document 2) and an extended average moving method (Extended Mean Shift: Non-Patent Document 3) improved from the mean moving method are also known.

In these conventional clustering methods, it is necessary to read the entire data set into the memory before execution. When each piece of data is expressed as a vector having a predetermined number of dimensions (= D), the memory capacity for holding a data set composed of N pieces of data is proportional to D × N. Then, it becomes impossible to hold all data on the memory. For example, the memory capacity required to hold a 500-dimensional / one-million data set with 32-bit double precision is 4 GB, which cannot be handled by a general personal computer. Even if the target is not so large, many of the existing clustering methods mentioned above increase the computation time in proportion to N ² , so the processing cannot be completed in a practical time for large-scale data Fall into.

  As a technique for handling such large-scale data, a technique called data squashing has been developed. In data squashing, large data sets that do not fit in the memory are sequentially read one by one and converted into a data set significantly smaller than the original data set in one pass. Then, various existing algorithms are applied to the converted data. That is, data squashing is a kind of preprocessing technique for large-scale data.

BIRCH (Balanced Iterrative Reducing and Clustering using Hierarchies: Non-Patent Document 4) is known as one of typical techniques for data squashing. In BIRCH, the original data set X = {x1, x2, ..., xN} is converted into a set V = {v1, v2, ..., vM} of partial clusters (that is, a part of the cluster). . Since N >> M, the existing clustering method can be applied to V. Patent Document 1 describes a method in which data squashing and an existing clustering method are combined. This method consists of a large classification process by BIRCH (referred to as online microclustering in Patent Document 1) and a detailed classification process by k-means (referred to as offline macroclustering in Patent Document 1). Yes. Non-Patent Document 5 shows the technical level at the time of filing of the present invention.
JP 2005-100363 A Miyamoto Sadaaki, “Introduction to Cluster Analysis”, Morikita Publishing, 1999 K. Fukunaga, “Statistical Pattern Recognition (second edition)”, Academic Press, 1990 Toru Wakahara et al., “Hierarchical clustering by the extended average moving method”, IEICE technical report PRMU98-38, 1998 T. Zang et al., “BIRCH: A New Data Clustering Algorithm and its Applications”, Journal of DAta Mining and Knowledge Discovery, vol.1, pp.103 -114,1996 MP Wand et al., “Kernel Smoothing”, Chapman & Hall, 1995

  As described above, there is a problem that existing clustering methods are difficult to apply to large-scale data alone. Although it is possible to apply an existing clustering method by using data squashing as preprocessing (major classification processing), in the conventional method as described in Patent Document 1, the size of the cluster shape is small. In the case of non-uniformity, there is a problem that a highly accurate classification result cannot be obtained.

  This is because information regarding the local density and distribution shape of each partial cluster obtained by the large classification process is not used in the detailed classification process. For example, each partial cluster generated by BIRCH is called a CF vector (Cluster Feature vector), which is composed of the number of data belonging to each partial cluster, the sum vector of the data belonging to it, and the sum of squares. Contains information about the distribution. However, when the detailed classification process is performed by the k-average method, each piece of data targeted by the k-average method needs to be a D-dimensional vector represented as one point in the D-dimensional space. The detailed classification process is performed on the center of gravity position (that is, a vector obtained by dividing the sum vector in the CF vector by the number of data belonging to the partial cluster).

This is equivalent to performing the detailed classification process using only the position of each partial cluster, and information on the distribution shape of each partial cluster and the number of data belonging to it is not taken into consideration at all.
Even when other existing clustering methods are used as the detailed classification process, each clustering method is basically a D-dimensional vector (that is, a point in the D-dimensional space), so the existing method is simply classified as it is. Similarly, when used for processing, the local density and distribution shape of partial clusters are not considered.

  An object of the present invention is to solve the above-mentioned problem, when clustering a large-scale data set by a two-stage process of a large classification process and a detailed classification process, the local cluster of each partial cluster obtained as a result of the large classification process An object of the present invention is to provide a data processing system and a data processing program that can appropriately handle attributes related to density in the detailed classification process and thereby obtain a highly accurate clustering result.

In order to solve the above problems, the present invention is a data processing system for clustering an input data set given as a set of vector data of a predetermined number of dimensions, and a large classification for converting an input data set into a set of partial clusters And detailed classification means for clustering the set of partial clusters converted by the large classification means, wherein the detailed classification means considers an attribute relating to the local density of the partial clusters converted by the large classification means. to perform a detailed classification Te is a data processing system according to claim.

  When a data processing system is configured in this way, high-precision clustering results can be obtained when clustering large data sets that cannot be processed by a single method due to restrictions on calculation time and memory capacity. An obtainable data processing system can be provided. In particular, when large-scale data with a nonuniform cluster shape or size is targeted for clustering, the classification accuracy can be improved as compared with the prior art.

Also, the detailed classification means in said data processing system, the center of gravity position coordinates of each portion cluster as a representative point of the respective partial cluster, the local maximum point searching means for searching a maximum point of the probability density function of the representative point as the starting point And a convergence determination unit that determines whether or not the local maximum point search by the local maximum point search unit has converged based on distance information between local average vectors for each partial cluster of the previous value and current value in which the local maximum point search has been performed, Partial cluster classification means for classifying partial cluster groups converged to the same or nearby local maximum points as belonging to the same cluster, and the local maximum point searching means determines the number of samples belonging to each partial cluster for each partial cluster. A maximum point search is performed as a weight .

If the detailed classification means is configured as described above, even if the cluster distribution shape is irregular, it is possible to cope with it and improve the classification accuracy.
Further, in the local maximum point search means, a parameter for defining a local neighborhood range is variable for each partial cluster, and among the partial clusters, the partial parameter having a high local density is set to a small parameter, Control may be performed so that the parameter is increased for partial clusters having a low general density.

Further, the local maximum point search means may calculate a local density individually for each dimension of the input data, and control the parameter individually for each dimension according to the density.
The data processing program of the present invention is a data processing system program for clustering an input data set given as a set of vector data of a predetermined dimension number, and converts the input data set into a set of partial clusters. Functioning as a large classification means, and further functioning as a detailed classification means for clustering the set of partial clusters converted by the large classification means, and when the large classification means functions as the detailed classification means, the large classification means converted The gist is that detailed classification is performed in consideration of an attribute related to the local density of the partial cluster. According to the program of the present invention, it is possible to obtain a highly accurate clustering result when clustering a large-scale data set that cannot be processed by a single method due to restrictions on calculation time, memory capacity, and the like. A data processing program for a data processing system can be provided. In particular, when large-scale data with a nonuniform cluster shape or size is targeted for clustering, it is possible to provide a data processing program capable of improving the classification accuracy as compared with the prior art.

  According to the present invention, in the case of clustering a large-scale data set that cannot be processed by a single method due to restrictions on calculation time, memory capacity, etc., data processing that can obtain a highly accurate clustering result Can provide a system. In particular, when large-scale data with a nonuniform cluster shape or size is targeted for clustering, the classification accuracy can be improved as compared with the prior art.

(First embodiment)
Hereinafter, a first embodiment of the present invention will be described with reference to FIGS. FIG. 1 shows a block diagram of the data processing system 10, and the internal configuration of the data processing system 10 shows functional blocks.

  As shown in FIG. 1, the data processing system 10 includes an input device 11 such as a keyboard and a data processing device 12 that operates according to various programs. A storage device 13 as storage means for storing various data and an output device 14 are connected to the data processing device 12. The output device 14 includes a display and a printer, for example. Note that the storage device 13 may be an internal device of a computer or may be configured as an external storage device.

The data processing device 12 includes a computer 16 having a ROM 12a and a RAM 12b, and performs various processes described later by a data processing program stored in the ROM 12a.
The data processing apparatus 12 includes a major classification processing unit 20 including a nearest neighbor partial cluster search unit 22, an addability determination unit 24, and a partial cluster set update unit 26 as functions, a maximum point search unit 32, a convergence determination unit 34, and a partial A detailed classification processing unit 30 including the cluster classification unit 36 as a function is configured. The data processing device 12 corresponds to a large classification unit and a detailed classification unit.

(Function)
Now, the operation of the data processing system 10 configured as described above will be described.
FIG. 2 is a flowchart of the large classification process performed by the data processing device 12.

  The data processing device 12 is input with a data set to be processed by an input means such as the input device 11 and is stored in the storage device 13 in advance. Shall be. The data set corresponds to an input data set given as a set of vector data having a predetermined number of dimensions.

Figure 2 is a flowchart of outline of processing by the data processing unit 12, in S10, the large classification processing unit 20 is a data set to be processed consisting of N data _{X = {x 1, x 2} , ..., x N } Is converted into a set of partial clusters V = {v ₁ , v ₂ ,..., V _M } to be classified into M partial clusters. Subsequently, in S20, the detailed classification processing unit 30 classifies the set V of partial clusters into C clusters. Here, generally, N >> M and M >> C. However, the true cluster number C is usually unknown. The number of partial clusters M needs to be appropriately determined by controlling the operation of the large classification processing unit, which will be described later.

FIG. 3 is a flowchart of an example of processing of the large classification processing unit 20.
The major classification processing unit 20 performs major classification processing by repeating the procedure shown in FIG. 3 every time one piece of data x _i (1 ≦ i ≦ N) to be processed is read from the data set X.

As shown in the figure, in S11, when the large classification processing unit 20 reads the data x _i to be processed, in S12, the large classification processing unit 20 sets x _i in the current partial cluster set V stored in the storage device 13. Search for the nearest partial cluster v _k .

Then, in S13, the rough classification processing unit 20 calculates the v _k to the diameter of adding the _{x i T (v k ∪x i} ), in S14, a threshold value T ₀ that defines the value previously Compare. When determining that T (v _k ∪x _i ) is smaller than T ₀ (determined as “YES”), the large classification processing unit 20 adds x _i to the partial cluster v _k in S15. When determining that T (v _k ∪x _i ) is equal to or greater than T ₀ (determined as “NO”), the large classification processing unit 20 creates a new partial cluster including only x _i in S16. The large classification processing unit 20 performs the above processing on all the data in X, so that the finally obtained partial cluster set is obtained as the large classification processing result.

  Here, S12 is a process of the nearest neighbor partial cluster search unit 22, and the nearest neighbor partial cluster search unit 22 corresponds to the nearest neighbor partial cluster search means. S14 is a process of the addability determination unit 24, and the addability determination unit 24 corresponds to an addability determination unit. S15 and S16 are processes of the partial cluster set update unit 26, and the partial cluster set update unit 26 corresponds to a partial cluster set update unit. The storage device 13 corresponds to storage means for storing a set of partial clusters.

The partial cluster v is described by three elements of n, LS, and SS as follows.
v = (n, LS, SS)
n: Number of data composing v
LS: sum vector (linear sum) of n data composing v
SS: square sum of n data composing v
The diameter T (v) of the partial cluster v is an average value of the Euclidean distance between all data constituting v, and is obtained from (n, LS, SS) by the following equation (1).

Here, the derivation of Expression (1) will be described.

The elements LS (linear sum) and SS (square sum) of the CF vector of the partial cluster v composed of n individuals {x _i } (1 ≦ i ≦ n) are expressed as follows (d: number of dimensions).

Further, the diameter T (v) of the partial cluster v is defined as an average value of the Euclidean distance between all individuals belonging to v.
Therefore,

Therefore, T (v) is calculated only from n, LS, and SS.

  Returning to the original, the distance d (v, x) between the partial cluster v and the data x is an average value of the Euclidean distance between all the data constituting v and x, and is obtained by the following equation (4). . The partial cluster closest to the read data is determined based on this distance measure.

The above equation (4) is derived as follows. That is, the calculation method of the distance d between partial clusters from the CF vector is as follows. The distance d (v _a , v _b ) between the partial clusters v _a and v _b is defined by the following equation (5). (n _a and n _b are the number of individuals in v _a and v _b )

From the definition of T (v)

Since the CF vector of the sum cluster (v _a ∪v _b ) can be calculated from the CF vector of v _a , v _b , d (v _a , v _b ) can be obtained only from the CF vector of v _a , v _{b after all} It is done.

  The distance d (v, x) between a single data x and a partial cluster v is calculated from the following equation (7) derived from the above equation (6) by regarding x as a partial cluster consisting of only one piece of data. it can.

Here, an example of a processing result by the large classification processing unit 20 will be described below. 4A to 4C are plots of three types of data sets to be processed. In the figure, all three types (a) to (c) have dimension number D = 2, number of data N = 300000, number of clusters C = 3, and the clusters to which each data belongs are expressed in shades of color. . In the figure, N ₁ , N ₂ , and N ₃ are the numbers of data belonging to the clusters c ₁ , c ₂ , and c ₃ , respectively.

FIGS. 5A to 5C are examples of large classification processing results for the data sets shown in FIGS. 4A to 4C, respectively. Each circle represents a partial cluster, and the size and color density of the circle indicate the diameter T (v) of the partial cluster and the number of data belonging to the partial cluster, respectively. Although the number M of partial clusters varies depending on the threshold value T ₀ , the value of T ₀ is controlled so that M is approximately 300 in (a) to (c).

Next, the detailed classification processing unit 30 shown in FIG. 1 will be described below.
The detailed classification processing unit 30 in the present embodiment is based on the average shift method (Mean Shift) shown in Non-Patent Documents 2 and 3. In the known average moving method, the local average vector defined for each data point is sequentially moved to search in parallel for the local maximum points of the density distribution and determine that the data converged to the same local maximum point belongs to the same cluster. To do. FIG. 10 shows a conceptual diagram of maximum point search in the average moving method. As shown in the figure, individuals that converge to the same maximum point form a cluster.

In the case of FIG. 10, each data point (for example, x _i , x _j , x _k ) is set as the initial value of the local average vector, and the state of searching using this as the starting point is shown.
When the known average moving method is directly applied to the set of partial clusters V = {v ₁ , v ₂ ,..., V _M }, the center-of-gravity position coordinates g (v _k ) = LS _k / n of each partial cluster. _k (where LS _k and n _k are the sum vectors of the partial clusters v _{k and} the number of data), respectively, is regarded as the processing target data, and each partial cluster is processed according to the following equation (8). The local average vector m _k (l) for is sequentially moved.

Here, l (= 1, 2,...) Is the number of iterations of the moving operation, and ω (x, x '; h) = exp (− || x−x ′ || ² / h ² ) is the local neighborhood. This is a kernel function to be defined, and h is a separately defined bandwidth parameter. As shown in Non-Patent Document 3, the moving direction by the operation of Equation (8) is equal to the steepest gradient direction ∇p (x) of the probability density function p (x), and the magnitude is far from the maximum point. The smaller ((= p (x)) is smaller). The probability density function p (x) here is a probability density function by a kernel density estimation method using ω (x, x ′; h) as a kernel function, and is expressed by the following equation (9). The processing of the detailed classification processing unit 30 is performed by the local maximum point searching unit 32.

Next, the convergence determination unit 34 of the detailed classification processing unit 30 performs convergence determination according to the following equation (10) every time the moving operation for all partial clusters is completed. Expression (10) is obtained when the sum of Euclidean square distances between local average vectors m _k (l) and m _k (l−1) is obtained in all partial clusters, and the sum is less than the threshold Th. Means that it has converged to. In the present embodiment, the Euclidean square distance between the local average vectors mk (l) and mk (l-1) is the local average vector for each partial cluster of the previous value and the current value in the iteration operation of the local average vector. It corresponds to the distance information. In addition, you may use another distance scale, for example, a Euclidean distance and a city block distance, instead of the Euclidean square distance as the distance information.

The convergence determination unit 34 of the detailed classification processing unit 30 ends this process when it is determined that the convergence is achieved by the equation (10) (that is, when the above equation (10) is established). Then, the partial cluster classification unit 36 sets the data group converged to the same local maximum point (or extremely close coordinate points, that is, local maximum points) to belong to the same cluster, that is, one cluster.

As will be described later, when a known average moving method is applied as it is, a desired classification result may not be obtained.
Therefore, in the present embodiment, a highly accurate detailed classification result is obtained by appropriately handling information on the local distribution of each partial cluster obtained by the large classification process. Specifically, the local average vector m _k (l) for each partial cluster is sequentially moved by the procedure of the following equation (11).

The first expression of the above expression (11) represents that the center-of-gravity position coordinates g (v _k ) of each partial cluster is used as the representative point of each partial cluster, and this is used as the initial value of the local average vector, that is, the starting point. Yes.

Note that n _j is the number of samples belonging to the partial cluster v _j , h _j is a bandwidth parameter determined for each partial cluster according to the local density, and h _j ² corresponding to the partial cluster v _j is T (v _j ) ² proportional to, and the average of h _j ² weighted by the number of samples n _j is defined by the following equation (12) to match the h ^2.

In addition, derivation | leading-out of Formula (12) is as follows.

The radius of the partial cluster v _j represents T (v _j) ^2, a bandwidth parameter and h _j.
h _j ² is determined as follows so that h _j ² is proportional to T (v _j ) ² and the average of h _j ² weighted by n _j matches h ² . That is,

Find a constant α that satisfies. Where n _j is the number of samples belonging to v _j , N is the total number of samples,

It is. From the above formula (13) and formula (14)

And

Is obtained. Therefore, Expression (17) (= Expression (12))

It becomes.

  Returning the story to making the bandwidth parameter variable as shown in equation (12), when focusing on a certain local average vector, the radius is large in equation (11) (that is, the local density is small). ) The value of the parameter of the kernel function with the partial cluster is reduced, and the value of the parameter of the kernel function with the partial cluster having a small radius (that is, local density is large) is increased. As a result, in the local maximum search, the local average vector moves toward a partial cluster having a higher density. Note that the local maximum point search unit 32 performs the process of changing the bandwidth parameter for each partial cluster as described above.

In equation (11), the value of the kernel function is multiplied by the number of samples n _j belonging to each partial cluster and added. This corresponds to performing a local maximum search using the number of samples n _j as the weight of each partial cluster.

  The reference value of the bandwidth parameter h needs to be appropriately determined in consideration of the distribution of the entire data, and the number of clusters obtained as a result of the detailed classification changes depending on the value of the reference value. Non-Patent Document 3 shows a method of gradually increasing the average value of the distances between nearest neighbor points of all samples as an initial value of a reference value. Bandwidth selection in the average shift method is essentially the same problem as the kernel density estimation method, and h may be determined by a plug-in rule shown in pages 71 to 72 of Non-Patent Document 5.

  The result of detailed classification performed by the detailed classification processing unit of the present embodiment on the partial cluster set of FIGS. 5A to 5C obtained by the large classification process as described above is shown in FIG. ) To (c). Here, the result when the value of h is changed and the number of clusters matches the true number of clusters 3 is shown, and the partial clusters classified into the same cluster are illustrated with the same brightness.

  FIGS. 7A to 7C show the results of performing detailed classification by the known average moving method on the large classification processing results of FIGS. 5A to 5C. In FIG. 6, the desired classification results are obtained in all of (a) to (c), whereas in FIG. 7, the shape and the number of samples of each cluster are not uniform as in (b) and (c). The correct classification result is not obtained. This indicates that, in the present embodiment, by performing detailed classification in consideration of the local density of each partial cluster, a classification result with higher accuracy than in the conventional technique can be obtained.

The data processing system 10 configured as described above has the following characteristics.
(1) The data processing device 12 of the data processing system according to the present embodiment converts a data set (input data set) as a large classifying unit into a set of partial clusters, and the detailed classifying unit converts the data set as a detailed classifying unit. When clustering a set of partial clusters, detailed classification is performed considering attributes related to the local density of the partial clusters.

  As a result, we provide a data processing system that can obtain highly accurate clustering results when clustering large-scale data sets that cannot be processed by a single method due to constraints on computation time and memory capacity. it can. In particular, when large-scale data with a nonuniform cluster shape or size is targeted for clustering, the classification accuracy can be improved as compared with the prior art.

  (2) The data processing device 12 of the present embodiment is a storage device 13 (storage unit) that stores the partial cluster set after the input data set is converted into a set of partial clusters as the large classification unit, and the storage A nearest neighbor cluster search unit 22 (nearest neighbor cluster search means) that searches for a nearest neighbor cluster that is the nearest cluster to the processing target data from the set of partial clusters stored in the device 13; , And an addition availability determination unit 24 (addition availability determination means) for determining whether or not the processing target data should be added to the nearest neighbor cluster. Further, the data processing device 12 performs a process of either adding the process target data to the nearest partial cluster or generating a new partial cluster for the process target data based on the determination result. A cluster set update unit 26 (partial cluster set update means) is provided. Then, each time one piece of data is read from the data set (input data set), the addability determination unit 24 performs the nearest neighbor partial cluster search and the addability determination, and the partial cluster set update unit 26 moves to the nearest neighbor cluster. Any one of the process for adding the data to be processed and the process for generating a new partial cluster is performed on the data to be processed. As a result, when the large classification means is configured in this way, it is not necessary to hold the entire input data set on the memory, and thus it is possible to handle large-scale data that cannot be held on the memory.

  (3) Further, the data processing apparatus 12 of the present embodiment uses, as the detailed classification means, the center-of-gravity position coordinates of each partial cluster as a representative point of each partial cluster, and uses the representative point as a starting point to determine the maximum point of the probability density function. The local maximum vector for each partial cluster of the previous value and the current value in which the local maximum search is performed as to whether the local maximum search by the local maximum point searching unit 32 (local maximum point searching means) to be searched and whether the local maximum point search by the local maximum point searching unit 32 has converged. And a partial cluster classification unit 36 (partial cluster classification unit) that classifies a partial cluster group that has converged to the same or nearby maximum point as belonging to the same cluster. ). The maximum point search unit 32 performs a maximum point search using the number of samples belonging to each partial cluster as the weight of each partial cluster. As a result, even when the cluster distribution shape is irregular, it can be dealt with, and the classification accuracy can be improved.

  (4) In the data processing apparatus 12 of the present embodiment, in the local maximum point search unit 32 (local maximum point searching means), a parameter (bandwidth parameter) that defines a local neighborhood range is variable for each partial cluster. Among the partial clusters, control is performed such that the parameter is reduced for a partial cluster having a large local density, and the parameter is increased for a partial cluster having a low local density. As a result, the effects (1), (2), and (3) can be easily realized.

(Second Embodiment)
Next, a second embodiment will be described with reference to FIG. In addition, about the same structure as 1st Embodiment, the same code | symbol is attached | subjected and a different structure is demonstrated.

In the second embodiment, the processing of the large classification processing unit 20 in the first embodiment is replaced with another method.
Specifically, in the method of the second embodiment, instead of describing each partial cluster with three elements (n, LS, SS), the partial cluster is described with two elements of the barycentric position coordinate and the number of samples. This method is a simplified version that uses only the center-of-gravity position coordinates without using the amount corresponding to the diameter T of the partial cluster when obtaining the distance between the data x and the partial cluster. Diameter T is calculated at the end of major classification.

FIG. 8 is a flowchart processed by the large classification processing unit 20. As shown in the figure, the major classification processing unit 20 repeats the procedure shown in FIG. 8 every time one piece of data x _i (1 ≦ i ≦ N) is read from the data set X. When the data x _i is read in S 11, first, in 12, the partial cluster v _k closest to x _i in the current partial cluster set V is searched. Then, in S13A, v _k and the distance d (v _k, x _i) of x _i is calculated. In subsequent S14A, v _k and x _i as the distance d (v _k, x _i) of the comparison with the threshold value d ₀ which predetermined a.

In _{S14A, d (v k, x} i) is the case d ₀ less, the process proceeds to S15A, add the x _i to the partial cluster v _k, updating the v _k of the center of gravity g (v _k). If d (v _k , x _i ) is greater than or equal to d ₀ in S14A, a new partial cluster consisting only of x _i is created in S16. By performing the above processing on all the data in X, the finally obtained partial cluster set is obtained as a result of the large classification processing.

In this method, the partial cluster v is described by two elements n and g (v) as follows.
v = (n, g (v))
n: Number of data composing v
g (v): Center-of-gravity position coordinates of n data constituting v Partial cluster v-centroid g (v) is an average vector of all data constituting v and is obtained by the following equation (18).

The distance d (v, x) between the partial cluster v and the data x is the Euclidean distance between g (v) and x, and is obtained by the following equation (19). The partial cluster closest to the read data is determined based on this distance measure.

When the assignment to the partial clusters is completed for all data, the diameter T is calculated for each partial cluster. The diameter T (v) of the partial cluster v is an average value of the Euclidean distance between all the data belonging to v, and is calculated directly from the coordinate value of each data by the following equation (20).

In the second embodiment, S12 is processing of the nearest neighbor cluster search unit 22, and the nearest neighbor cluster search unit 22 corresponds to nearest neighbor cluster search means. S14A is the process of the addability determination unit 24, and the addability determination unit 24 corresponds to an addability determination unit. S15A and S16 are processes of the partial cluster set update unit 26, and the partial cluster set update unit 26 corresponds to a partial cluster set update unit. The storage device 13 corresponds to storage means for storing the partial cluster set.
(Third embodiment)
Next, a third embodiment will be described with reference to FIG. In the present embodiment, only the number of samples of each partial cluster is considered. That is, in the first embodiment, the maximum point search is performed by considering both the number of samples and the radius of each partial cluster by the update formula of Formula (11), but it may be simplified to consider only the number of samples. Is possible. In this case, the update formula in the moving operation of the local average vector is expressed by the following formula (21).

The result of performing the detailed classification process using this method is shown in FIG. Although slightly inferior to the result of FIG. 6, it is significantly improved compared to the conventional method.
(Fourth embodiment)
Next, a fourth embodiment corresponding to claim 4 will be described. The present embodiment is characterized in that the local maximum point search unit 32 of the detailed classification processing unit 30 controls the bandwidth parameter h independently for each dimension of the input data.

  As will be described in detail below, in the present embodiment, the bandwidth parameter h is expanded anisotropically by making the SS (Square Sum) of the CF vector a vector whose element is the sum of squares for each dimension. (Bandwidth matrix H with zero off-diagonal elements).

Here, the elements LS (linear sum) and SS (square sum) of the CF vector of the partial cluster v consisting of n individuals {x _i } (1 ≦ i ≦ n) are expressed as follows (d: dimension number).

In the above, SS is changed to d-dimensional vector SS as follows.

Since SS (scalar amount) is obtained as the sum of the elements of vector SS, it is an extension of the conventional CF vector. That is, the diameter T when the partial cluster is assumed to be hyperspherical, the distance between the two partial clusters, and the sum of the two partial clusters can be calculated as in the conventional case.

Here, a partial cluster represented by a CF vector v obtained by extending SS into a vector is considered to be a super ellipsoid, and a diameter T _k (v) of the partial cluster v is considered for each dimension k (1 ≦ k ≦ d).
The diameter T _k (v) can be calculated as follows, as in the case where SS is a scalar.

(Calculation of bandwidth matrix H)
Here, the bandwidth parameter corresponding to the CF vector v _j is a bandwidth matrix H _j with zero off-diagonal elements, and the (k, k) element of H _j is written as h _jk ² .

Then, h _jk is determined as follows according to the local density in the kth dimension (1 ≦ k ≦ d). That is, h _jk ² is proportional to T _k (v _j ) ² and the average of h _jk ² considering the number of samples belonging to v _j matches the square of fixed bandwidth h _k ² in the _k- th dimension. It is defined by the following equation (26).

(Mean shift clustering using bandwidth matrix H)
Then, the local average vector m _i with respect to the center of gravity CF (v _i ) of the CF vector is sequentially moved by the procedure of the following equation (27) to search for a local maximum point.

Where ω _H (x, x '; H) is an anisotropic kernel function that defines the local neighborhood,

It is expressed.

In addition, you may change embodiment of this invention as follows.
In the first embodiment, the partial cluster obtained by the processing of the large classification processing unit 20 is described by three elements (n, LS, SS), but this configuration is not necessarily required, and the centroid of the partial cluster What is necessary is just to be comprised so that the quantity equivalent to a position coordinate and dispersion | distribution may be obtained. In the extreme case, the center of gravity and variance can be calculated as long as the correspondence between the coordinate points of all data and the partial clusters is given. However, in order to efficiently perform large classification on a large-scale data set, the above-described configuration is desirable. This is because the distance d (v, x) between the partial cluster v and the data x can be calculated, the diameter T (v) of the partial cluster can be calculated, and the parameters can be updated efficiently when the data x is added to the partial cluster v. It is.

In the configuration of the first embodiment, the method for determining the bandwidth parameter h _j for each partial cluster in the detailed classification processing unit 30 is not limited to the method of the above embodiment (= expression (12)). For example, the radius of the partial cluster is large (i.e. less local density) sometimes h _j is large, a small radius (i.e., greater local density) Sometimes it is also possible to use any method, such as h _j becomes smaller. Further, in the detailed classification process, the kernel function that defines the local neighborhood is not limited to the Gaussian function, and an arbitrary kernel function may be used.

  In the first embodiment, the data processing device 12 composed of one computer constitutes the large classification means and the detailed classification means, but the two computers may constitute the large classification means and the detailed classification means, respectively. Further, the nearest neighbor cluster search unit 22, the addability determination unit 24, the partial cluster set update unit 26, the local maximum point search unit 32 of the detailed classification processing unit 30, the convergence determination unit 34, and the partial cluster classification unit 36 are each configured by a computer. May be.

In the present specification, technical ideas that can be grasped other than the claims are listed below .

( 1 ) In claim 4 ,
When making a computer function as the local maximum search means,
The parameter that defines the local neighborhood range is variable for each partial cluster, and among the partial clusters, the parameter is small for a partial cluster with a high local density, and for the partial cluster with a low local density. A program for a data processing system, characterized in that control is performed to increase the parameter.

( 2 ) In the above ( 1 ),
When making a computer function as the local maximum search means,
A program for a data processing system, wherein a local density is calculated individually for each dimension of input data, and the parameters are individually controlled for each dimension according to the density.

1 is a block diagram constituting a data processing system 10. FIG. 5 is a flowchart of an outline of processing by the data processing device 12; 6 is a flowchart of an example of processing of a large classification processing unit 20. (A)-(c) is the figure which plotted three types of data sets used as a process target. (A)-(c) is explanatory drawing of the example of the large classification | category process result with respect to the data set shown to Fig.4 (a)-(c). (A) to (c) are the results of the detailed classification performed by the detailed classification processing unit of the first embodiment on the partial cluster set of FIGS. 5A to 5C obtained by the large classification processing. Illustration. (A)-(c) is explanatory drawing of the result of having performed the detailed classification | category by the well-known average moving method with respect to the large classification | category processing result of Fig.5 (a)-(c). Flowchart processed by the classification processing unit 20 of the second embodiment (A)-(c) is explanatory drawing of the result of having performed the detailed classification | category process in 3rd Embodiment. The conceptual diagram of the local maximum search in an average moving method.

Explanation of symbols

10: Data processing system,
11 ... Input device,
12 ... Data processing device (major classification means, detailed classification means),
12a ... ROM,
12b ... RAM,
13: Storage device (storage means),
14 ... output device,
20 ... major classification processing part,
22 ... search part nearest neighbor partial cluster search part,
24 ... Additional availability determination unit,
26: Partial cluster set update unit,
30 ... Detailed classification processing unit,
32 ... Maximum point search part,
34 ... Convergence determining unit,
36... Partial cluster classification unit.

Claims (4)

A data processing system for clustering an input data set given as a set of vector data of a predetermined number of dimensions,
A large classification means for converting an input data set into a set of partial clusters;
Detailed classification means for clustering the set of partial clusters converted by the large classification means,
The detailed classification means,
The center-of-gravity position coordinates of each partial cluster is used as the representative point of each partial cluster, and the local maximum point search means for searching for the local maximum point of the probability density function starting from this representative point, and the local maximum point search by the local maximum point searching means converge Convergence determination means that determines whether or not the local maximum vector for each partial cluster of the previous value and current value for which the local maximum point search has been performed is the same as the partial cluster group that has converged to the same or nearby local maximum point A partial cluster classification means for classifying as belonging to a cluster,
The data processing system, wherein the local maximum point searching means performs local maximum point search using the number of samples belonging to each partial cluster as the weight of each partial cluster .   In the local maximum point search means, a parameter for determining a local neighborhood range is variable for each partial cluster, and among the partial clusters, the partial parameter having a high local density is set to a small parameter, The data processing system according to claim 1, wherein the parameter is controlled to be increased for a partial cluster having a low density. The maximum point searching means calculates a local density individually for each dimension of the input data, according to claim 2, wherein the controller controls the parameters individually for each dimension according to the density Data processing system. A data processing system program for clustering an input data set given as a set of vector data of a predetermined number of dimensions,
Computer
Function as a large classification means to convert the input data set to a set of partial clusters,
Further, it functions as detailed classification means for clustering the set of partial clusters converted by the large classification means,
And when functioning as the detailed classification means,
The centroid position coordinates of each partial cluster as a representative point of each partial cluster, the local maximum point search means for searching for the local maximum point of the probability density function starting from this representative point,
Convergence determination means for determining whether or not the local maximum search has converged based on distance information between local average vectors for each partial cluster of the previous value and current value in which the local maximum search has been performed,
Function as a partial cluster classification means for classifying a partial cluster group converged to the same or nearby maximum point as belonging to the same cluster,
A data processing program that, when functioning as the maximum point search means, performs a maximum point search using the number of samples belonging to each partial cluster as a weight of each partial cluster .