JP5855594B2 - Clustering apparatus, clustering processing method and program thereof - Google Patents

Description

  The present invention relates to a clustering device, a clustering processing method, and a program thereof.

  As a conventional clustering technique for graph data, a clustering device (computer) selects nodes for clustering processing in any order using all edge data included in the input graph data, and generates an intermediate result of clustering. ,Summarize. Furthermore, there is a technique in which the clustering apparatus repeatedly performs clustering processing on an aggregated cluster to make the cluster size uniform and reduce the number of nodes to be processed (for example, see Non-Patent Document 1).

  As illustrated in FIG. 1, the clustering apparatus 210 includes an input unit 211, a control unit 212, and an output unit 213. The input unit 211 develops the graph data read from the graph data storage device 200 on the main storage device (see FIG. 4) included in the clustering device 210, and the control unit 212 performs the processing of the flow shown in FIG. The data is passed to the output unit 213. The output unit 213 outputs the clustering result obtained by the control unit 212 to an arbitrary device such as the clustering result storage device 220.

  The processing in the control unit 212 will be described below.

  First, when graph data as shown in FIG. 3A is input, the control unit 212 selects one arbitrary node from the graph data using a random number, and a list of nodes adjacent to the selected node. Is inserted into the adjacent node queue on the main memory (S100).

  Next, the control unit 212 selects an adjacent node using a random number from the adjacent node queue, and deletes the node from the adjacent node queue (S110).

  The control unit 212 calculates a clustering accuracy improvement amount when the two nodes, the node selected in S100 and the adjacent node selected in S110, are classified into the same cluster, and a set including the adjacent node and the clustering accuracy improvement amount is calculated. ({Adjacent node: clustering accuracy improvement amount}) is generated and inserted into the processed queue on the main storage device (S120). As shown in Non-Patent Document 1, the clustering accuracy improvement amount is obtained by the following equation.

  [Definition 1] The clustering accuracy improvement amount ΔQ is calculated as follows.

Where ΔQ is the clustering accuracy improvement amount, e _ij is the number of edges existing between cluster i and cluster j, m is the total number of edges existing in the graph, and C is a set of clusters.

  If there is an adjacent node in the adjacent node queue, that is, if there is an unprocessed node in the adjacent node of the node (No in S130), the process returns to S110. On the other hand, when there is no adjacent node in the adjacent node queue, that is, when the control unit 212 has processed all the adjacent nodes of the node (Yes in S130), the adjacent storage having the maximum clustering accuracy improvement amount is selected as the main storage device. Select from the above processed queue, assign the same cluster label to the node and the adjacent node, and store in the cluster correspondence table on the main storage device. That is, the control unit 212 performs labeling so that the node is classified into the same cluster as the adjacent node with the highest clustering accuracy improvement amount (S140). An arbitrary cluster label is assigned to the cluster label name. In the cluster correspondence table on the main storage device, in the case of the adjacent node b of the node a, a set of {cluster label: node a, node b} is stored in the form of a hash map.

  Next, the control unit 212 determines whether there is room for improving clustering accuracy between pairs of nodes (S150). That is, the control unit 212 first displays the graph data developed on the main storage device. And the cluster correspondence table on the main storage device is read, and the clustering accuracy of the entire graph data is calculated by the pair of nodes. As shown in Non-Patent Document 1, the clustering accuracy of the entire graph data is obtained by the following equation.

  [Definition 2] The clustering accuracy Q is calculated as follows.

  And the control part 212 calculates | requires the difference with the clustering precision at the time of performing the said process last time. In the first execution, it is determined that the previous clustering accuracy is regarded as 0. FIG. 3B shows a processing image from S100 to S150. If there is a difference, that is, if there is a room for improving clustering accuracy with a pair of nodes (No in S150), the process returns to S100. On the other hand, when there is no difference, that is, when there is no room for improving the clustering accuracy between the pairs of nodes (Yes in S150), the control unit 212 performs the processing for each cluster as shown in FIG. Then, the nodes and edges included in the cluster are aggregated into one node, and the graph data of the main storage device is updated (S160). Furthermore, the control unit 212 calculates the clustering accuracy of the entire graph data with pairs of clusters (that is, aggregated nodes), and determines the difference when the previous processing is executed. That is, the control unit 212 determines whether there is a room for improving clustering accuracy between pairs of clusters (S170). Note that the control unit 212 determines that the previous clustering accuracy is 0 at the first execution. If there is a difference (No in S170), the process returns to S100. If there is no difference (Yes in S170), the control unit 212 outputs the final cluster correspondence table to the clustering result storage device 220 and performs the process. finish.

The above-mentioned conventional technology can reduce the amount of processing by aggregation of intermediate results by making the cluster size uniform and reducing the number of edges and nodes used for processing. Can be executed in about two and a half hours.

Louvain method [Vincent D Blondel, Journal of Statistical Mechanics: Theory and Experiment, October 9, 2008]

  However, the prior art described above has the following problems. That is, the conventional clustering apparatus executes a clustering process on a large amount of graph data in a non-parallel environment on one thread of one CPU (Central Processing Unit), and thus all data must be processed in order. For this reason, the clustering processing time increases dramatically as the graph data increases. For example, the clustering processing time increases exponentially with respect to the number of nodes in the graph, and increases linearly with respect to the number of edges in the graph. In view of the above, an object of the present invention is to solve the above-described problem and to reduce the clustering time of graph data.

  In order to solve the above-described problem, the present invention selects nodes of clustering processing target nodes of input graph data in an arbitrary order, and acquires a set Γ of adjacent nodes of the selected node from the graph data. Then, arbitrary N adjacent nodes are selected from adjacent nodes of the acquired set Γ, the data of the selected N adjacent nodes are expanded on a SIMD (Single Instruction Multiple Data) register, and A process of calculating the clustering accuracy improvement amount for each of the expanded N adjacent node data using the SIMD instruction is executed for all the adjacent nodes included in the set Γ, and the calculated clustering accuracy improvement is performed. A control unit is provided that performs labeling so that the adjacent node having the maximum amount and the selected node are classified into the same cluster. It was clustering apparatus according to claim Rukoto. However, N is a value such that N = [R / T] with respect to the SIMD register size R of the control unit and the data type size T of the graph data. According to such an invention, since the SIMD instruction is used when the clustering apparatus performs the clustering process, the performance of the clustering process can be improved up to N times and the clustering process time can be reduced. Note that the clustering accuracy improvement amount is calculated by the mathematical formula described in [Definition 1].

  According to the present invention, it is possible to reduce graph data clustering processing time.

FIG. 1 is a diagram showing a configuration of a conventional clustering apparatus. FIG. 2 is a flowchart showing a clustering processing procedure in the conventional clustering apparatus. FIG. 3 is an aggregate image in conventional clustering. FIG. 4 is a hardware configuration diagram of the clustering apparatus according to the present embodiment. FIG. 5 is a flowchart showing a processing procedure of the clustering apparatus of FIG. FIG. 6 is a flowchart showing details of S210 in FIG.

  Hereinafter, embodiments of the present invention will be described with reference to the drawings. The embodiment described below is merely an example, and the present invention can be implemented in various other forms. The following description is based on the configuration of FIG. 1 and the flowchart of FIG.

  A computer used as the clustering device 210 includes a CPU (Central Processing Unit) 100, a main storage device 110 (memory), a secondary storage device 120 (HDD (Hard Disk Drive), SSD (Solid State Drive), etc.), and an input / output device. 130. Note that the control unit 212 in FIG. 1 is realized by the CPU 100. Further, adjacent nodes in the graph data input by the input unit 211 are expanded in an adjacent node queue on the main storage device 110. Further, the processed queue is also arranged on the main storage device 110. The graph data storage device 200 and the clustering result storage device 220 are recording media such as HDDs and SSDs included in the secondary storage device 120. The input unit 211 and the output unit 213 are included in the input / output device 130.

  The configuration of the clustering apparatus 210 in the following embodiment is the same as that in FIG. 1, but the CPU 100 used in the control unit 212 is a CPU that can execute a SIMD (Single Instruction Multiple Data) instruction.

  When a program that realizes the function of the clustering device 210 is installed from the input / output device 130 to the secondary storage device 120 and activated, this program is executed in parallel by the CPU 100 using SIMD instructions. That is, when executing the clustering process, the clustering device 210 develops the graph data on the main storage device 110 and executes it by the SIMD instruction.

  Note that the program module and program data related to the program for realizing the functions of the clustering apparatus 210 are the program module and program data related to the program are the network (LAN (Local Area Network), WAN (Wide Area Network), etc.). May be stored in another computer connected via the network, and read by the CPU 100 via the network interface.

  Hereinafter, processing in the control unit 212 (CPU 100) will be described with reference to the flowchart of FIG. In the following description, it is assumed that the clustering accuracy improvement amount is calculated by the calculation formula described in [Definition 1].

  First, when graph data as shown in FIG. 3A is input, the CPU 100 selects one arbitrary node u from the graph data using a random number (S200).

  Next, the CPU 100 executes the clustering process in parallel on the node u using the SIMD instruction (S210). Details of S210 will be described later with reference to FIG.

  If it is determined in S220 that there is no room for further improvement in clustering accuracy, that is, there is no room for improvement in clustering accuracy in a pair of nodes (a pair of the node selected in S210 and its adjacent node). When the determination is made (Yes in S220), the CPU 100 aggregates these nodes into one node that belongs to the same cluster (S230). Finally, when it is confirmed that there is no room for improvement in clustering accuracy between pairs of clusters (that is, aggregated nodes) (Yes in S240), the CPU 100 ends the process.

  In S220 and S240, when it is determined that there is still room for improvement in clustering accuracy (No in S200, No in S240), the process returns to S200.

  Details of S210 will be described with reference to the flowchart of FIG. First, in S210, the CPU 100 acquires a set Γ of adjacent nodes of the node u selected in S200 from the input graph data (S211).

  Next, the CPU 100 selects any N nodes from the set of adjacent nodes Γ (S212). Note that this N depends on the CPU performance used by the CPU 100. This N is N = [R / T] with respect to the SIMD register size R of the CPU 100 and the data type size T of the graph data.

  Thereafter, the CPU 100 expands the N adjacent nodes selected in S212 on the SIMD register (S213). Then, the CPU 100 calculates the clustering accuracy improvement amount in parallel for the N adjacent nodes using the SIMD instruction for the data on the SIMD register. That is, the CPU 100 calculates in parallel the clustering accuracy improvement amount between the node u in S200 and the N adjacent nodes selected in S212 on the SIMD register (S214).

  Then, the CPU 100 acquires, from the SIMD register, the adjacent node having the maximum clustering accuracy improvement amount and the clustering accuracy improvement amount (improvement amount) between the adjacent nodes (S215), and the acquired improvement amount is If it is the largest among the nodes included in the set Γ, that is, if it is larger than the improvement amount (maximum accuracy improvement amount) held on the main storage device 110 (No in S216), the maximum accuracy on the main storage device 110 The improvement amount and the maximum improvement amount node are updated (S217). That is, the CPU 100 updates the maximum accuracy improvement amount stored in the main storage device 110 with the value of the improvement amount acquired in S215, and updates the maximum improvement amount node as an adjacent node having the maximum improvement amount. Then, the process proceeds to S218.

  On the other hand, when the CPU 100 determines that the improvement obtained in S215 is less than or equal to the maximum accuracy improvement stored in the main storage device 110 (Yes in S216), there is no unprocessed node in the set Γ (S218). Yes), labeling is performed so that the node u selected in S200 and the maximum improvement node held on the main storage device 110 are classified into the same cluster (S219), and the process of S210 is terminated. On the other hand, when there is an unprocessed node in S218 (No in S218), the process returns to S213.

  As described above, since the clustering apparatus 210 uses the SIMD instruction in the clustering processing of the graph data, the clustering processing time can be improved by up to about N times and the clustering processing time can be reduced.

  The clustering processing performance of the clustering apparatus 210 depends on the SIMD register size of the CPU 100, but as of December 2012, the SIMD register size of the CPU 100 that is generally used is 128 bits, and the data type of the graph data is 32 bits. Therefore, even as of December 2012, a four-fold improvement in performance can be expected.

100 CPU
DESCRIPTION OF SYMBOLS 110 Main storage device 120 Secondary storage device 130 Input / output device 200 Graph data storage device 210 Clustering device 211 Input unit 212 Control unit 213 Output unit 220 Clustering result storage device

Claims (4)

Select the nodes for the clustering processing target node of the input graph data in any order,
Obtaining a set Γ of adjacent nodes of the selected node from the graph data;
Selecting any N neighboring nodes from neighboring nodes of the acquired set Γ,
The data of the selected N adjacent nodes is expanded on a SIMD (Single Instruction Multiple Data) register,
A process of calculating a clustering accuracy improvement amount using SIMD instructions for each of the expanded N adjacent node data,
Execute for all neighboring nodes included in the set Γ,
A clustering apparatus, comprising: a control unit that performs labeling so that the adjacent node having the maximum improvement in the calculated clustering accuracy and the selected node are classified into the same cluster.
However, N is a value such that N = [R / T] with respect to the SIMD register size R of the control unit and the data type size T of the graph data. The process of calculating the clustering accuracy improvement amount using the SIMD instruction for each of the expanded N adjacent node data,
The control unit is
2. The clustering apparatus according to claim 1, wherein the clustering apparatus is a process of calculating in parallel a clustering accuracy improvement amount for each pair of the selected node and the N adjacent nodes selected on the SIMD register. . The controller of the clustering device
A step of selecting nodes of clustering processing target nodes of input graph data in an arbitrary order;
Obtaining a set Γ of neighboring nodes of the selected node from the graph data;
Selecting any N neighboring nodes from neighboring nodes of the acquired set Γ;
Expanding the data of the selected N adjacent nodes on a SIMD register;
A processing step of calculating a clustering accuracy improvement amount using a SIMD instruction for each of the expanded N adjacent node data,
Performing for all neighboring nodes in the set Γ;
A clustering processing method comprising: performing a labeling step so as to classify the adjacent node having the maximum improvement in the calculated clustering accuracy and the selected node into the same cluster. In the clustering device,
A step of selecting nodes of clustering processing target nodes of input graph data in an arbitrary order;
Obtaining a set Γ of neighboring nodes of the selected node from the graph data;
Selecting any N neighboring nodes from neighboring nodes of the acquired set Γ;
Expanding the data of the selected N adjacent nodes on a SIMD register;
A processing step of calculating a clustering accuracy improvement amount using a SIMD instruction for each of the expanded N adjacent node data,
Performing for all neighboring nodes in the set Γ;
A program for executing the step of performing labeling to classify the adjacent node having the maximum improvement in the calculated clustering accuracy and the selected node into the same cluster.

Images