JP6160445B2 - Analysis apparatus, analysis method, and analysis program - Google Patents

Description

  The present invention relates to an analyzer and the like.

  Cluster analysis is a process of classifying a collection of data into a plurality of clusters based on the similarity between the data. For example, cluster analysis includes hierarchical cluster analysis and non-hierarchical cluster analysis.

  In the hierarchical cluster analysis, for example, individual data is set as one cluster, the similarity between the clusters is calculated, and the process of merging the most similar clusters is repeatedly executed.

  Non-hierarchical cluster analysis uses a function representing a classification state and performs a search so that the value of the function becomes an optimal solution.

JP 2007-179143 A JP 2005-293048 A

  However, the above-described conventional technique has a problem that it takes time to perform cluster analysis on large-scale data.

  For example, hierarchical cluster analysis and non-hierarchical cluster analysis assume that cluster analysis is performed on small-scale data and medium-scale data, respectively. For this reason, if hierarchical cluster analysis or non-hierarchical cluster analysis is performed on large-scale data in a realistic computer environment, calculation may not be possible within realistic calculation time.

  An object of one aspect is to provide an analysis apparatus, an analysis method, and an analysis program that can reduce the time required for cluster analysis.

  In the first plan, the analysis apparatus includes a sampling execution unit, a cluster analysis unit, a cluster prediction unit, a determination unit, and a final cluster calculation unit. The sampling execution unit generates a plurality of sampling data by repeatedly performing a process of sampling the input data and extracting a part of the data from the input data. The cluster analysis unit performs cluster analysis on a plurality of sampling data, and classifies data included in the sampling data into different clusters for each sampling data. The cluster prediction unit generates a plurality of prediction data indicating data obtained by predicting a cluster to which the data included in the input data belongs, based on the plurality of classification results of the cluster analysis unit for the plurality of sampling data and the input data. The determination unit calculates an evaluation value for each prediction data based on the inter-cluster distance and the intra-cluster distance of the prediction data, and determines prediction data corresponding to the evaluation value that is a Pareto solution. The final cluster calculation unit classifies the data included in the input data into clusters based on the prediction data corresponding to the evaluation value that is the Pareto solution. Note that the prediction data that is the Pareto solution has, for example, an evaluation value that is superior to other prediction data.

  According to one embodiment of the present invention, it is possible to reduce the time required for cluster analysis.

FIG. 1 is a functional block diagram illustrating the configuration of the analyzer according to the first embodiment. FIG. 2 is a diagram illustrating an example of a data structure of analysis target data. FIG. 3 is a diagram illustrating an example of the data structure of the sampling data table. FIG. 4 is a diagram illustrating an example of the data structure of the prediction data table. FIG. 5 is a diagram illustrating an example of a data structure of the evaluation value data table. FIG. 6 is a diagram illustrating an example of the data structure of the intermediate data table. FIG. 7 is a diagram illustrating an example of the data structure of the final data. FIG. 8 is a diagram (1) showing the relationship between the intra-cluster distance and the inter-cluster distance of each prediction data. FIG. 9 is a diagram illustrating an example of the final data candidate table. FIG. 10 is a flowchart of the process procedure of the analyzer according to the first embodiment. FIG. 11 is a functional block diagram of the configuration of the analyzer according to the second embodiment. FIG. 12 is a diagram (2) illustrating the relationship between the intra-cluster distance and the inter-cluster distance of each prediction data. FIG. 13 is a flowchart (1) illustrating a processing procedure of the analyzer according to the second embodiment. FIG. 14 is a flowchart (2) illustrating the processing procedure of the analyzer according to the second embodiment. FIG. 15 is a diagram illustrating an example of a computer that executes an analysis program.

  Hereinafter, embodiments of an analysis apparatus, an analysis method, and an analysis program disclosed in the present application will be described in detail with reference to the drawings. Note that the present invention is not limited to the embodiments.

  FIG. 1 is a functional block diagram illustrating the configuration of the analyzer according to the first embodiment. As illustrated in FIG. 1, the analysis apparatus 100 includes a communication unit 110, an input unit 120, an output unit 130, a storage unit 140, and a control unit 150.

  The communication unit 110 is a processing unit that connects to a network wirelessly or by wire and performs data communication with other devices via the network. The communication unit 110 corresponds to a communication device.

  The input unit 120 is an input device that inputs various types of information. The input unit 120 corresponds to, for example, a keyboard, a mouse, a touch panel, and the like.

  The output unit 130 is a display device that displays information output from the control unit 150. For example, the output unit 130 corresponds to a monitor, a liquid crystal display, a touch panel, or the like.

  The storage unit 140 includes analysis target data 141, a sampling data table 142, a prediction data table 143, an evaluation value table 144, an intermediate data table 145, and final data 146. The storage unit 140 corresponds to, for example, a semiconductor memory device such as a RAM (Random Access Memory), a ROM (Read Only Memory), and a flash memory (Flash Memory), and a storage device such as an HDD (Hard Disk Drive).

  The analysis target data 141 is data to be subjected to cluster analysis. FIG. 2 is a diagram illustrating an example of a data structure of analysis target data. As shown in FIG. 2, the analysis target data 141 includes an identification number, age, sex, height, weight, and the like. The identification number is information for uniquely identifying each record. Age, sex, height, and weight are information indicating the age, sex, height, and weight of a specific person, respectively. In the example shown in FIG. 2, the sex is represented by 1 or 2. For example, gender “1” indicates that the gender is male, and gender “2” indicates that the gender is female.

  The sampling data table 142 is a table having a plurality of sampling data. Each sampling data is generated by a sampling execution unit 151 described later. Each sampling data is produced | generated when the sampling execution part 151 samples the analysis object data 141. FIG. FIG. 3 is a diagram illustrating an example of the data structure of the sampling data table. As shown in FIG. 3, the sampling data table 142 has sampling data 142a, 142b, 142c. In FIG. 3, sampling data 142a, 142b, and 142c are shown as an example, but other sampling data may be included.

  In FIG. 3, for example, the sampling data 142a has an identification number, age, sex, height, weight, and cluster number. The explanation regarding the identification number, age, sex, height, and weight is the same as the explanation of age, sex, height, and weight described in FIG.

  The prediction data table 143 is a table having a plurality of prediction data. Each prediction data is generated by a cluster prediction unit 153 described later. The cluster prediction unit 153 generates prediction data by predicting the cluster number of each record of the analysis target data 141 based on the sampling data. Prediction data is generated for each sampling data. FIG. 4 is a diagram illustrating an example of the data structure of the prediction data table. As shown in FIG. 4, the prediction data table 143 includes prediction data 143a, 143b, and 143c. FIG. 4 shows the prediction data 143a, 143b, and 143c as an example, but other prediction data may be included.

  The evaluation value data table 144 is a table that holds the evaluation value of each prediction data. FIG. 5 is a diagram illustrating an example of a data structure of the evaluation value data table. As shown in FIG. 5, this evaluation value data table 144 associates predicted data identification information with evaluation values. The prediction data identification information is information that uniquely identifies the prediction data.

  In FIG. 5, the evaluation value includes an inter-cluster distance and an intra-cluster distance. The inter-cluster distance indicates the distance between different clusters. In general, the larger the inter-cluster distance, the higher the evaluation for the cluster analysis result. The intra-cluster distance indicates the diameter of the cluster. In general, the smaller the intra-cluster distance, the higher the evaluation for the cluster analysis result. That is, the larger the inter-cluster distance and the smaller the intra-cluster distance, the better the cluster analysis result.

  The intermediate data table 145 is a table having a plurality of intermediate data. Each intermediate data is created corresponding to prediction data having a good evaluation value. FIG. 6 is a diagram illustrating an example of the data structure of the intermediate data table. As shown in FIG. 6, the intermediate data table 145 includes intermediate data 145a, 145g, and 145z. In FIG. 6, the intermediate data 145a, 145g, and 145z are shown as an example, but other intermediate data may be included.

  In FIG. 6, the intermediate data associates an identification number with each cluster number. The identification number corresponds to the identification number of the analysis target data 141. For example, the first row of the intermediate data 145a indicates that the record corresponding to the identification number 1001 is classified into the cluster number “1”. Here, the record corresponding to the identification number “1001” corresponds to the record corresponding to the identification number “1001” of the analysis target data 141 illustrated in FIG. 2. Therefore, it is indicated that the record corresponding to the identification number “1001” of the analysis target data 141 illustrated in FIG. 2 belongs to the cluster having the cluster number “1”.

  The final data 146 indicates the final cluster analysis result of the analysis target data 141. FIG. 7 is a diagram illustrating an example of the data structure of the final data. As shown in FIG. 7, the final data 146 associates an identification number with each cluster number. The identification number corresponds to the identification number of the analysis target data 141. For example, the first row of the final data 146 indicates that the record corresponding to the identification number 1001 is classified into the cluster number “1”.

  Returning to the description of FIG. The control unit 150 includes a sampling execution unit 151, a cluster analysis unit 152, a cluster prediction unit 153, a determination unit 154, and a final cluster calculation unit 155. The control unit 150 corresponds to an integrated device such as an application specific integrated circuit (ASIC) or a field programmable gate array (FPGA). The control unit 150 corresponds to an electronic circuit such as a CPU (Central Processing Unit) or an MPU (Micro Processing Unit).

  The sampling execution unit 151 is a processing unit that generates a plurality of sampling data by repeatedly performing sampling on the analysis target data 141 a plurality of times. The sampling execution unit 151 stores the generated sampling data in the sampling data table 142.

  For example, the sampling execution unit 151 acquires the number of calculations and the number of samplings via the input unit 120, and performs sampling for the acquired number of calculations. In addition, the sampling execution unit 151 may change the sampling interval each time sampling is executed. Further, random sampling may be performed.

  The sampling execution unit 151 matches the number of sampling data records with the number of samplings acquired from the input unit 120. For example, when the designated sampling number is N2, the number of each sampling data is N2. For example, if the number of records of the analysis target data is N1, the magnitude relationship between N1 and N2 is “N1> N2.”

  The cluster analysis unit 152 is a processing unit that acquires each sampling data stored in the sampling data table 142 and performs cluster analysis on each sampling data. The cluster analysis unit 152 assigns a cluster number to each record of the sampling data according to the cluster analysis result.

  The sampling data table 142 shown in FIG. 3 will be described as an example. The cluster analysis unit 152 first performs cluster analysis on the sampling data 142a, classifies each record of the sampling data 142a into a plurality of clusters, and assigns cluster numbers according to the classification results. Similarly, the cluster analysis unit 152 performs cluster analysis on the sampling data 142b and 142c, classifies each record into a plurality of clusters, and assigns cluster numbers according to the classification results. It is assumed that the number of clusters classified by the cluster analysis unit 152 is set in advance.

  The cluster analysis performed by the cluster analysis unit 152 may be hierarchical cluster analysis or non-hierarchical cluster analysis. Here, as an example, a case where the cluster analysis unit 152 executes hierarchical cluster analysis will be described.

  When the cluster analysis unit 152 performs hierarchical cluster analysis, first, each data is set as one cluster, and the similarity between the clusters is calculated. The cluster analysis unit 152 merges the most similar clusters. The cluster analysis unit 152 repeatedly executes the above processing until the number is equal to the number of clusters set in advance.

  For example, the cluster analysis unit 152 calculates the Euclidean distance between the clusters for each combination of clusters, and merges the combinations of the clusters that minimize the Euclidean distance. In this case, the Euclidean distance between the clusters corresponds to the similarity between the clusters, and the similarity is higher as the Euclidean distance is shorter.

  The cluster prediction unit 153 is a processing unit that predicts the cluster number of each record of the analysis target data 141 based on the cluster analysis result of the sampling data in the sampling data table 142 and generates the prediction data table 143. The cluster prediction unit 153 generates prediction data by the number of sampling data included in the sampling data table 142, and registers the generated prediction data in the prediction data table 143.

  For example, the cluster prediction unit 153 generates the prediction data 143a based on the sampling data 142a in the sampling data table 142. The cluster prediction unit 153 generates prediction data 143b based on the sampling data 142b. The cluster prediction unit 153 generates the prediction data 143c based on the sampling data 142c. When there are N pieces of sampling data, the cluster prediction unit 153 creates N pieces of prediction data.

  Here, an example of processing when the cluster prediction unit 153 generates the prediction data 143a based on the sampling data 142a will be described. First, the cluster prediction unit 153 sets the relationship between the identification number included in the sampling data 142 and the cluster number as it is in the prediction data 143a.

  For example, if the cluster number of the record with the identification number “1001” in the sampling data 142a is “1”, the cluster prediction unit 153 sets the cluster number with the identification number “1001” in the prediction data 143a to “1”. . Similarly, the cluster prediction unit 153 sets the relationship between all identification numbers and cluster numbers existing in the sampling data 142a in the prediction data 143a.

  Subsequently, the cluster predicting unit 153 performs the following process on the record in which the cluster number is not set as a result of the above process. First, the cluster prediction unit 153 detects a representative record from records classified into each cluster. For example, a record having an average numerical value among the records having the cluster number “1” is detected as a representative record. The cluster prediction unit 153 similarly detects representative records corresponding to other cluster numbers.

  The cluster prediction unit 153 calculates the Euclidean distance between the record in which the cluster number is not set and each representative record, and identifies the combination that minimizes the Euclidean distance. The cluster prediction unit 153 sets the cluster number of the representative record of the identified set as the cluster number of the corresponding record.

  For example, when the Euclidean distance between the record in which the cluster number is not set and each representative record is calculated, and the Eugrid distance between the record that has not been set and the representative record with the cluster number “1” is the minimum, this is applicable. Set the cluster number of the record to “1”. The cluster prediction unit 153 generates the prediction data table 143 by repeatedly executing the above process for the unset records.

  The determination unit 154 is a processing unit that generates the evaluation value data table 144 based on the prediction data table 143. The evaluation unit 154 calculates an evaluation value for each prediction data included in the prediction data table 143.

  The determination unit 154 calculates the inter-cluster distance and the intra-cluster distance for each prediction data, and uses the inter-cluster distance and the intra-cluster distance as evaluation values of the prediction data. An example of processing for calculating the inter-cluster distance of the prediction data will be described. Here, it is assumed that clusters with cluster numbers “1 to 3” exist. The determination unit 154 detects the first representative record belonging to the cluster number “1”, the second representative record belonging to the cluster number “2”, and the third representative record belonging to the cluster number “3”. An example of processing for detecting a representative record is to detect, for example, a record having an average numerical value as a representative record among records belonging to the same cluster number.

  The determination unit 154 calculates the Euclidean distance between the first representative record and the second representative record, and calculates the Euclidean distance between the first representative record and the third representative record. The determination unit 154 sets the Euclidean distance obtained by averaging the calculated Euclidean distances as the intercluster distance of the prediction data.

  For example, the values of age, sex, height, and weight of the first representative record are a1, a2, a3, and a4, respectively. The values of age, sex, height, and weight of the second representative record are b1, b2, b3, and b4, respectively. Assume that the values of the age, sex, height, and weight of the third representative record are c1, c2, c3, and c4, respectively. In this case, the Euclidean distance X1 between the first representative record and the second representative record is calculated by Expression (1), and the Euclidean distance X2 between the first representative record and the third representative record is calculated by Expression (2). ). In this case, the inter-cluster distance of the prediction data is as shown in Expression (3).

Eugrit distance X1 = ((a1-b1) ² + (a2-b2) ² + (a3-b3) ² + (a4-b4) ² ) ^1/2 (1)

Eugrid distance X2 = ((a1-c1) ² + (a2-c2) ² + (a3-c3) ² + (a4-c4) ² ) ^1/2 (2)

Distance between clusters = (X1 + X2) / 2 (3)

  Next, a process for calculating the intra-cluster distance will be described. First, the determination unit 154 calculates the Euclidean distance between records belonging to the same cluster number. Then, the determination unit 154 sets the Euclidean distance obtained by averaging the calculated Euclidean distances as the intra-cluster distance of the prediction data. The determination unit 154 calculates the intra-cluster distance of the prediction data by averaging the intra-cluster distance corresponding to the cluster of each cluster number.

  For example, when there is a cluster with the cluster number “1-3”, the determination unit 154 calculates the intra-cluster distance of each cluster number “1-3”. The determination unit 154 calculates the intra-cluster distance of the prediction data by averaging the intra-cluster distances of the respective cluster numbers “1 to 3”.

  For example, an example of calculating the intra-cluster distance of the cluster number “1” will be described. Assume that there are three first records, second record, and third record in the cluster. For example, assume that the values of age, sex, height, and weight of the first record are d1, d2, d3, and d4, respectively. Assume that the values of age, sex, height, and weight of the second record are e1, e2, e3, and e4, respectively. The values of the age, sex, height, and weight of the third representative record are f1, f2, f3, and f4, respectively. In this case, the Euclidean distance Y1 between the first record and the second record is calculated by Expression (4), and the Euclidean distance Y2 between the first record and the third record is calculated by Expression (5). The In this case, the intra-cluster distance of the cluster with the cluster number “1” is as shown in Expression (6).

Eugrid distance Y1 = ((d1−e1) ² + (d2−e2) ² + (d3−e3) ² + (d4−e4) ² ) ^1/2 (4)

Eugrid distance Y2 = ((d1-f1) ² + (d2-f2) ² + (d3-f3) ² + (d4-f4) ² ) ^1/2 (5)

Distance within cluster = (Y1 + Y2) / 2 (6)

  The determination unit 154 calculates the intra-cluster distance for other clusters in the same manner, and calculates the intra-cluster distance of the prediction data by averaging the intra-cluster distance of each cluster.

  The determination unit 154 performs the above processing for each prediction data included in the prediction data table 143, thereby calculating an evaluation value of each prediction data, and generates an evaluation value data table 144.

  The final cluster calculation unit 155 is a processing unit that generates final data 146 that is a final cluster analysis result of the analysis target data 141. The final cluster calculation unit 155 generates the final data 146 based on the intermediate data table 145 after performing the process of generating the intermediate data table 145 from the evaluation value data table 144.

  An example of processing in which the final cluster calculation unit 155 generates the intermediate data table 145 from the evaluation value data table 144 will be described. The final cluster calculation unit 155 compares the evaluation values for each prediction data in the evaluation value data table 144, specifies the prediction data to be the Pareto solution, and sets the prediction data to be the specified Pareto solution in the intermediate data table 145 To do. For example, the prediction data that is a Pareto solution is superior to other prediction data for one or more items.

  FIG. 8 is a diagram (1) showing the relationship between the intra-cluster distance and the inter-cluster distance of each prediction data. In FIG. 8, the vertical axis represents the intra-cluster distance, and the horizontal axis represents the inter-cluster distance. In general, it can be said that the prediction data is better prediction data as the inter-cluster distance is larger and the intra-cluster distance is smaller. Therefore, in the example illustrated in FIG. 8, the final cluster calculation unit 155 identifies the prediction data 143a, 143g, and 143z as Pareto solutions.

  Subsequently, a process in which the final cluster calculation unit 155 generates the final data 146 from the intermediate data table 145 will be described. First, the final cluster calculation unit 155 generates a final data candidate table. FIG. 9 is a diagram illustrating an example of the final data candidate table. As shown in FIG. 9, the final data candidate table 10 includes final data candidates 10a, 10b, and 10c. Here, as an example, the final data candidates 10a, 10b, and 10c are shown, but other final data candidates may be included.

  The final cluster calculation unit 155 sets each cluster number of the final data candidates 10a, 10b, and 10c to an initial value of 0. Then, the final cluster calculation unit 155 randomly assigns “1” so that any one of the values of the cluster numbers of the identification numbers is “1”. For example, in the example shown in FIG. 9, by randomly assigning “1” to the identification number “1001” of the final data candidate 10a, the one corresponding to the cluster number “1” is set to “1”. “0” is set for the cluster number.

  The final cluster calculator 155 calculates the similarity between each final data candidate 10a, 10b, 10c in the final data candidate table 10 and each intermediate data in the intermediate data table 145, and determines the final data candidate with the highest similarity as It is specified as final data 146.

  The final cluster calculator 155 compares the identification number of the intermediate data and the cluster number corresponding to the identification number with the identification number of the final data candidate and the cluster number corresponding to the identification number, and counts the number of matches. The final cluster calculation unit 155 calculates the similarity by dividing the number of matches by the total number of records. In the following description, the number of matches is expressed as the number of matches.

  For example, a case where the final cluster calculation unit 155 calculates the similarity of the final data candidate 10a will be described. The final cluster calculation unit 155 compares the final data candidate 10a and the intermediate data 145a, and if the number of matches is “L1” and the total number of records of the final data candidate 10a is “M1”, the final data candidate 10a And the intermediate data 145a is “L1 / M1”. The final cluster calculation unit 155 compares the final data candidate 10a with the intermediate data 145g, and if the number of matches is “L2” and the total number of records in the final data candidate 10a is “M2”, the final data candidate 10a And the intermediate data 145a is “L2 / M2”. The final cluster calculation unit 155 compares the final data candidate 10a with the intermediate data 145z. If the number of matches is “L3” and the total number of records of the final data candidate 10a is “M3”, the final data candidate 10a And the intermediate data 145z are “L3 / M3”. In this case, the final cluster calculation unit 155 specifies the similarity of the final data candidate 10a as “L1 / M1 + L2 / M2 + L3 / M3”.

  The final cluster calculation unit 155 calculates the similarity for the final data candidates 10b and 10c in the same manner as the final data candidate 10a. The final cluster calculation unit 155 compares the similarity of the final data candidate 10a, the similarity of the final data candidate 10b, and the similarity of the final data candidate 10c, and specifies the final data candidate having the maximum similarity. The final cluster calculation unit 155 sets the specified final data candidate as the final data 146. The final cluster calculation unit 155 may output the final data 146 to the output unit 130.

  Next, a processing procedure of the analyzer 100 according to the first embodiment will be described. FIG. 10 is a flowchart of the process procedure of the analyzer according to the first embodiment. As shown in FIG. 10, the analysis apparatus 100 accepts analysis target data 141 (step S101). Moreover, the analyzer 100 receives the number of repeated calculations (step S102) and receives the number of samplings. Also, the count value is initialized (step S103). The analyzer 100 adds 1 to the count value (step S104). The initial value of the count value is 0.

  The analysis apparatus 100 samples the analysis target data 141 and generates sampling data (step S105). Each sampling data is stored in the sampling data table 142. The analysis apparatus 100 performs a cluster analysis process on the sampling data, and assigns a cluster number to each record (step S106).

  The analysis apparatus 100 compares the sampling data to which the cluster number is assigned and the analysis target data 141, and generates prediction data by assigning a cluster number to each record included in the analysis target data 141 (step S107). ). Each prediction data is stored in the prediction data table 143.

  The analysis apparatus 100 calculates the intra-cluster distance and the inter-cluster distance based on the prediction data, and generates the evaluation value data table 144 (step S108). The analysis apparatus 100 determines whether or not the number of repeated calculations is less than the count value (step S109). When the number of repeated calculations is less than the count value (step S1090, Yes), the analysis apparatus 100 proceeds to step S104.

  On the other hand, when the number of repeated calculations is equal to or greater than the count value (No at Step S109), the analysis apparatus 100 selects prediction data corresponding to the Pareto solution and creates the intermediate data table 145 (Step S110). .

  The analysis apparatus 100 generates a plurality of final data candidates randomly assigned with cluster numbers (step S111). In step S111, the analysis apparatus 100 assigns cluster numbers so that the degree of similarity increases. For example, the analysis apparatus 100 randomly assigns cluster numbers and calculates the similarity. Then, the analysis apparatus 100 changes the cluster number allocation with a high degree of similarity a little, and repeats the trial processing as many times as the degree of similarity increases or the user sets.

  The analysis apparatus 100 compares the intermediate data with each final data candidate, and determines the final data candidate having the maximum similarity (step S112). The final data candidate having the maximum similarity determined in step S112 is the final data 146. The analyzer 100 outputs the determination result (step S113).

  Next, effects of the analyzer 100 according to the first embodiment will be described. The analysis apparatus 100 performs cluster analysis on the sampling data extracted from the analysis target data 141, and generates a plurality of prediction data in which the cluster to which each data of the analysis target data belongs is predicted based on the cluster analysis result of the sampling data. And the analysis apparatus 100 specifies the final cluster classification | category result of the analysis object data 141 using the cluster classification | category result of prediction data with a favorable evaluation value among several prediction data. Thereby, according to the analyzer 100, the time which cluster analysis requires can be reduced.

  In addition, it is possible to perform cluster analysis within a realistic time with a realistic computer for large-scale data that cannot be calculated within a realistic time with a realistic computer.

  FIG. 11 is a functional block diagram of the configuration of the analyzer according to the second embodiment. As illustrated in FIG. 11, the analysis device 200 includes a communication unit 210, an input unit 220, an output unit 230, a storage unit 240, and a control unit 250.

  The description regarding the communication unit 210, the input unit 220, and the output unit 230 is the same as the description regarding the communication unit 110, the input unit 120, and the output unit 130 illustrated in FIG.

  The storage unit 240 includes analysis target data 241, a sampling data table 242, a prediction data table 243, an evaluation value data table 244, an intermediate data table 245, and final data 246. The storage unit 240 corresponds to, for example, a semiconductor memory element such as a RAM, a ROM, or a flash memory, or a storage device such as an HDD.

  The analysis target data 241 is data to be subjected to cluster analysis. The data structure of the analysis target data 241 is the same as the data structure of the analysis target data 141 shown in FIG.

  The sampling data table 242 is a table having a plurality of sampling data. Each sampling data is generated by a sampling execution unit 251 described later. The sampling execution unit 251 samples the analysis target data 241 so that each sampling data is generated. The data structure of the sampling data table 242 is the same as the data structure of the sampling data table 142 shown in FIG.

  The prediction data table 243 is a table having a plurality of prediction data. Each prediction data is generated by a cluster prediction unit 253 described later. The cluster prediction unit 253 generates prediction data by predicting the cluster number of each record of the analysis target data 241 based on the sampling data. Prediction data is generated for each sampling data. The data structure of the prediction data table 243 is the same as the data structure of the prediction data table 143 shown in FIG.

  The evaluation value data table 244 is a table that holds the evaluation value of each prediction data. The data structure of the evaluation value data table 244 is the same as the data structure of the evaluation value data table 144 shown in FIG.

  The intermediate data table 245 is a table having a plurality of intermediate data. Each intermediate data is created corresponding to prediction data having a good evaluation value. The data structure of the intermediate data table 245 is the same as the data structure of the intermediate data table 145 shown in FIG.

  The final data 246a, 246b, and 246c indicate final cluster analysis results of the analysis target data 241. The data structure of each final data 246a, 246b, 246c is the same as the data structure of the final data 146 shown in FIG.

  Returning to the description of FIG. The control unit 250 includes a sampling execution unit 251, a cluster analysis unit 252, a cluster prediction unit 253, a determination unit 254, and a final cluster calculation unit 255. The control unit 250 corresponds to, for example, an integrated device such as an ASIC or FPGA. Moreover, the control part 250 respond | corresponds to electronic circuits, such as CPU and MPU, for example.

  The sampling execution unit 251 is a processing unit that generates a plurality of sampling data by repeatedly performing sampling on the analysis target data 241 a plurality of times. The sampling execution unit 251 stores each generated sampling data in the sampling data table 242. Specific processing of the sampling execution unit 251 is the same as that of the sampling execution unit 151 shown in FIG.

  The cluster analysis unit 252 is a processing unit that acquires each sampling data stored in the sampling data table 242 and performs cluster analysis on each sampling data. The cluster analysis unit 252 assigns a cluster number to each record of sampling data according to the cluster analysis result. The specific processing of the cluster analysis unit 252 is the same as that of the cluster analysis unit 152 shown in FIG.

  The cluster prediction unit 253 is a processing unit that predicts the cluster number of each record of the analysis target data 241 based on the cluster analysis result of the sampling data in the sampling data table 242, and generates the prediction data table 243. The specific processing of the cluster prediction unit 253 is the same as that of the cluster prediction unit 153 illustrated in FIG.

  The determination unit 254 is a processing unit that generates the evaluation value data table 244 based on the prediction data table 243. The determination unit 254 calculates an evaluation value for each prediction data included in the prediction data table 243. For example, the determination unit 254 calculates the inter-cluster distance and the intra-cluster distance for each prediction data, and uses the inter-cluster distance and the intra-cluster distance as evaluation values of the prediction data.

  The final cluster calculation unit 255 is a processing unit that generates final data 246 that is a final cluster analysis result of the analysis target data 241. The final cluster calculation unit 255 performs processing for generating the intermediate data table 245 from the evaluation value data table 244, and then generates final data 246a, 246b, and 246c based on the intermediate data table 245.

  A process in which the final cluster calculation unit 255 generates the intermediate data table 245 from the evaluation value data table 244 will be described. The final cluster calculation unit 255 compares the evaluation values for each prediction data in the evaluation data table 244, specifies the prediction data to be the Pareto solution, and sets the prediction data to be the specified Pareto solution in the intermediate data table 245.

  FIG. 12 is a diagram (2) illustrating the relationship between the intra-cluster distance and the inter-cluster distance of each prediction data. In FIG. 12, the vertical axis represents the intra-cluster distance, and the horizontal axis represents the inter-cluster distance. In general, it can be said that the intermediate data is better prediction data as the inter-cluster distance is larger and the intra-cluster distance is smaller. Therefore, in the example illustrated in FIG. 12, the final cluster calculation unit 255 identifies the intermediate data 243a, 243c, 243f, 243g, and 243z as Pareto solutions.

  Next, a process in which the final cluster calculation unit 255 generates the final data 246 from the intermediate data table 245 will be described. The final cluster calculation unit 255 compares the evaluation values of the respective prediction data in the intermediate data table 245 and classifies similar prediction data into the same group. For example, the final cluster calculation unit 255 classifies the prediction data in which the difference between the inter-cluster distances of each prediction data is less than the threshold and the difference in the intra-cluster distance of each prediction data is less than the threshold into the same group.

  In the example illustrated in FIG. 12, the final cluster calculation unit 255 classifies the prediction data 243a and 243c into the group 50a, classifies the prediction data 243f and 243g into the group 50b, and classifies the prediction data 243i and 243z into the group 50c. The final cluster calculation unit 255 generates final data 246 for each classified group.

  For example, the final cluster calculation unit 255 generates final data 246a based on the prediction data 243a and 243c included in the group 50a. The final cluster calculation unit 255 generates final data 246b based on the prediction data 243f and 243g included in the group 50b. The final cluster calculation unit 255 generates final data 246c based on the prediction data 243i and 243z included in the group 50c.

  The final cluster calculation unit 255 specifies the final data based on the prediction data. The final cluster calculation unit 155 in FIG. 1 specifies the final data based on the prediction data in the intermediate data table 145. It is the same. In the example shown in FIG. 12, final data is specified for the groups 50a, 50b, and 50c, and final data 246a, 246b, and 246c are generated.

  Next, a processing procedure of the analyzer 200 according to the second embodiment will be described. 13 and 14 are flowcharts illustrating the processing procedure of the analysis apparatus according to the second embodiment. As illustrated in FIG. 13, the analysis apparatus 200 receives analysis target data 241 (step S201). Moreover, the analyzer 200 receives the number of repeated calculations (step S202) and receives the number of samplings. Also, the count value is initialized (step S203). The analyzer 200 adds 1 to the count value (step S204). The initial value of the count value is 0.

  The analysis device 200 samples the analysis target data 241 and generates sampling data (step S205). Each sampling data is stored in the sampling data table 242. The analysis apparatus 200 performs cluster analysis processing on the sampling data and assigns a cluster number to each record (step S206).

  The analysis apparatus 200 compares the sampling data to which the cluster number is assigned and the analysis target data 241 and generates prediction data by assigning the cluster number to each record included in the analysis target data 241 (step S207). ). Each prediction data is stored in the prediction data table 243.

  The analysis apparatus 200 calculates the intra-cluster distance and the inter-cluster distance based on the prediction data, and generates the evaluation value data table 244 (step S208). The analyzing apparatus 200 determines whether or not the number of repeated calculations is less than the count value (step S209). When the number of repeated calculations is less than the count value (step S2090, Yes), the analyzer 2100 proceeds to step S204.

  On the other hand, when the number of repeated calculations is equal to or greater than the count value (No at Step S209), the analyzer 200 proceeds to Step S210 in FIG.

  The description shifts to the description of FIG. The analysis apparatus 200 selects prediction data corresponding to the Pareto solution and creates the intermediate data table 245 (step S210). The analysis device 200 calculates the similarity of each prediction data corresponding to the Pareto solution (step S211). The analysis apparatus 200 classifies each similar prediction data into a group (step S212).

  The analysis apparatus 200 selects an unselected group (step S213), and generates a plurality of final data candidates randomly assigned with cluster numbers (step S214). In step S214, the analysis apparatus 200 assigns a cluster number so that the degree of similarity increases. For example, the analysis apparatus 200 randomly assigns cluster numbers and calculates the similarity. Then, the analysis apparatus 200 changes the cluster number allocation with a high degree of similarity a little, and repeats the trial processing as many times as the degree of similarity increases or the user sets.

  The analysis apparatus 200 compares the prediction data included in the group with each final data candidate, and determines the final data candidate having the maximum similarity (step S215).

  The analysis apparatus 200 determines whether there is an unselected group (step S216). If there is an unselected group (step S216, Yes), the analysis apparatus 200 proceeds to step S213. On the other hand, when there is no unselected group (step S216, No), the analysis apparatus 200 outputs the determination result of each group (step S217).

  Next, the effect of the analyzer 200 according to the second embodiment will be described. The analysis apparatus 200 groups prediction data having a good evaluation value among a plurality of prediction data with similar prediction data, and generates final data 246 for each group. For this reason, according to the analyzer 200, the time required for cluster analysis can be reduced. In addition, a plurality of final data candidates corresponding to similar prediction data can be obtained.

  Next, an example of a computer that executes an analysis program that realizes the same function as the analysis apparatuses 100 and 200 shown in the above embodiment will be described. FIG. 15 is a diagram illustrating an example of a computer that executes an analysis program.

  As illustrated in FIG. 15, the computer 300 includes a CPU 301 that executes various arithmetic processes, an input device 302 that receives input of data from a user, and a display 303. The computer 300 also includes a reading device 304 that reads a program or the like from a storage medium, and an interface device 305 that exchanges data with other computers via a network. The computer 300 also includes a RAM 306 that temporarily stores various types of information and a hard disk device 307. The devices 301 to 307 are connected to the bus 308.

  The hard disk device 307 includes a sampling program 307a, a cluster analysis program 307b, a cluster prediction program 307c, a determination program 307d, and a final cluster calculation program 307e. The CPU 301 reads out each program 307 a to 307 e and develops it in the RAM 306.

  The sampling program 307a functions as a sampling process 306a. The cluster analysis program 307b functions as a cluster analysis process 306b. The cluster prediction program 307c functions as a cluster prediction process 306c. The determination program 307d functions as a determination process 306d. The final cluster calculation program 307e functions as a final cluster calculation process 306e.

  For example, the sampling process 306a corresponds to the sampling execution units 151 and 251. The cluster analysis process 306 b corresponds to the cluster analysis units 152 and 252. The cluster prediction process 306 c corresponds to the cluster prediction units 153 and 253. The determination process 306d corresponds to the determination units 154 and 254. The final cluster calculation process 306e corresponds to the final cluster calculation units 155 and 255.

  Note that the programs 307a to 307e are not necessarily stored in the hard disk device 307 from the beginning. For example, each program is stored in a “portable physical medium” such as a flexible disk (FD), a CD-ROM, a DVD disk, a magneto-optical disk, and an IC card inserted into the computer 300. Then, the computer 500 may read and execute each of the programs 307a to 307e from these.

  The following supplementary notes are further disclosed with respect to the embodiments including the above examples.

(Supplementary Note 1) A sampling execution unit that performs sampling on input data and repeatedly executes a process of extracting some data from the input data to generate a plurality of sampling data;
A cluster analysis unit that performs cluster analysis on the plurality of sampling data, and classifies the data included in the sampling data into different clusters for each sampling data;
Cluster prediction for generating a plurality of prediction data indicating data obtained by predicting a cluster to which data included in the input data belongs based on a plurality of classification results of the cluster analysis unit and the input data for the plurality of sampling data And
A determination unit that calculates an evaluation value for each prediction data based on the inter-cluster distance and the intra-cluster distance of the prediction data, and determines prediction data corresponding to the evaluation value that is a Pareto solution;
An analysis device comprising: a final cluster calculation unit that classifies data included in the input data into clusters based on prediction data corresponding to the evaluation value that is the Pareto solution.

(Supplementary Note 2) The final cluster calculation unit groups similar prediction data corresponding to the evaluation value that is a Pareto solution, and sets the data included in the input data to different clusters based on the prediction data included in the same group. The analysis apparatus according to appendix 1, wherein the process of classifying is performed for each group.

(Supplementary Note 3) The final cluster calculation unit generates a plurality of final cluster data in which clusters are randomly assigned to the input data, and specifies based on the similarity between each final cluster data and predicted data The analyzer according to appendix 1 or 2, wherein the last cluster data is selected.

(Supplementary Note 4) An analysis method executed by a computer,
Sampling the input data and repeatedly executing a process of extracting a part of the data from the input data to generate a plurality of sampling data,
Cluster analysis is performed on the plurality of sampling data, and the data included in the sampling data is classified into different clusters for each sampling data,
Based on a plurality of classification results of the cluster analysis unit for the plurality of sampling data and the input data, to generate a plurality of prediction data indicating data predicted clusters to which the data included in the input data belongs,
Based on the inter-cluster distance and intra-cluster distance of the prediction data, calculate an evaluation value for each prediction data, determine the prediction data corresponding to the evaluation value to be a Pareto solution,
An analysis method comprising: performing each process of classifying data included in the input data into clusters based on prediction data corresponding to the evaluation value serving as the Pareto solution.

(Additional remark 5) The process which classifies the data contained in the said input data into a cluster groups the similar prediction data corresponding to the evaluation value used as a Pareto solution, and performs said input based on the prediction data contained in the same group The analysis method according to appendix 4, wherein the process of classifying the data included in the data into different clusters is executed for each group.

(Additional remark 6) The process which classifies the data contained in the said input data into a cluster produces | generates several final cluster data which allocated the cluster at random with respect to the said input data, and each final cluster data and prediction data 6. The analysis method according to appendix 4 or 5, wherein specific final cluster data is selected based on the similarity.

(Appendix 7)
Sampling the input data and repeatedly executing a process of extracting a part of the data from the input data to generate a plurality of sampling data,
Cluster analysis is performed on the plurality of sampling data, and the data included in the sampling data is classified into different clusters for each sampling data,
Based on a plurality of classification results of the cluster analysis unit for the plurality of sampling data and the input data, to generate a plurality of prediction data indicating data predicted clusters to which the data included in the input data belongs,
Based on the inter-cluster distance and intra-cluster distance of the prediction data, calculate an evaluation value for each prediction data, determine the prediction data corresponding to the evaluation value to be a Pareto solution,
An analysis program for executing each process for classifying data included in the input data into clusters based on prediction data corresponding to an evaluation value serving as the Pareto solution.

(Additional remark 8) The process which classifies the data contained in the said input data into a cluster groups the similar prediction data corresponding to the evaluation value used as a Pareto solution, and performs said input based on the prediction data contained in the same group The analysis program according to appendix 7, wherein the process of classifying the data included in the data into different clusters is executed for each group.

(Additional remark 9) The process which classifies the data contained in the said input data into a cluster produces | generates several final cluster data which allocated the cluster at random with respect to the said input data, and each final cluster data and prediction data 6. The analysis method according to appendix 4 or 5, wherein specific final cluster data is selected based on the similarity.

100,200 Analysis device 151,251 Sampling execution unit 152,252 Cluster analysis unit 153,253 Cluster prediction unit 154,254 Determination unit 155,255 Final cluster calculation unit

Claims (5)

A sampling execution unit that performs sampling on input data and repeatedly executes a process of extracting some data from the input data to generate a plurality of sampling data;
A cluster analysis unit that performs cluster analysis on the plurality of sampling data, and classifies the data included in the sampling data into different clusters for each sampling data;
Cluster prediction for generating a plurality of prediction data indicating data obtained by predicting a cluster to which data included in the input data belongs based on a plurality of classification results of the cluster analysis unit and the input data for the plurality of sampling data And
A determination unit that calculates an evaluation value for each prediction data based on the inter-cluster distance and the intra-cluster distance of the prediction data, and determines prediction data corresponding to the evaluation value that is a Pareto solution;
An analysis device comprising: a final cluster calculation unit that classifies data included in the input data into clusters based on prediction data corresponding to the evaluation value that is the Pareto solution.   The final cluster calculation unit is a process of grouping similar prediction data corresponding to evaluation values that are Pareto solutions and classifying the data included in the input data into different clusters based on the prediction data included in the same group The analysis apparatus according to claim 1, wherein the analysis is executed for each group.   The final cluster calculation unit generates a plurality of final cluster data in which clusters are randomly assigned to the input data, and based on the similarity between each final cluster data and predicted data, specific final cluster data The analyzer according to claim 1, wherein the analyzer is selected. An analysis method performed by a computer,
Sampling the input data and repeatedly executing a process of extracting a part of the data from the input data to generate a plurality of sampling data,
Cluster analysis is performed on the plurality of sampling data, and the data included in the sampling data is classified into different clusters for each sampling data,
Based on a plurality of classification results of the cluster analysis unit for the plurality of sampling data and the input data, to generate a plurality of prediction data indicating data predicted clusters to which the data included in the input data belongs,
Based on the inter-cluster distance and intra-cluster distance of the prediction data, calculate an evaluation value for each prediction data, determine the prediction data corresponding to the evaluation value to be a Pareto solution,
An analysis method comprising: performing each process of classifying data included in the input data into clusters based on prediction data corresponding to the evaluation value serving as the Pareto solution. On the computer,
Sampling the input data and repeatedly executing a process of extracting a part of the data from the input data to generate a plurality of sampling data,
Cluster analysis is performed on the plurality of sampling data, and the data included in the sampling data is classified into different clusters for each sampling data,
Based on a plurality of classification results of the cluster analysis unit for the plurality of sampling data and the input data, to generate a plurality of prediction data indicating data predicted clusters to which the data included in the input data belongs,
Based on the inter-cluster distance and intra-cluster distance of the prediction data, calculate an evaluation value for each prediction data, determine the prediction data corresponding to the evaluation value to be a Pareto solution,
An analysis program for executing each process for classifying data included in the input data into clusters based on prediction data corresponding to an evaluation value serving as the Pareto solution.

Images