JP2011034457A - Data mining system, data mining method and data mining program - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a data mining system that can change the granularity of attributes efficiently without causing a user's trial and error. <P>SOLUTION: The data mining system includes a grouping means 102 for selecting an attribute from data including a plurality of attributes and attribute values and grouping the attribute values of the selected attribute according to a pre-stored classification hierarchy hierarchically representing a classification corresponding to the attribute, a test statistic calculation means 103 for calculating a test statistic indicating the strength of relevance of the grouped attribute values to an attribute under analysis, and a determination means 104 for determining whether the grouped attribute values are characteristic in terms of relationship to the attribute under analysis according to the calculated test statistic. If the determination means 104 determines that they are not characteristic, the grouping means 102 executes the grouping again according to a level of classification in the classification hierarchy higher than that used in the preceding grouping out of the classification hierarchy. <P>COPYRIGHT: (C)2011,JPO&INPIT

Description

  The present invention relates to a data mining system, a data mining method, and a data mining program for determining an attribute related to an attribute to be analyzed from data including a plurality of attributes and attribute values.

  In general, a data mining system calculates (extracts) a law of relation between attributes by using data having various attributes as input. For example, the data mining system calculates (extracts) what attributes are related to the success / failure of a sales activity by using, as an input, historical data of customer visits in the sales activity.

  FIG. 1 is an explanatory diagram showing an example of customer visit history data. In the example illustrated in FIG. 1, the history data includes data having attributes of visited place, visited date, visited time, visitor, and success / failure. In addition, "..." in a figure shows abbreviation. When such data is given (input), the data mining system searches (extracts) an attribute closely related to the success / failure attribute from the history data. In order to determine whether there is a relationship between attributes, the data mining system uses, for example, a well-known test method such as t-test.

  However, in order to perform such a test, it is necessary to discretize numerical attributes or group attributes. As an apparatus intended to solve this problem, for example, there is an apparatus described in Patent Document 1.

  In the apparatus described in Patent Document 1, numerical attributes are discretized and attributes are grouped by applying a rule called a procedure file to input data. For example, in the procedure file, “visit time before 8:00, 8:00 to 09:59, 10:00 to 11:59, 12:00 to 14:00, 14:00 to 16:00, 16: “Discrete from 00”, “Distribute destinations to A industry and B Kosan, C Kosan and D company, E company and F company, and other discretization”, etc. Perform grouping.

JP-A-11-250084

  However, the problem with the device described in Patent Document 1 is that it is not efficient. The reason is that a lot of trial and error is required because a good result cannot be obtained by describing only one discretization or grouping for one attribute.

  In the apparatus described in Patent Document 1, for example, “visit time from 8:00 or before, 8:00 to 09:59, 10:00 to 11:59, 12:00 to 14:00, 14:00 After describing the rule of “discretization after 16:00, 16:00”, if a reasonable result cannot be obtained, “visit visit times from 0:00 to 11:59 and from 12:00 to 23:59 It is necessary to rewrite a new rule such as “discretization”. Even if a plurality of rules can be described, the apparatus described in Patent Document 1 needs to perform a test after applying all the rules.

  Accordingly, an object of the present invention is to provide a data mining system, a data mining method, and a data mining program that can efficiently change the granularity of attributes without trial and error by a user in data mining. .

  The data mining system according to the present invention is a data mining system that determines an attribute related to an attribute to be analyzed from data including a plurality of attributes and attribute values, and selects an attribute from the data and corresponds to an attribute stored in advance Grouping means for grouping the attribute values of the selected attributes based on the classification hierarchy representing the classification to be performed, and the strength of the relationship between the attribute values grouped by the grouping means and the attributes to be analyzed A test amount calculating means for calculating a test amount to be displayed, and a determination for determining whether the grouped attribute value is characteristic in relation to the analysis target attribute based on the test amount calculated by the test value calculating means And when the determination means determines that the determination means is not characteristic, the grouping means classifies the classification hierarchy to a higher hierarchy than the lower hierarchy used in the previous grouping. Zui and executes a grouping process again.

  The data mining method according to the present invention is a data mining method for determining an attribute related to an attribute to be analyzed from data including a plurality of attributes and attribute values, and selects an attribute from the data and corresponds to an attribute stored in advance A grouping step that groups the attribute values of the selected attribute based on the classification hierarchy that represents the classification to be performed hierarchically, and a test quantity that indicates the strength of the association between the grouped attribute value and the attribute to be analyzed A test amount calculation step for calculating, and a determination step for determining whether or not the grouped attribute value is characteristic in relation to the attribute to be analyzed based on the calculated test amount. If it is determined that it is not characteristic, the grouping process is executed again based on the classification of the higher hierarchy than the lower hierarchy used in the previous grouping in the classification hierarchy. It is characterized in.

  The data mining program according to the present invention selects an attribute from data and determines a classification corresponding to an attribute stored in advance in a computer that determines an attribute related to the attribute to be analyzed from data including a plurality of attributes and attribute values. A grouping process that groups the attribute values of the selected attributes based on the hierarchical classification, and a test that calculates the test amount that indicates the strength of the relationship between the grouped attribute values and the attributes to be analyzed Based on the calculated amount and the calculated amount, the grouped attribute value is judged to determine whether the grouped attribute value is characteristic in relation to the attribute to be analyzed. If it is determined that it is not appropriate, the grouping process is performed again based on the classification of the higher hierarchy than the lower hierarchy used in the previous grouping in the classification hierarchy. .

  According to the present invention, in data mining, the attribute granularity can be changed efficiently without trial and error by the user.

It is explanatory drawing which shows the example of the historical data of a customer visit. It is a block diagram which shows an example of a structure of the data mining system by this invention. It is a flowchart which shows the process example which a data mining system performs. It is explanatory drawing which shows an example of the classification | category hierarchy corresponding to the attribute which shows visit time. It is explanatory drawing which shows an example of the classification hierarchy corresponding to the attribute which shows a visited place. It is explanatory drawing which shows an example of the classification hierarchy corresponding to the attribute which shows a visitor. It is explanatory drawing which shows an example of the result of a grouping means. It is explanatory drawing which shows an example of the result progress of a test amount calculation means. It is explanatory drawing which shows an example of the result progress of a test amount calculation means. It is explanatory drawing which shows an example of the result of a grouping means. It is explanatory drawing which shows an example of the result of an output means. It is a block diagram which shows an example of a structure of the data mining system in 2nd Embodiment. It is explanatory drawing which shows an example of the classification | category hierarchy for every department corresponding to the attribute which shows a visitor. It is a flowchart which shows the process example which the data mining system in 2nd Embodiment performs. It is a block diagram which shows an example of a structure of the data mining system in 3rd Embodiment. It is explanatory drawing which shows an example of the result of a gathering means. It is a block diagram which shows an example of a structure of the data mining system in 4th Embodiment. It is a flowchart which shows the process example which the data mining system in 4th Embodiment performs. It is explanatory drawing which shows an example of the result of a grouping means. It is explanatory drawing which shows an example of the result progress of a test amount calculation means. It is a block diagram which shows the minimum structural example of a data mining system.

  Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings. FIG. 2 is a block diagram showing an example of the configuration of the data mining system according to the present invention. As shown in FIG. 2, the data mining system according to the first embodiment of the present invention receives a data processing device 1001 that operates according to a program, a data storage device 1002, and a target attribute for which a data group and characteristics are to be determined (data Input means 101 for inputting to the processing apparatus 1001 and output means 105 for outputting a list of attributes related to the target attribute. In the present embodiment, the data mining system is realized by an information processing apparatus such as a personal computer that operates according to a program.

  Specifically, the data processing device 1001 is realized by a CPU of an information processing device that operates according to a program. The data processing apparatus 1001 refers to a classification hierarchy that hierarchically represents classifications corresponding to attributes, and grouping means 102 that groups attribute values, and a verification amount that performs a predetermined test with the data group and the target attribute as inputs. The calculation means 103 and the determination means 104 which determines whether an attribute is characteristic (it is a value which shows that a test quantity is strong relevance) based on a test result.

  Specifically, the data storage device 1002 is realized by a storage device such as a magnetic disk device or an optical disk device. As shown in FIG. 2, the data storage device 1002 includes attribute classification hierarchy storage means 106.

  Each of these means generally operates as follows.

  Specifically, the input unit 101 is realized by an input device such as a keyboard. The input unit 101 has a function of receiving a data group and a target attribute (inputting to the data processing apparatus 1001). The data group includes attributes and attribute values. In addition, the purpose attribute is an attribute that is subject to a test of regularity and relationship with the attribute included in the data group by performing data mining (in this embodiment, such an attribute is included in the data group. It is called a target attribute to distinguish it from the included attributes). For example, when customer visit history data (data group) is input and an attribute related to success / failure is to be obtained, the input unit 101 inputs success / failure as the target attribute (input to the data processing apparatus 1001). ). In the present embodiment, the data mining system performs processing for deriving regularity and relationship between attributes included in the input data group and target attributes by performing data mining.

  Specifically, the attribute classification hierarchy storage means 106 is realized by a storage device such as a magnetic disk device or an optical disk device. The attribute classification hierarchy storage unit 106 stores attributes and classification hierarchies corresponding to the attributes. For example, if the attribute is a visit time, the attribute classification hierarchy storage means 106 classifies the classification hierarchy as AM or PM, and further classifies AM as working hours AM and early morning and stores them. The attribute classification hierarchy storage means 106 stores these classification hierarchies for each attribute. These data (classification hierarchy) may be stored in the attribute classification hierarchy storage means 106 using data created in advance by the user, or stored in the attribute classification hierarchy storage means 106 using publicly available data. May be. For example, when the attribute is a company, a company classification used by the Tokyo Stock Exchange can be used. And if it is a customer visit history as shown in FIG. 1, you may use this company classification | category as a classification hierarchy of a visited place.

  Specifically, the grouping means 102 is realized by a CPU of an information processing apparatus that operates according to a program. The grouping means 102 has a function of referring to the attribute classification hierarchy storage means 106 and grouping input data in order from the lower hierarchy based on the classification hierarchy corresponding to the attribute. Specifically, the grouping unit 102 selects one attribute from the data group input by the input unit 101, and if there is a classification hierarchy corresponding to the selected attribute, the grouping unit 102 is based on the classification of the lowest layer of the classification hierarchy. , Group attribute values. For example, when the grouping means 102 selects a visit time attribute from the customer visit history data group, if the lowest layer of the classification hierarchy corresponding to the visit time attribute is a classification every two hours, the actual visit that is the attribute value Create data with time grouped every two hours.

  When the grouping unit 102 receives (inputs) an instruction to regroup from the determination unit 104, the grouping unit 102 is based on the classification of the hierarchy one level above the classification hierarchy corresponding to the previously grouped attribute. A function for grouping attribute values is provided. Further, when the grouping unit 102 receives (inputs) an instruction to group another attribute from the determination unit 104, the grouping unit 102 selects one other attribute and enters the classification hierarchy corresponding to the selected attribute. Based on this, a function for grouping attribute values is provided.

  Specifically, the test amount calculation means 103 is realized by a CPU of an information processing apparatus that operates according to a program. Based on the data group output from the grouping means 102 and the objective attribute (grouped input data), the test amount calculation means 103 determines the strength of the relevance between the objective attribute and other attributes and attribute values. It has a function to calculate the test amount shown. The test amount calculation means 103 uses, for example, a method such as a general chi-square test when calculating the test amount. For example, when the data group is customer visit history data and the objective attribute is success / failure, the verification amount calculation means 103 is based on the success / failure attribute and the visit time attribute if the chi-square test is used. Create a frequency table. Then, the test amount calculation means 103 performs a chi-square test using the created frequency table.

  Specifically, the determination unit 104 is realized by a CPU of an information processing apparatus that operates according to a program. The determination unit 104 has a function of receiving a test result from the test amount calculation unit 103 and determining whether the test result is characteristic based on a predetermined threshold. If it is determined that it is not characteristic, the determination unit 104 transmits (outputs) to the grouping unit 102 an instruction to perform grouping by raising the hierarchy by one. If it is determined that it is characteristic, the determination unit 104 transmits (outputs) an instruction to group the other attributes to the grouping unit 102. When all the attributes are grouped, the determination unit 104 passes (outputs) the characteristic attribute to the output unit 105.

  Specifically, the output unit 105 is realized by a display device such as a network interface unit or a display device of an information processing apparatus that operates according to a program. The output unit 105 has a function of receiving the attribute list from the determination unit 104 and outputting it. For example, when the output unit 105 is realized by a display device, the output unit 105 displays an attribute list according to an instruction from the determination unit 104. The output unit 105 outputs the attribute list to another data processing device, for example, when realized by a network interface unit.

  Next, the operation of the data mining system will be described with reference to FIGS. FIG. 3 is a flowchart showing an example of processing executed by the data mining system.

  In order to extract information related to a predetermined attribute from the data group, when the user operates the input unit 101 to input the data group and the target attribute, the input unit 101 performs the data group and the target attribute according to the user operation. Are input to the data processing apparatus 1001. Here, an example of a data group input by the input unit 101 to the data processing apparatus 1001 in this embodiment is shown in FIG. The customer visit history data shown in FIG. 1 includes data indicating what kind of customers the sales department employees visited and when the sales activities succeeded or failed. Referring to FIG. 1, the data group is tabular data having attributes of visited place, visited date, visited time, visitor, and success / failure. Each value in the table shown in FIG. 1 indicates an attribute value for the attribute. As shown in FIG. 1, an identifier ID is written (applied) to each data. The data of ID1 represents that the attribute value of the attribute “visit destination” is “A industry”. In the present embodiment, the input unit 101 inputs “success / failure” as the target attribute. Thus, the user can know which behavior is closely related to the success or failure of the sales activity.

  When the input unit 101 inputs a data group and a target attribute to the data processing apparatus 1001 in accordance with a user input operation, the grouping unit 102 refers to the input data group and selects one attribute to be grouped. (Step S11 in FIG. 3). The grouping means 102 performs attribute selection at random. For example, the grouping means 102 selects a visit time from each attribute included in the data group shown in FIG.

  Next, the grouping means 102 refers to the attribute classification hierarchy storage means 106 and acquires (extracts) a classification hierarchy corresponding to the selected attribute (step S12 in FIG. 3).

  Examples of classification hierarchies stored in the attribute classification hierarchy storage unit 106 are shown in FIGS. In the present embodiment, it is assumed that the classification hierarchy storage unit 106 stores all of these classification hierarchies.

  FIG. 4 is an explanatory diagram illustrating an example of a classification hierarchy corresponding to an attribute indicating a visit time. Referring to FIG. 4, the visit time is classified into morning and afternoon. Furthermore, morning is classified into early morning and working hours in the morning, and afternoon is classified into lunch break, working hours in the afternoon and late at night. Furthermore, the working hours in the morning are classified into 8: 00-10: 00 and 10: 00-12: 00.

  FIG. 5 is an explanatory diagram illustrating an example of a classification hierarchy corresponding to an attribute indicating a visited place. In addition, "..." in a figure represents abbreviation. FIG. 5 shows a classification of visited companies. As such a company classification, a public company classification can be used. For example, the example shown in FIG. 5 is an example extracted from the classification of the list of Japanese companies in Wikipedia widely released on the Internet.

  FIG. 6 is an explanatory diagram illustrating an example of a classification hierarchy corresponding to an attribute indicating a visitor. In the figure, "..." represents omission. FIG. 6 shows the job classification of visitors. Such classification can be obtained from, for example, a company personnel database.

  In the present embodiment, in step S12, the grouping unit 102 selects (extracts) the classification hierarchy corresponding to the visit time shown in FIG. 4 from the attribute classification hierarchy storage unit 106.

  Next, the grouping unit 102 groups attribute values based on the classification of the hierarchy that is the current processing target among the selected (extracted) classification hierarchies (step S13 in FIG. 3). Here, immediately after the classification hierarchy corresponding to the attribute is selected, the lowest hierarchy among the classification hierarchies becomes the current processing target hierarchy. The grouping means 102 groups the data group (attribute value of the attribute selected in step S11) based on this hierarchy classification.

  FIG. 7 shows an example in which the data group shown in FIG. 1 is grouped. Referring to FIG. 7, it can be seen that the attribute value of the attribute indicating the visit time has changed. The grouping means 102 groups attribute values based on the classification for every two hours, which is the lowest hierarchy among the classification hierarchies corresponding to the attribute indicating the visit time. For example, the attribute value of the attribute indicating the visit time of the data ID 1 has been replaced with a group from “11:30” to “10: 00-12: 00”. In addition, the attribute value of the attribute indicating the visit time of the data ID2 is in a group of “13:10” to “13: 00-15: 00” and the attribute value of the attribute indicating the visit time of the data ID3 is “13: 20 ”to“ 13: 00-15: 00 ”groups.

  Next, the test amount calculation unit 103 calculates a test amount between the target attribute and other attributes or grouped attribute values based on the grouped data output from the grouping unit 102 and the target attribute. . The test amount calculation means 103 uses an existing method that can measure whether or not the relationship between two variables is strong in calculating the test amount. Existing methods include, for example, chi-square test and Fisher's exact test, information gain ratio often used in decision trees, and support and confidence used in association rules.

  Here, an example in which the test amount calculation means 103 uses a chi-square test in calculating the test amount will be described. First, when the chi-square test is used, the test amount calculation unit 103 creates a frequency table based on the objective attribute and the grouping target attribute. FIG. 8 shows an example of a frequency table. In the example illustrated in FIG. 8, the attribute values “success” and “failure” of the success / failure attribute that are the target attributes are displayed horizontally, and one attribute value “ It is a 2 × 2 frequency table that takes “12: 00-13: 00” and “other than 12: 00-13: 00”. The value in the table is the number of data that matches the condition. The test amount calculation means 103 performs the following chi-square test using this frequency table.

Chi-square = Σ {(F (i, j) −D (i, j)) ² / F (i, j)},
D (i, j): value of i row j column of the frequency table,
F (i, j): a (i) × b (j) / N, N: total frequency,
a (i): total value of i rows,
b (j): Total value of j columns

  When applied to this equation, it can be calculated as follows.

Chi-square test = (250 × 60 / 1260−50) ² / (250 × 60/1260) + (1010 × 60 / 1260−10) ² / (1010 × 60/1260) + (250 × 1200 / 1260−200) ) ² / (250 × 1200/1260) + (1010 × 1200 / 1260−1000) ² /(1010×1200/1260)=121.9047619+30.17444602+6.095238095+1.508722301=159.68316383

  The value calculated as described above is a verification amount relating to success / failure of sales activities (objective attribute) and whether or not the visit time is 12: 00-13: 00 (grouped attribute value). Similarly, the test amount calculation means 103 calculates a test amount for each of the other attribute values grouped. When the test amount is calculated for all the grouped attribute values, the test amount calculation unit 103 passes (outputs) a list including each grouped attribute value and the test amount to the determination unit 104.

  Further, as shown in FIG. 9, the test amount calculation means 103 creates a frequency table based on the attribute values of success / failure and visit time. Based on this, processing is performed in the same manner as described above, so that the test amount calculation means 103 calculates a test amount related to the summation of visit times every two hours (a specific layer in the classification layer) and success / failure (target attribute). To do. In this case, the test amount calculation unit 103 passes (outputs) the hierarchy and the test amount to the determination unit 104.

  Next, the determination unit 104 determines whether or not the output verification amount is characteristic (step S15 in FIG. 3). The determination means 104 determines whether or not the test amount is characteristic based on a predetermined threshold. Then, the determination unit 104 causes the data processing device to store characteristic grouped attribute values in the list including the grouped attribute values and the test amount. If all grouped attribute values are characteristic, the determination unit 104 determines that the output hierarchy is characteristic in relation to the target attribute.

  When it is determined in step S15 that there is an attribute value that is not characteristic (that is, even one attribute value grouped in the processing target hierarchy), the determination unit 104 proceeds to step S16. To do. On the other hand, if it is determined in step S15 that all grouped attribute values are characteristic, the determination unit 104 shifts the process to step S17.

  Here, in determining whether or not it is characteristic, for example, in the case of a chi-square test amount, a significance level of 5% is set as a threshold value. In the case of the example shown in FIG. 8, since the degree of freedom is 1, if the test amount is 3.84 or more, the determination unit 104 rejects the null hypothesis (the two attributes are independent), and the two attributes Is determined to be characteristic because it is not independent. In the case of the example shown in FIG. 8, since the verification amount exceeds this, the determination means 104 determines that it is characteristic. The determination unit 104 also determines other grouped attribute values, and if there is any grouped attribute value that is not characteristic, the process proceeds to step S16.

  If it is determined in step S15 that there is a non-characteristic attribute, the determination unit 104 outputs an instruction to the grouping unit 102 to group the processing target classification hierarchy up (step S15). S16). In the present embodiment, the determination unit 104 outputs an instruction to the grouping unit 102 to group the attribute values of the visit time based on the business time hierarchy classification.

  When the determination unit outputs the above instruction in step S16, the grouping unit 102 groups the attribute values based on the classification of the current layer (that is, the layer one level higher than the previous layer) (FIG. 3). Step S13). In the present embodiment, the grouping means 102 groups the visit time attribute values based on the business time hierarchy classification. An example of the result of the grouping process here is shown in FIG. Referring to FIG. 10, it can be seen that the attribute value of the visit time is changed to “working time am”, “working time afternoon”, or the like, as compared with the example shown in FIG.

  Next, the test amount calculation means 103 again calculates the test amount of the target attribute and other attributes and attribute values after grouping (step S14). Then, the determination unit 104 determines whether the grouped attribute values are characteristic based on the test amount (step S15). As a result of the determination, if all the attributes are characteristic, the determination unit 104 shifts the process to step S17. On the other hand, if it is determined that all the attributes are not characteristic, the determination unit 104 shifts the process to step S16 again. Thereafter, in step S15, the data processing device 1001 repeats the processing of steps S13 to S16 until the determination unit 104 determines that all attributes are characteristic.

  Next, the determination unit 104 determines whether or not determination has been made for all attributes of the input data group (step S17). And when it determines with there being an undetermined attribute, the determination means 104 transfers a process to step S11. Thereafter, in step S17, the data processing device 1001 repeats the processing of steps S11 to S17 until the determination unit 104 determines that determination has been made for all attributes.

  On the other hand, if it is determined in step S17 that the determination for all the attributes has been completed, the output unit 105 displays the characteristic attributes stored in the data processing device 1001 and the verification amount thereof according to the instruction of the determination unit 104. The data is output to the device (step S18). An example of the attribute list output by the output means 105 and the test amount is shown in FIG. For example, as shown in FIG. 11, the output unit 105 outputs a list including the attributes determined to be characteristic and their test amounts.

  Next, the effect of this embodiment will be described. In the present embodiment, the granularity of the attribute can be changed efficiently without trial and error by the user. This is because the grouping means performs grouping in order from the lower hierarchy based on the attribute classification hierarchy, and controls the grouping process to stop when the determination means determines that it is a characteristic hierarchy.

  In the present embodiment, in the data mining, the granularity of the attribute to be processed can be changed without using a rule. The reason is that the grouping means groups in order from the lower hierarchy based on the attribute classification hierarchy stored in the attribute classification hierarchy storage means 106.

Embodiment 2. FIG.
Next, a second embodiment of the present invention will be described in detail with reference to the drawings. Referring to FIG. 12, the data mining system according to the second exemplary embodiment of the present invention includes a second grouping unit 202 instead of the grouping unit 102 and replaces the determination unit 104 as compared with the first exemplary embodiment. The second determination unit 204 is different.

  Hereinafter, differences from the first embodiment will be mainly described. Specifically, the second grouping unit 202 is realized by a CPU of an information processing apparatus that operates according to a program. The second grouping means 202 has a function of referring to the attribute classification hierarchy storage means 106 and grouping input data in order from the lower hierarchy of the classification hierarchy. Specifically, when the second grouping unit 202 receives (inputs) a data group from the input unit 101, the second grouping unit 202 selects one attribute from the data group and has a classification hierarchy corresponding to the selected attribute. For example, attribute values are grouped based on the classification of the lowest layer of the classification hierarchy. When there are a plurality of classification hierarchies corresponding to the attribute, the second grouping means 202 selects any one.

  For example, when the classification hierarchy shown in FIG. 6 and the classification hierarchy shown in FIG. 13 correspond to the visitor attribute of the customer visit history data, the second grouping means 202 selects either classification hierarchy. FIG. 13 is an explanatory diagram showing a classification hierarchy for each department corresponding to an attribute indicating a visitor.

  When the second grouping unit 202 receives (inputs) an instruction to execute regrouping from the second determination unit 204, the second grouping unit 202 determines the level of the processing target in the previous grouping. A grouping function is provided based on the classification of the hierarchy one level above. Further, upon receiving (inputting) an instruction to select another classification hierarchy from the second determination means 204, the second grouping means 202 selects a classification hierarchy other than the classification hierarchy currently being processed. It has a function. Furthermore, when the second grouping unit 202 receives (inputs) an instruction to execute grouping with another attribute from the second determination unit 204, the second grouping unit 202 selects one other attribute from the data group. And a function for grouping attribute values included in the selected attribute.

  Specifically, the second determination unit 204 is realized by a CPU of an information processing apparatus that operates according to a program. The second determination unit 204 has a function of determining whether or not the test result is characteristic based on a predetermined threshold when receiving the test result from the test amount calculation unit 103 (input). When it is determined that the test result is not characteristic, the second determination unit 204 transmits (outputs) an instruction to the second grouping unit 202 to raise the hierarchy by one and perform grouping. On the other hand, if it is determined that the test result is characteristic, the second determination unit 204 instructs the second group to perform grouping based on the other classification hierarchy corresponding to the attribute, if any. Is transmitted (output) to the conversion means 202. If there is no other classification hierarchy corresponding to the attribute, the second determination unit 204 transmits (outputs) an instruction to group the other attribute to the second grouping unit 202. When determining that all attributes have been grouped, the second determination unit 204 passes (outputs) the attribute determined to be characteristic to the output unit 105.

  Next, with reference to FIG. 12 and FIG. 14, the operation of the data mining system in the present embodiment will be described in detail mainly for parts different from the first embodiment. FIG. 14 is a flowchart illustrating an example of processing executed by the data mining system according to the second embodiment.

  The difference from the first embodiment is that step S21 for determining whether there is another classification hierarchy corresponding to the attribute is added.

  In step S <b> 21, the second determination unit 204 determines whether there is another classification hierarchy corresponding to the attribute in the classification hierarchy stored in the attribute classification hierarchy storage unit 106. If it is determined that there is, the second determination unit 204 proceeds to step 12 and outputs an instruction to the second grouping unit 202 to perform grouping based on another classification hierarchy. To do. On the other hand, if it is determined that there is no attribute, the second determination unit 204 moves the process to step S17 and determines whether or not there is an undetermined attribute as in the first embodiment.

  In step S21, when the second determination unit 204 outputs an instruction to perform grouping based on another classification hierarchy, in step S12, the second grouping unit 202 determines that the classification hierarchy that is the current processing target. Select a different classification hierarchy. After that, the second determination unit 204 performs grouping of attribute values based on the selected classification hierarchy, similarly to the processing in the first embodiment.

  As described above, in the present embodiment, when a plurality of classification hierarchies correspond to one attribute, the second grouping unit selects any one of the classification hierarchies. Then, the second determination unit outputs an instruction to select an unadopted classification hierarchy to the second grouping unit. Then, the second grouping unit executes the grouping process based on another classification hierarchy according to the instruction output from the second determination unit. Therefore, even if a plurality of classification hierarchies correspond, it can be determined whether or not all the classification hierarchies are characteristic.

Embodiment 3. FIG.
Next, a third embodiment of the present invention will be described in detail with reference to the drawings. Referring to FIG. 15, the data mining system according to the third exemplary embodiment of the present invention is different from the first exemplary embodiment in that a collecting unit 301 is additionally included. Hereinafter, differences from the first embodiment will be mainly described.

  Specifically, the bundling unit 301 is realized by a CPU of an information processing apparatus that operates according to a program. When the attributes to be output are ranked based on the test amount, the grouping unit 301 has all the classifications that are in the sibling relationship (same hierarchy) in the classification hierarchy with a close rank or a test amount with a small difference. In such a case, a function is provided for outputting these to the output means 105 as a classification of the parent hierarchy. Thus, in the present embodiment, to summarize means to replace a plurality of classifications having a predetermined relationship with one attribute.

  The gathering unit 301 determines whether the ranking is near or whether the difference is a small test amount based on a predetermined threshold. For example, let us consider a case where the attributes shown in FIG. 11 and the ranking of the test amount are input, and the threshold value is within 3rd order. Referring to the classification hierarchy, the attributes of the ranking 3 and the ranking 4 are “visiting time 13: 00-15: 00” and “visiting time 15: 00-17: 00” and are in a sibling relationship, It is a classification that has all the sibling relationships that belong to the lower hierarchy of “visit time working hours afternoon”. Therefore, the summarizing means 301 puts these two together as “visit time working hour afternoon”. At this time, the summarizing means 301 employs a test rank having a high rank (that is, a rank 3 test value 23.4 shown in FIG. 11). The method for determining the test amount is not limited to this, and for example, an average value or a method of writing two values together may be used.

  As described above, in the present embodiment, the grouping unit 301 collects the characteristic attribute lists to be output that are close in the classification hierarchy. Therefore, it is possible to output a classification that is easy for the user to see.

Embodiment 4 FIG.
Next, a fourth embodiment of the present invention will be described in detail with reference to the drawings. Referring to FIG. 17, the data mining system according to the fourth exemplary embodiment of the present invention is different from the first exemplary embodiment in that a combination unit 401 is additionally included.

  Specifically, the combination means 401 is realized by a CPU of an information processing apparatus that operates according to a program. The combination unit 401 has a function of creating a combination of any two or more attributes in the data group when the data group and the target attribute are input from the input unit 101. The combination unit 401 has a function of passing (outputting) the data group, the target attribute, and a combination of attributes other than the target attribute to the grouping unit 102.

  The grouping unit 102 has a function of grouping attribute values of attributes when a data group, a target attribute, and a combination of attributes other than the target attribute are output from the combining unit 401.

  Next, with reference to FIG. 17 and FIG. 18, the operation of the data mining system in the present embodiment will be described in detail mainly on the differences from the first embodiment. FIG. 18 is a flowchart illustrating an example of processing executed by the data mining system according to the fourth embodiment.

  The combination unit 401 refers to the data group and selects a combination of two or more attributes from attributes other than the target attribute (step S41 shown in FIG. 18). For example, when the data group shown in FIG. 1 is input, the combination unit 401 selects a visited place and a visited time as a combination of attributes other than the objective attribute.

  Next, the grouping means 102 acquires (extracts) the attribute classification hierarchy (step S12 shown in FIG. 18).

  Next, the grouping means 102 groups the attribute values based on the classification of the hierarchy that is the current processing target (step S13 shown in FIG. 18). For example, the data group is as shown in FIG. 1, the combination of attributes other than the objective attribute is the visit destination and the visit time, and the classification hierarchy corresponding to the visit time is as shown in FIG. 4, corresponding to the visit destination. Assume that the classification hierarchy is as shown in FIG. In this case, the grouping means 102 refers to the classification hierarchy corresponding to the visit time and the visit destination, and groups the attribute values based on the lowest classification of the corresponding classification hierarchy as shown in FIG. . Referring to FIG. 19, it can be seen that the attribute values of the visited place and the visited time are grouped as compared with FIG. Referring to the data in the first row, the visited place is converted from the A industry to the food category group, and the visiting time is converted from 11:10 to 10: 00-12: 00.

  Next, the test amount calculation means 103 calculates a test amount (S14 in FIG. 18). For example, when the chi-square test is used, the test amount calculation unit 103 creates a frequency table with combinations of attributes. An example of the frequency table created by the test amount calculation means is shown in FIG. FIG. 20 is a frequency table when the visit destination is food and the visit time is 12: 00-13: 00. Other operations are the same as those in the first embodiment.

  As described above, in this embodiment, in addition to the effects of the first embodiment, more detailed analysis can be realized by further considering the combination of attributes.

  Next, the minimum configuration of the data mining system according to the present invention will be described. FIG. 21 is a block diagram illustrating a minimum configuration example of the data mining system. As shown in FIG. 21, the data mining system includes a grouping unit 102, a test amount calculation unit 103, and a determination unit 104 as minimum components.

  In the data mining system with the minimum configuration shown in FIG. 21, the grouping unit 102 selects an attribute from data including a plurality of attributes and attribute values, and classifies the classification corresponding to the attribute stored in the storage unit in a hierarchical manner. Group attribute values of selected attributes based on hierarchy. Then, the test amount calculation unit 103 calculates the test value indicating the attribute value grouped by the grouping unit 102 and the strength of relevance between the selected attribute and the attribute to be analyzed. Then, the determination unit 104 determines whether the grouped attribute value and the selected attribute are characteristic of the analysis target attribute based on the test amount calculated by the test value calculation unit 103. Then, when the determination unit 104 determines that it is not characteristic, the grouping unit 102 executes the grouping process again based on the classification of the hierarchy higher than the previous one among the classification hierarchies.

  Therefore, according to the data mining system having the minimum configuration shown in FIG. 21, in the data mining, the analysis granularity can be changed efficiently without trial and error by the user.

  In this embodiment, the characteristic configuration of the data mining system as shown in the following (1) to (6) is shown.

  (1) The data mining system relates to an analysis target attribute (for example, success / failure illustrated in FIG. 1) from data including a plurality of attributes and attribute values (for example, customer visit history data illustrated in FIG. 1). A data mining system for determining an attribute, wherein an attribute is selected from data and stored in advance (for example, realized by the attribute classification hierarchy storage means 106) based on a classification hierarchy that hierarchically represents a classification corresponding to the attribute Grouping means for grouping the attribute values of the selected attributes (for example, realized by the grouping means 102), and the strength of relevance between the attribute value grouped by the grouping means and the attribute to be analyzed Based on the test amount calculated by the test value calculation unit (for example, realized by the test amount calculation unit 103) and the test value calculation unit that calculates the test amount shown Determining means (for example, realized by the determining means 104) for determining whether or not the grouped attribute values are characteristic in relation to the attribute to be analyzed. If it is determined that it is not characteristic, the grouping process is performed again based on the classification of the classification hierarchy higher than the hierarchy used in the previous grouping (for example, the business time shown in FIG. 4). And

  (2) In the data mining system, when the grouping unit determines that all grouped attribute values are characteristic of the analysis target attribute, the grouping unit selects other attributes from the data and groups them You may be comprised so that a process may be performed again.

  (3) A data mining system is a data mining system that receives data including a plurality of attributes and attribute values and attributes to be analyzed and outputs a list of attributes related to the attributes to be analyzed. Attribute classification hierarchy storage means (for example, realized by the attribute classification hierarchy storage means 106) that stores the classification hierarchy that represents the classification to be performed, and one attribute to be compared is selected from the data, and the selected attribute Grouping means (for example, grouping attribute values corresponding to the attributes to be compared among the data based on the classification of the lower hierarchy of the extracted classification hierarchy) And at least the attribute to be analyzed and the attribute to be compared, or the attribute to be analyzed and the group The attribute to be analyzed and the attribute to be compared, or the attribute value of the attribute to be analyzed and the attribute to be compared with Whether or not the test amount calculated by the test amount calculation means (for example, realized by the test amount calculation means 103) that calculates the test amount indicating the strength of the relationship exceeds a specific threshold Based on the classification of the hierarchy one level above the classification hierarchy in the grouping means when it is determined whether there is a test result indicating a non-characteristic attribute or a non-characteristic attribute value. Grouping process, and when it is determined that all the comparison target attributes or attribute value test results are characteristic, the grouping means does not change the classification hierarchy and assigns another undecided attribute. Let all select Characteristic list and test amounts of attributes and outputs the determination means (e.g., as implemented by the decision means 104), characterized in that it comprises a.

  (4) In the data mining system, the attribute classification hierarchy storage unit stores a classification hierarchy corresponding to one attribute, and the grouping unit (for example, realized by the second grouping unit 202) corresponds to the attribute. One of the classification hierarchies to be selected is selected from the attribute classification hierarchy storage unit, and the determination unit (for example, realized by the second determination unit 204) determines that all the attributes to be compared or the attribute result of the attribute are tested. If there is another classification hierarchy corresponding to the attribute to be compared and determined to be characteristic, the grouping unit may be configured to select an unselected classification hierarchy.

  (5) In the data mining system, a grouping unit (for example, the grouping unit 301 is realized by combining attributes satisfying a predetermined condition into one attribute by inputting a list of characteristic attributes and a test amount. The summarizing means includes, as a predetermined condition, a plurality of different attributes included within a predetermined rank range as a result of ranking in the order of the test amount and within a range of the predetermined test amount. And when these attributes are sibling hierarchies in the corresponding classification hierarchy and share the same parent classification, the parent classification is summarized as a new characteristic attribute. It may be configured to output.

  (6) The data mining system may include a combination unit (for example, realized by the combination unit 401) that generates a combination of attributes to be grouped by the grouping unit.

  The present invention can be applied to a usage analysis using a customer visit history. Moreover, it is applicable also to the use which analyzes sales goods using sales history data.

DESCRIPTION OF SYMBOLS 101 Input means 102 Grouping means 103 Test amount calculation means 104 Judgment means 105 Output means 106 Attribute classification hierarchy storage means 202 2nd grouping means 204 2nd determination means 301 Summarizing means 401 Combining means 1001 Data processing apparatus 1002 Data Storage device

Claims (14)

A data mining system for determining an attribute related to an attribute to be analyzed from data including a plurality of attributes and attribute values,
Grouping means for selecting an attribute from the data and grouping attribute values of the selected attribute based on a classification hierarchy that hierarchically represents a classification corresponding to the attribute stored in advance;
A test amount calculating means for calculating a test amount indicating the strength of relevance between the attribute value grouped by the grouping means and the attribute to be analyzed;
Determination means for determining whether the grouped attribute values are characteristic in relation to the analysis target attribute based on the test amount calculated by the test value calculation means,
When the grouping unit determines that the determination unit is not characteristic, the grouping unit executes the grouping process again based on a classification of a hierarchy higher than the hierarchy used in the previous grouping among the classification hierarchies. A featured data mining system. The grouping unit selects another attribute from the data and executes the grouping process again when the determining unit determines that all grouped attribute values are characteristic in relation to the attribute to be analyzed. 1. A data mining system according to 1. A data mining system that outputs data including a plurality of attributes and attribute values and an attribute to be analyzed, and outputs a list of attributes related to the attribute to be analyzed,
Attribute classification hierarchy storage means for storing a classification hierarchy representatively representing the classification corresponding to the attribute;
From the data, select one attribute to be compared, extract the classification hierarchy corresponding to the selected attribute from the attribute classification hierarchy storage means, based on the classification of the lower hierarchy of the extracted classification hierarchy, the Grouping means for grouping attribute values corresponding to the attributes to be compared in the data;
At least one of the analysis target attribute and the comparison target attribute, or the analysis target attribute and all the attribute values of the comparison target attribute grouped by the grouping unit, the analysis target attribute A test amount calculating means for calculating a test amount indicating the strength of the relationship between the attribute and the attribute to be compared, or the attribute value of the attribute to be analyzed and the attribute to be compared;
It is determined whether or not the test amount calculated by the test amount calculation means is characteristic depending on whether or not it exceeds a specific threshold, and it is determined that there is a test result indicating an uncharacteristic attribute or an uncharacteristic attribute value. In the case where the grouping means is caused to execute a grouping process based on the classification of the hierarchy one level above the classification hierarchy, and it is determined that all comparison target attributes or attribute value test results are characteristic In addition, the grouping means includes selecting means for selecting another undecided attribute without changing the classification hierarchy, and outputting a list of all characteristic attributes and the test amount. Mining system. The attribute classification hierarchy storage means stores a classification hierarchy corresponding to one attribute,
The grouping means selects one of the classification hierarchies corresponding to the attribute from the attribute classification hierarchy storage means,
The determination unit determines that all the attributes to be compared or the test result of the attribute value of the attribute is characteristic, and if there is another classification hierarchy corresponding to the attribute to be compared, the grouping unit The data mining system according to claim 3, wherein an unselected classification hierarchy is selected. Including a list of characteristic attributes and a test amount as input, and a grouping unit that combines attributes satisfying a predetermined condition into one attribute,
The summarizing means, as a predetermined condition, as a result of ranking in the order of the test amount, as a result, a plurality of different attributes are included within a predetermined rank range and at least included in the range of the predetermined test amount When any one of these attributes is satisfied and these attributes are sibling hierarchies in the corresponding classification hierarchy and share the same parent classification, the parent classification is collected and output as a new characteristic attribute. Item 5. The data mining system according to item 3 or claim 4. The data mining system according to any one of claims 3 to 5, further comprising a combination unit that generates a combination of attributes to be grouped by the grouping unit. A data mining method for identifying an attribute related to an attribute to be analyzed from data including a plurality of attributes and attribute values,
A grouping step of selecting an attribute from the data and grouping attribute values of the selected attribute based on a classification hierarchy that hierarchically represents a classification corresponding to the attribute stored in a storage unit;
A test amount calculation step of calculating a test amount indicating the strength of the relationship between the grouped attribute value and the attribute to be analyzed;
A determination step of determining whether or not the grouped attribute values are characteristic in relation to the analysis target attribute based on the calculated verification amount;
If it is determined in the grouping step that the characteristic is not characteristic in the determination step, a grouping process is executed again based on a classification of a hierarchy higher than the previous one of the classification hierarchies. The data according to claim 7, wherein in the grouping step, when it is determined that all the grouped attribute values are characteristic in relation to the attribute to be analyzed, another attribute is selected from the data and the grouping process is executed. Mining method. A data mining method for inputting data including a plurality of attributes and attribute values and an attribute to be analyzed, and outputting a list of attributes related to the attribute to be analyzed,
A storage step of storing in the storage unit a classification hierarchy that hierarchically represents the classification corresponding to the attribute;
A selection step of selecting one attribute to be compared from the data;
An extraction step of extracting the classification hierarchy corresponding to the selected attribute from the storage unit;
A grouping step of grouping attribute values corresponding to attributes of the data based on the classification of the lower hierarchy of the extracted classification hierarchy;
The analysis target attribute and the comparison target for at least one of the attribute of the analysis target and the attribute of the comparison target, or all attribute values of the attribute of the previous comparison target grouped with the analysis target attribute Or a test amount calculation step for calculating a test amount indicating the strength of the relationship between the attribute to be analyzed and the attribute value of the attribute to be compared;
The classification hierarchy is determined when it is determined whether there is a test result indicating a non-characteristic attribute or a non-characteristic attribute value by determining whether the calculated test amount exceeds a specific threshold or not. A regrouping step for grouping based on a hierarchy above
When it is determined that the test results of all the comparison target attributes or attribute values are characteristic, another undecided attribute is selected without changing the classification hierarchy, and all characteristic attribute lists and tests are performed. A data mining method comprising: an output step for outputting the quantity. In the storing step, the classification hierarchy corresponding to one attribute is stored,
In the extraction step, one of the classification hierarchies corresponding to the attribute is extracted,
When it is determined that all the attributes to be compared or the test result of the attribute value of the attribute is characteristic, and there is another classification hierarchy corresponding to the attribute to be compared, an unselected classification hierarchy is The data mining method according to claim 9, further comprising a reselecting step of selecting. Including a list of characteristic attributes and a test amount as input, and a step of combining attributes that satisfy a predetermined condition into one attribute,
In the summarizing step, as a predetermined condition, as a result of ranking in the order of the test amount, any of a plurality of different attributes is included within a predetermined rank range or included in a range of a predetermined test amount When one or both of them are satisfied and these attributes are sibling hierarchies in the corresponding classification hierarchy and share the same parent classification, the parent classification is collected and output as a new characteristic attribute. The data mining method according to claim 9 or claim 10. The data mining method according to claim 9, further comprising a combination step of generating a combination of attributes to be grouped. To identify attributes related to the attribute being analyzed from data containing multiple attributes and attribute values,
A grouping process for selecting an attribute from the data and grouping attribute values of the selected attribute based on a classification hierarchy that hierarchically represents a classification corresponding to the attribute stored in the storage unit;
A test amount calculation process for calculating a test amount indicating the strength of relevance between the grouped attribute value and the attribute to be analyzed;
A determination process for determining whether or not the grouped attribute values are characteristic in relation to the analysis target attribute based on the calculated verification amount;
A data mining program for causing a grouping process to be executed again based on a classification of a hierarchy higher than the previous one of the classification hierarchies when it is determined that the grouping process is not characteristic in the determination process. On the computer,
14. The data according to claim 13, wherein when the grouping process determines that all grouped attribute values are characteristic in relation to the attribute to be analyzed, the grouping process is performed by selecting another attribute from the data. Mining program.

Images