JP2008299382A - Data division program, recording medium with the program recorded thereon, data distribution device and data distribution method - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To automatically distribute multi-dimensional data into proper groups matched with an analytic purpose in distributing the multi-dimensional data based on items having a hierarchical structure, and to present characteristics considered by a user to the distributed groups. <P>SOLUTION: A data distribution device 100 is configured: to execute input processing for accepting the input of explanation variables and target variables by an input part 201; and to execute decision processing by a decision part 202 and distribution processing by a distribution part 203 (decision/distribution processing), and to, when hierarchical data and a decision table as distribution processing results are acquired, execute integration processing by an integration part 204, and to, when the hierarchical structure data and the decision table as the integration processing results are acquired by the integration part 204, output the integration processing results by an output part 205. <P>COPYRIGHT: (C)2009,JPO&INPIT

Description

  The present invention relates to a data division program that divides a multidimensional data group having explanatory variables representing hierarchies and target variables that can represent distribution bias, a recording medium on which the program is recorded, a data division apparatus, and a data division method About.

  Conventionally, in the analysis of multidimensional data, the task of analyzing the relationship between items has been widely performed. In many cases, the one or more items have a hierarchical structure. For example, in patent gazette data, a classification code having a hierarchy called an IPC code is given together with an item having no hierarchy such as an applicant.

  In this patent publication data analysis, a task of analyzing the relationship between the applicant and the IPC code is assumed (prior art 1). The IPC code is obtained by encoding the technical content of the patent publication filed. By analyzing the relationship between the IPC code and the applicant, the technical strength / weakness of each applicant can be grasped, so this analysis is widely performed.

  For example, if a patent publication having a certain IPC code is found to have been filed in large quantities by a certain applicant, it can be understood that the applicant is strong in the technical field represented by the IPC code. Can devise an appropriate strategy.

  As this analysis method, for example, patent gazette data is divided by an IPC code, and the distribution of applicants in each group is observed. Specifically, by creating a graph of the number of applications by applicant with the IPC code on the horizontal axis and the applicant on the vertical axis, the IPC code where a specific applicant accounts for the majority of applications, Information such as IPC codes that have been applied for evenly by humans can be read.

  In the above description, items having a hierarchy such as “IPC code” are referred to as “explanatory variables”. On the other hand, items such as “applicant” who want to know the distribution bias are called “objective variables”. In particular, a fixed value other than a numerical value such as “applicant” is referred to as a fixed value.

  Further, among the known data division techniques, the following patent document 1 (conventional technique 2) can be cited as something similar to the above-mentioned viewpoint. This prior art 2 determines whether the difference in the distribution of the objective variable is significant with respect to the divided results using a statistical test, and integrates groups having no significant difference. By using this technique, it is ensured that the original division results are integrated into several groups, and that there is a statistically significant difference in the distribution of objective variables between the created groups.

  In addition, as a well-known division technique for hierarchical data, conventional technique 3 for creating groups with uniform granularity by dividing the numbers belonging to each group so as to fall within a certain range (for example, Patent Document 2 below) See).

  Also, there is an analysis for selecting an item that takes numerical information (continuous value or discrete value) that can be ordered, instead of an item that takes a fixed value such as “Applicant” in the patent gazette data of Prior Art 1 as an objective variable (prior art) 4).

  For example, in the task of analyzing the relationship between the number of sales of a certain product and other items using POS data as analysis target data, the objective variable of interest is the number of sales. As a result of this analysis, for example, if it is found that there is a large difference in the number of sales of product A between weekdays and holidays, it is possible to make use of determining the appropriate number of purchases according to the day of the week.

  However, there are usually variations in daily sales. Therefore, useful knowledge cannot be obtained simply by comparing individual daily sales data. Therefore, it is necessary to group individual data and perform statistical analysis, for example, under the assumption that the number of sales follows a Poisson distribution under exactly the same conditions (this assumption is valid).

  In addition, instead of adopting items described in the data as explanatory variables with a hierarchical structure, by applying a technique called hierarchical clustering to free description items (text description items) in the data, There is an analysis that creates a hierarchical structure of data and uses the hierarchical structure as an explanatory variable (prior art 5).

  For example, in a monitor questionnaire on a product, when analyzing data consisting only of items in which one record freely describes the gender and impressions about the product, what is the target product due to gender differences? An analysis that wants to know the difference.

  In this analysis, create a hierarchical structure based on the similarity of the description contents of the free description items, and divide it into groups at an appropriate hierarchy.For example, a group with many impressions focusing on product design has a high percentage of women, A group with a lot of feedback focusing on usability gains knowledge that there are many male ratios.

  As a method of dividing the result of the hierarchical clustering, a method of dividing the data with the same granularity so as to suppress the difference in the number of data of the divided results as much as possible, or a method of dividing by a portion having a low similarity calculated at the time of clustering. Such a method is general and widely used (prior art 6).

JP 2004-157814 A Japanese Patent Laid-Open No. 10-124528

  However, in the above-described prior art 1, since the IPC code has a hierarchy, it is unclear which layer is used for the horizontal axis of the above-mentioned graph and the information to be read appears in the graph. As for the entire data, even if it is applied evenly by each applicant, when it is clearly biased when it is divided into one layer of the IPC code, the layer where such characteristics can be seen is the data Therefore, it is not possible to determine an appropriate hierarchy before creating the graph.

  Therefore, the analyst had to create a graph for each hierarchy on a trial and error basis and examine the entire graph. Considering the original analysis purpose, it is meaningless to align the horizontal axis of the graph with the IPC code of the same layer. For example, it is fully conceivable that a feature can be read in the second level of a certain group, but a feature can be obtained only by dividing another group up to the third level. Thus, it is desirable that each group is divided into granularities having characteristics.

  In addition, even if the lowermost layer is divided, there is a possibility that there is a group having no characteristics (“IPC code group that has been applied evenly by each applicant” in the above example). Similar to groups with features, groups without such features are important information, but in order for analysts to find such groups, examine multiple graphs created for each hierarchy in detail. There was a need.

  Further, in the above-described prior art 2, the case of having a hierarchy is not considered, and it cannot be applied as it is to the above-described prior art 1 task.

  Also, in the above-described prior art 3, it is unclear whether the created group has a feature with respect to the distribution of the objective variable. Note that even if the above-described known techniques are combined with an idea that takes into account the hierarchy of explanatory variables, the intended result cannot be obtained.

  Further, in the above-described prior art 4, since the purchase recorded in the POS data is performed under various conditions, there is a problem that it is extremely difficult to define “exactly the same conditions”. Therefore, it is necessary to create a group by ignoring differences that can be ignored in the analysis of individual purchase data under various conditions. If the items used for this division have a hierarchical structure, it is unclear which group will create a group with different conditions when the division is performed. The same problem occurs.

  For example, it is unclear before the analysis whether the number of sales of a product changes on a seasonal basis, on a monthly basis, or on a weekly or weekday basis. Therefore, it is necessary to divide the data in units that depend on the common sense of the analyst and verify the results, or to compare the number of sales in all divided units.

  Also in the above-described prior art 5, it is not clear at which hierarchy the useful knowledge can be obtained by dividing the data. For example, in a group with many impressions focusing on product design, there is no significant difference in the gender ratio, but in the lower hierarchy, there are groups where the design is “good” and “bad”. The knowledge that the two groups are largely biased is useful, but this knowledge cannot be acquired when divided by the upper group. That is, also in this case, it is unclear how useful knowledge can be obtained by dividing the hierarchical clustering result.

  Further, even if the division method of Conventional Technique 6 is used for Conventional Technique 5, it is unclear whether the result of the division has a feature with respect to the objective variable.

  In order to solve the above-described problems caused by the prior art, the first aspect of the present invention is to automatically divide multidimensional data into appropriate groups that match the purpose of analysis when dividing into items having a hierarchical structure. The purpose. Here, the “appropriate group meeting the purpose of analysis” is a group that is characterized by, for example, the distribution of the item of interest of the user, or a group that can be regarded as having the same conditions regarding the variation of the numeric item of interest. is there. In addition, a second object is to present a feature that the user pays attention to the divided groups.

  In order to solve the above-described problems and achieve the object, a data division program, a recording medium on which the program is recorded, a data division device, and a data division method are used as explanatory variables representing the hierarchy of a multidimensional data group. When the group of the corresponding multidimensional data is set as the division target group, the attribute of the division target group is obtained by performing a statistical test based on the distribution of the objective variable that can represent the distribution bias in the division target group. Determining, based on the determination result, dividing the group to be divided into child group groups belonging to the child hierarchy of the hierarchy to which the group to be divided belongs, and newly selecting the child group from among the child group groups. Divided target group, and any group that composes the obtained group set is consolidated target group If, based on the attributes of the child group belonging to the child layer of the hierarchical of the integration target group, and wherein the obtaining the integration result by integrating the child group set to the integration target group.

  According to the present invention, multidimensional data can be divided with a granularity suitable for an objective variable.

  Further, in the above invention, when it is determined that the statistical test can be used based on the number of multidimensional data belonging to the division target group, the attribute of the division target group is determined by performing the statistical test. May be determined.

  According to the present invention, the efficiency of the statistical test can be improved.

  Further, in the above invention, when it is determined that the statistical test cannot be used based on the number of multidimensional data belonging to the division target group, the division target group is a child group belonging to a child hierarchy of the hierarchy. It may be determined that the group is a group that stops the transition to a group, and a group that is not selected as the division target group may be newly set as the division target group.

  According to the present invention, it is possible to improve the efficiency of attribute determination and to suppress meaningless division.

  Moreover, in the said invention, it is good also as determining whether distribution of the said objective variable in the said division | segmentation object group is characteristic distribution.

  In particular, when the objective variable is a fixed value, the distribution of the objective variable in the group to be divided is determined by a significance test between the distribution of the objective variable in the group to be divided and the distribution of the objective variable in the entire group set. It may be determined whether or not is a characteristic distribution.

  According to the present invention, it is possible to distinguish between characteristic groups and non-characteristic groups.

  In the above invention, when the two distributions are significantly different from each other, the distribution of the objective variable in the group to be divided is compared with the distribution of the entire group set to determine that the group to be divided is a characteristic distribution. A group that is not selected as the division target group may be newly set as the division target group.

  According to the present invention, it is possible to detect a characteristic group and suppress excessive division into lower layers.

  Further, in the above invention, when the objective variable is a numerical value, by performing a fitness test between the distribution of the objective variable in the division target group and a known probability distribution model estimated from the distribution, It may be determined whether the distribution of the objective variable in the group is a characteristic distribution.

  In particular, when the distribution of the objective variable in the division target group is not different from the probability distribution model, it is determined that the distribution of the objective variable in the division target group is a characteristic distribution according to the probability distribution model, A group that is not selected as the division target group may be newly set as the division target group.

  According to the present invention, it is possible to distinguish between characteristic groups and non-characteristic groups.

  Further, in the above invention, when the distribution of the objective variable in the group to be divided is not a characteristic distribution according to the probability distribution model and there is no child group group belonging to the child hierarchy, the group to be divided is: It is also possible to determine that the group is a group that stops the division into a group group belonging to a child hierarchy of the hierarchy, and a group not selected as the division target group may be newly set as the division target group.

  According to the present invention, it is possible to detect a characteristic group and suppress excessive division into lower layers.

  Further, in the above invention, when the distribution of the objective variable in the group to be divided is not a characteristic distribution according to the probability distribution model and there is a child group group belonging to the child hierarchy, the group to be divided is The group may be determined as a group that can be determined by the attribute of the child group belonging to the child hierarchy of the hierarchy, and the child group may be newly set as the division target group.

  According to the present invention, it is possible to suppress overdivision into lower hierarchies.

  Further, in the above invention, when it is determined that all the child groups belonging to the child hierarchy of the hierarchy of the integration target group are groups that stop the division, the child group group is deleted, and the integration target group is It is good also as making it the same attribute as a child group group.

  According to the present invention, it is possible to integrate over-divided groups.

  In the above invention, when it is determined that all distributions of the objective variable in each child group belonging to a child hierarchy of the hierarchy of the integration target group are characteristic distributions, the child group group is set as the integration target. It may not be integrated into a group.

  According to the present invention, it is possible to leave only a group having a characteristic distribution.

  According to the data division program, the recording medium on which the program is recorded, the data division device, and the data division method according to the present invention, when dividing multi-dimensional data into items having a hierarchical structure, the data division program The effect is that it can be automatically divided into various groups. Moreover, the effect that the feature which a user pays attention to can be shown with respect to the divided | segmented group is produced.

  Exemplary embodiments of a data division program, a recording medium recording the program, a data division device, and a data division method according to the present invention will be explained below in detail with reference to the accompanying drawings.

(Hardware configuration of data division device)
First, the hardware configuration of the data dividing apparatus according to the embodiment of the present invention will be described. FIG. 1 is an explanatory diagram showing the hardware configuration of the data dividing apparatus according to the embodiment of the present invention.

  In FIG. 1, a data dividing device 100 includes a computer main body 110, an input device 120, and an output device 130. The data dividing device 100 is connected to a network 140 such as a LAN, WAN, or the Internet via a router or a modem (not shown). Connectable.

  The computer main body 110 has a CPU, a storage device, and an interface. The CPU governs overall control of the data dividing device 100. The storage device includes a ROM, RAM, HD, optical disk 111, and flash memory. The RAM is used as a work area for the CPU.

  In addition, various programs are stored in the storage device, and loaded according to instructions from the CPU. Data read / write of the HD and the optical disk 111 is controlled by a disk drive. The optical disk 111 and the flash memory are detachable from the computer main body 110. The interface controls input from the input device 120, output to the output device 130, and transmission / reception with respect to the network 140.

  The input device 120 includes a keyboard 121, a mouse 122, a scanner 123, and the like. The keyboard 121 includes keys for inputting characters, numbers, various instructions, and the like, and inputs data. Further, it may be a touch panel type. The mouse 122 performs cursor movement, range selection, window movement, size change, and the like. The scanner 123 optically reads an image. The read image is captured as image data and stored in a storage device in the computer main body 110. Note that the scanner 123 may have an OCR function.

  The output device 130 includes a display 131, a printer 132, a speaker 133, and the like. The display 131 displays data such as a document, an image, and function information as well as a cursor, an icon, or a tool box. The printer 132 prints image data and document data. The speaker 133 outputs sound such as sound effects and reading sounds.

(Functional configuration of data partitioning device)
Next, a functional configuration of the data dividing apparatus according to the embodiment of the present invention will be described. FIG. 2 is a block diagram showing a functional configuration of the data dividing apparatus according to the embodiment of the present invention.

  The data dividing apparatus 100 includes an input unit 201, a determination unit 202, a dividing unit 203, an integrating unit 204, an output unit 205, and a DB 210, and is an objective variable that can represent an explanatory variable representing a hierarchy and a distribution bias. Are divided into multidimensional data groups. Each function is realized by causing the CPU to execute a program stored in a storage device such as the memory or the HD shown in FIG. 1 or by an I / F.

  First, the input unit 201 has a function of receiving input of an explanatory variable that represents a hierarchy of multidimensional data and an objective variable that can represent a distribution bias. Multidimensional data is data having explanatory variables and objective variables. For example, as described above, when the multidimensional data is patent publication data, it has an IPC code as an explanatory variable, a fixed value such as “applicant” and “inventor” as an objective variable, From what number of patent publications the patent publication is cited)).

  In multidimensional data, a textual free description item may be included in the multidimensional data instead of the explanatory variable. This may be achieved by creating a hierarchical structure of a multidimensional data group by applying a known technique called hierarchical clustering to the text described in the free description item, and using the hierarchical structure as an explanatory variable. Further, the input unit 201 may accept input of parameters used for the division process (for example, a risk factor when performing a statistical test or the number of data items that are not further divided).

  In this way, when an explanatory variable is input, a group of multidimensional data corresponding to the explanatory variable is specified. This identified group becomes the initial division target group. For example, in the analysis of patent gazette data, if the IPC code is “G06”, “G06” is the initial division target group. FIG. 3 shows hierarchical structure data whose root is the explanatory variable which is the initial division target group.

  FIG. 3 is an explanatory diagram showing hierarchical structure data having the explanatory variable as a root. In the hierarchical structure data 300, the entire group R represents an explanatory variable. In FIG. 3, as an example, the descendant groups branch to the first hierarchy, the second hierarchy, and the third hierarchy. The groups A, B, and C in the first hierarchy are child groups of the entire group R.

  Hereinafter, a group belonging to a child hierarchy of the group G (G is a group name) hierarchy is denoted as G # (# is a number), and a group belonging to the child hierarchy is denoted as G ##. For example, the groups belonging to the second hierarchy of the group A of the first hierarchy are A1, A2, and A3, respectively, and the groups belonging to the third hierarchy that is a child hierarchy of the group A1 of the second hierarchy are A11 and A12, respectively. .

  The hierarchical structure data 300 may be directly given to the input unit 201. When an arbitrary explanatory variable is input, the hierarchical structure data 300 is automatically referred to by referring to the explanatory variables included in the multidimensional data group in the DB 210. It may be created. Further, when the hierarchical data 300 of the entire multidimensional data group is already stored in the DB 210, a partial hierarchy having the explanatory variable as a root by inputting an arbitrary explanatory variable to the input unit 201 The structure data 300 may be extracted.

  Further, only the group corresponding to the explanatory variable may be given, and then the child group of the lower hierarchy may be added each time the division unit 203 divides the group. In this case, the hierarchical structure data 600 shown in FIG. 6 is directly generated.

  Further, the entire group R that is the root of the hierarchical structure data 300 is an initial division target group. The division target group is a group to be determined by the determination unit 202 and a group to be divided by the division unit 203, and a group (child group) of a child hierarchy obtained by the division is newly set as a division target group, so that the determination unit The determination process 202 and the dividing process of the dividing unit 203 can be recursively executed.

  This input unit 201 is, for example, an I / F in which a user selects a data item displayed on the display 131 shown in FIG. 1 using the input device 120 such as a mouse 122 and a keyboard 121, or a network 140 from an external device. It can be realized by an I / F such as receiving via the I / F.

  In FIG. 2, when the determination unit 202 sets a group of multidimensional data corresponding to the explanatory variable input by the input unit 201 as a division target group, the determination unit 202 performs statistical processing based on the distribution of the objective variable in the division target group. It has a function to determine the attribute of the group to be divided by performing the test. Specifically, it is determined what attribute group the division target group is. The determination result is held in the determination table as a flag.

  FIG. 4 is an explanatory diagram showing the determination table 400 (initial state). The determination table 400 is a table for setting a flag for each group of each hierarchy of the hierarchical structure data 300. Since the determination table 400 shown in FIG. 4 represents the initial state before the start of determination, all the flags that are determination results are set to “0”. The flag: 0 is a flag indicating the re-division condition, and remains “0” when the stop condition and the matching condition are not matched.

  When the determination process is executed by the determination unit 202, a flag is set in the determination table 400 as the determination result. Several types of flags are prepared according to group attributes. The flag: 1 is a flag indicating that the division target group is a “conforming group”. Here, the condition for setting the flag to 1 is referred to as a conforming condition.

  The conforming condition is a condition for determining whether or not the distribution of the objective variable in the group to be divided is a characteristic distribution. If the distribution is characteristic, the conforming condition is matched and the flag is set to “1”. Is done. On the other hand, if it is not characteristic, the matching condition is not met.

  Judgment based on conformance conditions varies depending on the type of objective variable. First, the case where the objective variable is a fixed value such as “applicant” will be described. In the case of a fixed value, whether or not the matching condition is met is determined by whether or not the distribution of the objective variable in the entire group R and the distribution of the objective variable in the group to be divided are significantly different by statistical test. If it is characteristic, the flag is set to “1” in accordance with the matching condition. On the other hand, if it is not characteristic, the matching condition is not matched.

  On the other hand, if the objective variable is a numerical value, whether the numerical value that is the objective variable of the group to be divided conforms to the matching condition depending on whether or not it follows a known probability distribution model estimated from the multidimensional data group of the group to be divided Determine whether or not. If the known probability distribution model is followed, the flag is set to “1” in accordance with the matching condition. On the other hand, if it is not characteristic, the matching condition is not matched.

  The flag: 2 is a flag indicating that the division target group is a “stop group”. Here, the condition for setting the flag: 2 is referred to as a stop condition. The stop condition is a condition for determining whether or not to divide into groups that belong to a child hierarchy, and two types of stop conditions are prepared. Here, in order to distinguish, one is referred to as “stop condition 1” and the other as “stop condition 2”.

  First, the stop condition 1 is a condition to which the statistical test method used for the determination of the conformity condition can be applied to the multidimensional data group included in the division target group. Is set to “2”. On the other hand, if the condition is satisfied, the statistical test method can be applied, so that the stop condition 1 is not matched.

  The stop condition 1 is, for example, a condition that the number of multidimensional data in the division target group is smaller than the number of multidimensional data that guarantees the accuracy of the statistical test method. When the number is small, the stop condition 1 is matched and the flag is set to “2”.

  On the other hand, the stop condition 2 is a condition in which the division target group has no child hierarchy and cannot be further divided. If there is no child hierarchy, the flag is set to “2” in accordance with the stop condition 2. On the other hand, if there is a child hierarchy, the stop condition 2 is not matched, and the child hierarchy can be divided.

  Further, based on the determination result determined by the determination unit 202, the dividing unit 203 divides the division target group into groups (child group groups) belonging to the child hierarchy of the hierarchy to which the division target group belongs, and It has a function of newly setting a child group selected from the group group as a group to be divided. Specifically, when the division target group matches the subdivision condition, the group is divided into child group groups.

  Note that division means transition from a division target group to its child group when the hierarchical structure data 300 shown in FIG. 3 is given prior to division. Then, an arbitrary group in the child group group that is the transition destination is newly set as a division target group. If it is not necessary to divide (transition) into child group groups even though there are child group groups, the groups below the child group groups are deleted.

  On the other hand, when the hierarchical structure data 300 shown in FIG. 3 is not given prior to the division and only the entire group R is given, the division is to read the child group group of the division target group from the DB 210. It means adding to the group to be divided. In this case, since only the divided child group group is added to the child hierarchy, the hierarchical structure data 600 shown in FIG. 6 is directly generated. Specific examples of the determination process of the determination unit 202 and the division process of the division unit 203 will be described later.

  Further, the integration unit 204, when an arbitrary group constituting the hierarchical structure data 300 that is a group set after the division result is an integration target group, a group group (child group group) belonging to a child hierarchy of the hierarchy of the integration target group ) Based on the attribute, the child group group is integrated into the integration target group.

  Specifically, the integration unit 204 detects the group overdivided by the division unit 203 using the attribute of the group as a clue, and deletes the overdivided group to integrate it with its parent group. In this case, the flag representing the attribute of the parent group is changed from the intermediate group (flag: 0) representing the re-division condition to the flag: 2 representing the stop condition. Details of the integration process of the integration unit 204 will be described later.

  The output unit 205 has a function of outputting the integration result integrated by the integration unit 204. The integration result may be the content of the hierarchical structure data 1100 or the determination table 400 finally obtained (FIG. 11), or may be output data in which the objective variables are aggregated for each integration result and the distribution is graphed. Although it is good, it is not restricted to these output forms.

  Further, since the attribute of “adaptation group (flag: 1)” or “stop group (flag: 2)” is assigned to the output hierarchical structure data 1100, the attribute can also be presented. is there. There are various forms of the display method of this attribute. For example, in the case of the above-described graphed output data, a group characterized by the distribution of the objective variable compared to the whole (“matching group (flag: 1)” ) And a group having no distribution difference (“stop group (flag: 2)”) are displayed in different colors.

(Specific examples of judgment processing and division processing)
Next, a specific example of the determination process of the determination unit 202 and the division process of the division unit 203 described above will be described. First, the determination unit 202 executes determination processing in the order of stop condition 1 ⇒ conforming condition ⇒ stop condition 2 regardless of the type of the objective variable. First, the determination process when the objective variable is a fixed value will be specifically described.

  For example, items that are objective variables are set to fixed values of X, Y, and Z. These X, Y, and Z correspond to, for example, applicant X, applicant Y, and applicant Z in the analysis of patent gazette data. In the case where the group to be divided is the entire group R serving as the root, when the stop condition 2 is satisfied, the dividing unit 203 divides the group A into C of the first hierarchy that is a child hierarchy.

  The determination unit 202 selects an undetermined group (for example, group A) from the groups A to C and sets it as a division target group. The table shows the number of multidimensional data for each objective variable X, Y, Z in the group A, which is the group to be divided, and the number of multidimensional data for each objective variable X, Y, Z in the entire group R.

  FIG. 5 is a chart showing the number of multidimensional data for each objective variable X, Y, Z in the group A and the number of multidimensional data for each objective variable X, Y, Z in the entire group R. First, a process for determining whether or not the stop condition 1 is matched is executed. An example of the stop condition 1 is a condition that “the number of multidimensional data of the group to be divided is 10 or less”.

In addition, a statistical test method used for checking the next matching condition and a condition determined by the property of multidimensional data to be statistically tested may be used. For example, in the case of a general χ ² test as a statistical test method, “when the null hypothesis is established, if the expected value of the number of data for one objective variable is less than 5% exceeds 40%, the test It is known to have the property that the accuracy is significantly reduced. Such a complicated condition can be adopted as the stop condition 1.

  As shown in FIG. 5, when the null hypothesis is established, that is, when there is no difference in the distribution of the objective variables of the group A and the entire group R, the three objective variables X, Y, Since Z has a uniform distribution (both are 100), it is considered that each objective variable X, Y, Z is also uniform in group A.

  Further, since the total number of multidimensional data of group A is 90, the expected value of the number of multidimensional data for each objective variable X, Y, Z is 30 (90/3). Since no one of the three groups divided by the objective variable has an expected value less than 5, group A does not match stop condition 1. Therefore, in this case, the process proceeds to determination of the conforming condition.

  In addition, when the objective variable is a numerical value, a goodness-of-fit test with a known probability distribution model is performed. In the goodness-of-fit test, “the group whose expected value of the number of multidimensional data belonging to the group to be divided is less than 1” Since there is known a property that the accuracy of the test decreases when there is at least one, it can be adopted as the stop condition 1.

Next, a process for determining whether or not the matching condition is met is executed. When the above-described χ ² test is used as a statistical test method, a χ ² statistic represented by the following formula (1) is calculated.

χ ² = Σ {(actual measurement value−expected value) ² / expected value} (1)

In the case of FIG. 5, the expected value in each objective variable is 30, and the actual measurement values are 32, 28, and 30 in this order. Therefore, when substituting into the above equation (1), the χ ² statistic is 0.27. Is calculated.

Next, the significance of the difference in distribution is determined based on the calculated χ ² statistic. In the case of FIG. 5, since the number of possible values of the objective variable is 3, the degree of freedom is 2, which is 1 less than that. In the χ ² test with ² degrees of freedom, if the risk factor (probability of determining that there is no significance but 5%) is, for example, 5%, the determination reference value is 5.991.

Therefore, it can be said that the calculated χ ² statistic is significantly different when it is larger than the criterion value 5.991, and the flag is set to “1” in accordance with the matching condition. Since the χ ² statistic (= 0.27) of group A, which is the current group to be divided, is equal to or less than the criterion value 5.991, group A does not match the matching condition. Therefore, in this case, the process proceeds to the determination of the stop condition 2.

  Further, a process for determining whether or not the stop condition 2 is matched is executed. Since the group A shown in FIG. 5 has child hierarchies, it can be further divided and does not match the stop condition 2. A group that does not match any of the above conditions is determined to match the subdivision condition. Since the current group A matches the subdivision condition, it is divided into three groups A1 to A3 in the second hierarchy, and the same determination process is executed for each of the groups A1 to A3.

  Since the group A1 matches the matching condition, the attribute of “matching group (flag: 1)” is given, and no further division is performed. Since the groups A2 and A3 match the re-division condition, “intermediate group (flag: 0)” is assigned and further division is performed. As a result, the group A2 is divided into groups A21 and A22, and each of the groups A21 and A22 matches the matching condition, so the attribute of “matching group (flag: 1)” is given.

  Further, the group A3 is divided into groups A31 and A32, and the groups A31 and A32 match the stop condition, so that “stop group (flag: 2)” is assigned to each of them. By performing the above-described processing in group A on groups B and C in the same manner, the multidimensional data group is divided. The division result and the determination result are shown.

  FIG. 6 is an explanatory diagram showing the hierarchical structure data 600 after the division process (before the integration process), and FIG. 7 is a chart showing the determination table 400 after the division process (before the integration process). In FIG. 6, the number on the lower right of each group is a flag value representing the attribute of the group.

  In addition, when an item that takes a numerical value as the objective variable is selected, only the fitting condition changes, and it is determined whether the objective variable of the group to be divided follows a known probability distribution model estimated from multidimensional data (in general, “ (Referred to as “goodness-of-fit test”). For example, it is assumed that an item having a discrete value such as the number of sales per day of a product is selected as an objective variable. At this time, for example, the following chart shows the distribution of the data frequency of each objective variable in group A1 (number of multidimensional data: 50).

  FIG. 8 is a chart showing the data frequency distribution of each objective variable in group A1. On the other hand, when the Poisson distribution is adopted as the probability distribution function for comparison, the expected value of the objective variable is 2.4, and therefore the probability p (x) that the objective variable is x is defined by the following equation (2). .

p (x) = exp (−2.4) × 2.4 ^x / x! ... (2)

  Next, when the objective variable follows the probability distribution of Expression (2), the expected value of the number of multidimensional data taking each objective variable is shown below.

  FIG. 9 is a chart showing the data frequency distribution (added expected value frequency) of each objective variable in group A1. In the six groups divided by the value of each objective variable, there is no group whose expected frequency is less than 1, so the stop condition 1 is not matched. Therefore, the determination of the conforming condition is executed.

Next, a modified χ ² statistic that is a statistical value often used in the fitness test is calculated. Since the modified χ ² statistic is defined by the above equation (1), the current modified χ ² statistic is 2.13.

On the other hand, the standard value of the degree of freedom 5 under the risk rate of 5% is 9.488. Since the modified χ ² statistic in this data is below its reference value, the two distributions are not different. That is, it can be said that the distribution of the objective variable in this division target group follows the Poisson distribution. As described above, the group A1 matches the matching condition, and no further division is performed.

In the above-described example, a statistical test method based on a general χ ² statistic is adopted for the statistical test, but is not limited to this statistical test method. Further, although the test is performed using the Poisson distribution as a probability distribution function for comparison when the objective variable is a numerical value, the present invention is not limited to this. An appropriate probability distribution model can be selected in consideration of the expected probability distribution from the properties of the objective variable such as whether the objective variable takes a continuous value or a discrete value.

  For example, when an operation of creating a rust-preventing film by spraying a paint on a metal product, the film thickness under the same condition is expected to follow a normal distribution. Although it is useful to find conditions for ensuring sufficient film thickness while reducing the number of paints as much as possible, in conditions with a hierarchical structure, ignoring differences in negligible conditions, Useful for partitioning multidimensional data with influencing granularity.

(Specific example of integration processing)
Next, a specific example of integration processing by the integration unit 204 described above will be described. There is a possibility that excessive division has occurred in the division result by the division unit 203. The integration unit 204 detects this overdivided group and integrates them in the parent hierarchy. Specifically, the over-divided group is a group group that matches all the stop conditions. The group group that matches the stop condition is a group in which no feature is found with respect to the objective variable. Since it is not meaningful to separate such a group having no feature, the integration unit 204 performs integration.

  In the integration process by the integration unit 204, in the initial state, the group (total group R) that becomes the root of the group set (hierarchical structure data 600 after the division) that is the division result is set as the integration target group, and the following procedures (1) to (1) to Realized according to (4).

(1) It is determined whether or not the integration target group has an attribute of “intermediate group (flag: 0)” (that is, has a group belonging to a child hierarchy of the hierarchy). End the integration process for the integration target group.

(2) On the other hand, when having an attribute, the above (1) is recursively executed with each group belonging to the child hierarchy as an integration target group. As a result of this integration processing, each group is given either the “stop group (flag: 2)” or “intermediate group that cannot be integrated (flag: 3)” attribute.

(3) When a group having an attribute of “applicable group (flag: 1)” or “intermediate group that cannot be integrated (flag: 3)” is included in the group belonging to the child hierarchy of the integration target group, The attribute of “intermediate group that cannot be integrated (flag: 3)” is assigned to the integration target group.

(4) As a result of performing the integration process on all groups (descendant groups) in the lower hierarchy of the integration target group hierarchy, the attribute of “intermediate group that cannot be integrated (flag: 3)” is assigned to the integration target group. If not, the “stop group (flag: 2)” attribute is assigned to this group, and all descendant groups are deleted. The procedures (1) to (4) will be described using the hierarchical structure data 600 that is the division result shown in FIG.

  10 is an explanatory diagram showing the hierarchical structure data 600 during the integration process, FIG. 11 is an explanatory diagram showing the hierarchical structure data 1100 after the integration process, and FIG. 12 is a determination table after the integration process. FIG.

[1] In FIG. 6, since the entire group R is divided into three child groups of groups A to C (flag: 0), the process proceeds to [2].

[2] Group A (flag: 0) is further divided into three child groups of groups A1, A2 and A3. Therefore, the recursive processing of [2a] to [2c] below is performed with group A as the integration target group. Execute.

  [2a] Since the group A1 which is a child group has the attribute of “conformity group (flag: 1)”, the attribute of the group A which is an integration target group is changed to “intermediate group which cannot be integrated (flag: 3)” (See FIG. 10).

  [2b] The group A2 (flag: 0), which is a child group, has a group that belongs to its child hierarchy, and therefore, an integration process that uses the group A2 as an integration target group is applied. Since both of the groups A21 and A22 which are child groups of the group A2 have the attribute of “conforming group (flag: 1)”, the attribute of the group A2 is changed to “intermediate group that cannot be integrated (flag: 3)”. It is changed (see FIG. 10). As a result, the attribute of “intermediate group that cannot be integrated (flag: 3)” is also given to group A, but it is not different from the attribute given to group A1.

  [2c] Since the group A3 (flag: 0) has a group that belongs to its child hierarchy, an integration process in which the group A3 is the integration target group is applied. Since both the groups A31 and A32 which are child groups of the group A3 have the attribute of “stop group (flag: 2)”, the attribute of the group A3 is changed to “stop group (flag: 2)”. (See FIG. 10). As a result, the child groups A31 and A32 are deleted.

  As described above, in FIG. 10, the attribute of “intermediate group that cannot be integrated (flag: 3)” is assigned to group A, and group A includes group A1 (flag: 1), group A21 (flag: 1), and group A22. (Flag: 1) and Group A3 (Flag: 2). Further, since the attribute of the group A is “intermediate group that cannot be integrated (flag: 3)”, the attribute of “intermediate group that cannot be integrated (flag: 3)” is also given to the entire group R.

[3] In FIG. 6, the group B (flag: 0) also has a group belonging to its child hierarchy, so that the integration process is recursively performed as in the case of the group A. As a result of this processing, the attribute of “stop group (flag: 2)” is given to group B, and the descendant groups B1, B2, B21, and B22 are deleted (see FIG. 11).

  More specifically, since the attribute of the group B1 is “stop group (flag: 2)”, the recursive process is not performed. Further, since the group B2 has the attribute of “intermediate group (flag: 0)”, the integration process is recursively performed with the group B2 as the integration target group.

  Since both the groups B21 and B22 which are child groups of the group B2 have the attribute “stop group (flag: 2)”, the attribute of the group B2 is changed to “stop group (flag: 2)” and its children Groups B21 and B22, which are groups, are deleted (see FIG. 10).

  As a result, both of the groups B1 and B2 have the attribute of “stop group (flag: 2)”, so that the parent group, group B, is changed to “stop group (flag: 2)”. B1 and B2 are deleted (see FIG. 11).

[4] Also, in FIG. 6, since the group C (flag: 0) also has a group belonging to its child hierarchy, the integration processing is recursively performed as in the case of the groups A and B. As a result of this processing, the attribute of group C is changed to “intermediate group that cannot be integrated (flag: 3)”, and is divided into group C1 (flag: 1) and group C2 (flag: 2) that are child groups ( (See FIGS. 10 and 11).

  Further, since the attribute of the group C is “intermediate group that cannot be integrated (flag: 3)”, the attribute of the entire group R is maintained (the already assigned intermediate group that cannot be integrated (flag: 3)).

[5] Since the attribute of “intermediate group that cannot be integrated (flag: 3)” is assigned to the entire group R, a series of processing ends. Thereby, the hierarchical structure data 1100 after the integration process as shown in FIG. 11 and the determination table 400 after the integration process as shown in FIG. 12 are obtained.

  Here, when the division result (FIGS. 6 and 7) and the integration result (FIGS. 11 and 12) are compared, if all the child group groups are stop groups (flag: 2), the attribute of the parent group is stopped. It becomes a group (flag: 2), and the child group group has been deleted. Further, the attribute of the group (where the integration process is not performed) is flag: 0, but when the integration process is performed, the attribute is “intermediate group that cannot be integrated (flag: 3)”.

  In addition, if one child group has an attribute of “applicable group (flag: 1)” or “intermediate group that cannot be integrated (flag: 3)”, no integration is performed. However, it is possible to relax the condition and perform the integration process.

  For example, the number of child groups is compared with the number of groups having the attribute of “matching group (flag: 1)” or “intermediate group that cannot be merged (flag: 3)”, and the ratio is less than a certain number. In some cases, integration is performed, and the ratio between the number of multidimensional data belonging to the “adapted group (flag: 1)” or “intermediate group that cannot be integrated (flag: 3)” and the number of multidimensional data of the parent group is constant. Processing may be performed such that integration is performed when the number is less than the number.

  In addition to the integration to the parent group, if a child group of a certain group (parent group) is divided into “adaptation group (flag: 1)” and “stop group (flag: 2)” attributes, A process of integrating the “stop group (flag: 2)” into one group (a process of combining a plurality of stop groups into one) may be added. In this process, the group that becomes the “conformity group (flag: 1)” is particularly important, and no further details need to be analyzed for the “stop group (flag: 2)” that has no characteristics. Suitable for analytical use. As a result of this processing, one or more matching groups and one stop group are created as child groups immediately below the parent group.

  For example, in the analysis of patent gazette data, for groups with IPC codes characterized by the distribution of applicants, the analysis proceeds further in terms of which applicants have applied for more (or fewer) applications. In this case, all the groups that are not present are “groups of patent publications that have been applied evenly by each applicant”, and therefore no further examination is performed.

(Data division processing procedure)
Next, a data division processing procedure according to the embodiment of the present invention will be described. FIG. 13 is a flowchart showing a data division processing procedure according to the embodiment of the present invention. In FIG. 13, first, input processing for receiving input of explanatory variables and objective variables is executed by the input unit 201 (step S1301), and determination processing by the determination unit 202 and division processing (determination / division processing) by the division unit 203 are executed. (Step S1302).

  When the division processing result (the hierarchical structure data 600 shown in FIG. 6 and the determination table 400 shown in FIG. 7) is obtained, the integration processing is executed by the integration unit 204 (step S1303). When the integration unit 204 obtains the integration process result (the hierarchical structure data 1100 shown in FIG. 11 and the determination table 400 shown in FIG. 12), the output unit 205 executes the output process for outputting the integration process result (step S1304). ).

(Judgment / division procedure)
Next, a detailed processing procedure of the determination / division processing shown in FIG. 13 will be described. FIG. 14 is a flowchart showing a detailed processing procedure of the determination / division processing shown in FIG. 14, first, hierarchical structure data 300 (see FIG. 3) having an explanatory variable as a root is acquired (step S1401), and flag: 0 is set as an attribute of all the groups (step S1402).

  Next, it is determined whether or not the entire group R matches the stop condition 2 (step S1403). When the stop condition 2 is matched (step S1403: Yes), since there is no child group, the attribute of the entire group R is changed to flag: 2 (step S1404), and the process proceeds to step S1303.

  On the other hand, when the stop condition 2 is not matched (step S1403: No), it is divided into child groups (step S1405). It is determined whether or not there is a child group having an attribute of flag: 0 in this child group group (step S1406). If a child group having an attribute of flag: 0 exists (step S1406: Yes), the flag : One child group having the attribute of 0 is selected as a division target group (step S1407), and the determination process is executed (step S1408). Note that the child group for which the determination process has been executed is not selected as the group to be divided. This determination process (step S1408) will be described later.

  The division target group flag by this determination processing is checked (step S1409). When the flag is 0 (step S1409: YES), since this division target group has a child group group, the process returns to step S1405. Thereby, division can be performed recursively.

  On the other hand, when the flag is not 0 (step S1409: No), that is, when the flag is 1 or 2, the descendant group of the group to be divided is deleted from the hierarchical structure data 300 shown in FIG. 3, and the process returns to step S1406.

  In step S1406, if there is no child group having the attribute of flag: 0 in the child group group (step S1406: No), since the determination process for the child group group has been completed, transition to the parent group is performed ( Step S1410). Then, it is determined whether or not the parent group is the entire group R (step S1411).

  If it is not the parent group (step S1411: NO), the process returns to step S1406. Thereby, a child group having the attribute of flag: 0 can be searched. On the other hand, when it is a parent group (step S1411: Yes), it transfers to step S1303. As a result, the series of determination / division processing ends.

(Judgment processing procedure)
Next, a detailed processing procedure of the determination process (step S1408) illustrated in FIG. 14 will be described. FIG. 15 is a flowchart showing a detailed processing procedure of the determination process shown in FIG. In FIG. 15, first, it is determined whether or not the child group selected as the group to be divided in step S1407 matches the stop condition 1 (step S1501). When the stop condition 1 is matched (step S1501: Yes), the division target group flag is changed to 2 (step S1502), and the process proceeds to step S1409.

  On the other hand, when the stop condition 1 is not matched (step S1501: No), it is determined whether or not the matching condition is matched (step S1503). When the matching condition is met (step S1503: Yes), the division target group flag is changed to 1 (step S1504), and the process proceeds to step S1409.

  On the other hand, if the matching condition is not matched (step S1503: No), it is determined whether the stop condition 2 is matched (step S1505). When the stop condition 2 is matched (step S1505: Yes), the division target group flag is changed to 2 (step S1506), and the process proceeds to step S1409. On the other hand, when the stop condition 2 is not matched (step S1505: No), the division target group flag remains 0, and the process proceeds to step S1409. Thereby, a series of determination processes is completed.

(Integration process)
Next, a detailed processing procedure of the integration processing shown in FIG. 13 will be described. FIG. 16 is a flowchart showing a detailed processing procedure of the integration processing shown in FIG. In FIG. 16, first, the flag of the entire group R is checked (step S1601). If the flag is not 0 (step S1601: No), the process proceeds to step S1304.

  On the other hand, when the flag is 0 (step S1601: Yes), the process shifts to the child hierarchy (step S1602), and determines whether there is an unselected child group in the group group (child group group) belonging to the child hierarchy. (Step S1603). If there is an unselected child group (step S1603: Yes), one child group is selected (step S1604), and its flag is checked (step S1605).

  If the flag is 0 (step S1605: YES), the process returns to step S1602, and further transitions to a child hierarchy. On the other hand, if the flag is not 0 (step S1605: NO), the process returns to step S1603 to determine whether there is an unselected child group.

  If there is no unselected child group in the child group group in step S1606 (step S1603: No), the flag of the child group group is checked with the parent group of the child group group as the integration target G loop (step S1606). .

  When all the flags of the child group group are 2 (step S1606: Yes), the flag of the parent group to be integrated is changed to 2 (step S1607), and the child group group is deleted (step S1608). At this time, if the child group group has a group in the lower hierarchy, these groups are also collected, that is, the descendant group is deleted. Then, control goes to a step S1610.

  On the other hand, if all the flag of the child group group is not 2 (step S1606: No), the flag of the parent group is changed to 3 (step S1609), and the process proceeds to step S1610. In step S1610, it is determined whether or not the parent group is the entire group R (step S1610). When it is not the entire group (step S1610: No), the process transits to the parent hierarchy (step S1611) and returns to step S1603. On the other hand, when it is the entire group R (step S1610: Yes), the process proceeds to step S1304. As a result, the series of integration processes is completed.

  Thus, according to the embodiment of the present invention, multidimensional data can be appropriately divided according to the purpose. In addition, when the objective variable takes a fixed value, attributes such as “a group that is characteristic of the distribution compared to the whole” and “a group that is not different from the whole” are assigned to the distribution. Attributes can also be presented. In addition, when the objective variable takes a numerical value, attributes such as “a group that follows a known probability distribution model” and “a group that does not follow a known probability distribution model” are given. The parameters of the probability distribution model can also be presented.

  Here, the present embodiment is compared with a known technique (Patent Document 1 described above). The publicly known technique is a technique for integrating two divided result sets when there is no significant difference in the distribution of objective variables. Processing according to the present embodiment and processing using a known technique will be described using the following example. In this example, the processing is performed when an item that takes a fixed value as the objective variable is specified. However, when an item that takes a numerical value as the objective variable is specified, only the test method is different, and the same result is obtained.

First layer: The explanatory variable takes two values of group A and group B as the first layer.
Second layer: Groups A and B in the first layer take the values of groups A1 to A50 and groups B1 to B50, respectively.
-Characteristics of each group: Groups A and B have characteristics in comparison with the overall group R in terms of the distribution of objective variables. In addition, regarding the lower groups of groups A and B (groups A1 to A50 and groups B1 to B50), there are some differences in the distribution of objective variables.

  In such a case, in the determination / division processing, the division into the groups A and B is carried out by division in the first hierarchy. Since each group A and B matches the matching condition, the attribute of “matching group (flag: 1)” is given and no further division is performed. The number of determinations in the above processing is two.

  In the integration process, the entire group R is divided into groups A and B. Since these groups A and B have the attribute of “matching group (flag: 1)”, the attribute of “intermediate group (flag: 3) that cannot be integrated” is given to the entire group R. The process ends. The number of determinations in the above processing is two.

  Note that this determination process simply determines whether to integrate by referring to the attributes in the two lower groups, and does not perform complicated processing such as statistical testing. As described above, the multidimensional data is divided into groups A and B in the first hierarchy, and these two groups A and B are respectively compared with the whole group R (under the risk factor used in the test) as the objective variable. The attribute that there is a difference in the distribution of is given.

  Next, a processing result by a known technique will be described. First, groups A1 to A50 and B1 to B50 are created as division results up to the lowest group. Next, as a contrivance in consideration of the hierarchical nature of multidimensional data, the integration process is performed under the condition that “groups with different upper groups are not integrated”. When this condition is not added, in the known technique, it is determined whether or not integration is possible with respect to all combinations of the group. Therefore, the determination is performed 4950 times, which is the number of combinations for selecting two out of 100. Will do.

  Under the above conditions, for example, first, integration processing is performed on the groups A1 to A50 which are lower groups of the group A. Here, if there is no significant difference between the two selected groups, and if the integration is performed sequentially, the two groups are integrated in one determination if there is no mistake in the determination. 49 determination processes are performed, and 50 groups A1 to A50 are integrated into one group.

  However, the statistical test used for judgment does not guarantee that a correct judgment can be made. For example, in a 5% risk ratio test that is generally widely adopted, it is determined that there is a significant difference at a rate of 5% even though there is no significant difference. Also in the processing of group A, if a determination error occurs with this probability, it is considered that the determination is incorrect one or two times out of 49 times. When this error occurs, a group that is not integrated is created in the lower groups A1 to A50 of the group A.

  In this case, since the number of groups is not reduced by the integration process because the number of groups that are not integrated by one determination is generated, the number of determinations further increases (the worst case is that all the determinations are not integrated and the number of determinations is 1225). . When such an unintegrated group is created in the lower hierarchy, the integration into the group A is not finally performed, and thus an appropriate result cannot be obtained. Further, the possibility of such an error occurring does not change even in the processing in group B.

  Since the risk rate in the test can be controlled by the user, the error probability can be lowered by making this value as low as possible. However, by reducing the risk factor, this time, errors that determine that there is no significant difference even though there is actually a significant difference increase. As a result, for example, there is a high possibility that it is determined that there is no significant difference between the groups A and B integrated by the integration processing in the second hierarchy, and eventually there is a high possibility that no division will be performed.

  Even if the groups A and B can be correctly divided, it is unclear whether the two groups A and B have a bias in the distribution of objective variables compared to the whole group R, that is, have characteristics. For example, even though the group A is multidimensional data having characteristics compared to the entire group R, but the group B has no characteristics, in the present embodiment, the group A is distinguished by the flag assigned to each group. Although it is possible, in the known technique, the distinction is not made and only the division result is obtained. From the above, the processing results of the known techniques are summarized as follows.

The number of determinations is at least 99 (49 determinations in the second layer group and 1 determination in the first layer).
-Since the number of verifications is large, there is a high possibility that a verification error will occur, and there is a high possibility that an appropriate division result will not be obtained.
・ Characteristics for the entire group of segmentation results are unknown.

  Such a tendency becomes worse as the number of layers of multidimensional data increases and the number of groups in the lowest layer increases. On the other hand, in this embodiment, since the determination is performed from the highest hierarchy, even if the number of hierarchies increases, no further division is performed as long as the condition is met in the higher hierarchy, which is efficient.

  In addition, since the present embodiment performs the test in a state where the number of multi-dimensional data of the upper hierarchy group is large, the test is performed in comparison with the test performed between the groups having a small number of multi-dimensional data performed by the known technique. There is also an advantage of high accuracy.

  Further, in the case of another data having no characteristics in group B and some of its subgroups having characteristics, in the present embodiment, the same division result can be obtained regardless of the order in which the tests are performed. On the other hand, when using known technology, when processing such as sequentially integrating groups that did not produce a significant difference in a single test result, the result of the division may change depending on the order in which the group group to be tested is selected. There is sex.

  However, it is impossible to predetermine an appropriate order of the test results. In order to obtain a segmentation result that does not depend on the order of the data in any case using known techniques, for example, tests are performed on all 50 groups, and the group with the closest distribution of the objective variable is selected. It is necessary to repeat the process of integration.

  In that case, after performing the test of 1225, which is a combination of selecting two out of 50 for the first integration, one group having the closest distribution is integrated, and in the next integration, The process of further calculating the statistic for the newly created group and the remaining group combinations (48 sets) and determining the group to be integrated next is repeated until it cannot be integrated.

  In the case of this method, the same division result is obtained regardless of the order of the groups B1 to B50, but the number of statistics calculations, the search for the nearest group that can be integrated, and the number of integration processes are very large. . For example, in this case, when the number of groups that cannot be further integrated is small, the above-described series of processing is performed about 2500 times.

  On the other hand, the test processing in the 50 subgroups of group B in the present embodiment can be completed once with a total of 50 times, so that the processing can be completed with a significantly smaller number of tests compared to the processing using the known technology. it can.

  In this case, the output result differs between the known technique and the present embodiment. In the known technique, in the groups belonging to the same hierarchy, those determined to have the same distribution of objective variables are integrated into one group. However, in this embodiment, partial integration in the same hierarchy group is not performed. (Note that groups without features can be combined into one if no further analysis is performed).

  When this difference is examined, for example, when used for the purpose of analyzing the tendency of the applicant by the IPC code of the patent gazette data cited as an application example, the division result in this embodiment (strictly, the integration result) It turns out that is more appropriate.

  For example, for two IPC codes X and Y in the same hierarchy, if there are 1000 patent publication data with IPC code X and 100 patent publication data with IPC code Y, the distribution of applicants is the same in the known technology When it can be considered, a group of 1100 is created in total, but the breakdown is unknown.

  On the other hand, in the present embodiment, since these two groups (IPC codes X and Y) are not integrated, it is important that the number of applications for IPC code X technology is large, and the technology for IPC code Y is relatively small in number. Information is retained.

  As described above, according to the embodiment of the present invention, when multidimensional data is divided into items having a hierarchical structure, it can be automatically divided into appropriate groups that match the analysis purpose. Moreover, the feature which a user pays attention with respect to the divided | segmented group can be shown.

  The data division method described in this embodiment can be realized by executing a program prepared in advance on a computer such as a personal computer or a workstation. This program is recorded on a computer-readable recording medium such as a hard disk, a flexible disk, a CD-ROM, an MO, and a DVD, and is executed by being read from the recording medium by the computer. The program may be a transmission medium that can be distributed via a network such as the Internet.

  In addition, the data division apparatus 100 described in the present embodiment includes a PLD (Programmable) such as an application specific integrated circuit (hereinafter referred to simply as “ASIC”) such as a standard cell or a structured ASIC (Application Specific Integrated Circuit). (Logic Device) can also be realized. Specifically, for example, by functionally defining the functional configuration of the above-described data dividing device 100 (inserting each code of the functional block) by HDL description, logically synthesizing the HDL description and giving it to the ASIC or PLD, The data dividing device 100 can be manufactured.

(Appendix 1) Computer
When a group of multidimensional data that corresponds to an explanatory variable that represents the hierarchy of a multidimensional data group is used as a division target group, a statistical test is performed based on the distribution of objective variables that can represent the distribution bias in the division target group. Determination means for determining the attribute of the division target group by performing,
Based on the determination result determined by the determining means, the child group selected from the child group group by dividing the group to be divided into child group groups belonging to the child hierarchy of the hierarchy to which the group to be divided belongs. Dividing means for newly setting the dividing target group,
When an arbitrary group constituting the group set obtained by the dividing unit is an integration target group, the integration of the child group group based on the attribute of the child group group belonging to the child hierarchy of the hierarchy of the integration target group Integration means to integrate the target group and obtain the integration result,
A data division program characterized by functioning as

(Appendix 2) The determination means includes
When it is determined that the statistical test is available based on the number of multidimensional data belonging to the division target group, the attribute of the division target group is determined by performing the statistical test. The data division program according to appendix 1.

(Supplementary Note 3) The determination means includes
If it is determined that the statistical test is not available based on the number of multidimensional data belonging to the division target group, the division target group stops transitioning to a child group group belonging to a child hierarchy of the hierarchy To determine that the group
The dividing means includes
The data division program according to appendix 2, wherein a group not selected as the division target group is newly set as the division target group.

(Supplementary Note 4)
The data division program according to appendix 1 or 2, wherein it is determined whether or not the distribution of the objective variable in the division target group is a characteristic distribution.

(Supplementary Note 5) The determination means includes:
When the objective variable is a fixed value, the distribution of the objective variable in the group to be divided is characterized by a significance test between the distribution of the objective variable in the group to be divided and the distribution of the objective variable in the entire group set. The data division program according to appendix 4, wherein it is determined whether or not the distribution is a typical distribution.

(Appendix 6) The determination means includes
If the two distributions are significantly different, the distribution of the objective variable in the group to be divided is determined to be a characteristic distribution of the group to be divided by comparing with the distribution of the entire group set,
The dividing means includes
The data division program according to appendix 5, wherein a group not selected as the division target group is newly set as the division target group.

(Appendix 7) The determination means includes
When the objective variable is a numerical value, by performing a fitness test between the distribution of the objective variable in the division target group and a known probability distribution model estimated from the distribution, the objective variable of the division target group The data division program according to appendix 4, wherein it is determined whether or not the distribution is a characteristic distribution.

(Appendix 8) The determination means includes
If the distribution of the objective variable in the group to be divided and the probability distribution model are not different, determine that the distribution of the objective variable in the group to be divided is a characteristic distribution according to the probability distribution model,
The dividing means includes
The data division program according to appendix 7, wherein a group not selected as the division target group is newly set as the division target group.

(Supplementary note 9)
When the distribution of the objective variable in the division target group is not a characteristic distribution according to the probability distribution model and there is no child group group belonging to the child hierarchy, the division target group is included in a child hierarchy of the hierarchy. It is determined that the group stops dividing into groups to which it belongs,
The dividing means includes
The data division program according to appendix 4, wherein a group not selected as the division target group is newly set as the division target group.

(Supplementary Note 10) The determination means includes:
When the distribution of the objective variable in the division target group is not a characteristic distribution according to the probability distribution model and there are child group groups belonging to the child hierarchy, the division target group is included in the child hierarchy of the hierarchy. It is determined as a group that can be determined by the attribute of the child group to which it belongs,
The dividing means includes
The data division program according to appendix 4, wherein the child group is newly set as the division target group.

(Appendix 11) The integration means
When all the child groups belonging to the child hierarchy of the hierarchy of the integration target group are determined to be groups to be divided by the determination unit, the child group group is deleted, and the integration target group is changed to the child group The data division program according to any one of appendices 1 to 10, wherein the same attribute as that of the group is used.

(Supplementary Note 12) The integration means includes:
When it is determined that the distribution of the objective variable in each child group belonging to the child hierarchy of the hierarchy of the integration target group is all characteristic distribution, the child group group is not integrated into the integration target group. The data division program according to any one of supplementary notes 1 to 11, characterized in that the data division program is characterized.

(Additional remark 13) The said computer-readable recording medium which recorded the data division | segmentation program as described in any one of additional marks 1-12.

(Supplementary note 14) When a group of multidimensional data corresponding to an explanatory variable representing a hierarchy of a multidimensional data group is set as a division target group, it is based on the distribution of objective variables that can represent the distribution bias in the division target group. A determination means for determining an attribute of the division target group by performing a statistical test;
Based on the determination result determined by the determining means, the child group selected from the child group group by dividing the group to be divided into child group groups belonging to the child hierarchy of the hierarchy to which the group to be divided belongs. Dividing means for newly setting the group to be divided;
When an arbitrary group constituting the group set obtained by the dividing unit is an integration target group, the integration of the child group group based on the attribute of the child group group belonging to the child hierarchy of the hierarchy of the integration target group Integration means to integrate into the target group and obtain the integration results;
A data dividing device comprising:

(Supplementary Note 15) When a group of multidimensional data corresponding to an explanatory variable representing a hierarchy of a multidimensional data group is set as a division target group, it is based on a distribution of objective variables that can represent a distribution bias in the division target group. A determination step of determining an attribute of the division target group by performing a statistical test;
Based on the determination result determined in the determination step, the child group selected from the child group group by dividing the child group group into child group groups belonging to child hierarchies of the hierarchy to which the target group to be divided belongs. A dividing step to newly set the dividing target group,
When an arbitrary group constituting the group set obtained by the dividing step is an integration target group, the integration of the child group group based on the attribute of the child group group belonging to the child hierarchy of the hierarchy of the integration target group An integration process to integrate the target group and obtain the integration results;
A data partitioning method characterized by comprising:

  As described above, the data division program according to the present invention, the recording medium on which the program is recorded, the data division device, and the data division method relate to multidimensional data division processing having a plurality of items, such as patent publication data, One of the items is suitable for analyzing data having a hierarchical structure. It is also suitable for analyzing data in which one item is described in text.

It is explanatory drawing which shows the hardware constitutions of the data division | segmentation apparatus concerning embodiment of this invention. It is a block diagram which shows the functional structure of the data division | segmentation apparatus concerning embodiment of this invention. It is explanatory drawing which shows the hierarchical structure data which makes an explanatory variable a root. It is explanatory drawing which shows the determination table (initial state). 5 is a chart showing the number of multidimensional data for each objective variable X, Y, Z in group A and the number of multidimensional data for each objective variable X, Y, Z in overall group R. It is explanatory drawing which shows the hierarchical structure data after a division process (before integration process). It is a chart which shows the determination table after a division process (before an integration process). It is a graph which shows distribution of the data frequency of each objective variable in group A1. It is a graph which shows distribution of the data frequency of each objective variable in group A1 (an expected value frequency is added). It is explanatory drawing which shows the hierarchical structure data in integration processing. It is explanatory drawing which shows the hierarchical structure data after an integration process. It is a chart which shows the determination table after an integration process. It is a flowchart which shows the data division | segmentation processing procedure concerning embodiment of this invention. It is a flowchart which shows the detailed process sequence of the determination / division | segmentation process shown in FIG. It is a flowchart which shows the detailed process sequence of the determination process shown in FIG. It is a flowchart which shows the detailed process sequence of the integration process shown in FIG.

Explanation of symbols

DESCRIPTION OF SYMBOLS 100 Data division apparatus 201 Input part 202 Judgment part 203 Division part 204 Integration part 205 Output part 300,600,1100 Hierarchical structure data 400 Determination table

Claims (8)

Computer
When a group of multidimensional data that corresponds to an explanatory variable that represents the hierarchy of a multidimensional data group is used as a division target group, a statistical test is performed based on the distribution of objective variables that can represent the distribution bias in the division target group. Determination means for determining the attribute of the division target group by performing,
Based on the determination result determined by the determining means, the child group selected from the child group group by dividing the group to be divided into child group groups belonging to the child hierarchy of the hierarchy to which the group to be divided belongs. Dividing means for newly setting the dividing target group,
When an arbitrary group constituting the group set obtained by the dividing unit is an integration target group, the integration of the child group group based on the attribute of the child group group belonging to the child hierarchy of the hierarchy of the integration target group Integration means to integrate the target group and obtain the integration result,
A data division program characterized by functioning as The determination means includes
When it is determined that the statistical test is available based on the number of multidimensional data belonging to the division target group, the attribute of the division target group is determined by performing the statistical test. The data division program according to claim 1. The determination means includes
If it is determined that the statistical test is not available based on the number of multidimensional data belonging to the division target group, the division target group stops transitioning to a child group group belonging to a child hierarchy of the hierarchy To determine that the group
The dividing means includes
The data division program according to claim 2, wherein a group not selected as the division target group is newly set as the division target group. The determination means includes
The data division program according to claim 1 or 2, wherein it is determined whether or not the distribution of the objective variable in the division target group is a characteristic distribution. The determination means includes
When the objective variable is a fixed value, the distribution of the objective variable in the group to be divided is characterized by a significance test between the distribution of the objective variable in the group to be divided and the distribution of the objective variable in the entire group set. The data division program according to claim 4, wherein it is determined whether or not the distribution is a typical distribution.   6. A computer-readable recording medium on which the data division program according to claim 1 is recorded. When a group of multidimensional data that corresponds to an explanatory variable that represents the hierarchy of a multidimensional data group is used as a division target group, a statistical test is performed based on the distribution of objective variables that can represent the distribution bias in the division target group. Determination means for determining the attribute of the group to be divided by performing,
Based on the determination result determined by the determining means, the child group selected from the child group group by dividing the group to be divided into child group groups belonging to the child hierarchy of the hierarchy to which the group to be divided belongs. Dividing means for newly setting the group to be divided;
When an arbitrary group constituting the group set obtained by the dividing unit is an integration target group, the integration of the child group group based on the attribute of the child group group belonging to the child hierarchy of the hierarchy of the integration target group Integration means to integrate into the target group and obtain the integration results;
A data dividing device comprising: When a group of multidimensional data that corresponds to an explanatory variable that represents the hierarchy of a multidimensional data group is used as a division target group, a statistical test is performed based on the distribution of objective variables that can represent the distribution bias in the division target group. A determination step of determining an attribute of the division target group by performing,
Based on the determination result determined in the determination step, the child group selected from the child group group by dividing the child group group into child group groups belonging to child hierarchies of the hierarchy to which the target group to be divided belongs. A dividing step to newly set the dividing target group,
When an arbitrary group constituting the group set obtained by the dividing step is an integration target group, the integration of the child group group based on the attribute of the child group group belonging to the child hierarchy of the hierarchy of the integration target group An integration process to integrate the target group and obtain the integration results;
A data partitioning method characterized by comprising:

Images