JP2010153588A - Program, apparatus and method for analyzing data - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To efficiently extract remarkable characteristics, statistical significant difference and correlation between data. <P>SOLUTION: In a graph of response variables by kind after splitting, an axis of abscissas refers to an assignment record number and an axis of ordinates refers to a response variable (temperature T1). In addition, a group split in a first layer is indicated as a thick dotted oval. Thus, in regression tree analysis, an explanatory variable most affecting a value of the response variable by two-dividing a set and its range are automatically extracted. Since the explanatory variable is only the assignment record number E when sorted in the order of magnitudes of the response variables, it can be known which assignment record number E endures the most clear grouping when the response variables are split into two large and small record groups. That is, grouping as shown in Fig.9-1 results in set branching with the largest significant difference for the temperature T1. <P>COPYRIGHT: (C)2010,JPO&INPIT

Description

  The present invention relates to a data analysis program, a data analysis apparatus, and a data analysis method for analyzing a relationship between data widely handled in the industry.

  Conventionally, a large amount of data has been accumulated in computer systems at many sites including semiconductor processes. Simply accumulating these data does not generate revenue. Therefore, data mining is one of the data analysis techniques for efficiently finding out the regularity and features hidden in these large amounts of data, and is often used in the industry.

  Data mining has been widely used and has been successful in the fields of finance and distribution, but has recently been applied to the field of process data analysis including semiconductors.

  Data mining techniques include correlation analysis, clustering analysis, clustering analysis, neural network analysis, neural net analysis, MBR (Memory Based Reasoning) analysis, and the like.

  For example, there is data analysis (conventional example 1) in which all target data is analyzed at once. Further, there is a data analysis (conventional example 2) in which all records are divided into several record groups and analyzed, and the correlation existing for each record group is extracted (for example, Patent Documents 1 and 2 below) reference.).

  There is also a data analysis (conventional example 3) in which analysis is performed by dividing into record groups with different conditions, and the correlation existing for each record group is extracted. For example, data is analyzed according to conditions including equipment used in the manufacturing process.

JP 2006-086403 A JP 2006-338265 A

  However, the above-described data analysis of Conventional Example 1 has a problem in that data collected under various conditions are mixed and noises are generated from each other, so that originally existing features cannot be extracted. In the data analysis of Conventional Example 2 described above, the record order can be changed by sorting with a certain variable, but in principle, record groups are created in the order in which records exist, but the records that make up the record group The number is fixed and is not flexible according to the data contents. Therefore, there is a problem that a significant correlation is not often extracted.

  Furthermore, when the explanatory variable is an attribute value of a nominal variable such as a process using device, it is effective because the set division criterion is clear. However, as in the data analysis of Conventional Example 3 described above, when grouping with explanatory variables that are numerical variables, it is inefficient to group by the combination of all the numerical values because of a large number of division patterns. There was a problem.

  The present invention eliminates the problems caused by the prior art described above, and a data analysis program, a data analysis apparatus, and a data analysis method capable of efficiently extracting remarkable features, statistically significant differences, and correlations between data The purpose is to provide.

  In order to solve the above-described problems and achieve the object, the data analysis program, the data analysis apparatus, and the data analysis method are configured to analyze a sequence number as an explanatory variable and a target variable corresponding to the sequence number as a record. Acquire data, sort the records of the acquired data to be analyzed in ascending or descending order of the objective variable, assign a new sequence number to the sorted records, and assign the order based on regression tree analysis It is a requirement to divide the sorted records into two divided groups with a specific number selected from the numbers (hereinafter referred to as “allocation order number”) as a boundary, and to output the two divided groups. To do.

  According to the data analysis program, the data analysis apparatus, and the data analysis method, the allocation sequence number is set as the only explanatory variable, so that the location where the objective variable greatly varies can be grouped as a boundary.

  According to the data analysis program, the data analysis apparatus, and the data analysis method, it is possible to efficiently extract remarkable features, statistically significant differences, and correlations between data.

  Exemplary embodiments of a data analysis program, a data analysis apparatus, and a data analysis method will be described below in detail with reference to the accompanying drawings.

  FIG. 1 is an explanatory diagram illustrating an example of analysis target data. The analysis target data has explanatory variables and a plurality of types of objective variables. The explanatory variable is a record number (order number) indicating the order. For example, it is a series of numbers assigned to the same kind of manufacturing apparatus. Here, the record number is e, and e = 1, 2, 3,. That is, the same kind of manufacturing apparatuses from No. 1 to No. 25 are shown.

  The objective variable is a parameter obtained for each record number. In this example, four types of objective variables are prepared. These objective variables indicate, for example, temperatures t1 to t4 under different conditions (for example, by time zone) for each manufacturing apparatus. Here, a combination of a record number as an explanatory variable and its objective variable is referred to as a record. Therefore, a set of records for each type of objective variable becomes analysis target data. In FIG. 1, four types of analysis target data of temperatures t1 to t4 are collected.

  FIGS. 2-1 to 2-4 are explanatory diagrams illustrating graphs for different types of objective variables. FIG. 2-1 is a graph for the temperature t1, FIG. 2-2 is the temperature t2, FIG. 2-3 is the temperature t3, and FIG. 2-4 is the temperature t4. In the graphs of FIGS. 2-1 to 2-4, the horizontal axis indicates the record number, and the vertical axis indicates the temperature.

FIGS. 3-1 to 3-6 are explanatory diagrams illustrating correlation diagrams for each combination of objective variables for all record numbers. 3-1 is a correlation diagram between temperature t1 and temperature t2, FIG. 3-2 is a correlation diagram between temperature t1 and temperature t3, FIG. 3-3 is a correlation diagram between temperature t1 and temperature t4, and FIG. FIG. 3-5 is a correlation diagram between the temperature t2 and the temperature t4, and FIG. 3-6 is a correlation diagram between the temperature t3 and the temperature t4. The straight lines in FIGS. 3-1 to 3-6 are linear regression equations, and R ² is the contribution rate. It can be seen that no significant correlation is found in any of the graphs of FIGS. 2-1 to 2-4 and FIGS. 3-1 to 3-6.

(Hardware configuration of data analyzer)
FIG. 4 is a block diagram showing a hardware configuration of the data analysis apparatus according to the present embodiment. In FIG. 4, the data analysis apparatus includes a CPU (Central Processing Unit) 401, a ROM (Read-Only Memory) 402, a RAM (Random Access Memory) 403, a magnetic disk drive 404, a magnetic disk 405, and an optical disk drive. 406, an optical disk 407, a display 408, an I / F (Interface) 409, a keyboard 410, a mouse 411, a scanner 412, and a printer 413. Each component is connected by a bus 400.

  Here, the CPU 401 controls the entire data analysis apparatus. The ROM 402 stores programs such as a boot program. The RAM 403 is used as a work area for the CPU 401. The magnetic disk drive 404 controls the reading / writing of the data with respect to the magnetic disk 405 according to control of CPU401. The magnetic disk 405 stores data written under the control of the magnetic disk drive 404.

  The optical disk drive 406 controls reading / writing of data with respect to the optical disk 407 according to the control of the CPU 401. The optical disk 407 stores data written under the control of the optical disk drive 406, or causes the computer to read data stored on the optical disk 407.

  The display 408 displays data such as a document, an image, and function information as well as a cursor, an icon, or a tool box. As the display 408, for example, a CRT, a TFT liquid crystal display, a plasma display, or the like can be adopted.

  An interface (hereinafter abbreviated as “I / F”) 409 is connected to a network 414 such as a LAN (Local Area Network), a WAN (Wide Area Network), and the Internet through a communication line, and the other via the network 414. Connected to other devices. The I / F 409 controls an internal interface with the network 414 and controls data input / output from an external apparatus. For example, a modem or a LAN adapter may be employed as the I / F 409.

  The keyboard 410 includes keys for inputting characters, numbers, various instructions, and the like, and inputs data. Moreover, a touch panel type input pad or a numeric keypad may be used. The mouse 411 moves the cursor, selects a range, moves the window, changes the size, and the like. A trackball or a joystick may be used as long as they have the same function as a pointing device.

  The scanner 412 optically reads an image and takes in the image data into the data analysis apparatus. The scanner 412 may have an OCR (Optical Character Reader) function. The printer 413 prints image data and document data. As the printer 413, for example, a laser printer or an ink jet printer can be adopted.

(Functional configuration of data analysis device)
FIG. 5 is a block diagram illustrating a functional configuration of the data analysis apparatus. The data analysis apparatus 500 includes an acquisition unit 501, a sorting unit 502, an allocation unit 503, a dividing unit 504, a specifying unit 505, a selecting unit 506, a correlation information calculating unit 507, and a unity degree calculating unit 508. , A determination unit 509 and an output unit 510. Specifically, each of these functions is executed by, for example, causing the CPU 401 to execute a program stored in a storage area such as the ROM 402, the RAM 403, the magnetic disk 405, and the optical disk 407 illustrated in FIG. 4 or by the I / F 409. Realize its function. Each function reads data generated by the preceding function from a storage device such as the ROM 402, the RAM 403, and the magnetic disk 405, and stores the information generated by the function in the storage device.

  The acquisition unit 501 has a function of acquiring analysis target data in which a sequence number serving as an explanatory variable and a target variable corresponding to the sequence number are records. Specifically, for example, the analysis target data shown in FIG. 1 is acquired. More specifically, the analysis target data is received from the outside, or the analysis target data stored in a storage device such as the ROM 402, the RAM 403, and the magnetic disk 405 is read.

  The sort unit 502 has a function of sorting the records of the analysis target data in ascending order or descending order of the objective variable. FIG. 6A to FIG. 6D are explanatory diagrams illustrating sorting results for each type of objective variable. 6A to 6D, the temperatures T1 to T4 are data strings in which the temperatures t1 to t4 are sorted in ascending order. According to this sort, the record number e which is an explanatory variable is also rearranged.

  The assigning unit 503 has a function of assigning a new sequence number to the records after sorting by the sorting unit 502. Since the record numbers rearranged by the sorting unit 502 are not sequential, as shown in FIGS. 6-1 to 6-4, E is a record number that represents a new order, and E = 1, 2, 3, ..., 25. This record number E is referred to as an assigned record number. The assigned record numbers are assigned objective variables in ascending order from E = 1. Thereby, the combination of the original record number e, the assigned record number E, and the objective variable becomes one record.

  FIG. 7A to FIG. 7D are explanatory diagrams illustrating graphs for each type of objective variable after sorting. The horizontal axis is the assigned record number E, and the vertical axis is the objective variable (temperatures T1 to T4).

  In FIG. 5, the dividing unit 504 has a function of dividing the sorted objective variable into two divided groups with a specific number selected from the allocation record numbers based on the regression tree analysis as a boundary. That is, in order to divide based on the allocation record number E instead of the record number e before allocation, the regression tree analysis is executed using the allocation record number E as an explanatory variable. In regression tree analysis, S0 is the sum of squares of the set of objective variables before division, S1 is the sum of squares of the set of one objective variable after division, and S2 is the sum of squares of the other set of objective variables after division. , ΔS is divided into S1 and S2 so as to be maximum.

ΔS = S0− (S1 + S2) (1)

  Specifically, for example, in FIG. 6A, a set of temperatures T1 with allocation record numbers E = 1 to 25 is group G0, and a set of temperatures T1 with allocation record numbers E = 1 to i is group G1 (i is 1 ≦ 1). i ≦ 25), and a set of temperatures T1 of assigned record numbers E = i + 1 to 25 is set as a group G2, and the respective square sums S0 to S2 are calculated.

  Then, grouping is performed so that ΔS is maximized. That is, by specifying the allocation record number E = i serving as the boundary, it can be divided into two groups. This division processing is recursively executed with the divided group as a division source. As a result, a regression tree is generated.

  8A to 8D are explanatory diagrams illustrating regression trees generated for each objective variable. The level to be divided is set in advance. For example, the user can set according to the analysis purpose, such as dividing up to the k-th hierarchy, or dividing until the maximum value of ΔS becomes a threshold value or less. If the root of the regression tree (group G) is the 0th hierarchy, the regression trees shown in FIGS. 8-1 to 8-4 are divided to the 2nd hierarchy.

  In FIG. 8A, focusing on the first hierarchy, the group G of objective variables having allocation record numbers E = 1 to 25 is divided into a group G10-1 and a group G10-2. The group G10-1 is a set of objective variables with allocation record numbers E = 1 to 15 and the group G10-2 is a set of objective variables with allocation record numbers E = 16-25.

  Also, in FIG. 8B, focusing on the first hierarchy, the group G of objective variables with allocation record numbers E = 1 to 25 is divided into a group G20-1 and a group G20-2. The group G20-1 is a set of objective variables with allocation record numbers E = 1 to 15, and the group G20-2 is a set of objective variables with allocation record numbers E = 16-25.

  Further, in FIG. 8C, focusing on the first hierarchy, the group G of the objective variables having the allocation record numbers E = 1 to 25 is divided into a group G30-1 and a group G30-2. The group G30-1 is a set of objective variables with allocation record numbers E = 1 to 20, and the group G30-2 is a set of objective variables with allocation record numbers E = 21 to 25.

  Also, in FIG. 8-4, focusing on the first hierarchy, the group G of objective variables with allocation record numbers E = 1 to 25 is divided into a group G40-1 and a group G40-2. The group G40-1 is a set of objective variables with allocation record numbers E = 1 to 14, and the group G40-2 is a set of objective variables with allocation record numbers E = 15 to 25.

  FIG. 9A to FIG. 9D are explanatory diagrams illustrating graphs for each type of objective variable after division. The horizontal axis is the assigned record number, and the vertical axis is the objective variable (temperatures T1 to T4). In each graph, the groups divided in the first hierarchy are indicated by bold dotted ellipses.

  As described above, in the regression tree analysis, the explanatory variable that most influences the value of the objective variable and its range are automatically extracted by dividing the set into two. Here, since the explanatory variable is only the allocation record number E when the objective variable is sorted in the order of the size of the objective variable, when the objective variable is divided into two large and small record groups, the allocation record number E is most clearly divided. Shows whether it can be divided. That is, the grouping as shown in FIGS. 9-1 to 9-4 is a set branch in which the most significant difference occurs between the temperatures T1 to T4.

  Further, in FIG. 5, the specifying unit 505 has a function of specifying a sequence number corresponding to the objective variable constituting each divided group. Specifically, in each record shown in FIGS. 6-1 to 6-4, the original sequence number e corresponding to the temperatures T1 to T4 is read. Thus, by specifying the original sequence number e, information that characterizes the divided groups is given to the graphs shown in FIGS. 2-1 to 2-4.

  10A to 10D are explanatory diagrams illustrating graphs according to the types of objective variables to which divided groups are assigned. In FIGS. 10-1 to 10-4, the groups divided in the first hierarchy are indicated by thick dotted ovals. This ellipse is information that characterizes the divided group.

  Taking FIG. 10-1 as an example, the temperature t1 of the original record numbers e = 1 to 5 and 16 to 25 corresponding to the assigned record numbers E = 1 to 15 corresponds to the divided group G10-1. Similarly, the temperature t1 of the original record numbers e = 6 to 15 corresponding to the allocation record numbers E = 16 to 25 corresponds to the divided group G10-2.

  There is some difference between the two divided groups G10-1 and G10-2, which affects other variable values, and when analyzed at the same time, the remarkable correlation that originally existed may be hidden. It is done. In this case, a significant correlation is detected for each of the record groups of the divided groups G10-1 and G10-2 by the selection unit and the correlation information calculation unit described later.

  In FIG. 5, the selection unit 506 has a function of selecting two types of analysis target data from a plurality of types of analysis target data having a common explanatory variable. Specifically, for example, analysis target data having a temperature t1 as an objective variable and analysis target data having a temperature t2 as an objective variable are selected. The combination is selected based on the brute force type of the objective variable.

The correlation information calculation unit 507 correlates the sequence number identified by the identification unit 505 and the objective variable corresponding to the sequence number among the objective variables of the two types of analysis target data selected by the selection unit 506. It has a function of calculating information representing the relationship. Specifically, for example, a target variable common to the original record number corresponding to the divided group is extracted from the two types of analysis target data selected, and a correlation such as a linear regression equation, a correlation coefficient R, and a contribution rate R ^{2 is obtained.} Calculate relationship information.

  FIG. 11A is an explanatory diagram of a correlation diagram of temperatures t1 and t2 for record numbers e = 6 to 15. FIG. 11B is an explanatory diagram of a correlation diagram of the temperatures t1 and t3 for the record numbers e = 6 to 15. FIG. 11C is an explanatory diagram of a correlation diagram of the temperatures t1 and t4 for the record numbers e = 6 to 15. FIG. 11-4 is an explanatory diagram of a correlation diagram of temperatures t2 and t3 for record numbers e = 6 to 15. FIG. 11-5 is an explanatory diagram of a correlation diagram between temperatures t2 and t4 for record numbers e = 6 to 15. FIG. 11-6 is an explanatory diagram of a correlation diagram of temperatures t3 and t4 for record numbers e = 6 to 15. These are correlation diagrams specialized for the divided group G10-2 corresponding to the record numbers e = 6 to 15.

  FIG. 12A is an explanatory diagram of a correlation diagram of temperatures t1 and t2 for record numbers e = 1 to 5 and 16 to 25. FIG. 12B is an explanatory diagram of a correlation diagram of the temperatures t1 and t3 for the record numbers e = 1 to 5 and 16 to 25. FIG. 12C is an explanatory diagram of a correlation diagram of the temperatures t1 and t4 with respect to the record numbers e = 1 to 5 and 16 to 25. FIG. 12-4 is an explanatory diagram of a correlation diagram of the temperatures t2 and t3 for the record numbers e = 1 to 5 and 16 to 25. FIG. 12-5 is an explanatory diagram of a correlation diagram of the temperatures t2 and t4 for the record numbers e = 1 to 5 and 16 to 25. FIG. 12-6 is an explanatory diagram of a correlation diagram of the temperatures t3 and t4 for the record numbers e = 1 to 5 and 16 to 25. These are correlation diagrams specialized for the divided group G10-1 corresponding to the record numbers e = 1 to 5, 16 to 25.

Among the correlation diagrams of FIGS. 11-1 to 11-6 and FIGS. 12-1 to 12-6, in the correlation diagram of FIG. 11-4, the linear regression equation is y = x and the contribution rate R ² = 1. Therefore, it can be seen that the temperatures t2 and t3 for the record numbers e = 6 to 15 have a significant correlation. Thus, it is good also as calculating | requiring a correlation about all the combinations of temperature, and taking out a remarkable correlation.

  In FIG. 5, the unity degree calculation unit 508 has a function of calculating a degree of unity that represents the clarity of set division of two division groups for each of a plurality of types of analysis target data having a common explanatory variable. Specifically, for example, the sum of squares S0 is calculated for the set of objective variables that are the division sources. In addition, the sum of squares S1 is calculated for the set of objective variables constituting one of the divided groups. Further, the sum of squares S2 is calculated for the set of objective variables that constitute the other divided group. The unity degree U is calculated by the following equation (2).

U = [{S0− (S1 + S2)} / (S1 + S2)] × 100 (2)

  The range of the unity degree U is 0 ≦ U ≦ 100, and the larger the value, the clearer the set partitioning is performed. Here, in the division group of the first hierarchy shown in FIGS. 8-1 to 8-4, the unity degree when dividing into the division groups G30-1 and G30-2 shown in FIG. = 86.7813).

  In FIG. 5, the determination unit 509 has a function of determining an output target from a plurality of types of analysis target data based on each unit degree calculated by the unit degree calculation unit 508. Specifically, in a plurality of types of analysis target data, it is determined which set division of analysis target data is clear among divisions in a certain hierarchy of regression trees. For example, a divided group in which a set division of the maximum value among the calculated unity degrees is performed is an output target. More specifically, a flag for identifying an output target is set for the divided group that has undergone the set division that maximizes the unity.

  Taking the regression tree shown in FIGS. 7-1 to 7-4 as an example, the case where all records are divided into two by the value of temperature t3 in the first hierarchy (FIG. 7-3) is most clearly divided into large and small. Therefore, there is a high possibility that the factor is clearly present. In other words, since there are clearly different conditions, analyzing with all data results in analyzing a noisy state. Therefore, by dividing and analyzing, the hidden correlation is easily extracted.

  Further, the correlation information calculation unit 507 may calculate the correlation information for the divided group determined as the output target by the determination unit 509. Thereby, it is not necessary to calculate correlation information other than the output target, and the calculation process can be made more efficient. Further, since only necessary information can be obtained, the efficiency of data analysis can be improved.

  FIG. 13A is an explanatory diagram of a correlation diagram of temperatures t1 and t2 with respect to record numbers e = 1 to 5 and 11 to 25. FIG. 13-2 is an explanatory diagram of a correlation diagram of the temperatures t1 and t3 for the record numbers e = 1 to 5 and 11 to 25. FIG. 13C is an explanatory diagram of a correlation diagram of the temperatures t1 and t4 with respect to the record numbers e = 1 to 5 and 11 to 25. FIG. 13-4 is an explanatory diagram of a correlation diagram of the temperatures t2 and t3 for the record numbers e = 1 to 5 and 11 to 25. FIG. 13-5 is an explanatory diagram of a correlation diagram of temperatures t2 and t4 with respect to record numbers e = 1 to 5 and 11 to 25. FIG. 13-6 is an explanatory diagram of a correlation diagram of the temperatures t3 and t4 for the record numbers e = 1 to 5 and 11 to 25. These are correlation diagrams specialized for the division group G30-1 corresponding to the record numbers e = 1 to 5, 11 to 25.

  FIG. 14A is an explanatory diagram of a correlation diagram of temperatures t1 and t2 for record numbers e = 6 to 10. FIG. 14-2 is an explanatory diagram of a correlation diagram of temperatures t1 and t3 for record numbers e = 6 to 10. FIG. 14C is an explanatory diagram of a correlation diagram of the temperatures t1 and t4 for the record numbers e = 6 to 10. FIG. 14-4 is an explanatory diagram of a correlation diagram of temperatures t2 and t3 for record numbers e = 6 to 10. FIG. 14-5 is an explanatory diagram of a correlation diagram of temperatures t2 and t4 for record numbers e = 6 to 10. FIG. 14-6 is an explanatory diagram of a correlation diagram of temperatures t3 and t4 for record numbers e = 6 to 10. These are correlation diagrams specialized for the divided group G30-2 corresponding to the record numbers e = 6-10.

  In FIG. 5, the output unit 510 has a function of outputting various data. The output unit 510 outputs to a display, a printer, or an I / F. As output data, for example, data to be analyzed shown in FIG. 1, graphs before analysis shown in FIGS. 2-1 to 2-4, and data before analysis shown in FIGS. Correlation diagram, analysis target data after sorting and allocation shown in FIGS. 6-1 to 6-4, temperature characteristic graphs of allocation record numbers shown in FIGS. 7-1 to 7-4, FIGS. The regression tree and the degree of unity shown in 8-4, the temperature characteristic graph of the allocation record number after the division shown in FIGS. 9-1 to 9-4, and the division group shown in FIGS. FIG. 11-1 to FIG. 11-6, FIG. 12-1 to FIG. 12-6, FIG. 13-1 to FIG. 13-6, and FIG. 14-1 to FIG. 14-6. The correlation diagram shown in is output.

(Data analysis procedure)
FIG. 15 is a flowchart showing a data analysis processing procedure by the data analysis apparatus 500 according to this embodiment. First, an acquisition process of data to be analyzed is executed by the acquisition unit 501 (step S1501), and an analysis process is executed (step S1502). Next, a determination process by the determination unit 509 is executed (step S1503), and a correlation analysis process is executed (step S1504). Then, output processing is executed by the output unit 510 (step S1505), and a series of processing ends.

  FIG. 16 is a flowchart showing a detailed processing procedure of the analysis processing (step S1502) shown in FIG. First, the record group of the analysis target data is sorted in ascending order (or descending order) of the objective variable by the sorting unit 502 (step S1601). Next, the allocation unit 503 allocates allocation record numbers to the sorted record group in the sort order (step S1602). Then, the dividing process is executed by the dividing unit 504 (step S1603), and the process proceeds to step S1503.

  FIG. 17 is a flowchart showing a detailed processing procedure of the dividing process (step S1603) shown in FIG. First, it is determined whether there is unprocessed analysis target data (step S1701). For example, it is determined whether or not there is an unprocessed objective variable group for the analysis target data shown in FIG. If there is unprocessed analysis target data (step S1701: Yes), unprocessed analysis target data is acquired (step S1702).

  Next, it is determined whether there is an unprocessed objective variable group that can be divided (step S1703). If there is an unprocessed objective variable group (step S1703: YES), one unprocessed objective variable group that can be divided is selected (step S1704). For example, in the case of division for the first time (0th hierarchy), the data group of the allocation record number E = 1 to 25 of the temperature T1 is selected (step S1704). Then, the sum of squares S0 of the selected objective variable group is calculated (step S1705). Here, the variable of the allocation record number E is set to i, and i = m is set (step S1706). m is the first record number of the selection target variable group. In the case of the data group of the allocation record number E = 1 to 25 of the temperature T1, m = 1.

  Then, the sum of squares S1 of the first divided groups from the allocation record numbers E = m to i is calculated (step S1707). Further, the sum of squares S2 of the second divided groups from the allocation record number E = i + 1 to n is calculated (step S1708). Here, n is the record number at the end of the selection target variable group. For example, in the case of the data group of the allocation record number E = 1 to 25 of the temperature T1, n = 25. Thereafter, ΔS is calculated and stored in the storage device (step S1709), and the process returns to step S1703.

  In step S1703, if there is no unprocessed record group that can be divided (step S1703: No), the divided group having the maximum ΔS calculated so far is specified (step S1710). In the case of the data group of the allocation record number E = 1 to 25 of the temperature T1, it is divided into a group G10-1 and a group G10-2 as shown in FIGS. Then, the record number before sorting corresponding to the objective variable constituting the specified divided group is specified (step S1711), and the process returns to step S1701.

  FIG. 18 is a flowchart showing a detailed processing procedure of the determination process (step S1503) shown in FIG. First, it waits for the hierarchy designation of the regression tree (step S1801: No), and if there is a hierarchy designation (step S1801: Yes), it is determined whether there is unprocessed analysis target data (step S1802).

  If there is unprocessed data to be analyzed (step S1803: Yes), the division group of the designated hierarchy is selected (step S1803). Then, the unity degree is calculated (step S1804), and the process returns to step S1802. On the other hand, in step S1802, when there is no unprocessed analysis target data (step S1802: No), a division group to be output is determined from the unity (step S1805). Then, control goes to a step S1504.

  FIG. 19 is a flowchart showing a detailed processing procedure of the correlation analysis processing (step S1504) shown in FIG. First, it is determined whether there is an unprocessed divided group among the divided groups determined as output targets (step S1901). If there is an unprocessed divided group (step S1901: Yes), an unprocessed divided group is determined. A group is selected (step S1902). Next, the record number of the selected divided group before sorting is specified (step S1903).

  Then, it is determined whether or not there is an unprocessed combination of objective variables for the specified record number (step S1904). When there is an unprocessed combination (step S1904: Yes), a linear regression equation, a correlation coefficient, and a contribution rate are calculated (step S1905), and the process returns to step S1904. On the other hand, if there is no unprocessed combination in step S1904 (step S1904: No), the process returns to step S1901. In step S1901, if there is no unprocessed divided group (step S1901: No), the process proceeds to output processing (step S1505).

  As described above, according to the present embodiment, each numerical variable that is a target variable is sorted in ascending order by its value, and a record number (assigned record number) that becomes a new order for each record of the sorting result. , The regression tree analysis is executed with the assigned record number as the only explanatory variable. Since the regression tree analysis is a process of extracting the influence of the explanatory variable on the size of the objective variable, the combination of the explanatory variable and the objective variable as the constituent elements of one record is changed.

  As a result, in the data analysis of this embodiment, when the objective variable value is divided into a high group and a low group, instead of extracting a large section and a small section of the target variable, it is most obvious where to divide. You can ask for it. Therefore, it can be checked whether there is a significant correlation.

  One of the effective methods for extracting useful information from data is to visualize as much information as possible on a graph or the like. In particular, in order to extract information related to the correlation between data, the relevant variables are displayed on the vertical and horizontal axes, and the relationship is grasped. From this two-dimensional distribution, information on the correlation coefficient value and the presence or absence of bias in the data distribution is obtained. For this purpose, it is not necessary to separate the data, it can be used as it is, and it is not necessary to create a new record name.

  On the other hand, if the records are sorted by the value of the objective variable and the horizontal axis is displayed as the newly created variable, the graph shows information for only one variable, and the relationship with other variables is ignored and the amount of information Decreases overwhelmingly. In the present embodiment, it is possible to extract a correlation existing between a plurality of variables based on such a graph relating to only one variable.

  When analyzing the causal relationship of regression tree analysis, the cause explanatory variable and the objective variable resulting are clearly distinguished and analyzed. If you make this mistake, you won't be looking at the causal relationship, which is a meaningless analysis. Therefore, analyzing (dividing) the objective variable itself using the objective variable is considered meaningless for the purpose of analyzing the causal relationship. In this embodiment, analysis that is considered meaningless, that is, highly correlated variables that do not require distinction between explanatory variables (causes) and objective variables (results) are extracted.

  That is, it is to extract how the objective variable itself is divided and the distribution of the objective variable is clearly divided. Therefore, by performing a regression tree analysis on the objective variable group to which the assigned record numbers are assigned after sorting, the values of the objective variables are arranged in order of magnitude on the horizontal axis (assigned record number). Therefore, where the explanatory variable is divided into two sets, it is extracted whether the size of the objective variable is most clearly defined, that is, whether the objective variables having similar values are grouped.

  In addition, in order to find a partially occurring correlation, it is necessary to divide analysis target data into various patterns, but there are very many division patterns, and it is difficult to cover all of them. Moreover, even if all the divided patterns can be created and a strong correlation can be extracted, if you do not know what the divided pattern means, you do not know what to improve.

  Therefore, in order to use some meaningful division, as a conventional method, there is a method in which a certain variable is used and arranged in the order of the values and divided at an arbitrary interval. It is possible to use regression tree analysis for the creation of the division pattern by a combination of conventional techniques. However, when regression tree analysis is used as it is, the division pattern that can be created is a two-division pattern of explanatory variables that clearly divides the objective variable. Limited.

  On the other hand, in the present embodiment, by creating a division pattern using the objective variable itself, it is possible to create a division pattern having the meaning that the value of the objective variable, which has been difficult to make in the past, is close. It becomes like this. Therefore, it is possible to detect a correlation in a division whose meaning has been difficult to find in the past.

  Further, by analyzing the division pattern of the explanatory variable using the explanatory variable itself by using the value of the new explanatory variable, it becomes possible to perform a clearer analysis with emphasized features. Further, by using a data mining technique such as regression tree analysis to create a division pattern using the variable itself, a division pattern can be created efficiently. By such different purposes, a method for utilizing and combining analysis methods that are conventionally considered meaningless are also effective.

  As described above, according to the present embodiment, it is possible to efficiently extract a remarkable feature, statistical significance difference, and correlation between data.

  The data analysis method described in this embodiment can be realized by executing a program prepared in advance on a computer such as a personal computer or a workstation. This program is recorded on a computer-readable recording medium such as a hard disk, a flexible disk, a CD-ROM, an MO, and a DVD, and is executed by being read from the recording medium by the computer. This program may be distributed via a network such as the Internet.

It is explanatory drawing which shows an example of analysis object data. It is explanatory drawing (the 1) which shows the graph according to the kind of objective variable. It is explanatory drawing (the 2) which shows the graph according to the kind of objective variable. It is explanatory drawing (the 3) which shows the graph according to the kind of objective variable. It is explanatory drawing (the 4) which shows the graph according to the kind of objective variable. It is explanatory drawing (the 1) which shows the correlation diagram for every combination of the objective variable about all the record numbers. It is explanatory drawing (the 2) which shows the correlation diagram for every combination of the objective variable about all the record numbers. It is explanatory drawing (the 3) which shows the correlation diagram for every combination of the objective variable about all the record numbers. It is explanatory drawing (the 4) which shows the correlation figure for every combination of the objective variable about all the record numbers. It is explanatory drawing (the 5) which shows the correlation figure for every combination of the objective variable about all the record numbers. It is explanatory drawing (the 6) which shows the correlation diagram for every combination of the objective variable about all the record numbers. It is a block diagram which shows the hardware constitutions of the data analysis apparatus concerning this Embodiment. It is a block diagram which shows the functional structure of a data analyzer. It is explanatory drawing (the 1) which shows the sort result according to the kind of objective variable. It is explanatory drawing (the 2) which shows the sort result according to the kind of objective variable. It is explanatory drawing (the 3) which shows the sort result according to the kind of objective variable. It is explanatory drawing (the 4) which shows the sorting result according to the kind of objective variable. It is explanatory drawing (the 1) which shows the graph according to the kind of objective variable after sorting. It is explanatory drawing (the 2) which shows the graph according to the kind of objective variable after sorting. It is explanatory drawing (the 3) which shows the graph according to the kind of objective variable after sorting. It is explanatory drawing (the 4) which shows the graph according to the kind of objective variable after sorting. It is explanatory drawing (the 1) which shows the regression tree produced | generated according to the objective variable. It is explanatory drawing (the 2) which shows the regression tree produced | generated according to the objective variable. It is explanatory drawing (the 3) which shows the regression tree produced | generated according to the objective variable. It is explanatory drawing (the 4) which shows the regression tree produced | generated according to the objective variable. It is explanatory drawing (the 1) which shows the graph according to the kind of objective variable after a division | segmentation. It is explanatory drawing (the 2) which shows the graph according to the kind of objective variable after a division | segmentation. It is explanatory drawing (the 3) which shows the graph according to the kind of objective variable after a division | segmentation. It is explanatory drawing (the 4) which shows the graph according to the kind of objective variable after a division | segmentation. It is explanatory drawing (the 1) which shows the graph according to the kind of the objective variable which allocated the division group. It is explanatory drawing (the 2) which shows the graph according to the kind of the objective variable which allocated the division group. It is explanatory drawing (the 3) which shows the graph according to the kind of the objective variable which allocated the division group. It is explanatory drawing (the 4) which shows the graph according to the kind of the objective variable which allocated the division group. It is explanatory drawing which shows the correlation diagram of temperature t1, t2 about record number e = 6-15. It is explanatory drawing which shows the correlation diagram of temperature t1, t3 about record number e = 6-15. It is explanatory drawing which shows the correlation diagram of temperature t1, t4 about record number e = 6-15. It is explanatory drawing which shows the correlation diagram of temperature t2, t3 about record number e = 6-15. It is explanatory drawing which shows the correlation diagram of temperature t2, t4 about record number e = 6-15. It is explanatory drawing which shows the correlation diagram of temperature t3, t4 about record number e = 6-15. It is explanatory drawing which shows the correlation diagram of temperature t1, t2 about record number e = 1-5 and 16-25. It is explanatory drawing which shows the correlation diagram of temperature t1, t3 about record number e = 1-5 and 16-25. It is explanatory drawing which shows the correlation diagram of temperature t1, t4 about record number e = 1-5, 16-25. It is explanatory drawing which shows the correlation diagram of temperature t2, t3 about record number e = 1-5 and 16-25. It is explanatory drawing which shows the correlation diagram of temperature t2, t4 about record number e = 1-5, 16-25. It is explanatory drawing which shows the correlation diagram of temperature t3, t4 about record number e = 1-5, 16-25. It is explanatory drawing which shows the correlation diagram of temperature t1, t2 about record number e = 1-5, 11-25. It is explanatory drawing which shows the correlation diagram of temperature t1, t3 about record number e = 1-5, 11-25. It is explanatory drawing which shows the correlation diagram of temperature t1, t4 about record number e = 1-5, 11-25. It is explanatory drawing which shows the correlation diagram of temperature t2, t3 about record number e = 1-5, 11-25. It is explanatory drawing which shows the correlation diagram of temperature t2, t4 about record number e = 1-5, 11-25. It is explanatory drawing which shows the correlation diagram of temperature t3, t4 about record number e = 1-5, 11-25. It is explanatory drawing which shows the correlation diagram of temperature t1, t2 about record number e = 6-10. It is explanatory drawing which shows the correlation diagram of temperature t1, t3 about record number e = 6-10. It is explanatory drawing which shows the correlation diagram of temperature t1, t4 about record number e = 6-10. It is explanatory drawing which shows the correlation diagram of temperature t2, t3 about record number e = 6-10. It is explanatory drawing which shows the correlation diagram of temperature t2, t4 about record number e = 6-10. It is explanatory drawing which shows the correlation diagram of temperature t3, t4 about record number e = 6-10. It is a flowchart which shows the data-analysis processing procedure by the data-analysis apparatus 500 concerning this Embodiment. 16 is a flowchart showing a detailed processing procedure of the analysis process (step S1502) shown in FIG. It is a flowchart which shows the detailed process sequence of the division | segmentation process (step S1603) shown in FIG. 16 is a flowchart showing a detailed processing procedure of the determination process (step S1503) shown in FIG. 16 is a flowchart showing a detailed processing procedure of correlation analysis processing (step S1504) shown in FIG.

Explanation of symbols

500 Data Analysis Device 501 Obtaining Unit 502 Sorting Unit 503 Allocation Unit 504 Division Unit 505 Identification Unit 506 Selection Unit 507 Correlation Information Calculation Unit 508 Grouping Level Calculation Unit 509 Determination Unit 510 Output Unit

Claims (7)

Computer
An acquisition means for acquiring analysis target data having a sequence number as an explanatory variable and a target variable corresponding to the sequence number as records;
Sort means for sorting records of data to be analyzed acquired by the acquisition means in ascending or descending order of the objective variable;
Assigning means for assigning a new sequence number to the records after sorting by the sorting means;
A division that divides the sorted records into two division groups with a specific number selected from the order numbers assigned by the assignment means based on regression tree analysis (hereinafter referred to as “assignment order numbers”) as boundaries. means,
Output means for outputting two divided groups divided by the dividing means;
Data analysis program characterized by functioning as The computer,
Function as specifying means for specifying the sequence number corresponding to the objective variable constituting each of the divided groups;
The output means includes
The data analysis program according to claim 1, wherein the data analysis program outputs information that characterizes the analysis target data, a sequence number specified by the specifying unit, and an objective variable corresponding to the sequence number. The computer,
Selecting means for selecting two types of analysis target data from a plurality of types of analysis target data having the same explanatory variable;
Correlation for calculating information indicating the correlation between the order number specified by the specifying means and the objective variable corresponding to the order number among the objective variables of the two types of analysis target data selected by the selection means Function as a relation information calculation means,
The output means includes
The data analysis program according to claim 2, wherein the calculation result calculated by the correlation information calculation means is output. The computer,
Based on the sum of squares of the objective variables and the sum of squares of the objective variables constituting the two divided groups, a set partitioning of the two divided groups is performed for each of a plurality of types of analysis target data having the same explanatory variable. A unity degree calculation means for calculating a unity degree expressing clarity,
Based on each unity degree calculated by the unity degree calculation unit, function as a determination unit that determines an output target from the plurality of types of analysis target data,
The output means includes
3. The analysis target data determined as the output target by the determining unit, and information characterizing the sequence number specified by the specifying unit and an objective variable corresponding to the sequence number are output. The data analysis program described. The computer,
Selecting means for selecting two types of analysis target data from the plurality of types of analysis target data;
Correlation for calculating information indicating the correlation between the order number specified by the specifying means and the objective variable corresponding to the order number among the objective variables of the two types of analysis target data selected by the selection means Function as a relation information calculation means,
The output means includes
The data analysis program according to claim 4, wherein the calculation result calculated by the correlation information calculation means is output. An acquisition means for acquiring analysis target data having a record of a sequence number serving as an explanatory variable and a target variable corresponding to the sequence number;
Sorting means for sorting the records of the data to be analyzed acquired by the acquisition means in ascending or descending order of the objective variable;
Assigning means for assigning a new sequence number to the records after sorting by the sorting means;
A division that divides the sorted records into two division groups with a specific number selected from the order numbers assigned by the assignment means based on regression tree analysis (hereinafter referred to as “assignment order numbers”) as boundaries. Means,
Output means for outputting two divided groups divided by the dividing means;
A data analysis apparatus comprising: Computer
An acquisition step of acquiring analysis target data having a sequence number as an explanatory variable and a target variable corresponding to the sequence number as records;
A sorting step of sorting the records of data to be analyzed acquired by the acquisition step in ascending or descending order of the objective variable;
An assigning step of assigning a new sequence number to the records after sorting by the sorting step;
A division that divides the sorted records into two division groups with a specific number selected from the order numbers assigned by the assignment step based on regression tree analysis (hereinafter referred to as “assignment order numbers”) as boundaries. Process,
An output step of outputting the two divided groups divided by the dividing step;
The data analysis method characterized by performing.

Images