JP7396213B2 - Data analysis system, data analysis method, and data analysis program - Google Patents

Description

The present invention relates to a data analysis system, a data analysis method, and a data analysis program.

A known data analysis method is to create a predictive model that represents the relationship between the objective variable and explanatory variables based on the dataset to be analyzed, and then perform analysis based on the created predictive model (e.g. (See Patent Document 1).

JP2020-24544A

In the data analysis method described above, for example, by using machine learning, it is possible to generate a model that predicts a target variable with high accuracy from explanatory variables. However, data analysis may be difficult because the resulting model is not easy to interpret. Further, for example, if the analysis target includes not only numerical data but also character data, it may not be possible to perform the analysis appropriately, and there is also a problem that the degree of freedom of the analysis target is low.

An object of the present invention is to provide a data analysis system, a data analysis method, and a data analysis program that can facilitate data analysis and improve the degree of freedom of an analysis target.

The data analysis system of the present invention includes at least one processor, and the at least one processor receives a data set including a plurality of data units that are a collection of data of a plurality of items, and the at least one processor receives a data set including a plurality of data units that are a collection of data of a plurality of items. Based on the dataset, the relationship between the objective variable and each of the multiple explanatory variables is calculated using the calculation method in hypothesis testing based on the data set. The significance probability of the explanatory variables is calculated, and the plurality of explanatory variables are displayed on the display unit in descending order of significance probability.

In this data analysis system, the significance probability between the target variable and each of multiple explanatory variables is calculated using the calculation method used in hypothesis testing, and the multiple explanatory variables are displayed on the display in order of decreasing significance probability. Ru. This allows the user to easily understand explanatory variables with a low significance probability, that is, explanatory variables that are expected to have a strong relationship with the target variable. Furthermore, this data analysis system allows the user to compare multiple explanatory variables based on significance probability. Since the significance probability is calculated in common even when calculation methods of different hypothesis testing methods are used, by using the significance probability as the standard, even if calculation methods of different hypothesis testing methods are used, multiple calculations using the same standard can be performed. It becomes possible to compare the explanatory variables of As a result, for example, even when the analysis target includes both numerical data and character data, it is possible to suitably compare a plurality of explanatory variables. Therefore, according to this data analysis system, data analysis can be facilitated, and the degree of freedom of analysis targets can be improved. Note that although this data analysis system calculates the significance probability using a calculation method in hypothesis testing, the hypothesis testing itself does not need to be performed. Hypothesis testing is a statistical procedure for deciding based on observed values whether to reject the null hypothesis and support an alternative hypothesis, or not to reject the null hypothesis.

The calculation method used by the at least one processor may include a calculation method in a non-parametric testing technique. In this case, the calculation method used in the nonparametric test method does not easily reduce accuracy even if the data to be analyzed contains abnormal values such as outliers, and does not require assumptions such as population distribution. The degree of freedom of targeting can be further improved.

The calculation method used by the at least one processor includes a calculation method in a first hypothesis testing method and a calculation method in a second hypothesis testing method different from the first hypothesis testing method, and the at least one processor When both the objective variable and the explanatory variable are composed of numerical data, the significance probability is calculated using the calculation method in the first hypothesis testing method, and when at least one of the objective variable and the explanatory variable is composed of character data, The significance probability may be calculated using the calculation method in the second hypothesis testing method. In this case, even if the analysis target includes both numerical data and character data, the significance probability can be suitably calculated.

The second hypothesis testing method includes a third hypothesis testing method and a fourth hypothesis testing method different from the third hypothesis testing method, and the at least one processor is configured such that one of the objective variable and the explanatory variable is composed of numerical data. and the other of the objective variable and explanatory variable is composed of text data, the significance probability is calculated using the calculation method in the third hypothesis testing method, and both the objective variable and explanatory variable are composed of text data. If configured, the significance probability may be calculated using the calculation method in the fourth hypothesis testing method. In this case, even if the analysis target includes both numerical data and character data, the significance probability can be calculated more suitably.

The at least one processor calculates a plurality of significance probabilities between the objective variable and the explanatory variable using calculation methods in a plurality of mutually different hypothesis testing methods, and explains the smallest significance probability among the plurality of significance probabilities as the objective variable. It may also be a significance probability between variables. In this case, the significance probability can be calculated with higher accuracy.

The plurality of explanatory variables include a first explanatory variable and a second explanatory variable, and the at least one processor calculates the significance probability between the target variable and the first explanatory variable using a calculation method in the first hypothesis testing method. However, the significance probability between the objective variable and the second explanatory variable may be calculated using a calculation method in a second hypothesis testing method that is different from the first hypothesis testing method. In this case, the significance probability can be calculated using the calculation methods in the first hypothesis testing method and the second hypothesis testing method, and the degree of freedom of the analysis target can be further improved.

The at least one processor may cause the display unit to display a graph showing a relationship between the objective variable and one explanatory variable selected from the plurality of explanatory variables displayed on the display unit. In this case, the user can easily understand the relationship between the selected explanatory variable and the target variable.

The data analysis method of the present invention is a data analysis method executed by a data analysis system equipped with at least one processor, and includes the steps of receiving a data set including a plurality of data units that are a collection of data of a plurality of items; For the objective variable consisting of one of the items, and the multiple explanatory variables consisting of two or more of the multiple items, the objective variable and The method includes the steps of calculating the significance probability between each of the plurality of explanatory variables, and displaying the plurality of explanatory variables on the display unit in order of decreasing significance probability. According to this data analysis method, for the reasons mentioned above, data analysis can be facilitated and the degree of freedom of the analysis target can be improved.

The data analysis program of the present invention includes the steps of accepting a dataset including a plurality of data units that are a collection of data of a plurality of items, an objective variable consisting of one item among the plurality of items, and a target variable consisting of one item among the plurality of items. A step of calculating the significance probability between the target variable and each of the plurality of explanatory variables based on the data set using a calculation method in hypothesis testing for the plurality of explanatory variables consisting of two or more other items; A computer is caused to display the explanatory variables on a display unit in order of decreasing significance probability. According to this data analysis program, for the reasons mentioned above, data analysis can be facilitated and the degree of freedom of the analysis target can be improved.

According to the present invention, it is possible to provide a data analysis system, a data analysis method, and a data analysis program that can facilitate data analysis and improve the degree of freedom of analysis targets.

FIG. 1 is a diagram illustrating an example of a functional configuration of a data analysis system according to an embodiment. 1 is a diagram illustrating an example of a hardware configuration of a computer that constitutes a data analysis system. 2 is a flowchart showing an example of the operation of the data analysis system. FIG. 3 is a diagram showing an example of a data set. FIG. 3 is a diagram illustrating an example of a method of converting character data into numerical data. It is a figure which shows the example of a display of a display part. It is a figure which shows the example of a graph. It is a figure which shows the example of a graph. It is a figure which shows the example of a graph. (a) to (c) are diagrams showing examples of graphs.

Hereinafter, one embodiment of the present invention will be described in detail with reference to the drawings. In the following description, the same reference numerals will be used for the same or equivalent elements, and overlapping description will be omitted.
[System configuration]

As shown in FIG. 1, the data analysis system 1 according to the embodiment includes a reception section 11, a calculation section 12, and a display control section 13 as functional elements. The reception unit 11 receives the data set 30. The calculation unit 12 calculates the significance probability between the objective variable and each of the plurality of explanatory variables based on the data set 30. The display control unit 13 causes the display unit 26 described below to display a plurality of explanatory variables in order of decreasing significance probability.

The data analysis system 1 includes, for example, a computer 20. As shown in FIG. 2, the computer 20 includes a processor 21, a main storage section 22, an auxiliary storage section 23, a communication control section 24, an input section 25, and a display section 26. The processor 21 is, for example, a CPU, and executes an operating system, application programs, and the like. The main storage unit 22 is composed of, for example, a ROM, a RAM, and the like. The auxiliary storage section 23 is composed of, for example, a hard disk, a flash memory, etc., and stores a larger amount of data than the main storage section 22. The communication control unit 24 includes, for example, a network card, a wireless communication module, and the like. The input unit 25 includes, for example, a keyboard, a mouse, a touch panel, and the like. The display unit 26 is composed of, for example, a monitor, a touch panel display, or the like.

Each functional element of the data analysis system 1 is realized by executing a data analysis program 27 stored in advance in the auxiliary storage unit 23. Specifically, by loading the data analysis program 27 into the processor 21 or the main storage unit 22 and having the processor 21 execute the data analysis program 27, each of the reception unit 11, calculation unit 12, and display control unit 13 is Function is realized. The processor 21 operates the communication control section 24, the input section 25, and the display section 26 according to the data analysis program 27, and reads and writes data in the main storage section 22 and the auxiliary storage section 23. Data or databases necessary for processing are stored in the main storage section 22 or the auxiliary storage section 23.

The data analysis program 27 may be provided after being permanently recorded on a tangible recording medium such as a CD-ROM, DVD-ROM, or semiconductor memory. That is, the data analysis program 27 may be provided after being recorded on a computer-readable recording medium. Alternatively, the data analysis program 27 may be provided via a communication network as a data signal superimposed on a carrier wave.

The data analysis system 1 may be composed of one computer 20 or a plurality of computers 20. When using a plurality of computers 20, one data analysis system 1 may be logically constructed by connecting these computers 20 to each other via a communication network such as the Internet or an intranet.
[System operation]

An example of a data analysis method executed by the data analysis system 1 will be described with reference to FIG. 3. First, the receiving unit 11 receives the data set 30 (step S1). The data set 30 is input to the reception unit 11 by the user via the input unit 25 and the display unit 26, for example. For example, when the user specifies a data set 30 stored in the auxiliary storage unit 23, the specified data set 30 is read and accepted by the reception unit 11.

The data set 30 is an analysis target and includes a plurality of data units 31 that are a collection of data of a plurality of items. The items included in the data unit 31 may be set arbitrarily. The items may be, for example, the characteristics, composition, etc. of materials, compounds, etc., or the characteristics, dimensions, materials, etc. of apparatuses, devices, etc. The data of each item is numerical data or character data. Character data is data other than numerical data and is data represented by characters or symbols. As will be described later, character data is used after being converted to numerical data.

The data set 30 may be, for example, a collection of data acquired during a manufacturing process at a factory. It is expected that the promotion of the Internet of Things (IOT) will make it possible to acquire large amounts of data in the manufacturing process. Items may include quality, characteristics, manufacturing conditions, etc. of manufactured products. Examples of product quality or characteristics include defect rate, breakdown voltage, short circuit rate, and the like. Examples of manufacturing conditions include unique numbers/symbols assigned to manufacturing equipment, average value or variance of material thickness, process execution time/number of times, and the like. Data set 30 may be time series data. In this case, one item may be composed of numerical data representing the time or order in which the data units 31 were acquired.

Data set 30 may include data units 31 that include missing values. Missing values mean missing data. The data set 30 may include a data unit 31 that includes an abnormal value (outlier). An abnormal value is a value that extremely deviates from other data in the item, and may occur due to measurement or recording errors. Processing of missing values and abnormal values will be described later. The number of data units 31 is not limited, but may be several hundred or more, for example. The number of items is not limited, but may be several thousand or more, for example.

FIG. 4 is a diagram showing an example of the data set 30. In this example, data set 30 is represented in tabular form. Each row corresponds to a data unit 31, and each column corresponds to an item. Each data unit 31 includes item A, item B, item C, item D, item E, item F, item G, and item H as items. For example, items A to D are composed of numerical data, and items E to H are composed of character data.

Following step S1, the receiving unit 11 receives analysis conditions (step S2). The analysis conditions are input to the reception unit 11 by the user via the input unit 25 and the display unit 26, for example. The analysis conditions include designation of objective variables and explanatory variables. The user selects one item from among the plurality of items in the data set 30 as a target variable, and selects a plurality of items from the remaining items as explanatory variables. For example, a selection box for selecting a target variable is displayed on the display unit 26, and the user selects the target variable by selecting an item in the selection box. Along with this, items other than the items selected as objective variables are selected as explanatory variables. Note that it may be possible to select an item to be set as an explanatory variable from among items other than the item selected as the objective variable.

Furthermore, the analysis conditions may include designation of numerical ranges for the objective variable and each explanatory variable. Furthermore, the analysis conditions may include a designation as to which data unit 31 is to be analyzed.

Following step S2, the calculation unit 12 calculates the significance probability (P value) between the target variable and each explanatory variable based on the data set 30 using the calculation method used in hypothesis testing (step S3). . Significance probability is the probability that a statistical test quantity is realized under a null hypothesis in statistical hypothesis testing. A small significance probability indicates that the null hypothesis is unlikely to hold true. The null hypothesis in this case varies depending on the hypothesis testing method, but is, for example, a hypothesis that there is no relationship between the objective variable and the explanatory variable. The calculation unit 12 calculates the significance probability between each of the plurality of explanatory variables and the objective variable. The calculation method used by the calculation unit 12 to calculate the significance probability includes calculation methods for a plurality of mutually different hypothesis testing methods. The calculation unit 12 determines which hypothesis testing method calculation method to use for each combination of objective variable and explanatory variable, as described below.

The calculation unit 12 calculates the significance probability using the calculation method in the first hypothesis testing method when both the objective variable and the explanatory variable are composed of numerical data. On the other hand, when at least one of the objective variable and the explanatory variable is composed of character data, the calculation unit 12 calculates the significance probability using the calculation method in the second hypothesis testing method.

More specifically, when one of the objective variable and the explanatory variable is composed of numerical data, and the other of the objective variable and the explanatory variable is composed of character data, the calculation unit 12 uses the third hypothesis testing method. Calculate the significance probability using the calculation method in . The calculation unit 12 calculates the significance probability using the calculation method in the fourth hypothesis testing method when both the objective variable and the explanatory variable are composed of character data. That is, the calculation method used by the calculation unit 12 includes the calculation method in the first hypothesis testing method and the second hypothesis testing method, and the second hypothesis testing method includes the calculation method in the third hypothesis testing method and the fourth hypothesis testing method. Contains. The first hypothesis testing method, the third hypothesis testing method, and the fourth hypothesis testing method are different hypothesis testing methods. Regardless of the calculation method used in any hypothesis testing method, the significance probability is calculated in common.

The first hypothesis testing method is applicable when both variables to be tested are numerical data. In the first hypothesis testing method, correlations between variables are tested. Examples of the first hypothesis testing method include Spearman's rank correlation coefficient test and Kendall's rank correlation coefficient test. Spearman's rank correlation coefficient test and Kendall's rank correlation coefficient test are nonparametric test methods. A nonparametric testing method is a method of performing a statistical test without assuming a specific distribution such as a normal distribution as the population distribution. With non-parametric testing methods, accuracy is unlikely to decrease even if the data to be analyzed contains abnormal values such as outliers.

The third hypothesis testing method is a method applicable when one of the variables to be tested is numerical data and the other is character data. In the third hypothesis testing method, a test is performed on the difference in representative values (numeric data) between levels (between character data). Examples of the third hypothesis testing method include the Kruskal-Wallis test and the Flinger-Killen test. The Kruskal-Wallis test and the Fligner-Killen test are nonparametric test methods.

The fourth hypothesis testing method is a method applicable when both variables to be tested are character data. In the fourth hypothesis testing method, the independence of contingency tables created from each variable is tested. Examples of the fourth hypothesis testing method include chi-square test of independence and Fisher's exact test. Chi-square test of independence and Fisher's exact test are non-parametric testing techniques.

A plurality of different hypothesis testing methods may be set as at least one of the first hypothesis testing method, the third hypothesis testing method, and the fourth hypothesis testing method. In this case, the calculation unit 12 calculates a plurality of significance probabilities between the objective variable and the explanatory variable using calculation methods in the plurality of set hypothesis testing methods. Then, the calculation unit 12 sets the smallest significance probability among the plurality of calculated significance probabilities as the significance probability between the objective variable and the explanatory variable. For example, when the Kruskal-Wallis test and the Fligner-Killen test are set as the third hypothesis testing method, the calculation unit 12 calculates that one of the objective variable and the explanatory variable is composed of numerical data, and the other is composed of character data, calculate the significance probability of the difference in representative values between levels between the objective variable and explanatory variable using the Kruskal-Wallis test and Fligner-Killen test. calculate. Then, the calculation unit 12 sets the smaller of the two calculated significance probabilities as the significance probability of the difference in representative value between levels between the target variable and the explanatory variable.

Further, the calculation unit 12 performs the following missing value processing when calculating the significance probability. When a missing value is included in the objective variable or design variable composed of numerical data, the calculation unit 12 excludes the data unit 31 containing the missing value from the analysis target, and calculates the significance probability using the remaining data unit 31. Calculate. When a missing value is included in the objective variable or design variable made up of character data, the calculation unit 12 replaces the missing value with a predetermined word (for example, "NA") and calculates the significance probability.

Further, the calculation unit 12 may calculate the significance probability after converting character data into numerical data. Any method may be used to convert character data into numerical data. For example, in the example of FIG. 5, the item "device" includes column data consisting of three types of characters, "A", "B", and "C", and the column data includes the numerical values "0" and " 1" is converted into three-column matrix data. Through such conversion, character data can be converted into numerical data. Further, the calculation unit 12 may calculate the significance probability without converting character data into numerical data. For example, in the calculation method for the Kruskal-Wallis test, significance probability is calculated without converting character data into numerical data.

Following step S3, the display control unit 13 causes the display unit 26 to display a plurality of explanatory variables in order of decreasing significance probability (step S4).

FIG. 6 is a diagram showing a display example of the display section 26. As shown in FIG. In this example, a table 40 showing calculation results is displayed on the display section 26. Table 40 shows an example in which the objective variable is item A and the explanatory variables are items B to H. Items B to H, which are explanatory variables, are arranged in descending order of the minimum significance probability from the top. That is, in this example, the minimum value of the significance probability of item H is the smallest, and the minimum value of the significance probability of item F is the largest. To the right of the item name, information indicating whether the data constituting the explanatory variable (item) is numerical data or character data (that is, the data type of the explanatory variable) is displayed in characters. To the right of the data type, the number of explanatory variables (items) is displayed as a numerical value.

On the right side of the number of data, the significance probability calculated using each of the calculation methods of hypothesis testing method A, hypothesis testing method B, hypothesis testing method C, and hypothesis testing method D is displayed as a numerical value. Hypothesis testing method A and hypothesis testing method B are the first hypothesis testing methods described above. That is, in this example, two different hypothesis testing methods are set as the first hypothesis testing method. Hypothesis testing method C is the third hypothesis testing method (second hypothesis testing method) described above. Hypothesis testing method D is the fourth hypothesis testing method (second hypothesis testing method) described above.

In this example, item A, which is the objective variable, and items B to D, which are explanatory variables, are composed of numerical data, and items E to H are composed of character data. Therefore, the significance probabilities between item A and items B to D are calculated using the calculation methods in each of hypothesis testing method A and hypothesis testing method B, which are the first hypothesis testing methods. The calculated significance probabilities are displayed numerically in columns of "Significance Probability (Method A)" and "Significance Probability (Method B)", respectively.

The significance probabilities between item A and items E to H are calculated using the calculation method in hypothesis testing method C, which is the third hypothesis testing method. The calculated significance probability is displayed numerically in the "Significance Probability (Method C)" column. In this example, since item A, which is the target variable, is numerical data, the calculation method in hypothesis testing method D is not used. Therefore, the column "Significance Probability (Method D)" is blank. When the significance probability is calculated using the calculation method in hypothesis testing method D, the calculated significance probability is displayed as a numerical value in the "Significance probability (method D)" column. Note that in the columns of "Significance Probability (Method A)," "Significance Probability (Method B)," and "Significance Probability (Method C)," the columns other than those for the corresponding items are also blank.

In the rightmost column, the minimum value of the significance probability is displayed numerically. In this example, for items B to D, the minimum significance probability is the significance probability calculated using the calculation method in hypothesis testing method A, and the significance probability calculated using the calculation method in hypothesis testing method B. This is the smaller of the two. For items E to H, the minimum significance probability is the significance probability calculated using the calculation method in hypothesis testing method C. In other words, if multiple significance probabilities are calculated between the objective variable and explanatory variables using calculation methods in multiple hypothesis testing methods, the smallest significance probability among the multiple significance probabilities will be used to determine the relationship between the objective variable and explanatory variable. It is considered to be the significance probability between.

After step S4, the user can select one explanatory variable for displaying a graph 50, which will be described later, from among the plurality of explanatory variables (items) displayed on the display unit 26. For example, a selection box is displayed on the display unit 26, and when the user presses the selection box, the selection box is expanded. In the expanded state, labels indicating a plurality of explanatory variables are displayed in text in the selection box, arranged in descending order of significance from the top. The user selects one explanatory variable by selecting a label indicating the explanatory variable in the selection box. Upon receiving this selection, the display control unit 13 displays a graph 50 showing the relationship between the selected explanatory variable and the objective variable on the display unit 26. Although the graph 50 and the selection box are displayed on a different screen (tab) from the table 40, for example, they may be displayed together with the table 40 on the same screen. In the selection box, multiple explanatory variables are displayed in descending order of significance probability, so the user can proceed with the analysis efficiently by, for example, selecting the explanatory variables in order from the top and checking the graph 50. can.

7 to 10 are diagrams showing examples of the graph 50. 7 to 10, a graph 50 is shown in which the explanatory variable is item X and the objective variable is item Y. In the example of FIG. 7, the relationship between item X and item Y is shown by a scatter diagram. The significance probability (P) and the number of data (n) are displayed at the upper left, and a smooth line 51 is also displayed. The significance probability, the number of data, and whether or not to display the smooth line 51 may be selectable using check boxes. In the example of FIG. 8, the relationship between item X and item Y is shown by a box plot . Item X includes character data of "H1", "H2", "H3", "H4", "H5", and so on.

In the example of FIG. 9, the relationship between item X and item Y is displayed as time series information using a line graph. The horizontal axis represents a numerical value indicating the time or order in which the data unit 31 was acquired, and the vertical axis represents the numerical value of item X and item Y. In this way, the relationship between item X and item Y may be displayed as time series information. If there is an item consisting of numerical data representing the time or order in which the data unit 31 was acquired, the horizontal axis may be the data of the item. Alternatively, the horizontal axis may be the row number of the data unit 31.

The relationship between item X and item Y may be displayed using a scatter diagram as shown in FIG. , may be displayed using a violin plot as shown in FIG. 10(c). In the examples shown in FIGS. 10(a) to 10(c), item X is composed of two types of character data: "a" and "b". Note that numerical data greater than 0 may be displayed after being logarithmically converted. If both item X and item Y are composed of character data, a mosaic plot may be used. A plurality of graphs 50 may be displayed on the display unit 26. In this case, the plurality of graphs 50 may be displayed in descending order of significance probabilities for the corresponding explanatory variables. The data analysis system 1 may be configured to be able to output analysis results including a display 40 and a graph 50 to a file in a predetermined format.
[Action and effect]

In the data analysis system 1, the significance probability between the objective variable and each of the plurality of explanatory variables is calculated using the calculation method in hypothesis testing, and the plurality of explanatory variables are displayed on the display unit 26 in order of decreasing significance probability. be done. This allows the user to easily understand explanatory variables with a low significance probability, that is, explanatory variables that are expected to have a strong relationship with the objective variable. Furthermore, in the data analysis system 1, the user can compare multiple explanatory variables based on significance probability. The significance probability is calculated in common even when calculation methods for different hypothesis testing methods are used. It becomes possible to compare the explanatory variables of As a result, for example, even when the analysis target includes both numerical data and character data, it is possible to suitably compare a plurality of explanatory variables. Therefore, according to the data analysis system 1, data analysis can be facilitated, and the degree of freedom of the analysis target can be improved.

As mentioned above, a large amount of data can be acquired every day in the manufacturing process. However, since the amount of data is enormous, it is not easy to find data related to product quality. Furthermore, by using machine learning, it is sometimes possible to predict a target variable from explanatory variables with high accuracy, but it is not easy to interpret the resulting model. When analyzing manufacturing process data, there is no point in predicting the defective rate with high accuracy, but rather to reduce the defective rate. In this regard, when an abnormality occurs in the quality of a product during the manufacturing process, it is often the result of a single cause rather than a combination of multiple causes. For example, an event may occur where the number of defects increases when manufactured with specific equipment or when specific raw materials are used. Furthermore, commercially available software can also calculate the correlation coefficient, but there are problems in that the calculation cannot be performed if there are missing values, and the accuracy decreases significantly if there are abnormal values. Further, there are also problems in that it is impossible to calculate accurately unless the relationship between numerical values is linear, and it is impossible to calculate a correlation coefficient between numerical values and characters or between characters. On the other hand, as described above, in the data analysis system 1, the user can easily grasp explanatory variables that have a small significance probability, that is, explanatory variables that are expected to have a strong relationship with the target variable. As a result, even if, for example, an abnormality occurs in the quality of the product during the manufacturing process, the cause can be easily identified. Further, the data analysis system 1 can be applied even when the analysis target includes both numerical data and character data, and can also be applied when there are missing values or abnormal values. Therefore, according to the data analysis system 1, data analysis can be facilitated and the degree of freedom of the analysis target can be improved.

The calculation methods used by the processor 21 include only calculation methods in non-parametric testing methods. As a result, the calculation method used in nonparametric test methods does not easily reduce accuracy even if the data to be analyzed contains abnormal values such as outliers, and does not require assumptions such as population distribution. The degree of freedom of targeting can be further improved.

The calculation method used by the processor 21 includes a calculation method in a first hypothesis testing method and a calculation method in a second hypothesis testing method different from the first hypothesis testing method. Then, when both the objective variable and the explanatory variable are composed of numerical data, the processor 21 calculates the significance probability using the calculation method in the first hypothesis testing method, and at least one of the objective variable and the explanatory variable is composed of characters. If it is composed of data, the significance probability is calculated using the calculation method in the second hypothesis testing method. Thereby, even if the analysis target includes both numerical data and character data, the significance probability can be suitably calculated.

The second hypothesis testing method includes a third hypothesis testing method and a fourth hypothesis testing method different from the third hypothesis testing method. When one of the objective variable and the explanatory variable is composed of numerical data, and the other of the objective variable and the explanatory variable is composed of character data, the processor 21 uses the calculation method in the third hypothesis testing method. If both the objective variable and the explanatory variable are composed of character data, the significance probability is calculated using the calculation method in the fourth hypothesis testing method. Thereby, even when the analysis target includes both numerical data and character data, the significance probability can be calculated more suitably.

The processor 21 calculates a plurality of significance probabilities between the objective variable and the explanatory variable using calculation methods in a plurality of mutually different hypothesis testing methods, and selects the smallest significance probability among the plurality of significance probabilities as the objective variable and the explanatory variable. Let the significance probability be between . Thereby, the significance probability can be calculated with higher accuracy.

The processor 21 causes the display unit 26 to display a graph 50 showing the relationship between the target variable and one explanatory variable selected from the plurality of explanatory variables displayed on the display unit 26. Thereby, the user can easily understand the relationship between the selected explanatory variable and the objective variable.

In the above embodiment, the calculation method used to calculate the significance probability for one explanatory variable among the plurality of explanatory variables is different from the calculation method used to calculate the significance probability for the other explanatory variables. That is, the plurality of explanatory variables include a first explanatory variable and a second explanatory variable, and the processor 21 calculates the significance probability between the target variable and the first explanatory variable using the calculation method in the first hypothesis testing method. Then, the significance probability between the objective variable and the second explanatory variable is calculated using a calculation method in the second hypothesis testing method that is different from the first hypothesis testing method. Thereby, the significance probability can be calculated using the calculation method in the first hypothesis testing method and the second hypothesis testing method, and the degree of freedom of the analysis target can be further improved.

The present invention is not limited to the above embodiments. For example, in the above embodiment, the calculation method used by the processor 21 is only a calculation method for a non-parametric test method, but it may further include a calculation method for a parametric test method, or a calculation method for a parametric test method may be included. It may be only the calculation method in the method. An example of a parametric first hypothesis testing method that is applicable when both variables to be tested are numerical data is Pearson's correlation coefficient test. An example of a parametric third hypothesis testing method that can be applied when one of the variables to be tested is numerical data and the other is character data is analysis of variance.

DESCRIPTION OF SYMBOLS 1... Data analysis system, 20... Computer, 21... Processor, 26... Display part, 27... Data analysis program, 30... Data set, 31... Data unit, 50... Graph.

Claims (9)

comprising at least one processor;
The at least one processor includes:
Accepts a dataset containing multiple data units that are a collection of data for multiple items,
For an objective variable consisting of one item among the plurality of items and a plurality of explanatory variables consisting of two or more other items among the plurality of items, the objective variable and the explanation are determined based on the data set. Calculating the significance probability between the target variable and each of the plurality of explanatory variables using a calculation method in a hypothesis test in which a null hypothesis is a hypothesis that there is no relationship between the variables,
A data analysis system that displays the plurality of explanatory variables on a display unit in descending order of the significance probability. The data analysis system according to claim 1, wherein the calculation method used by the at least one processor includes a calculation method in a non-parametric testing method. The calculation method used by the at least one processor includes a calculation method in a first hypothesis testing method and a calculation method in a second hypothesis testing method different from the first hypothesis testing method,
The at least one processor includes:
When both the objective variable and the explanatory variable are composed of numerical data, calculating the significance probability using the calculation method in the first hypothesis testing method,
The data analysis system according to claim 1 or 2, wherein when at least one of the objective variable and the explanatory variable is composed of character data, the significance probability is calculated using a calculation method in the second hypothesis testing method. . The second hypothesis testing method includes a third hypothesis testing method and a fourth hypothesis testing method different from the third hypothesis testing method,
The at least one processor includes:
When one of the objective variable and the explanatory variable is composed of numerical data, and the other of the objective variable and the explanatory variable is composed of character data, using the calculation method in the third hypothesis testing method. Calculate the significance probability,
4. The data analysis system according to claim 3, wherein when both the objective variable and the explanatory variable are composed of character data, the significance probability is calculated using a calculation method in the fourth hypothesis testing method. The at least one processor includes:
calculating a plurality of significance probabilities between the objective variable and the explanatory variable using calculation methods in a plurality of mutually different hypothesis testing methods;
5. The data analysis system according to claim 1, wherein the smallest significance probability among the plurality of significance probabilities is set as the significance probability between the target variable and the explanatory variable. The plurality of explanatory variables include a first explanatory variable and a second explanatory variable,
The at least one processor includes:
Calculating the significance probability between the objective variable and the first explanatory variable using a calculation method in a first hypothesis testing method,
The data analysis according to claim 1 or 2, wherein the significance probability between the objective variable and the second explanatory variable is calculated using a calculation method in a second hypothesis testing method different from the first hypothesis testing method. system. The at least one processor includes:
Any one of claims 1 to 6, wherein a graph showing a relationship between one explanatory variable selected from the plurality of explanatory variables displayed on the display unit and the target variable is displayed on the display unit. The data analysis system according to item (1). A data analysis method performed by a data analysis system comprising at least one processor, the method comprising:
accepting a dataset including multiple data units that are a collection of data of multiple items;
For an objective variable consisting of one item among the plurality of items and a plurality of explanatory variables consisting of two or more other items among the plurality of items, the objective variable and the explanation are determined based on the data set. calculating the significance probability between the target variable and each of the plurality of explanatory variables using a calculation method in a hypothesis test in which a null hypothesis is a hypothesis that there is no relationship between the variables;
A data analysis method, comprising the step of displaying the plurality of explanatory variables on a display unit in descending order of the significance probability. accepting a dataset including multiple data units that are a collection of data of multiple items;
For an objective variable consisting of one item among the plurality of items and a plurality of explanatory variables consisting of two or more other items among the plurality of items, the objective variable and the explanation are determined based on the data set. calculating the significance probability between the target variable and each of the plurality of explanatory variables using a calculation method in a hypothesis test in which a null hypothesis is a hypothesis that there is no relationship between the variables;
A data analysis program that causes a computer to execute the step of displaying the plurality of explanatory variables on a display unit in descending order of the significance probabilities.

Images