JP2018147280A - Data analysis device and data analysis method - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a technique that allows a user to easily determine what kind of measures to take for a prediction result is effective.SOLUTION: A data analysis device 1 includes: a data set generation unit 124 for generating data of an explanatory variable input to a machine learning model and inputting the generated data to the machine learning model to obtain data of a target variable; and a model evaluation unit 125 for calculating, based on the explanatory variable data generated by the data set generation unit 124 and the target variable data, a relationship between the explanatory variable and the target variable.SELECTED DRAWING: Figure 1

Description

  The present disclosure relates to a data analysis apparatus and a data analysis method for analyzing the relationship between input data and output data in machine learning.

  Conventionally, machine learning techniques such as neural networks have attracted attention. Various problems are solved by using a machine learning model obtained by machine learning.

  Patent Document 1 proposes a technique that can be used for applications such as financial credit problems, finding unauthorized customers of credit cards, and finding unauthorized access in a network through machine learning techniques. The method described in Patent Document 1 is a mechanism that supports a user's subsequent judgment on a prediction result by adding a reliability measure indicating the reliability of the certainty factor to the certainty factor of a prediction result based on similar cases. have.

JP 2003-323601 A

  However, when the method described in Patent Document 1 is used, the user cannot know the degree to which individual explanatory variables contribute to the prediction result. In other words, the user cannot know what factor led to the prediction result from the input data. In other words, the user uses the machine learning model while the relevance between the explanatory variable and the objective variable that is the prediction result is unknown in the neural network. For this reason, it is difficult for the user to know what kind of judgment should be made based on the prediction result.

  The present disclosure has been made in view of such a situation, and provides a technique by which a user can easily know what judgment should be made based on a prediction result.

  In order to solve the above problem, one of the representative data analysis devices of the present disclosure generates data of explanatory variables to be input to a machine learning model, and inputs the generated data of the explanatory variables to the machine learning model. Based on the data of the explanatory variable generated by the data set generating unit and the data of the objective variable, the relationship between the explanatory variable and the objective variable is obtained. A model evaluation unit for calculating.

  Also, one of the representative data analysis methods of the present disclosure is to generate data of explanatory variables to be input to a machine learning model, and input the generated data of the explanatory variables to the machine learning model to obtain data of objective variables. And calculating the relationship between the explanatory variable and the objective variable based on the generated data of the explanatory variable and the data of the objective variable.

According to the present disclosure, it is possible to calculate the degree of influence on an objective variable for each explanatory variable with respect to a learned machine learning model. As a result, it is easy to infer what factor the machine learning model has obtained the output result, and it is easy for the user to make a determination when performing subsequent measures.
Problems, configurations, and effects other than those described above will become apparent from the following embodiments for implementing the present invention and the accompanying drawings.

It is a functional block diagram which shows schematic structure of the data analyzer which concerns on embodiment. It is a figure which shows an example of model data. It is a figure which shows an example of input-output data. It is a figure which shows an example of simulation data. It is a figure which shows an example of the display screen of evaluation data. It is a flowchart for demonstrating a simulation process. It is a flowchart for demonstrating an evaluation process.

  Hereinafter, embodiments of the present invention will be described with reference to the accompanying drawings. However, it should be noted that this embodiment is merely an example for realizing the present invention, and does not limit the technical scope of the present invention. In each drawing, the same reference numerals are assigned to common components. In the present specification, the learned model is a model obtained by machine learning, and is also referred to as a machine learning model.

<Configuration of data analyzer>
FIG. 1 is a functional block diagram illustrating a schematic configuration of a data analysis apparatus 1 according to the embodiment. The data analysis apparatus 1 includes a central processing unit (processor) 100 that performs necessary arithmetic processing and control processing, an input / output device 110 for inputting and outputting data, and a program necessary for processing by the central processing unit 100. And a storage device 130 for storing data to be processed by the central processing unit 100 or data processed by the central processing unit 100.

  The input / output device 110 includes an output device configured by a display unit 111 for displaying data, a printer (not shown), and the like, and a keyboard for performing operations such as selecting a menu for the displayed data. 112, a pointing device 113 such as a mouse.

  The program memory 120 inputs a variety of input data to a learned model generated by machine learning, obtains an output result, an evaluation program 122 that analyzes the result of the simulation process, and an analysis of the simulation process An output program 123 for displaying the result on the display unit 111 is stored. Each processing program is stored in the program memory 120 as a program code, and each processing is realized by the central processing unit 100 executing each program code.

  The central processing unit 100 functions as the data set generation unit 124 by executing the simulation program 121, functions as the model evaluation unit 125 by executing the evaluation program 122, and executes the evaluation output unit by executing the output program 123. It functions as 126.

  The data set generation unit 124 generates explanatory variable data to be input to the machine learning model, and inputs the data generated to the machine learning model to obtain target variable data. The model evaluation unit 125 calculates the relationship between the explanatory variable and the objective variable based on the explanatory variable data and the objective variable data generated by the data set generation unit 124. The relationship between the explanatory variable and the objective variable indicates, for example, a statistical correlation between the explanatory variable and the objective variable. The evaluation output unit 126 displays the relationship between the explanatory variable and the objective variable calculated by the model evaluation unit 125 on the display unit 111 together with an interface for inputting the explanatory variable of the machine learning model.

  The storage device 130 includes model data 131 which is data of a learned model generated in advance by machine learning, input / output data 132 which is an input / output format of teacher data for machine learning used when generating the model data 131, and input data 132. Simulation data 133 generated for simulation based on the output data 132 and evaluation data 134 obtained after analyzing the simulation data 133 are stored. The storage device 130 may be a storage system that is remotely arranged via a network.

  The processing programs, data, programs, and the like described above can be provided by being stored in various recording media such as a CD-ROM, a DVD-ROM, and a USB memory.

<Model data>
FIG. 2 is a diagram illustrating an example of the model data 131 in the storage device 130. The model data includes a learned model setting data obtained by machine learning and a data type 201 composed of learned models. The setting data and the learned model are composed of a data item 202 and its value 203, respectively.

  The data items of the setting data include, for example, the number of input dimensions and the number of output dimensions. The number of input dimensions is the number of explanatory variables when inputting to a machine learning model. The number of output dimensions is the number of objective variables output by the machine learning model. For example, in the case of a machine learning model in which sales on a certain day, probability of precipitation on the next day and day of the week are explanatory variables and sales on the next day are objective variables, the number of input dimensions is 3 and the number of output dimensions is 1.

  The data item of the learned model includes parameters that characterize the learned model for obtaining output data by inputting the input data as described above. Note that the parameters of the learned model do not need to be grasped by the user, and are automatically used in the program when the machine learning model is called.

<Input / output data>
FIG. 3 is a diagram illustrating an example of the input / output data 132 in the storage device 130. Input / output data is input to and output from the model, that is, a data item 301 that stores explanatory variables and objective variables, a data type 302 that represents the data type of each variable stored in the data item 301, and a data item 301. The value range 303 represents the range of the data value of each variable.

  The data type 302 stores a value such as a floating point type or a binary type. The floating point type is a data type having a real value as a value, and stores a value such as 0.1. The binary type is a data type in which only two types of values are stored, such as sex and purchase / non-purchase, and a value such as 0 or 1 is stored to distinguish the two types. The value range 303 stores a range of possible values. For example, if the data type is a floating point type and 0-1, real values in the range of 0 to 1 are stored.

<Simulation data>
FIG. 4 is a diagram illustrating an example of the simulation data 133 in the storage device 130. The simulation data includes data that stores a combination of explanatory variable groups 401, 402, and 403 and a target variable 404 that is output when the explanatory variable is input to the model. The number of explanatory variables and objective variables is determined by the number of input dimensions and the number of output dimensions in the model data 131. The numerical value of the explanatory variable group is determined so as not to overlap among the values included in the value range of each explanatory variable in the input / output data 132.

  In the example shown in FIG. 4, a data set when a simulation is performed based on the input / output data shown in FIG. 3 is shown. In the simulation example shown in FIG. 4, for example, the explanatory variables 1 and 3 are randomly generated from the range of 0−1, and the explanatory variable 2 is randomly generated as either 0 or 1. In the objective variable 1, a value obtained by inputting the generated explanatory variables 1 to 3 into the machine learning model is recorded.

<Evaluation data>
FIG. 5 is a diagram illustrating an example of a display screen of the evaluation data 134 in the storage device 130. The evaluation data includes an evaluation value for each explanatory variable in the evaluation target model and an evaluation value for the entire model. Specifically, the evaluation data 134 stores an explanatory variable 501 that is an ID of each explanatory variable and one or more evaluation items for each explanatory variable. In FIG. 5, standard partial regression coefficient 502 is stored as evaluation item 1, and significance probability p-value is stored as evaluation item 2. Also, model evaluation items 504 and model evaluation values 505 that are the values are stored as evaluation values for the entire model.

  In the evaluation data display screen shown in FIG. 5, for example, when the user selects a machine learning model that the user wants to use, the evaluation output unit 126 reads the evaluation data 134 related to the machine learning model, and the value of the explanatory variable is read into the machine learning model. Is displayed on the display unit 111 by being output together with the interface for inputting.

<Outline of processing in data analyzer>
Processing performed in the data analysis apparatus 1 having the above-described configuration will be described. First, the central processing unit 100 executes the simulation program 121 and functions as the data set generation unit 124. The data set generation unit 124 reads the model data 131 and the input / output data 132 stored in the storage device 130 and performs a simulation. The central processing unit 100 stores the simulation data 133 obtained by the simulation in the storage device 130.

  Next, the central processing unit 100 executes the evaluation program 122 and functions as the model evaluation unit 125. The model evaluation unit 125 reads the simulation data 133 from the storage device 130, calculates an evaluation value for each explanatory variable of the machine learning model and an evaluation value for the entire machine learning model, and uses the calculated evaluation value as evaluation data 134. To store.

  The central processing unit 100 executes the output program 123 and functions as the evaluation output unit 126. Then, the evaluation output unit 126 displays the contents of the evaluation data 134 on the display unit 111. Details of each of the above processes will be described below.

<Simulation process>
FIG. 6 is a flowchart for explaining the simulation processing executed by the data set generation unit 124. The data set generation unit 124 generates a set of explanatory variable values as shown in FIG. 4 in the simulation process. The data set generation unit 124 performs processing for obtaining a combination pattern of the value of the explanatory variable and the value of the objective variable obtained when the generated value of the explanatory variable is input to the machine learning model. For example, millions to hundreds of millions of patterns of combinations of explanatory variables and objective variables are generated.

  In step 601, the data set generation unit 124 reads model data 131 and input / output data 132.

  In step 602, the data set generation unit 124 first reads the number of input dimensions and the number of output dimensions from the model data 131, and reads the data type and value range of each explanatory variable from the input / output data 132. Next, based on these data, a pattern of combinations of explanatory variables is comprehensively generated in the value range of each explanatory variable.

  For example, in the example of the input / output data in FIG. 3, the combination of the values of the explanatory variable 1, the explanatory variable 2, and the explanatory variable 3 is expressed as (value of explanatory variable 1, value of explanatory variable 2, value of explanatory variable 3). As shown in FIG. 4, combinations such as (0.1, 0, 0.2), (0.1, 0, 0.3), (0.1, 1, 0.2), etc. A pattern is output. At this time, the value of the objective variable 1 is not set. The degree of completeness of the combination pattern, that is, the data granularity can be arbitrarily set. In general, the higher the comprehensiveness, the better the model evaluation accuracy, but the processing time increases. Conversely, the lower the completeness, the worse the evaluation accuracy of the model, but the shorter the processing time.

  In step 603, the data set generation unit 124 reads the learned model from the model data, inputs the simulation data generated in step 602 to the learned model, and obtains an output result, that is, an objective variable.

  In step 604, the data set generation unit 124 stores the value of the objective variable obtained in step 603 in the corresponding explanatory variable group record, and updates the simulation data.

<Evaluation process>
FIG. 7 is a flowchart for explaining an evaluation process executed by the model evaluation unit 125 and the evaluation output unit 126. In the evaluation process, for example, the evaluation result shown in FIG. 5 is output based on the simulation data as shown in FIG.

  In step 701, the model evaluation unit 125 reads the simulation data 133.

  In step 702, the model evaluation unit 125 applies an evaluation model to the simulation data 133. The evaluation model is used to obtain the degree of influence on the objective variable for each explanatory variable. For example, multiple regression analysis can be used for the evaluation model. Any evaluation model can be applied if this purpose can be realized.

  Hereinafter, the case where multiple regression analysis is used for the evaluation model will be described. When multiple regression analysis is applied to a record of simulation data, a standard partial regression coefficient and a significance probability p value can be calculated for each explanatory variable. The standard partial regression coefficient represents the strength of the influence of the explanatory variable on the objective variable, and the scale of each explanatory variable is unified. That is, the magnitude of the influence of the explanatory variable can be grasped by comparing the standard partial regression coefficients for each explanatory variable. In addition, the significance probability p-value represents the probability of the standard partial regression coefficient calculated by multiple regression analysis. In general, when the significance probability p-value is less than 5%, it can be determined that the explanatory variable is “related” to the objective variable.

  Further, when multiple regression analysis is applied to simulation data records, determination coefficients and determination coefficients with adjusted degrees of freedom can be calculated as evaluation items for the entire model. The coefficient of determination represents the ratio that can be explained by all explanatory variables out of the total variation of the objective variable, and is a value indicating the goodness of fit between the regression equation and the sample data. The degree-of-freedom-adjusted determination coefficient is a determination coefficient that takes into account the number of explanatory variables, and compensates for the disadvantage that the normal determination coefficient increases as the number of explanatory variables increases. Each output value obtained in this way is stored in the storage device 130 as evaluation data 134.

  In step 703, the evaluation output unit 126 displays the evaluation data 134 on the display unit 111. By displaying the screen in this way, it is possible to quantify and grasp the degree of influence on the objective variable in the learned model by machine learning.

<Summary>
As described above, according to the present embodiment, the values of the explanatory variables are input to the machine learning model in a comprehensive or sufficiently large number, the corresponding objective variable is obtained, and the evaluation model is applied to the result. By doing so, it becomes possible to quantify the degree of influence on the objective variable for each explanatory variable. Thereby, it becomes easy to grasp which explanatory variable is strongly influenced by the objective variable output for a certain explanatory variable group. In addition, compared to the case where the evaluation model is applied to the learned model using only the teacher data and correct answer data used when learning the model, it has already been learned by performing a comprehensive simulation as in this embodiment. The model can be evaluated accurately.

  In addition, the data analysis apparatus 1 according to the embodiment displays, for example, the relationship between the explanatory variable and the objective variable on the display unit 111 together with the interface for inputting the explanatory variable of the machine learning model. Therefore, when the user uses the machine learning model, the value of the explanatory variable can be input while checking the evaluation of the machine learning model displayed on the display unit 111. Therefore, the user can more quantitatively verify the reliability of the output of the machine learning model.

  Note that the present invention is not limited to the embodiments as they are, and can be embodied by modifying the constituent elements without departing from the scope of the invention in the implementation stage. Various inventions can be formed by appropriately combining a plurality of constituent elements disclosed in the embodiments. For example, some components may be deleted from all the components shown in the embodiment. Furthermore, constituent elements over different embodiments may be appropriately combined.

  In addition, each configuration, function, processing unit, processing unit, and the like described in the embodiments may be realized in hardware by designing a part or all of them with, for example, an integrated circuit. Further, each of the above-described configurations, functions, etc. may be realized by software by the processor interpreting and executing a program that realizes each function. Information such as programs, tables, and files for realizing each function is stored in a recording or storage device such as a memory, a hard disk, or an SSD (Solid State Drive), or a recording or storage medium such as an IC card, SD card, or DVD. be able to. In addition, a part or all of the program executed by the data analysis apparatus 1 according to the present embodiment may be realized by dedicated hardware or may be modularized. Various programs may be installed in each computer by a program distribution server or a storage medium.

  Furthermore, in the above-described embodiment, control lines and information lines are those that are considered necessary for explanation, and not all control lines and information lines on the product are necessarily shown. All the components may be connected to each other.

100: Central processing unit (processor)
DESCRIPTION OF SYMBOLS 110 ... Input / output device 111 ... Display unit 112 ... Keyboard 113 ... Mouse 120 ... Program memory 121 ... Simulation program 122 ... Evaluation program 123 ... Output program 124 ... Data set generation unit 125 ... Model evaluation unit 126 ... Evaluation output unit 130 ... Memory Device 131 ... Model data 132 ... Input / output data 133 ... Simulation data 134 ... Evaluation data

Claims (8)

A data set generation unit that generates data of explanatory variables to be input to the machine learning model, inputs the generated data of the explanatory variables to the machine learning model, and obtains data of the objective variable;
A model evaluation unit that calculates the relationship between the explanatory variable and the objective variable based on the explanatory variable data and the objective variable data generated by the data set generation unit;
A data analysis apparatus comprising: An evaluation output unit for displaying the relationship between the explanatory variable calculated by the model evaluation unit and the objective variable together with an interface for inputting the explanatory variable of the machine learning model on a display unit;
The data analysis apparatus according to claim 1. The model evaluation unit calculates at least one of a significance probability or a standard partial regression coefficient for each explanatory variable by performing multiple regression analysis on a data set of the data of the explanatory variable and the data of the objective variable.
The data analysis apparatus according to claim 1. The model evaluation unit calculates a determination coefficient indicating a degree by which the objective variable can be explained by the explanatory variable;
The data analysis apparatus according to claim 1. Generating explanatory variable data to be input to the machine learning model, inputting the generated explanatory variable data to the machine learning model to obtain target variable data; and
Calculating a relationship between the explanatory variable and the objective variable based on the generated explanatory variable data and the objective variable data;
Data analysis method including. A step of displaying the relationship between the calculated explanatory variable and the objective variable on a display unit together with an interface for inputting the explanatory variable of the machine learning model;
The data analysis method according to claim 5. The step of calculating the relationship between the explanatory variable and the objective variable includes performing a multiple regression analysis on a data set of the explanatory variable data and the objective variable data to obtain a significant probability or Calculating at least one of the standard partial regression coefficients,
The data analysis method according to claim 5. The step of calculating the relationship between the explanatory variable and the objective variable is a step of calculating a determination coefficient indicating a degree that the objective variable can be explained by the explanatory variable.
The data analysis method according to claim 5.

Images