JP2021182329A - Learning model selection method - Google Patents

Abstract

To select a highly accurate learning model for solving problems for new data out of existing models.SOLUTION: A model selection device separates input data into test data and learning data. Subsequently, the learning data obtained from the input data is input to create a model by transferring learning from one or more existing models. Subsequently, the test data obtained from the input data is input to the model by the transferring learning to calculate an objective variable of the data obtained by inference and a mean square error with the true value of the item corresponding to the objective variable of the input data. Then, a transition model with a small mean square error is selected as the optimum model. When selecting an existing model, a statistic of a data item (variance, mean) is compared with the same data item statistic of the input data, and the smaller one is selected.SELECTED DRAWING: Figure 3

Description

The present invention relates to a learning model selection method, and more particularly to a learning model selection method suitable for selecting an optimum model from a small amount of data in transfer learning.

With the development of information processing technology and the base technology that supports it, expectations for machine learning are increasing in various fields. Machine learning is a method of constructing a learning model (also referred to simply as a "model" in the present specification) with a large amount of data or a certain rule, and deriving an optimum problem-solving solution according to the learning model.

In general, the accuracy of machine learning can be improved as the amount of data is sufficiently large. On the contrary, when the data is small, it is difficult to create a model with sufficient accuracy. Therefore, even if an attempt is made to build a forecasting system with the same purpose, it is possible that the accuracy is sufficient for a project in which a large amount of data can be obtained, but the accuracy is insufficient in a project with a small amount of data.

In the field of machine learning, as a method to achieve high accuracy with a small amount of data, one of the existing models can be reused entirely from a model created with other data (existing model), or transfer learning using an existing model and new data. A method of reusing a part is known. Transfer learning using an existing model and new data is, for example, a method of inputting new data into an existing model, combining the inference result with the new data, and creating a new model using the combined data.

In this transfer learning method, it is important how to select a model that fits the new data. For example, Patent Document 1 discloses a technique for selecting an optimum model for such machine learning. In the prediction model selection system described in Patent Document 1, a plurality of prediction models are created by changing initial values from a single data. Then, the elements constituting the prediction model are quantified, and the prediction model whose numerical value does not meet the selection criteria is excluded. As a result, even if the amount of training data is not sufficient, an appropriate prediction model can be selected.

International Publication No. 2017/168458

In general, when selecting the optimum model and constructing a new learning system, the characteristics of the existing model used for reuse or transfer learning determine the accuracy of the new model. For example, when building a learning system that predicts house prices for each region, it is possible to build a highly accurate model in urban areas with a large number of samples, but sufficient accuracy cannot be obtained in rural areas with a small number of samples. In this case, it is conceivable to use the urban model for transfer learning, but it is necessary to decide which city model should be selected from a large number of models constructed in multiple cities.

According to Patent Document 1, a technique is disclosed in which a plurality of predictive models are created by changing initial values from a single data, the elements constituting the predictive model are quantified, and the predictive models whose numerical values do not meet the selection criteria are excluded. ing. However, it is not assumed that a model similar to the new data is selected from the models generated from different data.

An object of the present invention is to provide a learning model selection method for selecting a learning model with high accuracy in solving a problem for new data from existing models.

The learning model selection method of the present invention preferably selects a model suitable for a transfer learning method for creating a highly accurate model from new input data from a plurality of existing models stored in the information processing apparatus. One or more training model selection methods, in which the information processing device separates the input data into test data and training data, and the information processing device inputs the training data obtained from the input data into the existing model. The step of creating a model by transfer learning from each of the existing models of the above, the objective variable of the data obtained by inputting the test data obtained from the input data to the model by transfer learning, and the input data The data processing apparatus has a step of calculating an error from the true value of the item corresponding to the objective variable of the above, and a step of selecting a predetermined number of transfer models having a small error as the optimum model.

INDUSTRIAL APPLICABILITY According to the present invention, it is possible to provide a learning model selection method for selecting a learning model with high accuracy in solving a problem for new data from existing models.

It is a figure which shows the functional structure and data flow of a model selection apparatus. It is a figure which shows the hardware software configuration of a model selection apparatus. It is a figure explaining the concept of the model selection method of Embodiment 1 by using a specific example. It is a figure which shows an example of the house data table of a house analysis system. It is a figure which shows an example of a statistic item table. It is a figure which shows an example of a model statistic information table. It is a figure which shows an example of a statistic comparison table. It is a figure which shows an example of a transfer model result table. FIG. 5 is a flowchart showing a series of processes from the acquisition of input data to the selection of a model by the model selection device of the first embodiment. It is a flowchart which shows the data analysis process of Embodiment 1. FIG. It is a flowchart which shows the candidate model search process of Embodiment 1. FIG. It is a figure which shows an example of the important item input screen. It is a flowchart which shows the candidate model selection process of Embodiment 1. FIG. It is a flowchart which shows the transfer learning process. It is a flowchart which shows the transition model evaluation process. It is a flowchart which shows the optimum model evaluation process. It is a figure explaining the concept of the model selection method of Embodiment 2 by a concrete example. It is a figure which shows an example of a model result table. FIG. 5 is a flowchart showing a series of processes from the acquisition of input data to the selection of a model by the model selection device of the second embodiment. It is a flowchart which shows the data analysis process of Embodiment 2. It is a flowchart which shows the candidate model search process of Embodiment 2. It is a figure which shows an example of the model FI management table. It is a figure which shows an example of a new model FI result table. It is a figure which shows an example of a model FI comparison table. 6 is a flowchart showing a series of processes from the acquisition of input data to the selection of a model by the model selection device of the third embodiment. It is a flowchart which shows the candidate model search process of Embodiment 3. It is a flowchart which shows the candidate model selection process of Embodiment 3. It is a figure which shows an example of the transfer candidate selection screen.

Hereinafter, each embodiment of the present invention will be described with reference to FIGS. 1 to 28.

[Embodiment 1]
Hereinafter, the first embodiment according to the present invention will be described with reference to FIGS. 1 to 16.
In the learning model selection method of the first embodiment, the optimum model for adapting to the new model is selected from the existing models according to the difference from the value of the input data.

First, the configuration of the model selection device will be described with reference to FIGS. 1 and 2.
As shown in FIG. 1, the model selection device 100 includes a candidate model analysis unit 110, a transfer learning unit 120, a model evaluation unit 130, an optimum model selection unit 140, a model management unit 150, and a storage unit 160 as functional configurations. ..

The candidate model analysis unit 110 is a functional unit that extracts model candidates for analyzing the input data 10 extracted from the model management DB (DataBase) 200 via the model management unit 150. The transfer learning unit 120 is a functional unit that performs transfer learning for the input data 10 for model candidates. The model evaluation unit 130 is a functional unit that evaluates the model constructed by transfer learning. The optimum model selection unit 140 is a functional unit that selects the optimum model as a learning model for the input data 10 according to the evaluation of the model evaluation unit 130. The model management unit 150 is a functional unit that takes out a necessary model from the model management DB 200 and inputs / outputs / processes information to the model according to the instructions of each unit. The storage unit 160 is a functional unit that stores data.

The storage unit 160 stores the model management DB 200, the input data DB 210, and the calculation result output DB 220.
The model management DB 200 is a DB that stores a model to be selected as a model. It is assumed that the model to be selected is already created in another learning process. The input data DB 210 is a DB that stores input data that is a target for constructing a learning system. The calculation result DB 220 is a DB that stores the calculation result of the processing performed by the model selection device 100.
The data structure of the data used in the processing of the model selection device 100 will be described in detail later.

Next, the hardware / software configuration of the model selection device will be described with reference to FIG.
The hardware configuration of the model selection device 100 is realized by, for example, a general information processing device such as the personal computer shown in FIG.

In the model selection device 100, a CPU (Central Processing Unit) 502, a main storage device 504, a network I / F (InterFace) 506, a display I / F 508, an input / output I / F 510, and an auxiliary storage I / F 512 are connected by a bus. It is in the form of.

The CPU 502 controls each part of the model selection device 100, loads and executes a program required for the main storage device 504.
The main storage device 504 is usually composed of a volatile memory such as a RAM, and stores a program executed by the CPU 502 and data to be referred to.

The network I / F506 is an interface for connecting to the network.

The display I / F 508 is an interface for connecting a display device 520 such as an LCD (Liquid Crystal Display).

The input / output I / F 510 is an interface for connecting an input / output device. In the example of FIG. 2, a keyboard 530 and a pointing device mouse 532 are connected.

The auxiliary storage I / F 512 is an interface for connecting an auxiliary storage device such as an HDD (Hard Disk Drive) 550 or an SSD (Solid State Drive).
The HDD 550 has a large storage capacity, and stores a program for executing the present embodiment.

A candidate model analysis program 560, a transfer learning program 561, a model construction evaluation program 562, and an optimum model selection program 563 are installed in the model selection device 100. The candidate model analysis program 560, the transfer learning program 561, the model construction evaluation program 562, and the optimum model selection program 563 have the functions of the candidate model analysis unit 110, the transfer learning unit 120, the model evaluation unit 130, and the optimum model selection unit 140, respectively. It is a program to be executed.
Further, the HDD 550 stores a model management DB 200, an input data DB 210, and a calculation result output DB 220.

Next, the concept of the model selection method of the present embodiment will be specifically described with reference to FIG.
In this embodiment, as shown in FIG. 3, an example of a housing analysis system that analyzes housing data in Hakodate as input data is taken as an example, and an example of selecting the optimum model for transfer learning from existing models will be described. In this embodiment, in order to predict the house price, which is the objective variable, an example is a system in which a model is constructed and inferred using the factors of the house price in the house data (age, walking distance from the nearest station, area) as explanatory variables. To take.

Here, the explanatory variable is a machine learning term, and is a variable that affects the result, and the objective variable is a variable that indicates a value desired to be obtained by the system.

First, it is assumed that an existing model is constructed from housing data of other cities such as Tokyo, Sapporo, and Asahikawa. Next, consider constructing a Hakodate housing price prediction model using the learning data of Hakodate housing data as input data. Since Hakodate has little learning data, we want to improve the accuracy by transfer learning. At this time, the issue is which model should be selected from among a plurality of existing models.

In this embodiment, statistics (for example, variance, average, etc.) related to a specified item are calculated from Hakodate housing data, and a model having a close statistic of an existing model is selected as a candidate model. Next, the learning data of the Hakodate housing data, which is the input data, is input to an existing model (for example, model (Tokyo), model (Sapporo), ...), transfer learning is performed, and a new transfer learning model (hereinafter referred to as). , "Transition model") (for example, transfer model (Tokyo), transfer model (Sapporo), ...). After that, the test data of Hakodate housing data, which is the input data, is input to multiple transfer models, and the objective variable (house price) of the inferred data and the objective variable (house price) of the input data are calculated. By evaluating the error, we try to select the optimum model.

Next, the data structure used in the model selection apparatus of the first embodiment will be described with reference to FIGS. 4 to 8.
The house data table 300 is a table that stores the basic house data of the house analysis system. For example, as shown in FIG. 4, the house ID 300a, the age of 300b, the nearest station walk 300c, the restricted area 300d, and the area. It consists of fields of 300e, floor plan 300f, form 300g, and house price 300h.

The house ID 300a stores an ID that uniquely identifies the house. The age of 300b stores the number of years since the house was erected. The nearest station walk 300c stores the time it takes to walk from the house to the nearest station. This time is generally converted into a distance on the basis of 80 m / min. In the use area 300d, a use area (area classification defined by the Town Planning and Zoning Act) in which the house exists, for example, a character string representing "house-only" or "commercial area", a code, or the like is stored. The building area of the house is stored in the area 300e. In the floor plan 300f, a character string and a code representing the room configuration of the house, for example, 2LDK, 1DK, etc. are stored. In the form 300g, for example, an apartment, a condominium (divided ownership), a character string representing a detached house, a code, and the like are stored as the building form of the house. The evaluation price of the house is stored in the house price 300h.

In the present embodiment, the input data 10 and the data used for learning will be described by taking the data represented by the housing data table 300 as an example. In applying machine learning, the house price is the objective variable, and the other items are the explanatory variables.

The statistic item table 420 is a table for associating each item of housing data with each statistic. The item field stores the name of each item shown in FIG. 4, for example, age, walking distance from the nearest station, area, and so on. Then, a character string and a code representing the statistic are stored in each field for each correspondence of the histogram, the average value, and the variance. In FIG. 5, a symbol called statistic i-j is stored in each of the i-th record and the j-th field representing the statistic.

The model statistic information table 430 is a table to which the statistic information is associated with each model, and as shown in FIG. 6, the model ID 430a, the model file path 430b, the statistic 1-1, and the statistic 1-2. , Statistics 1-3, ....

The model ID 430a stores an ID that uniquely identifies the model. The model file path 430b stores the path in the file system of the model. In the statistic 1-1, the statistic 1-2, the statistic 1-3, ..., The values for the items associated with the statistic item table 420 in FIG. 5 are stored. For example, statistic 1-1 stores the variance for age.

The statistic comparison table 400 stores the result of the difference between the statistic of the input data and the statistic of the training data when each model is created (also referred to simply as "model statistic" in the present specification). It is a table, and as shown in FIG. 7, it is composed of a model ID 400a, a variance σ400b, and a difference 400c from the variance of the input data. Here, as an example of the statistic, the variance of the distribution of the data items is used, and the difference between the variance of the specified items of the model and the variance of the specified items of the input data is stored in this table.

An ID that uniquely identifies the model is stored in the model ID 400a. The variance σ400b stores the variance of the distribution of certain data items in the model. The difference 400c from the variance of the input data stores the difference between the variance of the distribution of a certain data item of each model and the variance of the distribution of the data item having the input data.

The transfer model result table 410 is a table that stores the difference between the value of the item that is the objective variable obtained from the transfer model and the true value of the item of the input data, and is a transfer model as shown in FIG. The ID 410a, the model file path 410b, and the error 410c are stored.

The transfer model ID 410a stores an ID that uniquely identifies the transfer model. The model file path 410b stores the path in the file system of the transfer model. The error 410c stores the root mean square error between the value of the item that is the objective variable obtained from the inference of the transition model and the true value of the item in the input data (details will be described later).

Next, the processing performed by the model selection device will be described with reference to FIGS. 9 to 16.

First, a series of processes from the acquisition of input data to the selection of a model by the model selection device will be described with reference to FIG.
First, the model selection device 100 performs data analysis processing (S01). The details of the data analysis process will be described later with reference to FIG.

Next, the model selection device 100 performs a candidate model search process (S02). The details of the candidate model search process will be described later with reference to FIG.

Next, the model selection device 100 performs a candidate model selection process (S03). The details of the candidate model selection process will be described later with reference to FIG.

Next, the model selection device 100 performs a transfer learning process (S04). The details of the transfer learning process will be described later with reference to FIG.

Next, the model selection device 100 performs a transfer model evaluation process (S05). The details of the transfer model evaluation process will be described later with reference to FIG.

Next, the model selection device 100 performs the optimum model selection process (S06). The details of the optimum model selection process will be described later with reference to FIG.

Next, the data analysis process will be described with reference to FIG.
First, the candidate model analysis unit 110 of the model selection device 100 acquires the input data 10 from the input data DB 210 via the model management unit 150 (S100).
Next, the items in the statistic item correspondence table 420 of the model management DB 200 and the correspondence information of the statistic are acquired via the model management unit 150 (S101).

Next, each statistic of the item of the input data 10 is calculated according to the correspondence information of the statistic item correspondence table 420 (S102).
Next, the calculation result is stored in the calculation result DB 220 (S103).
Next, the acquired input data 10 is separated into test data and training data (S104). Separating data into test data and training data is a common practice in the field of machine learning. The method of separation may be randomly selected, but the test data is generally smaller than the training data. The processing using the test data and the training data will be described later.

Next, the candidate model search process will be described with reference to FIGS. 11 and 12.
First, from the priority item input screen 500, the analysis items that the user emphasizes and the statistics used for the model analysis are input (S200).

As shown in FIG. 12, the important item input screen 500 includes an important item list 510 and a statistical list 520, and the user can select the item by a pointing device such as a mouse. In this example, it is assumed that "year of construction" is selected as the important item and variance is selected as the statistic.

Next, with respect to the selected statistic of the selected priority item, the calculation result of the statistic calculated in S103 in the priority item of the input data stored in the calculation result DB is acquired (S201).

Next, among the statistics of the model N, the model statistic information table 430 of FIG. 6 is referred to, and the selected statistic of the selected important item is acquired (S202). Here, the initial value of N is 1, and in the flowchart, it is described that the processing is performed as model 1, model 2, ....

Next, with respect to the input data 10, the difference between the calculation result S103 of the statistic calculated in S103 of the important item and the selected statistic of the selected important item of the model N is calculated and shown in FIG. It is stored in the statistic comparison table 400 (S203).

Then, it is determined in the model management DB whether all the target models have been calculated (S204), when all the models have been calculated (S204: YES), the processing is terminated, and all the models have not been calculated (S204). S204: NO), N is incremented (N ++), the next model N is taken out (S210), and the process returns to S202.

Next, the candidate model selection process will be described with reference to FIG.
First, sorting is performed in ascending order of difference from the variance of the input data stored in the statistic comparison table 400 (S300).

Next, upper M (where M is a predetermined number) models, that is, M models having a small difference from the variance of the input data are selected as candidate models (S301).

Next, the transfer learning process will be described with reference to FIG.
First, the learning data separated in S104 is acquired from the input data (S401).
Next, the transfer learning unit 120 of the model selection device 100 acquires the predetermined upper M models selected by the candidate model selection process of FIG. 13 from the model management unit 150 via the model management unit 150 (. S401).

Next, transfer learning is performed using the acquired model and training data, a transfer model ID is assigned to the created transfer model, the transfer model ID is stored in the model management DB 200, and the transfer model ID and file path are shown in the figure. It is stored in the transfer model result table 410 of 7 (S402).

Next, it is determined whether all of the selected upper M elements have been processed (S403), when all have been processed (S403: YES), the processing has been completed, and all have not been processed. (S403: NO), the next model is taken out (S410), and the process returns to S402.

Next, the transfer model evaluation process will be described with reference to FIG.
First, the model evaluation unit 130 of the model selection device 100 acquires the test data separated by S105 from the input data DB (S500).

Next, the transfer model M of the transfer model ID stored in the transfer model statistic table 410 is acquired (S501). Here, the initial value of M is 1, and in the flowchart, it is described that the transfer model 1, the transfer model 2, ...

Next, the test data is inferred by the transfer model M, the mean square error of the value of the objective variable of the data obtained by the inference of the transfer model and the true value of the item corresponding to the objective variable of the input data is calculated, and the error is calculated. It is stored in the transfer model result table 410 (S502). The average squared error is obtained by taking the arithmetic mean of the square of the difference between the measured value and the true value, and quantitatively expresses the degree of variation in the measured value. The smaller the average squared error, the more. The measurement accuracy is considered to be good. That is, it is an amount obtained by the following (Equation 1). Here, the smaller the error, the more suitable the model is for the input data.

Next, it is determined whether all the transition models have been calculated (S503), when all have been calculated (S503: YES), the processing is terminated, and when all have not been calculated (S503: NO), M is incremented (S503: NO). Return to M ++) (S510) and S502.

Next, the optimum model evaluation process will be described with reference to FIG.
Next, the optimum model selection unit 140 of the model selection device 100 selects the transfer model having the smallest error in the transfer model result table 410 as the optimum model (S600).
Here, K predetermined optimum models may be selected.

As described above, according to the present embodiment, the error between the true value of the desired item (house price) of the input data and the value of the objective variable (house price) obtained by inference from the model by transfer learning is evaluated. Therefore, the optimum model for the input data can be obtained.

[Embodiment 2]
Hereinafter, the second embodiment according to the present invention will be described with reference to FIGS. 5 to 18.

In the first embodiment, the optimum model for inferring from the input data was obtained by evaluating the error between the true value of the item to be obtained in the input data and the value of the objective variable obtained by inference from the model by transfer learning. ..

In this embodiment, in order to infer from the input data by evaluating the error between the true value of the desired item of the input data and the value of the objective variable obtained by inputting the input data into the existing model and inferring. It is an attempt to obtain the optimum model of.

In this embodiment, the points different from those in the first embodiment will be mainly described.

Hereinafter, the concept of the model selection method of the present embodiment will be described with reference to FIG. 17 by using a specific example.
Similar to the first embodiment, this embodiment is also most suitable for transfer learning from an existing model by taking a housing analysis system that analyzes the housing data of Hakodate as input data as an example, as shown in FIG. An example of selecting an existing model will be described.

First, it is assumed that an existing model is constructed from housing data of other cities such as Tokyo, Sapporo, and Asahikawa.
At this time, the issue is which model should be selected as the model used for transfer learning.

In this embodiment, the error between the true value of the item (house price) to be obtained from the Hakodate house data and the objective variable (house price) obtained by inputting the input data into the existing model and inferring it is evaluated. This is an attempt to select the optimum model.

First, the data structure used in the model selection device of the second embodiment will be described with reference to FIG.
The model result table 440 is a table that stores the difference between the statistics of the existing model and the statistics of the distribution of the input data, and as shown in FIG. 18, the transfer model ID 440a, the model file path 440b, and the error 440c Stored.

The transfer model ID 440a stores an ID that uniquely identifies the model. The model file path 440b stores the path in the file system of the model. The error 440c stores the average squared error between the value of the objective variable obtained from the model and the true value corresponding to the item of the objective variable with the input data (details will be described later).

Next, the processing performed by the model selection device will be described with reference to FIGS. 19 to 21.

First, a series of processes from the acquisition of input data to the selection of a model by the model selection device will be described with reference to FIG.
First, the model selection device 100 performs a data analysis process (S11). This data analysis process will be described later with reference to FIG. 20.
Next, the model selection device 100 performs a candidate model search process (S12). The details of the candidate model search process will be described later with reference to FIG. 21.
Next, the model selection device 100 performs the optimum model selection process (S06). This optimum model selection process is the same as the process of FIG. 16 of the first embodiment.

Next, the data analysis process of the second embodiment will be described with reference to FIG.
First, the candidate model analysis unit 110 of the model selection device 100 acquires the input data 10 from the input data DB 210 via the model management unit 150 (S100).
Next, the acquired input data 10 is separated into test data and training data (S104).
Each of these processes is the same as the corresponding step of the data analysis process shown in FIG. 10 of the first embodiment.

Next, the candidate model search process of the second embodiment will be described with reference to FIG.
First, the test data of the input data 10 separated in S104 of FIG. 20 is acquired from the input data DB (S1000).
Next, the model N is acquired from the model management DB 200 via the model management unit 150 (S1001). Here, the initial value of N is 1, and in the flowchart, it is described that the processing is performed as model 1, model 2, ....

Next, the mean square error of the value of the objective variable of the data obtained by inferring the test data by the model N and the true value of the item corresponding to the objective variable of the input data is calculated, and the error is stored in the model result table 440. (S1002).

Then, it is determined in the model management DB whether all the target models have been calculated (S1003), when all the models have been calculated (S1003: YES), the processing is finished, and when all the models have not been calculated (S1003). S1003: NO), N is incremented (N ++) (S1010), and the process returns to S1001.

In the optimum model selection process, the model having the smallest error value in the model result table of FIG. 18 is selected as the optimum model.

As described above, according to the present embodiment, the true value of the item (house price) to be obtained for the input data and the test data of the input data are input, and the objective variable (house price) obtained by inference from the existing model is input. By evaluating the error of the value of), the optimum model for the input data can be obtained.

[Embodiment 3]
Hereinafter, the third embodiment according to the present invention will be described with reference to FIGS. 22 to 28.

In the first embodiment, the optimum model for inferring from the input data was obtained by evaluating the error between the true value of the item to be obtained in the input data and the value of the objective variable obtained by inference from the model by transfer learning. ..

This embodiment attempts to obtain an optimum model by comparing the FI (Feature Importance) of a new model obtained from input data with the FI of an existing model. FI is an index showing the contribution of the objective variable of each explanatory variable in the learning model of the tree system.

In this embodiment, the points different from those in the first embodiment will be mainly described.

First, the data structure used in the model selection device of the embodiment will be described with reference to FIGS. 22 to 24.

The model FI management table 450 is a table that holds the FI information of each model from the upper level to the lower level, and as shown in FIG. 22, the model ID 450a, the model file path 450b, the FI 1st place item name 450c1, and the FI 1st place item name. It has each field of the value 450c2, the value of the FI second place item name 450d1, the value of the FI second place item name 450d2, and so on.

The model ID 450a stores an ID that uniquely identifies the model. The model file path 450b stores the path in the file system of the model. Subsequent fields are stored in descending order of FI, with the item name of FI and the value of the item name as a pair. In the example of FIG. 22, the value of the first record is "age" as the item name of the first place in FI, the value is 4.5, the item name of the second place in FI is "walking to the nearest station", and the value is. , 2.5 are stored.

The new model FI result table 460 is a table that stores the result of calculating the FI of the new model newly generated from the training data of the input data, and as shown in FIG. 23, the FI 1st place item name 460a1 and the FI 1st place item. It has the fields of the name value 460a2, the FI second place item name 460b1, the FI second place item name value 460b2, and so on.

The FI 1st place item name 460a1 and the FI 1st place item name value 460a2 store the item name and the value of the FI having the largest value in the FI of the new model newly generated from the learning data of the input data. Similarly, the subsequent fields are stored in descending order of FI, with the item name of FI and the value of the item name as a pair.

The model FI comparison table 470 is a table that stores the results when the FIs of the new model and the existing model are selected, and has the fields of the model ID 470a and the candidate selection flag 470b as shown in FIG. 24.

The model ID 470a stores an ID that uniquely identifies the model. The candidate selection flag 470b stores a flag (details will be described later) indicating the result when compared with the new model. The initial value of the candidate selection flag 470b is 0 (not selected).

Next, the processing performed by the model selection device of the third embodiment will be described with reference to FIGS. 25 to 28.

First, a series of processes from the acquisition of input data to the selection of a model by the model selection device will be described with reference to FIG. 25.
First, the model selection device 100 performs data analysis processing (S01). This data analysis process is the same as the process of FIG. 9 of the first embodiment.

Next, the model selection device 100 performs a candidate model search process (S22). The details of the candidate model search process will be described later with reference to FIG. 26.

Next, the model selection device 100 performs a candidate model selection process (S23). The details of the candidate model selection process will be described later with reference to FIG. 27.

Next, the model selection device 100 performs a transfer learning process (S04). The transfer learning process is the same as the process of FIG. 9 of the first embodiment.

Next, the model selection device 100 performs a transfer model evaluation process (S05). The transfer model evaluation process is the same as the process of FIG. 9 of the first embodiment.

Next, the model selection device 100 performs the optimum model selection process (S06). The optimum model selection process is the same as the process of FIG. 9 of the first embodiment.

Next, the candidate model search process of the third embodiment will be described with reference to FIG. 26.
First, the learning data of the input data 10 separated in S206 of FIG. 10 is acquired from the input data DB (S1200).
Next, a new model is created using the training data, and the FI is calculated (S1201).

Next, the model ID and FI of the model N are acquired from the model FI management table 450 of the model management DB 200 via the model management unit 150 (S1202). Here, the initial value of N is 1, and in the flowchart, it is described that the processing is performed as model 1, model 2, ....

Next, it is determined whether or not the FI of the new model created in S2001 and the item name of the FI of the model N are the same by a predetermined number (for example, the top 10) (S1203), and when they are the same (S1203: YES). Set 1 to the candidate selection flag 470b of the model FI result comparison table 470 (S1210), and when different (S1203: NO), go to S1204.

Then, it is determined whether or not all the target models have been processed in the model management DB (S1204), and when all the models have been processed (S1204: YES), the processing is terminated and all the models have not been processed (S1204). S1204: NO), N is incremented (N ++) (S1210), and the process returns to S1202.

Next, the candidate model selection process of the third embodiment will be described with reference to FIGS. 27 and 28.

First, by comparing the FI of the new model of S1201 in FIG. 26 and the comparison process of S1203, the FI of the model ID whose candidate selection flag of the model FI comparison table 470 is 1 is displayed on the transfer candidate selection screen 340 (S1300). ).

As shown in FIG. 28, the transfer candidate selection screen 340 is a screen for selecting a transfer model candidate. The transfer candidate selection screen 340 includes a new model FI display field 350 and an existing model display field 360.

The existing model display field 360 includes a selection check box 361, a model ID display field 362, and an FI display field 363. The user confirms the FI of each model, and the selection check box 361 is used to select a transfer model candidate with the mouse. Etc. can be instructed and selected by the pointing device.
Next, the user selects a candidate for the transfer model (S1301).

As described above, in the present embodiment, a more optimal model candidate for transfer learning can be selected by clearly indicating an index representing the model contribution of the item FI and selecting it for the user.

In the present embodiment, the FI related to the item of the model is calculated and presented to the user. Similarly, as an index showing the contribution of the item, the SHAP summary plot is used and presented to the user. May be good. Here, the SHAP summary plot is a visual representation of the contribution of the explanatory variables for model construction to the objective variable.

10 ... Input data 110 ... Candidate model analysis unit 120 ... Transfer learning unit 130 ... Model evaluation unit 140 ... Optimal model selection unit 150 ... Model management unit 160 ... Storage unit 200 ... Model management DB
210 ... Input data DB
220 ... Calculation result output DB

Claims (8)

An information processing device is a learning model selection method that selects a learning model suitable for transfer learning from existing learning models when generating a learning model for input data by transfer learning.
The input data includes the objective variable and the explanatory variables.
A step in which an information processing device generates a model by transfer learning using an existing learning model and input data.
A step of calculating an error between the inference result in which the input data is input to the model by the transfer learning and the true value which is the objective variable included in the input data.
A learning model selection method comprising: a step of selecting a learning model having a small error. The learning model selection method according to claim 1, wherein the error is a mean square error. An information processing device is a learning model selection method that selects a learning model suitable for transfer learning from existing learning models when generating a learning model for input data by transfer learning.
The step of calculating the statistic about the value of the specified item of the input data, and
The step of calculating the difference from the statistic regarding the value of the corresponding item of the existing model, and
The learning model selection method according to claim 1, further comprising a step of selecting a learning model having a small difference as a candidate for a learning model used for transfer learning. The learning model selection method according to claim 3, wherein a mean or a variance is used as the statistic. An information processing device is a learning model selection method that selects a learning model suitable for transfer learning from existing learning models when generating a learning model for input data by transfer learning.
A step of calculating an error between an inference result in which the input data is input to an existing learning model and a true value which is an objective variable included in the input data.
The learning model selection method according to claim 1, further comprising a step of selecting a learning model having a small error as a candidate for a learning model used for transfer learning. The learning model selection method according to claim 5, wherein the error is a mean square error. An information processing device is a learning model selection method that selects a learning model suitable for transfer learning from existing learning models when generating a learning model for input data by transfer learning.
A step of generating a new learning model from the input data and calculating the contribution of each item of the input data to the inference result of the new learning model.
Comparing the contribution of each item of the input data used when generating the existing model, the upper item with the large contribution of each item of the data of the new model and each item of the data of the existing model The learning model selection method according to claim 1, wherein a higher-ranking item having a large contribution includes a step of selecting an existing model that matches a predetermined number as a candidate for a learning model used for transfer learning. The learning model selection method according to claim 7, wherein the contribution is a summary plot of FI (Feature Importance) or SHAP.

Images