JP6708295B2 - Feature selection system, feature selection method, and feature selection program - Google Patents

Description

The present invention relates to a feature selection system, a feature selection method, and a feature selection program for selecting features used for prediction.

When constructing a prediction model or a discriminant model, a feature selection process for selecting a meaningful feature from a plurality of features is generally performed. By performing feature selection, it becomes possible to represent which features of the observation data are important and how they are related.

For example, Patent Literature 1 describes a feature selection device that selects a feature used for malware determination. The feature selection device described in Patent Document 1 performs machine learning in advance on a readable character string included in an executable file of malware to extract words often used in malware. Further, in the feature selection device described in Patent Document 1, of the feature candidate groups, a feature group appearing in a set in the verification data is represented by any feature, and a feature other than the representative (redundant feature) is represented. delete.

JP, 2016-31629, A

Once the target can be predicted, future optimization strategies can be considered based on that prediction. For example, when a prediction model is generated, optimization based on this prediction model can be performed. It can be said that the optimization based on the prediction model is to optimize the features included in the prediction model so as to maximize the value of the objective function represented by the prediction model. An example of such optimization is optimizing the price using a sales number prediction model.

It is possible to construct the above-mentioned prediction model by using a general learning method based on past data. At that time, in a general learning method, as described in Patent Document 1, redundant features are generally excluded from the prediction model and not selected. By eliminating redundant features, you can mitigate the effects of dimension curses, speed up learning, and improve model readability without significantly affecting prediction accuracy. Excluding redundant features is also useful from the viewpoint of preventing overlearning.

Here, there is a case where one feature used for optimization of the prediction target is influenced by another feature used for prediction of the prediction target. In other words, there may be a causal relationship between one feature and another feature. When a feature is selected without considering such a causal relationship, a problem may occur in optimization even if the prediction accuracy does not occur. Hereinafter, a situation where a problem occurs will be described using a specific example.

Here we consider the problem of optimizing the price of an umbrella. It is assumed that x is the price of the umbrella, y is the number of sales of the umbrella, and z is a variable representing the weather, and the number of sales y is predicted. Here, x and z are one of the characteristics that are likely to affect the number of umbrella sales. In the past data, the number of umbrellas sold is high in the case of rain, so the store owners set the price of the umbrella high in anticipation of this, and on the contrary, when it is sunny, the number of umbrellas sold is low, In anticipation, the store owner sets the price of the umbrella low.

When this situation is expressed using the above variables, a rainy day is (x, y, z)=(“high”, “more”, “rain”), and a sunny day is (x, y, z). = (“Low”, “low”, “fine”). At this time, y is predicted using x and z. On the other hand, since there is a strong correlation between x and z, when predicting y in such a situation, it is sufficient to explain y only by x (that is, when x=high, z=rain Z is considered to be a redundant feature by the feature selection process. That is, z is excluded by the feature selection process. Therefore, in the prediction, the probability that p (y=high|x=high)=1 holds.

Since z, which is a feature, is not selected, it can be said from the above probability equation that y increases as x increases. Therefore, the optimization result for increasing y is “always sell umbrellas at a high price”. Can be judged. This result is counterintuitive, which means that even on sunny days, selling umbrellas at higher prices will increase the number of sales. This is the difference between the result of intervention by optimization and the prediction, and in the above example, the amount that can be sold naturally when the price is high and the amount that can be sold when the price itself is increased are different. That is, when the value obtained by performing the intervention is represented as do (variable), the relationship of the following Expression 1 is established.

p(y=many|x=high)≠p(y=many|do(x=high)) (Equation 1)

The prediction formula p (y=many|x=high) illustrated in Formula 1 has high accuracy in past data. However, it should be noted that there is no actual data that "sold an umbrella at a high price on a sunny day". In this case, the optimizer is performing optimization based on high prediction accuracy even though there is no combination of strategies (x=high, z=clear) in the past data. This can be understood as a phenomenon in which the information that the strategy is high-risk is not input by the feature amount selection, and the optimizer cannot appropriately judge. If optimization is performed without considering the situation shown in Expression 1, there is a possibility that a dangerous strategy may be selected as an optimization strategy. That is, in the prediction scene, the prediction accuracy in the unobserved situation is not guaranteed, while in the optimization scene, the situation not observed in the past is also taken into consideration.

It is assumed that there is a prediction model that is trained using only the selected features by performing appropriate feature selection from the perspective of prediction, that is, feature selection that excludes redundant features from the perspective of prediction. This predictive model appears to perform well as long as it is used for predictive purposes. However, when this predictive model is used for the purpose of optimization, there is a case where an appropriate optimization cannot be performed as a result of selecting a dangerous strategy. The set of features needed to learn a prediction model used only for prediction purposes and the set of features needed to learn a prediction model used for prediction-based optimization do not necessarily match. The present inventor has found out that When performing optimization based on a prediction model, it is preferable that features that are redundant for the purpose of prediction can be selected without omission for features that are necessary for proper optimization.

Therefore, in the present invention, when selecting a feature used for learning a prediction model, a feature selection system, a feature selection method, and a feature selection that can know the feature required for appropriate optimization performed using the prediction model The purpose is to provide the program.

A feature selection system according to the present invention is a feature selection that selects a feature used for learning of a prediction model when calculating an operation variable for optimizing an objective function represented by using the prediction model under a constraint condition. In the system, a reception unit that receives designation of a prediction target and designation of an operation variable, a first feature set that is a set of features that affect the prediction target from a set of features that can affect the prediction target, a feature selecting unit for selecting a second feature set and a set of features that affect the operation variable, a first feature set and an output unit for outputting a second feature set, the feature selection unit, as the manipulated variable The selected feature is selected to be included in the first feature set .

The feature selection method according to the present invention is a feature selection for selecting a feature used for learning of a prediction model when calculating an operation variable for optimizing an objective function represented by using the prediction model under a constraint condition. A method of accepting designation of a prediction target and designation of an operation variable, and selecting a first feature set, which is a set of features affecting the prediction target, and an operation variable from a set of features that may affect the prediction target. A second feature set, which is a set of influential features, is selected, and at the time of selection, a feature specified as an operation variable is selected to be included in the first feature set, and the first feature set and the second feature set are selected. Is output.

A feature selection program according to the present invention is a computer that selects a feature used for learning a prediction model when calculating an operation variable for optimizing an objective function represented by using the prediction model under a constraint condition. A feature selection program to be applied, which is a reception process for receiving specification of a prediction target and specification of an operation variable in a computer, and a set of features that affect the prediction target from a set of features that may affect the prediction target. A feature selection process of selecting a certain first feature set and a second feature set that is a set of features that affect the manipulated variables, and an output process of outputting the first feature set and the second feature set , In the feature selection process, the feature specified as the manipulated variable is selected to be included in the first feature set .

According to the present invention, when a feature used for learning a prediction model is selected, it is possible to know the feature required for appropriate optimization performed using the prediction model.

1 is a block diagram showing an embodiment of a price optimization system according to the present invention. It is a flow chart which shows an example of operation when a price optimization system performs price optimization. It is a flow chart which shows an example of processing in which a price optimizing system selects a feature according to specification of a prediction object and an operation variable. It is explanatory drawing which shows the example of the sales record of the store recorded on the database. It is a block diagram which shows the outline|summary of the feature selection system by this invention. It is explanatory drawing which shows a specific example of the data input into the feature selection system by this invention. It is a schematic block diagram which shows the structure of the computer which concerns on at least 1 embodiment.

First, terms used in the present invention will be described. In the present embodiment, “feature” is used to mean an attribute name. A specific value indicated by the attribute will be referred to as an attribute value. An example of the attribute is price, and an example of the value of the attribute in this case is 500 yen. In the following description, when "feature" is described, its role is not limited and may mean an explanatory variable, a prediction target, or an operation variable described later, in addition to the meaning of the attribute name.

The explanatory variable means a variable that can influence the prediction target. In the example of the umbrella price optimization problem described above, "whether it rains in the morning", "whether it rains in the afternoon", etc., and "whether it is the end of the month", etc. are explained. It corresponds to a variable. In this embodiment, a candidate for an explanatory variable is input as an input when performing feature selection. That is, in the feature selection, an explanatory variable that can influence the prediction target is selected as a feature from the explanatory variable candidates, and is output as a result. In other words, the explanatory variable selected in the feature selection is a subset of candidate explanatory variables.

The prediction target is also called an “objective variable” in the field of machine learning. In addition, in order to avoid confusion with an “objective variable” generally used in an optimization process described later, a variable representing a prediction target is described as an explained variable in the following description. Therefore, it can be said that the prediction model is a model in which the explained variable is represented by using one or more explanatory variables. In the present embodiment, a model obtained as a result of the learning process may be referred to as a learned model. In the present embodiment, the prediction model is a specific aspect of the learned model.

The manipulated variable means a variable into which some (for example, human) intervention occurs during operation. Specifically, it means a variable to be optimized in the optimization process. Note that the manipulated variable is a variable that is generally called an "objective variable" in the optimization process, but as described above, the term "objective variable" is used to avoid confusion with the objective variable used in machine learning. Instead, the present invention will be described. In the example of the umbrella price optimization problem described above, "umbrella price" corresponds to the manipulated variable.

The manipulated variable is a part of the explanatory variable. In the following description, when it is not necessary to distinguish between explanatory variables and manipulated variables, they are simply described as explanatory variables, and when distinguished between explanatory variables and manipulated variables, explanatory variables mean variables other than manipulated variables. When distinguishing between explanatory variables and manipulated variables, explanatory variables other than manipulated variables may be expressed as external variables.

The objective function means a target function for obtaining a maximum or minimum value by optimizing an operation variable under given constraints in the optimization processing. In the example of the umbrella price optimization problem described above, the function that calculates the sales amount (number of sales x price) corresponds to the objective function.

Embodiments of the present invention will be described below with reference to the drawings.

FIG. 1 is a block diagram showing an embodiment of a price optimization system according to the present invention. The price optimization system 100 according to the present embodiment is a system that performs prediction-based optimization, and includes a reception unit 10, a feature selection unit 20, a learning unit 30, an optimization unit 40, and an output unit 50. ing. Since the price optimizing system 100 of the present embodiment performs feature selection as a specific mode, the price optimizing system 100 can be called a feature selecting system.

That is, the price optimization system of the present embodiment is a system for learning a prediction model used for prediction of a prediction target, and optimizes an objective function represented by using the prediction model under constraint conditions. This is a system for calculating manipulated variables for. Here, the objective function represented by using the prediction model is either an objective function defined by using a prediction value predicted by using the prediction model as an argument, or an objective function defined by using a parameter of the prediction model as an argument. Also means.

The reception unit 10 receives a prediction target (in other words, an explained variable), a set of features that can affect the prediction target (in other words, an explanatory variable candidate), and an optimization target (in other words, an operation variable). Accept. Specifically, the receiving unit 10 receives designation of which feature is the explained variable y and designation of which feature is the operation variable x. The reception unit 10 also receives candidates for the explanatory variable z. In addition, when the price optimization system 100 holds the candidate for the explanatory variable z in advance, the reception unit 10 may receive two types of specification of the prediction target which is the explained variable y and specification of the operation variable x. Good.

As described above, since the manipulated variable x is a part of the explanatory variable z, the reception unit 10 receives the candidate for the explained variable z and the identifier of the manipulated variable x included in the explained variable z. Good. In the case of the umbrella price optimization problem described above, the explained variable y represents the number of umbrella sales, the manipulated variable x represents the umbrella price, and the explanatory variable z represents the weather. Also. The receiving unit 10 also receives various parameters required for the subsequent processing.

The feature selection unit 20 selects a feature used for learning the prediction model. Specifically, the feature selection unit 20 selects a set of features that affect the prediction target from the set of features that the reception unit 10 has received that may affect the prediction target. Hereinafter, the set of features that affect the prediction target is referred to as a first feature set. For example, in the case of the umbrella price optimization problem described above, the price is calculated as a set (first feature set) that influences the number of sales from the set of features that can affect the number of sales of the umbrella (product) that is the prediction target. To be elected. At this time, when there are a plurality of features that are redundant to each other for explaining the prediction target, some of the redundant features are excluded from the first feature set. In the above-mentioned example, the price and the weather are regarded as mutually redundant features as the features for explaining the prediction target (the number of sales), and one of the price and the weather is excluded from the first feature set. In the above example, the weather is excluded.

Further, the feature selection unit 20 of the present embodiment selects a set of features that affect the manipulated variable from the set of features that can affect the prediction target received by the reception unit 10. Hereinafter, the set of features that affect the manipulated variable will be referred to as the second feature set. For example, in the case of the umbrella price optimization problem described above, weather is selected as a set (second feature set) that affects the price, which is an operation variable. At this time, when there are a plurality of features that are redundant to explain the manipulated variable, some of the redundant features are excluded from the second feature set.

In this way, the feature selection unit 20 determines from the set of features that can affect the number of sales of the product that is the prediction target to the first feature set that affects the prediction target (the number of sales) and the operation variable (the price of the product). Select a second feature set to influence. Here, the first feature set is a necessary and sufficient feature set when learning the prediction model used only for the purpose of prediction. The features that are not included in the first feature set and that are included in the second feature set are not necessarily required when learning a prediction model used only for the purpose of prediction, but are used for prediction-based optimization. This is a necessary feature when learning a model. The feature selection unit 20 does not exclude the manipulated variable itself (that is, the manipulated variable always remains in either the first feature set or the second feature set).

Although the case where the features are selected using a specific example has been described above, the feature selection unit 20 may select the first feature set and the second feature set using a generally known feature selection technique. An example of the feature selection technique is L1 regularization. However, the method by which the feature selection unit 20 selects a feature is not limited to L1 regularization.

The feature selection includes, for example, greedy feature selection such as matching orthogonal purchase, and selection based on various information amount criteria. The regularization method is a method in which a penalty is added every time a large number of feature quantities are selected. The greedy method is a method of selecting a predetermined number of feature amounts from influential feature amounts. The information amount criterion is a method of imposing a penalty based on a generalization error generated by selecting many feature amounts. A specific method of feature selection using L1 regularization will be described later.

The learning unit 30 learns a prediction model in which the features included in the first feature set and the features included in the second feature set are used as explanatory variables, and the feature of the prediction target is used as the explained variable. In the case of the price example, the learning unit 30 learns a prediction model in which the number of sales is a prediction target with the features included in the first feature set and the features included in the second feature set as explanatory variables. At that time, the learning unit 30 learns the prediction model using at least one feature included in the second feature set but not included in the first feature set as an explanatory variable. In addition, it is preferable that the learning unit 30 sets all the features included in the first feature set and the features included in the second feature set as explanatory variables.

In general feature selection, since features included in the second feature set are not selected, it is difficult to perform learning including features that affect optimization processing described later. On the other hand, in the present embodiment, since the learning unit 30 learns the model by using the features included in the second feature set but not included in the first feature set as explanatory variables, the optimization process that is the post-process is considered. Can generate models.

The optimization unit 40 optimizes the value of the manipulated variable so as to maximize or minimize the function of the explained variable defined by using the prediction model generated by the learning unit 30 as an argument. In the case of the sales example, the optimization unit 40 optimizes the price of the product under the constraint condition so that the sales amount defined by using the prediction model as an argument is high. More specifically, the optimizing unit 40 optimizes the price of the product under the constraint condition so that the sales amount defined by using the sales number predicted using the prediction model as an argument becomes high.

When optimizing using the prediction model, it is possible to input information indicating the distribution of the prediction error to the optimizing unit 40 and perform the optimization based on the information. In other words, by imposing a penalty on a strategy with a large prediction error, optimization that avoids a risky strategy can be performed. This is called robust optimization, stochastic optimization, etc., as opposed to optimization that does not use prediction errors. For example, if the prediction model is represented by y=a ₁ x ₁ +b, the distribution of prediction errors is the distribution for a ₁ and b. The distribution of prediction errors is, for example, a variance-covariance matrix. The distribution of the prediction error input here depends on the content of the prediction model, more specifically, on the features included in the second feature set but not included in the first feature set.

For example, the manipulated variable is x ₁ , a feature that is an explanatory variable and is included in the first feature set is z ₁ , and a feature that is an explanatory variable that is included in the second feature set but is not included in the first feature set is Let z ₂ be y and the explained variable be y. When general feature selection is performed that does not consider features (that is, z ₂ ) that are included in the second feature set but not included in the first feature set, for example, the prediction model shown in Equation 2 below. Is generated.

y=a ₁ x ₁ +a ₂ z ₁ +b (Formula 2)

On the other hand, when the feature selection considering z ₂ is performed as in the present embodiment, for example, the prediction model shown in the following Expression 3 is generated.

y=a ₁ x ₁ +a ₂ z ₁ +a ₃ z ₂ +b (Formula 3)

As described above, even if the feature (z ₂ ) is not necessarily required to generate the prediction model, the features are selected so as to be included in the prediction model. Therefore, a more appropriate prediction error distribution is optimized by the optimization unit 40. Can be entered in.

In the problem of optimizing the price of the umbrella described above, the above expression 2 corresponds to the case where the feature quantity z related to the weather is not selected, and the above expression 3 corresponds to the case where the feature quantity z related to the weather is selected. The above expression 2 shows that the prediction error distribution has high prediction accuracy both when the price is high and when the price is low. On the other hand, the above expression 3 includes a prediction error distribution indicating that the prediction accuracy is good when the price is high due to rain, but the prediction accuracy is low when the price is sunny and the price is high. Therefore, by performing optimization in consideration of the situation as shown in Expression 3, it is possible to avoid a situation in which a high-risk strategy is selected due to the feature amount selection.

The optimizing unit 40 may perform any optimization method, and the operation variable (price) may be optimized using a method for solving a general optimization problem.

The output unit 50 outputs the optimization result. For example, when the price optimization is performed so as to increase the sales amount, the output unit 50 may output the optimum price and the sales amount at that time.

The output unit 50 may output not only the optimization result but also the first feature set and the second feature set selected by the feature selection unit 20. At this time, the output unit 50 may output the features included in the first feature set and the features included in the second feature set but not included in the first feature set in a distinguishable manner. .. As an example of the method of outputting in a distinguishable manner, a method of changing the color of a feature included in the second feature set but not included in the first feature set, a method of highlighting, a method of changing the size, and italics Examples include a method of displaying. The output destination of the output unit 50 is arbitrary, and may be, for example, a display device (not shown) such as a display device included in the price optimization system 100.

The first feature set is a feature selected by a general feature selection process, and the second feature set is a feature selected in consideration of an optimization process which is a post-process, and appears in a general feature selection process. There is no feature. By displaying such features separately, it becomes possible for the user to grasp and select the appropriate features used when executing the optimization processing. As a result, the user can browse the displayed information and adjust the characteristics by utilizing the domain knowledge.

The receiving unit 10, the feature selecting unit 20, the learning unit 30, the optimizing unit 40, and the output unit 50 are realized by a CPU of a computer that operates according to a program (price optimizing program, feature selecting program).

For example, the program is stored in a storage unit (not shown) included in the price optimization system 100, the CPU reads the program, and in accordance with the program, the reception unit 10, the feature selection unit 20, the learning unit 30, and the It may operate as the optimization unit 40 and the output unit 50.

Further, each of the reception unit 10, the feature selection unit 20, the learning unit 30, the optimization unit 40, and the output unit 50 may be realized by dedicated hardware.

Next, an operation example of the price optimization system 100 of this embodiment will be described. FIG. 2 is a flowchart showing an operation example when the price optimization system 100 performs price optimization.

The feature selection unit 20 selects the first feature set that affects the number of sales (that is, the explained variable y) from the set of features that can affect the number of sales of the product (that is, the candidate for the explanatory variable z) (step S11). Further, the feature selecting unit 20 selects the second feature set that affects the price of the product (that is, the operation variable x) from the set of features that can affect the number of sales of the product (step S12).

The learning unit 30 learns a prediction model in which the number of sales is a prediction target using the features included in the first feature set and the second feature set as explanatory variables. At that time, the learning unit 30 learns a prediction model having at least one feature included in the second feature set but not included in the first feature set as an explanatory variable (step S13).

The optimizing unit 40 optimizes the price of the product under the constraint condition so that the sales defined by using the prediction model as an argument are high (step S14).

Further, FIG. 3 is a flowchart showing an example of a process in which the price optimizing system 100 selects a feature in accordance with the designation of the prediction target and the operation variable.

The reception unit 10 receives the designation of the prediction target (that is, the explained variable y) and the designation of the manipulated variable (that is, the manipulated variable x) (step S21). The feature selecting unit 20 selects a first feature set that affects the prediction target and a second feature set that affects the operation variable from the set of features that can affect the prediction target (that is, the candidate for the explanatory variable z). Yes (step S22). The feature selection unit 20 may input the selected first feature set and second feature set to the learning unit 30.

The output unit 50 outputs the first feature set and the second feature set (step S23). At this time, the output unit 50 outputs the features included in the first feature set and the features included in the second feature set but not included in the first feature set in a distinguishable manner. Good.

As described above, in the present embodiment, the feature selection unit 20 selects the first feature set that influences the number of sales and the second feature set that influences the price of the product from the set of features that can influence the number of sales of the product. Is selected, the learning unit 30 learns a prediction model in which the features included in the first feature set and the second feature set are used as explanatory variables, and the number of sales is a prediction target, and the optimization unit 40 selects the prediction model. The price of the product is optimized under the constraint condition so that the sales defined as an argument becomes high. At that time, the learning unit 30 learns a prediction model having at least one feature included in the second feature set but not included in the first feature set as an explanatory variable.

Therefore, when optimizing the price based on the prediction, it is possible to select a feature for optimizing the price so as to avoid a dangerous strategy.

Further, in the present embodiment, the reception unit 10 receives the designation of the prediction target and the designation of the operation variable, and the feature selection unit 20 selects the first feature that influences the prediction target from the set of features that can affect the prediction target. The output unit 50 outputs the feature set and the second feature set that affects the manipulated variable.

Therefore, when selecting the features used for learning the prediction model, it is possible to know the features required for appropriate optimization performed using the prediction model.

Next, the process in which the price optimization system 100 of the present embodiment selects a feature will be described using a specific example of L1 regularization. As described above, the L1 regularization is only one specific example of many feature selection techniques, and the feature selection technique that can be used in the present invention is not limited to the L1 regularization. Here, let's consider an example in which an umbrella sells on a rainy afternoon. The manipulated variable x represents the price of the umbrella, the dependent variable y represents the number of sales of the umbrella, and the explanatory variables z _{1 to} z ₃ are “rain in the morning” and “rain in the afternoon”, respectively. Whether it is “the end of the month (15th or later)” is represented by a 0-1 variable. Here, it is assumed that the true sales number y is generated as the following Expression 4.

y=−7z ₁ +14z ₂ −x/50+15+noise (Equation 4)

In Equation 4, sales increase when it is raining in the afternoon (ie, z ₂ =1), but if it rains in the morning (for example, because the customer already bought an umbrella in the morning), the sales It is assumed that the model will fall. The explanatory variable z ₃ is a candidate for the explanatory variable, but can be said to be a variable not related to sales. It should be noted that noise has a random value of (0, 1, 2) in order to simplify the description.

On the other hand, it is assumed that the shop owner who knows that the umbrella can be sold on a rainy day sets the price of the umbrella based on the following Expression 5.

_{x = -100z 1 + 200z 2 +500} ( Equation 5)

FIG. 4 is an explanatory diagram showing an example of the sales record of the store recorded in the database. In the example shown in FIG. 4, the price x, the number of sales y in the afternoon at the time of aggregation, and the presence or absence of characteristics at the time of aggregation are recorded for each aggregation unit identified by Id. For example, the sales record identified by Id=1 indicates that when the price was set to 500 yen at the end of the month when neither morning nor afternoon was raining, there were 6 umbrellas sold in the afternoon. Show.

It is assumed that feature selection for prediction is performed based on such data. In the following description, the feature selection unit 20 performs feature selection by using L1 regularization (Lasso) to select a non-zero w _i that minimizes Equation 6 below. In Expression 6, the coefficient of the Lasso penalty is set to 1/10 in order to simplify the description below.

Assuming that sufficient data are obtained, w _i (and appropriately selected c) satisfying the relationship shown in the following Equation 7 or Equation 8 and their linear combination (a×(equation Similarly, the data of w _i )+(1−a)×(w _i shown in Expression 8) described in FIG. 7 explains the data well, and the first term in Expression 6 becomes the minimum. However, the constraint on the sparsity of the second term in Equation 6 yields the set of w _i shown in Equation 7. This means that the penalty calculated from the second term is 1/200 in the set of w _i shown in Expression 7, while the penalty calculated from the second term is 1.20 in the set of w _i shown in Expression 8. This is because it becomes 5. Therefore, x is selected as the feature.

w ₀ =1/20, w ₁ =w ₂ =w ₃ =0 (Equation 7)
w ₀ =0, w ₁ =−5, w ₂ =10, w ₃ =0 (Equation 8)

In this specific example, the case where the ideal w ₀ is obviously small is illustrated. However, even when w ₀ is large, the same is obtained by designating that w ₀ is always selected in the feature selection setting. You can observe the phenomenon. This setting is made especially when the post-processing optimization is assumed and it is assumed that the price indicating feature should remain.

Further, the feature selection unit 20 further selects the feature that explains x in addition to the feature selected based on Expression 6. Specifically, the feature selection unit 20 performs feature selection by selecting a non-zero w′ _i that minimizes Equation 9 below.

When w′ ₁ =−100 and w′ ₂ =−200, the first term in Expression 9 becomes the minimum. If the frequency of rainy days is high enough, for example, that there is independent rain in the morning and afternoon once every five days, the effect of minimizing the first term is sufficiently larger than the penalty of the second term. Become. As a result, since w′ ₁ =−100 and w′ ₂ =−200 are solutions, z ₁ and z ₂ are selected as features. Heretofore, the invention according to the present embodiment has been described with respect to a specific example executed by using the L1 regularization. The feature selection technique that can be used in the present invention is not limited to L1 regularization, and other feature selection techniques can also be used.

By the above-described feature selection process, that is, by the feature selection process that further selects the feature that describes the manipulated variable in addition to the feature that describes the prediction target, x, z ₁ and z ₂ are selected as features. In other words, since the optimization unit 40 can recognize x, z ₁ and z ₂ as features required for optimization, it can determine that the weather should be taken into consideration for optimization. You can avoid choosing dangerous strategies such as "selling at a high price."

Here, the reason why it is possible to avoid selecting the above-mentioned dangerous strategy will be described in more detail. Assuming that the features x, z ₁ and z ₂ are correctly selected, the prediction formula shown in the following Formula 10 is created, and w ₀ hat, w ₁ hat and w ₂ hat (hat is superscript ^) are obtained by estimation. Think about it.

When the x vector and the w hat vector are represented by the following equation 11, the y hat is represented by the following equation 12.

It is assumed that the past strategy x is generated as in the following Expression 13 based on the above Expression 5.

x=−100z ₁ +200z ₂ +500+ε ₂ (Equation 13)

In the equation 10 and equation _{13, ε 1 ~N (0,} σ 1 2), ε 2 ~N (0, σ 2 2) in, sigma ₂ ² is sufficiently than the sigma ₁ ² and the data number n Suppose it is small. Note that N(0, σ ² ) represents a normal distribution with mean 0 and variance σ ² .

Here, the vectors v ₁ , v ₂ , and v ₃ are defined. First, v ₁ is defined as in Expression 14 below. v ₁ satisfies the following formula 15 while (x z ₁ z ₂ ) satisfies the above formula 13.

It is assumed that the least squares method is used as the estimation method. At this time, assuming that the true coefficient w ^*T =(-1/50-71415), the estimated value approximately follows the probability distribution shown in Expression 16 below. Here, for simplification of description, the approximate expression shown in Expression 17 is assumed.

In Expression 17, σ ₂ ′=0 (σ ₂ ) and γ ₂ , γ ₃ and γ ₄ are constants. Further, v ₂ , v ₃ , and v ₄ are standardized vectors that are orthogonal to each other including v ₁ .

During _{optimization,} z 1, realizations tilde _{z 1} of _{z 2,} tilde _{z 2} (tilde ~ superscripts) and was obtained. At this time, consider the robust optimization method in the elliptical uncertainty region shown in the following Expression 18.

In Equation 18, it is assumed that the estimated value w vector hat and the variance-covariance matrix Σ of its prediction error are obtained. Σ may also be replaced by the estimated value. Also, λ is a properly selected positive parameter. At this time, the following Expression 19 is established.

Now, since 1/σ ₂ ′ is sufficiently larger than σ ₁ /√n, the price strategy x that does not satisfy the above expression 15 suffers a large penalty in the above expression 18. Therefore, it is easy to select a price that satisfies the following Expression 20.

Expression 20 above is equivalent to satisfying Expression 13 above. Therefore, in the above specific example, it corresponds to "putting a low price on a sunny day".

The above contents are generalized as follows. The optimization problem of the strategy x with respect to the true parameter θ ^* is defined by Equation 21 shown below.

In Expression 21, X is a domain and v is a function. Here, consider a robust optimization problem when the estimated value θ hat and the error distribution are obtained instead of θ ^* . If the error is assumed to be normal, the following equation 22 is typically defined using the error covariance matrix Σ. Note that the robust optimization method may be used by a method different from Expression 22. In Equation 22, the second term acts as a penalty for strategies with large prediction variances.

So far, I explained why you can avoid choosing a dangerous strategy. The following will also be described from the description of the present embodiment. As shown in Equation 1 above, p (y=many|x=high) and p(y=many|do(x=high)) are not equal. On the other hand, even when the value (do (x=high)) obtained by performing the intervention is used, not only the feature amount that can explain the prediction target y but also the feature amount that can explain the manipulated variable x should be left. Good. This means the content represented by the following Expression 23.

p (y=many|x=high, z=rain)=p(y=many|do(x=high), z=rain)
(Equation 23)

Next, the outline of the present invention will be described. FIG. 5 is a block diagram showing an outline of the feature selection system according to the present invention. The feature selection system 90 according to the present invention receives input of data (more specifically, a set of features) from the outside by the user. FIG. 6 is an explanatory diagram showing a specific example of data that the feature selection system illustrated in FIG. 5 receives from the outside by the user. The data illustrated in FIG. 6 is data indicating the price of the umbrella and the number of sales of the umbrella for each day, and the situation of the day (for example, weather, maximum temperature, whether an advertisement is placed, whether it is a holiday, etc.). Is.

The feature selection system 90 according to the present invention is used for learning a prediction model when calculating an operation variable (for example, an operation variable x) for optimizing an objective function represented by using the prediction model under a constraint condition. A feature selection system for selecting a feature to be used, which receives a designation of a prediction target (for example, an explained variable y) and a designation of an operation variable, and an influence on the prediction target. From a set of possible features (for example, candidates for the explanatory variable z), a first feature set, which is a set of features that affect the prediction target, and a second feature set, which is a set of features that affect the operation variable, are selected. The feature selection unit 92 (for example, the feature selection unit 20) and the output unit 93 (for example, the output unit 50) that outputs the first feature set and the second feature set.

With such a configuration, when a feature used for learning the prediction model is selected, it is possible to know the feature required for proper optimization performed using the prediction model.

For example, in the case of the umbrella price optimization problem, in the example shown in FIG. 6, the feature selection system 90 (more specifically, the reception unit 91) specifies the umbrella sales to the prediction target (explained variable y). The number of the umbrellas is accepted, and the price of the umbrella is accepted to specify the manipulated variable (the manipulated variable x). Further, the feature selection system 90 (more specifically, the reception unit 91) designates a set of features (candidates for the explanatory variable z) that can influence the prediction target, by specifying the situation of the day (for example, weather, maximum temperature, Whether or not an advertisement has been placed, whether or not it is a holiday, etc.) are accepted. Note that the feature selection system 90 receives the price of the umbrella and the situation of the day as a set of features, and among the features, identification information indicating that the price of the umbrella is an operation variable (other than that, candidates for the explanatory variable z). (Information indicating that) is also accepted.

The feature selection system 90 also includes a learning unit (for example, the learning unit 30) that learns a prediction model having at least one feature included in the second feature set but not included in the first feature set as an explanatory variable. May be.

Then, the learning unit may learn a model in which all the features included in the first feature set and the features included in the second feature set are used as explanatory variables, and the feature that is the prediction target is used as the explained variable. With such a configuration, the prediction model used as the input of the optimization process can be appropriately learned in consideration of the optimization process that is the post-process.

The output unit 93 also displays the features included in the first feature set and the features included in the second feature set but not included in the first feature set on the display device in a distinguishable manner. May be. With such a configuration, it is possible to distinguish between a feature selected in the case of specializing in prediction and a feature first selected in the case of considering an optimization process that is a post-process.

Further, the feature selecting unit 92 obtains the first feature set from the set of features that may affect the prediction target by performing the feature selection process with the prediction target as the explained variable, and sets of features that may affect the prediction target. Therefore, the second feature set may be acquired by performing the feature selection process using the manipulated variable as the explained variable. Then, the output unit 93 may output the union set of the acquired first feature set and second feature set.

The feature selecting unit 92 selects the feature designated as the operation variable so as to be included in the first feature set. In this way, by always leaving the feature designated as the manipulated variable, the post-processing optimization processing becomes possible.

The feature selection system 90 may be implemented as, for example, package software or an API (Application Program Interface) having a feature selection function for learning a prediction model used for optimization based on prediction. Such a feature selection function may be provided in the SaaS (Software as a Service) format.

FIG. 7 is a schematic block diagram showing the configuration of a computer according to at least one embodiment. The computer 1000 includes a CPU 1001, a main storage device 1002, an auxiliary storage device 1003, and an interface 1004.

The information processing system described above is implemented in the computer 1000. The operation of each processing unit described above is stored in the auxiliary storage device 1003 in the form of a program (feature selection program). The CPU 1001 reads the program from the auxiliary storage device 1003, expands it in the main storage device 1002, and executes the above-described processing according to the program.

Note that in at least one embodiment, the auxiliary storage device 1003 is an example of a non-transitory tangible medium. Other examples of non-transitory tangible media include magnetic disks, magneto-optical disks, CD-ROMs, DVD-ROMs, semiconductor memories, etc., connected through the interface 1004. Further, when this program is distributed to the computer 1000 via a communication line, the computer 1000 that has received the distribution may expand the program in the main storage device 1002 and execute the above processing.

Further, the program may be for realizing some of the functions described above. Further, the program may be a so-called difference file (difference program) that realizes the function described above in combination with another program already stored in the auxiliary storage device 1003.

INDUSTRIAL APPLICABILITY The present invention is suitably applied to a feature selection system that selects features used for prediction. For example, the present invention is preferably applied to a system for selecting a feature used for optimizing a hotel price. Further, the present invention is preferably applied to, for example, a system that is connected to a database and outputs a result (optimal solution) optimized based on prediction. In this case, for example, it may be provided as a system for collectively performing the feature amount selection process and the optimization process based on the feature amount selection process.

10 reception unit 20 feature selection unit 30 learning unit 40 optimization unit 50 output unit 100 price optimization system

Claims (9)

A feature selection system for selecting a feature used for learning of a prediction model when calculating an operation variable for optimizing an objective function represented using a prediction model under a constraint condition,
A receiving unit that receives the designation of the prediction target and the designation of the manipulated variable;
Feature selection for selecting, from a set of features that can affect the prediction target, a first feature set that is a set of features that affect the prediction target and a second feature set that is a set of features that affect the operation variable Department,
An output unit that outputs the first feature set and the second feature set ,
The feature selecting system, wherein the feature selecting unit selects a feature designated as the manipulated variable so as to be included in the first feature set . The feature selection system according to claim 1, further comprising: a learning unit that learns a prediction model having at least one feature included in the second feature set but not included in the first feature set as an explanatory variable. The feature selection according to claim 2, wherein the learning unit learns a model in which all the features included in the first feature set and the features included in the second feature set are used as explanatory variables, and the feature that is the prediction target is used as the explained variable. system. The output unit displays on the display device a feature included in the first feature set and a feature included in the second feature set but not included in the first feature set in a distinguishable manner. To the feature selection system according to claim 3. The feature selection unit obtains a first feature set from the set of features that may affect the prediction target by performing feature selection processing using the prediction target as the explained variable, and operates from the set of features that may affect the prediction target. A second feature set is obtained by performing feature selection processing using the variable as the explained variable,
The feature selection system according to any one of claims 1 to 4, wherein the output unit outputs a set of the acquired first feature set and the acquired second feature set. A feature selection method for selecting a feature used for learning of the prediction model when calculating an operation variable for optimizing an objective function represented by using the prediction model under a constraint condition,
Accept the prediction target specification and the operation variable specification,
From the set of features that can affect the prediction target, select a first feature set that is a set of features that affect the prediction target and a second feature set that is a set of features that affect the manipulated variable,
At the time of the selection, a feature specified as the manipulated variable is selected to be included in the first feature set,
A feature selection method comprising outputting the first feature set and the second feature set. The feature selection method according to claim 6 , wherein a prediction model having at least one feature included in the second feature set but not included in the first feature set as an explanatory variable is learned. It is a feature selection program applied to a computer that selects features used for learning of a prediction model when calculating an operation variable for optimizing an objective function represented by the prediction model under constraint conditions. hand,
On the computer,
Acceptance process for accepting the designation of the prediction target and the designation of the manipulated variable,
Feature selection for selecting, from a set of features that can affect the prediction target, a first feature set that is a set of features that affect the prediction target and a second feature set that is a set of features that affect the operation variable Processing, and
Output processing for outputting the first feature set and the second feature set ,
A feature selection program for causing the feature selection process to select a feature specified as the manipulated variable to be included in the first feature set . On the computer,
The feature selection program according to claim 8, which executes a learning process of learning a prediction model having at least one feature included in the second feature set but not included in the first feature set as an explanatory variable.

Images