JP6193428B1 - Feature selection device, feature selection method, and program - Google Patents

Abstract

A feature selection device, a feature selection method, and a program capable of selecting a subset so as to achieve a target value set for each of a plurality of evaluation indexes as much as possible are provided. A feature selection device according to an embodiment includes a model evaluation unit that calculates an evaluation value of each of a plurality of evaluation indexes related to the performance of the model, and an evaluation for a set target value for each of the plurality of evaluation indexes. An evaluation value integration unit 8 that calculates a degree of achievement of a value and calculates an integrated evaluation value that is higher as the degree of achievement of each of the plurality of evaluation indices is higher and the variation in the degree of achievement among the plurality of evaluation indices is smaller. And searching for a subset having a high integrated evaluation value. [Selection] Figure 6

Description

  Embodiments described herein relate generally to a feature selection device, a feature selection method, and a program.

  There is a technique called feature selection that selects a subset of features useful for machine learning from an arbitrary feature set when building a model using a machine learning algorithm. There are several methods for feature selection, one of which is the Wrapper method. The Wrapper method is a method of searching for a subset having a high evaluation value by repeatedly generating and evaluating a model while changing the subset.

  There are various evaluation indexes for evaluating the performance of the model. For example, as an evaluation index of a prediction model for predicting the occurrence of an event, there are a “reproducibility” indicating the completeness of prediction and a “matching rate” indicating the accuracy of prediction. The recall rate and the matching rate are basically in a trade-off relationship, and it is difficult to select a subset in which both evaluation values have the maximum value. Therefore, the “F value” that is the harmonic average of the recall rate and the precision rate may be used as an evaluation index. When the model is evaluated using the F value, the evaluation value of the model having a good balance between the recall rate and the matching rate becomes high.

  However, depending on the use of the model, for example, there is a request to build a model that prioritizes the precision while maintaining a certain degree of recall, or conversely, a model that prioritizes the recall while securing a certain degree of precision. is there. In the conventional technique, when the target is different in each of the plurality of evaluation indexes as described above, the subset cannot be selected so as to achieve them as much as possible, and improvement is required.

JP-A-5-197813

  A problem to be solved by the present invention is to provide a feature selection device, a feature selection method, and a program capable of selecting a subset so as to achieve a target value set for each of a plurality of evaluation indexes as much as possible. It is.

  A feature selection device according to an embodiment is a feature selection device that repeatedly generates and evaluates a model using a subset of a feature set, and searches for the subset with a high evaluation value. A section. The model evaluation unit calculates an evaluation value of each of a plurality of evaluation indexes related to the performance of the model. The evaluation value integration unit calculates a degree of achievement of the evaluation value with respect to a set target value for each of the plurality of evaluation indices, the degree of achievement of each of the plurality of evaluation indices is high, and the plurality of the plurality of evaluation indices An integrated evaluation value is calculated such that the smaller the variation in the achievement level between evaluation indexes, the higher the evaluation. The feature selection device according to the embodiment searches for the subset in which the integrated evaluation value is high.

FIG. 1 is a diagram illustrating an example in which estimation using a model is performed at a constant period. FIG. 2 is a diagram for explaining an example of predicting data in a future period using time-series data. FIG. 3 is a diagram illustrating an example of a data set. FIG. 4 is a diagram illustrating an example of cost setting data. FIG. 5 is a diagram illustrating an example of the correction setting data. FIG. 6 is a block diagram illustrating a functional configuration example of the feature selection device according to the embodiment. FIG. 7 is a diagram illustrating an example of target value data. FIG. 8 is a diagram illustrating an example of cost constraint data. FIG. 9 is a diagram illustrating an example of the end condition data. FIG. 10 is a diagram illustrating an example of feature candidate data. FIG. 11 is a diagram illustrating an example of selected set data. FIG. 12 is a diagram illustrating an example of mode data. FIG. 13 is a diagram illustrating an example of the evaluation target set data. FIG. 14 is a diagram illustrating a relationship between the process of the cost evaluation unit and data. FIG. 15 is a diagram illustrating an example of cost data. FIG. 16 is a diagram illustrating an example of model data. FIG. 17 is a diagram illustrating an example of evaluation value data. FIG. 18 is a diagram illustrating a relationship between processing of the evaluation value integration unit and data. FIG. 19 is a diagram illustrating an example of the integrated evaluation value data. FIG. 20 is a diagram illustrating a relationship between processing of the evaluation value correction unit and data. FIG. 21 is a diagram illustrating an example of corrected evaluation value data. FIG. 22 is a diagram illustrating a relationship between the process of the exclusion loss evaluation unit and data. FIG. 23 is a diagram illustrating an example of final evaluation data. FIG. 24 is a diagram illustrating an example of provisional first-order data. FIG. 25 is a diagram illustrating an example of the best set data. FIG. 26A is a flowchart illustrating an example of a processing procedure of the feature selection device in the addition mode. FIG. 26B is a flowchart illustrating an example of a processing procedure of the feature selection device in the addition mode. FIG. 27A is a flowchart illustrating an example of a processing procedure of the feature selection device in the exclusion mode. FIG. 27-2 is a flowchart illustrating an example of a processing procedure of the feature selection device in the exclusion mode. FIG. 28 is a block diagram illustrating an example of a hardware configuration of the feature selection device.

  Hereinafter, a feature selection device, a feature selection method, and a program according to embodiments will be described in detail with reference to the drawings. The feature selection device according to the present embodiment is a feature selection device based on the Wrapper method that repeatedly generates and evaluates a model using a subset of an arbitrary feature set and searches for a subset having a high evaluation value.

<Outline of Embodiment>
First, an outline of the present embodiment will be described. A conventional feature selection apparatus based on the Wrapper method does not have a function of performing feature selection so as to achieve target values as much as possible in consideration of target values set for a plurality of evaluation indexes related to model performance. Therefore, there is a problem that it is difficult to obtain a model desired by the user.

  Therefore, in the present embodiment, an “integrated evaluation value” obtained by integrating the evaluation values of the plurality of evaluation indexes is calculated based on the degree of achievement of the evaluation values with respect to the target values of the plurality of evaluation indexes. Then, a subset with a higher integrated evaluation value is searched. The integrated evaluation value is an evaluation value that becomes higher as the achievement level of each of the plurality of evaluation indexes is higher and the variation in the achievement level among the plurality of evaluation indexes is smaller. Thereby, the subset can be selected so as to achieve the target value set for each of the plurality of evaluation indexes as much as possible, and a model desired by the user can be obtained.

  In addition, a conventional feature selection apparatus based on the Wrapper method does not have a function for performing feature selection in consideration of factors other than model performance (such as prediction accuracy and estimation accuracy). However, considering actual data analysis, there are cases in which factors other than performance are considered when evaluating a model.

  For example, as shown in FIG. 1, a case where prediction / estimation by a model is performed at a constant period is considered. In this case, it is necessary to complete feature extraction and model application from observation data for each prediction / estimation period. At this time, if the feature set to be used (subset selected by the feature selection device) contains a large amount of features that require a long time for extraction, prediction / estimation at a fixed period cannot be performed. Therefore, when performing feature selection, not only the performance of the model obtained from the subset to be evaluated, but also whether the processing time required to extract all the features in the subset falls within a predetermined time. It is required to check.

  Therefore, in this embodiment, the processing time required to extract all the features in the subset is defined as the cost of the subset, while satisfying the constraint that the cost of the subset is equal to or lower than a predetermined upper limit value. The feature selection is performed so that the integrated evaluation value of becomes higher. This makes it possible to obtain a high-performance model that is useful in actual data analysis.

  Further, for example, as shown in FIG. 2, consider a case where occurrence of an event in a future period is predicted using time-series data (hereinafter, this period is referred to as a “prediction target period”). When it is desired to take some countermeasures by the prediction target period according to the prediction result, it is desirable to obtain the prediction result as early as possible. Even if the accuracy of the prediction is high, the value of the prediction model decreases if the prediction result is obtained only immediately before the prediction target period. On the other hand, even if the prediction accuracy is a little worse, the prediction model may be more valuable if the prediction result is obtained early.

  As illustrated in FIG. 2, when features are extracted from windows placed at various positions in a prediction target period and are used as candidates for feature selection, a feature set to be used (a portion selected by the feature selection device) The time when the prediction result is obtained also depends on the set. Therefore, in the present embodiment, the time when the prediction result is obtained by the model obtained from the subset to be evaluated (the length of the period T in FIG. 2) is defined as the score of the subset, and the score of the subset is set. Based on the above, the integrated evaluation value is corrected. The correction is performed so that the evaluation value becomes higher even in the model having the same prediction accuracy as the period T in FIG. 2 is longer. This makes it possible to obtain a high-performance model that is useful in actual data analysis.

<Example>
Hereinafter, more specific examples of the present embodiment will be described. In this embodiment, for example, a greedy method for forward selection and a greedy method for backward selection are assumed as approaches for searching for a subset having a high evaluation value. The forward selection greedy method repeats the process of adding features to a selected feature set (hereinafter referred to as a “selected set”). The feature selection apparatus according to the present embodiment is based on the feature selection mode based on the forward selection greedy method, and is switched to the feature selection mode based on the backward selection greedy method that excludes the feature from the selected set as necessary. Switch to process. Hereinafter, the former mode is referred to as “addition mode”, and the latter mode is referred to as “exclusion mode”.

  Note that the embodiment described below is an example, and if the apparatus has the same function, there is no limitation on how to divide the function. In addition, the various data shown below may be expressed in any form as long as the same information as in the present embodiment is included.

  In this embodiment, feature selection / model generation is performed according to the following preconditions and settings. It should be noted that conditions and settings other than those shown here can be dealt with, and the conditions and settings can be switched in accordance with user input or the like during initial setting described later. For example, it may be possible to select an evaluation index to be used in accordance with a user input at the time of initial setting.

<Model to be generated>
In this embodiment, it is assumed that a model (prediction model) that predicts whether or not a failure of a certain device occurs by 2-class identification is generated. For example, a model is generated that estimates and classifies whether the given input data belongs to one of two classes, “a class in which a failure occurs” and “a class in which no failure occurs”. This model outputs data (hereinafter referred to as “label”) indicating whether or not a failure occurs in the prediction target period for the input feature value. Such a model can be generated by a machine learning algorithm such as Support Vector Machine, decision tree, and logistic regression.

<Learning data set and evaluation data set>
The learning data set is a data set used for model generation by machine learning, and the evaluation data set is a data set used for evaluation of the generated model. The learning data set and the evaluation data set are collections of samples, and both include similar samples. Hereinafter, when the learning data set and the evaluation data set are not distinguished, they are simply referred to as “data sets”.

  FIG. 3 is a diagram illustrating an example of the data set D1. In the data set D1 shown in FIG. 3, one row corresponds to one sample. Each sample includes a set of feature values that are candidates for selection and a correct prediction label (hereinafter referred to as “correct answer label”). A data set D1 illustrated in FIG. 3 illustrates an example in which the number of features that are selection candidates (the total number of features included in the feature set) is 300. The correct label of each sample represents “1” that a failure has occurred in the device and “−1” that a failure has not occurred in the device. Note that the samples included in the learning data set and the evaluation data set may be randomly replaced each time before model generation or may be determined using cross-validation. In this embodiment, for simplicity, it is assumed that the samples included in the learning data set and the evaluation data set are not changed.

<Termination condition for feature selection processing>
In this embodiment, when the number of features included in the selected set reaches the maximum number determined at the time of initial setting, which will be described later, or when the processing in the exclusion mode is executed an upper limit number of times determined at the time of initial setting. Shall be terminated. Note that this is an example, and other termination conditions may be used, such as termination when the achievement rate of the model reproduction rate and the matching rate by the selected set becomes a certain level or more.

<Evaluation index of model>
The performance (prediction accuracy of the prediction model) of the model generated using the subset to be evaluated (hereinafter referred to as “evaluation target set”) X is predicted for the evaluation data set using the model. The prediction result obtained is evaluated by calculating the recall represented by the following formula (1) and the precision represented by the following formula (2). In other words, out of the two classes of “classes where failure occurs” and “classes where failure does not occur”, focus on the “class where failure occurs” as the evaluation target, and change it to “class where failure occurs” In other words, whether the data to be classified is classified without being obtained is evaluated by the reproducibility, and the correctly classified ratio of the data classified as “the class in which the failure occurs” is evaluated by the relevance ratio. Here, the evaluation target class is only “a class in which a failure occurs”, but the recall ratio and the conformity ratio may be evaluated for a plurality of classes.

  In this embodiment, the user sets target values at the time of initial setting for each of the recall ratio and the matching ratio. Then, the recall value (evaluation value) and the precision value (evaluation value) obtained by the evaluation of the model are integrated in consideration of the respective target values to calculate an integrated evaluation value merged_eval (X). The evaluation index shown here is an example, and any type of evaluation index may be used as long as it is calculated as a numerical value and can set a target value. Three or more kinds of evaluation indexes may be used.

<Costs and constraints>
In this embodiment, it is assumed that cost feature_cost (x) is set for each feature x included in the feature set. The cost feature_cost (x) may be, for example, an average processing time required for extracting the feature quantity x. Assume that the cost cost (X) of the evaluation target set X is defined by the sum of the cost feature_cost (x) of each feature x included in the evaluation target set X, for example, as shown in the following equation (3).

In the present embodiment, a constraint condition that the cost cost (X) of the evaluation target set X is equal to or lower than the upper limit value max_cost is set as shown in the following formula (4). The upper limit value max_cost is set by the user at the time of initial setting.

  These settings are only examples, and the cost cost (X) of the evaluation target set X may be any definition as long as it is a numerical value that tends to increase with an increase in the number of elements of X. For example, if feature extraction is performed in parallel, a method of using the maximum value instead of the sum of the cost feature_cost (x) of each feature x included in the evaluation target set X may be considered.

  It is assumed that the setting related to the cost feature_cost (x) is defined before the start of processing by the feature selection device of this embodiment, for example, by cost setting data D2 as shown in FIG. This cost setting data D2 indicates the cost feature_cost (x) of each feature x included in the feature set. Note that the value of the cost feature_cost (x) of the feature x, the calculation method of the cost cost (X) of the evaluation target set X, and the like may be set in accordance with user input at the time of initial setting or the like.

<Score and correction method>
In this embodiment, it is assumed that a score feature_score (x) is set for each feature x included in the feature set. The score feature_score (x) includes, for example, the length of the grace period t for each feature shown in FIG. Assume that the score score (X) of the evaluation target set X is defined by the minimum value of the score feature_score (x) of each feature x included in the evaluation target set X, for example, as shown in the following formula (5).

ここで、関数ｆ（・）は、例えば下記式（７）に示すようなものが考えられる。なお、下記式（７）のα（score（Ｘ））は、score（Ｘ）が大きいほど大きい値となる係数である。本実施例では、score（Ｘ）の値に応じたα（score（Ｘ））の値が事前に設定されているものとするが（図５参照）、score（Ｘ）とα（score（Ｘ））の関係に関して、他の任意の関数を定義してもよい。

In this embodiment, after calculating the integrated evaluation value merged_eval (X), the integrated evaluation value merged_eval (X) is corrected according to the score score (X) of the evaluation target set X as shown in the following formula (6). Then, a corrected evaluation value corrected_eval (X) is obtained.

Here, as the function f (•), for example, a function as shown in the following formula (7) is conceivable. Note that α (score (X)) in the following formula (7) is a coefficient that increases as the score (X) increases. In the present embodiment, it is assumed that the value of α (score (X)) corresponding to the value of score (X) is set in advance (see FIG. 5), but score (X) and α (score (X Any other function may be defined with respect to the relationship of)).

  The correction method shown here is merely an example, and the correction method is such that the corrected evaluation value corrected_eval (X) becomes a better value as the score score (X) of the evaluation target set X becomes a better value. Any method may be used. For example, in the above equation (7), the integrated evaluation value merged_eval (X) is multiplied by a coefficient α (score (X)) corresponding to the score score (X) of the evaluation target set X. A method of adding the coefficient α (score (X)) corresponding to (X) is also conceivable. A more complicated function may be defined.

  The setting relating to the score feature_score (x) and the coefficient α (score (X)) corresponding to the score score (X) is set by, for example, correction setting data D3 as shown in FIG. As defined in The correction setting data D3 includes a table of the score feature_score (x) of each feature x included in the feature set and the coefficient α (score (X)) corresponding to the score score (X) of the evaluation target set X. It is shown. It should be noted that in accordance with user input at the time of initial setting or the like, the value of the score feature_score (x) of the feature x, the score score (X) of the evaluation target set X, and the corrected evaluation value corrected_eval (X) are calculated. You may enable it to set a method etc.

<Device configuration>
Next, a functional configuration of the feature selection device according to the present embodiment will be described. FIG. 6 is a block diagram illustrating a functional configuration example of the feature selection device 1 according to the present embodiment. As shown in FIG. 6, the feature selection device 1 of the present embodiment includes an input reception unit 2, an initial setting unit 3, an evaluation target set generation unit 4, a cost evaluation unit 5, a model generation unit 6, a model Evaluation unit 7, evaluation value integration unit 8, evaluation value correction unit 9, exclusion loss evaluation unit 10, provisional first update unit 11, selected set update unit 12, end determination unit 13, output unit 14.

  The input receiving unit 2 receives an input by a user for initial setting. Here, it is assumed that the user inputs the target value of the recall rate / matching rate, the upper limit value max_cost of the cost cost (X) of the evaluation target set X, the maximum number of features and the maximum number of exclusion mode executions in the end condition. . As for the upper limit value max_cost of the cost cost (X) of the evaluation target set X, a value that is less than the minimum value among the score feature_score (x) of each feature x is not accepted.

  The initial setting unit 3 reflects the input contents of the user received by the input receiving unit 2, and includes target value data D4, cost constraint data D5, end condition data D6, feature candidate data D7, selected set data D8, and mode data D9. Generate and initialize.

  FIG. 7 is a diagram illustrating an example of the target value data D4. The target value data D4 is data in which target values of the recall rate and the matching rate are recorded in accordance with user input. The target value data D4 illustrated in FIG. 7 indicates that the target value for the recall is set to 0.9 and the target value for the precision is set to 0.4.

  FIG. 8 is a diagram illustrating an example of the cost constraint data D5. The cost constraint data D5 is data in which the upper limit value max_cost of the cost cost (X) of the evaluation target set X is recorded in accordance with a user input. The cost constraint data D5 illustrated in FIG. 8 indicates that the upper limit value max_cost is set to 5.0.

  FIG. 9 is a diagram illustrating an example of the end condition data D6. The end condition data D6 is data in which the maximum number of features to be selected and the maximum number of executions of the exclusion mode are recorded in accordance with a user input. The end condition data D6 illustrated in FIG. 9 indicates that the maximum number of features to be selected is set to 50 and the maximum number of executions of the exclusion mode is set to 10.

  FIG. 10 is a diagram illustrating an example of the feature candidate data D7. The feature candidate data D7 is data in which all features that are candidates for selection (all features included in the feature set) and their states are recorded. FIG. 10 shows an example of the feature candidate data D7 when the number of features that are selection candidates (total number of features included in the feature set) is 300. The state of each feature takes one of “not evaluated”, “pending”, “evaluated”, “selected”, “violation”, and “excluded”. At the initial setting, the state of all the features in the feature candidate data D7 is set to “not evaluated”.

  FIG. 11 is a diagram illustrating an example of the selected set data D8. The selected set data D8 includes all features included in the selected set, a corrected evaluation value corrected_eval (X) calculated for the selected set, and a cost cost (X) calculated for the selected set. Is recorded data. At the initial setting, the selected set of the selected set data D8 is an empty set, and arbitrary values are set for the corrected evaluation value corrected_eval (X) and the cost cost (X).

  FIG. 12 is a diagram illustrating an example of the mode data D9. The mode data D9 is data that records whether the current mode is the addition mode or the exclusion mode. At the initial setting, the mode of the mode data D9 is set to the additional mode.

  The evaluation target set generation unit 4 refers to the mode data D9, the feature candidate data D7, and the selected set data D8, and adds or excludes one feature whose state is “unevaluated” to the selected set in the add mode. In the mode, an evaluation target set is generated by excluding one feature whose state indicated in the feature candidate data D7 is not “evaluated” from the selected set. Further, the state of the feature candidate data D7 is set to “hold” for the added feature in the addition mode and the excluded feature in the exclusion mode. Then, the evaluation target set generation unit 4 generates evaluation target set data D10 in which each feature included in the evaluation target set is recorded, for example, as shown in FIG. At this time, if the evaluation target set data D10 already exists, the data is deleted and new evaluation target set data D10 is generated.

  The cost evaluation unit 5 calculates the cost cost (X) of the evaluation target set generated by the evaluation target set generation unit 4 and determines whether the calculated cost cost (X) exceeds the upper limit value max_cost. FIG. 14 is a diagram illustrating a relationship between the process of the cost evaluation unit 5 and data. In the figure, Refer means data reference, Create means data generation, and Update means data update.

  First, the cost evaluation unit 5 refers to the evaluation target set data D10 and the cost setting data D2, and calculates the cost cost (X) of the evaluation target set generated by the evaluation target set generation unit 4 by the above formula (3). Next, the cost evaluation unit 5 refers to the cost constraint data D5, and determines whether or not the calculated cost cost (X) of the evaluation target set exceeds the upper limit value max_cost. As a result of this determination, if the cost cost (X) of the set to be evaluated does not exceed the upper limit value max_cost, the cost evaluation unit 5 refers to the feature candidate data D7 and selects the feature whose state is “pending”. Update the status to “evaluated”. Then, for example, as shown in FIG. 15, cost data D11 in which the cost cost (X) of the evaluation target set is recorded is generated. At this time, if the cost data D11 already exists, the data is deleted and new cost data D11 is generated.

  On the other hand, when the cost cost (X) of the set to be evaluated exceeds the upper limit value max_cost, the cost evaluation unit 5 refers to the feature candidate data D7 and changes the state of the feature whose state is “pending” to “violation”. Update. At this time, if all the feature states of the feature candidate data D7 are “selected”, “violation”, or “exclusion”, the cost evaluation unit 5 updates the mode data D9 to the exclusion mode.

  The model generation unit 6 refers to the evaluation target set data D10 and the learning data set (data set D1), and generates a model using only the features recorded in the evaluation target set data D10. And the model production | generation part 6 produces | generates the model data D12 showing the rule, parameter, etc. of the produced | generated model. At this time, if the model data D12 already exists, the data is deleted and new model data D12 is generated. FIG. 16 is a diagram illustrating an example of the model data D12. The model data D12 illustrated in FIG. 16 is an example of model data when a linear discriminator is used, and the weight w and the bias b for each feature are recorded.

  The model evaluation unit 7 refers to the evaluation target set data D10, the model data D12, and the evaluation data set (data set D1), and the model generation unit 6 generates only the features recorded in the evaluation target set data D10. The reproducibility of the model is calculated by the above equation (1), and the precision is calculated by the above equation (2). And the model evaluation part 7 produces | generates the evaluation value data D13 which recorded the calculated reproduction rate and the precision as an evaluation value. At this time, if the evaluation value data D13 already exists, the data is deleted and new evaluation value data D13 is generated. FIG. 17 is a diagram illustrating an example of the evaluation value data D13. The evaluation value data D13 illustrated in FIG. 17 indicates that the calculated evaluation value of the reproduction rate is 0.6 and the evaluation value of the matching rate is 0.3.

  The evaluation value integration unit 8 calculates a degree of achievement of the evaluation value calculated by the model evaluation unit 7 with respect to the target value for each of a plurality of evaluation indexes, ie, a reproduction rate and a relevance rate. An integrated evaluation value is calculated based on the degree of achievement. FIG. 18 is a diagram illustrating a relationship between the process of the evaluation value integration unit 8 and data. Refer in the figure means data reference, and Create means data generation.

なお、本実施例では、値が大きいほど高い評価となる再現率および適合率を評価指標として扱うため、目標値に対する評価値の達成度を上記のように定義できる。平均二乗誤差のように、値が小さいほど高い評価となる評価指標を扱う場合は、分子・分母を反転して（評価値を分母、目標値を分子とする）目標値に対する評価値の達成度を定義すればよい。 The evaluation value integration unit 8 first refers to the evaluation value data D13 and the target value data D4, achieves the achievement rate of reproduction rate (reproduction rate achievement rate) by the following equation (8), and achieves the precision rate by the following equation (9). Each degree (accomplishment rate achievement) is calculated.

In the present embodiment, the higher the value, the higher the reproduction rate and the matching rate, which are treated as evaluation indexes, so that the degree of achievement of the evaluation value with respect to the target value can be defined as described above. When dealing with an evaluation index that becomes higher as the value is smaller, such as the mean square error, the degree of achievement of the evaluation value for the target value by inverting the numerator and denominator (with the evaluation value as the denominator and the target value as the numerator) Should be defined.

上記式（１０）のα_ｒと上記式（１１）のα_ｐをともに１．０より小さい正の値（例えば０．１など）にすることで、変換後再現率達成度および変換後適合率達成度が１．０を大幅に超えないようにでき、上記の問題を防ぐことができる。これらα_ｒとα_ｐの値は予め定められているものとするが、初期設定時にユーザの入力に応じて設定してもよい。 Here, when the above formula (8) and the above formula (9) are used as they are, when one of the achievement rate of the recall rate and the achievement rate of the matching rate greatly exceeds 1.0, the achievement level of the other is very small. However, the integrated evaluation value may become a high value. Therefore, the reproduction rate achievement is converted as shown in the following equation (10), and the accuracy rate achievement is converted as shown in the following equation (11). It is desirable to calculate the integrated evaluation value by using the achievement level and the conversion rate achievement level after conversion.

By making α _{r in the} above formula (10) and α _p in the above formula (11) both positive values smaller than 1.0 (for example, 0.1, etc.), the post-conversion recall achievement rate and the post-conversion precision The achievement level can be made not to greatly exceed 1.0, and the above problem can be prevented. The values of these alpha _r and alpha _p are assumed to be predetermined, but may be set in accordance with at initialization to user input.

そして、評価値統合部８は、算出した統合評価値merged_eval（Ｘ）を記録した統合評価値データＤ１４を生成する。このとき、統合評価値データＤ１４がすでに存在する場合は、そのデータを削除して新規に統合評価値データＤ１４を生成する。図１９は、統合評価値データＤ１４の一例を示す図である。図１９に例示する統合評価値データＤ１４は、算出した統合評価値merged_eval（Ｘ）が０．７０６であったことを示している。 Next, the evaluation value integration unit 8 calculates the integrated evaluation value merged_eval (X) by the following equation (12) using the post-conversion recall rate achievement and the post-conversion precision achievement.

Then, the evaluation value integration unit 8 generates integrated evaluation value data D14 in which the calculated integrated evaluation value merged_eval (X) is recorded. At this time, if the integrated evaluation value data D14 already exists, the integrated evaluation value data D14 is newly generated by deleting the data. FIG. 19 is a diagram illustrating an example of the integrated evaluation value data D14. The integrated evaluation value data D14 illustrated in FIG. 19 indicates that the calculated integrated evaluation value merged_eval (X) is 0.706.

  The integrated evaluation value merged_eval (X) calculated by the above equation (12) is a harmonic average of the degree of achievement of the conversion rate after conversion and the degree of achievement of the conversion rate after conversion. That is, the integrated evaluation value merged_eval (X) takes a large value when both the post-conversion recall rate achievement level and the post-conversion precision rate achievement level are large and similar values. By selecting subsets so that this integrated evaluation value merged_eval (X) becomes high (takes a large value), the target values of the recall and precision rates can be achieved as much as possible, and the variation in achievement level is A small (balanced) model will be obtained.

  The evaluation value correction unit 9 calculates the score score (X) of the evaluation target set generated by the evaluation target set generation unit 4, and the evaluation value integration unit 8 so that the higher the score score (X), the higher the evaluation. The integrated evaluation value merged_eval (X) calculated by is corrected. FIG. 20 is a diagram illustrating a relationship between the process of the evaluation value correction unit 9 and data. Refer in the figure means data reference, and Create means data generation.

  First, the evaluation value correction unit 9 refers to the evaluation target set data D10 and the correction setting data D3, and calculates the score score (X) of the evaluation target set generated by the evaluation target set generation unit 4 by the above formula (5). . Next, the evaluation value correction unit 9 refers to the integrated evaluation value data D14 and the correction setting data D3, and converts the integrated evaluation value merged_eval (X) into the score score (X) by the above formulas (6) and (7). A corrected evaluation value corrected_eval (X) corrected accordingly is calculated. Then, the evaluation value correction unit 9 generates corrected evaluation value data D15 in which the calculated corrected evaluation value corrected_eval (X) is recorded. At this time, if the corrected evaluation value data D15 already exists, the data is deleted and new corrected evaluation value data D15 is generated. FIG. 21 is a diagram illustrating an example of the corrected evaluation value data D15. The corrected evaluation value data D15 illustrated in FIG. 21 indicates that the calculated corrected evaluation value corrected_eval (X) is 0.776.

  The exclusion loss evaluation unit 10 excludes an exclusion loss indicating a ratio between a decrease in cost cost (X) and a decrease in model performance when one feature is excluded from the selected set in the exclusion mode. To calculate. FIG. 22 is a diagram illustrating a relationship between the processing of the exclusion loss evaluation unit 10 and data. Refer in the figure means data reference, and Create means data generation.

The exclusion loss evaluation unit 10 refers to the mode data D9, the selected set data D8, the corrected evaluation value data D15, and the cost data D11, and calculates an exclusion loss in the exclusion mode. Specifically, the excluded loss evaluation unit 10 corrects the corrected evaluation value corrected_eval (X _s ) and the cost cost (X _s ) of the selected set X _s recorded in the selected set data D8, and the corrected evaluation value data. Using the corrected evaluation value corrected_eval (X) of the evaluation target set X recorded in D15 and the cost cost (X) recorded in the cost data D11, the exclusion loss is calculated by the following equation (13).

  Then, the excluded loss evaluation unit 10 generates final evaluation data D16 in which the corrected evaluation value corrected_eval (X) of the evaluation target set X, the cost cost (X), and the excluded loss calculated by the above equation (13) are recorded. FIG. 23 is a diagram illustrating an example of the final evaluation data D16. The final evaluation data D16 illustrated in FIG. 23 indicates that the calculated exclusion loss is 0.0021. In addition, since the value of the exclusion loss is not used in the addition mode, the final evaluation data D16 in which an arbitrary value is recorded as the exclusion loss may be generated.

  The temporary first update unit 11 refers to the mode data D9, the final evaluation data D16, and the temporary first data D17, and in the addition mode, the correction recorded in the final evaluation data D16 rather than the temporary first data D17. When the already evaluated value corrected_eval (X) is higher, the provisional first data D17 is updated when the exclusion loss recorded in the final evaluation data D16 is smaller than the provisional first data D17 in the exclusion mode. To do.

  FIG. 24 is a diagram illustrating an example of the provisional first place data D17. As shown in FIG. 24, the temporary first-rank data D17 includes an evaluation object set X whose provisional value is temporarily first, a corrected evaluation value corrected_eval (X) of the evaluation object set X, and a cost cost (X). And excluded losses are recorded. If the provisional first place data D17 does not exist, the provisional first place update unit 11 newly generates the provisional first place data D17, and has been corrected recorded in the evaluation target set X and the final evaluation data D16 at that time. The evaluation value corrected_eval (X), the cost cost (X), and the exclusion loss are recorded in the temporary first place data D17.

The selected set update unit 12 refers to the mode data D9, the selected set data D8, and the feature candidate data D7, and first checks whether there is a feature whose state is “unevaluated” in the add mode. . Then, when there is no feature whose state is “unevaluated”, the selected set update unit 12 sets the evaluation target set X, the corrected evaluation value corrected_eval (X), and the cost cost (X) recorded in the temporary first-rank data D17. ), The selected set data D8 is updated, and the provisional first data D17 is deleted. In addition, the selected set update unit 12 updates the state of each feature of the feature candidate data D7 as follows.
I. Features included in the selected set data D8: The state is set to “selected”.
II. Features with status “excluded”: Leave as is.
III. Other state characteristics: The state is set to “not evaluated”.

  Further, the selected set update unit 12 refers to the best set data D18, and the corrected evaluation value corrected_eval (X) of the updated selected set data D8 is corrected corrected evaluation value corrected_eval recorded in the best set data D18. If it is higher than (X), the best set data D18 is updated with the contents of the selected set data D8.

  FIG. 25 is a diagram illustrating an example of the best set data D18. As shown in FIG. 25, in the best set data D18, the best set that is the subset with the highest evaluation at that time, the corrected evaluation value corrected_eval (X) and the cost cost (X) of the best set are recorded. ing. When the best set data D18 does not exist, the selected set update unit 12 newly generates the best set data D18 and records the contents recorded in the selected set data D8 in the best set data D18.

  In the exclusion mode, the selected set update unit 12 first checks whether there is a feature whose state is not “evaluated” in the selected set Xs. When there is no feature whose state is not “evaluated”, the selected set update unit 12 refers to the selected set data D8 and the provisional first-order data D17, and exists in the selected set data D8 and the provisional first-order data. A feature that does not exist in D17 is specified. Then, the state in the feature candidate data D7 for the feature is set to “excluded”, and the temporary first-order data D17 is deleted.

  Thereafter, the selected set update unit 12 updates the selected set data D8 and the best set data D18 as in the addition mode. In addition, the selected set update unit 12 performs the above-described I.D. To III. The state of each feature in the feature candidate data D7 is updated according to the above.

End determining unit 13, mode data D9, with reference to the selected set data D8 and end condition data D6, if additional mode, the number of features included in the selected set X _s is recorded in the termination condition data D6 When the maximum number is reached, the output unit 14 is instructed to output the data recorded in the best set data D18, and the process of the feature selection device 1 is terminated.

  In the exclusion mode, the end determination unit 13 counts the number of executions of the exclusion mode, and the counted number of executions of the exclusion mode reaches the maximum number of executions of the exclusion mode recorded in the end condition data D6. Then, the output unit 14 is instructed to output the data recorded in the best set data D18, and the process of the feature selection device 1 is ended.

  The output unit 14 outputs the data recorded in the best set data D18 in accordance with the instruction from the end determination unit 13. Data output by the output unit 14 can be performed by any one or combination of display by a display device, data storage in an external storage device, and data transmission to an external device, for example. When data other than the data recorded in the best set data D18 is output, processing may be added so as to hold them separately.

<Description of operation>
Next, the operation of the feature selection device 1 of this embodiment will be described. The feature selection apparatus 1 according to the present embodiment first performs various initial settings in accordance with a user input, and then adds an addition mode for adding features to the selected set, and an exclusion mode for excluding features from the selected set. Processing is performed while switching. Basically, it operates in the addition mode, but when it becomes impossible to satisfy the constraint condition shown in the above equation (4), the mode shifts to the exclusion mode. When a certain number of features (one in this embodiment) is excluded from the selected set, the process proceeds to the additional mode. Hereinafter, an example of each processing procedure will be described according to a flowchart, divided into processing in the addition mode and processing in the exclusion mode.

  First, a processing procedure in the addition mode will be described. FIGS. 26A and 26B are flowcharts illustrating an example of a processing procedure of the feature selection device 1 in the addition mode. The addition mode is a mode that is a basis of the feature selection device 1 of the present embodiment, in which a loop for adding features to a selected set is repeated.

  Step S101: When the processing in the addition mode is started, first, the evaluation target set generation unit 4 confirms the mode data D9. If the current mode indicated by the mode data D9 is the addition mode, the process proceeds to the next step S102. If the current mode is the exclusion mode, the process proceeds to the exclusion mode described later.

  Step S102: The evaluation target set generation unit 4 refers to the feature candidate data D7 and confirms whether or not there is a feature whose state is “not evaluated”. If there is a feature “not evaluated” (step S102: Yes), the process proceeds to the next step S103. If there is no feature “not evaluated” (step S102: No), the process proceeds to step S115.

  Step S103: The evaluation target set generation unit 4 refers to the selected set data D8 and the feature candidate data D7, and among the features included in the feature candidate data D7, the feature whose state is “unevaluated” is set to 1 in the selected set. The evaluation target set X added is generated. Then, the evaluation target set generation unit 4 generates evaluation target set data D10 in which the features included in the evaluation target set X are recorded, and updates the state of the added features in the feature candidate data D7 to “pending”. Then, the process proceeds to the next step S104.

  Step S104: The cost evaluation unit 5 refers to the evaluation target set data D10 and the cost setting data D2, calculates the cost cost (X) of the evaluation target set X generated in step S103 by the above formula (3), Proceed to the next Step S105.

  Step S105: The cost evaluation unit 5 refers to the cost constraint data D5 and determines whether or not the cost cost (X) calculated in step S104 satisfies the constraint condition according to the above equation (4). If the cost cost (X) calculated in step S104 satisfies the constraint condition (step S105: Yes), the cost evaluation unit 5 generates cost data D11 in which the cost cost (X) calculated in step S104 is recorded. Then, the state of the feature whose state is “pending” in step S103 is updated to “evaluated”, and the process proceeds to the next step S106. On the other hand, when the cost cost (X) calculated in step S104 does not satisfy the constraint condition (step S105: No), the state of the feature whose state is “pending” in step S103 is updated to “violation”. The process proceeds to step S113.

  Step S106: The model generation unit 6 refers to the evaluation target set data D10 and the learning data set (data set D1), and uses only the features included in the evaluation target set data D10 of each sample included in the learning data set. A model is generated by learning. And the model production | generation part 6 produces | generates the model data D12 showing the rule, parameter, etc. of the produced | generated model, and progresses to the following step S107.

  Step S107: The model evaluation unit 7 refers to the evaluation target set data D10, the model data D12, and the evaluation data set (data set D1), and is included in the evaluation target set data D10 of each sample included in the evaluation data set. Model evaluation using only features is performed, and the evaluation value (reproducibility / matching rate) of the model generated in step S106 is calculated by the above formulas (1) and (2). And the model evaluation part 7 produces | generates the evaluation value data D13 which recorded the calculated reproduction rate and the precision, and progresses to the following step S108.

  Step S108: The evaluation value integration unit 8 refers to the evaluation value data D13 and the target value data D4, and calculates the achievement rate and the achievement rate of the relevance rate by the above formulas (8) and (9). Further, the evaluation value integration unit 8 calculates the post-conversion recall rate achievement and the post-conversion precision achievement by the above formulas (10) and (11), and the integrated evaluation value merged_eval ( X) is calculated. Then, the evaluation value integration unit 8 generates integrated evaluation value data D14 in which the calculated integrated evaluation value merged_eval (X) is recorded, and the process proceeds to the next step S109.

  Step S109: The evaluation value correction unit 9 refers to the evaluation target set data D10 and the correction setting data D3, and calculates the score score (X) of the evaluation target set X generated in step S103 by the above equation (5). . Further, the evaluation value correction unit 9 refers to the integrated evaluation value data D14 and the correction setting data D3, and based on the integrated evaluation values merged_eval (X) and α (score (X)), the above equation (6) and the equation From (7), a corrected evaluation value corrected_eval (X) is calculated. Then, the evaluation value correction unit 9 generates corrected evaluation value data D15 in which the calculated corrected evaluation value corrected_eval (X) is recorded, and proceeds to the next step S110.

  Step S110: The excluded loss evaluation unit 10 refers to the corrected evaluation value data D15 and the cost data D11, and generates final evaluation data D16 in which the corrected evaluation value corrected_eval (X), the cost cost (X), and the exclusion loss are recorded. Then, the process proceeds to the next step S111. In addition, in the addition mode, an arbitrary value is recorded as the exclusion loss.

  Step S111: The provisional first place update unit 11 refers to the final evaluation data D16 and the provisional first place data D17, and the final evaluation data D16 has a corrected evaluation value corrected_eval (X) higher than the provisional first place data D17. It is determined whether or not. If the corrected evaluation value corrected_eval (X) recorded in the final evaluation data D16 is higher than the temporary first-order data D17 (step S111: Yes), the process proceeds to the next step S112, and is recorded in the final evaluation data D16. If the corrected evaluation value corrected_eval (X) is lower than the temporary first-order data D17 (step S111: No), the process returns to step S101 and the subsequent processing is repeated.

  Step S112: The temporary first-rank update unit 11 evaluates the evaluation target set X recorded in the evaluation target set data D10, the corrected evaluation value corrected_eval (X), the cost cost (X), and the exclusion recorded in the final evaluation data D16. The provisional first place data D17 is updated according to the loss. And it returns to step S101 and repeats the subsequent processes.

  Step S113: If the cost cost (X) calculated in step S104 does not satisfy the constraint condition (No in step S105), the cost evaluation unit 5 checks whether or not an evaluation target set that satisfies the constraint condition can be generated anymore. That is, the cost evaluation unit 5 cannot generate an evaluation target set that satisfies the constraint condition when all of the feature states included in the feature candidate data D7 are “selected”, “violation”, or “exclusion”. (Step S113: Yes), the process proceeds to the next step S114. On the other hand, if there is a possibility that an evaluation target set that satisfies the constraint conditions can be generated (step S105: No), the process returns to step S101 and the subsequent processing is repeated.

  Step S114: When the cost evaluation unit 5 determines that the evaluation target set that satisfies the constraint condition cannot be generated (step S105: Yes), the current mode of the mode data D9 is updated from the addition mode to the exclusion mode, and the process proceeds to step S101. Return and repeat the subsequent processing.

  Step S115: When it is determined in step S102 that the feature candidate data D7 does not have a feature whose state is “not evaluated” (step S102: No), the selected set update unit 12 is recorded in the temporary first-rank data D17. The selected set data D8 is updated with the evaluation target set (X), the corrected evaluation value corrected_eval (X), and the cost cost (X). In addition, the selected set update unit 12 deletes the temporary first rank data D17, updates the state of each feature of the feature candidate data D7 as described above, and proceeds to the next step S116.

  Step S116: The selected set update unit 12 refers to the best set data D18, and whether the updated set value D8 after the update has a corrected evaluation value corrected_eval (X) higher than that of the best set data D18. Determine whether or not. If the corrected evaluation value corrected_eval (X) recorded in the updated selected set data D8 is higher than the best set data D18 (step S116: Yes), the process proceeds to the next step S117, and the updated selected value is selected. If the corrected evaluation value corrected_eval (X) recorded in the set data D8 is equal to or less than the best set data D18 (step S116: No), the process proceeds to step S118.

  Step S117: The selected set update unit 12 updates the best set data D18 with the selected feature, the corrected evaluation value corrected_eval (X) and the cost cost (X) recorded in the updated selected set data D8. The process proceeds to the next step S118.

Step S118: The end determination unit 13 refers to the selected set data D8 and the end condition data D6, and whether the number of features included in the selected set X _s reaches the maximum number recorded in the end condition data D6, That is, it is determined whether or not the end condition is satisfied. If the end condition is satisfied (step S118: Yes), the process proceeds to the next step S119. If the end condition is not satisfied (step S118: No), the process returns to step S101 and the subsequent processing is repeated.

  Step S119: The output unit 14 outputs the data recorded in the best set data D18, and a series of processes by the feature selection device 1 of this embodiment is completed.

  Next, a processing procedure in the exclusion mode will be described. FIGS. 27A and 27B are flowcharts illustrating an example of a processing procedure of the feature selection device 1 in the exclusion mode. The exclusion mode is a mode that is executed when a cost constraint cannot be satisfied no matter which feature amount is added in the addition mode. In the exclusion mode, one feature is excluded from the selected feature set. In this embodiment, the mode returns to the addition mode when one feature amount is excluded, but the exclusion mode may be continued until a plurality of features are excluded. Below, it demonstrates focusing on the part from which a behavior differs from an addition mode.

  Step S201: When processing in the exclusion mode is started, first, the evaluation target set generation unit 4 confirms the mode data D9. If the current mode indicated by the mode data D9 is the exclusion mode, the process proceeds to the next step S202, and if it is the addition mode, the process proceeds to the above-described addition mode.

  Step S202: The evaluation target set generation unit 4 refers to the feature candidate data D7 and confirms whether or not there is a feature whose state is “selected”. If there is a “selected” feature (step S202: Yes), the process proceeds to the next step S203. If there is no “selected” feature (step S202: No), the process proceeds to step S211.

  Step S203: The evaluation target set generation unit 4 selects one feature whose state is “selected” among the features included in the feature candidate data D7, and updates the state of the feature to “pending”. Then, a set including only features whose states are “selected” or “evaluated” is generated. This process corresponds to generating the evaluation target set X by excluding the feature whose state is “pending” from the selected set. And the evaluation object set production | generation part 4 produces | generates evaluation object set data D10 which recorded this. Thereafter, the cost evaluation unit 5 updates the features included in the feature candidate data D7 with the status “pending” to “evaluated”. Then, the process proceeds to the next step S204.

  Step S204: The model generation unit 6 generates a model in the same manner as in step S106 in the addition mode, generates model data D12 representing rules and parameters of the generated model, and proceeds to the next step 205.

  Step S205: The model evaluation unit 7 calculates the evaluation value (reproduction rate / relevance rate) of the model similarly to step S107 in the addition mode, and generates evaluation value data D13 in which the calculated reproducibility / relevance rate is recorded. The process proceeds to the next step S206.

  Step S206: The evaluation value integration unit 8 calculates the integrated evaluation value merged_eval (X) as in step S108 in the addition mode, and generates integrated evaluation value data D14 in which the calculated integrated evaluation value merged_eval (X) is recorded. The process proceeds to the next step S207.

  Step S207: The evaluation value correction unit 9 calculates the corrected evaluation value corrected_eval (X) in the same manner as in step S109 in the addition mode, and the corrected evaluation value data D15 in which the calculated corrected evaluation value corrected_eval (X) is recorded. And proceed to the next step S208.

Step S208: Exclude loss evaluation unit 10, selected aggregate data D8, with reference to the corrected evaluation value data D15 and cost data D11, corrected evaluation value corrected_eval of the recorded selected set X _s to the selected set data D8 (X _s ) and cost cost (X _s ), corrected evaluation value corrected_eval (X) of the evaluation target set X recorded in the corrected evaluation value data D 15, and cost cost (X) recorded in the cost data D 11 And the exclusion loss is calculated by the above equation (13). Then, the excluded loss evaluation unit 10 generates final evaluation data D16 in which the corrected evaluation value corrected_eval (X) and cost cost (X) of the evaluation target set X and the calculated excluded loss are recorded, and the next step S209 is performed. Proceed to

  Step S209: The provisional first update unit 11 refers to the final evaluation data D16 and the provisional first data D17, and determines whether or not the final evaluation data D16 has a smaller exclusion loss than the provisional first data D17. If the exclusion loss recorded in the final evaluation data D16 is smaller than the provisional first-order data D17 (step S209: Yes), the process proceeds to the next step S210, and the exclusion loss recorded in the final evaluation data D16 is provisional first-order. If it is larger than the data D17 (step S209: No), the process returns to step S201 and the subsequent processing is repeated.

  Step S210: The provisional first-order update unit 11 evaluates the evaluation target set X recorded in the evaluation target set data D10, the corrected evaluation value corrected_eval (X), the cost cost (X), and the exclusion recorded in the final evaluation data D16. The provisional first place data D17 is updated according to the loss. And it returns to step S201 and repeats the subsequent processes.

  Step S211: When it is determined in step S202 that the feature candidate data D7 does not have a feature whose state is “selected” (step S202: No), the selected set update unit 12 tentatively ranks with the selected set data D8. With reference to the data D17, a feature that exists in the selected set data D8 but does not exist in the temporary first-order data D17 is specified, the state of the feature in the feature candidate data D7 is set to “excluded”, and the temporary first-order data D17 Is deleted. Then, the selected set update unit 12 updates the selected set data D8 in the same manner as in step S115 in the addition mode, updates each state of the feature candidate data D7, and proceeds to the next step S212.

  Step S212: As in step S116 in the addition mode, the selected set update unit 12 has the corrected evaluation value corrected_eval (X) higher in the updated selected set data D8 than in the best set data D18. It is determined whether or not. If the corrected evaluation value corrected_eval (X) recorded in the updated selected set data D8 is higher than the best set data D18 (step S212: Yes), the process proceeds to the next step S213, and the updated selected value is selected. If the corrected evaluation value corrected_eval (X) recorded in the set data D8 is equal to or less than the best set data D18 (step S212: No), the process proceeds to step S214.

  Step S213: As with step S117 in the add mode, the selected set update unit 12 selects the selected feature, the corrected evaluation value corrected_eval (X), and the cost cost (recorded in the updated set data D8 after update. X) updates the best set data D18, and proceeds to the next step S214.

  Step S214: The end determination unit 13 refers to the end condition data D6 and determines whether the number of executions of the exclusion mode has reached the maximum number recorded in the end condition data D6, that is, whether the end condition is satisfied. If the end condition is not satisfied (step S214: No), the process proceeds to the next step S215. If the end condition is satisfied (step S214: Yes), the process proceeds to step S216.

  Step S215: The end determination unit 13 updates the current mode of the mode data D9 from the exclusion mode to the addition mode, returns to step S201, and repeats the subsequent processing.

  Step S216: The output unit 14 outputs the data recorded in the best set data D18, and a series of processes by the feature selection device 1 of this embodiment is completed.

<Hardware configuration>
The function of the feature selection device 1 described above can be realized by, for example, cooperation between general computer hardware and software (program). An example of the hardware configuration of the feature selection device 1 in this case is shown in FIG.

  For example, as shown in FIG. 28, the feature selection device 1 of this embodiment includes a CPU (Central Processing Unit) 101 that performs information processing, a ROM (Read Only Memory) 102 that is a read-only memory that stores BIOS, and various data. RAM (Random Access Memory) 103 that stores data in a rewritable manner, HDD (Hard Disk Drive) 104 that functions as various databases and stores various programs, and uses a storage medium 110 to store information and distribute information to the outside The medium driving device 105 for obtaining information from the outside or the user, the input device 106 such as a keyboard and mouse for the user to input commands and information to the CPU 101, and the LCD for displaying the processing progress and results to the user (Liquid Cristal Display) and other display devices 107, etc. are transmitted and received between these parts The bus controller 108 arbitrates the data and operates.

  In such a feature selection device 1, when the user turns on the power, the CPU 101 activates a program called a loader in the ROM 102, and loads a program for managing the hardware and software of an OS (Operation System) from the HDD 104 into the RAM 103. This OS is started. Such an OS activates a program, reads data, or stores data in accordance with a user operation. As typical OS, Windows (registered trademark), UNIX (registered trademark), and the like are known. Programs that run on these OSs are called application programs. The application program is not limited to one that runs on a predetermined OS, and may be one that causes the OS to execute some of the various processes described below, or constitutes predetermined application software, an OS, or the like. It may be included as part of a group of program files.

  The feature selection device 1 stores, in the HDD 104, a program for generating the functional components shown in FIG. Application programs installed in the HDD 104 of the feature selection device 1 are generally of various types such as various optical disks such as CD-ROM and DVD, various magnetic disks such as various magneto-optical disks and flexible disks, and semiconductor memories. The program is provided by being recorded on a storage medium 110 such as a medium. Further, this program may be imported from the outside by communication using a network, for example, and installed in the HDD 104.

  When the hardware configuration as described above is adopted, the CPU 101 executes various arithmetic processes in accordance with the above-described program operating on the OS, thereby generating the functional components shown in FIG. The computer can function as the feature selection device 1. Note that some or all of the functional components shown in FIG. 6 may be realized by using dedicated hardware such as an application specific integrated circuit (ASIC) or a field-programmable gate array (FPGA). it can.

  In this embodiment, it is assumed that the feature selection device 1 is configured as a single device. However, the feature selection device 1 does not have to be configured as a single device, and is physically separated from the network. It may be configured by a plurality of devices connected via the. Further, the feature selection device 1 may be realized as a virtual machine that operates on the cloud system.

<Effect of embodiment>
As described above in detail with reference to specific examples, the feature selection device according to the present embodiment indicates the degree of achievement of the evaluation value with respect to the set target value for each of the plurality of evaluation indexes related to the performance of the model. calculate. Then, an integrated evaluation value that is highly evaluated as the achievement level for each evaluation index is high and the variation in the achievement level between evaluation indexes is small, and a subset in which the integrated evaluation value is high is searched. This makes it possible to generate a model that satisfies the requirements desired by the user in consideration of a plurality of evaluation indexes and target values.

  In addition, the feature selection device according to the present embodiment determines whether or not the constraint that the cost of the evaluation target set is equal to or lower than the upper limit value is satisfied. When an evaluation target set that satisfies the constraint conditions cannot be generated, the mode shifts to an exclusion mode in which features are excluded from the selected set. At this time, an exclusion loss is calculated that indicates the ratio between the reduction in cost when the feature is excluded and the reduction in the corrected evaluation value. By using this exclusion loss, it is possible to select features to be excluded from the selected set so that the cost can be significantly reduced without significantly reducing the performance of the model.

  As described above, in this embodiment, the cost constraint is checked, and if the constraint is not satisfied, the selected set can be updated so as to reduce the cost without reducing the performance of the model as much as possible. As a result, for example, feature selection considering factors other than model performance, such as a restriction on processing time required for feature extraction, can be performed.

  In addition, the feature selection device of the present embodiment corrects the integrated evaluation value according to the score defined for the evaluation target set, and calculates the corrected evaluation value. Then, a subset with a higher corrected evaluation value is searched. Thereby, for example, it is possible to select a feature considering factors other than the performance of the model, for example, a feature set that can be predicted earlier than the prediction target time can be set to be highly evaluated.

<Supplementary explanation>
In the above-described embodiment, it is assumed that a prediction model for predicting whether or not a device failure occurs is generated, and the example in which the recall rate and the matching rate are used as the evaluation indexes related to the model performance has been described. However, as an example of a model to be generated, there is a model (identification model) that classifies data into a plurality of classes. When generating an identification model, the recall used as an evaluation index for the model is that a class to be evaluated is set among multiple classes, and for each class to be evaluated, data that should be classified into that class leaks. Indicates whether it is classified. The relevance ratio indicates the proportion of data classified correctly among the data classified into the evaluation target class. The evaluation target may be one class. Alternatively, a plurality of classes may be evaluated at a time. Further, the evaluation index of the model is not limited to the reproduction rate and the matching rate, and various evaluation indexes can be used. Further, the number of evaluation indexes to be used is not limited to two, and may be three or more.

  For example, when a plurality of outputs are obtained with one model, the accuracy rate required for each output may be different. Specifically, for example, when considering a model for estimating gender and age from a face image, it may be required to estimate the gender strictly to some extent and the age roughly. In such a case, the accuracy rate of each output can be handled as a plurality of evaluation indexes. That is, a target value may be set for each output accuracy rate, and the integrated evaluation index may be calculated based on the degree of achievement of the target value.

  In the above-described embodiment, the cost of the evaluation target set is defined based on the processing time for extracting the features included in the evaluation target set. However, if not only the processing time for extracting features included in the evaluation target set but also the model application processing time fluctuates significantly, the cost of the evaluation target set should be defined in consideration of the model application processing time. Also good.

  In the above-described embodiment, the constraint condition that the cost of the evaluation target set is equal to or lower than the upper limit value is imposed. However, this constraint condition can be invalidated. In order to invalidate this restriction condition, for example, the upper limit value max_cost of the cost set at the initial setting may be set to a very large value. When the constraint condition is invalidated, the transition from the addition mode to the exclusion mode does not occur.

  In the above-described embodiment, the integrated evaluation value is corrected according to the score of the set to be evaluated. However, such a correction may not be performed. For example, if the scores of all the features in the correction setting data are set to the same value, the correction of the integrated evaluation value according to the score of the evaluation target set is invalidated.

  As mentioned above, although embodiment of this invention was described, embodiment described here is shown as an example and is not intending limiting the range of invention. The novel embodiments described herein can be implemented in various other forms, and various omissions, replacements, and changes can be made without departing from the spirit of the invention. The embodiments and modifications described herein are included in the scope and gist of the invention, and are included in the invention described in the claims and the equivalents thereof.

DESCRIPTION OF SYMBOLS 1 Feature selection apparatus 2 Input reception part 3 Initial setting part 4 Evaluation object set production | generation part 5 Cost evaluation part 6 Model production | generation part 7 Model evaluation part 8 Evaluation value integration part 9 Evaluation value correction part 10 Exclusion loss evaluation part 11 Temporary first place update Unit 12 selected set update unit 13 end determination unit 14 output unit

Claims (8)

A feature selection device that repeats generation and evaluation of a model using a subset of a feature set and searches for the subset with a high evaluation value,
A model evaluation unit that calculates an evaluation value of each of a plurality of evaluation indexes related to the performance of the model;
For each of the plurality of evaluation indexes, the degree of achievement of the evaluation value with respect to a set target value is calculated, the degree of achievement of each of the plurality of evaluation indexes is high, and the achievement among the plurality of evaluation indexes An evaluation value integration unit that calculates an integrated evaluation value that becomes higher as the degree of variation is smaller,
A feature selection device that searches for the subset in which the integrated evaluation value is high. The model is an identification model that classifies data into a plurality of classes,
The plurality of evaluation indicators are subject to evaluation of one or more of the plurality of classes, and for each of the evaluation target classes, a recall indicating whether or not all data to be classified into the class is classified, 2. The feature selection device according to claim 1, further comprising: a matching ratio indicating a correctly classified ratio of the data classified into the evaluation target class. The model is a prediction model for predicting the occurrence of an event,
The feature selection apparatus according to claim 1, wherein the plurality of evaluation indexes include a recall rate indicating the completeness of prediction and a matching rate indicating the accuracy of prediction. Each feature included in the feature set is given a first cost that increases as the time required for feature extraction increases.
A cost evaluation unit that calculates a second cost that is a cost of the subset based on the first cost of each feature included in the subset;
4. The feature according to claim 1, wherein the subset that satisfies the constraint condition that the second cost is equal to or lower than a set upper limit value and that has a higher integrated evaluation value is searched for. Selection device. An exclusion loss evaluation unit that calculates, for each feature to be excluded, an exclusion loss indicating a ratio between a decrease in the second cost and a decrease in the integrated evaluation value when one feature is excluded from the subset;
The feature selection device according to claim 4, wherein when the second cost exceeds the upper limit value, a feature to be excluded from the subset is selected based on the exclusion loss. The model is a prediction model for predicting the occurrence of an event,
A score corresponding to the time when the prediction result by the model is obtained for the subset is calculated, and an evaluation value correction unit that corrects the integrated evaluation value based on the calculated score is further provided.
The feature selection device according to claim 1, wherein the subset in which the corrected integrated evaluation value is increased is searched. A feature selection method for repeatedly searching for a subset having a high evaluation value by repeatedly generating and evaluating a model using a subset of the feature set,
Calculating an evaluation value of each of a plurality of evaluation indexes related to the performance of the model;
For each of the plurality of evaluation indexes, the degree of achievement of the evaluation value with respect to a set target value is calculated, the degree of achievement of each of the plurality of evaluation indexes is high, and the achievement among the plurality of evaluation indexes A step of calculating an integrated evaluation value that is higher as the degree of variation is smaller,
A feature selection method for searching for the subset in which the integrated evaluation value is high. A program for causing a computer to function as a feature selection device that repeatedly generates and evaluates a model using a subset of a feature set and searches for the subset with a high evaluation value,
In the computer,
A function of calculating an evaluation value of each of a plurality of evaluation indexes related to the performance of the model;
For each of the plurality of evaluation indexes, the degree of achievement of the evaluation value with respect to a set target value is calculated, the degree of achievement of each of the plurality of evaluation indexes is high, and the achievement among the plurality of evaluation indexes And a function to calculate an integrated evaluation value that becomes higher as the degree of variation is smaller,
A program for searching for the subset in which the integrated evaluation value increases.

Images