WO2020044814A1

WO2020044814A1 - Model updating device, model updating method, and model updating program

Info

Publication number: WO2020044814A1
Application number: PCT/JP2019/027687
Authority: WO
Inventors: 智之西山; 江藤　力
Original assignee: 日本電気株式会社
Priority date: 2018-08-27
Filing date: 2019-07-12
Publication date: 2020-03-05

Abstract

A data extraction unit 81 extracts data classified under a target condition in a hierarchical mixed model. A data supplementation unit 82 accepts supplementation of the extracted data. A model generation unit 83 uses the supplemented data and generates a discriminant model. A model updating unit 84 generates a model having a joint node arranged at the top of the hierarchical mixed model, said joint node classifying data fulfilling the target condition. The model updating unit 84 also: applies data not satisfying the condition, to the hierarchical mixed model corresponding to a leaf node for the joint node; and generates a model indicating that data satisfying the condition is applied to a discriminant model corresponding to a leaf node for the joint node.

Description

Model updating device, model updating method, and model updating program

<< The present invention relates to a model updating device, a model updating method, and a model updating program for updating a discrimination model.

(4) In order to process data efficiently, the task of determining target data using a determination model is generally performed. Although there are various types of discriminating models, a discriminating condition is clear, and a model with high interpretability can be said to be a model that can be easily operated.

階層 A hierarchical mixed model is known as a model whose determination conditions are clear and easy to interpret. The hierarchical mixed model is a model having a tree structure in which branch conditions (sometimes referred to as discriminant conditions and gate tree conditions) are set for root nodes and node nodes, and discriminants are set for leaf nodes. For example, in a discrimination problem with a clear rule in the background, the conditions for discriminating the target data are generally clear, so that the data can be discriminated only by the branch condition indicated by the node.

FIG. 28 is an explanatory diagram showing an example of a discrimination model for performing a binary decision. The discrimination model shown in FIG. 28 has a tree structure, and is a model for discriminating target data at leaf nodes. For example, when performing a binary judgment (for example, true / false judgment) on input information in a certain task, it is necessary to return the same result (object variable) for the same input information. For example, in the business of assessing approval or rejection represented by credit or the like, it is desired to determine the same result (acceptance of financing) for information on customers having the same conditions.

対象 If all the conditions for performing the binary judgment on the input information are known, the target data can be completely determined. For example, if the conditions 1 to n shown in FIG. 28 are all the conditions used for discrimination, the discrimination model shown in FIG. 28 can be said to be a model that can completely discriminate the target data.

However, usually, it is rare that all the conditions used for discrimination are known, and other information is required to perform a complete discrimination. FIG. 29 is an explanatory diagram showing another example of the discrimination model for performing the binary determination. The model shown in FIG. 29 is also a model that makes a determination based on the conditions 1 to n similarly to the model shown in FIG. By using the

conditions

1 and 3 illustrated in FIG. 29, it is possible to completely separate the input information by the determination 1. On the other hand, if a condition based on information that is not in the teacher data is hidden under the condition n, the result will be different even if the data satisfies the same condition n. Information that is not in the teacher data is generally determined from knowledge and other information.

再 In order to perform prediction by the processing of determination 2 shown in FIG. 29, it is necessary to perform re-learning based on learning data including “information not in teacher data”. However, in practice, the learning data often does not include this "information not in the teacher data", that is, other explanatory variables. In other words, if it is known that “information that is not in the teacher data” is required for prediction, complete prediction cannot be performed only with the conditions 1 to n.

On the other hand, a method of determining target information using a score is also known. Based on the assumption that the analysis result is normally distributed with respect to the population (that is, the law of large numbers), as a method of discriminating input information into binary values, for example, discriminating based on the degree of contribution to the classification value (score) And a method in which an occurrence probability (score) is assigned to each classification based on a reference value to make a determination. As the occurrence probability, for example, logit (logarithm of odds (= P / (1−P))) is used.

Patent Document 1 describes a system state determination support device that determines the state of a system. The apparatus described in Patent Literature 1 discloses a method of generating a discrimination model for discriminating whether a system is in a predetermined state, the reliability of the monitoring information of the system (model reliability), and the monitoring information of the monitoring target to be discriminated by the system. Calculate the reliability (target reliability). Then, based on the model reliability and the target reliability, the threshold used by the determination model for determination is corrected.

FIG. 30 is an explanatory diagram showing experimental results of analysis by binary discrimination. For example, it is assumed that the score of the above-described discriminant is output as a discrimination result. Before the analysis, the positive and negative examples were assumed to have the distribution shown in FIG. 30A. However, when the analysis was actually performed, the distribution was close to the distribution illustrated in FIG. 30B. In order to properly remove the negative example in this state, it is necessary to shift the boundary score for determining the positive example to a higher score direction (ie, increase the boundary score from the score S1 to the score S2). This can be seen (see FIG. 30 (c)).

International Publication No. WO2014 / 020908

(4) When data that cannot be clearly discriminated by the discrimination model (hereinafter sometimes referred to as gray data) increases, manual confirmation is required. Therefore, when operating using an existing discriminant model, it is desired that the influence of such gray data be suppressed and the accuracy of the discriminant model be ensured.

For example, in a discrimination problem with a clear rule in the background, it is preferable that the result of the discrimination using the discrimination model be completed as much as possible without human intervention. On the other hand, even if the determination problem is based on a clear rule, the determination accuracy of the determination model may decrease due to some external factors such as the presence of an unknown explanatory variable, and the gray data may increase.

閾値 In order to increase the discrimination accuracy, it is conceivable to correct the threshold value used for discrimination by the discrimination model as in the method described in Patent Document 1. For example, as shown in FIG. 30 described above, it is possible to exclude negative example data from the discrimination result by the discrimination model by shifting the boundary score toward a higher score. However, when the boundary score is increased, positive example data having a low score (negative example-side positive example data) is also excluded from the prediction target (that is, the judgment result by the model cannot be used). There is a problem that more manpower is required for confirmation.

It is also conceivable that a large number of data can be determined by combining a large number of determination conditions. However, if the discrimination conditions are complicated, the clarity and interpretability of the discrimination conditions are reduced, so that there is a problem that it is difficult for the user to determine whether or not the discrimination is appropriate based on the discrimination model.

Therefore, an object of the present invention is to provide a model updating apparatus, a model updating method, and a model updating program that can update a discrimination model so as to improve the discrimination accuracy while maintaining the discrimination conditions of an existing discrimination model.

A model updating apparatus according to the present invention is a model updating apparatus for updating a hierarchical mixed model, comprising: a data extracting unit for extracting data classified under a target condition in the hierarchical mixed model; A model in which a data replenishing unit that receives replenishment for a model, a model generating unit that generates a discriminant model using the replenished data, and a node that classifies data that satisfies the target condition are arranged at the top of the hierarchical mixed model And a model updating unit that applies the data that does not satisfy the condition to the hierarchical mixed model corresponding to the leaf node for the node node, and converts the condition to the discriminant model corresponding to the leaf node for the node node. The method is characterized in that a model indicating that the satisfying data is applied is generated.

A model updating method according to the present invention is a model updating method for updating a hierarchical mixed model, in which data classified under a target condition in the hierarchical mixed model is extracted, and replenishment for the extracted data is received. , A discriminant model is generated using the supplemented data, a node node for classifying data satisfying the target condition is generated at the top of the hierarchical mixed model, and a model is generated. Applying data that does not satisfy the condition to a hierarchical mixed model corresponding to a leaf node with respect to, and generating a model indicating that data satisfying the condition is applied to a discriminant model corresponding to a leaf node with respect to a node node .

The model update program according to the present invention is a model update program that is applied to a computer that updates a hierarchical mixed model.The computer updates the computer with data classified under the conditions targeted in the hierarchical mixed model. Hierarchical data extraction processing to extract, data replenishment processing to accept replenishment for the extracted data, model generation processing to generate a discriminant model using the replenished data, and node nodes that classify data that satisfies the target conditions A model updating process for generating a model arranged at the top of the type mixed model is executed, and in the model updating process, data that does not satisfy the conditions is applied to the hierarchical mixed model corresponding to the leaf node for the node node, Applying data that satisfies the conditions to the discriminant model corresponding to a leaf node Characterized in that to generate the model shown.

According to the present invention, the discrimination model can be updated so as to improve the discrimination accuracy while maintaining the discrimination conditions of the existing discrimination model.

FIG. 1 is a block diagram illustrating a configuration example of a first embodiment of a model updating device according to the present invention. It is explanatory drawing which shows the example of the discrimination model by a hierarchical mixture model. FIG. 9 is an explanatory diagram illustrating an example of a data determination result. FIG. 14 is an explanatory diagram illustrating an example of a process in which data is excluded. 6 is a flowchart illustrating an operation example of the model updating device. FIG. 9 is an explanatory diagram illustrating another example of a data determination result. 13 is a flowchart illustrating another operation example of the model updating device. FIG. 9 is an explanatory diagram illustrating an example of a process of determining gray data. FIG. 13 is an explanatory diagram illustrating another example of the process of determining gray data. It is a block diagram showing the example of composition of the second embodiment of the model updating device by the present invention. FIG. 9 is an explanatory diagram illustrating an example of a discrimination model serving as a reference. FIG. 9 is an explanatory diagram illustrating an example of a hierarchical mixture model generated using first learning data. FIG. 9 is an explanatory diagram illustrating an example of a determination result. FIG. 9 is an explanatory diagram illustrating an example of a determination result. FIG. 9 is an explanatory diagram illustrating an example of a process of generating a new hierarchical mixed model. FIG. 9 is an explanatory diagram illustrating an example of a generated hierarchical mixed model. It is an explanatory view showing an example of a discrimination system. It is a flowchart which shows the operation example of a clean zone extraction process. It is a flowchart which shows the operation example of a gray zone extraction process. It is a flowchart which shows the operation example of gray data exclusion processing. It is a flow chart which shows an example of operation of gray zone deep excavation processing. It is explanatory drawing which shows the example of the discrimination model by a hierarchical mixture model. FIG. 9 is an explanatory diagram showing an example of displaying the properties of each discriminant. FIG. 11 is an explanatory diagram illustrating an example of a result of classifying data using a discrimination model. FIG. 11 is an explanatory diagram illustrating an example of a result of classifying data using an updated discrimination model. It is a block diagram showing the outline of the model updating device by the present invention. FIG. 2 is a schematic block diagram illustrating a configuration of a computer according to at least one embodiment. FIG. 4 is an explanatory diagram illustrating an example of a discrimination model for performing a binary determination. FIG. 11 is an explanatory diagram illustrating another example of a discrimination model that performs a binary determination. FIG. 9 is an explanatory diagram showing experimental results of analysis by binary discrimination.

Hereinafter, embodiments of the present invention will be described with reference to the drawings.

Embodiment 1 FIG.
FIG. 1 is a block diagram showing a configuration example of a first embodiment of a model updating device according to the present invention. The model updating apparatus according to the present invention has a function of updating a hierarchical mixed model among the discrimination models. The hierarchical mixture model is represented by a tree structure, and has a structure in which components are arranged in leaf nodes and a gate function (branch function) indicating a branch condition is arranged in another upper node. The branch condition of the gate function is described using explanatory variables. When data is input to the discriminant model, the input data is branched by a gate function, and is assigned to any of a plurality of components following a root node and each node.

In the present embodiment, it is assumed that a discriminant model is created in advance, and a discriminant model to be used as a reference has been determined. In other words, by generating a hierarchical mixed model based on teacher data, it becomes possible to make a determination using a discriminant provided for a leaf node. In the following description, a case in which binary discrimination is performed using a hierarchical mixed model will be described as an example.

FIG. 2 is an explanatory diagram illustrating an example of a discrimination model based on a hierarchical mixed model. In the example shown in FIG. 2, the input data is classified into any of four types of leaf nodes based on the conditions 1 to 3, and is determined based on the discriminants Y ₁ to Y ₄ allocated to each leaf node. Indicates that For example, when the data satisfies the condition 1 (data satisfying the condition 1 = true) is input, the data discriminant _{Y 1} is classified into the leaf nodes arranged, discriminant _Y 1 ₌ F 1 (X) Is determined based on

(4) It is preferable to select a discriminant model as a reference that is suitable for operation. Specifically, it is preferable to select a discrimination model from a plurality of generated discrimination models not only from the viewpoint of accuracy but also from the viewpoint of easy understanding of the user. That is, a model constituted by variables and coefficients that the user is most satisfied with can be said to be a model that is easy for the user to understand.

Referring to FIG. 1, a model updating apparatus 100 according to the present embodiment includes a storage unit 10, an input unit 20, a data extracting unit 30, a data replenishing unit 40, a model generating unit 50, a model updating unit 60, And an output unit 70.

The storage unit 10 stores data to be determined. Further, the storage unit 10 may store various parameters necessary for the model generation unit 50 described later to generate a model. The storage unit 10 may store supplementary data received by the data supplementing unit 40 described later. The storage unit 10 is realized by, for example, a magnetic disk or the like.

The input unit 20 inputs data to be determined. The input unit 20 may read, for example, data to be determined stored in the storage unit 10 and input the data to the data extraction unit 30. Further, the input unit 20 may receive an instruction to select a branch condition to be extracted by the data extracting unit 30 described later.

The 抽出 data extraction unit 30 determines the data classified into the leaf nodes by using the discriminant formula allocated to each leaf node. The data extraction unit 30 performs the same determination on the data classified into other leaf nodes, and totals the determination results of the data classified into each leaf node. In the present embodiment, since the data classified into each leaf node is data classified according to the branch condition, the data extraction unit 30 determines the classification result of the data classified under the condition traced from the node. It can be said that the total is calculated. That is, the data extraction unit 30 can be referred to as a determination result totalization unit.

The data extraction unit 30 extracts data classified under the target condition in the hierarchical mixed model. Specifically, the data extraction unit 30 may extract data classified under a condition that the data determination result does not satisfy the criteria.

The data extraction unit 30 calculates, for each leaf node (that is, a condition under which the data is classified), the ratio of the correctness of the prediction result of the classified data. Specifically, the data extraction unit 30 totals the correct answer ratio, which is the ratio of data that is truly positive and the determination result is also positive among the data classified under each condition. By totalizing the prediction results for each leaf node, it is possible to determine whether or not the condition for classifying the leaf node can be explained by a known explanatory variable X. This is because, for example, as shown in FIG. 28 described above, if all the conditions for performing the binary determination on the input information are known, the target data can be completely determined.

基準 Criteria for determining whether to extract data are set appropriately according to the operation. The Gini coefficient may be used as a reference. In the case where the rule is highly dependent (the case where a clear rule is in the background), for example, a Gini coefficient of 99% is set as a reference.

FIG. 3 is an explanatory diagram illustrating an example of a data determination result. The table illustrated in FIG. 3 shows a result of discrimination of each data classified into the conditions (leaf nodes) determined by each of the discriminants Y ₁ to Y ₄ . The TP (True Positive) illustrated in FIG. 3 is the number of cases where the data of the positive example is determined as a positive example, and the TN (True Negative) is the number of cases where the data of the negative example is determined as a negative example. Further, FP (False Positive) is the number of cases where the data of the positive example is determined as a negative example, and FN is the number of cases where the data of the negative example is determined as a positive example. In the following description, a subscript may be added to each determination result.

For example, a result of the determination by the discriminant _{Y 1} illustrated in Figure 3, TP is 100, since TN is 1, the ratio of TP of at least 99%. Similarly, a result of the determination by the discriminant Y ₄ illustrated in FIG. 3, only the TP can be said because it is 50, and can be explained only by the explanatory variable X with respect to the determination result. Therefore, for example, a criterion “the ratio of TP is 99% or more” may be provided.

Meanwhile, a result of the determination by the discriminant _{Y 2} illustrated in Figure 3, TP is 20, FP is 30, FN 15, and, TN is 20. As a result of determination by the discriminant _{Y 3} illustrated in FIG. 3, TP is 0, FP is 20, FN 40, and, TN is 10. From these results, it can be determined that it is difficult to explain or categorize using the explanatory variable X, and that there is a possibility that an unknown explanatory variable X ′ exists.

As described above, the data extraction unit 30 may extract data classified under the condition that the correct answer ratio is equal to or less than the predetermined threshold. The data extracted in this way is used as learning data for performing more appropriate discrimination. In other words, it can be said that the data extracting unit 30 extracts, as gray data, data classified under the condition that the determination result of the data satisfies the standard. In the following description, “gray data” does not mean data that cannot be distinguished or not, but means data that is difficult to distinguish only by given explanatory variables. In the example shown in FIG. 3, the data extraction unit 30 satisfies the condition data classified to the determination target by satisfying data and discriminant Y ₃ are classified into the discrimination object by discriminant Y _2, extracted, respectively.

The data extraction unit 30 may extract data classified under the conditions specified by the user via the input unit 20. Specifically, the data extracting unit 30 may output the totalized determination result to the output unit 70 described later, and may extract data classified under the condition pointed out by the user according to the determination result. .

場合 In addition, when the number of targets satisfying the criterion by the known explanatory variables is limited with respect to the total number of cases, it is suspected that the explanatory variables used for learning are insufficient. On the other hand, it is also possible to classify the subordinates under the target conditions in more detail by adding further conditions. Therefore, the data extraction unit 30 may extract data classified based on more detailed conditions. A method of classifying the subordinates of the target condition in more detail (that is, performing deep excavation of the condition) will be described in more detail in an embodiment described later.

The data replenishment unit 40 receives replenishment for the data extracted by the data extraction unit 30. Here, replenishment of data means so-called machine teaching, which includes adding a value based on a new explanatory variable to the data, updating a teacher label of the data, and the like. For example, when data relating to a certain user does not include information indicating a work location, the data supplementing unit 40 may receive information using the work location as an explanatory variable. When changing the teacher label for a certain user, the data supplementing unit 40 may receive the teacher label to be changed.

The data supplementing unit 40 may output the data group extracted by the data extracting unit 30 in, for example, a file format, and accept a data group in which the user supplements the output data group with information.

The model generation unit 50 generates a discrimination model using the supplemented data. The mode of the model generated by the model generation unit 50 is arbitrary. The model generation unit 50 may generate, for example, a simple linear regression model as a discrimination model, or may learn a discrimination model represented by a hierarchical mixed model. By learning the discriminant model represented by the hierarchical mixed model, it becomes possible to clarify the conditions for classifying gray data while maintaining the clarity and interpretability of the discriminant conditions.

(4) The model generation unit 50 may generate a plurality of types of discrimination models. In this case, the model generation unit 50 causes the output unit 70 described later to output the generated plural types of discrimination models, and allows the user (for example, via the input unit 20) to select a desired discrimination model from the plural types of discrimination models. May be selected.

The model updating unit 60 updates the existing discrimination model using the generated new discrimination model. Specifically, the model updating unit 60 generates a model in which node nodes that classify data that satisfy the conditions for which the data extracting unit 30 extracts data are arranged at the top of the hierarchical mixed model. More specifically, the model updating unit 60 applies the data that does not satisfy the condition to the hierarchical mixed model corresponding to the leaf node for the node, and converts the data that satisfies the condition to the discriminant model corresponding to the leaf node for the node. Generate a model that indicates the application.

(4) By using the model generated in this way, an effect is obtained in which data that is difficult to discriminate with an existing discriminant model is excluded in advance by a newly generated discriminant model. The model updating unit 60 may generate a discriminant model that uses the discriminant model generated by the model generating unit 50 as a model for filtering data input to the existing discriminant model in advance. Further, the model updating unit 60 may generate a discrimination model in which the generated discrimination model is directly combined with the existing discrimination model. In this way, the model updating unit 60 arranges the generated discrimination model at the top of the hierarchical mixed model.

FIG. 4 is an explanatory diagram showing an example of a process in which data is excluded by a new determination model. The new discriminant model M1 illustrated in FIG. 4 is a discriminant model generated by the model generating unit 50, and the existing discriminant model M2 is a discriminant model created in advance as a reference. The model updating unit 60 generates a model in which node nodes are arranged at the highest level such that the new discrimination model M1 and the existing discrimination model M2 are respectively leaf nodes. Specifically, data that satisfies the conditions for the data extraction unit 30 to extract data is applied to the new discrimination model M1.

For example, in the example shown in FIGS. 2 and 3, the model generating unit 50 has been extracted by the data extraction unit 30, based on data that meets conditions are classified into the discrimination object by discriminant Y ₂ or discriminants Y ₃ Assume that a new discriminant model is generated. At this time, data to be determined by the new determination model M1 is data that satisfies (not (condition 1) and (condition 2)) or (not (condition 1) and not (condition 2) and (condition 3)). become. Since the data to be discriminated by the new discrimination model M1 is excluded from the discrimination processing of the discrimination model M2, the above-described condition can be called an exclusion condition (exclusion rule) of the existing discrimination model.

When the teacher data D1 is input to the models illustrated in FIG. 4 (specifically, the new discrimination model M1 and the discrimination model M2), the data extracting unit 30 converts the data satisfying the target condition into a new discrimination model. The determination is made using M1. For example, when the new discrimination model M1 is a hierarchical mixed model, data satisfying the target condition is further classified according to each branch condition, and discrimination processing is performed at each leaf node.

The data extraction unit 30 extracts data classified under the condition that the result of the data determination does not satisfy the criteria. In other words, the data extraction unit 30 extracts data that is difficult to explain using the known explanatory variable X, and excludes data that is difficult to discriminate using the discrimination model M2 as preprocessing of the discrimination processing using the existing discrimination model M2. It can be said that.

The data extraction unit 30 determines data other than the extracted data using the determination model M2. The data to be subjected to the discrimination processing by the discrimination model M2 can be said to be data that can be explained by the explanatory variable X. Therefore, the result determined by the determination model M2 can be said to be almost 100% reliable, and this result can be applied to various cases.

The output unit 70 outputs the information of the discrimination model to an output device such as a display device (not shown). The output unit 70 may output, for example, an updated model. In addition, as described above, the output unit 70 may output the determination result by the data extraction unit 30 or may output the determination conditions and the discriminant in a plurality of types of the determination models. The specific output mode will be described later.

The input unit 20, the data extraction unit 30, the data replenishment unit 40, the model generation unit 50, the model update unit 60, and the output unit 70 include a computer processor (eg, a computer that operates according to a program (model update program)). It is realized by a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), and an FPGA (field-programmable gate array).

For example, the program is stored in the storage unit 10, and the processor reads the program, and according to the program, the input unit 20, the data extraction unit 30, the data supplement unit 40, the model generation unit 50, the model update unit 60, and the output unit 70. It may operate as. Further, the function of the model updating device may be provided in SaaS (Software @ as \ a \ Service \) format.

The input unit 20, the data extraction unit 30, the data replenishment unit 40, the model generation unit 50, the model update unit 60, and the output unit 70 may each be realized by dedicated hardware. In addition, some or all of the components of each device may be realized by a general-purpose or dedicated circuit (circuitry II), a processor, or a combination thereof. These may be configured by a single chip, or may be configured by a plurality of chips connected via a bus. Some or all of the components of each device may be realized by a combination of the above-described circuit and the like and a program.

When some or all of the components of the model updating device are realized by a plurality of information processing devices or circuits, the plurality of information processing devices or circuits may be centrally arranged or distributed. It may be arranged. For example, the information processing device, the circuit, and the like may be realized as a form in which each is connected via a communication network, such as a client server system and a cloud computing system.

Next, the operation of the model updating apparatus according to the present embodiment will be described. FIG. 5 is a flowchart illustrating an operation example of the model updating device according to the present embodiment. The input unit 20 inputs data to be determined (step S11). The data extraction unit 30 extracts data classified under the target condition (step S12). The data extracting unit 30 may extract, for example, data classified under the condition that the correct answer ratio is equal to or less than a predetermined threshold. The data replenishing unit 40 receives replenishment for the extracted data (step S13).

The model generation unit 50 generates a discrimination model using the supplemented data (step S14). Then, the model updating unit 60 generates a model in which node nodes for classifying data satisfying the target condition are arranged at the top of the hierarchical mixed model (step S15). Specifically, the model updating unit 60 applies a data that does not satisfy the condition to the hierarchical mixed model, and generates a model indicating that the data that satisfies the condition is applied to the discrimination model.

As described above, in the present embodiment, the data extraction unit 30 extracts data classified under the target conditions in the hierarchical mixed model, and the data replenishment unit 40 receives replenishment for the extracted data. , The model generation unit 50 generates a discrimination model using the supplemented data. Then, the model updating unit 60 generates a model in which node nodes for classifying data satisfying the target condition are arranged at the top of the hierarchical mixed model. Specifically, the model updating unit 60 applies the data that does not satisfy the condition to the hierarchical mixed model corresponding to the leaf node for the node node, and applies the data that satisfies the condition to the discriminant model corresponding to the leaf node for the node node. Generate a model that indicates Therefore, the discrimination model can be updated so as to improve the discrimination accuracy while maintaining the discrimination condition of the existing discrimination model.

For example, if the discriminant model is simply updated (reconstructed), the structure, branching condition, discriminant, and the like of the discriminant model selected by the user may change. When such an update is performed, even if the accuracy of the discrimination model can be improved, the model becomes difficult for the user to use.

On the other hand, in the present embodiment, the model updating unit 60 sets a node indicating that the discriminant model is applied to data that reduces the accuracy of the existing discriminant model (that is, data that satisfies the target condition). , A model arranged at the top of the hierarchical mixed model is generated. Therefore, a model to which the newly generated discriminant model is added can be generated without changing the structure of the existing discriminant model. Therefore, the discrimination accuracy of the discrimination model can be improved as a whole while satisfying the user's desire to keep using the existing discrimination model.

Next, a modified example of the model updating device of the present embodiment will be described. In the above embodiment, the case where the model updating unit 60 updates the existing discriminant model using the new discriminant model generated by the model generating unit 50 has been described. At this time, the data extraction unit 30 may further extract data classified under the condition that the data discrimination result does not satisfy the criteria, using the discrimination model generated by the model generation unit 50.

In this case, the data replenishing unit 40 may receive replenishment for the data further extracted by the data extracting unit 30, and the model generating unit 50 may further generate a discriminant model using the replenished data. The data extraction unit 30 may repeat the above processing until the number of extracted data does not decrease (in other words, only data that cannot be determined by a known explanatory variable remains).

FIG. 6 is an explanatory diagram illustrating another example of the data determination result. The table illustrated in FIG. 6 shows the determination result of each data classified into the conditions (leaf nodes) determined by each of the discriminants Y ₁₁ to Y ₁₄ similarly to the table illustrated in FIG. The discrimination result illustrated in FIG. 6 has a low ratio of being discriminated as TP, and thus the discrimination result has low validity and is considered to be an unreliable result. In other words, the determination result illustrated in FIG. 6 is difficult to determine using the known explanatory variable X, and can be determined to be data affected by the unknown explanatory variable X ′. The data extraction unit 30 may repeat the above processing until only the data as illustrated in FIG. 6 remains.

条件 The condition applied to the newly generated discriminant model is an AND combination of the condition specified for each process. For example, when the discriminant illustrated in FIG. 6 is a discriminant performed on data satisfying “condition 4” in addition to the conditions illustrated in FIGS. 2 and 3, the exclusion condition (exclusion rule) (Not (condition 1) and (condition 2)) or (not (condition 1) and not (condition 2) and (condition 3)) and (condition 4).

Next, the operation of the present modified example will be described. FIG. 7 is a flowchart illustrating an operation example of the model updating device of the present modified example. The process of updating the reference discriminant model using the generated discriminant model (specifically, the processes of steps S11 to S15 illustrated in FIG. 5) is performed in advance.

The input unit 20 inputs the data to be determined again (step S16). The data extracting unit 30 extracts data classified under the target condition in the model generated by the model updating unit 60 (Step S17). The data replenishing unit 40 further receives replenishment for the extracted data (step S18). The model generation unit 50 generates another discrimination model using the supplemented data (Step S19). The model updating unit 60 generates a model indicating that data satisfying the target condition is applied to another discrimination model (step S20).

Thereafter, the processing from step S16 to step S20 is repeated until only data that cannot be determined by a known explanatory variable is extracted. In this way, by repeating the processing, it is possible to generate a discrimination model that more appropriately excludes data that cannot be discriminated by a known explanatory variable.

Embodiment 2. FIG.
Next, a second embodiment of the model updating device according to the present invention will be described. In the first embodiment, the method of updating the model by supplementing data extracted as gray data has been described. In the present embodiment, the conditions for extracting gray data are further deepened to refine the conditions, thereby determining whether the explanatory variables for prediction are satisfied. Further, by using the conditions refined in this way and extracting gray data from input data in advance, the discrimination accuracy of the reference discrimination model is improved.

First, gray data used in the present embodiment will be described. FIG. 8 is an explanatory diagram illustrating an example of a process of determining gray data. Each rectangle in FIG. 8 represents data to be determined. As described above, in the general method, a threshold value for determining a positive example and a negative example is provided, and data having a score smaller than the threshold value is not automatically determined to be a positive example. In the example shown in FIG. 8, a rectangle above the horizontal axis represents data of a positive example, and a rectangle below the horizontal axis represents data of a negative example. In the example illustrated in FIG. 8, it is assumed that the threshold value S3 is set to a value larger than the score = 0 and larger than the data of the negative example having the highest score.

In the case of the method of discriminating based on the threshold value, the data group S4 smaller than the threshold value S3 is not automatically determined to be a positive example even if the data group is smaller than the threshold value S3 even if the data is greater than 0. Become. That is, although the data group S4 is statistically correct, in the AI determination, it becomes data that is not subject to automatic determination due to the setting of the threshold.

FIG. 9 is an explanatory diagram showing another example of the process of determining gray data. In the present embodiment, similarly to the first embodiment, the determination results of the data classified into each leaf node are totaled. The unit of the data to be aggregated is sometimes referred to as a zone. Also, in the present embodiment, a zone (that is, a data group under the condition) that includes data having the same or similar prediction formula but different determination results is described as a gray zone. Data belonging to the gray zone is referred to as gray data. On the other hand, when the data and the prediction formula are the same or similar, a zone for which the determination result is uniquely determined (that is, a data group under known explanatory variables and under the condition that all results can be predicted) is referred to as a clean zone. Data belonging to the clean zone is referred to as clean data. For example, a zone in which a positive example and a negative example are mixed is a zone in which a determination result is not uniquely determined, and thus can be called a gray zone.

In the example shown in FIG. 9, each area surrounded by a dotted line indicates a gray zone, and each area surrounded by an elliptical solid line indicates a clean zone. In other words, even if the scores calculated by the general method are the same, whether the data belongs to the gray zone or the clean zone differs for each data depending on the process of calculating the score (difference between nodes). By handling data in units of zones, it is not necessary to set a threshold. In addition, a part of the negative example having a high score can be treated as a gray zone. However, the use of the score in discriminating a positive example from a negative example is not prevented from the operational convenience.

FIG. 10 is a block diagram showing a configuration example of the second embodiment of the model updating device according to the present invention. The model updating device 200 according to the present embodiment includes a storage unit 10, an input unit 20, a learning data generation unit 31, a model learning unit 32, a score calculation unit 33, a condition extraction unit 34, a condition generation unit 35, , A filter generation unit 61, and an output unit 70. That is, instead of the data extracting unit 30, the data replenishing unit 40, the model generating unit 50, and the model updating unit 60, the model updating device 200 of the present embodiment includes a learning data generating unit 31, a model learning unit 32, and a score calculating unit 33. , A condition extracting unit 34, a condition generating unit 35, and a filter generating unit 61 are different from the model updating apparatus 100 of the first embodiment. Other configurations are the same as in the first embodiment.

The storage unit 10 stores data to be determined and various parameters, as in the first embodiment. The input unit 20 inputs the data to be determined as in the first embodiment.

The learning data generation unit 31 generates learning data used when the model learning unit 32 described later learns the hierarchical mixed model. The model learning unit 32 generates a hierarchical mixed model by heterogeneous machine learning using the generated learning data. More specifically, with regard to heterogeneous machine learning, the model learning unit 32 preferably uses FAB (Factorized Asymptotic, Bayesian, inference) that maximizes the lower limit of the information criterion FIC (Factorized, Information, Criterion). However, with a similar technique, the method by which the model learning unit 32 learns the hierarchical mixture model is not limited to the heterogeneous mixture machine learning.

The 算出 score calculation unit 33 calculates a data determination result for each leaf node in the hierarchical mixture model. The condition extraction unit 34 extracts a branch condition for each leaf node based on a predetermined criterion. The condition generator 35 generates a condition combining the extracted conditions.

The learning data generation unit 31, the model learning unit 32, the score calculation unit 33, the condition extraction unit 34, and the condition generation unit 35 in the present embodiment differ in target data and criteria to be used depending on the progress of processing. In the present embodiment, a case will be described in which each component unit operates by changing target data and a reference to be used according to the progress of processing. However, each component may be implemented as a separate component according to the content of each process. Hereinafter, the operation of each component will be described along the processing flow.

[(1) Clean zone extraction processing]
First, as a first process, a process of extracting a clean zone in learning data will be described. As described above, the clean zone is a zone in which the determination result is uniquely determined, and can be said to be a zone in which explanatory variables for predicting data are satisfied. As a premise, as in the first embodiment, it is assumed that a reference discrimination model has been determined. FIG. 11 is an explanatory diagram illustrating an example of a discrimination model serving as a reference. In the discriminant model M20 illustrated in FIG. 11, based on the conditions C1 and C2, input data is classified into any of three types of leaf nodes, and discriminants Y ₂₁ to Y ₂₃ arranged in each leaf node. Is determined based on

教師 Further, it is assumed that teacher data including a label indicating a determination result is stored in the storage unit 10 in advance. Here, it is assumed that there are 100,000 teacher data, and the ratio of positive examples to negative examples in the teacher data is 19: 1. The input unit 20 reads the teacher data stored in the storage unit 10 and inputs the data to the learning data generation unit 31.

The learning data generation unit 31 applies the teacher data including the label indicating the input discrimination result to the discrimination model serving as a reference. Then, the learning data generating unit 31 sets learning data in which the discrimination result of the discrimination model matches the label as a positive example, and sets learning data in which the discrimination result and the label differ from each other as a negative example (hereinafter referred to as first learning data). .) Is generated.

For example, it is assumed that as a result of applying the teacher data to the discrimination model, it is determined that TP ₁ = 93.5K, TN ₁ = 0.3K, FP ₁ = 5.6K, and FN ₁ = 0.6K. In this case, the learning data generation unit 31 generates learning data using the teacher data determined as TP ₁ and TN ₁ as a positive example. Similarly, the learning data generation unit 31 generates learning data using the teacher data determined as FP ₁ and FN ₁ as a negative example.

The model learning unit 32 generates a hierarchical mixed model (hereinafter, referred to as a first hierarchical mixed model) by heterogeneous machine learning using the generated first learning data. Note that the first hierarchical mixed model generated here is a model different from the reference discriminant model. By generating a hierarchical mixture model by heterogeneous machine learning, it becomes possible to separate and extract patterns and regularities mixed in the original data from the automatically divided data. In addition, by using the generated hierarchical mixture model, the data group classified by the gate tree can be appropriately determined by the regression equation arranged on the leaf. Further, a model generated by heterogeneous machine learning is a technique that can be analyzed at various angles, and can be analyzed using explanatory variables of a discriminant model used as a reference.

FIG. 12 is an explanatory diagram illustrating an example of a hierarchical mixture model generated using the first learning data. The model learning unit 32 generates, for example, a hierarchical mixture model illustrated in FIG. In the discriminant model illustrated in FIG. 12, the first learning data is classified into any of three types of leaf nodes based on the conditions C3 and C4, and the discriminant expressions Y ₃₁ to Y ₃₃ arranged in each leaf node are used. Indicates that it is determined based on

Data Specifically, in the example shown in FIG. 12, data that is determined in discriminant Y ₃₁ is data that satisfies the condition "C = 0" and "D ≠ 0", which is determined by the discriminant Y ₃₂ is data that satisfies the condition "C = 0" and "D = 0", the data is determined by the discriminant Y ₃₃ is data that satisfies the condition "C ≠ 0".

The score calculation unit 33 calculates, for each leaf node in the generated first hierarchical mixed model, among the first learning data classified into the leaf node, data in which the first learning data set as a positive example is correctly determined. Is calculated (that is, the ratio of TP). Hereinafter, the calculated ratio is referred to as a first score.

FIG. 13 is an explanatory diagram illustrating an example of the determination result. The example shown in FIG. 13 shows that the determination results by the discriminants Y ₃₁ to Y ₃₃ are classified into TP ₂ , FP ₂ , TN ₂ and FN ₂ respectively. For example, the discriminant Y _31, since the first learning data of five, which is a positive cases were all correctly determined, explanatory variables to predict for the first learning data of the five is the they meet I can say.

The condition extraction unit 34 extracts a branch condition to a leaf node whose first score has been calculated so as to satisfy a predetermined criterion. The criterion defined here is a criterion for determining whether or not data that can be determined by using the explanatory variables used in the first hierarchical mixture model is a leaf node to be classified. In the following description, this criterion is referred to as a first criterion. That is, the first criterion can be said to be a criterion for determining whether or not a zone (clean zone) for which the determination result is uniquely determined as described above. For example, a criterion of “satisfying 100%” may be set as the first criterion. This indicates that the explanatory variables for predicting data classified as the target leaf node are satisfied. However, the first criterion is not limited to “satisfying 100%”, and a predetermined value less than 100 (for example, 0.995) may be set as the first criterion.

For example, when the first reference to "meet 100%", in the example shown in FIG. 13, determination result by the discriminant _{Y 31} to satisfy the TP / CNT = 5/5 = 1.0 = 100% . Therefore, condition extracting unit 34 extracts the discriminant Y ₃₁ is the branch condition to set the leaf node ( "C = 0" and "D ≠ 0"). Although FIG. 13 illustrates a case where only one branch condition is extracted, the number of branch conditions to be extracted is not limited to one and may be two or more.

The model learning unit 32, the score calculation unit 33, and the condition extraction unit 34 repeat the above processing. The number of repetitions depends on the machine resources and the like, but is preferably repeated in units of, for example, several hundred to several thousand. For example, if an attempt is made to classify data using a general binary tree and a machine learning algorithm that learns a discriminant to be assigned to a leaf node of the binary tree, if the data is executed several hundred to several thousand times, It is difficult to classify data. On the other hand, in the present embodiment, the model learning unit 32 generates a hierarchical mixed model (first hierarchical mixed model) that can be regularized as a whole. Therefore, the data can be classified by performing the learning several hundred to several thousand times.

More specifically, the model learning unit 32 generates a plurality of types of first hierarchical mixed models using the same generated first learning data, for example, by changing initial parameters in heterogeneous machine learning. . The score calculation unit 33 calculates the ratio of TP for each leaf node in the generated first hierarchical mixed model, and the condition extracting unit 34 determines a first criterion for each generated first hierarchical mixed model. The branch condition to the leaf node for which the first score to be satisfied is calculated is extracted.

(4) The condition generator 35 generates a condition (hereinafter, referred to as a distinguishable condition) in which branch conditions satisfying the first criterion are combined. Specifically, the condition generating unit 35 generates a distinguishable condition by combining all the extracted branch conditions. Since the discriminable condition is a condition in which zones in which the discrimination result is uniquely determined are combined, it can be said that it is a clean zone specific condition. For example, when a branch condition to Z leaf nodes is extracted, it can be said that Z zones (segments) that can be completely ruled (that is, can be predicted using a known explanatory variable) have been extracted. .

[(2) Gray zone extraction processing]
Next, as a second process, a process of extracting a gray zone in learning data will be described. The gray zone extraction process is an auxiliary process for efficiently proceeding with the above-described first process (that is, the clean zone extraction process). Since the clean zone has been extracted in the first processing, the learning data generating unit 31 determines the learning data (hereinafter, referred to as the second learning data) excluding the learning data corresponding to the discriminable condition from the first learning data. .) Is generated. For example, suppose that there are 40,000 first learning data items corresponding to the discriminable condition out of 100,000. In this case, the learning data generation unit 31 excludes the corresponding 40,000 cases from 100,000 cases and generates 60,000 second learning data.

クリーン By removing clean data from the learning data, the ratio of the remaining learning data that is difficult to determine increases. For example, if the number of positive cases becomes 54,000 and the number of negative cases becomes 65,000 by this process, the ratio of the positive case to the negative case becomes 9: 1.

The model learning unit 32 generates a hierarchical mixed model (hereinafter, referred to as a second hierarchical mixed model) by heterogeneous machine learning using the generated second learning data.

For each leaf node in the generated second hierarchical mixed model, the score calculation unit 33 has correctly determined the second learning data that is regarded as a positive example among the second learning data classified into the leaf node. The sum of the ratio and the ratio at which the second learning data set as a negative example was not correctly determined (that is, the ratio of (TP + FN)) is calculated. Hereinafter, the calculated ratio is referred to as a second score.

FIG. 14 is an explanatory diagram illustrating an example of the determination result. The example shown in FIG. 14 shows that the discrimination results by the discriminants Y ₄₁ to Y ₄₃ are classified into TP ₃ , FP ₃ , TN ₃ and FN ₃ respectively. Score calculation unit 33, for example, the discriminant Y _41, the proportion second learning data is a positive case was correctly discriminated 0 (= 0/6), the second learning data is a negative example The sum of the incorrectly determined ratio 0.16 (= 1/6) is calculated as 0.16.

The condition extraction unit 34 extracts a branch condition to a leaf node whose second score has been calculated so as to satisfy a predetermined criterion. The criterion determined here is a criterion for judging whether or not the data is a leaf node into which data that is difficult to determine only by using an explanatory variable used in the second hierarchical mixed model is classified. In the following description, this criterion is referred to as a second criterion. That is, the second criterion can be said to be a criterion for determining whether or not the determination result is a zone (gray zone) for which the determination result is not uniquely determined, as described above. If the value of the second criterion is reduced, a branch condition indicating a zone with a large proportion of gray data can be extracted, and if the value of the second criterion is increased, more branch conditions can be extracted. As the second criterion, for example, a criterion of “less than 0.5” can be set. The value to be set is not limited to 0.5, but may be set to a value of 0.7 to 0.8, for example.

The model learning unit 32, the score calculation unit 33, and the condition extraction unit 34 repeat the above-described processing as in the case of the first processing. The number of repetitions depends on the machine resources and the like, but is preferably repeated in units of, for example, several hundred to several thousand. In the present embodiment, similarly to the generation of the first hierarchical mixed model, in the present embodiment, the model learning unit 32 generates a hierarchical mixed model (second hierarchical mixed model) that can be regularized as a whole. Therefore, the data can be classified by performing the learning several hundred to several thousand times.

Specifically, the model learning unit 32 generates a plurality of types of second hierarchical mixed models using the same generated second learning data. The score calculation unit 33 calculates the ratio of TP + FN for each leaf node in the generated second hierarchical mixed model, and the condition extracting unit 34 determines the second reference for each generated second hierarchical mixed model. The branch condition to the leaf node for which the second score that satisfies is satisfied is extracted.

The condition generator 35 generates a condition (hereinafter, referred to as a difficult-to-discriminate condition) obtained by combining conditions satisfying the second criterion. Specifically, the condition generating unit 35 generates a difficult-to-discriminate condition by combining all the extracted branch conditions. Since the difficult-to-discriminate condition is a condition combining zones that are difficult to determine only with the given explanatory variables, it can be said that it is a specific condition of the gray zone.

[(3) Gray data exclusion processing]
Next, as a third process, a process of excluding gray data from the second learning data will be described. The learning data generation unit 31 generates data (hereinafter, referred to as third learning data) excluding the learning data corresponding to the difficult-to-discriminate condition from the second learning data.

For example, in FIG. 14, positive cases and negative examples percentage of data as determined by the discriminant Y ₄₁ is 1: 5, positive cases and negative cases the proportion of data which is determined in discriminant Y ₄₂ is 1: It is one. Here, if the extraction of data as determined by the discriminant data and discriminant Y ₄₂ is determined in Y ₄₁ as gray data, but also to exclude negative example 6 by simply excluding positive example 2 Will be possible. This makes it possible to increase the ratio of clean data in the learning data.

By the above-described (1) clean zone extraction processing, (2) gray zone extraction processing, and (3) gray data exclusion processing, data determined as clean data or gray data is excluded from the learning data. . In order to further exclude clean data or gray data, (1) clean zone extraction processing, (2) gray zone extraction processing, and (3) gray data exclusion processing may be repeated. That is, the model learning unit 32 may generate the first hierarchical mixture model using the generated third learning data.

[(4) Gray zone deep moat treatment]
Next, as a fourth process, a process for further refining the conditions for specifying the gray zone (gray zone deep digging process) will be described. Here, a case will be described in which (3) gray zone deep excavation processing is performed on (3) learning data on which gray data exclusion processing has been performed (that is, third learning data). However, (4) gray zone deep excavation processing may be performed on the first learning data.

スコア Similar to the processing shown in (1) clean zone extraction processing, the score calculation unit 33 calculates the first score for each leaf node in the first hierarchical mixed model generated using the third learning data. Further, the condition extracting unit 34 extracts a branch condition to a leaf node for which a first score that does not satisfy the first criterion is calculated. In other words, the condition extracting unit 34 extracts a branch condition to a leaf node that is not determined as a clean zone.

The model learning unit 32 uses learning data classified into leaf nodes from which branch conditions have been extracted, and uses a hierarchical mixed model (hereinafter, referred to as a third hierarchical mixed model) that branches conditionally under the leaf nodes. ).

FIG. 15 is an explanatory diagram showing an example of processing for generating a new hierarchical mixed model under a leaf node. The hierarchical mixed model M21 illustrated in FIG. 15 is the same as the hierarchical mixed model illustrated in FIG. In FIG. 15, each balloon indicates the classification results D51 to D53 of the classified data at each leaf node. In the discrimination result, the number of “○” indicates the ratio of TP data, and the number of “×” indicates the ratio of other (that is, TN, FP, FN) data.

For example, it is assumed that, for the leaf node C7 illustrated in FIG. 15, the ratio of the TP is approximately 50% as indicated by the determination result D51. In this case, since the first score calculated at this leaf node does not satisfy the first criterion, the condition extracting unit 34 extracts a branch condition to the leaf node C7. Then, the model learning unit 32 uses the data of the determination result D51 to generate a third hierarchical mixed model that branches conditionally under the leaf node C7.

FIG. 16 is an explanatory diagram illustrating an example of the generated third-layer mixed model. As illustrated in FIG. 16, the model learning unit 32 generates a third hierarchical mixed model M23 that branches conditionally under the leaf node C7 illustrated in FIG. By setting a new third-layer mixed model under the leaf nodes, more detailed branch conditions for each leaf node can be defined, and the discrimination results D61 to D63 of each leaf node are calculated. As a result, it is possible to further narrow down the conditions of the data group that can be predicted with the known explanatory variables and the conditions of the data group that cannot be predicted with the known explanatory variables. For example, the determination result D63 of the learning data classified into discriminant _{Y 63} indicates a TP100%. Therefore, it can be said that the data classified into this node can be predicted by a known explanatory variable. Therefore, the condition extracting unit 34 specifies this leaf node as a leaf node satisfying the first criterion (that is, a clean zone), and extracts a branch condition.

Further, the same processing may be further performed on each leaf node of the generated third hierarchical mixed model. That is, the score calculation unit 33 calculates the first score for each leaf node in the third hierarchical mixed model, and the condition extraction unit 34 determines whether the branch to the leaf node for which the first score satisfying the first criterion has been calculated. The condition may be extracted, and the model learning unit 32 may use the learning data classified into the leaf nodes to generate a third hierarchical mixed model that branches conditionally under the leaf nodes.

In the example shown in FIG. 16, a case is described in which the branch condition is digged only for one leaf node. However, the target for which the branch condition is to be excavated is not limited to one leaf node, but may be two or more leaf nodes. For example, also with respect to the leaf node C9 illustrated in FIG. 16, the calculated first score does not satisfy the first criterion, so the condition extracting unit 34 extracts a branch condition to the leaf node C9.

{Circle around (5)} Then, the condition generating unit 35 generates a discriminable condition in which branch conditions extracted as leaf nodes (that is, clean zones) satisfying the first criterion are further combined.

As described above, even when the learning data cannot be sufficiently classified only by the branch condition to a certain leaf node, the model learning unit 32 determines the model including the deeper branch condition (that is, the third hierarchical mixed model). ) Makes it possible to separate the clean zone and the gray zone under more detailed conditions. As described above, by performing more detailed segmentation, it is possible to further specify nodes that are satisfied with explanatory variables for prediction and nodes that are not satisfied.

The filter generation unit 61 generates a condition (hereinafter, referred to as a filter condition) for removing gray zone data (that is, gray data). In other words, the filter generation unit 61 generates a filter condition for removing data satisfying a condition that cannot be predicted by a known explanatory variable. Specifically, the filter generation unit 61 compares a branch condition to a leaf node for which a first score that does not satisfy the first criterion is calculated and a branch condition that satisfies the second criterion (that is, a difficult-to-discriminate condition). Combined to generate filter conditions. Note that the condition extracting unit 34 may extract a branch condition to a leaf node for which a first score that does not satisfy the first criterion is calculated, and the condition generating unit 35 may determine a branch condition that satisfies the second criterion. Combination difficulties may be generated.

The output unit 70 outputs the filter condition generated by the filter generation unit 61.

FIG. 17 is an explanatory diagram showing an example of the determination system. The discrimination system 500 illustrated in FIG. 17 includes a discrimination device 510 and a gray zone removal device 520.

The discrimination device 510 discriminates the input data 521 based on the discrimination model M20 used as a reference. The gray zone removal device 520 removes the gray data 522 from the input data based on the filter condition generated by the filter generation unit 61, and inputs the clean data 523 to the determination device 510. As described above, the gray zone removal device 520 removes the gray data 522 in advance, so that the determination result of the clean data input to the determination device 510 is guaranteed.

Note that the discrimination system illustrated in FIG. 17 can be said to be a device that selects whether data can be discriminated only by setting conditions using known explanatory variables. Therefore, the discrimination system illustrated in FIG. It can be called a sorting system. Further, since the discriminating system illustrated in FIG. 17 can be realized by the model updating device of the present embodiment, the model updating device of the second embodiment can also be called a discriminable data selection system.

The input unit 20, the learning data generation unit 31, the model learning unit 32, the score calculation unit 33, the condition extraction unit 34, the condition generation unit 35, the filter generation unit 61, and the output unit 70 include a program ( (A discriminable data selection program).

Next, the operation of the model updating apparatus according to the present embodiment will be described. FIG. 18 is a flowchart illustrating an operation example of the clean zone extraction process performed by the model updating device 200 according to the present embodiment. The input unit 20 inputs the teacher data to the learning data generation unit 31 (Step S21). The learning data generation unit 31 applies the teacher data including the label indicating the input discrimination result to the discrimination model serving as a reference (step S22). Then, the learning data generating unit 31 generates first learning data in which the teacher data whose discrimination result and the label match each other is set as a positive example, and the teacher data whose discrimination result is different from the label is set as a negative example (step S23).

The model learning unit 32 generates a first hierarchical mixed model by heterogeneous machine learning using the generated first learning data (step S24). The score calculation unit 33 calculates the ratio of TP for each leaf node in the generated first hierarchical mixture model (Step S25). The condition extraction unit 34 extracts a branch condition to the leaf node for which the first score has been calculated so as to satisfy the first criterion (Step S26).

If the number of repetitions of the processing from step S24 to step S26 (that is, the processing from generation of the model to extraction of the branch condition) has not reached the predetermined number (No in step S27), steps S24 to S26 The processing up to is repeated. On the other hand, when the number of repetitions has reached the predetermined number (Yes in step S27), the condition generating unit 35 generates a discriminable condition in which branch conditions satisfying the first criterion are combined (step S28).

FIG. 19 is a flowchart illustrating an operation example of a gray zone extraction process performed by the model updating apparatus 200 according to the present embodiment. The learning data generation unit 31 generates second learning data excluding the learning data corresponding to the discriminable condition from the first learning data (step S31). The model learning unit 32 generates a second hierarchical mixed model by heterogeneous machine learning using the generated second learning data (step S32).

The score calculation unit 33 calculates the ratio of (TP + FN) for each leaf node in the generated second hierarchical mixed model (step S33). The condition extraction unit 34 extracts a branch condition to the leaf node for which the second score has been calculated so as to satisfy the second criterion (step S34).

If the number of repetitions of the processing from step S32 to step S34 (that is, the processing from generation of the model to extraction of the branch condition) does not reach the predetermined number (No in step S35), steps S32 to S34 The processing up to is repeated. On the other hand, when the number of repetitions has reached the predetermined number (Yes in step S35), the condition generating unit 35 generates a difficult-to-discriminate condition combining conditions satisfying the second criterion (step S36).

FIG. 20 is a flowchart illustrating an operation example of gray data exclusion processing performed by the model updating apparatus 200 of the present embodiment. The learning data generation unit 31 generates third learning data from which the learning data corresponding to the difficult-to-discriminate condition is excluded from the second learning data (step S41).

FIG. 21 is a flowchart showing an operation example of the gray zone deep excavation processing performed by the model updating apparatus 200 of the present embodiment. The model learning unit 32 generates a first hierarchical mixture model using the generated third learning data (Step S51). The score calculation unit 33 calculates a first score (percentage of TP) for each leaf node in the first hierarchical mixed model generated using the third learning data (step S52). The condition extracting unit 34 extracts a branch condition to the leaf node for which the first score that does not satisfy the first criterion has been calculated (step S53). The model learning unit 32 uses the learning data classified into the leaf nodes from which the branch condition has been extracted, and generates a third hierarchical mixed model that branches conditionally under the leaf nodes (step S54).

As described above, in the present embodiment, the learning data generation unit 31 applies the reference teacher data to the discrimination model, compares the discrimination result with the label, sets the matched teacher data as a positive example, First learning data is generated using the data as a negative example. In addition, the model learning unit 32 generates a first hierarchical mixed model by heterogeneous machine learning using the generated first learning data, and the score calculation unit 33 determines the ratio of the TP for each leaf node (first score). Is calculated. The condition extraction unit 34 extracts a branch condition to the leaf node for which the first score that satisfies the first criterion has been calculated, and the condition generation unit 35 determines a identifiable condition obtained by combining the branch conditions that satisfy the first criterion. Generate.

Accordingly, it is possible to determine whether or not the data can be determined only by setting conditions using known explanatory variables. Further, as in the first embodiment, without replenishing the data or learning a new discriminant model, the discriminating condition of the existing discriminant model is maintained by selecting the data according to the filter condition. It can be said that the discrimination model has been updated so as to improve the discrimination accuracy while improving the discrimination accuracy.

Note that the model updating device 200 of the present embodiment may include the data replenishing unit 40 and the model generating unit 50 of the first embodiment. That is, the model updating apparatus 200 may perform supplementation of data or learning of a new discrimination model. According to such a configuration, the discrimination accuracy of the existing discrimination model can be further improved.

Also, in the present embodiment, the model learning unit 32 learns a hierarchical mixture model by heterogeneous mixture machine learning. Therefore, it is possible to extract data conditions that can be predicted almost completely with the learning data only, and data conditions that cannot be predicted completely with the learning data. More specifically, data corresponding to the former condition can be subjected to automatic determination by relying on the result of determination by the determination model, and data corresponding to the latter condition is difficult to determine by the determination model. Items can be individually determined. For example, in the case of a pre-examination of a mortgage loan, a case that can be completely distinguished from application information based on rules and a case that is preferably distinguished by a person can be separated.

Embodiment 3 FIG.
Next, a third embodiment of the model updating device according to the present invention will be described. In the first embodiment, the case where the model updating apparatus of the present invention is applied to a general discrimination problem including multi-value discrimination has been described. In the present embodiment, a description will be given focusing on a binary discrimination problem in which a clear rule exists behind, and 0/1 can be specified only by a branch condition.

(4) In the case of performing discrimination by machine learning, input information (explanatory variables) is all information used for discrimination. However, actually, the result is derived by other external factors (external factors). When the determination is made due to an external factor, it is difficult to make a complete determination using only input information.

Therefore, the data (TP, TN) for which the device for performing the determination using the discriminant model (hereinafter, referred to as AI (artificial {intelligence})) and the data for which the AI could not be determined (FP, FN) are utilized. . If the AI can be separated into data that can be determined only by input information and data that requires an external cause, the data that requires an external cause is excluded from the AI determination, and the AI determination result is almost 100% reliable. become.

Hereinafter, the following hypothesis is defined for the determination using AI (sometimes referred to as AI prediction).

1. The explanatory variables include "known variables" used in learning and "unknown variables" not used in learning.
2. Unknown variables are either present or not present at the time of analysis.
3. If the prediction target can be determined only by the known variables, the prediction target is completely correct (determined).
4. When the prediction target is affected by the unknown variable, (1) the prediction target does not completely answer correctly, and (2) even if the learning is performed a plurality of times, the accuracy deteriorates in the discrimination part of the unknown variable. That is, even if the branching is performed correctly with known variables, if an unknown variable appears at the end, the model created from the learning data will not completely match.
5. By repeatedly dividing and predicting the prediction target from various angles with known information a plurality of times, data that can be distinguished from known data is separated.

That is, there should be clear rules for credit-related tasks such as credit mentioned above. In other words, so-called 0/1 determination should be possible only by the condition for classifying the target data. On the other hand, so-called gray data in which 0/1 cannot be determined can be said to be data with arbitrariness, so that the user performs 0/1 determination separately rather than outputting any result using a determination model. Is preferred. In the model updated by the model updating apparatus of the present embodiment, it is possible to set a criterion for data (gray data) that cannot be clearly classified. The judgment can be automated.

Hereinafter, a specific example will be described in a case where the model updating apparatus of the present embodiment is used as an apparatus for updating a discriminating model for crediting a business partner. Here, a case where the discrimination model is updated using the model updating device of the first embodiment will be described.

The storage unit 10 stores, as an existing discriminant model, a discriminant for discriminating whether or not credit is given to a trading partner in a leaf node of the hierarchical mixed model, and performs binary branching based on an explanatory variable representing information on the credit trading partner. The model in which the condition is set to each node of the hierarchical mixed model is stored.

(4) The user determines the model that he / she wants to use for work from among a plurality of existing discrimination models generated in advance. In general, a user selects a model that matches not only the discrimination accuracy but also the branching condition and the prediction formula from the plurality of discrimination models to the operation.

FIG. 22 is an explanatory diagram showing an example of a discrimination model based on a hierarchical mixed model. In the discriminant model illustrated in FIG. 22, a double-frame rectangle is a root node and a node indicating a branch condition, and a normal rectangle is a leaf node indicating a discriminant (prediction formula). For example, a transaction partner whose age is less than 30 and whose loan balance is 10,000,000 or more is determined by the prediction formula of prediction formula number 1.

FIG. 22 illustrates the number of samples classified into each leaf node at the time of learning, evaluation, and prediction.

FIG. 23 is an explanatory diagram showing an example of displaying the properties of each discriminant. The graph illustrated in FIG. 23 shows the result of accumulating (adding) the weights (coefficients) of the explanatory variables in the case where each discriminant (predictive formula) is represented in a linear format for each predictive formula. For example, the output unit 70 may display the hierarchical mixture model in the format illustrated in FIG. 22 or may display the prediction formula in the format illustrated in FIG.

(4) The input unit 20 inputs data to be used for determining whether credit is permitted.

The 抽出 data extraction unit 30 tallies up the correct answer ratios in which the value of the labeled correct answer label is positive and the credit determination result is also positive among the business partner data classified into each leaf node. Then, the data extracting unit 30 extracts the partner data classified under the condition that the correct answer ratio is equal to or less than a predetermined threshold.

The data replenishment unit 40 accepts replenishment of at least one of adding an explanatory variable and updating a correct answer label to the extracted business partner data.

The model generation unit 50 generates a discrimination model by using the supplemented business partner data. As in the first embodiment, the model generation unit 50 may generate an arbitrary discrimination model.

The model updating unit 60 applies data that does not satisfy the conditions under which the extracted trading partner data is classified to the hierarchical mixed model, and generates a model indicating that data that satisfies the above conditions is applied to the discrimination model.

The output unit 70 outputs the generated discrimination model. The output unit 70 may output the generated discrimination model in a format exemplified in FIGS. 22 and 23 described above.

As described above, in the present embodiment, the model updating device updates the discriminating model for crediting the trading partner, so that while maintaining the discriminating conditions of the existing discriminating model, the trading partner to be manually confirmed is determined. The data (gray data) shown can be extracted.

More specifically, it is possible to extract data (that is, gray data) that cannot be determined by the explanatory variable used while maintaining the interpretability of the discriminant model and the like. By clarifying such data, it becomes possible to consider necessary explanatory variables.

FIG. 24 is an explanatory diagram showing an example of the result of classifying data using the discrimination model. In the example illustrated in FIG. 24, it is assumed that the data group D11 is classified into the leaf node of the prediction formula number 5 via the classification processing indicated by the thick arrow when the existing discrimination model M12 illustrated in FIG. 22 is used. . In the example shown in FIG. 24, since four incorrect data are included in the data group D11, the discrimination accuracy at the leaf node is reduced.

FIG. 25 is an explanatory diagram showing an example of a result of classifying data by the updated discrimination model. In the example shown in FIG. 25, as a result of updating the discrimination model illustrated in FIG. 24, a new discrimination model M11 is generated, and the discrimination model is updated as a whole including the new branch condition M13. According to the new branch condition M13, the data group D12 including data determined to be incorrect at the leaf node of the prediction formula number 5 is classified into the new discrimination model M11, so that the discrimination accuracy of the existing discrimination model M12 is improved. It becomes possible to do.

Next, the outline of the present invention will be described. FIG. 26 is a block diagram showing an outline of a model updating device according to the present invention. The model updating apparatus 80 according to the present invention is a model updating apparatus (for example, the model updating apparatus 100) that updates a hierarchical mixed model, and extracts data classified under a target condition in the hierarchical mixed model. A data extraction unit 81 (for example, the data extraction unit 30), a data replenishment unit 82 (for example, the data replenishment unit 40) that receives replenishment for the extracted data, and a model generation that generates a discriminant model using the replenished data A unit 83 (for example, the model generating unit 50) and a model updating unit 84 (for example, the model updating unit 60) that generates a model in which node nodes for classifying data satisfying a target condition are arranged at the top of the hierarchical mixed model. ).

The model updating unit 84 is a model indicating that the data that does not satisfy the condition is applied to the hierarchical mixed model corresponding to the leaf node for the node node, and the data that satisfies the condition is applied to the discriminant model corresponding to the leaf node for the node node. Generate

により With such a configuration, the discrimination model can be updated so as to improve the discrimination accuracy while maintaining the discrimination conditions of the existing discrimination model.

The model updating device 80 may include a determination result totalizing unit (for example, the data extracting unit 30) that totalizes the determination results of the data classified under each condition. Then, the data extracting unit 81 may extract data classified under the condition that the result of the data determination does not satisfy the criterion. With such a configuration, it is possible to specify and extract a condition portion where the explanatory variables are assumed to be insufficient.

{Also, as a specific mode, a condition of binary discrimination (for example, a clear rule) defined based on the explanatory variable may be set for each node of the hierarchical mixed model. At this time, the determination result totaling unit may total the correct answer ratio, which is the ratio of data that is truly positive and the determination result is also positive among the data classified under each condition. Then, the data extraction unit 81 extracts data classified under the condition that the correct answer ratio is equal to or less than a predetermined threshold, applies data that does not satisfy the condition to the hierarchical mixed model, and determines data that satisfies the condition as a discrimination model. May be generated.

As a further specific aspect, a discriminant for determining whether credit to a counterparty is permitted is set in a leaf node of the hierarchical mixed model, and a binary branching condition based on an explanatory variable representing information on credit counterparties is used as a hierarchical mixed condition. It may be set for each node of the model. At this time, the discrimination result tallying unit tallies the correct answer ratio in which the value of the labeled correct answer label is positive and the credit discrimination result is also positive among the partner data classified into each leaf node. The data extraction unit may extract partner data classified under the condition that the correct answer ratio is equal to or less than a predetermined threshold. Then, the data supplementing unit 82 accepts supplementation of at least one of the addition of the explanatory variable and the update of the correct answer label to the extracted business partner data, and the model generation unit 83 uses the supplemented business partner data. A model indicating that a discriminant model is generated, and the model updating unit 84 applies data that does not satisfy the condition under which the extracted trading partner data is classified to the hierarchical mixed model, and applies data that satisfies the condition to the discriminant model. May be generated.

(4) The model generation unit 83 may learn a discrimination model represented by a hierarchical mixed model. With such a configuration, it is possible to deeply determine the condition of the portion where the explanatory variable is assumed to be insufficient.

The model updating device 80 may include an output unit (for example, the output unit 70) that outputs information on the discrimination model. Then, the model generating unit 83 generates a plurality of types of discriminating models, and the output unit outputs the discriminating conditions and the discriminants (for example, the results illustrated in FIGS. 22 and 23) in the plural types of discriminating models. Good. With such a configuration, it is possible to present the content under the deeply digged condition to the user for selection.

Specifically, the data supplementing unit 82 may receive addition of an explanatory variable or update of a teacher label to the extracted data.

(4) The data extracting unit 81 may extract data classified under the target condition in the model generated by the model updating unit, and the data replenishing unit 82 may receive replenishment for the extracted data. Further, the model generation unit 83 generates another discrimination model using the supplemented data, and the model update unit 84 generates a model indicating that data satisfying a target condition is applied to another discrimination model. May be. As described above, by generating the repetitive discrimination model, it is possible to further improve the accuracy of the existing discrimination model.

FIG. 27 is a schematic block diagram showing a configuration of a computer according to at least one embodiment. The computer 1000 includes a processor 1001, a main storage device 1002, an auxiliary storage device 1003, and an interface 1004.

モデル The above-described model updating device is implemented in the computer 1000. The operation of each processing unit described above is stored in the auxiliary storage device 1003 in the form of a program (model update program). The processor 1001 reads out the program from the auxiliary storage device 1003, expands the program in the main storage device 1002, and executes the above processing according to the program.

In at least one embodiment, the auxiliary storage device 1003 is an example of a non-transitory tangible medium. Other examples of non-transitory tangible media include a magnetic disk, a magneto-optical disk, a CD-ROM (Compact Disc Read-only memory), a DVD-ROM (Read-only memory), A semiconductor memory and the like are included. When the program is distributed to the computer 1000 via a communication line, the computer 1000 that has received the program may load the program into the main storage device 1002 and execute the above-described processing.

The program may be for realizing a part of the functions described above. Further, the program may be a program that realizes the above-described function in combination with another program already stored in the auxiliary storage device 1003, that is, a so-called difference file (difference program).

一部 A part or all of the above-described embodiment can be described as in the following supplementary notes, but is not limited to the following.

(Supplementary Note 1) A model updating apparatus that updates a hierarchical mixed model, the data updating unit extracting data classified under a target condition in the hierarchical mixed model, and a supplement for the extracted data. A model in which a data replenishing unit to be accepted, a model generating unit that generates a discriminant model using the replenished data, and a node node for classifying data that satisfies the target condition are arranged at the top of the hierarchical mixed model. A model updating unit for generating, wherein the model updating unit applies data that does not satisfy the condition to the hierarchical mixed model that is a leaf node for the node node, and the discrimination model is a leaf node for the node node. A model updating apparatus for generating a model indicating that data satisfying the above condition is applied to the model.

(Supplementary Note 2) A judgment result summation unit that sums up the judgment results of the data classified under each condition is provided, and the data extraction unit extracts the data classified under the condition that the judgment result of the data does not satisfy the criterion. The model updating device according to supplementary note 1 to be extracted.

(Supplementary Note 3) Binary discrimination conditions defined based on the explanatory variables are set for each node of the hierarchical mixed model, and the discrimination result tallying unit determines whether the data classified under each condition is true. Positive, and the determination result is also tabulated the correct answer ratio that is the ratio of data that is also positive, the data extraction unit, the correct answer ratio is extracted data classified under the condition of a predetermined threshold or less, 3. The model updating apparatus according to claim 2, wherein the model updating unit applies data that does not satisfy the condition to a hierarchical mixed model and generates a model indicating that data that satisfies the condition is applied to a discriminant model.

(Supplementary Note 4) A discriminant for judging whether or not credit is given to the trading partner is set in the leaf node of the hierarchical mixed model, and the binary branching condition based on the explanatory variable representing the information on the credit trading partner is set in the hierarchical mixed model. Is set in each node of the answer table, and the determination result summarizing unit determines that the value of the labeled correct answer label is positive among the partner data classified into each leaf node, and that the determination result of the credit is also correct. Aggregating the ratios, the data extracting unit extracts the partner data classified under the condition that the correct answer ratio is equal to or less than a predetermined threshold, and the data replenishing unit explains the extracted partner data. Receiving a supplement of at least one of adding a variable and updating a correct answer label, the model generating unit generates a discrimination model using the supplemented partner data, and the model updating unit executes the extracted transaction. Applying the data that does not meet the conditions for hand data is classified into a hierarchical mixture model, the model updating device according to Note 2 or appendix 3, wherein generating a model illustrating applying the satisfying data to determine model.

(Supplementary note 5) The model updating device according to any one of Supplementary notes 1 to 4, wherein the model generation unit learns a discrimination model represented by a hierarchical mixed model.

(Supplementary Note 6) An output unit that outputs information of the discrimination model is provided, the model generation unit generates a plurality of types of discrimination models, and the output unit outputs discrimination conditions and discriminants in the plurality of types of discrimination models. The model updating device according to any one of supplementary notes 1 to 5.

(Supplementary note 7) The model updating device according to any one of Supplementary notes 1 to 6, wherein the data supplementing unit receives addition of an explanatory variable or update of a teacher label to the extracted data.

(Supplementary Note 8) The data extraction unit extracts data classified under the target condition in the model generated by the model update unit, the data replenishment unit receives replenishment for the extracted data, and the model generation unit Generates another discriminant model using the supplemented data, and the model updating unit generates a model indicating that data satisfying the target condition is applied to the other discriminant model. 8. The model updating device according to any one of 7 above.

(Supplementary Note 9) A model updating method for updating a hierarchical mixed model, the method including extracting data classified under a target condition in the hierarchical mixed model, receiving replenishment for the extracted data, and replenishing the extracted data. Generating a discriminant model using the extracted data, generating a model in which node nodes for classifying data satisfying the target condition are arranged at the top of the hierarchical mixed model, and generating the model, Applying data that does not satisfy the condition to the hierarchical mixed model that is a leaf node for a node, and generating a model indicating that data that satisfies the condition is applied to the discriminant model that is a leaf node for the node node. A model updating method characterized by the following.

(Supplementary note 10) The model updating method according to supplementary note 9, wherein the determination results of the data classified under each condition are totaled, and the data classified under the condition that the data determination result does not satisfy the criteria is extracted.

(Supplementary Note 11) A model update program applied to a computer that updates a hierarchical mixed model, the data updating processing extracting data classified under a target condition in the hierarchical mixed model to the computer. A data replenishment process for receiving replenishment for extracted data, a model generation process for generating a discriminant model using the replenished data, and a node node for classifying data satisfying the target condition. Executing a model update process for generating a model arranged at the top of the model node, applying data that does not satisfy the condition to the hierarchical mixed model that is a leaf node for the node node in the model update process, Indicates that data that satisfies the condition is applied to the discriminant model that is a leaf node for Model update program for generating the Dell.

(Supplementary Note 12) The computer is caused to execute a discrimination result summarization process for summarizing the discrimination results of the data classified under each condition, and in the data extraction process, the data is classified under the condition that the discrimination result does not satisfy the criterion. The model update program according to supplementary note 11, which causes the extracted data to be extracted.

Although the present invention has been described with reference to the exemplary embodiments and examples, the present invention is not limited to the exemplary embodiments and examples. Various changes that can be understood by those skilled in the art can be made to the configuration and details of the present invention within the scope of the present invention.

This application claims priority based on Japanese Patent Application No. 2018-158155 filed on Aug. 27, 2018, the entire disclosure of which is incorporated herein.

DESCRIPTION OF SYMBOLS 10 Storage part 20 Input part 30 Data extraction part 40 Data supplement part 50 Model generation part 60 Model update part 70 Output part 100 Model update device

Claims

A model updating device for updating a hierarchical mixed model,
A data extraction unit that extracts data classified under the condition to be targeted in the hierarchical mixed model,
A data replenishment unit that receives replenishment for the extracted data,
A model generation unit that generates a discriminant model using the supplemented data,
A model updating unit that generates a model in which a node that classifies data that satisfies the target condition is arranged at the top of the hierarchical mixed model,
The model updating unit applies data that does not satisfy the condition to the hierarchical mixed model corresponding to a leaf node for the node node, and transmits data that satisfies the condition to the discriminant model corresponding to a leaf node for the node node. A model updating device for generating a model indicating application.
A determination result totaling unit that totals the determination results of the data classified under each condition is provided,
The model updating device according to claim 1, wherein the data extracting unit extracts data classified under a condition where the determination result of the data does not satisfy a criterion.
Binary discrimination conditions defined based on explanatory variables are set for each node of the hierarchical mixed model,
The determination result totaling unit totals a correct answer ratio, which is a ratio of data that is truly positive among the data classified under each condition and the determination result is also positive,
The data extraction unit extracts the data classified under the condition that the correct answer ratio is equal to or less than a predetermined threshold,
The model updating device according to claim 2, wherein the model updating unit applies data that does not satisfy the condition to a hierarchical mixed model, and generates a model indicating that data that satisfies the condition is applied to a discriminant model.
A discriminant for determining whether credit is possible for the counterparty is set in the leaf node of the hierarchical mixed model, and a binary branching condition based on an explanatory variable representing information about the credit counterparty is set for each node of the hierarchical mixed model. Is set,
The discrimination result totaling unit tallies the correct answer ratio in which the value of the labeled correct answer label is positive and the credit discrimination result is also positive among the partner data classified into each leaf node,
The data extraction unit extracts the partner data classified under the condition that the correct answer ratio is equal to or less than a predetermined threshold,
The data replenishment unit accepts replenishment of at least one of addition of explanatory variables and update of the correct answer label to the extracted partner data,
The model generation unit generates a discrimination model using the supplemented counterparty data,
The model updating unit applies data that does not satisfy a condition under which the extracted trading partner data is classified to a hierarchical mixed model, and generates a model indicating that data that satisfies the condition is applied to a discrimination model. The model updating device according to claim 2 or 3.
The model updating device according to claim 1, wherein the model generation unit learns a discrimination model represented by a hierarchical mixed model.
An output unit that outputs information of the discrimination model is provided,
The model generation unit generates a plurality of types of discrimination models,
The model updating device according to claim 1, wherein the output unit outputs a discriminant condition and a discriminant in the plurality of types of discriminant models.
The model updating device according to any one of claims 1 to 6, wherein the data supplementing unit receives addition of an explanatory variable or update of a teacher label for the extracted data.
The data extraction unit extracts data classified under the target condition in the model generated by the model update unit,
The data replenishment unit receives replenishment for the extracted data,
The model generation unit generates another discrimination model using the supplemented data,
The model updating device according to any one of claims 1 to 7, wherein the model updating unit generates a model indicating that data satisfying the target condition is applied to the another discriminant model.
A model updating method for updating a hierarchical mixed model,
Extract data classified under the conditions of interest in the hierarchical mixed model,
Accept replenishment for the extracted data,
Generate a discriminant model using the supplemented data,
Generate a model in which a node node for classifying data satisfying the target condition is arranged at the top of the hierarchical mixed model,
When generating the model, data that does not satisfy the condition is applied to the hierarchical mixed model corresponding to a leaf node for the node node, and data that satisfies the condition is applied to the discriminant model corresponding to a leaf node for the node node. A model updating method, characterized by generating a model indicating that a model is applied.
The model updating method according to claim 9, wherein the determination results of the data classified under each condition are totaled, and the data classified under the condition that the determination result of the data does not satisfy a criterion is extracted.
A model update program applied to a computer that updates a hierarchical mixed model,
On the computer,
A data extraction process for extracting data classified under the target condition in the hierarchical mixed model;
Data replenishment processing to accept replenishment for extracted data,
A model generation process of generating a discriminant model using the supplemented data, and
Causing the node node that classifies the data satisfying the target condition to execute a model update process of generating a model in which the node is arranged at the top of the hierarchical mixed model;
In the model update process, data that does not satisfy the condition is applied to the hierarchical mixed model corresponding to a leaf node for the node, and data that satisfies the condition is applied to the discriminant model corresponding to a leaf node for the node. A model update program to generate a model indicating that it applies.
On the computer,
The discrimination result summarization process of summing up the judgment results of the data classified under each condition is executed,
The model update program according to claim 11, wherein the data extraction processing causes data classified under a condition that a result of the data determination does not satisfy a standard to be extracted.