JP6896590B2 - Predictive model maintenance system, predictive model maintenance method and predictive model maintenance program - Google Patents

Description

The present invention relates to a predictive model maintenance system, a predictive model maintenance method, and a predictive model maintenance program.

A method is known in which a prediction model is used to predict whether or not a predetermined requirement is satisfied after the process is performed, based on a predetermined condition in the process. For example, there is known a method of predicting the apparatus state of a plasma processing apparatus and the processing result of the plasma processing apparatus using a model created via a multivariate analysis program (see, for example, Patent Document 1).

Japanese Unexamined Patent Publication No. 2004-039805

Here, changes related to the process, that is, changes in the process such as deterioration, repair, and design change of the equipment or equipment performing the process, and changes including improvement of the process itself, changes in the material in which the process is executed, and the process. There is a problem that the accuracy of prediction using the same prediction model deteriorates due to changes in the environment in which the above is executed. Therefore, Patent Document 1 attempts to deal with this problem by creating a new model based on the residual of the detection data obtained by using the model.

However, the method of Patent Document 1 may not sufficiently cope with the problem that the accuracy of prediction deteriorates because the accuracy of prediction of the new model may not exceed that of the previous model.

The present invention has been made in view of the above, and provides a predictive model maintenance system, a predictive model maintenance method, and a predictive model maintenance program that sufficiently reduces the deterioration of the prediction accuracy due to changes in the process. The purpose is to provide.

In order to solve the above-mentioned problems and achieve the purpose, the prediction model maintenance system is a prediction model maintenance system that maintains the prediction probability of the prediction model that predicts whether or not the predetermined requirements are satisfied based on the predetermined conditions. The control unit includes a control unit, and the control unit uses a learning data creation unit that creates learning data used for creating a new prediction model and the learning data created by the learning data creation unit. A new prediction model creation unit that creates a prediction model, an evaluation data creation unit that creates evaluation data used to evaluate the prediction model and the new prediction model, and the evaluation data created by the evaluation data creation unit. It is characterized by including an operation prediction model selection unit that evaluates the prediction model and the new prediction model and selects the one with the higher evaluation as the operation prediction model.

According to this configuration, the newly created prediction model and the prediction model originally used are evaluated, and the one with the higher evaluation is selected as the operation prediction model, so that the prediction model originally used is used. Since it is not necessary to newly use the one whose prediction accuracy has deteriorated, it is possible to sufficiently reduce the deterioration of the prediction accuracy due to the change in the process.

In this configuration, a deterioration detection data creation unit that creates deterioration detection data used to detect whether or not the prediction probability of the prediction model has deteriorated, and the deterioration detection data created by the deterioration detection data creation unit. Further includes a deterioration detection determination unit that determines whether or not the prediction probability of the prediction model has decreased, and the learning data creation unit is such that the deterioration detection determination unit predicts the prediction model. When it is determined that the probability is low, it is preferable to create the training data. According to this configuration, a new prediction model is created only when it is determined that the prediction model currently in use has deteriorated. It is possible to sufficiently reduce the deterioration of accuracy.

In these configurations, when the higher evaluation is equal to or higher than the evaluation threshold, the operation prediction model selection unit selects the higher evaluation as the operation prediction model, and the higher evaluation is less than the evaluation threshold. If so, it is preferable not to select any of them as the operation prediction model. According to this configuration, since the prediction model for evaluation below the threshold value is not selected as the operation prediction model, it is possible to reduce the adverse effect on the process due to the use of the prediction model with low prediction accuracy.

In these configurations, it is preferable that the predetermined conditions are determined based on the equipment data in the process, and the predetermined requirement is that the quality data of the product that has undergone the process is within the predetermined range. According to this configuration, equipment data and quality data generally used for process control and evaluation can be preferably used, so that the accuracy of prediction deteriorates efficiently due to changes in the process. It can be sufficiently reduced.

In these configurations, the prediction model and the new prediction model are preferably set to issue an alert when there is a possibility that a predetermined requirement is not satisfied. According to this configuration, the alert can alert the user that there is a possibility that the predetermined requirement is not met, so that there is a sufficient possibility that the case where the predetermined requirement is not met in the process is overlooked. Can be reduced.

In these configurations, it is preferable that the learning data creation unit creates the learning data using data acquired in the past in time series rather than the data used by the evaluation data creation unit to create the evaluation data. According to this configuration, the new prediction model created based on the training data created using the data acquired in the past in the time series can also be used in the evaluation data created using the data acquired in the future in the time series. Since it is evaluated whether or not the prediction is possible with sufficient accuracy, it is possible to create a new prediction model having sufficient prediction accuracy in both the training data and the evaluation data. Therefore, it is possible to efficiently reduce the deterioration of the prediction accuracy due to the change in the process.

In a configuration in which the training data creation unit creates training data using data acquired in the past in a time series rather than the data used by the evaluation data creation unit to create the evaluation data, the training data creation unit creates the evaluation data. It is preferable to create the training data using the same amount or more data than the data used by the unit to create the evaluation data. According to this configuration, it is possible to use a larger ratio of data among the data for which the new prediction model is considered to have sufficient prediction accuracy for creating the new prediction model. Therefore, it is possible to more efficiently reduce the deterioration of the prediction accuracy due to changes in the process.

In a configuration in which the training data creation unit creates training data using the same amount or more data than the data used by the evaluation data creation unit to create the evaluation data, the training data creation unit is the evaluation data creation unit. For one type of evaluation data created by, a plurality of types of the training data are created using a plurality of types of data, and the new prediction model creation unit uses the plurality of types of the training data. It is preferable to create a plurality of types of the new prediction models. According to this configuration, a plurality of types of training data can be created at one time, and a plurality of types of new prediction models can be created. Therefore, if there is a change in the process while the data used for creating the training data is acquired, a new prediction model more suitable for the current process can be created. It is possible to further sufficiently reduce the deterioration of the prediction accuracy.

In order to solve the above-mentioned problems and achieve the purpose, the prediction model maintenance method is a prediction model maintenance method that maintains the prediction probability of the prediction model that predicts whether or not a predetermined requirement is satisfied, and is a new prediction model. A training data creation step for creating training data used to create a new prediction model, a new prediction model creation step for creating a new prediction model using the training data created in the training data creation step, and the prediction model. The prediction model and the new prediction model are evaluated using the evaluation data creation step for creating the evaluation data used for evaluating the new prediction model and the evaluation data created in the evaluation data creation step. It is characterized by having an operation prediction model selection step in which the one with the highest evaluation is selected as the operation prediction model.

According to this configuration, the newly created prediction model and the prediction model originally used are evaluated, and the one with the higher evaluation is selected as the operation prediction model, so that the prediction model originally used is used. Since it is not necessary to newly use the one whose prediction accuracy has deteriorated, it is possible to sufficiently reduce the deterioration of the prediction accuracy due to the change in the process.

In order to solve the above-mentioned problems and achieve the purpose, the prediction model maintenance program maintains the prediction model to maintain the prediction probability of the prediction model that predicts whether or not the predetermined requirements are satisfied based on the predetermined conditions. A prediction model maintenance program for causing a computer to execute a method, in which a learning data creation step for creating training data used for creating a new prediction model and the training data created in the training data creation step are performed on the computer. Created in the new prediction model creation step for creating the new prediction model, the evaluation data creation step for creating the evaluation data used to evaluate the prediction model and the new prediction model, and the evaluation data creation step. Using the evaluation data obtained, the prediction model and the new prediction model are evaluated, and the operation prediction model selection step of selecting the one with the higher evaluation as the operation prediction model is executed.

According to this configuration, the newly created prediction model and the prediction model originally used are evaluated, and the one with the higher evaluation is selected as the operation prediction model, so that the prediction model originally used is used. Since it is not necessary to newly use the one whose prediction accuracy has deteriorated, it is possible to sufficiently reduce the deterioration of the prediction accuracy due to the change in the process.

According to the present invention, it is possible to provide a predictive model maintenance system, a predictive model maintenance method, and a predictive model maintenance program that sufficiently reduce deterioration of prediction accuracy due to changes in the process.

FIG. 1 is an explanatory diagram illustrating the properties of the prediction model used in the prediction model maintenance system according to the embodiment of the present invention. FIG. 2 is an explanatory diagram illustrating the quality of the prediction model used in the prediction model maintenance system according to the embodiment of the present invention. FIG. 3 is a schematic configuration diagram of a prediction model maintenance system according to an embodiment of the present invention. FIG. 4 is a flowchart of a prediction model maintenance method according to an embodiment of the present invention. FIG. 5 is an explanatory diagram illustrating the details of the deterioration detection data creation step of FIG. FIG. 6 is an explanatory diagram illustrating an example of a prediction model that receives a determination in the deterioration detection determination step of FIG. FIG. 7 is an explanatory diagram illustrating an example of data processing in the deterioration detection determination step of FIG. FIG. 8 is an explanatory diagram illustrating an example of a determination criterion in the deterioration detection determination step of FIG. FIG. 9 is an explanatory diagram illustrating an example of a determination result in the deterioration detection determination step of FIG. FIG. 10 is an explanatory diagram illustrating the details of the learning data creation step of FIG. FIG. 11 is an explanatory diagram illustrating an example of extraction of a prediction rule in the new prediction model creation step of FIG. FIG. 12 is an explanatory diagram illustrating an example of evaluation of the prediction rule in the new prediction model creation step of FIG. FIG. 13 is an explanatory diagram illustrating the details of the evaluation data creation step of FIG. FIG. 14 is an explanatory diagram illustrating an example of a prediction model and a new prediction model to be evaluated in the operation prediction model selection step of FIG. FIG. 15 is an explanatory diagram illustrating an example of evaluation of the prediction model and the new prediction model in the operation prediction model selection step of FIG. FIG. 16 is an explanatory diagram illustrating an example of comparison of evaluation of the prediction model and the new prediction model in the operation prediction model selection step of FIG. FIG. 17 is an explanatory diagram illustrating an example of the operation prediction model selected in the operation prediction model selection step of FIG.

Hereinafter, embodiments according to the present invention will be described in detail with reference to the drawings. The present invention is not limited to this embodiment. In addition, the components in the embodiment include those that can be easily replaced by those skilled in the art, or those that are substantially the same. Furthermore, the components described below can be combined as appropriate.

[Embodiment]
FIG. 1 is an explanatory diagram illustrating the properties of the prediction model 1 used in the prediction model maintenance system 10 (see FIG. 3) according to the embodiment of the present invention. As shown in FIG. 1, in the prediction model 1, when the equipment data 21 (see FIG. 3) and the quality data 22 (see FIG. 3) that satisfy the predetermined conditions in the predetermined process are input, the product that has undergone this process is subjected to this process. It predicts that the quality data 22 (see FIG. 3) is within the desired predetermined range, that is, is likely to meet the predetermined requirement, and outputs that the quality data 22 is likely to meet the predetermined requirement. On the other hand, in the prediction model 1, as shown in FIG. 1, when the equipment data 21 (see FIG. 3) and the quality data 22 (see FIG. 3) that do not satisfy the predetermined conditions are input in the predetermined process, this process is performed. It is predicted that the quality data 22 (see FIG. 3) of the product that has passed through the above is not within the desired predetermined range, that is, it is highly likely that the predetermined requirement is not satisfied, and the output that the predetermined requirement is not satisfied is likely to be output. To do. The prediction model 1 is created by using the equipment data 21 (see FIG. 3) and the quality data 22 (see FIG. 3) as explanatory variables and the quality data 22 (see FIG. 3) as objective variables. The quality data 22 (see FIG. 3) is data that can be both an explanatory variable and an objective variable.

FIG. 2 is an explanatory diagram illustrating the quality of the prediction model 1 used in the prediction model maintenance system 10 (see FIG. 3) according to the embodiment of the present invention. Prediction Model 1, as indicated by a broken line L2 in FIG. 2, model quality showing the accuracy of prediction of the prediction model 1, the initial value is Q _F, with the passage of time, operates a predictive model 1 Process changes, that is, process changes such as deterioration, repair, and design changes of the equipment or equipment that executes the process, as well as changes that include improvement of the process itself, changes in the materials under which the process is performed, and execution of the process. It tends to gradually decrease due to changes in the environment. Here, the model quality, for example, used predictability predictive model 1, in this case, Q _F, depending on the nature of the process or the like, 85%, 90%, determined at 95%, etc..

Therefore, the prediction model 1 is revised at predetermined time intervals by using the prediction model maintenance system 10 (see FIG. 3) according to the embodiment of the present invention, and as shown in the solid line L1 of FIG. 2, the prediction model 1 model quality indicating the accuracy of the first prediction is, so as not to less than Q _T is a predetermined threshold value, can be maintained. That is, in the prediction model maintenance system 10 (see FIG. 3) according to the embodiment of the present invention, the prediction model 1 is revised at predetermined time intervals, so that the model quality indicating the prediction accuracy of the prediction model 1 is predetermined. It can be maintained so that it does not _fall below the threshold value of QT. Q _T, when the prediction probability prediction model 1 is used as a model quality, depending on the nature of the process or the like, 80% is defined to 85% or the like.

Here, the process using the prediction model 1 includes each manufacturing process of the turbine moving blade used for the aircraft engine, but the process is not limited to this, and the equipment data 21 which has a certain number of mass production or more and can be acquired. (See FIG. 3) and quality data 22 (see FIG. 3) may be used in any step as long as a sufficient amount can be obtained so as to be statistically superior. Here, the process using the prediction model 1 is preferably finely separated with respect to the entire process. In this case, the correlation between the equipment data 21 (see FIG. 3) and the quality data 22 (see FIG. 3). Therefore, the model quality of the prediction model 1 used in each process can be maintained high.

An example of the prediction model 1 is a set of prediction rules having a certain level of prediction accuracy or higher. The prediction rule is created by performing analysis processing using equipment data 21 (see FIG. 3) and quality data 22 (see FIG. 3) as explanatory variables and quality data 22 (see FIG. 3) as objective variables. Is. Here, the analysis process is one of determination tree analysis, analysis by Support Vector Machine (SVM), discriminant analysis, logistic regression analysis, and analysis by Mahalanobis Taguchi Method (MT method). Processing based on the analysis method is exemplified. In the present embodiment, a case in which the prediction model 1 is created by performing an analysis process based on the decision tree analysis will be mainly described in detail, but the present invention is not limited to this, and the prediction model 1 is exemplified above. Any analysis process including the above can be preferably used.

FIG. 3 is a schematic configuration diagram of the prediction model maintenance system 10 according to the embodiment of the present invention. As shown in FIG. 3, the prediction model maintenance system 10 includes a control unit 11. As shown in FIG. 3, the control unit 11 operates with a deterioration detection data creation unit 12, a deterioration detection determination unit 13, a learning data creation unit 14, a new prediction model creation unit 15, and an evaluation data creation unit 16. Prediction model selection unit 17 and the like are included. The control unit 11, the database 20, and the prediction unit 30 are electrically connected to each other so as to be able to communicate with each other.

The database 20 is a storage device such as a RAM, a ROM, and a flash memory, and stores equipment data 21 and quality data 22 as shown in FIG. The equipment data 21 is historical data or the like related to the environment of the process, and various physical quantities acquired for the control of the process such as temperature and pressure are exemplified. The quality data 22 is data related to the quality of the product that has undergone this process, and exemplifies various physical quantities acquired for controlling the quality of the product, such as size, shape, and strength. The quality data 22 also includes information on whether or not the various physical quantities contained in the quality data 22 are within a desirable predetermined range, in accordance with these various physical quantities. The equipment data 21 and the quality data 22 are both traced based on the serial number, are associated with each other, and are stored in the database 20. Specifically, the quality of the product manufactured through the process when the serial number is the equipment data 21 of XXX is the quality data 22 of XXX having the same serial number.

As shown in FIG. 3, the database 20 is electrically connected to the control unit 11 in the prediction model maintenance system 10 so as to be capable of information communication. The database 20 transmits the equipment data 21 and the quality data 22 in response to the request from the control unit 11. As shown in FIG. 3, the database 20 is electrically connected to the prediction unit 30 so as to be capable of information communication. The database 20 transmits the equipment data 21 in response to a request from the prediction unit 30.

The database 20 is preferably electrically connected to equipment in a process (not shown) so that information and communication can be performed. In this case, equipment data 21 can be acquired and stored in real time from the equipment in this process. Further, it is preferable that the database 20 is electrically connected to a quality control facility (not shown) so that information and communication can be performed. In this case, the quality data 22 can be acquired and accumulated in real time from the quality control facility. it can. The database 20 does not have to be electrically connected to equipment in a process (not shown) and equipment for quality control so as to be able to communicate with each other. It is preferable to update in real time.

As shown in FIG. 3, the prediction unit 30 is electrically connected to the control unit 11 in the prediction model maintenance system 10 so as to be capable of information communication, and receives the prediction model 1 used for prediction from the control unit 11. As shown in FIG. 3, the prediction unit 30 is electrically connected to the database 20 so as to be capable of information communication, and receives the equipment data 21 used for prediction from the database 20. The prediction unit 30 is not limited to acquiring the equipment data 21 from the database 20, and may directly acquire the equipment data 21 from equipment in a process (not shown).

The prediction unit 30 uses the prediction model 1 to predict the quality data 22 that may be obtained based on the equipment data 21 received from the database 20. The prediction unit 30 uses the prediction model 1 to determine whether or not the predicted quality data 22 satisfies a predetermined condition, and outputs the determination result as a prediction result by various methods.

As shown in FIG. 3, the prediction unit 30 includes an alert unit 32. In the present embodiment, the prediction unit 30 outputs a prediction result to that effect only when the predicted quality data 22 does not satisfy a predetermined condition by issuing a command to the alert unit 32 to issue an alert. When the prediction unit 32 uses the prediction model 1 to predict that the quality data 22 of the product that has undergone the process is not within the desired predetermined range, that is, it is highly likely that the quality data 22 does not meet the predetermined requirements. Issue an alert. Examples of the alert unit 32 include a sound alarm that issues an alert by sound, an optical alarm that issues an alert by lighting or blinking light, a compound alarm that issues an alert by sound and light, and the like. When the prediction unit 30 operates each prediction model 1 for a plurality of processes, the alert unit 32 alerts according to the prediction model 1 caused by predicting that there is a high possibility that the predetermined requirements are not satisfied. You can change the type of. In this case, the alert unit 32 can output what kind of situation has occurred in which process depending on the type of alert.

The prediction unit 30 includes a storage unit and a processing unit. A computer is exemplified as the prediction unit 30. The storage unit has, for example, a storage device such as a RAM, a ROM, and a flash memory, and stores a software program processed by the processing unit, data referenced by the software program, and the like. The storage unit also functions as a storage area in which the processing unit temporarily stores the processing result and the like. The processing unit reads a software program or the like from the storage unit and processes it, so that the function corresponding to the content of the software program, specifically, the prediction model 1 received from the control unit 11 is used from the database 20. The function of the prediction unit 30 is to predict the quality data 22 that may be obtained based on the received equipment data 21, and to predict whether or not the predicted quality data 22 satisfies a predetermined condition.

The deterioration detection data creation unit 12 creates deterioration detection data which is data used for detecting whether or not the prediction probability as the model quality of the prediction model 1 has deteriorated. Specifically, the deterioration detection data creation unit 12 adopts the equipment data 21 and the quality data 22 acquired within a certain period until the timing of creating the deterioration detection data, and is based on the adopted equipment data 21 and the quality data 22. And create deterioration detection data.

The deterioration detection determination unit 13 determines whether or not the prediction probability of the prediction model 1 is reduced by using the deterioration detection data created by the deterioration detection data creation unit 12. Specifically, the deterioration detection determination unit 13 first determines, in the equipment data 21, whether or not there is a change in the tendency, which is a probability of not satisfying a predetermined condition, for each prediction rule adopted in the prediction model 1. Next, the deterioration detection determination unit 13 satisfies a predetermined requirement for each prediction rule adopted in the prediction model 1 in the quality data 22 associated with the equipment data 21 that does not satisfy the predetermined condition of the prediction model 1. It is determined whether or not there is a change in accuracy, which is a non-probability. After that, the deterioration detection determination unit 13 determines that the prediction probability of the prediction model 1 has not decreased only when it is determined that there is no change in the tendency or no change in the accuracy in all the prediction rules. In other cases, it is determined that the prediction probability of the prediction model 1 is low, that is, deterioration is detected.

The learning data creation unit 14 creates learning data to be used for creating a new prediction model. Here, the new prediction model is generally a model different from the prediction model 1, but may be the same model as the prediction model 1. The learning data creation unit 14 may create learning data only when the deterioration detection determination unit 13 determines that the prediction probability of the prediction model 1 is low, or the deterioration detection determination unit 13 may create the prediction model 1. The learning data may be created regardless of whether or not the prediction probability of the above is lowered. Specifically, the learning data creation unit 14 adopts the equipment data 21 and the quality data 22 acquired in the past for a certain period or more in chronological order from the timing of creating the learning data, and the adopted equipment data 21 and the quality data 22 Create training data based on. The equipment data 21 and the quality data 22 acquired within a certain period until the timing of creating the learning data are used for creating the evaluation data by the evaluation data creating unit 16 described later. It is preferable that the training data creation unit 14 creates training data using the equipment data 21 and the quality data 22 in the same amount or more than the equipment data 21 and the quality data 22 used for the evaluation data. Further, it is preferable that the learning data creation unit 14 creates a plurality of types of learning data by using a plurality of types of equipment data 21 and quality data 22 in one learning data creation.

The new prediction model creation unit 15 creates a new prediction model using the learning data created by the learning data creation unit 14. Specifically, the new prediction model creation unit 15 first extracts prediction rules that may be adopted in the new prediction model. Next, the new prediction model creation unit 15 calculates a tendency of probability that a predetermined condition is not satisfied for each extracted prediction rule. Next, the new prediction model creation unit 15 calculates the accuracy, which is the probability of not satisfying the predetermined requirement, in the quality data 22 associated with the equipment data 21 that does not satisfy the predetermined condition for each extracted prediction rule. .. After that, the new prediction model creation unit 15 determines the prediction rule having a high tendency value and a high accuracy value as the prediction rule to be adopted in the new prediction model. If there is a prediction rule that is decided to be adopted in the new prediction model after that, the new prediction model creation unit 15 creates one or a plurality of types of new prediction models by combining these prediction rules. When the learning data creation unit 14 creates a plurality of types of learning data, the new prediction model creation unit 15 preferably creates a plurality of types of new prediction models using each type of learning data.

The evaluation data creation unit 16 creates evaluation data used for evaluating the prediction model 1 and the new prediction model created by the new prediction model creation unit 15. Specifically, the evaluation data creation unit 16 adopts the equipment data 21 and the quality data 22 acquired in the future in chronological order rather than the equipment data 21 and the quality data 22 adopted as the learning data, and adopts the equipment data 21 and the adopted equipment data 22. Evaluation data is created based on the quality data 22. In detail, the evaluation data creation unit 16 uses the equipment data 21 and the quality data 22 acquired within a certain period until the timing of creating the evaluation data to create the evaluation data. The evaluation data creation unit 16 creates evaluation data using the equipment data 21 and the quality data 22 in the same amount or less than the equipment data 21 and the quality data 22 used for the training data created by the training data creation unit 14. It is preferable to do so.

The operation prediction model selection unit 17 evaluates the prediction model 1 and the new prediction model using the evaluation data created by the evaluation data creation unit 16, and selects the one with the higher evaluation as the operation prediction model. Specifically, the operation prediction model selection unit 17 first uses the evaluation data created by the evaluation data creation unit 16 for the prediction model 1 and the new prediction model created by the new prediction model creation unit 15. To calculate the prediction probability. Next, the operation prediction model selection unit 17 selects the prediction model 1 or the new prediction model having the highest calculated prediction probability as the operation prediction model newly operated by the prediction unit 30.

When the higher-evaluated prediction model 1 or the new prediction model is equal to or higher than the evaluation threshold, the operation prediction model selection unit 17 selects the higher-evaluation prediction model 1 or the new prediction model as the operation prediction model and evaluates it. If the higher prediction model 1 or the new prediction model is less than the evaluation threshold, neither the prediction model 1 or the new prediction model may be selected as the operational prediction model. Operation prediction model selection unit 17, specifically, the highest when predictability is the predetermined threshold value Q _T or more as shown in FIG. 2, predictive model 1 or new predictive model of higher rating of the calculated predictability It was selected as operational prediction model, when the highest predictability of the calculated prediction probability is below a predetermined threshold Q _T, and will not be selected in any of the predictive model 1 or new predictive model production prediction model.

In the state where the operation prediction model is not selected by the operation prediction model selection unit 17, the training data creation unit 14 newly creates the training data, and the new prediction model creation unit 15 creates and evaluates the new prediction model. The data creation unit 16 creates evaluation data, and the operation prediction model selection unit 17 evaluates and selects a new prediction model in sequence.

The control unit 11 includes a storage unit and a processing unit. A computer is exemplified as the control unit 11. The storage unit has, for example, a storage device such as a RAM, a ROM, and a flash memory, and stores a software program processed by the processing unit, data referenced by the software program, and the like. The storage unit also functions as a storage area in which the processing unit temporarily stores the processing result and the like. The processing unit reads a software program or the like from the storage unit and processes the function according to the contents of the software program. Specifically, the deterioration detection data creation unit 12, the deterioration detection determination unit 13, and the learning data creation. The function of the control unit 11 including the functions of the unit 14, the new prediction model creation unit 15, the evaluation data creation unit 16, and the operation prediction model selection unit 17 is exhibited, and the prediction model maintenance method according to the embodiment of the present invention is exhibited. Execute. The software program that the processing unit reads from the storage unit and processes is the prediction model maintenance program according to the embodiment of the present invention.

In the present embodiment, the storage unit and the processing unit of the prediction unit 30 and the storage unit and the processing unit of the control unit 11 are separately provided, but the storage unit is not limited thereto. And the processing unit may be integrated. That is, in the present embodiment, the computer exemplified by the prediction unit 30 and the computer exemplified by the control unit 11 are separately provided, but the computer is not limited to this, and these computers are integrated. It may have been done.

The operation of the prediction model maintenance system 10 according to the embodiment having the above configuration will be described below. FIG. 4 is a flowchart of a prediction model maintenance method according to an embodiment of the present invention. The predictive model maintenance method according to the embodiment is a processing method executed by the predictive model maintenance system 10 according to the embodiment. The method for maintaining the prediction model according to the embodiment will be described with reference to FIG. As shown in FIG. 4, the prediction model maintenance method according to the embodiment includes deterioration detection data creation step S11, deterioration detection determination step S12, learning data creation step S13, new prediction model creation step S14, and evaluation data creation. It has a step S15 and an operation prediction model selection step S16.

The deterioration detection data creation step S11 is a step of creating deterioration detection data, which is data used by the deterioration detection data creation unit 12 to detect whether or not the prediction probability as the model quality of the prediction model 1 has deteriorated. is there. FIG. 5 is an explanatory diagram illustrating the details of the deterioration detection data creation step S11 of FIG. In deterioration detection data creation step S11, specifically, as shown in FIG. 5, the detection timing _t 11 is the timing to detect the deterioration provided for each fixed _{time, t 12, t 13, t} 14, t 15 Execute the process every time. In the present embodiment, _{the intervals between the detection timings t 11} , t ₁₂ , t ₁₃ , t ₁₄ , and t ₁₅ are set to a fixed time, but the interval is not limited to this, and the detection timings t ₁₁ , t ₁₂ , t ₁₃ , and t _{14 are not limited to this.} , T ₁₅ may be set at a predetermined time, or may be set every time a fixed amount of equipment data 21 and quality data 22 are acquired.

In deterioration detection data creation step S11, for example, as shown in FIG. 5, the deterioration detection data generating unit 12, the detection timing t _15, facility data 21 acquired within a certain period until the detection timing t ₁₅ and quality data 22 _{Is adopted, and deterioration detection data D 15} is created based on the adopted equipment data 21 and quality data 22. Similarly, in the deterioration detection data creation step S11, as shown in FIG. 5, the deterioration detection data creation unit 12 has detection timings t ₁₁ , t ₁₂ , t ₁₃ , t ₁₄ , t ₁₆ , t ₁₇ , and t _{18, respectively.} In, the equipment data 21 and the quality data 22 acquired within a certain period up to the detection timings t ₁₁ , t ₁₂ , t ₁₃ , t ₁₄ , t ₁₆ , t ₁₇ , and t _{18 are adopted, and the equipment data 21 and the equipment data 22 are adopted. Deterioration detection data D 11} , D ₁₂ , D ₁₃ , D ₁₄ , D ₁₆ , D ₁₇ , and D ₁₈ are created based on the quality data 22.

The deterioration detection determination step S12 is a step in which the deterioration detection determination unit 13 determines whether or not the prediction probability of the prediction model 1 is reduced by using the deterioration detection data created in the deterioration detection data creation step S11. FIG. 6 is an explanatory diagram illustrating an example of the prediction model 1 that receives the determination in the deterioration detection determination step S12 of FIG. As shown in FIG. 6, the prediction model 1 is created by the decision tree analysis, and among the four types of prediction rules by the decision tree analysis, the second prediction rule and the third prediction rule are The model adopted is illustrated.

FIG. 7 is an explanatory diagram illustrating an example of data processing in the deterioration detection determination step S12 of FIG. In the deterioration detection determination step S12, the deterioration detection determination unit 13 first uses the learning data and the evaluation data when the prediction model 1 is created, as shown in FIG. 7 for each prediction rule included in the prediction model 1. , The number of rule appearances, the number of rule correct answers, the rule correct answer rate, and the rule appearance rate. In the deterioration detection determination step S12, the deterioration detection determination unit 13 then uses the deterioration detection data created in the deterioration detection data creation step S11 for each prediction rule included in the prediction model 1 as shown in FIG. , The number of rule appearances, the number of rule correct answers, the rule correct answer rate, and the rule appearance rate. Here, in the prediction model 1 shown in FIG. 6, the second prediction rule and the third prediction rule are adopted, and in FIG. 7, only the first prediction rule is shown. However, in the deterioration detection determination step S12, the deterioration detection determination unit 13 also describes the first prediction rule and the fourth prediction rule that are paired with the second prediction rule and the third prediction rule, respectively. , Performs the process of obtaining each numerical value.

Here, the number of rule appearances is the number of times that the equipment data 21 does not satisfy the predetermined condition determined by the prediction rule. The number of correct answers to the rule is the number of times that the quality data 22 does not satisfy the predetermined requirements set by the rule. The rule correct answer rate is the ratio of the number of rule correct answers to the number of rule appearances, that is, among those determined not to meet the predetermined conditions based on the equipment data 21, the rule correct answer rate must meet the predetermined requirements based on the quality data 22. It is the determined ratio and is an index of the accuracy of detection of the prediction rule. The rule appearance rate is the ratio of the number of rule appearances to the total number of equipment data 21, that is, the ratio of determining that the predetermined condition is not satisfied based on the equipment data 21 for the entire process, and the tendency of the prediction rule. It is an index of.

In the deterioration detection determination step S12, the deterioration detection determination unit 13 uses two types of data, that is, when the prediction model 1 is created and when the prediction model 1 is detected to deteriorate, and the number of rule appearances, the number of correct rule answers, and the rules for each prediction rule After obtaining the four numerical values of the correct answer rate and the rule appearance rate, each of the rule correct answer rate and the rule appearance rate for each prediction rule is significant between the time when the prediction model 1 is created and the time when the deterioration detection of the prediction model 1 is detected. Determine if changes are seen. In the deterioration detection determination step S12, specifically, as shown in FIG. 7, the deterioration detection determination unit 13 calculates the test statistic for each of the rule correct answer rate and the rule appearance rate for each prediction rule, and the significance level. For example, 5% is used to determine whether or not a significant change is observed.

In the deterioration detection determination step S12, when the deterioration detection determination unit 13 determines that a significant change is observed in the rule accuracy rate on the lower side in a certain prediction rule, it is determined that the detection accuracy of the prediction rule has decreased. .. In the deterioration detection determination step S12, when the deterioration detection determination unit 13 determines that a significant change is observed in the rule accuracy rate on the upper side in a certain prediction rule, it is determined that the detection accuracy of the prediction rule is higher. In the deterioration detection determination step S12, when the deterioration detection determination unit 13 determines that there is no significant change in the rule accuracy rate in a certain prediction rule, it is determined that the detection accuracy of the prediction rule has not changed.

In the deterioration detection determination step S12, when the deterioration detection determination unit 13 determines that a significant change in the rule appearance rate is observed in a certain prediction rule, it is determined that the tendency of the prediction rule has changed. In the deterioration detection determination step S12, when the deterioration detection determination unit 13 determines that there is no significant change in the rule appearance rate in a certain prediction rule, it is determined that the tendency of the prediction rule does not change.

FIG. 8 is an explanatory diagram illustrating an example of a determination criterion in the deterioration detection determination step S12 of FIG. In the deterioration detection determination step S12, as shown in FIG. 8, when the deterioration detection determination unit 13 determines that the detection accuracy is lower or higher in a certain prediction rule, it is determined that the tendency has changed in the rule. Even if it is determined that there is no change, it is determined that the prediction rule has deteriorated. In the deterioration detection determination step S12, as shown in FIG. 8, when the deterioration detection determination unit 13 determines that the detection accuracy does not change in a certain prediction rule, and determines that the tendency has changed in the rule. If so, it is determined that the prediction rule has deteriorated. In the deterioration detection determination step S12, as shown in FIG. 8, when the deterioration detection determination unit 13 determines that the detection accuracy does not change in a certain prediction rule, and the tendency does not change in the rule. If it is determined, it is determined that the prediction rule has not deteriorated.

Here, unless it is determined that the detection accuracy does not change in a certain prediction rule and the tendency does not change in the rule, the detection accuracy is assumed to be higher in the prediction rule. Even if it is determined that the rule has deteriorated, the reason for determining that the rule has deteriorated is that if there is a change in at least one of the accuracy or tendency of the rule, there is a significant change in the process in which the prediction model 1 is operated. This is because it is highly likely that it is preferable to revise the prediction model 1 because it can be considered that there was.

FIG. 9 is an explanatory diagram illustrating an example of the determination result in the deterioration detection determination step S12 of FIG. In the deterioration detection determination step S12, as shown in FIG. 9, the deterioration detection determination unit 13 determines that the detection accuracy of the first prediction rule has not changed, and the tendency has not changed. Since it is judged, it is judged that it has not deteriorated. In the deterioration detection determination step S12, as shown in FIG. 9, the deterioration detection determination unit 13 determines that the detection accuracy of the second prediction rule is higher than that of the second prediction rule, and determines that the tendency has changed. Therefore, it is judged that it has deteriorated. In the deterioration detection determination step S12, as shown in FIG. 9, the deterioration detection determination unit 13 determines that the detection accuracy of the third prediction rule is lower and that the tendency has changed. Therefore, it is judged that it has deteriorated. In the deterioration detection determination step S12, as shown in FIG. 9, the deterioration detection determination unit 13 determines that the detection accuracy of the fourth prediction rule is higher and that the tendency has changed. Therefore, it is judged that it has deteriorated. In this way, in the deterioration detection determination step S12, as shown in FIG. 9, the deterioration detection determination unit 13 determines that the first prediction rule has not deteriorated, and each of the second to fourth Since it is determined that the prediction rule has deteriorated, it is detected that the prediction model in which the second prediction rule and the third prediction rule are adopted has deteriorated.

As described above, in the deterioration detection determination step S12, the prediction probability of the prediction model 1 is limited to the case where the deterioration detection determination unit 13 determines that there is no change in the tendency and no change in the accuracy in all the prediction rules. Is not reduced, and in other cases, it is determined that the prediction probability of the prediction model 1 is reduced, that is, deterioration is detected.

The deterioration detection data creation step S11 and the deterioration detection determination step S12 are executed in the present embodiment, but are not limited to these, and may be omitted together. When the deterioration detection data creation step S11 and the deterioration detection determination step S12 are executed, the learning data creation step S13 and the new prediction model creation are performed only when it is determined in the deterioration detection determination step S12 that the prediction probability of the prediction model 1 is low. A form in which step S14, evaluation data creation step S15, and operation prediction model selection step S16 are executed is exemplified as a preferable form. On the other hand, when the deterioration detection data creation step S11 and the deterioration detection determination step S12 are not executed, the training data creation step S13, the new prediction model creation step S14, and the evaluation are performed regardless of whether or not the prediction probability of the prediction model 1 is lowered. An example is a mode in which the data creation step S15 and the operation prediction model selection step S16 are executed.

Further, when the prediction unit 30 does not operate any of the prediction models 1, since there is no prediction model 1 to be determined in the deterioration detection determination step S12, the deterioration detection data creation step S11 and the deterioration detection determination step S12 are omitted, and the prediction is made. The process in the model maintenance method shifts to the training data creation step S13 from the beginning.

The learning data creation step S13 is a step in which the learning data creation unit 14 creates learning data to be used for creating a new prediction model. FIG. 10 is an explanatory diagram illustrating the details of the learning data creation step S13 of FIG. Specifically, in the learning data creation step S13, as shown in FIG. 10, the detection timings t ₂₁ , t ₂₂ , t ₂₃ , t ₂₄ , and t _{25, which are also the timings for creating the learning data provided at regular intervals.} , T ₂₆ , t ₂₇ , t ₂₈ , t ₂₉ , t ₃₀ , t ₃₁ , t ₃₂ , t ₃₃ . In this embodiment, the intervals between the _{detection timings t 21} , t ₂₂ , t ₂₃ , t ₂₄ , t ₂₅ , t ₂₆ , t ₂₇ , t ₂₈ , t ₂₉ , t ₃₀ , t ₃₁ , t ₃₂ , and t _{33 are constant.} The time is not limited to this, but the detection timing is not limited to t ₂₁ , t ₂₂ , t ₂₃ , t ₂₄ , t ₂₅ , t ₂₆ , t ₂₇ , t ₂₈ , t ₂₉ , t ₃₀ , t ₃₁ , t ₃₂ , t. ₃₃ may be set at a predetermined time, or may be set every time a fixed amount of equipment data 21 and quality data 22 are acquired.

In the learning data creation step S13, for example, as shown in FIG. 10, the learning data creation section 14, the detection timing _{t 25,} a certain period from the middle of the detection timing _{t 23} and the detection timing _{t 24} to the detection timing _{t 25} The equipment data 21 and quality data 22 acquired in the past in chronological order are adopted rather than the equipment data 21 and quality data 22 acquired in the room, and the first training data L_N1 is adopted based on the adopted equipment data 21 and quality data 22. , ..., L_Nn is created.

Incidentally, the equipment data 21 and the quality data 22 acquired within a certain period until the detection timing t _25, the Otte in the evaluation data generation step S15 to be described is intended to be employed in making the first evaluation data. In the training data creation step S13, the first training data is created using the same or larger amount of equipment data 21 and quality data 22 than the first evaluation data created in the evaluation data creation step S15, which will be described later. It is preferable to do so.

For example, in the learning data creation step S13, as shown in FIG. 10, the learning data creation unit 14 acquired the detection timing t ₂₅ from the detection timing t ₂₂ to the middle between the detection timing t ₂₃ and the detection timing t _24. The equipment data 21 and the quality data 22 are adopted, and the first learning data L_N1 is created based on the adopted equipment data 21 and the quality data 22. Further, in the learning data creation step S13, as shown in FIG. 10, the learning data creation unit 14 sets the detection timing t ₂₃ and the detection timing t ₂₄ from the timing before the detection timing t _{21 at the detection timing t 25.} The equipment data 21 and quality data 22 acquired up to the middle are adopted, and the first learning data L_Nn is created based on the adopted equipment data 21 and quality data 22. As described above, in the learning data creation step S13, as shown in FIG. 10, the learning data creation unit 14 has a past time in the time series more than the equipment data 21 and the quality data 22 acquired within a certain period until the detection timing. By setting the period for acquiring the equipment data 21 and the quality data 22 to be adopted to a plurality of types in the band, it is possible to create a plurality of types of learning data according to these settings.

Further, in the learning data creation step S13, as shown in FIG. 10, the learning data creation unit 14 acquired the detection timing t ₃₁ from the detection timing t ₂₈ to the middle between the detection timing t ₂₉ and the detection timing t _30. The equipment data 21 and the quality data 22 are adopted, and the second learning data L_N1 is created based on the adopted equipment data 21 and the quality data 22. Further, in the learning data creation step S13, as shown in FIG. 10, the learning data creation unit 14 performs the detection timing t ₂₉ and the detection timing t from between the detection timing t ₂₆ and the detection timing t _{27 at the detection timing t 31. The} equipment data 21 and the quality data 22 acquired up to the middle of 30 are adopted, and the second learning data L_Nn is created based on the adopted equipment data 21 and the quality data 22.

Further, in the learning data creation step S13, as shown in FIG. 10, the learning data creation unit 14 acquired the detection timing t ₃₂ from the detection timing t ₂₉ to the middle between the detection timing t ₃₀ and the detection timing t _31. The equipment data 21 and the quality data 22 are adopted, and the third learning data L_N1 is created based on the adopted equipment data 21 and the quality data 22. Further, in the learning data creation step S13, as shown in FIG. 10, the learning data creation unit 14 sets the detection timing t ₃₀ and the detection timing t from between the detection timing t ₂₇ and the detection timing t _{28 at the detection timing t 32. The} equipment data 21 and quality data 22 acquired up to the middle of 31 are adopted, and the third learning data L_Nn is created based on the adopted equipment data 21 and quality data 22.

As described above, in the learning data creation step S13, the detection timings t ₂₁ , t ₂₂ , t ₂₃ , t ₂₄ , t ₂₅ , t ₂₆ , t ₂₇ , t ₂₈ , t ₂₉ , t ₃₀ , t ₃₁ , t ₃₂ , t. _For each 33, it is possible to create a plurality of types of learning data according to the setting of the period in which the equipment data 21 and the quality data 22 to be adopted are acquired.

The new prediction model creation step S14 is a step in which the new prediction model creation unit 15 creates a new prediction model using the learning data created in the learning data creation step S13. FIG. 11 is an explanatory diagram illustrating an example of extraction of the prediction rule in the new prediction model creation step S14 of FIG. Specifically, in the new prediction model creation step S14, first, the new prediction model creation unit 15 extracts a prediction rule that may be adopted in the new prediction model, as shown in FIG. In the new prediction model creation step S14, as shown in FIG. 11, the new prediction model creation unit 15 analyzes the learning data L_N1 by the decision tree analysis and newly extracts four or more types of prediction rules.

FIG. 12 is an explanatory diagram illustrating an example of evaluation of the prediction rule in the new prediction model creation step S14 of FIG. In the new prediction model creation step S14, next, as shown in FIG. 12, the new prediction model creation unit 15 has 4 rules appearance number, rule correct answer number, rule correct answer rate, and rule appearance rate for each extracted prediction rule. Calculate two numbers. Note that in FIG. 12, the description of the rule appearance rate is omitted.

In the new prediction model creation step S14, next, as shown in FIG. 12, the new prediction model creation unit 15 may adopt the extracted rules whose rule accuracy rate is less than the threshold value for the new prediction model. It may be excluded from a certain prediction rule, and in addition, a rule whose number of correct answers is less than the threshold value may be further excluded. In the new prediction model creation step S14, as shown in FIG. 12, the new prediction model creation unit 15 has the fourth prediction rule in which the number of correct rule answers is less than 10 and the rule correct answer rate is 10% of the threshold value. The first prediction rule, which is less than, is excluded from the prediction rules that may be adopted in the new prediction model. Then, in the new prediction model creation step S14, as shown in FIG. 12, the new prediction model creation unit 15 adopts the second prediction rule and the third prediction rule that are not excluded into the new prediction model. Decide to do.

Here, the reason for excluding those whose rule accuracy rate is less than the threshold value from the prediction rules that may be adopted in the new prediction model is that even if the number of rule accuracy answers is sufficiently high, the rate of lowering the prediction probability is high. This is because it is considered. In addition, the reason why it is preferable to exclude those whose number of correct rule answers is less than the threshold value from the prediction rules that may be adopted in the new prediction model is that those whose number of correct answer answers is less than the threshold value have a sufficiently high rule correct answer rate. Even in this case, it is considered that the proportion that does not contribute to prediction is high.

After that, in the new prediction model creation step S14, the new prediction model creation unit 15 combines the second prediction rule and the third prediction rule that have been determined to be adopted in the new model, so that one type or a plurality of types can be used. Create a new type of prediction model. In the new prediction model creation step S14, when a plurality of types of training data are created in the training data creation step S13, a plurality of types of new prediction models are created using each type of training data.

The evaluation data creation step S15 is a step in which the evaluation data creation unit 16 creates evaluation data to be used for evaluating the prediction model 1 and the new prediction model created in the new prediction model creation step S14. FIG. 13 is an explanatory diagram illustrating the details of the evaluation data creation step S15 of FIG. Specifically, in the evaluation data creation step S15, as shown in FIG. 13, the detection timings t ₂₁ , t ₂₂ , t ₂₃ , t ₂₄ , and t _{25, which are also the timings for creating the evaluation data provided at regular intervals.} , T ₂₆ , t ₂₇ , t ₂₈ , t ₂₉ , t ₃₀ , t ₃₁ , t ₃₂ , t ₃₃ . Regarding the detection timings t ₂₁ , t ₂₂ , t ₂₃ , t ₂₄ , t ₂₅ , t ₂₆ , t ₂₇ , t ₂₈ , t ₂₉ , t ₃₀ , t ₃₁ , t ₃₂ , and t ₃₃ , the learning data creation step S13. It is the same as that described in the item of the explanation of.

In evaluation data generation step S15, for example, as shown in FIG. 13, the evaluation data generation section 16, the detection timing _{t 25,} a certain period from the middle of the detection timing _{t 23} and the detection timing _{t 24} to the detection timing _{t 25} The first evaluation data is created based on the T_N equipment data 21 and the quality data 22 acquired in the above.

In the evaluation data creation step S15, the first evaluation data can be created by using the equipment data 21 and the quality data 22 in the same amount or less than the first training data created in the training data creation step S13. preferable.

For example, in the evaluation data creation step S15, as shown in FIG. 13, the evaluation data creation unit 16 acquired the detection timing t ₂₅ from the middle between the detection timing t ₂₃ and the detection timing t ₂₄ to the detection timing t ₂₅ . The equipment data 21 and the quality data 22 are adopted, and the first evaluation data is created based on the adopted equipment data 21 and the quality data 22. Further, the evaluation data generation step S15, as shown in FIG. 13, the evaluation data generation section 16, the detection timing _{t 31,} was obtained from the intermediate between the detection timing _{t 29} and the detection timing _{t 30} to the detection timing _{t 31} The equipment data 21 and the quality data 22 are adopted, and the second evaluation data is created based on the adopted equipment data 21 and the quality data 22. Further, the evaluation data generation step S15, as shown in FIG. 13, the evaluation data generation section 16, the detection timing _{t 32,} was obtained from the intermediate between the detection timing _{t 30} and the detection timing _{t 31} to the detection timing _{t 32} The equipment data 21 and the quality data 22 are adopted, and a third evaluation data is created based on the adopted equipment data 21 and the quality data 22.

As described above, in the evaluation data creation step S15, the detection timings t ₂₁ , t ₂₂ , t ₂₃ , t ₂₄ , t ₂₅ , t ₂₆ , t ₂₇ , t ₂₈ , t ₂₉ , t ₃₀ , t ₃₁ , t ₃₂ , t. _For each 33, one type of evaluation data can be created according to the setting of the period for which the equipment data 21 and the quality data 22 to be adopted are acquired.

The evaluation data creation step S15 does not have to be performed after the training data creation step S13 and the new prediction model creation step S14. For example, the evaluation data creation step S15 is performed before the training data creation step S13 and the new prediction model creation step S14. Alternatively, it may be performed in parallel with the learning data creation step S13.

In the operation prediction model selection step S16, the operation prediction model selection unit 17 evaluates the prediction model 1 and the new prediction model created in the new prediction model creation step S14 by using the evaluation data created in the evaluation data creation step S15. , It is a step to select the one with the highest evaluation as the operation prediction model. FIG. 14 is an explanatory diagram illustrating an example of the prediction model 1 and the new prediction model to be evaluated in the operation prediction model selection step S16 of FIG. In the operation prediction model selection step S16, specifically, as shown in FIG. 14, the operation prediction model selection unit 17 performs the first prediction model 1 and the second and subsequent new predictions. The evaluation data created in the evaluation data creation step S15 is used for each of the plurality of new prediction models created in the model creation step S14.

FIG. 15 is an explanatory diagram illustrating an example of evaluation of the prediction model 1 and the new prediction model in the operation prediction model selection step S16 of FIG. In the operation prediction model selection step S16, the operation prediction model selection unit 17 then evaluates each of the prediction model 1 and the plurality of new prediction models created in the new prediction model creation step S14, as shown in FIG. Using the evaluation data created in the data creation step S15, four numerical values of rule appearance number, rule correct answer number, rule correct answer rate, and rule appearance rate for each prediction rule included in the prediction model 1 and the plurality of new prediction models. Is calculated. In the operation prediction model selection step S16, and as shown in FIG. 15, the operation prediction model selection unit 17 describes the entire model for each of the prediction model 1 and the plurality of new prediction models created in the new prediction model creation step S14. Calculate four numerical values: the number of rule appearances, the number of rule correct answers, the rule correct answer rate, and the rule appearance rate. Note that, in FIG. 15, the description of the rule appearance rate is omitted as in FIG.

FIG. 16 is an explanatory diagram illustrating an example of comparison of evaluation of the prediction model 1 and the new prediction model in the operation prediction model selection step S16 of FIG. In the operation prediction model selection step S16, as shown in FIG. 16, the operation prediction model selection unit 17 correctly answers the rules of the entire model of the prediction model 1 and the plurality of new prediction models created in the new prediction model creation step S14. The rates, that is, the prediction probabilities are compared, and the one with the highest rule accuracy rate for the entire model is newly selected as the most promising candidate for the prediction model operated by the prediction unit 30.

Here, the operation prediction model selection step S16, the operation prediction model selection unit 17, when predictability predictive model that was selected the most likely candidate is the predetermined threshold value Q _T or more as shown in FIG. 2, the leading candidate and selecting a predictive model to the operation prediction model and be operated in the prediction unit 30, when predictability predictive model that was selected the most likely candidate is less than a predetermined threshold Q _T, operate prediction model of the best candidates It is possible to prevent the prediction unit 30 from operating any prediction model without selecting it as the prediction model.

FIG. 17 is an explanatory diagram illustrating an example of the operation prediction model selected in the operation prediction model selection step S16 of FIG. In the operation prediction model selection step S16, as shown in FIG. 17, the operation prediction model selection unit 17 creates a new prediction model 1 based on the first learning data and the first evaluation data at _{the detection timing t 41.} for the probability of prediction is less than a predetermined threshold Q _T, it does not select a new predictive model 1 to the operation prediction model. In operation prediction model selection step S16, the operation prediction model selection unit 17, as shown in FIG. 17, the detection timing t _42, was created on the basis of the evaluation data of the learning data and the 2-A of the 2-A predictability of new predictive models 2-a is higher than the predictability of new predictive model 2-B, which is created based on the evaluation data of the learning data and the 2-B of the 2-B, a predetermined threshold Q _T For the above reasons, the new prediction model 2-A has been selected as the operational prediction model. As a result, _{before the detection timing t 42} , no prediction model is selected as the operation prediction model, and at the detection timing t ₄₂ , the new prediction model 2-A is selected as the operation prediction model. It has switched to the state.

In the operation prediction model selection step S16, as shown in FIG. 17, the operation prediction model selection unit 17 selects the prediction model 2-A as the operation prediction model at _{the detection timing t 43} and the detection timing t _42. Created based on the prediction probability of the new prediction model 3-A created based on the training data of 3-A and the evaluation data of 3-A, and the learning data of 3-B and the evaluation data of 3-B. are novel predictive model 3-B both higher than the predictability of still to maintain the predetermined threshold value or more Q _T, have continued chosen predictive model 2-a to the operation prediction model.

In operation prediction model selection step S16, the operation prediction model selection unit 17, as shown in FIG. 17, the detection timing t _44, the predictive model 2-A, which is selected in the operation prediction model in detection timing t _42, the Created based on the prediction probability of the new prediction model 4-A created based on the training data of 4-A and the evaluation data of 4-A, and the training data of 4-B and the evaluation data of 4-B. and predictability of new predictive models 4-B, which is a predetermined threshold value Q _T, in order to below than either excludes predictive model 2-a from the operational prediction model. Further, the operation prediction model selection step S16, the operation prediction model selection unit 17, as shown in FIG. 17, the detection timing t _44, the even new predictive model 4-A and the new prediction model 4-B, its predictability but because they are lower than the predictability or predetermined threshold Q _T predictive model 2-a, we do not select the operation prediction model. Thus, the detection timing t ₄₂ to the detection timing t ₄₄ is in a state in which predictive model 2-A is selected in the operation prediction model, the detection timing t _44, any prediction model is selected in the operation prediction model It has been switched to the non-existent state.

In operation prediction model selection step S16, the operation prediction model selection unit 17, as shown in FIG. 17, the detection timing t _45, was created on the basis of the evaluation data of the learning data and the 5-B of the 5-B predictability of new predictive model 5-B is higher than the predictability of the 5-a of the learning data and the 5-a new prediction model 5-a was created based on the evaluation data, the predetermined threshold Q _T For the above reasons, the new prediction model 5-B has been selected as the operational prediction model. As a result, _{before the detection timing t 45} , no prediction model is selected as the operation prediction model, and at the detection timing t ₄₅ , the new prediction model 5-B is selected as the operation prediction model. It has switched to the state.

Thus, predictive models maintained method according to the embodiment executed by the predictive model maintenance system 10 according to the embodiment, the predetermined time intervals, prediction model predictability in the state of the process is not less than the predetermined threshold value Q _T causes the operation of the predictive model to predict unit 30 when it was confirmed that there is any prediction if the prediction probability in the state of the process is confirmed that there are no predictive models it is greater than or equal to a predetermined threshold Q _T It is possible to maintain a state in which the prediction unit 30 does not operate the model. Therefore, prediction model maintains method according to the embodiment executed by the predictive model maintenance system 10 according to the embodiment, it is possible not continue to use the prediction models falls below a predetermined threshold Q _T.

Since the prediction model maintenance system 10 according to the embodiment, the prediction model maintenance method according to the embodiment, and the prediction model maintenance program according to the embodiment have the above configurations, they are originally used with the newly created new prediction model. Evaluate the prediction model that was used, and select the one with the highest evaluation as the operation prediction model. For this reason, the prediction model maintenance system 10 according to the embodiment, the prediction model maintenance method according to the embodiment, and the prediction model maintenance program according to the embodiment are newly introduced in that the prediction accuracy is deteriorated as compared with the prediction model originally used. Since it is not used for the above, it is possible to sufficiently reduce the deterioration of the prediction accuracy due to the change in the process.

The prediction model maintenance system 10 according to the embodiment, the prediction model maintenance method according to the embodiment, and the prediction model maintenance program according to the embodiment are further deteriorated used to detect whether or not the prediction probability of the prediction model is deteriorated. Detect data is created, and the created deterioration detection data is used to determine whether or not the prediction probability of the prediction model is decreasing. If it is determined that the prediction probability of the prediction model is decreasing, training data is created. It is supposed to be done. Therefore, the predictive model maintenance system 10 according to the embodiment, the predictive model maintenance method according to the embodiment, and the predictive model maintenance program according to the embodiment are used when it is determined that the predictive model currently used is deteriorated. Only, since a new prediction model is newly created, it is possible to efficiently reduce the deterioration of the prediction accuracy due to changes in the process.

In the prediction model maintenance system 10 according to the embodiment, the prediction model maintenance method according to the embodiment, and the prediction model maintenance program according to the embodiment, when the higher evaluation is equal to or higher than the evaluation threshold, the higher evaluation is used as the operation prediction model. If the one with the higher evaluation is less than the evaluation threshold, none of them will be selected as the operation prediction model. Therefore, the prediction model maintenance system 10 according to the embodiment, the prediction model maintenance method according to the embodiment, and the prediction model maintenance program according to the embodiment do not select the prediction model for evaluation below the threshold as the operation prediction model. It is possible to reduce the adverse effect on the process due to the use of a prediction model with low accuracy.

In the predictive model maintenance system 10 according to the embodiment, the predictive model maintenance method according to the embodiment, and the predictive model maintenance program according to the embodiment, predetermined conditions are determined based on equipment data in the process, and predetermined requirements are passed through the process. It is assumed that the quality data of the product is within the specified range. Therefore, the predictive model maintenance system 10 according to the embodiment, the predictive model maintenance method according to the embodiment, and the predictive model maintenance program according to the embodiment preferably use equipment data and quality data generally used for process management and evaluation. Therefore, it is possible to efficiently reduce the deterioration of the prediction accuracy due to changes in the process.

The predictive model maintenance system 10 according to the embodiment, the predictive model maintenance method according to the embodiment, and the predictive model maintenance program according to the embodiment are set to issue an alert when there is a possibility that the predetermined requirements are not met. There is. Therefore, the predictive model maintenance system 10 according to the embodiment, the predictive model maintenance method according to the embodiment, and the predictive model maintenance program according to the embodiment may not satisfy the predetermined requirements by the alert. Since it is possible to call attention, it is possible to sufficiently reduce the possibility of overlooking a case where a predetermined requirement is not satisfied in the process.

The predictive model maintenance system 10 according to the embodiment, the predictive model maintenance method according to the embodiment, and the predictive model maintenance program according to the embodiment learn using the data acquired in the past in chronological order rather than the data used for creating the evaluation data. Creating data. Therefore, the predictive model maintenance system 10 according to the embodiment, the predictive model maintenance method according to the embodiment, and the predictive model maintenance program according to the embodiment are based on the learning data created by using the data acquired in the past in the time series. Since the new prediction model created in the above process evaluates whether or not it is possible to predict with sufficient accuracy even in the evaluation data created using the data acquired in the future in the time series, the evaluation data is practically also in the training data. It is also possible to create a new prediction model having sufficient prediction accuracy. Therefore, the prediction model maintenance system 10 according to the embodiment, the prediction model maintenance method according to the embodiment, and the prediction model maintenance program according to the embodiment are efficient, and the accuracy of prediction deteriorates due to changes in the process. Can be sufficiently reduced.

The predictive model maintenance system 10 according to the embodiment, the predictive model maintenance method according to the embodiment, and the predictive model maintenance program according to the embodiment further use the same amount or more data than the data used for creating the evaluation data. Creating training data. Therefore, the prediction model maintenance system 10 according to the embodiment, the prediction model maintenance method according to the embodiment, and the prediction model maintenance program according to the embodiment are among the data in which the new prediction model is considered to have sufficient prediction accuracy. More proportions of data can be used to create new predictive models. Therefore, the prediction model maintenance system 10 according to the embodiment, the prediction model maintenance method according to the embodiment, and the prediction model maintenance program according to the embodiment are more efficient, and the accuracy of prediction deteriorates due to changes in the process. This can be sufficiently reduced.

The predictive model maintenance system 10 according to the embodiment, the predictive model maintenance method according to the embodiment, and the predictive model maintenance program according to the embodiment are further described by using a plurality of types of data for one type of evaluation data. Various types of training data are created, and multiple types of new prediction models are created using multiple types of training data. Therefore, the prediction model maintenance system 10 according to the embodiment, the prediction model maintenance method according to the embodiment, and the prediction model maintenance program according to the embodiment create a plurality of types of learning data at a time, and a plurality of types of new prediction models. Can be created. As a result, the predictive model maintenance system 10 according to the embodiment, the predictive model maintenance method according to the embodiment, and the predictive model maintenance program according to the embodiment change the process while the data used for creating the training data is acquired. In that case, a new prediction model more suitable for the current process can be created, so that it is possible to further sufficiently reduce the deterioration of the prediction accuracy due to the change in the process.

1 Prediction model 10 Prediction model maintenance system 11 Control unit 12 Deterioration detection data creation unit 13 Deterioration detection judgment unit 14 Learning data creation unit 15 New prediction model creation department 16 Evaluation data creation department 17 Operation prediction model selection department 20 Database 21 Equipment data 22 Quality data 30 Prediction department 32 Alert department

Claims (9)

A predictive model maintenance system that maintains the predictive probability of a predictive model that predicts whether or not a predetermined requirement is satisfied based on a predetermined condition.
Including the control unit
The control unit
A learning data creation unit that creates training data used to create a new prediction model,
A new prediction model creation unit that creates the new prediction model using the learning data created by the learning data creation unit, and a new prediction model creation unit.
An evaluation data creation unit that creates evaluation data used to evaluate the prediction model and the new prediction model, and an evaluation data creation unit.
Using the evaluation data created by the evaluation data creation unit, the operation prediction model selection unit evaluates the prediction model and the new prediction model, and selects the one with the higher evaluation as the operation prediction model.
Only including,
The operation prediction model selection unit selects the one with the higher evaluation as the operation prediction model when the higher evaluation is equal to or higher than the evaluation threshold, and when the higher evaluation is less than the evaluation threshold. Prediction model maintenance system characterized by not being selected as an operational prediction model. A deterioration detection data creation unit that creates deterioration detection data used to detect whether or not the prediction probability of the prediction model has deteriorated, and a deterioration detection data creation unit.
Using the deterioration detection data created by the deterioration detection data creation unit, a deterioration detection determination unit that determines whether or not the prediction probability of the prediction model is reduced, and a deterioration detection determination unit.
Including
The prediction model maintenance according to claim 1, wherein the learning data creation unit creates the training data when the deterioration detection determination unit determines that the prediction probability of the prediction model is low. system. Predetermined conditions are determined based on equipment data in the process.
The predictive model maintenance system according to claim 1 or 2 , wherein the predetermined requirement is that the quality data of the product that has undergone the process is within a predetermined range. The prediction model and the new prediction model are set to issue an alert when there is a possibility that the predetermined requirements are not satisfied, according to any one of claims 1 to 3. The described predictive model maintenance system. Claimed from claim 1, wherein the training data creation unit creates the training data using data acquired in the past in a time series rather than the data used by the evaluation data creation unit to create the evaluation data. Item 4. The predictive model maintenance system according to any one of items 4. The fifth aspect of claim 5, wherein the training data creation unit creates the training data using the same amount or more data than the data used by the evaluation data creation unit for creating the evaluation data. Predictive model maintenance system. The learning data creation unit creates a plurality of types of the training data using a plurality of types of data for one type of the evaluation data created by the evaluation data creation unit.
The prediction model maintenance system according to claim 6 , wherein the new prediction model creating unit creates a plurality of types of the new prediction model using the plurality of types of the learning data. It is a prediction model maintenance method that maintains the prediction probability of a prediction model that predicts whether or not a predetermined requirement is satisfied.
Training data creation steps to create training data used to create a new prediction model,
A new prediction model creation step for creating the new prediction model using the training data created in the training data creation step, and a new prediction model creation step.
An evaluation data creation step for creating evaluation data used for evaluating the prediction model and the new prediction model, and
Using the evaluation data created in the evaluation data creation step, the prediction model and the new prediction model are evaluated, and the one with the higher evaluation is selected as the operation prediction model.
Have a,
In the operation prediction model selection step, if the higher evaluation is equal to or higher than the evaluation threshold, the higher evaluation is selected as the operation prediction model, and the higher evaluation is less than the evaluation threshold. A predictive model maintenance method characterized by not being selected as an operational predictive model. A predictive model maintenance program that causes a computer to execute a predictive model maintenance method for maintaining a predictive probability of a predictive model that predicts whether or not a predetermined requirement is satisfied based on a predetermined condition.
On the computer
Training data creation steps to create training data used to create a new prediction model,
A new prediction model creation step for creating the new prediction model using the training data created in the training data creation step, and a new prediction model creation step.
An evaluation data creation step for creating evaluation data used for evaluating the prediction model and the new prediction model, and
Using the evaluation data created in the evaluation data creation step, the prediction model and the new prediction model are evaluated, and the one with the higher evaluation is selected as the operation prediction model.
To run ,
In the operation prediction model selection step, if the higher evaluation is equal to or higher than the evaluation threshold, the higher evaluation is selected as the operation prediction model, and the higher evaluation is less than the evaluation threshold. prediction model maintains program characterized Rukoto also be executed not to select the operation prediction model.

Images