JP7371696B2 - Deterioration detection method, deterioration detection program, and information processing device - Google Patents

Description

The present invention relates to a deterioration detection method, a deterioration detection program, and an information processing device.

Machine learning models (hereinafter sometimes simply referred to as "models") are increasingly being introduced into information systems used by companies, etc. for data judgment and classification functions. Machine learning models perform judgment and classification according to the training data trained during system development, so if the trend of input data (data distribution) changes during system operation, the accuracy of the machine learning model will deteriorate.

In general, the method of detecting model accuracy deterioration during system operation is to calculate the accuracy rate by periodically manually checking whether the model output results are correct or incorrect, and detect accuracy deterioration from the decrease in the accuracy rate. is used.

In recent years, ^T2 statistics (Hotelling's T-square) have become known as a technique for automatically detecting deterioration in accuracy of machine learning models during system operation. For example, input data and a group of normal data (training data) are subjected to principal component analysis, and the ^T2 statistic of the input data, which is the sum of the squares of the distances from the origin of each standardized principal component, is calculated. Then, based on the distribution of ^T2 statistics of the input data group, changes in the ratio of abnormal value data are detected, and deterioration in model accuracy is automatically detected.

A. Shabbak and H. Midi, “An Improvement of the Hotelling Statistic in Monitoring Multivariate Quality Characteristics”, Mathematical Problems in Engineering (2012) 1-15.

However, with the above technology, there are many restrictions on the machine learning model used to detect accuracy deterioration, making it difficult to use it for general purposes.

For example, when applied to a model that processes high-dimensional data of thousands to tens of thousands of dimensions, which originally has a very large amount of information, most of the information will be lost when the dimensions are reduced to a few dimensions using principal component analysis. . As a result, feature quantities that are important information for classification and determination are also lost, making it impossible to effectively detect abnormal data and detecting deterioration in model accuracy.

In addition, the ^T2 statistic uses the distance of the principal component from the training data group for measurement, so if the training data contains data groups of multiple categories (multiclasses), it will be judged as normal data. The scope of what is being done becomes wider. For this reason, abnormal data cannot be detected and detection of model accuracy degradation cannot be realized.

One aspect of the present invention is to provide a deterioration detection method, a deterioration detection program, and an information processing device that can detect deterioration in accuracy even for machine learning models that perform high-dimensional data classification or multi-class classification. shall be.

In the first proposal, in the deterioration detection method, a computer performs a process of acquiring a first output result when data is input to a trained model. In the deterioration detection method, a computer performs a process of acquiring a second output result when data is input to a detection model with a narrowed model application area of the learned model. In the deterioration detection method, a computer performs a process of detecting deterioration in accuracy of a trained model due to a change in data trend over time, based on the first output result and the second output result.

According to one embodiment, deterioration in accuracy can also be detected for machine learning models that perform high-dimensional data classification or multi-class classification.

FIG. 1 is a diagram illustrating an accuracy deterioration detection device according to a first embodiment. FIG. 2 is a diagram illustrating accuracy deterioration. FIG. 3 is a diagram illustrating an inspector model according to the first embodiment. FIG. 4 is a functional block diagram showing the functional configuration of the accuracy deterioration detection device according to the first embodiment. FIG. 5 is a diagram showing an example of information stored in the teacher data DB. FIG. 6 is a diagram showing an example of information stored in the input data DB. FIG. 7 is a diagram showing the relationship between the number of training data and the applicable range. FIG. 8 is a diagram illustrating detection of accuracy deterioration. FIG. 9 is a diagram illustrating changes in the distribution of matching rates. FIG. 10 is a flowchart showing the flow of processing. FIG. 11 is a diagram illustrating a comparison result of accuracy deterioration detection of high-dimensional data. FIG. 12 is a diagram illustrating a comparison result of detection of accuracy deterioration in multi-class classification. FIG. 13 is a diagram illustrating a specific example using an image classifier. FIG. 14 is a diagram illustrating a specific example of teacher data. FIG. 15 is a diagram illustrating the execution results of precision deterioration detection. FIG. 16 is a diagram illustrating an example of controlling the model application area. FIG. 17 is a diagram illustrating an example of generating an inspector model according to the second embodiment. FIG. 18 is a diagram illustrating changes in validation accuracy. FIG. 19 is a diagram illustrating generation of an inspector model using validation accuracy. FIG. 20 is a diagram illustrating an example in which the boundary position between the machine learning model and the inspector model does not change. FIG. 21 is a diagram illustrating the inspector model of the third embodiment. FIG. 22 is a diagram illustrating deterioration detection according to the third embodiment. FIG. 23 is a diagram illustrating an example of teacher data for the other class (class 10). FIG. 24 is a diagram illustrating the effects of the third embodiment. FIG. 25 is a diagram illustrating an example of a hardware configuration.

Embodiments of a deterioration detection method, a deterioration detection program, and an information processing device according to the present invention will be described in detail below with reference to the drawings. Note that the present invention is not limited to this example. Moreover, each embodiment can be combined as appropriate within a consistent range.

[Description of accuracy deterioration detection device]
FIG. 1 is a diagram illustrating an accuracy deterioration detection device 10 according to a first embodiment. The accuracy deterioration detection device 10 shown in FIG. This is an example of a computer device that monitors the accuracy of a model and detects deterioration in accuracy.

For example, during training, a machine learning model is trained using image data as an explanatory variable and training data that uses clothing names as an objective variable. During operation, when image data is input as input data, it is trained using training data such as "shirt". This is an image classifier that outputs judgment results. In other words, a machine learning model is an example of an image classifier that performs classification of high-dimensional data or multi-class classification.

Here, machine learning models learned by machine learning, deep learning, etc. are trained based on teacher data that is a combination of training data and labeling, so they function only within the range included in the teacher data. On the other hand, machine learning models are assumed to be input with the same type of data as during training after operation, but in reality, the state of the input data changes and the machine learning model does not function properly. It may disappear. In other words, "model accuracy deterioration" occurs.

FIG. 2 is a diagram illustrating accuracy deterioration. FIG. 2 shows information that has been organized by removing unnecessary data from the input data, and shows a feature space in which the machine learning model classifies the input data. FIG. 2 illustrates a feature space classified into class 0, class 1, and class 2.

As shown in FIG. 2, at the initial stage of system operation (when learning is completed), all input data are in normal positions and are classified inside the decision boundaries of each class. As time progresses thereafter, the distribution of input data of class 0 changes. In other words, input data that is difficult to classify as class 0 using the learned feature values of class 0 begins to be input. Furthermore, after that, the input data of class 0 crosses the decision boundary, and the accuracy rate of the machine learning model decreases. In other words, the feature amount of the input data to be classified as class 0 changes.

In this way, if the distribution of input data changes from the time of learning after the system starts operating, the accuracy rate of the machine learning model decreases, resulting in deterioration of the accuracy of the machine learning model.

Therefore, as shown in FIG. 1, the accuracy deterioration detection device 10 according to the first embodiment uses at least one inspector generated using a DNN (Deep Neural Network) that solves the same problem as the machine learning model to be monitored. A model (monitor, sometimes simply referred to as "inspector" below) is used. Specifically, the accuracy deterioration detection device 10 aggregates the match rate between the output of the machine learning model and the output of each inspector model for each output class of the machine learning model, thereby detecting a change in the distribution of the match rate, that is, the input Detect changes in data distribution.

Here, the inspector model will be explained. FIG. 3 is a diagram illustrating an inspector model according to the first embodiment. The inspector model is an example of a detection model that is generated under different conditions (different model applicability domain) than the machine learning model. In other words, each region (each feature amount) that the inspector model determines as class 0, class 1, and class 2 has a narrower range than each region that the machine learning model determines as class 0, class 1, and class 2. , an inspector model is generated.

This is because the narrower the model application area, the more sensitive the output changes to small changes in input data. Therefore, by making the model application area of the inspector model narrower than that of the machine learning model to be monitored, the output value of the inspector model will fluctuate due to small changes in the input data, and the matching rate with the output value of the machine learning model will change the data. Changes in trends can be measured.

Specifically, as shown in Figure 3, if the input data is within the model application area of the inspector model, the machine learning model determines that the input data is class 0, and the inspector model also determines that the input data is class 0. It is determined that In other words, both fall within the model application area of class 0, and the output values always match, so the matching rate does not decrease.

On the other hand, if the input data is outside the model application range of the inspector model, the machine learning model will judge the input data to be class 0, but the inspector model will judge that the input data is outside the model application range of each class. Therefore, it is not necessarily determined that it is class 0. In other words, since the output values do not necessarily match, the matching rate decreases.

In this way, the accuracy deterioration detection device 10 according to the first embodiment performs class determination using the inspector model, which has been trained to have a model application area narrower than the model application area of the machine learning model, in parallel with class determination using the machine learning model. Execute the judgment and calculate the match rate of both class judgments. Since the accuracy deterioration detection device 10 detects a change in the distribution of input data based on a change in the matching rate, it is possible to detect deterioration in the accuracy of a machine learning model that performs classification of high-dimensional data or multi-class classification.

[Functional configuration of accuracy deterioration detection device]
FIG. 4 is a functional block diagram showing the functional configuration of the accuracy deterioration detection device 10 according to the first embodiment. As shown in FIG. 4, the accuracy deterioration detection device 10 includes a communication section 11, a storage section 12, and a control section 209.

The communication unit 11 is a processing unit that controls communication with other devices, and is, for example, a communication interface. For example, the communication unit 11 receives various instructions from an administrator terminal or the like. The communication unit 11 also receives input data to be determined from various terminals.

The storage unit 12 is an example of a storage device that stores data, programs executed by the control unit 20, and the like, and is, for example, a memory or a hard disk. This storage unit 12 stores a teacher data DB 13, an input data DB 14, a machine learning model 15, and an inspector model DB 16.

The teacher data DB 13 is a database that stores teacher data used for learning the machine learning model and also used for learning the inspector model. FIG. 5 is a diagram showing an example of information stored in the teacher data DB 13. As shown in FIG. 5, the teacher data DB 13 stores data IDs and teacher data in association with each other.

The data ID stored here is an identifier that identifies the teacher data. The teacher data is training data used for learning or verification data used for verification during learning. In the example of FIG. 5, training data X whose data ID is "A1" and verification data Y whose data ID is "B1" are illustrated. Note that the training data and verification data are data in which image data, which is an explanatory variable, and correct answer information (label), which is a target variable, are associated with each other.

The input data DB 14 is a database that stores input data to be determined. Specifically, the input data DB 14 stores image data that is input to a machine learning model and is a target for image classification. FIG. 6 is a diagram showing an example of information stored in the input data DB 14. As shown in FIG. 6, the input data DB 14 stores data IDs and input data in association with each other.

The data ID stored here is an identifier that identifies input data. The input data is image data to be classified. In the example of FIG. 6, input data 1 whose data ID is "01" is illustrated. The input data does not need to be stored in advance and may be sent as a data stream from another terminal.

The machine learning model 15 is a learned machine learning model, and is a model to be monitored by the accuracy deterioration detection device 10. Note that it is also possible to store a machine learning model 15 such as a neural network or support vector machine in which trained parameters are set, and to store trained parameters that can be constructed by the trained machine learning model 15. Good too.

The inspector model DB 16 is a database that stores information regarding at least one inspector model used for accuracy deterioration detection. For example, the inspector model DB 16 stores various parameters of the DNN, which are parameters for constructing each of the five inspector models, and are generated (optimized) by machine learning by the control unit 20, which will be described later. Note that the inspector model DB 16 can also store learned parameters, and can also store the inspector model itself (DNN) in which the learned parameters are set.

The control unit 20 is a processing unit that controls the entire precision deterioration detection device 10, and is, for example, a processor. The control section 20 includes an inspector model generation section 21, a threshold value setting section 22, and a deterioration detection section 23. Note that the inspector model generation unit 21, the threshold value setting unit 22, and the deterioration detection unit 23 are an example of an electronic circuit included in a processor, an example of a process executed by the processor, and the like.

The inspector model generation unit 21 is a processing unit that generates an inspector model that is an example of a monitor or detection model that detects deterioration in accuracy of the machine learning model 15. Specifically, the inspector model generation unit 21 generates a plurality of inspector models with different model application ranges by deep learning using the teacher data used for learning the machine learning model 15. Then, the inspector model generation unit 21 stores various parameters for constructing each inspector model (each DNN) having a different model application range, obtained by deep learning, in the inspector model DB 16.

For example, the inspector model generation unit 21 generates a plurality of inspector models with different application ranges by controlling the number of training data. FIG. 7 is a diagram showing the relationship between the number of training data and the applicable range. FIG. 7 illustrates a feature space for classification into three classes: class 0, class 1, and class 2.

As shown in Figure 7, in general, the larger the number of training data, the more features will be learned, so more comprehensive learning will be performed, and a model with a wider range of model application will be generated. be done. On the other hand, the smaller the number of training data, the fewer the features of the training data to be learned, so the range (features) that can be covered becomes limited, and a model with a narrow model application range is generated.

Therefore, the inspector model generation unit 21 generates a plurality of inspector models by changing the number of training data while keeping the number of trainings the same. For example, consider a case where five inspector models are generated in a state where the machine learning model 15 has been trained with the number of training times (100 epochs) and the number of training data (1000 pieces/class). In this case, the inspector model generation unit 21 sets the number of training data for inspector model 1 to ``500 pieces/class'', the number of training data for inspector model 2 to ``400 pieces/class'', and the number of training data for inspector model 3 to ``500 pieces/class''. The number of training data for Inspector Model 4 is determined to be "300 pieces/class", the number of training data for Inspector Model 5 is determined as "100 pieces/1 class", and the training data is downloaded from the teacher data DB 13. Each is randomly selected and trained for 100 epochs.

Thereafter, the inspector model generation unit 21 stores the various parameters of the learned inspector models 1, 2, 3, 4, and 5 in the inspector model DB 16. In this way, the inspector model generation unit 21 can generate five inspector models that have a narrower model application range than the application range of the machine learning model 15 and have different model application ranges.

Note that the inspector model generation unit 21 can learn each inspector model using a method such as error back propagation, and can also employ other methods. For example, the inspector model generation unit updates the parameters of the DNN so that the error between the output result obtained by inputting training data into the inspector model and the label of the input training data becomes smaller, thereby generating the inspector model. (DNN) learning.

Returning to FIG. 4, the threshold value setting unit 22 sets a threshold value for determining accuracy deterioration of the machine learning model 15, and a threshold value used for determining the matching rate. For example, the threshold setting unit 22 reads the machine learning model 15 from the storage unit 12 and reads various parameters from the inspector model DB 16 to construct five learned inspector models. Then, the threshold setting unit 22 reads each verification data stored in the teacher data DB 13, inputs it to the machine learning model 15 and each inspector model, and applies the data to the model application area based on the output results (classification results) of each. Get distribution results.

Thereafter, the threshold setting unit 22 determines the match rate of each class between the machine learning model 15 and the inspector model 1, the match rate of each class between the machine learning model 15 and the inspector model 2, the match rate of each class between the machine learning model 15 and the inspector model 2, and the match rate of each class between the machine learning model 15 and the inspector model 2 for the verification data. The matching rate of each class between the inspector model 3, the matching rate of each class between the machine learning model 15 and the inspector model 4, and the matching rate of each class between the machine learning model 15 and the inspector model 5 are calculated.

Then, the threshold value setting unit 22 sets a threshold value using each matching rate. For example, the threshold value setting unit 22 displays each matching rate on a display or the like and receives threshold settings from the user. Further, the threshold value setting unit 22 can arbitrarily select and set an average value of each matching rate, a maximum value of each matching rate, a minimum value of each matching rate, etc., depending on the deterioration state that the user requests detection. I can do it.

Returning to FIG. 4, the deterioration detection unit 23 includes a classification unit 24, a monitoring unit 25, and a notification unit 26, and compares the output results of the machine learning model 15 with respect to input data and the output results of each inspector model, and performs machine learning. This is a processing unit that detects deterioration in the accuracy of the model 15.

The classification unit 24 is a processing unit that inputs input data to the machine learning model 15 and each inspector model, and obtains respective output results (classification results). For example, when the learning of each inspector model is completed, the classification unit 24 acquires the parameters of each inspector model from the inspector model DB 16 to construct each inspector model, and executes the machine learning model 15.

Then, the classification unit 24 inputs the input data to the machine learning model 15 and obtains the output results, and also inputs the input data to each of the five inspector models from inspector model 1 (DNN1) to inspector model 5 (DNN5). input and get each output result. Thereafter, the classification section 24 stores the input data and each output result in correspondence with each other in the storage section 12, and outputs them to the monitoring section 25.

The monitoring unit 25 is a processing unit that monitors accuracy deterioration of the machine learning model 15 using the output results of each inspector model. Specifically, the monitoring unit 25 measures a change in the distribution of the match rate between the output of the machine learning model 15 and the output of the inspector model for each class. For example, the monitoring unit 25 calculates the match rate between the output result of the machine learning model 15 and the output result of each inspector model for each input data, and detects a deterioration in the accuracy of the machine learning model 15 when the match rate decreases. do. Note that the monitoring unit 25 outputs the detection result to the notification unit 26.

FIG. 8 is a diagram illustrating detection of accuracy deterioration. FIG. 8 illustrates the output results of the machine learning model 15 to be monitored and the output results of the inspector model for input data. Here, to make the explanation easier to understand, we will use one inspector model as an example and use the data distribution in the model application area in the feature space to compare the output of the inspector model with respect to the output of the machine learning model 15 to be monitored. Explain the probability of matching.

As shown in FIG. 8, at the start of operation, the monitoring unit 25 determines that six input data belong to the model application area of class 0 and six input data belong to the model application area of class 1 from the machine learning model 15 to be monitored. It is acquired that the input data belongs and that eight input data belong to the model application area of class 2. On the other hand, the monitoring unit 25 determines from the inspector model that 6 input data belong to the model application area of class 0, 6 input data belong to the model application area of class 1, and 8 input data belong to the model application area of class 2. Get which input data belongs.

In other words, the monitoring unit 25 calculates the matching rate as 100% since the matching rate of each class between the machine learning model 15 and the inspector model matches. At this timing, the respective classification results match.

As time progresses, the monitoring unit 25 determines from the machine learning model 15 to be monitored that six input data belong to the model application area of class 0, six input data belong to the model application area of class 1, It is acquired that eight input data belong to the model application area of class 2. On the other hand, the monitoring unit 25 determines from the inspector model that three input data belong to the model application area of class 0, six input data belong to the model application area of class 1, and eight input data belong to the model application area of class 2. Get which input data belongs.

That is, the monitoring unit 25 calculates the match rate for class 0 as 50% ((3/6)×100), and calculates the match rate as 100% for classes 1 and 2. That is, a change in the data distribution of class 0 is detected. At this timing, the inspector model is in a state where the three input data that are not classified into class 0 are not necessarily classified into class 0.

As time progresses further, the monitoring unit 25 determines that three input data belong to the model application area of class 0 and six input data belong to the model application area of class 1 from the machine learning model 15 to be monitored. , it is acquired that eight input data belong to the model application area of class 2. On the other hand, the monitoring unit 25 determines from the inspector model that one input data belongs to the model application area of class 0, six input data belong to the model application area of class 1, and eight input data belong to the model application area of class 2. Get which input data belongs.

That is, the monitoring unit 25 calculates the match rate for class 0 as 33% ((1/3)×100), and calculates the match rate as 100% for classes 1 and 2. That is, it is determined that the data distribution of class 0 has changed. At this timing, the machine learning model 15 does not classify the input data that should be classified as class 0, and the inspector model classifies the five input data that were not classified as class 0 as class 0. This is a condition that cannot necessarily be classified.

Here, changes in the matching rate distribution will be explained. FIG. 9 is a diagram illustrating changes in the distribution of matching rates. In FIG. 9, the horizontal axis is each inspector model, and the vertical axis is the match rate (matching rate), and shows changes in the match rate between each of the five inspector models and the machine learning model 15 for a certain class.

Assume that the size of the model application area of inspector models 1, 2, 3, 4, and 5 is such that inspector model 1 is the widest and inspector model 5 is the narrowest. In this case, as time passes from the initial stage of operation, the match rate of inspector models 5 and 4 decreases because the inspector model with a narrower model application area responds more sensitively to data distribution. The monitoring unit 25 can detect the occurrence of accuracy deterioration by detecting that the matching rate of the inspector models 5 and 4 has fallen below a threshold value. Furthermore, the monitoring unit 25 can detect a change in the trend of input data by detecting that the matching rate of most inspector models has fallen below a threshold value.

Returning to FIG. 4, the notification unit 26 is a processing unit that notifies a predetermined device of an alert or the like when deterioration in accuracy of the machine learning model 15 is detected. For example, the notification unit 26 notifies an alert when an inspector model whose match rate is below a threshold is detected, or when a predetermined number or more of inspector models whose match rate is below a threshold are detected.

Further, the notification unit 26 can also notify an alert for each class. For example, the notification unit 26 notifies an alert when a predetermined number or more of inspector models whose matching rate is below a threshold is detected for a certain class. Note that monitoring items can be arbitrarily set for each class, each inspector model, etc. Furthermore, for each inspector model, the average of the match rates for each class can be taken as the match rate for each inspector model.

[Processing flow]
FIG. 10 is a flowchart showing the flow of processing. As shown in FIG. 10, when the process is started (S101: Yes), the inspector model generation unit 21 generates training data for each inspector model (S102), and uses the training data in the generated training data. Then, training for each inspector model is executed to generate each inspector model (S103).

Next, the threshold setting unit 22 inputs the verification data in the teacher data into the machine learning model 15 and each inspector model, calculates the match rate of the output results obtained (S104), and sets the threshold based on the match rate. (S105).

After that, the deterioration detection unit 23 inputs the input data to the machine learning model 15 to obtain an output result (S106), and inputs the input data to each inspector model to obtain the output result (S107).

Then, the deterioration detection unit 23 compares the output results, that is, accumulates the distribution of the model application area in the feature space (S108), and repeats S106 and subsequent steps until the accumulated number reaches the specified number (S109: No).

Thereafter, when the accumulated number reaches the specified number (S109: Yes), the deterioration detection unit 23 calculates the match rate between each inspector model and the machine learning model 15 for each class (S110).

Here, if the matching rate does not satisfy the detection condition (S111: No), the steps from S106 onwards are repeated, and if the matching rate satisfies the detection condition (S111: Yes), the deterioration detection unit 23 notifies an alert ( S112).

[effect]
As described above, the accuracy deterioration detection device 10 generates at least one inspector model whose model application area is narrower than that of the machine learning model to be monitored. Then, the accuracy deterioration detection device 10 measures a change in the distribution of the match rate between the output of the machine learning model and the output of each inspector model for each class. As a result, the accuracy deterioration detection device 10 can detect model accuracy deterioration even in multi-class classification problems of high-dimensional data, and can detect trends in input data without using correctness information of the output of the machine learning model 15. It is possible to detect functional deterioration of a trained model due to changes over time.

FIG. 11 is a diagram illustrating a comparison result of accuracy deterioration detection of high-dimensional data. In FIG. 11, the machine learning model 15 is learned using image data of a cat whose background is often green, as training data, and accuracy deterioration detection is performed using general techniques such as ^T2 statistics, and the method according to Example 1 ( (Use of inspector model) to compare accuracy degradation detection. Note that the horizontal and vertical axes of each graph in FIG. 11 also indicate feature amounts.

As shown in FIG. 11, the machine learning model 15 learns that the training data has many green components and white components as a feature quantity. Therefore, in the above-mentioned general technique that performs principal component analysis, even if image data of a dog with a large green component is input, it is determined to be in the cat class. Furthermore, in the case of image data with an abnormally large amount of white, even if it is an image of a cat, it cannot be detected as a cat class because there are too many white features.

On the other hand, the inspector model according to the first embodiment has a narrower model application area than the machine learning model 15. For this reason, the inspector model can determine that even if image data of a dog with a large amount of green components is input, it is not in the cat class. Since the feature values have been learned accurately, the cat class can be detected.

In this way, the inspector model of the accuracy deterioration detection device 10 can detect input data with different feature amounts from learning data with high accuracy compared to general techniques. Therefore, the accuracy deterioration detection device 10 can follow changes in the distribution of input data based on the match rate between the machine learning model 15 and the inspector model, and can detect a decrease in the accuracy of the machine learning model 15.

FIG. 12 is a diagram illustrating a comparison result of detection of accuracy deterioration in multi-class classification. In FIG. 12, similarly to FIG. 11, accuracy deterioration detection using a general technique such as the ^T2 statistic is compared with accuracy deterioration detection using the method according to the first embodiment (using the inspector model).

In the general technique shown in Figure 12, the distance of the principal component from the training data group is used for measurement, so if the training data contains data from multiple classes, the range of normal data will be wide. As a result, abnormal data cannot be detected. In other words, once the range of normal data for class 0, class 1, and class 2 is determined, most of the data will fall within that range, making it difficult to judge abnormal value data that should not belong to any of them as abnormal. difficult. For this reason, it is not possible to detect that the input data has changed to the abnormal value data shown in FIG. 12, and therefore it is not possible to detect deterioration in model accuracy.

On the other hand, the inspector model according to the first embodiment has a narrower model application area than the machine learning model 15. Therefore, it is possible to distinguish between a class 0 model application area, a class 1 model application area, and a class 2 model application area. Therefore, data belonging to a region other than the model application area can be accurately detected as abnormal. Therefore, since it is possible to detect that the input data has changed to the abnormal value data shown in FIG. 12, it is possible to realize the detection of model accuracy deterioration.

[Concrete example]
Next, a specific example will be described in which an image classifier is used as the machine learning model 15 to detect accuracy deterioration due to the inspector model. An image classifier is a machine learning model that classifies input images into classes (categories). For example, on apparel mail order sites, auction sites for buying and selling clothing between individuals, images of clothing items are uploaded to the site, and the categories of the clothing items are registered on the site. In order to automatically register the category of images uploaded to the site, a machine learning model is used to predict the clothing category from the image. During system operation, if the trends (data distribution) of uploaded clothing images change, the accuracy of the machine learning model will deteriorate. In conventional technology, deterioration in model accuracy is detected by manually checking the accuracy of prediction results and calculating the accuracy rate. Therefore, by applying the method according to the first embodiment, deterioration in model accuracy is detected without using information on whether the prediction results are correct or incorrect.

FIG. 13 is a diagram illustrating a specific example using an image classifier. As shown in FIG. 13, the system shown in the specific example inputs input data to each of the image classifier, inspector model 1, and inspector model 2, and calculates the data distribution of the model application area of the image classifier and each inspector model. This is a system that uses matching rates to detect deterioration in the accuracy of image classifiers and outputs alerts.

Next, the teaching data will be explained. FIG. 14 is a diagram illustrating a specific example of teacher data. As shown in FIG. 14, the training data for the specific example shown in FIG. 13 includes a T-shirt with a label of class 0, pants with a label of class 1, professional overshirts with a label of class 2, and a class 3 label with a label of class 3. Here, image data of a dress and a coat whose label is class 4 are used. Further, image data of sandals with a label of class 5, a shirt with a label of class 6, sneakers with a label of class 7, a bag with a label of class 8, and ankle boots with a label of class 9 are used.

Here, the image classifier is a classifier using DNN that performs 10-class classification, and is trained using 1000 pieces of teacher data/class and 100 epochs of training. The inspector model 1 is a detector using a DNN that performs 10-class classification, and is trained with 200 pieces of teacher data per class and 100 epochs of training. Inspector model 2 is a detector using DNN that performs 10-class classification, and is trained using 100 pieces of teacher data/class and 100 epochs of training.

In other words, the model application area becomes narrower in the order of image classifier, inspector model 1, and inspector model 2. Note that the training data was randomly selected from the training data of the image classifier. Further, the matching rate threshold for each class is set to 0.7 for both Inspector Model 1 and Inspector Model 2.

In such a state, the input data of the system shown in FIG. 13 uses an image (grayscale) of clothing (any of the 10 classes), similar to the teacher data. Note that the input image may be in color. Input data suitable for the image classifier (machine learning model 15) to be monitored is used.

In such a state, the accuracy deterioration detection device 10 inputs the data input to the image classifier to be monitored to each inspector model, compares the outputs, and performs the comparison for each output class of the image classifier. Accumulate the results (match or non-match). Then, the accuracy deterioration detection device 10 calculates the match rate for each class from the accumulated comparison results (for example, the most recent 100 results/class), and determines whether the match rate is less than a threshold value. Then, the accuracy deterioration detection device 10 outputs an alert for detecting accuracy deterioration when the value is less than the threshold value.

FIG. 15 is a diagram illustrating the execution results of precision deterioration detection. FIG. 15 shows the execution results of a case in which only the image of class 0 (T-shirt) among the input data is gradually rotated and the tendency changes. The precision deterioration detection device 10 notifies an alert when the match rate (0.69) of the inspector model 2 falls below a threshold (for example, 0.7) when the class 0 data is rotated by 10 degrees. Note that when the data of class 0 is rotated by 15 degrees, not only the match rate of inspector model 2 but also the match rate of inspector model 1 decreases. In other words, the accuracy deterioration detection device 10 was able to detect the accuracy deterioration of the model at the stage when the correct answer rate of the image classifier decreased slightly.

By the way, in Example 1, each inspector model is generated in which the model application area is reduced by reducing training data, which is the opposite of data expansion, which is a method of increasing training data in order to expand the model application area. explained. However, even if the number of training data is reduced, the model application area may not necessarily become narrower.

FIG. 16 is a diagram illustrating an example of controlling the model application area. As shown in the upper diagram of FIG. 16, in Example 1, the training data of the inspector model is randomly reduced, and the number of training data to be reduced is changed for each inspector model, thereby reducing the model application area. Generated inspector model. However, since it is unknown how much the model application area will be narrowed by reducing which training data, intentionally adjusting the model application area to an arbitrary width is not necessarily successful. Therefore, as shown in the lower diagram of FIG. 16, there are cases where the model application area of the inspector model generated by reducing the training data is not narrowed. In this way, if the model application area is not narrowed, it will take a lot of man-hours to re-create it.

Therefore, in the second embodiment, the model application area is reliably narrowed by performing overfitting using the same training data as the machine learning model to be monitored. At this time, the width of the model application area is arbitrarily adjusted using the value of validation accuracy (accuracy rate for validation data).

FIG. 17 is a diagram illustrating an example of generating an inspector model according to the second embodiment. As shown in FIG. 17, in the second embodiment, at the timing when learning of the inspector model is executed for 30 epochs using the training data, the validation accuracy at that time is calculated and retained using the validation data. Furthermore, at the timing when the inspector model learning has been executed for 70 epochs using the training data, the validation accuracy at that time is calculated and retained using the validation data, and at the timing when the inspector model learning has been executed for 100 epochs. Calculate and maintain the validation accuracy of. Then, the state of the inspector model (for example, DNN feature amount) at each validation accuracy is held.

In this way, in the process of performing learning using training data, we monitor the validation accuracy of the inspector model that is in the middle of learning, and intentionally overfit it until the validation accuracy drops to a desired value. Generates a condition that degrades generalization performance. In other words, by maintaining the state of the inspector model with an arbitrary validation accuracy value, an inspector model with the width of the model application area arbitrarily adjusted is generated.

FIG. 18 is a diagram illustrating changes in validation accuracy. FIG. 18 shows the relationship between the number of training sessions and the learning curve during learning. The inspector model generation unit 21 of the accuracy deterioration detection device 10 according to the second embodiment reliably narrows the model application area by performing overfitting using the same training data as the machine learning model to be monitored. Generally, the more overfitted the DNN used for the inspector model is, the more optimized it becomes for training data, and the model application area will be reduced.

As shown in FIG. 18, the correct answer rate gradually increases as the number of training increases until the correct answer rate reaches 0.9. However, if the number of trainings is further increased from the number of training sessions where the accuracy rate reaches 0.9, the training accuracy (accuracy rate for the training data) will gradually increase, but the validation accuracy will decrease as overfitting progresses. . In other words, the more the model is overfitted, the narrower the model's application area becomes, and the accuracy rate decreases with small changes in input data. This is because generalization performance is lost due to overfitting, and the accuracy rate for data other than training data decreases. It can be confirmed that the model application area is narrowed by this decrease in the validation accuracy value, so by monitoring the validation accuracy value, it is possible to generate multiple inspector models with different model application areas.

FIG. 19 is a diagram illustrating generation of an inspector model using validation accuracy. FIG. 19 shows the relationship between the number of training sessions and the learning curve during learning. As described above, the size of the model application area of the inspector model can be measured by the level of the validation accuracy value. By creating multiple inspector models with different validation accuracy values, it is possible to guarantee that each inspector model has a different model application area.

As shown in FIG. 19, the inspector model generation unit 21 learns the inspector model (DNN) using the training data, and acquires various parameters of the DNN 1 when the validation accuracy value becomes 0.9. and hold it. The inspector model generation unit 21 further continues training, and various parameters of DNN2 at the time when the value of validation accuracy becomes 0.8, various parameters of DNN3 at the time when the value of validation accuracy becomes 0.6, validation accuracy Various parameters of the DNN 4 at the time when the value of validation accuracy becomes 0.4, and various parameters of the DNN 5 at the time when the value of validation accuracy becomes 0.2 are acquired and held.

As a result, the inspector model generation unit 21 can generate DNN1, DNN2, DNN3, DNN4, and DNN5 that have different model application areas with certainty. If the input data has the same distribution as the learning data, "concordance rate ≒ (validation accuracy) x correct answer rate of the model to be monitored". In other words, the distribution of matching rates is proportional to the validation accuracy of the inspector model, as shown in the lower graph of FIG.

Since the accuracy deterioration detection device 10 according to the second embodiment can definitely narrow the model application area of the inspector model, it is possible to reduce the number of man-hours such as rebuilding the inspector model that would be required if the model application area was not narrowed. In addition, the accuracy deterioration detection device 10 can measure the size of the model application area by the value of validation accuracy, so by changing the value of validation accuracy, it is possible to create an inspector model with a different model application area. It can definitely meet the requirement of "multiple inspector models in different model application areas" required for accuracy degradation detection.

In addition, the accuracy deterioration detection device 10 according to the second embodiment detects accuracy deterioration of the machine learning model 15 using a plurality of inspector models generated by the above-described method, thereby improving accuracy compared to the first example. It is possible to achieve high detection.

By the way, in Example 2, we explained an example of narrowing the model application area due to overfitting, but even if the model application area becomes narrower, the position of the decision boundary for each class does not change, and changes in the trends of input data can be detected. There is a possibility that something that cannot be done may occur.

For example, in the case of training data in which the characteristics of each class are clearly separated, the position of the decision boundary of each class may not change even if the number of training data is reduced and training is performed. If the position of the decision boundary does not change, that is, if the output of the inspector model is exactly the same as the output of the machine learning model to be monitored even outside the model application area, and all of them match, it is difficult to detect changes in the trend of the input data. Undetectable.

FIG. 20 is a diagram illustrating an example in which the boundary position between the machine learning model and the inspector model does not change. In the case of the OK example in FIG. 20, if the number of training data is reduced and training is performed, the position of the decision boundary changes, so deterioration in model accuracy can be detected from a change in the matching rate. On the other hand, in the case of the NG example shown in FIG. 20, since the position of the decision boundary does not change, the outputs of all input data match, and deterioration in model accuracy cannot be detected.

Therefore, in the third embodiment, an "other (unknown) class" is newly added to the classification classes of the inspector model. Then, the inspector model is trained using the same training data set as the machine learning model to be monitored, plus training data for other classes. Training data for other classes uses data unrelated to the original training dataset. Specifically, data that has the same format but is randomly extracted from unrelated data sets, or data that is automatically generated by setting random values for each item, etc. is used. If the output of the inspector model is any other class, the input data is determined to be outside the model application area.

FIG. 21 is a diagram illustrating the inspector model of the third embodiment. As shown in FIG. 21, the normal inspector model explained in Examples 1 and 2 classifies the feature space into a class 0 model application area, a class 1 model application area, and a class 2 model application area. This is what I did. For this reason, a normal inspector model can guarantee the class to which data that falls within these model application areas will be classified, but it is difficult to determine which class to classify data that does not fall into these model application areas. We cannot guarantee that. For example, if input data that should be classified as class 0 is classified as class 1 in the machine learning model 15 and classified as class 1 in the inspector model, the classification result will match as class 1, and the matching rate will decrease. do not.

In contrast, the inspector model of Example 3 classifies the feature space into a model application area of class 0, a model application area of class 1, and a model application area of class 2, and does not belong to any class. The area is classified as a model application area of class 10 (other class). Therefore, the inspector model of Example 3 can guarantee the classification destination class for data that falls within the model application area of each class, and also guarantees the classification destination class for data that does not fall within the model application area of each class. I can guarantee that it will be classified.

As described above, the accuracy deterioration detection device 10 according to the third embodiment provides, for each inspector model, in addition to the output class of the machine learning model 15 that is the monitoring target, the other class ( For example, create a new class 10). The accuracy deterioration detection device 10 according to the third embodiment treats input data determined to belong to other classes as "non-matching" in the model accuracy deterioration detection mechanism.

FIG. 22 is a diagram illustrating deterioration detection according to the third embodiment. As shown in FIG. 22, in the initial stage of operation, the deterioration detection unit 23 has a match rate of remains high.

Then, as time passes, the distribution of input data begins to change. In this case, for the machine learning model 15 to be monitored, each input data belongs to the model application range of each class, but for the inspector model, the deterioration detection unit 23 detects input data classified into class 10 (other classes). appears. Here, the input data classified into class 10 is a class that is not classified by the machine learning model 15, so there is no match. In other words, the matching rate gradually decreases.

After that, as more time passes, the distribution of input data begins to change further. In this case, for the machine learning model 15 to be monitored, each input data belongs to the model application range of each class, but for the inspector model, the deterioration detection unit 23 detects input data classified into class 10 (other classes). occurs frequently. Therefore, the deterioration detection unit 23 can detect that the matching rate is less than the threshold and the accuracy has deteriorated.

Here, an example of generating an inspector model according to the third embodiment will be described using the specific example described in FIG. 14. FIG. 23 is a diagram illustrating an example of teacher data for the other class (class 10). As shown in FIG. 23, the inspector model generation unit 21 uses the image data shown in FIG. 23 in addition to the image data explained in FIG. Train the inspector model. In other words, the inspector model generation unit 21 randomly sets features different from the first training data used in the machine learning model 15, and also labels data indicating that data that has not been learned by the machine learning model 15 is to be determined. An inspector model is generated by learning using second training data having .

Specifically, as teacher data for class 10, 1000 images randomly extracted from images of 1000 categories published on the Internet are used. For example, image data in a category different from the clothing shown in Figure 14, such as an apple image, a baby image, a bear image, a bed image, a bicycle image, and a fish image; in other words, it is not included in the clothing item. The inspector model is made to learn the model application area of class 10 by using image data for which a label that is not specified is set.

In Example 3, the image classifier is a classifier using DNN that performs 10 class classification, and is trained with 1000 pieces of teacher data/class and the number of training times as 100 epochs. Further, the inspector model is a detector using DNN that performs 11 class classification, and is trained with 1000 pieces of teacher data/one class and 1000 other classes, and the number of training times is 100 epochs. Note that the training data was randomly selected from the training data of the image classifier.

In such a state, the accuracy deterioration detection device 10 inputs the data input to the image classifier to be monitored into the inspector model, compares the outputs, and compares the comparison results ( matches or non-matches). Then, the accuracy deterioration detection device 10 calculates the match rate for each class from the accumulated comparison results (for example, the most recent 100 results/class), and determines whether the match rate is less than a threshold value. Then, the accuracy deterioration detection device 10 outputs an alert for detecting accuracy deterioration when the value is less than the threshold value.

FIG. 24 is a diagram illustrating the effects of the third embodiment. FIG. 24 shows the execution results of a case in which only the image of class 0 (T-shirt) among the input data is gradually rotated and the tendency changes. The accuracy deterioration detection device 10 notifies an alert when the match rate (0.68) of the inspector model falls below a threshold (for example, 0.7) when the class 0 data is rotated by 5 degrees. In other words, deterioration in model accuracy could be detected when the accuracy rate of the image classifier decreased slightly.

As described above, the accuracy deterioration detection device 10 according to the third embodiment is a highly accurate system that can detect accuracy deterioration even in the case of training data in which the characteristics of each class are clearly separated, that is, even when the decision boundary does not change. Inspector models can be generated. Further, the accuracy deterioration detection device 10 according to the third embodiment can sensitively detect changes in the distribution of input data by using an inspector model that can detect other classes. Note that the accuracy deterioration detection device 10 according to the third embodiment can also detect accuracy deterioration based on the matching rate of each class, and can also detect accuracy deterioration when the number of occurrences of other classes exceeds a threshold value. .

Now, the embodiments of the present invention have been described so far, but the present invention may be implemented in various different forms in addition to the embodiments described above.

[Numeric values, etc.]
Furthermore, the data examples, numerical values, threshold values, feature space, number of labels, number of inspector models, specific examples, etc. used in the above embodiments are just examples, and can be changed as desired. Furthermore, the input data, learning method, etc. are just examples, and can be changed arbitrarily. Furthermore, various methods such as neural networks can be adopted as the learning model.

[Model application range, etc.]
In Example 1, an example was explained in which multiple inspector models with different model application ranges are generated by reducing the number of training data, but the invention is not limited to this. By reducing it, it is also possible to generate multiple inspector models with different model application ranges. Furthermore, by reducing the number of training data included in the teacher data rather than the number of teacher data, it is also possible to generate a plurality of inspector models with different model application ranges.

[Match rate]
For example, in the above embodiment, an example was explained in which the matching rate of input data belonging to the model application area of each class is calculated, but the present invention is not limited to this. For example, deterioration in accuracy can be detected based on the match rate between the output results of the machine learning model 15 and the output results of the inspector model.

Furthermore, in the example of FIG. 8, the matching rate was calculated focusing on class 0, but it is also possible to focus on each class. For example, in the example shown in FIG. 8, after the elapse of time, the monitoring unit 25 determines that six input data belong to the model application area of class 0 and six input data belong to the model application area of class 1 from the machine learning model 15 to be monitored. It is acquired that eight input data belong to the model application area of class 2. On the other hand, the monitoring unit 25 determines from the inspector model that three input data belong to the model application area of class 0, nine input data belong to the model application area of class 1, and eight input data belong to the model application area of class 2. Get which input data belongs. In this case, the monitoring unit 25 can detect a decrease in the match rate for each of class 0 and class 1.

[Other classes]
In Example 3, a specific example was explained in which the training data for the other classes uses image data that has the same format as the original training dataset but is randomly extracted from an unrelated dataset; however, the present invention is not limited to this. isn't it. For example, in the case of data such as tables, it is also possible to generate teacher data for other classes in which random values are set for each item.

[Re-learning]
Further, when accuracy deterioration is detected, the accuracy deterioration detection device 10 can also re-learn the machine learning model 15 using the determination result of the inspector model as correct information. For example, the accuracy deterioration detection device 10 can also relearn the machine learning model 15 by generating relearning data in which each input data is used as an explanatory variable and the determination result of the inspector model for each input data is used as an objective variable. Note that when there are multiple inspector models, an inspector model with a low matching rate with the machine learning model 15 can be adopted.

[system]
Information including processing procedures, control procedures, specific names, and various data and parameters shown in the above documents and drawings can be changed arbitrarily unless otherwise specified.

Furthermore, each component of each device shown in the drawings is functionally conceptual, and does not necessarily need to be physically configured as shown in the drawings. That is, the specific form of distributing and integrating each device is not limited to what is shown in the drawings. In other words, all or part of them can be functionally or physically distributed and integrated into arbitrary units depending on various loads and usage conditions. For example, a device that executes the machine learning model 15 to classify input data and a device that detects accuracy deterioration can be implemented in separate housings.

Furthermore, all or any part of each processing function performed by each device can be realized by a CPU and a program that is analyzed and executed by the CPU, or can be realized as hardware using wired logic.

[hardware]
FIG. 25 is a diagram illustrating an example of a hardware configuration. As shown in FIG. 25, the accuracy deterioration detection device 10 includes a communication device 10a, an HDD (Hard Disk Drive) 10b, a memory 10c, and a processor 10d. Furthermore, the parts shown in FIG. 25 are interconnected by a bus or the like.

The communication device 10a is a network interface card or the like, and communicates with other devices. The HDD 10b stores programs and DB that operate the functions shown in FIG.

The processor 10d reads a program that executes the same processing as each processing unit shown in FIG. 4 from the HDD 10b, etc., and deploys it in the memory 10c, thereby operating a process that executes each function described in FIG. 4, etc. For example, this process executes the same functions as each processing unit included in the accuracy deterioration detection device 10. Specifically, the processor 10d reads a program having the same functions as the inspector model generation section 21, the threshold value setting section 22, the deterioration detection section 23, etc. from the HDD 10b. The processor 10d then executes a process that performs the same processing as the inspector model generation unit 21, the threshold value setting unit 22, the deterioration detection unit 23, and the like.

In this way, the accuracy deterioration detection device 10 operates as an information processing device that executes the accuracy deterioration detection method by reading and executing the program. Furthermore, the accuracy deterioration detection device 10 can also realize the same functions as in the above-described embodiments by reading the program from a recording medium using a medium reading device and executing the read program. Note that the programs in other embodiments are not limited to being executed by the precision deterioration detection device 10. For example, the present invention can be similarly applied when another computer or server executes the program, or when these computers or servers cooperate to execute the program.

10 Accuracy deterioration detection device 11 Communication unit 12 Storage unit 13 Teacher data DB
14 Input data DB
15 Machine learning model 16 Inspector model DB
20 Control unit 21 Inspector model generation unit 22 Threshold setting unit 23 Deterioration detection unit 24 Classification unit 25 Monitoring unit 26 Notification unit

Claims (7)

The computer is
Obtain the first output result when inputting the input data to the learned model trained using the training data ,
Extract a smaller number of training data than the number of training data from the training data, and obtain a second output result when inputting the input data to a detection model trained using the extracted training data. death,
Deterioration characterized by executing a process of detecting accuracy deterioration of the trained model due to time change in data trend based on a match rate between the first output result and the second output result. Detection method. Unlearned training data is acquired for the trained model whose number is smaller than the number of training data, and the second detection model trained using the acquired training data is 2. The deterioration detection method according to claim 1 , wherein the computer executes the process of acquiring the third output result . The acquiring process is performed for each of a plurality of detection models trained using training data that is smaller than the number of training data used for the learned model and that has a different number of training data. Get each second output result when entering the input data,
2. The detecting process detects deterioration in accuracy of the trained model based on each matching rate between the first output result and each of the second output results. Deterioration detection method. The acquiring process acquires each second output result when inputting the input data for each of the plurality of detection models,
The detecting process calculates a match rate between the first output result and the second output result for each output class of each of the plurality of detection models, and when a match rate less than a threshold occurs, 4. The deterioration detection method according to claim 3, further comprising detecting deterioration in accuracy of the trained model. When deterioration in accuracy of the learned model is detected, generating relearning data using the second output result as correct answer information,
2. The deterioration detection method according to claim 1, wherein the computer executes a process of relearning the trained model using the relearning data. to the computer,
Obtain the first output result when inputting the input data to the learned model trained using the training data ,
Extract a smaller number of training data than the number of training data from the training data, and obtain a second output result when inputting the input data to a detection model trained using the extracted training data. death,
Deterioration characterized by executing a process of detecting accuracy deterioration of the trained model due to time change in data trend based on a match rate between the first output result and the second output result. detection program. a first acquisition unit that acquires a first output result when input data is input to the learned model trained using the training data ;
Extract a smaller number of training data than the number of training data from the training data, and obtain a second output result when inputting the input data to a detection model trained using the extracted training data. a second acquisition section,
and a detection unit that detects deterioration in accuracy of the learned model due to temporal changes in data trends based on a match rate between the first output result and the second output result. Information processing device.

Images