JP7422621B2 - Recognition management device, recognition management system, and recognition management method - Google Patents

Description

The present invention relates to a recognition management device, a recognition management system, and a recognition management method.

In recent years, with the advancement of IT, a large number of sensors have been placed in society, and an extremely large amount of data has been accumulated. Under these circumstances, various measures are being considered to utilize the accumulated image data. In particular, as video content such as photos, videos, and images increases, a machine learning model that can freely detect and accurately identify objects and activities in the video is desired.

In order to train a machine learning model that can recognize arbitrary objects and activities with high accuracy, it needs to be trained on data from various classes (categories). However, in reality, it may be difficult to obtain appropriate learning data depending on the class. Therefore, a large amount of learning data for classes that are relatively easy to obtain is obtained, whereas a small amount of learning data for classes that are difficult to obtain is obtained. If a machine learning model is trained using learning data in which there is a bias in the number of samples between different classes, the machine learning model will be biased toward classes with a large number of samples, and recognition accuracy for classes with a small number of samples will be limited. Ru. Generally, a situation where there is a bias between training data depending on the class is called an "imbalanced data problem" and causes a decline in the recognition accuracy of machine learning models.

Several countermeasures to the problem of imbalanced data have been proposed.
For example, a study by Tsung Yi Lin (Non-Patent Document 1) states that ``The most accurate object detectors to date are based on a two-step approach generalized by R-CNN. A classifier is applied to a sparse set of candidate positions for an object, whereas a one-stage detector applied to a regular dense sampling of possible object positions could be faster and simpler. , falls short of the accuracy of the two-stage detector. In this paper, we investigate the cause of this. Results show that the extreme foreground-background class imbalance that occurred during training of the dense detector is the central cause. To address this class imbalance, we propose to reshape the standard cross-entropy loss to reduce the loss assigned to well-classified examples.Focal Loss The method focuses training on a sparse set of hard examples, preventing the detector from being overwhelmed by a huge number of negative examples during training.To evaluate the effectiveness of the loss, we use a simple method called RetinaNet. ``Design and train a high-density detector.'' techniques are described.

Tsung-Yi Lin et al. , “Focal loss for dense object detection,” ICCV2017

Non-Patent Document 1 describes a method of calculating weighting for each sample in the process of machine learning. More specifically, samples that are difficult to recognize are assigned higher weights, and samples that are easier to recognize are assigned lower weights.
However, in the method described in Non-Patent Document 1, since weighting is calculated for each sample, there are, for example, class A with a large amount of training data and class B with a small amount of training data, and it is difficult to recognize class A. If the absolute number of samples is greater than the absolute number of difficult samples in class B, the recognition model will be biased toward class A, which has a large amount of training data. Therefore, even if the method described in Non-Patent Document 1 is used, the so-called "imbalanced data problem" is not solved, and the accuracy of the recognition model is limited.

Therefore, the present invention calculates the recognition difficulty level for each class based on the recognition performance of the recognition model, calculates the weighting for each class based on the recognition difficulty level for each class, and uses the recognition model according to the weighting. The objective is to provide a highly accurate recognition model that can perform equal recognition processing without being biased toward a particular class by updating the model parameters of the model.

In order to solve the above problems, one of the typical recognition management devices of the present invention performs class recognition processing on data to be analyzed that includes at least one class, and Based on a recognition model that determines a predicted label that specifies a class, an error of the recognition model calculated from the predicted label determined for the analysis target data, and an input label that specifies the true class of the analysis target data. , a performance calculation unit that calculates the recognition performance of the recognition model, a difficulty calculation unit that calculates the recognition difficulty of each of the classes included in the analysis target data based on the recognition performance, and the recognition of each of the classes. and a weight calculation unit that calculates and allocates weights to each of the classes included in the data to be analyzed based on the difficulty level.

According to the present invention, the recognition difficulty level for each class is calculated based on the recognition performance of the recognition model, the weighting for each class is calculated based on the recognition difficulty level for each class, and the recognition model is By updating the model parameters, it is possible to provide a highly accurate recognition model that can perform equal recognition processing without being biased toward a particular class.
Problems, configurations, and effects other than those described above will be made clear by the description of the embodiments below.

FIG. 1 is a diagram illustrating an example of preprocessing of analysis target data according to Example 1 of the present disclosure. FIG. 2 is a diagram illustrating an example of a functional configuration of a recognition management device according to Example 1 of the present disclosure. FIG. 3 is a diagram illustrating an example of the flow of recognition performance calculation processing by the performance calculation unit in the recognition management means according to the first embodiment of the present disclosure. FIG. 4 is a diagram illustrating an example of an input data table according to Example 1 of the present disclosure. FIG. 5 is a diagram illustrating an example of a prediction data table in the recognition management means according to Example 1 of the present disclosure. FIG. 6 is a diagram illustrating an example of a recognition performance table in the recognition management means according to Example 1 of the present disclosure. FIG. 7 is a diagram illustrating an example of the flow of difficulty calculation processing by the difficulty calculation unit in the recognition management means according to the first embodiment of the present disclosure. FIG. 8 is a diagram illustrating an example of a difficulty level table in the recognition management means according to Example 1 of the present disclosure. FIG. 9 is a diagram illustrating an example of the flow of weighting calculation processing by the weighting calculation unit in the recognition management means according to the first embodiment of the present disclosure. FIG. 10 is a diagram illustrating an example of a class-based weighting table in the recognition management unit according to the first embodiment of the present disclosure. FIG. 11 is a diagram showing a functional configuration of a recognition management device according to Example 2 of the present disclosure. FIG. 12 is a diagram illustrating an example of the flow of weighting normalization processing by the weighting normalization unit in the recognition management means according to the second embodiment of the present disclosure. FIG. 13 is a diagram illustrating an example of a normalized weighting table for each class in the recognition management unit according to the second embodiment of the present disclosure. FIG. 14 is a diagram showing a functional configuration of a recognition management device according to Example 3 of the present disclosure. FIG. 15 is a diagram illustrating an example of the flow of weighted parameter determination processing by the dynamic weighted parameter determination unit in the recognition management means according to Example 3 of the present disclosure. FIG. 16 is a diagram illustrating an example of the flow of weighted parameter determination processing by the dynamic weighted parameter determination unit in the recognition management means according to Example 3 of the present disclosure. FIG. 17 is a diagram illustrating an example of a weighted parameter table in the recognition management means according to Example 3 of the present disclosure. FIG. 18 is a diagram showing a functional configuration of a recognition management device according to Example 4 of the present disclosure. FIG. 19 is a diagram illustrating an example of preprocessing of input data according to Example 4 of the present disclosure. FIG. 20 is a diagram showing the first screen of the GUI of the recognition management means according to the embodiment of the present disclosure. FIG. 21 is a diagram showing a second screen of the GUI of the recognition management means according to the embodiment of the present disclosure. FIG. 22 is a diagram showing the third screen of the GUI of the recognition management means according to the embodiment of the present disclosure. FIG. 23 is a diagram showing a fourth screen of the GUI of the recognition management means according to the embodiment of the present disclosure. FIG. 24 is a diagram showing the fifth screen of the GUI of the recognition management means according to the embodiment of the present disclosure. FIG. 25 is a diagram illustrating a computer system for implementing an embodiment of the present disclosure.

Embodiments of the present invention will be described below with reference to the drawings. Note that the present invention is not limited to this example. In addition, in the description of the drawings, the same parts are denoted by the same reference numerals.
(Background and overview of this disclosure)

As mentioned above, in machine learning, the "imbalanced data problem" in which there is bias between learning data depending on the class is a serious problem that limits the recognition accuracy of recognition models.
To illustrate an example of the problem of unbalanced data, consider a recognition system that performs activity detection of, for example, worker activity. Workers, such as carpenters, typically do not perform all activities on the job with the same frequency. For example, a worker may perform the activity of driving a nail many times a day, but only perform the activity of climbing a ladder once a week. Therefore, the unequal frequency of these activities makes it difficult to collect the same amount of training data samples for all activities, resulting in bias among training data by class.

An activity recognition model trained using such an unbalanced training dataset can achieve good recognition accuracy for classes with rich training data (e.g., the nail-driving activity class); Recognition accuracy for a small number of classes (for example, a class of cuactivities climbing a ladder) is insufficient. This is because in conventional machine learning methods, all samples are weighted equally. As a result, the recognition model is updated more from the class that has more samples, and is biased toward that class.
Note that "weighting" here is a measure that defines the influence of a certain class on the model parameters of a recognition model. Classes with higher weights have more influence on the recognition model, and classes with lower weights have less influence on the recognition model.

The easiest way to solve this problem of unbalanced data is to collect more data samples for classes with a small amount of training data. Collecting more data samples is not practical when responding to hazardous activities.

There are traditional methods that have attempted to resolve class imbalance without collecting more data.
For example, so-called "weighted loss" is one of the techniques used in the past. In weighted loss,
Different weights are given to different samples in order to balance the number of updates of the recognition model with classes of rich training data and the number of updates of the recognition model with classes of small amount of training data. As one method using weighted loss, there is the above-mentioned non-patent document 1.

A static class-based weighting method is also possible. In such static class-based weighting methods, the weights for each class are calculated before the start of the machine learning process and are kept fixed throughout the learning process. As a result, a higher weighting is assigned to a class with a small number of data samples, so that a balance can be achieved with a class with a large number of data samples.

However, none of the conventional techniques satisfactorily solves the problem of unbalanced data. For example, as mentioned above, in the method described in Non-Patent Document 1, weighting is calculated for each sample, so there are, for example, class A with a large amount of training data and class B with a small amount of training data; If the absolute number of samples that are difficult to recognize in class A is greater than the absolute number of difficult samples in class B, the recognition model will be biased toward class A, which has a large amount of training data.
Furthermore, in static class-based weighting methods, the weighting of each class is fixed throughout the learning process, so the recognition model may be biased toward classes with a small amount of training data.

Therefore, in the present invention, the recognition difficulty level for each class is calculated based on the recognition performance of the recognition model, the weighting for each class is calculated based on the recognition difficulty level for each class, and the recognition model is By updating the model parameters, it is possible to provide a highly accurate recognition model that can perform equal recognition processing without being biased toward a particular class.

In the description of the embodiments described below, an example will be described in which an image is used as input data, but the present disclosure is not limited to this, and any media data such as text, video, etc. may be used. Furthermore, for convenience of explanation, a case will be described below in which a task based on machine learning is a task related to recognition, but the present disclosure is not limited to this, and a task based on machine learning may be any task related to recognition. It may be of.

Embodiment 1 of the present disclosure will be described below with reference to FIGS. 1 to 10.

FIG. 1 is a diagram illustrating an example of preprocessing of analysis target data according to Example 1 of the present disclosure. After undergoing the preprocessing shown in FIG. 1, the data to be analyzed is input to a recognition model described later.

First, data to be analyzed is stored in the data storage unit 101, which is either a hard disk, a drive memory, a solid state drive, or a server memory. This data to be analyzed may be, for example, data prepared in advance for training a recognition model, or may be data provided to a third party and which is the subject of inference.

The data reading unit 102 reads the analysis target data stored in the data storage unit 101 as well as the input label corresponding to the analysis target data. The input label here is, for example, a label that becomes the ground truth and indicates the true class of the data to be analyzed. Here, the data reading unit 102 may convert the input label into a numerical format. For example, in a cat/dog binary classification task, images of cats may be labeled as "0" and images of dogs as "1". Similarly, for a C-value classification task, the data is labeled from 0 to C-1. These input labels are stored in the input label DB 107 shown in FIG.
Note that the format of the input label is not particularly limited, and may be expressed in any format such as one-hot encoding, for example.

Next, the analysis target data acquired by the data reading unit is processed by the data normalization unit 103. The data normalization unit 103 normalizes the image intensity value of the analysis target data to a range of 0 to 1 by processing the analysis target data using a general normalization algorithm such as minimum/maximum normalization.
Note that the normalization algorithm used here is not particularly limited, and any normalization algorithm such as average subtraction or standardization may be used.

Next, the normalized data to be analyzed is processed by the data correction unit 104. Here, the data correction unit 104 may perform various correction processes such as horizontal flipping, random image cropping, and image rotation on the normalized data to be analyzed. The purpose of these correction processes is to improve the diversity of the data to be analyzed and to prevent overfitting in the learning process. This can prevent the recognition model described below from being biased towards specific general features in the data.
Note that the type of correction processing used here is not particularly limited, and any method may be used.

After the correction process is completed, the data to be analyzed is transferred to the graphics processing unit (GPU) memory 105. If the GPU memory 105 is not available, the data to be analyzed may be processed by the CPU memory, but from the viewpoint of improving processing speed, a configuration using the GPU memory 105 is desirable.
Note that after the processing by the GPU memory 105 is completed, the preprocessed analysis target data is stored in the preprocessed analysis target data DB 106.

After the above-described processing is completed, the preprocessed data to be analyzed and the input label corresponding to the preprocessed data to be analyzed are input to the recognition model described below.

FIG. 2 is a diagram showing a functional configuration of the recognition management device 200 according to the first embodiment of the present disclosure.

First, the analysis target data stored in the preprocessed analysis target data DB 106 described with reference to FIG. 1 is input to the recognition model 203. The type and configuration of the recognition model 203 here may be selected as appropriate depending on the recognition task. For example, the recognition model 203 may be a classification model such as ResNet, LeNet, or ImageNet, a detection model such as SSD or YOLO, or any machine learning model such as SVM or kNN. Here, for convenience of explanation, the recognition model 203 shown in FIG. 2 is assumed to be a classification model in order to explain the case where the recognition management means according to the embodiment of the present disclosure is applied to a classification task as an example.

The model parameter DB 204 is configuration variables that control the behavior of the recognition model 203. The recognition model 203 performs recognition processing on the data to be analyzed according to the model parameters stored in the model parameter D204B. The initial values of these model parameters may be, for example, random values or values obtained from a specific distribution.

The recognition model 203 performs class recognition processing on the analysis target data input from the preprocessed analysis target data DB 106 according to the model parameters stored in the model parameter DB 204, thereby generating a predicted label that identifies the class of the analysis target data. Determine. The class here is information that defines the category of objects and activities included in the data to be analyzed.
For classification tasks, the output of recognition model 203 is a probability distribution over a predetermined number of classes/labels. In this probability distribution, the class/label with the highest probability is selected as the "predicted label" of the data to be analyzed and stored in the predicted label DB 205.
Note that the configuration of the output of the recognition model may differ depending on the task to be executed.

Next, the error calculation unit 206 compares the predicted label stored in the predicted label DB 205 and the input label stored in the input label DB 107 (that is, the ground truth indicating the true class of the data to be analyzed). , calculate the error in the prediction of the recognition model 203. Here, in order to calculate the error of the recognition model 203, the error calculation unit 206 may calculate the error using an error function selected by the user. The error function here may be, for example, a so-called cross-entropy function. In principle, the higher the accuracy of the recognition model 203, the lower the prediction error should be.

Next, the performance calculation unit 207 calculates the recognition accuracy of the recognition model 203 for each class included in the data to be analyzed. Recognition performance here is a measure of the accuracy with which a recognition model can correctly predict the class of data to be analyzed.
Furthermore, the difficulty level calculation unit 208 calculates the recognition difficulty level of each class in the analysis target data based on the recognition performance of the recognition model 203 calculated by the performance calculation unit 207. The recognition difficulty level (hereinafter referred to as "difficulty level") is a measure of how difficult it is for the recognition model 203 to correctly recognize a specific class.
Further, a dynamic weighting calculation unit (hereinafter referred to as “weighting calculation unit”) 209 calculates weighting for each class in the data to be analyzed based on the difficulty level of each class calculated by the difficulty level calculation unit 208. ,assign. This gives higher weighting to classes that are more difficult to recognize.
Note that details of the performance calculation unit 207, difficulty level calculation unit 208, and weighting calculation unit 209 will be described later.

Furthermore, as shown in FIG. 2, the weighting calculation unit 209 may use weighting parameters. This weighting parameter is a hyperparameter that defines relative differences between classes. By setting this weighting parameter, it is possible to specify how much the weighting of classes that are easier to recognize is reduced compared to classes that are more difficult to recognize.
This weight parameter is set by the weight parameter setting section 210 shown in FIG. In the first embodiment of the present disclosure, a fixed value set by the user via the weight parameter setting unit 210 is used as the weight parameter. The value of this weighting parameter may be selected from the range of 0 to 5, for example, but is not limited to this range.

By multiplying the weighting calculated by the weighting calculation unit 209 and the error calculated by the error calculation unit 206, the specific gravity error can be obtained. As an example, assume that an image I belonging to class Y ₁ is input to the recognition model 203, and as a result, the prediction error by the recognition model 203 is E _I. In this case, if the weighting calculated for class Y ₁ is W ₁ , the specific gravity error of image I is W ₁ E _I . The updating unit 211 shown in FIG. 2 uses this specific gravity error to update the model parameters stored in the model parameter DB 204.
The flow of inputting an image to the recognition model 203 and updating the model parameters using the specific gravity error calculated from the class recognition process for the image is performed iteratively, and as the number of iterations increases, the recognition model 203 is updated. Recognition accuracy improves.
Although the functional configuration for learning and training the recognition management device 200 according to the embodiment of the present disclosure has been described above, the recognition model learned through the above process can be applied to any recognition task. It goes without saying that you can do it. For example, one possible application of the recognition management means according to the embodiment of the present disclosure is human activity detection. When the recognition management means according to the embodiments of the present disclosure is applied to human activity detection, the data to be analyzed may be videos, sounds, images, etc. of humans performing specific activities, and the data disclosed in the present disclosure may be As a result of the recognition management means according to the embodiment processing this data to be analyzed, the class of activity being performed in the data to be analyzed (driving a car, holding a specific object, etc.) is recognized. Can be done.
As an example, the recognition management device according to the embodiment of the present disclosure is connected to a client terminal via a communication network, and analyzes inference data for activity detection received from the client terminal using a trained recognition model. By doing so, the activity class corresponding to the inference data may be predicted, and an activity detection result indicating the predicted activity class may be transmitted to the client terminal.

FIG. 3 is a flowchart showing the flow of recognition performance calculation processing 360 in the recognition management means according to the first embodiment of the present disclosure. The recognition performance calculation process 360 shown in FIG. 3 is executed by the performance calculation unit 207 shown in FIG. 2, for example, and is a process for calculating the performance of the recognition model 203.

First, in steps 361 and 362, the performance calculation unit calculates the predicted label (for example, the predicted label stored in the predicted label DB 205 shown in FIG. 2) and the input label (for example, the predicted label stored in the input label DB 107 shown in FIG. 2). input label). These input labels are labels indicating the true class of the data to be analyzed, which are provided by the user or administrator, as shown in FIG. 1, for example. Further, the predicted label is a label predicted by the recognition model 203 shown in FIG. 2 by performing class recognition processing on the data to be analyzed.
Note that the data structure of these labels will be described later with reference to FIGS. 4 and 5.

Next, in step 363, the performance calculation unit declares a variable i pointing to the class number, and sets the initial value of the variable i to "1" so that the variable i points to the first class.

Next, in step 364, the performance calculation unit determines the number of samples correctly predicted by the recognition model in the i-th class (eg, n _i ) by comparing the predicted label and the input label. Here, "predicted correctly" means that the predicted label predicted by the recognition model matches the input label indicating the actual class.

Next, in step 365, the performance calculation unit calculates the recognition performance of the recognition model for each class included in the analysis target data using the number of samples correctly predicted by the recognition model determined in step 364. do. Here, the recognition performance A _i of the recognition model in the i-th class is determined by the following equation 1.

Next, in step 366, the performance calculation unit increments the value of the variable i by one in order to proceed to the next class included in the data to be analyzed.

Next, in step 367, the performance calculation unit determines whether the value of variable i is less than or equal to C, which is a value indicating the total number of classes. If the value of variable i is less than or equal to C, the process proceeds to step 364, and steps 364 and 365 are performed for the next class. If the value of variable i is greater than C, the process proceeds to step 368.

Next, after the recognition performance of the recognition model for all classes included in the data to be analyzed is calculated, in step 368, the performance calculation unit calculates the recognition performance for each class {A ₁ , A ₂ , ...A _C } is output.

Through the recognition performance calculation process 360 described above, it is possible to calculate the recognition performance of the recognition model for each class included in the data to be analyzed.

FIG. 4 is a diagram illustrating an example of an input data table 400 including analysis target data and input labels in the recognition management means according to the first embodiment of the present disclosure. As shown in FIG. 4, each row included in the input data table 400 is composed of one image data number 401 and one input label 402.

Each image included in the analysis target data is associated with an image data number 401 for uniquely identifying the image. For example, if the data to be analyzed includes M images, the column of image data number 401 has entries from 0 to M-1. Further, each image included in the analysis target data is associated with an input label 402 indicating the class of the image. As described above, the input label 402 shown in FIG. 4 is a label that serves as the ground truth, and is used together with the predicted label to calculate the performance of the recognition model. In FIG. 4, the label of input label 402 is indicated by Y, and the value of the subscript indicates the number of the label. As an example, Y ₁ may indicate a first class label and Y ₂ may indicate a second class label. In the case of a cat/dog binary classification task, if the class of the image is determined to be cat, the input label of the image may be recorded as "cat". One row of the image data table consists of one image data serial number and one actual label.

FIG. 5 is a diagram illustrating an example of a prediction data table 500 in the recognition management means according to Example 1 of the present disclosure. As shown in FIG. 5, each row included in the prediction data table 500 is composed of one image data number 401 and one prediction label 501.

Each image included in the analysis target data is associated with an image data number 401 for uniquely identifying the image. Furthermore, each image included in the analysis target data is also associated with a predicted label 501. These predicted labels 501 are labels predicted by a recognition model by performing class recognition processing on each image included in the analysis target data (that is, the image corresponding to image data number 401). For the C value recognition task, the predicted labels will be the labels selected from C and belong to {Y ₁ , Y ₂ ,...Y _C }.

FIG. 6 is a diagram illustrating an example of a recognition performance table 600 in the recognition management means according to the first embodiment of the present disclosure. As shown in FIG. 6, each row included in the recognition performance table 600 is composed of one class label 601 and one recognition performance 602.

The recognition performance table 600 shown in FIG. 6 is a table that shows the recognition performance of the recognition model for each class included in the analysis target data, which is generated by the recognition performance calculation process 360 described with reference to FIG. 3, for example. As shown in FIG. 6, the recognition performance table 600 has a class label 601 indicating a specific class label. In the case of a C value recognition task, the recognition performance table 600 includes class labels from 1 to C. Furthermore, the recognition performance table 600 shows, for each class label 601, the recognition performance 602 of the recognition model for the class label. For example, as shown in FIG. 6, the recognition performance for each class of the recognition model is listed as {A ₁ , A ₂ , . . . A _C }.
As described above, the recognition performance for each class shown in the recognition performance table 600 is used for calculating the difficulty level for each class, which will be described later.

FIG. 7 is a diagram illustrating an example of the flow of difficulty level calculation processing 700 by the difficulty level calculation unit in the recognition management means according to the first embodiment of the present disclosure. The difficulty level calculation process 700 shown in FIG. 7 is executed by, for example, the difficulty level calculation unit 208 shown in FIG. 2, and is a process for calculating the difficulty level for each class.

First, in step 701, the difficulty calculation unit inputs the recognition performance {A ₁ , A ₂ , . . . A _C } for each class of the recognition model calculated by the performance calculation unit.

Next, in step 702, the difficulty calculation unit declares a variable i that indicates the class number, and sets the initial value of the variable i to "1" so that the variable i indicates the first class.

Next, in step 703, the difficulty level calculation unit calculates the difficulty level D _i of the i-th class specified by the variable i. The difficulty level D _i of the i-th class is determined by Equation 2 below. The difficulty level determined by Formula 2 is a numerical value within the range of 0 to 1.

When the recognition performance of the recognition model for a particular class is high, the class is considered to be an "easy to recognize" class, and the difficulty level calculated by Equation 2 becomes a lower value. On the other hand, if the recognition performance of the recognition model for a particular class is low, the class is considered to be a "difficult to recognize" class, and the difficulty level calculated by Equation 2 becomes a higher value. The difficulty level calculation in step 703 is dynamically performed for each epoch in the learning process. Epoch here refers to the period during which a recognition model processes a particular data set. Furthermore, one epoch includes multiple batches. One batch is a collection of multiple samples. For example, 1000 samples are divided into 10 batches of 100 samples each, and the period of processing these 10 batches is one epoch.
The recognition model is trained iteratively and the recognition performance for each class of the recognition model increases as epochs pass. As a result, the difficulty level of each class should decrease.

Next, in step 704, the difficulty level calculation unit increments the value of the variable i by one in order to proceed to the next class included in the data to be analyzed.

Next, in step 705, the difficulty level calculation unit determines whether the value of the variable i is less than or equal to C, which is a value indicating the total number of classes. If the value of variable i is less than or equal to C, the process proceeds to step 703, and step 703 is performed for the next class. If the value of variable i is greater than C, the process proceeds to step 706.

Next, after the difficulty levels for all classes included in the data to be analyzed are calculated, in step 706, the difficulty level calculation unit calculates the difficulty level for each class {D ₁ , D ₂ ,...D _C }. Output.
Note that, as described above, the difficulty level calculation here may be performed dynamically for each epoch in the learning process, but is not limited to this, and may be calculated at a frequency specified by the user. .

The difficulty level calculation process 700 described above allows the difficulty level of each class included in the analysis target data to be calculated.

Note that although an example of the process of calculating the difficulty level for each class has been described above, the difficulty level calculation in the present disclosure is not limited to this. For example, the difficulty level for each class may be the average value of the difficulty levels of samples belonging to the class. In this case, there are several possible ways to calculate the difficulty level of each sample in a particular class. In one method, the error calculated for each sample by the error calculation unit described above (for example, the error calculation unit 206 shown in FIG. 2) may be used as the difficulty level of the sample. This is because, as a general rule, samples with higher errors tend to have higher recognition difficulty.

これにより、個別のサンプルの難易度から、クラス全体の難易度を計算することができる。 Another method for calculating the difficulty level of each sample is that when sample s belonging to class c included in the data to be analyzed is input to the recognition model, the recognition model calculates the probability distribution over all possible classes. Output. Here, let p _y be the probability that sample s belongs to class c. In this case, the recognition difficulty of sample s is calculated as (1-p _y ). Furthermore, the difficulty level of the entire class c can be calculated from the difficulty level of the sample s. When class c includes N _c samples, the difficulty level D _C of class c is determined by Equation 3 below.

This allows the difficulty level of the entire class to be calculated from the difficulty level of each individual sample.

FIG. 8 is a diagram illustrating an example of a difficulty level table 800 in the recognition management means according to the first embodiment of the present disclosure. As shown in FIG. 8, each row included in the difficulty level table 800 includes one class label 601 and one difficulty level 801.

A difficulty level table 800 shown in FIG. 8 is a table showing the difficulty level of each class included in the analysis target data, which is generated by the difficulty level calculation process 700 described with reference to FIG. 7, for example. As shown in FIG. 8, the difficulty level table 800 has a class label 601 indicating a specific class label. For the C value recognition task, the difficulty level table 800 includes class labels from 1 to C. Further, the difficulty level table 800 shows, for each class label 601, a difficulty level 801 indicating the difficulty level of the class label. For example, as shown in FIG. 8, the difficulty level for each class of the recognition model is listed as {D ₁ , D ₂ , . . . D _C }.
As described above, the difficulty level for each class shown in the difficulty level table 800 is used in the weighting calculation for each class, which will be described later.

FIG. 9 is a diagram illustrating an example of the flow of weighting calculation processing 900 by the weighting calculation unit in the recognition management means according to the first embodiment of the present disclosure. Weighting calculation processing 900 shown in FIG. 9 is executed, for example, by the weighting calculation unit 209 shown in FIG. 2, and is a process for calculating weighting for each class.

First, in step 901, the weighting calculation unit inputs the difficulty level {D ₁ , D ₂ , . . . D _C } of each class calculated by the difficulty calculation unit.

Next, in step 902, the weight calculation unit obtains the weight parameters set by the weight parameter setting unit. As described above, this weighting parameter is a hyperparameter that defines the relative difference between classes, and in the first embodiment, it is a parameter selected by the user. By setting this weighting parameter, it is possible to specify how much the weighting of classes that are easier to recognize is reduced compared to classes that are more difficult to recognize.

Next, in step 903, the weight calculation unit declares a variable i that indicates the class number, and sets the initial value of the variable i to "1" so that the variable i indicates the first class.

Next, in step 904, the weighting calculation unit calculates the weighting of each class based on the weighting parameter and the difficulty level for each class. Here, the i-th weighting W _i is determined by Equation 4 below.

Next, in step 905, the weight calculation unit increments the value of the variable i by one in order to proceed to the next class included in the data to be analyzed.

Next, in step 906, the weight calculation unit determines whether the value of variable i is less than or equal to C, which is a value indicating the total number of classes. If the value of variable i is less than or equal to C, the process proceeds to step 904, and step 904 is performed for the next class. If the value of variable i is greater than C, the process proceeds to step 907.

Next, after the weights for all classes included in the data to be analyzed are calculated, in step 907, the weight calculation unit outputs the weights {W ₁ , W ₂ , . . . W _C } for each class.

By the weighting calculation process 900 described above, a class that is more difficult to recognize is assigned a higher weight than a class that is easier to recognize. Additionally, the weighting parameter defines the relative difference between classes that are easy to recognize and classes that are difficult to recognize. The weighting here is performed dynamically for each epoch in the learning process. Furthermore, the weighting of each class calculated here is used to calculate a specific weight error for updating the recognition model.
Note that since the difficulty level of each class changes as the learning of the recognition model progresses, the weighting of each class also changes according to the difficulty level of each class.

FIG. 10 is a diagram illustrating an example of a class weighting table 1000 in the recognition management unit according to the first embodiment of the present disclosure. As shown in FIG. 10, each row included in the class weighting table 1000 is composed of one class label 601 and one class weighting 1001.

The class-based weighting table 1000 shown in FIG. 10 is a table that shows the weighting of each class included in the analysis target data, which is generated by the weighting calculation process described with reference to FIG. 9, for example. As shown in FIG. 10, the class weighting table 1000 has a class label 601 indicating a specific class label. In the case of the C value recognition task, the class weighting table 1000 includes class labels from 1 to C. Further, the class weighting table 1000 shows, for each class label 601, a class weighting 1001 indicating the weighting of the class label. For example, as shown in FIG. 10, the weights for each class are listed as {W ₁ , W ₂ , . . . W _C }.

Embodiment 2 of the present disclosure will be described below with reference to FIGS. 11 to 13. In the first embodiment described above, the difficulty level calculation unit 208 calculated the difficulty level of each class using the recognition performance for each class of the recognition model 203 calculated by the performance calculation unit 207. Then, the weighting calculation unit 209 calculated the weighting to be assigned to each class using these difficulty levels. These weights are used to calculate the specific gravity error described above. Moreover, since this process is repeated for each epoch, as the training progresses, the recognition performance of the recognition model 203 improves more and more, and as a result, the difficulty level of each class decreases. Further, as the difficulty level of the class decreases, the weighting of each class calculated by the weighting calculation unit 209 naturally decreases. Since the specific gravity error is the product of the calculated weighting and the prediction error of the recognition model 203, as the weighting progresses, the specific gravity error becomes small compared to the actual prediction error. When the model parameters of the recognition model 203 are updated using such a small specific gravity error, the learning speed of the recognition model 203 becomes slow. Example 2 of the present disclosure, which will be described below, takes this problem into consideration.

FIG. 11 is a diagram showing a functional configuration of a recognition management device 1100 according to Example 2 of the present disclosure. The recognition management device 1100 according to the second embodiment of the present disclosure differs from the recognition management device 200 according to the first embodiment of the present disclosure in that it includes a weighting normalization unit 1101, and the other configurations are substantially the same. .

The flow of processing by the recognition management device 1100 is also substantially the same as that of the recognition management device 200 described with reference to FIG. The solution target data stored in the preprocessed solution target data DB and the input labels stored in the input label DB 107 are created by the same process as shown in FIG. 1 and input to the recognition model 203. Further, the predicted label output from the recognition model 203 is stored in the predicted label DB 205. The error calculation unit 206 calculates the prediction error of the recognition model 203 by comparing these predicted labels with the input label indicating the actual class of the data to be analyzed. The performance calculation unit 207 calculates the recognition performance of the recognition model 203 for each epoch based on the prediction error of the recognition model 203. Further, the difficulty level calculation unit 208 uses the recognition performance of the recognition model 203 calculated by the performance calculation unit 207 to calculate the difficulty level of each class included in the data to be analyzed. The weighting calculation unit 209 calculates the weighting of each class using the weighting parameter selected by the user via the weighting parameter setting unit 210 and the difficulty level for each class calculated by the difficulty level calculation unit 208.

However, the recognition management device 1100 according to the second embodiment is different from the recognition management device according to the first embodiment of the present disclosure in that the weighting of each class is processed by the weighting normalization unit 1101 after the weighting of each class is calculated. This is different from device 200. The weighting normalization unit 1101 is a functional unit that normalizes the weighting of each class using a method selected by the user. Details of the processing by the weighted normalization unit 1101 will be described later.
Next, by multiplying the normalized weighting of each class by the prediction error of the recognition model 203 calculated by the error calculation unit 206, a specific gravity error can be obtained. Thereafter, the updating unit 211 updates the model parameters of the recognition model 203 stored in the model parameter DB 204 using this specific gravity error.

By normalizing the weighting of each class using the recognition management device 1100 according to the second embodiment described above, the weighting of each class is prevented from decreasing excessively as the recognition model 203 learns, and the recognition management device 1100 of the recognition model 203 is It is possible to prevent the learning speed from decreasing.

FIG. 12 is a diagram illustrating an example of the flow of weighting normalization processing 1200 by the weighting normalization unit in the recognition management means according to the second embodiment of the present disclosure. Weighting normalization processing 1200 shown in FIG. 12 is executed by, for example, the weighting normalization unit 1101 shown in FIG. 11, and is processing for normalizing the weighting of each class.

First, in step 1201, the weight normalization unit inputs the weights {W ₁ , W ₂ , . . . , W _C } of each class calculated by the weight calculation unit described above. Here, the weighting data structure is as described with reference to FIG. 10, for example.

Next, in step 1202, the weighted normalization unit declares a variable i pointing to the class number, and sets the initial value of the variable i to "1" so that the variable i points to the first class.

ここでは、Ｃは、クラスの数であり、Ｗ_ｉは、正規化されていない重み付けである。
なお、Ｃとの乗算は、この場合の重み付けの合計を、重み付けが全て１の場合と同様に保つために行われる。 Next, in step 1203, the weighting normalization unit normalizes the weighting of the i-th class. Here, the normalized weighting NW _i (hereinafter referred to as "normalized weighting") of the i-th class is obtained by the following Equation 5.

Here C is the number of classes and W _i are the unnormalized weights.
Note that the multiplication with C is performed to keep the total weighting in this case the same as when the weightings are all 1.

Next, in step 1204, the weighting normalization unit increments the value of the variable i by one in order to proceed to the next class included in the data to be analyzed.

Next, in step 1205, the weighting normalization unit determines whether the value of the variable i is less than or equal to C, which is a value indicating the total number of classes. If the value of variable i is less than or equal to C, the process proceeds to step 1203, and step 1203 is performed for the next class. If the value of variable i is greater than C, the process proceeds to step 1206.

Next, after the weights of all classes included in the data to be analyzed are normalized, in step 1206, the performance calculation unit calculates the normalized weights for each class {NW ₁ , NW ₂ ,...NW _C }. Output.

The weighting normalization process 1200 described above allows the weighting of each class included in the data to be analyzed to be normalized.

FIG. 13 is a diagram illustrating an example of a class-specific normalization weighting table 1300 in the recognition management unit according to the second embodiment of the present disclosure. As shown in FIG. 13, each row included in the normalized weighting table 1300 for each class is composed of one class label 601 and one normalized weighting 1301.

The normalized weighting table 1300 for each class shown in FIG. 13 is a table showing the normalized weighting of each class included in the data to be analyzed, which is generated by the weighted normalization process described with reference to FIG. 12, for example. As shown in FIG. 13, the normalized weighting table 1300 for each class has a class label 601 indicating a specific class label. In the case of the C value recognition task, the class-specific normalized weighting table 1300 includes class labels from 1 to C. Further, the class-specific normalization weighting table 1300 shows, for each class label 601, a normalization weighting 1301 indicating the normalization weighting of the class label. For example, as shown in FIG. 13, the normalized weights for each class are listed as {NW ₁ , NW ₂ , . . . NW _C }.

Embodiment 3 of the present disclosure will be described below with reference to FIGS. 14 to 17. In the first and second embodiments described above, weighting parameters are used. Although the weighting parameters according to the first and second embodiments described above are hyperparameters selected by the user to define relative differences between classes, the present disclosure is not limited thereto. For example, in some cases, the weighting parameters selected by the user may not result in weightings that lead to good model parameters.
Therefore, in view of this problem, the third embodiment of the present disclosure relates to automatically determining weighting parameters for obtaining weighting that leads to good model parameters.

FIG. 14 is a diagram showing a functional configuration of a recognition management device according to Example 3 of the present disclosure. The recognition management device 1400 according to the third embodiment of the present disclosure differs from the recognition management device 200 according to the first embodiment of the present disclosure in that the weight parameter setting section shown in FIG. 2 is replaced with a dynamic weight parameter determination section 1401. , the other configurations are substantially the same.

The dynamic weight calculation unit 209 is a functional unit that dynamically calculates weight parameters for each epoch using the recognition performance of the recognition model 203 for each class calculated by the performance calculation unit 207. Details of the dynamic calculation of the weighting parameters here will be explained with reference to FIG. 15.
Furthermore, as described above, the difficulty level calculation unit 208 uses the recognition performance of the recognition model 203 calculated by the performance calculation unit 207 to calculate the difficulty level of each class included in the analysis target data for each epoch. . The weight calculation unit 209 calculates the weight of each class using the weight parameter determined by the dynamic weight parameter determination unit 1401 and the difficulty level for each class calculated by the difficulty level calculation unit 208. Furthermore, the weighting of each class is used to calculate the specific weight error. The updating unit 211 uses this specific gravity error to update the model parameters stored in the model parameter DB 204.

By dynamically and automatically determining the weighting parameters using the recognition management device 1400 according to the third embodiment described above, no intervention by a human user is required, and the weighting parameters are compared to the weighting parameters selected by the human user. In this way, weighting that leads to better model parameters can be obtained.

FIG. 15 is a diagram illustrating an example of the flow of weighted parameter determination processing 1500 by the dynamic weighted parameter determination unit in the recognition management unit according to the third embodiment of the present disclosure. Weighted parameter determination processing 1500 shown in FIG. 15 is executed, for example, by the dynamic weighted parameter determination unit shown in FIG. 14, and is a process for dynamically and automatically determining weighted parameters.

First, in step 1501, the dynamic weight parameter determining unit inputs the recognition performance {A ₁ , A ₂ , . . . , A _C } of the recognition model for each class, which is calculated by the performance calculation unit.

Next, in step 1502, the dynamic weight parameter determining unit declares a variable ε, and sets the initial value of the variable ε to, for example, “0.0001”. The initial value of the variable ε may be determined by the user's selection. In addition, the variable ε is a hyperparameter that takes a small positive value, and is used to prevent the denominator from becoming 0 when min _k=1,...,c A _k =0 in Equation 5, which will be described later. .

ここでは、Ａ_kは、特定のクラスｋに対する認識モデルの認識性能であり、Ｃは、クラスの合計の数である。認識モデルの認識性能が各エポック毎に変化するため、この加重パラメータは各エポック毎に計算される。 Next, in step 1503, the dynamic weighting parameter determining unit sets the ratio between the maximum value and the minimum value in the recognition performance of the recognition model as a weighting parameter. This weighting parameter is determined by Equation 6 below.

Here, A _k is the recognition performance of the recognition model for a particular class k, and C is the total number of classes. This weighting parameter is calculated for each epoch since the recognition performance of the recognition model changes for each epoch.

Next, in step 1504, the dynamic weight parameter determination unit outputs the weight parameter calculated in step 1503.

According to the weighted parameter determination process 1500 described above, if the recognition performance of the recognition model is unbalanced between classes that are difficult to recognize and classes that are easy to recognize, the weighted parameter will have a higher value. has a higher value. Similarly, when the recognition performance of the recognition model is relatively balanced between classes that are difficult to recognize and classes that are easy to recognize, the weighting parameter will be low, so the weighting of the difficult to recognize class will be a lower value.

FIG. 16 is a diagram illustrating an example of the flow of weighted parameter determination processing 1600 by the dynamic weighted parameter determination unit in the recognition management unit according to the third embodiment of the present disclosure. Weighted parameter determination processing 1600 shown in FIG. 16 is executed, for example, by the dynamic weighted parameter determination unit 1401 shown in FIG. 14, and is a process for dynamically and automatically determining weighted parameters.
Note that the weighted parameter determination process 1600 is a weighted parameter determination process that is different from the weighted parameter determination process 1500 described with reference to FIG. More specifically, weight parameter determination processing 1600 differs from weight parameter determination processing 1500 described above in that a different weight parameter is calculated for each class.

First, in step 1601, the dynamic weight parameter determination unit inputs the recognition performance {A ₁ , A ₂ , . . . , _AC } of the recognition model for each class, which is calculated by the performance calculation unit.

Next, in step 1602, the dynamic weight parameter determination unit declares a variable ε, and sets the initial value of the variable ε to, for example, “0.0001”. The initial value of the variable ε may be determined by the user's selection. In addition, the variable ε is a hyperparameter that takes a small positive value, and is used to prevent the denominator from becoming 0 when min _k=1,...,c A _k =0 in Equation 5, which will be described later. .

Next, in step 1603, the dynamic weight parameter determination unit declares a variable i that indicates the class number, and sets the initial value of the variable i to "1" so that the variable i indicates the first class.

Next, in step 1604, the dynamic weight parameter determination unit calculates the weight parameter of the i-th class. The weighting parameter τ _i of the i-th class is determined by Equation 7 below.

Next, in step 1605, the dynamic weight parameter determination unit increments the value of the variable i by one in order to proceed to the next class included in the data to be analyzed.

Next, in step 1606, the dynamic weight parameter determining unit determines whether the value of variable i is less than or equal to C, which is a value indicating the total number of classes. If the value of variable i is less than or equal to C, the process proceeds to step 1604, and step 1604 is performed for the next class. If the value of variable i is greater than C, the process proceeds to step 1607.

Next, after the weight parameters for all classes included in the data to be analyzed are calculated, in step 1607, the dynamic weight parameter determination unit calculates the weight parameters for each class {τ ₁ , τ ₂ , ... τ _C } is output.
Note that the calculation of the weighted parameter for each class described above is calculated for each epoch.

According to the weighted parameter determination process 1600 described above, the weighted parameter for classes that are difficult to recognize becomes high, and the weighted parameter for classes that are easy to recognize become low. As mentioned above, the weighting of each class is calculated as ^{a weighted parameter} (difficulty level of each class), and the difficulty level is a numerical value within the range of 0 to 1, so the weighting of classes that are easy to recognize will be higher. In contrast, the weighting of classes that are difficult to recognize remains almost the same. As a result, the relative difference between classes that are difficult to recognize and classes that are easy to recognize becomes larger.

FIG. 17 is a diagram illustrating an example of a weighted parameter table 1700 in the recognition management means according to the third embodiment of the present disclosure. As shown in FIG. 17, each row included in the weight parameter table 1700 is composed of one class label 601 and one weight parameter 1701.

A weighted parameter table 1700 shown in FIG. 17 is a table showing weighted parameters calculated for each class included in the analysis target data, which is generated by the weighted parameter determination process described with reference to FIG. 16, for example. As shown in FIG. 17, the weighted parameter table 1700 has a class label 601 indicating a specific class label. For the C value recognition task, the weight parameter table 1700 includes class labels from 1 to C. Further, the weight parameter table 1700 shows, for each class label 601, the weight parameter 1701 calculated for the class label. For example, as shown in FIG. 17, the weighting parameters for each class are listed as {τ ₁ , τ ₂ , ... τ _C }.

Also, the weighting parameters for each class are used to calculate the weighting for each class. Further, as described above, the weighting of each class is calculated as ^{the weighting parameter of class c} (difficulty level of class c).

Next, a fourth embodiment of the present disclosure will be described with reference to FIGS. 18 and 19. In the first and second embodiments described above, the same input data (data to be analyzed) was used for learning the recognition model and calculating the performance of the recognition model. In this case, the recognition model may overfit the data used as learning data. This overfitting refers to a situation in which a recognition model is overfitted to training data, resulting in a loss of adaptability to other data sets. If the recognition model overfits the learning data, even if the difficulty level of each class is calculated using the performance of the recognition model, an accurate difficulty level for each class cannot be obtained.
Therefore, in view of this problem, Embodiment 4 of the present disclosure relates to means for preventing overfitting of a recognition model to learning data.

FIG. 18 is a diagram showing a functional configuration of a recognition management device 1800 according to Example 4 of the present disclosure. The recognition management device 1800 according to the fourth embodiment of the present disclosure is different from the recognition management device 200 according to the first embodiment of the present disclosure in that two different data sets are used for learning the recognition model and calculating the performance of the recognition model. However, the other configurations are substantially the same.

More specifically, the data set input to the recognition management device 1800 according to Example 4 of the present disclosure includes analysis target data that is learning data for learning a recognition model, and data for calculating the performance of the recognition model. Contains verification data. The analysis target data and the verification data here are extracted from the same data distribution, but are different data sets. Further, the verification data here is preprocessed and stored in the preprocessed verification data DB 1801 similarly to the analysis target data described with reference to FIG. Further, a verification label indicating the true class of this verification data is stored in the verification label DB 1802.
Note that the preprocessing performed on the verification data is substantially the same as the preprocessing described with reference to FIG. 1, and the details will be described later with reference to FIG. 19.

For each epoch, preprocessed verification data is input to the recognition model 1803. The model parameters of this recognition model 1803 are recognition models that use the same model parameters as those of the recognition model 203. Furthermore, the recognition model 1803 predicts the class of the verification data based on the verification data, and outputs a predicted label indicating the predicted class to the predicted label DB 205.

The performance calculation unit 207 calculates the recognition performance of the recognition model 1803 for each epoch based on the verification labels stored in the verification label DB 1802 and the prediction labels stored in the prediction label DB 205. The recognition performance of the recognition model 1803 output here may have substantially the same data structure as the recognition performance table 600 shown in FIG. 6, for example.

The difficulty calculation unit 208 uses the recognition performance of the recognition model 1803 calculated by the performance calculation unit 207 to calculate the recognition difficulty of each class. Further, the weighting calculation unit 209 calculates the weighting of each class using the recognition difficulty level of each class and the weighting parameter set by the weighting parameter setting unit 210.

The error calculation unit 206 also calculates the predicted label predicted by the recognition model 203 by processing the analysis target data stored in the preprocessed analysis target data DB 106 and the input label stored in the input label DB 107. Based on this, the prediction error of the recognition model 203 is calculated.

The updating unit 211 calculates a specific gravity error using the prediction error calculated by the error calculation unit 206 and the weighting calculated by the weighting calculation unit 209, and then stores the specific gravity error in the model parameter DB 204 using this specific gravity error. Update model parameters.

In addition, although the case where the performance of the recognition model is calculated "for each epoch" was described above as an example, the present disclosure is not limited to this. In reality, the performance of the recognition model may change rapidly during the learning process, and therefore the difficulty level of each class may also change rapidly. For example, even during one epoch, the performance of the recognition model and the difficulty level of each class may change significantly. Therefore, even if the performance of the recognition model and the difficulty level of each class are calculated for each epoch, changes between one epoch cannot be observed. Therefore, in this embodiment, it is desirable to calculate the performance of the recognition model and the difficulty level of each class more frequently. As an example, the performance of the recognition model and the difficulty level of each class may be calculated for each predetermined number of batches specified by the user. This makes it possible to measure more detailed changes in the performance of the recognition model and the difficulty level of each class.

FIG. 19 is a diagram illustrating an example of preprocessing of input data according to Example 4 of the present disclosure. The preprocessing of the verification data shown in FIG. 19 differs from the preprocessing of the input data according to the first embodiment described with reference to FIG. and the processing is substantially similar.

The cropping unit 1901 is a functional unit that crops (cuts out) a part of the image included in the verification data. The position and size of the image area extracted by the cropping unit 1901 may be determined by the user, for example, or may be determined randomly.

When the preprocessing of the verification data is completed, the preprocessed verification data is stored in the preprocessed verification data DB 1801, and the verification label indicating the true class of the preprocessed verification data is stored in the verification label DB 1802. is stored in
(About the GUI of recognition management means)

Next, a GUI (Graphical User Interface) of the recognition management means according to the embodiment of the present disclosure will be described with reference to FIGS. 20 to 24.

FIG. 20 is a diagram showing a first screen 2000 of the GUI of the recognition management means according to the embodiment of the present disclosure. The first screen 2000 of the GUI may show, for example, an initial state of the GUI that is displayed when learning the recognition model of the recognition management means according to the embodiment of the present disclosure.

First, a first screen 2000 of the GUI allows a human user, such as a system administrator, to select values for some specific hyperparameters to begin the learning process of the recognition model. For example, as shown in FIG. 20, the user may input the number of classes 2001, which is the total number of classes included in the data to be analyzed. The user may also input the number of epochs 2002, which is the number of epochs for learning the recognition model. By increasing the number of epochs, better learning results can be obtained, but it takes longer time.

The user may also input a data route 2003 to the storage location where the data to be analyzed is stored. As an example, this data path 2003 may be a data path to a storage location of a preprocessed analysis target data DB in which analysis target data is stored. The user may also input a batch size 2004 that specifies the number of data samples included in one batch. Generally, recognition model learning is performed for each batch, but the present disclosure is not limited thereto, and recognition model learning may be performed for each predetermined number of batches.

The user may also input a loss calculation method 2005 used for learning the recognition model. As an example, the user may select one of cross-entropy loss 2006, focal loss 2007, or class difficulty-based dynamically weighted cross-entropy loss 2008 as the loss calculation method 2005.
Note that the cross-entropy loss 2006 and the focal loss 2007 are conventionally used loss calculation methods, and the class difficulty-based dynamic weighted cross-entropy loss 2008 is a loss calculation method according to the embodiment of the present disclosure described above. .
After the hyperparameter input described above is completed, the GUI screen changes to a second screen 2100 shown in FIG. 21.

FIG. 21 is a diagram showing a second screen 2100 of the GUI of the recognition management means according to the embodiment of the present disclosure. When the hyperparameter selection is completed on the first screen 2000 of the GUI described with reference to FIG. 20, a pop-up window 2101 of the second screen 2100 of the GUI shown in FIG. 21 is displayed. A pop-up window 2101 is a screen for setting the above-mentioned weighting parameters. As explained in Example 1 above, if the user wants to set the weighting parameter to a fixed value, the user selects "manual input" in the pop-up window 2101 and sets the weighting parameter as a numerical value within the range of 0 to 10. You can. Alternatively, as described in Example 3 above, the user may select "Auto Calculate" in the pop-up window 2101 if the user wants the weight parameters to be automatically determined.
After the weight parameter settings are completed, the GUI screen changes to the third screen 2200 shown in FIG. 22.

FIG. 22 is a diagram showing a third screen 2200 of the GUI of the recognition management means according to the embodiment of the present disclosure. After the weight parameter settings are completed, a new pop-up window 2201 is displayed on the third GUI screen 2200 shown in FIG. 22. A pop-up window 2201 is a screen for setting the update frequency of weighting in the learning process of the recognition model. Here, the user can set weighting updates for every predetermined number of epochs or for every predetermined number of batches. After the setting of the weighting update frequency is completed, the GUI screen changes to a fourth screen 2300 shown in FIG. 23.

FIG. 23 is a diagram showing a fourth screen 2300 of the GUI of the recognition management means according to the embodiment of the present disclosure.
As described above, by applying the recognition model learned by the recognition management means according to the embodiment of the present disclosure to a predetermined recognition task, highly accurate recognition results can be obtained. For example, one possible application of the recognition management means according to the embodiment of the present disclosure is human activity detection.

Further, when the recognition management means according to the embodiment of the present disclosure is applied to human activity detection, different weighting parameters may be set based on the importance of the activity class. The importance here is a measure indicating the priority for correctly detecting a specific activity, and may be defined by the user for each class of activity. This importance may also be expressed, for example, as a number in the range of 0 to 10 (higher numbers indicate higher importance).

The importance of each class may be set by the user via the importance input window 2301 displayed on the fourth screen 2300 of the GUI shown in FIG. For example, a user may give an importance of ``9'' to the activity class ``A person holds a weapon'' and a ``1'' to an activity class ``A person holds a book.'' You can. Thereby, a higher weighting parameter is set for a class of activities that are more important, and a lower weighting parameter is set for a class of activities that are less important.
After the importance of each class is set, the GUI screen changes to the fifth screen 2400 shown in FIG. 24.

FIG. 24 is a diagram showing a fifth screen 2400 of the GUI of the recognition management means according to the embodiment of the present disclosure. After the importance setting for each class is completed on the fourth screen 2300 of the GUI described above, a fifth screen 2400 of the GUI showing the progress of learning of the recognition model is displayed. For example, the status display window 2401 displays information such as that learning has started and that training has ended. When learning of the recognition model is completed, the learned model is stored in a predetermined storage location.
(Hardware configuration)

Next, with reference to FIG. 25, a computer system 300 for implementing an embodiment of the present disclosure will be described. The various example mechanisms and apparatus disclosed herein may be applied to any suitable computing system. The main components of computer system 300 include one or more processors 302 , memory 304 , terminal interface 312 , storage interface 314 , I/O (input/output) device interface 316 , and network interface 318 . These components may be interconnected via memory bus 306, I/O bus 308, bus interface unit 309, and I/O bus interface unit 310.

Computer system 300 may include one or more general purpose programmable central processing units (CPUs) 302A and 302B, collectively referred to as processors 302. In some embodiments, computer system 300 may include multiple processors, and in other embodiments, computer system 300 may be a single CPU system. Each processor 302 executes instructions stored in memory 304 and may include onboard cache.

In some embodiments, memory 304 may include random access semiconductor memory, storage devices, or storage media (either volatile or nonvolatile) for storing data and programs. Memory 304 may store all or a portion of the programs, modules, and data structures that perform the functions described herein. For example, memory 304 may store recognition management application 350. In some embodiments, recognition management application 350 may include instructions or writing to perform functions described below on processor 302.

In some embodiments, the recognition management application 350 may operate on semiconductor devices, chips, logic gates, circuits, circuit cards, and/or other physical hardware devices instead of or in addition to processor-based systems. It may also be implemented in hardware via. In some embodiments, recognition management application 350 may include data other than instructions or descriptions. In some embodiments, a camera, sensor, or other data input device (not shown) may be provided to communicate directly with bus interface unit 309, processor 302, or other hardware of computer system 300. .

Computer system 300 may include a bus interface unit 309 that provides communication between processor 302 , memory 304 , display system 324 , and I/O bus interface unit 310 . I/O bus interface unit 310 may be coupled to I/O bus 308 for transferring data to and from various I/O units. The I/O bus interface unit 310 connects a plurality of I/O interface units 312, 314, 316, also known as I/O processors (IOPs) or I/O adapters (IOAs), via the I/O bus 308. and 318.

Display system 324 may include a display controller, display memory, or both. A display controller may provide video, audio, or both data to display device 326. Computer system 300 may also include devices, such as one or more sensors, configured to collect data and provide the data to processor 302.

For example, the computer system 300 may include a biometric sensor that collects heart rate data, stress level data, etc., an environmental sensor that collects humidity data, temperature data, pressure data, etc., and a motion sensor that collects acceleration data, exercise data, etc. May include. Other types of sensors can also be used. Display system 324 may be connected to a display device 326, such as a standalone display screen, a television, a tablet, or a handheld device.

The I/O interface unit has the ability to communicate with various storage or I/O devices. For example, the terminal interface unit 312 may include a user output device such as a video display device, a speaker television, or a user input device such as a keyboard, mouse, keypad, touchpad, trackball, buttons, light pen, or other pointing device. It is possible to attach user I/O devices 320 such as: Using the user interface, a user operates a user input device to input input data and instructions to user I/O device 320 and computer system 300, and to receive output data from computer system 300. Good too. The user interface may be displayed on a display device, played through a speaker, or printed through a printer, for example, via the user I/O device 320.

Storage interface 314 may include one or more disk drives or direct access storage devices 322 (typically magnetic disk drive storage devices, but also an array of disk drives or other storage devices configured to appear as a single disk drive). ) can be installed. In some embodiments, storage device 322 may be implemented as any secondary storage device. The contents of memory 304 may be stored in storage device 322 and read from storage device 322 as needed. I/O device interface 316 may provide an interface to other I/O devices such as printers, fax machines, etc. Network interface 318 may provide a communication pathway so that computer system 300 and other devices can communicate with each other. This communication path may be, for example, network 330.

In some embodiments, computer system 300 is a device that receives requests from other computer systems (clients) that do not have a direct user interface, such as a multi-user mainframe computer system, a single-user system, or a server computer. There may be. In other examples, computer system 300 may be a desktop computer, a portable computer, a laptop, a tablet computer, a pocket computer, a telephone, a smart phone, or any other suitable electronic device.

Although the embodiments of the present invention have been described above, the present invention is not limited to the embodiments described above, and various changes can be made without departing from the gist of the present invention.

106 Preprocessed analysis target data DB
107 Input label DB
203 Recognition model 204 Model parameter DB
205 Predicted label DB
206 Error calculation section 207 Performance calculation section 208 Difficulty calculation section 209 Dynamic weight calculation section 210 Weight parameter setting section 211 Update section

Claims (8)

a recognition model that performs class recognition processing on data to be analyzed that includes at least one class, and determines a predicted label that identifies each class included in the data to be analyzed;
a performance calculation unit that calculates recognition performance of the recognition model based on an error of the recognition model calculated from the predicted label determined for the analysis target data and an input label specifying the true class of the analysis target data; ,
a difficulty calculation unit that calculates the recognition difficulty of each of the classes included in the analysis target data based on the recognition performance;
a weighting calculation unit that calculates and assigns a weight to each of the classes included in the analysis target data based on the recognition difficulty level of each of the classes;
a weighting parameter setting unit for setting a weighting parameter that defines a relative difference in weighting assigned to each of the first class and the second class included in the analysis target data based on user input;
a dynamic weighting parameter determination unit that sets a ratio between a maximum value and a minimum value in the recognition performance as the weighting parameter;
A recognition management device comprising: The recognition management device includes:
further comprising a weighted normalization unit;
The weighted normalization unit includes:
By normalizing the weights assigned to each class,
defining a weighting range between each of the classes;
The recognition management device according to claim 1, characterized in that: calculating a specific gravity error based on the error of the recognition model and the weighting;
The method further includes an updating unit that trains the recognition model by updating model parameters that control the behavior of the class recognition process by the recognition model using the specific gravity error, and generates a trained recognition model. The recognition management device according to claim 1 . By analyzing inference data for activity detection using the trained recognition model, a class of an activity corresponding to the inference data is predicted, and an activity detection result indicating the predicted class of the activity is output. ,
The recognition management device according to claim 3 , characterized in that: The dynamic weighting parameter determination unit includes:
Based on the importance corresponding to each class included in the inference data,
individually setting weight parameters for each class included in the inference data;
The recognition management device according to claim 4 , characterized in that: The difficulty level calculation section is
Calculating the difficulty level of each sample corresponding to the first class included in the analysis target data,
The average value of the difficulty levels calculated for each of the samples is set as the recognition difficulty level of the first class.
The recognition management device according to claim 1, characterized in that: A recognition management system in which a client terminal and a recognition management device are connected via a communication network,
The recognition management device includes:
a recognition model that performs class recognition processing on data to be analyzed that includes at least one class, and determines a predicted label that identifies each class included in the data to be analyzed;
a performance calculation unit that calculates recognition performance of the recognition model based on an error of the recognition model calculated from the predicted label determined for the analysis target data and an input label specifying the true class of the analysis target data; ,
a difficulty calculation unit that calculates the recognition difficulty of each of the classes included in the analysis target data based on the recognition performance;
a weighting calculation unit that calculates and assigns a weight to each of the classes included in the analysis target data based on the recognition difficulty level of each of the classes;
a weighting parameter setting unit for setting a weighting parameter that defines a relative difference in weighting assigned to each of the first class and the second class included in the analysis target data based on user input;
a dynamic weighting parameter determination unit that sets a ratio between a maximum value and a minimum value in the recognition performance as the weighting parameter;
calculating a specific gravity error based on the error of the recognition model and the weighting;
an updating unit that uses the specific gravity error to train the recognition model by updating model parameters that control the behavior of the class recognition process by the recognition model, and generates a trained recognition model;
Analyzing inference data for activity detection received from the client terminal using the trained recognition model, predicting a class of activity corresponding to the inference data, and indicating the predicted class of the activity. transmitting an activity detection result to the client terminal;
A recognition management system characterized by: A recognition management method implemented by computer software in a recognition management device, the method comprising:
The recognition management device includes:
including a memory and a processor;
The memory is
performing class recognition processing using a recognition model on data to be analyzed that includes at least one class, and determining predicted labels that identify each class included in the data to be analyzed;
Calculating the recognition performance of the recognition model based on the error of the recognition model calculated from the predicted label determined for the analysis target data and an input label specifying the true class of the analysis target data;
calculating the recognition difficulty level of each of the classes included in the data to be analyzed based on the recognition performance;
Calculating and assigning a weight to each class included in the analysis target data based on the recognition difficulty level of each class;
setting a weighting parameter that defines a relative difference in weighting assigned to each of the first class and the second class included in the analysis target data based on user input;
setting a ratio between a maximum value and a minimum value in the recognition performance as the weighting parameter;
A recognition management method comprising : a processing instruction for causing the processor to execute .

Images