JP7422548B2 - Label noise detection program, label noise detection method, and label noise detection device - Google Patents

Description

The present disclosure relates to a label noise detection program, a label noise detection method, and a label noise detection device.

The training images used in image classification learning are labeled with correct teacher labels. Teacher labels are attached to learning images through annotation work, but at this time, incorrect teacher labels may be attached to learning images. Erroneous teacher labels given to training images are called label noise. If learning is performed using training data that includes label noise, the learning accuracy of image classification may decrease.

Here, as an apparatus related to image classification learning, for example, an information processing apparatus disclosed in Patent Document 1 is known. The information processing device of Patent Document 1 specifies the class to which the learning data belongs based on the feature amount of the learning data, and arranges an image of the learning data at a position according to the identified class and the reliability of the class. A distribution map of training data is displayed.

Japanese Patent Application Publication No. 2018-142097

Although the information processing apparatus of Patent Document 1 arranges images of learning data at positions according to reliability, it does not detect label noise.

An object of the present disclosure is to provide a label noise detection program, a label noise detection method, and a label noise detection device that can efficiently detect label noise.

A label noise detection program according to one aspect uses an image data set that includes a learning image that is a learning target image in image classification, and a teacher label corresponding to the learning image, and detects noise in the teacher label. A label noise detection program executed in a label noise detection device for detecting a label noise, the program learning a learning model using the image data set, classifying the learning image into a class using the trained learning model, Obtain an output label and likelihood attached to the learning image based on the result of class classification, and determine whether or not the teacher label is attached incorrectly based on the output label, the likelihood, and the teacher label. and extracting, as a label noise data set, the image data set including the learning image that has been determined to have been incorrectly assigned the teacher label.

A label noise detection method according to one aspect uses an image data set that includes a learning image that is a learning target image in image classification, and a teacher label corresponding to the learning image, and removes noise in the teacher label. A label noise detection method for detecting label noise, in which a learning model is trained using the image data set, the training image is classified into classes using the trained learning model, and the training image is classified into classes based on the result of the class classification. Obtain the output label and likelihood attached to the learning image, determine whether or not the teacher label is attached incorrectly based on the output label, the likelihood, and the teacher label, and add the teacher label. Extracting the image data set including the learning image determined to be incorrect as a label noise data set.

A label noise detection device according to one aspect uses an image data set that includes a learning image that is a learning target image in image classification and a teacher label corresponding to the learning image, and detects noise in the teacher label. A label noise detection device for detecting label noise, which uses the image data set to learn a learning model, classifies the learning image using the trained learning model, and classifies the learning image based on the result of the class classification. Obtain the output label and likelihood attached to the learning image, determine whether or not the teacher label is attached incorrectly based on the output label, the likelihood, and the teacher label, and add the teacher label. The image data set including the learning image determined to be incorrect is extracted as a label noise data set.

According to the present disclosure, label noise can be detected efficiently.

FIG. 1 is a diagram showing an overview of a label noise detection device according to an embodiment. FIG. 2 is a diagram illustrating an example of a learning image to which a teacher label is attached in the image data set. FIG. 3 is a diagram illustrating an example of a learning image to which an output label is attached in the image data set. FIG. 4 is a diagram illustrating an example of processing related to label noise detection by the label noise detection device. FIG. 5 is a diagram illustrating an example of displaying a classified image data set.

Embodiments according to the present disclosure will be described in detail with reference to the drawings. In the following description, similar components may be denoted by the same reference numerals. Furthermore, duplicate explanations may be omitted. Furthermore, descriptions and illustrations of matters that are not closely related to the description of the embodiments of the present disclosure may be omitted. Note that the present disclosure is not limited to the following embodiments. Furthermore, the following embodiments include those that can be easily imagined by those skilled in the art, those that are substantially the same, and those that are within the so-called equivalent range.

(Embodiment)
FIG. 1 is a diagram showing an overview of a label noise detection device according to an embodiment. The label noise detection device 1 is a device that detects label noise included in the image data set D and corrects the detected label noise. The label noise detection device 1 detects label noise by performing image classification using a learning model such as a CNN (Convolutional Neural Network). Image classification is to classify the image (class inference), determine which class it belongs to, and attach an identifier (label) to identify the class the image represents. It is to classify by label. Note that the label noise detection device 1 is not particularly limited to image classification using a learning model, and may be applied to object detection in which objects are detected by image classification using a learning model. Here, prior to explaining the label noise detection device 1, the image data set D will be explained with reference to FIG.

FIG. 2 is a diagram illustrating an example of a learning image to which a teacher label is attached in the image data set. The image data set D includes a learning image G, which is an image to be learned in image classification, and a teacher label T corresponding to the learning image G. Further, the image data set D includes a plurality of learning images G to which teacher labels T are attached. The teacher label T is a label that identifies an object included in the learning image G, and is a correct (correct) label. As shown in FIG. 2, as an example of the image data set D, the object included in the learning image G is a car, and the correct teacher label T is a car. The teacher label T is attached to the learning image G by human annotation work. The teacher label T may be assigned an incorrect label. The incorrect teacher label T given to the learning image G becomes label noise. The image data set D including label noise becomes a label noise data set Dn. As shown in FIG. 2, as an example of the label noise data set Dn, the object included in the learning image G is a car, and the incorrect teacher label T is a person.

A label noise detection device 1 will be described with reference to FIG. The label noise detection device 1 includes an input section 3, an output section 4, a control section (processing section) 5, and a storage section 6.

The input unit 3 includes an operation input device such as a keyboard and a mouse, and an audio input device such as a microphone. The input unit 3 generates an input signal when an input operation is performed by an operator, and outputs the input signal to the control unit 5 .

The output unit 4 includes a display device such as a display, and an audio output device such as a speaker. The output unit 4 receives an output signal as a processing result processed by the control unit 5. The output unit 4 displays a display screen on a display device or outputs audio from an audio output device based on the output signal.

The storage unit 6 stores programs and data. Furthermore, the storage unit 6 may also be used as a work area for temporarily storing processing results of the control unit 5. The storage unit 6 may include any storage device such as a semiconductor storage device and a magnetic storage device. Furthermore, the storage unit 6 may include multiple types of storage devices. Furthermore, the storage unit 6 may include a combination of a portable storage medium such as a memory card and a storage medium reading device.

The storage unit 6 includes a label noise detection program P as a program. The label noise detection program P is a program for detecting label noise. Furthermore, the storage unit 6 includes the above image data set D as data. The image data set D is annotated data, and is the image data set D shown in FIG. 2 as an example.

The control unit 5 comprehensively controls the operation of the label noise detection device 1 to realize various functions. The control unit 5 includes, for example, an integrated circuit such as a CPU (Central Processing Unit). Specifically, the control unit 5 implements various functions by executing instructions included in programs stored in the storage unit 6. The control unit 5 detects the label noise data set Dn included in the image data set D, for example, by executing the label noise detection program P.

Next, with reference to FIGS. 3 and 4, processing related to label noise detection by the label noise detection apparatus 1 using the image data set D will be described. FIG. 3 is a diagram illustrating an example of a learning image to which an output label is attached in the image data set. FIG. 4 is a diagram illustrating an example of processing related to label noise detection by the label noise detection device.

In the label noise detection of this embodiment, an example will be described in which classification is performed into two classes, with the case where the teacher label T is a car as correct, and the case where the teacher label T is not a car as an error.

First, the control unit 5 executes learning of a learning model using the annotated image data set D (step S1). When training the learning model, shallow learning is performed. That is, since the image data set D includes the label noise data set Dn, shallow learning is performed to avoid excessive fitting to the label noise. Note that the shallow learning is, for example, once.

Further, in learning the learning model, the control unit 5 performs learning so that the value of the loss function included in the learning model is minimized. As the loss function, mean absolute error (MAE) is applied.

After executing step S1, the control unit 5 extracts and selects the label noise data set Dn (step S2). Specifically, in step S2, the control unit 5 performs image classification (class classification) on the learning image G using the learning model learned in step S1. The control unit 5 obtains the output label O and likelihood attached to the learning image G as a result of image classification. Here, with reference to FIG. 3, a learning image G to which an output label O is attached, which is obtained by image classification, will be described.

As shown in FIG. 3, the learning image G is associated with an output label O given as a result of image classification using a trained learning model and a likelihood. The object included in the learning image G is a car, and the output label O is a car. Therefore, in FIG. 3, the correct output label O is attached to the learning image G. Further, the learning image G is associated with a likelihood i for the output label O. The likelihood i is the degree of plausibility (likelihood) of the output label O.

After executing the image classification, the control unit 5 outputs the teacher label T (see FIG. 2) attached to the learning image G of the image data set D and the output label O (see FIG. 3) attached to the learning image G by image classification. ) to determine whether the labels match. If the result of the determination is that the teacher label T and the output label O match, the control unit 5 determines that there is no label noise, that is, the label is clean. On the other hand, if the result of the determination is that the teacher label T and the output label O do not match, the control unit 5 determines that there is label noise, that is, the teacher label T is incorrectly attached. Then, the control unit 5 extracts the image data set D determined to have label noise as a label noise data set Dn.

The control unit 5 selects a predetermined label noise data set Dn from the extracted label noise data sets Dn based on the likelihood. As an example of selecting the label noise data set Dn based on the likelihood, selection is performed using a threshold value. The control unit 5 determines whether the likelihood i assigned to the learning image G of the label noise data set Dn is greater than or equal to a predefined threshold of the likelihood i. The control unit 5 selects the label noise data set Dn for which the likelihood i is equal to or greater than the threshold value. On the other hand, the control unit 5 does not select the label noise data set Dn whose likelihood i is smaller than the threshold value.

Note that the selection of the label noise data set Dn based on the likelihood is not particularly limited to the above selection. As another example of selecting the label noise data set Dn based on the likelihood, the label noise data set Dn may be selected in descending order of likelihood. The control unit 5 selects a predetermined number of label noise data sets Dn in descending order of the likelihood i assigned to the learning image G of the label noise data sets Dn. The predetermined number is arbitrary, but is a number that allows annotation work, for example, in replacing the output label O, which will be described later.

After executing step S2, the control unit 5 replaces the output label O of the selected label noise data set Dn (step S3). Specifically, the control unit 5 outputs information on the selected label noise data set Dn to the output unit 4 after executing step S2. The operator inputs information on the correct output label O to be replaced via the input unit 3 based on the information on the output label noise data set Dn. The control unit 5 changes the output label O of the label noise data set Dn based on the input information input from the input unit 3.

After executing step S3, the control unit 5 updates the replaced label noise data set Dn as a new image data set D (step S4). The control unit 5 sets the output label O after the replacement as the teacher label T by executing step S4. After executing step S4, the control unit 5 determines whether label noise removal is completed (step S5). As the determination in step S5, the control unit 5 may determine that label noise removal has been completed, for example, when the label noise data set Dn has not been extracted in step S2. Further, as the determination in step S5, for example, when the likelihood i of the label noise data set Dn extracted in step S2 is smaller than the threshold value, the control unit 5 determines that label noise removal has been completed. Good too. The determination as to whether label noise removal is completed is not particularly limited.

When the control unit 5 determines in step S5 that label noise removal is completed (step S5: Yes), it ends the processing related to label noise detection. On the other hand, if the control unit 5 determines in step S5 that the removal of label noise is not completed (step S5: No), the control unit 5 proceeds to step S1 again to remove the label using the updated image data set D. Executes processing related to noise detection. Therefore, the control unit 5 repeatedly executes steps S1 to S5 until label noise removal is completed.

Here, the control unit 5 may display the image data set D subjected to image classification in step S2 on the output unit 4 as a histogram distribution as shown in FIG. FIG. 5 is a diagram illustrating an example of displaying a classified image data set. In FIG. 5, the horizontal axis represents the likelihood (class likelihood), and the vertical axis represents the number of learning images G. On the horizontal axis of FIG. 5, the likelihood is, for example, between 0 and 1, in increments of 0.1, and divided into 10 equal parts. Note that the number of divisions and the interval of the likelihood on the horizontal axis in FIG. 5 are not particularly limited to the above. Further, in FIG. 5, the classified image data set D includes an image data set D of car_clean, an image data set D of Ped_clean, and an image data set D of Ped_noise. The image data set D of car_clean is an image data set D in which the teacher label T and the output label O match the car. The Ped_Clean image data set D is an image data set D in which the teacher label T and the output label O match for each person. The image data set D of Ped_noise is an image data set D (label noise data set Dn) in which the teacher label T and the output label O do not match for each person.

The control unit 5 performs image classification on the image data set D including the label noise data set Dn using the learning model learned in step S1. The control unit 5 displays a histogram distribution as shown in FIG. 5 on the output unit 4 by integrating the number of learning images G in association with the likelihood i based on the result of image classification. As shown in Figure 5, in image classification using a learning model that performs shallow learning, the output label O of the label noise dataset Dn (image dataset D of Ped_noise) is assigned with a high likelihood (for example, 1). be done.

As described above, in the label noise detection according to the embodiment, label noise data is extracted from the image data set D based on the teacher label T shown in FIG. 2 and the output label O and likelihood i shown in FIG. Set Dn can be extracted. According to this embodiment, label noise can be efficiently detected from the image data set D.

In addition, in the label noise detection according to the embodiment, after extracting the label noise data set Dn, it is determined whether the likelihood i assigned to the learning image G is equal to or greater than a predefined threshold of the likelihood i. Then, a label noise data set Dn including the teacher label T associated with the learning image G that is equal to or higher than the threshold value is selected. According to this embodiment, even if the number of extracted label noise data sets Dn is large, a label noise data set Dn with a high likelihood i can be selected. Therefore, the work of replacing the output label O in step S3 can be reduced.

In addition, in the label noise detection according to the embodiment, after extracting the label noise data set Dn, teacher labels associated with a predetermined number of learning images G are assigned to the learning images G in descending order of likelihood i. A label noise data set Dn containing T is selected. According to this embodiment, even if a large number of label noise data sets Dn are extracted, a predetermined number of label noise data sets Dn can be selected in descending order of likelihood i. Therefore, the work of replacing the output label O in step S3 can be reduced.

Further, in the label noise detection according to the embodiment, the teacher label T of the selected label noise data set Dn is replaced with an output label O based on the input information input to the label noise detection device 1. According to this embodiment, since the output label O after replacement can be updated as the correct teacher label T, the image data set D can be obtained from which label noise has been purified.

In addition, in the label noise detection according to the embodiment, since the learning model is trained using the image data set D that includes the learning image G after the teacher label T is replaced with the output label O, the teacher label T is replaced with the output label O. Do the same thing over and over again. According to this embodiment, since label noise can be purified each time the repetition is repeated, it is possible to obtain an image data set D in which label noise is further purified.

Furthermore, in the label noise detection according to the embodiment, the learning model is trained so that the value of the loss function included in the learning model is minimized, and the loss function is a mean absolute error (MAE). Error). According to this embodiment, label noise can be efficiently purified by using MAE as a loss function.

1 Label noise detection device 3 Input section 4 Output section 5 Control section 6 Storage section P Label noise detection program D Image dataset Dn Label noise dataset G Learning image T Teacher label O Output label i Likelihood

Claims (6)

Label noise performed in a label noise detection device that detects noise in the teacher label using an image data set that includes a learning image that is a learning target image in image classification and a teacher label corresponding to the learning image. A detection program,
Learning a learning model using the image dataset,
classifying the learning image using the trained learning model, obtaining an output label and likelihood attached to the learning image based on the result of the class classification,
Based on the output label, the likelihood, and the teacher label, it is determined whether the teacher label is incorrectly attached, and the image data includes the learning image for which it is determined that the teacher label is incorrectly attached. extract the set as a label noise dataset ,
After extracting the label noise data set, it is determined whether the likelihood assigned to the learning image is equal to or greater than a predefined threshold of the likelihood, and becomes equal to or greater than the threshold; A label noise detection program that selects the label noise data set including a predetermined number of the teacher labels associated with the learning images in descending order of likelihood . The label noise detection program according to claim 1 , further executing the step of replacing the teacher label of the selected label noise data set with the output label based on input information input to the label noise detection device. 2. A process from learning the learning model to replacing the teacher label with the output label is repeatedly performed using the image data set including the learning image after the teacher label is replaced with the output label. Label noise detection program described in . In learning the learning model, learning is performed so that the value of a loss function included in the learning model is minimized, and the loss function is a mean absolute error (MAE). 3. The label noise detection program according to any one of 3 . A label noise detection method for detecting noise in the teacher label using an image dataset including a learning image that is a learning target image in image classification and a teacher label corresponding to the learning image, the method comprising:
Learning a learning model using the image dataset,
classifying the learning image using the trained learning model, obtaining an output label and likelihood attached to the learning image based on the result of the class classification,
Based on the output label, the likelihood, and the teacher label, it is determined whether the teacher label is incorrectly attached, and the image data includes the learning image for which it is determined that the teacher label is incorrectly attached. extract the set as a label noise dataset ,
After extracting the label noise data set, it is determined whether the likelihood assigned to the learning image is equal to or greater than a predefined threshold of the likelihood, and becomes equal to or greater than the threshold; A label noise detection method comprising: selecting the label noise data set including a predetermined number of the teacher labels associated with the learning images in descending order of likelihood . A label noise detection device that detects noise in the teacher label using an image data set that includes a learning image that is a learning target image in image classification and a teacher label corresponding to the learning image,
Learning a learning model using the image dataset,
classifying the learning image using the trained learning model, obtaining an output label and likelihood attached to the learning image based on the result of the class classification,
Based on the output label, the likelihood, and the teacher label, it is determined whether the teacher label is incorrectly attached, and the image data includes the learning image for which it is determined that the teacher label is incorrectly attached. extract the set as a label noise dataset ,
After extracting the label noise data set, it is determined whether the likelihood assigned to the learning image is equal to or greater than a predefined threshold of the likelihood, and becomes equal to or greater than the threshold; A label noise detection device comprising: a processing unit configured to select the label noise data set including the teacher labels associated with a predetermined number of the learning images in descending order of likelihood .

Images