CN113361576A

CN113361576A - Picture labeling method and equipment

Info

Publication number: CN113361576A
Application number: CN202110601687.4A
Authority: CN
Inventors: 赵薇
Original assignee: Spreadtrum Communications Tianjin Co Ltd
Current assignee: Spreadtrum Communications Tianjin Co Ltd
Priority date: 2021-05-31
Filing date: 2021-05-31
Publication date: 2021-09-07

Abstract

The invention relates to the field of machine vision, in particular to a picture marking method and picture marking equipment. Wherein, the method comprises the following steps: inputting a test picture into a pre-training model to obtain a first labeling result of the test picture; displaying the test picture and the first labeling result in a first software tool; when the correction operation acting on the first labeling result is detected, correcting the first labeling result according to the correction operation to obtain a second labeling result; and adding the test picture and the second labeling result into a training data set, and performing first picture labeling training on the pre-training model through the training data set to obtain an optimization model, wherein the optimization model is used for automatically labeling pictures. The image annotation scheme provided by the embodiment of the invention can obtain high-precision annotation results and simultaneously reduce labor cost and time cost.

Description

Picture labeling method and equipment

[ technical field ] A method for producing a semiconductor device

The invention relates to the field of machine vision, in particular to a method and equipment for marking pictures.

[ background of the invention ]

Deep learning trains a model by using a training data set, and aims to enable the model to have the analysis and learning capacity. The training data set needs to contain a large amount of training data and labeling results corresponding to the training data. The accuracy of the annotation result has a direct impact on the learning ability of the model.

In the machine vision domain, pictures are used as training data. If the pictures are manually marked, a high-precision marking result can be obtained, but a large amount of labor and time cost are required to be invested. If the model is used for automatically labeling the pictures, when training data are less, the learning capacity of the model is often not good enough, and the obtained labeling result has the problem of low accuracy at a high probability.

[ summary of the invention ]

In view of this, embodiments of the present invention provide a method and an apparatus for image annotation, which can improve accuracy of an annotation result and reduce labor and time costs.

In a first aspect, an embodiment of the present invention provides a method for labeling a picture, including:

inputting a test picture into a pre-training model to obtain a first labeling result of the test picture;

displaying the test picture and the first labeling result in a first software tool;

when the correction operation acting on the first labeling result is detected, correcting the first labeling result according to the correction operation to obtain a second labeling result;

and adding the test picture and the second labeling result into a training data set, and performing first picture labeling training on the pre-training model through the training data set to obtain an optimization model, wherein the optimization model is used for automatically labeling pictures.

In one possible implementation manner, before inputting the test picture into the pre-training model, the method further includes:

performing second picture labeling training on the initial network model based on the training pictures;

and according to the second picture marking training, updating the parameter weight of the initial network model to obtain the pre-training model.

In one possible implementation manner, the first labeling result includes: the coordinate information of a first target area identified by the pre-training model in the test picture and the label information of the first target area.

In one possible implementation manner, the detecting of the correction operation on the first annotation result includes:

detecting a correction operation on a coordinate point included in the first target area and/or a correction operation on tag information of the first target area.

In one possible implementation manner, the second annotation result includes at least one of the following:

after the coordinate points included in the first target area are corrected, coordinate information of a second target area is obtained;

and after the label information of the first target area is corrected, obtaining the label information of a second target area.

In one possible implementation manner, after obtaining the first annotation result, before displaying the test picture and the first annotation result in the first software tool, the method further includes:

and converting the test picture and the first labeling result into a format supported by the first software tool.

In one possible implementation manner, after obtaining a second labeling result and before adding the test picture and the second labeling result to a training data set, the method further includes:

and converting the second labeling result into a label format supported by the pre-training model.

In a second aspect, an image annotation apparatus according to an embodiment of the present invention includes:

the marking module is used for inputting a test picture into the pre-training model to obtain a first marking result of the test picture;

the display module is used for displaying the test picture and the first annotation result in a first software tool;

the correction module is used for correcting the first labeling result according to the correction operation when the correction operation acting on the first labeling result is detected, so as to obtain a second labeling result;

and the optimization module is used for adding the test picture and the second labeling result into a training data set so as to perform first picture labeling training on the pre-training model through the training data set to obtain an optimization model, and the optimization model is used for automatically labeling pictures.

In a third aspect, an embodiment of the present invention provides an electronic device, including:

at least one processor; and

at least one memory communicatively coupled to the processor, wherein:

the memory stores program instructions executable by the processor, the processor calling the program instructions to be able to perform the method provided by the first aspect.

In a fourth aspect, an embodiment of the present invention provides a computer-readable storage medium, where the computer-readable storage medium includes a stored program, and when the program runs, the apparatus on which the computer-readable storage medium is located is controlled to execute the method in the first aspect.

It should be understood that the second to fourth aspects of the embodiment of the present invention are consistent with the technical solution of the first aspect of the embodiment of the present invention, and the beneficial effects obtained by the aspects and the corresponding possible implementation manners are similar, and are not described again.

The image labeling method and the image labeling equipment provided by the embodiment of the invention can improve the accuracy of the image labeling result and save the labor and time cost.

[ description of the drawings ]

In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings needed to be used in the embodiments will be briefly described below, and it is obvious that the drawings in the following description are only some embodiments of the present specification, and it is obvious for those skilled in the art that other drawings can be obtained according to the drawings without creative efforts.

Fig. 1 is a flowchart of a picture labeling method according to an embodiment of the present invention;

fig. 2 is a schematic diagram illustrating an effect of a test picture according to an embodiment of the present invention;

FIG. 3-a is a schematic diagram illustrating the effect of the first annotation result provided by the embodiment of the invention;

FIG. 3-b is a schematic diagram illustrating the effect of another first annotation result provided by the embodiment of the invention;

FIG. 4-a is a schematic diagram illustrating the effect of a second annotation result provided by the embodiment of the invention;

FIG. 4-b is a schematic diagram illustrating the effect of a second annotation result provided by the embodiment of the invention;

FIG. 5 is a flowchart of another method for annotating pictures according to an embodiment of the present invention;

fig. 6 is a schematic diagram of a picture labeling apparatus according to an embodiment of the present invention;

FIG. 7 is a schematic diagram of another apparatus for annotating pictures according to an embodiment of the present invention;

fig. 8 is a schematic structural diagram of an electronic device according to an embodiment of the present invention.

[ detailed description ] embodiments

For better understanding of the technical solutions of the present invention, the following detailed descriptions of the embodiments of the present disclosure are provided with reference to the accompanying drawings.

It should be understood that the described embodiments are only a few embodiments of the present specification, and not all embodiments. All other embodiments obtained by a person skilled in the art based on the embodiments in the present specification without any inventive step are within the scope of the present specification.

The terminology used in the embodiments of the specification is for the purpose of describing particular embodiments only and is not intended to be limiting of the specification. As used in the specification examples and the appended claims, the singular forms "a", "an", and "the" are intended to include the plural forms as well, unless the context clearly indicates otherwise.

In the field of machine vision, if the model is used for automatically labeling the pictures, the learning capability of the model is often not good enough, so that the obtained labeling result has the problem of low accuracy at a high probability. If the picture is marked manually, a high-precision marking result can be obtained, but a large amount of labor and time cost is needed. The image annotation method and the image annotation equipment provided by the embodiment of the invention can obtain high-precision annotation results, and simultaneously reduce labor cost and time cost.

Fig. 1 is a flowchart of a method for labeling a picture according to an embodiment of the present invention. As shown in fig. 1, the method for labeling a picture may include:

step 101, inputting a test picture into a pre-training model to obtain a first labeling result of the test picture.

The test picture may be a picture to be labeled. The pre-training model can be a neural network model with the function of labeling the pictures. Alternatively, the pre-trained model may be a convolutional neural network model. In this step, the test picture is input into the pre-training model, and the pre-training model can label the test picture. Specifically, the pre-training model may identify a first target region in the test picture, and label information is labeled on the first target region. That is, inputting the test picture into the pre-training model to obtain the first labeling result may include: coordinate information of a first target area identified by the pre-training model in the test picture and label information of the first target area. The first target area may be a target object in the test picture, such as a human face, a building, and the like. After the pre-training model identifies the target object, label information can be set for the target object. Certainly, in some embodiments, the pre-training model may perform region segmentation on the test picture, some or all of the segmented regions may be used as the first target region, and label information is set for each first target region. For example, the pre-training model may divide the test picture into a foreground region and a background region, and use the foreground region and/or the background region as the first target region and label the label information. That is, the pre-training model may perform a target detection or region segmentation task, and determine a first target region and set label information for the first target region based on a task execution result.

In one specific example, a test picture containing a car and a white background is shown in fig. 2. When the pre-training model performs the target detection task on the test picture shown in fig. 2, as shown in fig. 3-a, the pre-training model may identify the car from the test picture and set tag information for the car; when the pre-trained model performs the region segmentation task on the test shown in fig. 2, as shown in fig. 3-b, the pre-trained model can segment the background region and the car region, and label each region.

FIG. 3-a is a schematic diagram illustrating an effect of the first annotation result according to the embodiment of the invention. As shown in fig. 3-a, the target object is the car in fig. 2, the pre-trained model identifies the car in fig. 2, the area where the car is located is marked with a rectangular target frame, and the tag information "car" is set for the target object. The first target area in FIG. 3-a, the area within the rectangular target box; with the bottom left corner of fig. 3-a as the origin (0,0), the coordinate information of the first target region may be represented using the bottom left corner coordinates (x1, y1), width value w1, and height value h1 of the rectangular target frame; the tag information of the first target area is "car".

FIG. 3-b is a schematic diagram illustrating an effect of another first annotation result provided by the embodiment of the invention. As shown in fig. 3-b, the pre-training model segments the test picture shown in fig. 2 to obtain a background region and a car region, and the two regions are labeled with different colors. As shown in fig. 3-b, the black area is the area where the car is located, and the white area is the area where the white background is located. The first target area in fig. 3-b may be a black area; the coordinate information of the first target area can be represented by coordinates of each point on the outline of the black area; the label information of the first target area represents the black color of the area where the car is located.

After the first labeling result is obtained, the process continues to step 102.

And 102, displaying the test picture and the first labeling result in a first software tool.

In the implementation of the present invention, there may be a certain error in the labeling result output by the pre-training model in step 101. Therefore, after the step 101 is executed, the first labeling result of the test picture is further modified, so that the labeling result of the test picture is more accurate. In this step, the test picture and the first annotation result in step 101 are displayed in a first software tool. The first software tool may be software capable of modifying the first annotation result, such as labelme. In this step, the first software tool displays the coordinate information and the tag information of the first target area included in the first labeling result at the corresponding position of the test picture, and then may continue to execute step 103.

And 103, when the correction operation acting on the first labeling result is detected, correcting the first labeling result according to the correction operation to obtain a second labeling result.

It should be noted that, because the first labeling result obtained in step 101 may have a certain error, in order to obtain a labeling result with higher accuracy, the first labeling result may be modified by manually moving, adding, or deleting the coordinate point included in the first target area, and modifying, adding, or deleting the tag information of the first target area, so as to obtain a second labeling result with higher accuracy. Wherein the second labeling result comprises at least one of the following: after the coordinate points included in the first target area are corrected, coordinate information of a second target area is obtained; and after the label information of the first target area is corrected, obtaining the label information of a second target area. In this step, when the first software tool detects the correction operation applied to the first labeling result, the first labeling result is corrected according to the correction operation, and a second labeling result is obtained.

FIG. 4-a is a diagram illustrating an effect of a second annotation result provided in the embodiment of the present invention.

Specifically, the second labeling result shown in fig. 4-a is obtained by modifying the first labeling result shown in fig. 3-a. Through manual judgment, it is found that the first target region in fig. 3-a includes many blank backgrounds in addition to the target object, and the rectangular target frame in fig. 3-a is not completely attached to the target object, that is, it is found that the accuracy of the first labeling result in fig. 3-a is not high. The size of the rectangular target frame can be changed by manually moving the coordinate point on the boundary of the rectangular target frame, so that the rectangular target frame is more attached to the target object. 4-a, the second target area is the area within the rectangular target box shown in FIG. 4-a; with the lower left corner of fig. 4-a as the origin (0,0), the coordinate information of the second target region may be represented using the coordinates (x2, y2), width value w2, and height value h2 of the lower left corner of the rectangular target frame in fig. 4-a; the tag information of the second target area is "car". It can be seen that the rectangular target frame in FIG. 4-a fits better to the target object than the rectangular target frame in FIG. 3-a, and therefore, the second labeling result in FIG. 4-a has higher accuracy than the first labeling result in FIG. 4-a.

FIG. 4-b is a schematic diagram illustrating the effect of the second annotation result provided by the embodiment of the invention.

Specifically, the second labeling result shown in fig. 4-b is obtained by modifying the first labeling result shown in fig. 3-b. Through manual judgment, the outline of the black area in fig. 3-b is found not to be completely overlapped with the outline of the automobile in fig. 2, i.e. the first labeling result in fig. 3-b is found to have low accuracy. The shape of the black area can be changed by manually moving each coordinate point on the outline of the black area, so that the outline of the black area is more fit with the outline of the automobile. 4-b, the second target area may be a black area in FIG. 4-b, and the coordinate information of the second target area may be represented by coordinates of points on the outline of the black area in FIG. 4-b; the label information of the second target area represents the black color of the car. It can be seen that the second labeling result in FIG. 4-b has higher accuracy than the first labeling result in FIG. 3-b.

After the second labeling result with higher accuracy is obtained, the process continues to step 104.

And 104, adding the test picture and the second labeling result into a training data set, and performing first picture labeling training on the pre-training model through the training data set to obtain an optimized model, wherein the optimized model is used for automatically labeling pictures.

The training data set comprises a test picture and a second labeling result, and can be used for labeling and training the first picture of the pre-training model. The first picture labeling training aims at improving the picture labeling capacity of the pre-training model to obtain the optimized model. Compared with a pre-training model, the optimization model has better picture labeling capability and can obtain a labeling result with higher accuracy. Further, after step 104 is completed, the obtained optimization model may be used as a pre-training model, and the step of obtaining the optimization model is repeated, so that the accuracy of the automatic labeling result obtained by the optimization model is continuously improved.

Fig. 5 is a flowchart of another picture labeling method according to an embodiment of the present invention. As shown in fig. 5, in the embodiment of the present invention shown in fig. 1, step 203 may further include:

step 201, performing second picture labeling training on the initial network model based on the training picture.

And step 202, according to the second picture marking training, updating the parameter weight of the initial network model to obtain a pre-training model.

Wherein, the training picture in step 201 is a marked picture; the initial network model may be a neural network model that has learning capabilities, but not yet picture tagging capabilities. The second picture labeling training aims at training the initial network model, so that the initial network model learns to label the pictures. The parameter weight in step 202 is a purpose of performing second image annotation training, and is also a key index for distinguishing the initial network model from the pre-trained model, and the pre-trained model has image annotation capability only because the parameter weight is updated. Step 201 to step 202, performing second picture labeling training on the initial network model based on the training pictures, updating the parameter weight to obtain a pre-training model with picture labeling capability, and then executing step 203.

As shown in fig. 5, in the embodiment of the present invention shown in fig. 1, after step 203, step 205 may further include:

step 204, converting the test picture and the first labeling result into a format supported by the first software tool.

It should be noted that the first labeling result obtained in step 203 is a label format supported by the pre-training model, and is not compatible with the first labeling software. Therefore, it is necessary to convert the first annotation result from the tag format into a format supported by the first software tool. Specifically, when the first software tool is labelme, the format supported by the first software tool is json format. The first annotation result after format conversion can be directly displayed in the first software tool, i.e., step 205 is performed.

As shown in fig. 5, in the embodiment of the present invention shown in fig. 1, after step 206, step 208 may further include:

and step 207, converting the second labeling result into a label format supported by the pre-training model.

It should be noted that the second labeling result obtained in step 206 is a format supported by the first software tool, for example, the second labeling result may be a json format supported by labelme, and is incompatible with the pre-training model and cannot be directly used in the pre-training model. The second labeling result obtained in step 103 needs to be converted into a label format supported by the pre-training model, so that step 208 can be executed continuously.

Fig. 6 is a schematic diagram of a picture labeling apparatus according to an embodiment of the present invention. As shown in fig. 6, the image annotation device may include:

the labeling module 61 is used for inputting a test picture into the pre-training model to obtain a first labeling result of the test picture;

a display module 62, configured to display the test picture and the first annotation result in a first software tool;

a correcting module 63, configured to, when a correcting operation acting on the first labeling result is detected, correct the first labeling result according to the correcting operation, so as to obtain a second labeling result;

and the optimization module 64 is configured to add the test picture and the second labeling result into a training data set, so as to perform first picture labeling training on the pre-training model through the training data set to obtain an optimization model, where the optimization model is used to perform automatic labeling on pictures.

The image annotation device provided in the embodiment shown in fig. 7 can be used to implement the technical solution of the embodiment of the method shown in fig. 1 of the present invention, and the implementation principle and technical effects thereof can be further referred to the related description in the embodiment of the method.

Fig. 7 is a schematic diagram of another picture labeling apparatus according to an embodiment of the present invention. As shown in fig. 7, the image annotation device may further include:

the pre-training module 71 is configured to perform second picture labeling training on the initial network model based on the training picture; and according to the second picture marking training, updating the parameter weight of the initial network model to obtain the pre-training model.

A format conversion module 72, configured to convert the test picture and the first annotation result into a format supported by the first software tool;

the format conversion module 72 is further configured to convert the second labeling result into a label format supported by the pre-training model.

The image annotation device provided in the embodiment shown in fig. 7 can be used to implement the technical solution of the embodiment of the method shown in fig. 5 of the present invention, and the implementation principle and the technical effects thereof can be further referred to the related description in the embodiment of the method.

Fig. 8 is a schematic structural diagram of an electronic device according to an embodiment of the present invention. The electronic device shown in fig. 8 is only an example, and should not bring any limitation to the functions and the scope of use of the embodiments of the present invention.

As shown in fig. 8, the electronic device may include at least one processor; and at least one memory communicatively coupled to the processor, wherein: the memory stores program instructions executable by the processor, and the processor calls the program instructions to execute the method for tagging pictures provided by the embodiment of fig. 1 or fig. 5. The electronic device is in the form of a general purpose computing device. Components of the electronic device may include, but are not limited to: one or more processors 410, a communication interface 420, a memory 430, and a communication bus 440 that connects the various system components (including the memory 430 and the processing unit 410).

Communication bus 440 represents one or more of any of several types of bus structures, including a memory bus or memory controller, a peripheral bus, an accelerated graphics port, and a processor or local bus using any of a variety of bus architectures. These architectures include, but are not limited to, Industry Standard Architecture (ISA) bus, Micro Channel Architecture (MAC) bus, enhanced ISA bus, Video Electronics Standards Association (VESA) local bus, and Peripheral Component Interconnect (PCI) bus, to name a few.

Electronic devices typically include a variety of computer system readable media. Such media may be any available media that is accessible by the electronic device and includes both volatile and nonvolatile media, removable and non-removable media.

Memory 430 may include computer system readable media in the form of volatile Memory, such as Random Access Memory (RAM) and/or cache Memory. The electronic device may further include other removable/non-removable, volatile/nonvolatile computer system storage media. Memory 430 may include at least one program product having a set (e.g., at least one) of program modules that are configured to carry out the functions of embodiments of the invention.

A program/utility having a set (at least one) of program modules, including but not limited to an operating system, one or more application programs, other program modules, and program data, may be stored in memory 430, each of which examples or some combination may include an implementation of a network environment. The program modules generally perform the functions and/or methodologies of the described embodiments of the invention.

The processor 410 executes programs stored in the memory 430 to perform various functional applications and data processing, such as implementing the picture method provided by the embodiment of fig. 1 or 5 of the present invention.

An embodiment of the present invention provides a computer-readable storage medium, where the computer-readable storage medium includes a stored program, where when the program runs, a device where the computer-readable storage medium is located is controlled to execute the picture annotation method provided in the embodiment shown in fig. 1 or fig. 5 in the present invention.

A computer readable signal medium may include a propagated data signal with computer readable program code embodied therein, for example, in baseband or as part of a carrier wave. Such a propagated data signal may take any of a variety of forms, including, but not limited to, electro-magnetic, optical, or any suitable combination thereof. A computer readable signal medium may also be any computer readable medium that is not a computer readable storage medium and that can communicate, propagate, or transport a program for use by or in connection with an instruction execution system, apparatus, or device.

Program code embodied on a computer readable medium may be transmitted using any appropriate medium, including but not limited to wireless, wireline, optical fiber cable, RF, etc., or any suitable combination of the foregoing.

Computer program code for carrying out operations for aspects of the present invention may be written in any combination of one or more programming languages, including an object oriented programming language such as Java, Smalltalk, Python, C + +, and conventional procedural programming languages, such as the "just" programming language or similar programming languages. The program code may execute entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer or entirely on the remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer through any type of Network, including a Local Area Network (LAN) or a Wide Area Network (WAN), or the connection may be made to an external computer (for example, through the Internet using an Internet service provider).

The foregoing description of specific embodiments of the present invention has been presented. Other embodiments are within the scope of the following claims. In some cases, the actions or steps recited in the claims may be performed in a different order than in the embodiments and still achieve desirable results. In addition, the processes depicted in the accompanying figures do not necessarily require the particular order shown, or sequential order, to achieve desirable results. In some embodiments, multitasking and parallel processing may also be possible or may be advantageous.

In the description of the present invention, reference to the description of the terms "one embodiment," "some embodiments," "an example," "a specific example," or "some examples," etc., means that a particular feature, structure, material, or characteristic described in connection with the embodiment or example is included in at least one embodiment or example of the present invention. In the present disclosure, the schematic representations of the terms used above are not necessarily intended to be the same embodiment or example. Furthermore, the particular features, structures, materials, or characteristics described may be combined in any suitable manner in any one or more embodiments or examples. Furthermore, various embodiments or examples and features of different embodiments or examples described in this disclosure can be combined and combined by one skilled in the art without contradiction.

Furthermore, the terms "first", "second" and "first" are used for descriptive purposes only and are not to be construed as indicating or implying relative importance or implicitly indicating the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include at least one such feature. In the description of the present invention, "a plurality" means at least two, e.g., two, three, etc., unless specifically limited otherwise.

Any process or method descriptions in flow charts or otherwise described herein may be understood as representing modules, segments, or portions of code which include one or more executable instructions for implementing steps of a custom logic function or process, and alternate implementations are included within the scope of the preferred embodiment of the present invention in which functions may be executed out of order from that shown or discussed, including substantially concurrently or in reverse order, depending on the functionality involved, as would be understood by those reasonably skilled in the art of the present invention.

The word "if" as used herein may be interpreted as "at … …" or "when … …" or "in response to a determination" or "in response to a detection", depending on the context. Similarly, the phrases "if determined" or "if detected (a stated condition or event)" may be interpreted as "when determined" or "in response to a determination" or "when detected (a stated condition or event)" or "in response to a detection (a stated condition or event)", depending on the context.

In the embodiments provided in the present invention, it should be understood that the disclosed system, apparatus and method may be implemented in other ways. For example, the above-described apparatus embodiments are merely illustrative, and for example, the division of the units is only one logical division, and there may be other divisions in actual implementation, for example, a plurality of units or components may be combined or integrated into another system, or some features may be omitted, or not executed. In addition, the shown or discussed mutual coupling or direct coupling or communication connection may be an indirect coupling or communication connection through some interfaces, devices or units, and may be in an electrical, mechanical or other form.

In addition, functional units in the embodiments of the present invention may be integrated into one processing unit, or each unit may exist alone physically, or two or more units are integrated into one unit. The integrated unit can be realized in a form of hardware, or in a form of hardware plus a software functional unit.

The above description is only for the purpose of illustrating the preferred embodiments of the present invention and is not to be construed as limiting the invention, and any modifications, equivalents, improvements and the like made within the spirit and principle of the present invention should be included in the scope of the present invention.

Claims

1. A picture marking method is characterized by comprising the following steps:

2. The method of claim 1, wherein before inputting the test picture into the pre-training model, further comprising:

3. The method of claim 1, wherein the first annotation result comprises: the coordinate information of a first target area identified by the pre-training model in the test picture and the label information of the first target area.

4. The method of claim 3, wherein detecting a corrective action on the first annotated result comprises:

5. The method of any of claims 1 to 4, wherein the second annotation result comprises at least one of:

6. The method of claim 1, wherein after obtaining the first annotation result, before displaying the test picture and the first annotation result in the first software tool, further comprising:

7. The method of claim 1, wherein after obtaining the second labeling result and before adding the test picture and the second labeling result to the training data set, further comprising:

8. A picture labeling apparatus, comprising:

9. An electronic device, comprising:

at least one processor; and

at least one memory communicatively coupled to the processor, wherein:

the memory stores program instructions executable by the processor, the processor invoking the program instructions to perform the method of any of claims 1 to 7.

10. A computer-readable storage medium, comprising a stored program, wherein the program, when executed, controls an apparatus on which the computer-readable storage medium resides to perform the method of any one of claims 1 to 7.