CN112418287B - Image pre-labeling method, device, electronic equipment and medium - Google Patents

Image pre-labeling method, device, electronic equipment and medium Download PDF

Info

Publication number
CN112418287B
CN112418287B CN202011280039.5A CN202011280039A CN112418287B CN 112418287 B CN112418287 B CN 112418287B CN 202011280039 A CN202011280039 A CN 202011280039A CN 112418287 B CN112418287 B CN 112418287B
Authority
CN
China
Prior art keywords
image
labeling
model
target
marked
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
CN202011280039.5A
Other languages
Chinese (zh)
Other versions
CN112418287A (en
Inventor
蔡永辉
程骏
庞建新
熊友军
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Beijing Youbixuan Intelligent Robot Co ltd
Original Assignee
Ubtech Robotics Corp
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Ubtech Robotics Corp filed Critical Ubtech Robotics Corp
Priority to CN202011280039.5A priority Critical patent/CN112418287B/en
Publication of CN112418287A publication Critical patent/CN112418287A/en
Application granted granted Critical
Publication of CN112418287B publication Critical patent/CN112418287B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/24—Classification techniques
    • G06F18/241—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/045—Combinations of networks
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods

Abstract

The embodiment of the application discloses an image pre-labeling method, an image pre-labeling device, electronic equipment and a medium. The method comprises the following steps: predicting a target annotation object in the image to be annotated by adopting an image pre-annotation model; the target labeling object and the size change image of the target labeling object are respectively added into another unlabeled image to obtain at least two remapped images; and determining a pre-labeling result of the image to be labeled according to the prediction of the image pre-labeling model on the at least two remapped images. By executing the technical scheme, the problem of error accumulation in the pre-labeling of the algorithm can be effectively relieved, and the pre-labeling efficiency of the model is improved.

Description

Image pre-labeling method, device, electronic equipment and medium
Technical Field
The embodiment of the application relates to the field of data processing, in particular to an image pre-labeling method, an image pre-labeling device, electronic equipment and a medium.
Background
At present, a convolutional neural network model is used for detecting a target object or belongs to the category of supervised learning, namely, a large number of target frames in images are marked manually, then a marked image data set is sent into the convolutional neural network model, the model learns a mapping relation for the marked images, and finally, the purpose of detecting the target object in the real world can be achieved.
The current main current method is to manually mark a small number of pictures, then train a model by using the pictures, pre-mark part of the pictures by using the model, and finally mix the pre-marked pictures with the original pictures and then continue to train the model in an iterative way.
The problem in the prior art is that a model trained by using a small number of pictures has no good learning ability on a target object to be detected, so that a predicted class result is easy to have class errors and a positioning frame has larger deviation, if the pre-marked sample is directly added into a training sample for iterative training, error accumulation is caused, the robustness of the trained model is poor, and the recognition ability and the positioning ability on the target object can not reach the expected result.
Disclosure of Invention
The embodiment of the application provides an image pre-labeling method, an image pre-labeling device, electronic equipment and a medium, so that the technical effects of effectively relieving the problem of error accumulation in algorithm pre-labeling and improving model pre-labeling efficiency are achieved.
In a first aspect, an embodiment of the present application provides an image pre-labeling method, including:
predicting a target annotation object in the image to be annotated by adopting an image pre-annotation model;
the target labeling object and the size change image of the target labeling object are respectively added into another unlabeled image to obtain at least two remapped images;
and determining a pre-labeling result of the image to be labeled according to the prediction of the image pre-labeling model on the at least two remapped images.
In a second aspect, an embodiment of the present application further provides an image pre-labeling device, where the device includes:
the prediction labeling module is used for predicting a target labeling object in the image to be labeled by adopting the image pre-labeling model;
the image reproducing module is used for respectively adding the target marked object and the size change image of the target marked object into another unlabeled image to obtain at least two reproduced images;
and the result determining module is used for determining the pre-marking result of the image to be marked according to the prediction of the image pre-marking model on the at least two reproduced images.
In a third aspect, an embodiment of the present application further provides an electronic device, including:
one or more processors;
a storage means for storing one or more programs;
the one or more programs are executed by the one or more processors to cause the one or more processors to implement an image pre-labeling method as provided in any embodiment of the application.
In a fourth aspect, there is also provided in an embodiment of the present application a computer-readable storage medium having stored thereon a computer program which, when executed by a processor, implements an image pre-labeling method as provided in any embodiment of the present application.
The embodiment of the application provides an image pre-labeling method, which adopts an image pre-labeling model to predict and determine a target labeling object in an image to be labeled; the target labeling object and the size change image of the target labeling object are respectively added into another unlabeled image to obtain at least two remapped images; and determining a pre-labeling result of the image to be labeled according to the prediction of the image pre-labeling model on the at least two remapped images, and achieving the purpose of improving the model and the labeling efficiency through image pre-labeling.
By adopting the technical scheme of the application, the image to be marked can be predicted and marked according to the model, and the pre-marking result of the image to be marked by the model can be automatically determined according to the predicted result of the reproduced image, so that the problem of error accumulation in algorithm pre-marking is effectively solved, the technical effect of model pre-marking efficiency is improved, the image is pre-marked with high quality, and the labor and time resource cost is reduced.
The foregoing summary is merely an overview of the present application, and is intended to be implemented in accordance with the teachings of the present application in order that the same may be more fully understood, and in order that the same or additional objects, features and advantages of the present application may be more fully understood.
Drawings
Other features, objects and advantages of the present application will become more apparent upon reading of the detailed description of non-limiting embodiments made with reference to the following drawings. The drawings are only for purposes of illustrating the preferred embodiments and are not to be construed as limiting the application. Also, like reference numerals are used to designate like parts throughout the figures. In the drawings:
FIG. 1 is a flowchart of an image pre-labeling method according to an embodiment of the present application;
FIG. 2 is a flowchart of another image pre-labeling method according to a second embodiment of the present application;
FIG. 3 is a schematic diagram of model training of a method for pre-labeling pictures according to an embodiment of the present application;
fig. 4 is a schematic structural diagram of an image pre-labeling device according to a third embodiment of the present application;
fig. 5 is a schematic structural diagram of an electronic device according to a fourth embodiment of the present application.
Detailed Description
The application is described in further detail below with reference to the drawings and examples. It is to be understood that the specific embodiments described herein are merely illustrative of the application and are not limiting thereof. It should be further noted that, for convenience of description, only some, but not all of the structures related to the present application are shown in the drawings.
Before discussing exemplary embodiments in more detail, it should be mentioned that some exemplary embodiments are described as processes or methods depicted as flowcharts. Although a flowchart depicts operations (or steps) as a sequential process, many of the operations can be performed in parallel, concurrently, or at the same time. Furthermore, the order of the operations may be rearranged. The process may be terminated when its operations are completed, but may have additional steps not included in the figures. The processes may correspond to methods, functions, procedures, subroutines, and the like.
Example 1
Fig. 1 is a flowchart of an image pre-labeling method provided in a first embodiment of the present application, where the method may be applied to pre-labeling a picture containing a target object area, and the method may be performed by an image pre-labeling device, where the device may be implemented by software and/or hardware and may be integrated into an electronic device. As shown in fig. 1, the image pre-labeling method in this embodiment includes the following steps:
s110, predicting a target annotation object in the image to be annotated by adopting an image pre-annotation model.
In order to solve the problem of target labeling of a large number of targets in images, an image pre-labeling model is generally adopted to predict target labeling objects in images to be labeled. The image pre-labeling model can be a neural network model, and the target labeling objects in the image to be labeled can be characters, animals, objects, outlines and the like.
In an alternative of this embodiment, the image pre-labeling model may be trained based on at least a preset number of manually labeled images when the first pre-labeling operation is performed. The manually marked image may be an image obtained by manually judging target information of the target object and marking the image. For example, a certain number of pictures of each category are manually marked, such as 100 pictures of each category are manually marked, and then the neural network model training is performed by manually marking the certain number of pictures of each category, so as to obtain a first-time image pre-marking model.
By adopting the technical scheme, the image pre-marking model is obtained through training of a small amount of manual marking images, so that a large amount of manual marking images are not needed for training of the image pre-marking model to be used at first, and the labor and time resource cost is reduced.
In another alternative of this embodiment, when the non-first pre-labeling operation is performed, the image pre-labeling model may be obtained by iteratively updating the previous image pre-labeling model based on at least a preset number of manually labeled images and pre-labeled images. For example, a certain number of pictures of each category are manually marked, a pre-marked image obtained by other model pre-marking operation before non-first pre-marking operation is obtained, and then iteration update is carried out on the image pre-marking model used last time by utilizing at least a preset number of manually marked images and the pre-marked images, so that the image pre-marking model used this time is obtained.
By adopting the technical scheme, the previous image pre-marking model is iterated based on a small amount of manual marking images and pre-marking images, the image pre-marking model used in the previous time can be corrected, the prediction accuracy of the image pre-marking model is higher, and the technical effects of reducing the labor and time cost can be achieved.
In this embodiment, optionally, the image pre-labeling model is a lightweight neural network model, such as Pelee or other lightweight neural network models, which is obtained by training under a Caffe deep learning framework, and when the image pre-labeling model is deployed, the Pelee model file is converted into a model file suitable for deployment at a mobile end, such as an NCNN model file. When the image pre-labeling model is deployed on the hardware equipment, a lightweight model Pelee is used, and a cache version Pelee model file is converted into a NCNN reasoning engine file and then deployed. For example, the hardware device may be a service robot, an intelligent detection device, etc., and for the service robot, most of the service robot does not have an inflight video card or a neural network processing chip (NPU), so the computing performance of the service robot is very limited, and the service robot is not suitable for deployment and use of a heavyweight neural network model.
By adopting the technical scheme, the image pre-labeling model is obtained through training by using the lightweight neural network model Pelee, the Pelee model file of the Caffe edition is converted into the file suitable for NCNN, and then the converted image pre-labeling model is deployed on the service robot, so that the reasoning speed of the image pre-labeling model on the service robot with limited hardware performance can be effectively improved, and the occupation rate of a CPU and a memory can be reduced.
And S120, respectively adding the target marked object and the size change image of the target marked object into another unmarked image to obtain at least two remarked images.
The hardware device adds the information of the target labeling object and the image of the target labeling object after the size change to another unlabeled image respectively to obtain at least two remapped images, wherein the other unlabeled image can be one unlabeled image randomly selected from a picture database or a preset unlabeled image, and the unlabeled image can be used as a base map of the target labeling object to play a role in changing the environment of the target labeling object and increasing the pre-labeling difficulty. The remapped image is a new image obtained by adding the target labeling object and the size change result of the target labeling object to the unlabeled image.
In one alternative of this embodiment, this embodiment may be combined with each of the alternatives of one or more of the embodiments described above. The step of adding the target labeling object and the dimensional change of the target labeling object to the unlabeled image respectively may include the following steps A1-A2:
and A1, carrying out the reduction and the enlargement of the target labeling object in the same proportion to obtain a reduced image and an enlarged image of the target labeling object.
And A2, respectively adding the target annotation object, the reduced image and the enlarged image of the target annotation object to another unlabeled image randomly selected from a picture database.
The target labeling object and the size change result of the target labeling object can be added into the same image in a mapping mode, the target labeling object is reduced and enlarged in the same proportion, the proportion can be one third, other scaling proportions can be adopted, and the reduction and enlargement result is added into the same image randomly selected from the picture library.
In the foregoing embodiment, optionally, before the target labeling object and the size change image of the target labeling object are respectively added to another unlabeled image, in order to increase the generalization capability of the model, operations such as flipping, rotating, adding random noise, and changing illumination may also be performed on the target labeling object. Therefore, the model can adapt to more scenes to a greater extent, the scene adaptability of model prediction labeling is increased, the accuracy of model prediction labeling is improved, the problem of error accumulation in each prediction labeling process is further reduced, and the image pre-labeling efficiency is higher.
S130, according to the image pre-marking model, predicting the at least two remapped images, and determining a pre-marking result of the image to be marked.
According to the technical scheme, a lightweight neural network model Pelee is used, a Pelee model file of a Caffe version is converted into a file suitable for NCNN, an image pre-labeling model is adopted to predict a target labeling object in an image to be labeled, the target labeling object and a size change image of the target labeling object are respectively added into another unlabeled image, at least two remarked images are obtained, a pre-labeling result of the image to be labeled is determined according to the prediction of the image pre-labeling model on the at least two remarked images, the technical effects of improving the reasoning speed of the model on a service robot with limited hardware performance, reducing the occupancy rate of a CPU and a memory, effectively relieving the error accumulation problem in algorithm pre-labeling and improving the model pre-labeling efficiency are achieved.
Example two
Fig. 2 is a flowchart of another image pre-labeling method according to a second embodiment of the present application. Embodiments of the present application may be further optimized on the basis of the foregoing embodiments, and may be combined with each of the alternatives of one or more of the foregoing embodiments. As shown in fig. 2, the image pre-labeling method provided in the embodiment of the present application may include the following steps:
s210, selecting an image from the unlabeled image set as an image to be annotated, and carrying out prediction annotation on the image to be annotated by adopting the image pre-annotation model.
The computer or the processor selects one image from the unlabeled image set as the image to be annotated, wherein the image can be selected randomly from the unlabeled image set, or the images in the unlabeled image set can be sequentially arranged and then sequentially selected, and the image to be annotated is predicted and annotated by adopting an image pre-annotation model. In an alternative example, when the first pre-labeling operation is performed, the image pre-labeling model is obtained based on at least a preset number of manually labeled images, and when the non-first pre-labeling operation is performed, the image pre-labeling model may be obtained by iteratively updating the image pre-labeling model used last time based on at least the preset number of manually labeled images and the pre-labeled images.
S220, if the confidence coefficient of the first predicted result of the image to be marked is larger than a first preset confidence coefficient threshold value, extracting a region image corresponding to the first predicted result in the image to be marked, and determining the region image as a target marking object.
If the confidence coefficient of the first prediction result of the image to be marked is greater than a first preset confidence coefficient threshold value, extracting an area image corresponding to the first prediction result in the image to be marked as a target marking object, wherein the first prediction result is a result of performing prediction marking on the image to be marked by using an image pre-marking model, and the confidence coefficient can be the similarity or the fitting degree of the prediction marking result and the actual information of the target marking object, or can be the difference information between the prediction result and the actual result of the target marking object obtained by performing prediction marking on the image to be marked. The first preset confidence threshold may be a preset confidence level when the preset image pre-labeling model predicts and labels the image to be labeled, and in order to reduce an error of a predicted labeling result of the image to be labeled, the first preset confidence level is generally set to be high, for example, the first preset confidence level may be set to be 0.95.
S230, respectively adding the target labeling object and the size change image of the target labeling object to another unlabeled image to obtain at least two remarked images.
S240, according to the image pre-marking model, predicting the at least two remapped images, and determining a pre-marking result of the image to be marked.
In one alternative of this embodiment, this embodiment may be combined with each of the alternatives of one or more of the embodiments described above. The determining the pre-labeling result of the image to be labeled according to the prediction of the image pre-labeling model on the at least two remapped images may include the following steps B1-B2:
b1: and respectively predicting each of the at least two reproduced images by adopting the image pre-labeling model to obtain at least two second prediction results.
B2: and if the marked object position and the object category of the at least two second prediction results are correct and the confidence coefficient of the at least two second prediction results is larger than a second preset confidence coefficient threshold value, adding the image to be marked and the target marked object information of the image to be marked into the marked image database.
And respectively predicting each of the at least two reproduced images by adopting an image pre-labeling model to obtain at least two second prediction results, wherein the reproduced images can be new images with target labeling objects with the same enlarged or reduced proportion, and the second prediction results are the results of the image pre-labeling model for performing prediction labeling on the at least two reproduced images.
The second preset result may be a result obtained by pre-labeling at least two remarked images by the image pre-labeling model, the second preset confidence threshold may be a preset confidence of the result obtained by pre-labeling at least two remarked images by the image pre-labeling model, and the second preset confidence threshold may be set to 0.75 due to the fact that the remarked images replace the background environment of the target labeling object and the size of the target labeling object. Adding the image to be annotated and the target annotation object information of the image to be annotated into the annotated picture database, wherein the target annotation object information can be the position coordinates of the target annotation object and the category of the target annotation object.
In the foregoing embodiment, optionally, if the confidence coefficient of any one of the at least two second prediction results is smaller than a second preset confidence coefficient threshold, performing manual calibration on the target labeling object information of the image to be labeled, where if the confidence coefficient of any one of the at least two second prediction results is smaller than the second preset confidence coefficient threshold, performing manual calibration labeling on the prediction labeling result of the image with the confidence coefficient smaller than the threshold by manual operation. As shown in FIG. 3, the predicted labeling result of the remapped image can be used for carrying out iterative training on the original image pre-labeling model after passing through a threshold value and manual calibration, so that the performance of the image pre-labeling model is improved.
By adopting the technical scheme, the technical scheme that the model is marked as the main part and the manual marking is used as the auxiliary part, and the self-supervision learning mechanism is added to check the model pre-marking result for a plurality of times is adopted, so that the problem of error accumulation in the algorithm pre-marking is effectively solved, the model pre-marking result is improved, and the technical effect of the high-performance image pre-marking model is trained.
In the foregoing embodiment, optionally, the first preset confidence threshold and the second preset confidence threshold gradually decrease as the number of images in the annotated image database increases.
By adopting the technical scheme, the image data of the marked image database is continuously updated, and the performance of the model trained by iteration is also continuously improved, so that the first preset confidence threshold value and the second preset confidence threshold value can be set to be gradually reduced along with the increase of the number of images in the marked image database, the model training efficiency is improved, and the technical effect of improving the image pre-marking speed while considering the image pre-marking accuracy is achieved.
According to the technical scheme, the first and second preset confidence thresholds of the confidence level of the image pre-labeling result are set, the prediction result of which the confidence level of any second prediction result is smaller than the second preset confidence threshold is manually calibrated, the first preset confidence threshold and the second preset confidence threshold are set to gradually decrease along with the increase of the number of images in the labeled image database, the problem of error accumulation in the algorithm pre-labeling is effectively solved, the accuracy of the model pre-labeling result is improved, and the training efficiency of the model is improved.
Example III
Fig. 4 is a schematic structural diagram of an image pre-labeling device according to a third embodiment of the present application. The device can be suitable for the case of pre-marking a picture containing a target object area, can be realized by software and/or hardware and is integrated in an input device. The device is used for realizing the image pre-marking method provided by the embodiment. As shown in fig. 4, the image pre-labeling device provided in this embodiment includes:
the prediction labeling module 410 is configured to predict a target labeling object in an image to be labeled by using an image pre-labeling model;
the image reproducing module 420 is configured to add the target labeling object and the size change image of the target labeling object to another unlabeled image respectively, so as to obtain at least two reproduced images;
the result determining module 430 is configured to determine a pre-labeling result of the image to be labeled according to the prediction of the image pre-labeling model on the at least two remapped images.
Based on the above embodiment, optionally, the prediction labeling module 410 is configured to:
the image pre-labeling model is obtained based on at least a preset number of manual labeling images in a training mode; or, based on at least a preset number of manually marked images and the pre-marked images, iteratively updating the pre-marked model of the previous image.
Optionally, on the basis of the above embodiment, the prediction labeling module 410 is further configured to:
predicting a target annotation object in an image to be annotated by adopting an image pre-annotation model, comprising:
selecting an image from an unlabeled image set as an image to be annotated, and carrying out predictive annotation on the image to be annotated by adopting the image pre-annotation model;
and if the confidence coefficient of the first predicted result of the image to be marked is larger than a first preset confidence coefficient threshold value, extracting a region image corresponding to the first predicted result in the image to be marked, and determining the region image as a target marking object.
On the basis of the above embodiment, optionally, the image reproduction module 420 is configured to:
the target labeling object and the size change image of the target labeling object are respectively added into another unlabeled image, and the method comprises the following steps:
the target labeling object is reduced and enlarged in the same proportion, and a reduced image and an enlarged image of the target labeling object are obtained;
and respectively adding the target annotation object, the reduced image and the enlarged image of the target annotation object into another unlabeled image randomly selected from a picture database.
On the basis of the above embodiment, optionally, the result determining module 430 is configured to:
according to the prediction of the image pre-labeling model to the at least two remapped images, determining a pre-labeling result of the image to be labeled comprises the following steps:
predicting each of the at least two reproduced images by adopting the image pre-labeling model to obtain at least two second prediction results;
and if the marked object position and the object category of the at least two second prediction results are correct and the confidence coefficient of the at least two second prediction results is larger than a second preset confidence coefficient threshold value, adding the image to be marked and the target marked object information of the image to be marked into the marked image database.
On the basis of the above embodiment, optionally, the result determining module 430 is further configured to:
and if the confidence coefficient of any one second predicted result in the at least two second predicted results is smaller than a second preset confidence coefficient threshold value, manually calibrating the target labeling object information of the image to be labeled.
On the basis of the above embodiment, optionally, the result determining module 430 is further configured to:
the first preset confidence threshold and the second preset confidence threshold gradually decrease with the increase of the number of images in the marked image database.
Optionally, on the basis of the above embodiment, the prediction labeling module 410 is further configured to:
the image pre-labeling model is a lightweight model, and the lightweight model file is converted into a model file suitable for mobile terminal deployment when the image pre-labeling model is deployed.
The image pre-marking device provided by the embodiment of the application can execute the image pre-marking method provided by any embodiment of the application, has the corresponding functions and beneficial effects of executing the image pre-marking method, and the detailed process refers to the related operation of the image pre-marking method in the embodiment.
Example IV
Fig. 5 is a schematic structural diagram of an electronic device according to a fourth embodiment of the present application. The embodiment of the application provides electronic equipment, and the image pre-marking device provided by the embodiment of the application can be integrated in the electronic equipment. As shown in fig. 5, the present embodiment provides an electronic device 500, which includes: one or more processors 520; the storage 510 is configured to store one or more programs, where the one or more programs are executed by the one or more processors 520, so that the one or more processors 520 implement the image pre-labeling method provided by the embodiment of the present application, and the method includes:
predicting a target annotation object in the image to be annotated by adopting an image pre-annotation model;
the target labeling object and the size change image of the target labeling object are respectively added into another labeling image to obtain at least two remade images;
and determining a pre-labeling result of the image to be labeled according to the prediction of the image pre-labeling model on the at least two remapped images.
Of course, those skilled in the art will appreciate that the processor 520 also implements the technical solution of the image pre-labeling method provided in any embodiment of the present application.
The electronic device 500 shown in fig. 5 is merely an example, and should not be construed as limiting the functionality and scope of use of embodiments of the present application.
As shown in fig. 5, the electronic device 500 includes a processor 520, a storage device 510, an input device 530, and an output device 540; the number of processors 520 in the electronic device may be one or more, one processor 520 being exemplified in fig. 5; the processor 520, the storage 510, the input 530, and the output 540 in the electronic device may be connected by a bus or other means, as exemplified by connection via bus 550 in fig. 5.
The storage device 510 is a computer readable storage medium, and can be used to store a software program, a computer executable program, and a module unit, such as program instructions corresponding to the image pre-labeling method in the embodiment of the present application.
The storage device 510 may mainly include a storage program area and a storage data area, wherein the storage program area may store an operating system, at least one application program required for functions; the storage data area may store data created according to the use of the terminal, etc. In addition, the storage 510 may include high-speed random access memory, and may also include non-volatile memory, such as at least one magnetic disk storage device, flash memory device, or other non-volatile solid-state storage device. In some examples, storage 510 may further include memory located remotely from processor 520, which may be connected via a network. Examples of such networks include, but are not limited to, the internet, intranets, local area networks, mobile communication networks, and combinations thereof.
The input means 530 may be used to receive input numeric, character information or voice information and to generate key signal inputs related to user settings and function control of the electronic device. Output 540 may include an electronic device such as a display screen, speaker, etc.
The electronic equipment provided by the embodiment of the application can achieve the technical effects of effectively relieving the problem of error accumulation in the pre-labeling of the algorithm and improving the pre-labeling efficiency of the model.
Example five
In a fifth embodiment of the present application, there is provided a computer-readable medium having stored thereon a computer program for executing an image pre-labeling method when executed by a processor, the method including:
predicting a target annotation object in the image to be annotated by adopting an image pre-annotation model;
the target labeling object and the size change image of the target labeling object are respectively added into another unlabeled image to obtain at least two remapped images;
and determining a pre-labeling result of the image to be labeled according to the prediction of the image pre-labeling model on the at least two remapped images.
Optionally, the program may be further configured to perform the image pre-labeling method provided in any embodiment of the present application when executed by a processor.
The computer storage media of embodiments of the application may take the form of any combination of one or more computer-readable media. The computer readable medium may be a computer readable signal medium or a computer readable storage medium. The computer readable storage medium can be, for example, but not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or a combination of any of the foregoing. More specific examples (a non-exhaustive list) of the computer-readable storage medium would include the following: an electrical connection having one or more wires, a portable computer diskette, a hard disk, a random access Memory (Random Access Memory, RAM), a Read-Only Memory (ROM), an erasable programmable Read-Only Memory (Erasable Programmable Read Only Memory, EPROM), a flash Memory, an optical fiber, a portable CD-ROM, an optical storage device, a magnetic storage device, or any suitable combination of the foregoing. A computer readable storage medium may be any tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, apparatus, or device.
The computer readable signal medium may include a propagated data signal with computer readable program code embodied therein, either in baseband or as part of a carrier wave. Such a propagated data signal may take many forms, including, but not limited to: electromagnetic signals, optical signals, or any suitable combination of the preceding. A computer readable signal medium may also be any computer readable medium that is not a computer readable storage medium and that can communicate, propagate, or transport a program for use by or in connection with an instruction execution system, apparatus, or device.
Program code embodied on a computer readable medium may be transmitted using any appropriate medium, including but not limited to: wireless, wire, fiber optic cable, radio frequency (RadioFrequency, RF), and the like, or any suitable combination of the foregoing.
Computer program code for carrying out operations of the present application may be written in any combination of one or more programming languages, including an object oriented programming language such as Java, smalltalk, C ++ and conventional procedural programming languages, such as the "C" programming language or similar programming languages. The program code may execute entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer or entirely on the remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer through any kind of network, including a Local Area Network (LAN) or a Wide Area Network (WAN), or may be connected to an external computer (for example, through the Internet using an Internet service provider).
In the description of the present specification, a description referring to terms "one embodiment," "some embodiments," "examples," "specific examples," or "some examples," etc., means that a particular feature, structure, material, or characteristic described in connection with the embodiment or example is included in at least one embodiment or example of the present application. In this specification, schematic representations of the above terms do not necessarily refer to the same embodiments or examples. Furthermore, the particular features, structures, materials, or characteristics described may be combined in any suitable manner in any one or more embodiments or examples.
Note that the above is only a preferred embodiment of the present application and the technical principle applied. It will be understood by those skilled in the art that the present application is not limited to the particular embodiments described herein, but is capable of various obvious changes, rearrangements and substitutions as will now become apparent to those skilled in the art without departing from the scope of the application. Therefore, while the application has been described in connection with the above embodiments, the application is not limited to the embodiments, but may be embodied in many other equivalent forms without departing from the spirit or scope of the application, which is set forth in the following claims.

Claims (9)

1. An image pre-labeling method, comprising the steps of:
predicting a target annotation object in the image to be annotated by adopting an image pre-annotation model;
the target labeling object and the size change image of the target labeling object are respectively added into another unlabeled image to obtain at least two remapped images;
determining a pre-labeling result of the image to be labeled according to the prediction of the image pre-labeling model on the at least two remapped images;
the predicting the at least two remapped images according to the image pre-labeling model to determine the pre-labeling result of the image to be labeled comprises the following steps:
predicting each of the at least two reproduced images by adopting the image pre-labeling model to obtain at least two second prediction results;
if the marked object position and the object category of the at least two second prediction results are correct and the confidence coefficient of the at least two second prediction results is larger than a second preset confidence coefficient threshold value, adding the image to be marked and the target marked object information of the image to be marked into a marked image database;
and the image pre-labeling model is obtained by carrying out iterative updating on the previous image pre-labeling model based on the pre-labeling results of the image to be labeled and the reproduced image.
2. The method of claim 1, wherein the image pre-annotation model is trained based on at least a predetermined number of manually annotated images; or, based on at least a preset number of manually marked images and the pre-marked images, iteratively updating the pre-marked model of the previous image.
3. The method of claim 1, wherein predicting the target annotation object in the image to be annotated using the image pre-annotation model comprises:
selecting an image from an unlabeled image set as an image to be annotated, and carrying out predictive annotation on the image to be annotated by adopting the image pre-annotation model;
and if the confidence coefficient of the first predicted result of the image to be marked is larger than a first preset confidence coefficient threshold value, extracting a region image corresponding to the first predicted result in the image to be marked, and determining the region image as a target marking object.
4. The method of claim 1, wherein adding the target annotation object and the size-change image of the target annotation object to the unlabeled image, respectively, comprises:
the target labeling object is reduced and enlarged in the same proportion, and a reduced image and an enlarged image of the target labeling object are obtained;
and respectively adding the target annotation object, the reduced image and the enlarged image of the target annotation object into another unlabeled image randomly selected from a picture database.
5. The method according to claim 1, wherein the method further comprises:
and if the confidence coefficient of any one second predicted result in the at least two second predicted results is smaller than a second preset confidence coefficient threshold value, manually calibrating the target labeling object information of the image to be labeled.
6. The method of claim 3, wherein the step of,
the first preset confidence threshold and the second preset confidence threshold gradually decrease with the increase of the number of images in the marked image database.
7. The method of claim 1, wherein the image pre-annotation model is a lightweight model and the lightweight model file is converted to a model file suitable for mobile end deployment when the image pre-annotation model is deployed.
8. An image pre-labeling apparatus, the apparatus comprising:
the prediction labeling module is used for predicting a target labeling object in the image to be labeled by adopting the image pre-labeling model;
the image reproducing module is used for respectively adding the target marked object and the size change image of the target marked object into another unlabeled image to obtain at least two reproduced images;
the result determining module is used for determining a pre-marking result of the image to be marked according to the prediction of the image pre-marking model on the at least two remapped images;
the result determining module is specifically configured to:
predicting each of the at least two reproduced images by adopting the image pre-labeling model to obtain at least two second prediction results;
if the marked object position and the object category of the at least two second prediction results are correct and the confidence coefficient of the at least two second prediction results is larger than a second preset confidence coefficient threshold value, adding the image to be marked and the target marked object information of the image to be marked into a marked image database;
and the image pre-labeling model is obtained by carrying out iterative updating on the previous image pre-labeling model based on the pre-labeling results of the image to be labeled and the reproduced image.
9. An electronic device, comprising:
one or more processors;
a storage means for storing one or more programs;
the one or more programs, when executed by the one or more processors, cause the one or more processors to implement the image pre-labeling method of any of claims 1-7.
CN202011280039.5A 2020-11-16 2020-11-16 Image pre-labeling method, device, electronic equipment and medium Active CN112418287B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202011280039.5A CN112418287B (en) 2020-11-16 2020-11-16 Image pre-labeling method, device, electronic equipment and medium

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202011280039.5A CN112418287B (en) 2020-11-16 2020-11-16 Image pre-labeling method, device, electronic equipment and medium

Publications (2)

Publication Number Publication Date
CN112418287A CN112418287A (en) 2021-02-26
CN112418287B true CN112418287B (en) 2023-10-13

Family

ID=74832274

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202011280039.5A Active CN112418287B (en) 2020-11-16 2020-11-16 Image pre-labeling method, device, electronic equipment and medium

Country Status (1)

Country Link
CN (1) CN112418287B (en)

Families Citing this family (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN113129303B (en) * 2021-05-18 2022-01-18 广州市吉华勘测股份有限公司 Automatic marking method and device for inspection pictures, storage medium and electronic equipment

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN109753978A (en) * 2017-11-01 2019-05-14 腾讯科技（深圳）有限公司 Image classification method, device and computer readable storage medium
CN110378420A (en) * 2019-07-19 2019-10-25 Oppo广东移动通信有限公司 A kind of image detecting method, device and computer readable storage medium
CN110472593A (en) * 2019-08-20 2019-11-19 重庆紫光华山智安科技有限公司 Training image acquisition methods, model training method and relevant apparatus
CN111274426A (en) * 2020-01-19 2020-06-12 深圳市商汤科技有限公司 Category labeling method and device, electronic equipment and storage medium
CN111814820A (en) * 2020-05-18 2020-10-23 北京迈格威科技有限公司 Image processing method and device

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN109753978A (en) * 2017-11-01 2019-05-14 腾讯科技（深圳）有限公司 Image classification method, device and computer readable storage medium
CN110378420A (en) * 2019-07-19 2019-10-25 Oppo广东移动通信有限公司 A kind of image detecting method, device and computer readable storage medium
CN110472593A (en) * 2019-08-20 2019-11-19 重庆紫光华山智安科技有限公司 Training image acquisition methods, model training method and relevant apparatus
CN111274426A (en) * 2020-01-19 2020-06-12 深圳市商汤科技有限公司 Category labeling method and device, electronic equipment and storage medium
CN111814820A (en) * 2020-05-18 2020-10-23 北京迈格威科技有限公司 Image processing method and device

Also Published As

Publication number Publication date
CN112418287A (en) 2021-02-26

Similar Documents

Publication Publication Date Title
CN109117831B (en) Training method and device of object detection network
CN107808004B (en) Model training method and system, server and storage medium
CN108920135B (en) User-defined service generation method and device, computer equipment and storage medium
CN113971727A (en) Training method, device, equipment and medium of semantic segmentation model
CN108229675B (en) Neural network training method, object detection method, device and electronic equipment
CN112287994A (en) Pseudo label processing method, device, equipment and computer readable storage medium
CN111753863A (en) Image classification method and device, electronic equipment and storage medium
CN113449821B (en) Intelligent training method, device, equipment and medium fusing semantics and image characteristics
CN114723646A (en) Image data generation method with label, device, storage medium and electronic equipment
CN112418287B (en) Image pre-labeling method, device, electronic equipment and medium
CN112712036A (en) Traffic sign recognition method and device, electronic equipment and computer storage medium
CN114330588A (en) Picture classification method, picture classification model training method and related device
CN113239883A (en) Method and device for training classification model, electronic equipment and storage medium
CN111930891B (en) Knowledge graph-based search text expansion method and related device
CN116894188A (en) Service tag set updating method and device, medium and electronic equipment
CN116580407A (en) Training method of text detection model, text detection method and device
CN115861809A (en) Rod detection and training method and device for model thereof, electronic equipment and medium
CN115049003A (en) Pre-training model fine-tuning method, device, equipment and storage medium
CN110909797B (en) Image detection method and device, equipment and storage medium
CN114330542A (en) Sample mining method and device based on target detection and storage medium
CN115375657A (en) Method for training polyp detection model, detection method, device, medium, and apparatus
CN114898379A (en) Method, device and equipment for recognizing curved text and storage medium
CN114663751A (en) Power transmission line defect identification method and system based on incremental learning technology
CN111353464B (en) Object detection model training and object detection method and device
CN114187586B (en) Travel itinerary identification method and device, electronic equipment and storage medium

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant
TR01 Transfer of patent right

Effective date of registration: 20231206

Address after: Room 601, 6th Floor, Building 13, No. 3 Jinghai Fifth Road, Beijing Economic and Technological Development Zone (Tongzhou), Tongzhou District, Beijing, 100176

Patentee after: Beijing Youbixuan Intelligent Robot Co.,Ltd.

Address before: 518000 16th and 22nd Floors, C1 Building, Nanshan Zhiyuan, 1001 Xueyuan Avenue, Nanshan District, Shenzhen City, Guangdong Province

Patentee before: Shenzhen Youbixuan Technology Co.,Ltd.

TR01 Transfer of patent right