CN113378830A - Domain-adaptation-based autonomous learning data label generation method - Google Patents
Domain-adaptation-based autonomous learning data label generation method Download PDFInfo
- Publication number
- CN113378830A CN113378830A CN202110488161.XA CN202110488161A CN113378830A CN 113378830 A CN113378830 A CN 113378830A CN 202110488161 A CN202110488161 A CN 202110488161A CN 113378830 A CN113378830 A CN 113378830A
- Authority
- CN
- China
- Prior art keywords
- image
- data set
- domain
- data
- target
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 30
- 238000001514 detection method Methods 0.000 claims abstract description 134
- 238000012549 training Methods 0.000 claims abstract description 32
- 238000002372 labelling Methods 0.000 claims abstract description 25
- 238000001914 filtration Methods 0.000 claims abstract description 18
- 230000015572 biosynthetic process Effects 0.000 claims abstract description 9
- 238000003786 synthesis reaction Methods 0.000 claims abstract description 9
- 230000006978 adaptation Effects 0.000 claims abstract description 7
- 125000004122 cyclic group Chemical group 0.000 claims abstract description 7
- 238000006243 chemical reaction Methods 0.000 claims description 27
- 238000013135 deep learning Methods 0.000 claims description 12
- 230000006870 function Effects 0.000 claims description 12
- 238000013136 deep learning model Methods 0.000 claims description 10
- 238000013507 mapping Methods 0.000 claims description 6
- 238000005457 optimization Methods 0.000 claims description 4
- 238000012360 testing method Methods 0.000 claims description 4
- 230000003042 antagnostic effect Effects 0.000 claims description 3
- 238000004364 calculation method Methods 0.000 claims description 2
- 238000010276 construction Methods 0.000 claims description 2
- 230000003044 adaptive effect Effects 0.000 description 6
- 238000005516 engineering process Methods 0.000 description 6
- 239000002131 composite material Substances 0.000 description 4
- 230000008569 process Effects 0.000 description 4
- 239000008186 active pharmaceutical agent Substances 0.000 description 2
- 238000004458 analytical method Methods 0.000 description 2
- 238000010586 diagram Methods 0.000 description 2
- 235000013399 edible fruits Nutrition 0.000 description 2
- 238000005286 illumination Methods 0.000 description 2
- 230000000007 visual effect Effects 0.000 description 2
- 238000011161 development Methods 0.000 description 1
- 230000014759 maintenance of location Effects 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 239000002420 orchard Substances 0.000 description 1
- 238000012545 processing Methods 0.000 description 1
- 239000004576 sand Substances 0.000 description 1
- 230000009466 transformation Effects 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
- G06F18/214—Generating training patterns; Bootstrap methods, e.g. bagging or boosting
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
Abstract
The invention discloses a domain adaptation-based self-learning data label generation method, which comprises the steps of combining source domain data set label data to construct a labeled intermediate domain synthetic data set; training a target detection network by using the labeled intermediate domain synthesis data set, and inputting image data in the label-free target domain data set to obtain a corresponding image detection frame result; and dynamically updating the confidence coefficient threshold parameter value of the current target detection network, performing noise filtering and cyclic updating operation on the detection frame result, and converting the detection frame result into an image tag data format. And outputting image label data, and constructing a labeled target domain data set by combining corresponding target domain image data to realize the automatic labeling function of the target domain data set. The method and the device are applied to automatic labeling work of the data sets of various different target scenes or categories, so that a large amount of manual data set labeling work does not need to be carried out on the image data sets under different application scenes, labor cost is saved, and work efficiency is improved.
Description
Technical Field
The invention belongs to the field of computer vision and image processing, and particularly relates to a domain-adaptation-based autonomous learning data label generation method.
Background
In recent years, with the rapid development of modern information technology, the application demand of computer technology in various fields is increasing. In the aspect of visual detection application, the existing target detection technology based on deep learning has the advantages of high precision, good robustness and the like, gradually replaces the traditional visual detection technology, and is widely applied to detection work in various fields. In the using process of the target detection technology based on deep learning, an image data set with labeled information is generally required to be manufactured, and training learning of a supervised learning signal support model is provided. In an actual application scene, when a scene or category of a target object to be detected is changed (including aspects of target object appearance, background, illumination and the like), image data distribution in a training set and a testing set is different, so that the precision of a deep learning detection model obtained through the training set is greatly reduced in the actual scene application; therefore, the image data set under the corresponding target scene or category generally needs to be replaced, and the training and learning of the deep learning detection model is restarted, and each time the data set is re-made, a large amount of expensive manual labor is needed to label the data labels, and the whole process is time-consuming and labor-consuming.
At present, in the deep learning model at the present stage, a strong supervision label labeling method is mainly adopted, the position and category information of each visible target object in an image is labeled in a manner of drawing a bounding box, a strong supervision learning signal of the model is provided, and a high-precision deep learning detection model is obtained, but the data labeling form is complex and easily causes long labeling time. Some researchers propose that a weak supervision label labeling method can be adopted, such as an image-level label (only providing target object category information in an image and no specific position information), a point label (only labeling target object position information in a point drawing mode), and the labeling cost of the whole data set is reduced by reducing the labeling time of single data, but the method still has a certain data labeling workload. In addition, some researchers put forward an unsupervised deep learning detection model, so that any data labeling work is not needed; in practical application work, due to factors such as complexity of a background in a scene, target diversity and the like, the accuracy and the effectiveness of the unsupervised deep learning detection model cannot be achieved by the unsupervised deep learning detection model method. Therefore, in the practical application work of target detection, how to reduce the labeling of the data set is a key problem in deep learning.
Representative technique 1 item:
(1) the patent name: method for constructing target detection self-adaptive model based on cycleGAN and pseudo label (application number: 202010540046.8)
The invention provides a method for constructing a target detection self-adaptive model based on a cycleGAN and a pseudo label, which can solve the problem of domain drift of target detection caused by distribution difference between two domains, and the specific implementation steps are shown in figure 2 and comprise the following steps: the data reading module is used for reading related image data and label data in the labeled source domain data set and the non-labeled target domain data set; the image conversion module is used for converting the image data of the source domain data set into intermediate domain synthetic data set image data close to the image data of the target domain data set by utilizing a cycleGAN network; the intermediate domain synthetic data set generating module is used for combining the label data of the source domain data set and the image data of the intermediate domain data set to construct a labeled intermediate domain synthetic data set; the target detection network module firstly inputs the intermediate domain synthetic data set into a fast R-CNN deep learning model for training and acquiring a primary domain adaptive model, then inputs image data of a label-free target domain data set into the domain adaptive model, manually sets a relevant confidence coefficient threshold value hyper-parameter and outputs a target domain data set image detection result; the pseudo label cyclic updating module receives the detection frame result of the image in the non-labeled target domain data set, converts the detection frame result into a data set label format and updates label data in the non-labeled target domain data set; the marked target domain data set generation module receives the image data of the target domain data set and the updated image label data and constructs a marked target domain data set; the module receives the intermediate domain synthesis data set and the labeled target domain data set, constructs a model training set, inputs the model training set to a target detection network, and performs iterative update and optimization of a network model; and finally, outputting the target detection network after the iteration updating is finished.
However, in the method, confidence threshold value hyper-parameters need to be manually set to obtain the target domain data set pseudo-label, while in the iterative updating and optimizing process of the domain adaptive model, the model continuously learns the target domain image characteristic information and the detection performance is continuously improved, and the fixed confidence threshold value hyper-parameters cannot well measure the quality of the obtained pseudo-label, so that the loss of part of positive sample pseudo-labels and the retention of part of negative sample label noise are caused; in addition, when the image distribution of the source domain data set and the target domain data set has a large difference, the image quality of the intermediate domain synthetic data set obtained by the CycleGAN network cannot be guaranteed, and when the method is used for model iteration updating and optimizing, the intermediate domain synthetic data set and the labeled target domain data set are alternately input into the domain adaptive model, so that the detection performance of the trained domain adaptive model in an actual target domain detection task is reduced.
Disclosure of Invention
The technical problem to be solved by the invention is as follows: in the current target detection task, in the actual application process of the target detection technology based on deep learning, most of image data under the actual application scene needs to be collected and a large amount of data labeling work is carried out, and training learning of a supervised learning signal support model is provided. When a target scene or category in an actual detection task is changed (including aspects such as appearance, background and illumination of a target object), image data needs to be collected again to label and train a model, so that a large amount of manual labeling labor is consumed, and the overall working efficiency is low.
In order to solve the problems, the invention realizes an autonomous learning data label generation method based on domain adaptation. The method comprises the steps of firstly, utilizing a generated countermeasure network to realize image conversion from a source domain image to a target domain image, and combining with label data of a source domain data set to construct a labeled intermediate domain synthetic data set. Then, the labeled intermediate domain synthesis data set is used for training a target detection network, and image data in the label-free target domain data set is input to obtain a corresponding image detection frame result. And then, dynamically updating the confidence coefficient threshold parameter value of the current target detection network, simultaneously carrying out noise filtering and cyclic updating operations on the detection frame result, and converting the detection frame result into an image tag data format. And finally, outputting image label data, and constructing a labeled target domain data set by combining corresponding target domain image data to realize the automatic labeling function of the target domain data set.
Referring to FIG. 1, the method of the present invention includes a data reading module step S10, an image transformation module step S20, an intermediate domain composite data set generation module step S30, an object detection network module step S40, a pseudo tag generation module step S50, an annotated target domain data set generation module step S60, and an output annotated target domain data set step S70.
The specific functions of each module are as follows:
data set reading module step S10: the image data set library is composed of a marked source domain data set and a non-marked target domain data set, and the module realizes a data reading function. Firstly, reading the labeled source domain data set, and respectively outputting the source domain image data and the label data to an image conversion module step S20 and an intermediate domain synthesis data set generation module step S30; next, the label-free target domain data set is read, and the target domain image data is output to the image conversion module step S20 and the target detection network module step S40, respectively.
Image conversion module step S20: the module step realizes the image conversion function between different target scenes or categories by utilizing the generation countermeasure network. Firstly, reading image data of a labeled source domain data set and image data of a non-labeled target domain data set from the data reading module S10, constructing an image training set, and inputting the image training set to a generation countermeasure network such as DiscaoAGN, DualGAN, cycleGAN and the like; secondly, training and generating a domain mapping relation for the image distribution between two image domains (a source domain and a target domain) of the confrontation network learning, and realizing image conversion operation between different target scenes or classes; then, reading the image data in the labeled source domain data set output by the data reading module in step S120, and inputting the image data into the trained generation countermeasure network to obtain intermediate domain synthetic image data; finally, the intermediate domain synthesized image data is output to the intermediate domain synthesized data set generating module step S30.
Intermediate domain synthetic data set generating module step S30: the module realizes the construction of the labeled intermediate domain synthesis data set. First, the step S10 of the data reading module and the step S20 of the image conversion module read the tag data and the intermediate domain synthetic image data in the labeled source domain data set, respectively, construct a labeled intermediate domain synthetic data set, and output the labeled intermediate domain synthetic data set to the target detecting network module step S40.
Target detection network module step S40: the module obtains the result of the image detection frame of the unmarked target domain by applying a deep learning detection model. Firstly, a step S30 of the intermediate domain synthesized data set generation module acquires a labeled intermediate domain synthesized data set, and inputs the labeled intermediate domain synthesized data set to a deep learning model such as SSD, fast R-CNN or Yolov3, etc. to perform training and learning of the model; then, the data reading module step S120 obtains image data in the non-labeled target domain data set, inputs a deep learning model for testing, and obtains a detection frame result of the image data in the non-labeled target domain data set; finally, the target domain image detection frame result is output to the pseudo label generation module step S50.
Pseudo tag generation module step S50: the module automatically updates the model confidence coefficient threshold parameter, and simultaneously carries out noise filtering on the detection frame result of the image in the unmarked target domain to obtain the corresponding label data of the image in the unmarked target domain data set. Firstly, obtaining the image detection frame result in the non-labeled target domain data set from the target detection network step S40, counting the sum of confidence scores of all detection frames, calculating the confidence average score of each detection frame, and setting and updating the confidence average score into the confidence threshold value hyperparameter of the subsequent model iterative update; secondly, setting the average confidence score value of the image detection frames obtained by current calculation as a filtering threshold value, and carrying out filtering operation on the image detection frames with the confidence scores lower than the filtering threshold value; and finally, converting the image detection frame result obtained by filtering into a label data format of the corresponding image, updating the label data of the current target domain data set, and outputting the label data to the labeled target domain data set generation module in step S60.
The labeled target domain data set generating module step S60: firstly, respectively acquiring tag data of a target domain data set and image data of the target domain data set from the step S50 of the pseudo tag generation module and the step S10 of the data reading module, and constructing a labeled target domain data set; then, inputting the obtained labeled target domain data set into a target detection network step S40 for iterative updating and optimization, and updating the label data in the labeled target domain data set again; and finally, stopping the iterative updating of the model when the number of times of the cyclic updating of the model is reached, and outputting the labeled target domain data set.
Outputting labeled target domain data set step S70: and step S60 of the pseudo label generating module acquires the labeled target domain data set, so as to implement the automatic labeling function of the target domain data set image.
The principle of the invention is as follows:
firstly, a generation countermeasure network such as DiscoGAN, DualGAN, CycleGAN and the like is utilized, the network is trained in a countermeasure learning mode to learn the domain mapping relation between image domains of different target scenes or classes, the image conversion operation between a source domain image and a target domain image is realized, an intermediate domain synthetic image with similar characteristics to the target domain image is generated, and a labeled intermediate domain synthetic data set is constructed by combining label data of a source domain data set.
Secondly, training a deep learning model such as SSD, fast R-CNN or Yolov3 and the like by using the intermediate domain synthesis data set to obtain the detection capability of the target domain image, inputting the target domain image to obtain a detection frame result, and converting the detection frame information into corresponding target domain image label information, thereby realizing the automatic acquisition of the target domain image label data.
And finally, constructing a labeled target domain data set by using the acquired target domain image label data, iteratively updating and optimizing the deep learning model, and filtering noise labels of the acquired target domain image to acquire a more accurate target domain image detection frame result and realize high-quality target domain image label data.
Compared with the prior art, the domain-adaptation-based self-learning data label generation method can be applied to automatic labeling work of data sets of various different target scenes or categories on the basis of the existing labeled source domain data set, so that a large amount of manual data set labeling work of image data sets in different application scenes is not needed, labor cost is saved, and work efficiency is improved. In addition, the invention further adopts noise removal and cyclic updating operation on the acquired data set label data set, thereby effectively improving the labeling quality of the data set and generating the label data of the data set with higher quality.
Drawings
FIG. 1 is a representative diagram of a domain adaptation-based autonomously learnable data tag generation method provided by the present patent.
Fig. 2 is a flow chart of an adaptive model for target detection based on CycleGAN and pseudo tags according to the related patent.
Fig. 3 is a flowchart of an image conversion module according to an embodiment of the present disclosure.
Fig. 4 is a flowchart of an object detection network module according to an embodiment of the present disclosure.
Fig. 5 is a flowchart of an automatic confidence threshold updating sub-module according to an embodiment of the present disclosure.
Fig. 6 is a flowchart of a pseudo tag noise removal submodule provided in an embodiment of the present patent.
FIG. 7 is a flowchart of a module for generating a labeled target domain data set according to an embodiment of the present disclosure.
Detailed Description
In order to make the objects, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention. It is to be understood that the embodiments described are only a few embodiments of the present invention, and not all embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention. The present invention will be described in detail below with reference to specific examples.
The schematic diagram of the method provided by the embodiment of the invention is shown in fig. 1, and the method comprises the following steps:
step S10: data reading module
Step S20: image conversion module
Step S30: intermediate domain synthetic data set generation module
Step S40: object detection network module
Step S50: pseudo label generation module
Step S60: labeled target domain data set generation module
Step S70: outputting tagged target domain data sets
In this embodiment, the labeled source domain data set is labeled
ISAnd lSRespectively representing source domain data set image data and corresponding label data, NSRepresenting the number of pictures of the source domain data set; annotate-free target domain data sets
ITAnd NTRespectively representing the image data and the number of pictures of the data set of the label-free target domain. The source domain data set and the target domain data set may represent image data sets of different target scenes or different target categories, such as vehicle data sets in sunny and foggy days, fruit data sets of different categories in an orchard, and the like.
In this embodiment, the data reading module step S10: from a labeled source domain data set DSAnd label-free target domain data set DTAcquiring related image data and label data, specifically comprising the following steps:
step S110: from the stationSaid labeled source domain data set DSReading the marked source domain data set DSImage data of
And tag data
Respectively to the image conversion module step S20 and the intermediate domain composite data set generation module step S30.
Step S120: from the label-free target domain dataset DTReading, and collecting the data set D of the unmarked target domainTImage data of
Respectively to the image conversion module step S20, the object detection network module step S40, and the annotated object domain data set generation module step S60.
Image conversion module step S20 in the present embodiment: utilizing labeled source domain data sets DsImage data of
And label-free target domain data set DTImage data of
And (3) generating an antagonistic network model such as DiscogAN, DualGAN and CycleGAN by joint input, training and learning the domain mapping relation among different image domains, and realizing the conversion operation of different types of images. The implementation steps are shown in fig. 3, and specifically include the following steps:
step S210: obtaining the image data in the labeled source domain data set from the step S110
It is output to step S230.
Step S220: obtaining the image data in the data set of the non-labeling target domain from the step S120
It is output to step S230.
Step S230: respectively obtaining source domain image data from the step S210 and the step S220
And target domain image data
Jointly constructing an image training set; then, adjusting and setting to generate the information of parameters related to the confrontation network cycleGAN, inputting an image training set into the cycleGAN network, training the image distribution mapping relationship between the learning source domain image and the target domain image, and recording the cycleGAN network as M1The training obtains the related network weight parameter as w1。
Step S240: obtaining the labeled source domain data set D from the step S110sMiddle image data
Inputting the data into the CycleGAN network M trained in the step S2301And performing image conversion operation from the source domain image to the target domain image.
Step S250: network M utilizing cycleGAN1According to the formula
Acquiring intermediate domain composite image data having similar characteristics to the target domain image
Step S260: acquiring intermediate domain composite image data from the step S250
Output to the intermediate domain synthesis data set generation moduleBlock step S30.
The intermediate domain synthesized data set generating module of the present embodiment, step S30: first, the labeled source domain data set D is obtained from the step S110 and the step S260 respectivelysMedium image label data
And intermediate domain synthesized image data
Then, as the conversion between the source domain image and the synthetic image is mainly the conversion of low-level characteristics such as target fruit color, texture and the like, the position information and the contour characteristics are not changed; thus, the labeled source domain data set D can be utilizedsConstructing labeled intermediate domain synthetic data set by using intermediate image label data
Finally, the intermediate domain is combined into a data set DSynOutput to the object detection network module step S40.
In this embodiment, the target detection network module step S40: synthesis of a data set D Using an intermediate DomainSynOr a labeled target domain data set
And as a target detection network training set, training or fine-tuning a target detection network Yolov3 to obtain the image detection capability in the data set of the unmarked target domain. The implementation steps are shown in fig. 4, and specifically include the following steps:
step S410: step S30 of obtaining a middle-domain synthetic data set D from the middle-domain synthetic data set generating moduleSynOr obtaining the labeled target domain data set from the step S660
It is output to step S420.
Step S420: acquiring an image data set from the step S410, inputting a target detection network Yolov3, setting Yolov3 network training iteration times, adjusting the network for training, and acquiring a non-labeled target domain data set DTRecording the target detection model Yolov3 as M for the detection capability of the image of the middle target area2The training obtains the related network weight parameter as w2。
Step S430: obtaining a label-free target domain data set D from the step S120TImage data of
The target detection network is input to the target detection network Yolov3 trained in step S420.
Step S440: and adjusting the target detection network Yolov3, and setting an initial or dynamically updated confidence coefficient threshold parameter theta value, wherein theta is more than or equal to 0 and less than or equal to 1.
Step S450: testing the target detection network Yolov3 according to the formula
Obtaining a label-free target domain data set DTMiddle target detection frame information, wherein NiIndicating the number of detection frames obtained by the ith target domain image,
j-th target detection frame result representing i-th target domain image, b ═ xmin,ymin,xmax,ymax,Conf)，xmin,ymin,xmax,ymaxRespectively representing the abscissa of the upper left corner point, the ordinate of the upper left corner point, the abscissa of the lower right corner point and the ordinate of the lower right corner point of the target detection frame in the corresponding image pixel coordinate system, wherein Conf represents the confidence score of the target detection frame, namely the probability that the detection frame contains the positive sample target, and the score value range is more than or equal to 0 and less than or equal to 1.
Step S460: from the step ofS450, acquiring a data set D of the label-free target domainTDetection result of middle target domain image
It is input to step S510-1.
In this embodiment, the pseudo tag generation module is step S50: first, for the unlabeled target domain data set D output in step S460TDetection result of middle target domain image
Performing confidence score analysis, and dynamically updating the current target detection network confidence threshold parameter; then, according to the obtained confidence threshold parameter value, the data set D of the unmarked target domainTMiddle whole image detection frame result
Carrying out noise filtering operation; and finally, converting the filtered target domain image detection frame result into an image label format to obtain corresponding target domain image label data. The step includes a confidence threshold value automatic updating submodule step S510, a pseudo tag noise removing submodule step S520, and a pseudo tag cyclic updating submodule step S530, and specifically includes the following steps:
confidence threshold automatic update submodule step S510: for the data set D of the label-free target domain output in the step S460TAnd performing confidence score analysis on the image detection frame result, and dynamically updating the current target detection network confidence threshold parameter value. The implementation steps are shown in fig. 5, and specifically include the following steps:
step S510-1: receiving the label-free target domain data set D output by the step S460TIntermediate image detection result
Step S510-2: according to the formula
Statistics of label-free target domain data set DTWherein the Score () function represents a statistical sum of all target domain image detection frame confidence scores.
Step S510-3: according to the formula
Computing a label-free target domain dataset DTConfidence coefficient average score S of image detection frame of middle target domainaver。
Step S510-4: obtaining a label-free target domain data set D from the step S510-3TConfidence coefficient average score S of image detection frame of middle target domainaverAnd dynamically updating the confidence coefficient threshold parameter theta of the current target detection network, and applying the confidence coefficient threshold parameter theta to iterative updating and optimization of a subsequent target detection network.
Step S510-5: and obtaining the updated target detection network confidence threshold parameter value theta from the step S510-4, and outputting the parameter value theta to a step S520-1.
Pseudo tag noise removal submodule S520: for the label-free target domain data set DtAnd carrying out noise filtering operation on the image detection frame result to obtain a higher-quality image detection frame result. The implementation steps are shown in fig. 6, and specifically include the following steps:
step S520-1: and obtaining the current target detection network confidence threshold parameter value theta from the step S510-5.
Step S520-2: for the data set D of the label-free target domaintAnd (4) performing traversal operation on all the image detection frame results, inputting the image detection frame results into the step (S520-3) for condition judgment, and performing subsequent operation.
Step S520-3: obtaining a label-free target domain data set D from the step S520-2TSequentially carrying out condition judgment operation on the detection frame results of all the images; when the confidence score value of the detection frame in the image is higher than the current confidence threshold parameter value, executing the step S520-4; otherwise, step S520-5 is executed.
Step S520-4: for the satisfaction of step S520-3Conditional unlabeled target domain dataset DTAnd storing the result of the middle image detection frame.
Step S520-5: for the non-labeling target domain data set D which does not meet the judgment condition of the step S520-3TAnd filtering the image detection frame result.
Step S520-6: obtaining the filtered data set D of the unmarked target domain from the step S520-4TThe image detection information of (2) is input to step S530.
The pseudo tag loop update submodule step S530: firstly, the filtered data set D of the unlabeled target domain obtained from the step S520-6TThe image detection frame information of (1); then, the image detection frame information format b is set to (x)min,ymin,xmax,ymaxConf) into the target Domain image data Label Format lT＝(xMIn,ymin,xmax,ymax) And recording the obtained label-free target domain data set DTThe tag data of
Finally, the obtained target domain image label data
And outputting to the labeled target domain data set generation module step S60.
In this embodiment, the labeled target domain data set generation module step S60: the module receives a label-free target domain data set DTMiddle image data
And corresponding tag data
Building labeled target domain data sets
And the target detection network Yolov3 is subjected to iterative updating operation, so that the target domain image label quality is improved.The implementation steps are shown in fig. 7, and specifically include the following steps:
step S610: obtaining a label-free target domain data set D from the step S530TImage tag data of
It is output to step S630.
Step S620: obtaining a label-free target domain data set D from the step S120TImage data of
It is output to step S630.
Step S630: respectively obtaining the data sets D of the unmarked target domain from the step S610 and the step S620tImage tag data of
And image data
Building labeled target domain data sets
Step S640: judging the fine tuning times of the target detection network, and executing the step S650 when the network reaches the fine tuning times; otherwise, step S660 is performed.
Step S650: obtaining the labeled target domain data set from the step S630
It is output to step S70.
Step S660: obtaining the labeled target domain data set from the step S630
Output itProceeding to step S40, the update target detection network is fine-tuned, and the update operation of the image tag data in the target domain data set is further performed.
Outputting labeled target domain data set step S70: obtaining the labeled target domain data set from the step S650
And outputting the data to realize the automatic labeling function of the data set of the label-free target domain.
The above description is only a preferred embodiment of the present invention, and should not be taken as limiting the invention, and all modifications, equivalents, improvements, etc. that are made within the spirit and principle of the present invention should be included in the scope of the present invention.
Claims (5)
1. A domain-adaptation-based autonomous learning data tag generation method is characterized in that: firstly, utilizing a generated countermeasure network to realize image conversion from a source domain image to a target domain image, and combining with label data of a source domain data set to construct a labeled intermediate domain synthetic data set; then, training a target detection network by using the labeled intermediate domain synthesis data set, and inputting image data in the data set of the label-free target domain to obtain a corresponding image detection frame result; then, dynamically updating the current target detection network confidence coefficient threshold parameter value, simultaneously carrying out noise filtering and cyclic updating operations on the detection frame result, and converting the detection frame result into an image tag data format; and finally, outputting image label data, and constructing a labeled target domain data set by combining corresponding target domain image data to realize the automatic labeling function of the target domain data set.
2. The method for generating the autonomously learnable data label based on the domain adaptation according to claim 1, wherein: the method comprises a data reading module step S10, an image conversion module step S20, an intermediate domain synthetic data set generating module step S30, a target detection network module step S40, a pseudo tag generating module step S50, a labeled target domain data set generating module step S60 and an output labeled target domain data set step S70;
the specific functions of each module are as follows:
data set reading module step S10: the image data set library is composed of a marked source domain data set and a non-marked target domain data set, and the module realizes the data reading function; reading the annotated source domain data set, outputting source domain image data and tag data to an image conversion module step S20 and an intermediate domain synthetic data set generation module step S30, respectively; reading the label-free target domain data set, and respectively outputting target domain image data to an image conversion module step S20 and a target detection network module step S40;
image conversion module step S20: the module realizes the image conversion function between different target scenes or categories by utilizing a generated countermeasure network; firstly, reading image data of a labeled source domain data set and image data of a label-free target domain data set from the data reading module S10, constructing an image training set, and inputting the image training set to a generation countermeasure network; secondly, training and generating a domain mapping relation for the confrontation network to learn the image distribution between two image domains, namely a source domain and a target domain, so as to realize image conversion operation between different target scenes or classes; then, reading the image data in the labeled source domain data set output by the data reading module in step S120, and inputting the image data into the trained generation countermeasure network to obtain intermediate domain synthetic image data; finally, the intermediate domain synthesized image data is output to the intermediate domain synthesized data set generating module step S30;
intermediate domain synthetic data set generating module step S30: the module realizes the construction of the marked intermediate domain synthetic data set; first, reading label data and intermediate domain synthetic image data in the labeled source domain data set from the data reading module step S10 and the image conversion module step S20, respectively, constructing a labeled intermediate domain synthetic data set, and outputting the labeled intermediate domain synthetic data set to the target detection network module step S40;
target detection network module step S40: the module obtains the result of the image detection frame of the unmarked target domain by applying a deep learning detection model; firstly, a step S30 of the intermediate domain synthetic data set generation module acquires a labeled intermediate domain synthetic data set, and inputs the labeled intermediate domain synthetic data set to a deep learning model for model training and learning; then, the data reading module step S120 obtains image data in the non-labeled target domain data set, inputs a deep learning model for testing, and obtains a detection frame result of the image data in the non-labeled target domain data set; finally, outputting the target domain image detection frame result to a pseudo label generation module step S50;
pseudo tag generation module step S50: the module realizes automatic updating operation of model confidence coefficient threshold parameters, and simultaneously carries out noise filtering on the detection frame result of the image of the unmarked target domain to obtain corresponding label data of the image in the data set of the unmarked target domain; firstly, obtaining the image detection frame result in the non-labeled target domain data set from the target detection network step S40, counting the sum of confidence scores of all detection frames, calculating the confidence average score of each detection frame, and setting and updating the confidence average score into the confidence threshold value hyperparameter of the subsequent model iterative update; secondly, setting the average confidence score value of the image detection frames obtained by current calculation as a filtering threshold value, and carrying out filtering operation on the image detection frames with the confidence scores lower than the filtering threshold value; finally, the image detection frame result obtained by filtering is converted into a label data format of a corresponding image, the label data of the current target domain data set is updated, and the label data is output to the labeled target domain data set generation module in step S60;
the labeled target domain data set generating module step S60: firstly, respectively acquiring tag data of a target domain data set and image data of the target domain data set from the step S50 of the pseudo tag generation module and the step S10 of the data reading module, and constructing a labeled target domain data set; then, inputting the obtained labeled target domain data set into a target detection network step S40 for iterative updating and optimization, and updating the label data in the labeled target domain data set again; finally, when the model reaches the cycle updating times, stopping the iterative updating of the model and outputting the labeled target domain data set;
outputting labeled target domain data set step S70: and step S60 of the pseudo label generating module acquires the labeled target domain data set, so as to implement the automatic labeling function of the target domain data set image.
3. The method for generating the autonomously learnable data label based on the domain adaptation according to claim 1, wherein: the method comprises the steps of generating an antagonistic network comprising DiscogAN, DualGAN and CycleGAN, training the network to learn the domain mapping relation between image domains of different target scenes or classes in an antagonistic learning mode, realizing the image conversion operation between a source domain image and a target domain image, generating a middle domain synthetic image with similar characteristics to the target domain image, and constructing a labeled middle domain synthetic data set by combining with label data of a source domain data set.
4. The method for generating the autonomously learnable data label based on the domain adaptation according to claim 1, wherein: the intermediate domain synthetic data set training deep learning model comprises SSD, fast R-CNN or Yolov3, the detection capability of the target domain image is obtained, the target domain image is input to obtain the detection frame result, the detection frame information is converted into the corresponding target domain image label information, and the automatic acquisition of the target domain image label data is realized.
5. The method for generating the autonomously learnable data label based on the domain adaptation according to claim 1, wherein: and constructing a labeled target domain data set by using the acquired target domain image label data, iteratively updating and optimizing the deep learning model, and filtering noise labels of the acquired target domain image to acquire a more accurate target domain image detection frame result and realize high-quality target domain image label data.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110488161.XA CN113378830A (en) | 2021-05-06 | 2021-05-06 | Domain-adaptation-based autonomous learning data label generation method |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110488161.XA CN113378830A (en) | 2021-05-06 | 2021-05-06 | Domain-adaptation-based autonomous learning data label generation method |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN113378830A true CN113378830A (en) | 2021-09-10 |
Family
ID=77570555
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202110488161.XA Pending CN113378830A (en) | 2021-05-06 | 2021-05-06 | Domain-adaptation-based autonomous learning data label generation method |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN113378830A (en) |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113869307A (en) * | 2021-09-16 | 2021-12-31 | 中国农业科学院农业资源与农业区划研究所 | Cross-category fruit data set automatic labeling system and method |
| CN114612744A (en) * | 2022-03-10 | 2022-06-10 | 平安科技（深圳）有限公司 | Detection model training method, vehicle damage detection method and terminal equipment |
Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109034125A (en) * | 2018-08-30 | 2018-12-18 | 北京工业大学 | Pedestrian detection method and system based on scene complexity |
| US20200125982A1 (en) * | 2018-02-06 | 2020-04-23 | Hrl Laboratories, Llc | System and method for unsupervised domain adaptation via sliced-wasserstein distance |
| CN111191732A (en) * | 2020-01-03 | 2020-05-22 | 天津大学 | Target detection method based on full-automatic learning |
| CN111882055A (en) * | 2020-06-15 | 2020-11-03 | 电子科技大学 | Method for constructing target detection self-adaptive model based on cycleGAN and pseudo label |
| CN112489076A (en) * | 2020-12-06 | 2021-03-12 | 北京工业大学 | Multi-target tracking method and system |
| CN112699892A (en) * | 2021-01-08 | 2021-04-23 | 北京工业大学 | Unsupervised field self-adaptive semantic segmentation method |
-
2021
- 2021-05-06 CN CN202110488161.XA patent/CN113378830A/en active Pending
Patent Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20200125982A1 (en) * | 2018-02-06 | 2020-04-23 | Hrl Laboratories, Llc | System and method for unsupervised domain adaptation via sliced-wasserstein distance |
| CN109034125A (en) * | 2018-08-30 | 2018-12-18 | 北京工业大学 | Pedestrian detection method and system based on scene complexity |
| CN111191732A (en) * | 2020-01-03 | 2020-05-22 | 天津大学 | Target detection method based on full-automatic learning |
| CN111882055A (en) * | 2020-06-15 | 2020-11-03 | 电子科技大学 | Method for constructing target detection self-adaptive model based on cycleGAN and pseudo label |
| CN112489076A (en) * | 2020-12-06 | 2021-03-12 | 北京工业大学 | Multi-target tracking method and system |
| CN112699892A (en) * | 2021-01-08 | 2021-04-23 | 北京工业大学 | Unsupervised field self-adaptive semantic segmentation method |
Non-Patent Citations (1)
| Title |
|---|
| 姚明海;黄展聪;: "基于主动学习的半监督领域自适应方法研究", 高技术通讯, no. 08, 15 August 2020 (2020-08-15) * |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113869307A (en) * | 2021-09-16 | 2021-12-31 | 中国农业科学院农业资源与农业区划研究所 | Cross-category fruit data set automatic labeling system and method |
| CN114612744A (en) * | 2022-03-10 | 2022-06-10 | 平安科技（深圳）有限公司 | Detection model training method, vehicle damage detection method and terminal equipment |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN110781262B (en) | Semantic map construction method based on visual SLAM | |
| CN110457982B (en) | Crop disease image identification method based on feature migration learning | |
| CN113378830A (en) | Domain-adaptation-based autonomous learning data label generation method | |
| CN110458022B (en) | Autonomous learning target detection method based on domain adaptation | |
| CN114067173A (en) | Small sample low-quality image target detection method based on multi-definition integrated self-training | |
| CN110853070A (en) | Underwater sea cucumber image segmentation method based on significance and Grabcut | |
| CN112613428B (en) | Resnet-3D convolution cattle video target detection method based on balance loss | |
| CN111079826A (en) | SLAM and image processing fused construction progress real-time identification method | |
| CN114596477A (en) | Foggy day train fault detection method based on field self-adaption and attention mechanism | |
| CN114092699A (en) | Method and system for cluster pig image segmentation based on transfer learning | |
| CN116310293B (en) | Method for detecting target of generating high-quality candidate frame based on weak supervised learning | |
| CN116912673A (en) | Target detection method based on underwater optical image | |
| CN116681961A (en) | Weak supervision target detection method based on semi-supervision method and noise processing | |
| CN114067359B (en) | Pedestrian detection method integrating human body key points and visible part attention characteristics | |
| CN114758135A (en) | Unsupervised image semantic segmentation method based on attention mechanism | |
| CN115049933A (en) | Unsupervised cross-domain plant organ counting method based on minimized characteristic difference | |
| CN114120359A (en) | Method for measuring body size of group-fed pigs based on stacked hourglass network | |
| CN113361530A (en) | Image semantic accurate segmentation and optimization method using interaction means | |
| CN114494332B (en) | Unsupervised synthesis to real LiDAR point cloud scene flow estimation method | |
| Meena Deshpande | License Plate Detection and Recognition using YOLO v4 | |
| CN113642500B (en) | Low-illumination target detection method based on multi-stage domain self-adaption | |
| CN113869307A (en) | Cross-category fruit data set automatic labeling system and method | |
| CN117079103B (en) | Pseudo tag generation method and system for neural network training | |
| CN117274750B (en) | Knowledge distillation semi-automatic visual labeling method and system | |
| CN113780524B (en) | Weather self-adaptive target detection system and method |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |