CN116385879A - Semi-supervised sea surface target detection method, system, equipment and storage medium - Google Patents
Semi-supervised sea surface target detection method, system, equipment and storage medium Download PDFInfo
- Publication number
- CN116385879A CN116385879A CN202310369693.0A CN202310369693A CN116385879A CN 116385879 A CN116385879 A CN 116385879A CN 202310369693 A CN202310369693 A CN 202310369693A CN 116385879 A CN116385879 A CN 116385879A
- Authority
- CN
- China
- Prior art keywords
- model
- sea surface
- surface target
- target detection
- supervised
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000001514 detection method Methods 0.000 title claims abstract description 110
- 238000012549 training Methods 0.000 claims abstract description 47
- 238000000034 method Methods 0.000 claims abstract description 38
- 230000008569 process Effects 0.000 claims abstract description 16
- 230000006870 function Effects 0.000 claims description 69
- 230000000694 effects Effects 0.000 claims description 12
- 238000004590 computer program Methods 0.000 claims description 11
- 238000013508 migration Methods 0.000 claims description 7
- 230000005012 migration Effects 0.000 claims description 7
- 230000003190 augmentative effect Effects 0.000 claims description 3
- 238000010276 construction Methods 0.000 claims description 3
- 230000001105 regulatory effect Effects 0.000 claims description 2
- 238000013519 translation Methods 0.000 claims description 2
- 230000007246 mechanism Effects 0.000 abstract description 4
- 238000004422 calculation algorithm Methods 0.000 description 8
- 230000000875 corresponding effect Effects 0.000 description 4
- 238000011161 development Methods 0.000 description 3
- 238000010586 diagram Methods 0.000 description 3
- 238000004364 calculation method Methods 0.000 description 2
- 238000012790 confirmation Methods 0.000 description 2
- 238000013527 convolutional neural network Methods 0.000 description 2
- 238000012216 screening Methods 0.000 description 2
- 238000007476 Maximum Likelihood Methods 0.000 description 1
- 238000013473 artificial intelligence Methods 0.000 description 1
- 238000013528 artificial neural network Methods 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 230000002596 correlated effect Effects 0.000 description 1
- 238000013135 deep learning Methods 0.000 description 1
- 230000001419 dependent effect Effects 0.000 description 1
- 238000002474 experimental method Methods 0.000 description 1
- 230000004438 eyesight Effects 0.000 description 1
- 238000001914 filtration Methods 0.000 description 1
- 238000011835 investigation Methods 0.000 description 1
- 238000002372 labelling Methods 0.000 description 1
- 238000005259 measurement Methods 0.000 description 1
- 238000012544 monitoring process Methods 0.000 description 1
- 238000010606 normalization Methods 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 238000004091 panning Methods 0.000 description 1
- 230000008092 positive effect Effects 0.000 description 1
- 230000000750 progressive effect Effects 0.000 description 1
- 238000007619 statistical method Methods 0.000 description 1
- 238000010200 validation analysis Methods 0.000 description 1
- 230000016776 visual perception Effects 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/10—Terrestrial scenes
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
- G06N3/0895—Weakly supervised learning, e.g. semi-supervised or self-supervised learning
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
- G06V10/77—Processing image or video features in feature spaces; using data integration or data reduction, e.g. principal component analysis [PCA] or independent component analysis [ICA] or self-organising maps [SOM]; Blind source separation
- G06V10/774—Generating sets of training patterns; Bootstrap methods, e.g. bagging or boosting
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V2201/00—Indexing scheme relating to image or video recognition or understanding
- G06V2201/07—Target detection
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02A—TECHNOLOGIES FOR ADAPTATION TO CLIMATE CHANGE
- Y02A90/00—Technologies having an indirect contribution to adaptation to climate change
- Y02A90/10—Information and communication technologies [ICT] supporting adaptation to climate change, e.g. for weather forecasting or climate simulation
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Evolutionary Computation (AREA)
- General Health & Medical Sciences (AREA)
- Artificial Intelligence (AREA)
- Software Systems (AREA)
- Computing Systems (AREA)
- Health & Medical Sciences (AREA)
- Multimedia (AREA)
- Databases & Information Systems (AREA)
- Medical Informatics (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Life Sciences & Earth Sciences (AREA)
- Biomedical Technology (AREA)
- Biophysics (AREA)
- Computational Linguistics (AREA)
- Data Mining & Analysis (AREA)
- Molecular Biology (AREA)
- General Engineering & Computer Science (AREA)
- Mathematical Physics (AREA)
- Image Analysis (AREA)
Abstract
The invention discloses a semi-supervised sea surface target detection method, a semi-supervised sea surface target detection system, semi-supervised sea surface target detection equipment and a storage medium, and relates to the field of target detection. Establishing a confidence threshold adjustment model of the sea surface target class, determining a classification loss function of the label-free data according to the confidence threshold adjustment model, and adjusting a regression loss function of the label-free data through the consistency of the bounding box, wherein the confidence threshold is utilized in the model training process to improve the quality of the pseudo labels. The confidence threshold value of each sea surface target class is dynamically adjusted, the quality of the pseudo tag is improved, the strong dependence on the cross-correlation ratio in the traditional positive and negative sample matching mechanism is relieved through the consistency regularization of the boundary box, the quality of the boundary box is improved, and further the sea surface target detection performance is improved.
Description
Technical Field
The present invention relates to the field of target detection, and in particular, to a method, a system, an apparatus, and a storage medium for semi-supervised sea surface target detection.
Background
With the continuous development of artificial intelligence, more and more unmanned devices are coming out. The advent of unmanned aerial vehicles, unmanned vehicles and surface unmanned boats for use in aerial operations has played a significant role in high-risk scenarios. The unmanned surface vessel is important intelligent equipment for replacing people to perform sea surface operation, and the existing unmanned surface vessel does not need manual driving and can autonomously complete dangerous and boring tasks such as tracking, submarine investigation, port patrol, mine removal and the like.
Meanwhile, the unmanned surface vessel can be provided with a visual perception system to realize sea surface target detection, and targets such as vessels and ships are positioned and identified when the targets appear in the detection image and are used for tracking and manual analysis in the later period.
Object detection is an important task in computer vision. Currently, the mainstream target detection task is based on a deep learning algorithm, and the detection performance of the algorithm depends on the number of labeled image samples in model training, namely, the position, the category and other label information of the detected target object. However, because the manual labeling requires a certain cost, researchers need to consider both the performance of the model and the development cost when carrying out experiments, which limits the development of the target detection algorithm to a certain extent, so that a semi-supervised learning algorithm is extended, and the core idea of the semi-supervised target detection algorithm is to utilize label-free data to improve the performance of the model.
Most of the current semi-supervised target detection algorithms are based on the methods of improving loss functions, data enhancement techniques, model training and the like. The most advanced semi-supervised target detection algorithm at present is mainly a method combining pseudo tag learning and consistency learning. However, when the pseudo tag learning is performed, there is often a serious problem of confirmation deviation. In order to improve the quality of the pseudo tag, the main stream target detection algorithm mainly measures the quality of the pseudo tag according to whether the maximum likelihood result of the classification result obtained by predicting the label-free image by the pre-trained detection model is larger than a preset fixed confidence threshold. However, the proportion of each category of data in the data set is different, so that the detection difficulty is different, and the same confidence threshold is unreasonable to set for filtering the detection results of all categories, so that serious confirmation deviation problems can be generated.
Studies have shown that the accuracy of classification confidence and target location is not positively correlated, and that the IoU (the intersection ratio of the prediction bounding box and the truth bounding box) of bounding boxes with higher classification confidence tends to be lower, so that it is limited to measure the quality of pseudo tags by considering only classification confidence. For the target positioning task, the strong dependence on IoU in the conventional positive and negative sample matching mechanism is not in accurate conflict with the positioning information of the pseudo tag in the semi-supervised detection task.
Disclosure of Invention
The invention aims to provide a semi-supervised sea surface target detection method, a semi-supervised sea surface target detection system, semi-supervised sea surface target detection equipment and a semi-supervised sea surface target detection storage medium.
In order to achieve the above object, the present invention provides the following solutions:
a semi-supervised sea surface target detection method, comprising:
constructing a semi-supervised sea surface target detection model; the semi-supervised sea surface target detection model comprises a teacher model and a student model which have the same initialization parameters and network architecture;
establishing a confidence threshold adjustment model of the sea surface target class, and perfecting a classification loss function of the label-free data according to the confidence threshold adjustment model;
the regression loss function of the label-free data is regulated through the consistency regularization of the boundary frame;
determining a total loss function of the semi-supervised sea surface target detection model according to the classification loss function and the adjusted regression loss function;
acquiring a sea surface target detection image dataset; the sea surface target detection image data set comprises a labeled image sample and an unlabeled image sample;
pre-training the teacher model by using the labeled image sample, and transferring the pre-trained teacher model parameters to a student model;
training a pre-trained teacher model and a student model with parameters transferred by adopting a sea surface target detection image data set based on the total loss function, dynamically adjusting the confidence threshold value of each sea surface target class by utilizing the confidence threshold value adjustment model in the training process, training the student model by utilizing labeled data and unlabeled data with pseudo labels, and transmitting the trained network weight parameters to the teacher model in an exponential moving average mode after the student model finishes the current parameter updating to obtain a trained teacher model;
inputting a sea surface target image to be detected into a trained teacher model, and outputting a detection result of each sea surface target in the sea surface target image to be detected.
Optionally, the confidence threshold adjustment model is
τ′＝T s (c)·τ；
Wherein τ' is the adjusted class c confidence threshold, T s (c) For the learning effect parameters of class c in step s, τ is the confidence threshold of class c before adjustment, ρ is the maximum value of the class prediction of the model in step s, G is the total number of unlabeled data, G is the G-th unlabeled data,
for the prediction of the unlabeled image samples I' by the model at step s,
for weak enhancement operations, y is a category.
Optionally, the classification loss function of the unlabeled data is
In the method, in the process of the invention,
for the classification loss function of unlabeled data, α is the ratio of unlabeled image samples to labeled image samples in the small lot, n is the number of labeled image samples in the small lot, σ is the statistical coefficient, H is the cross entropy loss function,
as pseudo tag, pm is the output probability of the model, +.>
For strong enhancement operations, I' k Is the kth unlabeled image sample.
Optionally, the regularizing the regression loss function of the unlabeled data through the consistency of the bounding box specifically includes:
constructing a boundary frame set for each false true value candidate frame by the teacher model by adopting a method based on cross-ratio distribution;
using the formula q=s γ ×D 1-γ Calculating the quality of the bounding box; wherein Q is the quality of the bounding box, S is the final classification score of the bounding box on the teacher model R-CNN, D is the intersection ratio between the bounding box and the false true value candidate box, and gamma is the control parameter;
sorting the boundary frames in each boundary frame set in descending order according to the size of Q, selecting the first K boundary frames as positive samples of the respective false true value candidate frames, wherein the boundary frames after the Kth boundary frame are negative samples of the respective false true value candidate frames;
using the formula
For each pseudoDefining regression consistency factors by a truth value candidate box; in E-shape i For the regression consistency factor of the ith false true value candidate box, K is the number of positive sample bounding boxes allocated to the ith false true value candidate box, +.>
The j positive sample of the i false true value candidate frame;
taking the regression consistency factor as an example-based regression loss weight, and obtaining a regression loss function of the label-free data as follows
In (1) the->
Reg and +.>
The regression output and the true value are respectively, M is the number of false true value candidate frames, and N is the number of positive samples of the ith false true value candidate frame.
Optionally, the total loss function of the semi-supervised sea surface target detection model is
Wherein L is a total loss function, and beta is a weighting coefficient;
for a cross entropy loss function with tag data,
I l for the first labeled image sample, +.>
Representing the first hard tag with tag data.
Optionally, pre-training the teacher model by using the labeled image sample specifically includes:
performing weak enhancement on the tagged image samples to expand the number of tagged image samples; the weak enhancement includes translation and flipping operations;
and pre-training the teacher model by using the weakly enhanced labeled image sample to obtain a pre-trained teacher model.
Optionally, based on the total loss function, training a pre-trained teacher model and a pre-parameter-migrated student model by adopting a sea surface target detection image dataset, and dynamically adjusting a confidence threshold value of each sea surface target class by using the confidence threshold value adjustment model in the training process to obtain a trained student model, which specifically comprises the following steps:
inputting the unlabeled image samples in the sea surface target detection image data set into a pre-trained teacher model, training the pre-trained teacher model, and outputting unlabeled image samples with pseudo labels when the accuracy rate of output results of the teacher model is greater than or equal to an accuracy rate threshold value;
extracting a preset number of labeled image samples from the sea surface target detection image data set;
weak enhancement is carried out on the extracted labeled image samples;
mixing a label-free image sample with a pseudo label and a weakly enhanced labeled image sample according to a proportion z by adopting a Mixup method to generate an enhanced data set; wherein z is a random value in the β distribution;
inputting the augmented data set into the student model after parameter migration, training the student model after parameter migration based on the total loss function, dynamically adjusting the confidence threshold value of each sea surface target class by using the confidence threshold value adjusting model in the training process, and updating the teacher model through the index moving average index of the student model to obtain the trained student model.
A semi-supervised sea surface target detection system, comprising:
the detection model construction module is used for constructing a semi-supervised sea surface target detection model; the semi-supervised sea surface target detection model comprises a teacher model and a student model which have the same initialization parameters and network architecture;
the classification loss determining module is used for establishing a confidence threshold adjustment model of the sea surface target class and determining a classification loss function of the label-free data according to the confidence threshold adjustment model;
the regression loss adjustment module is used for regularly adjusting the regression loss function of the label-free data through the consistency of the boundary frame;
the total loss determining module is used for determining a total loss function of the semi-supervised sea surface target detection model according to the classification loss function and the adjusted regression loss function;
the sample data set acquisition module is used for acquiring a sea surface target detection image data set; the sea surface target detection image data set comprises a labeled image sample and an unlabeled image sample;
the model migration module is used for pre-training the teacher model by using the labeled image sample and migrating the pre-trained teacher model parameters to the student model;
the training module is used for training the pre-trained teacher model and the student model with the parameter transferred by adopting the sea surface target detection image data set based on the total loss function, dynamically adjusting the confidence threshold value of each sea surface target class by utilizing the confidence threshold value adjusting model in the training process, training the student model by utilizing the labeled data and the unlabeled data with the pseudo labels, and transmitting the trained network weight parameters to the teacher model in an exponential moving average mode after the student model finishes the current parameter updating to obtain a trained teacher model;
the application module is used for inputting a sea surface target image to be detected into the trained teacher model and outputting the category of each sea surface target in the sea surface target image to be detected.
An electronic device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, the processor implementing a semi-supervised sea surface target detection method as described above when executing the computer program.
A computer readable storage medium having stored thereon a computer program which when executed implements a semi-supervised sea surface target detection method as previously described.
According to the specific embodiment provided by the invention, the invention discloses the following technical effects:
the invention discloses a semi-supervised sea surface target detection method, a system, equipment and a storage medium, wherein a confidence threshold adjustment model of sea surface target categories is established, a classification loss function of label-free data is determined according to the confidence threshold adjustment model, the regression loss function of the label-free data is adjusted through a boundary frame consistency regular, the confidence threshold of each sea surface target category is dynamically adjusted by the confidence threshold adjustment model in the model training process, the quality of pseudo labels is improved, the strong dependence on the cross-correlation ratio in a traditional positive and negative sample matching mechanism is relieved through the boundary frame consistency regular, the quality of the boundary frame is improved, and further the sea surface target detection performance is improved.
Drawings
In order to more clearly illustrate the embodiments of the present invention or the technical solutions of the prior art, the drawings that are needed in the embodiments will be briefly described below, it being obvious that the drawings in the following description are only some embodiments of the present invention, and that other drawings may be obtained according to these drawings without inventive effort for a person skilled in the art.
FIG. 1 is a flow chart of a semi-supervised sea surface target detection method provided by an embodiment of the invention;
FIG. 2 is a schematic diagram of a semi-supervised sea surface target detection method according to an embodiment of the present invention;
FIG. 3 is a schematic diagram of a class consistency method according to an embodiment of the present invention;
fig. 4 is a schematic diagram of a positioning consistency method according to an embodiment of the present invention.
Detailed Description
The following description of the embodiments of the present invention will be made clearly and completely with reference to the accompanying drawings, in which it is apparent that the embodiments described are only some embodiments of the present invention, but not all embodiments. All other embodiments, which can be made by those skilled in the art based on the embodiments of the invention without making any inventive effort, are intended to be within the scope of the invention.
The invention aims to provide a semi-supervised sea surface target detection method, a semi-supervised sea surface target detection system, semi-supervised sea surface target detection equipment and a semi-supervised sea surface target detection storage medium.
In order that the above-recited objects, features and advantages of the present invention will become more readily apparent, a more particular description of the invention will be rendered by reference to the appended drawings and appended detailed description.
As shown in fig. 1, an embodiment of the present invention provides a semi-supervised sea surface target detection method, including:
step 1: constructing a semi-supervised sea surface target detection model; the semi-supervised sea surface target detection model comprises a teacher model and a student model with the same initialization parameters and network architecture.
The teacher model and the student model are both based on the target detection network, and the network architecture adopts a Faster-RCNN neural network.
Step 2: and establishing a confidence threshold adjustment model of the sea surface target class, and determining a classification loss function of the unlabeled data according to the confidence threshold adjustment model.
For unlabeled data, first, the learning effect of each category is determined by the number of unlabeled image samples belonging to that category and above a fixed threshold, which are used to flexibly adjust the threshold to let the best unlabeled data pass the confidence threshold. At the same time, learning efficiency does not always increase, and learning effects may also decrease if predictions of unlabeled data fall into other categories in subsequent iterations.
The existing confidence threshold tau is constant, and the invention dynamically adjusts the fixed confidence threshold for each category with different learning effects. The student model adjusts the confidence threshold for each category based on the learning efficiency for that category: in the target detection model, the number of samples reaching the initial confidence coefficient threshold value in each category is subjected to statistical analysis to obtain a learning effect of each category, and the confidence coefficient threshold value is correspondingly adjusted by utilizing the learning effect, so that the category confidence coefficient threshold value which is more in line with the actual situation for each category is obtained.
Specifically, the invention scales the threshold by the model's predicted effect for each class, assuming that when the threshold is high, the learning effect of a class can be reflected by the number of samples it predicts to fall into that class and above the threshold. That is, a class with fewer samples and a smaller number of predictive confidence levels reaching a threshold is considered to have greater difficulty in learning or worse learning, and this process does not introduce additional reasoning processes, nor does it require additional validation sets, and the formula for determining learning is as follows:
τ′＝T s (c)·τ
wherein τ' is the adjusted class c confidence threshold, T s (c) For the learning effect parameters of class c in step s, τ is the confidence threshold of class c before adjustment, ρ is the maximum value of the class prediction of the model in step s, G is the total number of unlabeled data, G is the G-th unlabeled data,
for the prediction of the unlabeled image samples I' by the model at step s,
for weak enhancement operations, y is a category.
When the categories in the unlabeled dataset are balanced, i.e., the number of unlabeled data belonging to different categories is equal or close), a larger T s (c) Indicating that the estimated learning effect is better. By T of s (c) The following normalization process is performed to range from 0 to 1, which can then be used to measure the fixed threshold τ.
The principle of the category consistency method is shown in fig. 3.
The classification loss function of the unlabeled data is:
in the method, in the process of the invention,
for the classification loss function of unlabeled data, α is the ratio of unlabeled image samples to labeled image samples in the small lot, n is the number of labeled image samples in the small lot, σ is the statistical coefficient, H is the cross entropy loss function,
is a pseudo tag, P m Output probability of model, +.>
For strong enhancement operations, I' k Is the kth unlabeled image sample. Pseudo tag here->
Pseudo tags in one-hot form, i.e. all negative tags are 0 except that the positive tag is 1.
Step 3: and (5) regularizing the regression loss function of the unlabeled data through the consistency of the bounding box.
Since the classification score does not represent the quality of the regression frame. In the semi-supervised target detection, the traditional method for screening the bounding box by utilizing the IoU (cross ratio) size between the prediction bounding box and the true value is too dependent on the accuracy of the true value, and the false label of the label-free sample data in the semi-supervised target detection is obtained through model prediction, so that the accuracy of the false label cannot be guaranteed. To solve this problem, the present invention provides a simple and effective method of: and positioning consistency to screen out a high-quality bounding box and carrying out certain constraint on the influence of the bounding box in regression loss. A plurality of bounding boxes are assigned to each pseudo-true bounding box, and the consistency of regression results for these bounding boxes can reflect the target positioning quality of the corresponding pseudo-true bounding box. The quality of the boundary frame is improved according to the boundary frame consistency rules, and the positioning quality of the pseudo tag boundary frame is reflected by the regression consistency of the positive samples.
For unlabeled data, first, we measure the quality of the bounding box by setting Q, as follows:
Q＝S γ ×D 1-γ
where S represents the final classification score of the bounding box on the teacher network R-CNN, D represents the IoU value between the bounding box and the false true value candidate box, and γ is used to control the contribution of S and IOU to the final calculated Q. Here, the student network shares the RPN generated bounding box of the teacher network.
For unlabeled sample data, the teacher model firstly builds a bounding box set for each false true value candidate box by using a traditional IoU allocation method, then sorts the bounding boxes in each bounding box set according to the quality Q of the bounding boxes, and finally selects the first K as positive samples and the rest as negative samples. In this way, a high quality bounding box is screened out. For calculation of K, we perform summation operation through the positive sample and IoU value of the true value corresponding to the positive sample, and perform down-rounding if the result is decimal.
Second, for the filtered bounding box, some noise will still be present, in order to improve the edgesThe locating accuracy of the bounding box is achieved by using all positive samples corresponding to each false true value candidate box, calculating the average value of IoU with the false true value candidate box, and naming the average value as a measurement standard to be a regression consistency factor epsilon i 。
Where i denotes the index of the false true value candidate box, K denotes the number of positive sample bounding boxes allocated to the ith false true value candidate box, and where the consistency factor E is obtained i Then, the regression loss weight is used as an example-based regression loss weight to adjust the contribution degree of each false true value candidate box to the regression loss of the label-free data. If the consistency factor is small, it indicates that the positioning information of the false true candidate box is relatively inaccurate, so its contribution to the loss calculation should be reduced, and vice versa. The regression loss function formula for unlabeled data is as follows:
wherein reg and
the regression output and ground-trunk (true value) are represented, respectively. M represents the number of false true value candidate boxes, and N is the number of positive samples of the ith false true value candidate box.
The principle of the positioning consistency method is shown in fig. 4.
Step 4: and determining the total loss function of the semi-supervised sea surface target detection model according to the classification loss function and the adjusted regression loss function.
And finally, directly adopting conventional classification and bounding box regression loss for monitoring the tagged data. And for the unlabeled sample data, screening the pseudo labels of the unlabeled data through the confidence threshold of the corresponding category so as to improve the quality of the pseudo labels. While the regression loss is adjusted by bounding box consistency regularization. The bounding box consistency is used as a positioning loss to form an overall loss function in combination with the improved classification loss.
For tagged data, the teacher model performs weak enhancement on the tagged data, and then directly monitors the tagged data by adopting cross entropy loss, and the formula is as follows:
wherein n represents the number of tagged image samples in the small lot, I represents tagged data, p m The output probability of the model is expressed, y is the category,
hard tag representing the first tagged data, < >>
Representing weak enhancement operations.
The student model directly monitors tagged data by adopting smoothL1 function loss as a loss function.
The final loss function is a weighted sum of supervised and unsupervised losses, where the weighting coefficients are β:
step 5: acquiring a sea surface target detection image dataset; the sea surface target detection image data set comprises a labeled image sample and an unlabeled image sample.
Step 6: and pre-training the teacher model by using the labeled image sample, and migrating the pre-trained teacher model parameters to a student model.
Step 7: based on the total loss function, training a pre-trained teacher model and a student model with parameters transferred by adopting a sea surface target detection image data set, dynamically adjusting the confidence threshold value of each sea surface target class by utilizing the confidence threshold value adjusting model in the training process, training the student model by utilizing labeled data and unlabeled data with pseudo labels, and transmitting the trained network weight parameters to the teacher model in an exponential moving average mode after the student model finishes the current parameter updating to obtain a trained teacher model.
The student network performs inverse random gradient descent according to the loss function, and the teacher network updates with EMA (Exponential Moving Average, index average index) of the student network.
The teacher model is trained:
1. the labeled image samples in the dataset are weakly enhanced to augment the labeled image sample dataset, the weakly enhancement including panning and flipping operations. And pre-training the teacher model by using the expanded labeled image samples, so that the teacher model preliminarily has a certain target detection capability.
2. And inputting the unlabeled image sample into a pre-trained teacher model to obtain a prediction result of the unlabeled image sample.
The student model is trained:
a certain number of tagged image samples and untagged image samples with pseudo tags are extracted from the original dataset according to a certain proportion to form a small batch of data, as shown in fig. 2. Performing weak enhancement on the extracted labeled image sample, and performing strong enhancement operation on the unlabeled image sample with the pseudo label and the labeled image sample after weak enhancement; the data intensity enhancement adopts Mixup operation, and a non-label image with a pseudo label and a labeled image are mixed according to a certain proportion z (z is a random value in beta distribution) to generate a new enhanced data set; then, inputting the augmented data set into a student model, wherein the student model and the pre-trained teacher model have the same initialization parameters and network architecture; and then, the quality of the pseudo tag is improved through adjustment of the confidence threshold value of each category to realize consistency of category classification, and meanwhile, the positioning accuracy of the target is improved by utilizing the positioning consistency regularization of the candidate frame, so that the influence caused by the positioning deviation of the pseudo tag is reduced. Finally, the weighted and improved positioning loss and classification loss form an overall loss function to randomly gradient down to update the student model, and the teacher model is updated through the index moving average EMA of the student model; at this time, when the student network converges, training is completed, and the student network can be independently used for target detection of pictures.
Step 8: inputting a sea surface target image to be detected into a trained teacher model, and outputting the category of each sea surface target in the sea surface target image to be detected.
According to the method, the confidence coefficient threshold value of each sea surface target class is correspondingly adjusted according to the learning efficiency of the class, a class confidence coefficient threshold value set is formed, and meanwhile, in order to relieve the conflict that the strong dependence on IoU in a traditional positive and negative sample matching mechanism and the positioning information of the pseudo tag in a semi-supervision detection task are inaccurate, the quality of the boundary box is further improved through the consistency and the regularization of the boundary box. Therefore, the quality of the pseudo tag is improved, and the positive effect of the non-tag data on model training is improved.
The embodiment of the invention also provides a semi-supervised sea surface target detection system, which comprises:
the detection model construction module is used for constructing a semi-supervised sea surface target detection model; the semi-supervised sea surface target detection model comprises a teacher model and a student model which have the same initialization parameters and network architecture;
the classification loss determining module is used for establishing a confidence threshold adjustment model of the sea surface target class and determining a classification loss function of the label-free data according to the confidence threshold adjustment model;
the regression loss adjustment module is used for regularly adjusting the regression loss function of the label-free data through the consistency of the boundary frame;
the total loss determining module is used for determining a total loss function of the semi-supervised sea surface target detection model according to the classification loss function and the adjusted regression loss function;
the sample data set acquisition module is used for acquiring a sea surface target detection image data set; the sea surface target detection image data set comprises a labeled image sample and an unlabeled image sample;
the model migration module is used for pre-training the teacher model by using the labeled image sample and migrating the pre-trained teacher model parameters to the student model;
the training module is used for training the pre-trained teacher model and the student model with the parameter transferred by adopting the sea surface target detection image data set based on the total loss function, dynamically adjusting the confidence threshold value of each sea surface target class by utilizing the confidence threshold value adjusting model in the training process, training the student model by utilizing the labeled data and the unlabeled data with the pseudo labels, and transmitting the trained network weight parameters to the teacher model in an exponential moving average mode after the student model finishes the current parameter updating to obtain a trained teacher model;
the application module is used for inputting a sea surface target image to be detected into the trained teacher model and outputting the category of each sea surface target in the sea surface target image to be detected.
The semi-supervised sea surface target detection system provided by the embodiment of the invention is similar to the semi-supervised sea surface target detection method described in the above embodiment in terms of working principle and beneficial effects, so that details are not described herein, and specific details can be found in the description of the above method embodiment.
The invention also provides an electronic device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, the processor implementing the semi-supervised sea surface target detection method as described above when executing the computer program.
Furthermore, the computer program in the above-described memory may be stored in a computer-readable storage medium when it is implemented in the form of a software functional unit and sold or used as a separate product. Based on this understanding, the technical solution of the present invention may be embodied essentially or in a part contributing to the prior art or in the form of a software product stored in a storage medium, comprising several instructions for causing a computer device (which may be a personal computer, a server or a network device, etc.) to perform all or part of the steps of the method according to the embodiments of the present invention. And the aforementioned storage medium includes: various media capable of storing program codes, such as a U disk, a mobile hard disk, a read-only memory, a random access memory, a magnetic disk or an optical disk.
Further, the present invention also provides a computer readable storage medium having stored thereon a computer program which when executed implements a semi-supervised sea surface target detection method as described above.
In the present specification, each embodiment is described in a progressive manner, and each embodiment is mainly described in a different point from other embodiments, and identical and similar parts between the embodiments are all enough to refer to each other.
The principles and embodiments of the present invention have been described herein with reference to specific examples, the description of which is intended only to assist in understanding the methods of the present invention and the core ideas thereof; also, it is within the scope of the present invention to be modified by those of ordinary skill in the art in light of the present teachings. In view of the foregoing, this description should not be construed as limiting the invention.
Claims (10)
1. A semi-supervised sea surface target detection method, comprising:
constructing a semi-supervised sea surface target detection model; the semi-supervised sea surface target detection model comprises a teacher model and a student model which have the same initialization parameters and network architecture;
establishing a confidence threshold adjustment model of the sea surface target class, and determining a classification loss function of the label-free data according to the confidence threshold adjustment model;
the regression loss function of the label-free data is regulated through the consistency regularization of the boundary frame;
determining a total loss function of the semi-supervised sea surface target detection model according to the classification loss function and the adjusted regression loss function;
acquiring a sea surface target detection image dataset; the sea surface target detection image data set comprises a labeled image sample and an unlabeled image sample;
pre-training the teacher model by using the labeled image sample, and transferring the pre-trained teacher model parameters to a student model;
training a pre-trained teacher model and a student model with parameters transferred by adopting a sea surface target detection image data set based on the total loss function, dynamically adjusting the confidence threshold value of each sea surface target class by utilizing the confidence threshold value adjustment model in the training process, training the student model by utilizing labeled data and unlabeled data with pseudo labels, and transmitting the trained network weight parameters to the teacher model in an exponential moving average mode after the student model finishes the current parameter updating to obtain a trained teacher model;
inputting a sea surface target image to be detected into a trained teacher model, and outputting the category of each sea surface target in the sea surface target image to be detected.
2. The semi-supervised sea surface target detection method of claim 1, wherein said confidence threshold adjustment model is
τ′＝T s (c)·τ；
Wherein v' is the adjusted confidence threshold of class c, T s (c) For the learning effect parameters of class c in step s, τ is the confidence threshold of class c before adjustment, ρ is the maximum value of the class prediction of the model in step s, G is the total number of unlabeled data, G is the G-th unlabeled data,
prediction of unlabeled image samples I' for model at step s,/for model>
For weak enhancement operations, y is a category.
3. The semi-supervised sea surface target detection method of claim 2, wherein the classification loss function of the unlabeled data is
In the above-mentioned method, the step of,
for the classification loss function of unlabeled data, α is the ratio of unlabeled image samples to labeled image samples in the small lot, n is the number of labeled image samples in the small lot, σ is the statistical coefficient, H is the cross entropy loss function,
is a pseudo tag, p m Output probability of model, +.>
For strong enhancement operations, I' k Is the kth unlabeled image sample.
4. A semi-supervised sea surface target detection method according to claim 3, wherein the regularized regression loss function of unlabeled data by bounding box consistency specifically comprises:
constructing a boundary frame set for each false true value candidate frame by the teacher model by adopting a method based on cross-ratio distribution;
using the formula q=s γ ×D 1-γ Calculating the quality of the bounding box; wherein Q is the quality of the bounding box, S is the final classification score of the bounding box on the teacher model R-CNN, D is the intersection ratio between the bounding box and the false true value candidate box, and gamma is the control parameter;
sorting the boundary frames in each boundary frame set in descending order according to the size of Q, selecting the first K boundary frames as positive samples of the respective false true value candidate frames, wherein the boundary frames after the Kth boundary frame are negative samples of the respective false true value candidate frames;
using the formula
Defining a regression consistency factor for each false true value candidate box; in E-shape i For the regression consistency factor of the ith false true value candidate box, K is the number of positive sample bounding boxes allocated to the ith false true value candidate box, +.>
The j positive sample of the i false true value candidate frame;
taking the regression consistency factor as an example-based regression loss weight, and obtaining a regression loss function of the label-free data as follows
In (1) the->
Reg and
the regression output and the true value are respectively, M is the number of false true value candidate frames, and N is the number of positive samples of the ith false true value candidate frame.
5. The semi-supervised sea surface target detection method of claim 4, wherein the total loss function of the semi-supervised sea surface target detection model is
Wherein L is a total loss function, and beta is a weighting coefficient;
for a cross entropy loss function with tag data,
I l for the first labeled image sample, +.>
Representing the first hard tag with tag data.
6. The semi-supervised sea surface target detection method of claim 1, wherein the teacher model is pre-trained with the labeled image samples, comprising:
performing weak enhancement on the tagged image samples to expand the number of tagged image samples; the weak enhancement includes translation and flipping operations;
and pre-training the teacher model by using the weakly enhanced labeled image sample to obtain a pre-trained teacher model.
7. The semi-supervised sea surface target detection method of claim 6, wherein the training of the pre-trained teacher model and the parameter-migrated student model using the sea surface target detection image dataset based on the total loss function, and the dynamic adjustment of the confidence threshold of each sea surface target class using the confidence threshold adjustment model during training, specifically comprises:
inputting the unlabeled image samples in the sea surface target detection image data set into a pre-trained teacher model, training the pre-trained teacher model, and outputting unlabeled image samples with pseudo labels when the accuracy rate of output results of the teacher model is greater than or equal to an accuracy rate threshold value;
extracting a preset number of labeled image samples from the sea surface target detection image data set;
weak enhancement is carried out on the extracted labeled image samples;
mixing a label-free image sample with a pseudo label and a weakly enhanced labeled image sample according to a proportion z by adopting a Mixup method to generate an enhanced data set; wherein z is a random value in the β distribution;
inputting the augmented data set into the student model after parameter migration, training the student model after parameter migration based on the total loss function, dynamically adjusting the confidence threshold value of each sea surface target class by using the confidence threshold value adjusting model in the training process, and updating the teacher model through the index moving average index of the student model to obtain the trained student model.
8. A semi-supervised sea surface target detection system, comprising:
the detection model construction module is used for constructing a semi-supervised sea surface target detection model; the semi-supervised sea surface target detection model comprises a teacher model and a student model which have the same initialization parameters and network architecture;
the classification loss determining module is used for establishing a confidence threshold adjustment model of the sea surface target class and determining a classification loss function of the label-free data according to the confidence threshold adjustment model;
the regression loss adjustment module is used for regularly adjusting the regression loss function of the label-free data through the consistency of the boundary frame;
the total loss determining module is used for determining a total loss function of the semi-supervised sea surface target detection model according to the classification loss function and the adjusted regression loss function;
the sample data set acquisition module is used for acquiring a sea surface target detection image data set; the sea surface target detection image data set comprises a labeled image sample and an unlabeled image sample;
the model migration module is used for pre-training the teacher model by using the labeled image sample and migrating the pre-trained teacher model parameters to the student model;
the training module is used for training the pre-trained teacher model and the student model with the parameter transferred by adopting the sea surface target detection image data set based on the total loss function, dynamically adjusting the confidence threshold value of each sea surface target class by utilizing the confidence threshold value adjusting model in the training process, training the student model by utilizing the labeled data and the unlabeled data with the pseudo labels, and transmitting the trained network weight parameters to the teacher model in an exponential moving average mode after the student model finishes the current parameter updating to obtain a trained teacher model;
the application module is used for inputting a sea surface target image to be detected into the trained teacher model and outputting the category of each sea surface target in the sea surface target image to be detected.
9. An electronic device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, the processor implementing the semi-supervised sea surface target detection method as recited in any of claims 1-7 when the computer program is executed by the processor.
10. A computer readable storage medium, characterized in that a computer program is stored thereon, which computer program, when executed, implements the semi-supervised sea surface target detection method as claimed in any of claims 1 to 7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310369693.0A CN116385879A (en) | 2023-04-07 | 2023-04-07 | Semi-supervised sea surface target detection method, system, equipment and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310369693.0A CN116385879A (en) | 2023-04-07 | 2023-04-07 | Semi-supervised sea surface target detection method, system, equipment and storage medium |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN116385879A true CN116385879A (en) | 2023-07-04 |
Family
ID=86961233
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202310369693.0A Pending CN116385879A (en) | 2023-04-07 | 2023-04-07 | Semi-supervised sea surface target detection method, system, equipment and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN116385879A (en) |
Cited By (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN117058489A (en) * | 2023-10-09 | 2023-11-14 | 腾讯科技（深圳）有限公司 | Training method, device, equipment and storage medium of multi-label recognition model |
| CN117253071A (en) * | 2023-07-25 | 2023-12-19 | 山东建筑大学 | Semi-supervised target detection method and system based on multistage pseudo tag enhancement |
| CN117372819A (en) * | 2023-12-07 | 2024-01-09 | 神思电子技术股份有限公司 | Target detection increment learning method, device and medium for limited model space |
-
2023
- 2023-04-07 CN CN202310369693.0A patent/CN116385879A/en active Pending
Cited By (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN117253071A (en) * | 2023-07-25 | 2023-12-19 | 山东建筑大学 | Semi-supervised target detection method and system based on multistage pseudo tag enhancement |
| CN117253071B (en) * | 2023-07-25 | 2024-02-20 | 山东建筑大学 | Semi-supervised target detection method and system based on multistage pseudo tag enhancement |
| CN117058489A (en) * | 2023-10-09 | 2023-11-14 | 腾讯科技（深圳）有限公司 | Training method, device, equipment and storage medium of multi-label recognition model |
| CN117058489B (en) * | 2023-10-09 | 2023-12-29 | 腾讯科技（深圳）有限公司 | Training method, device, equipment and storage medium of multi-label recognition model |
| CN117372819A (en) * | 2023-12-07 | 2024-01-09 | 神思电子技术股份有限公司 | Target detection increment learning method, device and medium for limited model space |
| CN117372819B (en) * | 2023-12-07 | 2024-02-20 | 神思电子技术股份有限公司 | Target detection increment learning method, device and medium for limited model space |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN116385879A (en) | Semi-supervised sea surface target detection method, system, equipment and storage medium | |
| US11151417B2 (en) | Method of and system for generating training images for instance segmentation machine learning algorithm | |
| CN112150821B (en) | Lightweight vehicle detection model construction method, system and device | |
| CN109101108A (en) | Method and system based on three decision optimization intelligence cockpit human-computer interaction interfaces | |
| CN114241340A (en) | Image target detection method and system based on double-path depth residual error network | |
| CN116110022B (en) | Lightweight traffic sign detection method and system based on response knowledge distillation | |
| CN116894985B (en) | Semi-supervised image classification method and semi-supervised image classification system | |
| CN111127360A (en) | Gray level image transfer learning method based on automatic encoder | |
| CN113436174A (en) | Construction method and application of human face quality evaluation model | |
| CN114842238A (en) | Embedded mammary gland ultrasonic image identification method | |
| CN115527103A (en) | Unmanned ship perception experiment platform system | |
| CN116912568A (en) | Noise-containing label image recognition method based on self-adaptive class equalization | |
| CN115511892A (en) | Training method of semantic segmentation model, semantic segmentation method and device | |
| CN113989519B (en) | Long-tail target detection method and system | |
| CN116452810A (en) | Multi-level semantic segmentation method and device, electronic equipment and storage medium | |
| CN112270404A (en) | Detection structure and method for bulge defect of fastener product based on ResNet64 network | |
| CN116977710A (en) | Remote sensing image long tail distribution target semi-supervised detection method | |
| CN115205573A (en) | Image processing method, device and equipment | |
| US20220138573A1 (en) | Methods and systems for training convolutional neural networks | |
| CN114557670A (en) | Physiological age prediction method, apparatus, device and medium | |
| JP2022088341A (en) | Apparatus learning device and method | |
| CN114241227A (en) | VLAD (very-large-scale integration) -based image identification method and device | |
| CN113139612A (en) | Image classification method, training method of classification network and related products | |
| CA3070816A1 (en) | Method of and system for generating training images for instance segmentation machine learning algorithm | |
| CN111881833B (en) | Vehicle detection method, device, equipment and storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |