CN117649528A - Semi-supervised image segmentation method, system, electronic equipment and storage medium - Google Patents
Semi-supervised image segmentation method, system, electronic equipment and storage medium Download PDFInfo
- Publication number
- CN117649528A CN117649528A CN202410114897.4A CN202410114897A CN117649528A CN 117649528 A CN117649528 A CN 117649528A CN 202410114897 A CN202410114897 A CN 202410114897A CN 117649528 A CN117649528 A CN 117649528A
- Authority
- CN
- China
- Prior art keywords
- image segmentation
- image
- training
- supervised
- data set
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 238000003709 image segmentation Methods 0.000 title claims abstract description 131
- 238000000034 method Methods 0.000 title claims abstract description 88
- 238000012549 training Methods 0.000 claims abstract description 68
- 238000012545 processing Methods 0.000 claims abstract description 12
- 230000006870 function Effects 0.000 claims description 32
- 230000003044 adaptive effect Effects 0.000 claims description 9
- 238000013507 mapping Methods 0.000 claims description 3
- 238000012790 confirmation Methods 0.000 abstract description 6
- 230000002401 inhibitory effect Effects 0.000 abstract 1
- 230000008569 process Effects 0.000 description 13
- 238000010586 diagram Methods 0.000 description 6
- 238000002372 labelling Methods 0.000 description 6
- 238000004364 calculation method Methods 0.000 description 5
- 238000004590 computer program Methods 0.000 description 5
- 238000013459 approach Methods 0.000 description 4
- 239000003550 marker Substances 0.000 description 4
- 239000011159 matrix material Substances 0.000 description 3
- 239000012141 concentrate Substances 0.000 description 2
- 230000006872 improvement Effects 0.000 description 2
- 238000012986 modification Methods 0.000 description 2
- 230000004048 modification Effects 0.000 description 2
- 230000011218 segmentation Effects 0.000 description 2
- 230000008485 antagonism Effects 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 230000008859 change Effects 0.000 description 1
- 238000013527 convolutional neural network Methods 0.000 description 1
- 238000013135 deep learning Methods 0.000 description 1
- 230000007547 defect Effects 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 230000018109 developmental process Effects 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 230000007613 environmental effect Effects 0.000 description 1
- 230000003090 exacerbative effect Effects 0.000 description 1
- 238000010191 image analysis Methods 0.000 description 1
- 238000004519 manufacturing process Methods 0.000 description 1
- 238000007781 pre-processing Methods 0.000 description 1
- 238000012546 transfer Methods 0.000 description 1
- 230000009466 transformation Effects 0.000 description 1
- 238000010200 validation analysis Methods 0.000 description 1
Landscapes
- Image Analysis (AREA)
Abstract
The invention discloses a semi-supervised image segmentation method, a semi-supervised image segmentation system, electronic equipment and a storage medium, and belongs to the technical field of image segmentation. The method comprises the steps of obtaining an image to be segmented, inputting the image to be segmented into a trained image segmentation model for processing, and obtaining an image segmentation result; the training of the image segmentation model specifically comprises the following steps: constructing an annotation data set, and performing supervision training on the image segmentation model by using an annotation-driven sample self-adaptive weighting method based on the annotation data set; and constructing an unlabeled data set, and performing unsupervised training on the image segmentation model through a stability-driven sample self-adaptive weighting method based on the unlabeled data set. The utilization efficiency of valuable information contained in difficult samples in marked data can be improved while inhibiting confirmation deviation, and the utilization efficiency of the difficult samples in unmarked data sets is improved; the problem of difficult sample interference leads to the image segmentation precision low is solved.
Description
Technical Field
The present invention relates to the field of image segmentation technologies, and in particular, to a semi-supervised image segmentation method, system, electronic device, and storage medium.
Background
The statements in this section merely relate to the background of the present disclosure and may not necessarily constitute prior art.
Image segmentation is a fundamental problem in computer vision, whose purpose is to infer semantic labels for all pixels in an image. Image segmentation has made significant progress over the past decade with the advent of large-scale data sets and the rapid development of convolutional neural networks and segmentation models.
Because the image acquisition equipment has equipment variability and different environmental conditions of acquired images, the characteristics of unclear characteristic textures, large characteristic variability and the like of partial images to be segmented influence the accuracy of image segmentation. If the image segmentation method is improved, the image segmentation model is oversized, and the instantaneity and portability of image segmentation are affected.
The fully supervised image segmentation approach learns how to assign pixel-level semantic labels by generalizing from a large number of densely annotated images, and despite the rapid progress of such approaches, pixel-by-pixel manual labeling is expensive and laborious, impeding deployment of such approaches in scenarios such as medical image analysis.
In order to reduce the cost of image annotation, semi-supervised learning methods have further been proposed to train an image segmentation model with small amounts of annotated data and large amounts of unlabeled data. The existing semi-supervised learning method which is dominant in deep learning comprises self-training, consistency regularization, antagonism learning and the like. Although these methods have been somewhat constructive, how to more efficiently utilize unlabeled data remains one of the most interesting problems of these semi-supervised learning methods.
Most existing semi-supervised learning approaches utilize predictions of unlabeled data to obtain information. However, the quality of the model's predictions of unlabeled data is not guaranteed due to the lack of labeled data. Low quality predictions can lead to models learning in the wrong direction, thereby impeding the improvement of model performance. Difficult samples contain a lot of valuable information, but such samples often lead to low quality predictions of the model.
In order to solve the problem of low utilization efficiency of difficult samples, the current mainstream method is to perform adaptive weighting of samples, so that the network is more concerned about learning of the difficult samples. The most typical method is to measure the difficulty of samples from the angle of uncertainty of entropy values, and the sample self-adaptive weighting method based on the entropy values, on one hand, guides the network to pay more attention to samples with low prediction uncertainty in unsupervised learning; on the other hand, it guides the network to pay more attention to the high uncertainty samples in supervised learning.
However, although entropy values can evaluate the difficulty of a sample from an uncertainty point of view, it cannot evaluate the difficulty of a sample from a prediction accuracy point of view. However, prediction accuracy is also an angle considering difficulty of samples, and samples with low prediction accuracy generally represent a type of samples with poor network learning and should be regarded as difficult samples. Considering only the prediction uncertainty and not the prediction accuracy, the network may not be concerned with learning samples with low uncertainty and low prediction accuracy. For example, a sample is mispredicted with a high certainty, then the sample is a difficult sample for the network. Since the prediction uncertainty of this sample is low, calculating the sample weight from the uncertainty alone gives the sample a relatively small weight, which in turn makes the network neglect to learn such difficult samples.
In an unsupervised training scenario, the predictive quality of unlabeled samples by the network is not guaranteed. If the difficulty of evaluating the sample is only from the uncertainty point of view, but the difficulty of evaluating the sample from the prediction accuracy point of view is ignored, a situation that a sample with high prediction uncertainty and low prediction accuracy is given a larger weight occurs in the network training process, so that the occurrence of confirmation deviation is aggravated. For example, if the network's prediction of a certain unlabeled exemplar is inaccurate, then an erroneous pseudo tag may be generated for that exemplar. However, if the prediction of the unlabeled exemplar by the network is uncertain, this means that when the difficulty of evaluating the exemplar is evaluated from an uncertainty point of view, the unlabeled exemplar is given a greater weight, which in turn makes the network pay more attention to learning the unlabeled exemplar. However, the pseudo tag corresponding to the unlabeled exemplar is inaccurate, which can cause the network to deepen the learning of the error message, thereby exacerbating the occurrence of the validation bias.
In addition, for supervised training, labeled samples with low prediction uncertainty and low prediction accuracy often contain a large amount of valuable information as well; however, the existing sample self-adaptive weighting method based on entropy value just ignores the attention to the samples, reduces the utilization efficiency of difficult samples, and influences the precision of an image segmentation model.
Most of the existing semi-supervised methods based on threshold false labels train models by using false labels with prediction confidence higher than the threshold, and simply neglect the utilization of other false labels. From the perspective of web learning, the learning difficulty of different samples is different. Difficult samples in unlabeled datasets with reliable predictions are typically ignored by the network because their confidence in predictions is below a threshold.
Disclosure of Invention
In order to solve the defects in the prior art, the invention provides a semi-supervised image segmentation method, a semi-supervised image segmentation system, electronic equipment and a computer readable storage medium, and two different sample adaptive weighting strategies are provided according to the difference between supervised learning and unsupervised learning so as to force a model to concentrate on learning valuable areas and improve the prediction accuracy of image segmentation.
In a first aspect, the present invention provides a semi-supervised image segmentation method;
a semi-supervised image segmentation method, comprising:
acquiring an image to be segmented;
inputting an image to be segmented into a trained image segmentation model for processing, and obtaining an image segmentation result;
the training of the image segmentation model specifically comprises the following steps:
constructing an annotation data set, and performing supervision training on the image segmentation model by using an annotation-driven sample self-adaptive weighting method based on the annotation data set;
and constructing an unlabeled data set, and performing unsupervised training on the image segmentation model through a stability-driven sample self-adaptive weighting method based on the unlabeled data set.
Further, the performing supervised training on the image segmentation model by using a sample self-adaptive weighting method driven by the annotation based on the annotation data set comprises the following steps:
inputting the marked images in the marked data set into the image segmentation model to obtain an image segmentation prediction result;
according to the image segmentation prediction result, performing uncertainty estimation on the marked image by a Meng Daka Dropout method to obtain an uncertainty weight;
calculating cross entropy loss according to the image segmentation prediction result; acquiring importance weights of the marked samples according to the cross entropy loss and the uncertainty weights;
and acquiring a supervision loss function according to the importance weight and the cross entropy loss of the marked sample, and performing supervision training on the image segmentation model by using the supervision loss function.
Preferably, the supervised loss function is expressed as:
wherein->To supervise the loss function->For marking the size of the smallest lot of images, +.>(. Cndot.) is cross entropy loss, (. Cndot.)>For marking sample importance weights, +.>Label corresponding to the ith marked image, < >>The prediction result of the ith marked image.
Further, the performing, based on the non-labeling data set, non-supervision training on the image segmentation model by using a stability-driven sample adaptive weighting method includes:
after carrying out weak enhancement and strong enhancement on the unmarked images in the unmarked data set, respectively inputting two identical image segmentation models in parallel, and correspondingly obtaining an image segmentation prediction result;
calculating a prediction entropy value of each unmarked sample in a random scene by a Monte Carlo Dropout method, and calculating a prediction stability score of each unmarked image by KL divergence according to an image segmentation prediction result of a historical training round and a prediction result of a current training round to obtain an importance weight;
and according to the importance weight, self-adaptively adjusting a pseudo tag weight threshold value, and acquiring an unsupervised loss function so as to perform unsupervised training on the image segmentation model by using the unsupervised loss function.
Preferably, the performing the unsupervised training on the image segmentation model by using a stability-driven sample adaptive weighting method based on the unlabeled dataset further includes:
after carrying out weak enhancement and strong enhancement on the unmarked images in the unmarked data set, respectively inputting two identical image segmentation models in parallel, and correspondingly obtaining a first image segmentation prediction result and a second image segmentation prediction result;
mapping the first image segmentation prediction result into a pseudo tag, and calculating cross entropy loss by combining the second image segmentation prediction result to serve as initial unsupervised loss.
Preferably, the historical prediction results of the samples are obtained by using model checkpoints stored after each training round is completed.
Preferably, the unsupervised loss function is expressed as:
wherein, the method comprises the steps of, wherein,as an unsupervised loss function->Is the smallest lot of unlabeled data, +.>(. Cndot.) is a conditional function, (. Cndot.)>Importance weight for unlabeled samples, +.>Is the height of the unlabeled image, +.>Is the width of the unlabeled image, +.>For the j-th element in the uncertainty estimation result of the unlabeled image,/th element in the uncertainty estimation result of the unlabeled image>Is adaptive threshold value->For maximizing the function, for calculating the maximum value of the class prediction probability, +.>For cross entropy loss, < >>For the function of the parameters of the function, the index corresponding to the category with the highest prediction probability is obtained,/>For predicting probability +.>For weak enhancement operation, ++>For the j-th pixel value in the i-th unlabeled image,/th pixel value in the i-th unlabeled image>Is the predictive entropy value.
In a second aspect, the present invention provides a semi-supervised image segmentation system;
a semi-supervised image segmentation system, comprising:
an acquisition module configured to: acquiring an image to be segmented;
an image segmentation module configured to: inputting an image to be segmented into a trained image segmentation model for processing, and obtaining an image segmentation result;
the training of the image segmentation model specifically comprises the following steps:
constructing an annotation data set, and performing supervision training on the image segmentation model by using an annotation-driven sample self-adaptive weighting method based on the annotation data set;
and constructing an unlabeled data set, and performing unsupervised training on the image segmentation model through a stability-driven sample self-adaptive weighting method based on the unlabeled data set.
In a third aspect, the present invention provides an electronic device;
an electronic device comprising a memory and a processor and computer instructions stored on the memory and running on the processor, which when executed by the processor, perform the steps of the semi-supervised image segmentation method described above.
In a fourth aspect, the present invention provides a computer-readable storage medium;
a computer readable storage medium storing computer instructions which, when executed by a processor, perform the steps of the semi-supervised image segmentation method described above.
Compared with the prior art, the invention has the beneficial effects that:
1. according to the technical scheme provided by the invention, in order to improve the accuracy and the instantaneity of image segmentation and not influence the overall architecture of the image segmentation model, two different sample importance self-adaptive weighting methods are provided according to the difference between supervised training and unsupervised training, the model is forced to concentrate on learning valuable areas, the training scene of the model and the accuracy and the uncertainty of a prediction result are fully considered in the sample weighting process, and the efficient utilization of difficult samples is realized under the condition that larger confirmation deviation is not introduced.
2. According to the technical scheme, the sample self-adaptive weighting method driven by the labeling is provided for the supervision training process, labeling information is further introduced based on the prediction uncertainty information of the samples according to the characteristics of the supervision training scene, so that the high-efficiency weighting of the labeled samples is realized, and the problem that the utilization rate of valuable information contained in the labeled samples with low prediction uncertainty but low accuracy is low in the traditional method is solved.
3. For unsupervised training, unlabeled samples with high prediction uncertainty and high prediction accuracy typically also contain a large amount of valuable information; according to the technical scheme, aiming at an unsupervised training process, the stability-driven sample self-adaptive weighting method is provided for self-adaptive weighting of unlabeled samples, the prediction accuracy of the samples is evaluated by calculating the prediction stability score of the samples, and the importance weight of the unlabeled samples is calculated by combining the prediction accuracy information and the uncertain information, so that the utilization efficiency of valuable information contained in difficult samples is improved while confirmation deviation is restrained.
4. The technical scheme provided by the invention provides a threshold self-adaptive adjustment strategy based on sample difficulty and prediction reliability to further improve the utilization efficiency of unmarked difficult samples in the data set.
Drawings
The accompanying drawings, which are included to provide a further understanding of the invention and are incorporated in and constitute a part of this specification, illustrate embodiments of the invention and together with the description serve to explain the invention.
FIG. 1 is a schematic flow chart provided in an embodiment of the present invention;
fig. 2 is a schematic flow chart of model training according to an embodiment of the present invention.
Detailed Description
It should be noted that the following detailed description is exemplary and is intended to provide further explanation of the invention. Unless defined otherwise, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention belongs.
It is noted that the terminology used herein is for the purpose of describing particular embodiments only and is not intended to be limiting of exemplary embodiments according to the present invention. As used herein, unless the context clearly indicates otherwise, the singular forms also are intended to include the plural forms, and furthermore, it is to be understood that the terms "comprises" and "comprising" and any variations thereof are intended to cover non-exclusive inclusions, such as, for example, processes, methods, systems, products or devices that comprise a series of steps or units, are not necessarily limited to those steps or units that are expressly listed, but may include other steps or units that are not expressly listed or inherent to such processes, methods, products or devices.
Embodiments of the invention and features of the embodiments may be combined with each other without conflict.
Example 1
The image segmentation method in the prior art has poor segmentation precision when the image with unclear texture features and large feature differences is segmented; therefore, the invention provides a semi-supervised image segmentation method, which does not need to change a model framework, performs supervised training and unsupervised training on an image segmentation model, efficiently utilizes valuable information contained in a difficult sample under the condition of not introducing error information, optimizes network parameters of the image segmentation model, and improves the accuracy of image segmentation.
Next, a semi-supervised image segmentation method disclosed in this embodiment is described in detail with reference to fig. 1-2. The semi-supervised image segmentation method comprises the following steps:
s1, acquiring an image to be segmented.
S2, inputting the image to be segmented into a trained image segmentation model for processing, and obtaining an image segmentation result, wherein the image segmentation model is a U-NET network.
As one embodiment, the training image segmentation model specifically includes:
and step 1, preprocessing a data set.
Specifically, first, a random data enhancement method is used to perform a data enhancement operation on an existing data set to expand the data set. Then, the scale transformation is performed for all the images in the dataset, ensuring that all the images have a uniform size.
Specific modes of data enhancement operations include random clipping, horizontal flipping, vertical flipping, random rotation, and gaussian noise addition, among others.
And 2, constructing an annotation data set, and performing supervision training on the image segmentation model through an annotation-driven sample self-adaptive weighting method based on the annotation data set.
In the step, labeling information is further introduced on the basis of sample prediction uncertainty information aiming at the characteristics of the supervision training scene, so that efficient weighting of the labeled samples is realized. The specific flow is as follows:
step 201, inputting a marked image in the marked data set into an image segmentation model for training, and enabling the image segmentation model to conduct T times of forward propagation under a random Dropout situation in the training process through a Meng Daka Dropout method to obtain an image segmentation prediction result.
T dropouts are performed by a Meng Daka Dropout method, and forward propagation is performed once after each Dropout, so that T random forward propagation is performed, and finally T predicted results are obtained for each image, wherein the predicted results of random image segmentation are average values of the T predicted results obtained after T forward propagation is performed under the random Dropout situation.
Step 202, performing uncertainty estimation on a marked image through a Meng Daka Dropout method according to an image segmentation prediction result to obtain an uncertainty weight; the uncertain weight refers to the prediction entropy obtained by the Meng Daka Dropout method, and the larger the entropy value is, the higher the prediction uncertainty is, and the greater the prediction difficulty of the image segmentation model on the sample is.
The prediction entropy is expressed as:wherein->Prediction probability obtained for the t-th prediction, < >>(. Cndot.) represents a weak enhancement operation, (. Cndot.)>Representing the ith image in the marker dataset, c representing the category, T being the number of random forward propagates, +.>For the predictive entropy value corresponding to the c-th type in the ith marked image,/th>As a logarithmic function.
Uncertainty of the marked sample is estimated at pixel level, and uncertainty estimation result of the ith marked image in the marked data setThe j-th element of (a)>Where H represents the width of the image and W represents the height of the image.
In step 203, samples with low prediction uncertainty and low accuracy should also be regarded as difficult samples, in order to further screen out samples with low prediction uncertainty and low accuracy, and adaptively increase the weight of samples such as "samples with low uncertainty and low accuracy" according to the prediction accuracy, so that the network is more concerned about learning of such samples, and in this embodiment, labeling information is further introduced to evaluate the accuracy of sample prediction.
Specifically, the distance between the prediction of each pixel point and its corresponding marker is measured by using the cross entropy loss, and the distance is used together with the uncertainty estimation result (uncertainty weight) for the calculation of the marker sample importance weight.
The calculation formula for the marker sample importance weight can be expressed as:wherein->Representing a marked imageCross entropy loss value of j-th pixel point in (a)>Is->Importance weight matrix of->The j-th element of (a).
Will beAnd image->The cross entropy loss of (a) is multiplied to obtain a final supervision loss function, and the supervision loss function is expressed as: />Wherein (1)>Representing the size of the smallest lot of the marked image,/->(. Cndot.) represents cross entropy loss, (. Cndot.)>For marking sample importance weights, +.>Label corresponding to the ith marked image, < >>The prediction result of the ith marked image.
And 3, constructing an unlabeled data set, and performing unsupervised training on the image segmentation model through a stability-driven sample self-adaptive weighting method based on the unlabeled data set.
As an embodiment, step 3 specifically includes:
step 301, obtaining unlabeled data, and constructing an unlabeled data set.
Step 302, after the same unlabeled image in the unlabeled data set is subjected to weak enhancement operation and strong enhancement operation respectively, the enhanced image is respectively input into two identical image segmentation models in parallel, and a first image segmentation prediction result and a second image segmentation prediction result are correspondingly obtained. Mapping the predictions of the weakly enhanced image as pseudo labels for use in calculating cross entropy loss with the predictions of the strongly enhanced image and taking the loss as the initial unsupervised loss.
This step is the idea of typical semi-supervised learning, and uses such a semi-supervised learning framework as a basic framework, so as to transfer the information of the marked data into the unmarked data through consistency constraint, and further use the information of the unmarked data.
Step 303, estimating the prediction uncertainty of each unlabeled image by calculating the prediction entropy value of each unlabeled image under different scenes after the strong enhancement processing by using Monte Carlo Dropout.
Wherein, the prediction entropy value is expressed as:
wherein->Prediction probability obtained for the t-th prediction, < >>(. Cndot.) represents a strongly enhanced operation, (. Cndot.)>Representing the ith image in the unlabeled dataset, c represents the category. The method estimates uncertainty at the pixel level, image +.>Uncertainty estimation result ∈10->The j-th element of (a)>Wherein H and W represent the width and height of the image, respectively.
Step 304, calculating the prediction stability score of each unlabeled image through KL divergence.
Wherein, the formula of the predicted stability score may be expressed as:wherein K represents a historical round number, K represents a current round number, ++>Represents->The prediction result of the j-th pixel in the k-th training,represents->The prediction result of the j-th pixel in the K-th training.
The predicted result of the unlabeled image in the historical turns is obtained by using a model checkpoint saved after the end of each training turn, which refers to the intermediate state of the model saved periodically during training.
In order to make the network pay attention to learning of valuable information contained in difficult samples while suppressing occurrence of confirmation deviation, in the present embodiment, the prediction accuracy of samples is evaluated using the prediction stability score of each unlabeled image.
Step 305, calculating importance weights of the unlabeled images according to the prediction stability score and the prediction entropy value of each unlabeled image.
Exemplary, unlabeled imagesImportance weight matrix of->The calculation formula of each element in (a) can be expressed as: />Wherein (1)>。
Unlabeled imageImportance weight matrix of->The method can be further applied to calculation of unsupervised loss, so that the network pays attention to learning of the difficult sample with reliable prediction results in the training process, and further high-efficiency utilization of the difficult sample is realized without introducing large confirmation deviation.
Step 306, adaptively adjusting a threshold for pseudo tag generation based on the sample difficulty and the prediction reliability.
In the embodiment, in the process of generating the pseudo tag in the unsupervised training scene, the threshold value is dynamically adjusted according to the difficulty and the prediction reliability of the sample, and a threshold value self-adaptive adjustment method based on the difficulty and the prediction reliability of the sample is provided.
Importance weighting of unlabeled imagesMeanwhile, the difficulty and the prediction reliability of the samples are reflected, the difficult samples with reliable prediction results in the unlabeled data set have larger importance weights, and the difficult samples with unreliable prediction results and the easy samples have smaller weights.
Exemplary, willThe formula applied to the calculation of the threshold value can be expressed asWherein->An initial threshold value set manually, +.>And e is the current training round and is the maximum training round dynamically adjusted for the threshold value.
During the unsupervised training process, false labels with difficult samples of reliable predictors are assigned a small threshold and difficult and easy samples with unreliable predictors are assigned a large threshold. To this end, the expression for the unsupervised loss function can be expressed as:wherein->Is the smallest lot of unlabeled data, +.>(. Cndot.) represents a conditional function, (. Cndot.)>For maximizing the function, for calculating the maximum value of the class prediction probability, +.>For the function of the parameters, the index corresponding to the category with the highest prediction probability is obtained, and when the prediction confidence of the sample is larger than the corresponding threshold value, the corresponding pseudo tag is generated and used for calculating the unsupervised loss.
S4, network training.
Specifically, the total loss function in the network training process can be defined as:wherein->Weighting factors set for humans for balancing supervision losses +.>And unsupervised loss->。
The image segmentation model can repeatedly perform reverse propagation training based on the loss function L in the learning process, and the loss value can slowly decrease along with the increase of training rounds. When the loss value reaches the minimum value, the obtained network model is the best training result.
Example two
The embodiment discloses a semi-supervised image segmentation system, comprising:
an acquisition module configured to: acquiring an image to be segmented;
an image segmentation module configured to: inputting an image to be segmented into a trained image segmentation model for processing, and obtaining an image segmentation result;
the training of the image segmentation model specifically comprises the following steps:
constructing an annotation data set, and performing supervision training on the image segmentation model by using an annotation-driven sample self-adaptive weighting method based on the annotation data set;
and constructing an unlabeled data set, and performing unsupervised training on the image segmentation model through a stability-driven sample self-adaptive weighting method based on the unlabeled data set.
It should be noted that, the acquiring module and the image dividing module correspond to the steps in the first embodiment, and the modules are the same as the examples and the application scenarios implemented by the corresponding steps, but are not limited to the disclosure in the first embodiment. It should be noted that the modules described above may be implemented as part of a system in a computer system, such as a set of computer-executable instructions.
Example III
The third embodiment of the invention provides an electronic device, which comprises a memory, a processor and computer instructions stored on the memory and running on the processor, wherein the steps of the semi-supervised image segmentation method are completed when the computer instructions are run by the processor.
Example IV
A fourth embodiment of the present invention provides a computer readable storage medium storing computer instructions that, when executed by a processor, perform the steps of the semi-supervised image segmentation method described above.
The present invention is described with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems) and computer program products according to embodiments of the invention. It will be understood that each flow and/or block of the flowchart illustrations and/or block diagrams, and combinations of flows and/or blocks in the flowchart illustrations and/or block diagrams, can be implemented by computer program instructions. These computer program instructions may be provided to a processor of a general purpose computer, special purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instruction means which implement the function specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
The foregoing embodiments are directed to various embodiments, and details of one embodiment may be found in the related description of another embodiment.
The above description is only of the preferred embodiments of the present invention and is not intended to limit the present invention, but various modifications and variations can be made to the present invention by those skilled in the art. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention should be included in the protection scope of the present invention.
Claims (10)
1. A semi-supervised image segmentation method, comprising:
acquiring an image to be segmented;
inputting an image to be segmented into a trained image segmentation model for processing, and obtaining an image segmentation result;
the training of the image segmentation model specifically comprises the following steps:
constructing an annotation data set, and performing supervision training on the image segmentation model by using an annotation-driven sample self-adaptive weighting method based on the annotation data set;
and constructing an unlabeled data set, and performing unsupervised training on the image segmentation model through a stability-driven sample self-adaptive weighting method based on the unlabeled data set.
2. The semi-supervised image segmentation method as set forth in claim 1, wherein the supervised training of the image segmentation model by an annotation driven sample adaptive weighting method based on the annotation dataset comprises:
inputting the marked images in the marked data set into the image segmentation model to obtain an image segmentation prediction result;
according to the image segmentation prediction result, performing uncertainty estimation on the marked image by a Meng Daka Dropout method to obtain an uncertainty weight;
calculating cross entropy loss according to the image segmentation prediction result; acquiring importance weights of the marked samples according to the cross entropy loss and the uncertainty weights;
and acquiring a supervision loss function according to the importance weight and the cross entropy loss of the marked sample, and performing supervision training on the image segmentation model by using the supervision loss function.
3. The semi-supervised image segmentation method as set forth in claim 2, wherein the supervised loss function is expressed as:
，
wherein,to supervise the loss function->For marking the size of the smallest lot of images, +.>(. Cndot.) is cross entropy loss, (. Cndot.)>For marking sample importance weights, +.>Label corresponding to the ith marked image, < >>The prediction result of the ith marked image.
4. The semi-supervised image segmentation method as set forth in claim 1, wherein the non-supervised training of the image segmentation model by a stability driven sample adaptive weighting method based on the non-labeled dataset comprises:
after carrying out weak enhancement and strong enhancement on the unmarked images in the unmarked data set, respectively inputting two identical image segmentation models in parallel, and correspondingly obtaining an image segmentation prediction result;
calculating a prediction entropy value of each unmarked sample in a random scene by a Monte Carlo Dropout method, and calculating a prediction stability score of each unmarked image by KL divergence according to an image segmentation prediction result of a historical training round and a prediction result of a current training round to obtain an importance weight;
and according to the importance weight, self-adaptively adjusting a pseudo tag weight threshold value, and acquiring an unsupervised loss function so as to perform unsupervised training on the image segmentation model by using the unsupervised loss function.
5. The semi-supervised image segmentation method as set forth in claim 4, wherein the non-supervised training of the image segmentation model by a stability-driven sample adaptive weighting method based on the non-labeled dataset further comprises:
after carrying out weak enhancement and strong enhancement on the unmarked images in the unmarked data set, respectively inputting two identical image segmentation models in parallel, and correspondingly obtaining a first image segmentation prediction result and a second image segmentation prediction result;
mapping the first image segmentation prediction result into a pseudo tag, and calculating cross entropy loss by combining the second image segmentation prediction result to serve as initial unsupervised loss.
6. The semi-supervised image segmentation method as set forth in claim 4, wherein the historical predictions for the samples are obtained using model checkpoints maintained after the end of each training round.
7. The semi-supervised image segmentation method as set forth in claim 4, wherein the unsupervised loss function is expressed as:
，
wherein,as an unsupervised loss function->Is the smallest lot of unlabeled data, +.>(. Cndot.) is a conditional function, (. Cndot.)>Importance weight for unlabeled samples, +.>Is the height of the unlabeled image, +.>Is the width of the unlabeled image, +.>For the j-th element in the uncertainty estimation result of the unlabeled image,/th element in the uncertainty estimation result of the unlabeled image>Is adaptive threshold value->To maximize the function +.>For cross entropy loss, < >>For parameterizing functions, +.>For predicting probability +.>For weak enhancement operation, ++>For the j-th pixel value in the i-th unlabeled image,/th pixel value in the i-th unlabeled image>Is the predictive entropy value.
8. A semi-supervised image segmentation system, comprising:
an acquisition module configured to: acquiring an image to be segmented;
an image segmentation module configured to: inputting an image to be segmented into a trained image segmentation model for processing, and obtaining an image segmentation result;
the training of the image segmentation model specifically comprises the following steps:
constructing an annotation data set, and performing supervision training on the image segmentation model by using an annotation-driven sample self-adaptive weighting method based on the annotation data set;
and constructing an unlabeled data set, and performing unsupervised training on the image segmentation model through a stability-driven sample self-adaptive weighting method based on the unlabeled data set.
9. An electronic device comprising a memory and a processor and computer instructions stored on the memory and running on the processor, which when executed by the processor, perform the steps of the semi-supervised image segmentation method of any of claims 1-7.
10. A computer readable storage medium storing computer instructions which, when executed by a processor, perform the steps of the semi-supervised image segmentation method of any of claims 1-7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202410114897.4A CN117649528B (en) | 2024-01-29 | Semi-supervised image segmentation method, system, electronic equipment and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202410114897.4A CN117649528B (en) | 2024-01-29 | Semi-supervised image segmentation method, system, electronic equipment and storage medium |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN117649528A true CN117649528A (en) | 2024-03-05 |
| CN117649528B CN117649528B (en) | 2024-05-31 |
Family
ID=
Citations (15)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2019075771A1 (en) * | 2017-10-20 | 2019-04-25 | Huawei Technologies Co., Ltd. | Self-training method and system for semi-supervised learning with generative adversarial networks |
| CN113255892A (en) * | 2021-06-01 | 2021-08-13 | 上海交通大学烟台信息技术研究院 | Method and device for searching decoupled network structure and readable storage medium |
| CN113657455A (en) * | 2021-07-23 | 2021-11-16 | 西北工业大学 | Semi-supervised learning method based on triple network and labeling consistency regularization |
| CN113781491A (en) * | 2021-11-11 | 2021-12-10 | 阿里巴巴达摩院(杭州)科技有限公司 | Training of image segmentation model, image segmentation method and device |
| CN115761300A (en) * | 2022-10-18 | 2023-03-07 | 江门市卓立安全科技有限公司 | Method, system and detection device for dividing safety exit abnormity |
| WO2023038574A1 (en) * | 2021-09-10 | 2023-03-16 | Grabtaxi Holdings Pte. Ltd. | Method and system for processing a target image |
| CN115966302A (en) * | 2023-01-04 | 2023-04-14 | 河北工业大学 | Semi-supervised gonitis auxiliary analysis method based on deep contrast learning |
| CN116051574A (en) * | 2022-12-28 | 2023-05-02 | 河南大学 | Semi-supervised segmentation model construction and image analysis method, device and system |
| CN116258729A (en) * | 2023-03-17 | 2023-06-13 | 江苏科技大学 | Heart magnetic resonance image segmentation method based on improved collaborative training |
| CN116363144A (en) * | 2023-03-31 | 2023-06-30 | 南京大学 | Semi-supervised medical image segmentation method based on disturbance collaborative training |
| CN116486076A (en) * | 2023-03-31 | 2023-07-25 | 长沙理工大学 | Remote sensing image semantic segmentation method, system, equipment and storage medium |
| CN116597136A (en) * | 2023-03-31 | 2023-08-15 | 长沙理工大学 | Semi-supervised remote sensing image semantic segmentation method and system |
| CN116883791A (en) * | 2023-07-13 | 2023-10-13 | 青岛兴牧畜牧科技发展有限公司 | Pig house monitoring model training method and pig house monitoring model |
| CN116935054A (en) * | 2023-08-01 | 2023-10-24 | 重庆理工大学 | Semi-supervised medical image segmentation method based on hybrid-decoupling training |
| CN117409201A (en) * | 2023-10-24 | 2024-01-16 | 江苏省肿瘤医院 | MR medical image colorectal cancer segmentation method and system based on semi-supervised learning |
Patent Citations (15)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2019075771A1 (en) * | 2017-10-20 | 2019-04-25 | Huawei Technologies Co., Ltd. | Self-training method and system for semi-supervised learning with generative adversarial networks |
| CN113255892A (en) * | 2021-06-01 | 2021-08-13 | 上海交通大学烟台信息技术研究院 | Method and device for searching decoupled network structure and readable storage medium |
| CN113657455A (en) * | 2021-07-23 | 2021-11-16 | 西北工业大学 | Semi-supervised learning method based on triple network and labeling consistency regularization |
| WO2023038574A1 (en) * | 2021-09-10 | 2023-03-16 | Grabtaxi Holdings Pte. Ltd. | Method and system for processing a target image |
| CN113781491A (en) * | 2021-11-11 | 2021-12-10 | 阿里巴巴达摩院(杭州)科技有限公司 | Training of image segmentation model, image segmentation method and device |
| CN115761300A (en) * | 2022-10-18 | 2023-03-07 | 江门市卓立安全科技有限公司 | Method, system and detection device for dividing safety exit abnormity |
| CN116051574A (en) * | 2022-12-28 | 2023-05-02 | 河南大学 | Semi-supervised segmentation model construction and image analysis method, device and system |
| CN115966302A (en) * | 2023-01-04 | 2023-04-14 | 河北工业大学 | Semi-supervised gonitis auxiliary analysis method based on deep contrast learning |
| CN116258729A (en) * | 2023-03-17 | 2023-06-13 | 江苏科技大学 | Heart magnetic resonance image segmentation method based on improved collaborative training |
| CN116363144A (en) * | 2023-03-31 | 2023-06-30 | 南京大学 | Semi-supervised medical image segmentation method based on disturbance collaborative training |
| CN116486076A (en) * | 2023-03-31 | 2023-07-25 | 长沙理工大学 | Remote sensing image semantic segmentation method, system, equipment and storage medium |
| CN116597136A (en) * | 2023-03-31 | 2023-08-15 | 长沙理工大学 | Semi-supervised remote sensing image semantic segmentation method and system |
| CN116883791A (en) * | 2023-07-13 | 2023-10-13 | 青岛兴牧畜牧科技发展有限公司 | Pig house monitoring model training method and pig house monitoring model |
| CN116935054A (en) * | 2023-08-01 | 2023-10-24 | 重庆理工大学 | Semi-supervised medical image segmentation method based on hybrid-decoupling training |
| CN117409201A (en) * | 2023-10-24 | 2024-01-16 | 江苏省肿瘤医院 | MR medical image colorectal cancer segmentation method and system based on semi-supervised learning |
Non-Patent Citations (3)
| Title |
|---|
| LI, ZONGRUI等: "MTCSNet: Mean Teachers Cross-Supervision Network for Semi-Supervised Cloud Detection", 《REMOTE SENSING》, 15 May 2023 (2023-05-15) * |
| 刘雨溪;张铂;王斌;: "基于生成式对抗网络的遥感图像半监督语义分割", 红外与毫米波学报, no. 04, 15 August 2020 (2020-08-15) * |
| 贾颖霞;郎丛妍;冯松鹤;: "基于类别相关的领域自适应交通图像语义分割方法", 计算机研究与发展, no. 04, 14 April 2020 (2020-04-14) * |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN111192292B (en) | Target tracking method and related equipment based on attention mechanism and twin network | |
| CN113221905B (en) | Semantic segmentation unsupervised domain adaptation method, device and system based on uniform clustering and storage medium | |
| CN110120064B (en) | Depth-related target tracking algorithm based on mutual reinforcement and multi-attention mechanism learning | |
| CN109635763B (en) | Crowd density estimation method | |
| CN113852432B (en) | Spectrum Prediction Sensing Method Based on RCS-GRU Model | |
| CN111523422A (en) | Key point detection model training method, key point detection method and device | |
| CN113111716B (en) | Remote sensing image semiautomatic labeling method and device based on deep learning | |
| CN112052818A (en) | Unsupervised domain adaptive pedestrian detection method, unsupervised domain adaptive pedestrian detection system and storage medium | |
| WO2023207389A1 (en) | Data processing method and apparatus, program product, computer device, and medium | |
| CN115187772A (en) | Training method, device and equipment of target detection network and target detection method, device and equipment | |
| CN115861738A (en) | Category semantic information guided remote sensing target detection active sampling method | |
| CN115829983B (en) | High-speed industrial scene visual quality detection method based on knowledge distillation | |
| CN117649528B (en) | Semi-supervised image segmentation method, system, electronic equipment and storage medium | |
| CN112418149A (en) | Abnormal behavior detection method based on deep convolutional neural network | |
| CN107247996A (en) | A kind of Active Learning Method applied to different distributed data environment | |
| CN116681961A (en) | Weak supervision target detection method based on semi-supervision method and noise processing | |
| CN117649528A (en) | Semi-supervised image segmentation method, system, electronic equipment and storage medium | |
| CN116258861A (en) | Semi-supervised semantic segmentation method and segmentation device based on multi-label learning | |
| CN116385466A (en) | Method and system for dividing targets in image based on boundary box weak annotation | |
| CN113344005B (en) | Image edge detection method based on optimized small-scale features | |
| CN114220086A (en) | Cost-efficient scene character detection method and system | |
| CN115170414A (en) | Knowledge distillation-based single image rain removing method and system | |
| CN114943741A (en) | Visual SLAM method based on target detection and geometric probability in dynamic scene | |
| CN111815658B (en) | Image recognition method and device | |
| CN112613525A (en) | Target frame prediction method, device, equipment and medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant |