CN112669298A - Foundation cloud image cloud detection method based on model self-training - Google Patents
Foundation cloud image cloud detection method based on model self-training Download PDFInfo
- Publication number
- CN112669298A CN112669298A CN202011637512.0A CN202011637512A CN112669298A CN 112669298 A CN112669298 A CN 112669298A CN 202011637512 A CN202011637512 A CN 202011637512A CN 112669298 A CN112669298 A CN 112669298A
- Authority
- CN
- China
- Prior art keywords
- training
- sample
- super
- classification model
- pixel
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000012549 training Methods 0.000 title claims abstract description 157
- 238000001514 detection method Methods 0.000 title claims abstract description 22
- 238000013145 classification model Methods 0.000 claims abstract description 126
- 238000000034 method Methods 0.000 claims abstract description 53
- 230000011218 segmentation Effects 0.000 claims description 22
- 238000000605 extraction Methods 0.000 claims description 18
- 238000012163 sequencing technique Methods 0.000 claims description 3
- 125000004122 cyclic group Chemical group 0.000 claims description 2
- 230000008569 process Effects 0.000 description 8
- 238000010586 diagram Methods 0.000 description 7
- 238000005259 measurement Methods 0.000 description 7
- 238000004422 calculation algorithm Methods 0.000 description 5
- 238000004891 communication Methods 0.000 description 5
- 238000013528 artificial neural network Methods 0.000 description 4
- 238000012545 processing Methods 0.000 description 4
- 238000012706 support-vector machine Methods 0.000 description 4
- 238000004364 calculation method Methods 0.000 description 3
- 230000000694 effects Effects 0.000 description 3
- 238000002372 labelling Methods 0.000 description 3
- 230000000007 visual effect Effects 0.000 description 3
- 230000008859 change Effects 0.000 description 2
- 238000013527 convolutional neural network Methods 0.000 description 2
- 238000013136 deep learning model Methods 0.000 description 2
- 238000003709 image segmentation Methods 0.000 description 2
- 238000003384 imaging method Methods 0.000 description 2
- 238000010606 normalization Methods 0.000 description 2
- 238000007637 random forest analysis Methods 0.000 description 2
- 238000012512 characterization method Methods 0.000 description 1
- 238000013500 data storage Methods 0.000 description 1
- 238000011156 evaluation Methods 0.000 description 1
- 230000003993 interaction Effects 0.000 description 1
- 238000010801 machine learning Methods 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 238000005457 optimization Methods 0.000 description 1
Images
Landscapes
- Image Analysis (AREA)
Abstract
The invention discloses a model self-training-based foundation cloud image cloud detection method, which particularly discloses a method for determining a labeled superpixel sample and an unlabeled superpixel sample by obtaining a foundation cloud sample image, determining a training sample set according to the labeled superpixel sample, training a preset classifier model according to the training sample set to obtain a classification model, circularly selecting part of the unlabeled superpixel sample to update to the training sample set, repeating iterative training on the classification model, determining that the unlabeled superpixel sample is completely updated to the training sample set when the trained classification model is detected, and determining the trained classification model as a target classification model.
Description
Technical Field
The invention relates to the technical field of image processing, in particular to a foundation cloud image cloud detection method based on model self-training.
Background
Image processing is a technique that uses a computer to analyze an image to achieve a desired result. At present, the foundation cloud image cloud detection method mostly adopts an unsupervised learning image segmentation algorithm and a cloud detection method based on supervised learning, and converts the foundation cloud image segmentation problem into the two classification problems of image local areas. The cloud detection method based on supervised learning is mainly divided into two types, one type is based on a traditional machine learning model, for example, a foundation cloud image is divided into a plurality of small parts (generally called as superpixels), then the characteristics of the small parts are extracted, the superpixels are used as sample data or objects of classification prediction, the model is trained through the samples, and finally the ground cloud image to be analyzed is subjected to local two-classification by the trained model; the other type is based on a deep learning model, for example, a complex convolutional neural network is directly designed, the convolutional neural network is trained through a large amount of labeled data, and the network model directly analyzes a predicted image to obtain a result after detection and segmentation.
However, although the unsupervised learning segmentation algorithm is fast, the unsupervised learning segmentation algorithm has poor adaptability to the imaging environment and the imaging quality of the image. In the actual process of observing the foundation cloud, the visual difference of the foundation cloud image caused by various reasons such as time, geographic position, climate environment and the like is very complex, the characteristics used by an unsupervised segmentation algorithm are often simpler, the characterization capability is limited, the robustness is weak, and therefore the result is extremely unstable; the supervised learning method generally needs a large number of labeled samples, and particularly based on a deep learning model, because of huge model parameter quantity, the requirement on the number of samples is very high, and in the actual process, the labeling of the samples is very time-consuming and labor-consuming and has high cost. Meanwhile, as the climatic characteristics, air quality and other environments of different geographic positions are different and can change along with the time and the years, when the external conditions change, in order to better predict the effect of the model, retraining by using local or current real observation images is almost unavoidable, and the marked demand brings great cost and obstruction to the optimization training work of the model.
The above is only for the purpose of assisting understanding of the technical aspects of the present invention, and does not represent an admission that the above is prior art.
Disclosure of Invention
The invention mainly aims to provide a foundation cloud image cloud detection method based on model self-training, and aims to provide a foundation cloud image cloud detection method which is strong in robustness and adaptability and low in requirement on an annotation sample.
In order to achieve the above object, the present invention provides a training method for a ground-based cloud image classification model, comprising:
acquiring a foundation cloud sample image, and determining a marked super-pixel sample and an unmarked super-pixel sample;
determining a training sample set according to the marked super-pixel samples;
training a preset classifier model according to the training sample set to obtain a classification model;
circularly selecting part of the unlabeled super-pixel samples to update to the training sample set, and repeating iterative training on the classification model;
and when the unmarked super-pixel samples are detected to be completely updated to the training sample set, determining the trained classification model as a target classification model.
Preferably, after the step of training a preset classifier model according to the training sample set to obtain a classification model, before the step of cyclically selecting part of the unlabeled super-pixel samples to update to the training sample set and repeating iterative training on the classification model, the method further includes:
determining a pseudo label of the unlabeled super-pixel sample and a confidence coefficient thereof;
correspondingly, the step of circularly selecting part of the unlabeled super-pixel samples to update to the training sample set, and repeating iterative training on the classification model specifically includes:
and circularly selecting part of the unlabeled super-pixel samples to update to the training sample set according to the pseudo labels and the confidence degrees of the unlabeled super-pixel samples, and repeating iterative training on the classification model.
Preferably, the step of determining the pseudo label of the unlabeled super-pixel sample and the confidence thereof includes:
calculating a red-blue ratio Otsu threshold value of the foundation cloud sample image where the unmarked super-pixel sample is located;
predicting the unmarked super-pixel sample by using a red-blue ratio Otsu threshold value to obtain a predicted value St;
according to the classification model, performing class prediction on the unmarked superpixel sample to obtain a class prediction value Sc;
comparing the category predicted value Sc with the predicted value St;
and determining the pseudo label of the unlabeled super-pixel sample and the confidence thereof according to the comparison result.
Preferably, the step of circularly selecting a part of the unlabeled super-pixel samples to update to the training sample set according to the pseudo labels and the confidence degrees of the unlabeled super-pixel samples, and repeating iterative training on the classification model specifically includes:
sequencing the unmarked super-pixel samples according to the confidence degrees corresponding to the pseudo labels of the unmarked super-pixel samples;
sequentially extracting the unmarked superpixel samples with a preset proportion according to the sorted unmarked superpixel samples and updating the unmarked superpixel samples to the training sample set;
and repeating iterative training on the classification model according to the training sample set.
Preferably, the step of obtaining the ground-based cloud sample image and determining the annotated superpixel sample and the unlabeled superpixel sample comprises:
acquiring a foundation cloud sample image and performing super-pixel segmentation, wherein the foundation cloud sample image comprises a completely-labeled sample image, an unlabeled sample image and an incompletely-labeled sample image;
extracting the superpixel characteristics of the ground cloud sample image after superpixel segmentation;
determining the marked superpixel sample according to the completely marked sample image and the incompletely marked sample image after the superpixel feature extraction;
and determining that the residual ground cloud sample images after the extraction of the superpixel features are unmarked superpixel samples according to the marked superpixel samples.
Preferably, the step of determining the labeled superpixel sample according to the completely labeled sample image and the incompletely labeled sample image after the superpixel feature extraction includes:
according to the completely marked sample image and the incompletely marked sample image after extraction of the super-pixel characteristics, the marked pixel number in the pixels contained in the super-pixel accounts for the proportion of the total number of the pixels of the super-pixel;
and if the proportion is larger than a preset threshold value, determining the super pixel as a marked super pixel sample.
In order to achieve the purpose, the invention provides a model self-training-based foundation cloud image cloud detection method, which comprises the following steps:
acquiring a to-be-predicted image;
inputting the image to be predicted into a target classification model for image classification;
forming a cloud detection result graph according to the classification result;
the target classification model is obtained by training the foundation cloud image classification model according to any one of claims 1 to 6.
In order to achieve the above object, the present invention provides a training device for a ground-based cloud image classification model, comprising:
the acquisition sample unit is used for acquiring a foundation cloud sample image and determining a marked super-pixel sample and an unmarked super-pixel sample;
the determining sample unit is used for determining a training sample set according to the marked super-pixel sample;
the initial model unit is used for training a preset classifier model according to the training sample set to obtain a classification model;
the cyclic iteration unit is used for circularly selecting part of the unlabeled super-pixel samples to update to the training sample set and repeating iterative training on the classification model;
and the target determining unit is used for determining that the trained classification model is the target classification model when detecting that the unlabeled super-pixel samples are all updated to the training sample set.
In order to achieve the above object, the present invention provides a terminal, including: the ground-based cloud image classification model training program is configured to implement the steps of the ground-based cloud image classification model training method.
In order to achieve the above object, the present invention further provides a storage medium, wherein the storage medium stores a training program of a ground-based cloud image classification model, and the training program of the ground-based cloud image classification model, when executed by a processor, implements the steps of the above training method of the ground-based cloud image classification model.
According to the method, a foundation cloud sample image is obtained, a marked super-pixel sample and an unmarked super-pixel sample are determined, a training sample set is determined according to the marked super-pixel sample, a preset classifier model is trained according to the training sample set to obtain a classification model, a part of the unmarked super-pixel sample is selected in a circulating mode to be updated to the training sample set, iterative training is repeated on the classification model, the unmarked super-pixel sample is detected to be updated to the training sample set, and the trained classification model is determined to be a target classification model.
Drawings
Fig. 1 is a schematic structural diagram of a terminal in a hardware operating environment according to an embodiment of the present invention;
FIG. 2 is a flowchart illustrating a first embodiment of a method for training a ground-based cloud image classification model according to the present invention;
FIG. 3 is a flowchart illustrating a second embodiment of the method for training a ground-based cloud image classification model according to the present invention;
FIG. 4 is a flowchart illustrating a third embodiment of the training method for a ground-based cloud image classification model according to the present invention;
FIG. 5 is a flowchart illustrating a fourth embodiment of the training method for a ground-based cloud image classification model according to the present invention;
FIG. 6 is a flowchart illustrating a fifth embodiment of the training method for a ground-based cloud image classification model according to the present invention;
fig. 7 is a schematic flowchart of a first embodiment of the model self-training-based ground-based cloud image cloud detection method according to the present invention.
FIG. 8 is a block diagram of a training apparatus for a ground-based cloud image classification model according to a first embodiment of the present invention;
FIG. 9a is a fully labeled sample image according to an embodiment of the present invention;
FIG. 9b is a schematic diagram of an incompletely annotated sample image according to an embodiment of the present invention;
FIG. 9c is an unlabeled sample image according to an embodiment of the present invention;
FIG. 10a is a ground based cloud sample image according to an embodiment of the present invention;
FIG. 10b is a ground based cloud sample image of FIG. 10a after superpixel segmentation;
fig. 11 is a schematic diagram of an embodiment of a model self-training-based ground-based cloud image cloud detection method provided by the present invention.
The implementation, functional features and advantages of the objects of the present invention will be further explained with reference to the accompanying drawings.
Detailed Description
It should be understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention.
Referring to fig. 1, fig. 1 is a schematic terminal structure diagram of a hardware operating environment according to an embodiment of the present invention.
As shown in fig. 1, the terminal may include: a processor 1001, such as a Central Processing Unit (CPU), a communication bus 1002, a user interface 1003, a network interface 1004, and a memory 1005. Wherein a communication bus 1002 is used to enable connective communication between these components. The user interface 1003 may include a Display screen (Display), an input module such as a Keyboard (Keyboard), and the optional user interface 1003 may also include a standard wired interface, a wireless interface. The network interface 1004 may optionally include a standard wired interface, a WIreless interface (e.g., a WIreless-FIdelity (WI-FI) interface). The Memory 1005 may be a Random Access Memory (RAM) Memory, or may be a Non-Volatile Memory (NVM), such as a disk Memory. The memory 1005 may alternatively be a storage device separate from the processor 1001.
Those skilled in the art will appreciate that the configuration shown in fig. 1 is not intended to be limiting and may include more or fewer components than those shown, or some components may be combined, or a different arrangement of components.
As shown in fig. 1, a memory 1005, which is a storage medium, may include therein a training program of an operating system, a data storage module, a network communication module, a user interface module, and a ground-based cloud image classification model.
In the terminal shown in fig. 1, the network interface 1004 is mainly used for data communication with a network server; the user interface 1003 is mainly used for data interaction with a user; the processor 1001 and the memory 1005 in the terminal of the present invention may be disposed in the terminal, and the terminal invokes a training program of the ground-based cloud image classification model stored in the memory 1005 through the processor 1001 and executes the training method of the ground-based cloud image classification model provided in the embodiment of the present invention.
The embodiment of the invention provides a training method of a ground-based cloud image classification model, and referring to fig. 2, fig. 2 is a schematic flow diagram of a first embodiment of the training method of the ground-based cloud image classification model.
In this embodiment, the training method for the ground-based cloud image classification model includes the following steps:
step S10: acquiring a foundation cloud sample image, and determining a marked super-pixel sample and an unmarked super-pixel sample;
it should be understood that the execution subject is a terminal, and the ground-based cloud sample image includes a fully-labeled sample image (e.g., fig. 9a), an unlabeled sample image (e.g., fig. 9c), and an incompletely-labeled sample image (e.g., fig. 9 b). In the invention, the number of unlabeled sample images can be large, and the number of fully labeled sample images can be small, in other words, the demand of the invention for the number of labeled samples is not large.
Step S20: determining a training sample set according to the marked super-pixel samples;
it should be understood that the determined labeled superpixel samples are used to determine a training sample set for training a preset classification model.
Step S30: training a preset classifier model according to the training sample set to obtain a classification model;
it should be understood that the pre-set classifier may be a conventional classifier model, such as a Support Vector Machine (SVM), an Artificial Neural Network (ANN), a random forest decision (RF), etc. And the initialization of the classifier model can also be set by referring to a common mode, a preset classifier model is trained by taking the marked super-pixel sample in the training sample set as a training sample, and the trained model is taken as a classification model.
Step S40: circularly selecting part of the unlabeled super-pixel samples to update to the training sample set, and repeating iterative training on the classification model;
it should be understood that, the circularly selecting part of the unlabeled super-pixel samples to update to the training sample set means that circularly selecting part of the unlabeled super-pixel samples to update to the training sample set, where the unlabeled super-pixel samples are not repeatedly selected (in other words, selected samples are not selected again). And after selecting part of the remaining unmarked superpixel samples each time and updating the part of the remaining unmarked superpixel samples to the training sample set, iteratively training the classification model, and repeating the steps until all the samples are selected. The rule of selection may be according to a certain rule, or may be in a self-defined manner, for example, 5% of the remaining unlabeled superpixel samples are selected each time and updated to the training sample set.
Step S50: and when the unmarked super-pixel samples are detected to be completely updated to the training sample set, determining the trained classification model as a target classification model.
It should be understood that after all the unlabeled super-pixel samples are updated to the training sample set, the trained classification model is the target classification model.
According to the method, a foundation cloud sample image is obtained, a marked super-pixel sample and an unmarked super-pixel sample are determined, a training sample set is determined according to the marked super-pixel sample, a preset classifier model is trained according to the training sample set to obtain a classification model, a part of the unmarked super-pixel sample is selected in a circulating mode to be updated to the training sample set, iterative training is repeated on the classification model, the unmarked super-pixel sample is detected to be updated to the training sample set, and the trained classification model is determined to be a target classification model.
Referring to fig. 3, fig. 3 is a flowchart illustrating a second embodiment of the training method for a ground-based cloud image classification model according to the present invention.
Based on the first embodiment, in the present embodiment, after the step S30 and before the step S40, the method further includes:
step S60: determining a pseudo label of the unlabeled super-pixel sample and a confidence coefficient thereof;
correspondingly, the step of step S40 specifically includes:
step S401: and circularly selecting part of the unlabeled super-pixel samples to update to the training sample set according to the pseudo labels and the confidence degrees of the unlabeled super-pixel samples, and repeating iterative training on the classification model.
Referring to fig. 4, fig. 4 is a flowchart illustrating a third embodiment of the training method for a ground-based cloud image classification model according to the present invention.
Based on the second embodiment described above, in the present embodiment, the step S60 includes:
step S601: calculating a red-blue ratio Otsu threshold value of the foundation cloud sample image where the unmarked super-pixel sample is located;
in a specific implementation, the step S601 includes:
(1) converting the foundation cloud sample image where each unmarked super pixel sample is located into a red-blue ratio gray scale map Irb (x, y) ═ Ir (x, y)/Ib (x, y);
wherein Ir (x, y) and Ib (x, y) are components of image I (x, y) on the R channel and the B channel, respectively.
(2) Counting all pixel values in Irb (x, y) and carrying out normalization processing;
wherein, the normalization process is to normalize all pixel values in Irb (x, y) to be within the interval of 0-255.
(3) Assuming that the Otsu threshold in the image Irb (x, y) is t, the inter-class distance variance of the image foreground (cloud) and background (sky) can be expressed as:
G(t)＝ω1(μ1-μ)2+ω2(μ2-μ)2
wherein, ω is1，ω2Respectively, represent the foreground (Irb (x, y)>t, representing cloud) and background pixels (Irb (x, y)<t, representing sky) in the image, μ1，μ2The average Irb values of the foreground and background are represented, respectively, and μ represents the average Irb value of the entire image.
(4) Traversing the integer value of t ═ 0, 255, the value of t that makes g (t) the maximum is selected as the threshold.
Step S602: predicting the unmarked super-pixel sample by using a red-blue ratio Otsu threshold value to obtain a predicted value St;
in a specific implementation, the step S602 includes:
calculating a difference value Drb between the Irb average value of all pixel points contained in each unmarked superpixel sample and the red-blue ratio Otsu threshold t of the foundation cloud sample image in which the unmarked superpixel sample is located, wherein the difference value is Irb-t; let Drb/255 be the Otsu threshold evaluation score St of the unlabeled superpixel sample (i.e. the Otsu threshold gives the predicted value of the unlabeled superpixel sample), and in this embodiment, St takes a value within the range of [ -1,1] (for example, St >0 indicates that the prediction category is "cloud", and St <0 indicates that the prediction category is "sky").
Step S603: according to the classification model, performing class prediction on the unmarked superpixel sample to obtain a class prediction value Sc;
in specific implementation, the unlabeled super-pixel sample is classified by a classification model, and a classification score value Sc (i.e., a class prediction value Sc, a probability value belonging to a certain class given by the classification model) is calculated, in this embodiment, the Sc value is within the range of [ -1,1] (for example, Sc >0 indicates that the prediction class is "cloud", and Sc <0 indicates that the prediction class is "sky").
Step S604: comparing the category predicted value Sc with the predicted value St;
step S605: and determining the pseudo label of the unlabeled super-pixel sample and the confidence thereof according to the comparison result.
In a specific implementation, the step S605 includes:
1) if the classification model prediction class Sc is the same as the pseudo label (cloud or sky) represented by the threshold prediction class St, the class is determined to be the pseudo label of the prediction sample, the comprehensive confidence coefficient is S | (1- λ) Sc + λ St |, and λ is 0-1.
2) If the classification model prediction category Sc is different from the pseudo label (cloud or sky) represented by the threshold prediction category St, and Sc is greater than or equal to St, the prediction result predicted by the classification model is used as the pseudo label of the unlabeled super-pixel sample, and the comprehensive confidence is S | (1- λ) Sc + λ St |, λ is 0-1.
3) If the classification model prediction class Sc is different from the pseudo label (cloud or sky) represented by the threshold prediction class St, and Sc is smaller than St, the prediction result of the Otsu threshold is used as the pseudo label of the unlabeled super-pixel sample, the comprehensive confidence is S | (1- λ) Sc + λ St |, and λ is 0-1.
Referring to fig. 5, fig. 5 is a schematic flowchart of a fourth embodiment of the training method for a ground-based cloud image classification model according to the present invention.
Based on the third embodiment, in this embodiment, the step S401 includes:
step S4011: sequencing the unmarked super-pixel samples according to the confidence degrees corresponding to the pseudo labels of the unmarked super-pixel samples;
in specific implementation, the unlabeled super-pixel samples may be sorted from the greater confidence level to the lesser confidence level according to the confidence level corresponding to the pseudo-label of the unlabeled super-pixel sample.
Step S4012: sequentially extracting the unmarked superpixel samples with a preset proportion according to the sorted unmarked superpixel samples and updating the unmarked superpixel samples to the training sample set;
in a specific implementation, the unlabeled super-pixel samples with the preset proportion are extracted and updated to the training sample set every time according to the confidence coefficient from large to small, in this embodiment, the preset proportion is 5%, that is, the unlabeled super-pixel samples with the confidence coefficient from small to first 5% are extracted and merged to the training sample set.
It should be noted that after each extraction, the remaining unlabeled super-pixel samples do not include the extracted unlabeled super-pixel sample, and each extraction of the unlabeled super-pixel sample with the preset proportion means that the sample with the preset proportion in the remaining unlabeled super-pixel samples is extracted.
Step S4013: and repeating iterative training on the classification model according to the training sample set.
Referring to fig. 6, fig. 6 is a schematic flowchart of a fifth embodiment of the training method for a ground-based cloud image classification model according to the present invention.
Based on the first embodiment described above, in the present embodiment, the step S10 includes:
step S101: acquiring a foundation cloud sample image and performing super-pixel segmentation, wherein the foundation cloud sample image comprises a completely-labeled sample image, an unlabeled sample image and an incompletely-labeled sample image;
it should be understood that the labeling of the ground-based cloud sample image is the labeling of pixel points on the ground-based cloud image. The fully labeled sample image refers to the fact that all pixel points in the sample image are labeled, the incompletely labeled sample image refers to the fact that a part of the pixel points in the sample image are labeled, and the unlabeled sample image refers to the fact that the pixel points in the sample image are not labeled.
The super-pixel segmentation is to segment an image into a plurality of small regions, i.e. super-pixels, by using color information of image pixels. In this embodiment, the super-pixel segmentation method adopts a SLIC (simple linear iterative cluster) super-pixel segmentation algorithm, which converts a color image into a 5-dimensional feature vector in a CIELAB color space and XY coordinates, then constructs a distance metric standard for the 5-dimensional feature vector, and performs local clustering on image pixels to finally obtain a super-pixel segmentation result, and specifically includes the following steps:
(1) initializing a cluster center: setting a predetermined number K of superpixels (in this embodiment, different K values are set according to the size of the input image, where K is N/100 and an integer part is taken), the size of each superpixel (including the number of pixel points) is N/K, and then selecting a seed point in the image as an initial clustering center with a step size of S being sqrt (N/K).
(2) Fine adjustment of seed points: in order to avoid that the seed points fall at the edges in the image and the super-pixel segmentation result has a certain edge-preserving effect, the gradient values of all pixels are calculated in the neighborhood of each seed point n x n (in the embodiment, n takes 3 or 5, preferably, n takes 5), and the seed point is reset to the point with the minimum gradient in the neighborhood. The gradient G (x, y) of the pixel points in the image is calculated as follows:
wherein, I (x, y) is the pixel gray scale value of the coordinate point (x, y).
(3) Pixel clustering: and (3) allocating a label to each pixel point, namely assigning the label to the seed point with the closest distance measurement, wherein the used distance measurement needs to simultaneously consider the spatial distance and the color characteristic distance between the pixel and the seed point. The specific implementation method is that the seed points (a plurality of seed points are generally included because the selection step length of the seed points is S) are found in the 2S-2S field of each pixel point i, the distance measurement D (i, j) between the pixel point and each seed point is respectively calculated, the j with the minimum distance measurement D is taken, the pixel point is distributed to the cluster indicated by the seed point j, and the used distance measurement is calculated as follows:
wherein l, a and b are pixel values of three channels in an LAB color space of the pixel point;
i is a pixel point;
j is a seed point;
x and y are coordinate positions of the pixel points in the image;
Nsis the maximum spatial distance within the class, defined as Ns＝S＝sqrt(N/K)；
NcThe maximum distance of the color is taken as constant 40 in this example.
(4) Repositioning seed points: and (4) after assigning distribution points for all pixels in the image according to the step (3), re-determining the central point of each cluster according to all pixel points contained in each cluster. And (3) calculating the distance measurement D (i, j) between every two pixel points in the cluster, and taking the distance measurement to other points in the cluster and the minimum point as a new cluster center of the cluster.
(5) And (3) taking the new clustering center as the seed point in the step (2), and repeating the iteration steps (2), (3) and (4) until all clustering center points are not changed any more. In the actual implementation process of the method, the step is simplified into 10 fixed iterations, and the desired effect can be generally achieved.
(6) And finally, obtaining a segmentation graph, wherein the segmentation graph is consistent with the original input image in size and only has one channel, and the value of each pixel point is a given clustering label. And the corresponding area in the original image of the pixel point with the same label is the super pixel. The original location base cloud image is shown in FIG. 10a and the result obtained after superpixel segmentation is shown in FIG. 10 b.
Step S102: extracting the superpixel characteristics of the ground cloud sample image after superpixel segmentation;
in a specific implementation, in step S101, the ground cloud sample image is subjected to superpixel feature extraction, which is to extract visual features in this embodiment. The visual features may include color features, texture features inside the superpixel, texture features in the superpixel domain, superpixel position weighting, and the like, specifically as follows:
(1) color feature extraction
In the embodiment, three types of RBG color space, Lab color space and opponent color space are selected for selecting the color space, the color pixel value of each color space comprises 3 channels, the average value and the variance are respectively calculated, and 18-dimensional eigenvectors of the super-pixel related to the color are obtained in the aspect of color.
(2) Superpixel internal texture feature extraction
And for the texture features inside the super pixels, performing LBP coded statistical histogram calculation by using a rotation-invariant LBP descriptor. The principle of calculation of rotation invariant LBP coding is: for each pixel, 8 adjacent pixels are compared with the pixel, the pixel is larger than the pixel and is marked as 1, the pixel is smaller than the pixel and is marked as 0, the 80 or 1 are combined into an 8-bit binary number according to the clockwise sequence from the upper left, then the binary number is shifted to ensure that the corresponding decimal number is the minimum, and the decimal number is the LBP code of the pixel.
After obtaining the LBP codes of all pixel points in the super-pixel, the LBP codes totally contain 64 different decimal numbers, and the number of the super-pixels of each code value in the super-pixel is counted to form a 64-dimensional histogram feature vector for expressing the internal texture feature of the super-pixel.
(3) Superpixel domain texture feature extraction
In order to express the relationship between the superpixels and obtain the neighborhood texture features of the superpixels, in this embodiment, the 128-dimensional SIFT descriptor feature of the center point of the superpixel is used as the neighborhood texture feature of the superpixel. The specific implementation steps for extracting the SIFT descriptor feature of a certain fixed point in an image adopted in the invention are as follows:
calculating 8-direction gradients of all points in a super pixel central point 3S-3S neighborhood;
counting 8-direction gradient value histograms in the field and positioning the direction with the largest accumulated sum in the main direction;
dividing the field into 4 × 4 subregions, and calculating 8-direction gradient histograms of each subregion with the main direction as the starting direction;
the gradient histograms of all sub-regions are concatenated into a feature vector of 4 x 8-128 dimensions.
After the super-pixel features of the foundation cloud sample image are extracted, the extracted features are sequentially connected in series, and the comprehensive features of the super-pixels can be obtained. In the present embodiment, the total color feature of the super-pixel (16 d in the present embodiment) + the inner texture feature of the super-pixel (64 d in the present embodiment) + the texture feature of the super-pixel region (128 d in the present embodiment) can be obtained through the above calculation, that is, 18+64+128 is 210 d.
Compared with the prior art, the method adopts a superpixel comprehensive feature extraction mode, so that the extracted foundation cloud image has more comprehensive local features and stronger representation capability.
Step S103: determining the marked superpixel sample according to the completely marked sample image and the incompletely marked sample image after the superpixel feature extraction;
in a specific implementation, the step S103 includes:
step S1031: determining the proportion of the number of pixels marked in the pixels contained in the super pixels to the total number of the pixels of the super pixels according to the fully marked sample image and the incompletely marked sample image after the super pixel characteristics are extracted;
it should be understood that the step of determining the marked superpixel sample is to perform superpixel segmentation on the foundation cloud sample image, and on the basis of the result of the superpixel segmentation, the superpixels in the completely marked sample image and the incompletely marked sample image are judged.
Step S1032: and if the proportion is larger than a preset threshold value, determining the super pixel as a marked super pixel sample.
In this embodiment, the preset threshold is 1/2, and the number of pixels labeled in the pixels included in the superpixel in the fully labeled sample image and the incompletely labeled sample image after the superpixel feature extraction exceeds half of the number of pixels of the superpixel, which is considered to determine that the superpixel is the labeled superpixel sample.
Step S104: and determining that the residual ground cloud sample images after the extraction of the superpixel features are unmarked superpixel samples according to the marked superpixel samples.
It should be understood that, except for the superpixels that are confirmed to be the labeled superpixels in the ground-based cloud sample image after the superpixel feature extraction, the remaining superpixels are the unlabeled superpixels.
Referring to fig. 7, fig. 7 is a flowchart illustrating a method for cloud detection based on model self-training ground-based cloud images according to a first embodiment of the present invention.
In this embodiment, the model self-training based ground-based cloud image cloud detection method includes the following steps:
step S701: acquiring a to-be-predicted image;
it should be understood that the executing body is a terminal.
Step S702: inputting the image to be predicted into a target classification model for image classification;
it should be understood that the target classification model is a target classification model obtained by training through the above-mentioned training method of the ground-based cloud image classification model.
Step S703: forming a cloud detection result graph according to the classification result;
in specific implementation, please refer to fig. 11, where fig. 11 shows a flow of a model-based self-training ground cloud image cloud detection method, after a to-be-predicted image is input into a target classification model, superpixel segmentation is performed first, superpixels are classified to obtain superpixel types, a specific label is given to a corresponding pixel or region according to a position of the superpixel in an original image, and then a cloud detection result on the rightmost side of fig. 11 is obtained, and the cloud detection result is marked as a cloud pixel percentage to obtain cloud coverage.
Referring to fig. 8, fig. 8 is a block diagram illustrating a first embodiment of a training apparatus for a ground-based cloud image classification model according to the present invention.
As shown in fig. 8, the training apparatus for a ground-based cloud image classification model according to an embodiment of the present invention includes:
an acquiring sample unit 801, configured to acquire a foundation cloud sample image, and determine a super-pixel sample that has been labeled and a super-pixel sample that has not been labeled;
it should be understood that the ground-based cloud sample image includes a fully-annotated sample image (e.g., fig. 9a), an unlabeled sample image (e.g., fig. 9c), and an incompletely-annotated sample image (e.g., fig. 9 b). In the invention, the number of the unlabeled sample images can be large, and the number of the completely labeled sample images can be small, in other words, the required amount of the invention for the number of the labeled samples is not large.
A determining sample unit 802, configured to determine a training sample set according to the labeled superpixel sample;
it should be understood that the determined labeled superpixel samples are used to determine a training sample set for training a preset classification model.
An initial model unit 803, configured to train a preset classifier model according to the training sample set, so as to obtain a classification model;
it should be understood that the pre-set classifier may be a conventional classifier model, such as a Support Vector Machine (SVM), an Artificial Neural Network (ANN), a random forest decision (RF), etc. And the initialization of the classifier model can also be set by referring to a common mode, a preset classifier model is trained by taking the marked super-pixel sample in the training sample set as a training sample, and the trained model is taken as a classification model.
A loop iteration unit 804, configured to select part of the unlabeled super-pixel samples in a loop manner to update to the training sample set, and repeat iteration training on the classification model;
it should be understood that, the circularly selecting part of the unlabeled super-pixel samples to update to the training sample set means that circularly selecting part of the unlabeled super-pixel samples to update to the training sample set, where the unlabeled super-pixel samples are not repeatedly selected (in other words, selected samples are not selected again). And after selecting part of the remaining unmarked superpixel samples each time and updating the part of the remaining unmarked superpixel samples to the training sample set, iteratively training the classification model, and repeating the steps until all the samples are selected. The rule of selection may be according to a certain rule, or may be in a self-defined manner, for example, 5% of the remaining unlabeled superpixel samples are selected each time and updated to the training sample set.
And a target determining unit 805, configured to determine that the trained classification model is the target classification model when it is detected that all the unlabeled super-pixel samples are updated to the training sample set.
It should be understood that after all the unlabeled super-pixel samples are updated to the training sample set, the trained classification model is the target classification model.
According to the method, a foundation cloud sample image is obtained, a marked super-pixel sample and an unmarked super-pixel sample are determined, a training sample set is determined according to the marked super-pixel sample, a preset classifier model is trained according to the training sample set to obtain a classification model, a part of the unmarked super-pixel sample is selected in a circulating mode to be updated to the training sample set, iterative training is repeated on the classification model, the unmarked super-pixel sample is detected to be updated to the training sample set, and the trained classification model is determined to be a target classification model.
In addition, an embodiment of the present invention further provides a storage medium, where a training program of a ground-based cloud image classification model is stored, and when executed by a processor, the method for training the ground-based cloud image classification model includes the above steps.
Other embodiments or specific implementation manners of the training device for the ground-based cloud image classification model of the present invention may refer to the above method embodiments, and are not described herein again.
It should be noted that, in this document, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or system that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or system. Without further limitation, an element defined by the phrase "comprising an … …" does not exclude the presence of other like elements in a process, method, article, or system that comprises the element.
The above-mentioned serial numbers of the embodiments of the present invention are merely for description and do not represent the merits of the embodiments.
Through the above description of the embodiments, those skilled in the art will clearly understand that the method of the above embodiments can be implemented by software plus a necessary general hardware platform, and certainly can also be implemented by hardware, but in many cases, the former is a better implementation manner. Based on such understanding, the technical solutions of the present invention may be embodied in the form of a software product, which is stored in a storage medium (e.g., a rom/ram, a magnetic disk, an optical disk) and includes instructions for enabling a terminal device (e.g., a mobile phone, a computer, a server, an air conditioner, or a network device) to execute the method according to the embodiments of the present invention.
The above description is only a preferred embodiment of the present invention, and not intended to limit the scope of the present invention, and all modifications of equivalent structures and equivalent processes, which are made by using the contents of the present specification and the accompanying drawings, or directly or indirectly applied to other related technical fields, are included in the scope of the present invention.
Claims (10)
1. A training method of a ground cloud image classification model is characterized by comprising the following steps:
acquiring a foundation cloud sample image, and determining a marked super-pixel sample and an unmarked super-pixel sample;
determining a training sample set according to the marked super-pixel samples;
training a preset classifier model according to the training sample set to obtain a classification model;
circularly selecting part of the unlabeled super-pixel samples to update to the training sample set, and repeating iterative training on the classification model;
and when the unmarked super-pixel samples are detected to be completely updated to the training sample set, determining the trained classification model as a target classification model.
2. The method for training a ground-based cloud image classification model according to claim 1, wherein after the step of training a preset classifier model according to the training sample set to obtain a classification model, the step of cyclically selecting a part of the unlabeled superpixel samples to update to the training sample set and before the step of repeating iterative training on the classification model further comprises:
determining a pseudo label of the unlabeled super-pixel sample and a confidence coefficient thereof;
correspondingly, the step of circularly selecting part of the unlabeled super-pixel samples to update to the training sample set, and repeating iterative training on the classification model specifically includes:
and circularly selecting part of the unlabeled super-pixel samples to update to the training sample set according to the pseudo labels and the confidence degrees of the unlabeled super-pixel samples, and repeating iterative training on the classification model.
3. The method for training a ground-based cloud image classification model according to claim 2, wherein the step of determining the pseudo label and the confidence level thereof of the unlabeled super-pixel sample comprises:
calculating a red-blue ratio Otsu threshold value of the foundation cloud sample image where the unmarked super-pixel sample is located;
predicting the unmarked super-pixel sample by using a red-blue ratio Otsu threshold value to obtain a predicted value St;
according to the classification model, performing class prediction on the unmarked superpixel sample to obtain a class prediction value Sc;
comparing the category predicted value Sc with the predicted value St;
and determining the pseudo label of the unlabeled super-pixel sample and the confidence thereof according to the comparison result.
4. The method for training the ground-based cloud image classification model according to claim 3, wherein the step of cyclically selecting a part of the unlabeled super-pixel samples to update to the training sample set according to the pseudo labels and the confidence degrees of the unlabeled super-pixel samples and repeating iterative training on the classification model specifically comprises:
sequencing the unmarked super-pixel samples according to the confidence degrees corresponding to the pseudo labels of the unmarked super-pixel samples;
sequentially extracting the unmarked superpixel samples with a preset proportion according to the sorted unmarked superpixel samples and updating the unmarked superpixel samples to the training sample set;
and repeating iterative training on the classification model according to the training sample set.
5. The method for training a ground-based cloud image classification model according to claim 1, wherein the step of obtaining an image of a ground-based cloud sample and determining the labeled superpixel sample and the unlabeled superpixel sample comprises:
acquiring a foundation cloud sample image and performing super-pixel segmentation, wherein the foundation cloud sample image comprises a completely-labeled sample image, an unlabeled sample image and an incompletely-labeled sample image;
extracting the superpixel characteristics of the ground cloud sample image after superpixel segmentation;
determining the marked superpixel sample according to the completely marked sample image and the incompletely marked sample image after the superpixel feature extraction;
and determining that the residual ground cloud sample images after the extraction of the superpixel features are unmarked superpixel samples according to the marked superpixel samples.
6. The method for training the ground-based cloud image classification model according to claim 5, wherein the step of determining the labeled superpixel sample according to the completely labeled sample image and the incompletely labeled sample image after the extraction of the superpixel features comprises:
determining the proportion of the number of pixels marked in the pixels contained in the super pixels to the total number of the pixels of the super pixels according to the fully marked sample image and the incompletely marked sample image after the super pixel characteristics are extracted;
and if the proportion is larger than a preset threshold value, determining the super pixel as a marked super pixel sample.
7. A foundation cloud image cloud detection method based on model self-training is characterized by comprising the following steps:
acquiring a to-be-predicted image;
inputting the image to be predicted into a target classification model for image classification;
forming a cloud detection result graph according to the classification result;
the target classification model is obtained by training the foundation cloud image classification model according to any one of claims 1 to 6.
8. The utility model provides a training device of ground cloud image classification model which characterized in that includes:
the acquisition sample unit is used for acquiring a foundation cloud sample image and determining a marked super-pixel sample and an unmarked super-pixel sample;
the determining sample unit is used for determining a training sample set according to the marked super-pixel sample;
the initial model unit is used for training a preset classifier model according to the training sample set to obtain a classification model;
the cyclic iteration unit is used for circularly selecting part of the unlabeled super-pixel samples to update to the training sample set and repeating iterative training on the classification model;
and the target determining unit is used for determining that the trained classification model is the target classification model when detecting that the unlabeled super-pixel samples are all updated to the training sample set.
9. A terminal, characterized in that the terminal comprises: a memory, a processor, and a training program of a ground-based cloud image classification model stored on the memory and executable on the processor, the training program of the ground-based cloud image classification model configured to implement the steps of the method of training of a ground-based cloud image classification model as claimed in any one of claims 1 to 6.
10. A storage medium having stored thereon a training program for ground-based cloud image classification models, the training program for ground-based cloud image classification models, when executed by a processor, implementing the steps of the method for training ground-based cloud image classification models according to any one of claims 1 to 6.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011637512.0A CN112669298A (en) | 2020-12-31 | 2020-12-31 | Foundation cloud image cloud detection method based on model self-training |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011637512.0A CN112669298A (en) | 2020-12-31 | 2020-12-31 | Foundation cloud image cloud detection method based on model self-training |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN112669298A true CN112669298A (en) | 2021-04-16 |
Family
ID=75413757
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202011637512.0A Pending CN112669298A (en) | 2020-12-31 | 2020-12-31 | Foundation cloud image cloud detection method based on model self-training |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN112669298A (en) |
Cited By (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113065609A (en) * | 2021-04-22 | 2021-07-02 | 平安国际智慧城市科技股份有限公司 | Image classification method and device, electronic equipment and readable storage medium |
| CN113792653A (en) * | 2021-09-13 | 2021-12-14 | 山东交通学院 | Method, system, equipment and storage medium for cloud detection of remote sensing image |
| CN114463603A (en) * | 2022-04-14 | 2022-05-10 | 浙江啄云智能科技有限公司 | Training method and device for image detection model, electronic equipment and storage medium |
| CN114612744A (en) * | 2022-03-10 | 2022-06-10 | 平安科技（深圳）有限公司 | Detection model training method, vehicle damage detection method and terminal equipment |
| CN116257800A (en) * | 2023-05-12 | 2023-06-13 | 智慧眼科技股份有限公司 | Labeling method and system for training samples |
| CN117152138A (en) * | 2023-10-30 | 2023-12-01 | 陕西惠宾电子科技有限公司 | Medical image tumor target detection method based on unsupervised learning |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103258214A (en) * | 2013-04-26 | 2013-08-21 | 南京信息工程大学 | Remote sensing image classification method based on image block active learning |
| CN105260738A (en) * | 2015-09-15 | 2016-01-20 | 武汉大学 | Method and system for detecting change of high-resolution remote sensing image based on active learning |
| CN108764281A (en) * | 2018-04-18 | 2018-11-06 | 华南理工大学 | A kind of image classification method learning across task depth network based on semi-supervised step certainly |
| US20190122120A1 (en) * | 2017-10-20 | 2019-04-25 | Dalei Wu | Self-training method and system for semi-supervised learning with generative adversarial networks |
-
2020
- 2020-12-31 CN CN202011637512.0A patent/CN112669298A/en active Pending
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103258214A (en) * | 2013-04-26 | 2013-08-21 | 南京信息工程大学 | Remote sensing image classification method based on image block active learning |
| CN105260738A (en) * | 2015-09-15 | 2016-01-20 | 武汉大学 | Method and system for detecting change of high-resolution remote sensing image based on active learning |
| US20190122120A1 (en) * | 2017-10-20 | 2019-04-25 | Dalei Wu | Self-training method and system for semi-supervised learning with generative adversarial networks |
| CN108764281A (en) * | 2018-04-18 | 2018-11-06 | 华南理工大学 | A kind of image classification method learning across task depth network based on semi-supervised step certainly |
Cited By (10)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113065609A (en) * | 2021-04-22 | 2021-07-02 | 平安国际智慧城市科技股份有限公司 | Image classification method and device, electronic equipment and readable storage medium |
| CN113065609B (en) * | 2021-04-22 | 2024-04-09 | 深圳赛安特技术服务有限公司 | Image classification method, device, electronic equipment and readable storage medium |
| CN113792653A (en) * | 2021-09-13 | 2021-12-14 | 山东交通学院 | Method, system, equipment and storage medium for cloud detection of remote sensing image |
| CN113792653B (en) * | 2021-09-13 | 2023-10-20 | 山东交通学院 | Method, system, equipment and storage medium for cloud detection of remote sensing image |
| CN114612744A (en) * | 2022-03-10 | 2022-06-10 | 平安科技（深圳）有限公司 | Detection model training method, vehicle damage detection method and terminal equipment |
| CN114463603A (en) * | 2022-04-14 | 2022-05-10 | 浙江啄云智能科技有限公司 | Training method and device for image detection model, electronic equipment and storage medium |
| CN116257800A (en) * | 2023-05-12 | 2023-06-13 | 智慧眼科技股份有限公司 | Labeling method and system for training samples |
| CN116257800B (en) * | 2023-05-12 | 2023-08-25 | 智慧眼科技股份有限公司 | Labeling method and system for training samples |
| CN117152138A (en) * | 2023-10-30 | 2023-12-01 | 陕西惠宾电子科技有限公司 | Medical image tumor target detection method based on unsupervised learning |
| CN117152138B (en) * | 2023-10-30 | 2024-01-16 | 陕西惠宾电子科技有限公司 | Medical image tumor target detection method based on unsupervised learning |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN112669298A (en) | Foundation cloud image cloud detection method based on model self-training | |
| Seeland et al. | Plant species classification using flower images—A comparative study of local feature representations | |
| Xie et al. | Multilevel cloud detection in remote sensing images based on deep learning | |
| CN107424159B (en) | Image semantic segmentation method based on super-pixel edge and full convolution network | |
| Arteta et al. | Interactive object counting | |
| EP2701098B1 (en) | Region refocusing for data-driven object localization | |
| CN109522908A (en) | Image significance detection method based on area label fusion | |
| Rouhani et al. | Semantic segmentation of 3D textured meshes for urban scene analysis | |
| US11586863B2 (en) | Image classification method and device | |
| CN106980856B (en) | Formula identification method and system and symbolic reasoning calculation method and system | |
| CN109343920B (en) | Image processing method and device, equipment and storage medium thereof | |
| CN111783779B (en) | Image processing method, apparatus and computer readable storage medium | |
| JP4376145B2 (en) | Image classification learning processing system and image identification processing system | |
| JP6897749B2 (en) | Learning methods, learning systems, and learning programs | |
| CN112288761B (en) | Abnormal heating power equipment detection method and device and readable storage medium | |
| US8867851B2 (en) | Sparse coding based superpixel representation using hierarchical codebook constructing and indexing | |
| Chen et al. | A semisupervised context-sensitive change detection technique via Gaussian process | |
| CN110738672A (en) | image segmentation method based on hierarchical high-order conditional random field | |
| CN112926601A (en) | Image recognition method, device and equipment based on deep learning and storage medium | |
| CN113963295A (en) | Method, device, equipment and storage medium for recognizing landmark in video clip | |
| CN112241736A (en) | Text detection method and device | |
| Saha et al. | Application of a multiseed-based clustering technique for automatic satellite image segmentation | |
| Park et al. | An evaluation of statistical methods in handwritten hangul recognition | |
| CN113297411A (en) | Method, device and equipment for measuring similarity of wheel-shaped atlas and storage medium | |
| Zhang et al. | Robust Image Segmentation Using LBP Embedded Region Merging. |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |