CN111860571B - Cloud microparticle classification method based on CIP data quality control - Google Patents
Cloud microparticle classification method based on CIP data quality control Download PDFInfo
- Publication number
- CN111860571B CN111860571B CN202010495758.2A CN202010495758A CN111860571B CN 111860571 B CN111860571 B CN 111860571B CN 202010495758 A CN202010495758 A CN 202010495758A CN 111860571 B CN111860571 B CN 111860571B
- Authority
- CN
- China
- Prior art keywords
- image
- cloud
- cip
- line
- sliding window
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/24—Classification techniques
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
- G06F18/214—Generating training patterns; Bootstrap methods, e.g. bagging or boosting
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/0002—Inspection of images, e.g. flaw detection
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/20—Image preprocessing
- G06V10/26—Segmentation of patterns in the image field; Cutting or merging of image elements to establish the pattern region, e.g. clustering-based techniques; Detection of occlusion
- G06V10/267—Segmentation of patterns in the image field; Cutting or merging of image elements to establish the pattern region, e.g. clustering-based techniques; Detection of occlusion by performing operations on regions, e.g. growing, shrinking or watersheds
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/30—Subject of image; Context of image processing
- G06T2207/30168—Image quality inspection
Abstract
The invention relates to a CIP data quality control-based cloud microparticle classification method which mainly comprises the steps of preprocessing a CIP image, performing cloud microparticle subimage quality control on a binarized CIP image, performing segmentation processing on the CIP image, and establishing a deep neural network classification model based on transfer learning.
Description
Technical Field
The invention belongs to the field of digital image processing and artificial intelligence algorithms, and particularly relates to a method for performing quality control on Cloud microparticle sub-images detected by an airborne Cloud Imaging (CIP) detector and realizing rapid and high-precision morphological classification of Cloud microparticle images based on a transfer learning deep neural network model.
Background
CIP is an onboard cloud imaging probe with a 64 element optical array probe. The system mainly comprises a particle detection system, a laser transmitter, a linear optical array and a digital signal processing system. When the cloud particles pass through a sampling channel of CIP, shadows generated by the laser beams irradiating the particles are projected onto the optical linear array, and cloud particle sub-images are obtained through array optical induction intensity change and digital processing system reconstruction. The cloud particle sub-images are images acquired by 64 optical arrays, which are independent slice images. CIP can detect particle images in the range of 25 to 1550 μm and deduce the particle size distribution, while CIP image resolution can be chosen to be 15 μm or 25 μm. It may also record information such as atmospheric temperature, pressure, aircraft speed, altitude, and Liquid Water Content (LWC).
In the field of meteorology, the global climate and weather are influenced by the ice cloud through radiation transmission and cloud micro physical processes, the particle size distribution of the ice cloud needs to be obtained to know the property and the forming process of the ice cloud, and the more accurate particle size distribution of the ice cloud can be obtained from a two-dimensional image recorded by an airborne probe. Through the particle size distribution of the ice cloud and the shape of the ice crystal particles, the properties of different ice clouds, such as total number concentration, extinction coefficient, ice water content, average reduction rate, precipitation rate, effective diameter and single scattering performance, can be determined. Analysis of ice cloud physical parameters plays an important role in understanding cloud precipitation processes and radiation transmission. Furthermore, understanding of the ice crystal size distribution plays an important role in improving the atmospheric model parameterization scheme. In summary, classification of particle shape by two-dimensional images of airborne probes plays an important role in understanding cloud precipitation processes, radiation transmission and global climate change.
Currently, there are still some data problems in the images collected by optical array probes. Such as photosensitive edge occlusion and pixel loss problems. In past studies, CIP ice crystal data were shape classified based on a modal parameterization scheme. But due to data quality problems and low data resolution, the classification effect is affected. There is currently no research considering image quality control of CIP ice crystal data and classification of CIP ice crystal images in conjunction with deep neural networks.
Disclosure of Invention
Aiming at the defects of the prior art, the invention provides a cloud micro-particle classification method based on CIP data quality control, which comprises the following steps:
step 1: preprocessing the cloud particle image, comprising:
step 1.1: carrying out graying processing on the CIP image, wherein the specific method is to convert an RGB three-channel image into a single-channel grayscale image;
step 1.2: carrying out binarization on the CIP gray image, wherein the threshold value of the binarization is Th;
step 2: performing cloud particle sub-image quality control on the binarized CIP image, comprising:
step 2.1: obtaining image information based on sliding window, firstly, selecting a matrix with the size of M multiplied by l as the sliding window, and taking the matrix M as the sliding windowml1 st element Mml(1,1) is superposed with i rows and j columns of pixel points C (i, j) of the CIP image to form a sliding window matrix MmlStarting position of sliding on CIP image C, and sliding window matrix M in left-to-right ordermlSliding is carried out with the sliding step length of s, when the point M on the sliding window matrixml(1,1) after the coincidence with the point C (i, q-l) on the CIP image, the sliding window matrix MmlLine feed is carried out, and the sliding window matrix M isml1 st element Mml(1,1) and CIP image pixel point C (i + k (M-1), j) are superposed to be used as a line feed starting point position of the sliding window matrix sliding on the CIP image C, and the sliding window matrix M is sequentially arranged from left to rightmlSliding is carried out with the sliding step length of s, when the point M on the sliding window matrixml(1,1) is superimposed on the CIP image at point C (i + k (M-1), q-l), and then the pair of M is continuedmlPerforming line feed until the matrix traverses and slides all pixel points on the CIP image, wherein a variable k is the number of line feed, and p × q is the size of the CIP image;
step 2.2: judging whether the cloud particle sub-image loses pixels: preferably, three sliding window matrixes of large, medium and small are selected to extract CIP image information, the size of the large sliding window is x1 × y1, the first sliding step is s1, the size of the medium sliding window is x2 × y2, the second sliding step is s2, the size of the small sliding window is x3 × y3, and the third sliding step is s3, and then the following steps are executed:
step (a): detecting whether all the image information extracted by the large sliding window meets the condition that the pixel values of the whole line are all 0, and all the upper h1 line and the lower h1 line adjacent to the line have the line with the pixel value of 1 line all 1, recording the line numbers of all the lines meeting the condition into a1, judging that the CIP image does not have the quality problem of pixel loss when no record is recorded in a1, executing step 2.3, judging that the CIP image possibly has the quality problem of pixel loss when the record is recorded in a1, and executing step (b);
step (b): detecting whether all the line numbers recorded in a1 in all the image information extracted by the middle sliding window satisfy that the pixel values of the whole line are all 0, and whether a line with all the pixel values of 1 line exists in the adjacent upper h2 line and the lower h2 line of the line, recording the line numbers of all the lines satisfying the condition into a2, executing step (b1) when no record exists in a2, and executing step (c) when a2 has a record;
step (b 1): detecting whether all the line numbers recorded in a1 in all the image information extracted by the middle sliding window satisfy the condition that the pixel values of the whole line are all 0, and the adjacent upper h3 line and the lower h3 line of the line have the line with the pixel value of 1 line all 1, recording the line numbers of all the lines satisfying the condition into a21, when the a21 does not record, judging that the quality problem of pixel loss does not exist in the CIP image, entering step 2.3, when the a21 records, judging that the quality problem of pixel loss possibly exists in the CIP image, and executing step (d);
step (c): detecting whether all line numbers recorded in a2 in all image information extracted by a small sliding window satisfy that the pixel values of the whole line are all 0, and the line with the pixel values all 1 exists in the h4 line above and the h4 line below the line, recording the line numbers of all lines satisfying the condition into a3, and recording the coordinate position of the pixel point of the line satisfying the condition on the CIP image into a4, when the line is not recorded in a3, entering step 2.3, when the line is recorded in a3, judging that the CIP image has quality problem, and the cloud particle sub-image to which the pixel point recorded in a4 belongs has pixel loss;
step 2.3: judging whether the cloud particle sub-images are shielded or not, and recording the positions of the cloud particle sub-images when the cloud particle sub-images are judged to be shielded;
and step 3: segmenting the binary CIP image, firstly performing morphological processing on the binary CIP image, firstly performing corrosion operation on a matrix with a structural unit of g1 Xg 1, then selecting a matrix with a structural unit of g2 Xg 2 for expansion operation, then performing connected domain search on the morphologically processed binary CIP image, firstly traversing all points with a pixel value of 1 in the image, respectively taking the pixel points with the pixel value of 1 as central points, searching pixel points in eight directions, namely upper, lower, left, right, lower, upper left, upper right and lower right, when a certain point value is 1, the two pixel points belong to the same connected domain, continuously searching eight-direction adjacent pixels outside the connected domain by taking the point as the center, marking corresponding region rectangles to obtain the number of the connected domains after the searching of the connected domains is completed, and finally, extracting cloud particle sub-images of the corresponding marking areas by segmenting from left to right and from top to bottom the coordinates of the upper left corner of the marking areas;
and 4, step 4: filling the pixels of the cloud particle sub-images recorded in the step 2.2 and having the pixel loss, performing expansion corrosion operation by adopting a matrix with the structure size of g3 multiplied by g3, and removing the cloud particle sub-images recorded in the step 2.3 and having the blocked condition;
and 5: establishing a cloud particle sub-image data set;
step 6: sending the marked data set into a deep neural network model based on transfer learning;
and 7: inputting the real-time data to be identified without labels into the trained model, extracting image features by a neural network and obtaining a classification result, wherein the classification result can be further applied to the calculation of parameters such as a particle size distribution spectrum, liquid water content, number concentration, effective particle size and the like on the basis to realize the analysis of the cloud microstructure.
Further, in step 2.3, it is determined whether the cloud particle sub-image is occluded as:
selecting a medium and small sliding window matrix to extract CIP image information, wherein the size of the medium and small sliding window is x4 multiplied by y4, the sliding step length is s4, and then, executing the following steps:
step (d): detecting whether each image information extracted by the small and medium sliding windows has a condition that the 3 rd row has continuous non-zero values or not, if not, executing the step (e), if so, marking the number of the continuous non-zero values as c1, and detecting the number of the continuous non-zero values of each row of the 3 th row adjacent to and below the row as c2, c3 and c4, respectively, if c1 < round (beta 1 × c2), c2 < round (beta 2 × c3) and c3 < round (beta 3 × c4) are simultaneously satisfied, judging that the image information extracted by the small and medium sliding windows has a quality problem, and judging that the cloud microparticle image to which the 3 rd row continuous non-zero value pixel points belong in the sliding window has a shielded condition;
a step (e): detecting whether each image information extracted by the small and medium sliding windows has the condition that the 2 nd row from the last has continuous non-zero values or not, if not, judging that the cloud particle images are not blocked, if so, marking the quantity of the continuous non-zero values as d1, and detecting the quantity of the continuous non-zero values of each row of 3 rows above the row as d1, d2, d3 and d4, if d1 is more than round (beta 1 multiplied by d2), d2 is more than or equal to round (beta 2 multiplied by d3), and d3 is more than or equal to round (beta 3 multiplied by d4), judging that the image information extracted by the small and medium sliding windows has the quality problem, and judging that the cloud particle images of the 2 nd row continuous non-zero value pixel points in the sliding windows have the blocked condition.
Further, the establishing a cloud particle sub-image dataset comprises:
the method comprises the following steps of dividing the types of cloud particle sub-images, and dividing the cloud particles into the following types according to the particle characteristics in the cloud particle sub-images: micro, spherical cloud drop, spherical rain drop, columnar, needle-shaped, irregular, hexagonal plate-shaped, aragonite, dendritic, mixed and the like.
Further, the micro-type, the spherical cloud drop, the spherical raindrop, the columnar type, the needle-like type, the irregular type, the hexagonal plate-like type, the shot-like type, the dendritic type and the mixed type are specifically characterized in that the total number of pixel points of the particles is between 10 and 120 pixels, the spherical cloud drop type is characterized in that the particle shape is approximately circular and the particle diameter is less than 100 mu m, the spherical raindrop type is characterized in that the particle shape is approximately circular and the particle diameter is more than 100 mu m, the columnar type is characterized in that the basic structure is a linear structure, the width of two ends is 3 to 5 times of that of the linear structure, the needle-like type is characterized in that the basic structure is a linear structure, the ratio of the width of the two ends is more than 1.5, the total number of the pixel points of the particles is less than 700, the irregular type is characterized in that the shape is irregular, the total number of the pixels is between 1000 and 4000, the hexagonal plate-like structure is, the aragonite type is characterized in that the total number of the particle pixels is more than 10000, no obvious branch exists, the edge is smooth, the dendritic type is characterized in that six branch structures are uniformly distributed, no other ice crystal is condensed on a branch angle, the mixed type is characterized in that the basic structure is an aggregate, the shape of the aggregate is mainly the condensation growth of other types of cloud microparticles, the total number of the particle pixels is more than 8000, and then the cloud microparticle image obtained by segmentation is marked with a label of a corresponding type.
Further, in the deep neural network model based on the transfer learning, the specific setting parameters of the network model are as follows: firstly, dividing a cloud microparticle image data set into a training set and a testing set according to a ratio of 5:3, performing random rotation and mirror symmetry rotation operation on cloud microparticle images, unifying cloud microparticle image matrixes to 224 x 224, normalizing a cloud microparticle image digital matrix, setting batch size values of the training set and the testing set to be 10, setting sample iteration times to be 16, setting a cross entropy function to be a loss function, setting minimum batch gradient descent to be a parameter updating function, setting a learning rate to be 0.0001, setting a final output value of a complete connection layer to be 9, obtaining a pre-training model by using a Pytrch open source library, training data on six models, and finally evaluating the models through overall precision, accuracy, recall rate and F1 scores to obtain a classification model suitable for CIP microparticle cloud subimages.
Furthermore, the six models are respectively TL-AlexNet, TL-Vgg16, TL-Vgg19, TL-ResNet18, TL-ResNet34 and TL-Squeezenet models.
Compared with the traditional CIP data cloud microparticle classification method, the method has the following advantages, so that the corresponding technical problems are solved:
1. aiming at the classification and identification of CIP cloud microparticle subimages, a CIP data quality control-based cloud microparticle classification method is provided, and the cloud microparticle classification accuracy can be effectively improved through the method.
2. Aiming at the problem of pixel loss in cloud particle sub-images of CIP data, an effective judgment method is provided, and the reliability of a classification result is improved.
3. Aiming at the problem that whether the cloud particle sub-image of the CIP data is blocked or not, an effective judgment method is provided, and the reliability of the classification result is improved.
4. Specific division standards are provided for the classification of the cloud particle subimages of the CIP data, and more reliable data support is provided for researching meteorological changes such as radiation transmission and cloud micro physics.
Drawings
FIG. 1 is a flow chart of a cloud micro-particle classification method based on CIP data quality control;
fig. 2 is a graph of the best accuracy of six kinds of transfer learning models in the test.
Detailed Description
The technical solution in the embodiment of the present invention will be clearly and completely described below with reference to the accompanying drawings in the embodiment of the present invention, and a flowchart of the method is shown in fig. 1, and includes the following steps:
step 1: preprocessing the cloud particle image, comprising:
step 1.1: carrying out graying processing on the CIP image, wherein the specific method is to convert an RGB three-channel image into a single-channel grayscale image;
step 1.2: binarizing the CIP gray image, wherein the threshold value of binarization is Th 254;
step 2: performing cloud particle sub-image quality control on the binarized CIP image, comprising:
step 2.1: obtaining image information based on sliding window, firstly, selecting a matrix with the size of M multiplied by l as the sliding window, and taking the matrix M as the sliding windowml1 st element Mml(1,1) is superposed with i rows and j columns of pixel points C (i, j) of the CIP image to form a sliding window matrix MmlStarting position of sliding on CIP image C, and sliding window matrix M in left-to-right ordermlSliding is carried out with the sliding step length of s, when the point M on the sliding window matrixml(1,1) after the coincidence with the point C (i, q-l) on the CIP image, the sliding window matrix MmlLine feed is carried out, and the sliding window matrix M isml1 st element Mml(1,1) and CIP image pixel point C (i + k (M-1), j) are superposed to be used as a line feed starting point position of the sliding window matrix sliding on the CIP image C, and the sliding window matrix M is sequentially arranged from left to rightmlThe sliding is carried out with the step length of s,when point M on the sliding window matrixml(1,1) is superimposed on the CIP image at point C (i + k (M-1), q-l), and then the pair of M is continuedmlPerforming line feed until the matrix traverses and slides all pixel points on the CIP image, wherein a variable k is the number of line feed, and p × q is the size of the CIP image;
step 2.2: judging whether the cloud particle sub-image loses pixels: selecting three sliding window matrixes of large, medium and small to extract CIP image information, wherein the size of a large sliding window is 18 multiplied by 90, the sliding step length is s 1-18, the size of a medium sliding window is 10 multiplied by 90, the sliding step length is s 2-10, the size of a small sliding window is 5 multiplied by 90, and the sliding step length is s 3-5, and then executing the following steps:
step (a): detecting whether all image information extracted by the large sliding window has a line meeting the condition that the pixel values of the whole line are all 0, and all the upper 8 lines and the lower 8 lines adjacent to the line have the condition that the pixel values of some 1 line are all 1, recording the line numbers of all the lines meeting the condition into a1, judging that the CIP image has no quality problem of pixel loss when no record is recorded in a1, executing step 2.3, judging that the CIP image has the quality problem of pixel loss when a1 has a record, and executing step (b);
step (b): detecting whether all the line numbers recorded in a1 in all the image information extracted by the middle sliding window satisfy that the pixel values of the whole line are all 0, and whether a line with the pixel values of all 1 line exists in the adjacent upper 5 lines and the lower 5 lines of the line, recording the line numbers of all the lines satisfying the condition into a2, executing step (b1) when no record exists in a2, and executing step (c) when a record exists in a 2;
step (b 1): detecting whether all the line numbers recorded in a1 in all the image information extracted by the middle sliding window satisfy the condition that the pixel values of the whole line are all 0, and the adjacent upper 6 lines and lower 6 lines of the line have the line with the pixel value of 1 line all 1, recording the line numbers of all the lines satisfying the condition into a21, when the a21 does not record, judging that the CIP image does not have the quality problem of pixel loss, entering step 2.3, when the a21 records, judging that the CIP image possibly has the quality problem of pixel loss, and executing step (c);
step (c): detecting whether all the line numbers recorded in a2 in all the image information extracted by the small sliding window satisfy the condition that the pixel values of the whole line are all 0 and the pixel values of the upper 3 lines and the lower 3 lines adjacent to the line are all 1, recording the line numbers of all the lines satisfying the condition into a3, recording the coordinate positions of the pixel points of the lines satisfying the condition on the CIP image into a4, when the a3 does not record, entering step 2.2.2, when the a3 records, judging that the CIP image has quality problems, and judging that the cloud particle sub-images to which the pixel points recorded in a4 belong have pixel loss;
step 2.3: judging whether the cloud particle sub-image is shielded or not, selecting a medium and small sliding window matrix to extract CIP image information, wherein the size of the medium and small sliding window is 14 multiplied by 90, the sliding step length is s-14, and then executing the following steps:
step (d): detecting whether each image information extracted by the small and medium sliding windows meets the condition that the 3 rd row has continuous non-zero values or not, if not, executing the step (e), if so, marking the number of the continuous non-zero values as c1, and detecting the number of the continuous non-zero values of each row of the 3 th row adjacent to and below the row as c2, c3 and c4, respectively, if c1 < round (beta 1 × c2), c2 is less than or equal to round (beta 2 × c3) and c3 is less than or equal to round (beta 3 × c4) are met, judging that the image information extracted by the small and medium sliding windows has quality problems, and judging that the cloud microparticle image to which the 3 rd row continuous non-zero value pixel points in the sliding window belongs has a blocked condition;
a step (e): detecting whether each image information extracted by the small and medium sliding windows has the condition that the 2 nd row meets the condition that continuous non-zero values exist, if not, judging that the cloud particle images are not blocked, if so, marking the quantity of the continuous non-zero values as d1, and detecting the quantity of the continuous non-zero values of each row of the 3 rows above the row as d1, d2, d3 and d4, if d1 is more than round (beta 1 multiplied by d2), d2 is more than or equal to round (beta 2 multiplied by d3) and d3 is more than or equal to round (beta 3 multiplied by d4), judging that the image information extracted by the small and medium sliding windows has the quality problem, and judging that the cloud particle images of the 2 nd row continuous non-zero value pixel points in the sliding windows have the condition that the cloud particle images are blocked;
and step 3: segmenting a binarized CIP image, firstly performing morphological processing on the binarized CIP image, firstly performing corrosion operation by adopting a matrix with a structural unit of 4 multiplied by 4, then selecting a matrix with a structural unit of 3 multiplied by 3 to perform expansion operation, then performing connected domain search on the binarized CIP image after the morphological processing, firstly traversing all points with a pixel value of 1 in the image, respectively taking the pixel points with the pixel value of 1 as a central point, searching pixel points in eight directions, namely, upper, lower, left, upper left, upper right and lower right, and when a certain point value is 1, enabling the two pixel points to belong to the same connected domain, continuously searching eight-direction adjacent pixels outside the connected domain by taking the point as the center, marking corresponding region rectangles to obtain the number of the connected domains after the searching of the connected domains is completed, and finally, extracting cloud particle sub-images of the corresponding marking areas by segmenting from left to right and from top to bottom the coordinates of the upper left corner of the marking areas;
and 4, step 4: filling the pixels of the cloud particle sub-images recorded in the step 2.2 and having the pixel loss, performing expansion corrosion operation by adopting a matrix with the structure size of 3 multiplied by 3, and removing the cloud particle sub-images recorded in the step 2.3 and having the sheltered condition;
and 5: establishing a cloud particle image data set, firstly dividing the cloud particle image types into 10 types which are respectively micro, spherical cloud drop, spherical raindrop, columnar, needle-shaped, irregular, hexagonal plate-shaped, shot-shaped, dendritic and mixed, wherein the micro type is characterized in that the total number of pixel points of the particles is between 10 and 120 pixels, the spherical cloud drop is characterized in that the shape of the particles is approximate to a circle and the particle diameter is less than 100 mu m, the spherical raindrop is characterized in that the shape of the particles is approximate to a circle and the particle diameter is more than 100 mu m, the columnar type is characterized in that the basic structure is a linear structure and the width of two ends is 3 to 5 times of the linear structure, the needle-shaped type is characterized in that the basic structure is a linear structure, the ratio of the widths of the two ends is more than 1.5, the total number of the pixel points of the particles is less than 700, the shape of the irregular type is irregular, and the total number of the pixels is between 1000 and 4000, the hexagonal plate-shaped class is characterized in that the basic structure is plate-shaped, six vertex angles are visible, the plate-shaped region is smooth, the aragonite class is characterized in that the total number of the particle pixels is more than 10000, no obvious branch exists, the edge is smooth, the dendritic class is characterized in that six branch structures are uniformly distributed, no other ice crystals are condensed on branch angles, the mixed class is characterized in that the basic structure is an aggregate, the shape of the aggregate is mainly that other types of cloud microparticles are condensed and grow, the total number of the particle pixels is more than 8000, and then the cloud microparticle image obtained by segmentation is marked with a label of a corresponding type;
step 6: and sending the marked data set into a deep neural network model based on the transfer learning, wherein the specific setting parameters of the network model are as follows: firstly, dividing a cloud microparticle image data set into a training set and a test set according to a ratio of 5:3, performing random rotation and mirror symmetry rotation operations on cloud microparticle images, unifying a cloud microparticle image matrix into 224 × 224, normalizing a cloud microparticle image digital matrix, setting the batch size values of the training set and the test set to be 10, setting the sample iteration number to be 16, setting a cross entropy function to be a loss function, and setting the minimum batch gradient drop as a parameter updating function, further setting a learning rate to be 0.0001, setting the final output value of a complete connection layer to be 9, then obtaining a pre-training model by using a Pythroch open source library, then training data on six models TL-AlexNet, TL-Vgg16, TL-Vgg19, TL-ResNet18, TL-ResNet34 and TL-SqueezeNet, wherein the optimal accuracy graph of the six migration learning models in the test is shown in FIG. 2, finally, evaluating the model through overall precision, accuracy, recall rate and F1 score to obtain a classification model suitable for CIP cloud particle subimages;
and 7: inputting the real-time data to be identified without labels into the trained model, extracting image features by a neural network and obtaining a classification result, wherein the classification result can be further applied to the calculation of parameters such as a particle size distribution spectrum, liquid water content, number concentration, effective particle size and the like on the basis of the classification result, so that the analysis of the cloud microstructure is realized;
it is to be understood that the described embodiments are merely exemplary of the invention, and not restrictive of the full scope of the invention. Other embodiments, which can be derived by one of ordinary skill in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
Claims (6)
1. A cloud micro-particle classification method based on CIP data quality control comprises the following steps:
step 1: preprocessing the cloud particle image, comprising:
step 1.1: carrying out graying processing on the CIP image, wherein the specific method is to convert an RGB three-channel image into a single-channel grayscale image;
step 1.2: carrying out binarization on the CIP gray image, wherein the threshold value of the binarization is Th;
step 2: performing cloud particle sub-image quality control on the binarized CIP image, comprising:
step 2.1: obtaining image information based on sliding window, firstly, selecting a matrix with the size of M multiplied by l as the sliding window, and taking the matrix M as the sliding windowml1 st element Mml(1,1) is superposed with i rows and j columns of pixel points C (i, j) of the CIP image to form a sliding window matrix MmlStarting position of sliding on CIP image C, and sliding window matrix M in left-to-right ordermlSliding is carried out with the sliding step length of s, when the point M on the sliding window matrixml(1,1) after the coincidence with the point C (i, q-l) on the CIP image, the sliding window matrix MmlLine feed is carried out, and the sliding window matrix M isml1 st element Mml(1,1) and CIP image pixel point C (i + k (M-1), j) are superposed to be used as a line feed starting point position of the sliding window matrix sliding on the CIP image C, and the sliding window matrix M is sequentially arranged from left to rightmlSliding is carried out with the sliding step length of s, when the point M on the sliding window matrixml(1,1) is superimposed on the CIP image at point C (i + k (M-1), q-l), and then the pair of M is continuedmlPerforming line feed until the matrix traverses and slides all pixel points on the CIP image, wherein a variable k is the number of line feed, and p × q is the size of the CIP image;
step 2.2: judging whether the cloud particle sub-image loses pixels: selecting three sliding window matrixes of large, medium and small to extract CIP image information, wherein the size of a large sliding window is x1 multiplied by y1, the first sliding step length is s1, the size of a medium sliding window is x2 multiplied by y2, the second sliding step length is s2, the size of a small sliding window is x3 multiplied by y3, and the third sliding step length is s3, and then, executing the following steps:
step (a): detecting whether all the image information extracted by the large sliding window meets the condition that the pixel values of the whole line are all 0, and all the upper h1 line and the lower h1 line adjacent to the line have the line with the pixel value of 1 line all 1, recording the line numbers of all the lines meeting the condition into a1, judging that the CIP image does not have the quality problem of pixel loss when no record is recorded in a1, executing step 2.3, judging that the CIP image possibly has the quality problem of pixel loss when the record is recorded in a1, and executing step (b);
step (b): detecting whether all the line numbers recorded in a1 in all the image information extracted by the middle sliding window satisfy that the pixel values of the whole line are all 0, and whether a line with all the pixel values of 1 line exists in the adjacent upper h2 line and the lower h2 line of the line, recording the line numbers of all the lines satisfying the condition into a2, executing step (b1) when no record exists in a2, and executing step (c) when a2 has a record;
step (b 1): detecting whether all the line numbers recorded in a1 in all the image information extracted by the middle sliding window satisfy the condition that the pixel values of the whole line are all 0, and the adjacent upper h3 line and the lower h3 line of the line have the line with the pixel value of 1 line all 1, recording the line numbers of all the lines satisfying the condition into a21, when the a21 does not record, judging that the quality problem of pixel loss does not exist in the CIP image, entering step 2.3, when the a21 records, judging that the quality problem of pixel loss possibly exists in the CIP image, and executing step (d);
step (c): detecting whether all line numbers recorded in a2 in all image information extracted by a small sliding window satisfy that the pixel values of the whole line are all 0, and the line with the pixel values all 1 exists in the h4 line above and the h4 line below the line, recording the line numbers of all lines satisfying the condition into a3, and recording the coordinate position of the pixel point of the line satisfying the condition on the CIP image into a4, when the line is not recorded in a3, entering step 2.3, when the line is recorded in a3, judging that the CIP image has quality problem, and the cloud particle sub-image to which the pixel point recorded in a4 belongs has pixel loss;
step 2.3: judging whether the cloud particle sub-images are shielded or not, and recording the positions of the cloud particle sub-images when the cloud particle sub-images are judged to be shielded;
and step 3: segmenting the binary CIP image, firstly performing morphological processing on the binary CIP image, firstly performing corrosion operation on a matrix with a structural unit of g1 Xg 1, then selecting a matrix with a structural unit of g2 Xg 2 for expansion operation, then performing connected domain search on the morphologically processed binary CIP image, firstly traversing all points with a pixel value of 1 in the image, respectively taking the pixel points with the pixel value of 1 as central points, searching pixel points in eight directions, namely upper, lower, left, right, lower, upper left, upper right and lower right, when a certain point value is 1, the two pixel points belong to the same connected domain, continuously searching eight-direction adjacent pixels outside the connected domain by taking the point as the center, marking corresponding region rectangles to obtain the number of the connected domains after the searching of the connected domains is completed, and finally, extracting cloud particle sub-images of the corresponding marking areas by segmenting from left to right and from top to bottom the coordinates of the upper left corner of the marking areas;
and 4, step 4: filling the pixels of the cloud particle sub-images recorded in the step 2.2 and having the pixel loss, performing expansion corrosion operation by adopting a matrix with the structure size of g3 multiplied by g3, and removing the cloud particle sub-images recorded in the step 2.3 and having the blocked condition;
and 5: establishing a cloud particle sub-image data set;
step 6: sending the marked data set into a deep neural network model based on transfer learning;
and 7: inputting the real-time data to be identified without labels into the trained model, extracting image features by the neural network and obtaining a classification result.
2. The method for classifying cloud microparticles based on CIP data quality control according to claim 1, wherein in step 2.3, whether the cloud microparticle sub-image is occluded or not is determined as follows:
selecting a medium and small sliding window matrix to extract CIP image information, wherein the size of the medium and small sliding window is x4 multiplied by y4, the sliding step length is s4, and then, executing the following steps:
step (d): detecting whether each image information extracted by the small and medium sliding windows has a condition that the 3 rd row has continuous non-zero values or not, if not, executing the step (e), if so, marking the number of the continuous non-zero values as c1, and detecting the number of the continuous non-zero values of each row of the 3 th row adjacent to and below the row as c2, c3 and c4, respectively, if c1 < round (beta 1 × c2), c2 < round (beta 2 × c3) and c3 < round (beta 3 × c4) are simultaneously satisfied, judging that the image information extracted by the small and medium sliding windows has a quality problem, and judging that the cloud microparticle image to which the 3 rd row continuous non-zero value pixel points belong in the sliding window has a shielded condition;
a step (e): detecting whether each image information extracted by the small and medium sliding windows has the condition that the 2 nd row from the last has continuous non-zero values or not, if not, judging that the cloud particle images are not blocked, if so, marking the quantity of the continuous non-zero values as d1, and detecting the quantity of the continuous non-zero values of each row of 3 rows above the row as d1, d2, d3 and d4, if d1 is more than round (beta 1 multiplied by d2), d2 is more than or equal to round (beta 2 multiplied by d3), and d3 is more than or equal to round (beta 3 multiplied by d4), judging that the image information extracted by the small and medium sliding windows has the quality problem, and judging that the cloud particle images of the 2 nd row continuous non-zero value pixel points in the sliding windows have the blocked condition.
3. The CIP data quality control-based cloud particulate classification method of claim 1, wherein the establishing a cloud particulate sub-image dataset comprises:
the method comprises the following steps of dividing the types of cloud particle sub-images, and dividing the cloud particles into the following types according to the particle characteristics in the cloud particle sub-images: micro, spherical cloud drop, spherical rain drop, columnar, needle-shaped, irregular, hexagonal plate-shaped, aragonite, dendritic and mixed.
4. The CIP data quality control-based cloud microparticle classification method according to claim 3, wherein the micro-type, the spherical cloud droplet, the spherical raindrop, the columnar type, the needle-like type, the irregular type, the hexagonal plate-like type, the aragonite type, the dendritic type, and the mixed type are characterized in that the total number of the pixel points of the micro-type is between 10 and 120 pixels, the spherical cloud droplet type is characterized in that the particle shape is approximately circular and the particle diameter is less than 100 μm, the spherical raindrop type is characterized in that the particle shape is approximately circular and the particle diameter is more than 100 μm, the columnar type is characterized in that the basic structure is a linear structure and the width of both ends is 3 to 5 times of the linear structure, the needle-like type is characterized in that the basic structure is a linear structure, the ratio of the widths of both ends is more than 1.5, the total number of the pixel points of the particle is less than 700, the irregular type is characterized in that the shape is irregular, and the total number, the hexagonal plate-shaped type is characterized in that the basic structure is plate-shaped, six vertex angles are visible, the plate-shaped area is smooth, the aragonite type is characterized in that the total number of the particle pixels is more than 10000, no obvious branch exists, the edge is smooth, the dendritic type is characterized in that six branch structures are uniformly distributed, no other ice crystals are condensed on branch angles, the mixed type is characterized in that the basic structure is an aggregate, the shape of the aggregate is mainly that other types of cloud microparticles are condensed and grow, the total number of the particle pixels is more than 8000, and then the cloud microparticle image obtained by segmentation is marked with a label of a corresponding type.
5. The CIP data quality control-based cloud particle classification method according to claim 1, wherein the deep neural network model based on the transfer learning specifically sets parameters as follows: firstly, dividing a cloud microparticle image data set into a training set and a testing set according to a ratio of 5:3, performing random rotation and mirror symmetry rotation operation on cloud microparticle images, unifying cloud microparticle image matrixes to 224 x 224, normalizing a cloud microparticle image digital matrix, setting batch size values of the training set and the testing set to be 10, setting sample iteration times to be 16, setting a cross entropy function to be a loss function, setting minimum batch gradient descent to be a parameter updating function, setting a learning rate to be 0.0001, setting a final output value of a complete connection layer to be 9, obtaining a pre-training model by using a Pytrch open source library, training data on six models, and finally evaluating the models through overall precision, accuracy, recall rate and F1 scores to obtain a classification model suitable for CIP microparticle cloud subimages.
6. The CIP data quality control-based cloud micro-particle classification method according to claim 5, wherein the six models are TL-AlexNet, TL-Vgg16, TL-Vgg19, TL-ResNet18, TL-ResNet34 and TL-Squeezenet models respectively.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010495758.2A CN111860571B (en) | 2020-06-03 | 2020-06-03 | Cloud microparticle classification method based on CIP data quality control |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010495758.2A CN111860571B (en) | 2020-06-03 | 2020-06-03 | Cloud microparticle classification method based on CIP data quality control |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN111860571A CN111860571A (en) | 2020-10-30 |
| CN111860571B true CN111860571B (en) | 2021-05-25 |
Family
ID=72985593
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202010495758.2A Active CN111860571B (en) | 2020-06-03 | 2020-06-03 | Cloud microparticle classification method based on CIP data quality control |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111860571B (en) |
Families Citing this family (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112907475B (en) * | 2021-03-02 | 2023-05-23 | 珠海复旦创新研究院 | Automatic segmentation method for ice crystal pictures |
| CN114821136B (en) * | 2022-04-11 | 2023-04-21 | 成都信息工程大学 | Self-adaptive cloud microparticle image data processing method |
| CN114677499B (en) * | 2022-04-11 | 2023-04-18 | 成都信息工程大学 | Cloud microparticle image particle region positioning method |
| CN116895039B (en) * | 2023-09-11 | 2023-11-17 | 中国空气动力研究与发展中心低速空气动力研究所 | Icing cloud and fog pseudo particle image identification and characteristic parameter measurement method |
Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101498903A (en) * | 2008-02-01 | 2009-08-05 | 株式会社理光 | Toner, developer, image forming method, image forming apparatus and process cartridge |
| CN106054194A (en) * | 2016-05-10 | 2016-10-26 | 南京信息工程大学 | Spaceborne radar and ground-based radar reflectivity factor data three dimensional fusion method |
| US9933351B2 (en) * | 2015-03-06 | 2018-04-03 | Scanit Technologies, Inc. | Personal airborne particle monitor with quantum dots |
Family Cites Families (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| EP3040705A1 (en) * | 2014-12-30 | 2016-07-06 | Grundfos Holding A/S | Method for determining of particles |
| CN106683116B (en) * | 2016-08-31 | 2019-06-21 | 电子科技大学 | A kind of particle filter combined tracking method based on support vector machines |
| CN107992782B (en) * | 2016-10-27 | 2020-05-19 | 中国科学院大气物理研究所 | Cloud precipitation particle shape identification method based on image geometric characteristic parameters |
| WO2019190576A2 (en) * | 2017-06-13 | 2019-10-03 | The Trustees Of Princeton University | Fully automatic, template -free particle picking for electron microscopy |
| CN108198207A (en) * | 2017-12-22 | 2018-06-22 | 湖南源信光电科技股份有限公司 | Multiple mobile object tracking based on improved Vibe models and BP neural network |
| CN109800754B (en) * | 2018-12-06 | 2020-11-06 | 杭州电子科技大学 | Ancient font classification method based on convolutional neural network |
| CN110119780B (en) * | 2019-05-10 | 2020-11-27 | 西北工业大学 | Hyper-spectral image super-resolution reconstruction method based on generation countermeasure network |
-
2020
- 2020-06-03 CN CN202010495758.2A patent/CN111860571B/en active Active
Patent Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101498903A (en) * | 2008-02-01 | 2009-08-05 | 株式会社理光 | Toner, developer, image forming method, image forming apparatus and process cartridge |
| US9933351B2 (en) * | 2015-03-06 | 2018-04-03 | Scanit Technologies, Inc. | Personal airborne particle monitor with quantum dots |
| CN106054194A (en) * | 2016-05-10 | 2016-10-26 | 南京信息工程大学 | Spaceborne radar and ground-based radar reflectivity factor data three dimensional fusion method |
Also Published As
| Publication number | Publication date |
|---|---|
| CN111860571A (en) | 2020-10-30 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN111860571B (en) | Cloud microparticle classification method based on CIP data quality control | |
| CN108510467B (en) | SAR image target identification method based on depth deformable convolution neural network | |
| CN111598098B (en) | Water gauge water line detection and effectiveness identification method based on full convolution neural network | |
| CN113392775B (en) | Sugarcane seedling automatic identification and counting method based on deep neural network | |
| CN110728658A (en) | High-resolution remote sensing image weak target detection method based on deep learning | |
| US20210321963A1 (en) | Systems and methods for enhanced imaging and analysis | |
| CN106897681B (en) | Remote sensing image contrast analysis method and system | |
| CN104657706B (en) | The fracture of high ferro circuit line bar and draw bail body method for detecting abnormality based on image | |
| CN112836713A (en) | Image anchor-frame-free detection-based mesoscale convection system identification and tracking method | |
| CN111860570B (en) | Cloud particle image extraction and classification method | |
| CN109409205A (en) | Video road driveway line detecting method of taking photo by plane based on line pitch characteristics point cluster | |
| CN110598613B (en) | Expressway agglomerate fog monitoring method | |
| CN112069985B (en) | High-resolution field image rice spike detection and counting method based on deep learning | |
| CN114488194A (en) | Method for detecting and identifying targets under structured road of intelligent driving vehicle | |
| CN115115627B (en) | Soil saline-alkali soil monitoring method based on data processing | |
| CN105512622A (en) | Visible remote-sensing image sea-land segmentation method based on image segmentation and supervised learning | |
| CN111291818B (en) | Non-uniform class sample equalization method for cloud mask | |
| Ahlswede et al. | Hedgerow object detection in very high-resolution satellite images using convolutional neural networks | |
| CN112464933B (en) | Intelligent identification method for weak and small target through foundation staring infrared imaging | |
| CN113191271A (en) | Method for detecting surface particulate matter of weir dam based on deep learning | |
| CN112085725A (en) | Residual film residual quantity detection method and early warning system based on heuristic iterative algorithm | |
| Chen et al. | 2d tree detection in large urban landscapes using aerial lidar data | |
| CN116188943A (en) | Solar radio spectrum burst information detection method and device | |
| CN112036246B (en) | Construction method of remote sensing image classification model, remote sensing image classification method and system | |
| CN112508986B (en) | Water level measurement method based on deep convolutional network and random field |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |