CN110781954A - Adaptive fuzzy superpixel generation method for polarized SAR image classification - Google Patents

Abstract

The invention discloses a superpixel generation method for polarimetric SAR image classification, which mainly solves the problems of low precision of superpixel generation results in the prior art and inability to adaptively adjust the ratio of undetermined pixels. The implementation scheme is: 1) set parameters and input the polarimetric SAR image; 2) initialize the cluster center of the polarimetric SAR image to find the overlapping search area and non-overlapping search area; 3) iteratively calculate the clustering of pixels in the overlapping search area. Class center and membership matrix, until the maximum number of iterations is reached or the change of cluster center between two iterations is less than the set threshold; 4) Adaptively determine the proportion of undetermined pixels; 5) According to the ratio of membership matrix and undetermined pixels The fuzzy superpixel is generated; the invention improves the accuracy of the superpixel generation result, can adaptively adjust the ratio of the undetermined pixels, and can be applied to polarization SAR image classification.

Description

An Adaptive Fuzzy Superpixel Generation Method for Polarimetric SAR Image Classification

technical field

The invention belongs to the technical field of remote sensing images, and in particular relates to a method for generating self-adaptive fuzzy superpixels. Can be used to classify polarimetric SAR images.

Background technique

The purpose of the superpixel method is to segment the original image into smaller regions with similar features to preserve some spatial neighborhood information, so that the image classification is more computationally efficient than directly using pixel classification. Today, superpixels have been widely used in computer vision fields, such as image classification, image segmentation, image enhancement, foreground extraction, and visual object tracking.

With the development of Polarimetric Synthetic Aperture Radar SAR, PolSAR images have been widely used in land resource monitoring and damage assessment. Polarimetric SAR image classification is also an important research direction, since superpixels can reduce computational complexity while maintaining regional consistency. Degree, superpixel-based polarimetric SAR image classification methods have also been extensively studied.

Almost all superpixel generation methods are diverse and can be used as preprocessing methods for various applications. Diversity is an advantage, but since the specific nature of the application scene itself is not considered in the process of generating superpixels, the results of superpixel generation cannot achieve the best performance in specific application scenarios. That is to say, two main aspects should be considered for superpixels as the preprocessing of polarimetric SAR image classification, one is the characteristics of polarimetric SAR images, that is, polarimetric scattering information, and the other is the classification accuracy requirements.

R. Achanta et al. proposed a simple linear iterative clustering SLIC method in "Slic superpixels compared to state-of-the-artsuperpixel methods", which uses the k-means clustering algorithm to generate superpixels, the number of superpixels and The degree of compactness can be adjusted by parameters. As a general superpixel generation method, SLIC does not use the polarization scattering information in the polarization SAR image, so the accuracy of the superpixel generated on the polarization SAR image is not high.

Y. Guo et al. proposed the fuzzy superpixel FS method in "Fuzzy superpixels for polarimetric sar images classification". The FS proposed the concept of fuzzy superpixel based on the SLIC method. The fuzzy superpixel divides some pixels into undetermined pixels. Not classified into a specific superpixel, thus increasing the proportion of pure superpixels. However, since the FS method only uses the color and spatial information of the polarimetric SAR image, and does not use the polarimetric scattering information, the accuracy of the superpixels generated on the polarimetric SAR image still has room for improvement, and the FS method cannot automatically To adapt to adjust the ratio of undetermined pixels, it needs to be adjusted manually for different images. When the number of images to be processed increases, parameter adjustment will become very inconvenient.

SUMMARY OF THE INVENTION

The purpose of the present invention is to provide a superpixel generation method for polarimetric SAR image classification in view of the above-mentioned deficiencies of the prior art, so as to improve the superpixel classification accuracy and realize self-adaptive adjustment of the pixel ratio to be determined.

The technical idea of the present invention is: in addition to color and space information, superpixels are generated by polarization scattering information of SAR images; the ratio of undetermined pixels to overall pixels is adaptively determined by correlation between pixels, and the implementation steps include the following:

(1) Set the desired number of superpixels K, the threshold E, the maximum number of iterations I, and the distance weight parameter mf;

(2) Initialize the cluster centers of superpixels to find non-overlapping search areas and overlapping search areas;

(3) Generate fuzzy superpixels, that is, divide the pixels into two parts: superpixels and undetermined pixels:

其中c_j表示此重叠区域内像素对应于超像素j的聚类中心，n为该重叠搜索区域内的像素总数，x_i表示该重叠搜索区域内的第i个像素点的特征，u_ij表示像素点i与其对应于超像素j的聚类中心的隶属度,m为隶属度的正则化参数，i取值都是从1到n；(3a) Update the corresponding cluster centers of pixels in the overlapping search area as:

where c _j represents the cluster center of the pixels in the overlapping area corresponding to superpixel j, n is the total number of pixels in the overlapping search area, x _i represents the feature of the i-th pixel in the overlapping search area, and u _ij represents The membership degree of the pixel point i and the cluster center corresponding to the superpixel j, m is the regularization parameter of the membership degree, and the value of i is from 1 to n;

其中c表示该重叠搜索区域为c个聚类中心搜索区域的重叠部分，D_ij为第i个像素到第j个像素的距离，D_ik为第i个像素到第k个聚类中心的距离，m为隶属度的正则化参数，k取值为从1到c；(3b) Update the membership degree between pixel i and the jth superpixel cluster center in the overlapping search area as:

where c indicates that the overlapping search area is the overlapping part of the c cluster center search areas, D _ij is the distance from the i-th pixel to the j-th pixel, and D _ik is the distance from the i-th pixel to the k-th cluster center , m is the regularization parameter of membership degree, and k is from 1 to c;

(3c) Repeat steps (3a) to (3b) until the maximum number of iterations I or the change of the cluster center between two iterations is less than E;

表示第i个像素与第j个像素之间的模糊相似关系值，fea_t表示SAR特征图的第t个通道，fea_tmax，fea_tmin分别表示特征图第t个通道中的最大值和最小值，如果r_ij小于零，则令r_ij＝0；(3d) Calculate the fuzzy similarity relationship matrix: in:

Represents the fuzzy similarity relationship value between the ith pixel and the jth pixel, fea _t represents the t-th channel of the SAR feature map, fea _t max, fea _t min represent the maximum value in the t-th channel of the feature map and The minimum value, if ri _ij is less than zero, then let ri _ij =0;

其中U为该重叠搜索区域内所有像素与各个聚类中心之间的隶属度所构成的矩阵，矩阵大小为n×c；(3e) Calculate the relevance matrix of overlapping search regions:

where U is the matrix formed by the membership degrees between all pixels in the overlapping search area and each cluster center, and the size of the matrix is n×c;

其中E_tq为关联度矩阵E的第t行第q列的元素，sum(diag(E))表示关联度矩阵E对角元素之和；(3f) Calculate the degree of difference between cluster centers:

Where E _tq is the element of the t-th row and the q-th column of the correlation matrix E, and sum(diag(E)) represents the sum of the diagonal elements of the correlation matrix E;

(3g) Determine the proportion of undetermined pixels in the overlapping search area according to the equation P=0.5×Fand P<1;

(3h) Generate fuzzy superpixels according to the membership matrix U and the adaptively determined proportion P of undetermined pixels, that is, sort the largest elements of each row in the membership matrix U from large to small, and sort them before the n×Pth bit The pixel of , is classified into the superpixel corresponding to the column where the largest element of the row is located, and the pixel after the n×Pth bit is determined as the undetermined pixel, where n is the number of rows of the membership matrix U;

(4) Taking each undetermined pixel as the center, in the surrounding M×M area, find the number of superpixels in this area: if the number of superpixels is greater than 1, then mark all the pixels in this area as undetermined If the number of superpixels is equal to 1, the pending pixel is included in this superpixel.

Compared with the prior art, the present invention has the following advantages:

First, the present invention utilizes color features, spatial features and polarization scattering features of polarimetric SAR images to generate superpixels, so it is more suitable for superpixel generation of polarimetric SAR images.

Second, the present invention reduces the complexity of parameter adjustment and improves the accuracy of superpixel segmentation results because the proportion of undetermined pixels in the overlapping search area is adaptively adjusted by the degree of association between pixels in the overlapping search area.

Description of drawings

Fig. 1 is the realization flow chart of the present invention;

Fig. 2 is the simulation result figure that produces superpixel result on Flevoland image with the present invention;

Fig. 3 is the simulation result figure that produces superpixel on ESAR image with the present invention;

Fig. 4 is the simulation result figure that produces superpixel on San Francisco image with the present invention;

Detailed ways

The examples and effects of the present invention will be further described in detail below in conjunction with the accompanying drawings.

1, the specific implementation steps of the present invention are described in further detail:

Step 1. Set appropriate hyperparameters according to the data to be processed.

Obtain the polarimetric SAR image Flevoland with a size of 300 × 270, set the expected number of superpixels K in the image to be 500, the weight mf of the fuzzy similarity relationship is set to 0.6, the threshold E is set to 0.001, and the regularization parameter m of the membership degree is set to is 2, and the maximum number of iterations it is set to 500;

Step 2, initialize the cluster center, find out the non-overlapping search area and the overlapping search area.

(2a) According to the required number of superpixels, K cluster centers are selected at S pixel intervals on a regular grid, N is the total number of pixels in the image;

(2b) Move the cluster center to the lowest gradient position in the 3×3 neighborhood, and for each cluster center, set the size of the search area to find similar pixels to 2S×2S;

(2c) Form an overlapping search area with overlapping pixels in the search areas corresponding to two or more cluster centers; form a non-overlapping search area with pixels only included in the search area of one cluster center;

Step 3, generating fuzzy superpixels, that is, dividing the pixels into two parts: superpixels and undetermined pixels.

其中c_j表示此重叠区域内像素对应于超像素j的聚类中心，n为该重叠搜索区域内的像素总数，x_i表示该重叠搜索区域内的第i个像素点的特征，u_ij表示像素点i与其对应于超像素j的聚类中心的隶属度,m为隶属度的正则化参数，i，j取值都是从1到n；(3a) Update the corresponding cluster centers of pixels in the overlapping search area as:

where c _j represents the cluster center of the pixels in the overlapping area corresponding to superpixel j, n is the total number of pixels in the overlapping search area, x _i represents the feature of the i-th pixel in the overlapping search area, and u _ij represents The membership degree of the pixel point i and its cluster center corresponding to the superpixel j, m is the regularization parameter of the membership degree, and the values of i and j are from 1 to n;

其中c表示该重叠搜索区域内聚类中心的个数，D_ij为第i个像素到第j个像素的距离，D_ik为第i个像素到第k个聚类中心的距离，m为隶属度的正则化参数，k取值为从1到c；(3b) Update the membership between the i-th pixel and the j-th superpixel cluster center in the overlapping search area

where c represents the number of cluster centers in the overlapping search area, D _ij is the distance from the i-th pixel to the j-th pixel, D _ik is the distance from the i-th pixel to the k-th cluster center, and m is the membership Degree regularization parameter, k takes value from 1 to c;

(3b1) Calculate the spatial distance ds _ij from the i-th pixel to the j-th pixel and the color feature distance dc _ij :

where sx _i , sy _i represent the spatial features of the ith pixel, sx _j , sy _j represent the spatial features of the j th pixel, l _i , a _i , b _i represent the color features of the ith pixel in the color space lab , l _j , a _j , b _j represent the color feature of the jth pixel in the color space lab;

(3b2) According to the result of (3b1), calculate the distance D _ij between the i-th pixel and the j-th superpixel cluster center:

where dsc _ij =ds _ij +dc _ij is the sum of the spatial characteristic distance ds _ij and the color characteristic distance dc _ij , and r _ij represents the fuzzy similarity value between the ith pixel and the jth pixel calculated according to the polarization scattering information , mf is the weight of the fuzzy similarity value.

(3b3) Calculate the distance Di _ik from the i-th pixel to the k-th cluster center, and the calculation method is the same as that of D _ij ;

(3b4) Update the membership degree between the i-th pixel and the j-th superpixel cluster center in the overlapping search area

(3c) Repeat steps (3a) to (3b) until the maximum number of iterations it or the change of the cluster center between two iterations is less than the threshold E;

(3d) Calculate the fuzzy similarity relationship matrix

(3d1) Calculate the fuzzy similarity value between the ith pixel and the jth pixel:

Among them, fea _t represents the t-th channel of the SAR feature map, and fea _t max and fea _t min represent the maximum and minimum values in the t-th channel of the feature map, respectively. If r _ij is less than zero, then set r _ij =0, The value range of t is 1 to 9;

(3d2) Calculate the fuzzy similarity relationship value between all n pixels in the overlapping search area, and form a fuzzy similarity relationship matrix

(3e) Calculate the relevance matrix of overlapping search regions:

Among them, U is the membership matrix between all pixels in the overlapping search area and each cluster center, the size of the matrix is n×c, n is the number of pixels in the overlapping search area, and c represents the cluster center in the overlapping search area The number of , U ^T represents the transpose of the matrix U;

(3f) Calculate the degree of difference between cluster centers:

Among them, E _tq is the element of the t-th row and the q-th column of the correlation degree matrix E, and sum(diag(E)) represents the sum of the diagonal elements of the correlation degree matrix E;

(3g) Determine the proportion of undetermined pixels in the overlapping search area according to the equation P=0.5×Fand P<1;

(3h) Generate fuzzy superpixels according to the membership matrix U and the adaptively determined proportion P of undetermined pixels, that is, sort the largest elements of each row in the membership matrix U from large to small, and sort them before the n×Pth bit The pixels of , are classified into the superpixels corresponding to the column where the largest element of the row is located, and the pixels after the n×Pth bit are determined as undetermined pixels, where n is the number of rows of the membership matrix U.

Step 4: Perform category labeling according to the number of superpixels.

Taking each undetermined pixel as the center, in the M×M area around it, find the number of superpixels in this area:

If the number of superpixels is greater than 1, all the pixels in this area are marked as pending pixels;

If the number of superpixels is equal to 1, the pending pixel is classified into this superpixel;

If the number of superpixels is less than 1, no processing is performed.

The effects of the present invention will be further described in detail below in conjunction with simulation experiments.

1. Simulation experimental conditions

The simulation uses public data Flevoland, ESAR and San Francisco three polarimetric SAR images. The processor used for the simulation is AMD ^@ Ryzen5 2600, the simulation platform is windows10, and the simulation software is matlab 2018b.

The existing superpixel generation methods used in the simulation include: SLIC, LSC, USEAQ, LearnedS, FS.

2. Simulation experiment content and simulation experiment result analysis:

Simulation experiment one, using the present invention and the existing superpixel generation method to generate superpixels on the polarimetric SAR image Flevoland, and then use the superpixel results to classify, compare the classification accuracy, and generate the results when the number of superpixels is 500 Figure 2, where:

Fig. 2 (f) is the superpixel generation result on Flevoland data with the inventive method,

Figure 2(a) shows the result of superpixel generation on Flevoland data using the SLIC method,

Figure 2(b) is the result of superpixel generation on Flevoland data by LSC method,

Figure 2(c) shows the result of superpixel generation on Flevoland data using the USEAQ method,

Figure 2(d) shows the result of superpixel generation on Flevoland data using the LearnedS method,

Figure 2(e) shows the results of superpixel generation on Flevoland data using the FS method.

As can be seen from Figure 2, the superpixel generation result of the method of the present invention on Flevoland data is compared with the existing five methods, and all the pixels in the generated superpixels are the same type of superpixels. The class boundaries in the image fit better.

Statistical accuracy of classification using the results of the above six superpixel generation methods on Flevoland data, as shown in Table 1

Table 1. Comparison of classification accuracy using different superpixel generation methods in Flevoland

It can be seen from Table 1 that the classification accuracy using the superpixel results on Flevoland data of the present invention is higher than that of the existing SLIC, LSC, USEAQ, LearnedS, and FS methods, and the accuracy fluctuation range is smaller and more stable.

Simulation experiment 2, use the present invention and the existing superpixel generation method to generate superpixels on the polarimetric SAR image ESAR, then use the superpixel results to classify, compare the classification accuracy, and generate the results when the number of superpixels is 500 Figure 3, where:

Fig. 3 (f) is the superpixel generation result on ESAR data with the inventive method,

Figure 3(a) shows the result of superpixel generation on ESAR data using the SLIC method.

Figure 3(b) shows the result of superpixel generation on ESAR data by the LSC method,

Figure 3(c) shows the superpixel generation results on ESAR data using the USEAQ method,

Figure 3(d) shows the result of superpixel generation on ESAR data using the LearnedS method,

Figure 3(e) shows the result of superpixel generation on ESAR data using the FS method.

As can be seen from Figure 3, the superpixel generation result of the method of the present invention on ESAR data is compared with the existing five methods, and all the pixels in the generated superpixels are the same type of superpixel ratio higher, while the superpixel outline and The class boundaries in the image fit better.

Statistics on the classification accuracy of superpixel results on ESAR data using the above 6 methods, as shown in Table 2

Table 2. Comparison of the classification accuracy of ESAR superpixel results using different methods

It can be seen from Table 2 that the classification accuracy using the superpixel results on ESAR data of the present invention is higher than that of the existing SLIC, LSC, USEAQ, LearnedS, and FS methods, and the accuracy fluctuation range is smaller and more stable.

Simulation experiment three, use the present invention and the existing superpixel generation method to generate superpixels on the polarimetric SAR image San Francisco, and then use the superpixel results to classify, compare the classification accuracy, and generate the number of superpixels when 500. The results are shown in Figure 4, where:

Fig. 4 (f) is the superpixel generation result on San Francisco data with the inventive method,

Figure 4(a) is the result of superpixel generation on San Francisco data using SLIC method,

Figure 4(b) is the superpixel generation result on the San Francisco data using the LSC method,

Figure 4(c) is the result of superpixel generation on the San Francisco data using the USEAQ method,

Figure 4(d) shows the result of superpixel generation on the San Francisco data using the LearnedS method,

Figure 4(e) is the result of superpixel generation on San Francisco data by FS method,

As can be seen from Figure 4, the superpixel generation result of the method of the present invention on the San Francisco data is compared with the existing five methods, and all the pixels in the superpixels produced are the same type of superpixels. The ratio is higher, while the superpixel outline Fits more closely with the class boundaries in the image.

The accuracy of the above six methods when generating the same number of superpixels on San Francisco data is calculated, as shown in Table 3.

Table 3. Comparison of the classification accuracy of superpixel results in San Francisco using different methods

It can be seen from Table 3 that the classification accuracy using the superpixel results on the San Francisco data of the present invention is higher than that of the existing SLIC, LSC, USEAQ, LearnedS, and FS methods, and the accuracy fluctuation range is smaller and more stable.

To sum up, the performance of the present invention on the three polarimetric SAR images is higher than that of the existing superpixel generation methods.

Claims (3)

1. a superpixel generation method for polarimetric SAR image classification, is characterized in that, comprises as follows: (1) Set the desired number of superpixels K, the threshold E, the maximum number of iterations I, and the distance weight parameter mf; (2) Initialize the cluster centers of superpixels to find non-overlapping search areas and overlapping search areas; (3) Generate fuzzy superpixels, that is, divide the pixels into two parts: superpixels and undetermined pixels: (3a) Update the corresponding cluster centers of pixels in the overlapping search area as: where c _j represents the cluster center of the pixels in the overlapping area corresponding to superpixel j, n is the total number of pixels in the overlapping search area, x _i represents the feature of the i-th pixel in the overlapping search area, and u _ij represents The membership degree of pixel i and its cluster center corresponding to superpixel j, m is the regularization parameter of membership degree, and i takes value from 1 to n; (3b) Update the membership degree between the i-th pixel and the j-th superpixel cluster center in the overlapping search area as:

where c indicates that the overlapping search area is the overlapping part of the c search areas, D _ij is the distance from the i-th pixel to the j-th pixel, D _ik is the distance from the i-th pixel to the k-th cluster center, and m is The regularization parameter of the membership degree, the value of k is from 1 to c; (3c) Repeat steps (3a) to (3b) until the maximum number of iterations I or the change of the cluster center between two iterations is less than E; (3d) Calculate the fuzzy similarity relationship matrix:

in:

Represents the fuzzy similarity relationship value between the ith pixel and the jth pixel, fea _t represents the t-th channel of the SAR feature map, fea _t max, fea _t min represent the maximum value in the t-th channel of the feature map and The minimum value, if ri _ij is less than zero, then let ri _ij =0; (3e) Calculate the relevance matrix of overlapping search regions: where U is the matrix formed by the membership degrees between all pixels in the overlapping search area and each cluster center, and the size of the matrix is n×c; (3f) Calculate the degree of difference between cluster centers:

Where E _tq is the element of the t-th row and the q-th column of the correlation matrix E, and sum(diag(E)) represents the sum of the diagonal elements of the correlation matrix E; (3g) Determine the proportion of undetermined pixels in the overlapping search area according to the equation P=0.5×Fand P<1; (3h) Generate fuzzy superpixels according to the membership matrix U and the adaptively determined proportion P of undetermined pixels, that is, sort the largest elements of each row in the membership matrix U from large to small, and sort them before the n×Pth bit The pixel of , is classified into the superpixel corresponding to the column where the largest element of the row is located, and the pixel after the n×Pth bit is determined as the undetermined pixel, where n is the number of rows of the membership matrix U; (4) Taking each undetermined pixel as the center, in the surrounding M×M area, find the number of superpixels in this area: if the number of superpixels is greater than 1, then mark all the pixels in this area as undetermined If the number of superpixels is equal to 1, the pending pixel is included in this superpixel. 2. method according to claim 1, is characterized in that, in (2), initialize the cluster center of superpixel, find out non-overlapping search area and overlapping search area, realize as follows: (2a) According to the required number of superpixels, K cluster centers are selected at S pixel intervals on a regular grid,

N is the total number of pixels in the image; (2b) Move the cluster center to the lowest gradient position in the 3×3 neighborhood, and for each cluster center, set the size of the search area to find similar pixels to 2S×2S; (2c) Use the overlapping pixels in the search areas corresponding to two or more cluster centers to form an overlapping search area; use pixels only included in the search area of one cluster center to form a non-overlapping search area. 3. The method according to claim 1, wherein the distance D _ij between the pixels in (3b) is calculated as follows: (3b1) Calculate the spatial distance ds _ij from the i-th pixel to the j-th pixel and the color feature distance dc _ij :

where sx _i , sy _i represent the spatial features of the ith pixel, sx _j , sy _j represent the spatial features of the j th pixel, l _i , a _i , b _i represent the color features of the ith pixel in the color space lab , l _j , a _j , b _j represent the color feature of the jth pixel in the color space lab; (3b2) According to the result of (3b1), calculate the distance D _ij between the i-th pixel and the j-th pixel: where dsc _ij =ds _ij +dc _ij is the sum of the spatial characteristic distance ds _ij and the color characteristic distance dc _ij , and r _ij represents the fuzzy similarity value between the ith pixel and the jth pixel calculated according to the polarization scattering information , mf is the weight of the fuzzy similarity value.

Images