CN109741340B

CN109741340B - Ice cover radar image ice layer refined segmentation method based on FCN-ASPP network

Info

Publication number: CN109741340B
Application number: CN201811538248.8A
Authority: CN
Inventors: 蔡轶珩; 马杰; 胡绍斌; 李媛媛
Original assignee: Beijing University of Technology
Current assignee: Beijing University of Technology
Priority date: 2018-12-16
Filing date: 2018-12-16
Publication date: 2020-10-16
Anticipated expiration: 2038-12-16
Also published as: CN109741340A

Abstract

An ice cover radar image ice layer fine segmentation method based on an FCN-ASPP network relates to the field of computer vision and pattern recognition. The invention takes the radar amplitude image as a training sample of the network, and correspondingly performs data amplification aiming at the problem of less ice layer image data, thereby expanding the wide applicability of the invention. Lee filtering was performed on the ice-covered image. In order to save the edge information as much as possible, a threshold judgment process is added to the filtering process. According to the invention, the ice layer segmentation depth network of the FCN-ASPP is constructed, and the ASPP layer is improved, so that the extraction capability of the network on small-scale features is enhanced. And further processing the preliminary classification result by CRF, and further refining the segmentation result on the basis of realizing end-to-end pixel-level segmentation. In addition, the network greatly realizes the process of autonomous learning.

Description

Ice cover radar image ice layer refined segmentation method based on FCN-ASPP network

Technical Field

The invention belongs to the field of computer vision and pattern recognition, and relates to a method for ice cover radar image ice layer segmentation based on deep learning.

Background

In recent years, global warming has severely threatened the living environment on which we live. In recent decades, ice caps in the south pole have accelerated melting, causing the sea level to rise and have a considerable effect on ocean currents. And may even cause serious geological disasters. Therefore, the data of the thickness, distribution and the like of the polar ice cover and the information of how the data change along with time can be effectively understood and predicted to influence of the glacier melting. The radar sensor is one of instruments capable of penetrating an ice layer and providing large-area under-ice terrain information. Because air, ice and rock have different dielectric constants, the backscatter is different when the radar wave sees through different medium objects, consequently can be distinguished in the radar echo map, and then extracts key information such as ice sheet thickness, ice cover underground structure distribution.

The traditional radar classification image segmentation methods at present are divided into three types, namely segmentation based on a model, segmentation based on an edge and segmentation based on a region. However, these methods also have the problems of dependence on prior knowledge, manual setting of segmentation parameters, insufficient segmentation refinement and the like. Moreover, these methods typically require manual extraction of features, which require a significant amount of time to adjust to the new data set.

Compared with the traditional method, the deep learning method can automatically learn proper feature representation aiming at the current problem, and saves the labor time cost required by manually extracting the features. The full convolution neural network (FCN) is an end-to-end segmentation method developed by deep learning, has pixel-level segmentation capability, can achieve better segmentation effect than machine learning through a large amount of data learning, almost all parameters are obtained through continuous iterative learning, and therefore the number of parameters needing to be adjusted is small. However, the FCN segmentation network also has the defect of insufficient segmentation refinement, and the method improves the FCN segmentation network on the basis of the FCN segmentation network, strengthens the refinement segmentation capability of the network, and enables the FCN segmentation network to better meet the requirement of ice layer radar image segmentation.

Disclosure of Invention

The invention provides a segmentation method of an ice cover radar map based on a deep learning network.

1. Pretreatment of

The method comprises the steps of carrying out targeted preprocessing on an ice cover radar map according to characteristics of the ice cover radar map, and firstly converting a radar amplitude image into a 255-level gray scale map. Since different media in the ground reflect radar waves differently, different texture features and gray scale features appear in the converted gray scale map, which can be used as a basis for classification later.

The original radar amplitude image is acquired by an airplane airborne radar, and the radar image needs to be preprocessed according to the characteristics of the radar image, so that the acquired radar amplitude image is more suitable for ice cover segmentation, the accuracy of an algorithm is improved, and the logarithm conversion is performed on the pixel value of the original radar amplitude image. And calculating the converted pixel value corresponding to the radar amplitude. Then, the images are normalized to 255 pixel levels.

Due to the imaging mode specific to the radar system, a great amount of granular speckle noise exists in the radar original image, and the noise causes serious interference to segmentation. Therefore, the invention adopts the lee filtering algorithm to further process the radar gray level image, thereby eliminating the influence of speckle noise on classification. However, although the image after the traditional lee filtering eliminates the influence of speckle noise, the edge part of the image becomes fuzzy, and the boundary becomes difficult to be segmented. Therefore, the invention adds a threshold judgment process to the filtering process, introduces the variation coefficient C to judge whether the edge is the edge, and does not carry out filtering processing on the edge part to store the edge characteristic.

The filtered grayscale image is clipped to a size of 500 × 389 as an input training image. In addition, label images having the same size as the input training image are manually labeled, and noise (niose), an ice layer (layer), a bedrock layer (bedrock), and an air layer (air) are respectively labeled as

pixel gradations

0, 1, 2, and 3.

2. Data amplification and construction of training samples

Since training deep networks requires a large amount of data, the existing data alone is far from sufficient for training. In order to improve the training and detection effects of the algorithm, corresponding data amplification is performed for the problem that the image data of the ice cover is few. The method comprises translation transformation, scaling and the like, so that the data volume is increased, the learning and training of the method are facilitated, and the applicability and the generalization of the method are improved. The data amplification mode is as follows:

1) and (3) translating the preprocessed image in four directions of left, right, up and down, wherein the translation distance is 20 pixels, so that the network learns the translation invariance in the training process.

2) The preprocessed image is respectively reduced to 50% and 75% of the original image, and the sample data is further amplified.

3. FCN-ASPP network construction

The invention constructs the FCN-ASPP network.

Although the fcn (full volumetric network) network can realize end-to-end pixel-level image segmentation, a large amount of detail information is lost in a training picture through 5 downsampling processes due to a large number of downsampling links in an early stage, so that a classification result is not accurate enough. Therefore, the method simplifies the down-sampling link in the FCN, changes 5 times of down-sampling into 3 times of down-sampling processes, adds a BN (batch normalization) layer after each convolution process, performs batch standardization operation on data, and accelerates the convergence speed of the network. The specific network structure comprises an input layer, two convolution layers (C1, C2), two BN layers (B1, B2), two activation function layers (R1, R2), a first down-sampling layer (pool1), two convolution layers (C3, C4), two BN layers (B3, B4), two activation function layers (R3, R4), a second down-sampling layer (pool2), two convolution layers (C5, C6), two BN layers (B5, B6), two activation function layers (R5, R6) and a third down-sampling layer (pool 3). And (4) inputting training pictures into a down-sampling layer, and obtaining feature maps with different sizes through a down-sampling process to be used as the input of a next layer of network.

Because the edge part of the ice cover image is extremely irregular, in order to further enhance the classification capability of the network on the detail features, the classification accuracy of the ice layer is increased. A multi-scale expanded pyramid structure (Atrous Spatial pyramiding) was constructed herein. The ASPP network provided by the method comprises five parallel convolutional layers, and a common convolutional layer is added on the basis of the ASPP network with only four parallel expansion convolutional layers to better extract detailed information, so that the ASPP network with five expansion convolutional layers with different expansion rates is formed, and five parallel branch expansion convolutional characteristic graphs with different scales are obtained. And then the feature maps are fused into one feature map for subsequent processing. The ASPP network improves the perception degree of the whole network to local detail information.

4. Refined segmentation result

In order to further mine the relationship between pixels in the training image to improve the segmentation accuracy. And (4) performing detail processing on the training image and the segmentation result graph obtained by training as well as the input of a Markov Conditional Random Field (CRF) to obtain a final ice cover segmentation image.

Advantageous effects

1. The invention can rapidly carry out automatic feature extraction on the ice layer image by a method for constructing the FCN-ASPP network, can carry out feature extraction on the ice layer image from different levels, learns the relationship between each pixel in the ice layer image and a plurality of neighborhoods around the pixel, well expresses high-level features in the ice cover image and extracts more detailed information. The invention increases BN network layer to normalize data after each convolution layer, and improves network convergence. Therefore, the internal characteristics of the ice layer, the bedrock and the noise are better distinguished, end-to-end and pixel-level ice layer segmentation is realized, and the classification judgment efficiency is greatly improved compared with that of the traditional single pixel.

2. The invention further processes the tail end output of the FCN-ASPP network and the original image, and further excavates the relation between each pixel in the ice layer image and a plurality of neighborhoods around the pixel by using a Markov condition random field to improve the classification accuracy. So that the ice layer image segmentation result graph and the manual labeling segmentation result reach good consistency. The invention realizes the automation of ice layer image segmentation and greatly reduces the consumption of manpower and material resources.

3. The invention adopts the idea of multi-level feature extraction, fully utilizes the characteristics of high feature resolution of shallow features and embodies the characteristics of stronger semantic information of deep features. Therefore, refinement and robustness of ice layer segmentation are realized.

4. The invention improves the ASPP network structure, and adds a common convolution layer branch on the basis of four parallel expansion convolution layer branches of the original ASPP network structure. The extraction capability of the network for small-size features is further enhanced.

5. The invention carries out Lee filtering processing on the radar image. In order to ensure that the edge information of the image is stored, the invention adds a threshold judgment process to the filtering process, introduces a variation coefficient, and carries out filtering processing on the coefficient when the coefficient is lower than the threshold, otherwise, the original pixel value is kept. Thereby improving the robustness of the segmentation effect.

Drawings

FIG. 1 is an overall flow diagram of the present invention;

FIG. 2 is a schematic diagram of a dilated convolution jump structure;

FIG. 3 is a schematic diagram of a different expansion coefficient expansion convolution kernel;

FIG. 4 is an ASPP network structure;

FIG. 5 is an FCN-ASPP network structure;

FIG. 6 is the ice layer segmentation result;

Detailed Description

The following detailed description is made with reference to the accompanying drawings:

the technical block diagram of the invention is shown in figure 1. The specific implementation steps are as follows:

1. pretreatment of

In the first step, the acquired radar amplitude image is subjected to logarithmic conversion, and the radar amplitude a is calculated through a formula (1)_iCorresponding pixel value b_i

b_i＝20×log10(a_i) (1)

And in the second step, the image is normalized by formula (2) to be 255 pixel levels.

Wherein, b_iIs the logarithmically converted pixel value, c_iIs the normalized pixel value, max is the maximum value of the pixel before normalization, min is the minimum value of the pixel before normalization.

And thirdly, Lee filtering processing is carried out on the original image, coherent speckle noise of the radar image is eliminated, and the obtained image is stored as a training image. Speckle noise is a multiplicative noise, and is modeled as

y＝xv (3)

Where y represents the image after noise addition, x represents the image without noise addition, and v is the noise signal. A moving window of 7x7 was designed as a filter to filter the image. In order to prevent the edge blurring caused by Lee filtering from adversely affecting the segmentation, a simple threshold judgment is added, a variation coefficient C is introduced, and the threshold C of the image is firstly obtained

N is the view number of the radar image, and the variation coefficient C corresponding to the window center pixel I can be obtained by a formula_i。var(y_i) Is the variance of the pixels within the window,

is the mean of the pixels within the window.

When C is present_iAnd when the C is less than or equal to C, adopting a Lee filtering method. When C is present_iAnd when the pixel is more than C, the pixel is reserved and is not subjected to filtering processing.

The fourth step clips the normalized grayscale image to a size of 500 × 389 as the input training image. In addition, label images having the same size as the input training image are manually labeled, and noise (niose), an ice layer (layer), a bedrock layer (bedrock), and an air layer (air) are respectively labeled as

pixel gradations

0, 1, 2, and 3.

2. Data amplification and construction of training samples

Since training a deep network requires a large amount of data, only existing ice layer images are far from sufficient for training. Therefore, different modes of data amplification need to be performed on training data, so that the data volume is increased, and the training and detection effects are improved.

In the first step, the preprocessed image is translated into 20 pixels respectively in left, right, up and down directions, and the translation invariance of network learning is realized.

And in the second step, the preprocessed image and the image processed in the first step are subjected to reduction processing, and the image is reduced to 50% and 75% of the original image, so that the data is amplified.

And thirdly, expanding the same data of the label images (label) divided by the ice layer, thereby obtaining classification labels corresponding to the samples one by one. 3/4 of the constructed training sample data is used as a training set, and 1/4 is used as a verification set.

3. FCN-ASPP network construction and training and testing process

First step of building FCN-ASPP network

The FCN-ASPP network is constructed on the basis of the FCN network, the two-stage convolution layer and the pooling layer of the down-sampling part of the FCN network are removed, and the ASPP layer is added between the down-sampling part and the up-sampling part of the FCN network.

The network structure specifically comprises an input layer, two convolution layers (C1, C2), two BN layers (B1, B2), two activation function layers (R1, R2), a first downsampling layer (pool1), two convolution layers (C3, C4), two BN layers (B3, B4), two activation function layers (R3, R4), a second downsampling layer (pool2), two convolution layers (C5, C6), two BN layers (B5, B6), two activation function layers (R5, R6), and a third downsampling layer (pool 3). The convolution operation step size is carried out by applying two tightly connected 3 × 3 filter convolution kernels in the downsampling layers, the convolution is 1²The data is then normalized too much by the calculated mean and variance. The standardization process adopts a transformation reconstruction method.

y＝γx+β (6)

x is original data, y is normalized data, and the conversion parameters gamma and beta are obtained through training and learning. And (4) carrying out linear transformation on the input data of the BN layer through conversion parameters gamma and beta to obtain a final normalization result. The structure of the pooling layer is a max-pooling layer with a core of 2X 2 and a step size of 2. And inputting training pictures into a down-sampling layer, and obtaining feature maps with different sizes through a plurality of down-sampling processes.

ASPP layer section: because the low-level feature resolution of the FCN is high, and the high-level information embodies stronger semantic information, in order to fully utilize convolution features of different levels in a deep network, the original FCN is further improved, and output feature maps of pool1, pool2 and pool3 are used as the input of four parallel expansion convolution layers (C7-1, C7-2, C7-3 and C7-4) of the ASPP. Each layer respectively passes through two convolution layers, two activation function layers, a pooling layer and a full-connection layer, and a BN layer is added behind each convolution layer for normalizing data. (C7-1, BN4-1, R7-1, C8-1, BN5-1, R8-1, pool4-1, FC5-1), (C7-2, BN4-2, R7-2, C8-2, BN5-2, R8-2, pool4-2, FC5-2), (C7-3, BN4-3, R7-3, C8-3, BN 8-3, R8-3, pool 8-3, FC 8-3), (C8-4, BN 8-4, R8-4, C8-4, BN 8-4, R8-4, pool 8-4, FC 8-4), (C8-5, R8-5, C8-5, R8-5, and R8-5, C8-5. And fusing the FC5-1, FC5-2, FC5-3, FC5-4 and FC5-5 layer output characteristic maps to obtain a fused characteristic map (session).

The convolution layer of the ASPP layer adopts an expansion convolution method, and the edge part of the ice cover image is extremely irregular, so that the classification capability of the network on the detail characteristics is further enhanced, and the classification accuracy of the ice layer is improved. A multi-scale expanded Pyramid structure (Atrous Spatial Pyramid) was constructed herein. The ASPP network provided by the method comprises five parallel convolutional layers, and a common convolutional layer is added on the basis of the ASPP network with only four parallel expansion convolutional layers to better extract detailed information, so that the ASPP network with five expansion convolutional layers with different expansion rates is formed, and five parallel branch expansion convolutional characteristic graphs with different scales are obtained. These feature maps are then fused into one feature map. In the part, a BN layer is added after each convolution layer for data normalization, so that the convergence speed of the network is improved.

An up-sampling part: including an upsampling layer (upsampling) and an output layer. And taking the fusion characteristic graph obtained by the three ASPP layers as an up-sampling layer for input, and performing up-sampling through a bilinear interpolation algorithm. And fusing the feature maps into a feature map, and classifying pixels by a softmax classifier to obtain an ice cover segmentation image.

Training process: inputting the glacier radar image and the classification label image into an FCN-ASPP network to obtain a classification result image with the same size as the original image. And the loss function adopts a cross entropy function, and the network optimization is carried out by minimizing the difference value of the input label image and the classification predicted image. And storing the trained model.

4 refinement of the segmentation results

And (4) taking the segmentation result graph and the original gray level image obtained by the output of the front end FCN-ASPP as the input of a Markov Conditional Random Field (CRF) for further processing. The energy function used by the fully-connected CRF is

And the pixel x belongs to N, and N is all pixels of the picture. The first term of the formula (7) is

Is a unitary energy function, in which P_i(x) And the distribution probability of a classification mark i corresponding to the pixel x in a segmentation result graph obtained by front-end FCN-ASPP output is shown, and the value range of i in the invention is 0, 1, 2 and 3.

The second term of the formula (7) is

Is a binary energy function. x, y denote two different pixels, p_x，p_yRepresenting the x, y position, I, of the pixel_x，I_yAnd expressing the x, y gray scale of the pixel, and the gray scale and the pixel position of the pixel are obtained from the input original gray scale image. Mu.s_i(x, y) is a label compatibility function, where the label of pixel x is i, when pixel_yWhen they belong to the class i tag, mu_i(x, y) is 0, otherwise, is 1, pixel_yThe label of (2) is obtained from a segmentation result graph output by the front end FCN-ASPP. The closer the pixels x and y are, the smaller the gray scale difference,

the larger the value.

Four energy functions E of the pixel x are obtained through calculation₀(x),E₁(x),E₂(x),E₃(x) The label corresponding to the minimum energy value is taken as the new label of the pixel x. And traversing all pixels of the input segmentation result graph to obtain a new and more accurate segmentation result graph. Setting parameters according to the characteristics of the radar image, wherein the parameter omega is₁，ω₂Set to 49, parameter σ_α，σ_γSet to 5, parameter σ_βSet to 3.

Claims

1. An ice cover radar image ice layer fine segmentation method based on an FCN-ASPP network is characterized by comprising the following steps:

1) pretreatment of

Carrying out logarithmic conversion on the pixel value of the original radar amplitude image; calculating a converted pixel value corresponding to the radar amplitude; then, carrying out normalization processing on the image, and normalizing the image into 255 pixel levels;

adopting a lee filtering algorithm to further process the radar gray level image;

taking the filtered gray level image as an input training image; label images with the same size as the input training images are marked, and noise, an ice layer, a basement layer and an air layer are respectively marked as pixel gray levels of 0, 1, 2 and 3;

2) data amplification and construction of training samples

3) FCN-ASPP network construction

The FCN network structure specifically comprises an input layer, two convolution layers (C1, C2), two BN layers (B1, B2), two activation function layers (R1, R2), a first down-sampling layer (pool1), two convolution layers (C3, C4), two BN layers (B3, B4), two activation function layers (R3, R4), a second down-sampling layer (pool2), two convolution layers (C5, C6), two BN layers (B5, B6), two activation function layers (R5, R6) and a third down-sampling layer (pool 3);

the output characteristic diagrams of pool1, pool2 and pool3 are used as the input of four parallel expansion convolutional layers of ASPP, namely C7-1, C7-2, C7-3 and C7-4 and the input of a common convolutional layer C7-5; each layer respectively passes through two convolution layers, two activation function layers R, a pooling layer pool and a full connection layer FC, and a BN layer is added behind each convolution layer for normalizing data; the whole ASPP layer is: (C7-1, BN4-1, R7-1, C8-1, BN5-1, R8-1, pool4-1, FC5-1), (C7-2, BN4-2, R7-2, C8-2, BN5-2, R8-2, pool4-2, FC5-2), (C7-3, BN4-3, R7-3, C8-3, BN 8-3, R8-3, pool 8-3, FC 8-3), (C8-4, BN 8-4, R8-4, pool 8-4, FC 8-4), (C8-5, R8-5, C8-5, BN 8-5, R8-36; fusing FC5-1, FC5-2, FC5-3, FC5-4 and FC5-5 layer output characteristic diagrams to obtain fused characteristic diagrams;

inputting a fusion characteristic diagram obtained by the three ASPP layers as an up-sampling layer, and performing up-sampling through a bilinear interpolation algorithm; then, the feature maps are fused into a feature map, and pixels are classified through a softmax classifier to obtain an ice layer segmentation image;

inputting the glacier radar image and the classification label image into an FCN-ASPP network in the training process to obtain a classification result graph with the same size as the original image; the loss function adopts a cross entropy function, and network optimization is carried out by minimizing the difference value between the input label image and the classified prediction image; storing the trained model;

4) and further processing the preliminary segmentation result obtained by the FCN-ASPP by adopting a CRF algorithm.

2. The method of claim 1, wherein: the Lee filtering process specifically includes:

designing a 7x7 moving window as a filter to filter the image; adding a simple threshold judgment, introducing a variation coefficient C, and firstly calculating the threshold C of the image

N is the view number of the radar image, and the variation coefficient C corresponding to the window center pixel I is calculated by a formula_i；var(y_i) Is the variance of the pixels within the window,

is the mean value of the pixels in the window;

when C is present_iWhen the C is less than or equal to C, adopting a Lee filtering method; when C is present_iAnd when the pixel is more than C, the pixel is reserved and is not subjected to filtering processing.

3. The method of claim 1, wherein: the CRF treatment process specifically comprises the following steps;

taking a segmentation result image and an original gray level image output by the front end FCN-ASPP as the input of a Markov Conditional Random Field (CRF) for further processing; the energy function used by the fully-connected CRF is

The pixel x belongs to N, and N is all pixels of the picture; the first term of the formula is

Is a unitary energy function, in which P_i(x) Representing the distribution probability of a classification mark i corresponding to a pixel x in a segmentation result graph obtained by front-end FCN-ASPP output, wherein the value range of i is 0, 1, 2 and 3;

the second term of the formula is

Is a binary energy function; x, y denote two different pixels, p_x，p_yRepresenting the x, y position, I, of the pixel_x，I_yExpressing the x and y gray scales of the pixel, wherein the pixel gray scale and the pixel position are obtained from an input original gray scale image; mu.s_i(x, y) is a label compatibility function, where pixel x has a label of i, and when pixel y belongs to the i-class label, μ_i(x, y) takes a value of 0, otherwise, takes a value of 1, and the label of the pixel y is obtained by a segmentation result graph output by the front end FCN-ASPP; the closer the pixels x and y are, the smaller the gray scale difference,

the larger the value;

four energy functions E of the pixel x are obtained through calculation₀(x),E₁(x),E₂(x),E₃(x) Taking the label corresponding to the minimum energy value as the new label of the pixel x; traversing all pixels of the input segmentation result graph to obtain a new segmentation result graph with a parameter omega₁，ω₂Set to 49, parameter σ_α，σ_γSet to 5, parameter σ_βSet to 3.