WO2022083202A1

WO2022083202A1 - Fine water body extraction method based on u-net neural network

Info

Publication number: WO2022083202A1
Application number: PCT/CN2021/108847
Authority: WO
Inventors: 蔡玉林; 秦鹏; 孙孟昊; 王雪丽; 柳佳; 范璞然; 李龙坤; 徐慧宇
Original assignee: 山东科技大学
Priority date: 2020-10-20
Filing date: 2021-07-28
Publication date: 2022-04-28
Also published as: CN112232229A; CN112232229B

Abstract

Disclosed is a fine water body extraction method based on a U-net neural network, which method relates to the technical field of convolutional neural networks and water body extraction, and particularly relates to water body extraction by means of hyperspectral data. The method comprises: importing original images of all wavebands into ENVI for principal component analysis; forming a variety of combinations of different principal components; forming label data; dividing an optimal remote-sensing image into training data and test data; inputting all the training data into a U-net neural network for training; inputting the test data of the optimal remote-sensing image into the trained U-net neural network, so as to obtain an output image; performing threshold value segmentation and splicing on the output image, and restoring the output image to an original size; and comparing the output image which has been restored to the original size with the test data in the label data, so as to evaluate the precision of fine water body extraction.

Description

A Fine Water Body Extraction Method Based on U-net Neural Network

technical field

The invention discloses a fine water body extraction method based on U-net neural network, and belongs to the technical field of convolution neural network and water body extraction.

Background technique

With the continuous development of remote sensing technology, the use of remote sensing methods to automatically extract surface water information has become a hot spot in global information extraction research. Due to the different spectral reflection characteristics of water bodies near the near-infrared band, the water body can be identified by setting a threshold, but this method using a single band is difficult to completely distinguish the background from the water body; the combination of multiple bands is used for ratio calculation and index extraction. The method can highlight the water body information, but it is only suitable for large-scale water body extraction, and the recognition effect for some small water bodies is not ideal.

SUMMARY OF THE INVENTION

The invention discloses a fine water body extraction method based on U-net neural network, so as to solve the problem that the recognition effect of small water bodies in remote sensing images is not good in the prior art.

Fine water extraction method based on U-net neural network, including:

S1. Import the original images of all the bands into ENVI, and perform principal component analysis, so that the bands with strong correlation are converted into bands with weak correlation, and the bands with weak correlation are used as principal components;

S2. Perform multiple combinations of different principal components, each combination will synthesize a multi-band remote sensing image, and input the multi-band remote sensing images into the U-net network for testing to determine the multi-band remote sensing that can obtain the best water body classification effect. image, called the optimal remote sensing image;

S3. Visually interpret the water body of the optimal remote sensing image, use ENVI to delineate the area of interest, label the water body, and form label data;

S4. Divide the optimal remote sensing image into training data and test data, and use a part of the data in the training data as a validation set; divide the label data in the same way as the optimal remote sensing image in S4;

S5. Input all the training data into the U-net neural network for training. After convolution operation and maximum pooling downsampling operation, the first water body feature is obtained; after convolution operation and upsampling operation, the second water body feature is obtained. ; Connect the above two water body features to extract the final water body features; compare the final water body features with the real water body features, and continuously optimize the network parameters, so that the water body output by the U-net neural network is constantly close to the real water body, and the U-net neural network output is continuously close to the real water body. - training of net neural network;

S6. Input the test data of the optimal remote sensing image into the U-net neural network after training to obtain the output image; perform threshold segmentation and splicing, and restore the output image to the original size;

S7. Compare the output image restored to the original size with the test data in the label data to evaluate the precision of fine water body extraction.

Preferably, in the step S1, the original image format is .tiff, and the parameter used in the principal component analysis is a covariance matrix. At the maximum, output bands that are uncorrelated with each other are generated.

Preferably, in the step S1, after the principal components are formed, ENVI is output in the following order: the variance of the first principal component is the largest, the variance of the second principal component is the second largest, and so on, the final principal component has the smallest variance.

Preferably, in the step S3, the label data sets the water body value to 1 and the non-water body value to 0.

Preferably, in the step S4, the verification set does not participate in the training, and is used to calculate the accuracy of the U-net neural network after the training is completed.

Preferably, before the step S5 is performed, the training data of the optimal remote sensing image and the training data corresponding to the label data are divided, and a 64*64 image is output as the training data in step S5.

Preferably, in the step S5, the U-net neural network is deepened, and the deepening includes a 3×3 convolutional layer, a 2×2 downsampling layer, a 2×2 upsampling layer and a skip connection layer , after the convolution operation is performed, the Relu activation function is used.

Preferably, in the step S5, the setting of the U-net neural network uses the Adam optimizer to dynamically set the learning rate; each layer of the U-net neural network uses a random deactivation method to prevent overfitting, use Batch normalization prevents gradients from disappearing.

Preferably, in the step S5, the generations used for U-net neural network training are divided into several batches, and the training accuracy and the verification accuracy are respectively output after the training of one batch is completed. If the accuracy of a certain number of generations is continuously maintained at 90% The above and no major changes occur, then stop the training of the U-net neural network.

Preferably, before the step S6 is executed, the test data of the optimal remote sensing image is divided into 64*64 images; in the threshold segmentation, the value greater than 0.1 is assigned a value of 1, indicating a water body, and the rest is assigned a value of 0, indicating that non-water bodies.

Preferably, in the step S7, the recall rate and the accuracy rate are used as the precision evaluation criteria: the recall rate=(the number of pixels of the water body correctly identified by the neural network)/(the actual number of pixels of the water body), the accuracy rate=( The number of pixels that the neural network correctly identifies the water body)/(the total number of pixels that the neural network recognizes the water body).

Compared with the prior art, the present invention has the following beneficial effects: the depth of the U-net neural network is deepened, so that it can adapt to hyperspectral data with more bands; the method of principal component analysis is used to reduce the dimension of the input data and ensure that the information is as far as possible No loss; about 90% of the water body recognition rate is achieved on the 10-meter-resolution remote sensing data, some small water bodies are identified, and there is no misclassification with roads, buildings, vegetation, etc.

Description of drawings

Figure 1. Principal component analysis parameter setting diagram;

FIG. 2 is a comparison diagram of the actual water body distribution and the water body extracted by the method of the present invention.

Detailed ways

Below in conjunction with specific embodiment, the present invention is described in further detail:

Fine water extraction method based on U-net neural network, including:

S1. The data of this embodiment are selected from the Zhuhai No. 1 hyperspectral data in the area near Taihu Lake, Suzhou City, Jiangsu Province. Since all 32 bands are input into neural network training, it is easy to cause memory overflow and cannot complete the training. Therefore, the format of the 32 bands is The original image of .tiff is imported into ENVI, and the principal component analysis function is selected to perform principal component analysis; the parameter is the covariance matrix, and the parameter setting is shown in Figure 1; the band with strong correlation is converted to the band with weak correlation, and the correlation is weak The wavebands are used as principal components; the process of principal component analysis is as follows: establish a coordinate system with the origin as the mean value of the data, maximize the variance of the data through the rotation of the coordinate axis, and generate uncorrelated output bands; after forming the principal components, ENVI follows the Output in the following order: the first principal component has the largest variance, the second principal component has the second largest variance, and so on, the last principal component has the smallest variance, and the smaller the variance, the less information the principal component contains. The processing preserves the original multi-band information and has a good control over the data dimension, reducing the amount of computation and shortening the computation time.

S2. After performing principal component analysis, generally the first few principal components basically contain all the information of the original data, respectively input the first few principal components into the U-net network for testing, and perform various combinations of different principal components, each combination A multi-band remote sensing image is synthesized, and the multi-band remote sensing image is input into the U-net network for testing, and the multi-band remote sensing image that can obtain the best water classification effect is determined, which is called the optimal remote sensing image; the comparison results are shown in Table 1. It is shown that the classification effect achieved in the multi-band image input network composed of the first, second, third, fourth, fifth and sixth principal components is better, and the accuracy of the increase of the number of bands is smaller and the computational cost is larger.

Table 1 Classification accuracy of the first few principal components

主成分main ingredient	1-3主成分1-3 principal components	1-4主成分1-4 Principal Components	1-5主成分1-5 principal components	1-6主成分1-6 Principal Components	1-7主成分1-7 Principal Components	1-8主成分1-8 principal components
精度precision	86.41％86.41%	87.23％87.23%	89.45％89.45%	92.32％92.32%	92.65％92.65%	92.89％92.89%

S3. Visually interpret the optimal remote sensing image for water bodies, use ENVI to delineate areas of interest, and label water bodies to form label data; the label data is set to 1 for water bodies and 0 for non-water bodies.

S4. Divide the optimal remote sensing image into training data and test data. About 3/4 of the upper left corner is the training data, and the remaining 1/4 is the test data; 15% of the data in the training data is used as the validation set; The optimal remote sensing image is divided in the same way; the verification set does not participate in the training, and is used to calculate the accuracy of the U-net neural network after the training is completed; due to the large size of the original image, in order to reduce and prevent memory overflow, before the execution of step S5, The training data of the optimal remote sensing image and the training data corresponding to the label data are divided, and 64*64 images are output for inputting into the network for training, and finally 1640 pieces of training data are obtained as the training data in step S5.

S5. In order to make full use of the advantages of Zhuhai No. 1 hyperspectral data, extract useful information of each band, and deepen the U-net neural network. After deepening, it contains 22 3×3 convolutional layers, 5 2 ×2 downsampling layer, 5 2×2 upsampling layers and 5 skip connection layers, after the convolution operation is performed, the Relu activation function is used; the setting of the U-net neural network uses the Adam optimizer, and the learning rate is adjusted. Dynamic settings; each layer of the U-net neural network uses random deactivation to prevent overfitting and batch normalization to prevent vanishing gradients.

The training data is input into the U-net neural network for training once for one generation, with a total of 300 generations. In order to prevent memory overflow, every 8 generations is a batch. The accuracy of the neural network training is judged. The highest accuracy is 100%. If the accuracy of 10 generations is maintained above 90% continuously without major changes, the training of the U-net neural network will be stopped.

Input all training data into U-net neural network for training, after 11 convolution operations and 5 maximum pooling downsampling operations, the first water feature is obtained; after 11 convolution operations and 5 upsampling operations , obtain the second water body feature; connect the above two water body features to extract the final water body feature; compare the final water body feature with the real water body feature, and continuously optimize the network parameters to make the water body output by the U-net neural network match the real water body. The water body is constantly approaching, and the training of the U-net neural network is completed; after the training, the final training accuracy is 94.21%, and the verification accuracy is 92.32%.

The test data of the optimal remote sensing image is divided into 64*64 images, and a total of 620 pieces of test data are obtained; the test data of the optimal remote sensing image is input into the trained U-net neural network to obtain the output image; the output image is thresholded Segmentation, splicing, and restoration to the original size; in the threshold segmentation, the value greater than 0.1 is assigned a value of 1, indicating a water body, and the rest is assigned a value of 0, indicating a non-water body.

S7. Compare the output image restored to the original size with the test data in the label data, and the comparison result is shown in Figure 2; to evaluate the precision of fine water extraction, the recall rate and accuracy rate are used as the precision evaluation criteria: the recall rate display The completeness of the water body identification, the recall rate = (the number of pixels that the neural network correctly identifies the water body)/(the actual number of water body pixels); the accuracy rate shows the correct rate of the water body identified by the neural network, and the accuracy rate = ( The number of pixels that the neural network correctly identifies the water body)/(the total number of pixels that the neural network recognizes the water body); the final result shows that the recall rate of the U-net neural network is 89.03%, and the accuracy rate is 89.50%.

Of course, the above description is not intended to limit the present invention, and the present invention is not limited to the above examples. Changes, modifications, additions or substitutions made by those skilled in the art within the essential scope of the present invention should also belong to the present invention. the scope of protection of the invention.

Claims

The fine water body extraction method based on U-net neural network is characterized in that, it includes:

S1. Import the original images of all the bands into ENVI, and perform principal component analysis, so that the bands with strong correlation are converted into bands with weak correlation, and the bands with weak correlation are used as principal components;

S2. Perform multiple combinations of different principal components, each combination will synthesize a multi-band remote sensing image, and input the multi-band remote sensing images into the U-net network for testing to determine the multi-band remote sensing that can obtain the best water body classification effect. image, called the optimal remote sensing image;

S3. Visually interpret the water body of the optimal remote sensing image, use ENVI to delineate the area of interest, label the water body, and form label data;

S4. Divide the optimal remote sensing image into training data and test data, and use a part of the data in the training data as a validation set; divide the label data in the same way as the optimal remote sensing image in S4;

S5. Input all the training data into the U-net neural network for training. After convolution operation and maximum pooling downsampling operation, the first water body feature is obtained; after convolution operation and upsampling operation, the second water body feature is obtained. ; Connect the above two water body features to extract the final water body features; compare the final water body features with the real water body features, and continuously optimize the network parameters, so that the water body output by the U-net neural network is constantly close to the real water body, and the U-net neural network output is continuously close to the real water body. - training of net neural network;

S6. Input the test data of the optimal remote sensing image into the U-net neural network after training to obtain the output image; perform threshold segmentation and splicing, and restore the output image to the original size;

S7. Compare the output image restored to the original size with the test data in the label data to evaluate the precision of fine water body extraction.
The method for fine water body extraction based on U-net neural network according to claim 1, characterized in that, in the step S1, the original image format is .tiff, and the parameter used in the principal component analysis is a covariance matrix, and the process is: : Establish a coordinate system with the origin as the mean value of the data, and maximize the variance of the data through the rotation of the coordinate axis to generate uncorrelated output bands.
The method for fine water body extraction based on U-net neural network according to claim 2, characterized in that, in the step S1, after the principal components are formed, ENVI is output in the following order: the first principal component has the largest variance, and the second principal component has the largest variance. The variance is the second largest, and so on, and the final principal component has the smallest variance.
The method for fine water body extraction based on U-net neural network according to claim 1, characterized in that, in the step S3, the label data sets the water body value to 1 and the non-water body value to 0.
The method for extracting fine water bodies based on U-net neural network according to claim 1, characterized in that, in the step S4, the verification set does not participate in the training, and is used to calculate the accuracy of the U-net neural network after the training is completed.
The method for fine water body extraction based on U-net neural network according to claim 1, characterized in that, before the step S5 is executed, the training data of the optimal remote sensing image and the training data corresponding to the label data are divided, and the output 64 *64 images are used as training data in step S5.
The method for fine water body extraction based on U-net neural network according to claim 6, characterized in that, in the step S5, the U-net neural network is deepened, and after the deepening, a 3×3 convolutional layer is included , 2×2 downsampling layer, 2×2 upsampling layer and skip connection layer, after the convolution operation is performed, the Relu activation function is used.
The method for fine water body extraction based on U-net neural network according to claim 7, characterized in that, in the step S5, the setting of the U-net neural network uses an Adam optimizer to dynamically set the learning rate; Each layer of the net neural network uses random deactivation to prevent overfitting and batch normalization to prevent vanishing gradients.
The method for fine water body extraction based on U-net neural network according to claim 8, wherein in the step S5, the generations used for U-net neural network training are divided into several batches, and the training is completed in one batch Then output the training accuracy and verification accuracy respectively. If the accuracy of a certain number of generations is continuously maintained above 90% without major changes, the training of the U-net neural network is stopped.
The method for fine water body extraction based on U-net neural network according to claim 1, wherein, before the step S6 is executed, the test data of the optimal remote sensing image is divided into 64*64 images; the threshold segmentation In step S7, the recall rate and the accuracy rate are used as the precision evaluation criteria: the recall rate = (the neural network correctly identifies the water body The number of pixels of the actual water body)/(the actual number of pixels of the water body), the accuracy rate = (the number of pixels that the neural network correctly identifies the water body)/(the total number of pixels that the neural network identifies the water body).