CN109740485A - Reservoir or pond identification method based on spectral analysis and deep convolutional neural network - Google Patents

Abstract

The invention discloses a method for identifying a reservoir or a pond based on spectral analysis and a deep convolutional neural network, and relates to the fields of artificial intelligence and image recognition. First, use spectral analysis to process high-definition remote sensing images, extract water boundary information, map the water boundary information to the original image, and then use the water boundary information on the original image to divide the original image into several image blocks containing water storage bodies , take these picture blocks as the input of the constructed convolutional neural network, and use the convolutional neural network to identify and classify dams, gates, and spillways. Finally, use dams, gates, and spillways to identify and classify water bodies. The accuracy rate reaches more than 80%, which greatly improves the accuracy of water body identification. The convolutional neural network has obvious simplification in structure, and has greatly improved the running speed and robustness. The processing speed can reach 23fps, realizing Quick and accurate identification of reservoirs and dams.

Description

Reservoir or pond identification method based on spectral analysis and deep convolutional neural network

technical field

The invention relates to the field of artificial intelligence and image recognition, in particular to a reservoir or pond recognition method based on spectral analysis and deep convolutional neural network.

Background technique

Small and medium-sized rivers are widely distributed, with dense river networks, and their ability to accumulate and divert sudden precipitation is weak. In the flood forecasting of medium and small rivers, reservoirs and dams have an important impact on the flood process in the basin, and because reservoirs and ponds have different flood storage and discharge mechanisms, they need to be considered separately.

The identification and classification of reservoirs and ponds and dams is an important part of flood forecasting for small and medium-sized rivers. It is necessary to identify reservoirs and ponds and consider them separately. Through the corresponding digital elevation and high-definition remote sensing images, combined with the different flood storage and discharge mechanisms of reservoirs and ponds, Provide data support for flood forecasting of small and medium rivers.

At present, the identification and classification of reservoirs and ponds mainly adopts other methods based on deep learning. This type of method generally uses a region-based convolutional neural network (R-CNN), first generates a proposal frame (region proposal), then classifies the features in the proposal frame, and regresses the proposal frame to obtain the location and category of the target. . The problems of this type of method are mainly that the network structure is too complex, the computer hardware conditions are relatively high, the running speed is slow, and professional remote sensing practitioners are required to label the training set, so it will be greatly limited in the identification process. .

SUMMARY OF THE INVENTION

The purpose of the present invention is to provide a method for identifying reservoirs or ponds based on spectral analysis and deep convolutional neural network, so as to solve the aforementioned problems existing in the prior art.

In order to achieve the above object, the technical scheme adopted in the present invention is as follows:

A method for identifying reservoirs or ponds based on spectral analysis and deep convolutional neural networks, comprising the following steps:

S1, build a convolutional neural network, the network sequentially includes an input layer, five convolutional layers, a fusion layer, a fully connected layer, and an output layer, wherein the fusion layer is used to combine the second convolutional layer, the third convolutional layer, and the third convolutional layer. The features of the product layer, the fourth convolution layer, and the fifth convolution layer are fused;

S2, using the threshold-based multi-band spectral relationship method to analyze the high-resolution satellite remote sensing image, extract the water boundary information, frame the water body, and obtain the image after spectral analysis;

S3, uniformly map the coordinates of the original image and the image after spectral analysis processing to the same coordinate system, and map the water boundary information to the original image under the same coordinate, so as to realize the framing of the water body on the original image;

S4, using the water boundary information on the original image, the original image is divided into several image blocks containing the water storage body, and these image blocks are used as the input of the convolutional neural network;

S5, use the convolutional neural network to perform feature extraction on the input image block containing the water storage body, and identify the characteristic features in the input image block: dam, gate or floodway;

S6, according to the output result of the convolutional neural network, that is, whether there are dams, gates, and spillways, further classify the input image blocks to determine the specific type of the water body, that is, a reservoir or a dam, and combine the water boundary information to realize the water body. identify.

Preferably, S1 further includes the steps of: using a training set to train the convolutional neural network, and using a test set to evaluate the accuracy of the trained convolutional neural network.

Preferably, the training set and the test set are high-definition image data sets including dams, gates, and spillways, and the size of each image is 224×224 pixels, which are ordinary grayscale images. The training set and The ratio of the test set is 9:1, and the maximum number of iteration steps for training is set to 100K.

Preferably, the use of the training set to train the convolutional neural network is specifically: the number of pictures used for training each time is 128, that is, min_batch=128; a dynamic learning rate is used, and when the number of iteration steps is less than 30K, The value of the learning rate is 0.0001, when the number of iteration steps reaches 30K, the value of the learning rate is 0.00002, and when the number of iteration steps reaches 80K, the value of the learning rate is 0.00001.

Preferably, the evaluation criteria are recall, precision, F1 and mAP, which are respectively calculated according to the following formulas:

Among them, in formulas (1) and (2), Truepositive represents the positive samples predicted by the model as positive, that is, the true correct rate; Falsenegative represents the positive samples predicted as negative by the model, that is, the false negative rate; Falsepositive represents the The model predicts a positive negative sample, that is, the false positive rate;

In formula (4), N is the number of all pictures in the test set, M is the total number of categories of the target, p(k) represents the value of precision when k pictures can be recognized, and Δr(k) represents the number of recognized pictures from k. The change of the recall value when -1 changes to k.

Preferably, S2 is specifically, using the following formula to analyze the high-resolution satellite remote sensing image:

(b3+b4)-(b5+b6)>T (5)

In the formula, b3 is the gray value of band 3; b4 is the gray value of band 4; b5 is the gray value of band 5; b6 is the gray value of band 6; T is the threshold selected through experiments;

Use the above formula to operate on the band, and the water body is satisfied.

The beneficial effects of the present invention are as follows: the method for identifying reservoirs or ponds based on spectral analysis and deep convolutional neural network provided by the present invention firstly uses spectral analysis to process high-definition remote sensing images, extracts water boundary information, and maps the water boundary information to the original On the image, and then using the water boundary information on the original image, the original image is divided into several image blocks containing the water storage body, and these image blocks are used as the input of the constructed convolutional neural network. Identify and classify water bodies (reservoirs, ponds and dams) using dams, gates, and spillways. Therefore, by using the method provided by the present invention, the accuracy rate reaches more than 80%, which greatly improves the accuracy of water body identification. The structure of the convolutional neural network is obviously simplified, and the running speed and robustness are greatly improved. , the processing speed can reach 23fps, and the rapid and accurate identification of reservoirs and ponds and dams is realized.

Description of drawings

Fig. 1 is the schematic flow sheet of the method for identifying reservoirs or ponds based on spectral analysis and deep convolutional neural network provided by the present invention;

Figure 2 is a schematic diagram of the VGG-16 network structure;

Fig. 3 is the principle schematic diagram of the convolutional neural network constructed by the present invention;

Figure 4 is a schematic diagram of the principle of using the test set to evaluate the accuracy of the convolutional neural network;

Figure 5 is the original satellite picture;

Fig. 6 is the picture that the multi-band spectral relation method based on threshold is processed and obtained;

Fig. 7 is the structure schematic diagram of the convolutional neural network constructed by the present invention;

Figure 8 is a photo of three types of features including dams, gates and spillways;

Fig. 9 is the file structure schematic diagram of data set;

Figure 10 is a high-definition satellite image of the reservoir in the study area;

Figure 11 is a high-definition satellite image of the pond and dam in the study area;

Figure 12 is an electromagnetic spectrum diagram;

Figure 13 is the image of the reservoir after spectral analysis;

Figure 14 is the image of the pond after spectral analysis.

In Figure 7, ① represents the input data of the neural network, which is a high-resolution remote sensing image. ② represents the input layer of the convolutional neural network, and the structure in box ③ is five convolutional layers, which are used to extract the features of the input data. ④ is the feature fusion layer, which is used to combine the feature ⑦ of the second convolution layer (II), the feature ⑧ of the third convolution layer (III), the feature ⑨ of the fourth convolution layer (IV), and the fifth The features ⑩ of the convolutional layers (Ⅴ) are fused. ⑤ is a fully connected layer, this layer adopts a fully connected method, and each neuron establishes a connection with other neurons. ⑥ is the output layer, that is, the classification result of the output neural network.

Detailed ways

In order to make the objectives, technical solutions and advantages of the present invention clearer, the present invention will be further described in detail below with reference to the accompanying drawings. It should be understood that the specific embodiments described herein are only used to explain the present invention, but not to limit the present invention.

As shown in Figure 1, the present invention provides a method for identifying a reservoir or pond based on spectral analysis and deep convolutional neural network, comprising the following steps:

S1, build a convolutional neural network, the network sequentially includes an input layer, five convolutional layers, a fusion layer, a fully connected layer, and an output layer, wherein the fusion layer is used to combine the second convolutional layer, the third convolutional layer, and the third convolutional layer. The features of the product layer, the fourth convolution layer, and the fifth convolution layer are fused;

S2, using the threshold-based multi-band spectral relationship method to analyze the high-resolution satellite remote sensing image, extract the water boundary information, frame the water body, and obtain the image after spectral analysis;

S3, uniformly map the coordinates of the original image and the image after spectral analysis processing to the same coordinate system, and map the water boundary information to the original image under the same coordinate, so as to realize the framing of the water body on the original image;

S4, using the water boundary information on the original image, the original image is divided into several image blocks containing the water storage body, and these image blocks are used as the input of the convolutional neural network;

S5, use the convolutional neural network to perform feature extraction on the input image block containing the water storage body, and identify the characteristic features in the input image block: dam, gate or floodway;

S6, according to the output result of the convolutional neural network, that is, whether there are dams, gates, and spillways, further classify the input image blocks to determine the specific type of the water body, that is, a reservoir or a dam, and combine the water boundary information to realize the water body. identify.

Due to the small size of remote sensing image objects, it is greatly affected by the shooting angle and lighting conditions. With the deepening of the network, the extracted features are becoming more and more abstract, and the size of the image is also getting smaller and smaller. The traditional network structure is used in the later stage. Information about small features will be lost. Therefore, a reasonable network structure needs to be designed. To solve the problem of recognition of small objects, the method of backward propagation and re-fusion of the features of multiple convolutional layers can be adopted.

Experiments show that the combination of several small filter convolutional layers is better than one large filter convolutional layer. The VGG-16 network makes full use of the alternating structure of multiple convolutional layers and non-linear activation layers, which can extract deep features better than the structure of a single convolutional layer. VGG-16 (the network structure of VGG-16 is shown in Figure 2) The network includes 5 convolutional layers and 3 fully connected layers, each convolutional layer consists of several sub-layers, and the size of the convolution kernel used is 3 ×3. The number of feature maps output by the 5 convolutional layers are 64, 128, 256, 512, 512, respectively. The features are expanded into 4096-dimensional vectors in the first fully connected layer, and a 3-dimensional vector is obtained in the third fully connected layer. dimensional vector. The classification result is output at the output layer.

In the present invention, the VGG-16 network is used as the basic structure to construct a convolutional neural network, and on the basis of VGG-16, the features extracted from the second, third, fourth and fifth convolutional layers are fused to form a full-scale convolutional neural network. The input of the connection layer, as shown in Figure 3, improves the classification accuracy for small objects.

After the convolutional neural network is constructed, the convolutional neural network can be trained using the training set, and the accuracy of the trained convolutional neural network can be evaluated using the test set.

During training and testing, the datasets used are high-definition image datasets containing dams, gates, and spillways. The size of each image is 224 × 224 pixels, which is an ordinary grayscale image. The number of images contained in the data set is 60,000, and the ratio of training set and test set is 9:1, that is, the test set contains 54,000 images, and the test set contains 6,000 images. When the number of iteration steps reaches 100K, the training is stopped, and the trained model is saved for subsequent identification of water storage bodies.

Use the training set to train the convolutional neural network, specifically: the number of pictures used for training each time is 128; the dynamic learning rate is adopted, when the number of iteration steps is less than 30K, the value of the learning rate is 0.0001, and the value of the learning rate is 0.0001 in the iteration step. When the number of iterations reaches 30K, the value of the learning rate is 0.00002, and when the number of iteration steps reaches 80K, the value of the learning rate is 0.00001.

For the trained neural network, use the test set (the test set contains 6000 pictures) to evaluate the accuracy of the network. The evaluation criteria are recall, precision, F1 and mAP. As shown in Figure 4 and formula (1)(2)(3)(4).

Among them, in formulas (1) and (2), True positive represents the positive samples predicted by the model as positive, that is, the true correct rate; False negative represents the positive samples predicted as negative by the model, that is, the false negative rate; False positive represents the negative samples predicted by the model as positive, that is, the false positive rate;

In formula (4), N is the number of all pictures in the test set, M is the total number of categories of the target, p(k) represents the value of precision when k pictures can be recognized, and Δr(k) represents the number of recognized pictures from k. The change of the recall value when -1 changes to k.

In spectral analysis, the spectral characteristics of water bodies are: the reflection of water bodies is mainly concentrated in the blue-green band in the visible light band, and in the near-infrared band, the reflectivity of water bodies begins to rapidly decay to 0. However, other ground features such as vegetation and soil reflect significantly higher in the near-infrared band than water bodies. Therefore, in the near-infrared remote sensing images, the water body generally has a dark tone due to its low reflectivity, and the vegetation, soil, etc. generally have a bright tone due to its much higher reflectivity than the water body. differentiate.

The electromagnetic spectrum is shown in Figure 12, 400nm-700nm is the visible light band, 750nm and 800nm are the near-infrared band, in the 750nm band, the reflectivity of the water body is significantly attenuated, and in the 800nm band, the water body reflectivity attenuation is 0.

The present invention selects the threshold-based multi-band spectral relationship method to analyze the high-definition satellite remote sensing image of the research area. In the Landsat 8 OLI image, there are a total of 9 bands, among which, b3 is the green light band, b4 is the red light band, b5 is the near-infrared band, and b6 is the short-wave infrared band. The unique inter-spectral feature of the water body is b3 grayscale The sum of the gray value and the gray value of b4 is greater than the sum of the gray value of b5 and the gray value of b6, so the water body can be well extracted by using the relationship between the spectra.

S2 is specifically, the following formula is used to analyze the high-resolution satellite remote sensing image:

(b3+b4)-(b5+b6)>T (5)

In the formula, b3 is the gray value of band 3; b4 is the gray value of band 4; b5 is the gray value of band 5; b6 is the gray value of band 6; In the experiment, the value of T is 400.

Use formula (5) to operate on the band, and the water body that satisfies formula (5) is.

In the actual use process, the accuracy of spectral analysis can be evaluated. The input data is high-resolution remote sensing images. The evaluation indicators are the ratio of the number of water bodies detected by spectral analysis to the actual number of water bodies ρ and the accuracy of the water boundary frame φ. Suppose the actual number of water bodies is δ, the number of detected water bodies is ε, and the detection ratio ρ is defined as shown in formula (6):

The detection ratio ρ should be greater than 90%.

Accuracy of Water Boundary Boxing It is an index to measure the accuracy of water boundary information extraction. Let (x, y) represent the coordinates of the upper left corner of the actual water area bounding box α, and h, w represent the width and height of α, respectively. (x1, y1) represent the coordinates of the upper left corner of the bounding box β of the spectral analysis waters, and h1 and w1 represent the width and height of β, respectively. The transformation from β to α can be expressed by equation (7)(8)(9)(10):

Water Boundary Boxing Accuracy The definition of is shown in formula (11):

In the present invention, on the basis of extracting the water body by spectral analysis and training the network to classify the ground objects, the input of the unknown category is identified, and the information such as the category location of the water body is output. The convolutional neural network is used to identify dams, gates, and spillways, and the classification of water storage bodies is further determined by identifying dams, gates, and spillways.

The water body location information obtained by the spectral analysis and the water body type information obtained by the convolutional neural network are fused to determine the type and location of the water storage body, and realize the identification of the water storage body.

In actual use, the entire model can be evaluated. The spectral analysis and water body classification are jointly evaluated to test the water body identification accuracy of the entire model. The definition of water body identification accuracy τ is shown in formula (12).

in,

k ₁ +k ₂ +k ₃ =1 (13)

It can be seen that, by using the method of the present invention, rapid and accurate identification of reservoirs and ponds and dams is realized. Existing target detection methods are mainly target recognition methods based on region proposals. The method based on region proposal needs to use an additional network structure to generate proposals. For each image, the number of proposal boxes is generally around 300. The calculation cost of generating proposal boxes is huge, which makes the target detection network structure complex. The speed is slow, only around 5fps, and the accuracy is not high, around 70%.

The invention extracts the boundary information of the water body by using spectral analysis with less computational cost, defines the boundary of the water body, firstly determines the position information of the water body, and then uses the convolutional neural network to classify the water body. Compared with the traditional method of using suggestion box regression to predict the water body position, the present invention greatly improves the accuracy of water body identification, so that the accuracy rate reaches more than 80%. The water body classification network of the present invention uses a simple convolutional neural network. Compared with the region generation network RPN+region-based convolutional neural network R-CNN used in the traditional method, the network of the present invention is significantly simplified in structure, and it is The speed and robustness have been greatly improved, and the processing speed can reach 23fps.

Specific examples:

In this embodiment, the Danjiangkou area in the middle and upper reaches of the Han River is selected as the research area, and the high-definition satellite images of the reservoirs and ponds in the research area are shown in Figures 10 and 11 .

First, a convolutional neural network is constructed using the method provided by the present invention. The weight of the network is randomly initialized by using a normal distribution, the mean value is 0, the variance is 0.1, the initial value of the bias term is 0, and the initial value of the learning rate is 0.0001. The number of steps is set to 100K. The learning rate is 0.00002 after iteration 30K and 0.00001 after iteration 80K. The network has a total of 5 convolutional layers and a fully connected layer, and the number of feature maps output by each convolutional layer is 64, 128, 256, 512, 512, respectively. In the fully connected layer, the feature is expanded into a 4096-dimensional vector, and the final dimension of the network output is 3, which represents the probability of dam, gate, and spillway. The feature with the highest probability is taken as the result of this classification. The network structure is shown in Figure 7.

And use the data set for training and testing. The number of pictures contained in this data set is 60,000, and the ratio of training set and test set is 9:1, that is, the test set contains 54,000 pictures, and the test set contains 6,000 pictures. The size of each image is 224×224 pixels, and all are grayscale images. Both the training set and the test set contain three types of ground objects, namely dams, gates, and spillways. As shown in Figure 8, the first row is the gate, the second row is the dam, and the third row is the spillway. The file structure of the dataset is shown in Figure 9.

The neural network is trained using the data of the training set, and the number of images used for training each time is 128, that is, min_batch=128. The dynamic learning rate is adopted, that is, when the number of iteration steps is less than 30K, the value of the learning rate is 0.0001, when the number of iteration steps reaches 30K, the learning rate is 0.00002, and when the number of iteration steps reaches 80K, the learning rate is 0.00001. The model is evaluated using the data of the test set. The main indicators are recall rate, precision rate, F1 and mAP. The mAP of this model is greater than or equal to 0.7.

Use the test set to evaluate the accuracy of the trained network, when , the obtained evaluation criteria recall, precision, F1, and mAP are: 0.9, 0.82, 0.86, and 0.70, respectively.

In the present invention, the threshold-based multi-band spectral relationship method is used to analyze the high-definition satellite remote sensing images of the study area, and the obtained spectral analysis results are shown in Figures 13 and 14.

On the basis of extracting water bodies by spectral analysis and training the network to classify ground objects, the input of unknown categories is identified, and information such as the category location of water bodies is output. The convolutional neural network is used to identify dams, gates, and spillways, and the classification of water storage bodies is further determined by identifying dams, gates, and spillways.

The water body location information obtained by the spectral analysis and the water body type information obtained by the convolutional neural network are fused to determine the type and location of the water storage body, and realize the identification of the water storage body.

By adopting the above-mentioned technical solutions disclosed in the present invention, the following beneficial effects are obtained: the method for identifying reservoirs or ponds based on spectral analysis and deep convolutional neural network provided by the present invention firstly uses spectral analysis to process high-definition remote sensing images, and extracts water areas. Boundary information, map the water boundary information to the original image, and then use the water boundary information on the original image to divide the original image into several image blocks containing water storage bodies, and use these image blocks as the constructed convolutional neural network. Input, use the convolutional neural network to identify and classify dams, gates, and spillways. Finally, use dams, gates, and spillways to identify and classify water bodies (reservoirs, ponds and dams). Therefore, by using the method provided by the present invention, the accuracy rate reaches more than 80%, which greatly improves the accuracy of water body identification. The structure of the convolutional neural network is obviously simplified, and the running speed and robustness are greatly improved. , the processing speed can reach 23fps, and the rapid and accurate identification of reservoirs and ponds and dams is realized.

The above are only the preferred embodiments of the present invention. It should be pointed out that for those skilled in the art, without departing from the principles of the present invention, several improvements and modifications can be made. It should be regarded as the protection scope of the present invention.

Claims (6)

1. A reservoir or small reservoir identification method based on spectral analysis and a deep convolutional neural network is characterized by comprising the following steps:

s1, constructing a convolutional neural network, wherein the network sequentially comprises an input layer, five convolutional layers, a fusion layer, a full-connection layer and an output layer, and the fusion layer is used for fusing the characteristics of the second convolutional layer, the third convolutional layer, the fourth convolutional layer and the fifth convolutional layer;

s2, analyzing the high-resolution satellite remote sensing image by adopting a multi-band spectral relation method based on a threshold value, extracting water area boundary information, framing a water body, and obtaining an image after spectral analysis processing;

s3, mapping the coordinates of the original image and the image after spectral analysis to the same coordinate system, mapping the water area boundary information to the original image under the same coordinate system, and realizing framing of the water body on the original image;

s4, dividing the original picture into a plurality of picture blocks containing water storage bodies by using the water area boundary information on the original image, and taking the picture blocks as the input of a convolutional neural network;

s5, extracting the features of the input picture block containing the water storage body by using the convolutional neural network, and extracting the feature in the input picture block: identifying a dam, a gate or a spillway;

and S6, further classifying the input picture blocks according to the output result of the convolutional neural network, namely whether a dam, a gate and a spillway exist, determining the specific type of the water body, namely a reservoir or a small reservoir, and combining the water body boundary information to realize the identification of the water body.

2. The method for identifying a reservoir or a small reservoir based on spectral analysis and deep convolutional neural network as claimed in claim 1, further comprising the step of S1: and training the convolutional neural network by using a training set, and evaluating the accuracy of the trained convolutional neural network by using a testing set.

3. The method of claim 2, wherein the training set and the testing set are high-definition image data sets comprising dams, gates and spillways, each image is 224 x 224 pixels in size and is a common gray image, the ratio of the training set to the testing set is 9:1, and the maximum number of training iteration steps is set to 100K.

4. The method according to claim 3, wherein the convolutional neural network is trained using a training set, specifically: the number of pictures used for training at a time is 128, i.e., min _ batch is 128; the dynamic learning rate is adopted, when the number of iteration steps is less than 30K, the value of the learning rate is 0.0001, when the number of iteration steps reaches 30K, the value of the learning rate is 0.00002, and when the number of iteration steps reaches 80K, the value of the learning rate is 0.00001.

5. The method for reservoir or pond identification based on spectral analysis and deep convolutional neural network of claim 2, wherein the evaluation criteria are recall, precision, F1 and mAP, calculated according to the following formula:

in the formulas (1) and (2), True positive represents a positive sample predicted to be positive by the model, namely the True accuracy; false negative represents a positive sample predicted to be negative by the model, namely a False negative rate; false positive represents a negative sample predicted to be positive by the model, namely a False alarm rate;

in the formula (4), N is the number of all pictures in the test set, M is the total number of target categories, p (k) represents the value of precision when k pictures can be identified, and Δ r (k) represents the change of recall when the number of identified pictures changes from k-1 to k.

6. The method for identifying a reservoir or a small reservoir based on spectral analysis and deep convolutional neural network as claimed in claim 1, wherein S2 is specifically implemented by analyzing the high-resolution satellite remote sensing image by using the following formula:

(b3+b4)-(b5+b6)>T (5)

in the formula, b3 is the gray value of the wave band 3; b4 is the gray value of band 4; b5 is the gray value of wave band 5; b6 is the gray value of wave band 6; t is a threshold value selected through experiments;

the band is calculated by using the formula, and the water body is obtained when the requirement is met.