CN115331102A - Remote sensing image river and lake shoreline intelligent monitoring method based on deep learning - Google Patents
Remote sensing image river and lake shoreline intelligent monitoring method based on deep learning Download PDFInfo
- Publication number
- CN115331102A CN115331102A CN202210908207.3A CN202210908207A CN115331102A CN 115331102 A CN115331102 A CN 115331102A CN 202210908207 A CN202210908207 A CN 202210908207A CN 115331102 A CN115331102 A CN 115331102A
- Authority
- CN
- China
- Prior art keywords
- remote sensing
- model
- river
- deep learning
- sensing image
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 46
- 238000012544 monitoring process Methods 0.000 title claims abstract description 46
- 238000013135 deep learning Methods 0.000 title claims abstract description 21
- 238000012549 training Methods 0.000 claims abstract description 37
- 238000012545 processing Methods 0.000 claims abstract description 35
- 238000013136 deep learning model Methods 0.000 claims abstract description 17
- 238000013508 migration Methods 0.000 claims abstract description 16
- 230000005012 migration Effects 0.000 claims abstract description 16
- 230000006870 function Effects 0.000 claims abstract description 15
- 238000004364 calculation method Methods 0.000 claims abstract description 9
- 238000012805 post-processing Methods 0.000 claims abstract description 9
- 238000013528 artificial neural network Methods 0.000 claims abstract description 6
- 238000004519 manufacturing process Methods 0.000 claims abstract description 4
- 238000013526 transfer learning Methods 0.000 claims abstract description 3
- XLYOFNOQVPJJNP-UHFFFAOYSA-N water Substances O XLYOFNOQVPJJNP-UHFFFAOYSA-N 0.000 claims description 16
- 230000000903 blocking effect Effects 0.000 claims description 12
- 238000012360 testing method Methods 0.000 claims description 12
- 238000011176 pooling Methods 0.000 claims description 11
- 230000011218 segmentation Effects 0.000 claims description 11
- 238000006243 chemical reaction Methods 0.000 claims description 10
- 230000008569 process Effects 0.000 claims description 8
- 238000005457 optimization Methods 0.000 claims description 7
- 230000004913 activation Effects 0.000 claims description 6
- 238000005070 sampling Methods 0.000 claims description 6
- 230000000877 morphologic effect Effects 0.000 claims description 5
- 238000013527 convolutional neural network Methods 0.000 claims description 4
- 238000009412 basement excavation Methods 0.000 claims description 3
- 238000000605 extraction Methods 0.000 claims description 3
- 238000002360 preparation method Methods 0.000 claims description 2
- 230000008030 elimination Effects 0.000 claims 1
- 238000003379 elimination reaction Methods 0.000 claims 1
- 238000005192 partition Methods 0.000 claims 1
- 230000008901 benefit Effects 0.000 abstract description 6
- 238000003062 neural network model Methods 0.000 description 11
- 238000005516 engineering process Methods 0.000 description 5
- 238000011161 development Methods 0.000 description 4
- 230000018109 developmental process Effects 0.000 description 4
- 238000005520 cutting process Methods 0.000 description 3
- 238000010586 diagram Methods 0.000 description 3
- 230000000694 effects Effects 0.000 description 3
- 238000010276 construction Methods 0.000 description 2
- 238000013473 artificial intelligence Methods 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 230000002349 favourable effect Effects 0.000 description 1
- 230000006872 improvement Effects 0.000 description 1
- 238000010801 machine learning Methods 0.000 description 1
- 239000000463 material Substances 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000011160 research Methods 0.000 description 1
- 230000001502 supplementing effect Effects 0.000 description 1
- 238000012546 transfer Methods 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/10—Terrestrial scenes
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/20—Image preprocessing
- G06V10/28—Quantising the image, e.g. histogram thresholding for discrimination between background and foreground patterns
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/40—Extraction of image or video features
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
- G06V10/764—Arrangements for image or video recognition or understanding using pattern recognition or machine learning using classification, e.g. of video objects
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
- G06V10/77—Processing image or video features in feature spaces; using data integration or data reduction, e.g. principal component analysis [PCA] or independent component analysis [ICA] or self-organising maps [SOM]; Blind source separation
- G06V10/774—Generating sets of training patterns; Bootstrap methods, e.g. bagging or boosting
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
- G06V10/82—Arrangements for image or video recognition or understanding using pattern recognition or machine learning using neural networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/10—Terrestrial scenes
- G06V20/13—Satellite images
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/10—Terrestrial scenes
- G06V20/17—Terrestrial scenes taken from planes or by drones
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Multimedia (AREA)
- Evolutionary Computation (AREA)
- Health & Medical Sciences (AREA)
- Artificial Intelligence (AREA)
- Computing Systems (AREA)
- General Health & Medical Sciences (AREA)
- Software Systems (AREA)
- Databases & Information Systems (AREA)
- Medical Informatics (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Remote Sensing (AREA)
- Computational Linguistics (AREA)
- Biomedical Technology (AREA)
- Biophysics (AREA)
- Life Sciences & Earth Sciences (AREA)
- Data Mining & Analysis (AREA)
- Molecular Biology (AREA)
- General Engineering & Computer Science (AREA)
- Mathematical Physics (AREA)
- Astronomy & Astrophysics (AREA)
- Image Analysis (AREA)
- Image Processing (AREA)
Abstract
The invention discloses a remote sensing image river and lake shoreline intelligent monitoring method based on deep learning. The method comprises the following steps: step 1: manufacturing a multi-source heterogeneous remote sensing interpretation sample set aiming at typical land features related to river and lake shoreline monitoring; step 2: applying the deep learning model to river and lake shoreline monitoring; constructing a multi-scale deep convolution neural network, and designing a model loss function according to specific task requirements; and 3, step 3: deep learning model training; and 4, step 4: processing the large scene image data by adopting a block processing strategy; and 5: carrying out post-processing on the interpretation result; step 6: developing a model generalization based on transfer learning; by developing the multi-source remote sensing image intelligent interpretation processing strategy of joint migration learning, effective migration of the intelligent interpretation model from a source domain to a target domain is achieved, and the generalization capability of the model is improved. The method has the advantages of capability of accurately identifying typical ground objects of the river and lake shoreline, high calculation efficiency and strong generalization capability.
Description
Technical Field
The invention relates to the technical field of remote sensing image processing, in particular to a remote sensing image river and lake shoreline intelligent monitoring method based on deep learning.
Background
With the rapid development of the economic society, the development and utilization degree of the river and lake shoreline is continuously improved, and meanwhile, the protection of the river and lake shoreline also faces a severe challenge, so that the effective protection and reasonable utilization of the river shoreline have important influence on the regional ecological civilization construction and the economic society development. Therefore, the monitoring work of the river and lake shoreline is of great significance.
The traditional river and lake shoreline monitoring mainly depends on river and lake management personnel to regularly patrol the river on site, but the method has the problems of low patrol efficiency, large workload, large consumption of manpower, material resources and financial resources, difficulty in patrolling in partial areas, incomplete monitoring range and the like. With the continuous improvement of the remote sensing technology level, the remote sensing image data resources are increasingly abundant, and a mode of realizing river and lake shoreline monitoring by interpreting the image data of the target area gradually becomes a main trend. The remote sensing interpretation work of the river and lake shoreline is usually carried out manually by an interpreter with professional knowledge by combining the characteristics and the actual conditions of the remote sensing image, but the method is time-consuming and labor-consuming and is not beneficial to the efficient interpretation of large-scale large-scene images. In recent years, artificial intelligence technology has been developed rapidly, and the adoption of machine learning method to realize automatic, fast and accurate interpretation of remote sensing images has become the mainstream research direction.
Therefore, how to fully utilize the advantage of the big data of the remote sensing image and combine the deep learning theory to realize the intelligent monitoring of the river and lake shoreline of the remote sensing image is an important problem to be solved urgently.
Disclosure of Invention
The invention aims to provide a remote sensing image river and lake shoreline intelligent monitoring method based on deep learning, which fully utilizes the advantage of big data of the remote sensing image, combines the deep learning technology, realizes the remote sensing image river and lake shoreline intelligent monitoring and has strong generalization capability; the method realizes accurate recognition of typical ground features of the river and lake shoreline by constructing a large-scale remote sensing interpretation sample set, developing a multi-scale deep learning method, adopting a migration learning optimization strategy and the like, has recognition precision superior to 85 percent and higher calculation efficiency (the method realizes high-efficiency calculation by means of block processing, parallel calculation and the like), and has intelligent interpretation efficiency superior to 10 minutes for one-scene high-resolution two-number images (about 500 square kilometers), and is easy to put into practical use.
In order to realize the purpose, the technical scheme of the invention is as follows: a remote sensing image river and lake shoreline intelligent monitoring method based on deep learning is characterized in that: the method comprises the following steps:
step 1: manufacturing a multi-source heterogeneous remote sensing interpretation sample set aiming at typical land features related to river and lake shoreline monitoring;
constructing a multisource heterogeneous remote sensing image river, lake and shore line monitoring sample set, combining historical images and interpretation results related to developed engineering projects, carrying out data acquisition and sample manufacturing on remote sensing image river, lake and shore line monitoring typical ground object categories, constructing and obtaining multisource heterogeneous remote sensing interpretation sample set products containing hundred thousand orders of large-scale different sensors, different resolutions and different ground object elements, and dividing the multisource heterogeneous remote sensing interpretation sample set products into training data and testing data;
step 2: applying the deep learning model to river and lake shoreline monitoring;
constructing a multi-scale depth convolutional neural network model, and designing a multi-scale depth convolutional neural network model loss function according to specific task requirements, wherein the multi-scale depth convolutional neural network model loss function mainly comprises a multi-scale coding-decoding structure, a coding-decoding stacked structure and a binary cross entropy loss function;
and step 3: deep learning model training;
performing data enhancement on the training data constructed in the step 1, inputting the training data into the deep learning model constructed in the step 2 for training, and performing updating optimization on the multi-scale deep convolution neural network model parameters by adopting a random gradient descent algorithm and a back propagation algorithm;
and 4, step 4: testing a deep learning model;
inputting the large-scene remote sensing image test data constructed in the step 1 into the multi-scale depth convolution neural network model obtained by training in the step 3, processing the large-scene image data by adopting a block processing strategy, inputting the large-scene image data into the trained and optimized multi-scale depth convolution neural network model to obtain a preliminarily interpreted probability map (namely, obtaining an output large-scene probability map by adopting the block processing strategy), and obtaining a preliminarily extracted binarization result (namely, a preliminarily interpreted result) by adopting a threshold segmentation mode;
and 5: carrying out post-processing on the interpretation result;
optimizing the binary interpretation result extracted in the step 4 by adopting morphological post-processing, adding geographic coordinate information to the optimized result and carrying out grid-vector conversion to obtain a final river and lake shoreline monitoring interpretation result;
step 6: developing a model generalization based on transfer learning;
by developing a multi-source remote sensing image intelligent interpretation processing strategy of joint migration learning and supplementing a small amount of sample data under a specific scene, the optimized multi-scale depth convolution neural network model is finely adjusted, so that effective migration of the multi-scale depth convolution neural network model from a source domain (original scene data) to a target domain (specific scene data) is realized, and the generalization capability of the multi-scale depth convolution neural network model is improved.
The interpretation in the invention realizes the identification and monitoring of the ground features, and the whole process is to realize accurate interpretation. The steps 1-4 in the invention can realize the preliminary interpretation of the target on the remote sensing image, the step 5 carries out the optimized post-processing on the preliminary interpretation result of the step 4, the step 6 carries out the migration generalization and realizes the interpretation of the target under other scenes, the steps in the invention are interrelated, the former step is the basis of the latter step, and the latter step is the deepening of the former step.
In the technical scheme, in the step 1, the multi-source heterogeneous remote sensing interpretation sample set product comprises different sensors such as a high-resolution second satellite, a resource third satellite, a Beijing second satellite, a worldview satellite and an unmanned aerial vehicle, different spatial resolutions such as 0.2 meter, 0.8 meter and 2 meter, and different ground and feature elements such as a water body, suspected excavation, a building and a road.
In the technical scheme, in the step 2, the multi-scale depth convolution neural network is of a symmetrical U-shaped network structure, down sampling is performed in a coding mode, high-dimensional abstract features are extracted, up sampling is performed in a decoding mode, the image size is recovered, and a fine interpretation result is obtained;
in addition, the invention introduces a coding-decoding stacking structure to simultaneously integrate low-dimensional space information and high-dimensional semantic information, and for each symmetrical convolution-deconvolution pair, the characteristics of the convolution layer are stacked on the deconvolution layer, thereby providing more detailed ground object information; wherein, the pooling treatment in the network structure adopts maximum pooling (max pooling), the size of the pooling core is selected to be 2 × 2 according to experience, the step length is 2, and the formula is as follows:
wherein,
indicating that the k-th feature map is associated with a rectangular region R ij The maximum pooled output value of (a) is,
represents a rectangular region R ij The element located at (p, q).
The activation function is a nonlinear activation unit, reLU (corrected linear unit), taking the maximum value between 0 and the pixel value x, and the formula is:
f(x)＝max(0,x)
the loss function of the deep learning model is a binary cross entropy, and the formula is as follows:
wherein,
the confidence degree of the network prediction is used for measuring the probability that the pixel belongs to the ground feature; y is i Is a true value, wherein the identification object and the non-identification object are assigned a value of 1 and 0, respectively; i denotes an index of each pixel; n represents the total number of pixels;
the loss function is used for evaluating the difference degree of the predicted value and the true value of the deep learning model, so that the next training is guided to be carried out in the correct direction.
In the above technical solution, in step 3, the training data enhancement mode is image clipping, taking into account computational power resources and input size requirements of the multi-scale depth convolution neural network model, all training image data are uniformly clipped to 128 × 128 size, which is an empirical value, and other sizes can be selected according to actual situations, thereby effectively expanding the amount of training data; by cutting the image into small pieces of data with fixed sizes, the problem of insufficient computing resources of computer hardware can be avoided, and the training data volume is increased.
In the above technical solution, in step 4, the processing mode of the large-scene remote sensing image data is a block processing strategy, the size of the image blocks can be adjusted according to the specific hardware configuration, and the invention adopts 1500 × 1500 as the size of the blocks; in order to improve the false mark effect between blocks, a mode of increasing the overlapping degree between blocks is adopted for solving, wherein the overlapping degree is 30%, and the characteristic value of an overlapping area is calculated and averaged according to the corresponding overlapping area of each block; meanwhile, in the blocking processing process, a strategy of blocking reading, blocking testing and blocking storing is adopted, so that the processing efficiency of the large-scene remote sensing image data is effectively improved, the problem of insufficient computational resources is solved, and data with any image size can be processed; in addition, the threshold segmentation calculation formula is as follows:
wherein,
the output probability value of the multi-scale depth convolution neural network model is shown, y represents the binarization result after threshold segmentation, T represents the threshold, and 0.5 is adopted as the optimal segmentation threshold.
In the technical scheme, the morphological post-processing in the step 5 comprises small-area object removal and cavity filling, wherein the threshold value for small-area object removal is selected to be 400 pixels, and the threshold value for cavity filling is selected to be 200 pixels.
In the above technical solution, the grid-vector conversion in step 5 is performed by means of block processing and parallel operation, and the image is divided into blocks with a size larger than 40000 × 40000, and the block size is cut according to 40000 × 40000, so as to effectively improve the conversion efficiency.
In the above technical solution, the multi-source remote sensing image intelligent interpretation processing strategy of the joint migration learning in step 6 is a multi-scale depth convolution neural network model fine tuning strategy, that is, a large number of labeled samples in a source domain are firstly adopted to fully train the multi-scale depth convolution neural network model to obtain a pre-training model, and then a small number of labeled samples in a target domain are adopted to perform parameter fine tuning on the pre-training multi-scale depth convolution neural network model to realize migration generalization of the model.
The invention has the advantages that:
(1) Aiming at typical surface features related to river and lake shoreline monitoring, a multi-source heterogeneous remote sensing interpretation sample set product with large scale and different sensors, different resolutions and different surface feature elements is constructed and obtained, and data guarantee is provided for further development of a deep learning technology in the field;
(2) A multi-scale deep convolution neural network is developed, and high-precision recognition of typical ground objects of the river and lake shoreline is realized through optimized training and testing of a deep learning model;
(3) After the remote sensing interpretation result is obtained, a series of optimization strategies such as small-area object rejection, cavity filling, geographic coordinate adding, grid-vector conversion, block parallel processing, migration learning and the like are adopted, so that the interpretation precision and efficiency are improved, and the monitoring effect is improved.
The method provided by the invention can effectively identify the typical ground object type related to the river and lake shoreline monitoring of the remote sensing image, can accurately identify the typical ground object of the river and lake shoreline, has high calculation efficiency and strong generalization capability, and realizes high-efficiency high-precision automatic monitoring.
Drawings
FIG. 1 is a process flow diagram of the present invention.
FIG. 2 is a diagram illustrating the result of interpreting the water level of the area A according to the embodiment of the present invention.
Fig. 3 is an enlarged view of fig. 2 at E1.
Fig. 4 is an enlarged view of fig. 2 at E2.
FIG. 5 is a diagram illustrating the water level interpretation result of the area B according to the embodiment of the present invention.
Fig. 6 is an enlarged view of fig. 5 at F1.
Fig. 7 is an enlarged view of fig. 5 at F2.
In fig. 1, the high-resolution remote sensing image refers to an image from an unmanned aerial vehicle image of 0.2 m to a satellite image of 2 m, and mainly includes image data of three resolutions of 0.2 m, 0.8 m, and 2 m. The satellite data and the unmanned aerial vehicle data in the invention are high-resolution remote sensing image data.
In fig. 2, fig. 3, fig. 4, fig. 5, fig. 6 and fig. 7, the thickened black edge line is the water body extraction result obtained by the method, and as can be seen from fig. 2 to fig. 7, the method has the advantages of accurate interpretation result and high monitoring precision.
Detailed Description
The embodiments of the present invention will be described in detail with reference to the accompanying drawings, which are not intended to limit the present invention, but are merely exemplary. While the advantages of the invention will be clear and readily understood by the description.
According to the method, a typical ground object sample set for river and lake shoreline monitoring is constructed, a multi-scale depth convolution neural network is developed, and a series of optimization strategies such as small-area object removal, cavity filling, geographic coordinate adding, grid-vector conversion, block parallel processing, migration learning and the like are further adopted, so that accurate recognition of typical ground objects of the river and lake shoreline is achieved. Meanwhile, the construction of an intelligent model is realized through the step 2, the training optimization of the intelligent model is realized through the step 3, the intelligent processing of the image is realized through the step 4, and the intelligent transfer processing is realized through the step 6, so that the intelligent monitoring of the river, lake and shore lines of the remote sensing image is realized. The method can be used for large-scale large-scene data, and has the capacity of fully automatically realizing large-scale data batch processing and large-scene data block processing (for example, the method can fully automatically process dozens of scenes of image data in batch, and can also process large-scene image data facing to the whole market range).
The embodiment of the invention for monitoring the river shoreline of a certain remote sensing image is used for explaining the invention in detail, and the invention also has a guiding function for the application of the invention to other remote sensing image river and lake shoreline monitoring.
In this embodiment, the interpretation scene related to the river shoreline of a certain remote sensing image is a scene that moves from the area a to the area B.
Typical ground object types related to a certain remote sensing image river shoreline comprise water bodies, suspected excavation, buildings and the like; in this embodiment, the two pieces of image data belong to large-scale scene data, the coverage area of the area a is about 1300 kilometers squared, and the coverage area of the area B is about 500 kilometers squared.
In the prior art, remote sensing image data of a river shoreline is acquired, and the target area image data is interpreted so as to realize the monitoring of the river and lake shoreline, wherein the remote sensing interpretation work of the river and lake shoreline is usually carried out manually by an interpreter with professional knowledge by combining the characteristics and the actual condition of the remote sensing image; however, this method is time-consuming and labor-consuming, generally, the water body interpretation time of one scene of the second high-resolution image (more than 500 square kilometers) needs 2-3 days, which is not favorable for the high-efficiency interpretation of the large-scale scene image in this embodiment, the method provided by the present invention can usually implement the automatic interpretation of the water body of one scene of the second high-resolution image within 10 minutes, the water body interpretation time of the area a (the large-scale scene image data obtained by splicing the multiple scenes, the coverage area of which is about 1300 square kilometers) is about 16 minutes, the water body interpretation time of the area B (the coverage area of which is about 500 square kilometers) is about 8 minutes, the whole processing process is fully automatically implemented, no manual participation is required, and the identification precision is about 90%.
As shown in fig. 1, in this embodiment, the method for intelligently monitoring a river bank line of a remote sensing image based on deep learning according to the present invention is used for intelligently monitoring a river bank line of a remote sensing image, which is specifically described by taking intelligent interpretation of a water body as an example, and includes the following steps:
step 1: combining historical images and interpretation results related to developed engineering projects, carrying out data acquisition and sample preparation on water bodies in river, lake and shore line monitoring of remote sensing images, and constructing and obtaining large-scale different sensors and multi-source heterogeneous remote sensing interpretation sample set products with different resolutions, wherein the multi-source heterogeneous remote sensing interpretation sample set product in the embodiment is mainly a water body sample product with 0.8 m of resolution, the selected sensor is a high-resolution second-number sensor and 0.8 m of resolution, and is divided into training data and testing data;
and 2, step: and constructing a multi-scale deep convolution neural network, wherein the network is a symmetrical U-shaped network structure, firstly performing down-sampling in a coding mode, extracting high-dimensional abstract characteristics, then performing up-sampling in a decoding mode, recovering the image size and obtaining a fine interpretation result.
In addition, the invention introduces a coding-decoding stacking structure to simultaneously integrate low-dimensional space information and high-dimensional semantic information, and for each symmetrical convolution-deconvolution pair, the characteristics of the convolution layer are stacked on the deconvolution layer, thereby providing more detailed ground object information; wherein, the pooling treatment in the network structure adopts maximum pooling (max pooling), the size of the pooling core is selected to be 2 × 2 according to experience, the step length is 2, and the formula is as follows:
wherein,
indicating that the k-th feature map is associated with a rectangular region R ij The maximum pooled output value of (a) is,
represents a rectangular region R ij Is located at (p, q).
The activation function is a nonlinear activation unit, reLU (corrected linear unit), taking the maximum value between 0 and the pixel value x, and the formula is:
f(x)＝max(0,x)
the loss function of the deep learning model is a binary cross entropy, and the formula is as follows:
wherein,
the confidence degree of the network prediction is used for measuring the probability that the pixel belongs to the ground feature; y is i Is a true value, wherein the identification object and the non-identification object are assigned a value of 1 and 0, respectively; i denotes an index of each pixel; n denotes the total number of pixels.
The present embodiment adopts the above steps in the method of the present invention to construct a deep learning model.
And step 3: and (3) performing data enhancement on the training data constructed in the step (1), wherein the data enhancement mode is image cutting, namely cutting all the training image data into the size of 128 multiplied by 128 uniformly, so that the training data volume is effectively expanded. Inputting the expanded training data into the deep learning model constructed in the step 2 for training, and updating and optimizing model parameters by adopting a random gradient descent algorithm and a back propagation algorithm;
and 4, step 4: inputting the large-scene remote sensing image test data constructed in the step 1 into the network model obtained by training in the step 3, and obtaining an output large-scene probability map by adopting a block processing strategy, wherein 1500 × 1500 is adopted as the size of a block in the embodiment; in order to improve the false mark effect between blocks, a mode of increasing the overlapping degree between blocks is adopted for solving, wherein the overlapping degree is 30%, and the characteristic value of an overlapping area is calculated and averaged according to the corresponding overlapping area of each block; meanwhile, in the blocking processing process, a strategy of blocking reading, blocking testing and blocking storing is adopted, the processing efficiency of the large-scene remote sensing image data is effectively improved, the problem of insufficient computational resources is solved, and data with any image size can be processed; in addition, a threshold segmentation mode is adopted to obtain a preliminary extracted binarization result, and a threshold segmentation calculation formula is as follows:
wherein,
this indicates the model output probability value, y indicates the binarization result after threshold segmentation, and T indicates the threshold, and in this embodiment, 0.5 is used as the optimal segmentation threshold.
In this embodiment, a binarized water surface grid interpretation result is obtained through the above steps. That is, a pixel with a value of 0 is a background, and a pixel with a value of 1 is an object of interpretation.
And 5: and (3) optimizing the extraction result in the step (4) by adopting morphological post-processing such as small-area object removal, hole filling and the like, wherein the threshold selection of the small-area object removal is 400 pixels, the threshold selection of the hole filling is 200 pixels, geographic coordinate information and grid-vector conversion are added to the optimization result, the grid-vector conversion is carried out in a blocking processing and parallel operation mode, the image is blocked when the size of the image is larger than 40000 multiplied by 40000, and the blocking size is cut according to 40000 multiplied by 40000, so that the conversion efficiency is effectively improved, and the final water body interpretation result is obtained, as shown in fig. 2.
Step 6: the method comprises the steps of developing an intelligent interpretation processing strategy of the multi-source remote sensing image of joint migration learning, namely a model fine tuning strategy, fully training a model by adopting a large number of labeled samples of a source domain to obtain a pre-training model, and finely tuning parameters of the pre-training model by adopting a small number of labeled samples of a target domain to realize migration generalization of the model. Here the interpreted scene is migrated from area a to area B scene, the water level interpretation result is shown in fig. 3.
And (4) conclusion: the embodiment adopts the deep learning technology in the invention to realize the intelligent monitoring of the remote sensing image river and lake shoreline; the method can accurately identify the typical land features of the river and lake shoreline, the identification precision is better than 85 percent, the method has higher calculation efficiency, the intelligent interpretation efficiency of the one-scene high-score second image (about 500 square kilometers) is better than 10 minutes, and the method is easy to put into practical use.
It should be understood that the above description of the preferred embodiments is illustrative, and not restrictive, and that various changes and modifications may be made therein by those skilled in the art without departing from the scope of the invention as defined in the appended claims.
Other parts not described belong to the prior art.
Claims (9)
1. A remote sensing image river and lake shoreline intelligent monitoring method based on deep learning is characterized in that: the method comprises the following steps:
step 1: manufacturing a multi-source heterogeneous remote sensing interpretation sample set;
combining historical images and interpretation results related to developed engineering projects, carrying out data acquisition and sample preparation on remote sensing image river, lake and shore line monitoring typical surface feature types, constructing and obtaining a multisource heterogeneous remote sensing interpretation sample set product containing hundreds of thousands of magnitude of large-scale different sensors, different resolutions and different surface feature elements, and dividing the multisource heterogeneous remote sensing interpretation sample set product into training data and test data;
and 2, step: applying the deep learning model to river and lake shoreline monitoring;
constructing a multi-scale deep convolutional neural network model, and designing a model loss function according to specific task requirements, wherein the model loss function mainly comprises a multi-scale coding-decoding structure, a coding-decoding stacking structure and a binary cross entropy loss function;
and step 3: deep learning model training;
performing data enhancement on the training data constructed in the step (1), inputting the training data into the deep learning model constructed in the step (2) for training, and updating and optimizing model parameters by adopting a random gradient descent algorithm and a back propagation algorithm;
and 4, step 4: testing a deep learning model;
inputting the large-scene remote sensing image test data constructed in the step 1 into the network model obtained by training in the step 3, obtaining an output large-scene probability map by adopting a block processing strategy, and obtaining a preliminary extracted binarization result by adopting a threshold segmentation mode;
and 5: carrying out post-processing on the interpretation result;
optimizing the extraction result in the step 4 by adopting morphological post-processing, adding geographic coordinate information to the optimization result and performing grid-vector conversion to obtain a final river and lake shoreline monitoring interpretation result;
step 6: developing a model generalization based on transfer learning;
by developing the multi-source remote sensing image intelligent interpretation processing strategy of joint migration learning, effective migration of the intelligent interpretation model from a source domain to a target domain is achieved, and the generalization capability of the model is improved.
2. The remote sensing image river, lake and shore line intelligent monitoring method based on deep learning of claim 1, which is characterized in that: in the step 1, the multi-source heterogeneous remote sensing interpretation sample set product comprises different sensors such as a high-resolution second satellite, a resource third satellite, a Beijing second satellite, a world view satellite and an unmanned aerial vehicle, different spatial resolutions such as 0.2 meter, 0.8 meter and 2 meter, and different surface feature elements such as a water body, suspected excavation, a building and a road.
3. The remote sensing image river, lake and shore line intelligent monitoring method based on deep learning as claimed in claim 1 or 2, wherein: in the step 2, the multi-scale depth convolution neural network is of a symmetrical U-shaped network structure, down-sampling is carried out through a coding mode, high-dimensional abstract features are extracted, up-sampling is carried out through a decoding mode, the image size is recovered, and a fine interpretation result is obtained;
introducing a coding-decoding stacking structure to simultaneously integrate low-dimensional spatial information and high-dimensional semantic information, wherein for each symmetrical convolution-deconvolution pair, the features of the convolution layer are stacked on the deconvolution layer, thereby providing more detailed surface feature information; wherein, the pooling treatment in the network structure adopts maximum pooling, the size of a pooling core is selected to be 2 multiplied by 2 according to experience, the step length is 2, and the formula is as follows:
wherein,
indicating that the k-th feature map is associated with a rectangular region R ij The maximum pooled output value of (a) is,
represents a rectangular region R ij The element located at (p, q);
the activation function is a nonlinear activation unit ReLU, taking the maximum value between 0 and the pixel value x, and the formula is as follows:
f(x)＝max(0,x)
the loss function of the deep learning model is a binary cross entropy, and the formula is as follows:
wherein,
the confidence degree of the network prediction is used for measuring the probability that the pixel belongs to the ground feature; y is i Is a true value, whereinThe identification object and the non-identification object are respectively assigned with 1 and 0; i denotes an index of each pixel; n denotes the total number of pixels.
4. The remote sensing image river, lake and shore line intelligent monitoring method based on deep learning of claim 3 is characterized in that: in step 3, the training data enhancement mode is image cropping, and all training image data are uniformly cropped to 128 × 128 dimensions in consideration of computational resources and model input dimension requirements, so that the training data volume is effectively expanded.
5. The remote sensing image river, lake and shore line intelligent monitoring method based on deep learning of claim 4 is characterized in that: in step 4, the processing mode of the large-scene remote sensing image data is a block processing strategy; increasing the overlapping degree of each block, and calculating the mean value mode of the characteristic value of the overlapping area according to the corresponding overlapping area of each block; meanwhile, a strategy of block reading, block testing and block storing is adopted in the block processing process; in addition, the threshold segmentation calculation formula is as follows:
wherein,
the model output probability value is represented, y represents the binarization result after threshold segmentation, and T represents the threshold.
6. The remote sensing image river, lake and shore line intelligent monitoring method based on deep learning of claim 5 is characterized in that: in step 4, 1500 × 1500 is adopted as the size of the block in the block processing strategy; a 30% overlap between the partitions was used.
7. The remote sensing image river, lake and shore line intelligent monitoring method based on deep learning of claim 6, which is characterized in that: the morphological post-processing in the step 5 comprises small-area object elimination and cavity filling; the threshold value for removing the small-area objects is selected to be 400 pixels, and the threshold value for filling the cavities is selected to be 200 pixels.
8. The remote sensing image river, lake and shore line intelligent monitoring method based on deep learning of claim 7 is characterized in that: in step 5, the grid-vector conversion is performed by adopting a blocking processing and parallel operation mode, the image size is larger than 40000 multiplied by 40000 to be blocked, and the blocking size is cut according to 40000 multiplied by 40000.
9. The remote sensing image river, lake and shore line intelligent monitoring method based on deep learning of claim 8, which is characterized in that: and 6, the multi-source remote sensing image intelligent interpretation processing strategy of the joint migration learning is a model fine tuning strategy, namely, a large number of labeled samples of a source domain are adopted to fully train the model to obtain a pre-training model, and a small number of labeled samples of a target domain are adopted to perform parameter fine tuning on the pre-training model to realize the migration generalization of the model.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210908207.3A CN115331102A (en) | 2022-07-29 | 2022-07-29 | Remote sensing image river and lake shoreline intelligent monitoring method based on deep learning |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210908207.3A CN115331102A (en) | 2022-07-29 | 2022-07-29 | Remote sensing image river and lake shoreline intelligent monitoring method based on deep learning |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN115331102A true CN115331102A (en) | 2022-11-11 |
Family
ID=83919784
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202210908207.3A Pending CN115331102A (en) | 2022-07-29 | 2022-07-29 | Remote sensing image river and lake shoreline intelligent monitoring method based on deep learning |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN115331102A (en) |
Cited By (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN116343070A (en) * | 2023-05-22 | 2023-06-27 | 武汉天地鹰测绘科技有限公司 | Intelligent interpretation method for aerial survey image ground object elements |
| CN116580321A (en) * | 2023-07-12 | 2023-08-11 | 江苏省地质调查研究院 | Automatic recognition method for remote sensing image shoreline |
| CN116704197A (en) * | 2023-08-07 | 2023-09-05 | 水利部珠江水利委员会水文局 | Processing method and system for river and lake remote sensing image |
| CN118038223A (en) * | 2024-02-01 | 2024-05-14 | 中国科学院空天信息创新研究院 | AI-based remote sensing data sample information fusion and multi-scale reconstruction method and system |
| CN118038223B (en) * | 2024-02-01 | 2024-10-22 | 中国科学院空天信息创新研究院 | AI-based remote sensing data sample information fusion and multi-scale reconstruction method and system |
-
2022
- 2022-07-29 CN CN202210908207.3A patent/CN115331102A/en active Pending
Cited By (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN116343070A (en) * | 2023-05-22 | 2023-06-27 | 武汉天地鹰测绘科技有限公司 | Intelligent interpretation method for aerial survey image ground object elements |
| CN116343070B (en) * | 2023-05-22 | 2023-10-13 | 武汉天地鹰测绘科技有限公司 | Intelligent interpretation method for aerial survey image ground object elements |
| CN116580321A (en) * | 2023-07-12 | 2023-08-11 | 江苏省地质调查研究院 | Automatic recognition method for remote sensing image shoreline |
| CN116580321B (en) * | 2023-07-12 | 2023-09-08 | 江苏省地质调查研究院 | Automatic recognition method for remote sensing image shoreline |
| CN116704197A (en) * | 2023-08-07 | 2023-09-05 | 水利部珠江水利委员会水文局 | Processing method and system for river and lake remote sensing image |
| CN116704197B (en) * | 2023-08-07 | 2023-10-17 | 水利部珠江水利委员会水文局 | Processing method and system for river and lake remote sensing image |
| CN118038223A (en) * | 2024-02-01 | 2024-05-14 | 中国科学院空天信息创新研究院 | AI-based remote sensing data sample information fusion and multi-scale reconstruction method and system |
| CN118038223B (en) * | 2024-02-01 | 2024-10-22 | 中国科学院空天信息创新研究院 | AI-based remote sensing data sample information fusion and multi-scale reconstruction method and system |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN115331102A (en) | Remote sensing image river and lake shoreline intelligent monitoring method based on deep learning | |
| CN114444791A (en) | Flood disaster remote sensing monitoring and evaluation method based on machine learning | |
| CN111753682B (en) | Hoisting area dynamic monitoring method based on target detection algorithm | |
| CN111028255A (en) | Farmland area pre-screening method and device based on prior information and deep learning | |
| CN111985552B (en) | Method for detecting diseases of thin strip-shaped structure of airport pavement under complex background | |
| CN115841625B (en) | Remote sensing building image extraction method based on improved U-Net model | |
| CN114387521B (en) | Remote sensing image building extraction method based on attention mechanism and boundary loss | |
| CN116343053B (en) | Automatic solid waste extraction method based on fusion of optical remote sensing image and SAR remote sensing image | |
| CN115330720A (en) | Unmanned aerial vehicle image earth surface solid waste detection model and detection method | |
| CN115631162A (en) | Landslide hidden danger identification method, system, medium and equipment | |
| CN112949612A (en) | High-resolution remote sensing image coastal zone ground object classification method based on unmanned aerial vehicle | |
| CN115424059A (en) | Remote sensing land use classification method based on pixel level comparison learning | |
| CN114612450B (en) | Image detection segmentation method and system based on data augmentation machine vision and electronic equipment | |
| CN115170952A (en) | On-line automatic monitoring system for seepage safety of reservoir dam body | |
| CN117854027A (en) | Double-convolution network road extraction method based on hypergraph and multistage feature fusion | |
| CN115861260A (en) | Deep learning change detection method for wide-area city scene | |
| CN115497002A (en) | Multi-scale feature fusion laser radar remote sensing classification method | |
| CN113343861B (en) | Remote sensing image water body region extraction method based on neural network model | |
| CN117726954A (en) | Sea-land segmentation method and system for remote sensing image | |
| CN114387446A (en) | Automatic water body extraction method for high-resolution remote sensing image | |
| CN110853058B (en) | High-resolution remote sensing image road extraction method based on visual saliency detection | |
| CN112561808A (en) | Road boundary line restoration method based on vehicle-mounted laser point cloud and satellite image | |
| CN115147375B (en) | Concrete surface defect feature detection method based on multi-scale attention | |
| CN110751150A (en) | FPGA-based binary neural network license plate recognition method and system | |
| CN113723414B (en) | Method and device for dividing mask face shielding object |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |