CN113033390A - Dam remote sensing intelligent detection method based on deep learning - Google Patents
Dam remote sensing intelligent detection method based on deep learning Download PDFInfo
- Publication number
- CN113033390A CN113033390A CN202110310464.2A CN202110310464A CN113033390A CN 113033390 A CN113033390 A CN 113033390A CN 202110310464 A CN202110310464 A CN 202110310464A CN 113033390 A CN113033390 A CN 113033390A
- Authority
- CN
- China
- Prior art keywords
- dam
- remote sensing
- image
- sensing image
- target
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/10—Terrestrial scenes
- G06V20/176—Urban or other man-made structures
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
- G06F18/214—Generating training patterns; Bootstrap methods, e.g. bagging or boosting
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
- G06N3/082—Learning methods modifying the architecture, e.g. adding, deleting or silencing nodes or connections
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/10—Segmentation; Edge detection
- G06T7/11—Region-based segmentation
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/10—Image acquisition modality
- G06T2207/10032—Satellite or aerial image; Remote sensing
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20021—Dividing image into blocks, subimages or windows
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V2201/00—Indexing scheme relating to image or video recognition or understanding
- G06V2201/07—Target detection
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Data Mining & Analysis (AREA)
- Life Sciences & Earth Sciences (AREA)
- Artificial Intelligence (AREA)
- General Engineering & Computer Science (AREA)
- Evolutionary Computation (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Molecular Biology (AREA)
- Software Systems (AREA)
- Mathematical Physics (AREA)
- Computing Systems (AREA)
- Health & Medical Sciences (AREA)
- Biomedical Technology (AREA)
- Biophysics (AREA)
- Computational Linguistics (AREA)
- General Health & Medical Sciences (AREA)
- Evolutionary Biology (AREA)
- Multimedia (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Bioinformatics & Computational Biology (AREA)
- Image Analysis (AREA)
Abstract
The invention discloses a dam remote sensing intelligent detection method based on deep learning, which comprises the following steps: A. collecting dam remote sensing image samples and establishing a dam remote sensing image sample set; B. based on a deep learning model YOLOv3 framework, replacing an original trunk network Darknet53 with a lightweight network ShuffleNet2, training and testing the model to obtain a dam target detection model based on an improved YOLOv 3; C. taking a large-format remote sensing image as input, and performing sliding window segmentation according to a segmentation step length; D. performing dam target detection on the segmented image based on the improved dam target detection model of YOLOv 3; E. and recombining the segmented images into an image with the same size as the original remote sensing image according to the segmentation step length and the confidence coefficient of the detection boundary frame. The method can be used for rapidly and intelligently detecting the dam target on the optical remote sensing image, effectively improves the target interpretation efficiency, and provides high-quality target detection data for disaster emergency management, dam safety risk management and the like.
Description
Technical Field
The invention belongs to the technical field of remote sensing image processing and analysis, and particularly relates to a dam remote sensing intelligent detection method based on deep learning, which is suitable for optical remote sensing images with the resolution of 0.5m-2 m.
Background
The dam refers to a weir for intercepting rivers and blocking water, and a water blocking dam of reservoirs, rivers and the like. The dam is an important infrastructure in China and an important component of a comprehensive flood control engineering system in China. The dam has the main functions of improving the flood control standard of the downstream of the river, utilizing water energy to generate electricity, improving the shipping of the river channel, supplying water and irrigating for the downstream and serving the economic construction of areas. And the data of the targets are mostly dispersed in different departments, so that the requirement of important earthquake disaster emergency linkage and national defense safety on rapidly acquiring the dam target information cannot be met. With the development of earth observation technology, remote sensing is used as a comprehensive technology capable of rapidly acquiring remote sensing images in a large area range, and civil and military remote sensing applications are becoming wide and deep. Automatic target identification based on remote sensing images plays an increasingly important role in disaster emergency management and military information acquisition. However, the existing remote sensing image analysis technology still does not meet the requirement of practical application. Especially, with the improvement of the resolution of the image and the sharp increase of the data volume, especially the significant increase of the size of a single image, the utilization rate, the automatic processing efficiency and the reliability of the remote sensing data are all provided with serious challenges.
The high-resolution optical remote sensing image can acquire more shape texture information of ground objects, can more visually display the ground surface condition, and is the key point in the field of future remote sensing. The method is a key point of the invention for detecting the dam target by using the visible light high-resolution remote sensing image. The dam target detection aims to identify and locate the dam target from the visible light high-resolution remote sensing image. Firstly, accurately dividing a water area target, and then identifying the dam target by using the water area. Li Yong Fa et al propose an improved Fuzzy C-means (FCM) clustering method by introducing cross entropy distance measure, and obtain a dam target after segmenting a dam remote sensing image of a certain hydropower station. Shenye Jian et al utilize gradient histogram and region growing method, have extracted the upstream large-area waters of dam in the visible light image of medium-high resolution, reuse the chain code to represent the large-area waters and look for the downstream waters, cut apart the area that may include the dam, have extracted a set of characteristic parameters of the dam goal on this basis finally, have realized the recognition of the dam. The method is used for identifying a single dam target, has poor mobility and is not suitable for detecting dam targets of different types, different scales and large range; the existing machine learning method is low in speed and low in automatic detection degree, is basically an image oriented to a small range and a small target, and is not suitable for detecting large targets such as dams in large-format remote sensing images.
Disclosure of Invention
Aiming at the defects in the prior art, the invention aims to provide the dam remote sensing intelligent detection method based on deep learning, the method is easy to implement and convenient to operate, by acquiring dams in different regions and different types, targets can be quickly and intelligently identified and detected through optical remote sensing images, the targets are quickly locked, the automatic processing efficiency and reliability of target detection are improved, the dam condition can be quickly mastered at the first time after major disasters occur, and the disaster emergency management and military information collection capability is enhanced.
In order to further achieve the purpose, the invention adopts the following technical scheme:
the conception of the invention is as follows: firstly, establishing a high-resolution dam sample set; secondly, training and testing the dam sample to obtain a dam target detection model; then, segmenting the large-format remote sensing image to meet the input requirement of a deep learning model; then, carrying out dam target detection on the remote sensing image after segmentation; and finally, merging the remote sensing images after being divided into dam target detection results with the same size as the original large-format remote sensing images.
A dam remote sensing intelligent detection method based on deep learning comprises the following steps:
A. establishing a dam remote sensing image sample set: collecting and marking dam remote sensing image samples, and establishing a dam remote sensing image sample set;
B. establishing a dam target detection model based on improved YOLOv 3: based on a YOLOv3(YOLO (you Only Look one)) frame, a lightweight network ShuffleNet V2 is used for replacing an original trunk network Darknet53, and the model is trained and tested to obtain a dam target detection model based on an improved YOLOv 3;
C. dividing the large-format remote sensing image: according to the segmentation step length, segmenting the input large-format remote sensing image by using a sliding window;
D. and (3) dam target detection: performing dam target detection on the segmented remote sensing image data based on a dam target detection model of improved YOLOv 3;
E. post-processing a dam target detection result: and recombining the remote sensing images after segmentation into the size of the original remote sensing image according to the segmentation step length and the confidence coefficient of the detection boundary frame.
Optionally, in step a, a high-resolution satellite image, an aerial image and a remote sensing image shot by an unmanned aerial vehicle are collected, adaptability analysis of the remote sensing data to dam target detection is performed, and a remote sensing data source suitable for dam target detection is screened out. Meanwhile, a large amount of data is needed for the target detection deep neural network, so that a remote sensing image of the dam target is obtained from Google Earth to be used as supplement. 2000 images with the size of 416 x 416 pixels are obtained as a sample set through collection of a data set of the multi-source remote sensing platform, each image at least comprises one dam target, and 2302 dam samples are total. And then, marking the dam target by using a deep learning target detection marking tool LabelImg, and storing the circumscribed rectangular frame and the category of the dam in an XML file.
Optionally, in step B, first, the backbone network of YOLOv3 is changed from Darknet53 to a lightweight network, shefflenetv 2, so that the complexity of the model is reduced, and the running speed of the model is increased. And then, training the model, and finishing the establishment of the model when the loss function of the verification data is converged and the precision of the test data meets the requirement, or else, adjusting the parameters until the precision of the model meets the requirement.
Optionally, in step C, the size of the input image of the deep learning target detection model is generally hundreds or thousands of pixels, for example, the size of the input image of the YOLOv3 network adopted in the present invention is 416 × 416 pixels, which is very different from the size of Large Remote Sensing Images (Large Remote Sensing Images), so that a segmentation operation needs to be performed before the Remote Sensing Images are input into the network. However, because the dam target is large, when the large-format remote sensing image is input, the direct cutting image (with the cutting step length of 0) is input into the neural network, and the dam or hydropower station target may be cut, so that the dam target cannot be identified or is incompletely identified. Therefore, the invention adopts a sliding window cutting algorithm, and the adjacent image blocks are overlapped by using a certain segmentation step length, so that the model can detect the dam target more completely. The specific process of the sliding window cutting algorithm is that firstly, the times of dividing the large-format remote sensing image in the row direction and the column direction respectively are obtained, and then the large-format remote sensing image is divided according to the formula (1). Specifically, when the image is divided into columns or rows, the last image may be divided into images according to the division rule, in this case, the last divided block is shifted forward, so that the edge of the last divided block coincides with the edge of the original large-format remote sensing image, and finally the division of the large-format remote sensing image is realized.
Equation (1) is as follows:
the Image is an original large-format remote sensing Image, the cropImage is a segmented Image, i and j respectively represent the images segmented from the second block in the column direction and the row direction, nh and nw respectively represent the required segmentation times in the column direction and the row direction, and height and width represent the number of pixels in the column direction and the row direction of the large-format remote sensing Image; side represents the size of the target image (the length and width of the target image are the same); stride represents the segmentation step size for sliding window segmentation.
Furthermore, after a preprocessing scheme before large-format remote sensing image dam target identification is determined, the problem of selecting the segmentation step length needs to be solved. The method mainly comprises the steps of determining the range of the number of pixels occupied by the dam on remote sensing images with different resolutions according to the length of the top of the dam, and then determining the segmentation step length on the remote sensing images with different resolutions.
Furthermore, the dam crest length of more than 95% of the dams is 50-600m, which is obtained by counting the dam crest length of 100 dams randomly selected from dam data of the national energy agency dam safety supervision center. The determination of the segmentation step length was tested using a remote sensing image of 2008 "5, 12" Wenchuan seismic resolution of 2 m. In the 2m resolution remote sensing image, the pixel range occupied by the dam length of the dam is generally 20-300 pixels. When the segmentation step length is too long and the number of segmentation blocks is too large, the detection speed and the detection precision are reduced to some extent, and in order to select a proper segmentation step length, experiments are carried out through different segmentation step lengths of 0 pixel, 50 pixel, 100 pixel, 150 pixel, 200 pixel, 250 pixel and 300 pixel.
The test area is aerial remote sensing images of 5 and 15 days in 2008 and 5 and 17 days in 2008 in a part of Wenchuan county in Sichuan province. Through experiments, when the segmentation step length is 100, the dam target in the large-format remote sensing image can be detected, and the balance between precision and efficiency is realized, so that the segmentation step length of 100 pixels is relatively suitable for the remote sensing image with the resolution of 2 m. The segmentation step length can be calculated according to the formula (2) for the remote sensing image with higher resolution.
Where r is the resolution of the image and stride is the segmentation step size.
Optionally, in step D, the segmented remote sensing images are input into a trained dam target detection model based on improved YOLOv3, and dam target detection is performed for each segmented image.
Optionally, in step E, the segmented image is detected by a dam target detection model based on improved YOLOv3And the detected images are respectively merged according to rows and columns according to the segmentation step length. Since the divided sub-images overlap each other, it is necessary to determine which overlapping portion of two adjacent sub-images is completely reserved according to whether the overlapping portion detects a target or not during merging (the remaining sub-image to be merged needs to discard the overlapping portion and only reserve the portion that is not overlapped with the adjacent sub-image). P in the Yolov3 networkr(Object)*IOUtruth predWherein P isr(object) indicates the probability that the bounding box contains an object, IOUtruth predRepresenting the intersection ratio between the predicted bounding box and the true bounding box. When there is an object in the prediction box, the confidence is equal to IOUtruth predWhen there is no object in the prediction box, the confidence is 0. When the same dam target is detected in the overlapped part of two adjacent images, the reserved part is determined according to the confidence degree of the boundary frame of the target, the boundary frame with high confidence degree is reserved, and the boundary frame with low confidence degree is discarded.
From the above, by the technical measures of the foregoing five steps, the detection of the dam object based on the dam object detection model of the improved YOLOv3 can be realized. Because the original YOLOv3 model cannot directly perform target detection on a large-format remote sensing image, in the key steps C and E, the remote sensing image is segmented according to a certain segmentation step length, and the detection results of the deep learning model are combined according to a certain segmentation step length. The innovation point successfully solves the problem that the original Yolov3 target detection model cannot automatically extract dam target information from a large-format remote sensing image directly.
With the development of the remote sensing satellite technology, the resolution of the satellite remote sensing image is greatly improved. The high-resolution second-order satellite which is successfully launched in 2014 has the resolution ratio of 0.8 meter and the breadth of 45 kilometers, and marks that the satellite remote sensing technology of China also enters the sub-meter-level high-time-share generation. The high resolution and large breadth can bring about an improvement in recognition effect, but also increase the data volume. Meanwhile, the large breadth means that one remote sensing image contains more ground feature information, and the target needing to be identified exists in a complex ground feature background, so that the identification is easily interfered by a pseudo-target in a similar shape. Aiming at the problems, the dam sample collection system collects high-resolution satellite images, aerial photos and images shot by an unmanned aerial vehicle, obtains dam samples of different seasons, different illumination conditions and different sensors from Google Earth in a global range, and ensures the diversity of the dam sample collection. In addition, a YOLOv3 network is selected as a target detection network, the network is improved aiming at single target detection, and a backbone network Darknet53 of a YOLOv3 is changed into a lightweight network ShuffleNet V2. The deep learning model has robustness and mobility and is suitable for extracting dam targets at different time and different places.
In natural images, the object of interest usually occupies a large proportion of the whole picture. However, the remote sensing image has a large number of pixels and a large geographical range, and the target of interest is usually very small relative to the background. A large-format remote sensing image cannot be directly input into the deep learning target detection network. Aiming at the problems, before a large-breadth remote sensing image is input into a deep learning target detection network, the large-breadth remote sensing image can be normally input into the deep learning target detection and identification network through sliding window cutting, and the large-breadth remote sensing image is restored to the size of an original image through certain post-processing after detection, so that a dam information extraction result of the large-breadth remote sensing image is finally obtained. The problem that the deep learning network cannot directly carry out target detection on the large-format remote sensing image is effectively solved.
Due to the requirement of emergency management, the condition of the dam needs to be rapidly mastered at the first time, so that the dam information in the remote sensing image can be rapidly and accurately acquired.
Compared with the prior art, the invention at least has the following advantages and effects:
(1) the dam remote sensing image sample set established by the invention has the advantages of large quantity and high spatial resolution, and the target detection network can extract more abundant characteristics, so that the dam target detection model based on the improved YOLOv3 can more stably and automatically detect the dam target under the complex remote sensing background, and has the advantages of high speed, higher model precision and stronger robustness.
(2) According to the method, different types of dams are obtained, and dam targets can be rapidly and intelligently identified and detected on the optical remote sensing image; according to the invention, by performing sliding window cutting and merging post-processing on the large-format remote sensing image, the dam target can be directly identified and detected aiming at the large-format remote sensing image.
(3) According to the method, the dam target is intelligently extracted by using the dam target detection model based on the improved YOLOv3, so that the calculation efficiency is improved, and the time for detecting the dam target is shortened.
Drawings
The accompanying drawings, which are included to provide a further understanding of the application and are incorporated in and constitute a part of this application, illustrate embodiment(s) of the application and together with the description serve to explain the application and not to limit the application. In the drawings:
FIG. 1 is a flow chart of an intelligent dam remote sensing detection method based on deep learning;
FIG. 2 is a network structure of an improved Yolov3 target detection model;
FIG. 3 is a graph of loss values of training data and validation data during training of an improved YOLOv3 network model;
FIG. 4 is an example of preprocessing of a complete remote sensing image segmentation using a certain segmentation step;
FIG. 5 is a statistical scatter diagram of dam crest heights of 100 dams, which is obtained through statistics;
FIG. 6 is a remote sensing image taken by a test performed to determine an optimal segmentation step;
FIG. 7 is an example of post-processing of an intelligent detection result of a dam target with a certain segmentation step length;
FIG. 8 is a remote sensing image to be detected of the dam of the embodiment;
FIG. 9 is a diagram showing the results of dam inspection in the example;
FIG. 10 Algorithm 1 is pseudo code for the sliding window clipping algorithm of the present invention.
According to the method, a large-scale dam remote sensing image sample set with strong diversity is established, and deeper features are extracted from the dam target by using an improved YOLOv3 target detection network to carry out dam target detection and identification, so that the detection result has higher accuracy, stronger robustness and more flexible mobility. The method has the advantages that the detection of the small object from the large-scale image is one of the main problems of remote sensing image analysis, so that in the process of dividing the large-scale remote sensing image, the problem of target omission caused by the fact that a dam is cut is avoided by performing two steps of sliding window cutting according to the dividing step length before detection and combining with the confidence coefficient of a target detection frame according to the dividing step length after detection, and the detection precision is improved. Taking the application of emergency rescue in earthquake disasters as an example, for quickly locking targets of an earthquake-caused dam and helping rescuers to quickly master the condition of the dam and winning time for emergency rescue, a large-format remote sensing image after 2008 '5.12 Wenchuan earthquake' (example 1) is tested, two dam targets in the image can be accurately detected, the position of the dam and a boundary frame of the dam are obtained, and the effectiveness of the invention is verified.
Detailed Description
In order to make the objects, technical solutions and advantages of the present invention more apparent, embodiments of the present invention will be described in further detail below with reference to fig. 1 to 10.
As shown in fig. 1-10, in order to solve the problems of large workload, low timeliness and the like of traditional remote sensing disaster information extraction, the invention utilizes a Convolutional Neural Network (CNN) to automatically select optimal features and establish a dam target detection model. However, because the number of pixels of the remote sensing image is large, one large-format remote sensing image cannot be directly input into a deep learning target detection network, the invention provides a dam remote sensing intelligent detection method based on deep learning, a Yolov3 network with higher speed is selected as a frame, a main network Darknet53 is replaced by a light-weight network ShuffleNet V2, the target detection speed is increased, and the problem that CNN cannot directly detect the large-format remote sensing image is solved by carrying out two steps of segmentation before detection and combination after detection on the large-format remote sensing image. By the dam remote sensing intelligent detection method based on deep learning, provided by the invention, the intelligent extraction of the dam target can be realized, and the dam target information can be accurately obtained at the first time.
The process of the dam remote sensing intelligent detection method based on deep learning is shown in figure 1, and comprises 7 units, wherein the dam remote sensing intelligent detection method mainly comprises two parts of dam target detection model establishment and large-format remote sensing image dam target detection. The process of building the dam target detection model comprises the following steps: firstly, collecting and preprocessing the remote sensing image after earthquake, then establishing a dam remote sensing image sample set, and further establishing a dam target detection model based on improved YOLOv 3. In order to obtain a dam target detection result of the large-format remote sensing image, the large-format remote sensing image is firstly segmented, then dam target detection is carried out on the segmented image to obtain a dam information extraction result, and finally the dam information extraction result is subjected to post-processing.
The dam remote sensing intelligent detection method based on deep learning selects a YOLOv3 network as a basic framework, and the network structure is shown in FIG. 2. The YOLOv3 can effectively improve the accuracy of small target detection through multi-scale prediction. Fig. 3 shows the loss values of training data and validation data in the training process of the improved YOLOv3 network model. On the basis, the large-format remote sensing image is segmented before detection and combined after detection, the segmentation method before detection is sliding window cutting, as shown in figure 4, a specific algorithm flow is shown in algorithm 1 (figure 10), the problem that complete dam targets are possibly incomplete due to direct segmentation is solved, dam crest length statistics is carried out on 100 randomly selected dam targets, and a statistical scatter diagram is shown in figure 5. According to the dam crest length of the dam and the resolution of the remote sensing images, an optimal segmentation step length test is carried out through two remote sensing images of Wenchuan county in Sichuan as shown in fig. 6, test results are shown in table 1, and the purpose of merging small image detection results is to merge the small image detection results into an original image size as shown in fig. 7.
TABLE 1 statistical table of test results for determining optimal segmentation step length according to the present invention
Example 1:
a dam remote sensing intelligent detection method based on deep learning comprises the following steps:
100. collecting and preprocessing remote sensing images: taking 2008 '5.12 earthquakes' as an example, high-resolution satellite images, aerial photos and images shot by an unmanned aerial vehicle in Wenchuan earthquake disaster areas are collected, remote sensing data sources suitable for dam target detection are screened out, and the remote sensing images are preprocessed, wherein the preprocessing comprises radiation correction, geometric correction, image fusion and the like.
101. Establishing a dam remote sensing image sample set: and collecting and marking dam remote sensing image samples, and establishing a dam remote sensing image sample set.
102. Building a dam target detection model: based on a YOLOv3 framework, a lightweight network ShuffleNet2 is used for replacing an original trunk network Darknet53, the model is trained and tested, the model accuracy reaches 71.89%, and then the dam target detection model based on the improved YOLOv3 is obtained.
103. Dividing the large-format remote sensing image: the input image is a remote sensing image of Wenchuan disaster area of '5.12 earthquakes' in 2008, and the resolution is 1.5 m. Setting the segmentation step length as 133, and performing sliding window clipping on the input remote sensing image.
104. And (3) dam target detection: based on the dam target detection model of the improved YOLOv3, inputting the cut image of each sliding window into the dam target detection model based on the improved YOLOv3 to obtain the dam target detection result of each small image;
105. and (3) dam information extraction result post-processing: and according to the segmentation step length 133, the segmented small images are recombined into the original large-format remote sensing image size.
106. And outputting a dam target detection result graph, and giving the position of the dam target and the information of the external bounding box.
The embodiment of the invention is realized on a PC platform, and the test proves that an ideal result can be obtained.
The invention provides a dam remote sensing intelligent detection method based on deep learning, which solves the problems that the traditional method and the existing machine learning method are large in workload, low in timeliness and less in large-format remote sensing images in target detection and the like. The dam target detection test of a large-format remote sensing image of the '5.12 earthquake' Wenchuan catarch area in 2008 proves that the method can effectively improve the dam target detection efficiency, 2 dam targets in fig. 8 can be completely and accurately detected, and the dam target detection bounding box is shown in fig. 9.
The above examples are only preferred embodiments of the present invention, and are not intended to limit the embodiments of the present invention. Other variations and modifications may be made to the above-described embodiments, and not all embodiments may be exhaustive. All obvious changes and modifications of the technical scheme of the invention are within the protection scope of the invention.
Claims (8)
1. A dam remote sensing intelligent detection method based on deep learning comprises the following steps:
A. establishing a dam remote sensing image sample set: collecting dam remote sensing image samples and marking dam targets so as to establish a dam remote sensing image sample set;
B. establishing a dam target detection model based on improved YOLOv 3: based on a YOLOv3 framework, a light-weight network ShuffleNet V2 is used for replacing an original trunk network Darknet53, and the model is trained and tested to obtain a dam target detection model based on an improved YOLOv 3;
C. dividing the large-format remote sensing image: according to the segmentation step length, segmenting the input large-format remote sensing image by using a sliding window;
D. and (3) dam target detection: performing dam target detection on the segmented remote sensing image data based on a dam target detection model of improved YOLOv 3;
E. post-processing a dam target detection result: and recombining the remote sensing images after segmentation into an image with the same size as the original remote sensing image according to the segmentation step length and the confidence coefficient of the detection boundary frame.
2. The intelligent dam remote sensing detection method based on deep learning of claim 1, wherein in the step (A), firstly, a high-resolution satellite image, an aerial image and an unmanned aerial vehicle are used for shooting remote sensing images, adaptive analysis on dam target detection is carried out, a data source for dam target detection is screened out, a dam remote sensing image sample set is further established, then, on the basis, a deep learning target detection marking tool LabelImg is used for marking targets, and a circumscribed rectangular frame and a category of a dam are stored in an XML file.
3. The dam remote sensing intelligent detection method based on deep learning of claim 1, wherein in the step (B), based on a YOLOv3 model framework, a trunk network of the dam remote sensing intelligent detection method is changed from Darknet53 to a lightweight network ShuffleNet2 for model training; and after the model training is finished, inputting the image to be detected into the trained dam target detection model based on the improved YOLOv3 to generate a corresponding dam target detection result.
4. The dam remote sensing intelligent detection method based on deep learning of claim 1, wherein in the step (C), the size of the input image of the YOLOv3 network is 416 x 416 pixels, before the remote sensing image is input into the network, a sliding window clipping algorithm is adopted to perform a segmentation operation, and a corresponding segmentation step length is utilized to ensure that adjacent image blocks are overlapped, so that the model can detect a relatively complete dam target.
5. The dam remote sensing intelligent detection method based on deep learning of claim 4, wherein the sliding window clipping algorithm comprises the following specific processes: firstly, acquiring the times of dividing a large-format remote sensing image to be detected in the row and column directions respectively, and then dividing the image according to the formula (1) or the rows; if the last image is cut out of the image after the cutting, the last cutting block is translated forwards to enable the edge of the last cutting block to be overlapped with the edge of the original large-format remote sensing image, and finally the cutting of a complete remote sensing image is realized;
equation (1) is as follows:
in the formula (1), Image is an original large-format remote sensing Image; the cropImage is a divided image; i and j represent the images divided from the second block in the column direction and the row direction, respectively; nh and nw are the number of times of division in the column direction and the row direction, respectively; height and width represent the number of pixels in the column direction and the row direction of the large-format remote sensing image; the side represents the size of the target image, and the length and the width of the target image are the same; stride represents the segmentation step size for sliding window segmentation.
6. The intelligent dam remote sensing detection method based on deep learning of claim 4, wherein in the step (C), the pixel range and the segmentation step length of the dam on remote sensing images with different resolutions are determined according to the dam crest length of the dam; the dam crest length statistics of 100 dams are randomly selected from dam data disclosed by the national energy agency dam safety monitoring center, more than 95% of the dam crest length is 50-600m, the dam crest length is obtained by a 2m resolution remote sensing image segmentation step length test, and when the segmentation step length is 100, a complete dam target can be detected; for the remote sensing image with higher resolution, the step length of the segmentation can be calculated according to the formula (2):
in the formula (2), r is the resolution of the image, and stride is the segmentation step size.
7. The intelligent dam remote sensing detection method based on deep learning of claim 1, wherein in the step (D), the remote sensing images with the size of 416 x 416 pixels after being segmented are respectively input into the trained dam target detection model, and dam target detection results with corresponding sizes are obtained.
8. The intelligent dam remote sensing detection method based on deep learning as claimed in claim 1, wherein in the step (E), firstly, according to the segmentation stepThe images after the detection are respectively merged according to rows and columns, and the confidence level in the Yolov3 network is Pr(Object)*IOUtruth predWherein P isr(object) indicates the probability that the bounding box contains an object, IOUtruth predRepresenting the intersection ratio between the predicted bounding box and the real bounding box, and when there is an object in the predicted bounding box, the confidence is equal to IOUtruth predWhen no target exists in the prediction frame, the confidence coefficient is 0, when the overlapped part of two adjacent images detects the same dam target, the reserved part is determined according to the confidence coefficient of the boundary frame of the target, the boundary frame with high confidence coefficient is reserved, and the boundary frame with low confidence coefficient is discarded.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110310464.2A CN113033390B (en) | 2021-03-23 | 2021-03-23 | Dam remote sensing intelligent detection method based on deep learning |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110310464.2A CN113033390B (en) | 2021-03-23 | 2021-03-23 | Dam remote sensing intelligent detection method based on deep learning |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN113033390A true CN113033390A (en) | 2021-06-25 |
| CN113033390B CN113033390B (en) | 2022-12-13 |
Family
ID=76473055
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202110310464.2A Active CN113033390B (en) | 2021-03-23 | 2021-03-23 | Dam remote sensing intelligent detection method based on deep learning |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN113033390B (en) |
Cited By (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113763196A (en) * | 2021-08-23 | 2021-12-07 | 中国农业大学烟台研究院 | Orchard yield measuring system based on improved YOLOv3 |
| CN114170493A (en) * | 2021-12-02 | 2022-03-11 | 江苏天汇空间信息研究院有限公司 | Method for improving semantic segmentation precision of remote sensing image |
| CN115423829A (en) * | 2022-07-29 | 2022-12-02 | 江苏省水利科学研究院 | Method and system for rapidly extracting water body from single-band remote sensing image |
| CN115861861A (en) * | 2023-02-27 | 2023-03-28 | 国网江西省电力有限公司电力科学研究院 | Lightweight acceptance method based on unmanned aerial vehicle distribution line inspection |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107527352A (en) * | 2017-08-09 | 2017-12-29 | 中国电子科技集团公司第五十四研究所 | Remote sensing Ship Target contours segmentation and detection method based on deep learning FCN networks |
| CN107818326A (en) * | 2017-12-11 | 2018-03-20 | 珠海大横琴科技发展有限公司 | A kind of ship detection method and system based on scene multidimensional characteristic |
| CN107871124A (en) * | 2017-11-15 | 2018-04-03 | 陕西师范大学 | A kind of Remote Sensing Target detection method based on deep neural network |
| CN110175658A (en) * | 2019-06-26 | 2019-08-27 | 浙江大学 | A kind of distress in concrete recognition methods based on YOLOv3 deep learning |
-
2021
- 2021-03-23 CN CN202110310464.2A patent/CN113033390B/en active Active
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107527352A (en) * | 2017-08-09 | 2017-12-29 | 中国电子科技集团公司第五十四研究所 | Remote sensing Ship Target contours segmentation and detection method based on deep learning FCN networks |
| CN107871124A (en) * | 2017-11-15 | 2018-04-03 | 陕西师范大学 | A kind of Remote Sensing Target detection method based on deep neural network |
| CN107818326A (en) * | 2017-12-11 | 2018-03-20 | 珠海大横琴科技发展有限公司 | A kind of ship detection method and system based on scene multidimensional characteristic |
| US20200167601A1 (en) * | 2017-12-11 | 2020-05-28 | Zhuhai Da Hengqin Technology Development Co., Ltd. | Ship detection method and system based on multidimensional scene features |
| CN110175658A (en) * | 2019-06-26 | 2019-08-27 | 浙江大学 | A kind of distress in concrete recognition methods based on YOLOv3 deep learning |
Non-Patent Citations (1)
| Title |
|---|
| HAOJIE MA 等: "Detection of Collapsed Buildings in Post-Earthquake Remote Sensing Images Based on the Improved YOLOv3", 《REMOTE SENSING》 * |
Cited By (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113763196A (en) * | 2021-08-23 | 2021-12-07 | 中国农业大学烟台研究院 | Orchard yield measuring system based on improved YOLOv3 |
| CN114170493A (en) * | 2021-12-02 | 2022-03-11 | 江苏天汇空间信息研究院有限公司 | Method for improving semantic segmentation precision of remote sensing image |
| CN115423829A (en) * | 2022-07-29 | 2022-12-02 | 江苏省水利科学研究院 | Method and system for rapidly extracting water body from single-band remote sensing image |
| CN115423829B (en) * | 2022-07-29 | 2024-03-01 | 江苏省水利科学研究院 | Method and system for rapidly extracting water body of single-band remote sensing image |
| CN115861861A (en) * | 2023-02-27 | 2023-03-28 | 国网江西省电力有限公司电力科学研究院 | Lightweight acceptance method based on unmanned aerial vehicle distribution line inspection |
Also Published As
| Publication number | Publication date |
|---|---|
| CN113033390B (en) | 2022-12-13 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN113033390B (en) | Dam remote sensing intelligent detection method based on deep learning | |
| CN111062361B (en) | River and lake shoreline sand production monitoring and analyzing method and device | |
| CN109598290A (en) | A kind of image small target detecting method combined based on hierarchical detection | |
| CN111461291A (en) | Long-distance pipeline inspection method based on YO L Ov3 pruning network and deep learning defogging model | |
| CN111028255A (en) | Farmland area pre-screening method and device based on prior information and deep learning | |
| CN111126184B (en) | Post-earthquake building damage detection method based on unmanned aerial vehicle video | |
| CN106650812A (en) | City water body extraction method for satellite remote sensing image | |
| CN106156758B (en) | A kind of tidal saltmarsh method in SAR seashore image | |
| CN104809724A (en) | Automatic precise registration method for multiband remote sensing images | |
| CN110837839B (en) | High-precision unmanned aerial vehicle orthographic image manufacturing and data acquisition method | |
| CN105184317A (en) | License plate character segmentation method based on SVM classification | |
| CN115661032A (en) | Intelligent pavement disease detection method suitable for complex background | |
| CN113902792A (en) | Building height detection method and system based on improved RetinaNet network and electronic equipment | |
| CN114067245A (en) | Method and system for identifying hidden danger of external environment of railway | |
| CN112166688B (en) | Method for monitoring desert and desertification land based on minisatellite | |
| CN116993681A (en) | Substation inspection defect detection method and system | |
| CN106845498A (en) | With reference to the single width mountain range remote sensing images landslide detection method of elevation | |
| Yang et al. | Automatic Identification Method of Construction and Demolition Waste Based on Deep Learning and GAOFEN-2 Data | |
| CN115661932A (en) | Fishing behavior detection method | |
| CN112907567B (en) | SAR image ordered artificial structure extraction method based on spatial reasoning method | |
| CN115273009A (en) | Road crack detection method and system based on deep learning | |
| CN114140703A (en) | Intelligent recognition method and system for forest pine wood nematode diseases | |
| Kouchi et al. | Damage detection based on object-based segmentation and classification from high-resolution satellite images for the 2003 Boumerdes, Algeria earthquake | |
| CN109886142B (en) | Crop interpretation method based on SAR technology | |
| Zou et al. | Flood Depth Assessment with Location-Based Social Network Data and Google Street View-A Case Study with Buildings as Reference Objects |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |