CN116863357A - Unmanned aerial vehicle remote sensing dyke image calibration and intelligent segmentation change detection method - Google Patents

Abstract

The invention discloses a UAV remote sensing dam image calibration and intelligent segmentation change detection method. The steps include: 1) Complete multi-temporal high-resolution UAV remote sensing image collection of typical reservoir dams. 2) Perform calibration and correction processing on the collected UAV remote sensing images. 3) Intelligent extraction of reservoir dam image features based on an enhanced graph cutting algorithm based on a superpixel fusion edge operator. 4) After extracting the characteristics of the reservoir dam image, a high-precision matching technology of super-pixel block structure is used to detect changes in the dam using multi-temporal UAV remote sensing. The present invention proposes a set of UAV-based multi-temporal remote sensing dam image calibration and intelligent segmentation change detection methods with high detection accuracy, economy and convenience.

Description

A UAV remote sensing dam image calibration and intelligent segmentation change detection method

Technical field

The invention relates to a method for intelligent segmentation and change detection of dam images using UAV remote sensing technology. Belongs to the fields of computer vision, deep learning image processing and remote sensing.

Background technique

UAV remote sensing image change detection technology uses UAV airborne sensors to obtain change information in a certain area during a certain period of time. With the rapid development of UAV aircraft and airborne sensors in my country in recent years, UAV remote sensing technology has gradually improved and matured in change detection tasks. UAV remote sensing technology has many advantages, such as wide coverage, strong periodicity, and low cost. It is often used to monitor land use/cover changes, urban expansion planning, deforestation and protection, glacier melting, geographical disaster prevention and other issues. Change detection refers to extracting the change information of ground objects in the same geographical area at different time phases. Change detection is commonly used in urban planning, land use, vegetation coverage, disaster detection, etc.

Change detection based on Remote Sensing (RS) images is an important means of detecting surface changes. It is an effective method whether in urban planning, disaster prevention, environmental protection or agricultural census. Currently, the use of UAV remote sensing data to monitor changes in dams is an effective technical means. However, UAV remote sensing image data has the characteristics of being easily affected by the environment, large image data resolution, and huge data volume. It is difficult to use remote sensing images to monitor reservoir dam changes. There is currently no mature solution for change detection.

Contents of the invention

Based on this, the purpose of the present invention is to provide a UAV remote sensing dam image calibration and intelligent segmentation change detection method, aiming to provide a mature solution for using remote sensing images to detect changes in reservoir dams.

In order to achieve the above purpose, the present invention adopts the following technical solution: a UAV remote sensing dam image calibration and intelligent segmentation change detection method, the steps include: 1) Complete multi-temporal high-resolution UAV remote sensing of typical reservoir dams Image collection. 2) Perform calibration and correction processing on the collected UAV remote sensing images. 3) An enhanced graph cutting algorithm based on LSC (LinearSpectral Clustering) superpixel fusion edge operator to intelligently extract reservoir dam image features. 4) After extracting the characteristics of the reservoir dam image, a high-precision matching technology of super-pixel block structure is used to detect changes in the dam using multi-temporal UAV remote sensing.

The step 1) Completes high-resolution drone remote sensing image collection of typical reservoir dams through drone remote sensing. By setting three flight heights of 50m, 100m, and 200m, the ground dam remote sensing image pixels are 1:2:4. The size ratio is used to collect UAV remote sensing images of the dam body.

Said step 2) performs calibration and correction processing on the UAV remote sensing images after the data collection is completed, including the following steps: ① Use the radial-eccentric-image plane improved distortion model to calibrate the remote sensing images captured by the UAV airborne camera Correction; ② All corrected UAV remote sensing images need to be orthorectified; ③ After obtaining the orthoimage, the image needs to be rotated and intercepted; ④ Use methods based on histogram equalization and normalization to perform multi-temporal non-linear correction. Color unification of human-machine remote sensing images.

The step 3) uses the enhanced graph cutting algorithm based on the LSC superpixel fusion edge operator to intelligently extract the characteristics of the reservoir dam body, including the following steps: ① Use the idea of using points to replace surfaces, and divide each LSC superpixel segmentation result into The superpixel block is mapped to a point and rearranged into a matrix, and edge operators are introduced; ② The new mapping matrix integrated with the edge operator is input into the graph cut algorithm, and an enhanced graph cut algorithm is constructed for segmentation; ③ Based on the superpixel segmentation results , restore the resolution based on the index of the pixels within each superpixel block.

Step 4) After extracting the characteristics of the reservoir dam, detect changes in the dam from multi-temporal drone remote sensing images, including the following steps: ① Extract and match feature points on images of multiple phases. Using superpixels as feature points, feature matching adopts a high-precision matching technology of superpixel face block structure; ② The homography transformation matrix is calculated through the matched features, and the image is unified in a coordinate system through homography transformation (perspective transformation) Next; ③ Perform differential operation on the mask images of the dam extraction results in multiple phases in the same coordinate system to obtain the mask map of the dam change results, that is, the change results of the UAV remote sensing dam image.

The present invention adopts the above technical solution, which has the following advantages:

1. A set of UAV-based multi-temporal remote sensing dam image calibration and intelligent segmentation change detection methods are proposed. Through a series of processes such as calibration, orthorectification, color consistency adjustment, superpixel fusion edge segmentation, and high-precision matching of superpixel structures, this method has high detection accuracy and is economical and convenient compared with other existing methods.

2. The proposed enhanced graph cutting algorithm based on LSC superpixel fusion edge operator can effectively perform intelligent segmentation and extraction of large-resolution UAV remote sensing images. By mapping superpixels into points and rearranging them into mapping matrices and passing them into the segmentation algorithm, hardware consumption and computing time are greatly reduced; by introducing an enhanced graph cut algorithm with edge operators, the accuracy and smoothness of segmentation edges are increased.

Description of the drawings

In order to more clearly illustrate the specific embodiments of the present invention or the technical solutions in the prior art, the following will briefly introduce some specific embodiments or the drawings that need to be used in the description of the prior art. Obviously, the following description The drawings in are only some embodiments of the present invention. For those of ordinary skill in the art, other drawings can be obtained based on these drawings without exerting creative efforts.

Figure 1 is a technical roadmap for a UAV remote sensing dam image calibration and intelligent segmentation change detection method provided by the present invention;

Figure 2 is the flow chart of the Otsu method;

Figure 3 is the flow chart of dam target extraction using the enhanced graph cut algorithm of LSC superpixel fusion edge operator;

Figure 4 is a flow chart of dam change detection using two-phase images;

Figure 5 shows the two-phase change detection results of the reservoir dam.

Detailed ways

The present invention will be described in detail below with reference to the drawings and examples.

The invention is a UAV remote sensing dam image calibration and intelligent segmentation change detection method. The steps include:

1. Collect a large amount of multi-temporal high-resolution image data of typical reservoirs through UAV remote sensing. By setting three flight heights of 50m, 100m, and 200m, the 1:2:4 size ratio of the ground remote sensing image pixels of the dam body can be achieved. Field data UAV remote sensing image collection.

2. After the data collection is completed, perform calibration and correction processing of high-resolution UAV remote sensing images, including the following steps:

1) Use an improved camera distortion correction model to correct images captured by UAV airborne cameras. Due to camera distortion, there is an offset between the actual point coordinates and the ideal point coordinates. The following formula is used to establish an improved distortion model of radial-eccentricity-image plane to correct the images captured by the drone's airborne camera. Its correction model is: ,in: , in the formula are the ideal feature point coordinates, is the actual feature point coordinates, , respectively and Distortion in direction, , respectively and radial distortion in the direction, , respectively and Eccentric distortion in the direction, , for and plane distortion in the direction, is the radial distortion coefficient, is the eccentric distortion coefficient, is the image plane distortion coefficient, is the scale of the image plane distortion coefficient.

2) Perform orthorectification on all corrected UAV remote sensing images. Orthorectification can produce a digital elevation data model containing three-dimensional coordinate information by analyzing the GPS (Global Positioning System) geographical information of multiple images and combining it with camera parameters. Finally, an orthophoto is generated.

3) After obtaining the orthophoto, rotate and intercept the image, including the following steps:

① Use the Otsu method to obtain adaptive binarized images. The Otsu method process is shown in Figure 2. The specific steps are as follows:

for an image , the binary segmentation threshold of the front and rear background of the image is set to , the proportion of foreground pixels after binary segmentation is set to , its average gray level . After binary segmentation, the proportion of pixels belonging to the background is set to , its average gray level is . The total average grayscale of the image is recorded as , the inter-class variance is recorded as . Assuming that the contrast of the image is high, record the size of the image as , the gray value is less than The number of pixels is , the gray value is greater than The number of pixels is , then there is:

The pixel classification relationships are:

in:

Substitute the formula into:

At this time, the final inter-class variance is obtained after traversing the gray value of the entire image. pixel grayscale threshold under This is the desired binarization threshold.

② Morphological denoising and adhesion removal, and the main contour of the dam body is extracted;

③ Calculate the minimum circumscribed rectangle of the dam body outline;

④ Calculate the deflection angle of the rectangle and rotate the image;

⑤Cut out the main body of the reservoir dam.

4) Use methods based on histogram equalization and normalization to unify the colors of multi-temporal UAV remote sensing images. First, histogram equalization is performed on both the standard image and the target image to become the same standardized uniform histogram; then the histogram mapping table of the two images is obtained based on this uniform histogram (similar mapping according to the situation). The pixel mapping table is obtained through the histogram mapping table, and then each pixel value in the target image is recalculated through the pixel mapping table.

3. The enhanced graph cutting algorithm based on the LSC superpixel fusion edge operator intelligently extracts the image features of the reservoir dam. The specific process is shown in Figure 3, including the following steps:

1) Using the idea of replacing surfaces with points, map each superpixel block in the LSC superpixel segmentation result to a point and rearrange it into a matrix.

2) Input the new mapping matrix integrated with the edge operator into the graph cut algorithm, and construct an enhanced graph cut algorithm for segmentation. The specific steps are as follows:

Has a three-channel color image , all pixels of the image are constructed into a gray value matrix, a set is the grayscale value of the pixel in the BGR color channel, where is the index number; the segmentation of the image is determined by a fuzzy value set means that for the two-classification problem there is ∈ {0, 1}, where 0 means that the current pixel belongs to the background and 1 means that it belongs to the foreground.

Introduce edge operators into graph cut algorithms. The directional gradient is , The directional gradient is ,definition for pixels and The possibility value of an edge exists between them, using the gradient approximation as the edge operator:

Define a new smoothing term:

in, and is the gray value vector of the BGR color space, and uses the L2 norm to measure the color feature similarity of two pixels. When the difference between two pixels is greater, the more likely they are to be assigned different labels; Used to adjust the influence of the contrast of adjacent pixels when the image contrast is different, is the weighted harmonization factor of gray intensity and gradient intensity in the smoothing function, when When set to 1, it is the original graph cut algorithm, generally set to 0.5.

The data items in the graph cut algorithm are defined as follows:

;

in is a specific threshold, is the probability that the current pixel belongs to the foreground or background.

Finally, the energy function of the reconstructed enhanced graph cut algorithm is:

in, Is the balance factor, used to balance the weight of data items and smoothing items, generally 0.5. The segmentation results of the enhanced graph cut algorithm are more robust and accurate.

3) Restore the resolution based on the pixel index within each superpixel block in the segmentation result.

4. After extracting the characteristics of the reservoir dam, detecting changes in the dam using multi-temporal drone remote sensing images specifically includes the following steps:

1) Feature point extraction and matching of images of multiple phases; superpixels are used as feature points, and feature point matching adopts a high-precision matching technology of superpixel block structure. Based on the photometric consistency constraint in the optical flow method area matching and the pixel-by-pixel cost matching method in the stereo dense matching, the energy cost function of the internal face blocks of the superpixel structure is established.

in, Represents the superpixel structure area, Represents the arrangement number of the superpixel structure sequence, Represents color information, represents gradient information, Representation and reference pixels corresponding pixel. is the penalty function .

2) Calculate the homography transformation matrix through the matched superpixel feature points, through homography transformation (perspective transformation), relying on the formula , unifying the images in the same coordinate system. in It is a phase 1 image, It is a phase 2 image, is a homography matrix calculated based on four pairs of matching points from two images. Unifying images into a coordinate system specifically includes the following steps:

① Calculate the homography transformation matrix through the superpixel matching features of multi-temporal images;

Define homography matrix , represents the mapping between two planes, the expression is as follows:

Let any point in the left image of the two-dimensional image And the corresponding matching point in the picture on the right , through the mapping relationship:

Write the transformation as a matrix form:

It can be seen from the above formula that a set of matching feature points can obtain two sets of equations, and the homography matrix There are 9 unknown quantities. But there are actually only 8 degrees of freedom, and constraints can usually be added such that , because there is:

in is the scale factor, so there is:

It can be seen from the above formula that adding a scale factor has no effect at all on the mapping of points.

make , which makes have:

Get the homography matrix There are actually only 8 degrees of freedom, and it can be solved with at least 4 pairs of non-collinear matching points. Since more than 4 pairs of excellent matching points can be found in the actual feature point matching process, overdetermined problems are generally solved using the least squares method to construct a maximum likelihood function to improve registration accuracy;

②After calculating the homography matrix, reproject each pixel of the image through the homography transformation matrix to generate a new projection image;

③ Interpolate the pixels of the remapped image to unify the image in a coordinate system.

4) Perform differential operations on the mask images of the dam extraction results in multiple phases in the same coordinate system to obtain the mask map of the dam change results, that is, the change results of the drone remote sensing dam image.

The above descriptions are only preferred embodiments of the present invention and are not intended to limit the present invention. Those skilled in the art should understand that any modifications, equivalent substitutions or improvements, etc., made within the spirit and principles of the present invention, All should be included within the protection scope of the present invention, and the protection scope shall be defined by the claims.

Claims (5)

1. A UAV remote sensing dam image calibration and intelligent segmentation change detection method, the steps include: 1) Complete the collection of multi-temporal high-resolution UAV remote sensing images of typical reservoir dams; 2) Perform calibration and correction processing on the collected UAV remote sensing images; 3) Intelligent extraction of reservoir dam image features based on an enhanced graph cutting algorithm based on a superpixel fusion edge operator; 4) After extracting the characteristics of the reservoir dam image, a high-precision matching technology of super-pixel block structure is used to detect changes in the dam using multi-temporal UAV remote sensing. 2. The method according to claim 1, characterized in that by setting three flight heights of 50m, 100m, and 200m, the ground dam remote sensing image pixels can be collected in a 1:2:4 size ratio for the dam body UAV remote sensing image. . 3. The method of claim 1, wherein the step of performing calibration and correction processing on UAV remote sensing images includes: a) Calibration and correction processing of UAV remote sensing images; due to the distortion of UAV remote sensing images, there is a deviation between the actual point coordinates and the ideal point coordinates. Use the following formula to establish the improved distortion of the radial-eccentric-image plane calibration model ;in: , in the formula are the ideal feature point coordinates, is the actual feature point coordinates, , respectively and Distortion in direction, , respectively and radial distortion in the direction, , respectively and Eccentric distortion in the direction, , for and plane distortion in the direction, is the radial distortion coefficient, is the eccentric distortion coefficient, is the image plane distortion coefficient, is the scale of the image plane distortion coefficient; b) Perform orthorectification on the corrected UAV remote sensing images; c) After obtaining the orthophoto, the image needs to be rotated and intercepted, and finally the color of the multi-temporal UAV remote sensing image is unified using a method based on histogram equalization and normalization. 4. The method according to claim 1, characterized in that an enhanced graph cutting algorithm of LSC (Linear Spectral Clustering) super-pixel fusion edge operator is proposed to perform target extraction on the image features of the reservoir dam; the purpose of this extraction algorithm is Steps include: a) Using the idea of replacing surfaces with points, map each superpixel block in the LSC superpixel segmentation result to a point and rearrange it into a matrix, and introduce the following edge operators; The directional gradient is , The directional gradient is ,definition for pixels and Possibility value of edge between them, using gradient approximation as edge operator ; b) Input the new mapping matrix integrated into the edge operator into the graph cut algorithm, and construct an enhanced graph cut algorithm for segmentation; the characteristic of enhanced segmentation is to define a new smoothing term ; in, and is the gray value vector of the BGR color space, and uses the L2 norm to measure the color feature similarity of two pixels. When the difference between two pixels is greater, the more likely it is to be assigned to different labels. Used to adjust the influence of the contrast of adjacent pixels when the image contrast is different, is the weighted harmonization factor of gray intensity and gradient intensity in the smoothing function, when When set to 1, it is the original graph cut algorithm, generally set to 0.5; the data items in the graph cut algorithm are defined as: ,in is a specific threshold, is the probability that the current pixel belongs to the foreground or background, is the grayscale value of the pixel in the BGR color channel, ∈{0: background, 1: foreground}; the energy function of the reconstructed enhanced graph cut algorithm is ,in, Is the balance factor, used to balance the weight of data items and smoothing items, generally 0.5. The segmentation results of the enhanced graph cut algorithm are more robust and accurate; c) Based on the segmentation result of the previous step, restore the resolution based on the index of the pixels in each superpixel block. 5. The method of claim 1, wherein the step of detecting changes in the dam from multi-temporal UAV remote sensing images includes; a) Extract and match feature points from images of multiple phases. Using superpixels as feature points, feature point matching adopts a high-precision matching technology of superpixel block structure. The key to high-precision matching technology for super-pixel face block structures is to establish the energy cost function of the face blocks within the super-pixel structure. ; in, Represents the superpixel structure area, Represents the arrangement number of the superpixel structure sequence, Represents color information, represents gradient information, Representation and reference pixels corresponding pixel. is the penalty function ; b) Calculate the homography transformation matrix through the matched superpixel feature points, and unify the image under a coordinate system through homography transformation; c) Perform a differential operation on the dam extraction result mask images of multiple phases in the same coordinate system to obtain the dam change result mask image, that is, the UAV remote sensing dam image change results.