CN111666858A - Forest remote sensing image registration method and system based on single tree recognition - Google Patents

Abstract

The invention discloses a forest remote sensing image registration method and system based on single tree identification, and relates to the technical field of image processing. The method includes: recognizing and locating the single tree in the reference image and the image to be registered according to the local maximum method; calculating the local descriptor of the i-th single tree identified in the reference image, and calculating the image to be registered The local descriptor of the jth single tree identified in The similarity is determined to obtain the registration result of the image to be registered. The invention realizes the registration of images with high self-similarity such as forest images, and can obtain a registration result with higher reliability.

Description

Forest remote sensing image registration method and system based on single tree recognition

technical field

The invention relates to the technical field of image processing, in particular to a forest remote sensing image registration method and system based on single tree identification.

Background technique

At present, with the rapid development of UAV technology, UAVs are widely used in real-time monitoring of forests in the field of forestry. Since the remote sensing data obtained by UAV requires less manpower, simple operation and low cost, it can be used to assist the forest land survey on the ground, and can monitor the forest in real time, which greatly improves the time-consuming and labor-intensive traditional ground survey and the input of a large number of personnel. It can realize the research of forest biomass detection and estimation and the continuous detection of forest area changes.

However, the forest images collected by drones need to be registered first. Traditional images such as buildings, roads and other targets have distinct characteristics, so the realization is relatively simple. For forest images, vegetation areas account for the majority of the image area. Vegetation images belong to natural scene images and have self-similarity. Therefore, errors are prone to occur when registering feature points. The image is not universal, and the accuracy of the calculated registration point pair is low.

SUMMARY OF THE INVENTION

The technical problem to be solved by the present invention is to provide a forest remote sensing image registration method and system based on single tree identification, aiming at the deficiencies of the prior art.

The technical scheme that the present invention solves the above-mentioned technical problems is as follows:

A forest remote sensing image registration method based on single tree identification, comprising:

Obtain the benchmark image and the image to be registered of the target forest;

Identify and locate the single tree in the reference image and the to-be-registered image respectively according to the local maximum method;

Calculate the local descriptor of the ith single tree identified in the reference image, and calculate the local descriptor of the jth single tree identified in the to-be-registered image;

Using the Euclidean distance of the feature vector of a single tree as a similarity determination criterion, the local descriptor of the jth single tree and the local descriptor of the ith single tree are used to determine the similarity, and the to-be-registered tree is obtained. Image registration result;

Wherein, i=1,2,...,I,I is the number of single trees identified in the reference image, j=1,2,...,J,J is the number of trees identified in the image to be registered the number of single wood.

The beneficial effects of the present invention are as follows: the forest remote sensing image registration method provided by the present invention locates and identifies a single tree in the forest by using the local maximum method, and then uses the gradient direction distribution characteristics of the pixels in the neighborhood of the single tree position to be each Each point specifies the direction parameter, so that the operator has rotation invariance, and then generates a descriptor according to the feature description information of the single tree position point, and then uses the Euclidean distance of the single tree position point feature vector as the similarity of the key points in the two images. It realizes the registration of images with high self-similarity such as forest images, and can obtain registration results with higher reliability.

Another technical scheme that the present invention solves the above-mentioned technical problem is as follows:

A forest remote sensing image registration system based on single tree identification, comprising:

an image acquisition unit, used for acquiring the reference image of the target forest and the image to be registered;

A single tree identification unit, used to identify and locate the single tree in the reference image and the to-be-registered image respectively according to the local maximum method;

a descriptor computing unit, used to calculate the local descriptor of the i-th single tree identified in the reference image, and calculate the local descriptor of the j-th single tree identified in the to-be-registered image;

The similarity judgment unit is used to use the Euclidean distance of the feature vector of the single tree as the similarity judgment standard, and perform similarity judgment between the local descriptor of the jth single tree and the local descriptor of the ith single tree , to obtain the registration result of the to-be-registered image;

Wherein, i=1,2,...,I,I is the number of single trees identified in the reference image, j=1,2,...,J,J is the number of trees identified in the image to be registered the number of single wood.

Advantages of additional aspects of the invention will be set forth, in part, from the following description, and in part will become apparent from the following description, or may be learned by practice of the invention.

Description of drawings

1 is a schematic flowchart of an embodiment of a forest remote sensing image registration method according to the present invention;

2 is a schematic diagram of a forest remote sensing image provided by an embodiment of a forest remote sensing image registration method of the present invention;

3 is an exemplary local maximal graph provided by an embodiment of the forest remote sensing image registration method of the present invention;

4 is a schematic diagram of a key point 8*8 window provided by an embodiment of the forest remote sensing image registration method of the present invention;

5 is a schematic diagram of a weighted window provided by an embodiment of the forest remote sensing image registration method of the present invention;

FIG. 6 is a structural frame diagram provided by an embodiment of the forest remote sensing image registration system of the present invention.

Detailed ways

The principles and features of the present invention will be described below with reference to the accompanying drawings. The embodiments are only used to explain the present invention, but not to limit the scope of the present invention.

As shown in FIG. 1, it is a schematic flowchart of an embodiment of a forest remote sensing image registration method according to the present invention, and the forest remote sensing image registration method includes:

S1, obtain the reference image of the target forest and the image to be registered;

It should be noted that the reference image and the image to be registered of the target forest can be collected by drones or other aircraft. The drones can collect pictures or images. For pictures, directly select one of them. One image is sufficient. For images, you can select a frame of image with the required definition from the image sequence.

S2, according to the local maximum method, identify and locate the single tree in the reference image and the image to be registered, respectively;

It should be noted that those skilled in the art can use the specific implementation method of the local maximum method.

As shown in Figure 2, a schematic diagram of an exemplary forest remote sensing image is provided, which is realized by image processing technology. By performing the reconstruction operation based on opening and closing on the original image, local maximum processing can be realized. For the canopy of a single tree, because there are gaps between the crowns, the reference image and the image to be registered can be processed by the local maximum method to identify each single tree and its position.

For another example, for a scene with a small image or a small number of images, a search area can also be preset, and the entire image can be facilitated by sliding a window, so as to find the local largest area and determine it as a single tree.

For example, as shown in Figure 2, the pixel values between the single tree and the background are different. Assuming that the single side size of the single tree is usually between 10 and 15 pixels, the size of the search area can be set to 15*15, and the size of the search area can be set to 15*15. The average value of all pixels in the search area is used as the basis for comparison, and then the entire image is traversed through the sliding window, and the search area with the largest pixel value can be identified as a single tree.

S3, calculate the local descriptor of the i-th single tree identified in the reference image, and calculate the local descriptor of the j-th single tree identified in the image to be registered;

For example, the geometric center point of the single tree can be used as the key point, and the descriptor of the key point can be generated by the SIFT algorithm as the local descriptor of the single tree.

S4, using the Euclidean distance of the feature vector of the single tree as the similarity determination standard, the local descriptor of the jth single tree and the local descriptor of the ith single tree are used to determine the similarity, and the registration of the image to be registered is obtained. result;

Among them, i=1,2,...,I,I is the number of single trees identified in the reference image, j=1,2,...,J,J is the number of single trees identified in the image to be registered quantity.

In the forest remote sensing image registration method provided by this embodiment, a single tree in the forest is located and identified by the local maximum method, and then a direction parameter is specified for each point according to the gradient direction distribution characteristics of the pixels in the neighborhood of the single tree position point, Make the operator have rotation invariance, and then generate the descriptor according to the feature description information of the single tree position point, and then use the Euclidean distance of the feature vector of the single tree position point as the similarity judgment measure of the key points in the two images. The registration of images with high self-similarity such as forest images can obtain registration results with higher reliability.

Optionally, in some possible embodiments, after performing similarity determination between the local descriptor of the jth single tree and the local descriptor of the ith single tree, it also includes:

Through the least squares method and the RANSAC algorithm, the results of matching errors in the judgment process are eliminated.

For similarity determination, the single tree position point in the image to be registered is matched with the single tree position point in the reference image, but there may be errors in the matching due to various factors. Therefore, the least squares method can be used for matching. The two parameters are fitted, and the matching results that do not meet the expectations are eliminated through the RANSAC algorithm, so that more accurate matching results can be obtained, thereby improving the accuracy of registration.

Optionally, in some possible implementations, the single tree in the reference image and the to-be-registered image are respectively identified and positioned according to the local maximum method, specifically including:

Convert the reference image to a grayscale image, perform a reconstruction operation based on opening and closing, obtain a local maximal image, perform an erosion operation on the local maximal image, and superimpose the local maximal image processed by the erosion operation into the reference image. Identify and locate the single tree in the benchmark image;

Convert the image to be registered into a grayscale image, perform a reconstruction operation based on opening and closing, obtain a local maximal image, perform an erosion operation on the local maximal image, and superimpose the local maximal image processed by the erosion operation to the to-be-registered image. In the image, identify and locate the single tree in the image to be registered.

As shown in FIG. 3 , an exemplary local maximal image is provided. After performing the reconstruction operation based on opening and closing on the forest remote sensing image provided in FIG. 2 , the local maximal image shown in FIG. 3 is obtained.

Optionally, in some possible implementations, calculating the local descriptor of the i-th single tree identified in the reference image, specifically including:

Determine the position point of the i-th single tree identified in the benchmark image, and use the position point as a key point;

Divide the preset size area around the key point into blocks, and calculate the gradient histogram of each block separately;

Generate the feature vector of key points according to the gradient histogram;

According to the feature vector of the key point, the local descriptor of the key point is generated as the local descriptor of the ith single tree.

It should be understood that the location point of the single tree can be the center point of the single tree.

By dividing the area around the single wood location point and calculating the gradient histogram of each area separately, a unique vector can be generated. This vector is an abstraction of the image information of the area and is unique, so it can be It is used to uniquely characterize the single tree, so that the similarity of the single tree between the two images can be determined according to the local descriptor of the single tree.

Optionally, in some possible implementations, calculating the local descriptor of the jth single tree identified in the image to be registered, specifically including:

Determine the position point of the jth single tree identified in the image to be registered, and use the position point as a key point;

Divide the preset size area around the key point into blocks, and calculate the gradient histogram of each block separately;

Generate the feature vector of key points according to the gradient histogram;

According to the feature vector of the key point, the local descriptor of the key point is generated as the local descriptor of the jth single tree.

It should be understood that the location point of the single tree can be the center point of the single tree.

By dividing the area around the single wood location point and calculating the gradient histogram of each area separately, a unique vector can be generated. This vector is an abstraction of the image information of the area and is unique, so it can be It is used to uniquely characterize the single tree, so that the similarity of the single tree between the two images can be determined according to the local descriptor of the single tree.

It should be understood that the methods for generating local descriptors of a single tree in the reference image and in the image to be registered are the same. Therefore, the method for generating local descriptors is described below by taking the reference image as an example.

First, a direction is calculated for the key point, which is the basis for subsequent calculations. Using the gradient direction distribution characteristics of the pixels in the neighborhood of the key point to specify direction parameters for each key point, the operator can be rotationally invariant.

Specifically, it is possible to sample in a neighborhood window centered on the key point, and use a histogram to count the gradient directions of neighborhood pixels. The range of the gradient histogram is 0 to 360 degrees. For example, there can be a bar every 45 degrees, a total of 8 bars, or a bar every 10 degrees, a total of 36 bars, and the peak of the histogram represents the neighborhood at the key point The main direction of the gradient, which is the direction of the key point. The histogram can also be smoothed by a Gaussian function to reduce the impact of sudden changes.

Then, rotate the axes to the orientation of the key to ensure rotation invariance. For example, as shown in Figure 4, a window of 8*8 is selected with the key point as the center. Figure 5 shows the effect of Figure 4 weighted to 8 main directions. The center of Figure 4 is the position of the current key point. The grid represents a pixel in the scale space where the key point neighborhood is located. The gradient magnitude and gradient direction of each pixel are obtained by using the formula. The direction of the arrow represents the gradient direction of the pixel, and the length of the arrow represents the gradient modulus value. The weighting operation is performed. The circle in Figure 4 represents the range of Gaussian weighting, and the pixel gradient direction information that is closer to the key point contributes more. Then, the gradient direction histogram of 8 directions is calculated on each 4×4 small block, and the accumulated value of each gradient direction is drawn to form a seed point, as shown in Figure 5. A key point in Figure 5 consists of 2 × 2 seed points in total, and each seed point has 8 direction vector information. This idea of combining neighborhood directional information enhances the algorithm's ability to resist noise, and also provides better fault tolerance for feature matching with positioning errors.

In each 1/16 quadrant of 4*4, a gradient direction histogram is calculated by adding the weighted gradient value to one of the 8 direction intervals of the histogram. In this way, a 4*4*8=128-dimensional descriptor can be formed for each feature, and each dimension can represent the scale/orientation of one of the 4*4 grids. After normalizing this vector, it is further removed. the effect of light.

It can be understood that, in some embodiments, some or all of the optional implementations in the above-mentioned embodiments may be included.

As shown in FIG. 6, it is a structural framework diagram provided for an embodiment of the forest remote sensing image registration system of the present invention, and the forest remote sensing image registration system includes:

Image acquisition unit 1, for acquiring the reference image of the target forest and the image to be registered;

The single tree identification unit 2 is used to respectively identify and locate the single tree in the reference image and the image to be registered according to the local maximum method;

Descriptor computing unit 3 is used to calculate the local descriptor of the ith single tree identified in the reference image, and calculate the local descriptor of the jth single tree identified in the image to be registered;

The similarity judgment unit 4 is used to use the Euclidean distance of the feature vector of the single tree as the similarity judgment standard, and perform similarity judgment between the local descriptor of the jth single tree and the local descriptor of the ith single tree, and obtain the similarity judgment. The registration result of the registered image;

Among them, i=1,2,...,I,I is the number of single trees identified in the reference image, j=1,2,...,J,J is the number of single trees identified in the image to be registered quantity.

The forest remote sensing image registration system provided by this embodiment locates and identifies a single tree in the forest by using the local maximum method, and then specifies a direction parameter for each point according to the gradient direction distribution characteristics of the pixels in the neighborhood of the single tree position point, Make the operator have rotation invariance, and then generate the descriptor according to the feature description information of the single tree position point, and then use the Euclidean distance of the feature vector of the single tree position point as the similarity judgment measure of the key points in the two images. The registration of images with high self-similarity such as forest images can obtain registration results with higher reliability.

Optionally, in some possible implementations, the similarity judging unit 4 is further configured to eliminate the results of matching errors in the judging process through the least squares method and the RANSAC algorithm.

Optionally, in some possible implementations, the single wood identification unit 2 is specifically used to convert the reference image into a grayscale image, perform a reconstruction operation based on opening and closing, obtain a local maximum image, and erode the local maximum image. Operation processing, superimpose the local maximum image processed by the corrosion operation into the reference image, and identify and locate the single tree in the reference image;

Convert the image to be registered into a grayscale image, perform a reconstruction operation based on opening and closing, obtain a local maximal image, perform an erosion operation on the local maximal image, and superimpose the local maximal image processed by the erosion operation to the to-be-registered image. In the image, identify and locate the single tree in the image to be registered.

Optionally, in some possible implementations, the descriptor computing unit 3 is specifically used to determine the position point of the i-th single tree identified in the reference image, and the position point is used as a key point, and the preset size around the key point is determined. The area is divided into blocks, the gradient histogram of each area is calculated separately, the feature vector of the key point is generated according to the gradient histogram, and the local descriptor of the key point is generated according to the feature vector of the key point, as the local descriptor of the ith single tree .

Optionally, in some possible implementations, the descriptor computing unit 3 is specifically used to determine the position point of the j-th single tree identified in the image to be registered, using the position point as a key point, and predicting the surrounding area of the key point. Set the size area to be divided into blocks, calculate the gradient histogram of each block separately, generate the feature vector of the key point according to the gradient histogram, and generate the local descriptor of the key point according to the feature vector of the key point, as the local of the jth single tree descriptor.

It can be understood that, in some embodiments, some or all of the optional implementations in the above-mentioned embodiments may be included.

It should be noted that the above embodiments are product embodiments corresponding to the previous method embodiments, and for the description of each optional implementation in the product embodiments, reference may be made to the corresponding descriptions in the above method embodiments, which are not repeated here. Repeat.

The reader should understand that in the description of this specification, reference to the terms "one embodiment," "some embodiments," "example," "specific example," or "some examples", etc., is intended to incorporate the embodiment or example. A particular feature, structure, material, or characteristic described is included in at least one embodiment or example of the present invention. In this specification, schematic representations of the above terms are not necessarily directed to the same embodiment or example. Furthermore, the particular features, structures, materials or characteristics described may be combined in any suitable manner in any one or more embodiments or examples. Furthermore, those skilled in the art may combine and combine the different embodiments or examples described in this specification, as well as the features of the different embodiments or examples, without conflicting each other.

In the several embodiments provided in this application, it should be understood that the disclosed apparatus and method may be implemented in other manners. For example, the method embodiments described above are only illustrative. For example, the division of steps is only a logical function division. In actual implementation, there may be other division methods. For example, multiple steps may be combined or integrated into another A step, or some feature, can be ignored, or not performed.

The above are only specific embodiments of the present invention, but the protection scope of the present invention is not limited thereto. Any person skilled in the art can easily think of various equivalent modifications or modifications within the technical scope disclosed by the present invention. Replacement, these modifications or replacements should all be covered within the protection scope of the present invention. Therefore, the protection scope of the present invention should be subject to the protection scope of the claims.

Claims (10)

1. A forest remote sensing image registration method based on single tree recognition is characterized by comprising the following steps:

acquiring a reference image and an image to be registered of a target forest;

respectively identifying and positioning the single trees in the reference image and the image to be registered according to a local maximum method;

calculating a local descriptor of the ith single tree identified in the reference image, and calculating a local descriptor of the jth single tree identified in the image to be registered;

taking the Euclidean distance of the feature vector of the single tree as a similarity judgment standard, and carrying out similarity judgment on the local descriptor of the jth single tree and the local descriptor of the ith single tree to obtain a registration result of the image to be registered;

wherein I is 1, 2, … …, I is the number of singles identified in the reference image, and J is 1, 2, … …, J is the number of singles identified in the image to be registered.

2. The forest remote sensing image registration method based on single wood recognition according to claim 1, wherein after similarity determination is carried out on the local descriptor of the jth single wood and the local descriptor of the ith single wood, the method further comprises:

and eliminating the result of matching error in the judging process by a least square method and a RANSAC algorithm.

3. The forest remote sensing image registration method based on single tree recognition according to claim 1, wherein the single trees in the reference image and the image to be registered are respectively recognized and positioned according to a local maximum method, and the method specifically comprises the following steps:

converting the reference image into a gray image, performing reconstruction operation based on opening and closing to obtain a local maximum image, performing corrosion operation processing on the local maximum image, superposing the local maximum image subjected to the corrosion operation processing to the reference image, and identifying and positioning the single wood in the reference image;

converting the image to be registered into a gray image, performing reconstruction operation based on opening and closing to obtain a local maximum image, performing corrosion operation processing on the local maximum image, superposing the local maximum image subjected to the corrosion operation processing to the image to be registered, and identifying and positioning the single wood in the image to be registered.

4. The forest remote sensing image registration method based on single tree recognition according to any one of claims 1 to 3, wherein calculating a local descriptor of the ith single tree recognized in the reference image specifically comprises:

determining a position point of the ith single tree identified in the reference image, and taking the position point as a key point;

partitioning the preset size area around the key point, and respectively calculating the gradient histogram of each block area;

generating a feature vector of the key point according to the gradient histogram;

and generating a local descriptor of the key point according to the feature vector of the key point, wherein the local descriptor is used as a local descriptor of the ith single tree.

5. The forest remote sensing image registration method based on single tree recognition according to any one of claims 1 to 3, wherein calculating a local descriptor of a jth single tree recognized in the image to be registered specifically comprises:

determining a position point of the j-th single tree identified in the image to be registered, and taking the position point as a key point;

partitioning the preset size area around the key point, and respectively calculating the gradient histogram of each block area;

generating a feature vector of the key point according to the gradient histogram;

and generating a local descriptor of the key point according to the feature vector of the key point, wherein the local descriptor is used as a local descriptor of the jth single tree.

6. A forest remote sensing image registration system based on single wood recognition is characterized by comprising:

the image acquisition unit is used for acquiring a reference image and an image to be registered of the target forest;

the single-tree identification unit is used for respectively identifying and positioning the single trees in the reference image and the image to be registered according to a local maximum method;

the descriptor calculation unit is used for calculating a local descriptor of the ith single tree identified in the reference image and calculating a local descriptor of the jth single tree identified in the image to be registered;

the similarity judgment unit is used for taking the Euclidean distance of the feature vector of the single tree as a similarity judgment standard, and carrying out similarity judgment on the local descriptor of the jth single tree and the local descriptor of the ith single tree to obtain a registration result of the image to be registered;

wherein I is 1, 2, … …, I is the number of singles identified in the reference image, and J is 1, 2, … …, J is the number of singles identified in the image to be registered.

7. The forest remote sensing image registration system based on single wood recognition according to claim 6, wherein the similarity judgment unit is further used for eliminating the result of matching error in the judgment process through a least square method and a RANSAC algorithm.

8. The forest remote sensing image registration system based on single tree recognition according to claim 6, wherein the single tree recognition unit is specifically configured to convert the reference image into a gray image, perform reconstruction operation based on opening and closing to obtain a local maximum image, perform erosion operation processing on the local maximum image, superimpose the local maximum image after the erosion operation processing on the reference image, and recognize and position the single tree in the reference image;

and converting the image to be registered into a gray image, performing reconstruction operation based on opening and closing to obtain a local maximum image, performing corrosion operation processing on the local maximum image, superposing the local maximum image subjected to the corrosion operation processing to the image to be registered, and identifying and positioning the single wood in the image to be registered.

9. The forest remote sensing image registration system based on single tree recognition according to any one of claims 6 to 8, wherein the descriptor calculation unit is specifically configured to determine a position point of an ith single tree recognized in the reference image, use the position point as a key point, block regions of a preset size around the key point, respectively calculate a gradient histogram of each region, generate a feature vector of the key point according to the gradient histogram, and generate a local descriptor of the key point according to the feature vector of the key point, which is used as a local descriptor of the ith single tree.

10. The forest remote sensing image registration system based on single tree recognition according to any one of claims 6 to 8, wherein the descriptor calculation unit is specifically configured to determine a position point of a jth single tree recognized in the image to be registered, use the position point as a key point, block regions of a preset size around the key point, respectively calculate a gradient histogram of each region, generate a feature vector of the key point according to the gradient histogram, and generate a local descriptor of the key point according to the feature vector of the key point, as the local descriptor of the jth single tree.

Images