CN106875407B - Unmanned aerial vehicle image canopy segmentation method combining morphology and mark control - Google Patents

Abstract

The invention relates to an unmanned aerial vehicle image forest canopy segmentation method combined with morphology and marker control: using an unmanned aerial vehicle to obtain local remote sensing images of several forest areas, and obtaining a complete remote sensing image through mosaicking and orthorectification; The light band is smoothed and filtered; the adaptive local maximum search method is used to detect the position of the canopy apex from the green light band; the obtained canopy apex position information is imposed on the image through a forced minimum transformation by using morphological operations; Ortho-corrected true-color remote sensing images, using ISODATA clustering algorithm to obtain binary images that only contain forest canopy areas and non-forest canopy areas, and use the extracted non-forest canopy areas as external markers for segmentation; Perform watershed transformation segmentation on the image converted by the minimum value to obtain accurate information on the single tree canopy boundary of the stand. The invention effectively solves the problem of inaccurate segmentation of forest canopy boundaries caused by conventional methods.

Description

A UAV imagery canopy segmentation method combining morphology and marker control

technical field

The invention relates to an unmanned aerial vehicle image forest canopy segmentation method combined with morphology and marker control.

Background technique

As a place for trees to obtain light energy and perform energy conversion, tree canopy is of great significance for the study of forest growth and dynamic changes. However, due to the complexity and randomness of the forest structure, it is extremely difficult to obtain information on the shape and boundary of the tree canopy. In recent years, with the development of satellite remote sensing technology, especially the gradual improvement of the spatial resolution of satellite images, the estimation accuracy of forest canopy has been improved. However, affected by factors such as climate, period, resolution and cost, the obtained remote sensing The data are far from meeting the needs of forestry surveys. As an emerging remote sensing technology, UAV remote sensing has gradually become an effective supplement to satellite remote sensing technology due to its unique advantages of flexibility, timeliness, low cost, and easy access to data, and has been widely used in many fields. Although the research on UAV technology is increasing, the research on extracting forest canopy structure information from visible light UAV images is still in the experimental stage. For example, Díazvarela et al. An olive breeding plot in the Cordoba region of Spain was tested with an estimated crown size RMSE of 0.28. Chianucci et al. used the true color images obtained by the eBee unmanned aerial system, combined with the LAB2 image classification method to estimate the forest canopy coverage of the beech forest, and the coefficient of determination R2 reached about 0.7; in addition, Morgenroth, Mathews, etc. used UAVs. The point cloud data generated by the image is used to analyze the forest canopy structure, and certain results have been achieved. However, the conventional forest canopy segmentation method will cause the problem of inaccurate segmentation of the forest canopy boundary, which brings uncertainty to the accuracy of the forest parameters obtained by UAV remote sensing.

SUMMARY OF THE INVENTION

In view of this, the purpose of the present invention is to provide a UAV image canopy segmentation method combining morphology and marker control, which effectively solves the problem of inaccurate canopy boundary segmentation caused by conventional methods.

In order to achieve the above object, the present invention adopts the following technical scheme: a method for segmenting the canopy of an unmanned aerial vehicle image combined with morphology and marker control, which is characterized in that, comprising the following steps:

Step S1: using the UAV to obtain local remote sensing images of several forest areas, and performing mosaic and orthorectification on the several remote sensing images of forest areas to obtain complete remote sensing images of forest areas;

Step S2: using the Gaussian filtering method to perform smooth filtering processing on the green light band of the complete remote sensing image;

Step S3: using an adaptive local maximum search method to detect the position of the canopy vertex from the green light band of the complete remote sensing image;

Step S4: using morphological operations to impose the obtained canopy vertex position information on the green light band image after smooth filtering through a forced minimum conversion;

Step S5: For the complete remote sensing image obtained in Step S1, the ISODATA clustering algorithm is used to obtain a binary image that only includes two types of forest canopy areas and non-forest canopy areas, and the extracted non-forest canopy area is used as an external marker for segmentation;

Step S6: Based on the results obtained in Step S4 and Step S5, an external marker is imposed on the image transformed by the forced minimum value to perform watershed transformation segmentation, so as to obtain accurate stand single tree canopy boundary information.

Further, the local remote sensing image is a true color image, and the resolution is between 0.05-0.20m.

Further, the specific method of the step S2 is as follows: using a Gaussian distribution curve to process the green light band of the complete remote sensing image, reducing the noise level of the image and strengthening the radiation intensity value of the canopy vertex, the filtering formula is as follows:

In the formula, G(i,j) is the result of Gaussian filtering of image pixels at row i and column j, i and j are natural numbers, σ is the mean square error of Gaussian filtering, and the value of σ is selected as the minimum canopy size in the stand as window for image filtering.

Further, the specific method of the step S3 is as follows:

Step S31: Detect the position of the potential forest canopy vertex in the sample plot through a fixed window, and obtain the potential forest canopy vertex;

Step S32: Use an adaptive dynamic window to judge the acquired potential forest canopy vertices. If the current vertex is the maximum value of the corresponding window area, save it, otherwise delete it; The semi-variance calculation formula of the image pixel is:

In the formula, γ(h) is the empirical semivariance value, _xi is the pixel position of the image, h is the spatial separation distance between two pixels, Z(x) is the pixel value at the corresponding image _xi , N is the logarithm of cell pairs at a certain separation distance.

Further, the specific method of the step S4 is as follows:

Step S41: Perform an inversion operation on the filtered image f, and then mark the obtained canopy apex with a minimum value to obtain a marked image, as follows:

In the formula, f _m is the acquired marked image, p is each pixel of the image, and t _max is the maximum value of the image;

Step S42: Calculate the minimum value between the image f+1 and the marked image f _m pixel by pixel, and perform forced minimum conversion on the image;

In this step, the morphological calculation is expressed as: (f+1)∧f _m , and then the marked image f _m is used to reconstruct (f+1)∧f _m by morphological erosion, as follows:

描述(f+1)∧f_m在标记影像f_m掩膜下的形态学腐蚀重建过程。In the formula, f _mp is the image after forced minimum conversion,

Describe the morphological corrosion reconstruction process of (f+1)∧f _m under the mask of marked image f _m .

Further, the specific method of the step S5 is as follows: based on the obtained complete remote sensing image, the ISODATA unsupervised classification method is used for classification, the set number of classification categories is ≥ 10, and the maximum number of iterations is ≥ 5; the obtained classification results are visually inspected. The interpretations were merged to obtain a binary image that only contained the forest canopy area and the non-forest canopy area, and the extracted non-forest canopy area was used as an external marker for segmentation.

Further, the watershed transformation and segmentation in the step S6 adopts the following formula:

In the formula, WaterShed() is the watershed function; Mask is the mask function; B _OutMask is the external mark, that is, the non-forest canopy area, and W _canopy is the stand single tree canopy boundary.

Compared with the prior art, the present invention has the following beneficial effects: the present invention effectively solves the problem of inaccurate segmentation of forest canopy boundaries caused by conventional methods; it is conducive to the rapid and effective extraction of forest canopy information, which is the number of trees and canopy closures in forest resources surveys. Provides strong support for accurate and efficient estimation of degree.

Description of drawings

FIG. 1 is a flow chart of the method of the present invention.

FIG. 2A is a drone image according to the first embodiment of the present invention.

FIG. 2B is a result of filtering the green light band according to the first embodiment of the present invention.

FIG. 2C is a direct watershed segmentation result according to Embodiment 1 of the present invention.

FIG. 2D is a canopy vertex extracted by using a fixed window according to Embodiment 1 of the present invention.

FIG. 2E is a result of removing outliers by adopting a variable window according to Embodiment 1 of the present invention.

FIG. 2F is a binary map of forest canopy and non-forest canopy according to Embodiment 1 of the present invention.

FIG. 2G is the result of morphological reconstruction marking according to the first embodiment of the present invention.

FIG. 2H is the result of adding internal and external markers according to the first embodiment of the present invention.

FIG. 2I is a result of image segmentation combined with internal and external markers according to Embodiment 1 of the present invention.

FIG. 3A is a drone image of the second embodiment of the present invention.

FIG. 3B is the direct watershed segmentation result of the second embodiment of the present invention.

Fig. 3C The fixed point of the canopy extracted by the fixed window in the second embodiment of the present invention.

FIG. 3D shows the result of removing abnormal vertices from an adaptive window according to Embodiment 2 of the present invention.

FIG. 3E is an image of the top of the imposed forest canopy according to the second embodiment of the present invention.

FIG. 3F shows the morphological reconstruction results of the second embodiment of the present invention.

FIG. 3G is a binary map of forest canopy and non-forest canopy according to the second embodiment of the present invention.

FIG. 3H is the direct watershed segmentation result of the inner-marked image according to the second embodiment of the present invention.

FIG. 3I shows the result of image segmentation combined with internal and external markers according to the second embodiment of the present invention.

Detailed ways

The present invention will be further described below with reference to the accompanying drawings and embodiments.

Please refer to FIG. 1, the present invention provides a UAV image canopy segmentation method combining morphology and marker control, which is characterized in that it includes the following steps:

Step S1: using the unmanned aerial vehicle to obtain several local remote sensing images of the forest area with a resolution between 0.05-0.20m, and performing mosaic and orthorectification on the several remote sensing images of the forest area to obtain a complete remote sensing image of the forest area; The local remote sensing images should be at least true-color images containing red, green, and blue bands, and the heading and lateral overlap rates of the images should be ≥80%.

Step S2: use a Gaussian filtering method to smooth the green light band of the complete remote sensing image; the specific method is as follows: use a Gaussian distribution curve (bell curve) to process the green light band of the complete remote sensing image to reduce the noise of the image Radiation intensity values of horizontal and enhanced canopy vertices, the filtering formula is as follows:

In the formula, G(i,j) is the result of Gaussian filtering of image pixels at row i and column j, i and j are natural numbers, σ is the mean square error of Gaussian filtering, and the value of σ selects the minimum canopy size in the stand Image filtering is performed as a window.

Step S3: Use the adaptive local maximum search method to detect the position of the canopy vertex from the green light band of the complete remote sensing image; the specific method is as follows:

Step S31: First, detect the position of the potential forest canopy vertex in the sample plot through a small fixed window to obtain the potential forest canopy vertex;

Step S32: Use an adaptive dynamic window to judge the acquired potential forest canopy vertices. If the current vertex is the maximum value of the corresponding window area, save it, otherwise delete it; The semi-variance calculation formula of the image pixel is:

In the formula, γ(h) is the empirical semivariance value, _xi is the pixel position of the image, h is the spatial separation distance between two pixels, Z(x) is the pixel value at the corresponding image _xi , N is the logarithm of cell pairs at a certain separation distance.

Step S4: Using morphological operations, the obtained forest canopy vertex position information is imposed on the green light band image after smooth filtering through a forced minimum conversion; the specific method is as follows:

Step S41: First, perform an inversion operation on the image f after smoothing filtering, and then mark the obtained canopy vertex with a minimum value to obtain a marked image, as shown in the following formula:

In the formula, f _m is the acquired marked image, p is each pixel of the image, and t _max is the maximum value of the image;

Step S42: Then, calculate the minimum value between the image f+1 and the marked image f _m pixel by pixel, and perform forced minimum conversion on the image;

In this step, the morphological calculation is expressed as: (f+1)∧f _m , and then the marked image f _m is used to reconstruct (f+1)∧f _m by morphological erosion, as follows:

描述(f+1)∧f_m在标记影像f_m掩膜下的形态学腐蚀重建过程。In the formula, f _mp is the image after forced minimum conversion,

Describe the morphological corrosion reconstruction process of (f+1)∧f _m under the mask of marked image f _m .

Step S5: For the complete remote sensing image obtained in Step S1, the ISODATA clustering algorithm is used to obtain a binary image that only includes the forest canopy area and the non-forest canopy area, and the extracted non-forest canopy area is used as an external marker for segmentation; the specific method is as follows: Based on the obtained complete remote sensing images, the ISODATA unsupervised classification method is used for classification. The set number of classification categories is greater than or equal to 10, and the maximum number of iterations is greater than or equal to 5. The obtained classification results are merged through visual interpretation, and only the canopy area and non-forest canopy are obtained. Binary images of these two types of regions are used, and the extracted non-forest canopy regions are used as external markers for segmentation.

Step S6: Based on the results obtained in Step S4 and Step S5, an external marker is imposed on the image transformed by the forced minimum value to perform watershed transformation segmentation, so as to obtain accurate stand single tree canopy boundary information. The watershed transformation segmentation adopts the following formula:

In the formula, WaterShed() is the watershed function; Mask is the mask function; B _OutMask is the external mark, that is, the non-forest canopy area, and W _canopy is the stand single tree canopy boundary.

In order to make those skilled in the art better understand the technical solutions of the present invention, the present invention will be described in detail below with reference to two embodiments. Among them, the local remote sensing images obtained by the UAV are RGB true color images, which are preprocessed by PIX4D software. After mosaicking and orthorectification, the image resolution is 7cm.

Example 1:

Figure 2A is the original visible light image of Plot 1. Plot 1 is a coniferous forest plot with isolated tree crowns and overlapping tree crowns. Figure 2B shows the result of maximum filtering and Gaussian smoothing filtering in the green light band, which enhances the spectral difference between canopy and non-forest canopy and reduces the spectral heterogeneity within the canopy. Figure 2C directly performs watershed segmentation on the filtered green light band, and the phenomenon of over-segmentation occurs. This is because there are some noise values in the image in addition to the canopy vertices, and there are roads and open spaces in the image;

Figure 2D is the result of the canopy vertex detection using the fixed window local maximum method, and there is a problem of detecting multiple vertices in some canopies;

Figure 2E is the result of applying the variable window (adaptive window) maximum value method to detect the canopy vertices on the basis of the fixed window detection results. It can be found that the phenomenon of multiple vertices appearing in some canopies is eliminated;

Figure 2F is a binary map of forest canopy and non-forest canopy obtained by unsupervised classification;

Figure 2G: It is the green light band after morphological reconstruction and forced minimum conversion processing. At this time, the tree crown vertex marks obtained by searching are imposed on the image, that is, to ensure that the watershed segmentation will only be divided according to these tree top marks;

Figure 2H is the result of adding non-forest canopy outer markers to the results of Figure 2G;

Figure 2I shows the result of watershed segmentation combined with inner and outer canopy markers. Each closed polygon represents a canopy. By superimposing the canopy delineation results with the original image, it can be seen that the results obtained by this algorithm are relatively good.

Embodiment 2:

Figure 3A is the original visible light image of Plot 2, which is a broad-leaved forest plot. Figure 3B shows the results of direct watershed segmentation in the green light band. Over-splitting occurs. This is because there are some noise values in the image in addition to the canopy vertices, and the reason for the existence of shrubs and grasslands in the image;

Figure 3C is the result of using the fixed window local maximum method to detect canopy vertices. There is a problem of detecting multiple vertices in some canopies, as well as detecting vertices on shrubs and grasslands;

Figure 3D is the result of the canopy vertex detection using the variable window (adaptive window) maximum method on the basis of the fixed window detection results. It can be found that the phenomenon of multiple vertices in some canopies has been eliminated, but there are still shrub and grassland vertices. question;

Figure 3E is the result of the obtained tree crown vertex markers being imposed on the green light band image; Figure 3F is the result of morphological reconstruction. As a result, the result ensures that the watershed segmentation will only be segmented according to the tree top markers. In addition, in order to eliminate the influence of the non-forest canopy area on the canopy segmentation boundary, the segmentation result also needs to be masked. Figure 3G is a binary map of canopy and non-canopy obtained by unsupervised classification. Figure 3H is the result of watershed segmentation directly on the canopy marked image, which fails to eliminate the influence of shrubs and grasslands; Figure 3I is the result of watershed segmentation based on treetop marking and non-canopy mask marking, and the obtained results are relatively good.

According to the above experimental segmentation results, the results of single-tree canopy segmentation and visual interpretation of the two plots were compared and verified, and the results are shown in Table 1.

Table 1 Analysis of the extraction accuracy of single tree canopy in the sample plot

From Table 1, we can conclude that the canopy segmentation accuracy of the two sample plots is high, the plot-01 dominated by coniferous forest reaches 94.54%, while the sample plot dominated by broad-leaved forest plot-02 Then it reaches 95.56%.

The above descriptions are only preferred embodiments of the present invention, and all equivalent changes and modifications made according to the scope of the patent application of the present invention shall fall within the scope of the present invention.

Claims (4)

1. a UAV image canopy segmentation method combining morphology and marker control, is characterized in that, comprises the following steps: Step S1: using unmanned aerial vehicle to obtain several local remote sensing images of forest areas with a resolution between 0.05-0.20m, and performing mosaic and orthorectification on the several remote sensing images of forest areas to obtain complete remote sensing images of forest areas; The above-mentioned local remote sensing images shall at least be true color images containing red, green and blue bands, and the heading and side overlap ratio of the images shall be ≥80%; Step S2: using the Gaussian filtering method to perform smooth filtering processing on the green light band of the complete remote sensing image; Step S3: using an adaptive local maximum search method to detect the position of the canopy vertex from the green light band of the complete remote sensing image; Step S4: using morphological operations to impose the obtained canopy vertex position information on the green light band image after smooth filtering through a forced minimum conversion; Step S5: For the complete remote sensing image obtained in Step S1, the ISODATA clustering algorithm is used to obtain a binary image that only includes two types of forest canopy areas and non-forest canopy areas, and the extracted non-forest canopy area is used as an external marker for segmentation; Step S6: Based on the results obtained in steps S4 and S5, an external marker is imposed on the image after forced minimum conversion to perform watershed transformation segmentation to obtain accurate stand single tree canopy boundary information; The specific method of the step S3 is as follows: Step S31: Detect the position of the potential forest canopy vertex in the sample plot through a fixed window, and obtain the potential forest canopy vertex; Step S32: Use an adaptive dynamic window to judge the acquired potential forest canopy vertices. If the current vertex is the maximum value of the corresponding window area, save it, otherwise delete it; The range value is determined, and the semi-variance calculation formula of the image pixel is:

In the formula, γ(h) is the empirical semivariance value, x _i is the pixel position of the image, h is the spatial separation distance between two pixels, Z( _xi ) is the pixel value at the corresponding image x _i , N is the logarithm of pixel pairs at a certain separation distance; The specific method of the step S4 is as follows: Step S41: Perform an inversion operation on the filtered image f, and then mark the obtained canopy apex with a minimum value to obtain a marked image, as follows:

In the formula, f _m is the acquired marked image, p is each pixel of the image, and t _max is the maximum value of the image; Step S42: Calculate the minimum value between the image f+1 and the marked image f _m pixel by pixel, and perform forced minimum conversion on the image; In this step, the morphological calculation is expressed as: (f+1)∧f _m , and then the morphological erosion reconstruction of (f+1)∧f _m is performed by using the marked image f _m , as follows:

In the formula, f _mp is the image after forced minimum conversion,

Describe the morphological corrosion reconstruction process of (f+1)∧f _m under the mask of marked image f _m . 2. the unmanned aerial vehicle image forest canopy segmentation method combining morphology and mark control according to claim 1, is characterized in that: the concrete method of described step S2 is as follows: adopt a Gaussian distribution curve to the green light of complete remote sensing image The waveband is processed to reduce the noise level of the image and strengthen the radiation intensity value of the canopy vertex. The filtering formula is as follows:

In the formula, G(i,j) is the result of Gaussian filtering of the image pixel at row i and column j, i and j are natural numbers, σ is the mean square error of Gaussian filtering, and the value of σ selects the minimum canopy size in the stand as window for image filtering. 3. the unmanned aerial vehicle image forest canopy segmentation method combining morphology and marking control according to claim 1, is characterized in that: the concrete method of described step S5 is as follows: based on the complete remote sensing image obtained, adopt ISODATA unsupervised classification method Perform classification, set the number of classification categories ≥ 10, and the maximum number of iterations ≥ 5; the obtained classification results are merged through visual interpretation to obtain binary images that only include canopy areas and non-forest canopy areas, and will be extracted. The non-forest canopy area is used as an external marker for segmentation. 4. the unmanned aerial vehicle image forest canopy segmentation method combining morphology and mark control according to claim 1, is characterized in that: in described step S6, watershed transformation segmentation adopts formula as follows:

In the formula, WaterShed() is the watershed function; Mask is the mask function; B _OutMask is the external mark, that is, the non-forest canopy area, and W _canopy is the stand single tree canopy boundary.

Images