CN110717382B - Adhesion tree detection method based on inhibition optimization mechanism - Google Patents

Abstract

The method for detecting the adhered trees based on the inhibition optimization mechanism comprises the following steps: step 1, predicting bounding box suppression avoidance; step 2, detecting trees; and 3, elastic inhibition post-treatment. Compared with the existing method, the tree detection method based on the inhibition optimization mechanism has more excellent detection performance in the scene of tree adhesion, the scene with more serious adhesion is improved more obviously, and compared with the baseline method, the tree detection method based on the inhibition optimization mechanism has less false recognition and less missing recognition.

Description

Adhesion tree detection method based on inhibition optimization mechanism

Technical Field

The invention relates to a method for detecting trees adhered to a high-resolution remote sensing image.

Background

The high-resolution remote sensing image can help forestry personnel to realize large-range high-precision forest resource investigation, wood yield estimation and forest drawing, and can also provide decision support for urban greening planning. With the gradual improvement of the resolution of remote sensing images, the acquisition becomes easy, and many researchers are dedicated to the work of extracting information of a single tree. However, these tree detection methods generally rely on the prior knowledge of experts, and have low robustness to different scenes.

Although the current general target detection methods are more and more, the research on the method for detecting the ground objects in the remote sensing images, particularly detecting the trees, is relatively less. The reason is that the resolution of the remote sensing image is not high enough, the growth situation of trees on the ground is relatively complex, and particularly, in a complex scene with high canopy density, the trees are relatively crowded, and the crowns are staggered. In the remote sensing image, the situations are represented as unclear crown boundaries and serious mutual adhesion among trees, and no better solution exists at present for the problem in tree detection.

Disclosure of Invention

The invention aims to overcome the defects in the prior art and provides an adhesion tree detection method based on an inhibition optimization mechanism in a tree adhesion scene.

The invention designs an adhesion tree detection method based on an inhibition optimization mechanism. Firstly, a model is optimized by inhibiting and avoiding a prediction boundary frame, the distance loss of the prediction boundary frame outside an example and the distance loss of the reference boundary frame outside the example are respectively added to inhibit and avoid the model so as to reduce the problems of missing identification and positioning drift, then the optimized model tree is used for detection, and finally the detection precision of the adhered tree is further improved by carrying out elastic inhibition post-treatment.

The method for detecting the adhered trees based on the inhibition optimization mechanism comprises the following steps:

1. prediction bounding box suppression avoidance;

the model based on inhibition avoidance is based on Mask R-CNN, and the overall loss function is as follows:

L＝L_cls+L_box+L_mask (1)

wherein L is_clsIs the loss of classification, L_boxIs the regression loss, L_maskIs the mask loss; the three losses do not consider the influence of tree adhesion, wherein the regression loss only enables the prediction boundary box to approach the corresponding example reference tree infinitely, but the phenomenon that the prediction boxes are mutually inhibited cannot be avoided; the traditional non-maximum method only keeps a boundary box with extremely high confidence in the tree overlapping area, which can cause the problems of missing identification and the like; these problems can be solved by adding the following two losses, which can be used for tree detection in real images;

11) out-of-instance prediction bounding box distance loss avoidance suppression; given Box ∈ Box (Box is a bounding Box, Box is a set of bounding Boxes for all predictions are trees), Box is Box within an instance_innerIn the example, Box is Box_outerExample outer prediction bounding box distance penalty L_{outer_box}Can be expressed as the following equation:

where the condition for the calculation is that the intra-instance prediction bounding Box and the out-instance prediction bounding Box are stuck, i.e. IOU (Box)_inner，Box_outer) > 0, IOU is the degree of overlap, Smooth_lnIs a smoothed ln function;

the step avoids the phenomenon that prediction boundary frames corresponding to different reference boundary frames mutually inhibit in the tree adhesion problem; the optimized loss function is shown in equation (3):

L＝L_cls+L_box+L_mask+L_{outer_box} (3)

12) example outer reference bounding box distance loss avoidance suppression; given a prediction bounding Box Box ∈ Box, the reference bounding Box corresponding to Box is GT _ Box_innerDenoted GT _ Box_innerEither ae-B-C, or E-C, either ae-C, or E, or E_outerFor GT _ Box (GT _ Box is a set of reference bounding Boxes), the GT _ Box_outerIs to divide GT _ Box_innerEx-reference trees for the most severe cases of Ex-Box adhesion, i.e. except GT _ Box_innerExternal GT _ Box_outerThe maximum degree of overlap with Box is expressed as formula (4):

where GT _ Box' is the reference bounding Box;

example outer reference bounding box penalty L_{outer_gt}Can be expressed as formula (5):

where the condition for the computation is that the instance intra-prediction bounding Box is stuck with the instance outer reference bounding Box, i.e. IOU (Box)_inner，GT_Box_outer)＞0；

The aim of the step is to separate the prediction boundary frames corresponding to different reference boundary frames, and if each prediction boundary frame is far away from a reference example tree (especially a relatively close reference tree) different from the reference example tree corresponding to the prediction boundary frame, the mutual inhibition phenomenon of the prediction boundary frames of different examples can be avoided; the final optimized loss function is shown in equation (6):

L＝L_cls+L_box+L_mask+L_{outer_box}+L_{outer_gt} (6)

2. detecting trees;

using a model based on prediction frame suppression avoidance optimization, and collecting samples in a detection image by combining a sliding window technology, wherein the step length is set to be 2 pixels, and each sliding window detects whether the samples exist in the sliding window through a pre-trained model; in addition, the scale of the sliding window is changed along with the size of the tree when the sample is collected, so that one reference tree corresponds to the prediction labels of a plurality of sliding windows; sorting all the predicted labels in a descending order according to the classification probability, calculating the areas of the overlapping areas of the rest sliding windows from the sliding window of the first predicted label, removing the redundant windows when the area of the overlapping area exceeds a certain threshold value, removing the redundant windows of the sliding windows of the second predicted label, and repeating the steps till the last window;

3. elastic inhibition post-treatment;

the prediction boundary frames obtained by the model detection have the condition of mutual overlapping, some boundary frames are restrained according to the condition whether the prediction frames accord with the evolution growth of the trees or not and the overlapping degree elasticity, and some boundary frames are reserved to obtain the final tree detection result; according to the specific condition of tree adhesion, the confidence score of the bounding box is attenuated, the bounding box with the overlapping degree smaller than the threshold value is kept with the original confidence while the bounding box with the higher confidence score is kept based on the elastic inhibition post-treatment of tree evolution, the bounding box with the overlapping degree smaller than the threshold value is not directly inhibited, and the treatment is carried out by analyzing the two steps of the tree adhesion condition and the tree growth evolution condition;

31) analyzing the adhesion condition of the trees; consider bounding box b_iAdhesion to bounding boxes with higher confidence scores, heavyThe higher the degree of overlap, the higher the probability of detecting the same tree, and therefore the higher the probability of rejecting it, whereas the lower the degree of overlap, the more likely the bounding box is to be retained; that is, the more serious the bounding box adhesion is, the more the confidence coefficient is reduced; the smaller the adhesion, the smaller the confidence decrease;

32) analyzing the growth evolution situation of the trees; considering that the growth environment of the adhered trees is similar, the evolution of the crown of the tree accords with the Gaussian distribution rule; when the bounding box is very large or very small, it is considered that abnormal growth is rejected.

The invention has the advantages that: compared with the existing method, the provided detection algorithm has better detection performance; the detection model based on inhibition avoidance can effectively solve the problem of missed identification and improve the tree detection performance under the tree adhesion scene.

Drawings

FIG. 1 is a flow chart of the method of the present invention.

Detailed Description

The technical scheme of the invention is further explained by combining the attached drawings.

The high-resolution remote sensing image tree detection method based on deep learning comprises the following steps: 1. prediction bounding box suppression avoidance;

the model based on inhibition avoidance is based on Mask R-CNN, and the overall loss function is as follows:

L＝L_cls+L_box+L_mask (1)

wherein L is_clsIs the loss of classification, L_boxIs the regression loss, L_maskIs the mask loss; the three losses do not consider the influence of tree adhesion, wherein the regression loss only enables the prediction boundary box to approach the corresponding example reference tree infinitely, but the phenomenon that the prediction boxes are mutually inhibited cannot be avoided; the traditional non-maximum method only keeps a boundary box with extremely high confidence in the tree overlapping area, which can cause the problems of missing identification and the like; these problems can be solved by adding the following two losses, which can be used for tree detection in real images;

11) out-of-instance prediction bounding box distance loss avoidance suppression; given Box ∈ Box (Box is a bounding Box, BoxIs the set of bounding boxes for which all predictions are trees), Box is Box within an instance_innerIn the example, Box is Box_outerExample outer prediction bounding box distance penalty L_{outer_box}Can be expressed as the following equation:

where the condition for the calculation is that the intra-instance prediction bounding Box and the out-instance prediction bounding Box are stuck, i.e. IOU (Box)_inner，Box_outer) > 0, IOU is the degree of overlap, Smooth_lnIs a smoothed ln function;

the step avoids the phenomenon that prediction boundary frames corresponding to different reference boundary frames mutually inhibit in the tree adhesion problem; the optimized loss function is shown in equation (3):

L＝L_cls+L_box+L_mask+L_{outer_box} (3)

12) example outer reference bounding box distance loss avoidance suppression; given a prediction bounding Box Box ∈ Box, the reference bounding Box corresponding to Box is GT _ Box_innerDenoted GT _ Box_innerEither ae-B-C, or E-C, either ae-C, or E, or E_outerFor GT _ Box (GT _ Box is a set of reference bounding Boxes), the GT _ Box_outerIs to divide GT _ Box_innerEx-reference trees for the most severe cases of Ex-Box adhesion, i.e. except GT _ Box_innerExternal GT _ Box_outerThe maximum degree of overlap with Box is expressed as formula (4):

where GT _ Box' is the reference bounding Box;

example outer reference bounding box penalty L_{outer_gt}Can be expressed as formula (5):

where the condition for the computation is that the instance intra-prediction bounding Box is stuck with the instance outer reference bounding Box, i.e. IOU (Box)_inner，GT_Box_outer)＞0；

The aim of the step is to separate the prediction boundary frames corresponding to different reference boundary frames, and if each prediction boundary frame is far away from a reference example tree (especially a relatively close reference tree) different from the reference example tree corresponding to the prediction boundary frame, the mutual inhibition phenomenon of the prediction boundary frames of different examples can be avoided; the final optimized loss function is shown in equation (6):

L＝L_cls+L_box+L_mask+L_{outer_box}+L_{outer_gt} (6)

2. detecting trees;

using a model based on prediction frame suppression avoidance optimization, and collecting samples in a detection image by combining a sliding window technology, wherein the step length is set to be 2 pixels, and each sliding window detects whether the samples exist in the sliding window through a pre-trained model; in addition, the scale of the sliding window is changed along with the size of the tree when the sample is collected, so that one reference tree corresponds to the prediction labels of a plurality of sliding windows; sorting all the predicted labels in a descending order according to the classification probability, calculating the areas of the overlapping areas of the rest sliding windows from the sliding window of the first predicted label, removing the redundant windows when the area of the overlapping area exceeds a certain threshold value, removing the redundant windows of the sliding windows of the second predicted label, and repeating the steps till the last window;

3. elastic inhibition post-treatment;

the prediction boundary frames obtained by the model detection have the condition of mutual overlapping, some boundary frames are restrained according to the condition whether the prediction frames accord with the evolution growth of the trees or not and the overlapping degree elasticity, and some boundary frames are reserved to obtain the final tree detection result; according to the specific condition of tree adhesion, the confidence score of the bounding box is attenuated, the bounding box with the overlapping degree smaller than the threshold value is kept with the original confidence while the bounding box with the higher confidence score is kept based on the elastic inhibition post-treatment of tree evolution, the bounding box with the overlapping degree smaller than the threshold value is not directly inhibited, and the treatment is carried out by analyzing the two steps of the tree adhesion condition and the tree growth evolution condition;

31) analyzing the adhesion condition of the trees; consider bounding box b_iThe higher the degree of overlap with the blocking of the bounding box with the higher confidence score, the higher the probability of detecting the same tree, and thus the higher the probability of being rejected, on the contrary, the lower the degree of overlap, the more likely the bounding box is to be retained; that is, the more serious the bounding box adhesion is, the more the confidence coefficient is reduced; the smaller the adhesion, the smaller the confidence decrease;

32) analyzing the growth evolution situation of the trees; considering that the growth environment of the adhered trees is similar, the evolution of the crown of the tree accords with the Gaussian distribution rule; when the bounding box is very large or very small, it is considered that abnormal growth is rejected.

The embodiments described in this specification are merely illustrative of implementations of the inventive concept and the scope of the present invention should not be considered limited to the specific forms set forth in the embodiments but rather by the equivalents thereof as may occur to those skilled in the art upon consideration of the present inventive concept.

Claims (1)

1. The method for detecting the adhered trees based on the inhibition optimization mechanism comprises the following steps:

step 1, predicting bounding box suppression avoidance;

the model based on inhibition avoidance is based on Mask R-CNN, and the overall loss function is as follows:

L＝L_cls+L_box+L_mask (1)

wherein L is_clsIs the loss of classification, L_boxIs the regression loss, L_maskIs the mask loss; the three losses do not consider the influence of tree adhesion, wherein the regression loss only enables the prediction boundary box to approach the corresponding example reference tree infinitely, but the phenomenon that the prediction boxes are mutually inhibited cannot be avoided; the traditional non-maximum method only keeps a boundary box with extremely high confidence in the tree overlapping area, which can cause the problem of missed identification; these problems can be solved by adding two losses, which can be used in real imagesCarrying out tree detection;

11) out-of-instance prediction bounding box distance loss avoidance suppression; given Box ∈ Box, where Box is the bounding Box, Box is the set of bounding Boxes for all predictions are trees, and Box is Box within an instance_innerIn the example, Box is Box_outerExample outer prediction bounding box distance penalty L_{outer_box}Can be expressed as the following equation:

where the condition for the calculation is that the intra-instance prediction bounding Box and the out-instance prediction bounding Box are stuck, i.e. IOU (Box)_inner,Box_outer)>0, IOU is the degree of overlap, Smooth_lnIs a smoothed ln function;

the step avoids the phenomenon that prediction boundary frames corresponding to different reference boundary frames mutually inhibit in the tree adhesion problem; the optimized loss function is shown in equation (3):

L＝L_cls+L_box+L_mask+L_{outer_box} (3)

12) example outer reference bounding box distance loss avoidance suppression; given a prediction bounding Box Box ∈ Box, the reference bounding Box corresponding to Box is GT _ Box_innerDenoted GT _ Box_innerEither ae in step of ae, B

Where GT _ Box is the set of reference bounding Boxes, the GT _ Box_outerIs to divide GT _ Box_innerReference is made to bounding boxes outside the most severe cases of Box adhesion, i.e. except for GT _ Box_innerExternal GT _ Box_outerThe maximum degree of overlap with Box is expressed as formula (4):

where GT _ Box' is the reference bounding Box;

example outer reference bounding box penalty L_{outer_gt}Can be expressed as formula (5):

where the condition for the computation is that the instance intra-prediction bounding Box is stuck with the instance outer reference bounding Box, i.e. IOU (Box)_inner,GT_Box_outer)>0；

The aim of the step is to separate the prediction boundary frames corresponding to different reference boundary frames, if each prediction boundary frame is far away from a reference example tree different from the reference example tree corresponding to the prediction boundary frame, the mutual inhibition phenomenon of the prediction frames of different examples can be avoided; the final optimized loss function is shown in equation (6):

L＝L_cls+L_box+L_mask+L_{outer_box}+L_{outer_gt} (6)

step 2, detecting trees;

using a model based on prediction frame suppression avoidance optimization, and collecting samples in a detection image by combining a sliding window technology, wherein the step length is set to be 2 pixels, and each sliding window detects whether the samples exist in the sliding window through a pre-trained model; in addition, the scale of the sliding window is changed along with the size of the tree when the sample is collected, so that one reference tree corresponds to the prediction labels of a plurality of sliding windows; sorting all the predicted labels in a descending order according to the classification probability, calculating the areas of the overlapping areas of the rest sliding windows from the sliding window of the first predicted label, removing the redundant windows when the area of the overlapping area exceeds a certain threshold value, removing the redundant windows of the sliding windows of the second predicted label, and repeating the steps till the last window;

step 3, elastic inhibition post-treatment;

the prediction boundary frames obtained by the model detection have the condition of mutual overlapping, some boundary frames are restrained according to the condition whether the prediction frames accord with the evolution growth of the trees or not and the overlapping degree elasticity, and some boundary frames are reserved to obtain the final tree detection result; according to the specific condition of tree adhesion, the confidence score of the bounding box is attenuated, the bounding box with the overlapping degree smaller than the threshold value is kept with the original confidence while the bounding box with the higher confidence score is kept based on the elastic inhibition post-treatment of tree evolution, the bounding box with the overlapping degree smaller than the threshold value is not directly inhibited, and the treatment is carried out by analyzing the two steps of the tree adhesion condition and the tree growth evolution condition;

31) analyzing the adhesion condition of the trees; consider bounding box b_iThe higher the degree of overlap with the blocking of the bounding box with the higher confidence score, the higher the probability of detecting the same tree, and thus the higher the probability of being rejected, on the contrary, the lower the degree of overlap, the more likely the bounding box is to be retained; that is, the more serious the bounding box adhesion is, the more the confidence coefficient is reduced; the smaller the adhesion, the smaller the confidence decrease;

32) analyzing the growth evolution situation of the trees; considering that the growth environment of the adhered trees is similar, the evolution of the crown of the tree accords with the Gaussian distribution rule; when the bounding box is very large or very small, it is considered that abnormal growth is rejected.

Images