CN115063700A - Detection method based on small-scale pine wood nematode disease tree - Google Patents

Abstract

A detection method based on small-scale pine wood nematode disease trees comprises the following steps: step 1: acquiring a remote sensing image by using an unmanned aerial vehicle for aerial photography, marking the pine wood nematode disease tree, and making into a data set; step 2: inputting the data set into a bottom layer feature batch processing fusion and multiple feature multiplexing network model, performing feature extraction to obtain an identification model, and identifying the pine wood nematode disease tree; and step 3: vectorizing the identification result of the pine wood nematode disease tree to obtain a longitude and latitude coordinate file of the pine wood nematode disease tree; and 4, step 4: and uploading the longitude and latitude coordinate information of the pine wood nematode disease tree to a pine wood nematode disease tree supervision platform, checking the geographical distribution condition of the disease tree through the supervision platform, and manually felling. The invention aims to solve the problem that the detection of small-scale pine wood nematode disease trees is missed under the complex background of remote sensing images, and provides a pine wood nematode disease tree detection method based on bottom-layer feature batch processing fusion and a multiple-feature multiplexing network model.

Description

A detection method based on small-scale pine wood nematode diseased trees

technical field

The invention relates to the technical field of detection of pine wood nematode trees, in particular to a detection method based on small-scale pine wood nematode trees.

Background technique

Forest pests and diseases are the old enemies of forest health and forestry production. Pine wood nematode is one of the invasive alien species that is more harmful to pine trees. After being infected with pine wood nematode, it will cause the pine needles to be yellowish-brown or reddish-brown. It is an important pest that causes large areas of pine forests to be destroyed because of its fast speed, wide spread, wide spread and difficult control. Therefore, timely detection of pine wood nematode trees is crucial.

At present, the monitoring methods of pine wood nematode mainly include ground survey, satellite remote sensing monitoring, multi-spectral UAV image and UAV remote sensing monitoring. The traditional survey methods of pine wood nematode use artificial census with low efficiency, high cost, and easy to cause omission; satellite remote sensing has low spatial resolution and poor timeliness; multispectral UAV image acquisition efficiency is low and high resolution. It is currently the most effective to obtain panchromatic band images through UAVs. UAV panchromatic band images can not only obtain images with higher spatial resolution, but also ensure the efficiency of image acquisition. The color, shape, texture and other characteristics of a single diseased tree can be obtained with high detection accuracy through deep learning to meet the needs of governance.

Since AlexNet proposed by Hinton et al. won the ImageNet Large-scale Visual Recognition Challenge Image Classification Competition in 2012, deep learning has developed rapidly and has been used more and more widely in the field of computer vision. Learning is applied to object detection in remote sensing scenes. For the detection of small objects in remote sensing images, Wang et al. proposed an improved SSD algorithm combined with Feature Pyramid (FPN), which improved the speed and accuracy of detection to a certain extent. Yao Qunli et al. proposed a multi-scale network remote sensing target detection framework-MSCNN. This algorithm has better results for multi-scale remote sensing large-scale targets, but has poor detection effect on multi-scale small-scale targets.

In order to achieve efficient detection of remote sensing small targets in complex backgrounds, this patent proposes a detection method for pine wood nematode disease trees based on the underlying feature batch fusion and multiple feature multiplexing network model. This method considers and redesigns the feature fusion module, which combines the shallow feature map mainly responsible for small target detection with the deep feature map with rich semantic information, adopts the batch modular feature fusion method, and fuses the shallow feature information. In order to enhance and reuse, the feature extraction ability of small-scale targets is improved, and the idea of transfer learning is adopted. First, model training is performed on relatively small small-scale diseased tree samples at low resolution, and then the model obtained by training is used. As a pre-trained model trained on a higher resolution 1:500 dataset, the network model's ability to detect small-scale target diseased trees is improved.

SUMMARY OF THE INVENTION

The purpose of the patent of the present invention is to solve the problem of missed detection of small-scale pine wood nematode trees under the complex background of remote sensing images, and proposes a pine wood nematode based on the underlying feature batch fusion and multiple feature multiplexing network model. Methods for the detection of diseased trees.

A detection method based on a small-scale pine wood nematode disease tree, specifically comprising the following steps:

Step 1: Acquire images, mark pine wood nematode trees, and make a dataset;

Step 2: Input the dataset into the underlying feature batch fusion and multiple feature multiplexing network model, perform feature extraction, obtain a recognition model, and identify pine wood nematode trees;

Step 3: vectorize the identification result of the pine wood nematode tree to obtain the longitude and latitude coordinate information of the pine wood nematode tree;

Step 4: Upload the longitude and latitude coordinate information of the pine wood nematode tree to the pine wood nematode tree supervision platform, check the geographical distribution of the diseased trees through the supervision platform, and cut down manually.

In step 2, the dataset is input into the underlying feature batch fusion and multiple feature multiplexing network model, and feature extraction is performed to obtain a recognition model. Specifically, the following steps are used:

2-1: Select and preprocess the UAV image sample set;

2-2: Classify the dataset according to the sick tree images of different scales, and transfer the training network model separately;

2-3: Input the data set into the underlying feature batch fusion and multiple feature multiplexing network model, extract multi-feature information, obtain a pine wood nematode tree identification model, and identify pine wood nematode trees.

In step 2-2, the datasets are classified according to different scales, and the network models are transferred separately for training, which includes the following sub-steps:

2-2-1: In the dataset sample with the scale value A, the scale of the diseased tree is relatively small, which is conducive to the feature extraction of the small-scale diseased tree by the model. First, input the data set with the scale value of A into the network model , the pine wood nematode disease tree recognition model is obtained by training;

2-2-2: Then input the dataset samples with the scale value of B into the network model, and use the recognition model obtained by training with the scale value of A as the pre-training model trained with the scale value of the B dataset samples for migration training. The final pine wood nematode tree identification model was obtained.

In steps 2-3, the underlying feature batch fusion and multiple feature multiplexing network models are constructed, including the following sub-steps:

2-3-1: Input the image to be detected into the feature extraction backbone network, and extract features from shallow to deep in a bottom-up manner, as follows:

The input image is downsampled by convolution to obtain the feature map L1;

The feature map L1 is subjected to the maximum pooling downsampling operation through convolution, and then the feature map L2 is obtained through multiple convolution residual block operations in turn; the feature map L2 is obtained through multiple convolution residual block operations to obtain the feature map L3; The feature map L3 is operated by multiple convolution residual blocks to obtain the feature map L4; the feature map L4 is operated by multiple convolution residual blocks to obtain the feature map L5.

2-3-2: Integrate the underlying feature batch processing and deep feature multiplexing to enhance the feature extraction ability of the model for pine wood nematode trees of different scales, and then identify the pine wood nematode trees. The specific steps are as follows:

Convolve the underlying feature map L1, feature map L2, and feature map L3 to make their number of channels the same, and then perform feature fusion to obtain feature map F2;

Use a 3*3 convolution kernel and a convolution with a stride of 1 to perform a convolution operation on the feature map F2 to eliminate the aliasing effect after fusion, and obtain the final predicted feature map Ps1, which is used to detect small-scale diseased trees;

Convolve the deep feature map L4 and the feature map L5 so that the number of channels of the feature map L4 and the feature map L5 is the same as the number of channels of the feature map L3; then perform feature fusion to obtain the feature map F3;

Use a 3*3 convolution kernel and a convolution with a stride of 1 to perform a convolution operation on the feature map F3 to eliminate the aliasing effect after fusion, and obtain the final predicted feature map Ps2, which is used to detect that the predicted feature map Ps1 is not detected. to the small-scale diseased tree;

The feature map L4 is fused with the feature map F2 that has undergone the down-sampling operation and the feature map L5 that has undergone the up-sampling operation to obtain the feature map F4;

Use a 3*3 convolution kernel and a convolution with a stride of 1 to perform a convolution operation on the feature map F4 to eliminate the aliasing effect after fusion, and obtain the predicted feature map Pm, which is used to detect medium-scale diseased trees;

The feature map L5 is fused with the feature map F2 subjected to the downsampling operation to obtain the feature map F5, and the feature map F5 is subjected to a convolution operation to obtain the predicted feature map P1, which is used to detect large-scale diseased trees;

In step 2, the loss function used in the underlying feature batch fusion and multiple feature multiplexing network model is the focal loss function, as follows:

The meaning of each parameter is as follows: y is the real label value (positive sample value is 1, negative sample value is 0); y' is the predicted category probability given by the model; γ is a hyperparameter, γ>0 makes it easier to classify The loss of samples makes the model pay more attention to difficult and misclassified samples. Changing the size of γ can also adjust the rate of weight reduction of simple samples. When γ is 0, it is the cross entropy loss function. When γ increases, adjust the factor The influence of is also increasing. This γ will automatically reduce the weight of the contribution of simple samples (samples with clear categories) to the loss during training, and quickly make the model pay more attention to difficult samples (samples whose categories are not easy to distinguish), here γ A value of 2 is optimal; α is a balance factor hyperparameter used to balance the uneven proportion of positive and negative samples, and the value here is 0.25.

In step 4, the following sub-steps are specifically included:

4-1: Import the latitude and longitude coordinate information of the identified diseased trees into the monitoring platform for pine wood nematode disease trees;

4-2: The geographical distribution of diseased trees can be viewed on the supervision platform for pine wood nematode diseased trees. According to the diseased tree navigation information on the supervision platform, the diseased trees can be found, and the diseased pine trees can be surveyed and cut down manually.

Compared with the prior art, the present invention has the following technical effects:

The invention is a detection method based on small-scale pine wood nematode diseased trees. At present, only the whole pine wood nematode diseased tree is detected, and there is no research on the detection of small-scale diseased trees. Therefore, a network model detection method based on batch fusion of underlying features and multiplexing of multiple features is proposed. In this method, a low-level feature batch fusion module is designed, which fuses the adjacent features of the shallow feature maps mainly responsible for small target detection, and fuses the deep features with rich semantic information with the shallow features, which improves the model's ability to detect small targets. Feature extraction capability.

Description of drawings

Below in conjunction with accompanying drawing and embodiment, the present invention will be further described:

Fig. 1 is the flow chart of the present invention;

Fig. 2 is a network structure diagram in the present invention.

Detailed ways:

As shown in Figure 1, a detection method based on small-scale pine wood nematode trees includes the following steps:

Step 1: Use drone aerial photography to obtain remote sensing images, use labelimg tool to mark pine wood nematode trees, and make a data set;

Step 2: Input the dataset into the underlying feature batch fusion and multiple feature multiplexing network model, perform feature extraction, obtain a recognition model, and identify pine wood nematode trees;

Step 3: vectorize the identification result of the pine wood nematode tree to obtain the longitude and latitude coordinate file of the pine wood nematode tree;

Step 4: Upload the longitude and latitude coordinate information of the pine wood nematode tree to the pine wood nematode tree supervision platform, check the geographical distribution of the diseased trees through the supervision platform, and cut down manually.

In step 1, use drone aerial photography to obtain remote sensing images, use the labelimg tool to mark diseased trees in the remote sensing images, and create a data set. The specific steps are as follows:

(1) Use DJI drones to obtain clear 1:1000 and 1:500 images of pine trees during the peak period of pine wood nematode disease in September, and crop the obtained 1:500 high-definition pine tree images into 1000*1000 size , crop the obtained 1:1000 pine tree image into 1500*1500 size, and use the labelimg tool to label the sick tree in the picture;

(2) Divide the labeled samples into two datasets with different scales according to 1:500 and 1:1000.

As shown in Figure 2, in step 2, the data set is input into the underlying feature batch fusion and multiple feature multiplexing network model, and feature extraction is performed to obtain a recognition model, which specifically includes the following steps:

(1) Downsampling the input image through a 7*7 convolution with a stride of 2 to obtain a feature map L1; subject the feature map L1 to a 3*3 convolution with a stride of 2 for maximum pooling downsampling operation, and then pass through three 1*1, 3*3, 1*1 convolution residual blocks in turn to obtain the feature map L2; operate the feature map L2 through four convolution residual blocks to obtain the feature map L3; L3 obtains the feature map L4 through 6 convolution residual block operations; then the feature map L4 is obtained through 3 convolution residual block operations to obtain the feature map L5.

The underlying feature map L1, feature map L2, and feature map L3 are subjected to a 1*1 convolution operation to make their number of channels the same, and then feature fusion is performed to obtain a feature map F2, and a 3*3 convolution is performed on the feature map F2. The operation obtains the final predicted feature map Ps1, which is responsible for detecting small-scale diseased trees;

The deep feature map L4 and the feature map L5 are convolved by 1*1, so that the number of channels of the feature map L4 and L5 is the same as that of the feature map L3, and then feature fusion is performed to obtain the feature map F3, and the feature map F3 is 3*3 The convolution operation of Ps1 obtains the final predicted feature map Ps2, which is responsible for detecting small-scale diseased trees missing from the feature map Ps1;

The feature map L4 is fused with the feature map F2 that fuses the underlying feature map L1, feature map L2, feature map L3 and the feature map L5 that performs the up-sampling operation for down-sampling operation to obtain the feature map F4, and the feature map F4 is processed. The 3*3 convolution operation obtains the predicted feature map Pm, which is responsible for detecting medium-scale diseased trees;

The feature map L5 is fused with the feature map F2 that has undergone the downsampling operation to obtain the feature map F5, and a 3*3 convolution operation is performed on the feature map F5 to obtain the predicted feature map P1, which is responsible for detecting large-scale diseased trees;

The traditional prediction of small-scale targets is to output the prediction results of small-scale targets in the third layer, but the results of the first-layer convolution and the second-layer convolution are also helpful for small-scale feature fusion, and the second-layer small-scale target information More, so the features of the first layer and the third layer are fused and added to the second layer to make a prediction for small-scale targets, which enhances the ability of small-scale feature extraction.

On the fourth layer of the deep feature map, the traditional feature fusion method is to fuse directly through the convolution layer and the up-sampled feature map of the fifth layer, which loses the underlying geometric information. The present invention fuses on the basis of the existing fusion. The underlying information of the first, second and third layers is added, so that the fourth layer of the deep feature map adds the underlying geometric information, and the feature extraction ability is enhanced;

On the fifth layer of the deep feature map, the traditional method is to directly obtain the prediction map through the convolution operation on the feature map of the fifth layer, and the present invention also fuses the shallow features into the feature map of the fifth layer, which improves the performance of the model. Feature extraction capability;

(2) Input the data set samples into the constructed network model, repeat the training for many times, remove the wrongly recognized samples each time, and use them as negative samples, input them into the network model to continue training, in the process of recognition , due to the complex topography of mountain forests, there will be other red car fronts, roofs, bare ground and pine-like trees that interfere with the detection of pine wood nematode trees. In the process of each identification, it is necessary to remove the wrongly identified samples and re-mark them. It is a negative sample, put it into the data set, and then go to training until there is a stable and relatively high model recognition rate.

(3) In the 1:1000 data sample, the scale of the diseased tree is relatively small, which is conducive to the identification of small-scale diseased trees. First, the 1:1000 data set is input into the network model, and the pine wood nematode disease tree is obtained by training. Identify the model. Then, the model obtained by 1:1000 training is used as a pre-training model trained with 1:500 data samples, and the training is performed to obtain the final recognition model.

(4) The loss function used in the network model is the focal loss, as follows:

The focal loss is a modification on the basis of the cross-entropy loss function. The two-category cross-entropy loss is as follows:

Among them, y is the real label value (the real sample value is 1, the negative sample value is 0), y' is the predicted category probability given by the model, and the cross entropy loss For positive samples, the larger the output probability, the smaller the loss. For negative samples, the smaller the output probability, the smaller the loss. The loss function at this time is slow in the iterative process of a large number of simple samples and may not be optimized to the optimal.

The focal loss adds a factor to the original, as follows:

Among them, γ is a hyperparameter, and γ>0 reduces the loss of easy-to-classify samples and allows the model to focus more on difficult and misclassified samples. Changing the size of γ can also adjust the rate at which the weight of the simple sample decreases. When γ is 0, it is the cross-entropy loss function. When γ increases, the influence of the adjustment factor also increases. This γ will automatically reduce the weight of the contribution of simple samples (samples with clear categories) to loss during training, and quickly make the model pay more attention to difficult samples (samples with difficult categories), where γ is optimal when the value is 2 .

In addition, a balance factor α is added to balance the uneven proportion of positive and negative samples, as follows:

In steps 3 and 4, the identification results of the pine wood nematode tree are vectorized to obtain the longitude and latitude coordinate files of the pine wood nematode tree; Check the geographical distribution of diseased trees through the supervision platform, and cut down manually, which includes the following steps:

(1) Input the pine forest remote sensing image with coordinate information that needs to be identified into the detection model, obtain the identification result of the pine wood nematode tree and the location information of the diseased tree, and convert the identification result of the pine wood nematode tree into latitude and longitude coordinate information . Import the longitude and latitude information of pine wood nematode trees into ArcGis software, you can check the distribution of diseased trees for statistical analysis;

(2) Import the latitude and longitude coordinate information of the identified diseased tree into the monitoring platform for pine wood nematode diseased trees;

(3) The pine wood nematode disease tree supervision platform can check the geographical distribution of diseased trees, find diseased trees according to the diseased tree navigation information of the supervision platform, and conduct investigation and cleaning of diseased pine trees.

The present invention designs the low-level feature batch modular fusion and deep-level feature multiplexing and fusion, fuses the shallow-level feature map mainly responsible for small target detection with the deep-level feature map with rich semantic information, and adopts the low-level feature adjacency batch fusion and deep-level feature map fusion. The combination of feature multiplexing and fusion enhances the shallow information and improves the model's ability to extract small target features.

At the same time, the invention adopts the method of migration experiment, firstly trains small-scale diseased trees at low resolution, and then migrates to a data set at higher resolution for training, so that the model can first learn the characteristics of small-scale diseased trees and improve the network model. Detection ability for small-scale diseased trees.

Claims (6)

1. A detection method based on a small-scale pine wood nematode tree, specifically comprising the following steps: Step 1: Acquire images, mark pine wood nematode trees, and make a dataset; Step 2: Input the dataset into the underlying feature batch fusion and multiple feature multiplexing network model, perform feature extraction, obtain a recognition model, and identify pine wood nematode trees; Step 3: vectorize the identification result of the pine wood nematode tree to obtain the longitude and latitude coordinate information of the pine wood nematode tree; Step 4: Upload the longitude and latitude coordinate information of the pine wood nematode tree to the pine wood nematode tree supervision platform, check the geographical distribution of the diseased trees through the supervision platform, and cut down manually. 2. The method according to claim 1, characterized in that, in step 2, the data set is input into the underlying feature batch fusion and multiple feature multiplexing network model, and feature extraction is performed to obtain a recognition model, and the following steps are specifically adopted. : 2-1: Select and preprocess the UAV image sample set; 2-2: Classify the dataset according to the different scales of the diseased tree images, and transfer and train the network model separately; 2-3: Input the data set into the underlying feature batch fusion and multiple feature multiplexing network model, extract multi-feature information, obtain a pine wood nematode tree identification model, and identify pine wood nematode trees. 3. method according to claim 2 is characterized in that, in step 2-2, according to the different scales of diseased tree pictures, the data set is classified, and the migration training network model is carried out separately, specifically comprising the following substeps: 2-2-1: In the dataset sample with the scale value A, the scale of the diseased tree is relatively small, which is conducive to the feature extraction of the small-scale diseased tree by the model. First, input the data set with the scale value of A into the network model , the pine wood nematode disease tree recognition model is obtained by training; 2-2-2: Then input the dataset samples with scale value B into the network model, and use the network model obtained by training with scale value A as a pre-training model trained on dataset samples with scale value B for migration training. The final pine wood nematode tree identification model was obtained. 4. method according to claim 2, is characterized in that, in step 2-3, constructs bottom layer feature batch processing fusion and multiple feature multiplexing network model, specifically comprises the following sub-steps: 2-3-1: Input the image to be detected into the feature extraction backbone network, and extract features from shallow to deep in a bottom-up manner, as follows: The input image is downsampled by convolution to obtain the feature map L1; The feature map L1 is subjected to the maximum pooling downsampling operation through convolution, and then the feature map L2 is obtained through multiple convolution residual block operations in turn; the feature map L2 is obtained through multiple convolution residual block operations to obtain the feature map L3; Feature map L3 obtains feature map L4 through multiple convolution residual block operations; feature map L4 is obtained through multiple convolution residual block operations to obtain feature map L5; 2-3-2: Integrate the underlying feature batch processing and deep feature multiplexing to enhance the feature extraction ability of the model for pine wood nematode trees of different scales, and then identify the pine wood nematode trees. 5. method according to claim 4 is characterized in that, in step 2-3-2, concrete steps are as follows: Convolve the underlying feature map L1, feature map L2, and feature map L3 to make their number of channels the same, and then perform feature fusion to obtain feature map F2; Use convolution to perform the convolution operation on the feature map F2 to eliminate the mixed and superimposed effect of the fused features, and obtain the final predicted feature map Ps1, which is used to detect small-scale diseased trees; Convolve the deep feature map L4 and the feature map L5 so that the number of channels of the feature map L4 and the feature map L5 is the same as the number of channels of the feature map L3; then perform feature fusion to obtain the feature map F3; Use convolution to perform the convolution operation on the feature map F3 to eliminate the feature mixing and superposition effect after fusion, and obtain the final predicted feature map Ps2, which is used to detect the small-scale diseased trees that are not detected by the predicted feature map Ps1; The feature map L4 is fused with the feature map F2 that has undergone the down-sampling operation and the feature map L5 that has undergone the up-sampling operation to obtain the feature map F4; Use convolution to perform the convolution operation on the feature map F4 to eliminate the feature mixing and superposition effect after fusion, and obtain the predicted feature map Pm, which is used to detect medium-scale diseased trees; The feature map L5 is fused with the feature map F2 that has undergone the down-sampling operation to obtain the feature map F5, and the feature map F5 is convolved to obtain the predicted feature map P1, which is used to detect large-scale diseased trees. 6. The method according to claim 1, wherein step 4 specifically comprises the following substeps: 4-1: Import the latitude and longitude coordinate information of the identified diseased trees into the monitoring platform for pine wood nematode disease trees; 4-2: The geographical distribution of diseased trees can be viewed on the supervision platform for pine wood nematode diseased trees. According to the diseased tree navigation information on the supervision platform, the diseased trees can be found, and the diseased pine trees can be surveyed and cut down manually.

Images