CN114708492A - Fruit tree pest and disease damage image identification method - Google Patents

Abstract

The invention discloses a fruit tree pest image identification method, relates to the technical field of agricultural planting, and solves the technical problems of low accuracy and long detection time of the existing citrus pest identification and diagnosis method. The method comprises the following steps: acquiring a pest and disease damage image sample, and marking; carrying out enhancement processing on the pest image sample by using an image enhancement method to obtain an enhanced data set; pre-training the VGG19 model by using an ImageNet data set to obtain a pre-training weight parameter; the VGG19-INC model is constructed by reserving the first 4 convolutional layers and pooling layers of the VGG19 model, replacing the 5 th convolutional layer of the VGG19 with 1 batch of standardized convolutional layers and 2 Inception modules, replacing the full-link layer of the VGG19 model with 1 global pooling layer, and finally using a 1 x 4 Softmax layer; training the VGG19-INC model using an enhanced data set; and applying the trained VGG19-INC model to the classification prediction of unknown pest and disease images to finally obtain the recognition result of the pest and disease images.

Description

Fruit tree pest and disease damage image identification method

Technical Field

The invention relates to the technical field of agricultural planting, in particular to a fruit tree pest image identification method.

Background

The diseases and pests of the oranges can be divided into invasive diseases and non-invasive diseases and three types of pests, particularly, the citrus planting is influenced most greatly by yellow dragon diseases, leaf miners, nematode diseases and the like, and in severe cases, the citrus planting can cause the complete extinction of the whole citrus section area. Therefore, the rapid and accurate identification and diagnosis of citrus diseases and insect pests provide a basis for later treatment, and the problem to be solved urgently by the current citrus planting industry is solved. At present, two methods for identifying and diagnosing citrus diseases and pests mainly comprise a visual diagnosis method and a pathological characteristic detection method. The visual inspection diagnosis method depends on the field visual inspection and diagnosis of growers or agricultural technicians, which greatly depends on the professional technical level and experience of the growers or agricultural technicians, and the subjective difference causes the visual inspection diagnosis of the citrus diseases and insect pests to have large fluctuation of accuracy and low stability. The pathological characteristic detection method is a method for quantitative analysis of plant diseases and insect pests, and detection and analysis are carried out on a disease and insect pest sample by using biological, physical, chemical and other detection means, and a conclusion is given according to the method. The method has higher accuracy rate of identifying the citrus diseases and insect pests, but the detection can be completed only by related detection equipment and professional technicians, so the detection time is long, and the method is difficult to popularize in a large range.

Disclosure of Invention

The invention aims to solve the technical problem of the prior art, and aims to provide a fruit tree pest image identification method which can improve the accuracy and shorten the detection time.

The technical scheme of the invention is as follows: a fruit tree pest and disease damage image identification method comprises the following steps:

acquiring a pest and disease damage image sample, and marking;

enhancing the pest and disease damage image sample by using an image enhancement method to obtain an enhanced data set;

pre-training the VGG19 model by using an ImageNet data set to obtain a pre-training weight parameter;

the VGG19-INC model is constructed by reserving the first 4 convolutional layers and pooling layers of the VGG19 model, replacing the 5 th convolutional layer of the VGG19 with 1 batch of standardized convolutional layers and 2 Inception modules, replacing the full-connection layer of the VGG19 model with 1 global pooling layer, and finally using a 1 x 4 Softmax layer;

training the VGG19-INC model using the enhanced data set;

and applying the trained VGG19-INC model to the classification prediction of unknown pest and disease images to finally obtain the recognition result of the pest and disease images.

As a further improvement, the pest image sample comprises a pest sample and a health sample; and preprocessing the pest and disease damage image sample by a filtering method or an equalization method before enhancement processing.

Further, the enhancement processing comprises any one of horizontal turning, vertical turning, random horizontal translation and random vertical translation; and the number of the pest and disease damage image samples obtained after enhancement treatment is not less than 1000.

Further, the VGG19-INC model includes:

the pre-training module is used for extracting the outline characteristics and the texture characteristics of the image according to the pre-training weight parameters of the VGG 19;

the high-dimensional feature extraction module is used for realizing high-dimensional feature fusion of plant diseases and insect pests through the multi-feature fusion characteristic of the inclusion module;

and the feature matrix dimension reduction and classification module is used for realizing the functions of reducing calculation parameters and classifying.

Further, the VGG19-INC model adds the batch of standardized convolutional layers between the backbone network of the VGG19 and the inclusion module, and uses the Swish activation function instead of the Relu activation function in the batch of standardized convolutional layers.

Further, in the VGG19-INC model, extraction of high-dimensional features of an image is realized by an inclusion module, the inclusion module adopts two different convolution kernels of 1 × 1 and 3 × 3 for stacking, a plurality of convolution operations are executed in parallel, and finally, output results of the convolution operations are connected into a high-dimensional feature map.

Furthermore, the VGG19-INC model replaces the full link layer of the VGG19 model with a global pooling layer, averages all values of the whole feature map, fuses all values of each feature map of the convolutional layer into one feature value, enables the dimension of the feature values to be the same as the final classification number, forms column vectors with the same number as the classification number, and achieves dimension reduction of the feature matrix.

Further, let the feature map size be m × n, and the value of the k-th feature map be

Showing that after the global average pooling operation, the output characteristic value corresponding to the characteristic diagram is y^(k)：

Further, the VGG19-INC model is trained for no less than 100 times, and in the training process, a random gradient descent optimization algorithm is used, wherein a momentum factor in the random gradient descent optimization algorithm is set to be 0.8, a learning rate is 0.0001, an attenuation coefficient of the learning rate is 1e-6, and a batch size is set to be 24.

Further, the VGG19-INC model uses Accuracy, precision Accuracy, Recall, F1 score F1 score to evaluate model indexes, as shown in formulas (2), (3), (4), (5):

Accuracy＝(TP+TN)/(TP+TN+FP+FN) (2)

Prencision＝TP/(TP+FP) (3)

Recall＝TP/(TP+FN) (4)

F1＝2*Precision*Recall/(Precision+Recall) (5)

in the formula, TP: number of samples for which the true value is positive and the prediction is also positive; FP: the number of samples whose true value is negative but predicted to be positive; FN: the number of samples whose true value is positive but predicted to be negative; TN: number of samples whose true value is negative and whose prediction is also negative.

Advantageous effects

Compared with the prior art, the invention has the advantages that:

the VGG19-INC model of the invention takes a VGG19 network model as a backbone network and utilizes transfer learning to realize the sharing of pre-training weight parameters; the model structure uses 1 batch of standardized convolutional layers and 2 inclusion modules to replace the 5 th convolutional layer of VGG 19; and replacing the fully-connected layer of the VGG19 model by 1 global pooling layer, and finally using a 1 x 4 Softmax layer as a classification output layer. The model not only keeps the effective extraction of the image features by the VGG19, but also increases the depth and width of the network by using the inclusion module, so that the model obtains receptive fields with different sizes, and the fusion of multi-scale features is realized; in addition, the replacement of the global pooling layer for the full-connection layer enables the parameter reduction rate to reach 70.56%, effectively improves the training speed and the average testing speed of the model, and reduces the parameter load. The method can adjust the convolution module according to the specialized characteristics close to the top layer of the neural network, has more flexibility and pertinence, has better classification effect, ensures that the new model has higher identification precision and stronger generalization capability.

Drawings

FIG. 1 is a flow chart of the present invention;

FIG. 2 is a VGG19 model test classification confusion matrix;

FIG. 3 is a Resnet50 model test classification confusion matrix;

FIG. 4 is an Incepistationv 3 model test classification confusion matrix;

FIG. 5 is a class confusion matrix for the Densenet201 model test;

FIG. 6 is a VGG19-INC model test classification confusion matrix.

Detailed Description

The invention will be further described with reference to specific embodiments shown in the drawings.

VGG19 is a classical convolutional neural network developed by researchers at oxford university and google, inc. It adopts cascade network structure, which is composed of 16 convolution layers and 3 full-connection layers; each convolutional layer uses a 3 × 3 convolutional kernel, with a total of 5 maximum pooling layers between convolutional layers; the last layer of the network is the Softmax classifier. Since the VGG19 model uses a fixed 3 × 3 convolution kernel, the receptive field is fixed, some fine features are easily ignored, and thus, the fine-grained feature extraction is not sufficient. Therefore, when the model is used for identifying and classifying the early citrus greening disease images and the nematode disease images and the healthy leaves with higher similarity, the identification rate is lower. In addition, the VGG19 model uses 3 full connection layers, and the parameter amount reaches 143MB, so that the requirement on computing resources is large, the model training is slow, and the model is not favorable for being deployed at a mobile terminal.

Referring to fig. 1-6, a fruit tree pest and disease damage image identification method is characterized by comprising the following steps:

acquiring a pest and disease damage image sample, and marking;

carrying out enhancement processing on the pest and disease image sample by using an image enhancement method to obtain an enhanced data set;

pre-training the VGG19 model by using an ImageNet data set to obtain a pre-training weight parameter;

the method comprises the steps of reserving the first 4 convolutional layers and pooling layers of a VGG19 model, replacing the 5 th convolutional layer of a VGG19 with 1 batch of standardized convolutional layers and 2 inclusion modules, replacing the full-connection layer of a VGG19 model with 1 global pooling layer, and finally constructing by using a 1 x 4 Softmax layer to obtain a VGG19-INC model, wherein the VGG19-INC model is a pest and disease image recognition model with stronger generalization capability and high recognition accuracy;

training the VGG19-INC model using the enhanced data set;

and applying the trained VGG19-INC model to the classification prediction of unknown pest and disease images to finally obtain the recognition result of the pest and disease images.

In this embodiment, the pest image samples include pest samples and health samples, such as peach leaf spot disease, citrus greening disease, tomato yellow leaf curl disease, pumpkin powdery mildew, and healthy apple leaf images; and (3) preprocessing the pest and disease image sample by a filtering method or an equalization method before enhancement processing.

The enhancement treatment comprises any one of horizontal turning, vertical turning, random horizontal translation and random vertical translation; the number of the disease and insect damage image samples obtained after enhancement treatment is not less than 1000.

The VGG19-INC model includes:

the pre-training module is used for extracting the outline characteristics and the texture characteristics of the image according to the pre-training weight parameters of the VGG 19;

the high-dimensional feature extraction module is used for realizing high-dimensional feature fusion of plant diseases and insect pests through the multi-feature fusion characteristic of the inclusion module;

and the feature matrix dimension reduction and classification module is used for realizing the functions of reducing calculation parameters and classifying.

The VGG19-INC model adds a batch of standardized convolutional layers between a backbone network of VGG19 and an inclusion module, and uses a Swish activation function to replace a Relu activation function in the batch of standardized convolutional layers, so that the convergence speed of the model can be increased, the identification precision of the model is improved, and overfitting is reduced.

In the VGG19-INC model, extraction of high-dimensional features of an image is realized by an inclusion module, the inclusion module adopts two different convolution kernels of 1 × 1 and 3 × 3 for stacking, a plurality of convolution operations are executed in parallel, and finally output results of the convolution operations are connected into a high-dimensional feature map. Compared with a 3X 3 phase convolution layer adopted by VGG19, the inclusion module increases the depth and width of the network and improves the adaptability of the network to the scale, so that the sensing fields with different sizes are obtained.

The VGG19-INC model replaces the full link layer of the VGG19 model with the global pooling layer, averages all values of the whole feature map, fuses all values of each feature map (with the size of m × n) of the convolutional layer into one feature value, enables the dimension of the feature value to be the same as the final classification number, forms column vectors with the same number as the classification number, and forms 4 column vectors if the classification number is 4, thereby realizing the dimension reduction of the feature matrix, and reducing the number of calculation parameters while ensuring the classification effect.

Let the size of the feature map be m × n, and the value of the kth feature map be

Showing that after the global average pooling operation, the output characteristic value corresponding to the characteristic diagram is y^(k)：

The VGG19-INC model is trained for no less than 100 times, in the training process, an enhanced data set is divided into a training set and a testing set according to the proportion of 8:2, a random gradient descent optimization algorithm is used, the momentum factor in the random gradient descent optimization algorithm is set to be 0.8, the learning rate is 0.0001, the attenuation coefficient of the learning rate is 1e-6, and the batch size is set to be 24.

The VGG19-INC model uses Accuracy Accuracy, precision Prencion, Recall, F1 score F1 score to evaluate model indexes as shown in formulas (2), (3), (4), (5):

Accuracy＝(TP+TN)/(TP+TN+FP+FN) (2)

Prencision＝TP/(TP+FP) (3)

Recall＝TP/(TP+FN) (4)

F1＝2*Precision*Recall/(Precision+Recall) (5)

in the formula, TP: number of samples for which the true value is positive and the prediction is also positive; FP: the number of samples whose true value is negative but predicted to be positive; FN: the number of samples whose true value is positive but predicted to be negative; TN: the number of samples for which the true value is negative and the prediction is also negative.

In this example, images of 27 types of diseased leaves and 11 types of healthy leaves for 16 types of plants are provided in the plantavivollage public dataset. Due to the unbalanced distribution of the data set, the present study used a data enhancement method to make 1000 pictures of each type, for a total of 34200 pictures, and divided the training set and the test set in a ratio of 8: 2. During experimental verification, 5 types of 1000 images such as peach spot disease, citrus yellow dragon disease, tomato yellow leaf curl disease, pumpkin powdery mildew and apple healthy leaf image are selected as training and testing data.

Under the condition that the data of the PlantVillage data set is sufficient, after 70 times of training, the oscillation of the training and verification accuracy rate curve of each model becomes small and tends to be smooth, and the method for transfer learning can accelerate the training process of each model. After 100 rounds of training, the VGG19-INC model has 97.97% accuracy and 94.03% accuracy on the training set and the verification set respectively, and the loss values are 0.0179 and 0.188 respectively.

The evaluation results of each 100-piece test of 5 types of pictures of peach spot disease, citrus greening disease, tomato yellow leaf curl disease, pumpkin powdery mildew and apple healthy leaf image using the VGG19-INC model are shown in Table 1.

TABLE 1 evaluation index of VGG19-INC model test results

From table 1, the VGG19-INC model showed an average of 98.00% accuracy in image recognition of 5 types of plant diseases and insect pests, an average of 95.10% accuracy, an average of 95.00% recall, and an average of 95.00% F1 score. The evaluation indexes of the 5 types of images are uniform, and the model performance is good.

The citrus (Shatian pomelo) pest data set in the embodiment is from the agricultural scientific research institute of Yulin city, autonomous region of Guangxi Zhuang, and the pest image is collected from the Shatian pomelo planting base of Qianqiu village, Yulin city, in Guangxi, and comprises 3 common citrus diseases and pests such as yellow dragon disease, leaf miner, nematode and health and healthy leaf samples. Under sufficient light conditions, the images were taken using a nikon 3100 digital camera and the pixel size was 2992 × 2000. Randomly selecting 500 of the samples as an experimental data set; dividing the test sample into a training set and a testing set according to the ratio of 8:2, and labeling 4 types of health, yellow dragon disease, leaf miner and nematode by 1, 2, 3 and 4 respectively.

Under the conditions that the scale of the citrus disease and insect pest data set is small and the disease images are similar, through 100 rounds of training, the VGG19-INC model obtains higher accuracy and lower loss value; the accuracy on training and validation sets was 99.05% and 98.47%, respectively. The training set is 22.26%, 14.47%, 5.18% and 0.24% higher than the VGG19, Resnet50, Inceptionv3 and Densenet201 models, respectively. The test set is 22.36%, 7.92%, 10.84% and 0.55% higher respectively. Meanwhile, the VGG19-INC model has a loss function value of 0.0415, which is lower than other models by 1.1085, 0.2172, 0.3987 and 0.0654 respectively.

Table 2100 runs after each model performance and test results

In addition, as can be seen from table 2, in the 5 models, the number and weight of the parameters of the VGG19 model occupy the largest space, the average test time is the longest, and the average accuracy is 77.91%. The space occupied by the Densenet201 model parameters and the weight is minimum, the average accuracy is 81.38%, the average test time is 0.32/s, and the balance between the space and the average accuracy can be obtained. The average accuracy of the VGG19-INC model is the highest and is 95.25%, and the classification performance is the best. This is because the VGG19-INC uses the global average pooling layer instead of the fully connected layer of VGG19, compared to 1.42 × 10 in the VGG19 model⁸The weight parameter is reduced to 4.18 x 10⁷And the parameter reduction rate reaches 70.56 percent, the training speed and the average testing speed of the model are effectively improved, and the weight of the model is reduced to 163Mb, so that the model is favorably deployed at a mobile terminal.

2-6 show the shuffling matrix of classification tests of models, in order to test the generalization ability of the models, the present study compared the VGG19-INC model with the Densenet201 model which performs well in the training set and the validation set, and the accuracy, precision, recall, and F1 score of the two models are shown in Table 3.

Table 3 evaluation indexes of VGG19-INC and Densenet201 model detection results

From fig. 2-fig. 6 and table 3, it can be seen that the accuracy of the densenert 201 model is between 45.10% and 100%, and the average value is 75.18%; f1 score ranged from 56.64% to 67.54%, with an average value of 63.21%; the accuracy is 69.75-86.75%, and the average value is 81.38%; the recall rate is between 49.00% and 97.00%, and the average value is 62.75%. The evaluation indexes are not uniformly distributed, the difference is large, and the average value is low.

The accuracy of the VGG19-INC model is 83.62% -95.34%, and the average value is 90.98%; f1 is between 88.17% and 93.65%, and the average value is 90.45%; the accuracy is between 94.50% and 96.75%, and the average value is 95.25%; the recall rate is between 82.00% and 97.00%, and the average value is 90.5%. All evaluation indexes are uniformly distributed, and the average value is more than 90%, which shows that the model with improved structure is superior to the Densenet201 model and has good classification performance.

The training precision of the model is improved by using the BN layer with the Swish activation function, the training precision of the model using the Swish activation function is improved by 0.85 percent after 100 rounds of training, the verification precision is improved by 0.55 percent, and the training loss is reduced by 0.0196, as shown in Table 4.

TABLE 4 model accuracy and loss rate with Swish and Relu as activation functions

The invention utilizes an improved VGG19-INC model and a plurality of deep convolutional neural network models to carry out classification and identification tests on citrus disease and insect pest images, and obtains the following conclusion through comparative analysis:

(1) the deep convolutional neural network can be used for well and automatically extracting the citrus disease and insect pest characteristics, manual segmentation of citrus disease and insect pest image characteristics is not needed, and the classification performance is good on the whole.

(2) The transfer learning can fully utilize the learning knowledge on the large-scale data set, thereby accelerating the training speed of the model, saving the training time, improving the training precision and solving the problem of poor classification effect of the model caused by undersize data set.

(3) For a fine-grained classification task of the citrus disease images, the average accuracy of the designed VGG19-INC model reaches 95.25% by adjusting and combining the traditional model, and the model is superior to models such as Incept ion V3, VGG19, Resnet50 and Densenet 201. Meanwhile, the new model reduces the number of weight parameters, so that the occupied weight space is reduced, the model training speed is improved, the average detection time of a single image is reduced, and the characteristic extraction method can be further optimized by combining different convolutional neural network models, so that the characteristic extraction capability and the model generalization capability are improved, and the image identification requirements of different scenes are met.

The above is only a preferred embodiment of the present invention, and it should be noted that it is obvious to those skilled in the art that several variations and modifications can be made without departing from the structure of the present invention, which will not affect the effect of the implementation of the present invention and the utility of the patent.

Claims (10)

1. A fruit tree pest image identification method is characterized by comprising the following steps:

acquiring a pest and disease damage image sample, and marking;

enhancing the pest and disease damage image sample by using an image enhancement method to obtain an enhanced data set;

pre-training the VGG19 model by using an ImageNet data set to obtain a pre-training weight parameter;

the VGG19-INC model is constructed by reserving the first 4 convolutional layers and pooling layers of the VGG19 model, replacing the 5 th convolutional layer of the VGG19 with 1 batch of standardized convolutional layers and 2 Inception modules, replacing the full-link layer of the VGG19 model with 1 global pooling layer, and finally using a 1 x 4 Softmax layer;

training the VGG19-INC model using the enhanced data set;

and applying the trained VGG19-INC model to the classification prediction of unknown pest and disease images to finally obtain the recognition result of the pest and disease images.

2. The fruit tree pest image identification method according to claim 1, wherein the pest image sample comprises a pest sample and a health sample; and preprocessing the pest and disease damage image sample by a filtering method or an equalization method before enhancement processing.

3. The fruit tree pest image identification method according to claim 1, wherein the enhancement treatment comprises any one of horizontal turning, vertical turning, random horizontal translation and random vertical translation; and the quantity of the pest and disease damage image samples obtained after enhancement treatment is not less than 1000.

4. The fruit tree pest image identification method according to any one of claims 1-3, wherein the VGG19-INC model comprises:

the pre-training module is used for extracting contour features and texture features of the image according to pre-training weight parameters of VGG 19;

the high-dimensional feature extraction module is used for realizing high-dimensional feature fusion of plant diseases and insect pests through the multi-feature fusion characteristic of the inclusion module;

and the characteristic matrix dimension reduction and classification module is used for realizing the functions of reducing calculation parameters and classifying.

5. The fruit tree pest image identification method according to claim 4, wherein the VGG19-INC model adds the batch of standardized convolutional layers between the backbone network of VGG19 and the inclusion module, and uses a Swish activation function instead of a Relu activation function in the batch of standardized convolutional layers.

6. The fruit tree pest image identification method according to claim 4, wherein in the VGG19-INC model, extraction of high-dimensional features of an image is realized by an inclusion module, the inclusion module adopts two different convolution kernels of 1 x 1 and 3 x 3 for stacking, a plurality of convolution operations are executed in parallel, and finally, the output results of the convolution operations are connected into a high-dimensional feature map.

7. The fruit tree pest image identification method according to claim 4, wherein the VGG19-INC model replaces a full connection layer of the VGG19 model with a global pooling layer, averages all values of a whole feature map, fuses all values of each feature map of a convolutional layer into one feature value, enables the dimension of the feature value to be the same as the number of final classifications, forms column vectors with the same number as the number of classifications, and achieves dimension reduction of a feature matrix.

8. The fruit tree pest image identification method according to claim 7, wherein the size of the characteristic diagram is m x n, and the value of the kth characteristic diagram is used

Showing that after the global average pooling operation, the corresponding output characteristic value of the characteristic diagram is y^(k)：

9. The fruit tree pest image identification method according to claim 1, wherein the VGG19-INC model is trained for no less than 100 times, and in the training process, a random gradient descent optimization algorithm is used, wherein a momentum factor in the random gradient descent optimization algorithm is set to be 0.8, a learning rate is 0.0001, an attenuation coefficient of the learning rate is 1e-6, and a batch size is set to be 24.

10. The fruit tree pest image identification method according to claim 1, wherein the VGG19-INC model uses Accuracy, precision Accuracy, Recall, F1 score F1 score to evaluate model indexes, as shown in formulas (2), (3), (4) and (5):

Accuracy＝(TP+TN)/(TP+TN+FP+FN) (2)

Prencision＝TP/(TP+FP) (3)

Recall＝TP/(TP+FN) (4)

F1＝2*Precision*Recall/(Precision+Recall) (5)

in the formula, TP: number of samples for which the true value is positive and the prediction is also positive; FP: the number of samples whose true value is negative but predicted to be positive; FN: the number of samples whose true value is positive but predicted to be negative; TN: the number of samples for which the true value is negative and the prediction is also negative.

Images