CN117475230A - Grape disease leaf image classification prediction method by using GC-CapsNet - Google Patents
Grape disease leaf image classification prediction method by using GC-CapsNet Download PDFInfo
- Publication number
- CN117475230A CN117475230A CN202311485573.3A CN202311485573A CN117475230A CN 117475230 A CN117475230 A CN 117475230A CN 202311485573 A CN202311485573 A CN 202311485573A CN 117475230 A CN117475230 A CN 117475230A
- Authority
- CN
- China
- Prior art keywords
- disease
- grape
- feature map
- convolution
- leaf
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 201000010099 disease Diseases 0.000 title claims abstract description 151
- 208000037265 diseases, disorders, signs and symptoms Diseases 0.000 title claims abstract description 151
- 235000009754 Vitis X bourquina Nutrition 0.000 title claims abstract description 123
- 235000012333 Vitis X labruscana Nutrition 0.000 title claims abstract description 123
- 235000014787 Vitis vinifera Nutrition 0.000 title claims abstract description 123
- 238000000034 method Methods 0.000 title claims abstract description 58
- 240000006365 Vitis vinifera Species 0.000 title 1
- 241000219095 Vitis Species 0.000 claims abstract description 122
- 230000008569 process Effects 0.000 claims abstract description 20
- 230000011218 segmentation Effects 0.000 claims abstract description 17
- 238000004891 communication Methods 0.000 claims abstract description 7
- 238000012549 training Methods 0.000 claims description 36
- 238000011176 pooling Methods 0.000 claims description 27
- 239000002775 capsule Substances 0.000 claims description 21
- 238000010586 diagram Methods 0.000 claims description 18
- 238000012360 testing method Methods 0.000 claims description 12
- 238000012545 processing Methods 0.000 claims description 9
- 238000002372 labelling Methods 0.000 claims description 5
- 230000009466 transformation Effects 0.000 claims description 4
- 241000221785 Erysiphales Species 0.000 claims description 3
- 241000233679 Peronosporaceae Species 0.000 claims description 3
- 230000003213 activating effect Effects 0.000 claims description 3
- 208000024891 symptom Diseases 0.000 claims description 3
- 238000000605 extraction Methods 0.000 abstract description 6
- 238000010187 selection method Methods 0.000 abstract description 3
- 238000001514 detection method Methods 0.000 abstract description 2
- 241000196324 Embryophyta Species 0.000 description 7
- 230000000694 effects Effects 0.000 description 7
- 230000006870 function Effects 0.000 description 6
- 238000002474 experimental method Methods 0.000 description 5
- 238000003709 image segmentation Methods 0.000 description 4
- 238000005070 sampling Methods 0.000 description 4
- 230000009286 beneficial effect Effects 0.000 description 3
- 230000000052 comparative effect Effects 0.000 description 3
- 230000006872 improvement Effects 0.000 description 3
- 238000005457 optimization Methods 0.000 description 3
- 238000013459 approach Methods 0.000 description 2
- 238000013528 artificial neural network Methods 0.000 description 2
- 238000012937 correction Methods 0.000 description 2
- 230000010339 dilation Effects 0.000 description 2
- 235000013399 edible fruits Nutrition 0.000 description 2
- 238000005516 engineering process Methods 0.000 description 2
- 230000002708 enhancing effect Effects 0.000 description 2
- 238000011156 evaluation Methods 0.000 description 2
- 230000004927 fusion Effects 0.000 description 2
- 230000007246 mechanism Effects 0.000 description 2
- 238000012986 modification Methods 0.000 description 2
- 230000004048 modification Effects 0.000 description 2
- 241000894006 Bacteria Species 0.000 description 1
- 241000219094 Vitaceae Species 0.000 description 1
- 230000004913 activation Effects 0.000 description 1
- 230000005540 biological transmission Effects 0.000 description 1
- 238000004422 calculation algorithm Methods 0.000 description 1
- 238000004364 calculation method Methods 0.000 description 1
- 238000006243 chemical reaction Methods 0.000 description 1
- 238000013145 classification model Methods 0.000 description 1
- 238000013527 convolutional neural network Methods 0.000 description 1
- 238000013135 deep learning Methods 0.000 description 1
- 235000021021 grapes Nutrition 0.000 description 1
- 238000005286 illumination Methods 0.000 description 1
- 230000000050 nutritive effect Effects 0.000 description 1
- 238000011056 performance test Methods 0.000 description 1
- 230000002265 prevention Effects 0.000 description 1
- 230000004044 response Effects 0.000 description 1
- 238000005096 rolling process Methods 0.000 description 1
- 238000010206 sensitivity analysis Methods 0.000 description 1
- 235000019605 sweet taste sensations Nutrition 0.000 description 1
- 238000012546 transfer Methods 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
- G06V10/764—Arrangements for image or video recognition or understanding using pattern recognition or machine learning using classification, e.g. of video objects
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/0464—Convolutional networks [CNN, ConvNet]
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/0475—Generative networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
- G06N3/084—Backpropagation, e.g. using gradient descent
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/20—Image preprocessing
- G06V10/26—Segmentation of patterns in the image field; Cutting or merging of image elements to establish the pattern region, e.g. clustering-based techniques; Detection of occlusion
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/40—Extraction of image or video features
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
- G06V10/77—Processing image or video features in feature spaces; using data integration or data reduction, e.g. principal component analysis [PCA] or independent component analysis [ICA] or self-organising maps [SOM]; Blind source separation
- G06V10/774—Generating sets of training patterns; Bootstrap methods, e.g. bagging or boosting
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
- G06V10/82—Arrangements for image or video recognition or understanding using pattern recognition or machine learning using neural networks
Abstract
The invention relates to the technical field of classification and prediction of grape disease leaf images, in particular to a classification and prediction method of grape disease leaf images by using GC-CapsNet. The G-ENT segmentation model for segmenting the grape disease leaf image provided by the invention realizes automatic segmentation of the grape disease leaf image and extraction of related features, and improves the accuracy of subsequent prediction of the grape leaf disease type. In addition, by introducing a multi-scale loop communication convolution method to improve the feature learning process in the convolution layer, the feature information learned by images with different scales is reserved, so that errors in the classification process can be reduced to the greatest extent, and the accuracy of GC-CapsNet is improved. Finally, an inverse regression module is introduced as a feature selection method, so that the parameter quantity transmitted to a deep network is reduced, the time complexity of the whole model is reduced, and the detection precision of the model is improved by selecting important features and discarding features which have no meaning to a prediction result.
Description
Technical Field
The invention relates to the technical field of classification and prediction of grape disease leaf images, in particular to a classification and prediction method of grape disease leaf images by using GC-CapsNet.
Background
Grape has a large share in fruit trade markets in China. Has rich nutritive value and sweet taste, is deeply favored by people, and becomes one of daily edible fruits. However, in the grape planting process, diseases are generated by bacteria, environment and the like, and the diseases of grape leaves are most. Therefore, the identification of the grape leaf diseases is realized by utilizing the computer vision technology, timely and accurate information and prevention measures are provided for users, and the method has important significance for improving the yield and quality of the grapes.
Reference is made to the literature in recent years for classifying grape disease leaf images based on a deep learning method. For example, chinese patent application number CN202211397472.6 discloses a grape leaf disease identification method based on an improved Xception algorithm. The method mainly improves the Xattention network through a pre-training model, wherein the training speed of the model is accelerated and a channel attention mechanism is added into the model to improve the accuracy of model identification through changing an activation function in the original network. However, the method does not analyze the characteristics of the grape leaf diseases, but improves the classification performance of the model by singly adding a channel attention mechanism in the original Xreception network. The technical means can only lead the model to obtain a very small lifting effect, wherein a large degree of prediction error exists, and the accurate classification of the grape leaf diseases can not be achieved. In order to improve the classification accuracy of grape leaf diseases, a method for identifying grape leaf diseases is disclosed in literature ("grape leaf disease identification based on a multi-scale residual neural network", "computer engineering", he Xin, etc., 2021-05-07), the grape leaf positions are extracted by using Mask R-CNN after data enhancement and leaf area labeling are carried out on grape leaf images, and the response of a ResNet bottom layer to different scale features is changed by introducing multi-scale convolution, so that the feature extraction capability of the network is improved by using the added SENet. The method improves the characteristic extraction capability of the network and achieves the aim of improving the identification accuracy of grape leaf diseases. However, this approach is complex, requiring more computational resources and training time.
In order to ensure that the time consumption is reduced in the process of predicting the type of the grape leaf diseases and improve the accuracy of the type prediction of the grape leaf diseases, the method for classifying and predicting the grape leaf images by using GC-CapsNet is provided.
Disclosure of Invention
A grape disease leaf image classification prediction method by using GC-CapsNet is used for solving the technical problems. According to the invention, the automatic segmentation of the grape disease leaf image and the extraction of related features are realized through the G-ENT segmentation model for segmenting the grape disease leaf image, so that the accuracy of the subsequent prediction of the grape leaf disease type is improved. In addition, by introducing a multi-scale loop communication convolution method to improve the feature learning process in the convolution layer, the feature information learned by images with different scales is reserved, so that errors in the classification process can be reduced to the greatest extent, and the accuracy of GC-CapsNet is improved. Finally, an inverse regression module is introduced as a feature selection method, so that the parameter quantity transmitted to a deep network is reduced, the time complexity of the whole model is reduced, and the module discards features which have no meaning on a prediction result by selecting important features.
In order to achieve the above purpose, the present invention provides the following technical solutions:
a grape disease leaf image classification prediction method by using GC-CapsNet comprises the following steps.
Acquiring a grape leaf image with disease symptoms in a real background from the public data set, and constructing a grape leaf image set;
wherein the public data set includes a plant Vill data set and an AI changer data set.
Further, carrying out data enhancement processing on the grape leaf image set so as to achieve the effects of expanding samples and enhancing images; the data enhancement processing process comprises the following steps:
selecting a grape leaf image with low definition in the grape leaf image set;
inputting the low-definition grape leaf image into a generated countermeasure model;
outputting a high-definition grape leaf image;
an extended data set is obtained.
The training set is used for training of GC-CapsNet; the test set is used for performing performance test on the trained GC-CapsNet. The English of the GC-CapsNet is totally named as Grape Convolutional Capsule Network, namely a grape convolution capsule network;
labeling a foreground region and a background region of the grape leaves in the training set, and converting the labeled images into G-ENT identifiable files;
further, a G-ENT segmentation model is adopted to segment the background and the foreground of the grape leaf image in the extended data set, so that a disease leaf image is obtained;
the English of G-ENT is called Grape Efficient Network, namely grape high-efficiency network. The ene implements semantic segmentation on the image by assigning a specific label to each pixel in the image. The structure of the high-efficiency network is divided into an encoding module and a decoding module, and pixel-level classification and up-sampling operation are realized through down-sampling of a hidden layer, so that the positioning of a target in an image is realized. The ene utilizes smaller feature patterns, reduces network parameters, and improves the running speed of the network. In addition, the resolution of the small-scale feature map and the receptive field of the image can be balanced by means of dilation convolution.
The segmentation process of the disease leaf image specifically comprises the following steps:
inputting the grape leaf images in the training set to the coding module, and obtaining a first coding convolution characteristic diagram with the channel number of 32 after a convolution kernel of 1x 1;
the first coding convolution feature map is subjected to maximum pooling operation of 2x2 after passing through a filling and activating function with the step length of 1, so as to obtain a first pooling feature map;
after the first pooling feature map passes through two 3x3 convolution layers, a second coding convolution feature map with the channel number of 64 is obtained;
the second coding convolution feature map is subjected to 2x2 maximum pooling operation to obtain a second pooling feature map;
the second pooling feature map obtains a third coding convolution feature map with the channel number of 128 after two convolution layers of 3x 3;
the third coding convolution feature map is subjected to 2x2 maximum pooling operation to obtain a third pooling feature map;
after the third pooling feature map passes through two 3x3 convolution layers, a fourth coding convolution feature map with 256 channels is obtained;
inputting the fourth coding convolution feature map to the decoding module, and splicing the fourth coding convolution feature map with the third coding convolution feature map after deconvolution kernel of 2x2 to obtain a first decoding convolution feature map with 256 channels;
after the first decoding convolution feature map passes through two 3x3 convolution layers, a second decoding convolution feature map with the channel number of 128 is obtained;
after the deconvolution kernel of 2x2, the second decoding convolution feature map is spliced with the second coding convolution feature map to obtain a third decoding convolution feature map with the channel number of 128;
after passing through two 3x3 convolution layers, the third decoding convolution feature map obtains a fourth decoding convolution feature map with the channel number of 64;
after the deconvolution kernel of 2x2, the fourth decoding convolution feature map is spliced with the first coding convolution feature map to obtain a fifth decoding convolution feature map with the channel number of 64;
after passing through two 3x3 convolution layers, the fifth decoding convolution feature map obtains a sixth decoding convolution feature map with the channel number of 32;
and outputting a disease blade image after the sixth decoding convolution feature map passes through a convolution kernel of 1x1 to obtain a disease blade image set.
Further, dividing the disease leaf image set into a training set and a testing set;
further, labeling the image disease type label in the training set;
wherein the disease types of grape leaves include anthracnose, downy mildew, powdery mildew, brown spot, anthracnose, felty disease, black rot and yellow leaf.
Further, scaling the disease blade image by three images of different sizes, namely a first disease blade image, a second disease blade image and a third disease blade image;
wherein the first disease blade image size is 128x128, the second disease blade image size is 96x96, and the third disease blade image size is 64x64.
Further, inputting three disease leaf images into GC-Capsule for feature extraction;
training the GC-CapsNet according to the characteristics and disease type labels corresponding to the training set;
the GC-CapsNet specific training process comprises the following steps:
inputting the first disease blade image, inputting the first disease blade image into an imposition network after passing through a convolution kernel with a channel of 128,7x7, and obtaining a first disease blade characteristic diagram through an inverse regression module;
the second disease blade image is input and spliced with the first disease blade characteristic image through a convolution kernel with the channel of 256 and 5x5, and is input to an imposition network, and the second disease blade characteristic image is obtained through an inverse regression module;
the third disease blade image is input and spliced with the second disease blade characteristic image through a convolution kernel with the channel of 256 and 3x3, and is input to an imposition network, and the third disease blade characteristic image is obtained through an inverse regression module;
inputting the third disease leaf feature map to a main capsule layer, and outputting a transformation feature vector;
inputting the transformation feature vector, and updating the main capsule layer through a dynamic routing layer to obtain a prediction feature;
and outputting a grape leaf disease type prediction probability value.
The inverse regression module is used for eliminating the blade characteristic diagram which does not contain any important classification information, and the specific operation steps are as follows:
step one: setting a feature selection threshold to 0.5;
step two: fitting a model by using the output characteristics;
step three: selecting a characteristic value to compare with a threshold value;
step four: judging whether the characteristic value meets a threshold value or not, if not, neglecting, and continuing fitting; otherwise, the comparison is completed;
step five: the above steps are repeated until all significant blade features are identified.
The important blade characteristic identification mode is as follows:
y(m k )＝b 0 +b 1 x 1 +...+b 1 x n +ε
wherein y (m) k ) Representing the important blade features selected for the kth class model, b 0 Expressed as a bias constant, b 1 Denoted as superparameter, epsilon as correction parameter. X is x 1 And x n Represented as the 1 st and n-th features of the grape leaf, respectively.
Further, the GC-CapsNet model obtains a final GC-CapsNet model after multiple times of optimization by calculating errors between sample predicted values and real labels in a training set; the error calculating method comprises the following steps:
wherein L is a loss value of calculating grape leaf diseases, C is the number of grape leaf diseases, and r k Representing the proportion of the real number of the k-th grape leaf images to the total grape disease leaf images, N k The correct number, N ', of leaf images of the kth grape disease is detected' k Expressed as a kth grape diseaseThe actual number of blade images.
Further, inputting the grape disease leaf images in the test set to a final GC-CapsNet model for prediction;
and outputting the disease type and the predicted probability value.
Compared with the prior art, the invention has the beneficial effects that:
1. the invention provides a G-ENT segmentation model for segmenting a grape disease leaf image, and an ENT model is reconstructed through an optimization model frame. The model realizes automatic segmentation of grape disease leaf images and extraction of relevant features. Under different blade sizes, different illumination conditions and different backgrounds, the model can provide reliable segmentation results, and accuracy of subsequent prediction of grape leaf disease types is improved.
2. The invention provides a multi-scale loop communication convolution method in the learning process of disease features of grape leaves by GC-CapsNet. The method mainly carries out feature learning on three grape leaf images with different scales, and carries out transfer fusion on the features learned by the grape leaf images with different scales, thereby solving the problem of large calculation load caused by deepening deep features of network level learning and accelerating the learning efficiency of a network; in addition, the method adopts the idea of multi-scale feature fusion, and features of grape leaves with different scales are transmitted and fused, so that information under different scales is fully utilized. In the feature learning process of grape leaves, features with multiple scales, such as local details, textures, shapes and the like, are possibly involved, and play an important role in classification decision, so that the classification accuracy is improved.
3. The feature selection method is used for selecting the features in the feature learning process of the grape leaves by introducing the inverse regression module. The module can avoid the transmission of unimportant features to the GC-Capsule deep layer, reduce the generation of the parameter number and reduce the time complexity of the GC-capsule; and the module can well reserve important characteristics, so that the disease type prediction accuracy of the grape leaves is improved.
Drawings
FIG. 1 is a schematic diagram of a method for classifying and predicting grape disease leaf images by using GC-CapsNet according to an embodiment of the present invention;
fig. 2 is a schematic diagram of a grape disease leaf image provided by an embodiment of the present invention;
FIG. 3 is a schematic diagram of a G-ENT segmentation model according to an embodiment of the present invention;
FIG. 4 is a schematic diagram showing experimental results of a G-ENT segmentation model according to an embodiment of the present invention;
FIG. 5 is a schematic diagram of a GC-CapsNet model provided by an embodiment of the present invention;
FIG. 6 is a schematic flow chart of an inverse regression module according to an embodiment of the present invention;
fig. 7 is a schematic diagram of a characteristic threshold comparison experiment according to an embodiment of the present invention.
Detailed Description
The following description of the embodiments of the present invention will be made clearly and completely with reference to the accompanying drawings, in which it is apparent that the embodiments described are only some embodiments of the present invention, but not all embodiments. All other embodiments, which can be made by those skilled in the art based on the embodiments of the invention without making any inventive effort, are intended to be within the scope of the invention.
Example 1:
referring to fig. 1, the structure diagram of the classification and prediction method for grape disease leaf images by using GC-capsule net according to the present invention is shown. The method comprises the following steps:
s10, acquiring a grape leaf image and establishing a grape leaf image set; the invention acquires the grape leaf image with disease symptoms in the real background from the public data set (see figure 2 for details);
s20, carrying out data processing on the grape leaf image;
s30, segmenting the grape leaf image, and marking the disease type to obtain a disease leaf image;
s40, dividing a training set and a testing set;
s50, scaling and zooming the disease blade image;
s60, establishing GC-CapsNet;
s70, outputting a disease type result.
Preferably, the step S10 specifically includes: the grape leaf image set was from the Plant Village dataset and the AI changer dataset for a total of 3650 images. Wherein the Plant Village dataset is named as a Plant Village dataset and comprises Plant images from different regions of different Plant species, wherein each image is labeled with a corresponding disease category; the AI Challenger data set is named as a human intelligent challenge game data set, and is characterized in that the plant village data set is classified according to different disease degrees.
Further, the data enhancement processing is carried out on the images in the grape leaf image set, so that the functions of expanding data samples and enhancing images are achieved; the data enhancement processing process comprises the following steps:
selecting a grape leaf image with low definition in the grape leaf image set;
inputting the low-definition grape leaf image into a generated countermeasure model;
the generated countermeasure model is a generated model which learns in a manner of mutual countermeasure through two kinds of neural networks, and consists of two components of a generator and a discriminator. The generator is for generating a target sample similar to the original sample, and the discriminator is for discriminating the generated target sample from the original sample. The application fields of generating the countermeasure model are very wide, including image generation, text generation, video generation, and the like.
Outputting a high-definition grape leaf image;
an extended data set is obtained.
Further, the extended dataset has 4770 grape leaf images in total.
Further, in order to accurately classify the disease types of the grape leaves, referring to fig. 3, the example uses a G-ene model to segment the grape leaf images in the extended dataset. The G-ene is an improvement over ene, which implements semantic segmentation on an image by assigning a specific label to each pixel in the image. The structure of the ENT is divided into an encoding module and a decoding module, and the pixel-level classification and the up-sampling operation are realized through the down-sampling of the hidden layer, so that the positioning of the target in the image is realized. The ene utilizes smaller feature patterns, reduces network parameters, and improves the running speed of the network. In addition, the resolution of the small-scale feature map and the receptive field of the image can be balanced by means of dilation convolution.
Example 2:
the invention refers to the S30 for image segmentation;
inputting grape leaf image X i Obtaining a first coding convolution characteristic diagram with the channel number of 32 after a convolution kernel of 1x1 to a coding module;
the first coding convolution feature map is subjected to maximum pooling operation of 2x2 after passing through a filling and activating function with the step length of 1, so as to obtain a first pooling feature map;
after the first pooling feature map passes through two 3x3 convolution layers, a second coding convolution feature map with the channel number of 64 is obtained;
the second coding convolution feature map is subjected to 2x2 maximum pooling operation to obtain a second pooling feature map;
the second pooling feature map obtains a third coding convolution feature map with the channel number of 128 after two convolution layers of 3x 3;
the third coding convolution feature map is subjected to 2x2 maximum pooling operation to obtain a third pooling feature map;
after the third pooling feature map passes through two 3x3 convolution layers, a fourth coding convolution feature map with 256 channels is obtained;
inputting the fourth coding convolution feature map to the decoding module, and splicing the fourth coding convolution feature map with the third coding convolution feature map after deconvolution kernel of 2x2 to obtain a first decoding convolution feature map with 256 channels;
after the first decoding convolution feature map passes through two 3x3 convolution layers, a second decoding convolution feature map with the channel number of 128 is obtained;
after the deconvolution kernel of 2x2, the second decoding convolution feature map is spliced with the second coding convolution feature map to obtain a third decoding convolution feature map with the channel number of 128;
after passing through two 3x3 convolution layers, the third decoding convolution feature map obtains a fourth decoding convolution feature map with the channel number of 64;
after the deconvolution kernel of 2x2, the fourth decoding convolution feature map is spliced with the first coding convolution feature map to obtain a fifth decoding convolution feature map with the channel number of 64;
after passing through two 3x3 convolution layers, the fifth decoding convolution feature map obtains a sixth decoding convolution feature map with the channel number of 32;
after the sixth decoding convolution characteristic diagram is subjected to a convolution kernel of 1X1, a disease blade image X is output i '。
The G-ene proposed in this example can effectively segment the blade image area and improve the accuracy of classifying the disease types. To demonstrate the effectiveness of the proposed solution, referring to fig. 4, the G-ENet model and the ENet model were trained 500 times in the figure, each 100 times with accuracy recorded. The graph shows that the training model of 500 rounds achieves the optimal training, and the accuracy rate reaches more than 90%.
Example 3:
in order to illustrate that the accuracy of grape leaf disease prediction can be improved by adopting the G-ENT model. Comparative experiments of the prediction effect are performed in table 1, and experimental comparative results of three models, namely, a first model, a second model and a third model, are shown in the table, respectively. The first model represents a prediction model for image segmentation by adopting G-ENT; the second model represents a predictive model segmented by using ENet; the third model represents a predictive model that does not perform image segmentation.
TABLE 1 comparison of accuracy of different models
| Model | First model | Second model | Third model |
| Accuarcy(％) | 95.6％ | 91.7％ | 87.9％ |
As can be seen from the data in Table 1, the accuracy of the first model prediction is higher than that of the other two models, and the accuracy of 95.6% is achieved, and the first model prediction has good performance compared with the existing classification model. According to the third model experimental result, the data set processed by the image segmentation technology is beneficial to improving the model prediction performance.
Preferably, step S40 includes:
the disease leaf image set was processed according to 7:3, dividing a training set and a testing set; wherein the training set includes 3339 images and the test set 1431 images.
Further, marking disease types on the images in the training set; wherein the disease types (see figure 2 in detail) comprise anthracnose, downy mildew, powdery mildew, brown spot, anthracnose, felty disease, black rot and yellow leaf.
Preferably, the 50 includes:
scaling the images in the training set for training of a GC-capsule model; the specific implementation process is that the image I is scaled into a first disease blade image I1, a second disease blade image I2 and a third disease blade image I3 according to the sizes of 128x128, 96x96 and 64x64.
Example 4:
the step 60 refers to the step S50, and the disease type prediction is performed on the first disease blade image I1, the second disease blade image I2 and the third disease blade image I3 by using the GC-Capsule (see FIG. 5 in detail); the model improves a capsule network by introducing a multi-scale loop communication rolling and reverse regression module, learns grape leaf characteristics and is used for predicting grape leaf diseases; wherein the grape leaf feature comprises: local detail features, texture features, and shape features.
Inputting a first disease blade image I1, inputting the first disease blade image I1 into an importation network after passing through a convolution kernel with a channel of 128,7x7, and obtaining a first disease blade characteristic diagram F1 through an inverse regression module;
the second disease blade image I2 is input to be spliced with the F1 through a convolution kernel with a channel of 256 and 5x5, and is input to an imposition network, and a second disease blade characteristic diagram F2 is obtained through an inverse regression module;
the third disease blade image I3 is input to the imposition network through the convolution kernel with the channel of 256 and 3x3 and the F2 for splicing, and the third disease blade characteristic diagram F3 is obtained through the inverse regression module;
inputting the F3 to a main capsule layer, and outputting a conversion characteristic vector V;
inputting the V, and updating the main capsule layer through a dynamic routing layer to obtain a prediction characteristic P;
the GC-CapsNet is used for eliminating the blade characteristic diagram which does not contain any important classification information by referring to an inverse regression module; referring to fig. 6, the reverse regression module specifically includes the following steps:
step one: setting a feature selection threshold to 0.5;
step two: fitting a model by using the output characteristics;
step three: selecting a characteristic value to compare with a threshold value;
step four: judging whether the characteristic value meets a threshold value or not, if not, neglecting, and continuing fitting; otherwise, the comparison is completed;
step five: the above steps are repeated until all significant blade features are identified.
The important blade characteristic identification mode is as follows:
y(m k )＝b 0 +b 1 x 1 +...+b 1 x n +ε
wherein y (m) k ) Representing the important blade features selected for the kth class model, b 0 Expressed as a bias constant, b 1 Denoted as superparameter, epsilon as correction parameter. Represented as the 1 st and n-th features of the grape leaf, respectively.
Fifth embodiment:
preferably, the 70 includes:
referring to the above embodiment, according to the prediction feature P, a Softmax function is used to calculate, and a prediction probability result of the image I is output, and according to the result, the disease type of the image I is determined.
Further, the GC-CapsNet model obtains a final GC-CapsNet model after multiple times of optimization by calculating errors between sample predicted values and real labels in a training set; the error calculating method comprises the following steps:
wherein L is a loss value of calculating grape leaf diseases, C is the number of grape leaf diseases, and r k Representing the proportion of the real number of the k-th grape leaf images to the total grape disease leaf images, N k The correct number, N ', of leaf images of the kth grape disease is detected' k Expressed as the true number of kth grape disease leaf images.
In the multi-scale loop connected convolution method of the GC-CapsNet to improve the characteristic learning process in the convolution layer, the method takes the characteristic images learned by the upper layer as the input of the lower layer by inputting three grape disease leaf images with different scales, so that the characteristic information learned by the images with different scales is reserved, errors in the classification process can be reduced to the greatest extent, and the accuracy of the GC-CapsNet is improved. In this embodiment, by comparing the model effects under the normal convolution, it is illustrated that the multi-scale loop communication convolution mode is helpful to improve the prediction performance of the model. The comparison of experimental effects on the model in the multi-scale loop connected convolution mode and in the normal convolution mode is given in table 2, where the performance evaluation indicators shown in the table include accuracy P, recall R and F1 values.
TABLE 2 comparison of experiments on the convolution pattern of the multi-scale loop communication
As can be seen from the data in table 2, when GC-capsule net uses a scale convolution loop connected convolution approach, the model performance is optimal at this time, and the values of all evaluation indexes occupy the highest.
In the invention, an inverse regression module is designed as an important feature selection means, and the module is further processed by capturing important feature parameters. The module reduces the parameter quantity transmitted to the deep network, reduces the time complexity of the whole model, and discards the characteristics which have no meaning on the prediction result by selecting important characteristics, thereby positively influencing the detection precision of the model. Experiments on the beneficial effects of the inverse regression module in this example prove that the sensitivity analysis of the inverse regression module is performed in table 3, and the embodiment of the specific example is that the effects of the added regression module and the non-added regression module under different proportion training sets and test sets are compared by comparing GC-CapsNet.
TABLE 3 comparison of the performance of the reverse regression modules at different data ratios
As can be seen from table 3, the F1 value corresponding to the model also changes when the division ratio of the data set changes, but the addition or removal of the comparative inverse regression module also affects the performance of the model. It can be seen from the table that the model shows better F1 values when it adds an inverse regression module than when it does not. Therefore, experimental data show that the application of the inverse regression module has a certain prediction performance improvement effect on the model.
In this example, the inverse regression module is configured to select a feature of the prediction result, and set the feature threshold P to 0.5 in the inverse regression module, and perform an experimental description on the feature threshold to ensure the rationality of the set parameter. The P values are set to 0.45, 0.5, 0.65 and 0.8 in fig. 7, respectively, and these 4 threshold settings illustrate the selection of the optimal threshold by the loss function of GC-capsule trained over 500 rounds.
Example six:
in addition, this example section is explained by adding experiments. All the ideas presented in this invention are compared with other models to demonstrate the effectiveness of the invention. In table 4, by 3:7, respectively proving the effectiveness of the method in the training set and the test set.
TABLE 4 experimental comparison of different models on training and test sets
| Model | Trainingaccuracy(％) | Testingaccuracy(％) |
| ResNet | 87.6 | 85.9 |
| InceptionV3 | 89.6 | 86.3 |
| CapsNet | 90.5 | 88.7 |
| GC-CapsNet | 95.6 | 94.7 |
The foregoing describes specific embodiments of the present disclosure. Other embodiments are within the scope of the following claims. In some cases, the actions or steps recited in the claims can be performed in a different order than in the embodiments and still achieve desirable results. In addition, the processes depicted in the accompanying figures do not necessarily require the particular order shown, or sequential order, to achieve desirable results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.
The present disclosure is intended to embrace all such alternatives, modifications and variances which fall within the broad scope of the appended claims. Any omissions, modifications, equivalents, improvements, and the like, which are within the spirit and principles of the one or more embodiments of the disclosure, are therefore intended to be included within the scope of the disclosure.
Claims (10)
1. A grape disease leaf image classification prediction method by using GC-CapsNet is characterized by comprising the following steps:
dividing the background and the foreground of the grape leaf image by adopting a G-ENT division model to obtain a disease leaf image;
labeling the disease leaf image disease type label;
scaling the size of the disease blade image to obtain three disease blade images with different sizes, namely a first disease blade image, a second disease blade image and a third disease blade image;
extracting blade characteristics from the disease blade image through a GC-CapsNet model;
training the GC-CapsNet according to the leaf characteristics and disease type labels corresponding to the training set;
the GC-CapsNet specific training process comprises the following steps:
inputting the first disease blade image, inputting the first disease blade image into an imposition network after passing through a convolution kernel with a channel of 128,7x7, and obtaining a first disease blade characteristic diagram through an inverse regression module;
the second disease blade image is input and spliced with the first disease blade characteristic image through a convolution kernel with the channel of 256 and 5x5, and is input to an imposition network, and the second disease blade characteristic image is obtained through an inverse regression module;
the third disease blade image is input and spliced with the second disease blade characteristic image through a convolution kernel with the channel of 256 and 3x3, and is input to an imposition network, and the third disease blade characteristic image is obtained through an inverse regression module;
inputting the third disease leaf feature map to a main capsule layer, and outputting a transformation feature vector;
inputting the transformation characteristic vector, and updating the main capsule layer through a dynamic routing layer;
and outputting a grape leaf disease type prediction probability value.
2. The method for classifying and predicting a grape disease leaf image by using GC-capsule net according to claim 1, wherein the method for classifying and predicting the background and the foreground of the grape leaf image by using the G-ene segmentation model further comprises, before obtaining the disease leaf image:
acquiring a grape leaf image with disease symptoms under a real background through a public data set;
performing data enhancement processing on the grape leaf image to obtain an extended data set, and dividing the extended data set into a training set and a testing set;
the training set is used for training the GC-CapsNet;
the test set is used for testing the trained GC-CapsNet;
labeling a foreground region and a background region of the grape leaves in the training set, and converting the labeled images into the G-ENT identifiable file;
and training the segmentation model according to the training set and the identifiable file to obtain the G-ENT segmentation model.
3. The method for classifying and predicting grape disease leaf images by GC-capsule according to claim 2, wherein the data enhancement processing comprises:
selecting a low-definition grape leaf image in the training set;
inputting the low-definition grape leaf image into a generated countermeasure model;
outputting a high-definition grape leaf image;
obtaining the extended data set.
4. The method for classifying and predicting the grape disease leaf images by using the GC-CapsNet according to claim 2, wherein the G-ENT segmentation model comprises an encoding module and a decoding module, and the specific process comprises the following steps:
inputting the enhanced grape leaf image to the coding module, and obtaining a first coding convolution characteristic diagram with the channel number of 32 after a convolution kernel of 1x 1;
the first coding convolution feature map is subjected to maximum pooling operation of 2x2 after passing through a filling and activating function with the step length of 1, so as to obtain a first pooling feature map;
after the first pooling feature map passes through two 3x3 convolution layers, a second coding convolution feature map with the channel number of 64 is obtained;
the second coding convolution feature map is subjected to 2x2 maximum pooling operation to obtain a second pooling feature map;
the second pooling feature map obtains a third coding convolution feature map with the channel number of 128 after two convolution layers of 3x 3;
the third coding convolution feature map is subjected to 2x2 maximum pooling operation to obtain a third pooling feature map;
after the third pooling feature map passes through two 3x3 convolution layers, a fourth coding convolution feature map with 256 channels is obtained;
inputting the fourth coding convolution feature map to the decoder, and splicing the fourth coding convolution feature map with the third coding convolution feature map after deconvolution kernel of 2x2 to obtain a first decoding convolution feature map with 256 channels;
after the first decoding convolution feature map passes through two 3x3 convolution layers, a second decoding convolution feature map with the channel number of 128 is obtained;
after the deconvolution kernel of 2x2, the second decoding convolution feature map is spliced with the second coding convolution feature map to obtain a third decoding convolution feature map with the channel number of 128;
after passing through two 3x3 convolution layers, the third decoding convolution feature map obtains a fourth decoding convolution feature map with the channel number of 64;
after the deconvolution kernel of 2x2, the fourth decoding convolution feature map is spliced with the first coding convolution feature map to obtain a fifth decoding convolution feature map with the channel number of 64;
after passing through two 3x3 convolution layers, the fifth decoding convolution feature map obtains a sixth decoding convolution feature map with the channel number of 32;
and outputting a disease blade image after the sixth decoding convolution characteristic map passes through a convolution kernel of 1x 1.
5. The method for classifying and predicting grape disease leaf images by using GC-CapsNet according to claim 1, wherein the disease types comprise anthracnose, downy mildew, powdery mildew, brown spot, black pox, felt disease, black rot and yellow mosaic.
6. The method for classifying and predicting grape disease leaf image using GC-capsule of claim 1, wherein the first disease leaf image size is 128x128, the second disease leaf image size is 96x96, and the third disease leaf image size is 64x64.
7. The method for classifying and predicting grape disease leaf images by using GC-CapsNet according to claim 1, wherein the GC-CapsNet comprises the steps of introducing a multi-scale loop connected convolution and an inverse regression module to improve an original capsule network.
8. The method for classifying and predicting grape disease leaf images by using GC-CapsNet according to claim 7, wherein the multi-scale loop communication convolution adopts three convolution modules to deeply propagate grape leaf features with different scales to a network in a loop connection mode; wherein the grape leaf feature comprises: local detail features, texture features, and shape features.
9. The method for classifying and predicting the grape disease leaf images by using the GC-CapsNet according to claim 7, wherein the inverse regression is used for selecting important features of leaf images with different scales after passing through three convolution modules, and the next processing is performed by acquiring important feature parameters; wherein the process of inverse regression comprises:
step one: setting a feature selection threshold to 0.5;
step two: fitting a model by using the output characteristics;
step three: selecting a characteristic value to compare with a threshold value;
step four: judging whether the characteristic value meets a threshold value or not, if not, neglecting, and continuing fitting; otherwise, the comparison is completed;
step five: the above steps are repeated until all significant blade features are identified.
10. The method for classifying and predicting the grape disease leaf images by utilizing the GC-CapsNet according to claim 1, wherein the GC-CapsNet is obtained by calculating an error value between a prediction result and a true value of a training set and optimizing the error value for a plurality of times; the error value is calculated by adopting the following loss function, and the formula is as follows:
wherein L is a loss value of calculating grape leaf diseases, C is the number of grape leaf diseases, and r k Representing the proportion of the real number of the k-th grape leaf images to the total grape disease leaf images, N k The correct number, N ', of leaf images of the kth grape disease is detected' k Expressed as the true number of kth grape disease leaf images.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202311485573.3A CN117475230A (en) | 2023-11-09 | 2023-11-09 | Grape disease leaf image classification prediction method by using GC-CapsNet |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202311485573.3A CN117475230A (en) | 2023-11-09 | 2023-11-09 | Grape disease leaf image classification prediction method by using GC-CapsNet |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN117475230A true CN117475230A (en) | 2024-01-30 |
Family
ID=89632659
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202311485573.3A Pending CN117475230A (en) | 2023-11-09 | 2023-11-09 | Grape disease leaf image classification prediction method by using GC-CapsNet |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN117475230A (en) |
-
2023
- 2023-11-09 CN CN202311485573.3A patent/CN117475230A/en active Pending
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN110956185B (en) | Method for detecting image salient object | |
| CN110428428B (en) | Image semantic segmentation method, electronic equipment and readable storage medium | |
| CN110428432B (en) | Deep neural network algorithm for automatically segmenting colon gland image | |
| CN112183258A (en) | Remote sensing image road segmentation method based on context information and attention mechanism | |
| CN107871316B (en) | Automatic X-ray film hand bone interest area extraction method based on deep neural network | |
| CN112347970B (en) | Remote sensing image ground object identification method based on graph convolution neural network | |
| CN115661144A (en) | Self-adaptive medical image segmentation method based on deformable U-Net | |
| CN110287777B (en) | Golden monkey body segmentation algorithm in natural scene | |
| CN114187311A (en) | Image semantic segmentation method, device, equipment and storage medium | |
| CN114821069A (en) | Building semantic segmentation method for double-branch network remote sensing image fused with rich scale features | |
| CN113743505A (en) | Improved SSD target detection method based on self-attention and feature fusion | |
| CN111062381A (en) | License plate position detection method based on deep learning | |
| CN110991374B (en) | Fingerprint singular point detection method based on RCNN | |
| CN114897782B (en) | Gastric cancer pathological section image segmentation prediction method based on generation type countermeasure network | |
| CN113920468A (en) | Multi-branch pedestrian detection method based on cross-scale feature enhancement | |
| CN115359029A (en) | Semi-supervised medical image segmentation method based on heterogeneous cross pseudo-supervised network | |
| Zhang et al. | Residual attentive feature learning network for salient object detection | |
| Dong et al. | Field-matching attention network for object detection | |
| CN114841961A (en) | Wheat scab detection method based on image enhancement and improvement of YOLOv5 | |
| CN111666977B (en) | Shadow detection method of monochrome image | |
| CN115100509B (en) | Image identification method and system based on multi-branch block-level attention enhancement network | |
| CN114494284B (en) | Scene analysis model and method based on explicit supervision area relation | |
| CN116681921A (en) | Target labeling method and system based on multi-feature loss function fusion | |
| CN111242114A (en) | Character recognition method and device | |
| CN115330759A (en) | Method and device for calculating distance loss based on Hausdorff distance |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication |