CN114170137B

CN114170137B - Pepper disease identification method, identification system and computer readable storage medium

Info

Publication number: CN114170137B
Application number: CN202111306502.3A
Authority: CN
Inventors: 唐源; 余英浩; 李丽平; 唐有万; 谭华强; 李昱瑾; 林劼
Original assignee: Chengdu Univeristy of Technology
Current assignee: Chengdu Univeristy of Technology
Priority date: 2021-11-05
Filing date: 2021-11-05
Publication date: 2023-07-04
Anticipated expiration: 2041-11-05
Also published as: CN114170137A

Abstract

The invention belongs to the technical field of pest and disease detection, and discloses an identification system and a computer readable storage medium, wherein pictures related to pepper leaves, fruits and rhizomes are collected, and the pictures are subjected to label division to construct a data set; dividing the whole database into a training set and a verification set through random segmentation, and carrying out data enhancement operation processing on the image; constructing a network structure of a pepper plant disease and insect pest identification model by combining deep learning and transfer learning, and constructing a new convolution network model; training the model by utilizing a training set in the data set, and improving various parameters in the network model to obtain an optimal network model; and loading the trained model into a mobile phone APP to obtain a mobile application program, and identifying unclassified pepper pest images to obtain an identification result of the images to be identified. The invention has high portability and can optimally realize the pest and disease identification part in the whole process.

Description

Pepper disease identification method, identification system and computer readable storage medium

Technical Field

The invention belongs to the technical field of pest detection, and particularly relates to a pepper disease identification method, an identification system and a computer readable storage medium.

Background

At present, grain growers lack not only expert knowledge in pest and disease damage identification and prevention, but also expert first-line guidance, and when pest and disease damage occurs, great economic loss is easy to cause. Therefore, research on intelligent pest detection technology is important to control pest transmission. The invention relates to a method for identifying plant diseases and insect pests in a pepper growth process based on deep learning.

China is a large agricultural country, and capsicum is a necessary food material in daily life of people. In recent years, therefore, the area and area in which peppers are planted have been gradually enlarged. And the occurrence of the diseases and insect pests of the peppers can directly influence the yield and quality of the peppers. More than ten common diseases of capsicum are caused by leaves and stems, wherein powdery mildew, brown spot and the like of the leaves are the most common. At present, the identification of the pepper diseases mainly depends on expert to the site for naked eye identification, and the method is time-consuming, labor-consuming and low in efficiency and has a certain subjective speculation. Under such a background, intelligent identification of pepper diseases based on lesion images has been a challenging research topic in precision agriculture.

The current grain pest identification tool based on deep learning is Convolutional Neural Networks (CNNs). Lee et al propose a CNNs system based on leaf images for automatically identifying plants. Grinblat et al developed a neural network of relatively simple structure but powerful for successfully identifying three different leguminous plants based on the morphology pattern of the veins. Mohanty et al compared the application of two well-known, established CNN structures to 26 plant diseases using an open database of 14 different plant leaf images. Their result recognition accuracy is as high as 99.35%. However, one major drawback is that the entire photographic environment only includes images of laboratory scenes, not under cultivated real conditions. Sladojevic et al, using similar amounts of data obtained on the Internet, developed a similar method for detecting plant disease by leaf images, including a smaller number of diseases (13) and different plants (5). Their model success rate was between 91% and 98% based on the test data. Fuentes et al studied a CNN model for detecting 9 different tomato diseases and insect pests with satisfactory performance. However, in natural environments, it is often impractical to expect a designed classical algorithm to completely eliminate the impact of scene changes on recognition results. In a truly complex natural environment, plant pest detection faces many challenges such as small differences between lesion areas and backgrounds, low contrast, large scale variations of lesion areas and various types, large noise of lesion images, and the like.

Through the above analysis, the problems and defects existing in the prior art are as follows:

(1) At present, the identification of the pepper diseases mainly depends on expert to the site for naked eye identification, and the method is time-consuming, labor-consuming and low in efficiency and has a certain subjective speculation.

(2) In the current grain pest identification method based on deep learning, the whole shooting environment only comprises images of laboratory scenes, but not under the real cultivation condition.

(3) In a truly complex natural environment, plant pest detection faces many challenges, such as small difference between a lesion area and a background, low contrast, large scale variation of the lesion area and various types, large lesion image noise, and the like, and it is unrealistic to expect that a classical algorithm completely eliminates the influence of scene variation on a recognition result.

The difficulty of solving the problems and the defects is as follows:

(1) The subjective assumption is recognized by the pepper plant diseases and insect pests expert, and the efficiency of the expert to the field recognition is too low to be widely applied;

(2) The difference between the laboratory and the field actual environment is unpredictable, and has great interference to the identification of plant diseases and insect pests

The meaning of solving the problems and the defects is as follows:

(1) The portable lightweight model can be used for widely popularizing the identification of the disease and insect damage of the core part of the capsicum of the patent to farmers, and can be used for identifying capsicum directly in fields, so that the problem that capsicum growers can prevent and treat diseases under the condition of lacking first-line expert guidance is solved;

(2) The data set of the invention is more than 80% from the field real environment, so that the pepper plant diseases and insect pests identification model of the invention is trained by using real field data, the influence on the identification result is greatly reduced by directly using the model in the field after the model is transplanted to a mobile phone, the data set can regularly enrich the field real pictures, and the final data set is 100% from the field real environment, thereby improving the robustness of the model.

Disclosure of Invention

Aiming at the problems existing in the prior art, the invention provides a pepper disease identification method, an identification system and a computer readable storage medium, in particular to a pepper disease identification method and a system based on a lightweight convolutional neural network model.

The invention is realized in such a way that a pepper disease identification method comprises the following steps:

randomly dividing an original data set taking the characteristics of fruits, leaves and rhizomes as branches into a training set and a verification set, and carrying out image enhancement processing;

building a capsicum disease and insect pest identification network structure by using the constructed convolution network model;

training the convolutional network model according to a training set in the original data set, improving various parameters in the convolutional network model, and performing deep convolution to generate an countermeasure network judgment so as to obtain an optimal network model;

Implanting a trained model into a mobile phone APP, and identifying unclassified pepper plant diseases and insect pests images to obtain an identification result of the images to be identified; and uploading the field photos obtained by the mobile terminal to a cloud database, wherein the photo data can be further used for optimizing the network model.

Further, the pepper disease identification method specifically comprises the following steps:

step one, data set preparation: collecting pictures related to pepper leaves, fruits and rhizomes, carrying out label division on the pictures, dividing a data set into three branches of pepper fruit pictures, blade pictures and rhizome pictures according to the characteristics of the pepper fruits, the blades and the rhizomes, and marking the pepper fruit pictures, the blade pictures and the rhizome pictures as pictures of the same plant if the pepper fruits, the blade pictures and the rhizome pictures are three branches from the same plant of pepper, so as to construct a data set;

step two, data preprocessing and conversion: dividing the whole database into a training set and a verification set through random segmentation, preprocessing images, including data enhancement and conversion operations of downsizing and clipping to 256×256 pixels, normalizing (rescale), image rotation (rotation_range), random horizontal flip (horizontal flip), image displacement (width_shift_range), scaling (zoom_range), smoothing (smoothing);

Thirdly, constructing a network structure of a pepper plant disease and insect pest identification model: combining deep learning and transfer learning to construct a new convolutional network model;

step four, model training: the identification of the disease and insect damage of the capsicum is to obtain the disease and insect damage identification result of fruits, leaves and rhizomes on the basis of three branch structures, train the model by utilizing a training set in a data set, and improve various parameters in a network model; finally obtaining an optimal network model;

step five, image identification: implanting a trained model into a mobile phone APP, designing a mobile application program, and identifying unclassified pepper plant diseases and insect pests images to obtain an identification result of the images to be identified;

step six, enriching the data set: uploading the field photos acquired by the mobile terminal to a cloud database, and periodically downloading a field real photo rich data set from the cloud, so that 100% of the data set trained by the model comes from a field real scene, and the robustness of the training model is improved.

Further, in step one, the data set preparation includes:

collecting pictures related to pepper leaves, fruits and rhizomes, wherein the picture sources comprise: network-opened capsicum disease image library data set, disease pictures provided by the national academy of sciences of agriculture and forestry in Chengdu city and pictures shot by a team in the field; dividing the pictures into bacterial spot labels and health labels according to the guidance of the experts of the national academy of sciences;

Wherein the dataset comprises 9669 pepper leaves and fruit pictures, wherein the healthy and bacterial infected pictures are 6473 and 3196 respectively; 77% of the available images of healthy plants are photographed under real cultivation conditions in the field; the increased complexity of the image under real conditions includes the presence of multiple leaves and other parts of the plant in the picture, unrelated objects, different ground textures and shadow effects.

Further, in the second step, the data preprocessing and converting includes:

during training, the whole database is divided into two data sets, namely a training set and a verification set, wherein 80% of the data sets form the training set and 20% form the verification set by randomly dividing 9669 images; each part is divided into two sub-types of bacteria and health; the image is preprocessed, including data enhancement operations of downsizing and cropping to 256×256 pixel size, and performing normalization, image rotation, random horizontal flipping, image shifting, scaling, and smoothing.

Further, in the third step, the building of the network structure of the pepper plant disease and insect pest identification model includes:

using transfer learning, and combining deep learning and transfer learning to build a network structure of a pepper plant disease and insect pest identification model; the transfer learning firstly keeps the structure of the model convolution layer unchanged, and loads trained weights and parameters into the convolution layer; designing a full connection layer suitable for a new task, replacing the original full connection layer with the new full connection layer, and forming a new convolution network model with the previous convolution layer; the model structure framework starts from the data processing of the bottom layer and processes the data set; to data set partitioning; training the model, and adjusting model parameters according to a training result to obtain an optimal network model;

Loading a convolutional neural network model by using TF-Hub, and integrating a linear classifier on the feature_extralayer and Hub models; adding Dropout layers in the full connection layer, and adding a LeakyReLU activation function to each layer of Dropout, wherein the LeakyReLU is an activation function relatively superior to the ReLU;

regularizing the weight value; none in the Output layer of the Keraslayer layer refers to the size of each batch of samples, does not affect the process, can be changed, has 1280 Output dimension determined by the input dimension, and has param as an input parameter 2257984; none in the Output layer of the Dense layer refers to the sample size, 1280 the Output dimension determined by the input dimension, and param refers to the input parameter 1639680; none in the Output layer of the Dropout layer refers to the sample size, 1280 is the dimension of Output, and the parameter input is 0; none in the Output layer of the LeakyReLU refers to the sample size, 1280 is the dimension of Output, and the parameter input by param is 0; none in the dense_1 layer is the sample size, 512 is the dimension of the output, and the parameter input is 655872; none in the dropout_1 layer is the sample size, 512 is the dimension of the output, and the parameter input parameter is 0; none in the Leakyr_lu_1 layer is the sample size, 512 is the dimension of the output, and the parameter input parameter is 0; none in the dense_2 layer is the sample size, 512 is the dimension of the output, and the parameter input is 1026; the total amount of parameters of the input model was 4554562, wherein the parameters that participated in training were 2296578, and the parameters that did not participated in training were 2257984;

A neural network is built by using a Keras sequential model, a MobilenetV2 model is imported by calling a hub. KerasLayer () method, and a model. Add () method adds a Dropout layer and a LeakyReLU activation function.

Further, in the fourth step, the model training includes:

(1) Configuration: using an Adam optimizer, selecting a classification cross entropy function category_cross sentropy by a loss function, and marking a network evaluation index as accuracy;

(2) Training: generating data batch and training, setting training rounds, performing rounding and dividing operation on the batch size by the total number of samples, taking the obtained result as each round of training batch, and executing a generator in parallel with the model to improve the efficiency; the training mode is to calculate while training, and the total loss is not calculated after training is completed for one round; for a training set, inputting a batch into a model for training, in one round of training, calculating the loss value of the batch under all parameter conditions of the current model immediately after each batch is trained, and determining the loss value of the round by calculating the average value of the loss values of all batches after all batches of the round are finally trained;

(3) Optimizing: after the model frame is designed, parameters of a model Loss Function, an optimizer, an activation Function activation Function, a regularization term regulation term, a normalization Function and a callback Function callback Function are adjusted, and the model is optimized from the Loss Value, the precision Accuracy, the Confidence coefficient Confidence and the fitting goodness R square direction.

Further, the image recognition includes:

designing an APP program at a mobile phone end, and identifying diseases by taking pictures; converting the realized pepper disease identification model into a TFLite lightweight model, embedding the TFLite lightweight model into a mobile phone program APP, and creating an end-to-end android application program; the APP is written in the android Studio by Java language, and an apk format file is exported after the code is successfully operated; after the installation of a user, selecting a local photo or calling a mobile phone camera photographing identification function to detect pepper diseases, clicking the selected photo to select the photographed photo in a mobile phone album, clicking a camera to start photographing to select the photographed photo, wherein the middle photo is the selected or photographed photo, clicking an identification button, and the app gives a prediction result at the lower part; and finally, the pepper picture acquired by the mobile terminal is displayed.

Further, in step six, the enriching data set includes:

uploading the field photos acquired by the mobile terminal to a cloud database, and periodically downloading a field real photo rich data set from the cloud, so that 100% of the data set trained by the model comes from a field real scene, and the robustness of the training model is improved.

Another object of the present invention is to provide a pepper disease identification system comprising:

The data set preparation module is used for collecting pictures related to pepper leaves, fruits and rhizomes, and carrying out label division on the pictures to construct a data set;

the data preprocessing and converting module is used for dividing the whole database into a training set and a verification set through random segmentation and carrying out data enhancement operation processing on the image;

the model network structure building module is used for building a pepper plant disease and insect pest identification model network structure by combining deep learning and transfer learning through transfer learning, and building a new convolution network model;

the model training module is used for training the model by utilizing a training set in the data set, generating an antagonism network according to the depth convolution, selecting an optimal predicted value for the three-branch predicted result, and improving various parameters in the network model to obtain an optimal network model;

the image recognition module is used for loading the trained model into the mobile phone APP to obtain a mobile application program, recognizing unclassified pepper plant diseases and insect pests images, and obtaining recognition results of the images to be recognized; the acquired pictures are uploaded to enrich the existing data sets, and the final state is that 100% of the data sets used for model training come from a real scene, so that the robustness of the model is improved;

The data set enriching module is used for uploading field real pictures and periodically downloading enriched data sets, so that 100% of data trained by the model come from a real scene, and the robustness of the model is improved.

Another object of the present invention is to provide a computer readable storage medium storing a computer program which, when executed by a processor, causes the processor to perform the steps of:

collecting pictures related to pepper leaves, fruits and rhizomes, and carrying out label division on the pictures to construct a data set; dividing the whole database into a training set and a verification set through random segmentation, and carrying out data enhancement operation processing on the image; constructing a network structure of a pepper plant disease and insect pest identification model by combining deep learning and transfer learning through transfer learning, and constructing a new convolution network model; training the model by utilizing a training set in the data set, and improving various parameters in the network model to obtain an optimal network model; loading the trained model into a mobile phone APP to obtain a mobile application program, and identifying unclassified pepper plant diseases and insect pests images to obtain an identification result of the images to be identified; and uploading the acquired pictures to enrich the existing data set, wherein the final state is that 100% of the data set used for model training comes from a real scene, so that the robustness of the model is improved.

By combining all the technical schemes, the invention has the advantages and positive effects that: the invention 1 utilizes a convolutional neural network MobileNet V2 to extract the characteristics of diseases, solves the problems of complex image background, low contrast, different object sizes and the like, and realizes end-to-end image semantic segmentation. 2. And the time and space complexity of the convolution layer is effectively reduced by performing migration learning through Tensorfolw Hub. 3. The TensorFlow Lite is used for lightening the model, and the detection model is transplanted into the mobile phone APP, so that the model is favorable for wide popularization. 4. The most accurate result for judging the three branch prediction results by using the deep convolution generation countermeasure network effectively improves the model prediction result and makes the final prediction more accurate. 5. The mobile terminal identifies the plant diseases and insect pests, reserves the acquired photos and uploads the photos to the cloud database, periodically downloads data, enriches the existing data sets, and finally leads 100% of the data sets trained by the model to come from a real scene, so that the model robustness is enhanced.

The invention provides a capsicum disease identification method, which provides a complete set of flow frame of the capsicum disease and insect pest identification method, and under the TensorFlow frame, a Keras deep learning library is utilized to realize the construction of a capsicum disease detection model, and the processes of data preprocessing, data conversion, model training, parameter adjustment and the like are respectively carried out in the model design, so that the key (disease and insect pest identification) part in the whole flow is optimally realized, and the whole flow is also optimized.

The pepper disease and pest identification method provided by the invention has high portability, and an APP program at the mobile phone end is designed to identify the disease by taking pictures; and converting the realized pepper disease identification model into a TFLite lightweight model, embedding the TFLite lightweight model into a mobile phone program APP, and creating an end-to-end android application program.

The pepper plant diseases and insect pests identification method provided by the invention has very high data centralization capacity, a Dropout layer is added on a full-connection layer, and a LeakyReLU activation function is added after each layer of Dropout, wherein the LeakyReLU is an activation function relatively superior to a ReLU, and besides the advantages that the ReLU is inherited, the operand can be reduced, the problem of gradient disappearance is solved and the degree of overfitting is relieved, as the function gives a non-zero slope to all negative values of input, neurons can still keep a state capable of learning after entering a negative interval, and the parameter alpha value controls the gradient of a negative part linear function. In addition, in order to control the complexity of the model and reduce the overfitting, the invention carries out regularization treatment on the weight.

Drawings

In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings that are needed in the embodiments of the present invention will be briefly described below, and it is obvious that the drawings described below are only some embodiments of the present invention, and other drawings may be obtained according to these drawings without inventive effort for a person skilled in the art.

Fig. 1 is a flowchart of a method for identifying pepper diseases according to an embodiment of the present invention.

Fig. 2 is a block diagram of a pepper disease identification system according to an embodiment of the present invention;

in the figure: 1. a data set preparation module; 2. the data preprocessing and converting module; 3. building a model network structure building module; 4. a model training module; 5. an image recognition module; 6. a data set enriching module.

Fig. 3 is a schematic diagram of preprocessing picture data according to an embodiment of the present invention.

Fig. 3 (a) is an original image provided by an embodiment of the present invention.

Fig. 3 (b) is a schematic view of clipping according to an embodiment of the present invention.

Fig. 3 (c) is a schematic diagram illustrating flipping provided by an embodiment of the present invention.

Fig. 3 (d) is a schematic illustration of smoothing provided by an embodiment of the present invention.

Fig. 4 is a schematic diagram of a model building framework provided by an embodiment of the present invention.

Fig. 5 is a schematic diagram of a system design concept according to an embodiment of the present invention.

Fig. 6 is a schematic structural diagram of a pepper plant disease and insect pest identification model provided by the embodiment of the invention.

Fig. 7 is a schematic diagram of a training process of a pepper plant disease and insect pest identification model provided by an embodiment of the invention.

Fig. 8 is an APP schematic diagram of transplanting the pepper plant disease and insect pest identification model provided by the embodiment of the invention to a mobile phone terminal of an android system.

FIG. 9 is a graph of accuracy and loss values for a training set and a validation set in a comparative laboratory environment of the present invention.

FIG. 10 is a comparison of the present invention against the categorical prediction of a pepper leaf dataset in a laboratory environment.

FIG. 11 is a graph showing the loss value and the accuracy of the pepper leaves in the real environment of the comparative experiment field.

Fig. 12 is a view showing the predicted condition of the pepper leaves in the actual environment of the comparative experiment field of the present invention.

Detailed Description

The present invention will be described in further detail with reference to the following examples in order to make the objects, technical solutions and advantages of the present invention more apparent. It should be understood that the specific embodiments described herein are for purposes of illustration only and are not intended to limit the scope of the invention.

Aiming at the problems existing in the prior art, the invention provides a method, a system, equipment and a terminal for identifying pepper diseases, and the invention is described in detail below with reference to the accompanying drawings.

The invention provides a pepper disease identification method, which comprises the following steps:

collecting related pictures of pepper leaves, fruits and rhizomes, and constructing a raw data set by dividing the characteristics of the fruits, the leaves and the rhizomes into three branches;

randomly dividing the original data set into a training set and a verification set, wherein the proportion is 8:2, performing image enhancement and other operation treatments;

combining deep learning and transfer learning, constructing a new convolution network model, and constructing a pepper plant disease and insect pest identification network structure;

Training the model according to the training set in the data set, improving each parameter in the network model, and performing deep convolution on the accuracy of three-branch identification to generate one branch for resisting the network judgment to obtain the three-branch identification result, so as to finally obtain the optimal network model;

and implanting the trained model into a mobile phone APP, designing a mobile application program, and identifying unclassified pepper pest images to obtain an identification result of the images to be identified. Uploading the field photos obtained by the mobile terminal to a cloud database;

the field real photo enrichment data set is downloaded from the cloud end regularly, and 100% of the data set trained by the model is finally obtained from the field real scene, so that the original data set is enriched into the real scene data set, and the robustness of the training model is improved.

A set of complete pepper plant diseases and insect pests detection flow is formed, and the complete pepper plant diseases and insect pests detection flow comprises the steps of raw data set acquisition, data set processing, plant diseases and insect pests identification network model construction, model training parameter tuning, mobile terminal use transplanting, real picture enrichment of the raw data set collection, model identification accuracy improvement and model robustness enhancement.

Specifically, as shown in fig. 1, the method for identifying pepper diseases provided by the embodiment of the invention comprises the following steps:

S101, data set preparation: collecting pictures related to pepper leaves, fruits and rhizomes, and carrying out label division on the pictures to construct a data set;

s102, data preprocessing and conversion: dividing the whole database into a training set and a verification set through random segmentation, and carrying out data enhancement operation processing on the image;

s103, constructing a network structure of a pepper plant disease and insect pest identification model: combining deep learning and transfer learning by utilizing transfer learning to construct a new convolutional network model;

s104, model training: training the model by utilizing a training set in the data set, and improving various parameters in the network model to obtain an optimal network model;

s105, image recognition: and loading the trained model into a mobile phone APP to obtain a mobile application program, and identifying unclassified pepper pest images to obtain an identification result of the images to be identified.

S106, enriching the data set: the mobile terminal uploads the field real pictures, and enriches the data set by downloading periodically, so that 100% of data trained by the model come from a real scene, and the robustness of the model is improved.

The step S102 of performing data enhancement operation processing on the image includes: the size is reduced and clipped to 256×256 pixels, and data enhancement operations such as normalization (rescale), image rotation (rotation_range), random horizontal flip (horizontal flip), image shift (width_shift_range), scaling (zoom_range), smoothing (smoothing), and the like are performed.

As shown in fig. 2, the pepper disease identification system provided by the embodiment of the invention includes:

the data set preparation module 1 is used for collecting pictures related to pepper leaves, fruits and rhizomes, and carrying out label division on the pictures to construct a data set;

the data preprocessing and converting module 2 is used for dividing the whole database into a training set and a verification set through random segmentation and carrying out data enhancement operation processing on the image;

the model network structure building module 3 is used for building a pepper plant disease and insect pest identification model network structure by combining deep learning and transfer learning through transfer learning, and building a new convolution network model;

the model training module 4 is used for training the model by utilizing a training set in the data set, improving various parameters in the network model and obtaining an optimal network model;

the image recognition module 5 is used for loading the trained model into the mobile phone APP to obtain a mobile application program, and recognizing unclassified pepper plant diseases and insect pests images to obtain a recognition result of the images to be recognized.

And the data set enriching module 6 is used for uploading field real pictures and periodically downloading enriched data sets, so that 100% of data trained by the model come from a real scene, and the robustness of the model is improved.

The technical scheme of the invention is further described below with reference to specific embodiments.

Example 1

The invention discloses a capsicum disease and insect pest identification method based on a convolutional neural network model, which comprises the following steps: (1) preparing a dataset: (2) data preprocessing and conversion; (3) constructing a model network structure; (4) Training the model, namely training the model by utilizing a training set in the data set, and improving various parameters in the network model to obtain an optimal network model; (5) Image recognition, namely loading a trained model into a mobile phone APP to obtain a mobile application program, and recognizing unclassified pepper plant diseases and insect pests images to obtain recognition results of images to be recognized; (6) And enriching the data set, uploading the pictures acquired by the mobile terminal, enriching the existing data set regularly, and increasing the robustness of the model. The invention has higher identification accuracy, characterization capability and convergence rate, high portability and higher interactivity.

The following describes each step of the deep learning-based pepper plant disease and insect pest identification method in detail:

(1) Preparing a data set:

the invention firstly collects the pictures related to the pepper leaves, fruits and rhizomes, and the main picture sources are as follows: the network open data set and the integrated city agricultural and forestry academy of sciences provide, and at the same time, the team takes pictures in the field. According to the guidance of the expert of the national academy of sciences of agriculture and forestry, the invention classifies pictures into bacterial spot labels and health labels. The small data set volume of the present invention may cause problems in that the correlation accuracy after training is not high enough to cope with complex recognition environments, from the order of magnitude of the data set.

The dataset contained 9669 pepper leaf and fruit pictures, of which healthy and bacterial infection were 6473 and 3196, respectively. Data sources: (1) a network-opened pepper disease image library; (2) Disease pictures provided by the adult urban agriculture and forestry science institute (taken at the Wenchang pepper planting base at 2021, 4 to 5 months).

Table 1 shows the information of the dataset, the number of images available for capsicum (health and pathogen infection). The images (77%) of the healthy plants available in table 1 were taken under real cultivation conditions in the field. The increased complexity of the image under real conditions includes the presence of multiple leaves and other parts of the plant in the picture, unrelated objects (e.g., soil), different ground textures, shadow effects, etc.

TABLE 1 information on dataset, number of available images of Capsici fructus (health and pathogen infection)

(2) Data preprocessing and conversion:

during training, the whole database is divided into two data sets, a training set and a verification set, wherein 80% of the data sets form the training set and 20% of the data sets form the verification set by randomly dividing 9669 images. Each part is in turn divided into two sub-types, bacterial and health. First, the present invention performs preprocessing on an image, including data enhancement operations such as downsizing and cropping to 256×256 pixels, normalization (rescale), image rotation (rotation_range), random horizontal flip (horizontal flip), image shift (width_shift_range), scaling (zoom_range), smoothing (smoothing), and the like, as shown in fig. 3. An alternative to training using a gray-scale version of the image is not considered, as previous work has shown that this approach does not improve the final classification performance of the deep learning model. Neither is there any consideration in the process of segmenting pepper plant leaves from the image background, because the neural network has the ability to identify important and unimportant features of a set of images, and to some extent ignore the latter.

(3) Constructing a model network structure:

convolutional neural network model training is very time consuming. The invention combines deep learning and transfer learning by using transfer learning, thereby realizing the improvement of the accuracy of the model and saving the resources. The transfer learning firstly keeps the structure of the model convolution layer unchanged, and then loads trained weights and parameters into the convolution layer. Then designing a full connection layer suitable for a new task, replacing the original full connection layer with the new full connection layer, and forming a new convolution network model with the previous convolution layer, wherein the model structure framework is shown in fig. 4, and processing a data set from the data processing of the bottom layer; to data set partitioning; and training the model, and adjusting model parameters according to a training result to obtain an optimal network model.

The project design thought can be divided into two parts: (1) Model implementation and training, as shown in fig. 5 (a), the design and optimization of the pepper plant diseases and insect pests model is mainly designed; (2) The mobile phone terminal APP program is realized, and as shown in fig. 5 (b), the realized pepper plant diseases and insect pests model is mainly transplanted to An Zhuoduan.

The linear classifier is integrated on feature_extra_layer and Hub models using TF-Hub loading convolutional neural network models. In order to prevent the occurrence of the overfitting phenomenon and improve the generalization capability of the model in the data set, a Dropout layer is added at the full-connection layer, and meanwhile, a LeakyReLU activation function is added after each layer of Dropout, wherein the LeakyReLU is an activation function relatively superior to the ReLU, besides the advantages that the ReLU is inherited, the operand can be reduced, the problem of gradient disappearance is solved and the overfitting degree is relieved, as the function gives a non-zero slope to all negative values of the inputs, the neurons still keep a state which can be learned after entering a negative interval, and the parameter alpha value controls the gradient of a negative part linear function. In addition, in order to control the complexity of the model and reduce the overfitting, the invention carries out regularization treatment on the weight. As shown in fig. 6, which shows the training model structure, the specific structure of the model network layer can be seen, the shape (dimension) shape of the data Output by each layer and the specific number of parameters of each layer are described, the total number of parameters and the number of parameters participating in training and not participating in training are also described, the None in the Output layer (None, 1280) of the KerasLayer layer refers to the size of each batch of samples, the process is not affected, the process can be changed, so that the None is changed, the Output dimension of the next 1280 is determined by the input dimension, and the param refers to the input parameter 2257984; in the Output layer (None, 1280) of the Dense layer, none refers to the sample size, 1280 refers to the Output dimension determined by the input dimension, and param refers to the input parameter 1639680; none refers to the sample size in the Output layer (None, 1280) of the Dropout layer, 1280 is the dimension of the Output, and the parameter input is 0; none refers to the sample size in the Output layer of the LeakyReLU (None, 1280), 1280 is the dimension of the Output, and the parameter input is 0; none in the dense_1 layer (None, 512) is the sample size, 512 is the dimension of the output, and the parameter input is 655872; none in the dropout_1 layer (None, 512) is the sample size, 512 is the dimension of the output, and the parameter input is 0; in the Leakyr_lu_1 layer (None, 512), none is the sample size, 512 is the dimension of the output, and the parameter input parameter is 0; in the dense_2 layer (None, 512), none is the sample size, 512 is the dimension of the output, and the parameter input is 1026; the total amount of parameters entered into the model was 4554562, with 2296578 of parameters that participated in training and 2257984 of parameters that did not participated in training.

A neural network was built using a Sequential model (Sequential) of Keras, the mobiletv 2 model was imported by calling the hub. Keraslayer () method, and the model. Add () method added Dropout layers and the LeakyReLU activation function.

(4) Model training

Model training process:

1) Configuration: using an Adam optimizer, the loss function selects a categorical cross entropy function (categorical cross entropy), labeling the network evaluation index as accuracy.

2) Training: generating data batch, training, setting training rounds, performing rounding and dividing operation on the batch size by the total number of samples, taking the obtained result as each round of training batch, and executing the generator and the model in parallel to improve the efficiency. As can be seen in FIG. 7, the present invention works in a training-while-training manner, rather than computing the total loss after a round of training. Specifically, for the training set, the invention inputs a batch to the model one batch at a time for training, in one round of training, the loss value of the batch under all parameter conditions of the current model is calculated immediately after each batch (batch) is trained, and after all batches of the round are finally trained, the loss value of the round is determined by calculating the average value of all batch loss values. The present invention finds that the loss value of the previous entry model training is significantly higher than the loss value of the subsequent entry because the parameters are updated once each batch of training is completed, so the model accuracy will be higher and higher as the training proceeds until convergence. For the verification set, after one round of training is finished, the model performance is relatively good, so the loss value is naturally smaller.

3) Optimizing: simple optimization and linear stacking cannot construct an excellent model, and after a model frame is designed, parameters such as a model Loss Function (Loss Function), an optimizer (optimizer), an activation Function (activation Function), a regularization term (regularization), a normalization (normalization), a callback Function (callback Function) and the like are adjusted so as to optimize the model from the directions such as Loss Value (Loss Value), precision (Accuracy), confidence, fitting preference (R squared) and the like.

(5) And (3) image identification:

and designing an APP program at the mobile phone end, and identifying diseases by taking pictures. And converting the pepper disease identification model into a TFLite lightweight model, embedding the model into a mobile phone program APP, and creating an end-to-end android application program.

The APP is written in Java language in android Studio, and an apk format file is exported after the code is successfully operated. After the user installs, the method can use: (1) Selecting a local photo, (2) calling a mobile phone camera to take a photo and identify, wherein two functions are used for detecting the pepper diseases, as shown in fig. 8, the simple design effect is achieved, the photo can be selected in a mobile phone album by clicking the selected photo, the camera can be selected to take the photo by clicking the selected photo, the middle photo is the selected or taken photo, the identification button is clicked, and the app can give a prediction result at the lower part;

(6) Enriching a data set:

The invention provides a complete flow frame of a pepper disease and pest identification method, under a TensorFlow frame, a Keras deep learning library is utilized to realize the construction of a pepper disease detection model, and the processes of data preprocessing, data conversion, model training, parameter adjustment and the like are respectively carried out in the model design, so that the key (disease and pest identification) part in the whole flow is optimally realized, and the whole flow is also optimized.

The pepper disease and pest identification method provided by the invention has high portability, and the APP program at the mobile phone end is designed to identify the disease by taking pictures. And converting the realized pepper disease identification model into a TFLite lightweight model, embedding the TFLite lightweight model into a mobile phone program APP, and creating an end-to-end android application program.

Example 2

The pepper plant diseases and insect pests identification method based on deep learning provided by the embodiment of the invention comprises the following steps:

step 1: the method comprises the steps of preparing a data set, firstly collecting pictures related to pepper leaves, fruits and rhizomes, and mainly obtaining the pictures: the network open data set and the integrated city agricultural and forestry academy of sciences provide, and at the same time, the team takes pictures in the field. According to the guidance of the expert of the national academy of sciences of agriculture and forestry, the invention classifies pictures into bacterial spot labels and health labels. The small data set volume of the present invention may cause problems in that the correlation accuracy after training is not high enough to cope with complex recognition environments, from the order of magnitude of the data set.

Step 2: the data set is preprocessed, and during training, the whole database is divided into two data sets, namely a training set and a verification set, wherein 80% of the data sets form the training set and 20% form the verification set by randomly dividing 9669 images. Each part is in turn divided into two sub-types, bacterial and health. First, the present invention performs preprocessing on an image, including data enhancement operations such as downsizing and cropping to 256×256 pixels, normalization (rescale), image rotation (rotation_range), random horizontal flip (horizontal flip), image shift (width_shift_range), scaling (zoom_range), smoothing (smoothing), and the like, as shown in fig. 3. An alternative to training using a gray-scale version of the image is not considered, as previous work has shown that this approach does not improve the final classification performance of the deep learning model. Neither is there any consideration in the process of segmenting pepper plant leaves from the image background, because the neural network has the ability to identify important and unimportant features of a set of images, and to some extent ignore the latter.

Step 3: and constructing a pepper plant disease and insect pest identification network structure, loading a convolutional neural network model by using TF-Hub, and integrating a linear classifier on the feature_extralayer and Hub models. In order to prevent the occurrence of the overfitting phenomenon and improve the generalization capability of the model in the data set, a Dropout layer is added at the full-connection layer, and meanwhile, a LeakyReLU activation function is added after each layer of Dropout, wherein the LeakyReLU is an activation function relatively superior to the ReLU, besides the advantages that the ReLU is inherited, the operand can be reduced, the problem of gradient disappearance is solved and the overfitting degree is relieved, as the function gives a non-zero slope to all negative values of the inputs, the neurons still keep a state which can be learned after entering a negative interval, and the parameter alpha value controls the gradient of a negative part linear function. In addition, in order to control the complexity of the model and reduce the overfitting, the invention carries out regularization treatment on the weight. A neural network was built using a Sequential model (Sequential) of Keras, the mobiletv 2 model was imported by calling the hub. Keraslayer () method, and the model. Add () method added Dropout layers and the LeakyReLU activation function.

Step 4: model training, (1) configuration: using an Adam optimizer, the loss function selects a categorical cross entropy function (categorical cross entropy), labeling the network evaluation index as accuracy.

(2) Training: generating data batch, training, setting training rounds, performing rounding and dividing operation on the batch size by the total number of samples, taking the obtained result as each round of training batch, and executing the generator and the model in parallel to improve the efficiency. As can be seen in FIG. 7, the present invention works in a training-while-training manner, rather than computing the total loss after a round of training. Specifically, for the training set, the invention inputs a batch to the model one batch at a time for training, in one round of training, the loss value of the batch under all parameter conditions of the current model is calculated immediately after each batch (batch) is trained, and after all batches of the round are finally trained, the loss value of the round is determined by calculating the average value of all batch loss values. The present invention finds that the loss value of the previous entry model training is significantly higher than the loss value of the subsequent entry because the parameters are updated once each batch of training is completed, so the model accuracy will be higher and higher as the training proceeds until convergence. For the verification set, after one round of training is finished, the model performance is relatively good, so the loss value is naturally smaller.

(3) Optimizing: simple optimization and linear stacking cannot construct an excellent model, and after a model frame is designed, parameters such as a model Loss Function (Loss Function), an optimizer (optimizer), an activation Function (activation Function), a regularization term (regularization), a normalization (normalization), a callback Function (callback Function) and the like are adjusted so as to optimize the model from the directions such as Loss Value (Loss Value), precision (Accuracy), confidence, fitting preference (R squared) and the like.

Step 5: image recognition, designing a mobile phone terminal APP program, and performing disease recognition by taking pictures. And converting the pepper disease identification model into a TFLite lightweight model, embedding the model into a mobile phone program APP, and creating an end-to-end android application program.

The APP is written in Java language in android Studio, and an apk format file is exported after the code is successfully operated. After the user installs, the method can use: (1) selecting a local photograph; (2) Invoking the mobile phone camera to take a photograph and identify two functions to detect the pepper diseases, wherein the two functions are simple design effects as shown in fig. 8, clicking a selected photograph can select a photographed photograph in a mobile phone album and clicking to start photographing to select a camera to take a photograph, the middle picture is the selected or photographed photograph, clicking an identification button, and app can give a prediction result at the lower part;

And 6, enriching the data set, uploading the field photos obtained by the mobile terminal to a cloud database, periodically downloading the field real photo enriching data set from the cloud, and finally achieving that 100% of the data set for model training comes from a field real scene, thereby improving the robustness of the training model.

For classification problems, the Accuracy Accuracy and the Loss value Loss are used as evaluation indexes of model performance, the data set in the experiment is divided into a training set and a verification set according to the ratio of 8:2, and two experimental modes are adopted, namely, the identification of the capsicum leaf disease in a laboratory environment and the identification of the capsicum leaf disease in a field real environment. The results of the experiments under the two environments are specifically discussed below.

(1) In the laboratory setting, the dataset was divided into training and validation sets, 1982 and 496, respectively, limited to single leaf classification in a homogeneous background. The network model is evaluated through the verification set, the cross entropy loss value of the model gradually decreases under the training of sample data along with the increase of iteration times, the model finally converges to 0.08, and the accuracy reaches saturation when the iteration is performed 49 times. In this case, the accuracy of the validation set gradually fits to the accuracy of the training set, as shown in FIG. 9. When the label of the target value and the output value of the model are obtained, the label and the output value of the model can be compared, and fig. 10 is a prediction situation, and the file directory and the file name of the output picture, the true value category of the output picture, the classification after prediction and the confidence level can be intuitively displayed.

(2) The 4639 pictures of the pepper leaf dataset in the field real environment were tested, with 3711 and 928 pictures of the training set and the verification set in the dataset. When the background of the picture is complex, the higher the complexity of the model is, the more likely the generalization capability of the model to new samples is reduced, at this time, the overfitting phenomenon (overfitting) will easily occur, so that the Dropout layer is added in a stacked layer manner to effectively avoid overfitting, and in addition, the regularization processing is also performed on the weights, but in this case, no regularization is adopted when the loss value is calculated by using the verification set, no node is randomly shielded by Dropout, and the regularization and Dropout layers both make the accuracy lower, so that the accuracy of the obtained verification set is slightly greater than that of the training set, and finally the accuracy after saturation is 98.8%, and the prediction condition is shown in fig. 12.

From the test results, when the image recognition success rate is obviously higher than the success rate obtained by training in the field real environment only under laboratory conditions, the background (shadow, ground, branch and the like) of the image is found to have an influence on the accuracy. Experimental results indicate that image recognition under actual culture conditions is more difficult and complex than image recognition under laboratory conditions, and demonstrate the high importance of image data captured under actual culture conditions for developing systems for automatic detection and diagnosis of plant diseases.

In the above embodiments, it may be implemented in whole or in part by software, hardware, firmware, or any combination thereof. When used in whole or in part, is implemented in the form of a computer program product comprising one or more computer instructions. When loaded or executed on a computer, produces a flow or function in accordance with embodiments of the present invention, in whole or in part. The computer may be a general purpose computer, a special purpose computer, a computer network, or other programmable apparatus. The computer instructions may be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another computer-readable storage medium, for example, the computer instructions may be transmitted from one website, computer, server, or data center to another website, computer, server, or data center by a wired (e.g., coaxial cable, fiber optic, digital Subscriber Line (DSL), or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer readable storage medium may be any available medium that can be accessed by a computer or a data storage device such as a server, data center, etc. that contains an integration of one or more available media. The usable medium may be a magnetic medium (e.g., floppy Disk, hard Disk, magnetic tape), an optical medium (e.g., DVD), or a semiconductor medium (e.g., solid State Disk (SSD)), etc.

The foregoing is merely illustrative of specific embodiments of the present invention, and the scope of the invention is not limited thereto, but any modifications, equivalents, improvements and alternatives falling within the spirit and principles of the present invention will be apparent to those skilled in the art within the scope of the present invention.

Claims

1. The pepper disease identification method is characterized by comprising the following steps of:

implanting a trained model into a mobile phone APP, and identifying unclassified pepper plant diseases and insect pests images to obtain an identification result of the images to be identified; uploading field photos acquired by the mobile terminal to a cloud database, wherein the photo data can be further used for optimizing a network model;

The pepper disease identification method specifically comprises the following steps:

step six, enriching the data set: uploading the field photos obtained by the mobile terminal to a cloud database, and periodically downloading a rich data set of the field real photos from the cloud, wherein 100% of the data set for model training comes from the field real scene.

2. The pepper disease identification method as claimed in claim 1, wherein in the first step, the data set preparation comprises:

wherein the dataset comprises 9669 pepper leaves and fruit pictures, wherein the healthy and bacterial infected pictures are 6473 and 3196 respectively; 77% of the available images of healthy plants were taken under real cultivation conditions in the field; the increased complexity of the image under real conditions includes the presence of multiple leaves and other parts of the plant in the picture, unrelated objects, different ground textures and shadow effects.

3. The pepper disease identification method as claimed in claim 1, wherein in the second step, the data preprocessing and conversion comprises:

4. The pepper disease identification method as in claim 1, wherein in the third step, the constructing the pepper disease identification model network structure comprises:

using transfer learning, and combining deep learning and transfer learning to build a network structure of a pepper plant disease and insect pest identification model; the transfer learning firstly keeps the structure of the model convolution layer unchanged, and loads trained weights and parameters into the convolution layer; designing a full connection layer suitable for a new task, replacing the original full connection layer with the new full connection layer, and forming a new convolution network model with the previous convolution layer; the convolution network model structure framework starts from the data processing of the bottom layer and processes the data set; to data set partitioning; training the model, and adjusting model parameters according to a training result to obtain an optimal network model;

5. The pepper disease identification method as claimed in claim 1, wherein in the fourth step, the model training comprises:

(3) Optimizing: after the model framework is designed, the model is optimized according to the loss value LossValue, the precision Accument, the Confidence coefficient Convergence and the fitting goodness Rsquared of the model, wherein the loss function LossFunction, the optimizer, the activation function, the regularization term, the normalization and the callback function parameters of the model are adjusted.

6. The pepper disease identification method as claimed in claim 1, wherein in the fifth step, the image identification comprises:

7. The pepper disease identification method as in claim 1, wherein in step six, said rich dataset comprises: uploading the field photos obtained by the mobile terminal to a cloud database, and periodically downloading a rich data set of the field real photos from the cloud, wherein 100% of the data set for model training comes from the field real scene.

8. A pepper disease identification system using the pepper disease identification method as claimed in any one of the claims 1-7, characterized in that the pepper disease identification system comprises:

The image recognition module is used for loading the trained model into the mobile phone APP to obtain a mobile application program, recognizing unclassified pepper plant diseases and insect pests images, and obtaining recognition results of the images to be recognized; uploading the acquired pictures to enrich the existing data sets, wherein the final state is that 100% of the data sets used for model training come from a real scene;

9. A computer readable storage medium storing a computer program which, when executed by a processor, causes the processor to perform the pepper disease identification method as claimed in any one of claims 1-7.