CN110245720B - Deep learning-based intelligent citrus pest diagnosis method and system - Google Patents
Deep learning-based intelligent citrus pest diagnosis method and system Download PDFInfo
- Publication number
- CN110245720B CN110245720B CN201910546589.8A CN201910546589A CN110245720B CN 110245720 B CN110245720 B CN 110245720B CN 201910546589 A CN201910546589 A CN 201910546589A CN 110245720 B CN110245720 B CN 110245720B
- Authority
- CN
- China
- Prior art keywords
- citrus
- disease
- diagnosis
- diseases
- densenet
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/24—Classification techniques
- G06F18/241—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/40—Extraction of image or video features
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/60—Type of objects
- G06V20/68—Food, e.g. fruit or vegetables
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02A—TECHNOLOGIES FOR ADAPTATION TO CLIMATE CHANGE
- Y02A90/00—Technologies having an indirect contribution to adaptation to climate change
- Y02A90/10—Information and communication technologies [ICT] supporting adaptation to climate change, e.g. for weather forecasting or climate simulation
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- Data Mining & Analysis (AREA)
- General Physics & Mathematics (AREA)
- Life Sciences & Earth Sciences (AREA)
- Artificial Intelligence (AREA)
- General Engineering & Computer Science (AREA)
- Evolutionary Computation (AREA)
- General Health & Medical Sciences (AREA)
- Molecular Biology (AREA)
- Software Systems (AREA)
- Mathematical Physics (AREA)
- Computing Systems (AREA)
- Health & Medical Sciences (AREA)
- Biomedical Technology (AREA)
- Biophysics (AREA)
- Computational Linguistics (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Evolutionary Biology (AREA)
- Multimedia (AREA)
- Bioinformatics & Computational Biology (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Image Analysis (AREA)
Abstract
The invention discloses an intelligent citrus pest diagnosis method and system based on deep learning, wherein the method comprises the following steps: step 1: establishing an image data set of 6 citrus diseases such as yellow dragon disease, anthracnose, canker, scab, sandskin disease and scab based on expert experience; step 2: expanding a training set and a testing set by using five data enhancement methods; training the simplified DenseNet network by using the enhanced training set and the verification set, and storing the model into a system; and evaluating the performance of the model by using the test set. Step 3: the citrus disease diagnosis system is built on the basis of WeChat applet, a user uses the applet to photograph/upload images through a mobile phone, diagnosis is carried out through the uploaded trained convolution network model, and then an intelligent diagnosis result and a pest control suggestion are returned to the user, so that intelligent diagnosis of citrus pests is realized.
Description
Technical Field
The invention relates to an intelligent citrus pest diagnosis method and system based on deep learning.
Background
Citrus is one of the largest growing fruits in the world. However, the increasingly serious citrus diseases bring about a great economic loss to the vast majority of citrus farmers. With the rapid development of mobile devices, mobile services play an increasingly important role in the daily life of people. How to develop an intelligent citrus disease diagnosis system based on mobile service, build a bridge between orange farmers and experts, and realize that all orange farmers become the experts is a subject worthy of research.
According to incomplete statistics, citrus is planted in 140 countries around the world. Due to factors such as warming climate, forbidden use of highly toxic pesticides, aging of citrus trees, abuse of herbicides and the like, citrus diseases and insect pests are increasingly serious.
Citrus yellow longdisease (Citrus huanglongbing, HLB) is the most devastating worldwide disease in citrus production, which occurs in more than 40 countries in asia, africa, south america and north america. At present, many provinces in China are affected by yellow-dragon bacteria. The citrus yellow dragon disease has long incubation period, no obvious specific diseases in early stage, and is easy to mix with color change caused by the harm of element deficiency and soil pathogens, and misjudgment is often caused. Currently, polymerase Chain Reaction (PCR) technology is commonly used to detect pathogenic bacteria invading citrus to determine if it is infected with citrus yellow-tailed bacteria. However, due to uneven distribution of the citrus yellow-shoot bacteria in the citrus body, false negative is often caused by various factors such as sampling position, DNA extraction and the like.
Anthracnose (Anthracnose) is one of the most common diseases of citrus, and has the characteristics of wide harm and long harm time, and mainly damages leaves, branches, flowers, fruits and stalks. Anthracnose is serious and often causes a great deal of fallen leaves, fallen flowers, fallen fruits, dead branches and rotten fruits of citrus varieties.
Citrus Canker (Canker) is a major challenge in citrus cultivation, and it is a major hazard to citrus leaves, shoots and fruits. The damage of the sapling is particularly serious, which can cause fallen leaves and dead tips to influence the tree vigor; the fruit is damaged by the heavy person, the light person is not storage-resistant with the scab, the rot is easy to occur, the commodity value of the fruit is greatly reduced, the prevention and control cost of fruit farmers is increased, and the economic benefit is damaged.
Citrus scab (Black spot) occurs commonly in most citrus producing areas, mainly damaging the fruit, and symptoms appear most often on near-ripe fruits. The cladosporium cucumerinum mainly infects young fruits, but no obvious symptoms exist in the young fruit stage, and the disease spots appear on the epidermis from the fruit expansion stage to the maturity stage, so that the commodity of the fresh fruits is reduced, even completely lost.
Citrus sandpaper rust (sandpaper) is a fungal disease caused by diaporthecium citri (diaporthecetri), and is mainly harmful to tender leaves, tips and young fruits of citrus, and black brown colloid small particles are generated on the surface of a disease part, so that the surface becomes rough, and the marketability is affected.
Citrus scab (Scabis), is usually caused by Elsinoefawcettii. It is one of the important fungal diseases of citrus, not only endangering young fruit but also the calyx and petals.
The citrus diseases are wide in transmission path, rapid in infection, low in efficiency and low in accuracy, and are identified by naked eyes.
Therefore, aiming at the problems, it is necessary to design an intelligent citrus pest diagnosis method and system based on deep learning.
Disclosure of Invention
The invention aims to solve the technical problem of providing a practical deep learning-based intelligent diagnosis method and system for citrus plant diseases and insect pests, which are available at low temperature, and the intelligent diagnosis method and system for citrus plant diseases and insect pests based on deep learning has high diagnosis efficiency and high accuracy.
The technical proposal of the invention is as follows:
an intelligent citrus pest diagnosis method based on deep learning comprises the following steps:
step 1: establishing an image data set of 6 citrus diseases based on expert experience;
the 6 citrus diseases are as follows: yellow dragon disease, anthracnose, canker, scab, sandy skin disease, and scab;
step 2: training a convolutional network by adopting an image dataset of 6 citrus diseases;
randomly extracting images from the data set to form a training set, a verification set and a test set, wherein the proportion of each category is 6:2:2; the training set and the testing set are expanded by using five data enhancement methods, the simplified DenseNet network is trained by using the enhanced training set and the enhanced verification set, and a model is saved in the system; evaluating the performance of the model by using the test set;
step 3: the citrus disease diagnosis system is built on the basis of WeChat applets, a user uploads images by using the applets, diagnosis is carried out through the uploaded trained convolution network model, and intelligent diagnosis results and pest control suggestions are returned to the user. .
The training set and the test set are enhanced using 5 methods: horizontal flip, vertical flip, horizontal-vertical flip, changing brightness and contrast;
wherein, the formula for increasing brightness and contrast is:
dst＝img1×α+img2×β+γ；
wherein dst is a target image and is a linear combination of original images img1 and img 2; img1 and img2 are two images with the same size, the contrast and brightness of the images are changed by changing the values of alpha, beta and gamma, img1 is an original image, img2 is an image with the same size as the original image, all pixel values are 0, and the parameters for increasing the contrast of the image are as follows: α=1.5, β=3, γ=0, and parameters for increasing the brightness of an image are: α=1, β=2, γ=40; the overfitting problem can be alleviated by expanding the training set and the verification set by a data enhancement method.
Experiments were performed using a simplified DenseNet-201 network; denseNet-201 consists of 4 Dense blocks (DenseBlock), each of which consists of a bottleneck layer (Bottleneck layers) structure of 1×1 convolutions followed by 3×3 convolutions, the detailed structure of DenseNet-201 is composed of BN-ReLU-Conv (1×1) -BN-ReLU-Conv (3×3), where BN stands for batch normalization (Batch Normalization), reLU stands for linear rectification function, conv (1×1) stands for 1×1 convolutions, 5 bottleneck layer structures are deleted in the last DenseBlock, and batch normalization (Batch Normalization) is added, activation functions (activation function), global averaging pooling (global average pooling) and normalized exponential functions (softmax) have proven to be effective in convolutional neural networks. The resulting simplified DenseNet.
Combining the WeChat applet with a convolution network to realize the function of online identification on the smart phone;
uploading the shot citrus disease photo to a citrus disease diagnosis system by a user, and identifying the citrus disease diagnosis system at a server side through an uploaded model; the system sends intelligent diagnosis results of diseases of citrus to the smart phone; the diagnosis result includes: relevant information of diseases (such as symptoms and reasons of the diseases, and the like) and corresponding treatment schemes. Meanwhile, the user can search cases interested in the user; the system also has the expert consultation function and the monitoring point function. The user can accurately locate and set the monitoring points so as to discover diseases as early as possible and process the diseases in time.
An intelligent diagnosis system for citrus plant diseases and insect pests based on deep learning comprises MUC, and in MCU, the intelligent diagnosis for citrus plant diseases and insect pests based on deep learning is realized by adopting the intelligent diagnosis method for citrus plant diseases and insect pests based on deep learning.
The invention adopts the image recognition technology for the recognition of the plant diseases and insect pests, can effectively improve the efficiency and the accuracy and can save manpower and material resources. Recently, deep learning has been greatly successful in the field of image recognition, and can be applied to the fields of medical auxiliary diagnosis, agriculture, grain production, and the like. Compared with the traditional image recognition algorithm, the depth model has strong learning capability and efficient feature expression capability, and has the more important advantage of extracting information layer by layer from pixel-level original data to abstract semantic concepts, so that the depth model has outstanding advantages in the aspect of extracting global features and contextual information of images.
Early convolutional neural networks such as AlexNet and VGGNet discussed the relationship of network depth to classification performance, and the results indicated that the deeper the network, the better the performance in a structurally similar situation.
Unlike AlexNet and VGGNet, googLeNet deepens the network, introduces an indication structure, replaces the traditional simple convolution layer and activation function operation, and greatly reduces parameters.
As network depth increases, the network will theoretically have better performance. In practice, gradient disappearance and gradient explosion may occur, resulting in model non-convergence, reduced accuracy, and increased training errors and testing errors. To address these problems, resNet designed a residual block to train deeper networks.
DenseNet is a tightly connected convolutional neural network. The network connects each layer with all other layers, each layer having as input a feature map of the previous layer. This reduces the problem of gradient extinction and enhances the propagation of the features. Meanwhile, characteristic reuse is encouraged, and the number of parameters is greatly reduced.
In addition, the invention further simplifies and improves the structure of DenseNet, thereby obtaining more excellent model.
The beneficial effects are that:
the invention provides an intelligent mobile diagnosis system and method for citrus diseases, which train a citrus disease identification model based on DenseNet and realize intelligent diagnosis of citrus diseases by using a mobile WeChat applet. First, a citrus disease dataset was established, comprising six types of common citrus diseases. And secondly, data enhancement is utilized to simplify DenseNet so as to reduce overfitting and improve generalization capability of the model. Finally, the reduced network is trimmed using the dataset, and the model is saved in the intelligent system. The user may upload a photograph of the suspected citrus disease case to the system through a WeChat applet. And then feeding back the intelligent diagnosis result. Meanwhile, the system can provide positioning service, and a user can conveniently set monitoring points for further tracking. The system also allows the user to communicate directly online with the expert. The main contributions of the invention are as follows:
1. with the help of an expert, 6 image datasets of citrus diseases were constructed.
2. The intelligent diagnosis of citrus diseases is realized by utilizing the WeChat applet, and the gap between citrus growers and specialists is made up.
In summary, the invention is easy to implement and popularize, has high diagnosis accuracy and high diagnosis speed.
Drawings
FIG. 1 is a block diagram of an intelligent mobile diagnosis system for citrus diseases;
FIG. 2 is a schematic diagram of a 5-layer Dense block structure;
FIG. 3 is a schematic diagram of an intelligent mobile citrus disease diagnostic system;
FIG. 4 is a simplified DenseNet block diagram;
FIG. 5 is a schematic representation of the effect of cutting a compact block on the classification result;
FIG. 6 is a comparative schematic of time consumption;
FIG. 7 is a comparative schematic diagram of other evaluation criteria;
FIG. 8 is a flow chart of user interaction with a model;
fig. 9 is a detailed flowchart of front-end and back-end information interaction.
Detailed Description
The invention will be described in further detail with reference to the accompanying drawings and specific examples:
example 1:
the invention uses the WeChat applet to set the system on the mobile device, so that the user can use the system anytime and anywhere. Fig. 1 shows the structure of an intelligent mobile diagnosis system for citrus diseases. The system mainly realizes the following main functions:
1. the user can take a citrus disease photo through the system and upload it directly to the server. The system not only can feed back the disease type of the citrus, but also can display relevant information of the disease, such as symptoms, reasons and the like of the disease. In addition, the system also provides corresponding treatment schemes for the diseases, and helps to prevent and treat the diseases. At the same time, the user can search for cases of citrus diseases of interest to himself. By clicking on the expert's head, diagnostic information may be sent to the corresponding expert.
2. In order to facilitate the monitoring of citrus by a user, the system provides the function of setting monitoring points. The user can accurately locate and set the monitoring points so as to discover diseases as early as possible and process the diseases in time.
3. In order to reduce the gap between citrus growers and specialists, the system is provided with a specialist consultation function, and a user can send difficult and complicated symptoms to the specialists in the form of pictures and characters for specialist diagnosis.
The main method comprises the following steps:
step 1: establishing an image data set of 6 citrus diseases such as yellow dragon disease, anthracnose, canker, scab, sandskin disease and scab based on expert experience; step 2: expanding a training set and a testing set by using five data enhancement methods; training the simplified DenseNet network by using the enhanced training set and the verification set, and storing the model into a system; and evaluating the performance of the model by using the test set. Step 3: the citrus diagnosis system is built on the basis of the WeChat applet, the user uses the applet to upload images, diagnosis is carried out through the uploaded model, and an intelligent diagnosis result is returned. Based on the convolution network after training, inputting a new image to realize intelligent diagnosis of the citrus diseases and insect pests.
Establishment of a data set:
due to the lack of the citrus disease data set, related images are collected through a network, local data, expert access and the like, and the citrus disease data set is constructed. Table 1 shows the number and ratio of citrus disease images.
| Disease type | Quantity of | Proportion of |
| Yellow Dragon disease | 496 | 23.65％ |
| Anthracnose disease | 201 | 9.59％ |
| Ulcer disease | 746 | 35.57％ |
| Scab of stars | 279 | 13.30％ |
| Sandy skin disease | 198 | 9.44％ |
| Scab disease of the sore | 177 | 8.44％ |
TABLE 1 number and ratio of citrus disease images
The dataset contains six common citrus diseases: yellow dragon disease, anthracnose, canker, scab, sandskin disease, and scab. Each class includes images of the respective symptomatic fruit and leaf. The number and ratio distribution of each type of sample are shown in table 1, and the sample images of the experimental data set are shown in table 2.
TABLE 2 sample image
The images are randomly extracted from the data set to form a training set, a verification set and a test set, and the proportion of each category is 6:2:2. And training the network by using the training set and the verification set, and evaluating the performance of the model by using the test set. Because of the smaller data set, the training network may be overfitted. To avoid this problem, the data set may be processed, new images generated and the data set added. Thus, 5 methods are used to enhance the training set and the test set: horizontal flip, vertical flip, horizontal-vertical flip, changing brightness and contrast. Thus, the sizes of the training set and the test set are increased by 5 times. The data enhancement methods and examples are shown in tables 3 and 4. The number and ratio distribution of each category after the data increase is shown in table 5.
In the present embodiment, the formula 1 is used to increase the brightness and contrast:
dst＝img1×α+img2×β+γ (1)
where dst is the target image, corresponding to a linear combination of img1 and img 2. img1 and img2 are two images of the same size. The contrast and brightness of the image are changed by changing the values of α, β, γ. In the experiment, img1 is an original image, img2 is an image of the same size as the original image, and all pixel values are 0. The contrast of the image was increased in the experiment, and α=1.5, β=3, γ=0 was selected. In the experiment, the brightness of the image was increased, and α=1, β=2, γ=40 was selected.
TABLE 3 data enhancement method
TABLE 4 enhanced citrus disease data legend
| Disease type | Quantity of | Proportion of |
| Yellow Dragon disease | 2501 | 23.65％ |
| Anthracnose disease | 1016 | 9.61％ |
| Ulcer disease | 3761 | 35.56％ |
| Scab of stars | 1409 | 13.32％ |
| Sandy skin disease | 998 | 9.44％ |
| Scab disease of the sore | 892 | 8.43％ |
TABLE 5 number and ratio of images after citrus disease data enhancement
Introduction to convolutional neural networks:
with the development of convolutional neural networks, the number of layers of the network is continuously increased. But the information in the network training process may gradually disappear after repeated convolution. To solve this problem, denseNet has designed a Dense block (DenseBlock) structure [ see: H.Gao, L.Zhuang, M.D.V.Laurens. "Densely Connected Convolutional Networks," 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp.2261-2269, 2017]. To maximize the information gain of all network layers, any two layers in the network are directly connected so that each layer can accept as input the characteristics of all previous layers. The structure of the 5-layer Denseblock is shown in FIG. 2.
The input of the model is x 0 . Denseblock consists of L layers, each layer having a nonlinear transformation function H l (. Cndot.) wherein l represents the number of layers and the output of the first layer is denoted as x l . The relationship between layer l-1 and layer l is shown in equation 2:
x l ＝H l ([x 0 ，x 1 ，...，x l-1 ]) (2)
wherein [ x ] 0 ，x 1 ，...，x l-1 ]Representing that the Dense Block connects the output feature map from layer 0 to layer l-1.
DenseNet consists of multiple Denseblocks. DenseNet is connected by a transition layer every two Dense blocks. The core idea is to establish the connection between different layers, make full use of the characteristics, and further reduce the gradient disappearance. The invention further simplifies the DenseNet structure, deletes a certain number of network layers, reduces the time cost and improves the performance.
Convolutional neural network based on smart phone
A. System architecture
The intelligent mobile diagnosis system framework of citrus diseases based on DenseNet is shown in figure 3. Firstly, labeling a data set with the help of an expert, then training the simplified DenseNet by using the data set, and storing a model. And finally, uploading the information to a diagnosis system of the cloud server. The user collects the citrus disease picture through a WeChat Applet (i.e. WeChat Applet) on the mobile device and uploads the citrus disease picture to the system. The system recognizes the uploaded image through the trained model and returns the diagnosis and treatment advice to the user.
B. Simplified system
Because the data set is smaller, the original data set is enhanced first. And then fine tuning the DenseNet by utilizing the data set with enhanced data. To reduce the likelihood of overfitting, the number of layers in the network is suitably reduced.
Fig. 4 is a framework of the DenseNet model training process. Experiments were performed using a DenseNet-201 network. DenseNet-201 is composed of 4 Dense blocks (Dense blocks), wherein each Dense block is composed of a bottleneck layer (Bottleneck layers) structure, and the bottleneck layer structure is a 1×1 convolution layer followed by a 3×3 convolution, which can reduce the number of input feature maps, thereby improving the calculation efficiency. Whereas the detailed structure of DenseNet-201's Dense block consists of BN-ReLU-Conv (1×1) -BN-ReLU-Conv (3×3), where BN stands for batch normalization (Batch Normalization), reLU stands for linear rectification function and Conv (1×1) stands for a 1×1 convolutional layer. In the last Dense Block 5 bottleneck layer structures were deleted and Batch Normalization, (activation function) activation function, (global average pooling) global average pooling and (normalized exponential function) softmax layers were added, which have proven to be effective for convolutional neural networks for the prior art. The resulting simplified DenseNet is formed. Therefore, the number of network layers is reduced, the problem of over fitting can be relieved, and the classification accuracy is improved. . The specific network training method comprises the following steps: firstly, training a DenseNet-201 network by using a data set, reloading a stored model, deleting the model by using the network, and finally, further fine-tuning by using the data set, and then, testing images of a test set to obtain a classification result. The results show that the classification accuracy of the test set is improved and the time consumption is less.
Experimental results and analysis
A. Experimental environment
Experiments were performed using Intel (R) Core (TM) i7-7800X CPU@3.50GHz,64.00GB RAM and 1 Nvidia GeForce GTX 1080 Tigpu. To avoid repetitive development work, deep learning has a large number of frameworks. In the present invention, a Keras framework is applied.
The network was trained using random gradient descent with a momentum selection of 0.9. For the dataset, batch size was 8 and 50epochs. The initial learning rate was 0.001 and multiplied by 0.94 every second round. The initial weight is the weight of DenseNet on ImageNet.
B. Professional comparison and performance estimation
Comparative experiments were performed on DenseNet201, inceptionResNetV2, inceptionV3, resNet50 and simplified DenseNet201 using the original citrus disease dataset and the enhanced dataset. The accuracy of the different networks to the test set is shown in table 6.
The results show that the classification accuracy of the data-enhanced citrus disease data set is improved in different networks (+0.75%, +1.14%, +0.25%, +2%, +1%). The simplified classification accuracy of DenseNet201 is higher than the original classification accuracy of DenseNet201 by +1% and +0.26%. The classification accuracy is 88.53% at the highest, and a simplified DenseNet201 data enhancement set is adopted. This is because data expansion can alleviate the problem of limited data sets. Simplifying DenseNet can alleviate the problem of data set networks being overly complex, alleviating overfitting. By properly simplifying the network and enhancing the data set, the probability of overfitting can be reduced.
The influence of deleting different bottleneck layers on classification accuracy is studied through experiments. The experimental results are shown in FIG. 5. In the experiments, the effect of removing 1, 2, 3, 4, 5, 6 bottleneck layers was tested, respectively. Experimental results show that 5 bottleneck layers (88.78%) are removed, and the classification accuracy is highest. This is because properly reducing the number of network layers can alleviate the overfitting phenomenon and improve the classification accuracy.
The present invention also uses a simplified DenseNet201 network to make statistics on the classification results for each class, as shown in Table 7. It can be seen that the classification accuracy of yellow dragon disease and canker is higher, while that of anthracnose is lower. This is because some characteristics of citrus diseases are similar, increasing the difficulty in identifying citrus diseases. At the same time, different environments and devices can also affect the classification results.
In mobile services, the time consumption and real-time nature of the diagnostic system is a very important issue. The time consumption of different dense blocks of the original data set and the enhanced data set is compared in fig. 6.
The invention is also evaluated by adopting a Recall, F1-score and MC evaluation standard. Training the original DenseNet by using the original data set and the data set after the data expansion. Training the simplified DenseNet network by using the original data set and the data set after the data expansion.
As can be seen from fig. 7, all three evaluation values are in an ascending trend, which indicates that the proposed method is superior to other methods. The method has higher classification precision for citrus disease data sets in China.
TABLE 6 comparison of accuracy of different network classifications
TABLE 7 simplified classification accuracy of DenseNet201 for each class
Basic idea of the system:
a user uploads the citrus pictures in the album by using the smart phone or directly photographs the citrus affected by the diseases and insect pests, and the small program can intelligently diagnose the disease types of the citrus according to the pictures and provide a prevention and treatment scheme.
The complete citrus disease identification technology mainly comprises the following four steps:
(1) Training a disease identification model;
(2) Front and back end communication, uploading the image;
(3) Calling a model to identify a disease type;
(4) Returning the identification result to the user through https protocol;
the technical difficulties of the steps (1) and (3) are explained in detail in the foregoing, and the second step and the fourth step can be classified into information transmission, and the specific transmission flow is shown in fig. 8;
the intelligent detection WeChat applet for citrus diseases and insect pests based on deep learning not only can diagnose citrus diseases in real time, but also can collect citrus photos shot or uploaded by users, extract longitude and latitude information in the photos, and establish a disease and insect pest detection early warning system by analyzing the longitude and latitude and the uploading frequency of the users; meanwhile, the collected pictures are sorted into a citrus disease database, the collection and construction of a citrus disease and insect pest data set are carried out, and a citrus disease image database with more abundant types and larger specifications is provided for citrus disease and insect pest experts and deep learning. In addition, the system can help fruit farmers to locate disease occurrence places, know the disease and pest conditions of the orange trees in time, and provide a treatment scheme for fruit tree areas affected by the disease and pest, so that accurate control of fruit tree diseases is realized, excessive fertilization and pesticide application are avoided, and the edible safety of fruits is protected.
Front-end and back-end linking techniques
The client is used for connecting with the server, sending a request, receiving and analyzing data. When each WeChat applet is started, the WeChat applet is not necessarily connected with a server, and a part of data can be locally and temporarily stored and processed. However, because the small program occupies small space, the operation of the related database of the client needs to send a request to the server for processing, so that the server returns data to the client after executing corresponding operation according to the instruction, and the client analyzes the data again. The structure of client interaction with the background is very simple, first a wx.uploadfile function, which is the request-to-send entry of the client file, for creating connection and encapsulation transfer commands. The command is transferred by entering the current file address and the interface address. When receiving the data returned by the server, the server responds to the receiving request and then receives the corresponding data. The next is a wx.request interface call function for requesting the originating HTTPS network. Taking the "image recognition" function as an example, a detailed flowchart of front-end and back-end information interaction is shown in fig. 9.
Conclusion:
the invention applies deep learning to identification of citrus diseases. And establishing a citrus disease identification model by using the dense connection network. Meanwhile, the automatic identification of citrus diseases is realized by utilizing the WeChat applet, and professional disease control suggestions are provided for fruit growers.
In summary, the invention provides a deep learning-based intelligent citrus pest diagnosis method and system, and the method is characterized in that:
1. the communication bridge between orange farmers and specialists is opened through the micro-communication applet, so that the use is convenient;
the system is realized by adopting a WeChat Applet on the mobile device, and a user can upload images and receive diagnosis results and pest control suggestions.
2. The invention establishes an image data set of 6 citrus diseases with the help of experts, and realizes an intelligent diagnosis system of the citrus diseases by simplifying a densely connected convolution network (DenseNet). Experimental results show that the identification accuracy of the method on citrus diseases exceeds 88%. By simplifying the DenseNet structure, the prediction time consumption is reduced.
Claims (2)
1. An intelligent citrus pest diagnosis method based on deep learning is characterized by comprising the following steps:
step 1: establishing an image data set of 6 citrus diseases based on expert experience;
the 6 citrus diseases are as follows: yellow dragon disease, anthracnose, canker, scab, sandy skin disease, and scab;
step 2: training a convolutional network by adopting an image dataset of 6 citrus diseases;
randomly extracting images from the data set to form a training set, a verification set and a test set, wherein the proportion of each category is 6:2:2; the training set and the testing set are expanded by using five data enhancement methods, the simplified DenseNet network is trained by using the enhanced training set and the enhanced verification set, and a model is saved in the system; evaluating the performance of the model by using the test set;
step 3: the citrus disease diagnosis system is built on the basis of WeChat applets, a user uploads images by using the applets, diagnosis is carried out by the uploaded trained convolution network model, and an intelligent diagnosis result and pest control suggestions are returned to the user; the training set and the test set are enhanced using 5 methods, horizontal flip, vertical flip, horizontal-vertical flip, increasing brightness and contrast;
wherein, the formula for increasing brightness and contrast is:
dst＝img1×α+img2×β+γ；
wherein dst is a target image and is a linear combination of original images img1 and img 2; img1 and img2 are two images with the same size, the contrast and brightness of the images are changed by changing the values of alpha, beta and gamma, img1 is an original image, img2 is an image with the same size as the original image, all pixel values are 0, and the parameters for increasing the contrast of the image are as follows: α=1.5, β=3, γ=0; the parameters for increasing the brightness of the image are: α=1, β=2, γ=40; the training set and the verification set are expanded through a data enhancement method, so that the problem of overfitting can be relieved;
experiments were performed using a simplified DenseNet-201 network; denseNet-201 consists of 4 Dense blocks (DenseBlock), wherein each DenseBlock consists of a bottleneck layer (Bottleneck layers) structure, which is a 1×1 convolution layer followed by a 3×3 convolution, and the detailed structure of DenseNet-201 consists of BN-ReLU-Conv (1×1) -BN-ReLU-Conv (3×3), wherein BN represents a batch normalization (Batch Normalization), reLU represents a linear rectification function, conv (1×1) represents a 1×1 convolution layer, 5 bottleneck layer structures are deleted in the last DenseBlock, and a batch normalization (Batch Normalization), an activation function (activation function), global averaging pooling (global average pooling) and a normalization index function (softmax) have been demonstrated to be effective for convolutional neural networks, and finally a simplified DenseNet is formed; combining the WeChat applet with a convolution network to realize the citrus pest and disease damage on-line identification function on the smart phone;
uploading the shot citrus disease photo to a citrus disease diagnosis system by a user, and identifying the citrus disease diagnosis system at a server side through an uploaded model; the system feeds back the intelligent diagnosis result of the citrus diseases to the smart phone; the diagnosis result includes: disease related information and corresponding treatment regimen.
2. The intelligent diagnosis system for the citrus plant diseases and insect pests based on the deep learning is characterized by comprising an MCU, wherein the intelligent diagnosis for the citrus plant diseases and insect pests is realized by adopting the intelligent diagnosis method for the citrus plant diseases and insect pests based on the deep learning in the MCU.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910546589.8A CN110245720B (en) | 2019-06-22 | 2019-06-22 | Deep learning-based intelligent citrus pest diagnosis method and system |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910546589.8A CN110245720B (en) | 2019-06-22 | 2019-06-22 | Deep learning-based intelligent citrus pest diagnosis method and system |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN110245720A CN110245720A (en) | 2019-09-17 |
| CN110245720B true CN110245720B (en) | 2023-05-26 |
Family
ID=67888809
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201910546589.8A Active CN110245720B (en) | 2019-06-22 | 2019-06-22 | Deep learning-based intelligent citrus pest diagnosis method and system |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN110245720B (en) |
Families Citing this family (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111027436A (en) * | 2019-12-03 | 2020-04-17 | 吉林大学 | Northeast black fungus disease and pest image recognition system based on deep learning |
| CN111046964B (en) * | 2019-12-18 | 2021-01-26 | 电子科技大学 | Convolutional neural network-based human and vehicle infrared thermal image identification method |
| CN111761409A (en) * | 2020-07-09 | 2020-10-13 | 内蒙古工业大学 | Multi-sensor numerical control machine tool cutter wear monitoring method based on deep learning |
| CN111899234A (en) * | 2020-07-21 | 2020-11-06 | 仲恺农业工程学院 | Citrus disease and insect pest epidemic situation intelligent pushing system based on big data mining |
| CN112164041B (en) * | 2020-09-18 | 2023-05-12 | 南昌航空大学 | Automatic diagnosis and treatment system and method for yellow dragon disease based on multi-scale deep neural network |
| CN112949796A (en) * | 2021-03-23 | 2021-06-11 | 青岛农业大学 | Hybrid pest and disease identification device and method based on deep learning |
| CN113948220B (en) * | 2021-10-29 | 2022-12-09 | 广东技术师范大学 | Anthrax pathogen infection stage detection method with pre-analysis capability |
Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107576618A (en) * | 2017-07-20 | 2018-01-12 | 华南理工大学 | Pyricularia Oryzae detection method and system based on depth convolutional neural networks |
| CN108875787A (en) * | 2018-05-23 | 2018-11-23 | 北京市商汤科技开发有限公司 | A kind of image-recognizing method and device, computer equipment and storage medium |
| CN108921814A (en) * | 2018-05-16 | 2018-11-30 | 华南农业大学 | A kind of Citrus Huanglongbing pathogen on-line quick detection system and method based on deep learning |
Family Cites Families (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US10349584B2 (en) * | 2014-11-24 | 2019-07-16 | Prospera Technologies, Ltd. | System and method for plant monitoring |
| CN106250899B (en) * | 2016-07-29 | 2019-03-22 | 华东交通大学 | A kind of orange disease and insect pests monitoring and pre-alarming method based on distributed compression perception WSN |
| US11049018B2 (en) * | 2017-06-23 | 2021-06-29 | Nvidia Corporation | Transforming convolutional neural networks for visual sequence learning |
| US10747999B2 (en) * | 2017-10-18 | 2020-08-18 | The Trustees Of Columbia University In The City Of New York | Methods and systems for pattern characteristic detection |
| CN107798356A (en) * | 2017-11-24 | 2018-03-13 | 郑州大学西亚斯国际学院 | Crop leaf disease recognition method based on depth convolutional neural networks |
| CN109344883A (en) * | 2018-09-13 | 2019-02-15 | 西京学院 | Fruit tree diseases and pests recognition methods under a kind of complex background based on empty convolution |
-
2019
- 2019-06-22 CN CN201910546589.8A patent/CN110245720B/en active Active
Patent Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107576618A (en) * | 2017-07-20 | 2018-01-12 | 华南理工大学 | Pyricularia Oryzae detection method and system based on depth convolutional neural networks |
| CN108921814A (en) * | 2018-05-16 | 2018-11-30 | 华南农业大学 | A kind of Citrus Huanglongbing pathogen on-line quick detection system and method based on deep learning |
| CN108875787A (en) * | 2018-05-23 | 2018-11-23 | 北京市商汤科技开发有限公司 | A kind of image-recognizing method and device, computer equipment and storage medium |
Non-Patent Citations (2)
| Title |
|---|
| 密集连接卷积神经网络:让人工智能拥有更强大脑;黄高;上海信息化(第10期);全文 * |
| 深度学习在图像识别中的应用研究综述;郑远攀;李广阳;李晔;;计算机工程与应用(12);全文 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN110245720A (en) | 2019-09-17 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN110245720B (en) | Deep learning-based intelligent citrus pest diagnosis method and system | |
| CN112308158B (en) | Multi-source field self-adaptive model and method based on partial feature alignment | |
| Pan et al. | A smart mobile diagnosis system for citrus diseases based on densely connected convolutional networks | |
| Khan et al. | Deep learning for apple diseases: classification and identification | |
| CN109344883A (en) | Fruit tree diseases and pests recognition methods under a kind of complex background based on empty convolution | |
| CN112733744B (en) | Camouflage object detection model based on edge cooperative supervision and multi-level constraint | |
| CN111179216B (en) | Crop disease identification method based on image processing and convolutional neural network | |
| CN103020265B (en) | The method and system of image retrieval | |
| CN111400536B (en) | Low-cost tomato leaf disease identification method based on lightweight deep neural network | |
| Wang et al. | Mobile smart device-based vegetable disease and insect pest recognition method | |
| CN107563389A (en) | A kind of corps diseases recognition methods based on deep learning | |
| CN112434662B (en) | Tea leaf scab automatic identification algorithm based on multi-scale convolutional neural network | |
| CN115331104A (en) | Crop planting information extraction method based on convolutional neural network | |
| Astonkar et al. | Detection and analysis of plant diseases using image processing | |
| CN115909052A (en) | Hyperspectral remote sensing image classification method based on hybrid convolutional neural network | |
| Jain et al. | Detection of Guava Fruit Disease through a Unified Deep Learning Approach for Multi-classification | |
| Xiao et al. | Citrus greening disease recognition algorithm based on classification network using TRL-GAN | |
| CN112465038A (en) | Method and system for identifying disease and insect pest types of fruit trees | |
| Ibrahim et al. | Palm leaf nutrient deficiency detection using convolutional neural network (CNN) | |
| CN114596274A (en) | Natural background citrus greening disease detection method based on improved Cascade RCNN network | |
| Kamble | Plant disease detector | |
| CN112597907A (en) | Citrus red spider insect pest identification method based on deep learning | |
| CN116977863A (en) | Tomato plant disease and pest detection method and detection system | |
| CN116721060A (en) | Method for detecting red palm weevil beetles based on attention mechanism | |
| CN111523503A (en) | Apple target detection method based on improved SSD algorithm |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |