CN114898233B

CN114898233B - Method and related device for intelligently positioning diaphorina citri in citrus orchard

Info

Publication number: CN114898233B
Application number: CN202210485889.1A
Authority: CN
Inventors: 邓铁军; 刘丽辉; 刘吉敏; 蒋卓恩; 韦丽宋
Original assignee: Institute Of Plant Protection Guangxi Academy Of Agricultural Sciences
Current assignee: Institute Of Plant Protection Guangxi Academy Of Agricultural Sciences
Priority date: 2022-05-06
Filing date: 2022-05-06
Publication date: 2024-09-06
Anticipated expiration: 2042-05-06
Also published as: CN114898233A

Abstract

The application discloses an intelligent positioning method and a related device for diaphorina citri in a citrus orchard, which are used for improving the positioning accuracy of diaphorina citri. The method comprises the following steps: shooting a target citrus tree by using an unmanned aerial vehicle to obtain a citrus tip image group; selecting a citrus top image from the citrus top image group, and inputting the citrus top image into a target convolutional neural network model with training completed; performing feature extraction of different depths on the citrus tip image through the first, second, third and fourth feature extraction layers to generate a first depth feature set; fusing the first depth feature set through a residual layer to generate a first feature residual; classifying and calculating the first characteristic residual errors through a global average pooling layer and a softmax layer to generate a first probability value set of citrus fruit tip images belonging to citrus psyllid fruit tips; generating at least two first probability value sets corresponding to the citrus tip image group according to the method; generating a diaphorina citri alert according to the first set of probability values and the positioning information.

Description

Method and related device for intelligently positioning diaphorina citri in citrus orchard

Technical Field

The embodiment of the application relates to the field of citrus disease detection, in particular to a method and a related device for intelligently positioning diaphorina citri in a citrus orchard.

Background

With the continuous development of agriculture, more and more areas begin to plant some agricultural products which have high nutritional value and are concerned by consumers, and citrus is one of fruits loving for consumers.

Citrus, a plant of the genus citrus of the family rue. It has good heat and moisture resistance, and has better cold resistance than that of fructus Citri Grandis, lime and sweet orange. Diaphorina citri is the main pest in the young shoot stage of citrus, and is also the transmission medium of citrus yellow dragon disease. The adults lay eggs on the young shoots of the hosts, and after the nymphs are hatched, tender shoot juice is sucked until the adults emerge. The damaged young shoots of the host can be withered, deformed and the like. The psyllium can also secrete white honeydew and adhere to branches and leaves, and can cause the occurrence of soot disease.

Accurate and efficient diaphorina citri positioning detection is an important precondition for preventing and controlling citrus yellow dragon disease, and is also a key link for realizing accurate agriculture. Aiming at the problems of the field manual detection method, a great deal of research work has been done by a plurality of scholars at home and abroad in the field of crop pest identification based on convolutional neural network technology. However, due to the small size of the diaphorina citri, the morphology changes in the growth process, so that the characteristics of the diaphorina citri can be submerged in the background of the shot image in the process of training of the convolutional neural network, which is called as a multi-scale problem, the accuracy of detection is reduced when the convolutional neural network model detects the diaphorina citri infected tree, and the positioning accuracy of the diaphorina citri is further reduced.

Disclosure of Invention

The application provides a method for intelligently positioning diaphorina citri in a citrus orchard, which is characterized by comprising the following steps:

The method comprises the steps that a target citrus tree is shot in multiple directions through an unmanned aerial vehicle, a citrus tip image group is obtained, the citrus tip image group comprises at least one shot image of the target citrus tree, and the citrus tip image group comprises positioning information of the target citrus tree;

selecting a citrus tip image from the citrus tip image group;

inputting citrus tip images into a target convolutional neural network model with training completed, wherein the target convolutional neural network model comprises a first feature extraction layer, a second feature extraction layer, a third feature extraction layer, a fourth feature extraction layer, a residual error layer, a global average pooling layer and a softmax layer, the first feature extraction layer is formed by serially arranging convolution layers, normalization layers, convolution layers and normalization layers, and then is connected with a group of convolution layers, normalization layers and a maximum pooling layer in parallel, and the second feature extraction layer, the third feature extraction layer and the fourth feature extraction layer are identical to the first feature extraction layer;

performing feature extraction of different depths on the citrus tip image through the first feature extraction layer, the second feature extraction layer, the third feature extraction layer and the fourth feature extraction layer to generate a first depth feature set;

Fusing the first depth feature set through a residual layer to generate a first feature residual, wherein the first feature residual comprises a diaphorina citri feature residual or a affected tip residual;

classifying and calculating the first characteristic residual errors through a global average pooling layer and a softmax layer to generate a first probability value set of citrus fruit tip images belonging to citrus psyllid fruit tips;

generating at least two first probability value sets corresponding to the citrus tip image group according to the method;

And when probability values larger than a preset threshold exist in the at least two first probability value sets, generating a diaphorina citri alarm according to the positioning information.

Optionally, prior to acquiring the set of images of the citrus tip, the method further comprises:

Acquiring a citrus tip sample set, wherein the citrus tip sample set comprises at least two training samples of citrus tip, and the training samples are images marked with real infection conditions and target estimation thresholds of citrus psyllids;

An initial convolutional neural network model is built, the initial convolutional neural network model comprises a first feature extraction layer, a second feature extraction layer, a third feature extraction layer, a fourth feature extraction layer, a residual error layer, a global average pooling layer and a softmax layer, the first feature extraction layer is formed by serially arranging a convolutional layer, a normalizing layer, a convolutional layer and a normalizing layer, then parallelly connecting the first feature extraction layer with a group of convolutional layers, and finally serially connecting the first feature extraction layer with a maximum pooling layer, wherein the second feature extraction layer, the third feature extraction layer and the fourth feature extraction layer are identical to the first feature extraction layer;

selecting a training sample from the citrus tip strip sample set, and inputting the training sample into an initial convolutional neural network model;

performing feature extraction of different depths on the training sample through the first feature extraction layer, the second feature extraction layer, the third feature extraction layer and the fourth feature extraction layer to generate a second depth feature set;

fusing the second depth feature set through a residual layer to generate a second feature residual, wherein the second feature residual comprises a diaphorina citri feature residual or a affected tip residual;

classifying and calculating the second characteristic residual errors through a global average pooling layer and a softmax layer to generate a second probability value set of the training sample belonged to the diaphorina citri pins;

Calculating a loss value according to the second probability value set, the target estimation threshold value and a preset loss function of the initial convolutional neural network model to generate loss value data, wherein the loss value data is a statistical loss value set in the training process;

judging whether the loss value data is converged to 0 in a preset interval or not;

and if the loss value data is converged to 0 in the preset interval, determining the initial convolutional neural network model as a target convolutional neural network model.

Optionally, after determining whether the loss value data converges to 0 in the preset interval, the method further includes:

if the loss value data is not converged to 0 in the preset interval, judging whether the training times of the training sample reach the standard or not;

if the training times of the training samples reach the standard, updating the weight of the initial convolutional neural network model according to a small-batch gradient descent method, and re-selecting the training samples from the training sample set to input the training samples into the initial convolutional neural network model for training.

Optionally, after determining whether the training number of training samples meets the standard, the method further includes:

if the training times of the training samples do not reach the standard, updating the weight of the initial convolutional neural network model according to a small gradient descent method, and inputting the training samples into the initial convolutional neural network model again for training.

Optionally, obtaining a sample set of citrus fruit strips includes:

Acquiring a citrus tip strip shooting image set;

And carrying out sample expansion pretreatment on the shot images in the orange tip shot image set to generate an orange tip sample set, wherein the sample expansion pretreatment comprises scaling treatment, cutting treatment, rotation treatment and photo background gray level direct current component unification treatment.

Optionally, the feature extraction of different depths is performed on the citrus tip image by the first feature extraction layer, the second feature extraction layer, the third feature extraction layer and the fourth feature extraction layer, to generate a first depth feature set, including:

Inputting the citrus tip image into a real-time first feature extraction layer to generate a first residual error of the citrus tip image;

inputting the first residual error into a second feature extraction layer to generate a second residual error of the citrus tip strip image;

inputting the second residual error into a third feature extraction layer to generate a third residual error of the citrus tip strip image;

Inputting the third residual error into a fourth feature extraction layer to generate a fourth residual error of the citrus tip strip image;

the first, second, third, and fourth residuals are determined as a first depth feature set.

Optionally, fusing the first depth feature set through a residual layer to generate a first feature residual, including:

and sequentially fusing the first residual error to the fourth residual error through a residual error layer to generate a first characteristic residual error.

The second aspect of the application provides a device for intelligently positioning diaphorina citri in a citrus orchard, which is characterized by comprising:

the first acquisition unit is used for shooting the target citrus tree in multiple directions through the unmanned aerial vehicle to acquire a citrus tip image group, wherein the citrus tip image group comprises at least one shooting image of the target citrus tree, and the citrus tip image group comprises positioning information of the target citrus tree;

the first selecting unit is used for selecting one citrus tip image from the citrus tip image group;

The first input unit is used for inputting the citrus tip image into a target convolutional neural network model after training, the target convolutional neural network model comprises a first feature extraction layer, a second feature extraction layer, a third feature extraction layer, a fourth feature extraction layer, a residual layer, a global average pooling layer and a softmax layer, the first feature extraction layer is formed by serially arranging convolution layers, normalization layers, convolution layers and normalization layers, and then is connected with a group of convolution layers, normalization layers in parallel, and finally is serially connected with a maximum pooling layer, and the second feature extraction layer, the third feature extraction layer and the fourth feature extraction layer are identical with the first feature extraction layer;

The first generation unit is used for carrying out feature extraction of different depths on the citrus tip image through the first feature extraction layer, the second feature extraction layer, the third feature extraction layer and the fourth feature extraction layer to generate a first depth feature set;

The first fusion unit is used for fusing the first depth feature set through the residual layer to generate a first feature residual, wherein the first feature residual comprises a diaphorina citri feature residual or a affected tip strip residual;

the first computing unit is used for carrying out classification computation on the first characteristic residual errors through the global average pooling layer and the softmax layer to generate a first probability value set of the citrus fruit tip image belonging to the citrus psyllid fruit tip;

the first determining unit is used for generating at least two first probability value sets corresponding to the citrus tip image group according to the method;

and the second determining unit is used for generating a diaphorina citri alarm according to the positioning information when probability values larger than a preset threshold value exist in the at least two first probability value sets.

The second acquisition unit is used for acquiring a citrus tip sample set, wherein the citrus tip sample set comprises at least two training samples of citrus tip, and the training samples are images marked with real infection conditions and target estimation thresholds of the citrus psyllids;

The construction unit is used for constructing an initial convolutional neural network model, the initial convolutional neural network model comprises a first feature extraction layer, a second feature extraction layer, a third feature extraction layer, a fourth feature extraction layer, a residual layer, a global average pooling layer and a softmax layer, the first feature extraction layer is formed by serially arranging a convolutional layer, a normalizing layer, a convolutional layer and a normalizing layer, then parallelly connecting with a group of convolutional layers and a normalizing layer, and finally serially connecting with a maximum pooling layer, and the second feature extraction layer, the third feature extraction layer and the fourth feature extraction layer are identical to the first feature extraction layer;

The second selecting unit is used for selecting a training sample from the citrus tip sample set and inputting the training sample into the initial convolutional neural network model;

the second generating unit is used for carrying out feature extraction of different depths on the training sample through the first feature extraction layer, the second feature extraction layer, the third feature extraction layer and the fourth feature extraction layer to generate a second depth feature set;

the second fusion unit is used for fusing the second depth feature set through the residual layer to generate a second feature residual, and the second feature residual comprises a diaphorina citri feature residual or a affected tip strip residual;

The second calculation unit is used for carrying out classification calculation on the second characteristic residual errors through the global average pooling layer and the softmax layer to generate a second probability value set of the training sample belonging to the diaphorina citri pins;

The third calculation unit is used for calculating a loss value according to the second probability value set, the target estimation threshold value and a preset loss function of the initial convolutional neural network model so as to generate loss value data, wherein the loss value data is a statistical loss value set in the training process;

a first judging unit for judging whether the loss value data is converged to 0 in a preset interval;

And the third determining unit is used for determining the initial convolutional neural network model as a target convolutional neural network model if the loss value data is converged to 0 in a preset interval.

the second judging unit is used for judging whether the training times of the training sample reach the standard or not if the loss value data is not converged to 0 in the preset interval;

The first updating unit is used for updating the weight of the initial convolutional neural network model according to a small-batch gradient descent method if the training times of the training samples reach the standard, and re-selecting the training samples from the training sample set to input the training samples into the initial convolutional neural network model for training.

And the second updating unit is used for updating the weight of the initial convolutional neural network model according to the small-batch gradient descent method if the training times of the training samples do not reach the standard, and inputting the training samples into the initial convolutional neural network model again for training.

Optionally, the second obtaining unit specifically includes:

Acquiring a citrus tip strip shooting image set;

Optionally, the first generating unit specifically includes:

Optionally, the first fusion unit specifically is:

A third aspect of the present application provides an electronic apparatus, characterized by comprising:

a processor, a memory, an input-output unit, and a bus;

The processor is connected with the memory, the input/output unit and the bus;

the memory holds a program, and the processor invokes the program to perform the method according to any one of the first aspect and the first aspect.

A third aspect of the present application provides a computer readable storage medium having a program stored thereon, which when executed on a computer performs a method according to any of the first aspects.

From the above technical solutions, the embodiment of the present application has the following advantages:

According to the scheme, the unmanned aerial vehicle is used for carrying out fixed-point shooting, fixed-point information is a position point on the drip line of the target citrus tree, the drip line refers to an extension area of the crown of the fruit tree, the unmanned aerial vehicle shoots on the drip line to obtain a citrus tip image group, the citrus tip image group comprises at least one shooting image of the target citrus tree, and the citrus tip image group comprises positioning information of the target citrus tree. And selecting a citrus tip image from the citrus tip image group, inputting the citrus tip image into a target convolutional neural network model after training, wherein the target convolutional neural network model comprises a first feature extraction layer, a second feature extraction layer, a third feature extraction layer, a fourth feature extraction layer, a residual layer, a global average pooling layer and a softmax layer, the first feature extraction layer is formed by serially arranging a convolution layer-a normalization layer-a convolution layer-a normalization layer, then parallelly connecting the first feature extraction layer with a group of convolution layer-a normalization layer and serially connecting the first feature extraction layer with a maximum pooling layer, and the second feature extraction layer, the third feature extraction layer and the fourth feature extraction layer are identical with the first feature extraction layer. And performing feature extraction of different depths on the citrus tip image through the first feature extraction layer, the second feature extraction layer, the third feature extraction layer and the fourth feature extraction layer to generate a first depth feature set. And fusing the first depth feature set through a residual layer to generate a first feature residual, wherein the first feature residual comprises a diaphorina citri feature residual or a affected tip residual. And carrying out classification calculation on the first characteristic residual errors through the global average pooling layer and the softmax layer, and generating a first probability value set of the citrus tip image belonging to the citrus psyllid tip. And generating at least two first probability value sets corresponding to the citrus tip image group according to the method. And when probability values larger than a preset threshold exist in the at least two first probability value sets, generating a diaphorina citri alarm according to the positioning information. Because the first feature extraction layer, the second feature extraction layer, the third feature extraction layer and the fourth feature extraction layer sequentially extract the features of the citrus tip images in different depths, the extracted features cannot be submerged in the background of the shot images, and the accuracy of detecting the tree infected by the diaphorina citri is improved through feature fusion, so that the positioning accuracy of the diaphorina citri is further improved.

Drawings

FIG. 1 is a schematic view of one embodiment of a method for intelligent positioning of diaphorina citri in a citrus orchard according to the present application;

FIGS. 2-1 FIGS. 2-2 FIGS. 2-3 are schematic illustrations of another embodiment of a method for intelligent positioning of diaphorina citri in a citrus orchard according to the present application;

FIG. 3 is a schematic view of an embodiment of the device for intelligent positioning of diaphorina citri in citrus orchards according to the present application;

FIG. 4 is a schematic view of another embodiment of the device for intelligent positioning of diaphorina citri in citrus orchards according to the present application;

FIG. 5 is a schematic diagram of an embodiment of an electronic device of the present application.

Detailed Description

In the following description, for purposes of explanation and not limitation, specific details are set forth such as the particular system architecture, techniques, etc., in order to provide a thorough understanding of the embodiments of the present application. It will be apparent, however, to one skilled in the art that the present application may be practiced in other embodiments that depart from these specific details. In other instances, detailed descriptions of well-known systems, devices, circuits, and methods are omitted so as not to obscure the description of the present application with unnecessary detail.

It should be understood that the terms "comprises" and/or "comprising," when used in this specification and the appended claims, specify the presence of stated features, integers, steps, operations, elements, and/or components, but do not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and/or groups thereof.

It should also be understood that the term "and/or" as used in the present specification and the appended claims refers to any and all possible combinations of one or more of the associated listed items, and includes such combinations.

As used in the present description and the appended claims, the term "if" may be interpreted as "when..once" or "in response to a determination" or "in response to detection" depending on the context. Similarly, the phrase "if a determination" or "if a [ described condition or event ] is detected" may be interpreted in the context of meaning "upon determination" or "in response to determination" or "upon detection of a [ described condition or event ]" or "in response to detection of a [ described condition or event ]".

Furthermore, the terms "first," "second," "third," and the like in the description of the present specification and in the appended claims, are used for distinguishing between descriptions and not necessarily for indicating or implying a relative importance.

Reference in the specification to "one embodiment" or "some embodiments" or the like means that a particular feature, structure, or characteristic described in connection with the embodiment is included in one or more embodiments of the application. Thus, appearances of the phrases "in one embodiment," "in some embodiments," "in other embodiments," and the like in the specification are not necessarily all referring to the same embodiment, but mean "one or more but not all embodiments" unless expressly specified otherwise. The terms "comprising," "including," "having," and variations thereof mean "including but not limited to," unless expressly specified otherwise.

In the prior art, accurate and efficient diaphorina citri positioning detection is an important precondition for preventing and controlling citrus yellow dragon disease, and is also a key link for realizing accurate agriculture. Aiming at the problems of the field manual detection method, a great deal of research work has been done by a plurality of scholars at home and abroad in the field of crop pest identification based on convolutional neural network technology. However, due to the small size of the diaphorina citri, the morphology changes in the growth process, so that the characteristics of the diaphorina citri can be submerged in the background of the shot image in the process of training of the convolutional neural network, which is called as a multi-scale problem, the accuracy of detection is reduced when the convolutional neural network model detects the diaphorina citri infected tree, and the positioning accuracy of the diaphorina citri is further reduced.

Based on the method, the application discloses an intelligent positioning method and a related device for citrus psyllids in a citrus orchard, which are used for improving the positioning accuracy of the citrus psyllids.

The following description of the embodiments of the present application will be made clearly and fully with reference to the accompanying drawings, in which it is evident that the embodiments described are only some, but not all embodiments of the application. All other embodiments, which can be made by those skilled in the art based on the embodiments of the application without making any inventive effort, are intended to be within the scope of the application.

The method of the present application may be applied to a server, a device, a terminal, or other devices having logic processing capabilities, and the present application is not limited thereto. For convenience of description, the following description will take an execution body as an example of a terminal.

Referring to fig. 1, the present application provides an embodiment of a method for intelligently controlling diaphorina citri, including:

101. the method comprises the steps that a target citrus tree is shot in multiple directions through an unmanned aerial vehicle, a citrus tip image group is obtained, the citrus tip image group comprises at least one shot image of the target citrus tree, and the citrus tip image group comprises positioning information of the target citrus tree;

The terminal firstly sets a pointing device around the target citrus tree, so that the unmanned aerial vehicle can capture the positioning information of the pointing device and shoot. Specifically, the terminal sets up at least one pointing device on the drip of target citrus tree, pointing device can detect the tree drip to on sending location information to unmanned aerial vehicle, unmanned aerial vehicle carries out multi-direction shooting to target citrus tree through pointing device, acquires the citrus tip image group, and the citrus tip image group contains at least one shooting image of target citrus tree, contains the location information of target citrus tree in the citrus tip image group, specifically, carries out the flight through unmanned aerial vehicle in the citrus planting district, carries out image acquisition to every citrus fruit, mainly gathers the image of citrus fruit tip part. Because the diaphorina citri is mainly located at the young shoot part, the top of the citrus fruit tree needs to be shot through the unmanned aerial vehicle camera. Each shot image in the citrus tip strip image group is correspondingly marked with a citrus fruit tree.

The drip line of the fruit tree refers to the extension area of the crown of the fruit tree (the outer circle is drawn when the fruit tree is overlooked, and the circle is the drip line), and is the main growth area of the capillary root and the fibrous root of the fruit tree. The capillary root and the fibrous root bear the functions of most of the fruit trees of eating fertilizer and absorbing water. There is a need to prevent cash crops from growing on drip lines. In this embodiment, the drip line may also be used as a shooting location.

102. Selecting a citrus tip image from the citrus tip image group;

103. Inputting citrus tip images into a target convolutional neural network model with training completed, wherein the target convolutional neural network model comprises a first feature extraction layer, a second feature extraction layer, a third feature extraction layer, a fourth feature extraction layer, a residual error layer, a global average pooling layer and a softmax layer, the first feature extraction layer is formed by serially arranging convolution layers, normalization layers, convolution layers and normalization layers, and then is connected with a group of convolution layers, normalization layers and a maximum pooling layer in parallel, and the second feature extraction layer, the third feature extraction layer and the fourth feature extraction layer are identical to the first feature extraction layer;

and randomly selecting a citrus tip image from the citrus tip image group by the terminal, and inputting the citrus tip image into the trained target convolutional neural network model by the terminal. The target convolutional neural network model comprises a first feature extraction layer, a second feature extraction layer, a third feature extraction layer, a fourth feature extraction layer, a residual layer, a global average pooling layer and a softmax layer, wherein the first feature extraction layer is formed by serially arranging a convolutional layer, a normalizing layer, a convolutional layer and a normalizing layer, then parallelly connecting with a group of convolutional layers and the normalizing layer, and finally serially connecting with a maximum pooling layer, and the second feature extraction layer, the third feature extraction layer and the fourth feature extraction layer are identical to the first feature extraction layer.

The first feature extraction layer, the second feature extraction layer, the third feature extraction layer and the fourth feature extraction layer are mainly used for extracting the characteristics of the diaphorina citri with different depths and the characteristics of the infection tip of the diaphorina citri. In the detection process, the disease of the citrus fruit tree can be determined only by detecting the characteristics of the diaphorina citri or the characteristics of the infection tip of the diaphorina citri.

104. Performing feature extraction of different depths on the citrus tip image through the first feature extraction layer, the second feature extraction layer, the third feature extraction layer and the fourth feature extraction layer to generate a first depth feature set;

The terminal performs feature extraction of different depths on the citrus tip images through the first feature extraction layer, the second feature extraction layer, the third feature extraction layer and the fourth feature extraction layer to generate a first depth feature set, and the condition that the features caused by the growth stage are submerged due to the fact that the features of the citrus tip images are too small can be prevented through the feature extraction of different depths.

105. Fusing the first depth feature set through a residual layer to generate a first feature residual, wherein the first feature residual comprises a diaphorina citri feature residual or a affected tip residual;

the terminal fuses the first depth feature set through the residual layer to generate a first feature residual, wherein the first feature residual comprises a diaphorina citri feature residual or a affected tip residual. The residual fusion is mainly aimed at integrating the characteristics of the diaphorina citri or the characteristics of the infected tip of the diaphorina citri to obtain better characteristics.

106. Classifying and calculating the first characteristic residual errors through a global average pooling layer and a softmax layer to generate a first probability value set of citrus fruit tip images belonging to citrus psyllid fruit tips;

The terminal carries out classification calculation on the first characteristic residual error through a global average pooling layer and a softmax layer to generate a first probability value set of the citrus fruit image belonging to the citrus psyllid fruit, i.e. analyzing the probability value of the first residual characteristic being the insect characteristic of a diaphorina citri or the probability value of the first residual characteristic being the disease branch characteristic of a diaphorina citri affected tip strip.

And the terminal calculates the first characteristic residual error through the global average pooling layer and the softmax layer to generate a probability value set.

The specific calculation mode is as follows:

where e ^x is an exponential function, y _i represents the first input neuron in the output layer, the operation of the denominator represents a total of n output neurons in the output layer, and the sum of the indices of the input neurons in all output layers is calculated'

Y ⁱ denotes the output of the ith neuron, softmax (y _i) is a set of probability values.

107. Generating at least two first probability value sets corresponding to the citrus tip image group according to the method;

And the terminal determines the infection condition of the citrus psyllids of the target citrus tree according to the first probability value set, and determines that the citrus fruit tree corresponding to the citrus tip image is ill according to the fruit tree when the first probability value set shows that the infection probability of the citrus fruit tree is as high as a preset threshold. And the terminal acquires a first probability value set for a plurality of images shot in the citrus tip image group.

In the training process of the citrus tip image, the target convolutional neural network model is required to generate numerical value output, and only the result with high probability and the corresponding probability numerical value are required to be output in the actual application of the target convolutional neural network model. For example, in the training process, the model probability distribution P (the diseased probability value P1 and the non-diseased probability value P2) needs to be output, and in the actual application of the target convolutional neural network model, only max (P1, P2)) needs to be output.

108. And generating a diaphorina citri alarm according to the positioning information when probability values larger than a preset threshold exist in the at least two first probability value sets.

And when a probability value larger than a preset threshold exists in the first probability value set, generating a diaphorina citri alarm according to the positioning information.

Referring to fig. 2-1 fig. 2-2 fig. 2-3, another embodiment of a method for intelligent control of diaphorina citri according to the present application includes:

201. Acquiring a citrus tip strip shooting image set;

202. Carrying out sample expansion pretreatment on a shot image in a citrus tip shooting image set to generate a citrus tip sample set, wherein the sample expansion pretreatment comprises scaling treatment, cutting treatment, rotation treatment and photo background gray level direct current component unification treatment, and the citrus tip sample set comprises at least two citrus tip training samples which are images marked with real infection conditions and target estimation thresholds of citrus psyllids;

After the terminal acquires the captured image set of the citrus fruit tip, sample expansion preprocessing is carried out on the captured image in the captured image set of the citrus fruit tip, and a sample set of the citrus fruit tip is generated. The sample expansion pretreatment comprises scaling treatment, cutting treatment, rotation treatment and photo background gray level direct current component unification treatment, and the citrus tip sample set comprises at least two training samples of citrus tip, wherein the training samples are images marked with real infection conditions and target estimation thresholds of the citrus psyllids.

The terminal performs sample expansion pretreatment on the shot images in the citrus fruit shoot image set to generate a citrus fruit shoot sample set, wherein the sample expansion pretreatment comprises scaling treatment, cutting treatment, rotation treatment and photo background gray level direct current component unification treatment. For a citrus tip shooting image set obtained by shooting, sample expansion pretreatment is needed for a photo before the citrus tip shooting image set is sent into an initial convolutional neural network model for training.

When the initial convolutional neural network model is trained, a large number of photographs of the citrus psyllids exist and a large number of tips of the citrus psyllids are flattened, modeling is carried out through data features learned from a large number of training samples, sometimes a shot sample set is not enough, a defect sample is required to be artificially added through a data enhancement mode, the data enhancement comprises image operations such as rotation, offset, mirror image, cutting, stretching and the like on the photographs, so that new pictures are different from original pictures in a certain sense, and the data set is expanded.

Cutting: the image set of the citrus tip strip shot by using the industrial camera comprises a plurality of surrounding background parts besides the citrus psyllid part and the citrus psyllid affected tip strip part, the image of the surrounding background parts is unnecessary, the training and detection of the initial convolutional neural network model can be influenced, and the time cost and the GPU display memory consumption during the training and the testing of the initial convolutional neural network model can be increased by the extra image, so that the useless images are required to be removed in a cutting mode, and only the citrus psyllid part and the citrus psyllid affected tip strip part are reserved.

Besides the cutting process, the rotation process, the offset process, the mirror image process and the stretching process, the training sample can be expanded for the image in the image set of the citrus tip strip shooting.

Unifying gray-scale direct current components of the photo background: because the types of the diaphorina citri and the disease tip of the diaphorina citri are not necessarily identical, photographing conditions of different factories are different, background gray scales of different defect photos are different, the training and detection of an initial convolutional neural network model are inconvenient, and the background gray scales of images in a training sample set are different, so that a final detection result can be influenced. The orange tip shooting image set comprises a gray direct current component of a background, a orange psyllid part and a gray alternating current component of a tip affected by the orange psyllid part, and the gray direct current components of the background in all images are unified by reserving the gray alternating current components in the images, so that the initial convolutional neural network model can be adapted to images with different background gray in the orange tip shooting image set.

Secondly, for RGB three-channel color images existing in a shot sample set, the processing steps are slightly different, the green G channel is processed according to the above graph, after the average gray value of each channel pixel is subtracted from the red R channel pixel and the blue B channel pixel, the added unified gray value is different from the green channel pixel, and the unified gray value of corresponding proportion is added according to the proportion of the average gray value of each channel and the average gray value of the green channel.

For example: the average pixel gray level of the RGB three channels is 50, 75 and 100 respectively, the added unified gray level of the green channel is 128, namely, all pixel values of the green channel are subtracted by 50, 128 is added, all pixel values of the red channel are subtracted by 75, 128 (75/50) is added, all pixel values of the blue channel are subtracted by 100, and 128 (100/50) is added.

203. An initial convolutional neural network model is built, the initial convolutional neural network model comprises a first feature extraction layer, a second feature extraction layer, a third feature extraction layer, a fourth feature extraction layer, a residual error layer, a global average pooling layer and a softmax layer, the first feature extraction layer is formed by serially arranging a convolutional layer, a normalizing layer, a convolutional layer and a normalizing layer, then parallelly connecting the first feature extraction layer with a group of convolutional layers, and finally serially connecting the first feature extraction layer with a maximum pooling layer, wherein the second feature extraction layer, the third feature extraction layer and the fourth feature extraction layer are identical to the first feature extraction layer;

The terminal builds an initial convolutional neural network model, wherein the initial convolutional neural network model comprises a first feature extraction layer, a second feature extraction layer, a third feature extraction layer, a fourth feature extraction layer, a residual error layer, a global average pooling layer and a softmax layer, the first feature extraction layer is formed by serially arranging a convolutional layer, a normalizing layer, a convolutional layer and a normalizing layer, then parallelly connecting the first feature extraction layer with a group of convolutional layers, and finally serially connecting the first feature extraction layer with a maximum pooling layer, wherein the second feature extraction layer, the third feature extraction layer and the fourth feature extraction layer are identical to the first feature extraction layer.

The residual layers are also formed by a plurality of layers of detachable convolution layers, and the residual layers fuse two residual units at a time, so that the function of fusing all residual units to extract multi-scale features is achieved, the multi-scale features can be extracted, and the influence caused by partial shielding is restrained.

204. Selecting a training sample from the citrus tip strip sample set, and inputting the training sample into an initial convolutional neural network model;

the terminal selects a training sample from the citrus tip sample set, and inputs the training sample into the initial convolutional neural network model, wherein the terminal can randomly extract one training sample from the citrus tip sample set, or randomly extract a plurality of samples for training.

In this embodiment, a small batch of training samples with a batch size of 16 is used as the amount of one training, and the training effect is achieved through multiple iterations. In this embodiment, the number of iterations is about 25000.

For example: and inputting training samples in the citrus tip sample set into a multi-scale convolutional neural network model, wherein the training samples are tip images with citrus psyllids, and the target predicted value is more than ninety percent. After the first feature extraction layer, the second feature extraction layer, the third feature extraction layer and the fourth feature extraction layer are used for carrying out feature extraction with different depths, the residual layers are used for fusing the features with different depths, and the output layer is used for calculating the loss value of the features through global average pooling and combining the softmax layer and the cross entropy loss function and outputting the loss value.

205. Performing feature extraction of different depths on the training sample through the first feature extraction layer, the second feature extraction layer, the third feature extraction layer and the fourth feature extraction layer to generate a second depth feature set;

206. Fusing the second depth feature set through a residual layer to generate a second feature residual, wherein the second feature residual comprises a diaphorina citri feature residual or a affected tip residual;

207. classifying and calculating the second characteristic residual errors through a global average pooling layer and a softmax layer to generate a second probability value set of the training sample belonged to the diaphorina citri pins;

The terminal firstly performs feature extraction of different depths on the training sample through the first feature extraction layer, the second feature extraction layer, the third feature extraction layer and the fourth feature extraction layer to generate a second depth feature set.

And the terminal fuses the second depth feature set through the residual layer to generate a second feature residual, wherein the second feature residual comprises a diaphorina citri feature residual or a affected tip strip residual.

The residual error is calculated by the following steps: assuming that the first feature extraction layer input is X _l, two sets of convolution-normalization layers, which are arranged in series, and another set of parallel convolution-normalization layers, respectively, are formulated as follows:

y _l is the output of the first residual unit, F is the residual function, and F _xl is the residual calculation for Xl.

X _l+1 is the input to the second feature extraction layer:

Y _l+1 is the output of the 1 st residual unit, F is the residual function, and F _xl+1 is the residual calculation for X _l+1.

208. Calculating a loss value according to the second probability value set, the target estimation threshold value and a preset loss function of the initial convolutional neural network model to generate loss value data, wherein the loss value data is a statistical loss value set in the training process;

and the terminal calculates a loss value according to the second probability value set, the target estimation threshold value and a preset loss function of the initial convolutional neural network model to generate loss value data, wherein the loss value data is a statistical loss value set in the training process.

209. Judging whether the loss value data is converged to 0 in a preset interval or not;

The terminal determines whether the initial convolutional neural network model completes convergence according to the loss value data, if yes, the step 210 is executed, and if not, the step 211 is executed.

210. If the loss value data is converged to 0 in the preset interval, determining the initial convolutional neural network model as a target convolutional neural network model;

when the terminal determines that the initial convolutional neural network model completes convergence, the terminal can determine that the initial convolutional neural network model is a target convolutional neural network model, and the actual application can be achieved.

211. If the loss value data is not converged to 0 in the preset interval, judging whether the training times of the training sample reach the standard or not;

when the terminal determines that the initial convolutional neural network model does not complete convergence, the terminal judges whether the training times of the training samples reach the standard, if so, the step 212 is executed, and if not, the step 213 is executed.

212. If the training times of the training samples reach the standard, updating the weight of the initial convolutional neural network model according to a small batch gradient descent method, and re-selecting the training samples from the training sample set to input the training samples into the initial convolutional neural network model for training;

When the training times of the training samples reach the standard, the training samples reach the training amount, the terminal updates the weight of the initial convolutional neural network model according to a small gradient descent method, and reselects the training samples from the training sample set to input the training samples into the initial convolutional neural network model for training

213. If the training times of the training samples do not reach the standard, updating the weight of the initial convolutional neural network model according to a small gradient descent method, and inputting the training samples into the initial convolutional neural network model again for training;

If the training times of the training samples do not reach the standard, the training samples do not reach the training amount, the weight of the initial convolutional neural network model is updated according to a small gradient descent method, and the training samples are input into the initial convolutional neural network model again for training.

214. The method comprises the steps that a target citrus tree is shot in multiple directions through an unmanned aerial vehicle, a citrus tip image group is obtained, the citrus tip image group comprises at least one shot image of the target citrus tree, and the citrus tip image group comprises positioning information of the target citrus tree;

215. Selecting a citrus tip image from the citrus tip image group;

216. Inputting citrus tip images into a target convolutional neural network model with training completed, wherein the target convolutional neural network model comprises a first feature extraction layer, a second feature extraction layer, a third feature extraction layer, a fourth feature extraction layer, a residual error layer, a global average pooling layer and a softmax layer, the first feature extraction layer is formed by serially arranging convolution layers, normalization layers, convolution layers and normalization layers, and then is connected with a group of convolution layers, normalization layers and a maximum pooling layer in parallel, and the second feature extraction layer, the third feature extraction layer and the fourth feature extraction layer are identical to the first feature extraction layer;

Steps 214 to 216 in this embodiment are similar to steps 101 to 103 in the previous embodiment, and are not repeated here.

217. Inputting the citrus tip image into a real-time first feature extraction layer to generate a first residual error of the citrus tip image;

218. Inputting the first residual error into a second feature extraction layer to generate a second residual error of the citrus tip strip image;

219. inputting the second residual error into a third feature extraction layer to generate a third residual error of the citrus tip strip image;

220. inputting the third residual error into a fourth feature extraction layer to generate a fourth residual error of the citrus tip strip image;

221. determining the first residual, the second residual, the third residual and the fourth residual as a first depth feature set;

222. sequentially fusing the first residual error to the fourth residual error through a residual error layer to generate a first characteristic residual error;

The method comprises the steps that a terminal inputs a citrus tip image into a real-time first feature extraction layer to generate a first residual error of the citrus tip image, inputs the first residual error into a second feature extraction layer to generate a second residual error of the citrus tip image, inputs the second residual error into a third feature extraction layer to generate a third residual error of the citrus tip image, inputs the third residual error into a fourth feature extraction layer to generate a fourth residual error of the citrus tip image, and finally the terminal fuses the first residual error to the fourth residual error in sequence through a residual error layer to generate the first feature residual error.

223. Classifying and calculating the first characteristic residual errors through a global average pooling layer and a softmax layer to generate a first probability value set of citrus fruit tip images belonging to citrus psyllid fruit tips;

224. generating at least two first probability value sets corresponding to the citrus tip image group according to the method;

225. And generating a diaphorina citri alarm according to the positioning information when probability values larger than a preset threshold exist in the at least two first probability value sets.

Steps 223 to 225 in this embodiment are similar to steps 106 to 108 in the previous embodiment, and will not be repeated here.

In the embodiment of the application, the terminal firstly acquires the captured image set of the citrus fruit tip, and the terminal performs sample expansion pretreatment on the captured image in the captured image set of the citrus fruit tip to generate a sample set of the citrus fruit tip, so that the number of samples is increased. The terminal builds an initial convolutional neural network model, and the terminal selects training samples from the citrus tip sample set and inputs the training samples into the initial convolutional neural network model. The terminal firstly performs feature extraction of different depths on the training sample through the first feature extraction layer, the second feature extraction layer, the third feature extraction layer and the fourth feature extraction layer to generate a second depth feature set. And the terminal fuses the second depth feature set through the residual layer to generate a second feature residual, wherein the second feature residual comprises a diaphorina citri feature residual or a affected tip strip residual. And the terminal performs classification calculation on the second characteristic residual errors through the global average pooling layer and the softmax layer to generate a second probability value set of the training sample belonged to the diaphorina citri pins. And the terminal calculates a loss value according to the second probability value set, the target estimation threshold value and a preset loss function of the initial convolutional neural network model so as to generate loss value data. The terminal judges whether the loss value data is converged to 0 in a preset interval, if the loss value data is converged to 0 in the preset interval, the terminal determines that the initial convolutional neural network model is a target convolutional neural network model, and if the loss value data is not converged to 0 in the preset interval, the terminal judges whether the training times of the training samples reach the standard. If the training times of the training samples reach the standard, the terminal updates the weight of the initial convolutional neural network model according to a small gradient descent method, and reselects the training samples from the training sample set to input the training samples into the initial convolutional neural network model for training. If the training times of the training samples do not reach the standard, the terminal updates the weight of the initial convolutional neural network model according to a small gradient descent method, and re-inputs the training samples into the initial convolutional neural network model for training.

Firstly, a terminal shoots a target citrus tree in multiple directions through an unmanned aerial vehicle to obtain a citrus tip image group, wherein the citrus tip image group comprises at least one shot image of the target citrus tree, and the citrus tip image group comprises positioning information of the target citrus tree. And the terminal selects a citrus tip image from the citrus tip image group, inputs the citrus tip image into a target convolutional neural network model after training, wherein the target convolutional neural network model comprises a first feature extraction layer, a second feature extraction layer, a third feature extraction layer, a fourth feature extraction layer, a residual error layer, a global average pooling layer and a softmax layer, the first feature extraction layer is formed by serially arranging a convolution layer, a normalization layer, a group of convolution layers, a normalization layer, and the third feature extraction layer, the fourth feature extraction layer and the first feature extraction layer. The method comprises the steps that a terminal inputs a citrus tip image into a real-time first feature extraction layer to generate a first residual error of the citrus tip image, inputs the first residual error into a second feature extraction layer to generate a second residual error of the citrus tip image, inputs the second residual error into a third feature extraction layer to generate a third residual error of the citrus tip image, inputs the third residual error into a fourth feature extraction layer to generate a fourth residual error of the citrus tip image, and finally the terminal sequentially fuses the first residual error to the fourth residual error through a residual error layer to generate a first feature residual error, wherein the first feature residual error comprises a citrus psyllium feature residual error or a diseased citrus tip residual error.

The terminal performs classification calculation on the first feature residual through a global average pooling layer and a softmax layer, a first set of probability values for the citrus tip image attributed to the citrus psyllid tip is generated. And the terminal generates at least two first probability value sets corresponding to the citrus tip image group according to the method. When the terminal determines that probability values larger than a preset threshold exist in at least two first probability value sets, the terminal generates a diaphorina citri alarm according to the positioning information. Because the first feature extraction layer, the second feature extraction layer, the third feature extraction layer and the fourth feature extraction layer sequentially extract the features of the citrus tip images in different depths, the extracted features cannot be submerged in the background of the shot images, and the accuracy of detecting the tree infected by the diaphorina citri is improved through feature fusion, so that the positioning accuracy of the diaphorina citri is further improved.

Referring to fig. 3, the present application provides an embodiment of an apparatus for intelligently positioning diaphorina citri in a citrus orchard, including:

A first obtaining unit 301, configured to obtain, by using an unmanned aerial vehicle, a citrus top image set, where the citrus top image set includes at least one captured image of a target citrus tree, and includes positioning information of the target citrus tree;

A first selecting unit 302, configured to select a citrus tip image from the set of citrus tip images;

The first input unit 303 is configured to input the citrus tip image into a trained target convolutional neural network model, where the target convolutional neural network model includes a first feature extraction layer, a second feature extraction layer, a third feature extraction layer, a fourth feature extraction layer, a residual layer, a global average pooling layer, and a softmax layer, where the first feature extraction layer is formed by serially arranging a convolution layer, a normalization layer, a convolution layer, and a normalization layer, and then is connected in parallel with a group of convolution layer, and most serially connected with a maximum pooling layer, where the second feature extraction layer, the third feature extraction layer, and the fourth feature extraction layer are identical to the first feature extraction layer;

A first generating unit 304, configured to perform feature extraction of different depths on the citrus tip strip image through the first feature extraction layer, the second feature extraction layer, the third feature extraction layer, and the fourth feature extraction layer, to generate a first depth feature set;

the first fusion unit 305 is configured to fuse the first depth feature set through a residual layer to generate a first feature residual, where the first feature residual includes a diaphorina citri feature residual or a affected tip residual;

A first calculating unit 306, configured to perform classification calculation on the first feature residual through the global average pooling layer and the softmax layer, and generate a first probability value set of citrus fruit image belonging to citrus psyllid fruit;

A first determining unit 307, configured to generate at least two first probability value sets corresponding to the citrus tip strip image set according to the above method;

The second determining unit 308 is configured to generate a diaphorina citri alert according to the positioning information when there are probability values greater than a preset threshold value in the at least two first probability value sets.

Referring to fig. 4, another embodiment of an apparatus for intelligently positioning diaphorina citri in a citrus orchard according to the present application includes:

A second obtaining unit 401, configured to obtain a set of citrus fruit tip samples, where the set of citrus fruit tip samples includes at least two training samples of citrus fruit tip, and the training samples are images marked with real infection conditions and target estimation thresholds of citrus psyllids;

optionally, the second obtaining unit 401 specifically includes:

Acquiring a citrus tip strip shooting image set;

The construction unit 402 is configured to construct an initial convolutional neural network model, where the initial convolutional neural network model includes a first feature extraction layer, a second feature extraction layer, a third feature extraction layer, a fourth feature extraction layer, a residual layer, a global average pooling layer, and a softmax layer, the first feature extraction layer is arranged in series from a convolutional layer to a normalizing layer to a convolutional layer to a normalizing layer, and is connected in parallel with a group of convolutional layers to a normalizing layer, and is connected in series with a maximum pooling layer, where the second feature extraction layer, the third feature extraction layer, and the fourth feature extraction layer are identical to the first feature extraction layer;

A second selecting unit 403, configured to select a training sample from the set of citrus tip samples, and input the training sample into the initial convolutional neural network model;

A second generating unit 404, configured to perform feature extraction of different depths on the training sample through the first feature extraction layer, the second feature extraction layer, the third feature extraction layer, and the fourth feature extraction layer, to generate a second depth feature set;

the second fusing unit 405 is configured to fuse the second depth feature set through the residual layer to generate a second feature residual, where the second feature residual includes a diaphorina citri feature residual or a affected tip residual;

A second calculation unit 406, configured to perform classification calculation on the second feature residual through the global average pooling layer and the softmax layer, and generate a second probability value set of the training sample belonging to the diaphorina citri pins;

A third calculation unit 407, configured to calculate a loss value according to the second probability value set, the target estimation threshold value, and a preset loss function of the initial convolutional neural network model, so as to generate loss value data, where the loss value data is a statistical loss value set in the training process;

a first judging unit 408, configured to judge whether the loss value data converges to 0 in a preset interval;

A third determining unit 409, configured to determine, if the loss value data converges to 0 in a preset interval, that the initial convolutional neural network model is a target convolutional neural network model;

A second judging unit 410, configured to judge whether the training times of the training sample reach the standard if the loss value data does not converge to 0 in the preset interval;

A first updating unit 411, configured to update the weight of the initial convolutional neural network model according to a small-batch gradient descent method if the training times of the training samples reach the standard, and reselect the training samples from the training sample set to input the training samples into the initial convolutional neural network model for training;

A second updating unit 412, configured to update the weight of the initial convolutional neural network model according to the small-batch gradient descent method if the training frequency of the training sample does not reach the standard, and re-input the training sample into the initial convolutional neural network model for training;

A first obtaining unit 413, configured to obtain, by using the unmanned aerial vehicle, a citrus top image set, where the citrus top image set includes at least one captured image of the target citrus tree, and the citrus top image set includes positioning information of the target citrus tree;

A first selecting unit 414, configured to select a citrus tip image from the set of citrus tip images;

The first input unit 415 is configured to input the citrus tip image into a trained target convolutional neural network model, where the target convolutional neural network model includes a first feature extraction layer, a second feature extraction layer, a third feature extraction layer, a fourth feature extraction layer, a residual layer, a global average pooling layer, and a softmax layer, where the first feature extraction layer is formed by serially arranging a convolution layer, a normalization layer, a convolution layer, and a normalization layer, and then is connected in parallel with a group of convolution layer, and most serially connected with a maximum pooling layer, where the second feature extraction layer, the third feature extraction layer, and the fourth feature extraction layer are identical to the first feature extraction layer;

A first generating unit 416, configured to perform feature extraction of different depths on the citrus tip strip image through the first feature extraction layer, the second feature extraction layer, the third feature extraction layer, and the fourth feature extraction layer, to generate a first depth feature set;

optionally, the first generating unit 416 specifically is:

A first fusion unit 417, configured to fuse the first depth feature set through a residual layer to generate a first feature residual, where the first feature residual includes a diaphorina citri feature residual or a affected tip residual;

optionally, the first fusing unit 417 specifically includes:

A first calculation unit 418, configured to perform classification calculation on the first feature residual through the global average pooling layer and the softmax layer, and generate a first probability value set of citrus fruit image belonging to citrus psyllid fruit;

A first determining unit 419 configured to generate at least two first sets of probability values corresponding to the set of images of citrus fruit strips according to the method described above;

The second determining unit 420 is configured to generate a diaphorina citri alert according to the positioning information when there are probability values greater than a preset threshold value in the at least two first probability value sets.

Referring to fig. 5, the present application provides an electronic device, including:

A processor 501, a memory 502, an input-output unit 503, and a bus 504.

The processor 501 is connected to a memory 502, an input/output unit 503, and a bus 504.

The memory 501 holds a program that the processor 501 invokes to perform the method as in fig. 1-2-3.

The present application provides a computer readable storage medium having a program stored thereon, which when executed on a computer performs the method as in fig. 1 to 2-1, 2-3.

It will be clear to those skilled in the art that, for convenience and brevity of description, specific working procedures of the above-described systems, apparatuses and units may refer to corresponding procedures in the foregoing method embodiments, which are not repeated herein.

In the several embodiments provided in the present application, it should be understood that the disclosed systems, devices, and methods may be implemented in other manners. For example, the apparatus embodiments described above are merely illustrative, e.g., the division of the units is merely a logical function division, and there may be additional divisions when actually implemented, e.g., multiple units or components may be combined or integrated into another system, or some features may be omitted or not performed. Alternatively, the coupling or direct coupling or communication connection shown or discussed with each other may be an indirect coupling or communication connection via some interfaces, devices or units, which may be in electrical, mechanical or other form.

The units described as separate units may or may not be physically separate, and units shown as units may or may not be physical units, may be located in one place, or may be distributed on a plurality of network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

In addition, each functional unit in the embodiments of the present application may be integrated in one processing unit, or each unit may exist alone physically, or two or more units may be integrated in one unit. The integrated units may be implemented in hardware or in software functional units.

The integrated units, if implemented in the form of software functional units and sold or used as stand-alone products, may be stored in a computer readable storage medium. Based on such understanding, the technical solution of the present application may be embodied essentially or in part or all of the technical solution or in part in the form of a software product stored in a storage medium, including instructions for causing a computer device (which may be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method according to the embodiments of the present application. And the aforementioned storage medium includes: a U-disk, a removable hard disk, a read-only memory (ROM), a random-access memory (RAM, random access memory), a magnetic disk, or an optical disk, or other various media capable of storing program codes.

Claims

1. The intelligent positioning method for the diaphorina citri in the citrus orchard is characterized by comprising the following steps of:

Selecting a citrus tip image from the citrus tip image group;

Inputting the citrus tip image into a target convolutional neural network model with training completed, wherein the target convolutional neural network model comprises a first feature extraction layer, a second feature extraction layer, a third feature extraction layer, a fourth feature extraction layer, a residual error layer, a global average pooling layer and a softmax layer, the first feature extraction layer is formed by serially arranging convolution layers, normalization layers, convolution layers and normalization layers, and then is connected with a group of convolution layers, normalization layers and a maximum pooling layer in parallel, and the second feature extraction layer, the third feature extraction layer and the fourth feature extraction layer are identical to the first feature extraction layer;

Fusing the first depth feature set through the residual layer to generate a first feature residual, wherein the first feature residual comprises a diaphorina citri feature residual or a affected tip strip residual;

classifying and calculating the first characteristic residual errors through the global average pooling layer and the softmax layer to generate a first probability value set of the citrus fruit tree belonging to the citrus fruit tree image;

generating at least two first probability value sets corresponding to the citrus tip strip image group according to the method;

And generating a diaphorina citri alarm according to the positioning information when probability values larger than a preset threshold exist in the at least two first probability value sets.

2. The method of claim 1, wherein prior to the acquiring the set of images of the citrus tip, the method further comprises:

Obtaining a citrus tip sample set, wherein the citrus tip sample set comprises at least two training samples of citrus tip, and the training samples are images marked with real infection conditions and target estimation thresholds of citrus psyllids;

An initial convolutional neural network model is built, the initial convolutional neural network model comprises a first feature extraction layer, a second feature extraction layer, a third feature extraction layer, a fourth feature extraction layer, a residual layer, a global average pooling layer and a softmax layer, the first feature extraction layer is formed by serially arranging a convolutional layer, a normalizing layer, a convolutional layer and a normalizing layer, then parallelly connecting with a group of convolutional layers and a normalizing layer, and finally serially connecting with a maximum pooling layer, and the second feature extraction layer, the third feature extraction layer and the fourth feature extraction layer are identical to the first feature extraction layer;

Selecting a training sample from the citrus tip sample set, and inputting the training sample into the initial convolutional neural network model;

Fusing the second depth feature set through the residual layer to generate a second feature residual, wherein the second feature residual comprises a diaphorina citri feature residual or a affected tip strip residual;

Classifying and calculating the second characteristic residual error through the global average pooling layer and the softmax layer to generate a second probability value set of the training sample belonging to the diaphorina citri pins;

Calculating a loss value according to the second probability value set, the target estimation threshold value and a preset loss function of the initial convolutional neural network model to generate loss value data, wherein the loss value data is a statistical loss value set in a training process;

And if the loss value data is converged to 0 in a preset interval, determining the initial convolutional neural network model as a target convolutional neural network model.

3. The method according to claim 2, wherein after said determining whether the loss value data converges to 0 within a preset interval, the method further comprises:

if the loss value data is not converged to 0 in the preset interval, judging whether the training times of the training samples reach the standard or not;

And if the training times of the training samples reach the standard, updating the weight of the initial convolutional neural network model according to a small-batch gradient descent method, and re-selecting the training samples from the training sample set to input the training samples into the initial convolutional neural network model for training.

4. A method according to claim 3, wherein after determining whether the training sample has been trained for a number of times, the method further comprises:

If the training times of the training samples do not reach the standard, updating the weight of the initial convolutional neural network model according to a small-batch gradient descent method, and inputting the training samples into the initial convolutional neural network model again for training.

5. The method of claim 2, wherein the obtaining a sample set of citrus fruit strips comprises:

Acquiring a citrus tip strip shooting image set;

And carrying out sample expansion pretreatment on the shot images in the citrus fruit shoot image set to generate a citrus fruit shoot sample set, wherein the sample expansion pretreatment comprises scaling treatment, cutting treatment, rotation treatment and photo background gray level direct current component unification treatment.

6. The method of any of claims 1 to 5, wherein the generating a first set of depth features by the first, second, third, and fourth feature extraction layers performing feature extraction of different depths on the citrus strip image comprises:

inputting the first residual error into the second feature extraction layer to generate a second residual error of the citrus tip strip image;

Inputting the second residual error into the third feature extraction layer to generate a third residual error of the citrus tip strip image;

Inputting the third residual error into the fourth feature extraction layer to generate a fourth residual error of the citrus tip strip image;

the first, second, third and fourth residuals are determined to be a first depth feature set.

7. The method of claim 6, wherein the fusing the first depth feature set by the residual layer generates a first feature residual, comprising:

and sequentially fusing the first residual error to the fourth residual error through the residual error layer to generate a first characteristic residual error.

8. The utility model provides a device of intelligent prevention and cure of oranges and tangerines psyllid, its characterized in that includes:

The first acquisition unit is used for shooting the target citrus tree in multiple directions through the unmanned aerial vehicle, and acquiring a citrus tip image group, wherein the citrus tip image group comprises at least one shooting image of the target citrus tree, and the citrus tip image group comprises positioning information of the target citrus tree;

The first input unit is used for inputting the citrus tip image into a target convolutional neural network model with completed training, the target convolutional neural network model comprises a first feature extraction layer, a second feature extraction layer, a third feature extraction layer, a fourth feature extraction layer, a residual error layer, a global average pooling layer and a softmax layer, the first feature extraction layer is formed by serially arranging convolution layers, normalization layers, convolution layers and normalization layers, then parallelly connecting the first feature extraction layer with a group of convolution layers, normalization layers, and finally serially connecting the first feature extraction layer with a maximum pooling layer, and the second feature extraction layer, the third feature extraction layer and the fourth feature extraction layer are identical with the first feature extraction layer;

The first computing unit is used for carrying out classification computation on the first characteristic residual errors through the global average pooling layer and the softmax layer to generate a first probability value set of the citrus tip image belonging to the citrus psyllid tip;

the first determining unit is used for generating at least two first probability value sets corresponding to the citrus tip strip image group according to the method;

and the second determining unit is used for generating a diaphorina citri alarm according to the positioning information when probability values larger than a preset threshold exist in the at least two first probability value sets.

9. An electronic device, comprising:

a processor, a memory, an input-output unit, and a bus;

the processor is connected with the memory, the input/output unit and the bus;

The memory holds a program which the processor invokes to perform the method of any one of claims 1 to 7.

10. A computer readable storage medium having a program stored thereon, which when executed on a computer performs the method of any of claims 1 to 7.