KR101876397B1 - Apparatus and method for diagonising disease and insect pest of crops - Google Patents

Abstract

An apparatus and method for diagnosing pests of crops is disclosed. A method for diagnosing a pest of a crop according to an embodiment of the present invention includes the steps of receiving an image of a pest classification object, calculating a primary classification result of the image input using a convolutional neural network, And calculating a secondary classification result for the image using the primary classification result and the predetermined pest group information.

Description

FIELD OF THE INVENTION [0001] The present invention relates to an apparatus and a method for diagnosing a pest of a crop,

Embodiments of the present invention relate to techniques for accurately diagnosing pests of crops.

In order to effectively control disease or insect pests in the cultivation of crops, it is necessary to accurately diagnose the symptoms of insect pests. This is because the degree of control and damage varies greatly depending on the symptoms. For example, for the first time in Korea in April of 2015, burns of the burned sickness are similar to those of the black star disease that occurred frequently, but the extent of the damage is much more severe than the existing black star disease disease Do. In case of infection with a black star pattern disease, only the crop with the disease can be treated, but if a burn disease occurs and it is mutilated with other crops, the whole orchard must be disposed of. However, the two diseases are very small in size and similar in characteristics, so there is a limit to visually confirming the difference at the farmhouse.

In the past, when a disease or pest damage occurred, a diagnosis specialist such as the Rural Development Administration visited farmers to diagnose a problem and advised on the control, so that there was a limit to quick response to the pest and disease. Therefore, there is a need for a method for rapidly and accurately diagnosing pests and diseases in farm households.

Embodiments of the present invention are intended to provide a technical means for diagnosing the exact symptoms of crops suffering from pest or pest injuries.

According to an exemplary embodiment, there is provided a method performed in a computing device having one or more processors and a memory storing one or more programs executed by the one or more processors, the method comprising: receiving a pest classification image; Calculating a primary classification result for the input image using a convolutional neural network; And calculating a secondary classification result for the image using the primary classification result and the predetermined pest group information, wherein the composite neural network calculates a secondary classification result for the image based on the feature map a convolutional layer for generating a feature map, and a mask of a predetermined size to each pixel of the feature map, and sub-sampling the feature map using pixel values in the mask. A plurality of pairs of pulling layers for repeating a plurality of pairs of pulling layers for performing a plurality of pulling layers; And a feature classification layer group including a plurality of fully-connected layers for calculating the primary classification result from pixel values of a plurality of feature maps output from the feature extraction layer group / RTI >

The method may further include pre-processing the input image before performing the step of calculating the primary classification result.

The pre-processing may include: resizing the input image into a predetermined size; And applying a homomorphic filter to the resized image.

The resultant product layer may perform the product multiply operation using a Reed Linear Hidden Unit (ReLU) function.

The pooling layer may be a max pooling layer that outputs a maximum value of pixels in the mask.

The feature extraction layer group may be configured to generate a plurality of feature maps having a size that can be input to the foreground combining layer by repeatedly applying a pair of the composite product layer and the pooling layer to the image a plurality of times.

The mask size of the pooling layer of the first stage among the plurality of pooling layers included in the feature extraction layer group may be set to be larger than the mask size of the remaining pooling layers.

Wherein the feature classification layer group comprises: a first full combing layer for inputting pixel values of a plurality of feature maps output from the feature extraction layer group; A second prestraining layer connected to the first prestraining layer; And an output layer connected to the second pre-bonding layer.

The method may be configured to apply a predetermined dropout parameter to only the second full-combine layer of the feature classification layer group.

The output layer may be configured to calculate an output value for each node of the output layer using a hyperbolic tangent activation function.

The step of calculating the secondary classification result may be configured to set the pest classification value corresponding to the pest group information when the pest group information including the primary classification result exists, as the secondary classification result .

According to another exemplary embodiment, an input unit receives an image of a pest classification target image; A first calculation unit for calculating a primary classification result for the image input using a convolutional neural network; And a second calculation unit for calculating a secondary classification result for the image using the primary classification result and the predetermined pest group information, wherein the composite neural network calculates a composite product operation between the image and a predetermined filter A convolutional layer for generating a feature map through a feature map and applying a mask of a predetermined size to each pixel of the feature map and using subsampling for the feature map using pixel values in the mask a plurality of pairs of pulling layers for performing a plurality of sub sampling operations are repeated; And a feature classification layer group including a plurality of fully-connected layers for calculating the primary classification result from pixel values of a plurality of feature maps output from the feature extraction layer group, An apparatus for diagnosing a pest of a crop is provided.

The apparatus may further include a preprocessor for preprocessing the image input through the input unit.

The pre-processing unit may resize the input image to a predetermined size, and apply a hum mod filter to the resized image.

The resultant product layer may perform the product multiply operation using a Reed Linear Hidden Unit (ReLU) function.

The pooling layer may be a max pooling layer that outputs a maximum value of pixels in the mask.

The feature extraction layer group may be configured to generate a plurality of feature maps having a size that can be input to the foreground combining layer by repeatedly applying a pair of the composite product layer and the pooling layer to the image a plurality of times.

The mask size of the first pulling layer among the plurality of pulling layers included in the feature extraction layer group may be set to be larger than the mask size of the remaining pulling layers.

Wherein the feature classification layer group comprises: a first full combing layer for inputting pixel values of a plurality of feature maps output from the feature extraction layer group; A second prestraining layer connected to the first prestraining layer; And an output layer connected to the second pre-bonding layer.

The first calculation unit may apply a dropout parameter preset to only the second full-joining layer among the feature classification layer groups.

The output layer may be configured to calculate an output value for each node of the output layer using a hyperbolic tangent activation function.

The second calculation unit may set the pest classification value corresponding to the pest group information as the secondary classification result when the pest group information including the primary classification result exists.

According to the embodiments of the present invention, it is possible to effectively support the control of pests by diagnosing the exact symptom of the crops using the image data of the damaged crops or the damage suffered by the insects.

1 is a block diagram illustrating and illustrating a computing environment 10 including a computing device suitable for use in the exemplary embodiments.
2 is a flowchart illustrating a learning process 200 for a pest diagnosis of a crop according to an embodiment of the present invention.
FIG. 3 is a flow chart for describing image analysis step 208 in detail according to an embodiment of the present invention.
4 is an exemplary diagram for explaining a calculation process of a composite product layer according to an embodiment of the present invention.
5 is an exemplary diagram for explaining a calculation process of a pulling layer according to an embodiment of the present invention.
6 is an exemplary view for explaining a front bonding layer according to an embodiment of the present invention.
7 is a flow chart for explaining a process 700 for diagnosing a pest of a crop according to an embodiment of the present invention.
8 is a block diagram for explaining an apparatus 800 for diagnosing a pest of a crop according to an embodiment of the present invention.
9 is a block diagram for explaining a pest-insect detection system 900 of a crop according to an embodiment of the present invention.

Hereinafter, specific embodiments of the present invention will be described with reference to the drawings. The following detailed description is provided to provide a comprehensive understanding of the methods, apparatus, and / or systems described herein. However, this is merely an example and the present invention is not limited thereto.

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS Hereinafter, exemplary embodiments of the present invention will be described in detail with reference to the accompanying drawings. In the following description, well-known functions or constructions are not described in detail since they would obscure the invention in unnecessary detail. The following terms are defined in consideration of the functions of the present invention, and may be changed according to the intention or custom of the user, the operator, and the like. Therefore, the definition should be based on the contents throughout this specification. The terms used in the detailed description are intended only to describe embodiments of the invention and should in no way be limiting. Unless specifically stated otherwise, the singular form of a term includes plural forms of meaning. In this description, the expressions "comprising" or "comprising" are intended to indicate certain features, numbers, steps, operations, elements, parts or combinations thereof, Should not be construed to preclude the presence or possibility of other features, numbers, steps, operations, elements, portions or combinations thereof.

1 is a block diagram illustrating and illustrating a computing environment 10 that includes a computing device suitable for use in exemplary embodiments. In the illustrated embodiment, each of the components may have different functions and capabilities than those described below, and may include additional components in addition to those described below.

The illustrated computing environment 10 includes a computing device 12. In one embodiment, the computing device 12 may be a pest diagnostic device of a crop according to embodiments of the present invention. The computing device 12 includes at least one processor 14, a computer readable storage medium 16, The processor 14 may cause the computing device 12 to operate in accordance with the exemplary embodiment discussed above. For example, processor 14 may execute one or more programs stored on computer readable storage medium 16. The one or more programs may include one or more computer-executable instructions, which when executed by the processor 14 cause the computing device 12 to perform operations in accordance with the illustrative embodiment .

The computer-readable storage medium 16 is configured to store computer-executable instructions or program code, program data, and / or other suitable forms of information. The program 20 stored in the computer-readable storage medium 16 includes a set of instructions executable by the processor 14. In one embodiment, the computer-readable storage medium 16 may be any type of storage medium such as a memory (volatile memory such as random access memory, non-volatile memory, or any suitable combination thereof), one or more magnetic disk storage devices, Memory devices, or any other form of storage medium that can be accessed by the computing device 12 and store the desired information, or any suitable combination thereof.

Communication bus 18 interconnects various other components of computing device 12, including processor 14, computer readable storage medium 16.

The computing device 12 may also include one or more input / output interfaces 22 and one or more network communication interfaces 26 that provide an interface for one or more input / output devices 24. The input / output interface 22 and the network communication interface 26 are connected to the communication bus 18. The input / output device 24 may be connected to other components of the computing device 12 via the input / output interface 22. The exemplary input and output device 24 may be any type of device, such as a pointing device (such as a mouse or trackpad), a keyboard, a touch input device (such as a touch pad or touch screen), a voice or sound input device, An input device, and / or an output device such as a display device, a printer, a speaker, and / or a network card. The exemplary input and output device 24 may be included within the computing device 12 as a component of the computing device 12 and may be implemented as a separate device distinct from the computing device 12, Lt; / RTI >

Embodiments of the present invention are directed to receiving image data of crops suffering from pests or damages related to pests and diagnosing the exact symptoms of the crops through analysis of the images. The image data may be, for example, photograph data photographed on a leafing, stalk or fruit part of a diseased crop, photographic data of a pest, or photographic data of a pest. In the embodiments of the present invention, the image data is analyzed through a deep learning method, and it is determined through the analysis result whether the image data is related to which pests. Specifically, in the embodiments of the present invention, supervised learning is used to learn photographic images related to various diseases and injuries, and the pests are diagnosed based on the images.

2 is a flowchart illustrating a learning process 200 for a pest diagnosis of a crop according to an embodiment of the present invention. 2 may be performed by a computing device 12 having, for example, one or more processors and a memory for storing one or more programs executed by the one or more processors. In the illustrated flow chart, the method is described as being divided into a plurality of steps, but at least some of the steps may be performed in reverse order, combined with other steps, performed together, omitted, divided into detailed steps, One or more steps may be added and performed.

In step 202, the computing device 12 receives the pest classification learning learning image. As described above, the pest-screening classification image may be photographic data of a diseased state appearing on leaves, stalks or fruit parts of a diseased crop, or photographic data of a pest's top or side profile photographed.

In step 204, the computing device 12 performs a preprocessing operation on the input image. First, the computing device 12 may resize the size of the input image to a predetermined size. For example, the computing device 12 may be configured to resize the size of the input image to a size of 128x128 pixels. However, the size of the image to be resized may be appropriately determined in consideration of various factors such as the characteristics of the original image to be sorted, the processing performance of the computing device 12, or the number of the pests to be classified .

The computing device 12 then applies a pre-processing filter, such as a homomorphic filter, to the resized image to reduce the degree of influence of sunlight on the original image.

In step 206, the computing device 12 labels the preprocessed image. As described above, the method for diagnosing a pest of a crop according to an embodiment of the present invention is based on supervised learning. In order to learn an image, a correct answer to the image must be input through a labeling process.

In one embodiment, the label may consist of a positive integer starting from 0 or 1, and each label may be associated with a type of pest damage to be studied, respectively. Further, in embodiments of the present invention, the computing device 12 may have taken different parts of the crop, even in the same pest or stigma, or may give different labels if taken at different angles. For example, even though the same kind of disease, the patterns appearing on the leaves, stems, and fruits are all different, and classifying them into the same label makes the learning of the image very difficult or impossible. Likewise, in the case of pests, the appearance of the pests seen from above, and the appearance of the pests from the side are not so easy to judge by the same pests which are very different in shape. Therefore, in the embodiment of the present invention, different labels are given to the leaves, stems, fruits, pests of the crops where the disease has occurred, and the like. In addition, a plurality of labels corresponding to the same disease or insect pests can be grouped into one pest group, and then each image obtained by photographing leaves, stems, or fruits in the pest diagnosis process can be judged as the same disease Respectively. Likewise, in the case of insect pests, labels corresponding to the top and side profile of pests are grouped into the same pest group so that images taken on the top and side faces of pests can be judged as the same insect.

Next, in step 208, the computing device 12 performs an analysis on the labeled image. In one embodiment, the computing device 12 is configured to perform an analysis on the labeled image using a convolutional neural network. The composite neural network is largely composed of a feature extraction layer group and a feature classification layer group. Specifically, the feature extraction layer group includes a convolutional layer that generates a feature map by performing a product multiply operation between an input image and a predetermined filter (or mask), and a convolutional layer that is preset in each pixel of the feature map And a plurality of pairs of pulling layers for performing sub-sampling on the feature map using pixel values in the mask are repeated. The feature classification layer group is configured to include a plurality of fully-connected layers for calculating a classification result from each pixel value of a plurality of feature maps output from the feature extraction layer group. Hereinafter, an image analysis process according to an embodiment of the present invention will be described in detail.

FIG. 3 is a flow chart for explaining image analysis step 208 in detail according to an embodiment of the present invention. As described above, the image analysis step 208 according to an exemplary embodiment of the present invention may include a feature extraction layer group 302 and a feature classification layer group 304. [

The feature extraction layer group 302 is configured such that a plurality of pairs of a convolutional layer and a pulling layer are repeated. For example, as shown in FIG. 3, the feature extraction layer group 302 may include a first synthesis layer 306, a first pooling layer 308, and a second synthesis layer 306 so that the composite product layer and the pooling layer pair are repeated four times, A second pooling layer 312, a third composite product layer 314, a third pooling layer 316, a fourth composite product layer 318, a fourth pooling layer 320 ). ≪ / RTI > The number of iterations of the composite product layer and the pooling layer is determined by the size of the image input to the feature extraction layer group 302, the size of the mask applied to the operation, or the size of the output feature map And the like. In other words, it is noted that the scope of right of the present invention is not limited to the specific number of repetitions.

The composite product layer is a layer that repeatedly performs a process of outputting a new pixel value by multiplying the pixel value of the input image by the same filter having a size of 3x3, 5x5, and so on.

FIG. 4 is a diagram illustrating an operation process of a composite product layer according to an embodiment of the present invention. Referring to FIG. In the illustrated embodiment, for example, assume that a composite product operation is performed using a filter 406 as shown, for a pixel 404 at the (1, 1) position of the input image 402. Then, the computing device 12 multiplies the nine pixels corresponding to the filter size by the count value of the filter with the calculation target pixel 404 as a center, taking the filter size (3x3) into account, and adds the value to the new value . That is, the value of the (1, 1) pixel of the feature map 408 derived as a result of the convolution operation is

(1 * 0) + (1 * 0) + (0 * 0) + (0 * 0) + * (- 4)) = - 8

Can be calculated as follows. In the embodiment of the present invention, ReLU (Rectified Linear Hidden Unit) function is applied to the composite product layer, and the value of (1, 1) pixel of the actual feature map becomes ReLU (-8) = 0. The computing device 12 performs a convolution operation by repeating this process for all the pixels.

Meanwhile, in the embodiments of the present invention, the composite product layer can be configured to apply zero-padding for the product multiply operation. In order to solve the phenomenon of absence of peripheral pixels for performing a product multiply operation with a filter in the case of edge pixels, zero padding means performing a product multiply operation on the assumption that a pixel having a value of 0 is present at the edge . When zero padding is applied in this manner, the feature maps generated through the input image and the composite product operation become equal in size. The feature map, which is the result of performing the composite product operation, includes only the main features of the input image. The feature map is passed to the next layer, the pooling layer.

FIG. 5 is a diagram illustrating an operation process of a pulling layer according to an embodiment of the present invention. Referring to FIG. As shown, in the pooling layer, a mask is applied to each pixel of the feature map generated in the composite product layer, and the largest or average value among the values in the mask is set as a result value. This is for reducing the size of the feature map and selecting only the strongest stimulus among the plurality of stimuli to be transmitted to the next layer. As a pooling method of the pooling layer, there is a Max pooling method of setting the largest value in the mask as an output, and an average pooling method of outputting an average value of the values. In one embodiment, the computing device 12 may be configured to apply a Max pooling method to the pooling layer. In this way, when applying the max pooling, the calculation time can be shortened as compared with the average pooling.

(0, 0), (0, 1), (1, 0), (1, 1) of the input image 502 when the size of the mask is 2x2 and max pooling is applied ) Pixel is 6, the (0, 0) value of the output image 504 is 6. Likewise, since the maximum value of the (0, 2), (0, 3), (1, 2), (1, 3) pixels of the input image 502 is 8, The value is set to 8. The remaining pixels are subjected to the same process to perform a pooling operation.

In the feature extraction layer group 302, a plurality of feature maps are generated by repeatedly applying a pair of the composite product layer and the pooling layer to the image a plurality of times. At this time, the number of the filters or masks to be applied to each of the composite product layer and the pooling layer can be appropriately set in consideration of the characteristics of the classification object image and the like. For example, a 7x7x6 filter is used in the first composite product layer 306, a 5x5x16 filter is used in the second composite product layer 310, a 3x3x32 filter is used in the third composite product layer 314, ), A composite product operation can be performed using a 3x3x120 filter.

In the case of the pooling layer, the mask size of the pooling layer in the first stage among the plurality of pooling layers included in the feature extraction layer group 302 may be set to be larger than the mask size of the remaining pooling layers. For example, a 4x4 mask is applied to the first pulling layer 308, and a 2x2 mask is applied to the second pooling layer 312, the third pooling layer 316, and the fourth pooling layer 320 . When the mask size of the first stage is set to be larger than that of the other layers, the amount of calculation can be effectively reduced in the early stage of image analysis.

If the pair of the composite product layer and the pooling layer is repeatedly applied to the image a plurality of times, the size of the feature map is reduced and only a comprehensive feature that is strong and representative of the whole remains. Specifically, when an image is passed through a plurality of layers of the feature extraction layer group 302, a plurality of feature maps having sizes that can be input to the foreground combination layer in the feature classification layer group 304 are generated. For example, the feature extraction layer group 302 may be configured to generate a plurality of feature maps having a pixel size of 4x4 or less (e.g., 4x4 or 1x1, etc.) through the plurality of layers.

Next, the feature classification layer group 304 includes a plurality of fully-connected layers for calculating the classification result for the input image from each pixel value of the plurality of feature maps output from the feature extraction layer group 302, . Specifically, the feature classification layer group 304 includes a first pre-combination layer 322 for inputting pixel values of a plurality of feature maps output from the feature extraction layer group 302, A second pre-bonding layer 324 having the same number of nodes as the first pre-bonding layer 322, and an output layer 326 connected to the second pre-bonding layer.

In the embodiments of the present invention, a fully-connected layer means a node for performing image classification in a state where all the nodes of the previous step and all the nodes of the next layer are pre-combined with each other. FIG. 6 is an exemplary view for explaining a total bonding layer according to an embodiment of the present invention. FIG. In the illustrated embodiment, when there are three nodes on the left side and two nodes on the right side, the nodes are all combined. In this case, w ₁₁ , w ₁₂ , w ₁₃ , w ₂₁ , w ₂₂ , and w ₂₃ are weight values between connected nodes. In the learning process, the weight of each node and the bias of each node are adjusted through a process of comparing the learning result of the image and the correct answer, which will be described in detail below. The values of the output nodes y ₁ and y ₂ in the full-joining layer as shown in FIG. 6 are determined by the following Equation (1).

[Equation 1]

The first full combiner layer 322 has pixel values of the feature map output from the fourth pooling layer 320 as input nodes and weight values are applied to the input values in the same manner as in Equation (1) Layer < / RTI > The second pre-bonding layer 322 has the same number of nodes as the first pre-bonding layer 322, but a predetermined drop out parameter is applied, unlike the first pre-bonding layer 322. That is, in the embodiment of the present invention, a two-stage pre-bonding layer is included, and a dropout is applied only to the front bonding layer at the rear end. Dropout means a learning method to prevent overfitting that may occur in the learning process by performing learning by omitting some nodes at random. In the embodiments of the present invention, the overflow sum problem described above is solved by applying dropout to the two-level pre-bonding layer, and the weight of all the nodes is computed without applying dropout in the first- To prevent the learning accuracy from dropping due to dropout. The dropout parameter may be set appropriately in consideration of characteristics of an image to be classified, and a dropout parameter between 0.3 and 0.7 may be set, for example.

An output layer 326 is located at the rear end of the second pre-bonding layer 324. In the embodiment of the present invention, each node (class) of the output layer 326 is configured to correspond to the label set in step 206. In addition, hyperbolic tangent (tanh) function is applied as the activation function of the output layer.

The hyperbolic tangent function has a range of -1 to 1, and the closer the input image is to the class, the closer to 1, and vice versa. In the case of general classification functions such as Softmax functions, the class with the highest probability among each class is set as the result value. However, in the case of such a classification function, only the rank of the probability value of each class is considered, and the absolute probability value of each class is not considered. Therefore, even if a low probability value is derived for the entire class, the class having the highest value is uniformly returned as a result value. In addition, although there are common characteristics such as texture and color information even if the kinds of pests are different from each other, however, in the case of pest insects, there is no common feature because the range is comprehensive. Therefore, when Softmax is used as a classification function in determining whether a pest is pests, the probability of false detection increases. This is because Softmax judges that the non-pest is the most probable class.

In order to solve such a problem, the embodiment of the present invention is configured to apply the hyperbolic tangent function as the activation function in the output layer. In this case, the computing device 12 may determine the pest corresponding to the class having the highest hyperbolic tangent value of each class of the output layer as a pest matching with the corresponding image, like the soft max function. In addition, the computing device 12 may determine that the hyperbolic tangent of all classes is below a certain level and that the image is not associated with a pest. This is the biggest difference from softmax, which unconditionally returns the highest case even if the probability is low.

Again, back to FIG. 2, the computing device 12 may derive the classification result for the input learning image by performing an analysis on the labeled image as described above. Thereafter, in step 210, the computing device 12 compares the label (correct answer) given in step 206 with the classification result calculated in step 208 to adjust the analysis parameters. Specifically, the computing device 12 calculates a weight value for each node of the feature classification layer group 304 so that the cost function for the expected output (correct answer) and the actual output (classification result) the weight and the bias are adjusted repeatedly. As described above, the cost function is to obtain the difference between the expected output and the actual output when the input is input. The smaller the difference is, the better the learning is. In one embodiment, the computing device 120 may obtain parameters optimized for training data through backpropagation, which is a way of updating the weights and biases while propagating the error signal produced at the output to the opposite input side .

7 is a flowchart illustrating a process 700 for diagnosing a pest of a crop according to an embodiment of the present invention. In the pest diagnosis process 700 of the crop, the kind of pests is diagnosed from the image of the crop using the computing device 12 that has undergone the learning process described above. 7 may be performed by a computing device 12 having, for example, one or more processors and a memory for storing one or more programs executed by the one or more processors. In the illustrated flow chart, the method is described as being divided into a plurality of steps, but at least some of the steps may be performed in reverse order, combined with other steps, performed together, omitted, divided into detailed steps, One or more steps may be added and performed.

In step 702, the computing device 12 receives the pest diagnosis target image. The image of the pest diagnosis target image may be image data of a diseased state of a leaf, stem, or fruit of a diseased crop, or photograph data of a top or a side view of a pest.

In step 704, the computing device 12 performs a preprocessing operation on the input image. First, the computing device 12 may resize the size of the input image to a predetermined size. For example, the computing device 12 may be configured to resize the size of the input image to a size of 128x128 pixels. However, the size of the image to be resized may be appropriately determined in consideration of various factors such as the characteristics of the original image to be sorted, the processing performance of the computing device 12, or the number of the pests to be classified .

The computing device 12 then applies a pre-processing filter, such as a homomorphic filter, to the resized image to reduce the degree of influence of sunlight on the original image.

In step 706, the computing device 12 computes a primary classification result for the input image using a convolutional neural network. Specifically, in step 706, the computing device 12 is configured to calculate the primary classification result of the input image through the same process as step 208 and the image analysis process. The image analysis process has been described in detail above, and a repeated description thereof will be omitted here.

In step 708, the computing device 12 calculates the secondary classification result from the primary classification result in step 706. [ As described above, according to the embodiment of the present invention, different labels are given to the leaves, stems, fruits, pests, and tops of the diseased crops, and for a plurality of labels corresponding to the same disease or pests, And grouped into one pest group to be managed. Accordingly, the results of the primary classification result in different classification results depending on whether the image of the same pest is photographed with a leaf or a fruit. Therefore, in this step, when the pest group information including the primary classification result exists, the pest classification value corresponding to the pest group information is set as the secondary classification result.

FIG. 8 is a block diagram for explaining an apparatus 800 for diagnosing a pest of a crop according to an embodiment of the present invention. As shown, the apparatus for diagnosing a pest of a crop according to an embodiment of the present invention includes an input unit 802, a first calculation unit 804, and a second calculation unit 806.

The input unit 802 receives the pest classification target image.

The first calculation unit 804 calculates a primary classification result for the input image using a convolutional neural network. As described above, the composite neural network may include a feature extraction layer group and a feature classification layer group. The feature extraction layer group includes a convolutional layer that generates a feature map through a product multiply operation between the image and a predetermined filter, and a mask of a predetermined size to each pixel of the feature map And a plurality of pairs of pulling layers for performing sub-sampling on the feature map using pixel values in the mask may be repeated. The feature classification layer group may include a plurality of fully-connected layers for calculating the primary classification result from pixel values of a plurality of feature maps output from the feature extraction layer group. The above-described first-order classification process of the articulated-effect neural network and the image using the same is described above.

The second calculation unit 806 calculates a secondary classification result for the image using the primary classification result and the predetermined pest group information. Specifically, when the pest group information including the primary classification result exists, the second calculation unit 806 may set the pest classification value corresponding to the pest group information as the secondary classification result.

The apparatus for diagnosing a pest of a crop according to an embodiment of the present invention may further include a preprocessor (not shown) for preprocessing an image input through the input unit 802. [ The pre-processing unit may resize the input image to a predetermined size, and apply a call image filter to the resized image.

In one embodiment, the crop pest diagnosis device 800 may be implemented on a computing device that includes one or more processors and a computer readable recording medium coupled to the processors. The computer readable recording medium may be internal or external to the processor, and may be coupled to the processor by any of a variety of well known means. A processor in the computing device may cause each computing device to operate in accordance with the exemplary embodiment described herein. For example, a processor may execute instructions stored on a computer-readable recording medium, and instructions stored on the computer readable recording medium may cause a computing device to perform operations in accordance with the exemplary embodiments described herein For example.

9 is a block diagram for explaining a pest insect detection system 900 of a crop according to an embodiment of the present invention. The system for diagnosing a pest insect pest of a crop according to an embodiment of the present invention is a system for analyzing a pest related image of a crop received remotely from a client to discriminate kinds of pests and provide discrimination information to a client. As shown, the crop pest diagnosis system 900 according to an embodiment of the present invention includes a client 902 and a pest insect diagnosis server 904, and the client 902 and the pest- (906). In some embodiments, the network 906 may comprise one or more of the Internet, one or more local area networks, wire area networks, cellular networks, mobile networks, other types of networks, Combinations thereof.

The client 902 transmits an image necessary for the pest diagnosis to the pest detection server 904 and receives the diagnosis result corresponding to the image from the pest detection server 904. To this end, the client 902 may include image shooting means for shooting an image and output means for displaying the diagnosis result. Specifically, the client 902 shoots the leaves, stalks, or fruit parts of the damaged crop, photographed the top or side view of the pest, cuts out a background area or the like that is unnecessary for pest diagnosis in the photographed image crop pest diagnosis server 904 to the crop pest diagnosis server 904.

The pest-and-inspecting diagnosis server 904 receives the image from the client 902, diagnoses the type of the pest, and transmits the diagnosis result to the client 902. The process of diagnosing the kinds of pests has been described in detail in the foregoing, and a repeated explanation will be omitted here. The client 902 receiving the diagnosis result displays it on the screen so that the user can recognize the kind of the pest corresponding to the image.

On the other hand, an embodiment of the present invention may include a program for performing the methods described herein on a computer, and a computer-readable recording medium including the program. The computer-readable recording medium may include a program command, a local data file, a local data structure, or the like, alone or in combination. The media may be those specially designed and constructed for the present invention, or may be those that are commonly used in the field of computer software. Examples of computer-readable media include magnetic media such as hard disks, floppy disks and magnetic tape, optical recording media such as CD-ROMs and DVDs, and specifically configured to store and execute program instructions such as ROM, RAM, flash memory, Hardware devices. Examples of such programs may include machine language code such as those produced by a compiler, as well as high-level language code that may be executed by a computer using an interpreter or the like.

While the present invention has been particularly shown and described with reference to exemplary embodiments thereof, it is to be understood that the invention is not limited to the disclosed exemplary embodiments, but, on the contrary, . Therefore, the scope of the present invention should not be limited to the above-described embodiments, but should be determined by equivalents to the appended claims, as well as the appended claims.

302: Feature Extraction Layer Group
304: Feature Class Layer Group
306: first synthesis product layer
308: First pulling layer
310: second synthesis product layer
312: second pooling layer
314: third composite product layer
316: Third pulling layer
318: fourth composite product layer
320: Fourth pulling layer
322: first prime bond layer
324: second prestraining layer
326: Output layer
800: Plant pest diagnosis device
802: Input
804: first calculation unit
806: second calculation unit
900: Pest insect diagnosis system of crops
902: Client
904: Pest insect diagnosis server
906: Network

Claims (22)

One or more processors, and
A method performed in a computing device having a memory storing one or more programs executed by the one or more processors,
Receiving a pest classification target image;
Calculating a primary classification result for the input image using a convolutional neural network; And
And calculating a secondary classification result for the image using the primary classification result and the predetermined pest group information,
The composite-
A convolutional layer for generating a feature map through a product multiply operation between the image and a predetermined filter and a mask having a predetermined size for each pixel of the feature map, A feature extraction layer group in which a plurality of pairs of pulling layers for performing sub-sampling of the feature map are repeated using a feature extraction layer group; And
And a feature classification layer group including a plurality of fully-connected layers for calculating the primary classification result from pixel values of a plurality of feature maps output from the feature extraction layer group,
Wherein the step of calculating the secondary classification result is configured to set the pest classification value corresponding to the pest group information to the secondary classification result when the pest group information including the primary classification result is present .
The method according to claim 1,
Further comprising pre-processing the input image before performing the step of calculating the primary classification result.
The method of claim 2,
The pre-
Resizing the input image to a predetermined size; And
And applying a homomorphic filter to the resized image.
The method according to claim 1,
Wherein the composite product layer performs the composite product computation using a ReLU (Rectified Linear Hidden Unit) function.
The method according to claim 1,
Wherein the pooling layer is a max pooling layer that outputs a maximum value of pixels in the mask.
The method according to claim 1,
Wherein the feature extraction layer group includes:
Wherein the plurality of feature maps are configured to generate a plurality of feature maps having a pixel size that can be input to the foreground layer by repeatedly applying the composite product layer and the pooling layer pair to the image a plurality of times.
The method of claim 6,
Wherein the mask size of the pooling layer of the first stage among the plurality of pooling layers included in the feature extraction layer group is set to be larger than the mask size of the remaining pooling layers.
The method according to claim 1,
Wherein the feature classification layer group comprises:
A first prime-joining layer for inputting pixel values of a plurality of feature maps output from the feature extraction layer group;
A second prestraining layer connected to the first prestraining layer; And
And an output layer coupled to the second prestraining layer.
The method of claim 8,
And to apply a dropout parameter preset to only the second full-combine layer of the feature classification layer group.
The method of claim 8,
Wherein the output layer is configured to calculate an output value for each node of the output layer using a hyperbolic tangent activation function.
delete An input unit for receiving the pest classification target image;
A first calculation unit for calculating a primary classification result for the image input using a convolutional neural network; And
And a second calculation unit for calculating a secondary classification result for the image using the primary classification result and the predetermined pest group information,
The composite-
A convolutional layer for generating a feature map through a product multiply operation between the image and a predetermined filter and a mask having a predetermined size for each pixel of the feature map, A feature extraction layer group in which a plurality of pairs of pulling layers for performing sub-sampling of the feature map are repeated using a feature extraction layer group; And
And a feature classification layer group including a plurality of fully-connected layers for calculating the primary classification result from pixel values of a plurality of feature maps output from the feature extraction layer group,
And the second calculation unit sets the pest classification value corresponding to the pest group information when the pest group information including the primary classification result exists, as the secondary classification result.
The method of claim 12,
Further comprising a preprocessing unit for preprocessing the image input through the input unit.
14. The method of claim 13,
The pre-
Wherein the input image is resized to a predetermined size, and a reshaped image is subjected to a homomorph filter.
The method of claim 12,
Wherein the composite product layer performs the product multiplication operation using a ReLU (Rectified Linear Hidden Unit) function.
The method of claim 12,
Wherein the pooling layer is a max pooling layer for outputting a maximum value of pixels in the mask.
The method of claim 12,
Wherein the feature extraction layer group includes:
Wherein the plurality of feature maps are configured to generate a plurality of feature maps each having a size that can be input to the foreground layer by repeatedly applying a pair of the composite product layer and the pooling layer to the image a plurality of times.
18. The method of claim 17,
Wherein the mask size of the first pulling layer among the plurality of pulling layers included in the feature extraction layer group is set to be larger than the mask size of the remaining pooling layers.
The method of claim 12,
Wherein the feature classification layer group comprises:
A first prime-joining layer for inputting pixel values of a plurality of feature maps output from the feature extraction layer group;
A second prestraining layer connected to the first prestraining layer; And
And an output layer connected to the second pre-bonding layer.
The method of claim 19,
And to apply a dropout parameter preset to only the second pre-bonding layer of the feature classification layer group.
The method of claim 19,
Wherein the output layer is configured to calculate an output value for each node of the output layer using a hyperbolic tangent activation function.
delete

Images