CN116597312A - Crop leaf disease and pest identification method based on small sample image semantic segmentation - Google Patents

Crop leaf disease and pest identification method based on small sample image semantic segmentation Download PDF

Info

Publication number
CN116597312A
CN116597312A CN202310577254.9A CN202310577254A CN116597312A CN 116597312 A CN116597312 A CN 116597312A CN 202310577254 A CN202310577254 A CN 202310577254A CN 116597312 A CN116597312 A CN 116597312A
Authority
CN
China
Prior art keywords
feature map
images
query set
support set
image
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN202310577254.9A
Other languages
Chinese (zh)
Inventor
李雅琴
王丹丹
袁操
曾山
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Wuhan Polytechnic University
Original Assignee
Wuhan Polytechnic University
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Wuhan Polytechnic University filed Critical Wuhan Polytechnic University
Priority to CN202310577254.9A priority Critical patent/CN116597312A/en
Publication of CN116597312A publication Critical patent/CN116597312A/en
Pending legal-status Critical Current

Links

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V20/00—Scenes; Scene-specific elements
    • G06V20/10—Terrestrial scenes
    • G06V20/188—Vegetation
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/0464—Convolutional networks [CNN, ConvNet]
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • G06N3/082—Learning methods modifying the architecture, e.g. adding, deleting or silencing nodes or connections
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/20—Image preprocessing
    • G06V10/26—Segmentation of patterns in the image field; Cutting or merging of image elements to establish the pattern region, e.g. clustering-based techniques; Detection of occlusion
    • G06V10/267—Segmentation of patterns in the image field; Cutting or merging of image elements to establish the pattern region, e.g. clustering-based techniques; Detection of occlusion by performing operations on regions, e.g. growing, shrinking or watersheds
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/40—Extraction of image or video features
    • G06V10/44—Local feature extraction by analysis of parts of the pattern, e.g. by detecting edges, contours, loops, corners, strokes or intersections; Connectivity analysis, e.g. of connected components
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
    • G06V10/74—Image or video pattern matching; Proximity measures in feature spaces
    • G06V10/761—Proximity, similarity or dissimilarity measures
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
    • G06V10/764—Arrangements for image or video recognition or understanding using pattern recognition or machine learning using classification, e.g. of video objects
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
    • G06V10/77—Processing image or video features in feature spaces; using data integration or data reduction, e.g. principal component analysis [PCA] or independent component analysis [ICA] or self-organising maps [SOM]; Blind source separation
    • G06V10/774—Generating sets of training patterns; Bootstrap methods, e.g. bagging or boosting
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
    • G06V10/82—Arrangements for image or video recognition or understanding using pattern recognition or machine learning using neural networks
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V20/00—Scenes; Scene-specific elements
    • G06V20/70—Labelling scene content, e.g. deriving syntactic or semantic representations

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Multimedia (AREA)
  • Health & Medical Sciences (AREA)
  • General Health & Medical Sciences (AREA)
  • Evolutionary Computation (AREA)
  • Computing Systems (AREA)
  • Artificial Intelligence (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Software Systems (AREA)
  • Databases & Information Systems (AREA)
  • Medical Informatics (AREA)
  • Computational Linguistics (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Biomedical Technology (AREA)
  • Biophysics (AREA)
  • Data Mining & Analysis (AREA)
  • Molecular Biology (AREA)
  • General Engineering & Computer Science (AREA)
  • Mathematical Physics (AREA)
  • Image Analysis (AREA)

Abstract

The invention discloses a method for identifying crop leaf diseases and insect pests based on small sample image semantic segmentation, which is applied to the technical field of identifying crop leaf diseases and insect pests and comprises the following steps: collecting images of crop disease and insect pest leaves, dividing the images into two groups of images of a support set and a query set, and carrying out image segmentation labeling on the images in the support set to obtain a support set with labels and a query set without labels; carrying out feature extraction on images in a support set and a query set by adopting a pre-trained deep learning network model to obtain a feature map of the support set and a feature map of the query set; using a dense comparison module to carry out dense comparison on the support set feature map and the query set feature map so as to obtain a preliminary segmentation image of the query set; and optimizing the preliminary segmentation image of the query set by using an iterative optimization module to obtain an accurate segmentation image of the query set. The method can realize the identification of crop leaf diseases and insect pests by using a very small number of marked images.

Description

Crop leaf disease and pest identification method based on small sample image semantic segmentation
Technical Field
The invention relates to the technical field of crop leaf disease and insect pest identification, in particular to a method for identifying crop leaf disease and insect pest based on small sample image semantic segmentation.
Background
Small sample size is a significant challenge in agricultural image segmentation because acquiring large amounts of annotation data for training is often impractical and time consuming. However, accurate image segmentation is critical for various agricultural applications such as crop monitoring, yield estimation and disease diagnosis. Conventional supervised deep learning based segmentation methods can achieve good results when trained on large amounts of marker data. However, training of these networks on agronomic images is often difficult to achieve for two reasons: a sufficient amount of expert annotation data is lacking for training because these annotations require not only relevant lesion knowledge but also cost and time; in addition, it is impractical to train a new, specific model to cover every invisible topic, regardless of the number of agricultural plant lesion segmentation topics.
In recent years, development of a deep learning method provides a new idea for plant pest detection. The plant disease and pest detection method based on deep learning mainly comprises a Convolutional Neural Network (CNN) based method, a cyclic neural network (RNN) based method, an image segmentation based method and the like. The convolutional neural network method is widely applied to the segmentation task of plant diseases and insect pests, and remarkable performance improvement is achieved. However, acquiring large amounts of marker data for training is often impractical and time consuming, and convolutional neural network models are prone to overfitting when faced with small sample data, thereby affecting the accuracy of the classification and segmentation results.
To solve this problem, researchers have begun to try to improve the performance of plant pest segmentation based on a small sample learning approach. Small sample learning has become a promising approach to agricultural image segmentation. For a never seen class, the small sample model can learn its features from a very small amount of several annotation data. Allowing the model to effectively segment new targets using limited labeling data without requiring retraining the model with large amounts of labeling data. If small sample learning is applied to an agronomic image, the user can effectively segment rare or new lesions using only a small number of labeled samples. The small sample learning method comprises a data enhancement technology, meta learning, transfer learning and the like. The data enhancement technology expands the scale and diversity of the training data set by performing operations such as transformation, clipping, rotation, noise addition and the like on the original image, thereby alleviating the problem of overfitting. Meta-learning improves the generalization ability of a model on new tasks by pre-training a generic model or optimizer on a large-scale dataset and fine-tuning or updating on a small sample dataset. Transfer learning reduces the amount of training data required by the target domain by leveraging knowledge already in the source domain to assist in learning related tasks in the target domain.
Therefore, the invention provides a method for identifying crop leaf diseases and insect pests based on semantic segmentation of small sample images, which solves the problems of the prior art and is a problem to be solved by the technicians in the field.
Disclosure of Invention
In view of the above, the invention provides a method for identifying crop leaf diseases and insect pests based on semantic segmentation of small sample images, and by using the method, the identification of crop leaf diseases and insect pests can be realized by using a very small number of labeling images.
In order to achieve the above object, the present invention provides the following technical solutions:
a crop leaf disease and insect pest identification method based on small sample image semantic segmentation comprises the following steps:
s1, acquiring images of crop disease and insect pest leaves, dividing the images into two groups of images of a support set and a query set, and carrying out image segmentation labeling on the images in the support set to obtain a support set with labels and a query set without labels;
s2, carrying out feature extraction on images in a support set and a query set by adopting a pre-trained deep learning network model to obtain a feature map of the support set and a feature map of the query set;
s3, performing intensive comparison on the support set feature map and the query set feature map by using a intensive comparison module to obtain a preliminary segmentation image of the query set;
and S4, optimizing the preliminary segmentation image of the query set by using an iterative optimization module to obtain an accurate query set segmentation image.
Optionally, in S1, the support set image is segmented by a Canny edge detector and the segmented regions are manually labeled.
Optionally, in S2, feature extraction is performed on the support set and query set images using a deep learning network model res net.
Alternatively, the network in ResNet is represented by the following mathematical model:
(1) Convolution layer: for the input image I and the convolution kernel K, the convolution operation can be expressed as:
wherein C (i, j) is an element of the ith row and jth column of the output feature map;
(2) Pooling layer: for the input feature map F, the pooling operation can be expressed as:
where P (i, j) is the element of the ith row and jth column of the output feature map, s is the pooling step size, m and n are the size of the pooling window;
(3) Residual error module: for the input feature map X and residual function F, the residual module can be expressed as:
Y＝F(X)+X
wherein Y is an output feature map and F (X) is a nonlinear function composed of a convolution layer and an activation function;
(4) Global average pooling layer: for the input feature map G, the global average pooling layer can be expressed as:
wherein v is c Is the c-th element of the output vector, H and W are the height and width of the input feature map, and c is the number of channels.
Optionally, in S3, the pest category of the query set is calculated by comparing the similarity between the support set feature map and the query set feature map.
Optionally, in the dense comparison module of S3, a cosine distance is used as a measure function of the similarity of the feature map, and for two vectors a, B in space, the remaining chord distances are:
wherein, represents the dot product, the A represents the modular length of the vector A, and the calculation formula is as follows
Optionally, in S3, the support set has multiple images, and then a more accurate support set feature map is obtained by using the attention mechanism module; the attention mechanism module consists of two convolution blocks, the first having 256 3 x 3 filters followed by a 3 x 3 max pooling layer; the second convolution block has a 3 x 3 convolution layer followed by a global averaging pooling layer; the output of the attention branch is λ, representing the weight of each support sample.
Optionally, the SoftMax function is used to normalize the weights of all the support samples so that their sum is 1, and the specific calculation formula is as follows:
compared with the prior art, the invention discloses a crop leaf disease and pest identification method based on small sample image semantic segmentation, which has the following beneficial effects:
(1) Adopting a step 1, using an edge detection algorithm based on a Canny operator, smoothing an image by using a Gaussian filter, calculating the gradient strength and direction of each pixel point in the image, carrying out non-maximum suppression on the gradient amplitude, and detecting by using a double-threshold algorithm to determine the real and potential edges. The Canny operator is insensitive to noise, can detect a real weak edge, and is a very effective edge detection method.
(2) And 2, performing feature extraction on images in a support set and a query set, performing feature extraction on a ResNet deep learning network, and compared with a traditional CNN network, the ResNet introduces the concept of residual blocks (residual blocks) which add input and output through skip connections (skip connections), so that the problem of gradient disappearance or explosion is avoided. Thus, resNet can build very deep feature extraction networks, thereby mining more image features.
(3) And 3, comparing the similarity of the support set feature map and the query set feature map by using the cosine distance to obtain a preliminary query set segmentation image. The cosine distance (Cosine Similarity) is the similarity between two vectors determined by measuring the cosine value of the angle between the two vectors. It is usually used for positive space and thus gives a value between 0 and 1. It represents a relative difference in direction, irrespective of absolute differences in value. Its advantage is high Euclidean distance in high-dimensional data.
(4) And 4, optimizing the edges of the preliminary divided images by using an iterative residual error connection method based on the preliminary divided images of the query set in the step 3 to obtain accurate query set divided images.
Drawings
In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings that are required to be used in the embodiments or the description of the prior art will be briefly described below, and it is obvious that the drawings in the following description are only embodiments of the present invention, and that other drawings can be obtained according to the provided drawings without inventive effort for a person skilled in the art.
Fig. 1 is a flowchart of a method for identifying crop leaf diseases and insect pests based on semantic segmentation of small sample images.
Detailed Description
The following description of the embodiments of the present invention will be made clearly and completely with reference to the accompanying drawings, in which it is apparent that the embodiments described are only some embodiments of the present invention, but not all embodiments. All other embodiments, which can be made by those skilled in the art based on the embodiments of the invention without making any inventive effort, are intended to be within the scope of the invention.
Referring to fig. 1, the invention discloses a crop leaf pest identification method based on small sample image semantic segmentation, which comprises the following steps:
s1, acquiring images of crop disease and insect pest leaves, dividing the images into two groups of images of a support set and a query set, and carrying out image segmentation labeling on the images in the support set to obtain a support set with labels and a query set without labels;
s2, carrying out feature extraction on images in a support set and a query set by adopting a pre-trained deep learning network model to obtain a feature map of the support set and a feature map of the query set;
s3, performing intensive comparison on the support set feature map and the query set feature map by using a intensive comparison module to obtain a preliminary segmentation image of the query set;
and S4, optimizing the preliminary segmentation image of the query set by using an iterative optimization module to obtain an accurate query set segmentation image.
Further, S1 is specifically:
since the pest areas supporting the collection image need to be marked, the surrounding areas of the pest areas need to be segmented by using an image segmentation algorithm. Edge-based image segmentation algorithms refer to the process of achieving segmentation by detecting edges of different regions in an image, typically using first or second derivatives to find where the pixel gray values change drastically, and then connecting the edge points to form a closed region. The image segmentation algorithm based on the edge comprises a Canny edge detector, a Sobel operator, a Laplacian operator, a watershed algorithm and the like.
The Canny operator is a non-differential edge detection operator, the image is smoothed by a Gaussian filter, the gradient strength and direction of each pixel point in the image are calculated, the gradient amplitude is subjected to non-maximum suppression, and the real and potential edges are determined by detection of a double-threshold algorithm. The Canny operator is insensitive to noise, can detect a real weak edge, and is a very effective edge detection method. The main steps of the Canny edge detector are as follows:
gaussian smoothing filtering: convolving the image with a gaussian filter kernel to generate the equation:
where x and y are the center points of the kernels and σ is the standard deviation.
Calculating gradient amplitude and direction: the gradient amplitude G and the direction theta of each pixel point in the image are calculated by using a first-order finite difference operator (such as a Sobel operator), and the calculation formula is as follows:
θ＝arctan(G y /G x )
wherein G is x And G y The gradient values in the X and Y directions can be obtained by convolving the image with a Sobel operator.
Let a window of n x n in the image be A and Sobel operator be S x ,S y All are the same as the a dimension. Then there are:
G x ＝S x *A,G y ＝S y *A
where is a convolution operator.
Non-maximum suppression: and judging the local maximum value of the gradient amplitude according to the gradient direction, and inhibiting non-maximum value points to obtain a refined edge image. The method comprises the following specific steps:
1) The gradient intensity of the current pixel is compared with two pixels along the positive and negative gradient directions.
2) If the gradient intensity of the current pixel is maximum compared to the other two pixels, the pixel point remains as an edge point, otherwise the pixel point will be suppressed.
Double threshold detection: screening the edge image by using double thresholds (a high threshold T1 and a low threshold T2), reserving strong edge points, excluding points lower than the low threshold, and connecting weak edge points between the high threshold point and the low threshold point to obtain a final edge image. The method comprises the following specific steps:
1) If the gradient value of the edge pixel is higher than the high threshold T1, marking the edge pixel as a strong edge pixel;
2) If the gradient value of the edge pixel is less than the high threshold T1 and greater than the low threshold T2, it is marked as a weak edge pixel;
3) If the gradient value of the edge pixel is smaller than the low threshold T2, it is suppressed.
Suppressing isolated low threshold points: by looking at the weak edge pixels and their 8 neighborhood pixels, the weak edge points can remain as true edges as long as one is a strong edge pixel. The method comprises the following specific steps:
1) Traversing all weak edge pixels;
2) For each weak edge pixel, checking whether 8 neighborhood pixels thereof have strong edge pixels;
3) If so, reserving the weak edge pixels; if not, the weak edge pixels are suppressed.
Further, S2 is specifically:
in the field of image recognition, deep learning is a very effective tool, where convolutional neural networks have excellent results in feature extraction. The convolution neural network carries out convolution on pixels on the image through different convolution kernels, and the different convolution kernels correspond to the features of different dimensions. Through the convolutional neural network, invisible features on the images can be extracted, so that the model can classify specific images through the image features. On this basis, resNet introduces the concept of residual blocks (skip links) that add input and output via skip connections, thus avoiding the problem of gradient extinction or explosion. Thus, resNet can build very deep feature extraction networks, thereby mining more image features. The specific feature extraction structure of ResNet is different according to different depths, and can be generally divided into the following parts:
a first layer: a 7 x 7 convolutional layer, step size 2, output channel number 64, followed by a 3 x 3 max pooling layer, step size 2.
A second layer: and a plurality of residual modules, wherein each residual module comprises two 3×3 convolution layers, the number of output channels is 64, and the step size is 1.
Third layer: and each residual module comprises three convolution layers, namely 1 multiplied by 1,3 multiplied by 3 and 1 multiplied by 1, the number of output channels is 128, and the step size is 2.
Fourth layer: and each residual module comprises three convolution layers, namely 1 multiplied by 1,3 multiplied by 3 and 1 multiplied by 1, the number of output channels is 256, and the step size is 2.
Fifth layer: and each residual module comprises three convolution layers, namely 1 multiplied by 1,3 multiplied by 3 and 1 multiplied by 1, the number of output channels is 512, and the step size is 2.
Sixth layer: and a global averaging pooling layer for converting the feature map of the last layer into a one-dimensional vector.
ResNet-18 has 2,2 residual modules, resNet-34 has 3,4,6,3 residual modules, and ResNet-50 has 3,4,6,3 residual modules. In particular, the network in its steps can be represented by the following mathematical model:
convolution layer: for the input image I and the convolution kernel K, the convolution operation can be expressed as:
where C (i, j) is an element of the ith row and jth column of the output feature map.
Pooling layer: for the input feature map F, the pooling operation can be expressed as:
where P (i, j) is the element of the ith row and jth column of the output feature map, s is the pooling step size, and m and n are the size of the pooling window.
Residual error module: for the input feature map X and residual function F, the residual module can be expressed as:
Y＝F(X)+X
where Y is the output signature and F (X) is a nonlinear function consisting of a convolution layer and an activation function.
Global average pooling layer: for the input feature map G, the global average pooling layer can be expressed as:
wherein v is c Is the c-th element of the output vector, H and W are the height and width of the input feature map, and c is the number of channels.
Further, S3 is specifically:
in the feature extraction module, the invention uses the same feature extraction module to extract the features of the support set and the query set, and obtains the feature map of the support set and the feature map of the query set. After that, the present invention uses a dense comparison module to perform dense comparisons of the support set feature map and the query set feature map. Because the support set is an image marked by image segmentation, the disease and pest category of the support set image is known, and the disease and pest category of the query set can be calculated by comparing the similarity between the feature map of the support set and the feature map of the query set. In particular, in the dense comparison module of the present invention, cosine Distance (Cosine Distance) is used as a measure function of similarity of feature images.
In comparing the similarity between features, the Euclidean distance (Euclidean Distance) is typically chosen as the comparison function. Euclidean distance refers to the actual distance between two points in n-dimensional space. The straight line distance between two points is calculated by the Pythagorean theorem. For two vectors a (x 1 ,x 2 ,…,x n ),B(y 1 ,y 2 ,…,y n ) The euclidean distance can be expressed as:
its advantages are simple process, and easy implementation. But it has the disadvantage of not being scale-invariant, which means that the calculated distance may be skewed in terms of units of elements. Typically, the data needs to be normalized before this distance metric can be used. Furthermore, as the dimension of the data increases, the Euclidean distance is less useful.
Unlike Euclidean distance, cosine distance (Cosine Similarity) is the similarity between two vectors determined by measuring the cosine value of the angle between the two vectors. It is usually used for positive space and thus gives a value between 0 and 1. It represents a relative difference in direction, irrespective of absolute differences in value. Its advantage is high Euclidean distance in high-dimensional data. But it has the disadvantage that the size of the vectors is not considered, but only their direction. For two vectors a, B in space, the remaining chordal distance is:
wherein, represents the dot product, the A represents the modular length of the vector A, and the calculation formula is as follows
Further, S4 is specifically:
firstly, taking an output characteristic diagram of a dense comparison module and a prediction mask of the last iteration as inputs, and combining the output characteristic diagram and the prediction mask of the last iteration through a residual connection, wherein the formula of the residual connection is as follows:
M t ＝x+F(x,y t-1 )
wherein x is the output characteristic of the dense comparison module; y is t-1 Is the prediction mask of the last iteration step, M t Is the output of the residual block, and the function F (·) is the feature x and the prediction mask y t-1 Is a cascade of (a) and (b).
Then, two common residual blocks and a hole space pyramid pooling module (ASPP) are used to capture multi-scale information, outputting a finer prediction mask.
And finally, taking the prediction mask as the input of the next iteration, and repeating the process until the preset iteration times are reached.
Specifically, for the third step, if the support set has multiple images, a more accurate support set feature map is obtained by using the attention mechanism module, and the steps specifically include:
when more than one image is supported centrally, the present invention employs an attention mechanism to fuse the results of comparisons of different support samples. Of course, simply averaging OR logically OR (Logical OR) the comparison results of the different support samples is not reasonable, as the variability and relevance of the different support samples to the query sample is ignored. Thus, the present invention adds an attention branch to the third step dense comparison module for calculating the weight of each support sample. The attention branch consists of two convolution blocks, the first having 256 3 x 3 filters followed by a 3 x 3 max pooling layer; the second convolution block has a 3 x 3 convolution layer followed by a global averaging pooling layer. The output of the attention branch is λ, representing the weight of each support sample. The weights of all support samples are then normalized here with the SoftMax function so that their sum is 1, whose calculation formula is:
finally, the characteristics of the different support samples are weighted and summed by the normalized weights, so that the final output characteristics are obtained.
In the present specification, each embodiment is described in a progressive manner, and each embodiment is mainly described in a different point from other embodiments, and identical and similar parts between the embodiments are all enough to refer to each other. For the device disclosed in the embodiment, since it corresponds to the method disclosed in the embodiment, the description is relatively simple, and the relevant points refer to the description of the method section.
The previous description of the disclosed embodiments is provided to enable any person skilled in the art to make or use the present invention. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the generic principles defined herein may be applied to other embodiments without departing from the spirit or scope of the invention. Thus, the present invention is not intended to be limited to the embodiments shown herein but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims (8)

1. The method for identifying crop leaf diseases and insect pests based on small sample image semantic segmentation is characterized by comprising the following steps of:
s1, acquiring images of crop disease and insect pest leaves, dividing the images into two groups of images of a support set and a query set, and carrying out image segmentation labeling on the images in the support set to obtain a support set with labels and a query set without labels;
s2, carrying out feature extraction on images in a support set and a query set by adopting a pre-trained deep learning network model to obtain a feature map of the support set and a feature map of the query set;
s3, performing intensive comparison on the support set feature map and the query set feature map by using a intensive comparison module to obtain a preliminary segmentation image of the query set;
and S4, optimizing the preliminary segmentation image of the query set by using an iterative optimization module to obtain an accurate query set segmentation image.
2. The method for identifying crop leaf diseases and insect pests based on small sample image semantic segmentation according to claim 1, wherein the method comprises the following steps of,
in S1, the support set image is segmented through a Canny edge detector, and the segmented areas are marked manually.
3. The method for identifying crop leaf diseases and insect pests based on small sample image semantic segmentation according to claim 1, wherein the method comprises the following steps of,
the feature extraction of the support set and query set images in S2 uses a deep learning network model res net.
4. A method for identifying crop leaf diseases and insect pests based on semantic segmentation of small sample images according to claim 3,
the network in ResNet is represented by the following mathematical model:
(1) Convolution layer: for the input image I and the convolution kernel K, the convolution operation can be expressed as:
wherein C (i, j) is an element of the ith row and jth column of the output feature map;
(2) Pooling layer: for the input feature map F, the pooling operation can be expressed as:
where P (i, j) is the element of the ith row and jth column of the output feature map, s is the pooling step size, m and n are the size of the pooling window;
(3) Residual error module: for the input feature map X and residual function F, the residual module can be expressed as:
Y＝F(X)+X
wherein Y is an output feature map and F (X) is a nonlinear function composed of a convolution layer and an activation function;
(4) Global average pooling layer: for the input feature map G, the global average pooling layer can be expressed as:
wherein v is c Is the c-th element of the output vector, H and W are the height and width of the input feature map, and c is the number of channels.
5. The method for identifying crop leaf diseases and insect pests based on small sample image semantic segmentation according to claim 1, wherein the method comprises the following steps of,
s3, calculating the plant diseases and insect pests of the query set by comparing the similarity between the feature map of the support set and the feature map of the query set.
6. The method for identifying crop leaf diseases and insect pests based on small sample image semantic segmentation according to claim 5, wherein the method comprises the following steps of,
in the dense comparison module of S3, the cosine distance is used as a measurement function of the similarity of the feature map, and for two vectors a, B in space, the rest chord distances are:
wherein, represents the dot product, the A represents the modular length of the vector A, and the calculation formula is as follows
7. The method for identifying crop leaf diseases and insect pests based on small sample image semantic segmentation according to claim 5, wherein the method comprises the following steps of,
in S3, a plurality of images are collected in the support set, and then a more accurate support set feature map is obtained by using an attention mechanism module; the attention mechanism module consists of two convolution blocks, the first having 256 3 x 3 filters followed by a 3 x 3 max pooling layer; the second convolution block has a 3 x 3 convolution layer followed by a global averaging pooling layer; the output of the attention branch is λ, representing the weight of each support sample.
8. The method for identifying crop leaf diseases and insect pests based on small sample image semantic segmentation according to claim 7, wherein the method comprises the following steps of,
the weights of all support samples are normalized using the SoftMax function in S3 so that their sum is 1, and the specific calculation formula is as follows:
CN202310577254.9A 2023-05-22 2023-05-22 Crop leaf disease and pest identification method based on small sample image semantic segmentation Pending CN116597312A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202310577254.9A CN116597312A (en) 2023-05-22 2023-05-22 Crop leaf disease and pest identification method based on small sample image semantic segmentation

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202310577254.9A CN116597312A (en) 2023-05-22 2023-05-22 Crop leaf disease and pest identification method based on small sample image semantic segmentation

Publications (1)

Publication Number Publication Date
CN116597312A true CN116597312A (en) 2023-08-15

Family

ID=87611239

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202310577254.9A Pending CN116597312A (en) 2023-05-22 2023-05-22 Crop leaf disease and pest identification method based on small sample image semantic segmentation

Country Status (1)

Country Link
CN (1) CN116597312A (en)

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN116994076A (en) * 2023-09-28 2023-11-03 中国海洋大学 Small sample image recognition method based on double-branch mutual learning feature generation
CN117541623A (en) * 2023-11-23 2024-02-09 中国水产科学研究院黑龙江水产研究所 Fish shoal activity track monitoring system

Cited By (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN116994076A (en) * 2023-09-28 2023-11-03 中国海洋大学 Small sample image recognition method based on double-branch mutual learning feature generation
CN116994076B (en) * 2023-09-28 2024-01-19 中国海洋大学 Small sample image recognition method based on double-branch mutual learning feature generation
CN117541623A (en) * 2023-11-23 2024-02-09 中国水产科学研究院黑龙江水产研究所 Fish shoal activity track monitoring system
CN117541623B (en) * 2023-11-23 2024-06-07 中国水产科学研究院黑龙江水产研究所 Fish shoal activity track monitoring system

Similar Documents

Publication Publication Date Title
Albattah et al. A novel deep learning method for detection and classification of plant diseases
Upadhyay et al. A novel approach for rice plant diseases classification with deep convolutional neural network
Mahum et al. A novel framework for potato leaf disease detection using an efficient deep learning model
Francis et al. Disease detection and classification in agricultural plants using convolutional neural networks—a visual understanding
Oo et al. Plant leaf disease detection and classification using image processing
Zhao et al. An effective automatic system deployed in agricultural Internet of Things using Multi-Context Fusion Network towards crop disease recognition in the wild
Veerendra et al. Detecting plant diseases, quantifying and classifying digital image processing techniques
CN116597312A (en) Crop leaf disease and pest identification method based on small sample image semantic segmentation
Anandhakrishnan et al. Deep Convolutional Neural Networks for image based tomato leaf disease detection
CN115410088B (en) Hyperspectral image field self-adaption method based on virtual classifier
Pushpa et al. Plant disease detection and classification using deep learning model
CN111898621A (en) Outline shape recognition method
Nancy et al. Deep Learning and Machine Learning Based Efficient Framework for Image Based Plant Disease Classification and Detection
CN114155443A (en) Hyperspectral image classification method based on multi-receptive-field attention network
Khan et al. Machine learning-based detection and classification of walnut fungi diseases
Barman et al. Smartphone assist deep neural network to detect the citrus diseases in agri-informatics
CN115393719A (en) Hyperspectral image classification method combining space spectral domain self-adaption and ensemble learning
Abisha et al. Application of image processing techniques and artificial neural network for detection of diseases on brinjal leaf
Sharifi et al. A new deep learning approach for classification of hyperspectral images: Feature and decision level fusion of spectral and spatial features in multiscale CNN
Padmanabhuni et al. An extensive study on classification based plant disease detection system
Wakhare et al. Development of automated leaf disease detection in pomegranate using Alexnet algorithm
Khodaskar et al. Pattern recognition: Advanced development, techniques and application for image retrieval
Bhadur et al. Agricultural crops disease identification and classification through leaf images using machine learning and deep learning technique: a review
Kundur et al. Insect pest image detection and classification using deep learning
Sreedevi et al. Development of weighted ensemble transfer learning for tomato leaf disease classification solving low resolution problems

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination