CN114965501A - Peanut disease detection and yield prediction method based on canopy parameter processing - Google Patents

Peanut disease detection and yield prediction method based on canopy parameter processing Download PDF

Info

Publication number
CN114965501A
CN114965501A CN202210561268.7A CN202210561268A CN114965501A CN 114965501 A CN114965501 A CN 114965501A CN 202210561268 A CN202210561268 A CN 202210561268A CN 114965501 A CN114965501 A CN 114965501A
Authority
CN
China
Prior art keywords
peanut
disease
data
canopy
image
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN202210561268.7A
Other languages
Chinese (zh)
Inventor
尚书旗
王海清
王东伟
何晓宁
岳丹松
申世龙
朱浩
李成鹏
王悦涛
张梅
董�成
张春晓
谭营
纪瑞琪
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Qingdao Agricultural University
Original Assignee
Qingdao Agricultural University
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Qingdao Agricultural University filed Critical Qingdao Agricultural University
Priority to CN202210561268.7A priority Critical patent/CN114965501A/en
Publication of CN114965501A publication Critical patent/CN114965501A/en
Pending legal-status Critical Current

Links

Images

Classifications

    • G—PHYSICS
    • G01—MEASURING; TESTING
    • G01N—INVESTIGATING OR ANALYSING MATERIALS BY DETERMINING THEIR CHEMICAL OR PHYSICAL PROPERTIES
    • G01N21/00—Investigating or analysing materials by the use of optical means, i.e. using sub-millimetre waves, infrared, visible or ultraviolet light
    • G01N21/84—Systems specially adapted for particular applications
    • G01N21/88—Investigating the presence of flaws or contamination
    • G01N21/95—Investigating the presence of flaws or contamination characterised by the material or shape of the object to be examined
    • G—PHYSICS
    • G01—MEASURING; TESTING
    • G01B—MEASURING LENGTH, THICKNESS OR SIMILAR LINEAR DIMENSIONS; MEASURING ANGLES; MEASURING AREAS; MEASURING IRREGULARITIES OF SURFACES OR CONTOURS
    • G01B11/00—Measuring arrangements characterised by the use of optical techniques
    • G—PHYSICS
    • G01—MEASURING; TESTING
    • G01B—MEASURING LENGTH, THICKNESS OR SIMILAR LINEAR DIMENSIONS; MEASURING ANGLES; MEASURING AREAS; MEASURING IRREGULARITIES OF SURFACES OR CONTOURS
    • G01B11/00—Measuring arrangements characterised by the use of optical techniques
    • G01B11/28—Measuring arrangements characterised by the use of optical techniques for measuring areas
    • G—PHYSICS
    • G01—MEASURING; TESTING
    • G01N—INVESTIGATING OR ANALYSING MATERIALS BY DETERMINING THEIR CHEMICAL OR PHYSICAL PROPERTIES
    • G01N21/00—Investigating or analysing materials by the use of optical means, i.e. using sub-millimetre waves, infrared, visible or ultraviolet light
    • G01N21/17—Systems in which incident light is modified in accordance with the properties of the material investigated
    • G01N21/25—Colour; Spectral properties, i.e. comparison of effect of material on the light at two or more different wavelengths or wavelength bands
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/044—Recurrent networks, e.g. Hopfield networks
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/045—Combinations of networks
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/048—Activation functions
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/40—Extraction of image or video features
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
    • G06V10/74—Image or video pattern matching; Proximity measures in feature spaces
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
    • G06V10/82—Arrangements for image or video recognition or understanding using pattern recognition or machine learning using neural networks
    • Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
    • Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
    • Y02A—TECHNOLOGIES FOR ADAPTATION TO CLIMATE CHANGE
    • Y02A40/00—Adaptation technologies in agriculture, forestry, livestock or agroalimentary production
    • Y02A40/10—Adaptation technologies in agriculture, forestry, livestock or agroalimentary production in agriculture

Abstract

The invention discloses a peanut disease detection and yield prediction method based on canopy parameter processing, which comprises the steps of collecting hyperspectral images and leaf images of peanut in seedling stage, flowering stage and fruiting stage, extracting characteristic images of peanuts in different growth stages, calculating a normalized vegetation index and a leaf area index, and carrying out correlation analysis on the normalized vegetation index and the leaf area index; establishing a peanut leaf disease rapid detection model, and identifying the type and position of the disease of the peanut leaves in the picture; and finally, inputting the normalized vegetation index, the leaf area index, the correlation coefficient of the leaf area index and the disease category and position data into a CNN-LSTM network, establishing a peanut canopy parameter correlation analysis and yield prediction model, and outputting a peanut yield prediction value. The method utilizes an image processing technology and a data analysis method to monitor and predict the growth process of the peanuts from the aspects of the yield, the disease condition and the like of the peanuts, and achieves the purposes of obtaining the growth condition of the crops and guiding agricultural production through data analysis.

Description

Peanut disease detection and yield prediction method based on canopy parameter processing
Technical Field
The invention belongs to the field of peanut canopy parameter processing and application research, and particularly relates to a peanut disease detection and yield prediction method based on canopy parameter processing.
Background
The peanuts are one of important high-quality oil crops in China, and the improvement of the yield and the quality of the peanuts has important significance. Crop diseases are one of the direct threats to food safety, and the agricultural loss caused by diseases and insect pests is up to 2500 billion dollars each year around the world. Due to the influence of growth characteristics and external environmental factors, the roots and leaves of peanuts are easily infected by viruses, and the problems of slow growth, serious diseases and the like occur, so that the yield and the quality are reduced. Therefore, the method for monitoring the growth condition of the peanut crops and providing guidance for agricultural production is the key for improving the yield and the quality of the peanuts.
The traditional crop disease diagnosis method usually takes expert diagnosis and chemical tests as main parts, is limited by objective conditions such as experience, technology, time cost and the like, and has the problems of complex process, long consumed time, low recognition efficiency and precision and the like. Therefore, the difficulty is how to rapidly and accurately detect the type and the position of the peanut diseases and predict the yield of the peanut diseases. In the prior art, methods for detecting diseases by acquiring plant phenotypes and based on technologies such as image processing and the like are gradually emerging. However, most of the current plant phenotype acquisition modes are limited to hyperspectral characteristic data extraction or single-leaf disease detection and identification.
For example, in the aspect of hyperspectral characteristic data extraction, symptoms shown in different periods and different environments of the same disease may be different, and disease characteristics on plant leaves have the characteristics of small area, irregularity, uneven distribution and the like, so that a mature method for guiding agricultural production according to characteristic data extracted from hyperspectral images is lacked; in the aspect of disease identification, peanuts have different disease characteristics, and the differences are small, and the existing target detection model usually gives average attention to a whole picture, so that certain defects exist in the aspects of detection speed and precision of peanut diseases; in the aspects of data fusion and practical application, due to the fact that the dimension difference between hyperspectral characteristic data and disease characteristic data is large, accurate results are difficult to obtain in specific application, and therefore a method capable of achieving disease detection and yield prediction aiming at peanut canopy parameters is urgently needed to be provided.
Disclosure of Invention
The method comprises the steps of collecting images of peanuts in different growth periods, analyzing data such as normalized vegetation indexes and leaf area indexes of the peanuts in different growth periods by using an image processing and data analysis technology, detecting disease types and positions of peanut leaves by using a target identification technology, and carrying out peanut disease detection and yield prediction so as to provide reference and guidance for agricultural production.
The invention is realized by adopting the following technical scheme: a peanut disease detection and yield prediction method based on canopy parameter processing is characterized by comprising the following steps:
step A, collecting canopy spectral image information of a peanut seedling stage, a flowering stage and a fruiting stage and leaf image information of the peanut seedling stage, the flowering stage and the fruiting stage respectively, and recording collected time information;
b, based on the canopy spectral image information obtained in the step A, acquiring a normalized vegetation index and a leaf area index of the peanut through phenotypic characteristic processing, and performing correlation analysis on the normalized vegetation index and the leaf area index to obtain correlation coefficients of the normalized vegetation index and the leaf area index of the peanut and the growth of the peanut in different growth periods;
step C, establishing a peanut leaf disease identification model, and identifying the disease type and position of peanut leaves in the picture based on the leaf image information obtained in the step A;
and D, inputting the data of the normalized vegetation index, the leaf area index, the correlation coefficient of the leaf area index and the disease type and position into a CNN-LSTM network, and finally outputting a peanut yield predicted value through CNN-LSTM analysis processing.
Further, the step D specifically includes the following steps:
d1, extracting data features and reducing dimensions;
inputting the extracted normalized vegetation index, leaf area index and correlation coefficient thereof, and leaf disease category and position data into a CNN network according to the time sequence of the collected seedling stage, flowering stage and fruiting stage, wherein the CNN network comprises a convolution layer and a pooling layer, and the extraction and dimension reduction of data characteristics are realized through the convolution and pooling operation of the network;
d2, establishing a peanut canopy parameter correlation analysis model of the normalized vegetation index, the leaf area index and the correlation coefficient thereof and the disease category and position data;
Figure BDA0003656392690000021
i t ＝sigmoid(w ix x t +w is s t-1 +b i )
Figure BDA0003656392690000022
o t ＝sigmoid(w ox x t +w os s t-1 +b c )
Figure BDA0003656392690000023
wherein f is t Is the output signal of the forgetting gate, and determines the forgetting proportion of the memory unit c; i.e. i t Is the output signal of the output gate, determine how much current input information is input into the memory cell c;
Figure BDA0003656392690000024
is preliminary information to be input into the memory cell c, and i t Dot multiplication is carried out to obtain information in the memory unit c; o t Is the output signal of the output gate, determines the ratio of the memory cell c to the current state s;
Figure BDA0003656392690000025
is a preliminary information to be currently output to the hidden layer state s，o t The yield information is obtained by dot multiplication of the yield information and the point multiplication; s is the state information of the hidden layer; x is a radical of a fluorine atom t Is the input value at time t; w is a weight matrix; b t Is a bias parameter;
d3, fitting the trained data through the CNN-LSTM network, and outputting a peanut yield predicted value p t ；
Figure BDA0003656392690000026
Figure BDA0003656392690000027
Thereby realizing the prediction of the flower production quantity.
Further, the step D2 is specifically implemented by the following steps:
(1) the LSTM network comprises layer units, the number of hidden neurons of each layer is set according to input, a feature fusion operation of adding the hidden neurons point by point between peanut canopy parameter correlation analysis models is added, and a plurality of spectral features and a plurality of image features are added to obtain hyperspectral features;
Figure BDA0003656392690000028
wherein x and y respectively represent data of the hyperspectral characteristic matrix and the disease characteristic matrix, F represents a linear mapping function with characteristic fusion, and the expression is as follows:
F＝x i y i ,x i ,y i ∈R m
(2) inputting the data subjected to feature fusion into the LSTM network according to the time sequence of the seedling stage, the flowering stage and the fruiting stage, adjusting self parameters of a forgetting gate, an input gate and an output gate in the LSTM network through continuous iterative training of the data, enabling the LSTM network to learn the time fitting relationship among the data from the data information extracted by the CNN network, and outputting the correlation coefficient of the spectral data and the disease information at different periods
Figure BDA0003656392690000031
And further obtaining a peanut canopy parameter correlation analysis model.
Further, the step C is specifically realized by the following steps:
(1) making a peanut disease image data set, calibrating an image, dividing the image into a training set and a testing set, and enhancing a data sample, wherein the disease types comprise brown spot and peanut rust;
(2) constructing an improved YOLOv5 network model; on the basis of the original Yolov5 network model structure, a C3 module and an FPN + PANet module of a Yolov5 network model are correspondingly replaced by a CBAMC3 module and a Concat _ BiFPN module;
(3) and respectively inputting the training set and the test set into an improved YOLOv5 network model for training and testing, continuously adjusting model parameters, and outputting position information of a disease target label and a target marking frame.
Further, the step B is specifically realized by the following steps:
step B1, calculating the normalized vegetation index:
(1) acquiring feature matching points of the canopy spectral image through a Harris feature detector, splicing unmanned aerial vehicle images, and acquiring a canopy spectral image of the whole experimental field;
(2) preprocessing the canopy spectral image by using a wavelet analysis algorithm and a spectral normalization non-uniformity correction method to remove the influence of environmental noise; performing convolution and pooling operations by using a two-dimensional convolution model CNN, processing the canopy spectral image into a color image through one-dimensional convolution, extracting color features from red, green and blue spectral bands of the color image, outputting a feature map through two-dimensional convolution processing, and extracting texture and morphological features of the feature image from the feature map;
(3) calculating a normalized vegetation index NDVI based on the texture and morphological characteristics of the characteristic image, wherein the NDVI has the calculation formula:
NDVI＝(float(b2)-float(b1))/(float(b2)+float(b1))
wherein, b 1: near-infrared channel, b 2: a red channel;
step B2, calculating the leaf area index:
calculating the leaf area index LAI of the peanut in different periods of land coverage types, and storing the calculated LAI as an ENVI standard file, wherein the calculation formula of the LAI is as follows:
Figure BDA0003656392690000041
step B3, carrying out correlation analysis on the normalized vegetation index NDVI and the leaf area index LAI, analyzing the correlation between parameters such as the NDVI, the LAI and the like and the peanut growth in different growth periods of the peanut, and using a correlation coefficient r xy For the measurement, the specific calculation process is as follows:
Figure BDA0003656392690000042
Figure BDA0003656392690000043
Figure BDA0003656392690000044
Figure BDA0003656392690000045
wherein s is xy Represents the sample covariance, s x 、s y Respectively, the standard deviations of the samples x and y, r xy Representing the correlation coefficient, and n is the number of samples.
Further, in the step A, a hyperspectral sensor is carried on the unmanned aerial vehicle to acquire peanut canopy spectral image information, and a camera is used to acquire peanut leaf image information.
Compared with the prior art, the invention has the advantages and positive effects that:
according to the scheme, by means of fusion of information data of a hyperspectral image and a ground image of an unmanned aerial vehicle, a CNN-LSTM network is optimized and improved, a peanut canopy parameter correlation analysis model based on hyperspectral image data and disease image data is established, and conditions such as peanut yield and disease are judged and predicted; the method has the advantages that the self-made peanut disease data set is operated by image enhancement and the like, the original YOLOv5 model is optimized and improved by introducing a CBAM mechanism and a BiFPN structure, the detection speed and the detection precision of the model are improved, the peanut growth process is monitored and predicted from the aspects of peanut yield, disease conditions and the like, and the method has important significance for the development of precision agriculture.
Drawings
FIG. 1 is a schematic view of an overall process of disease detection and yield prediction according to an embodiment of the present invention;
FIG. 2 is a schematic diagram of a CNN-LSTM network model structure according to an embodiment of the present invention;
FIG. 3 is a schematic structural diagram of a peanut leaf disease identification model according to an embodiment of the invention;
FIG. 4 is a schematic structural diagram of CBAM introduced into the peanut leaf disease identification model in the embodiment of the invention;
FIG. 5 is a schematic diagram of a recognition result of a peanut leaf disease recognition model in the embodiment of the invention;
fig. 6 is a schematic view of a visualization training process according to an embodiment of the present invention.
Detailed Description
In order to make the above objects, features and advantages of the present invention more clearly understood, the present invention will be further described with reference to the accompanying drawings and examples. In the following description, numerous specific details are set forth in order to provide a thorough understanding of the present invention, however, the present invention may be practiced in other ways than those described herein, and thus, the present invention is not limited to the specific embodiments disclosed below.
The embodiment provides a peanut disease detection and yield prediction method based on canopy parameter processing, which comprises the following steps as shown in figure 1:
step A, respectively collecting canopy spectral image information of a peanut seedling stage, a flowering stage and a fruiting stage and leaf image information of the peanut seedling stage, the flowering stage and the fruiting stage, and recording collected time information;
b, acquiring a normalized vegetation index and a leaf area index of the peanut through phenotypic characteristic processing, and performing correlation analysis on the normalized vegetation index and the leaf area index to obtain correlation coefficients of the normalized vegetation index and the leaf area index and the peanut growth in different growth periods of the peanut;
step C, establishing a peanut leaf disease identification model, and identifying the disease type and position of peanut leaves in the picture;
and D, inputting the data of the normalized vegetation index, the leaf area index, the correlation coefficient of the leaf area index and the disease type and position into a CNN-LSTM network, analyzing and processing through the CNN-LSTM network, and outputting a peanut yield predicted value.
In the step A, an unmanned aerial vehicle is used for shooting peanut canopy spectrum images, in the embodiment, the Dajiang eidolon 4 is used for carrying a hyperspectral sensor to collect images, the Canon 80D digital camera is used for collecting peanut leaf images, the leaf images are classified according to disease types, and preparation is made for training and testing of a peanut disease detection model.
In the step B, based on the canopy spectral image information obtained in the step A, a method for processing phenotypic characteristics such as hyperspectral data processing and leaf area index calculation is used for obtaining normalized vegetation index and leaf area index data of peanuts, and the method specifically comprises the following steps:
step B1, calculating a normalized vegetation index (NDVI):
(1) by utilizing the local matching characteristic of the image, acquiring the characteristic matching points of the canopy spectrum image through a Harris characteristic detector, and carrying out unmanned aerial vehicle image splicing to acquire the global canopy spectrum image of the test field, so that the peanut characteristic information of the test field can be conveniently extracted;
(2) denoising, smoothing, sharpening and the like are carried out on the canopy spectral image by utilizing a wavelet analysis algorithm and a spectral normalization non-uniformity correction method, so that the influence of environmental noise is removed; in order to fully extract the spectral and spatial features of the canopy spectral image, a two-dimensional convolution model CNN is used for convolution and pooling, 240X 1 input canopy spectral image is processed into a 64X 64 color image through one-dimensional convolution, color features are extracted from red, green and blue spectral bands of the color image, 64 8X 8 feature maps are output through two-dimensional convolution processing, and then texture and morphological features of the feature image are extracted from the feature maps.
(3) Calculating a normalized vegetation index (NDVI) based on the texture and morphological characteristics of the characteristic image, calling an ENVI function in IDL, wherein the NDVI has the calculation formula:
NDVI＝(float(b2)-float(b1))/(float(b2)+float(b1))
wherein, b 1: near-infrared channel, b 2: a red channel;
step B2, Leaf Area Index (LAI) calculation:
calling a cal _ LAI function in the IDL to calculate the leaf area index of the peanut in different periods of land coverage types, and storing the calculated LAI as an ENVI standard file, wherein the calculation formula of the LAI is as follows:
Figure BDA0003656392690000061
step B3, carrying out correlation analysis on the normalized vegetation index NDVI and the leaf area index LAI, analyzing the correlation between parameters such as the NDVI, the LAI and the like and the peanut growth in different growth periods of the peanut, and using a correlation coefficient r xy For the measurement, the specific calculation process is as follows:
Figure BDA0003656392690000062
Figure BDA0003656392690000063
Figure BDA0003656392690000064
Figure BDA0003656392690000065
wherein n is the number of samples, x and y respectively represent the NDVI and LAI data of the extracted peanuts, and s xy Represents the covariance, s, of NDVI and LAI of peanut x 、s y Respectively represents the standard deviation r of NDVI and LAI of peanuts xy Represents the correlation coefficient between NDVI and LAI of peanut.
In the step C, the established peanut leaf disease identification model is an improved YOLOv5 network, a peanut disease image data set is established through image preprocessing methods such as image enhancement and the like, the data set is input into the improved YOLOv5 network, and the disease type and the position of the peanut leaf in the picture are identified, and the method specifically comprises the following steps:
(1) b, based on the peanut leaf image information obtained in the step A, carrying out image enhancement pretreatment and characteristic marking on the obtained peanut disease image, making a peanut disease image data set, and collecting 326 peanut brown spot images and 136 peanut rust images; the method comprises the following steps of carrying out image enhancement operations in the modes of channel separation, inversion, fuzzy operation, color enhancement, edge detection enhancement, logarithmic transformation, image denoising and the like on an acquired image, increasing the number of data sets, avoiding the overfitting condition of a model, and finally obtaining 2282 peanut brown spot images and 1224 peanut rust images, wherein the sizes of the data sets are unified to be 128 × 128px, and the data sets are 7: 3, dividing the training set and the test set in proportion, wherein the samples of the data enhancement operation are respectively as follows:
Figure BDA0003656392690000066
wherein x is b For image data with marked samples, B denotes the batch size, B represents the B-th sample of the batch,
Figure BDA0003656392690000071
representing image data of the marked sample after data enhancement;
using a LabelImg tool of a graphic image annotation tool written by Python language to calibrate the image, obtaining xml and txt files of the images of the training set and the test set by annotating the images in a PASCALVOC format (a format used by ImageNet), and then converting the format of the VOC stored after the annotation into a YOLO format, wherein the conversion process is as follows:
Figure BDA0003656392690000072
Figure BDA0003656392690000073
wherein the YOLO tag data comprises [ class, X ] center ,Y center ,width,height]Respectively representing the category of the detected target, the X coordinate and the Y coordinate of the central point, and the width and the height of the target frame. The coordinate information stored in the VOC label data is the coordinate position [ x ] of the calibration frame min ，x max ，y min ，y max ]Respectively representing the left and right values of the X coordinate and the upper and lower values of the Y coordinate of the target frame;
(2) constructing an improved YOLOv5 network model; on the basis of the original Yolov5 network structure, replacing a C3 module and an FPN + PANet module of a Yolov5 network model with a CBAM C3 module and a Concat _ BiFPN module;
the original YOLOv5 network model is introduced as follows: the method comprises an input layer, a backhaul layer, a neutral layer and a Prediction layer, wherein the input layer adopts a Mosaic data enhancement method to adaptively calculate an optimal anchor point frame according to the name of a data set, and adaptively adds a minimum black edge to a zoomed picture; the Backbone layer extracts features of different scales by using a Focus reference network and performs feature fusion by feature superposition; the Neck layer adopts a series FPN + PAN structure to perform feature fusion and multi-scale prediction between different layers from bottom to top, and positions pixels to form a mask; the Prediction layer outputs a target frame according to the characteristic part, and finally, the detection of the peanut diseases is completed;
the improved YOLOv5 network model structure improves the detection accuracy by introducing the information of the sensing direction and position of a CBAM attention mechanism, replaces the original FPN + PANet structure with a BiFPN structure, and performs weighted feature fusion by Fast normalized fusion (Fast normalized fusion) to improve the detection speed.
Loading a training configuration file, importing a model configuration file, modifying model parameters according to relevant configuration, improving an original YOLO v5 model by adding a CBAM module and a BiFPN structure, and replacing a C3 module and an FPN + PANet module of a YOLOv5 network model with a CBAM C3 module and a Concat _ BiFPN module; enabling the Concat layer to be connected with different layers, adopting Pairwise add operation to separate channels, calling modified yolov5l.yaml when training the model, and improving the detection precision of the model;
(3) and respectively inputting the training set and the test set into an improved YOLOv5 model for training and testing, continuously adjusting parameters such as the learning rate, the threshold value, the iteration times and the like of the model, improving the response speed and the detection precision of the network model detection, and outputting the position information of the disease target label and the target labeling frame.
The structure of the disease recognition model of the invention is shown in figure 3, and the main process is as follows: firstly, an input picture is equally divided into S x S grids, then the picture is sent into the network to predict whether targets, target types and target boundary frames exist in each grid, finally, the predicted boundary frames are subjected to non-maximum suppression (NMS) to select the best boundary frame and output, and the output dimension is S x S (B x 5+ C). As shown, the YOLOv5 structure is mainly divided into four levels: the input layer adopts a Mosaic data enhancement method to adaptively calculate an optimal anchor point frame according to the name of the data set; the backhaul layer uses the structures of the Focus reference network, the CSPDarknet53 and the SPP, and extracts the features in the pictures for feature fusion; the Neck layer adopts a series FPN + PAN structure to perform feature fusion and multi-scale prediction between different layers from bottom to top, and the propagation of semantic features and positioning information is enhanced; the Prediction layer adopts CIOU _ Loss as a Loss function of a Bounding box, adopts a weighted NMS mode to carry out screening operation of a target frame, and has better detection effect on a target with shielding overlapping.
The CBAM structure introduced into the peanut leaf disease identification model is shown in figure 4, the main process is to carry out convolution operation and average and maximum pooling operation on Inputs to obtain an intermediate feature F (H multiplied by W multiplied by C), then an activation function is used for calculation, and the multiplication of output channel attention Mc, Mc and MLP is used for obtaining a channel attention distribution map F', so that the importance of a channel in a feature map is explained. And carrying out operations such as averaging, maximum pooling, convolution and the like on the channel attention map F 'to obtain the spatial attention Ms, multiplying the Ms and the F' to obtain a spatial attention map F ", multiplying the channel attention map and the spatial attention map, and adding the multiplied channel attention map and the spatial attention map to the original input to obtain the final output Outputs.
In the step D, the data of the normalized vegetation index, the leaf area index, the correlation coefficient thereof, and the disease category and position are input into the CNN-LSTM network, the trained data are fitted through the CNN-LSTM network, a predicted peanut yield value is output through the full connection layer, and a peanut canopy parameter correlation analysis model based on the hyperspectral image data and the disease data is established, as shown in fig. 2, the method specifically includes:
d1, extracting data features and reducing dimensions;
inputting the extracted normalized vegetation index, leaf area index and correlation coefficient thereof, and data of leaf disease category and position into a CNN network according to the time sequence of the collected seedling stage, flowering stage and fruiting stage, wherein the CNN network comprises a convolution layer and a pooling layer, the core size of the convolution layer is set to 3 x 3, the pooling layer adopts maximum pooling operation, and the extraction and dimension reduction of data characteristics are realized through the convolution and pooling operation of the network;
Figure BDA0003656392690000081
Figure BDA0003656392690000082
wherein x is k l Represents the k-th convolution mapping of l layers; f is an activation function; n represents the number of input convolution mapping; denotes the convolution operation; w is a ik l Representing the weight of the ith operation of the kth convolution kernel of the l layer; b k l Representing the offset of the k-th convolution kernel corresponding to the l layers;
d2, establishing a peanut canopy parameter correlation analysis model based on the normalized vegetation index, the leaf area index and the correlation coefficient thereof, and the disease category and position data;
(1) the LSTM network comprises layer units, the number of hidden neurons of each layer is set according to input, in order to enable data between the CNN network and the LSTM network to be better correlated, feature fusion operation of adding the CNN network and the LSTM network point by point is added between peanut canopy parameter correlation analysis models, 2880 spectral features and 2880 image features are added, and the hyperspectral feature with the length of 5760 is obtained;
Figure BDA0003656392690000091
wherein x and y respectively represent data of the hyperspectral characteristic matrix and the disease characteristic matrix, F represents a linear mapping function with characteristic fusion, and the expression is as follows:
F＝x i y i ,x i ,y i ∈R m
(2) inputting the data subjected to feature fusion into the LSTM network according to the time sequence of the seedling stage, the flowering stage and the fruiting stage, adjusting self parameters of a forgetting gate, an input gate and an output gate in the LSTM network through continuous iterative training of the data, enabling the LSTM network to learn the time fitting relationship among the data from the data information extracted by the CNN network, and outputting the correlation coefficient of the spectral data and the disease information at different periods
Figure BDA0003656392690000092
Establishing a correlation analysis model of the hyperspectral image data and the data of disease categories and positions;
Figure BDA0003656392690000093
i t ＝sigmoid(w ix x t +w is s t-1 +b i )
Figure BDA0003656392690000094
o t ＝sigmoid(w ox x t +w os s t-1 +b c )
Figure BDA0003656392690000095
wherein, f t Is the output signal of the forgetting gate, and determines the forgetting proportion of the memory unit c; i.e. i t Is the output signal of the output gate, determine how much current input information is input into the memory cell c;
Figure BDA0003656392690000096
is preliminary information to be input into the memory cell c, and i t Dot multiplication is carried out to obtain information in the memory unit c; o t Is the output signal of the output gate, determines the ratio of the memory cell c to the current state s;
Figure BDA0003656392690000097
is the preparation information, o, currently to be output to the hidden layer state s t The yield information is obtained by dot multiplication of the yield information and the point multiplication; s is the state information of the hidden layer; x is a radical of a fluorine atom t Is the input value at time t; w is a weight matrix; b t Is a bias parameter;
d3, fitting the trained data through the CNN-LSTM network, and outputting a peanut yield predicted value p through the full-connection layer according to parameter values set in different periods t ；
Figure BDA0003656392690000098
Figure BDA0003656392690000099
Thereby realizing the prediction of the flower production quantity.
In the following, the scheme of the invention is described in detail with reference to specific examples, and the data acquisition site is peanut Yuhua No. 11 planted in modern agriculture science and technology demonstration garden (northern latitude N: 36 degrees 26 '24.61', east longitude E: 120 degrees 04 '51.30') of Qingdao agriculture university of Qingdao City, Qingdao, Shandong province; the hyperspectral image acquisition equipment carries a visible light remote sensing system consisting of a hyperspectral camera for an unmanned aerial vehicle remote sensing platform: an unmanned plane of Xinjiang spirit 4 carries a UHD185 imaging spectrometer with the wavelength range of 450-950nm, the instrument is ensured to be vertically downward during spectrum acquisition, and a black and white board is utilized to carry out hyperspectral image radiation correction on the ground before data acquisition; the ground image acquisition equipment is a digital single lens reflex camera Canon EOS 80D, the effective pixels are 2000-2999 ten thousand CMOS sensors, and the shutter speed is 1/8000-30 s.
The operating system used for model training of disease identification is Microsoft Windows 10 flagship edition (64 bits), the deep learning framework is Pytroch 1.4, CUDA4.10.1 edition, and the processor is
Figure BDA0003656392690000101
Core TM i7-7200CPU @2.50GHz, NVIDIARTX 2070S (8GB memory) as a graphics card, Python1.8 as a programming language, and YOLOv5 as an original model.
Respectively acquiring hyperspectral images and ground leaf images of peanut seedling stage, flowering stage and fruiting stage by using an unmanned aerial vehicle and a camera, recording acquired time information, acquiring feature matching points of the images by using matching features of adjacent images through a Harris feature detector, and carrying out unmanned aerial vehicle image splicing; denoising, smoothing and sharpening the image, and acquiring data such as a normalized vegetation index and a leaf area index of the peanut by utilizing a hyperspectral data processing and leaf area index calculation isophenotype feature processing method; performing graying, edge detection and other processing on the peanut leaf images by adopting image enhancement, increasing the number of data sets and unifying the image size to be 128 × 128 px;
replacing a C3 module and an FPN + PANet module of a YOLOv5 network model with a CBAM C3 module and a Concat _ BiFPN module; inputting the data set into an improved and optimized YOLOv5 network, establishing a peanut leaf disease rapid detection model, and identifying the disease type and position of peanut leaves in the picture; and inputting the normalized vegetation index, the normalized leaf area index correlation coefficient, the normalized disease category and disease position data into a CNN-LSTM neural network according to a time sequence, fitting the trained data through the CNN-LSTM neural network, outputting a peanut yield predicted value through a full connection layer, and establishing a correlation analysis model of CNN-LSTM hyperspectral image data and disease image data.
The disease identification result of the invention is shown in fig. 5, and it can be seen that the position and category characteristics of peanut diseases can be detected by respectively performing model training and optimization on data sets of a complex background and a simple background in a field. Under the condition of ensuring other parameters to be consistent, network models with different structures are selected: faster R-CNN, VGG16, YOLOv4, YOLOv5, and modified YOLOv5, and the like. The test results are shown in table 1, and the indexes such as Detection time (Detection time), Loss function (Loss), accuracy (P), recall rate (R), F1 score (F1) of the model are compared: (1) the improved model size was 17.8 MB; compared with a Faster R-CNN model, a VGG model and a YOLOv4 model, 147.2MB, 100.2MB and 15.9MB are saved respectively; the improved model can greatly save the memory occupation. (2) The average detection time of a single image is 15ms, which is respectively improved by 93.6%, 90.5%, 37.5% and 11.8% compared with the models of Faster R-CNN, VGG16, YOLO v4 and YOLO v 5; the improved model is shown to improve the detection speed. (3) The accuracy, the recall rate and the F1 score of the improved YOLO v5 model for detecting diseases are 93.73%, 92.94% and 92.97% respectively; the yield is respectively improved by 3.98%, 2.69% and 2.29% compared with the original YOLOv5 model; the improved model has better detection effect. By combining the analysis, the detection method provided by the research has obvious advantages in the aspects of model size, detection speed, detection accuracy and the like, and meets the requirements of disease detection tasks of peanuts.
TABLE 1 comparison of recognition Performance of different network models
Figure BDA0003656392690000102
Figure BDA0003656392690000111
The visualization training process of the invention is shown in fig. 6, and the parameter fluctuation of the model is large in the process of model iteration for 0-300 times. In the process of model iteration of 300 times and 600 times, the performance of the model is continuously optimized along with the increase of the iteration times. In the process of model iteration of 900 times and 1000 times, each index gradually tends to be stable, and the accuracy (P) reaches about 93% and is gradually stable. Therefore, considering the influence of the iteration number on the stability and efficiency of the model, the optimal iteration number of the model is selected to be 1000. After the model is trained for 1000 times, the learning rate is reduced by 0.1, and the training time is about 9.5 h.
The above description is only a preferred embodiment of the present invention, and not intended to limit the present invention in other forms, and any person skilled in the art may apply the above modifications or changes to the equivalent embodiments with equivalent changes, without departing from the technical spirit of the present invention, and any simple modification, equivalent change and change made to the above embodiments according to the technical spirit of the present invention still belong to the protection scope of the technical spirit of the present invention.

Claims (6)

1. The peanut disease detection and yield prediction method based on canopy parameter processing is characterized by comprising the following steps:
step A, collecting canopy spectral image information of a peanut seedling stage, a flowering stage and a fruiting stage and leaf image information of the peanut seedling stage, the flowering stage and the fruiting stage respectively, and recording collected time information;
b, based on the canopy spectral image information obtained in the step A, acquiring a normalized vegetation index and a leaf area index of the peanut through phenotypic characteristic processing, and performing correlation analysis on the normalized vegetation index and the leaf area index to obtain correlation coefficients of the normalized vegetation index and the leaf area index of the peanut and the growth of the peanut in different growth periods;
step C, establishing a peanut leaf disease identification model, and identifying the disease type and position of peanut leaves in the picture based on the leaf image information obtained in the step A;
and D, inputting the data of the normalized vegetation index, the leaf area index, the correlation coefficient of the leaf area index and the disease type and position into a CNN-LSTM network, and finally outputting a peanut yield predicted value through CNN-LSTM analysis processing.
2. The canopy parameter processing-based peanut disease detection and yield prediction method of claim 1, wherein: the step D specifically comprises the following steps:
d1, extracting data features and reducing dimensions;
inputting the extracted normalized vegetation index, leaf area index and correlation coefficient thereof, and leaf disease category and position data into a CNN network according to the time sequence of the collected seedling stage, flowering stage and fruiting stage, wherein the CNN network comprises a convolution layer and a pooling layer, and the extraction and dimension reduction of data characteristics are realized through the convolution and pooling operation of the network;
d2, establishing a peanut canopy parameter correlation analysis model of the normalized vegetation index, the leaf area index and the correlation coefficient thereof and the disease category and position data;
f t ＝sigmoid(w ft x t +w ft s t-1 +b t )
i t ＝sigmoid(w ix x t +w is s t-1 +b i )
Figure FDA0003656392680000011
o t ＝sigmoid(w ox x t +w os s t-1 +b c )
Figure FDA0003656392680000012
wherein f is t Is the output signal of the forgetting gate, and determines the forgetting proportion of the memory unit c; i.e. i t Is the output of the output gateA signal for determining how much current input information is input to the memory unit c;
Figure FDA0003656392680000013
is preliminary information to be input into the memory cell c, and i t Dot multiplication is carried out to obtain information in the memory unit c; o. o t Is the output signal of the output gate, determines the ratio of the memory cell c to the current state s;
Figure FDA0003656392680000014
is the preparation information, o, currently to be output to the hidden layer state s t The yield information is obtained by dot multiplication of the yield information and the point multiplication; s is the state information of the hidden layer; x is the number of t Is the input value at time t; w is a weight matrix; b t Is a bias parameter;
d3, fitting the trained data through a CNN-LSTM network, and outputting a peanut yield predicted value p t ；
Figure FDA0003656392680000015
Figure FDA0003656392680000016
Thereby realizing the prediction of the flower production quantity.
3. The canopy parameter processing-based peanut disease detection and yield prediction method of claim 1, wherein: the step D2 is specifically implemented by the following method:
(1) the LSTM network comprises layer units, the number of hidden neurons of each layer is set according to input, a feature fusion operation of adding the hidden neurons point by point between peanut canopy parameter correlation analysis models is added, and a plurality of spectral features and a plurality of image features are added to obtain hyperspectral features;
Figure FDA0003656392680000021
wherein x and y respectively represent data of the hyperspectral characteristic matrix and the disease characteristic matrix, F represents a linear mapping function with characteristic fusion, and the expression is as follows:
F＝x i y i ,x i ,y i ∈R m
(2) inputting the data subjected to feature fusion into the LSTM network according to the time sequence of the seedling stage, the flowering stage and the fruiting stage, adjusting self parameters of a forgetting gate, an input gate and an output gate in the LSTM network through continuous iterative training of the data, enabling the LSTM network to learn the time fitting relationship among the data from the data information extracted by the CNN network, and outputting the correlation coefficient of the spectral data and the disease information at different periods
Figure FDA0003656392680000022
And further obtaining a peanut canopy parameter correlation analysis model.
4. The method for detecting peanut diseases and predicting peanut yield based on canopy parameter processing as claimed in claim 1, wherein: the step C is specifically realized by the following steps:
(1) making a peanut disease image data set, calibrating an image, dividing the image into a training set and a testing set, and enhancing a data sample, wherein the disease types comprise brown spot and peanut rust;
(2) constructing an improved YOLOv5 network model; on the basis of the original Yolov5 network model structure, a C3 module and an FPN + PANet module of a Yolov5 network model are correspondingly replaced by a CBAM C3 module and a Concat _ BiFPN module;
(3) and respectively inputting the training set and the test set into an improved YOLOv5 network model for training and testing, continuously adjusting model parameters, and outputting position information of a disease target label and a target marking frame.
5. The method for detecting peanut diseases and predicting peanut yield based on canopy parameter processing as claimed in claim 1, wherein: the step B is specifically realized by the following steps:
step B1, calculating the normalized vegetation index:
(1) acquiring feature matching points of the canopy spectral image through a Harris feature detector, splicing unmanned aerial vehicle images, and acquiring a canopy spectral image of the whole experimental field;
(2) preprocessing the canopy spectral image by using a wavelet analysis algorithm and a spectral normalization non-uniformity correction method to remove the influence of environmental noise; performing convolution and pooling operations by using a two-dimensional convolution model CNN, processing the canopy spectral image into a color image through one-dimensional convolution, extracting color features from red, green and blue spectral bands of the color image, outputting a feature map through two-dimensional convolution processing, and extracting texture and morphological features of the feature image from the feature map;
(3) calculating a normalized vegetation index NDVI based on the texture and morphological characteristics of the characteristic image, wherein the NDVI has the calculation formula:
NDVI＝(float(b2)-float(b1))/(float(b2)+float(b1))
wherein, b 1: near-infrared channel, b 2: a red channel;
step B2, calculating the leaf area index:
calculating the leaf area index LAI of the peanut in different periods of land coverage types, and storing the calculated LAI as an ENVI standard file, wherein the calculation formula of the LAI is as follows:
Figure FDA0003656392680000031
step B3, carrying out correlation analysis on the normalized vegetation index NDVI and the leaf area index LAI, analyzing the correlation between parameters such as the NDVI, the LAI and the like and the peanut growth in different growth periods of the peanut, and using a correlation coefficient r xy For the measurement, the specific calculation process is as follows:
Figure FDA0003656392680000032
Figure FDA0003656392680000033
Figure FDA0003656392680000034
Figure FDA0003656392680000035
wherein s is xy Represents the sample covariance, s x 、s y Respectively, the standard deviations of the samples x and y, r xy Representing the correlation coefficient, and n is the number of samples.
6. The method for detecting peanut diseases and predicting peanut yield based on canopy parameter processing as claimed in claim 1, wherein: in the step A, a hyperspectral sensor is carried on the basis of an unmanned aerial vehicle to acquire peanut canopy spectrum image information, and a camera is adopted to acquire peanut leaf image information.
CN202210561268.7A 2022-05-23 2022-05-23 Peanut disease detection and yield prediction method based on canopy parameter processing Pending CN114965501A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202210561268.7A CN114965501A (en) 2022-05-23 2022-05-23 Peanut disease detection and yield prediction method based on canopy parameter processing

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202210561268.7A CN114965501A (en) 2022-05-23 2022-05-23 Peanut disease detection and yield prediction method based on canopy parameter processing

Publications (1)

Publication Number Publication Date
CN114965501A true CN114965501A (en) 2022-08-30

Family

ID=82985558

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202210561268.7A Pending CN114965501A (en) 2022-05-23 2022-05-23 Peanut disease detection and yield prediction method based on canopy parameter processing

Country Status (1)

Country Link
CN (1) CN114965501A (en)

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN116168223A (en) * 2023-04-20 2023-05-26 华南农业大学 Multi-mode-based peanut leaf spot disease grade detection method
CN117516639A (en) * 2024-01-08 2024-02-06 吉林农业大学 High-flux greenhouse plant phenotype measurement system based on multispectral point cloud fusion

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN116168223A (en) * 2023-04-20 2023-05-26 华南农业大学 Multi-mode-based peanut leaf spot disease grade detection method
CN117516639A (en) * 2024-01-08 2024-02-06 吉林农业大学 High-flux greenhouse plant phenotype measurement system based on multispectral point cloud fusion

Similar Documents

Publication Publication Date Title
CN109325431B (en) Method and device for detecting vegetation coverage in feeding path of grassland grazing sheep
CN113392775B (en) Sugarcane seedling automatic identification and counting method based on deep neural network
CA3125790A1 (en) Shadow and cloud masking for remote sensing images in agriculture applications using multilayer perceptron
CN114965501A (en) Peanut disease detection and yield prediction method based on canopy parameter processing
CN110569747A (en) method for rapidly counting rice ears of paddy field rice by using image pyramid and fast-RCNN
CN112766155A (en) Deep learning-based mariculture area extraction method
Liang et al. Improved estimation of aboveground biomass in rubber plantations by fusing spectral and textural information from UAV-based RGB imagery
CN111161362A (en) Tea tree growth state spectral image identification method
CN112836725A (en) Weak supervision LSTM recurrent neural network rice field identification method based on time sequence remote sensing data
CN110533100A (en) A method of CME detection and tracking is carried out based on machine learning
CN112446298A (en) Hyperspectral nondestructive testing method for wheat scab
CN110765977A (en) Method for extracting wheat lodging information based on multi-temporal remote sensing data of unmanned aerial vehicle
CN116258956A (en) Unmanned aerial vehicle tree recognition method, unmanned aerial vehicle tree recognition equipment, storage medium and unmanned aerial vehicle tree recognition device
CN114332534B (en) Hyperspectral image small sample classification method
CN114973024A (en) Method and system for monitoring and evaluating comprehensive evaluation index of mechanical cotton picking and defoliating effect
CN117576195A (en) Plant leaf morphology recognition method
Kuswidiyanto et al. Airborne hyperspectral imaging for early diagnosis of kimchi cabbage downy mildew using 3D-ResNet and leaf segmentation
CN116863341B (en) Crop classification and identification method and system based on time sequence satellite remote sensing image
CN112528726B (en) Cotton aphid pest monitoring method and system based on spectral imaging and deep learning
Menezes et al. Pseudo-label semi-supervised learning for soybean monitoring
CN109308464A (en) A kind of rice tassel recognition methods based on digital camera images
Yang et al. Rapid image detection and recognition of rice false smut based on mobile smart devices with anti-light features from cloud database
CN117058522A (en) Remote sensing earth surface ecological change detection method integrating spectral neural network and leaf area index
CN114863296A (en) Method and system for identifying and positioning wood damaged by pine wilt disease
CN112488230A (en) Crop water stress degree judging method and device based on machine learning

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination