CN114419439B - Wheat seedling monitoring method based on unmanned aerial vehicle remote sensing and deep learning - Google Patents
Wheat seedling monitoring method based on unmanned aerial vehicle remote sensing and deep learning Download PDFInfo
- Publication number
- CN114419439B CN114419439B CN202210049867.0A CN202210049867A CN114419439B CN 114419439 B CN114419439 B CN 114419439B CN 202210049867 A CN202210049867 A CN 202210049867A CN 114419439 B CN114419439 B CN 114419439B
- Authority
- CN
- China
- Prior art keywords
- wheat
- wheat seedling
- image
- seedling
- seedlings
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 241000209140 Triticum Species 0.000 title claims abstract description 206
- 235000021307 Triticum Nutrition 0.000 title claims abstract description 206
- 238000000034 method Methods 0.000 title claims abstract description 56
- 238000013135 deep learning Methods 0.000 title claims abstract description 42
- 238000012544 monitoring process Methods 0.000 title claims abstract description 22
- 238000005070 sampling Methods 0.000 claims abstract description 54
- 238000001514 detection method Methods 0.000 claims abstract description 46
- 238000012549 training Methods 0.000 claims abstract description 33
- 238000004422 calculation algorithm Methods 0.000 claims description 15
- 238000004519 manufacturing process Methods 0.000 claims description 12
- 230000008569 process Effects 0.000 claims description 12
- 230000011218 segmentation Effects 0.000 claims description 9
- 238000013145 classification model Methods 0.000 claims description 8
- 238000005520 cutting process Methods 0.000 claims description 6
- 238000003064 k means clustering Methods 0.000 claims description 5
- 230000007797 corrosion Effects 0.000 claims description 4
- 238000005260 corrosion Methods 0.000 claims description 4
- 238000004590 computer program Methods 0.000 claims description 3
- 238000007781 pre-processing Methods 0.000 claims description 3
- 238000005457 optimization Methods 0.000 abstract description 7
- 238000012360 testing method Methods 0.000 description 13
- 238000002474 experimental method Methods 0.000 description 6
- 238000010586 diagram Methods 0.000 description 5
- 238000010276 construction Methods 0.000 description 3
- 230000007547 defect Effects 0.000 description 3
- 230000000694 effects Effects 0.000 description 3
- 238000005516 engineering process Methods 0.000 description 3
- 238000002372 labelling Methods 0.000 description 3
- 238000012545 processing Methods 0.000 description 3
- 235000013339 cereals Nutrition 0.000 description 2
- 238000000605 extraction Methods 0.000 description 2
- 239000003550 marker Substances 0.000 description 2
- 241000199223 Elaeocarpus kirtonii Species 0.000 description 1
- 235000009414 Elaeocarpus kirtonii Nutrition 0.000 description 1
- 241000531897 Loma Species 0.000 description 1
- 235000013584 Tabebuia pallida Nutrition 0.000 description 1
- 238000012271 agricultural production Methods 0.000 description 1
- 238000004458 analytical method Methods 0.000 description 1
- 238000004364 calculation method Methods 0.000 description 1
- 238000007635 classification algorithm Methods 0.000 description 1
- 230000007423 decrease Effects 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 230000018109 developmental process Effects 0.000 description 1
- 238000005194 fractionation Methods 0.000 description 1
- 230000008014 freezing Effects 0.000 description 1
- 238000007710 freezing Methods 0.000 description 1
- 230000006872 improvement Effects 0.000 description 1
- 238000003973 irrigation Methods 0.000 description 1
- 230000002262 irrigation Effects 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 239000000575 pesticide Substances 0.000 description 1
- 238000003672 processing method Methods 0.000 description 1
- 238000011002 quantification Methods 0.000 description 1
- 238000005507 spraying Methods 0.000 description 1
- 230000007704 transition Effects 0.000 description 1
- 238000012795 verification Methods 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
- G06F18/214—Generating training patterns; Bootstrap methods, e.g. bagging or boosting
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/23—Clustering techniques
- G06F18/232—Non-hierarchical techniques
- G06F18/2321—Non-hierarchical techniques using statistics or function optimisation, e.g. modelling of probability density functions
- G06F18/23213—Non-hierarchical techniques using statistics or function optimisation, e.g. modelling of probability density functions with fixed number of clusters, e.g. K-means clustering
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- Data Mining & Analysis (AREA)
- Life Sciences & Earth Sciences (AREA)
- Artificial Intelligence (AREA)
- General Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- Evolutionary Computation (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Computational Linguistics (AREA)
- Bioinformatics & Computational Biology (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Health & Medical Sciences (AREA)
- Biomedical Technology (AREA)
- Biophysics (AREA)
- Evolutionary Biology (AREA)
- General Health & Medical Sciences (AREA)
- Molecular Biology (AREA)
- Computing Systems (AREA)
- Mathematical Physics (AREA)
- Software Systems (AREA)
- Probability & Statistics with Applications (AREA)
- Image Analysis (AREA)
Abstract
The invention discloses a wheat seedling monitoring method based on unmanned aerial vehicle remote sensing and deep learning, which comprises the following steps: step 1, training a deep learning network to obtain a wheat seedling detection model; step 2, training a deep learning network by using a wheat seedling grading data set to obtain a wheat seedling grading model; step 3, selecting a rectangular area in the wheat field as a sampling block, inserting a label at the vertex of the rectangle as a marking point, and calculating the area S of the sampling block; step 4, collecting sample block images by using an unmanned aerial vehicle, and detecting positions of all seedlings in the images by using a wheat seedling detection model; step 5, counting the number N of wheat seedlings in the sampling block, and calculating to obtain the seedling density P=N/S; and 6, grading all seedling individuals in the sampling block marking frame by using a wheat seedling grading model, and calculating the ratio Ri of the number Ni of the ith level seedlings to the number N of the seedlings. According to the invention, the wheat seedlings can be detected in a complex scene through the training of the deep learning network and the optimization of the priori frame, so that the accuracy and the reliability of the wheat seedling detection are greatly improved.
Description
Technical Field
The invention relates to the field of intelligent monitoring of crop growth, in particular to a wheat seedling monitoring method based on unmanned aerial vehicle remote sensing and deep learning.
Background
Wheat is one of grain crops with the largest planting area in China, is widely planted in various places in China, and has great and profound significance in guaranteeing the grain safety in China. The seedling period of wheat has important influence on the whole growth and development process of wheat, and the number of wheat seedlings, the growth condition of individual seedlings, the leaf number of seedlings, the ground coverage of seedlings and the like are key indexes for measuring the growth quality of the wheat in the seedling period. In order to ensure stable growth of wheat and ensure wheat yield, the condition information of wheat seedlings needs to be timely and effectively monitored in the planting process, so that decision basis is provided for a series of agricultural management such as reseeding, irrigation, pesticide spraying and the like in the later period of wheat. However, the conventional monitoring of wheat seedling condition mainly depends on eye observation and experience judgment, has strong subjectivity and lacks objective and accurate quantification standard.
In recent years, unmanned aerial vehicle remote sensing technology is increasingly commonly used for acquiring various agricultural information, and becomes a powerful tool for agricultural informatization. The unmanned aerial vehicle is used for acquiring information of the wheat in different periods, so that the unmanned aerial vehicle is flexible, and agricultural personnel can acquire wheat images without going deep into fields; the unmanned aerial vehicle shooting range is large, the angle adjustment is convenient, and the growth condition of the large-range wheat can be obtained. However, the unmanned aerial vehicle collects only shooting data, so that the implementation effect of the method depends on accurate extraction of the image to crop information, and if an algorithm cannot effectively convert the shooting image, the monitoring result is greatly discounted.
Therefore, the existing remote sensing monitoring still has the defects of complex method, unstable detection result, low detection precision, insufficient intelligent degree and the like, and cannot be well adapted to the growth characteristics of wheat. No means or technique for solving or improving the above problems is currently known.
Disclosure of Invention
Aiming at the defects or improvement demands of the existing method, the invention aims to provide the wheat seedling monitoring method based on unmanned aerial vehicle remote sensing and deep learning, and combines the unmanned aerial vehicle remote sensing technology with the deep learning technology, thereby providing an intelligent technical means for wheat field management, improving the stability and accuracy of detection results and accelerating the digitization and intelligent process of wheat planting.
In order to solve the technical problems, the invention provides a wheat seedling monitoring method based on unmanned aerial vehicle remote sensing and deep learning, which comprises the following steps in sequence:
step 1, acquiring a wheat seedling image by using an unmanned aerial vehicle-mounted camera, manufacturing a wheat seedling data set, and training a deep learning network through the data set to obtain a wheat seedling detection model;
Step 2, dividing wheat seedlings into four grades of 1 leaf, 2 leaf, 3 leaf and more than 3 leaf according to the number of the leaves, manufacturing a wheat seedling grading data set, and training a deep learning network through the grading data set to obtain a wheat seedling grading model;
Step 3, selecting a rectangular area in the wheat field as a sampling block, inserting a label at the vertex of the rectangular area, manually measuring the actual length L and the width W of the sampling block, and calculating the area S=L×W of the sampling block;
Step 4, using an unmanned aerial vehicle to carry a camera, a certain height is away from the ground, sampling block images I are collected at a certain angle, the sampling block images I are sent into a wheat seedling detection model, and the positions of all seedlings in the images are detected by using the wheat seedling detection model and the positions of the seedlings are extracted;
step 5, simultaneously, extracting mark points from the sampling block image I to obtain the position of the sampling block, counting the number N of wheat seedlings in the sampling block by combining the seedling position information, and calculating to obtain the seedling density P=N/S;
Step 6, sending the sampling block image I and seedling position information into a wheat seedling grading model, grading all seedling individuals in a sampling block marking frame by using the wheat seedling grading model, counting to obtain the number Ni (i=1, 2,3, 4) of the ith grade, and calculating the ratio Ri=Ni/N100% of the number Ni of the ith grade to the number N of the seedlings;
And 7, carrying a camera by using an unmanned aerial vehicle, and acquiring a wheat seedling nodding image F perpendicular to the ground at a certain height from the ground. Obtaining a wheat seedling binarization image of the nodding image F by using a threshold segmentation method, counting the pixel number Sw of the wheat seedling and the total number S0 of image pixels, and calculating the ground coverage cover=Sw/S0 of the wheat seedling;
And 8, outputting a wheat seedling monitoring result, wherein the wheat seedling monitoring result comprises a seedling density P, the number Ni and the proportion Ri (i=1, 2,3, 4) of seedlings at each level and a ground coverage Cover of the seedlings.
Further, in step 1, the construction process of the wheat seedling detection model specifically includes the following steps in order:
step 11, collecting images of wheat seedlings at a certain height and at multiple angles from the ground by using an unmanned aerial vehicle in the period of the wheat seedlings;
step 12, preprocessing a wheat image, cutting the image, removing the image with poor quality, marking the position of the wheat seedling in the image, and manufacturing a wheat seedling data set;
Step 13, defining a first deep learning network;
and 14, training the deep learning network by using the wheat seedling data set to obtain a wheat seedling detection model.
Preferably, the deep learning network for constructing the wheat seedling detection model is based on YOLOV network structure, and the optimal prior frame size is obtained through a K-means++ clustering algorithm so as to improve the detection accuracy. The method is that K cluster centers are selected, the distance between each labeling frame and the K cluster centers is calculated, the distances are divided into the cluster center categories closest to the K cluster centers, and the cluster centers are continuously updated until the cluster centers are not changed.
In step 2, the construction of the wheat seedling classification model specifically comprises the following steps in sequence:
step 21, collecting wheat seedling images at multiple angles by using an unmanned plane in a wheat seedling period;
And 22, detecting the wheat seedling image by using a wheat seedling detection model to obtain a wheat seedling individual image, removing the image with poor quality, marking the position and the grade of the wheat seedling in the image, dividing the wheat seedling into four grades of 1 leaf, 2 leaves, 3 leaves and more than 3 leaves according to the number of the seedling leaves, and manufacturing a wheat seedling grading data set.
Step 23, defining a second deep learning network;
And step 24, training the deep learning network by using the wheat seedling classification data set to obtain a wheat seedling classification model.
Preferably, the deep learning network for constructing the wheat seedling classification model adopts ResNet-18 network structure.
In step 3, the rectangular sampling block is used for measuring the density and the grading proportion of seedlings, a rectangular area is selected in the field, and white labels are inserted into the tops of the rectangles to serve as marking points, as shown in fig. 3.
In step 5, the specific method for extracting the positions of the marking points from the image of the sampling block is that firstly, the pixels of the marking points are extracted by using a threshold segmentation algorithm, noise points in the image are removed by corrosion expansion processing, then, the positions of the four marking points are found by using a K-means clustering algorithm, and finally, the marking frame of the sampling block is obtained.
Further, in step 7, a wheat seedling binarized image is obtained using the supergreen method. For wheat seedling images, the green factor for each pixel is calculated:
| (1) |
Is a green factor, which is used for the color of the green, Is the value of the pixel that is to be green,For the blue pixel value,Is the red pixel value. And replacing each pixel in the image with a corresponding green factor to obtain an ultra-green gray level image of the wheat seedlings. Then determining an optimal segmentation threshold by using a maximum inter-class variance method (OSTU)And binarizing each pixel of the ultra-green gray scale image:
| (2) |
when the pixel value is greater than the threshold value T, it is set to 255, indicating that it is a wheat seedling; when the pixel value is smaller than the threshold value T, the pixel value is set to 0, which represents that the pixel value is used as a background, so that a binarized image of the wheat seedling is obtained.
The invention also provides a processor, which is characterized in that the processor is used for running a program, wherein the program executes the steps.
The invention also provides corresponding computer equipment, comprising a memory, a processor and a computer program stored on the memory and capable of running on the processor, wherein the memory realizes the steps in the embodiment when the program is executed.
The invention provides a novel monitoring algorithm by utilizing the deep learning network, so that the detection is simple and flexible, the characteristics of wheat are better adapted, and the defect of poor traditional monitoring effect is overcome. The method provided by the invention is based on a novel wheat seedling detection model, and the wheat seedlings in a complex scene can be detected through training of a deep learning network and optimization of a priori frame, so that the accuracy and reliability of wheat seedling detection are greatly improved.
Additional features and advantages of the invention will be set forth in the description which follows, and in part will be obvious from the description, or may be learned by practice of the invention. The objectives and other advantages of the invention will be realized and attained by the structure particularly pointed out in the written description and claims hereof as well as the appended drawings.
Drawings
The accompanying drawings are included to provide a further understanding of the application and are incorporated in and constitute a part of this specification, illustrate and do not limit the application.
Fig. 1 is a schematic flow chart of a wheat seedling monitoring method based on unmanned aerial vehicle remote sensing and deep learning according to an embodiment of the invention;
FIG. 2 is a schematic diagram of a process for constructing a wheat seedling detection model according to an embodiment of the present invention;
FIG. 3 is a schematic diagram of a sample block arrangement according to an embodiment of the present invention;
FIG. 4 is a schematic illustration of an experiment using labelImg for marking wheat seedlings according to an embodiment of the present invention;
FIG. 5 is a schematic illustration of an experiment in which marker points and marker boxes are extracted from an image in accordance with an embodiment of the present invention;
FIG. 6 is a schematic diagram showing the test results of wheat seedlings in a sample block according to an embodiment of the present invention;
FIG. 7 is a schematic diagram of a wheat seedling classification procedure according to an embodiment of the present invention;
FIG. 8 is a schematic diagram of an original image, an ultra-green gray image and an OSTU binarization image obtained in the experimental process of the invention;
FIG. 9 is a histogram of wheat seedling ground coverage during the course of the experiment of the present invention.
Detailed Description
In order to enable those skilled in the art to better understand the present application, the objects, technical solutions and advantages of the present application will be more clearly understood, and the present application will be further described with reference to the following specific examples and the accompanying drawings. It should be understood that the specific embodiments described herein are for purposes of illustration only and are not intended to limit the scope of the application. It should be noted that, without conflict, the embodiments of the present application and features of the embodiments may be arbitrarily combined with each other.
The invention provides a wheat seedling monitoring method based on unmanned aerial vehicle remote sensing and deep learning, which is shown in figure 1. The method comprises the following steps in sequence:
step 1, acquiring a wheat seedling image by using an unmanned aerial vehicle carrying camera, manufacturing a wheat seedling data set, and training a deep learning network through the data set to obtain a wheat seedling detection model;
Step 2, dividing wheat seedlings into four grades of 1 leaf, 2 leaf, 3 leaf and more than 3 leaf according to the number of the leaves, manufacturing a wheat seedling grading data set, and training a deep learning network through the grading data set to obtain a wheat seedling grading model;
Step 3, selecting a rectangular area in the wheat field as a sampling block, inserting a label at the vertex of the rectangular area, manually measuring the actual length L and the width W of the sampling block, and calculating the area S=L×W of the sampling block;
Step 4, using an unmanned aerial vehicle to carry a camera, a certain height is away from the ground, sampling block images I are collected at a certain angle, the sampling block images I are sent into a wheat seedling detection model, and the positions of all seedlings in the images are detected by using the wheat seedling detection model and the positions of the seedlings are extracted;
step 5, simultaneously, extracting mark points from the sampling block image I to obtain the position of the sampling block, counting the number N of wheat seedlings in the sampling block by combining the seedling position information, and calculating to obtain the seedling density P=N/S;
Step 6, sending the sampling block image I and seedling position information into a wheat seedling grading model, grading all seedling individuals in a sampling block marking frame by using the wheat seedling grading model, counting to obtain the number Ni (i=1, 2,3, 4) of the ith grade, and calculating the ratio Ri=Ni/N100% of the number Ni of the ith grade to the number N of the seedlings;
And 7, carrying a camera by using an unmanned aerial vehicle, and acquiring a wheat seedling nodding image F perpendicular to the ground at a certain height from the ground. Obtaining a wheat seedling binarization image of the nodding image F by using a threshold segmentation method, counting the pixel number Sw of the wheat seedling and the total number S0 of image pixels, and calculating the ground coverage cover=Sw/S0 of the wheat seedling;
And 8, outputting a wheat seedling monitoring result, wherein the wheat seedling monitoring result comprises a seedling density P, the number Ni and the proportion Ri (i=1, 2,3, 4) of seedlings at each level and a ground coverage Cover of the seedlings.
Further, in step 1, the process of constructing the wheat seedling detection model is shown in fig. 2. The method specifically comprises the following steps in sequence:
step 11, collecting images of wheat seedlings at a certain height and at multiple angles from the ground by using an unmanned aerial vehicle in the period of the wheat seedlings;
step 12, preprocessing a wheat image, cutting the image, removing the image with poor quality, marking the position of the wheat seedling in the image, and manufacturing a wheat seedling data set;
Step 13, defining a first deep learning network;
and 14, training the deep learning network by using the wheat seedling data set to obtain a wheat seedling detection model.
Preferably, the deep learning network for constructing the wheat seedling detection model is based on YOLOV network structure, and the optimal prior frame size is obtained through a K-means++ clustering algorithm so as to improve the detection accuracy. The method is that K cluster centers are selected, the distance between each labeling frame and the K cluster centers is calculated, the distances are divided into the cluster center categories closest to the K cluster centers, and the cluster centers are continuously updated until the cluster centers are not changed.
In step 2, the construction of the wheat seedling classification model specifically comprises the following steps in sequence:
step 21, collecting wheat seedling images at multiple angles by using an unmanned plane in a wheat seedling period;
And 22, detecting the wheat seedling image by using a wheat seedling detection model to obtain a wheat seedling individual image, removing the image with poor quality, marking the position and the grade of the wheat seedling in the image, dividing the wheat seedling into four grades of 1 leaf, 2 leaves, 3 leaves and more than 3 leaves according to the number of the seedling leaves, and manufacturing a wheat seedling grading data set.
Step 23, defining a second deep learning network;
And step 24, training the deep learning network by using the wheat seedling classification data set to obtain a wheat seedling classification model.
Preferably, the deep learning network for constructing the wheat seedling classification model adopts ResNet-18 network structure.
In step 3, the rectangular sampling block is used for measuring the density and the grading proportion of seedlings, a rectangular area is selected in the field, and white labels are inserted into the tops of the rectangles to serve as marking points, as shown in fig. 3.
In step 5, the specific method for extracting the positions of the marking points from the image of the sampling block is that firstly, the pixels of the marking points are extracted by using a threshold segmentation algorithm, noise points in the image are removed by corrosion expansion processing, then, the positions of the four marking points are found by using a K-means clustering algorithm, and finally, the marking frame of the sampling block is obtained.
Further, in step 7, a wheat seedling binarized image is obtained using the supergreen method. For wheat seedling images, the green factor for each pixel is calculated:
| (1) |
Is a green factor, which is used for the color of the green, Is the value of the pixel that is to be green,For the blue pixel value,Is the red pixel value. And replacing each pixel in the image with a corresponding green factor to obtain an ultra-green gray level image of the wheat seedlings. Then determining an optimal segmentation threshold by using a maximum inter-class variance method (OSTU)And binarizing each pixel of the ultra-green gray scale image:
| (2) |
when the pixel value is greater than the threshold value T, it is set to 255, indicating that it is a wheat seedling; when the pixel value is smaller than the threshold value T, the pixel value is set to 0, which represents that the pixel value is used as a background, so that a binarized image of the wheat seedling is obtained.
The present invention also provides various types of programmable processors (FPGA, ASIC or other integrated circuit) for running a program, wherein the program when run performs the steps of the embodiments described above.
The invention also provides corresponding computer equipment, comprising a memory, a processor and a computer program stored on the memory and capable of running on the processor, wherein the memory realizes the steps in the embodiment when the program is executed.
The following describes the experimental conditions of the seedling condition monitoring performed by the above method in detail.
The experimental field is located in the area of the agricultural university Guo Heji of Guo He Zhenan, lujiang county, anhui province (30°57'-31°33' north latitude, 117°01'-117°34') east longitude. The wheat seedling condition image acquisition time is 9 to 12 am in 11 months and 20 am in 2020, the weather is cloudy, and the planting variety is loma 23.
In the experimental process, a desktop computer is adopted as an experimental platform, an operating system is Windows10 professional, a processor is Intel (R) Core (TM) i7-6700 CPU @3.4GHz, and the memory size is 16G. The code was debugged using PyCharm editor using an anaconda3.6 scientific computing package as the whole experimental code environment. The unmanned aerial vehicle of the model MAVIC AIR in the Xinjiang is used for data acquisition in the experiment, and has the advantages of being simple in debugging, flexible in operation and the like, the duration time of flight is about 21 minutes, and the furthest flight distance is about 4 kilometers. The flight specific parameters of the unmanned aerial vehicle are shown in table 1.
Table 1 unmanned aerial vehicle flight specific parameters
| Category(s) | Parameters (parameters) | Category(s) | Parameters (parameters) |
| Weight of take-off | 430g | Maximum horizontal flight speed | 68.4Km/h (sports mode) |
| Maximum time of flight | 21 Minutes (windless environment) | Operating ambient temperature | 0 ℃ To 40 DEG C |
| Operating frequency | 2.4-2.4835GHz | Controllable rotation range of cradle head | Pitching: -90 to 0 degree |
| Battery capacity | 2357mAh | Battery voltage | Rated 11.55V |
| Battery type | LiPo 3S | Battery energy | 27.43W |
The MAVIC AIR camera has small volume and high pixel, and is very suitable for wheat image acquisition, and specific specification parameters are shown in table 2.
Table 2 MAVIC AIR unmanned aerial vehicle camera specification parameters
And selecting a proper safe take-off and landing position after the unmanned aerial vehicle is debugged, and starting image data acquisition. The hovering height of the unmanned aerial vehicle is 1.3m, and the shooting angle of the camera is 45-degree included angle with the ground. When the unmanned aerial vehicle hovers to shoot, the unmanned aerial vehicle advances to shoot along the direction of the wheat row, and each time a picture is shot, the unmanned aerial vehicle advances forward by 40cm. 48 effective images are obtained through shooting, and the resolution of the images of the wheat seedlings obtained through shooting is 4056 multiplied by 3008. In order to reduce the influence of image scaling on training, the clearer characteristics of wheat seedlings are kept as far as possible, each original image is cut into 25 parts, and the size of the cut image is 811 multiplied by 601. After cutting, removing some images with poor quality, and actually reserving 1000 images with 45 degrees obliquely downwards.
In the original image, the positions and areas of wheat seedlings are marked by using labelImg software, and rectangular boxes are used for marking. When the object is covered, the rectangular frame can cover the covered part. After labeling, clicking for storing, and automatically storing as an xml file. The interface of wheat seedlings was labeled using labelImg as shown in figure 4.
In this example, YOLOV4 deep learning networks were used for wheat seedling detection. The prior frame adopted by YOLOV network is not suitable for being directly applied to the wheat seedling data set, and the prior frame size needs to be optimized. For the a priori block A, B, the distance L between them is defined:
| (3) |
And clustering the sizes of the prior frames by adopting a K-mean++ clustering algorithm. Firstly, K initial clustering centers are selected, the distance between each marking frame in the wheat seedling data set and the K clustering centers is calculated, the marking frames are divided into categories which are closest to the clustering centers, the clustering centers are continuously updated until the clustering centers are not changed, and the optimal prior frame size is obtained. In the experiment, the clustering center number K=9, and the prior frame sizes before and after optimization are shown in table 3.
Table 3 prior frame size comparison before and after optimization
| Small target | Middle target | Large target | |
| Before optimization | 12*16，19*36，40*28 | 36*75，76*55，72*146 | 142*110，192*243，459*401 |
| After optimization | 17*58，19*83，22*101 | 22*63，24*80，28*94 | 29*69，31*112，35*89 |
Training on a self-made data set by using the optimized prior frame, training 800 pieces of training set, 100 pieces of verification set and 100 pieces of test set, training 1500 times of iteration altogether, freezing a backbone network for training in the first 500 times of iteration, and training the whole network in the last 1000 times of iteration. The batch_size is set to 16 and the data enhancement is performed using the mosaics method. The initial learning rate was set to 0.0001, and the decay rate was set to 0.00115 using the exponential decay method. The trained model is used for detection on a test set, the test set has 100 images, the detection is carried out for 134.68s, the average time for each image is 1.3468s, and the average accuracy is 93.89%.
After the optimization frame is determined, the specific implementation process and experimental results of obtaining the wheat seedling density, individual classification of seedlings and ground coverage of seedlings are respectively described below.
Wheat seedling density
In order to facilitate the extraction of the sampling block by adopting an image processing method, four white wood strips are used as four marking points of the sampling block before the image of the sampling block is acquired, and the sampling block takes a square with a side length of 1.1 m and an area of 1.21 square meters. Selecting a rectangular sampling block, inserting labels at four vertexes of the sampling block, manually counting the number of wheat seedlings in the sampling block, and calculating the density of the wheat seedlings.
For the acquired sampling image, four mark points are extracted by using a maximum entropy threshold segmentation algorithm, and then the corrosion expansion processing is carried out on the image by using a convolution check image of 7x7, so that noise points in the image are removed. And then using a K-means clustering algorithm to find the position of the marking point, writing the position of the pixel point with the pixel value of (255 ) on the image into a two-dimensional array, clustering the two-dimensional array by using the K-means clustering algorithm, finding four clustering centers, wherein the clustering centers are the positions of the marking point, and connecting the marking points to obtain a marking frame. The process is shown in fig. 5.
And detecting the image of the sampling block by using a trained wheat seedling detection model, wherein the detection result is shown in fig. 6, and a red rectangular frame is the detected wheat seedling. It can be seen that most wheat seedlings can be accurately detected, and when the seedlings overlap, individual missing detection conditions exist, and no false detection condition exists. Three sample block images were detected separately using a human and model, and the number of seedlings per sample block was counted, with the results shown in table 4.
TABLE 4 wheat seedling number for human and model statistics
| Sampling block | Area of | Number of seedlings counted manually | Model statistics of seedling number |
| Sampling block 1 | 1.21 Square meters | 88 | 82 |
| Sampling block 2 | 1.21 Square meters | 124 | 111 |
| Sampling block 3 | 1.21 Square meters | 107 | 106 |
The data of three sampling blocks are synthesized, the average mu seedling number of the artificial statistics of the test field is 58616 strains, the average mu seedling number of the model statistics is 55280 strains, the relative error is 5.6%, and the seedling density information obtained by the method has a use value.
Individual classification of wheat seedlings
And detecting the wheat seedling pictures of the training set by using the trained wheat seedling detection model, cutting the detected wheat seedlings, grading by using ResNet deep learning network, and training an individual grading model of the wheat seedlings. Then, the individual wheat seedlings in the sample block image are detected by using a wheat seedling detection model, grading is performed by using an individual wheat seedling grading model, and finally the number of seedlings at each grade is counted. The wheat seedling classification flow is shown in FIG. 7.
Cutting off the wheat seedlings obtained by the wheat seedling detection model, and dividing the seedlings into 4 grades according to the quantity of wheat leaves: 1 leaf, 2 leaf, 3 leaf and more than 3 leaf. The data set was collected for a total of 2238 images, with 285 seedlings for 1 leaf, 1131 seedling for 2 leaves, 600 seedlings for 3 leaves, and 222 seedlings for more than 3 leaves. Since the image size is mostly between 29×130 pixels and 113×149 pixels, and the training network input size is 224×224 pixels, it is necessary to fill the data before training so that the size of all images is substantially the same as the input size required by the training network. The dataset was as follows 8: the scale of 2 is divided into training and test sets.
ResNet18, which is excellent in classification, was used as a classification network after comparing various classification algorithms in a preliminary experiment. Training was started from scratch without using a pre-training weight, the learning rate was set to 0.00012, and the momentum was 0.8. The training iteration number is 100 iterations, the optimizer is SGD, and the batch size is set to 16. In the training process, as the iteration number increases, the loss value of the classification model gradually decreases, and tends to converge at the 30 th iteration. The accuracy tended to stabilize after 40 th iteration, with a maximum accuracy of 96.30% and an average accuracy after model convergence of 94.62%. The test set is used for detecting 447 pictures, the trained model is used for detecting the test set, the 425 pictures are classified correctly, the accuracy is 95.07%, and the experimental effect is good. All sample block images are selected, seedling detection is firstly carried out, then the detected individual body of the wheat seedlings is cut off, calculation is carried out by using a grading model, and the number and proportion of seedlings in each level in the three sample blocks are shown in table 5.
TABLE 5 wheat seedling fractionation test results
As can be seen from the table, the wheat seedlings of the 2-leaf grade of the test field account for the most, and the average is 53.45%; 3-leaf wheat grade, 27.32%; the other levels together account for 18.74%. From this, it was confirmed that the wheat seedlings in the test field were in the transition period from the 2-leaf period to the 3-leaf period.
Wheat seedling ground coverage
And carrying out graying treatment on the wheat seedling image by using an ultra-green method, and converting the ultra-green gray level image into a binary image by using a maximum inter-class variance method (OSTU). The comparison of the original, the super green gray scale and the OSTU binarization map is shown in fig. 8.
And obtaining the wheat growth uniformity of the test field by using the ground coverage. The continuous 100 wheat seedling images taken vertically and nodding by the unmanned aerial vehicle are sequentially selected, and the ground coverage is calculated respectively, as shown in fig. 9.
The ground coverage of the test field is calculated to be 4.415 percent at the highest, 2.237 percent at the average and 3.411 percent. The coverage of most areas fluctuates up and down around the average coverage, and the reason for larger difference of the ground coverage can be found out through field analysis, so that guidance is improved for agricultural production.
According to the invention, through unmanned aerial vehicle remote sensing and deep learning network, required equipment is simple, the complexity of detection is greatly reduced, and meanwhile, the detection stability and accuracy are greatly improved.
Although the embodiments of the present invention are described above, the embodiments are only used for facilitating understanding of the present invention, and are not intended to limit the present invention. Any person skilled in the art to which the present invention pertains may make any modifications, changes, equivalents, etc. in form and detail of the implementation without departing from the spirit and principles of the present invention disclosed herein, which are within the scope of the present invention. Accordingly, the scope of the invention should be determined from the following claims.
Claims (9)
1. A wheat seedling monitoring method based on unmanned aerial vehicle remote sensing and deep learning is characterized by comprising the following steps in sequence:
step 1, acquiring a wheat seedling image by using an unmanned aerial vehicle, manufacturing a wheat seedling data set, and training a deep learning network through the data set to obtain a wheat seedling detection model;
Step 2, dividing wheat seedlings into four grades of 1 leaf, 2 leaf, 3 leaf and more than 3 leaf according to the number of the leaves, manufacturing a wheat seedling grading data set, and training a deep learning network through the grading data set to obtain a wheat seedling grading model;
Step 3, selecting a rectangular area in the wheat field as a sampling block, inserting a label at the vertex of the rectangle as a marking point, and calculating the area S of the sampling block;
Step 4, collecting a sampling block image I at a certain angle by using a certain height of the unmanned aerial vehicle from the ground, sending the sampling block image I into a wheat seedling detection model, and detecting the positions of all seedlings in the image by using the wheat seedling detection model;
step 5, simultaneously, extracting mark points from the sampling block image I to obtain sampling block positions, counting the number N of wheat seedlings in the sampling block, and calculating to obtain the seedling density P=N/S;
Step 6, grading all seedling individuals in a sampling block marking frame by using a wheat seedling grading model, and counting to obtain the number Ni of seedlings of the ith grade, wherein i=1, 2,3 and 4 are respectively in one-to-one correspondence with four grades above 1 leaf, 2 leaf, 3 leaf and 3 leaf, and the ratio Ri=Ni/N100% of the number Ni of the seedlings of the ith grade to the number N of the seedlings is calculated;
Step 7, collecting a wheat seedling nodding image F by using a certain height of the unmanned aerial vehicle from the ground and perpendicular to the ground, obtaining a wheat seedling binarization image by using a threshold segmentation method on the nodding image F, counting the pixel number Sw of the wheat seedling and the total number S0 of image pixels, and calculating the ground coverage cover=Sw/S0 of the wheat seedling;
And 8, outputting a wheat seedling monitoring result, wherein the wheat seedling monitoring result comprises a seedling density P, the number Ni and the proportion Ri of seedlings at each level and the ground coverage Cover of the seedlings.
2. The method according to claim 1, wherein said step 1 comprises the following steps in particular:
step 11, collecting images of wheat seedlings at a certain height and at multiple angles from the ground by using an unmanned aerial vehicle in the period of the wheat seedlings;
step 12, preprocessing a wheat image, cutting the image, removing the image with poor quality, marking the position of the wheat seedling in the image, and manufacturing a wheat seedling data set;
Step 13, defining a first deep learning network;
and 14, training a first deep learning network by using the wheat seedling data set to obtain a wheat seedling detection model.
3. The method of claim 2, wherein the first deep learning network for constructing the wheat seedling detection model is based on YOLOV network structure, and the optimal prior frame size is obtained through a K-means++ clustering algorithm to improve the accuracy of detection.
4. The method of claim 1, wherein step 2 comprises the following sequential steps: step 21, collecting wheat seedling images at multiple angles by using an unmanned plane in a wheat seedling period;
Step 22, detecting a wheat seedling image by using a wheat seedling detection model to obtain a wheat seedling individual image, removing an image with poor quality, marking the position and the grade of the wheat seedling in the image, dividing the wheat seedling into four grades of 1 leaf, 2 leaves, 3 leaves and more than 3 leaves according to the number of the seedling leaves, and manufacturing a wheat seedling grading data set;
Step 23, defining a second deep learning network;
And step 24, training a second deep learning network by using the wheat seedling classification data set to obtain a wheat seedling classification model.
5. The method of claim 4, wherein the second deep learning network employs a ResNet-18 network architecture.
6. The method according to any one of claims 1 to 5, wherein in step 5, the specific method for extracting the positions of the marking points from the image of the sampling block is to extract the pixels of the marking points by using a threshold segmentation algorithm, remove the noise points in the image by a corrosion expansion process, find the positions of the four marking points by using a K-means clustering algorithm, and finally obtain the marking frame of the sampling block.
7. The method according to any one of claims 1 to 5, wherein in step 7, a wheat seedling binarized image is obtained using the supergreen method.
8. A processor for running a program, wherein the program when run performs the method of any of the preceding claims 1-7.
9. A computer device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, characterized in that the memory implements the method of any of the preceding claims 1-7 when the program is executed.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210049867.0A CN114419439B (en) | 2022-01-17 | 2022-01-17 | Wheat seedling monitoring method based on unmanned aerial vehicle remote sensing and deep learning |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210049867.0A CN114419439B (en) | 2022-01-17 | 2022-01-17 | Wheat seedling monitoring method based on unmanned aerial vehicle remote sensing and deep learning |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN114419439A CN114419439A (en) | 2022-04-29 |
| CN114419439B true CN114419439B (en) | 2024-06-21 |
Family
ID=81274373
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202210049867.0A Active CN114419439B (en) | 2022-01-17 | 2022-01-17 | Wheat seedling monitoring method based on unmanned aerial vehicle remote sensing and deep learning |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN114419439B (en) |
Citations (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111340141A (en) * | 2020-04-20 | 2020-06-26 | 天津职业技术师范大学(中国职业培训指导教师进修中心) | Crop seedling and weed detection method and system based on deep learning |
Family Cites Families (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| PL3942465T3 (en) * | 2019-03-19 | 2023-08-21 | Bühler AG | Industrialized system for rice grain recognition and method thereof |
| CN113222991A (en) * | 2021-06-16 | 2021-08-06 | 南京农业大学 | Deep learning network-based field ear counting and wheat yield prediction |
-
2022
- 2022-01-17 CN CN202210049867.0A patent/CN114419439B/en active Active
Patent Citations (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111340141A (en) * | 2020-04-20 | 2020-06-26 | 天津职业技术师范大学(中国职业培训指导教师进修中心) | Crop seedling and weed detection method and system based on deep learning |
Non-Patent Citations (1)
| Title |
|---|
| 基于多元岭回归估计小麦种植密度;刘哲;乌伟;张善文;崔倩倩;李瑞洋;;中国农业科技导报;20200612(第06期);全文 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN114419439A (en) | 2022-04-29 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN110378909B (en) | Single wood segmentation method for laser point cloud based on Faster R-CNN | |
| CN106971167B (en) | Crop growth analysis method and system based on unmanned aerial vehicle platform | |
| CN104881865B (en) | Forest pest and disease monitoring method for early warning and its system based on unmanned plane graphical analysis | |
| CN111340826B (en) | Aerial image single tree crown segmentation algorithm based on super pixels and topological features | |
| CN110517311A (en) | Pest and disease monitoring method based on leaf spot lesion area | |
| Zhou et al. | An integrated skeleton extraction and pruning method for spatial recognition of maize seedlings in MGV and UAV remote images | |
| CN110956187A (en) | Unmanned aerial vehicle image plant canopy information extraction method based on ensemble learning | |
| CN113012150A (en) | Feature-fused high-density rice field unmanned aerial vehicle image rice ear counting method | |
| CN112418188A (en) | Crop growth whole-course digital assessment method based on unmanned aerial vehicle vision | |
| CN113822198B (en) | Peanut growth monitoring method, system and medium based on UAV-RGB image and deep learning | |
| CN106228136A (en) | Panorama streetscape method for secret protection based on converging channels feature | |
| CN117409339A (en) | Unmanned aerial vehicle crop state visual identification method for air-ground coordination | |
| CN117456358A (en) | Method for detecting plant diseases and insect pests based on YOLOv5 neural network | |
| CN115331100A (en) | Spatial distribution monitoring method and system for cultivated land planting attributes | |
| Zhong et al. | Identification and depth localization of clustered pod pepper based on improved Faster R-CNN | |
| Silva et al. | Sugarcane crop line detection from UAV images using genetic algorithm and Radon transform | |
| CN111241905A (en) | Power transmission line nest detection method based on improved SSD algorithm | |
| Lin et al. | A novel approach for estimating the flowering rate of litchi based on deep learning and UAV images | |
| He et al. | A calculation method of phenotypic traits of soybean pods based on image processing technology | |
| Zou et al. | The fusion of satellite and unmanned aerial vehicle (UAV) imagery for improving classification performance | |
| CN109166127A (en) | A kind of wearable plant phenotype sensory perceptual system | |
| CN114419439B (en) | Wheat seedling monitoring method based on unmanned aerial vehicle remote sensing and deep learning | |
| CN111950524A (en) | Orchard local sparse mapping method and system based on binocular vision and RTK | |
| CN116721344A (en) | Vegetation detection method, device and equipment based on aerial photographing equipment | |
| CN115035423B (en) | Hybrid rice parent and parent identification extraction method based on unmanned aerial vehicle remote sensing image |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant |