CN109886155B - Single-plant rice detection and positioning method, system, equipment and medium based on deep learning - Google Patents
Single-plant rice detection and positioning method, system, equipment and medium based on deep learning Download PDFInfo
- Publication number
- CN109886155B CN109886155B CN201910089817.3A CN201910089817A CN109886155B CN 109886155 B CN109886155 B CN 109886155B CN 201910089817 A CN201910089817 A CN 201910089817A CN 109886155 B CN109886155 B CN 109886155B
- Authority
- CN
- China
- Prior art keywords
- rice
- network
- image
- image data
- feature information
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Landscapes
- Image Analysis (AREA)
Abstract
The invention discloses a method, a system, equipment and a medium for detecting and positioning single-plant rice based on deep learning, wherein the method comprises the following steps: acquiring image data of a paddy field rice sample; preprocessing and labeling the image data of the paddy field rice sample to obtain preprocessed image data; establishing a deep convolutional neural network detection model; performing optimization training on the deep convolutional neural network detection model by using the preprocessed image data; and detecting and positioning the single rice of the image to be detected of the rice in the field by using the trained deep convolution neural network detection model. The invention adopts a detection method based on deep learning in computer vision, can greatly improve the defects of the prior art, extracts the plant height dimension space semantic features by designing a deep convolutional neural network model, still has good positioning precision and robustness in a complex environment, and can be widely applied to the automatic and intelligent production management of agriculture.
Description
Technical Field
The invention relates to a detection positioning method, in particular to a method, a system, equipment and a medium for detecting and positioning single-plant rice based on deep learning, belonging to the field of target detection of computer vision.
Background
The rice is one of the main grain crops in the world, and the planting area of the rice in the end of 20 world reaches 1.54 hundred million hm2. The yield of China accounts for 40% of the total grain yield, and rice production is responsible for ensuring the grain safety of China. The reduction of the input-output ratio and the increase of the economic benefit of rice planting are very important. Weeds are one of the main reasons for causing the yield reduction of rice, and compete with crops for resources such as water, nutrients, sunlight and the like, so that the yield reduction of rice is serious. On the other hand, the rice is subjected to mechanical operations such as accurate irrigation, fertilization or pesticide spraying, the utilization rate and the efficiency of agricultural materials can be improved, the waste of production data and the environmental pollution caused by the waste are reduced, and ecological water resources, air and soil are protected.
Therefore, the control and management of the intelligent agricultural operation are very important, and the intelligent rice precise positioning technology is adopted to realize mechanical intelligent weeding, intelligent fine spraying of agricultural pesticide and fertilizer and the like, so that the yield per unit of rice is increased, the fine and automatic level of agricultural production is improved, and the sustainable development of agriculture in China is promoted.
In recent years, the precise positioning technology is widely researched and applied in the agricultural field, and mainly comprises intelligent mechanical weeding, mechanical autonomous navigation, precise pesticide and fertilizer spraying, automatic agricultural product picking and the like. In 2011, Xuewen Wu and the like propose a rice detection method based on position and edge characteristics, which is difficult to be used in a paddy field environment, a green plant and a soil background are divided by using a color difference, a crop center is determined by using a pixel histogram, and a crop edge is used as an end point to fill a crop area for detection. In 2015, Kazmi and the like use color and edge features to fuse to provide a new local feature to perform weed segmentation, and a Support Vector Machine (SVM) classifier is combined to complete detection of rice, but the method is greatly influenced by complex illumination in natural environment. In 2015, based on image edge detection, a fuzzy-enhancement-based crop image contour extraction algorithm is provided for positioning by analyzing a method most suitable for distinguishing weeds and crops through preprocessing operations such as graying, binarization and denoising on field shot images. In 2017, a method for distinguishing and fitting edges of stem base parts is provided for plant positioning in Jiang Ji and the like, and the problem of inaccurate positioning caused by continuous rice canopies in a rice weeding period is solved. However, the method has poor robustness and poor positioning effect on paddy rice with various forms and reflective ponding soil. In summary, most of the methods studied by experts at home and abroad are based on the low-dimensional characteristics of the color, shape, texture and the like of crops, and few of the methods are combined with the characteristics of artificial design to perform detection and positioning. These methods are often susceptible to factors such as complex lighting background, plant shading, canine-tooth staggered canopy, etc., and thus cannot achieve the required positioning accuracy, even cause erroneous positioning.
Disclosure of Invention
In view of the above, the invention provides a method, a system, equipment and a medium for detecting and positioning single-plant rice based on deep learning, which adopt a detection method based on deep learning in computer vision, can greatly improve the defects of the prior art, extract the plant high-dimensional space semantic features by designing a deep convolutional neural network model, still have good positioning accuracy and robustness in a complex environment, and can be widely applied to the automatic and intelligent production management of agriculture.
The invention aims to provide a method for detecting and positioning single-plant rice based on deep learning.
The second purpose of the invention is to provide a single-plant rice detection and positioning system based on deep learning.
It is a third object of the invention to provide a computer apparatus.
It is a fourth object of the present invention to provide a storage medium.
The first purpose of the invention can be achieved by adopting the following technical scheme:
a method for detecting and positioning single-plant rice based on deep learning comprises the following steps:
acquiring image data of a paddy field rice sample;
preprocessing and labeling the image data of the paddy field rice sample to obtain preprocessed image data;
establishing a deep convolutional neural network detection model;
performing optimization training on the deep convolutional neural network detection model by using the preprocessed image data;
and detecting and positioning the single rice of the image to be detected of the rice in the field by using the trained deep convolution neural network detection model.
Further, the preprocessing and labeling of the image data of the rice sample in the field specifically includes:
cleaning the image data of the rice sample in the field, and abandoning the image data without rice sample or difficult rice plant visual identification of people;
according to proposed targetMarking the image data of the cleaned paddy field rice sample; wherein the proposed standard is: defining a stem base circular area A with the radius of gamma by taking the center of the stem base of the plant as the center of a circle1Circle A2Is a radius twice that of circle A1Taking the concentric circle of (A)2The circumscribed square of (a) is used as a detection real label of the single-plant rice example.
Further, the establishing of the deep convolutional neural network detection model specifically includes:
inputting the preprocessed image after scaling operation into an extracted image characteristic information subnetwork, and outputting a depth convolution high-dimensional spatial characteristic information map of the image through the extracted image characteristic information subnetwork;
inputting the feature information graph output by the sub-network for extracting the image feature information into a candidate region extraction sub-network, and outputting R high-quality candidate regions possibly containing rice through the candidate region extraction sub-network;
inputting the feature information graph output by the extracted image feature information sub-network and the candidate area output by the candidate area extraction sub-network into the detection positioning sub-network, and detecting the position of the rice in the feature information graph output by the detection positioning sub-network.
Further, the scaling operation includes: and scaling the long edge and the short edge of the preprocessed image in an equal proportion mode, so that the short edge of the preprocessed image is smaller than or equal to a first preset pixel value, and the long edge of the preprocessed image is smaller than or equal to a second preset pixel value.
Furthermore, the top end of the candidate region extraction sub-network is a convolutional layer, two convolutional layer branches are connected after the convolutional layer, and the two convolutional layer branches are respectively used for classification and regression of the candidate region;
the sub-network outputs R high-quality candidate regions possibly containing rice through the candidate region extraction, which specifically comprises the following steps: performing point-by-point sliding convolution on a feature information graph output by an extraction image feature information subnetwork through a convolutional layer, generating K candidate regions at each sliding center, sending the K candidate regions into two convolutional layer branches, wherein the number of convolutional layer branch output channels for candidate region classification is Kx 2, two classification scores for classifying the candidate regions into a foreground and a background are represented, the number of convolutional layer branch output channels for candidate region regression is Kx 4, and four correction values for K candidate region boundary frames are represented; carrying out non-maximum suppression on the first T candidate regions which are divided into the convolutional layer branches used for classifying the candidate regions and have higher foreground class scores to remove redundant candidate regions, and outputting fewer R (R < T) high-quality candidate regions which possibly contain rice;
wherein generating K candidate regions comprises: according to the width W and the height H of the characteristic information graph, setting an anchor point at every interval of D pixels on a row and a column of an input image after zooming operation, wherein the anchor points are W multiplied by H in total, each anchor point is used as a center to generate K candidate regions, the K candidate regions are divided into U groups according to different area sizes, the area sizes of the candidate regions in each group are the same, and the W multiplied by H multiplied by K candidate regions are generated by all anchor points in total.
Furthermore, the top end of the detection positioning sub-network is a pooling layer outputting a feature information map with a fixed size, a plurality of convolutional layers/full-link layers are sequentially stacked behind the pooling layer for further extracting R candidate region feature information, two convolutional layer/full-link layer branches are connected behind the plurality of convolutional layers/full-link layers, and the two convolutional layer/full-link layer branches are respectively used for classification and regression of candidate regions;
the method for detecting the position of the rice in the characteristic information graph output by the detection positioning sub-network specifically comprises the following steps: further extracting R candidate region characteristic information, wherein the number of the convolutional layer/full-link layer branch output channels for candidate region classification is 2, the score of the candidate region classification into rice and background is shown, the number of the convolutional layer/full-link layer branch output channels for candidate region regression is 4 multiplied by 2, and four regression correction values of the boundary frames of the two candidate regions of the rice and the background are shown; and (4) after carrying out non-maximum suppression and redundancy removal on all candidate regions belonging to the rice category scores, outputting the final detected and positioned rice candidate regions.
Further, the optimal training of the deep convolutional neural network detection model by using the preprocessed image data specifically comprises:
randomly extracting a plurality of preprocessed image data from the preprocessed image data training set every time of iteration to form a batch for updating parameters of the whole deep convolution neural network detection model, and performing optimization training by adopting a back propagation and random gradient descent algorithm; and iterating the optimization training process for E times, setting the initial learning rate to lr, reducing the learning rate to one tenth of the original learning rate after each step of iterative training, and ending the training convergence when the whole training process iterates until the loss function tends to be stable and does not decrease.
The second purpose of the invention can be achieved by adopting the following technical scheme:
a deep learning-based single-plant rice detection and positioning system, comprising:
the acquisition module is used for acquiring image data of a rice sample in a field;
the system comprises a preprocessing module, a data processing module and a data processing module, wherein the preprocessing module is used for preprocessing and labeling the image data of the paddy field rice sample to obtain preprocessed image data;
the establishing module is used for establishing a deep convolutional neural network detection model;
the training module is used for carrying out optimization training on the deep convolutional neural network detection model by using the preprocessed image data;
and the detection module is used for detecting and positioning the individual rice of the image to be detected of the rice in the field by utilizing the trained deep convolution neural network detection model.
The third purpose of the invention can be achieved by adopting the following technical scheme:
a computer device comprises a processor and a memory for storing a program executable by the processor, wherein when the processor executes the program stored in the memory, the method for detecting and positioning the rice on the single plant is realized.
The fourth purpose of the invention can be achieved by adopting the following technical scheme:
a storage medium stores a program, and when the program is executed by a processor, the single-plant rice detection and positioning method is realized.
Compared with the prior art, the invention has the following beneficial effects:
1. the invention adopts a detection method based on deep learning in computer vision, can accurately position the single-plant rice in the image, has profound guiding significance in field operations such as effective irrigation and fertilization of the single-plant rice, can improve the resource utilization rate and the efficiency, reduce the waste of production data and the pollution of pesticide and fertilizer to the environment, can also protect ecological water resources, air and soil, has universality and has wide application scenes.
2. The invention deeply learns the knowledge in the leading edge field in computer vision, applies to the automatic and intelligent production management of agriculture, extracts the high-dimensional spatial semantic features of rice plants by designing a deep convolutional neural network model, still has good positioning precision and robustness for rice detection and positioning in the environments of complex illumination background, plant mutual shielding, canine-tooth staggered canopy and the like, and is a novel agricultural automatic and intelligent technology with great development potential at present.
Drawings
In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the embodiments or the prior art will be briefly described below, it is obvious that the drawings in the following description are only some embodiments of the present invention, and for those skilled in the art, other drawings can be obtained according to the structures shown in the drawings without creative efforts.
Fig. 1 is a flowchart of a method for detecting and locating individual rice plants based on deep learning in embodiment 1 of the present invention.
Fig. 2 is a structural diagram of the application of the deep learning-based single-plant rice detection and positioning method in detecting and positioning to-be-detected images of rice in a field in embodiment 1 of the present invention.
Fig. 3 is a block diagram of the image feature information extraction sub-network according to embodiment 1 of the present invention.
Fig. 4 is a configuration diagram of a candidate area extraction sub-network according to embodiment 1 of the present invention.
Fig. 5 is a block diagram of a detection and positioning sub-network according to embodiment 1 of the present invention.
Fig. 6a to 6d are graphs showing the results of locating rice by outdoor testing according to the deep learning-based single-plant rice testing and locating method of embodiment 1 of the present invention.
Fig. 7 is a block diagram of a system for detecting and locating individual rice based on deep learning in embodiment 2 of the present invention.
Fig. 8 is a block diagram of a preprocessing module according to embodiment 2 of the present invention.
Fig. 9 is a block diagram of a building module according to embodiment 2 of the present invention.
Fig. 10 is a block diagram of a detection module according to embodiment 2 of the present invention.
Fig. 11 is a block diagram of a computer device according to embodiment 3 of the present invention.
Detailed Description
In order to make the objects, technical solutions and advantages of the embodiments of the present invention clearer and more complete, the technical solutions in the embodiments of the present invention will be described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are some embodiments of the present invention, but not all embodiments, and all other embodiments obtained by a person of ordinary skill in the art without creative efforts based on the embodiments of the present invention belong to the protection scope of the present invention.
Example 1:
as shown in fig. 1 and fig. 2, this embodiment provides a method for detecting and locating individual rice based on deep learning, which includes the following steps:
and S1, acquiring image data of the paddy field rice sample.
The field rice sample image data of this embodiment can acquire through image acquisition system collection, this image acquisition system includes color camera, the camera lens, a computer, shade, mounting platform etc, install this image acquisition system on agricultural weeding machine, it is 2 meters per second to set for mechanical speed of advance, image acquisition system frame rate is 10 frames per second, field operation processes such as simulation weeding, fertilize, use image acquisition system to gather the field rice image that awaits measuring of different forms, the diversity of fully increased image database, thereby the generalization ability and the positioning accuracy of reinforcing degree of depth convolution neural network model.
And S2, preprocessing and labeling the image data of the paddy field rice sample to obtain preprocessed image data.
The step S2 specifically includes:
s201, data cleaning: and (4) cleaning the image data of the rice sample in the field, and discarding the image data without rice sample or difficult rice plant visual identification of people.
S202, marking data: marking the image data of the cleaned paddy field rice sample according to a proposed standard; wherein, the proposed standard is as follows: defining a stem base circular area A with the radius of gamma by taking the center of the stem base of the plant as the center of a circle1Circle A2Is a radius twice that of circle A1Taking the concentric circle of (A)2The circumscribed square of (a) is used as a detection real label of the single-plant rice example. The standard refers to the industrial standard of weeding or accurately spraying pesticide and fertilizer in rice fields.
And S3, establishing a deep convolutional neural network detection model.
The deep convolutional neural network detection model of the embodiment comprises an image feature information extraction sub-network, a candidate region extraction sub-network and a detection positioning sub-network, wherein the three sub-networks form a whole, so that the detection process of the deep convolutional neural network detection model is finished end to end.
The step S3 specifically includes:
s301, inputting the preprocessed image data after the scaling operation into an image feature information extraction sub-network, and outputting a depth convolution high-dimensional space feature information map of the image with the size of W multiplied by H through the image feature information extraction sub-network.
The sub-network for extracting image feature information in this embodiment uses a deep convolutional neural network to extract image information, where the deep convolutional neural network generally uses a classical efficient structural model, for example, the series of network structural models of VGG13, VGG16, VGG19, and the like, which are invented by the Visual Geometry Group (Visual Geometry Group) of oxford university, Facebook artificial intelligence research institute (FAIR) hokeming scientistThe deep residual network series of the invention of et al (ResNet18, ResNet50, ResNet101), and so on. As shown in fig. 3, the sub-network for extracting image feature information of the present embodiment employs the VGG16 model. The scaling operation specifically includes: generally, an image input into the neural network model is small, for example, 224 pixels × 224 pixels, 480 pixels × 640 pixels, and the like, but in the embodiment, the long side and the short side of the preprocessed image data are scaled, so that the short side is smaller than or equal to a first preset pixel value, and the long side is smaller than or equal to a second preset pixel value, the first preset pixel value is 600 pixels, and the second preset pixel value is 1000 pixels. Specifically, if the original image (preprocessed image) is M1×N1Pixel without M1Is a shorter side, N1Is a longer edge. According to the original image M1:N1Such that the short side M is scaled1To the specified length of 600 pixels, the other edge is set to be N after scaling2Now the image size is 600 XN2A pixel.
S302, inputting the feature information map output by the image feature information extraction sub-network into a candidate region extraction sub-network, outputting 300 high-quality candidate regions possibly containing rice through the candidate region extraction sub-network, and providing the candidate regions for a subsequent detection positioning sub-network to further detect and position.
As shown in fig. 4, the top of the candidate region extraction sub-network of this embodiment is a convolution layer with convolution kernel size of 3 × 3, and two convolution layer branches with convolution kernel size of 1 × 1 are connected after the convolution layer, and the two convolution layer branches are respectively used for classification and regression of the candidate region.
Outputting 300 candidate regions possibly containing rice through the candidate region extraction sub-network, specifically: performing point-by-point sliding convolution on a feature information graph output by an extraction image feature information subnetwork through a convolutional layer, generating 4 candidate regions at each sliding center, sending the 4 candidate regions into two convolutional layer branches, wherein the number of convolutional layer branch output channels for candidate region classification is 4 multiplied by 2, two classification scores representing the classification of the candidate regions into a foreground and a background are represented, the number of convolutional layer branch output channels for candidate region regression is 4 multiplied by 4, and four correction values for 4 candidate region boundary frames are represented; and performing Non-maximum suppression (NMS) on the top T-6000 candidate regions with high foreground class scores in the convolutional layer branches used for candidate region classification to remove redundant candidate regions, and outputting less R-300 high-quality candidate regions possibly containing rice.
Wherein generating 4 candidate regions comprises: according to the width W and the height H of the feature information map, an anchor point is set for each 16 pixels at an interval D on the row and the column of the input image after the scaling operation, W × H anchor points are provided in total, each anchor point generates 4 candidate regions for the center, and the 4 candidate regions in the embodiment have the same area size, so that only U is divided into 1 group of candidate regions, and the length-width ratio of the 4 candidate regions is 1: 1, it is understood that the aspect ratio of the 4 candidate regions may be different, and all anchor points generate W × H × 4 candidate regions in total.
S303, inputting the feature information graph output by the sub-network for extracting the image feature information and the candidate region output by the sub-network for extracting the candidate region into a detection and positioning sub-network, and detecting the position of the rice in the feature information graph output by the detection and positioning sub-network, namely the expected output of the deep convolutional neural network detection model.
As shown in fig. 5, the top of the sub-network for location detection in this embodiment is a pooling layer outputting a feature information map with a fixed size, two full-connected layers are sequentially stacked behind the pooling layer for further extracting R candidate region feature information, and two full-connected layer branches are connected behind the two full-connected layers and are respectively used for classification and regression of candidate regions; it is understood that the fully-connected layer and the fully-connected layer branches may also be convolutional layers and convolutional layer branches.
The method comprises the following steps of detecting the position of rice in a positioning sub-network output characteristic information graph, specifically: further extracting R candidate region characteristic information, wherein the number of the convolutional layer/full-link layer branch output channels for candidate region classification is 2, the score of the candidate region classification into rice and background is shown, the number of the convolutional layer/full-link layer branch output channels for candidate region regression is 4 multiplied by 2, and four regression correction values of the boundary frames of the two candidate regions of the rice and the background are shown; and (3) after redundancy of all candidate regions belonging to the rice category score is removed through Non-maximum suppression (NMS), outputting the final detected and positioned rice candidate region.
And S4, optimally training the deep convolutional neural network detection model by using the preprocessed image data.
Specifically, from the training set of the preprocessed image data, 64 preprocessed image data are randomly extracted each time in an iteration mode to form a batch, parameters of the whole deep convolution neural network detection model are updated, and optimal training is carried out by adopting a back propagation and random gradient descent algorithm; the optimization training process is iterated for 1000 times, the initial learning rate is set to be lr equal to 0.0001, the learning rate is reduced to one tenth of the original learning rate after each step equal to 100 times of iterative training, the momentum parameter is set to be 0.9, the weight attenuation is 0.0005, and the image feature information subnetwork is extracted, and the top end part of the detection locator subnetwork is initialized by using an ImageNet classification pre-training model. The whole training process is iterated until the loss function tends to be stable and does not fall, and then the training convergence is finished.
And S5, detecting and positioning the individual rice of the to-be-detected image of the paddy field rice by using the trained deep convolutional neural network detection model.
The step S5 specifically includes:
s501, acquiring an image to be detected of the paddy field rice: the same image acquisition system as that in step S1 was used, and the system was mounted on a machine for agricultural weeding or the like, and the field rice image to be measured was acquired at a traveling speed of 2 meters per second and a frame rate of 10 frames per second, and the color camera transmitted the acquired field rice image to be measured to a computer.
S502, detecting the to-be-detected image of the paddy field through the trained deep convolutional neural network model, and outputting the specific position of the paddy field, so that intelligent field weeding operation of agricultural machinery is realized, resources such as irrigation water fertilizers and pesticides are intelligently controlled, the resource utilization rate and the efficiency are improved, waste of production data and pollution of the pesticide fertilizers to the environment are reduced, and ecological water resources, air and soil are protected.
Fig. 6a to 6d are result graphs of the single-plant rice detection method in this embodiment for detecting single-plant rice in an outdoor field, where the gray and black frames in the graph are the position and size of the rice to be detected and located, it can be seen that the center of the rice to be detected and located is substantially overlapped with the actual center of the plant, and the plants with different dimensions in the visual field range are detected and located, so that the effect is very accurate.
Those skilled in the art will appreciate that all or part of the steps in the method for implementing the above embodiments may be implemented by a program to instruct related hardware, and the corresponding program may be stored in a computer-readable storage medium.
It should be noted that although the method operations of the above-described embodiments are depicted in the drawings in a particular order, this does not require or imply that these operations must be performed in this particular order, or that all of the illustrated operations must be performed, to achieve desirable results. Rather, the depicted steps may change the order of execution. Additionally or alternatively, certain steps may be omitted, multiple steps combined into one step execution, and/or one step broken down into multiple step executions.
Example 2:
as shown in fig. 7, the embodiment provides a deep learning-based single-plant rice detection and positioning system, which includes an obtaining module 701, a preprocessing module 702, an establishing module 703, a training module 704, and a detecting module 705, where the specific functions of the modules are as follows:
the acquiring module 701 is used for acquiring field rice sample image data.
The preprocessing module 702 is configured to preprocess and label the image data of the paddy field rice sample to obtain preprocessed image data.
Further, as shown in fig. 8, the preprocessing module 702 specifically includes:
A labeling unit 7022, configured to label the image data of the washed rice sample in the field according to a proposed standard; wherein the proposed standard is: defining a stem base circular area A with the radius of gamma by taking the center of the stem base of the plant as the center of a circle1Circle A2Is a radius twice that of circle A1Taking the concentric circle of (A)2The circumscribed square of (a) is used as a detection real label of the single-plant rice example.
The establishing module 703 is configured to establish a deep convolutional neural network detection model.
Further, as shown in fig. 9, the establishing module 703 specifically includes:
first input/output section 7031 is configured to input the preprocessed image data subjected to the scaling operation into the extracted image feature information sub-network, and output the depth convolution high-dimensional spatial feature information map of the image by the extracted image feature information sub-network.
Second input/output section 7032 is configured to input the feature information map output from the sub-network for extracting image feature information into the sub-network for candidate region extraction, and output R high-quality candidate regions that may include rice through the sub-network for candidate region extraction.
Third input/output section 7033 is configured to input the feature information map output from the sub-network for extracting image feature information and the candidate region output from the candidate region extracting sub-network into the detection and localization sub-network, and output the feature information map by the detection and localization sub-network to determine the location of the rice.
The training module 704 is configured to perform optimization training on the deep convolutional neural network detection model by using the preprocessed image data.
The detection module 705 is configured to detect and locate individual rice plants in the image to be detected of the field rice by using the trained deep convolutional neural network detection model.
Further, as shown in fig. 10, the detecting module 705 specifically includes:
an obtaining unit 7051 is configured to obtain an image to be detected of the paddy field.
And the detection output unit 7052 is configured to detect an image to be detected of the rice in the field through the trained deep convolutional neural network model, and output a specific position of the rice.
For specific implementation of each module and unit in this embodiment, reference may be made to embodiment 1, which is not described herein again. It should be noted that the system provided in the foregoing embodiment is only illustrated by the division of the functional modules, and in practical applications, the above functions may be distributed by different functional modules as needed, that is, the internal structure is divided into different functional modules to complete all or part of the functions described above.
It will be understood that the terms "first," "second," and the like as used in the systems of the above embodiments may be used to describe various elements, but the elements are not limited by these terms. These terms are only used to distinguish one element from another. For example, a first input/output unit may be referred to as a second input/output unit, and similarly, a second input/output unit may be referred to as a first input/output unit, both of which are input/output units, but which are not the same input/output unit, without departing from the scope of the present invention.
Example 3:
as shown in fig. 11, the present embodiment provides a computer apparatus, which may be a computer, including a processor 1102, a memory, an input device 1103, a display 1104, and a network interface 1105 connected by a system bus 1101. The processor 1102 is configured to provide computing and control capabilities, the memory includes a nonvolatile storage medium 1106 and an internal memory 1107, the nonvolatile storage medium 1106 stores an operating system, a computer program, and a database, the internal memory 1107 provides an environment for operating the operating system and the computer program in the nonvolatile storage medium 1106, and when the computer program is executed by the processor 1102, the method for detecting and locating single-plant rice in embodiment 1 is implemented as follows:
acquiring image data of a paddy field rice sample;
preprocessing and labeling the image data of the paddy field rice sample to obtain preprocessed image data;
establishing a deep convolutional neural network detection model;
performing optimization training on the deep convolutional neural network detection model by using the preprocessed image data;
and detecting and positioning the single rice of the image to be detected of the rice in the field by using the trained deep convolution neural network detection model.
The computer device described in this embodiment may also be a server or other terminal devices with a computing function.
Example 4:
the present embodiment provides a storage medium, which is a computer-readable storage medium, and stores a computer program, and when the program is executed by a processor, and the processor executes the computer program stored in the memory, the method for detecting and locating individual rice in embodiment 1 is implemented as follows:
acquiring image data of a paddy field rice sample;
preprocessing and labeling the image data of the paddy field rice sample to obtain preprocessed image data;
establishing a deep convolutional neural network detection model;
performing optimization training on the deep convolutional neural network detection model by using the preprocessed image data;
and detecting and positioning the single rice of the image to be detected of the rice in the field by using the trained deep convolution neural network detection model.
The storage medium in this embodiment may be a magnetic disk, an optical disk, a computer Memory, a Read-Only Memory (ROM), a Random Access Memory (RAM), a usb disk, a removable hard disk, or other media.
In conclusion, the invention adopts a detection method based on deep learning in computer vision, accurately positions the single-plant rice in the image, has profound guiding significance in field operations such as effective irrigation and fertilization of the single-plant rice, can improve the resource utilization rate and the efficiency, reduce the waste of production data and the pollution of pesticide and fertilizer to the environment, can protect ecological water resources, air and soil, has universality and universality, and has wide application scenes.
The above description is only for the preferred embodiments of the present invention, but the protection scope of the present invention is not limited thereto, and any person skilled in the art can substitute or change the technical solution and the inventive concept of the present invention within the scope of the present invention.
Claims (7)
1. A method for detecting and positioning single-plant rice based on deep learning is characterized by comprising the following steps:
acquiring image data of a paddy field rice sample;
preprocessing and labeling the image data of the paddy field rice sample to obtain preprocessed image data;
establishing a deep convolutional neural network detection model;
performing optimization training on the deep convolutional neural network detection model by using the preprocessed image data;
detecting and positioning the single rice of the image to be detected of the rice in the field by utilizing the trained deep convolution neural network detection model;
the preprocessing and labeling of the image data of the paddy field rice sample specifically comprises the following steps:
cleaning the image data of the rice sample in the field, and abandoning the image data without rice sample or difficult rice plant visual identification of people;
marking the image data of the cleaned paddy field rice sample according to a proposed standard; wherein the proposed standard is: defining a stem base circular area A with the radius of gamma by taking the center of the stem base of the plant as the center of a circle1Circle A2Is a radius twice that of circle A1Taking the concentric circle of (A)2The circumscribed square of (a) is used as a detection real label of a single-plant rice example;
the method comprises the steps that image data of a paddy field rice sample are acquired through an image acquisition system, the image acquisition system comprises a color camera, a lens, a computer, a shading device and an installation platform, the image acquisition system is installed on agricultural weeding machinery, the mechanical advancing speed is set to be 2 meters per second, the frame rate of the image acquisition system is set to be 10 frames per second, and the field operation process is simulated;
the deep convolutional neural network detection model comprises an image feature information extraction sub-network, a candidate region extraction sub-network and a detection positioning sub-network, the three sub-networks form a whole, so that the detection process of the deep convolutional neural network detection model is finished end to end, the image feature information extraction sub-network adopts a VGG16 model, a preprocessed image after scaling operation is input into the image feature information extraction sub-network, and a deep convolutional high-dimensional spatial feature information map of the image is output through the image feature information extraction sub-network; inputting the feature information graph output by the sub-network for extracting the image feature information into a candidate region extraction sub-network, and outputting R high-quality candidate regions possibly containing rice through the candidate region extraction sub-network; inputting the feature information graph output by the extracted image feature information sub-network and the candidate region output by the candidate region extraction sub-network into a detection positioning sub-network, and detecting the position of the positioning rice in the feature information graph output by the detection positioning sub-network;
the scaling operation comprises: scaling the data of the preprocessed image in a manner of equal proportion of long edge and short edge to make the short edge smaller than or equal to a first preset pixel value and the long edge smaller than or equal to a second preset pixel value, and if the preprocessed image is M1×N1Pixel, set M1Is a shorter side, N1Is a longer edge, according to the preprocessed image M1:N1Such that the short side M is scaled1To the specified length of 600 pixels, the other edge is set to be N after scaling2Now the image size is 600 XN2A pixel.
2. The method for detecting and locating single-plant rice as claimed in claim 1, wherein the top of the candidate region extraction subnetwork is a convolutional layer, and two convolutional layer branches are connected to the convolutional layer and used for classification and regression of the candidate region respectively;
the sub-network outputs R high-quality candidate regions possibly containing rice through the candidate region extraction, which specifically comprises the following steps: performing point-by-point sliding convolution on a feature information graph output by an extraction image feature information subnetwork through a convolutional layer, generating K candidate regions at each sliding center, sending the K candidate regions into two convolutional layer branches, wherein the number of convolutional layer branch output channels for candidate region classification is Kx 2, two classification scores for classifying the candidate regions into a foreground and a background are represented, the number of convolutional layer branch output channels for candidate region regression is Kx 4, and four correction values for K candidate region boundary frames are represented; carrying out non-maximum suppression on the first T candidate regions which are divided into the convolutional layer branches used for classifying the candidate regions and have higher foreground class scores to remove redundant candidate regions, and outputting fewer R high-quality candidate regions which possibly contain rice;
wherein generating K candidate regions comprises: according to the width W and the height H of the characteristic information graph, setting an anchor point at every interval of D pixels on a row and a column of an input image after zooming operation, wherein the anchor points are W multiplied by H in total, each anchor point is used as a center to generate K candidate regions, the K candidate regions are divided into U groups according to different area sizes, the area sizes of the candidate regions in each group are the same, and the W multiplied by H multiplied by K candidate regions are generated by all anchor points in total.
3. The method for detecting and locating individual rice plants as claimed in claim 1, wherein the top of the sub-network for detecting and locating is a pooling layer outputting a feature information map with a fixed size, a plurality of convolutional layers/fully-connected layers are sequentially stacked behind the pooling layer for further extracting R candidate region feature information, two convolutional layer/fully-connected layer branches are connected behind the plurality of convolutional layers/fully-connected layers, and the two convolutional layer/fully-connected layer branches are respectively used for classification and regression of candidate regions;
the method for detecting the position of the rice in the characteristic information graph output by the detection positioning sub-network specifically comprises the following steps: further extracting R candidate region characteristic information, wherein the number of the convolutional layer/full-link layer branch output channels for candidate region classification is 2, the score of the candidate region classification into rice and background is shown, the number of the convolutional layer/full-link layer branch output channels for candidate region regression is 4 multiplied by 2, and four regression correction values of the boundary frames of the two candidate regions of the rice and the background are shown; and (4) after carrying out non-maximum suppression and redundancy removal on all candidate regions belonging to the rice category scores, outputting the final detected and positioned rice candidate regions.
4. The method for detecting and positioning the individual rice plants according to any one of claims 1 to 3, wherein the preprocessed image data is used for carrying out optimization training on the deep convolutional neural network detection model, and specifically comprises the following steps:
randomly extracting a plurality of preprocessed image data from the preprocessed image data training set every time of iteration to form a batch for updating parameters of the whole deep convolution neural network detection model, and performing optimization training by adopting a back propagation and random gradient descent algorithm; and iterating the optimization training process for E times, setting the initial learning rate to lr, reducing the learning rate to one tenth of the original learning rate after each step of iterative training, and ending the training convergence when the whole training process iterates until the loss function tends to be stable and does not decrease.
5. A single-plant rice detection and positioning system based on deep learning is characterized by comprising:
the acquisition module is used for acquiring image data of a rice sample in a field;
the system comprises a preprocessing module, a data processing module and a data processing module, wherein the preprocessing module is used for preprocessing and labeling the image data of the paddy field rice sample to obtain preprocessed image data;
the establishing module is used for establishing a deep convolutional neural network detection model;
the training module is used for carrying out optimization training on the deep convolutional neural network detection model by using the preprocessed image data;
the detection module is used for detecting and positioning the individual rice of the image to be detected of the field rice by utilizing the trained deep convolution neural network detection model;
the preprocessing module specifically comprises:
the cleaning unit is used for cleaning the image data of the rice sample in the field and abandoning the image data without rice examples or difficult rice plant visual identification of people;
the marking unit is used for marking the image data of the cleaned paddy field rice sample according to a proposed standard; wherein the proposed standard is: defining a stem base circular area A with the radius of gamma by taking the center of the stem base of the plant as the center of a circle1Circle A2Is a radius twice that of circle A1Taking the concentric circle of (A)2The circumscribed square of (a) is used as a detection real label of a single-plant rice example;
the method comprises the steps that image data of a paddy field rice sample are acquired through an image acquisition system, the image acquisition system comprises a color camera, a lens, a computer, a shading device and an installation platform, the image acquisition system is installed on agricultural weeding machinery, the mechanical advancing speed is set to be 2 meters per second, the frame rate of the image acquisition system is set to be 10 frames per second, and the field operation process is simulated;
the deep convolutional neural network detection model comprises an image feature information extraction sub-network, a candidate region extraction sub-network and a detection positioning sub-network, the three sub-networks form a whole, so that the detection process of the deep convolutional neural network detection model is finished end to end, the image feature information extraction sub-network adopts a VGG16 model, a preprocessed image after scaling operation is input into the image feature information extraction sub-network, and a deep convolutional high-dimensional spatial feature information map of the image is output through the image feature information extraction sub-network; inputting the feature information graph output by the sub-network for extracting the image feature information into a candidate region extraction sub-network, and outputting R high-quality candidate regions possibly containing rice through the candidate region extraction sub-network; inputting the feature information graph output by the extracted image feature information sub-network and the candidate region output by the candidate region extraction sub-network into a detection positioning sub-network, and detecting the position of the positioning rice in the feature information graph output by the detection positioning sub-network;
the scaling operation comprises: for the pre-processing chartScaling the long and short sides of the image data to make the short sides less than or equal to a first preset pixel value and the long sides less than or equal to a second preset pixel value, wherein the first preset pixel value is 600 pixels and the second preset pixel value is 1000 pixels, and if the preprocessed image is M1×N1Pixel, set M1Is a shorter side, N1Is a longer edge, according to the preprocessed image M1:N1Such that the short side M is scaled1To the specified length of 600 pixels, the other edge is set to be N after scaling2Now the image size is 600 XN2A pixel.
6. A computer device comprising a processor and a memory for storing a program executable by the processor, wherein the processor, when executing the program stored in the memory, implements the method for detecting and locating rice as claimed in any one of claims 1 to 4.
7. A storage medium storing a program, wherein the program, when executed by a processor, implements the method for detecting and locating individual rice according to any one of claims 1 to 4.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910089817.3A CN109886155B (en) | 2019-01-30 | 2019-01-30 | Single-plant rice detection and positioning method, system, equipment and medium based on deep learning |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910089817.3A CN109886155B (en) | 2019-01-30 | 2019-01-30 | Single-plant rice detection and positioning method, system, equipment and medium based on deep learning |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN109886155A CN109886155A (en) | 2019-06-14 |
| CN109886155B true CN109886155B (en) | 2021-08-10 |
Family
ID=66927519
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201910089817.3A Active CN109886155B (en) | 2019-01-30 | 2019-01-30 | Single-plant rice detection and positioning method, system, equipment and medium based on deep learning |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN109886155B (en) |
Families Citing this family (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110853080A (en) * | 2019-09-30 | 2020-02-28 | 广西慧云信息技术有限公司 | Method for measuring size of field fruit |
| CN111062871B (en) * | 2019-12-17 | 2023-01-24 | 腾讯科技（深圳）有限公司 | Image processing method and device, computer equipment and readable storage medium |
| CN111462058B (en) * | 2020-03-24 | 2024-01-05 | 华南农业大学 | Method for rapidly detecting effective rice ears |
| CN111723736A (en) * | 2020-06-19 | 2020-09-29 | 中国农业科学院农业信息研究所 | Fruit tree flowering phase monitoring method and device, computer equipment and storage medium |
| CN113128576A (en) * | 2021-04-02 | 2021-07-16 | 中国农业大学 | Crop row detection method and device based on deep learning image segmentation |
| CN113129281B (en) * | 2021-04-13 | 2022-06-21 | 广西大学 | Wheat stem section parameter detection method based on deep learning |
| CN115619286B (en) * | 2022-11-11 | 2023-10-03 | 中国农业科学院农业资源与农业区划研究所 | Method and system for evaluating quality of sample plot of breeding field district |
| CN116863340A (en) * | 2023-08-16 | 2023-10-10 | 安徽荃银超大种业有限公司 | Rice leaf disease identification method based on deep learning |
Citations (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107169421A (en) * | 2017-04-20 | 2017-09-15 | 华南理工大学 | A kind of car steering scene objects detection method based on depth convolutional neural networks |
| CN107316289A (en) * | 2017-06-08 | 2017-11-03 | 华中农业大学 | Crop field spike of rice dividing method based on deep learning and super-pixel segmentation |
| CN107576618A (en) * | 2017-07-20 | 2018-01-12 | 华南理工大学 | Pyricularia Oryzae detection method and system based on depth convolutional neural networks |
| CN108416353A (en) * | 2018-02-03 | 2018-08-17 | 华中农业大学 | Crop field spike of rice fast partition method based on the full convolutional neural networks of depth |
| CN108491854A (en) * | 2018-02-05 | 2018-09-04 | 西安电子科技大学 | Remote sensing image object detection method based on SF-RCNN |
| CN109117876A (en) * | 2018-07-26 | 2019-01-01 | 成都快眼科技有限公司 | A kind of dense small target deteection model building method, model and detection method |
| CN109242844A (en) * | 2018-09-04 | 2019-01-18 | 青岛大学附属医院 | Pancreatic tumour automatic recognition system based on deep learning, computer equipment, storage medium |
| CN109255352A (en) * | 2018-09-07 | 2019-01-22 | 北京旷视科技有限公司 | Object detection method, apparatus and system |
Family Cites Families (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107871124B (en) * | 2017-11-15 | 2018-08-10 | 陕西师范大学 | A kind of Remote Sensing Target detection method based on deep neural network |
-
2019
- 2019-01-30 CN CN201910089817.3A patent/CN109886155B/en active Active
Patent Citations (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107169421A (en) * | 2017-04-20 | 2017-09-15 | 华南理工大学 | A kind of car steering scene objects detection method based on depth convolutional neural networks |
| CN107316289A (en) * | 2017-06-08 | 2017-11-03 | 华中农业大学 | Crop field spike of rice dividing method based on deep learning and super-pixel segmentation |
| CN107576618A (en) * | 2017-07-20 | 2018-01-12 | 华南理工大学 | Pyricularia Oryzae detection method and system based on depth convolutional neural networks |
| CN108416353A (en) * | 2018-02-03 | 2018-08-17 | 华中农业大学 | Crop field spike of rice fast partition method based on the full convolutional neural networks of depth |
| CN108491854A (en) * | 2018-02-05 | 2018-09-04 | 西安电子科技大学 | Remote sensing image object detection method based on SF-RCNN |
| CN109117876A (en) * | 2018-07-26 | 2019-01-01 | 成都快眼科技有限公司 | A kind of dense small target deteection model building method, model and detection method |
| CN109242844A (en) * | 2018-09-04 | 2019-01-18 | 青岛大学附属医院 | Pancreatic tumour automatic recognition system based on deep learning, computer equipment, storage medium |
| CN109255352A (en) * | 2018-09-07 | 2019-01-22 | 北京旷视科技有限公司 | Object detection method, apparatus and system |
Non-Patent Citations (2)
| Title |
|---|
| 基于Faster_R_CNN的目标检测算法的研究;伍伟明;《中国优秀硕士学位论文全文数据库信息科技辑》;20180515;第I138-508页 * |
| 基于深度卷积神经网络的水稻穗瘟病检测方法;黄双萍 等;《农业工程学报》;20171031;第33卷(第20期);第169-176页 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN109886155A (en) | 2019-06-14 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN109886155B (en) | Single-plant rice detection and positioning method, system, equipment and medium based on deep learning | |
| Kamilaris et al. | Deep learning in agriculture: A survey | |
| Zhao et al. | Rapeseed seedling stand counting and seeding performance evaluation at two early growth stages based on unmanned aerial vehicle imagery | |
| Rivera et al. | LiDAR applications in precision agriculture for cultivating crops: A review of recent advances | |
| Vidović et al. | Crop row detection by global energy minimization | |
| Yu et al. | Automatic image-based detection technology for two critical growth stages of maize: Emergence and three-leaf stage | |
| Hung et al. | A feature learning based approach for automated fruit yield estimation | |
| CN110765916B (en) | Farmland seedling ridge identification method and system based on semantics and example segmentation | |
| Wang et al. | Tea picking point detection and location based on Mask-RCNN | |
| Huang et al. | Deep localization model for intra-row crop detection in paddy field | |
| Kalampokas et al. | Grape stem detection using regression convolutional neural networks | |
| CN111126287A (en) | Remote sensing image dense target deep learning detection method | |
| de Silva et al. | Towards agricultural autonomy: crop row detection under varying field conditions using deep learning | |
| Badhan et al. | Real-time weed detection using machine learning and stereo-vision | |
| Díaz et al. | Grapevine buds detection and localization in 3D space based on structure from motion and 2D image classification | |
| Gleason et al. | A Fusion Approach for Tree Crown Delineation from Lidar Data. | |
| Lu et al. | Citrus green fruit detection via improved feature network extraction | |
| Gurumurthy et al. | Mango Tree Net--A fully convolutional network for semantic segmentation and individual crown detection of mango trees | |
| Lyu et al. | Development of phenotyping system using low altitude UAV imagery and deep learning | |
| CN116977960A (en) | Rice seedling row detection method based on example segmentation | |
| Jia et al. | YOLOF-Snake: An efficient segmentation model for green object fruit | |
| Miao et al. | Crop weed identification system based on convolutional neural network | |
| Simon et al. | Weed Plant Identification and Removal Using Machine Vision and Robotics | |
| AHM et al. | A deep convolutional neural network based image processing framework for monitoring the growth of soybean crops | |
| Jakab et al. | Detection of Plastic Greenhouses Using High Resolution Rgb Remote Sensing Data and Convolutional Neural Network |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |