CN116188987A - Corn field multiple weed real-time identification algorithm based on Kalman filtering and deep learning - Google Patents

Corn field multiple weed real-time identification algorithm based on Kalman filtering and deep learning Download PDF

Info

Publication number
CN116188987A
CN116188987A CN202310189828.5A CN202310189828A CN116188987A CN 116188987 A CN116188987 A CN 116188987A CN 202310189828 A CN202310189828 A CN 202310189828A CN 116188987 A CN116188987 A CN 116188987A
Authority
CN
China
Prior art keywords
corn
matrix
kalman filtering
time
seedlings
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN202310189828.5A
Other languages
Chinese (zh)
Inventor
权龙哲
杨允欢
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Anhui Agricultural University AHAU
Original Assignee
Anhui Agricultural University AHAU
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Anhui Agricultural University AHAU filed Critical Anhui Agricultural University AHAU
Priority to CN202310189828.5A priority Critical patent/CN116188987A/en
Publication of CN116188987A publication Critical patent/CN116188987A/en
Pending legal-status Critical Current

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V20/00—Scenes; Scene-specific elements
    • G06V20/10—Terrestrial scenes
    • G06V20/188—Vegetation
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N20/00—Machine learning
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
    • G06V10/77—Processing image or video features in feature spaces; using data integration or data reduction, e.g. principal component analysis [PCA] or independent component analysis [ICA] or self-organising maps [SOM]; Blind source separation
    • G06V10/774—Generating sets of training patterns; Bootstrap methods, e.g. bagging or boosting
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V2201/00—Indexing scheme relating to image or video recognition or understanding
    • G06V2201/07—Target detection
    • Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
    • Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
    • Y02A—TECHNOLOGIES FOR ADAPTATION TO CLIMATE CHANGE
    • Y02A40/00—Adaptation technologies in agriculture, forestry, livestock or agroalimentary production
    • Y02A40/10—Adaptation technologies in agriculture, forestry, livestock or agroalimentary production in agriculture

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Software Systems (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Multimedia (AREA)
  • Artificial Intelligence (AREA)
  • Health & Medical Sciences (AREA)
  • Computing Systems (AREA)
  • Evolutionary Computation (AREA)
  • Medical Informatics (AREA)
  • General Health & Medical Sciences (AREA)
  • Databases & Information Systems (AREA)
  • Data Mining & Analysis (AREA)
  • General Engineering & Computer Science (AREA)
  • Mathematical Physics (AREA)
  • Image Analysis (AREA)

Abstract

The invention discloses a corn field multiple weed real-time identification algorithm based on Kalman filtering and deep learning, and belongs to the field of deep learning and precise agriculture. In the process of identifying the seedlings, a stable tracking algorithm is formed by combining Kalman filtering and IOU matching technology, and then the identification of weeds with various and multi-characteristic exposure degrees is realized by combining a color segmentation technology and morphological analysis treatment. Greatly improves the recognition rate of corn seedlings and weeds by an artificial intelligence algorithm, and lays a foundation for accurate agriculture.

Description

Corn field multiple weed real-time identification algorithm based on Kalman filtering and deep learning
Technical Field
The invention relates to the field of deep learning and accurate agriculture, in particular to a corn field multiple weed real-time identification algorithm based on Kalman filtering and deep learning.
Background
The harm of weeds to agricultural production is self-evident, and the precise target weed-spraying technology is the best solution at present. In the process of target weeding, the most basic and fundamental step is to accurately identify crops and weeds. The field environment is complex, the interference is much, and the weeder can vibrate the camera at intervals in the field action process, so that the finally shot picture is blurred, and the recognition loss phenomenon is caused. In the invention, the target detection network model and the Kalman filtering prediction model are combined, so that the recognition loss phenomenon is avoided as much as possible.
The weed species in the field are usually various, and if the collection of the data set and the establishment of the corresponding network model are carried out for each weed, the real-time performance of the identification task is greatly reduced. At the same time, the crop is usually above the weed and part of the weed is in a clustered state, which represents a shade between the crop and the weed, and the characteristic information of the weed is largely lost. The target detection network is unable to identify all weeds in such situations. In the invention, the target detection network, the color segmentation technology and the morphological analysis are combined, so that the efficient, accurate and various weed identification is realized.
Disclosure of Invention
The invention aims to provide a corn field multiple weed real-time identification algorithm based on Kalman filtering and deep learning, which can combine the early-stage movement trend of crops to accurately and continuously identify the crops when the identification loss phenomenon occurs in the target identification process, and can also realize high-precision and high-efficiency multiple weed identification.
The technical scheme adopted by the invention comprises the following steps:
(1) Network pre-training: and pre-training the target detection network to obtain a network model capable of accurately identifying corn seedlings.
(2) Corn seedling detection: and (3) detecting the position and the size of corn seedlings in the images input by the camera through the pre-training network in the step (1).
(3) Kalman filtering: establishing a field weeding machine motion model, setting a filtering initial value, if the current frame is a first frame, setting Kalman filtering initial estimation according to the position of the corn seedlings detected in the first frame, and predicting the position and the size of the corn seedlings corresponding to the next frame; if the current frame is not the first frame, the position and the size of the corn seedling corresponding to the next frame are directly predicted by the corn seedling position of the current frame.
(4) And (3) tracking and judging: and judging whether the current frame corresponding to the corn seedling identification is lost or not. If the corn seedlings are not lost, the detected corn seedling positions are directly regarded as the current frame corn seedling positions; if the corn seedlings are lost, the prediction result of the corresponding corn seedlings is regarded as the corn seedling positions.
(5) Identification of various weeds: firstly shielding corn seedling position image information in an image, obtaining the edge shape of a green area of the image except corn seedlings through an ultra-green model and morphological processing technology, solving the external rectangle of the edge shape, and regarding the edge shape as a weed position.
(6) Repeating (2) - (5) until the camera stops inputting the image.
Compared with the prior art, the invention has the advantages that: firstly, compared with the traditional target detection network model, in the process of identifying the target, the method can solve the identification loss phenomenon caused by the reasons of blurring, vibration, characteristic loss and the like, so that the identification result is more accurate, continuous and stable. Compared with the existing weed identification mode, the weed identification mode disclosed by the invention can realize the identification of various weeds, and is strong in instantaneity and high in accuracy.
Drawings
FIG. 1 is a flow chart of the algorithm
FIG. 2 Kalman filtering model building schematic diagram
FIG. 3 is a schematic flow chart of the stable tracking technique
FIGS. 4-9 multiple species, multiple feature exposure weed identification flow charts
Detailed Description
In order to make the technical problems, technical schemes and beneficial effects to be solved more clear, the invention is described in detail below with reference to the embodiments. It should be noted that the specific embodiments described herein are for the purpose of illustrating the invention only and are not to be construed as limiting the invention.
The implementation steps of the corn field multiple weed real-time identification algorithm based on Kalman filtering and deep learning are as follows:
(1) Network pre-training: and pre-training the target detection network to obtain a network model capable of accurately identifying corn seedlings.
(2) Corn seedling detection: and (3) detecting the position and the size of corn seedlings in the images input by the camera through the pre-training network in the step (1).
(3) Kalman filtering: establishing a field weeding machine motion model, setting a filtering initial value, if the current frame is a first frame, setting Kalman filtering initial estimation according to the position of the corn seedlings detected in the first frame, and predicting the position and the size of the corn seedlings corresponding to the next frame; if the current frame is not the first frame, the position and the size of the corn seedling corresponding to the next frame are directly predicted by the corn seedling position of the current frame.
(4) And (3) tracking and judging: and judging whether the current frame corresponding to the corn seedling identification is lost or not. If the corn seedlings are not lost, the detected corn seedling positions are directly regarded as the current frame corn seedling positions; if the corn seedlings are lost, the prediction result of the corresponding corn seedlings is regarded as the corn seedling positions.
(5) Identification of various weeds: firstly shielding corn seedling position image information in an image, obtaining the edge shape of a green area of the image except corn seedlings through an ultra-green model and morphological processing technology, solving the external rectangle of the edge shape, and regarding the edge shape as a weed position.
(6) Repeating (2) - (5) until the camera stops inputting the image.
The data set used in the pre-training in the step 1 is a three-leaf corn seedling data set with full angles, full weather and multiple light intensities, and the used target detection network model is Yolov5 with the best balance between the current efficiency and the precision.
Wherein, the kalman filtering in the step 2:
kalman filtering prediction model:
in the near-end identification and weeding process, when corn seedlings move relative to a weeding machine, the vertex angle position of the corn seedlings can be regarded as discretized data, and a discretized target position, a discretized size state model and an observation equation can be expressed as
X k ＝A k,k-1 X k-1 +BU k-1 +Γ k-1 W k-1
Z k ＝H k X k +V k
Wherein X is k The estimated value of the state of the target position and the size at the moment k comprises the central coordinates x and y of the position, the target size, the length h and the width w.
A k,k-1 -target motion state transition matrix from moment k-1 to moment k
B-control matrix
U k-1 -control input
Γ k-1 -a system noise influence matrix representing the degree to which each state quantity is influenced by each target state quantity noise from time k-1 to time k
W k-1 -systematic noise at time k-1
Z k -observation vector at time k
H k -observation matrix at time k
V k Observation noise at time k
According to Kalman filtering recurrence relation, relevant parameter prediction is carried out, namely
Figure BDA0004105033000000031
Figure BDA0004105033000000032
In the middle of
Figure BDA0004105033000000033
-current time target position and size prior estimation matrix
Figure BDA0004105033000000034
-current time target position and size prior estimation covariance matrix
Q-System Process covariance matrix
Kalman filtering parameters are updated to
Figure BDA0004105033000000035
Figure BDA0004105033000000036
K in the formula k -kalman gain
Figure BDA0004105033000000037
Figure BDA0004105033000000038
-optimal estimation matrix of target position and size
P k -updated covariance matrix
I-identity matrix
Building a crop motion model:
since the relative motion trail of crops and the weeder can be regarded as approximately uniform linear motion, the state transition matrix can be set as follows according to a kinematic formula:
Figure BDA0004105033000000041
when the camera is mounted on the robot for recognition tasks, the camera may vibrate, so that the photographed picture shakes, but in reality, the relative positions of the robot and crops are not changed. The covariance matrix of the process noise is used to solve this problem, and in general, the covariance matrix of the process noise is a diagonal matrix, and the smaller the value on the diagonal, the faster the convergence speed, so the process noise covariance matrix is set as:
Figure BDA0004105033000000042
the magnitude of the observed noise covariance matrix is related to the instrument and is not too large or too small, so a value which is moderate in magnitude and is close to the working process state of the weeding machine is taken as the value:
Figure BDA0004105033000000043
in the invention, all the directly observable values are only the central abscissa of the crop position, the central ordinate of the crop position and the height of the crop, so the state measurement matrix is set as follows:
Figure BDA0004105033000000044
wherein, a plurality of weed identification algorithms in step 4:
building an ultra-green model: in the invention, each frame of RGB color space image transmitted from a camera is converted into an HSV color space image, and the extraction and conversion of green pixels in a corn field into a binary image are realized by respectively setting the threshold values of H, S, V, wherein the three threshold values refer to the principle of HSV color space and the actual color distribution characteristics of the corn field, and are respectively set as follows: (45, 90), (0, 2555), (0, 255).
Morphological analysis treatment: in the invention, the binary image output by 4.1 contains a great amount of noise and loss because of complex actual field scene or error of the ultra-green color model, so the invention optimizes the binary image output according to the morphological characteristics of corn and weeds, wherein the binary image output is processed by an open operation and closed operation technology, wherein an open operator is a 10 x 10 identity matrix, and a closed operator is a 30 x 30 identity matrix.
The above examples are only one of the specific embodiments of the present invention, and the ordinary changes and substitutions made by those skilled in the art within the scope of the technical solution of the present invention should be included in the scope of the present invention.

Claims (4)

1. A corn field multiple weed real-time identification algorithm based on Kalman filtering and deep learning is characterized by comprising the following steps:
step 1: network pre-training: and pre-training the target detection network to obtain a network model capable of accurately identifying corn seedlings.
Step 2: corn seedling detection: and (3) detecting the position and the size of corn seedlings in the images input by the camera through the pre-training network in the step (1).
Step 3: kalman filtering: establishing a field weeding machine motion model, setting a filtering initial value, if the current frame is a first frame, setting Kalman filtering initial estimation according to the position of the corn seedlings detected in the first frame, and predicting the position and the size of the corn seedlings corresponding to the next frame; if the current frame is not the first frame, the position and the size of the corn seedling corresponding to the next frame are directly predicted by the corn seedling position of the current frame.
Step 4: and (3) tracking and judging: and judging whether the current frame corresponding to the corn seedling identification is lost or not. If the corn seedlings are not lost, the detected corn seedling positions are directly regarded as the current frame corn seedling positions; if the corn seedlings are lost, the prediction result of the corresponding corn seedlings is regarded as the corn seedling positions.
Step 5: identification of various weeds: firstly shielding corn seedling position image information in an image, obtaining the edge shape of a green area of the image except corn seedlings through an ultra-green model and morphological processing technology, solving the external rectangle of the edge shape, and regarding the edge shape as a weed position.
Step 6: repeating (2) - (5) until the camera stops inputting the image.
2. The Kalman filtering and deep learning based corn field weed real-time identification algorithm is characterized in that: the data set used in the pre-training in the step 1 is a three-leaf corn seedling data set with full angles, full weather and multiple light intensities, and the target detection network model used is Yolov5 with the best balance between the current efficiency and the precision.
3. The Kalman filtering and deep learning based corn field weed real-time identification algorithm is characterized in that: kalman filtering as described in step 2:
(1) Kalman filtering algorithm
In the near-end identification and weeding process, when corn seedlings move relative to a weeding machine, the vertex angle position of the corn seedlings can be regarded as discretized data, and a discretized target position, a discretized size state model and an observation equation can be expressed as
X k ＝A k,k-1 X k-1 +BU k-1 +Γ k-1 W k-1
Z k ＝H k X k +V k
Wherein X is k The estimated value of the state of the target position and the size at the moment k comprises the central coordinates x and y of the position, the target size, the length h and the width w.
A k,k-1 -target motion state transition matrix from moment k-1 to moment k
B-control matrix
U k-1 -control input
Γ k-1 -a system noise influence matrix representing the degree to which each state quantity is influenced by each target state quantity noise from time k-1 to time k
W k-1 -systematic noise at time k-1
Z k -observation vector at time k
H k -observation matrix at time k
V k Observation noise at time k
According to Kalman filtering recurrence relation, relevant parameter prediction is carried out, namely
Figure QLYQS_1
Figure QLYQS_2
In the middle of
Figure QLYQS_3
-current time target position and size prior estimation matrix
Figure QLYQS_4
-current time target position and size prior estimation covariance matrix
Q-System Process covariance matrix
Kalman filtering parameters are updated to
K k ＝P k - H T (HP k - H T +R) -1
Figure QLYQS_5
P k ＝(I-K k H)P k -
K in the formula k -kalman gain
Figure QLYQS_6
-optimal estimation matrix of target position and size
P k -updated covariance matrix
I-identity matrix
(2) Crop motion model building
Since the relative motion trail of crops and the weeder can be regarded as approximately uniform linear motion, the state transition matrix can be set as follows according to a kinematic formula:
Figure QLYQS_7
when the camera is mounted on the robot for recognition tasks, the camera may vibrate, so that the photographed picture shakes, but in reality, the relative positions of the robot and crops are not changed. The covariance matrix of the process noise is used to solve this problem, and in general, the covariance matrix of the process noise is a diagonal matrix, and the smaller the value on the diagonal, the faster the convergence speed, so the process noise covariance matrix is set as:
Figure QLYQS_8
the magnitude of the observed noise covariance matrix is related to the instrument and is not too large or too small, so in the invention, a value which is moderate in magnitude and is close to the working process state of the weeding machine is taken as follows:
Figure QLYQS_9
in the invention, all the directly observable values are only the central abscissa of the crop position, the central ordinate of the crop position and the height of the crop, so the state measurement matrix is set as follows:
Figure QLYQS_10
4. the Kalman filtering and deep learning based corn field weed real-time identification algorithm is characterized in that: various weed identification algorithms in step 4:
(1) Building an ultra-green model: in the invention, each frame of RGB color space image transmitted from a camera is converted into an HSV color space image, and the extraction and conversion of green pixels in a corn field into a binary image are realized by respectively setting the threshold values of H, S, V, wherein the three threshold values refer to the principle of HSV color space and the actual color distribution characteristics of the corn field, and are respectively set as follows: (45, 90), (0, 2555), (0, 255).
(2) Morphological analysis treatment: in the present invention, the binary image output by the method (1) in claim 4 contains a lot of noise and loss due to the complex actual field scene or the error of the ultra-green color model, so the present invention optimizes the binary image output according to the morphological characteristics of corn and weeds, wherein the binary image output is processed by using an open operation and a closed operation technology, wherein the open operation operator is a 10 x 10 identity matrix, and the closed operation operator is a 30 x 30 identity matrix.
CN202310189828.5A 2023-03-02 2023-03-02 Corn field multiple weed real-time identification algorithm based on Kalman filtering and deep learning Pending CN116188987A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202310189828.5A CN116188987A (en) 2023-03-02 2023-03-02 Corn field multiple weed real-time identification algorithm based on Kalman filtering and deep learning

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202310189828.5A CN116188987A (en) 2023-03-02 2023-03-02 Corn field multiple weed real-time identification algorithm based on Kalman filtering and deep learning

Publications (1)

Publication Number Publication Date
CN116188987A true CN116188987A (en) 2023-05-30

Family

ID=86450413

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202310189828.5A Pending CN116188987A (en) 2023-03-02 2023-03-02 Corn field multiple weed real-time identification algorithm based on Kalman filtering and deep learning

Country Status (1)

Country Link
CN (1) CN116188987A (en)

Similar Documents

Publication Publication Date Title
Xiong et al. Visual detection of green mangoes by an unmanned aerial vehicle in orchards based on a deep learning method
Bai et al. Vegetation segmentation robust to illumination variations based on clustering and morphology modelling
CN110689519B (en) Fog drop deposition image detection system and method based on yolo network
JP2019520631A (en) Weed recognition in the natural environment
CN109255302A (en) Object recognition methods and terminal, mobile device control method and terminal
CN109448023B (en) Satellite video small target real-time tracking method
Lin et al. Automatic detection of plant rows for a transplanter in paddy field using faster r-cnn
CN113344972B (en) Fish track prediction method based on intensive culture
de Silva et al. Towards agricultural autonomy: crop row detection under varying field conditions using deep learning
CN117115210B (en) Intelligent agricultural monitoring and adjusting method based on Internet of things
Ospina et al. Simultaneous mapping and crop row detection by fusing data from wide angle and telephoto images
Roggiolani et al. Hierarchical approach for joint semantic, plant instance, and leaf instance segmentation in the agricultural domain
Malekabadi et al. A comparative evaluation of combined feature detectors and descriptors in different color spaces for stereo image matching of tree
Wang et al. Research on image capture technology of intelligent terminal and multi exposure fusion to improve the resilience of agriculture production systems
Loresco et al. Segmentation of lettuce plants using super pixels and thresholding methods in smart farm hydroponics setup
Magistri et al. Towards in-field phenotyping exploiting differentiable rendering with self-consistency loss
CN115687850A (en) Method and device for calculating irrigation water demand of farmland
Buddha et al. Weed detection and classification in high altitude aerial images for robot-based precision agriculture
Fernando et al. Ai based greenhouse farming support system with robotic monitoring
Zhong et al. Identification and depth localization of clustered pod pepper based on improved Faster R-CNN
Shanmugam et al. A machine-vision-based real-time sensor system to control weeds in agricultural fields
CN115294562B (en) Intelligent sensing method for operation environment of plant protection robot
CN111695560A (en) Method for actively positioning and focusing crop diseases and insect pests based on convolutional neural network
CN117115769A (en) Plant detection and positioning method based on semantic segmentation network
CN116188987A (en) Corn field multiple weed real-time identification algorithm based on Kalman filtering and deep learning

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination