CN110989643B - Unmanned aerial vehicle plant protection system at night based on multi-view vision - Google Patents
Unmanned aerial vehicle plant protection system at night based on multi-view vision Download PDFInfo
- Publication number
- CN110989643B CN110989643B CN201911198570.5A CN201911198570A CN110989643B CN 110989643 B CN110989643 B CN 110989643B CN 201911198570 A CN201911198570 A CN 201911198570A CN 110989643 B CN110989643 B CN 110989643B
- Authority
- CN
- China
- Prior art keywords
- aerial vehicle
- platform
- unmanned aerial
- vision
- plant protection
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 239000000575 pesticide Substances 0.000 claims abstract description 22
- 238000005507 spraying Methods 0.000 claims abstract description 17
- 230000007246 mechanism Effects 0.000 claims abstract description 4
- 238000000034 method Methods 0.000 claims description 36
- 239000011159 matrix material Substances 0.000 claims description 28
- 230000008569 process Effects 0.000 claims description 23
- 238000013519 translation Methods 0.000 claims description 13
- 230000000007 visual effect Effects 0.000 claims description 13
- 238000004891 communication Methods 0.000 claims description 12
- 241000238631 Hexapoda Species 0.000 claims description 8
- 241000607479 Yersinia pestis Species 0.000 claims description 8
- 238000004422 calculation algorithm Methods 0.000 claims description 8
- 201000010099 disease Diseases 0.000 claims description 8
- 208000037265 diseases, disorders, signs and symptoms Diseases 0.000 claims description 8
- 238000004364 calculation method Methods 0.000 claims description 6
- 238000005516 engineering process Methods 0.000 claims description 6
- 230000004888 barrier function Effects 0.000 claims description 5
- 230000009466 transformation Effects 0.000 claims description 3
- RZVHIXYEVGDQDX-UHFFFAOYSA-N 9,10-anthraquinone Chemical compound C1=CC=C2C(=O)C3=CC=CC=C3C(=O)C2=C1 RZVHIXYEVGDQDX-UHFFFAOYSA-N 0.000 abstract description 2
- 238000007689 inspection Methods 0.000 description 4
- 238000002360 preparation method Methods 0.000 description 4
- 238000012271 agricultural production Methods 0.000 description 3
- 238000013461 design Methods 0.000 description 3
- 230000007547 defect Effects 0.000 description 2
- 238000010586 diagram Methods 0.000 description 2
- 238000012986 modification Methods 0.000 description 2
- 230000004048 modification Effects 0.000 description 2
- 239000007921 spray Substances 0.000 description 2
- 230000009286 beneficial effect Effects 0.000 description 1
- 230000010485 coping Effects 0.000 description 1
- 238000012937 correction Methods 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 230000008030 elimination Effects 0.000 description 1
- 238000003379 elimination reaction Methods 0.000 description 1
- XLYOFNOQVPJJNP-UHFFFAOYSA-N water Substances O XLYOFNOQVPJJNP-UHFFFAOYSA-N 0.000 description 1
Classifications
-
- G—PHYSICS
- G05—CONTROLLING; REGULATING
- G05D—SYSTEMS FOR CONTROLLING OR REGULATING NON-ELECTRIC VARIABLES
- G05D1/00—Control of position, course, altitude or attitude of land, water, air or space vehicles, e.g. using automatic pilots
- G05D1/08—Control of attitude, i.e. control of roll, pitch, or yaw
- G05D1/0808—Control of attitude, i.e. control of roll, pitch, or yaw specially adapted for aircraft
-
- G—PHYSICS
- G05—CONTROLLING; REGULATING
- G05D—SYSTEMS FOR CONTROLLING OR REGULATING NON-ELECTRIC VARIABLES
- G05D1/00—Control of position, course, altitude or attitude of land, water, air or space vehicles, e.g. using automatic pilots
- G05D1/10—Simultaneous control of position or course in three dimensions
- G05D1/101—Simultaneous control of position or course in three dimensions specially adapted for aircraft
Landscapes
- Engineering & Computer Science (AREA)
- Aviation & Aerospace Engineering (AREA)
- Radar, Positioning & Navigation (AREA)
- Remote Sensing (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Automation & Control Theory (AREA)
- Control Of Position, Course, Altitude, Or Attitude Of Moving Bodies (AREA)
Abstract
The invention relates to an unmanned aerial vehicle night plant protection operation system based on multi-view vision, and belongs to the technical field of aviation systems. The system comprises an unmanned aerial vehicle platform, a multi-vision platform, a pesticide spraying platform and an unmanned aerial vehicle ground control platform; the unmanned aerial vehicle platform is a flight platform for unmanned aerial vehicle plant protection operation; the multi-vision platform is used for capturing vision positioning identification points during night plant protection operation of the unmanned aerial vehicle and calculating the position of the unmanned aerial vehicle; the pesticide spraying platform is an executing mechanism for unmanned aerial vehicle plant protection operation; the unmanned aerial vehicle ground control platform is a ground control platform for plant protection unmanned aerial vehicle flight control, and is mainly used for receiving unmanned aerial vehicle position information fed back by the multi-vision platform and position and gesture information fed back by the unmanned aerial vehicle and planning a flight path of the unmanned aerial vehicle.
Description
Technical Field
The invention belongs to the technical field of aviation systems, and particularly relates to an unmanned aerial vehicle night plant protection system based on multi-view vision.
Background
The agricultural production and agricultural product consumption large country is provided with 20.25 hundred million mu of basic cultivated lands, a large amount of agricultural plant protection work is needed for each year on the cultivated lands, and particularly, a large amount of human hands are needed to gather in a short period to protect crop diseases and insect pests in the season of disease and insect pest outbreaks, but the traditional manual plant protection means have the defects of more human resources and low plant protection efficiency, so that the normal operation of the agricultural production in China is influenced, and the development of the agricultural production in China is restricted.
Along with the continuous progress of technology and the continuous promotion of the modern agricultural process, the modern agriculture with high efficiency, safety and precision is favored by all levels of governments in China. The unmanned aerial vehicle plant protection technology is an emerging technology in agricultural plant protection, can replace the traditional plant protection mode to a great extent to carry out plant protection operation, has more remarkable effects in aspects of convenience, safety, spraying efficiency, water and pesticide saving and the like, and becomes a main measure and important strength of agricultural modernization in China.
In the peak period of the outbreak of plant diseases and insect pests, the plant protection operation is used for rapidly, efficiently and timely coping with the damage of the plant diseases and insect pests, the timeliness of the plant protection operation is required to be higher, new requirements are provided for all-weather plant protection of the unmanned aerial vehicle, and how to improve the plant protection capability of the unmanned aerial vehicle under the night environment is particularly critical.
Disclosure of Invention
First, the technical problem to be solved
The invention provides an unmanned aerial vehicle night plant protection system based on multi-vision feedback, aiming at the problems of continuous operation, even night operation, of unmanned aerial vehicle plant protection during the peak period of diseases and insect pests related in the technical background, and the technical difficulties of low visibility, difficulty in obstacle avoidance and the like of unmanned aerial vehicle night operation.
(II) technical scheme
In order to solve the technical problems, the invention provides an unmanned aerial vehicle night plant protection operation system based on multi-vision, which comprises an unmanned aerial vehicle platform, a multi-vision platform, a pesticide spraying platform and a ground control platform;
the unmanned aerial vehicle platform is a flight platform for unmanned aerial vehicle plant protection operation and is used for carrying a pesticide spraying platform and a visual positioning identification point; when the unmanned aerial vehicle performs plant protection operation, the unmanned aerial vehicle platform carries a pesticide spraying platform to perform track real-time planning under the guidance of the multi-vision platform;
the multi-vision platform is used for capturing vision positioning identification points during night plant protection operation of the unmanned aerial vehicle, further performing unmanned aerial vehicle position calculation, sending unmanned aerial vehicle position information to the unmanned aerial vehicle platform and the ground control platform, and providing navigation position reference for the unmanned aerial vehicle;
the pesticide spraying platform is an execution mechanism of unmanned aerial vehicle plant protection operation, and is used for controlling the spraying intensity of pesticide in the unmanned aerial vehicle plant protection operation process, so that the pesticide acts on the roots of the damaged crops, and crop diseases and insect pests are better eliminated;
the ground control platform is used for receiving unmanned aerial vehicle position information fed back by the multi-vision platform and position and posture information fed back by the unmanned aerial vehicle, planning a flight path of the unmanned aerial vehicle, sending a control instruction to the unmanned aerial vehicle through wireless communication, and having unmanned aerial vehicle flight path planning, unmanned aerial vehicle posture display and unmanned aerial vehicle position display functions.
Further, the visual positioning identification point adopts the design of an infrared characteristic target point of the unmanned aerial vehicle, the infrared characteristic target point consists of an infrared characteristic identification point and an identification point support frame, the identification point support frame is in a cross shape, the infrared characteristic identification point is arranged at four end point positions of the identification point support frame, the infrared characteristic identification point is heated in the plant protection operation process of the unmanned aerial vehicle, and the identification degree of the infrared characteristic identification point in an infrared camera is improved.
Further, the unmanned aerial vehicle platform is still carried with an ultrasonic shielding device and a visual shielding device, and obstacles encountered in the process of plant protection operation are avoided in an ultrasonic and visual combined shielding mode in the process of unmanned aerial vehicle flight.
Further, the multi-vision platform consists of a plurality of groups of infrared binocular vision sub-platforms, is arranged around a farmland needing to perform plant protection operation, and is used for completing position calibration among the groups of binocular vision sub-platforms, and the unmanned aerial vehicle performs real-time calculation of the position of the unmanned aerial vehicle through the multi-vision platform at night in the farmland flight process.
Further, the position calibration algorithm between each group of binocular vision sub-platforms is as follows:
the multi-vision platform comprises n binocular vision platform subsystems which are respectively numbered 1,2,3 and …, n-1 and n; it is arranged around farmland, and the left camera in the binocular vision platform subsystem with the number of 1 is assumed to be used as the origin of the binocular vision platform coordinate system, and the camera parameter is M 1L ＝K 1L [I|0]Wherein K is 1L The parameters of the left camera of the binocular vision platform with the number of 1 are M, wherein the parameters are internal parameters of the left camera of the binocular vision platform with the number of 1, I is a 3 multiplied by 3 identity matrix, 0 is a 3 multiplied by 1 zero matrix 1R ＝K 1R [R 1 |t 1 ]，R 1 ,t 1 Respectively a rotation matrix and a translation matrix of the right camera relative to the left camera, K 1R Is the internal reference of the right camera in the binocular vision platform with the number of 1, and the binocular vision with the number of 2 is similarly carried outParameters of the left and right cameras of the platform are M respectively 2L ＝K 2L [I|0]，M 2R ＝K 2R [R n |t n ]The method comprises the steps of carrying out a first treatment on the surface of the And so on, the parameters of the left camera and the right camera of the binocular vision platform with the number n are M nL ＝K nL [I|0]，M nR ＝K nR [R n |t n ]；
Assuming that the numbers of two adjacent binocular vision platforms to be calibrated are 1 and 2, placing the infrared characteristic target points in a common field of view area of the two adjacent binocular vision platforms, moving the infrared characteristic target points by at least 3 positions in the calibration process, calculating the positions of the target points in respective binocular camera coordinate systems by using a formula 1,
in the formula 1, x, y and z are three-dimensional space coordinates of an infrared target point under a binocular vision platform coordinate system, and f l ,f r R is the focal length of the left and right cameras in the camera internal reference matrix 4 -r 9 Is a parameter in the camera matrix R, t y ,t z X is a parameter in the camera matrix t l 、Y l 、Y r The coordinate position of the target point in the camera image is obtained;
assume that the three-dimensional space position of the target point, calculated by using the formula 1, of the binocular vision platform with the number of 1 is P 1 ＝{X 1 ,Y 1 ,Z 1 ,1} T The binocular vision platform with the number of 2 uses the three-dimensional space position of the target point calculated by the formula 1 as P 2 ＝{X 2 ,Y 2 ,Z 2 1, due to a rotation-translation transformation between the binocular vision platform coordinate systems numbered 1 and 2, R can be rotated 21 Translation t 21 Matrix representation, thus
P 2 ＝{R 21 |t 21 }P 1 (2)
The three-dimensional space positions of a plurality of groups of target points form an equation set by utilizing the equation (2), and the rotation R between the binocular vision platform coordinate systems with the numbers of 1 and 2 can be calculated 21 Translation t 21 The parameters in the matrix finish the calibration of the positions between the binocular vision platforms with the numbers of 1 and 2;
and similarly, the calibration of the binocular vision platforms at other adjacent positions is repeated, so that the position calibration of the whole multi-vision platform is completed.
Further, the unmanned aerial vehicle position calculation algorithm is as follows:
after the position calibration of the multi-vision platform is finished, all binocular vision platforms in the multi-vision platform are unified to the same world coordinate system, so that in the process that the unmanned aerial vehicle platform performs plant protection operation in a farmland, the binocular vision platform captures infrared characteristic target points carried on the unmanned aerial vehicle platform in real time, and the three-dimensional space positions of the target points are calculated by using the following formula 3;
since the multi-vision platform comprises n groups of binocular vision platforms, three-dimensional space positions of n groups of targets are calculated, and since 4 identification points are arranged on the target frame, each calculated group of targets comprises 4 three-dimensional space positions of target points, and three-dimensional space coordinates of the four target points are averaged to be used as a position p of the unmanned aerial vehicle under the group of binocular vision platforms w ＝{x w ,y w ,z w Position of unmanned aerial vehicle under n groups of binocular vision platforms, and calculate their average value as position P of unmanned aerial vehicle in agricultural field calculated by multi-vision platform W ＝{X W ,Y W ,Z W }。
Further, the multi-vision platform transmits the position of the unmanned aerial vehicle to the unmanned aerial vehicle platform and the ground control platform through a wireless communication technology.
Further, the system also comprises a remote controller, the ground control platform is connected with the remote controller in a wired mode, and when an emergency occurs, the unmanned aerial vehicle can be controlled by the remote controller to land.
Further, the ground control platform and the unmanned plane platform are in two-way communication through wireless communication, the unmanned plane platform sends pose information to the ground control platform, the ground control platform plans a track according to the pose information of the unmanned plane and sends a control instruction, and the control instruction indicates track coordinates.
(III) beneficial effects
According to the unmanned aerial vehicle night plant protection method based on the multi-view visual feedback, the unmanned aerial vehicle position is fed back through the multi-view visual feedback method, position information is provided for unmanned aerial vehicle plant protection track planning under the night environment, the difficult problem of unmanned aerial vehicle night plant protection operation difficulty is overcome, and the unmanned aerial vehicle has all-weather plant protection operation capability.
The design of the unique multi-vision platform adopts a multi-group infrared binocular vision system formed by a plurality of groups of infrared binocular vision platforms, so that the position of the unmanned aerial vehicle can be resolved in real time, the unmanned aerial vehicle can be flexibly disassembled and randomly arranged around a farmland, and the defect that the single-group binocular vision system has vision blind areas and cannot capture the position of the unmanned aerial vehicle in all directions is avoided.
Unique unmanned aerial vehicle vision location identification point design, this identification point adds infrared characteristic capture point, discerns this vision location identification point under the multi-vision system night environment.
Drawings
FIG. 1 is a general system diagram of the present invention;
FIG. 2 is a diagram of the connection relationship of each sub-platform according to the present invention;
FIG. 3 is a flow chart of the operation of the present invention;
fig. 4 is a schematic view of the visual positioning mark point of the present invention.
In the figure:
the system comprises a 1-multi-vision platform, a 2-plant protection unmanned aerial vehicle platform, a 3-pesticide spraying platform, a 4-ground control platform, a 5-farmland to be plant protected, a 6-unmanned aerial vehicle, a 7-ground control operation platform, an 8-binocular vision sub-platform, 9-remote control equipment, 10-infrared characteristic identification points and 11-identification point support frames.
Detailed Description
For the purposes of clarity, content, and advantages of the present invention, a detailed description of the embodiments of the present invention will be described in detail below with reference to the drawings and examples.
As shown in FIG. 1, the unmanned aerial vehicle night plant protection method based on the multi-view vision is mainly applied to unmanned aerial vehicle position calculation and navigation in the unmanned aerial vehicle night plant protection operation process. The method mainly relates to the following objects: the plant protection system comprises a multi-vision platform 1, a plant protection unmanned aerial vehicle platform 2, a pesticide spraying platform 3, a ground control platform 4 and a farmland 5 to be plant protected.
The plant protection unmanned aerial vehicle platform 2 is a flight platform for unmanned aerial vehicle plant protection operation and is used for carrying a pesticide spraying platform 3, a visual positioning identification point, ultrasonic shielding equipment and a visual shielding device; when unmanned aerial vehicle carries out plant protection operation, plant protection unmanned aerial vehicle platform 2 carries on pesticide and sprays platform 3 and carries out the flight path real-time planning under the guide of visual platform 1 of many eyes, avoids the barrier that plant protection operation in-process encountered through supersound, vision combination barrier mode at the in-process of flight.
The multi-vision platform 1 is used for calculating the position of the unmanned aerial vehicle during the night plant protection operation of the unmanned aerial vehicle, and provides navigation position reference for the unmanned aerial vehicle. The multi-vision platform 1 is composed of a plurality of groups of infrared binocular vision sub-platforms 8, is arranged around a farmland 5 needing to perform plant protection operation, and is used for completing position calibration among the groups of binocular vision sub-platforms 8, and the unmanned aerial vehicle is used for performing real-time calculation on the unmanned aerial vehicle position through the multi-vision platform 1 at night in the farmland flight process, wherein the multi-vision platform 1 is used for sending the unmanned aerial vehicle position information to the plant protection unmanned aerial vehicle platform 2 and the ground control platform 4, and the ground control platform 4 is used for performing track planning by using the position information or forwarding the position information to the plant protection unmanned aerial vehicle platform 2 so as to provide a position basis for unmanned aerial vehicle navigation.
Pesticide spraying platform 3 is the actuating mechanism of unmanned aerial vehicle plant protection operation, and unmanned aerial vehicle plant protection in-process is used for the control pesticide spray intensity for pesticide acts on victim crops root, better elimination crop disease and insect pest.
The unmanned aerial vehicle ground control platform 4 is a ground control platform for plant protection unmanned aerial vehicle flight control, and is mainly used for receiving unmanned aerial vehicle position information fed back by the multi-vision platform and position and gesture information fed back by the unmanned aerial vehicle, planning a flight path of the unmanned aerial vehicle, sending a control instruction to the unmanned aerial vehicle through wireless communication, wherein the control instruction can be a flight path coordinate instruction, and has functions of unmanned aerial vehicle flight path planning, unmanned aerial vehicle gesture display, unmanned aerial vehicle position display and the like.
Unmanned aerial vehicle vision identification point carries on unmanned aerial vehicle platform 2, and the multi-vision system of being convenient for catches this identification point, and then solves the concrete position of unmanned aerial vehicle in the farmland.
During the process of executing the plant protection operation: firstly, arranging a plurality of binocular vision sub-platforms 8 around a farmland to be protected to form a multi-vision platform 1; parameter calibration is carried out on the binocular vision sub-platforms 8, and then the position relation among all groups of binocular vision sub-platforms is calibrated; after the calibration of the multi-vision platform 1 is finished, preparing for inspection before the operation of the plant protection unmanned aerial vehicle platform 2, and calculating the position of the plant protection unmanned aerial vehicle platform 2 relative to farmlands through a multi-vision system; after the inspection is finished, starting up the ground control platform 4 for inspection, and planning a flight path of the plant protection unmanned aerial vehicle by using the ground control platform 4 after the inspection is finished; and finally, starting plant protection operation, wherein the multi-vision platform 1 calculates the position of the plant protection unmanned aerial vehicle relative to a farmland in real time by capturing infrared characteristic identification points on the plant protection unmanned aerial vehicle platform 2 in the operation process, and feeds back position information to an autopilot and a ground control platform of the plant protection unmanned aerial vehicle for real-time correction of the unmanned aerial vehicle in the plant protection operation process.
As shown in fig. 2, the following relationship between the devices related to the unmanned aerial vehicle night plant protection method is: the multi-vision platform 1 formed by a plurality of groups of binocular vision sub-platforms 8 captures infrared identification points on a plant protection unmanned aerial vehicle platform in real time, the position of the unmanned aerial vehicle in a farmland is calculated, the position is transmitted to an unmanned aerial vehicle autopilot and a ground control operation platform 7 through a wireless communication technology, two-way communication is carried out between the ground control operation platform 7 and the unmanned aerial vehicle 6 through wireless communication, the unmanned aerial vehicle sends pose information to the ground control operation platform 7, the ground control operation platform 7 sends control instructions according to the pose information of the unmanned aerial vehicle, the ground control operation platform 7 is connected with a remote controller in a wired mode for preventing emergency in the operation process, and when the emergency happens, the unmanned aerial vehicle can be controlled by the remote controller to land.
As shown in FIG. 3, the night plant protection operation flow is as follows: firstly, arranging a multi-vision platform 1, then calibrating the multi-vision platform 1, carrying out unmanned aerial vehicle system preparation work after the calibration of the multi-vision platform 1 is finished, marking the position relationship between an unmanned aerial vehicle and a farmland in the process, carrying out ground control platform 4 preparation after the unmanned aerial vehicle system preparation is finished, and starting unmanned aerial vehicle night plant protection work after all preparation works are finished.
As shown in fig. 4, the visual feature identification point related to the invention is designed by adopting an infrared feature target point of the unmanned aerial vehicle, the infrared feature target point mainly comprises an infrared feature identification point 10 and an identification point support frame 11, the identification point support frame 11 is designed into a cross shape, the infrared feature identification point 10 is arranged at four end point positions of the identification point support frame 11, the infrared identification point is heated in the plant protection operation process of the unmanned aerial vehicle, and the recognition degree of the infrared identification point in an infrared camera is improved.
The position calibration algorithm among the binocular vision sub-platforms of each group is as follows:
the multi-vision platform comprises n binocular vision platform subsystems numbered 1,2,3 …, n-1, n respectively. It is arranged around farmland, and the left camera in the binocular vision platform subsystem with the number of 1 is assumed to be used as the origin of the binocular vision platform coordinate system, and the camera parameter is M 1L ＝K 1L [I|0]Wherein K is 1L The parameters of the left camera of the binocular vision platform with the number of 1 are M, wherein the parameters are internal parameters of the left camera of the binocular vision platform with the number of 1, I is a 3 multiplied by 3 identity matrix, 0 is a 3 multiplied by 1 zero matrix 1R ＝K 1R [R 1 |t 1 ]，R 1 ,t 1 Respectively a rotation matrix and a translation matrix of the right camera relative to the left camera, K 1R The parameters of the left and right cameras of the binocular vision platform with the number of 2 are respectively M 2L ＝K 2L [I|0]，M 2R ＝K 2R [R n |t n ]The method comprises the steps of carrying out a first treatment on the surface of the And so on, the parameters of the left camera and the right camera of the binocular vision platform with the number n are M nL ＝K nL [I|0]，M nR ＝K nR [R n |t n ]The invention mainly relates to an algorithm for calibrating a multi-vision platform under the condition that the camera parameters of the binocular vision platform are known (a rotation and translation matrix among all binocular vision platforms is calculated, and the camera parameters of a plurality of binocular vision platforms are unified into the same world coordinate system). The calibration algorithm is as follows:
taking the position relation between two adjacent binocular vision platforms with the calibration numbers of 1 and 2 as an example, the infrared characteristic target points related to the patent are placed in the public view field area of the two adjacent binocular vision platforms, the infrared characteristic target points are moved by at least 3 positions (assuming that the positions are moved by 3 positions) in the calibration process, and the positions of the target points in the respective binocular camera coordinate systems are calculated by using a formula 1.
In the formula 1, x, y and z are three-dimensional space coordinates of an infrared target point under a binocular vision platform coordinate system, and f l ,f r R is the focal length of the left and right cameras in the camera internal reference matrix 4 -r 9 Is a parameter in the camera matrix R, t y ,t z X is a parameter in the camera matrix t l 、Y l 、Y r Is the coordinate position of the target point in the camera image.
Assume that the three-dimensional space position of the target point, calculated by using the formula 1, of the binocular vision platform with the number of 1 is P 1 ＝{X 1 ,Y 1 ,Z 1 ,1} T The binocular vision platform with the number of 2 uses the three-dimensional space position of the target point calculated by the formula 1 as P 2 ＝{X 2 ,Y 2 ,Z 2 1, due to a rotation-translation transformation between the binocular vision platform coordinate systems numbered 1 and 2, R can be rotated 21 Translation t 21 Matrix representation, therefore;
P 2 ＝{R 21 |t 21 }P 1 (2)
the three-dimensional space positions of a plurality of groups of target points form an equation set by utilizing the equation (2), and the rotation R between the binocular vision platform coordinate systems with the numbers of 1 and 2 can be calculated 21 Translation t 21 And (5) completing the calibration of the positions between the binocular vision platforms with the numbers of 1 and 2 by using parameters in the matrix. And similarly, the calibration of the binocular vision platforms at other adjacent positions is repeated, so that the position calibration of the whole multi-vision platform is completed.
Unmanned aerial vehicle position calculation algorithm
After the calibration of the multi-vision platform is completed, all binocular vision platforms in the multi-vision platform are unified to the same world coordinate system, so that in the process that the unmanned aerial vehicle platform performs plant protection operation in farmlands, the binocular vision platform captures infrared characteristic target points carried on the unmanned aerial vehicle platform in real time, and the three-dimensional space positions of the target points are calculated by the following formula.
Since the multi-vision platform comprises n groups of binocular vision platforms, three-dimensional space positions of n groups of targets are calculated, and since 4 identification points are arranged on the target frame, each calculated group of targets comprises 4 three-dimensional space positions of target points, and three-dimensional space coordinates of the four target points are averaged to be used as a position p of the unmanned aerial vehicle under the group of binocular vision platforms w ＝{x w ,y w ,z w Then, the positions of the n groups of unmanned aerial vehicles are calculated, and the average value of the positions is calculated and used as the position P of the unmanned aerial vehicle in the farmland calculated by the multi-vision platform W ＝{X W ,Y W ,Z W }。
It will be understood by those skilled in the art that, unless otherwise defined, all terms (including technical and scientific terms) used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention belongs. It will be further understood that terms, such as those defined in commonly used dictionaries, should be interpreted as having a meaning that is consistent with their meaning in the context of the prior art and will not be interpreted in an idealized or overly formal sense unless expressly so defined herein.
The foregoing is merely a preferred embodiment of the present invention, and it should be noted that modifications and variations could be made by those skilled in the art without departing from the technical principles of the present invention, and such modifications and variations should also be regarded as being within the scope of the invention.
Claims (6)
1. Unmanned aerial vehicle night plant protection operating system based on multi-view vision, its characterized in that: the system comprises an unmanned plane platform, a multi-vision platform, a pesticide spraying platform and a ground control platform;
the unmanned aerial vehicle platform is a flight platform for unmanned aerial vehicle plant protection operation and is used for carrying a pesticide spraying platform and a visual positioning identification point; when the unmanned aerial vehicle performs plant protection operation, the unmanned aerial vehicle platform carries a pesticide spraying platform to perform track real-time planning under the guidance of the multi-vision platform;
the multi-vision platform is used for capturing vision positioning identification points during night plant protection operation of the unmanned aerial vehicle, further performing unmanned aerial vehicle position calculation, sending unmanned aerial vehicle position information to the unmanned aerial vehicle platform and the ground control platform, and providing navigation position reference for the unmanned aerial vehicle;
the pesticide spraying platform is an execution mechanism of unmanned aerial vehicle plant protection operation, and is used for controlling the spraying intensity of pesticide in the unmanned aerial vehicle plant protection operation process, so that the pesticide acts on the roots of the damaged crops, and crop diseases and insect pests are better eliminated;
the ground control platform is used for receiving unmanned aerial vehicle position information fed back by the multi-vision platform and position and posture information fed back by the unmanned aerial vehicle, planning a flight path of the unmanned aerial vehicle, sending a control instruction to the unmanned aerial vehicle through wireless communication, and having unmanned aerial vehicle flight path planning, unmanned aerial vehicle posture display and unmanned aerial vehicle position display functions;
wherein,
the visual positioning identification points are designed by adopting infrared characteristic target points of the unmanned aerial vehicle, the infrared characteristic target points consist of infrared characteristic identification points and identification point supporting frames, the identification point supporting frames are in a cross shape, the infrared characteristic identification points are arranged at four end point positions of the identification point supporting frames, the infrared characteristic identification points are heated in the plant protection operation process of the unmanned aerial vehicle, and the identification degree of the infrared characteristic identification points in an infrared camera is improved;
the position calibration algorithm among the binocular vision sub-platforms of each group is as follows:
the multi-vision platform comprises n binocular vision platform subsystems which are respectively numbered 1,2,3 and …, n-1 and n; it is arranged around farmland, and the left camera in the binocular vision platform subsystem with the number of 1 is assumed to be used as the origin of the binocular vision platform coordinate system, and the camera parameter is M 1L ＝K 1L [I|0]Wherein K is 1L The parameters of the left camera of the binocular vision platform with the number of 1 are M, wherein the parameters are internal parameters of the left camera of the binocular vision platform with the number of 1, I is a 3 multiplied by 3 identity matrix, 0 is a 3 multiplied by 1 zero matrix 1R ＝K 1R [R 1 |t 1 ]，R 1 ,t 1 Respectively a rotation matrix and a translation matrix of the right camera relative to the left camera, K 1R The parameters of the left and right cameras of the binocular vision platform with the number of 2 are respectively M 2L ＝K 2L [I|0]，M 2R ＝K 2R [R n |t n ]The method comprises the steps of carrying out a first treatment on the surface of the And so on, the parameters of the left camera and the right camera of the binocular vision platform with the number n are M nL ＝K nL [I|0]，M nR ＝K nR [R n |t n ]；
Assuming that the numbers of two adjacent binocular vision platforms to be calibrated are 1 and 2, placing the infrared characteristic target points in a common field of view area of the two adjacent binocular vision platforms, moving the infrared characteristic target points by at least 3 positions in the calibration process, calculating the positions of the target points in respective binocular camera coordinate systems by using a formula 1,
in the formula 1, x, y and z are three-dimensional space coordinates of an infrared target point under a binocular vision platform coordinate system, and f l ,f r R is the focal length of the left and right cameras in the camera internal reference matrix 4 -r 9 Is a parameter in the camera matrix R, t y ,t z X is a parameter in the camera matrix t l 、Y l 、Y r The coordinate position of the target point in the camera image is obtained;
assume that the three-dimensional space position of the target point, calculated by using the formula 1, of the binocular vision platform with the number of 1 is P 1 ＝{X 1 ,Y 1 ,Z 1 ,1} T The binocular vision platform with the number of 2 uses the three-dimensional space position of the target point calculated by the formula 1 as P 2 ＝{X 2 ,Y 2 ,Z 2 1, due to a rotation-translation transformation between the binocular vision platform coordinate systems numbered 1 and 2, R can be rotated 21 Translation t 21 Matrix representation, thus
P 2 ＝{R 21 |t 21 }P 1 (2)
The three-dimensional space positions of a plurality of groups of target points form an equation set by utilizing the equation (2), and the rotation R between the binocular vision platform coordinate systems with the numbers of 1 and 2 can be calculated 21 Translation t 21 The parameters in the matrix finish the calibration of the positions between the binocular vision platforms with the numbers of 1 and 2;
similarly, the calibration of binocular vision platforms at other adjacent positions is repeated, so that the position calibration of the whole multi-vision platform is completed;
the unmanned aerial vehicle position calculation algorithm is as follows:
after the position calibration of the multi-vision platform is finished, all binocular vision platforms in the multi-vision platform are unified to the same world coordinate system, so that in the process that the unmanned aerial vehicle platform performs plant protection operation in a farmland, the binocular vision platform captures infrared characteristic target points carried on the unmanned aerial vehicle platform in real time, and the three-dimensional space positions of the target points are calculated by using the following formula 3;
since the multi-vision platform comprises n groups of binocular vision platforms, three-dimensional space positions of n groups of targets are calculated, and since 4 identification points are arranged on the target frame, each calculated group of targets comprises 4 three-dimensional space positions of target points, and three-dimensional space coordinates of the four target points are averaged to be used as a position p of the unmanned aerial vehicle under the group of binocular vision platforms w ＝{x w ,y w ,z w Position of unmanned aerial vehicle under n groups of binocular vision platforms, and calculate their average value as position P of unmanned aerial vehicle in agricultural field calculated by multi-vision platform W ＝{X W ,Y W ,Z W }。
2. The unmanned aerial vehicle night plant protection operating system based on multi-vision according to claim 1, wherein: the unmanned aerial vehicle platform still carries on supersound and covers barrier equipment and vision and cover barrier device, and the in-process that flies at unmanned aerial vehicle is through supersound, vision combination shielding mode avoids the barrier that plant protection operation in-process met.
3. The unmanned aerial vehicle night plant protection operating system based on multi-vision according to claim 1, wherein: the multi-vision platform consists of a plurality of groups of infrared binocular vision sub-platforms, is arranged around a farmland needing to perform plant protection operation, and is used for completing position calibration among the groups of binocular vision sub-platforms, and the unmanned aerial vehicle performs real-time calculation of the position of the unmanned aerial vehicle through the multi-vision platform at night in the farmland flight process.
4. The unmanned aerial vehicle night plant protection operating system based on multi-vision according to claim 1, wherein: the multi-vision platform transmits the position of the unmanned aerial vehicle to the unmanned aerial vehicle platform and the ground control platform through a wireless communication technology.
5. The unmanned aerial vehicle night plant protection operating system based on multi-vision according to claim 1, wherein: the system also comprises a remote controller, the ground control platform is connected with the remote controller in a wired way, and when an emergency occurs, the unmanned aerial vehicle can be controlled by the remote controller to land.
6. The unmanned aerial vehicle night plant protection operating system based on multi-vision according to claim 1, wherein: the ground control platform and the unmanned plane platform are in two-way communication through wireless communication, the unmanned plane platform sends pose information to the ground control platform, the ground control platform plans a track according to the pose information of the unmanned plane and sends a control instruction, and the control instruction indicates track coordinates.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201911198570.5A CN110989643B (en) | 2019-11-29 | 2019-11-29 | Unmanned aerial vehicle plant protection system at night based on multi-view vision |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201911198570.5A CN110989643B (en) | 2019-11-29 | 2019-11-29 | Unmanned aerial vehicle plant protection system at night based on multi-view vision |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN110989643A CN110989643A (en) | 2020-04-10 |
| CN110989643B true CN110989643B (en) | 2023-12-19 |
Family
ID=70088286
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201911198570.5A Active CN110989643B (en) | 2019-11-29 | 2019-11-29 | Unmanned aerial vehicle plant protection system at night based on multi-view vision |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN110989643B (en) |
Families Citing this family (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110989664A (en) * | 2019-11-29 | 2020-04-10 | 北京特种机械研究所 | Unmanned aerial vehicle night plant protection method based on multi-view vision |
| CN111990378A (en) * | 2020-08-25 | 2020-11-27 | 淮阴工学院 | Spraying control method for spraying robot |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104457761A (en) * | 2014-11-18 | 2015-03-25 | 上海新跃仪表厂 | Characteristic relay method for relative position and attitude based on multi-vision |
| CN106096207A (en) * | 2016-06-29 | 2016-11-09 | 武汉中观自动化科技有限公司 | A kind of rotor wing unmanned aerial vehicle wind resistance appraisal procedure based on multi-vision visual and system |
| CN106295563A (en) * | 2016-08-09 | 2017-01-04 | 武汉中观自动化科技有限公司 | A kind of system and method airbound target flying quality assessed based on multi-vision visual |
| CN108519775A (en) * | 2017-10-30 | 2018-09-11 | 北京博鹰通航科技有限公司 | A kind of UAV system and its control method precisely sprayed |
| CN110189382A (en) * | 2019-05-31 | 2019-08-30 | 东北大学 | A kind of more binocular cameras movement scaling method based on no zone of mutual visibility domain |
Family Cites Families (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107747941B (en) * | 2017-09-29 | 2020-05-15 | 歌尔股份有限公司 | Binocular vision positioning method, device and system |
-
2019
- 2019-11-29 CN CN201911198570.5A patent/CN110989643B/en active Active
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104457761A (en) * | 2014-11-18 | 2015-03-25 | 上海新跃仪表厂 | Characteristic relay method for relative position and attitude based on multi-vision |
| CN106096207A (en) * | 2016-06-29 | 2016-11-09 | 武汉中观自动化科技有限公司 | A kind of rotor wing unmanned aerial vehicle wind resistance appraisal procedure based on multi-vision visual and system |
| CN106295563A (en) * | 2016-08-09 | 2017-01-04 | 武汉中观自动化科技有限公司 | A kind of system and method airbound target flying quality assessed based on multi-vision visual |
| CN108519775A (en) * | 2017-10-30 | 2018-09-11 | 北京博鹰通航科技有限公司 | A kind of UAV system and its control method precisely sprayed |
| CN110189382A (en) * | 2019-05-31 | 2019-08-30 | 东北大学 | A kind of more binocular cameras movement scaling method based on no zone of mutual visibility domain |
Also Published As
| Publication number | Publication date |
|---|---|
| CN110989643A (en) | 2020-04-10 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US12004443B2 (en) | Payload selection to treat multiple plant objects having different attributes | |
| CN110692352B (en) | Intelligent agricultural robot and control method thereof | |
| US11526997B2 (en) | Targeting agricultural objects to apply units of treatment autonomously | |
| CN111543413B (en) | Method for accurately applying pesticide by agricultural robot in cooperation with air and ground | |
| CN110989643B (en) | Unmanned aerial vehicle plant protection system at night based on multi-view vision | |
| US11812681B2 (en) | Precision treatment of agricultural objects on a moving platform | |
| CN108362285A (en) | A kind of plant protection drone precisely covers path planning method and system | |
| US11465162B2 (en) | Obscurant emission to assist image formation to automate agricultural management and treatment | |
| US11449976B2 (en) | Pixel projectile delivery system to replicate an image on a surface using pixel projectiles | |
| US20210186006A1 (en) | Autonomous agricultural treatment delivery | |
| US20210185942A1 (en) | Managing stages of growth of a crop with micro-precision via an agricultural treatment delivery system | |
| US11653590B2 (en) | Calibration of systems to deliver agricultural projectiles | |
| EP3878741A1 (en) | Unmanned aerial vehicle | |
| CN110989664A (en) | Unmanned aerial vehicle night plant protection method based on multi-view vision | |
| CN113273376A (en) | Modularization weeding device | |
| CN203762122U (en) | Indoor autonomous mobile pesticide spraying robot | |
| WO2022072219A1 (en) | Calibration of autonomous farming vehicle image acquisition system | |
| WO2019237412A1 (en) | Monitoring system and method for automatically operated unmanned aerial vehicle for plant protection | |
| CN116806799B (en) | Intelligent agricultural field weeding method and system | |
| Mashori et al. | Development of Pesticide Sprayer Robot Prototype for Chilli Farm Agricultural Application | |
| WO2024052757A1 (en) | Robotic sprayer | |
| CN116829460A (en) | Expert flying robot system for field autonomous operation |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |