CN110989664A - Unmanned aerial vehicle night plant protection method based on multi-view vision - Google Patents
Unmanned aerial vehicle night plant protection method based on multi-view vision Download PDFInfo
- Publication number
- CN110989664A CN110989664A CN201911199206.0A CN201911199206A CN110989664A CN 110989664 A CN110989664 A CN 110989664A CN 201911199206 A CN201911199206 A CN 201911199206A CN 110989664 A CN110989664 A CN 110989664A
- Authority
- CN
- China
- Prior art keywords
- unmanned aerial
- aerial vehicle
- platform
- plant protection
- vision
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 57
- 239000000575 pesticide Substances 0.000 claims abstract description 26
- 230000008569 process Effects 0.000 claims abstract description 24
- 230000007246 mechanism Effects 0.000 claims abstract description 6
- 239000007921 spray Substances 0.000 claims abstract description 6
- 238000012937 correction Methods 0.000 claims abstract description 4
- 230000000007 visual effect Effects 0.000 claims description 42
- 239000011159 matrix material Substances 0.000 claims description 24
- 238000005507 spraying Methods 0.000 claims description 16
- 238000004891 communication Methods 0.000 claims description 13
- 238000007689 inspection Methods 0.000 claims description 11
- 238000013519 translation Methods 0.000 claims description 10
- 241000607479 Yersinia pestis Species 0.000 claims description 8
- 238000004422 calculation algorithm Methods 0.000 claims description 8
- 201000010099 disease Diseases 0.000 claims description 8
- 208000037265 diseases, disorders, signs and symptoms Diseases 0.000 claims description 8
- 241000238631 Hexapoda Species 0.000 claims description 6
- 230000004888 barrier function Effects 0.000 claims description 6
- 238000013461 design Methods 0.000 claims description 6
- 238000002360 preparation method Methods 0.000 claims description 5
- 238000005516 engineering process Methods 0.000 claims description 4
- 238000004364 calculation method Methods 0.000 claims description 3
- 230000009466 transformation Effects 0.000 claims description 3
- 238000012369 In process control Methods 0.000 claims description 2
- 238000010965 in-process control Methods 0.000 claims description 2
- 238000012271 agricultural production Methods 0.000 description 2
- 238000010586 diagram Methods 0.000 description 2
- 230000000694 effects Effects 0.000 description 2
- 238000012986 modification Methods 0.000 description 2
- 230000004048 modification Effects 0.000 description 2
- RZVHIXYEVGDQDX-UHFFFAOYSA-N 9,10-anthraquinone Chemical compound C1=CC=C2C(=O)C3=CC=CC=C3C(=O)C2=C1 RZVHIXYEVGDQDX-UHFFFAOYSA-N 0.000 description 1
- 230000010485 coping Effects 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 239000003814 drug Substances 0.000 description 1
- 230000008030 elimination Effects 0.000 description 1
- 238000003379 elimination reaction Methods 0.000 description 1
- XLYOFNOQVPJJNP-UHFFFAOYSA-N water Substances O XLYOFNOQVPJJNP-UHFFFAOYSA-N 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G05—CONTROLLING; REGULATING
- G05D—SYSTEMS FOR CONTROLLING OR REGULATING NON-ELECTRIC VARIABLES
- G05D1/00—Control of position, course, altitude or attitude of land, water, air or space vehicles, e.g. using automatic pilots
- G05D1/10—Simultaneous control of position or course in three dimensions
- G05D1/101—Simultaneous control of position or course in three dimensions specially adapted for aircraft
-
- A—HUMAN NECESSITIES
- A01—AGRICULTURE; FORESTRY; ANIMAL HUSBANDRY; HUNTING; TRAPPING; FISHING
- A01G—HORTICULTURE; CULTIVATION OF VEGETABLES, FLOWERS, RICE, FRUIT, VINES, HOPS OR SEAWEED; FORESTRY; WATERING
- A01G13/00—Protecting plants
-
- A—HUMAN NECESSITIES
- A01—AGRICULTURE; FORESTRY; ANIMAL HUSBANDRY; HUNTING; TRAPPING; FISHING
- A01M—CATCHING, TRAPPING OR SCARING OF ANIMALS; APPARATUS FOR THE DESTRUCTION OF NOXIOUS ANIMALS OR NOXIOUS PLANTS
- A01M7/00—Special adaptations or arrangements of liquid-spraying apparatus for purposes covered by this subclass
- A01M7/0025—Mechanical sprayers
- A01M7/0032—Pressure sprayers
- A01M7/0042—Field sprayers, e.g. self-propelled, drawn or tractor-mounted
Landscapes
- Engineering & Computer Science (AREA)
- Life Sciences & Earth Sciences (AREA)
- Environmental Sciences (AREA)
- Aviation & Aerospace Engineering (AREA)
- Remote Sensing (AREA)
- Wood Science & Technology (AREA)
- Zoology (AREA)
- Insects & Arthropods (AREA)
- Mechanical Engineering (AREA)
- Radar, Positioning & Navigation (AREA)
- Pest Control & Pesticides (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Automation & Control Theory (AREA)
- Health & Medical Sciences (AREA)
- General Health & Medical Sciences (AREA)
- Toxicology (AREA)
- Catching Or Destruction (AREA)
- Control Of Position, Course, Altitude, Or Attitude Of Moving Bodies (AREA)
Abstract
The invention relates to a night plant protection operation method of an unmanned aerial vehicle based on multi-view vision, and belongs to the technical field of aviation systems. Arranging a plurality of binocular vision sub-platforms around a farmland to be protected to form a multi-view vision platform; calibrating parameters of the binocular vision sub-platforms, and then calibrating the position relation among each group of binocular vision sub-platforms; calculating the initial position of the plant protection unmanned aerial vehicle platform relative to the farmland through a multi-view vision system; planning the flight path of the plant protection unmanned aerial vehicle by using a ground control platform; starting plant protection operation, wherein the multi-view vision platform calculates the position of the plant protection unmanned aerial vehicle relative to a farmland in real time by capturing infrared characteristic identification points on a plant protection unmanned aerial vehicle platform in the operation process, and feeds back position information to an automatic pilot and a ground control platform of the plant protection unmanned aerial vehicle for real-time correction of the air route of the plant protection unmanned aerial vehicle in the operation process; pesticide sprays the platform and carries on the unmanned aerial vehicle platform, for the actuating mechanism of unmanned aerial vehicle plant protection operation.
Description
Technical Field
The invention belongs to the technical field of aviation systems, and particularly relates to a night plant protection method for an unmanned aerial vehicle based on multi-view vision.
Background
China is a large country for agricultural production and agricultural product consumption, China has 20.25 hundred million acres of basic farmlands, and the farmlands need a large amount of agricultural plant protection operation every year, particularly in the season of disease and pest outbreak, a large amount of hands need to be gathered in a short period to carry out crop disease and pest protection work, but the traditional artificial plant protection means needs more manpower resources and low plant protection efficiency, normal operation of agricultural production in China is influenced, and the development of agriculture in China is restricted.
With the continuous progress of science and technology and the continuous promotion of the agricultural modernization process, the high-efficiency, safe and accurate modern agriculture is favored by all levels of governments in China. Unmanned aerial vehicle plant protection technique is as a new technique in agricultural plant protection, and unmanned aerial vehicle can replace traditional plant protection mode to carry out the plant protection operation to a great extent to unmanned aerial vehicle plant protection all has more showing effect in the aspect of convenience, security, spraying efficiency, water conservation festival medicine etc. will become the main measure and the important strength of china's agricultural modernization.
In the peak period of outbreak of plant diseases and insect pests, plant protection operation puts forward higher requirements on the timeliness of plant protection operation for quick, efficient and timely coping with plant diseases and insect pests, puts forward new demands on all-weather plant protection of the unmanned aerial vehicle, and how to improve the plant protection capability of the unmanned aerial vehicle under the night environment is particularly critical.
Disclosure of Invention
Technical problem to be solved
The invention provides an unmanned aerial vehicle night plant protection method based on multi-view visual feedback, aiming at the technical difficulties that the unmanned aerial vehicle needs continuous operation and even night operation in the peak period of plant diseases and insect pests related to the technical background, the unmanned aerial vehicle plant protection operation is low in visibility, difficult in obstacle avoidance and the like.
(II) technical scheme
In order to solve the technical problems, the invention provides an unmanned aerial vehicle night plant protection operation method based on multi-view vision, which is realized based on an unmanned aerial vehicle night plant protection operation system, the system comprises four task platforms, namely an unmanned aerial vehicle platform, a multi-view vision platform, a pesticide spraying platform and an unmanned aerial vehicle ground control platform, and the unmanned aerial vehicle platform is a flight platform for unmanned aerial vehicle plant protection operation and is used for carrying the pesticide spraying platform and a visual positioning identification point; when the unmanned aerial vehicle carries out plant protection operation, the unmanned aerial vehicle platform carries a pesticide spraying platform to carry out real-time flight path planning under the guidance of the multi-view visual platform; the multi-view vision platform is used for capturing vision positioning identification points during night plant protection operation of the unmanned aerial vehicle, further performing position calculation of the unmanned aerial vehicle, sending position information of the unmanned aerial vehicle to the unmanned aerial vehicle platform and the ground control platform, and providing navigation position reference for the unmanned aerial vehicle; the pesticide spraying platform is an executing mechanism for unmanned aerial vehicle plant protection operation, and is used for controlling the spraying strength of pesticides in the unmanned aerial vehicle plant protection operation process, so that the pesticides act on the roots of damaged crops, and the crop diseases and insect pests are better eliminated; the ground control platform is used for receiving unmanned aerial vehicle position information fed back by the multi-view visual platform and unmanned aerial vehicle position and attitude information fed back by the unmanned aerial vehicle, planning the flight path of the unmanned aerial vehicle, and sending a control command to the unmanned aerial vehicle through wireless communication, and has the functions of unmanned aerial vehicle flight path planning, unmanned aerial vehicle attitude display and unmanned aerial vehicle position display; the unmanned aerial vehicle night plant protection operation method based on the system comprises the following steps:
arranging a plurality of binocular vision sub-platforms around a farmland to be protected to form a multi-view vision platform; calibrating parameters of the binocular vision sub-platforms, and then calibrating the position relation among each group of binocular vision sub-platforms;
after the calibration of the multi-view vision platform is completed, carrying out preparation inspection before the operation of the plant protection unmanned aerial vehicle platform, and calculating the position of the plant protection unmanned aerial vehicle platform relative to the farmland through a multi-view vision system;
after the inspection is finished, starting up the ground control platform for inspection, and planning the flight path of the plant protection unmanned aerial vehicle by using the ground control platform after the inspection is finished;
finally, plant protection operation is started, the multi-view vision platform calculates the position of the plant protection unmanned aerial vehicle relative to a farmland in real time by capturing infrared characteristic identification points on the plant protection unmanned aerial vehicle platform in the operation process, and feeds back position information to an automatic pilot and a ground control platform of the plant protection unmanned aerial vehicle for real-time correction of the unmanned aerial vehicle route in the plant protection operation process;
the pesticide sprays the platform and carries on the unmanned aerial vehicle platform, for the actuating mechanism of unmanned aerial vehicle plant protection operation, the intensity of spraying of unmanned aerial vehicle plant protection operation in-process control pesticide for the pesticide acts on the victim crops root.
Further, the ground control platform and the unmanned aerial vehicle platform are in two-way communication through wireless communication, the unmanned aerial vehicle platform sends pose information to the ground control platform, the ground control platform plans a flight path according to the pose information of the unmanned aerial vehicle and sends a control command, and the control command indicates a flight path coordinate.
Further, ground control platform still carries out wired connection with the remote controller, when emergency takes place, can descend through the artificial control unmanned aerial vehicle of remote controller.
Further, the multi-view vision platform transmits the position of the unmanned aerial vehicle to the unmanned aerial vehicle platform and the ground control platform through a wireless communication technology.
Further, the unmanned aerial vehicle platform still carries on the visual positioning identification point, this visual positioning identification point adopts the design of the infrared characteristic target point of unmanned aerial vehicle, this infrared characteristic target point comprises infrared characteristic identification point and identification point support frame, the identification point support frame designs into the cross shape, infrared characteristic identification point arranges in four extreme point positions of identification point support frame, unmanned aerial vehicle is at the plant protection operation in-process, infrared characteristic identification point is heated, improve the recognition degree of infrared characteristic identification point in infrared camera.
Furthermore, the unmanned aerial vehicle platform still carries on supersound shielding equipment and vision shielding device, avoids the barrier that the plant protection operation in-process meets through supersound, vision combination shielding mode at the in-process that unmanned aerial vehicle flies.
Further, the position calibration algorithm among each group of binocular vision sub-platforms is as follows:
the multi-view vision platform comprises n binocular vision platform subsystems which are respectively numbered as 1,2,3 …, n-1 and n; arranging the binocular vision platform subsystem around a farmland, and assuming that a left camera in the binocular vision platform subsystem with the number of 1 is used as the origin of a coordinate system of the binocular vision platform, wherein the camera parameter is M1L＝K1L[I|0]In which K is1LThe reference of the left camera in the binocular vision platform with the number of 1, I is a 3 multiplied by 3 identity matrix, 0 is a 3 multiplied by 1 zero matrix, and then the parameter of the right camera of the binocular vision platform with the number of 1 is M1R＝K1R[R1|t1]，R1,t1Respectively rotation and translation matrix of the right camera relative to the left camera, K1RThe parameters of the right camera in the binocular vision platform with the number of 1 are M respectively2L＝K2L[I|0]，M2R＝K2R[Rn|tn](ii) a By analogy, the parameter of the left camera and the right camera of the binocular vision platform with the number of n is MnL＝KnL[I|0]，MnR＝KnR[Rn|tn]；
Assuming that the numbers of two adjacent binocular vision platforms to be calibrated are 1 and 2, placing infrared characteristic target points in a common visual field area of the two adjacent binocular vision platforms, moving the infrared characteristic target points at least 3 positions in the calibration process, calculating the positions of the target points in respective binocular camera coordinate systems by using a formula 1,
in formula 1, x, y and z are three-dimensional space coordinates of the infrared target point in a binocular vision platform coordinate system, and fl,frIs the focal length of the left and right cameras in the camera internal reference matrix, R is the parameter in the camera matrix R, ty,tzAs a parameter in the camera matrix t, Xl、Yl、YrThe coordinate position of the target point in the camera image is taken;
suppose that the binocular vision platform numbered 1 is solved by formula 1The three-dimensional space position of the output target point is P1＝{X1,Y1,Z1,1}TAnd the three-dimensional space position of the target point calculated by the binocular vision platform with the number of 2 by using the formula 1 is P2＝{X2,Y2,Z21, since there is a rotational-translational transformation between the coordinate systems of the binocular vision platforms numbered 1 and 2, it is possible to rotate R21And translation t21Is represented by a matrix, thus
P2＝{R21|t21}P1(2)
The three-dimensional space positions of a plurality of groups of target points form an equation group by using a formula (2), and the rotation R between the coordinate systems of the binocular vision platforms numbered 1 and 2 can be solved21And translation t21The parameters in the matrix finish the calibration of the position between the binocular vision platforms numbered 1 and 2;
similarly, the calibration of the binocular vision platform at other adjacent positions repeats the above process, and then the position calibration of the whole binocular vision platform is completed.
Further, the unmanned plane position calculation algorithm is as follows:
after the position calibration of the multi-view vision platform is completed, all binocular vision platforms in the multi-view vision platform are unified to the same world coordinate system, so that the binocular vision platforms capture infrared characteristic target points carried on the unmanned aerial vehicle platform in real time in the process of plant protection operation of the unmanned aerial vehicle platform in a farmland, and the three-dimensional space position of the target points is calculated by using the following formula 3;
the multi-view visual platform comprises n groups of binocular visual platforms, so that three-dimensional space positions of n groups of targets can be calculated, 4 identification points are arranged on a target frame, each group of calculated targets comprises 4 target point three-dimensional space positions, and the three-dimensional space coordinates of the four target points are averaged to serve as the position p of the unmanned aerial vehicle under the group of binocular visual platformsw＝{xw,yw,zwSeeking the unmanned planen groups of positions under the binocular vision platform, and calculating the average value of the positions, wherein the average value is used as the position P of the unmanned aerial vehicle in the farmland, which is calculated by the binocular vision platformW＝{XW,YW,ZW}。
(III) advantageous effects
According to the unmanned aerial vehicle night plant protection method based on the multi-view visual feedback, the position of the unmanned aerial vehicle is fed back through the multi-view visual feedback method, position information is provided for unmanned aerial vehicle plant protection flight path planning in a night environment, the difficult problem of unmanned aerial vehicle night plant protection operation difficulty is solved, and the unmanned aerial vehicle has all-weather plant protection operation capacity.
The design of unique many meshes visual platform adopts the infrared binocular visual platform of multiunit to constitute many meshes visual system, can solve unmanned aerial vehicle's position in real time, can dismantle in a flexible way, arranges around the farmland at random, avoids singly organizing to have the vision blind area by the binocular visual system, can't all-round shortcoming of catching the unmanned aerial vehicle position.
Unique unmanned aerial vehicle visual positioning identification point design, this identification point adds infrared characteristic and catches the point, helps this visual positioning identification point of discernment under the many mesh visual system night environment.
Drawings
FIG. 1 is a diagram of the overall system of the present invention;
FIG. 2 is a connection diagram of the sub-platforms of the present invention;
FIG. 3 is a flow chart of the present invention;
FIG. 4 is a schematic view of the visual positioning of the identification points of the present invention.
In the figure:
1-multi-vision platform, 2-plant protection unmanned aerial vehicle platform, 3-pesticide spraying platform, 4-ground control platform, 5-farmland to be plant protected, 6-unmanned aerial vehicle, 7-ground control operation platform, 8-binocular vision sub-platform, 9-remote control equipment, 10-infrared characteristic identification point and 11-identification point support frame.
Detailed Description
In order to make the objects, contents, and advantages of the present invention clearer, the following detailed description of the embodiments of the present invention will be made in conjunction with the accompanying drawings and examples.
As shown in fig. 1, the night plant protection method for the unmanned aerial vehicle based on the multi-view vision is mainly applied to unmanned aerial vehicle position calculation and navigation in the night plant protection operation process of the unmanned aerial vehicle. The method mainly relates to the following objects: many meshes vision platform 1, plant protection unmanned aerial vehicle platform 2, pesticide spray platform 3, ground control platform 4, treat plant protection farmland 5.
The plant protection unmanned aerial vehicle platform 2 is a flight platform for unmanned aerial vehicle plant protection operation and is used for carrying a pesticide spraying platform 3, a visual positioning identification point, an ultrasonic barrier device and a visual barrier device; when unmanned aerial vehicle carries out the plant protection operation, plant protection unmanned aerial vehicle platform 2 carries on pesticide and sprays platform 3 and carry out the flight path real-time planning under the guide of many mesh visual platform 1, shelters from the barrier that the plant protection operation in-process meets through supersound, vision combination barrier mode at the in-process of flight.
The multi-view vision platform 1 is used for calculating the position of the unmanned aerial vehicle during night plant protection operation of the unmanned aerial vehicle and provides navigation position reference for the unmanned aerial vehicle. Many mesh visual platform 1 comprises infrared binocular vision sub-platform 8 of multiunit, arrange around 5 farmlands that need carry out the plant protection operation, and accomplish the position calibration between each group's binocular vision sub-platform 8, unmanned aerial vehicle is at the farmland flight in-process at night, carry out the real-time solution of unmanned aerial vehicle position through this many mesh visual platform 1, many mesh visual platform 1 sends unmanned aerial vehicle positional information for plant protection unmanned aerial vehicle platform 2 and ground control platform 4, use this positional information to carry out the flight path planning or with this positional information forward plant protection unmanned aerial vehicle platform 2 by ground control platform 4, provide the position foundation for unmanned aerial vehicle navigation.
Pesticide sprays platform 3 is the actuating mechanism of unmanned aerial vehicle plant protection operation, and the unmanned aerial vehicle plant protection operation in-process is used for controlling the intensity of spraying of control pesticide for the pesticide acts on victim crops root, better elimination crops plant diseases and insect pests.
Unmanned aerial vehicle ground control platform 4 is plant protection unmanned aerial vehicle flight control's ground control platform, mainly used receives unmanned aerial vehicle positional information and the position and the attitude information of unmanned aerial vehicle feedback of many mesh visual platform feedback, plans unmanned aerial vehicle's flight path to send control command to unmanned aerial vehicle through wireless communication, this control command can be for the flight path coordinate indication, possesses functions such as unmanned aerial vehicle flight path planning, unmanned aerial vehicle gesture shows, unmanned aerial vehicle position shows.
The unmanned aerial vehicle visual identification points are carried on the unmanned aerial vehicle platform 2, so that the identification points can be conveniently captured by a multi-view visual system, and then the specific positions of the unmanned aerial vehicle in the farmland can be calculated.
In the process of executing plant protection operation: firstly, arranging a plurality of binocular vision sub-platforms 8 around a farmland to be protected to form a multi-view vision platform 1; calibrating parameters of the binocular vision sub-platforms 8, and then calibrating the position relation among each group of binocular vision sub-platforms; after the calibration of the multi-view vision platform 1 is completed, the preparation check before the operation of the plant protection unmanned aerial vehicle platform 2 is carried out, and the position of the plant protection unmanned aerial vehicle platform 2 relative to the farmland is calculated through a multi-view vision system; after the inspection is finished, starting up the ground control platform 4 for inspection, and planning the flight path of the plant protection unmanned aerial vehicle by using the ground control platform 4 after the inspection is finished; and finally, starting plant protection operation, resolving the position of the plant protection unmanned aerial vehicle relative to the farmland in real time by capturing the infrared characteristic identification points on the plant protection unmanned aerial vehicle platform 2 by the multi-view vision platform 1 in the operation process, and feeding back the position information to an automatic pilot and a ground control platform of the plant protection unmanned aerial vehicle for real-time correction of the unmanned aerial vehicle route in the plant protection operation process.
As shown in fig. 2, the mutual communication relationship among the devices involved in the method for plant protection at night by the unmanned aerial vehicle is as follows: the multi-vision platform 1 composed of a plurality of groups of binocular vision sub-platforms 8 captures infrared identification points on the plant protection unmanned platform in real time, the position of the unmanned aerial vehicle in the farmland is resolved, the position is transmitted to an unmanned aerial vehicle automatic pilot and a ground control operation platform 7 through a wireless communication technology, two-way communication is carried out between the ground control operation platform 7 and the unmanned aerial vehicle 6 through wireless communication, the unmanned aerial vehicle sends pose information to the ground control operation platform 7, the ground control operation platform 7 sends control instructions according to the pose information of the unmanned aerial vehicle, in order to prevent emergency in the operation process, the ground control operation platform 7 is in wired connection with a remote controller, when the emergency happens, the unmanned aerial vehicle can be controlled to land through the remote controller.
As shown in fig. 3, the plant protection process at night of the present invention is as follows: firstly, arrange many mesh vision platform 1, then carry out many mesh vision platform 1's demarcation, carry out unmanned aerial vehicle system preparation work after many mesh vision platform 1 is markd and is accomplished, mark the position relation between unmanned aerial vehicle and the farmland at this in-process, carry out ground control platform 4 after the unmanned aerial vehicle system finishes preparing and prepare, after all preparation work are accomplished, begin unmanned aerial vehicle night plant protection operation.
As shown in fig. 4, the visual feature identification point according to the present invention is designed by using an infrared feature target point of an unmanned aerial vehicle, the infrared feature target point mainly comprises an infrared feature identification point 10 and an identification point support frame 11, the identification point support frame 11 is designed in a cross shape, the infrared feature identification point 10 is arranged at four end positions of the identification point support frame 11, and the infrared feature identification point is heated during the plant protection operation of the unmanned aerial vehicle, so as to improve the recognition degree of the infrared feature identification point in an infrared camera.
The position calibration algorithm among each group of binocular vision sub-platforms is as follows:
the multi-view vision platform comprises n binocular vision platform subsystems which are numbered as 1,2,3 …, n-1 and n respectively. Arranging the binocular vision platform subsystem around a farmland, and assuming that a left camera in the binocular vision platform subsystem with the number of 1 is used as the origin of a coordinate system of the binocular vision platform, wherein the camera parameter is M1L＝K1L[I|0]In which K is1LThe reference of the left camera in the binocular vision platform with the number of 1, I is a 3 multiplied by 3 identity matrix, 0 is a 3 multiplied by 1 zero matrix, and then the parameter of the right camera of the binocular vision platform with the number of 1 is M1R＝K1R[R1|t1]，R1,t1Respectively rotation and translation matrix of the right camera relative to the left camera, K1RThe parameters of the right camera in the binocular vision platform with the number of 1 are M respectively2L＝K2L[I|0]，M2R＝K2R[Rn|tn](ii) a By analogy, the number n is the parameter of the left camera and the right camera of the binocular vision platformNumber MnL＝KnL[I|0]，MnR＝KnR[Rn|tn]The invention mainly relates to an algorithm for calibrating a multi-view vision platform under the condition that parameters of cameras of the binocular vision platform are known (rotation and translation matrixes among the binocular vision platforms are calculated, and the parameters of the cameras of the binocular vision platforms are unified to the same world coordinate system). The calibration algorithm is as follows:
taking the position relation between two adjacent binocular vision platforms with the calibration numbers of 1 and 2 as an example, the infrared characteristic target points related to the patent are placed in the public view field areas of the two adjacent binocular vision platforms, the infrared characteristic target points are moved at least 3 positions (assuming that the 3 positions are moved) in the calibration process, and the positions of the target points in respective binocular camera coordinate systems are calculated by using a formula 1.
In formula 1, x, y and z are three-dimensional space coordinates of the infrared target point in a binocular vision platform coordinate system, and fl,frIs the focal length, r, of the left and right cameras in the camera reference matrix4-r9As a parameter in the camera matrix R, ty,tzAs a parameter in the camera matrix t, Xl、Yl、YrIs the coordinate position of the target point in the camera image.
Assuming that the three-dimensional spatial position of the target point solved by the binocular vision platform with the number of 1 by using the formula 1 is P1＝{X1,Y1,Z1,1}TAnd the three-dimensional space position of the target point calculated by the binocular vision platform with the number of 2 by using the formula 1 is P2＝{X2,Y2,Z21, since there is a rotational-translational transformation between the coordinate systems of the binocular vision platforms numbered 1 and 2, it is possible to rotate R21And translation t21A matrix representation, and thus;
P2＝{R21|t21}P1(2)
the three-dimensional space positions of a plurality of groups of target points form an equation group by using a formula (2), and the rotation R between the coordinate systems of the binocular vision platforms numbered 1 and 2 can be solved21And translation t21And (5) completing the calibration of the positions between the binocular vision platforms numbered 1 and 2 by using the parameters in the matrix. Similarly, the calibration of the binocular vision platform at other adjacent positions repeats the above process, and then the position calibration of the whole binocular vision platform is completed.
Unmanned plane position calculation algorithm
After the calibration of the multi-view vision platform is completed, all binocular vision platforms in the multi-view vision platform are unified to the same world coordinate system, so that in the process of plant protection operation of the unmanned aerial vehicle platform in a farmland, the binocular vision platforms capture infrared characteristic target points carried on the unmanned aerial vehicle platform in real time, and the three-dimensional space positions of the target points are calculated by using the following formula.
The multi-view visual platform comprises n groups of binocular visual platforms, so that three-dimensional space positions of n groups of targets can be calculated, 4 identification points are arranged on a target frame, each group of calculated targets comprises 4 target point three-dimensional space positions, and the three-dimensional space coordinates of the four target points are averaged to serve as the position p of the unmanned aerial vehicle under the group of binocular visual platformsw＝{xw,yw,zwAnd then, solving the positions of n groups of unmanned aerial vehicles, and calculating the average value of the positions, wherein the average value is used as the position P of the unmanned aerial vehicle in the farmland, which is calculated by the multi-view vision platformW＝{XW,YW,ZW}。
It will be understood by those skilled in the art that, unless otherwise defined, all terms (including technical and scientific terms) used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention belongs. It will be further understood that terms, such as those defined in commonly used dictionaries, should be interpreted as having a meaning that is consistent with their meaning in the context of the prior art and will not be interpreted in an idealized or overly formal sense unless expressly so defined herein.
The above description is only a preferred embodiment of the present invention, and it should be noted that, for those skilled in the art, several modifications and variations can be made without departing from the technical principle of the present invention, and these modifications and variations should also be regarded as the protection scope of the present invention.
Claims (8)
1. A night plant protection operation method of an unmanned aerial vehicle based on multi-view vision is characterized in that the method is realized based on a night plant protection operation system of the unmanned aerial vehicle, the system comprises four task platforms, namely an unmanned aerial vehicle platform, a multi-view vision platform, a pesticide spraying platform and an unmanned aerial vehicle ground control platform, and the unmanned aerial vehicle platform is a flight platform for unmanned aerial vehicle plant protection operation and is used for carrying a pesticide spraying platform and a visual positioning identification point; when the unmanned aerial vehicle carries out plant protection operation, the unmanned aerial vehicle platform carries a pesticide spraying platform to carry out real-time flight path planning under the guidance of the multi-view visual platform; the multi-view vision platform is used for capturing vision positioning identification points during night plant protection operation of the unmanned aerial vehicle, further performing position calculation of the unmanned aerial vehicle, sending position information of the unmanned aerial vehicle to the unmanned aerial vehicle platform and the ground control platform, and providing navigation position reference for the unmanned aerial vehicle; the pesticide spraying platform is an executing mechanism for unmanned aerial vehicle plant protection operation, and is used for controlling the spraying strength of pesticides in the unmanned aerial vehicle plant protection operation process, so that the pesticides act on the roots of damaged crops, and the crop diseases and insect pests are better eliminated; the ground control platform is used for receiving unmanned aerial vehicle position information fed back by the multi-view visual platform and unmanned aerial vehicle position and attitude information fed back by the unmanned aerial vehicle, planning the flight path of the unmanned aerial vehicle, and sending a control command to the unmanned aerial vehicle through wireless communication, and has the functions of unmanned aerial vehicle flight path planning, unmanned aerial vehicle attitude display and unmanned aerial vehicle position display; the unmanned aerial vehicle night plant protection operation method based on the system comprises the following steps:
arranging a plurality of binocular vision sub-platforms around a farmland to be protected to form a multi-view vision platform; calibrating parameters of the binocular vision sub-platforms, and then calibrating the position relation among each group of binocular vision sub-platforms;
after the calibration of the multi-view vision platform is completed, carrying out preparation inspection before the operation of the plant protection unmanned aerial vehicle platform, and calculating the position of the plant protection unmanned aerial vehicle platform relative to the farmland through a multi-view vision system;
after the inspection is finished, starting up the ground control platform for inspection, and planning the flight path of the plant protection unmanned aerial vehicle by using the ground control platform after the inspection is finished;
finally, plant protection operation is started, the multi-view vision platform calculates the position of the plant protection unmanned aerial vehicle relative to a farmland in real time by capturing infrared characteristic identification points on the plant protection unmanned aerial vehicle platform in the operation process, and feeds back position information to an automatic pilot and a ground control platform of the plant protection unmanned aerial vehicle for real-time correction of the unmanned aerial vehicle route in the plant protection operation process;
the pesticide sprays the platform and carries on the unmanned aerial vehicle platform, for the actuating mechanism of unmanned aerial vehicle plant protection operation, the intensity of spraying of unmanned aerial vehicle plant protection operation in-process control pesticide for the pesticide acts on the victim crops root.
2. The night planting operation method of the unmanned aerial vehicle based on the multi-vision as claimed in claim 1, characterized in that: the ground control platform and the unmanned aerial vehicle platform are in two-way communication through wireless communication, the unmanned aerial vehicle platform sends pose information to the ground control platform, the ground control platform plans a flight path according to the pose information of the unmanned aerial vehicle and sends a control command, and the control command indicates a flight path coordinate.
3. The night planting operation method of the unmanned aerial vehicle based on the multi-vision as claimed in claim 1, characterized in that: ground control platform still carries out wired connection with the remote controller, when emergency takes place, can descend through the unmanned aerial vehicle of remote controller artificial control.
4. The night planting operation method of the unmanned aerial vehicle based on the multi-vision as claimed in claim 1, characterized in that: the multi-view vision platform transmits the position of the unmanned aerial vehicle to the unmanned aerial vehicle platform and the ground control platform through a wireless communication technology.
5. The night planting operation method of the unmanned aerial vehicle based on the multi-vision as claimed in claim 1, characterized in that: the unmanned aerial vehicle platform still carries on the visual positioning identification point, this visual positioning identification point adopts the design of the infrared characteristic target point of unmanned aerial vehicle, this infrared characteristic target point comprises infrared characteristic identification point and identification point support frame, the identification point support frame designs into the cross shape, infrared characteristic identification point arranges in four extreme point positions of identification point support frame, unmanned aerial vehicle is at the plant protection operation in-process, infrared characteristic identification point is heated, improves the discernment degree of infrared characteristic identification point in infrared camera.
6. The night planting operation method of the unmanned aerial vehicle based on the multi-vision as claimed in claim 1, characterized in that: the unmanned aerial vehicle platform still carries on supersound shielding equipment and vision shielding device, avoids the barrier that the plant protection operation in-process meets through supersound, vision combination shielding mode at the in-process that unmanned aerial vehicle flies.
7. The night planting operation system of the unmanned aerial vehicle based on the multi-vision as claimed in claim 5, wherein: the position calibration algorithm among each group of binocular vision sub-platforms is as follows:
the multi-view vision platform comprises n binocular vision platform subsystems which are respectively numbered as 1,2,3 …, n-1 and n; arranging the binocular vision platform subsystem around a farmland, and assuming that a left camera in the binocular vision platform subsystem with the number of 1 is used as the origin of a coordinate system of the binocular vision platform, wherein the camera parameter is M1L＝K1L[I|0]In which K is1LThe reference of the left camera in the binocular vision platform with the number of 1, I is a 3 multiplied by 3 identity matrix, 0 is a 3 multiplied by 1 zero matrix, and then the parameter of the right camera of the binocular vision platform with the number of 1 is M1R＝K1R[R1|t1]，R1,t1Respectively a rotation and translation matrix of the right camera relative to the left camera,K1Rthe parameters of the right camera in the binocular vision platform with the number of 1 are M respectively2L＝K2L[I|0]，M2R＝K2R[Rn|tn](ii) a By analogy, the parameter of the left camera and the right camera of the binocular vision platform with the number of n is MnL＝KnL[I|0]，MnR＝KnR[Rn|tn]；
Assuming that the numbers of two adjacent binocular vision platforms to be calibrated are 1 and 2, placing infrared characteristic target points in a common visual field area of the two adjacent binocular vision platforms, moving the infrared characteristic target points at least 3 positions in the calibration process, calculating the positions of the target points in respective binocular camera coordinate systems by using a formula 1,
in formula 1, x, y and z are three-dimensional space coordinates of the infrared target point in a binocular vision platform coordinate system, and fl,frIs the focal length, r, of the left and right cameras in the camera reference matrix4-r9As a parameter in the camera matrix R, ty,tzAs a parameter in the camera matrix t, Xl、Yl、YrThe coordinate position of the target point in the camera image is taken;
assuming that the three-dimensional spatial position of the target point solved by the binocular vision platform with the number of 1 by using the formula 1 is P1＝{X1,Y1,Z1,1}TAnd the three-dimensional space position of the target point calculated by the binocular vision platform with the number of 2 by using the formula 1 is P2＝{X2,Y2,Z21, since there is a rotational-translational transformation between the coordinate systems of the binocular vision platforms numbered 1 and 2, it is possible to rotate R21And translation t21Is represented by a matrix, thus
P2＝{R21|t21}P1(2)
Three-dimensional space positions of a plurality of groups of target points form an equation set by using a formula (2)It is possible to resolve the rotation R between the coordinate systems of the binocular vision platforms numbered 1 and 221And translation t21The parameters in the matrix finish the calibration of the position between the binocular vision platforms numbered 1 and 2;
similarly, the calibration of the binocular vision platform at other adjacent positions repeats the above process, and then the position calibration of the whole binocular vision platform is completed.
8. The night planting operation system of the unmanned aerial vehicle based on the multi-vision as claimed in claim 7, wherein: the unmanned plane position calculation algorithm is as follows:
after the position calibration of the multi-view vision platform is completed, all binocular vision platforms in the multi-view vision platform are unified to the same world coordinate system, so that the binocular vision platforms capture infrared characteristic target points carried on the unmanned aerial vehicle platform in real time in the process of plant protection operation of the unmanned aerial vehicle platform in a farmland, and the three-dimensional space position of the target points is calculated by using the following formula 3;
the multi-view visual platform comprises n groups of binocular visual platforms, so that three-dimensional space positions of n groups of targets can be calculated, 4 identification points are arranged on a target frame, each group of calculated targets comprises 4 target point three-dimensional space positions, and the three-dimensional space coordinates of the four target points are averaged to serve as the position p of the unmanned aerial vehicle under the group of binocular visual platformsw＝{xw,yw,zwSolving the positions of the unmanned aerial vehicles under the n groups of binocular vision platforms, and calculating the average value of the positions, wherein the average value is used as the position P of the unmanned aerial vehicle in the farmland, which is calculated by the multi-view vision platformW＝{XW,YW,ZW}。
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201911199206.0A CN110989664A (en) | 2019-11-29 | 2019-11-29 | Unmanned aerial vehicle night plant protection method based on multi-view vision |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201911199206.0A CN110989664A (en) | 2019-11-29 | 2019-11-29 | Unmanned aerial vehicle night plant protection method based on multi-view vision |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN110989664A true CN110989664A (en) | 2020-04-10 |
Family
ID=70088375
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201911199206.0A Pending CN110989664A (en) | 2019-11-29 | 2019-11-29 | Unmanned aerial vehicle night plant protection method based on multi-view vision |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN110989664A (en) |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111543413A (en) * | 2020-05-28 | 2020-08-18 | 中国农业大学 | Method and system for accurately applying pesticide by agricultural robot in air-ground cooperation |
| CN113432603A (en) * | 2021-06-25 | 2021-09-24 | 中国船舶重工集团公司第七0七研究所 | Method for measuring relative pose of mother platform and son platform in complex environment |
Citations (10)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN105205824A (en) * | 2015-09-25 | 2015-12-30 | 北京航空航天大学 | Multi-camera global calibration method based on high-precision auxiliary cameras and ball targets |
| CN106296718A (en) * | 2016-09-22 | 2017-01-04 | 西北工业大学 | Camera array quick calibrating method based on stereoscopic vision navigation system |
| CN106774365A (en) * | 2016-12-20 | 2017-05-31 | 华中科技大学 | A kind of unmanned plane debugging system based on external view positioning |
| CN107358631A (en) * | 2017-06-27 | 2017-11-17 | 大连理工大学 | A kind of binocular vision method for reconstructing for taking into account three-dimensional distortion |
| CN107697302A (en) * | 2017-08-31 | 2018-02-16 | 广东容祺智能科技有限公司 | A kind of pesticide spraying UAS based on binocular vision |
| CN108279677A (en) * | 2018-02-08 | 2018-07-13 | 张文 | Track machine people's detection method based on binocular vision sensor |
| CN109029417A (en) * | 2018-05-21 | 2018-12-18 | 南京航空航天大学 | Unmanned plane SLAM method based on mixing visual odometry and multiple dimensioned map |
| CN109085594A (en) * | 2018-06-01 | 2018-12-25 | 北京农业智能装备技术研究中心 | A kind of unmanned aerial vehicle onboard system and application guidance system for being administered guiding |
| CN110371294A (en) * | 2019-07-12 | 2019-10-25 | 仲恺农业工程学院 | Plant protection drone system and plant protection drone operational method |
| CN110989643A (en) * | 2019-11-29 | 2020-04-10 | 北京特种机械研究所 | Unmanned aerial vehicle night plant protection system based on multi-view vision |
-
2019
- 2019-11-29 CN CN201911199206.0A patent/CN110989664A/en active Pending
Patent Citations (10)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN105205824A (en) * | 2015-09-25 | 2015-12-30 | 北京航空航天大学 | Multi-camera global calibration method based on high-precision auxiliary cameras and ball targets |
| CN106296718A (en) * | 2016-09-22 | 2017-01-04 | 西北工业大学 | Camera array quick calibrating method based on stereoscopic vision navigation system |
| CN106774365A (en) * | 2016-12-20 | 2017-05-31 | 华中科技大学 | A kind of unmanned plane debugging system based on external view positioning |
| CN107358631A (en) * | 2017-06-27 | 2017-11-17 | 大连理工大学 | A kind of binocular vision method for reconstructing for taking into account three-dimensional distortion |
| CN107697302A (en) * | 2017-08-31 | 2018-02-16 | 广东容祺智能科技有限公司 | A kind of pesticide spraying UAS based on binocular vision |
| CN108279677A (en) * | 2018-02-08 | 2018-07-13 | 张文 | Track machine people's detection method based on binocular vision sensor |
| CN109029417A (en) * | 2018-05-21 | 2018-12-18 | 南京航空航天大学 | Unmanned plane SLAM method based on mixing visual odometry and multiple dimensioned map |
| CN109085594A (en) * | 2018-06-01 | 2018-12-25 | 北京农业智能装备技术研究中心 | A kind of unmanned aerial vehicle onboard system and application guidance system for being administered guiding |
| CN110371294A (en) * | 2019-07-12 | 2019-10-25 | 仲恺农业工程学院 | Plant protection drone system and plant protection drone operational method |
| CN110989643A (en) * | 2019-11-29 | 2020-04-10 | 北京特种机械研究所 | Unmanned aerial vehicle night plant protection system based on multi-view vision |
Cited By (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111543413A (en) * | 2020-05-28 | 2020-08-18 | 中国农业大学 | Method and system for accurately applying pesticide by agricultural robot in air-ground cooperation |
| CN111543413B (en) * | 2020-05-28 | 2021-05-04 | 中国农业大学 | Method for accurately applying pesticide by agricultural robot in cooperation with air and ground |
| CN113432603A (en) * | 2021-06-25 | 2021-09-24 | 中国船舶重工集团公司第七0七研究所 | Method for measuring relative pose of mother platform and son platform in complex environment |
| CN113432603B (en) * | 2021-06-25 | 2022-09-16 | 中国船舶重工集团公司第七0七研究所 | Method for measuring relative pose of mother platform and son platform in complex environment |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| AU2019246795B2 (en) | Automatic target recognition and dispensing system | |
| US11526997B2 (en) | Targeting agricultural objects to apply units of treatment autonomously | |
| CN106200683B (en) | Unmanned plane plant protection system and plant protection method | |
| US11406052B2 (en) | Cartridges to employ an agricultural payload via an agricultural treatment delivery system | |
| US11812681B2 (en) | Precision treatment of agricultural objects on a moving platform | |
| US11465162B2 (en) | Obscurant emission to assist image formation to automate agricultural management and treatment | |
| CN106585992A (en) | Method and system for intelligent identification and accurate pesticide spraying using unmanned aerial vehicles | |
| US20230083872A1 (en) | Pixel projectile delivery system to replicate an image on a surface using pixel projectiles | |
| CN110989664A (en) | Unmanned aerial vehicle night plant protection method based on multi-view vision | |
| US20210185942A1 (en) | Managing stages of growth of a crop with micro-precision via an agricultural treatment delivery system | |
| CN110989643A (en) | Unmanned aerial vehicle night plant protection system based on multi-view vision | |
| US20230029636A1 (en) | Unmanned aerial vehicle | |
| CN108303963A (en) | A kind of Intelligent unattended driving equipment for plant protection earth station system | |
| CN113273376A (en) | Modularization weeding device | |
| CN203762122U (en) | Indoor autonomous mobile pesticide spraying robot | |
| CA3164825A1 (en) | Agricultural projectile delivery system | |
| US11653590B2 (en) | Calibration of systems to deliver agricultural projectiles | |
| CN108719242B (en) | Pesticide spraying system | |
| CN107711799A (en) | A kind of artificial intelligence pesticide spraying system based on unmanned machine testing orchard information | |
| CA3224120A1 (en) | Multi-device agricultural field treatment | |
| CN116829460A (en) | Expert flying robot system for field autonomous operation |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| AD01 | Patent right deemed abandoned | ||
| AD01 | Patent right deemed abandoned |
Effective date of abandoning: 20240507 |