CN110456811A - Unmanned plane selectivity obstacle avoidance system and method based on binocular vision and three axis holders - Google Patents
Unmanned plane selectivity obstacle avoidance system and method based on binocular vision and three axis holders Download PDFInfo
- Publication number
- CN110456811A CN110456811A CN201910776823.6A CN201910776823A CN110456811A CN 110456811 A CN110456811 A CN 110456811A CN 201910776823 A CN201910776823 A CN 201910776823A CN 110456811 A CN110456811 A CN 110456811A
- Authority
- CN
- China
- Prior art keywords
- unmanned plane
- angle
- barrier
- axis
- binocular
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Withdrawn
Links
Classifications
-
- G—PHYSICS
- G05—CONTROLLING; REGULATING
- G05D—SYSTEMS FOR CONTROLLING OR REGULATING NON-ELECTRIC VARIABLES
- G05D1/00—Control of position, course or altitude of land, water, air, or space vehicles, e.g. automatic pilot
- G05D1/08—Control of attitude, i.e. control of roll, pitch, or yaw
- G05D1/0808—Control of attitude, i.e. control of roll, pitch, or yaw specially adapted for aircraft
-
- G—PHYSICS
- G05—CONTROLLING; REGULATING
- G05D—SYSTEMS FOR CONTROLLING OR REGULATING NON-ELECTRIC VARIABLES
- G05D1/00—Control of position, course or altitude of land, water, air, or space vehicles, e.g. automatic pilot
- G05D1/10—Simultaneous control of position or course in three dimensions
- G05D1/101—Simultaneous control of position or course in three dimensions specially adapted for aircraft
Abstract
The invention discloses a kind of unmanned plane selectivity obstacle avoidance system based on binocular vision and three axis holders and method the following steps are included: opening eight road ultrasonic waves and binocular vision, and determine the heading angle of unmanned plane；Binocular camera is rotated into heading by three axis holders and is opened, the environment of real-time monitoring heading；Obtain depth information；The orientation that the pitch angle and roll angle comprehensive analysis that resulting obstructions chart picture is obtained with UAV Attitude identification module go out barrier is shot by binocular camera, barrier is at a distance from unmanned plane；Tree plum sends control panel to carry out judging whether avoidance；Selective avoidance mode；Plan unmanned plane surrounding environment and eight road avoiding obstacles by supersonic wave；The mentioned method of the present invention not only greatly reduces cost, more can 360 ° of loads without dead angle observation surrounding environment and when mitigating unmanned plane autonomous flight；The offset of pitch angle and roll angle to pixel flight safety frame when the use of three axis holders eliminates unmanned plane during flying, it is ensured that flight safety.
Description
Technical field
The present invention relates to unmanned plane avoidance technical fields more particularly to a kind of based on nobody of binocular vision and three axis holders
Machine selectivity obstacle avoidance system and method.
Background technique
With unmanned plane market development, the requirement of automatic sensing and control to unmanned plane is also higher and higher, especially exists
In terms of avoidance, unmanned function is made to complete aerial mission in complex environment.Common avoidance equipment has infrared, radar, ultrasound at present
Wave and binocular range finder module.It is infrared more sensitive to external conditions such as illumination；Radar range finding technology is not overripened at present, exists
Erroneous judgement problem；Ultrasonic wave presence can not be accurately positioned Obstacle Position, and ranging range is limited；Binocular ranging or cognitive disorders object
When binocular camera limited viewing angle and calculation amount it is larger, ambient is affected to it, and 360 ° difficult to realize are kept away without dead angle
Barrier.
When multi-rotor unmanned aerial vehicle flight forward, unmanned plane is heeling condition, distance measured by binocular vision or not
It is the horizontal actual range between unmanned plane and ground or barrier, it may appear that unnecessary avoidance or hovering, so that unmanned plane
Cannot according to it is anticipated that rule carry out avoidance, will be greatly promoted a possibility that so that unmanned plane is collided, to reduce
The practicability and reliability of binocular obstacle avoidance system.
Summary of the invention
In view of the drawbacks described above of the prior art, the present invention provides a kind of based on the choosing of the unmanned plane of binocular vision and three axis holders
Selecting property obstacle avoidance system and method.Some barriers for not needing avoidance can be excluded by the use of multiplex ultrasonic, reduce journey
The operand of sequence is convenient for unmanned plane during flying.When unmanned plane, which there are problems that, to need avoidance, ultrasonic wave can not be accurately positioned
The position of barrier makes up the impossible task of ultrasonic wave using binocular ranging.Pass through the yaw axis in three axis clouds terrace systems
It realizes 360 ° of binocular camera rotations, preferably observes unmanned plane surrounding environment.Compared to eight tunnel binocular cameras, not only significantly
Ground reduces cost, it is often more important that mitigates load when unmanned plane autonomous flight.In addition, pitch axis in three axis clouds terrace systems and
Roll axis remains at binocular camera on horizontal position, overcomes original binocular camera and is fixed on unmanned plane, when
Unmanned plane tilts existing flight error when flight, blends, overcomes the deficiencies of existing technologies with flight control system, thus fastly
The stable carry out automatic obstacle avoiding of speed.
The invention patent solves technical solution used by its technical problem:
Unmanned plane selectivity obstacle avoidance system and method based on binocular vision and three axis holders, which is characterized in that described to keep away
Barrier system includes multiplex ultrasonic module, binocular range finder module, raspberry pie control panel, three axis clouds terrace system modules, unmanned plane appearance
State identification module, flight control modules and transmission module.
The multiplex ultrasonic module includes that eight ultrasounds of unmanned plane surrounding involve a CD4051 chip, for detecting
Whether there are obstacles for surrounding environment, measures at a distance from barrier, simply judges whether avoidance and uses binocular camera；
Ultrasonic measurement range: 60 ° of horizontal view angle, 60 ° of vertical angle of view；Eight ultrasonic waves be placed on unmanned plane every 45 degree before, front left,
A left side, rear left, rear, rear right, right, eight orientation of front right.
The three axis clouds terrace system modules are made of three mutually perpendicular pitch axis, roll axis and yaw axis,
For keeping binocular camera horizontal position, yaw axis realizes 360 ° of binocular camera rotations for middle roll axis and pitch axis.
The binocular camera shooting head module is mounted on three axis clouds terrace systems immediately below unmanned plane, sends control panel with tree plum
It is connected, for shooting surrounding environment, handles disparity map, and rebuild to three-dimensional, calculate depth；The binocular is taken the photograph
As head measurement range: 60 ° of horizontal view angle, 60 ° of vertical angle of view.
The gesture recognition module includes accelerometer, gyroscope, magnetometer, barometer, for obtaining unmanned plane appearance
State angle and height.
The flight control modules are mainly embedded scm and earth station, for controlling the flight appearance of unmanned plane
State.
The transmission module is mainly digital transmission module, figure transmission module and remote controler.Digital transmission module and figure transmission module are used
In transmission data and video information, remote controler is for artificially controlling unmanned plane.
Further, the multi-rotor unmanned aerial vehicle selectivity avoidance obstacle method based on binocular vision and three axis holders, including with
Lower step:
Step 1 opens eight road ultrasonic waves and all the way binocular vision, and determines the heading angle of unmanned plane
1. opening eight road ultrasonic waves and all the way binocular vision simultaneously；
2. determining the angle γ in unmanned plane during flying direction and heading
The horizontal axis component F in horizontal plane is calculated by spin matrixaWith the longitudinal axis component F in horizontal planeb:
In formula, FcFor the longitudinal axis component in vertical plane, F is lift of the propeller to unmanned plane；β is roll angle, β ∈ (-
25 °, 25 °), angle beta is positive when the rolling to the right of unmanned plane body, to the left when β be negative；θ is pitch angle, θ ∈ (- 25 °, 25 °), nothing
θ is positive when man-machine body head is downward, and θ is negative when upward；
Fa=-F sin β (2)
Fb=-F cos β sin θ (3)
The angle γ of unmanned plane during flying direction and heading is
Wherein, sgn (x) is sign function, as x > 0, sgn (x)=1；As x < 0, sgn (x)=- 1.Using head as base
Directrix is rotated clockwise and is positive, and rotation is negative counterclockwise；The range of angle γ is [- π, π].
3. UAV Attitude module obtains unmanned plane pitch angle, roll angle and yaw angle；Three axis holders compensate unmanned plane
Pitch angle and roll angle, so that binocular camera is maintained at horizontal position；By the yaw axis in three axis clouds terrace systems by binocular
Camera rotates clockwise γ angle, so that the visual angle of binocular camera is consistent with unmanned plane during flying direction, is more advantageous to
Automatic obstacle avoiding；
Step 2 obtains depth information, and carries out judging whether avoidance by raspberry pie control panel
Based on unmanned plane current pose information and the barrier depth value obtained according to image；The UAV Attitude letter
Breath is unmanned plane pitch angle, roll angle and yaw angle acquired in the attitude transducer set on unmanned plane, and attitude transducer is to add
Speedometer, gyroscope and magnetometer；The depth value of the barrier is obtained by following steps:
1. it includes the first image and the second image that the binocular camera, which obtains the ambient image in front of unmanned plane,；
2. the binocular camera obtains the inside and outside parameter of camera by Zhang Shi standardization, then passes through the outer of camera
Portion's parameter and distortion factor correct the first image and the second image；By after correction the first image and the second image use
SGAM algorithm carries out Stereo matching, obtains the disparity map of two images；
3. obtaining depth image based on the disparity map, barrier depth value is extracted according to the depth image；
When binocular camera detects barrier, tree plum sends control panel according to obtained depth value, and by following several
A step come judge be carry out avoidance, be also to maintain original route and continue to fly:
1. wide by the length of unmanned plane, height calculates nothing under the influence of pitching, roll, through the projection on heading
Man-machine practical rectangle frame length L1With width W1:
L1=(h sin | β |+l cos | β |) cos | γ |+(b cos | θ |+h cos | β | sin | θ |) sin | and γ | (5)
In formula, h is unmanned plane height, and l is unmanned plane length；B is unmanned plane width；
W1=b sin | θ |+l sin | β |+h cos | θ | cos | β | (6)
2. calculating the horizontal distance d of unmanned plane and nearest barrier1:
d1=z1 (7)
In formula, z1For the vertical pivot coordinate of nearest barrier；
3. calculating the length △ U of pixel flight safety frame1With width △ V1, concrete form are as follows:
In formula, D1The parallax of image center when be depth value being d1, B is the distance between two cameras, and f is double
The focal length of lens camera；
In formula, Δ x1For L1The correspondingly-sized under image coordinate system in depth value d1；
In formula, Δ u1For L1In depth value d1When pixel coordinate system under corresponding size, dx be unit pixel in horizontal axis
Physical size；
In formula, Δ v1For W1In depth value d1When pixel coordinate system under corresponding size, dy be unit pixel in the longitudinal axis
Physical size；By Δ u1With Δ v1All expand as original r times, and r > 1；General r takes 1.2, can reduce extraneous factor influence and
Processing emergency case in time, then have
ΔU1=r* Δ u1, Δ V1=r* Δ v1 (12)
4. calculating the vertical axis component s in the plummet plane at unmanned plane center and binocular camera centera:
5. calculating saCorrespondingly-sized s under pixel coordinate system1:
6. pixel flight safety frame is offset up s1, traverse pixel flight safety frame: if 1) in pixel flight safety frame
There is no barrier, unmanned plane flies to pixel flight safety frame direction, and after unmanned plane passes through pixel flight safety frame, binocular is taken the photograph
As head rotate to it is consistent with unmanned plane during flying direction；2) if there are barrier in pixel flight safety frame, unmanned plane carries out corresponding
Selective avoidance.
Step 3 selectivity avoidance mode
Based on, there are barrier, unmanned plane carries out 2 kinds of selective avoidance modes in pixel flight safety frame:
It goes slowly 1. unmanned plane reduces speed now, the depth value deep of the nearest barrier of record pixel flight safety frame, and with
The threshold value set_d of setting compares: deep enters jogging area when being greater than set_d, unmanned plane adjusts pitch angle and roll angle, appropriate to drop
Low velocity；Deep enters alert zone when being less than set_d, and unmanned plane slows down hovering immediately, and passes through digital transmission module and figure transmission module
Information is sent, artificial remote control is prompted；
2. converting binary picture for disparity map, pixel flight safety frame is traversed, passes through point of the pixel value in analysis block
Cloth speculates the position of barrier in frame.If barrier left side, unmanned plane in frame fly to the right relative to pixel flight safety frame
Row；If barrier flies in frame inner underside, unmanned plane relative to pixel flight safety frame upwards；If barrier flies in pixel
Safe frame proportion is greater than 80%, then unmanned plane flies 5 meters upwards, if 8 road ultrasonic waves do not detect barrier within 5 meters
Hinder object, by the airline operation originally planned, binocular camera rotates to consistent with unmanned plane during flying direction unmanned plane；If 5 meters
Except 8 road ultrasonic waves at least 1 road detect in threshold value lmaxInside there is barrier, unmanned plane slows down hovering immediately, while will be double
Lens camera rotates to the region where the barrier, and sends information by digital transmission module and figure transmission module, prompts artificial distant
Control.
Step 4 plans unmanned plane surrounding environment and eight road avoiding obstacles by supersonic wave
[- π, π] is set by the horizontal plane angle range of unmanned plane, using head as benchmark line, rotates clockwise and is positive, it is inverse
Hour hands rotation is negative；Eight road ultrasonic wave places 1 every 45 °, is divided into eight regions, 60 ° of each region, with unmanned plane machine
Head direction is that be named as the 1st region and the center line in region be heading for the region of symmetry axis, demarcates the in the direction of the clock
2,3,4,5,6,7,8 regions, then corresponding center line angle is 0 °, 45 °, 90 °, 135 °, 180 °, -135 °, -90 °, -45 °；It is super
Acoustic measurement range: 60 ° of horizontal view angle, 60 ° of vertical angle of view, then there is overlapping region, each overlapping portion in adjacent two-way ultrasonic wave
The centric angle divided is 22.5 °, 67.5 °, 112.5 °, 157.5 °, -157.5 °, -112..5 °, -67.5 °, -22.5 °；Unmanned plane
By eight tunnel ultrasound examination surrounding environments, whether there are obstacles and measurement is at a distance from barrier；If ultrasound examination goes out
The region needs avoidance, then binocular camera is gone to the position of center line in the region, obtains unmanned plane by binocular camera
The ambient image in front；The ambient image is that two cameras of binocular camera distinguish captured first in synchronization
Image and the second image；To obtain depth information value, and flight control modules are uploaded to, makes corresponding avoidance measure；
Further, since the factors such as ambient influence, when the depth value fluctuation that binocular vision is read is larger, unmanned plane
Mainly take avoiding obstacles by supersonic wave.
The invention adopts the above technical scheme compared with prior art, has following technical effect that
1. the multiplex ultrasonic module based on CD4051, cheap, flexibility is high；In contrast pin resource is occupied
Reading that is low, selecting 1CD4051 chip just to complete 8 tunnel ultrasonic wave module data by one 8.Embedded processing is alleviated simultaneously
The load of device, so that the mobility of unmanned plane is more outstanding；
2. realizing 360 ° of rotation binocular cameras by three axis clouds terrace systems, unmanned plane surrounding environment can be shot；It compares
Yu Balu binocular camera, not only greatly reduces cost, it is often more important that can with 360 ° without dead angle observe surrounding environment and
Mitigate load when unmanned plane autonomous flight；
The offset of pitching and roll to pixel flight safety frame when 3. the use of holder eliminates unmanned plane during flying, therefore can
To ensure safety when unmanned plane during flying；But since binocular camera is hung over below unmanned plane, have centainly with unmanned plane center
Distance, it is therefore desirable to pixel flight safety frame center is translated up with unmanned plane center in same level；
4. ultrasonic wave module and using safety when more ensuring unmanned plane during flying while binocular range finder module；
5. when the depth value fluctuation that binocular vision is read is larger, unmanned plane is mainly adopted since the factors such as ambient influence
Take avoiding obstacles by supersonic wave；
6. tilting the interference to binocular ranging, and the range that will be detected in rectangle frame by angle modification unmanned plane
It is divided into three parts according to distance and unmanned plane speed and is respectively processed, keeps the avoidance route planning of unmanned plane more reasonable, quasi-
Really.
Detailed description of the invention
Fig. 1 is the structure chart of the unmanned plane selectivity avoidance obstacle method based on ultrasonic wave and binocular camera；
Fig. 2 is the flow chart of the unmanned plane selectivity avoidance obstacle method based on ultrasonic wave and binocular camera；
Fig. 3 is γ ° of angle of the schematic diagram in unmanned plane during flying direction Yu unmanned plane head；
Fig. 4 is the schematic diagram of the practical rectangle frame size of unmanned plane；
Fig. 5 is the schematic diagram of the horizontal distance of unmanned plane and nearest barrier；
Fig. 6 is the schematic diagram that the practical rectangle of unmanned plane is converted into pixel flight safety frame；
Fig. 7 is the schematic diagram between unmanned plane and barrier apart from processing mode；
Fig. 8 is unmanned plane surrounding environment schematic diagram.
Specific embodiment
In order to keep the objectives, technical solutions, and advantages of the present invention clearer, below in conjunction with attached drawing 1-8, to the present invention into
One step detailed description.
Referring to Fig.1, unmanned plane selectivity obstacle avoidance system and control method based on binocular vision and three axis holders, feature
It is, the obstacle avoidance system includes multiplex ultrasonic module, raspberry pie control panel, binocular range finder module, three axis clouds terrace system moulds
Block, UAV Attitude identification module, flight control modules and transmission module.
The multiplex ultrasonic module includes that eight ultrasounds of unmanned plane surrounding involve a CD4051 chip, for detecting
Whether there are obstacles for surrounding environment, measures at a distance from barrier, simply judges whether avoidance and uses binocular camera；
Ultrasonic measurement range: 60 ° of horizontal view angle, 60 ° of vertical angle of view；
The three axis clouds terrace system modules are made of three mutually perpendicular pitch axis, roll axis and yaw axis,
For keeping binocular camera horizontal position, yaw axis realizes 360 ° of binocular camera rotations for middle roll axis and pitch axis.
The binocular camera shooting head module is mounted on three axis clouds terrace systems immediately below unmanned plane, sends control panel with tree plum
It is connected, for shooting surrounding environment, handles disparity map, and rebuild to three-dimensional, calculate depth；The binocular is taken the photograph
As head measurement range: 60 ° of horizontal view angle, 60 ° of vertical angle of view.
The gesture recognition module includes accelerometer, gyroscope, magnetometer, barometer, for obtaining unmanned plane appearance
State angle and height.
The flight control modules are mainly embedded scm and earth station, for controlling the flight appearance of unmanned plane
State.
The transmission module is mainly digital transmission module, figure transmission module and remote controler.Digital transmission module and figure transmission module are used
In transmission data and video information, remote controler is for artificially controlling unmanned plane.
Further, as shown in Fig. 2, the unmanned plane selectivity avoidance obstacle method based on binocular vision and three axis holders,
The following steps are included:
Step 1 opens eight road ultrasonic waves and all the way binocular vision, and determines the heading angle of unmanned plane
1. opening eight road ultrasonic waves and all the way binocular vision simultaneously；
2. determining the angle γ in unmanned plane during flying direction and heading
The horizontal axis component F in horizontal plane is calculated by spin matrixaWith the longitudinal axis component F in horizontal planeb:
In formula, FcFor the longitudinal axis component in vertical plane, F is lift of the propeller to unmanned plane；β is roll angle, β ∈ (-
25 °, 25 °), angle beta is positive when the rolling to the right of unmanned plane body, to the left when β be negative；θ is pitch angle, θ ∈ (- 25 °, 25 °), nothing
θ is positive when man-machine body head is downward, and θ is negative when upward；
Fa=-F sin β (2)
Fb=-F cos β sin θ (3)
The angle γ (see Fig. 3) of unmanned plane during flying direction and heading is
Wherein, sgn (x) is sign function, as x > 0, sgn (x)=1；As x < 0, sgn (x)=- 1.Using head as base
Directrix is rotated clockwise and is positive, and rotation is negative counterclockwise；The range of angle γ is [- π, π].
3. UAV Attitude module obtains unmanned plane pitch angle, roll angle and yaw angle；Three axis holders compensate unmanned plane
Pitch angle and roll angle, so that binocular camera is maintained at horizontal position；By the yaw axis in three axis clouds terrace systems by binocular
Camera rotates clockwise γ angle, so that the visual angle of binocular camera is consistent with unmanned plane during flying direction, is more advantageous to
Automatic obstacle avoiding；
Step 2 obtains depth information, and carries out judging whether avoidance by raspberry pie control panel
Based on unmanned plane current pose information and the barrier depth value obtained according to image；The UAV Attitude letter
Breath is unmanned plane pitch angle and roll angle acquired in attitude transducer set on unmanned plane, attitude transducer be accelerometer,
Gyroscope and magnetometer；The depth value of the barrier is obtained by following steps:
1. it includes the first image and the second image that the binocular camera, which obtains the ambient image in front of unmanned plane,；
2. the binocular camera obtains the inside and outside parameter of camera by Zhang Shi standardization, then passes through the outer of camera
Portion's parameter and distortion factor correct the first image and the second image；By after correction the first image and the second image use
SGAM algorithm carries out Stereo matching, obtains the disparity map of two images；
3. obtaining depth image based on the disparity map, barrier depth value is extracted according to the depth image；
When binocular camera detects barrier, tree plum sends control panel according to obtained depth value, and by following several
A step come judge be carry out avoidance, be also to maintain original route and continue to fly:
1. wide by the length of unmanned plane, height calculates nothing under the influence of pitching, roll, through the projection on heading
Man-machine practical rectangle frame length L1With width W1(as shown in Figure 4):
L1=(h sin | β |+l cos | β |) cos | γ |+(b cos | θ |+h cos | β | sin | θ |) sin | and γ | (5)
In formula, h is unmanned plane height, and l is unmanned plane length；B is unmanned plane width；
W1=b sin | θ |+l sin | β |+h cos | θ | cos | β | (6)
2. calculating the horizontal distance d of unmanned plane and nearest barrier1(see Fig. 5):
d1=z1 (7)
In formula, z1For the vertical pivot coordinate of nearest barrier；
3. calculating the length △ U of pixel flight safety frame1With width △ V1(see Fig. 6), concrete form are as follows:
In formula, D1The parallax of image center when be depth value being d1, B is the distance between two cameras, and f is double
The focal length of lens camera；
In formula, Δ x1For L1The correspondingly-sized under image coordinate system in depth value d1；
In formula, Δ u1For L1In depth value d1When pixel coordinate system under corresponding size, dx be unit pixel in horizontal axis
Physical size；
In formula, Δ v1For W1In depth value d1When pixel coordinate system under corresponding size, dy be unit pixel in the longitudinal axis
Physical size；By Δ u1With Δ v1All expand as original r times, and r > 1；General r takes 1.2, can reduce extraneous factor influence and
Processing emergency case in time, then have
ΔU1=r* Δ u1, Δ V1=r* Δ v1 (12)
4. calculating the vertical axis component s in the plummet plane at unmanned plane center and binocular camera centera:
5. calculating saCorrespondingly-sized s under pixel coordinate system1:
6. pixel flight safety frame is offset up s1, traverse pixel flight safety frame: if 1) in pixel flight safety frame
There is no barrier, unmanned plane flies to pixel flight safety frame direction, and after unmanned plane passes through pixel flight safety frame, binocular is taken the photograph
As head rotate to it is consistent with unmanned plane during flying direction；2) if there are barrier in pixel flight safety frame, unmanned plane carries out corresponding
Selective avoidance.
Step 3 selectivity avoidance mode
Based on, there are barrier, unmanned plane carries out 2 kinds of selective avoidance modes in pixel flight safety frame:
1. going slowly as shown in fig. 7, unmanned plane reduces speed now, the depth value of the nearest barrier of pixel flight safety frame is recorded
Deep, and compared with the threshold value set_d of setting: deep enters jogging area when being greater than set_d, unmanned plane adjusts pitch angle and roll
Angle, suitably reduction speed；Deep enters alert zone when being less than set_d, and unmanned plane slows down hovering immediately, and by digital transmission module with
Figure transmission module sends information, prompts artificial remote control；
2. converting binary picture for disparity map, pixel flight safety frame is traversed, passes through point of the pixel value in analysis block
Cloth speculates the position of barrier in frame.If barrier left side, unmanned plane in frame fly to the right relative to pixel flight safety frame
Row；If barrier flies in frame inner underside, unmanned plane relative to pixel flight safety frame upwards；If barrier flies in pixel
Safe frame proportion is greater than 80%, then unmanned plane flies 5 meters upwards, if 8 road ultrasonic waves do not detect barrier within 5 meters
Hinder object, by the airline operation originally planned, binocular camera rotates to consistent with unmanned plane during flying direction unmanned plane；If 5 meters
Except 8 road ultrasonic waves at least 1 road detect in threshold value lmaxInside there is barrier, unmanned plane slows down hovering immediately, while will be double
Lens camera rotates to the region where the barrier, and sends information by digital transmission module and figure transmission module, prompts artificial distant
Control.
Step 4 plans unmanned plane surrounding environment and eight road avoiding obstacles by supersonic wave
As shown in figure 8, setting [- π, π] for the horizontal plane angle range of unmanned plane, using head as benchmark line, clockwise
Rotation is positive, and rotation is negative counterclockwise；Eight road ultrasonic wave places 1 every 45 °, is divided into eight regions, each region
60 °, the center line in the 1st region and region is named as heading, by suitable by the region of symmetry axis of unmanned plane heading
Clockwise demarcate the 2nd, 3,4,5,6,7,8 region, then corresponding center line angle be 0 °, 45 °, 90 °, 135 °, 180 ° ,-
135 °, -90 °, -45 °；Ultrasonic measurement range: 60 ° of horizontal view angle, 60 ° of vertical angle of view, then adjacent two-way ultrasonic wave weighs
Folded region, the centric angle of each lap are 22.5 °, 67.5 °, 112.5 °, and 157.5 °, -157.5 °, -112..5 °, -
67.5 °, -22.5 °；
Further, whether there are obstacles and measures and obstacle by eight tunnel ultrasound examination surrounding environments for unmanned plane
The distance of object；If ultrasound examination, which goes out the region, needs avoidance, binocular camera is gone to the position of center line in the region, is led to
Cross the ambient image in front of binocular camera acquisition unmanned plane；The ambient image is two cameras of binocular camera same
One moment difference captured the first image and the second image；To obtain depth information value, and flight control modules are uploaded to,
Make corresponding avoidance measure；
Further, since the factors such as ambient influence, when the depth value fluctuation that binocular vision is read is larger, unmanned plane
Mainly take avoiding obstacles by supersonic wave.
The foregoing is only a preferred embodiment of the present invention, is not intended to restrict the invention, for the skill of this field
For art personnel, the invention may be variously modified and varied.All within the spirits and principles of the present invention, made any to repair
Change, equivalent replacement, improvement etc., should all be included in the protection scope of the present invention.
Claims (5)
1. unmanned plane selectivity obstacle avoidance system and method based on binocular vision and three axis holders, which is characterized in that the avoidance
System includes multiplex ultrasonic module, binocular range finder module, raspberry pie control panel, three axis clouds terrace system modules, UAV Attitude
Identification module, flight control modules and transmission module；Wherein multiplex ultrasonic module includes eight ultrasonic waves of unmanned plane surrounding
And a CD4051 chip, for detecting surrounding environment, whether there are obstacles, measures at a distance from barrier, and simple judgement is
No avoidance and use binocular camera；Ultrasonic measurement range: 60 ° of horizontal view angle, 60 ° of vertical angle of view；Eight ultrasonic waves are every
Before being placed on unmanned plane every 45 degree, front left, a left side, rear left, rear, rear right, right, eight orientation of front right；Three axis clouds terrace system modules are
It is made of three mutually perpendicular pitch axis, roll axis and yaw axis, wherein roll axis and pitch axis are for keeping binocular camera shooting
Head horizontal position, yaw axis realize 360 ° of binocular camera rotations；Binocular camera shooting head module is mounted on three immediately below unmanned plane
It on axis clouds terrace system, sends control panel to be connected with tree plum, for shooting surrounding environment, handles disparity map, and carry out to three-dimensional
It rebuilds, calculates depth；The binocular camera measurement range: 60 ° of horizontal view angle, 60 ° of vertical angle of view；
Further, the unmanned plane selectivity barrier-avoiding method based on binocular vision and three axis holders, comprising the following steps:
Step 1 opens eight road ultrasonic waves and all the way binocular vision, and determines the heading angle of unmanned plane；
Step 2 obtains depth information, and sends control panel to carry out judging whether avoidance by setting plum；
Step 3 selectivity avoidance mode；
Step 4 plans unmanned plane surrounding environment and eight road avoiding obstacles by supersonic wave.
2. the unmanned plane selectivity obstacle avoidance system and method according to claim 1 based on binocular vision and three axis holders,
It is characterized in that, the step 1 is specific as follows:
1. opening eight road ultrasonic waves and all the way binocular vision simultaneously；
2. determining the angle γ in unmanned plane during flying direction and heading；
The horizontal axis component F in horizontal plane is calculated by spin matrixaWith the longitudinal axis component F in horizontal planeb:
In formula, FcFor the longitudinal axis component in vertical plane, F is lift of the propeller to unmanned plane；β is roll angle, β ∈ (- 25 °,
25 °), angle beta is positive when the rolling to the right of unmanned plane body, to the left when β be negative；θ is pitch angle, θ ∈ (- 25 °, 25 °), unmanned plane
Body head it is downward when θ be positive, θ is negative when upward；
Fa=-Fsin β, Fb=-Fcos β sin θ (2)
The angle γ of unmanned plane during flying direction and heading is
Wherein, sgn (x) is sign function, as x > 0, sgn (x)=1；As x < 0, sgn (x)=- 1；On the basis of head
Line is rotated clockwise and is positive, and rotation is negative counterclockwise；The range of angle γ is [- π, π]；
3. UAV Attitude module obtains unmanned plane pitch angle, roll angle and yaw angle；Three axis holders compensation unmanned plane is bowed
The elevation angle and roll angle, make binocular camera be maintained at horizontal position；By the yaw axis in three axis clouds terrace systems by binocular camera shooting
Head rotates clockwise γ angle, so that the visual angle of binocular camera is consistent with unmanned plane during flying direction, is more advantageous to autonomous
Avoidance.
3. the unmanned plane selectivity obstacle avoidance system and method according to claim 1 based on binocular vision and three axis holders,
It is characterized in that, the step 2 is specific as follows: based on unmanned plane current pose information and the obstacle obtained according to image
Object depth angle value；The UAV Attitude information is unmanned plane pitch angle and roll acquired in the attitude transducer set on unmanned plane
Angle, attitude transducer are accelerometer, gyroscope and magnetometer；The depth value of the barrier is obtained by following steps:
1. it includes the first image and the second image that the binocular camera, which obtains the ambient image in front of unmanned plane,；
2. the binocular camera obtains the inside and outside parameter of camera, then the external ginseng for passing through camera by Zhang Shi standardization
Several and distortion factor corrects the first image and the second image；By after correction the first image and the second image use SGAM
Algorithm carries out Stereo matching, obtains the disparity map of two images；
3. obtaining depth image based on the disparity map, barrier depth value is extracted according to the depth image；
When binocular camera detects barrier, tree plum sends control panel according to obtained depth value, and passes through following several steps
It is rapid to judge it is to carry out avoidance, it is also to maintain original route and continues to fly:
1. wide by the length of unmanned plane, height calculates unmanned plane under the influence of pitching, roll, through the projection on heading
Practical rectangle frame length L1With width W1:
L1=(hsin | β |+lcos | β |) cos | γ |+(bcos | θ |+hcos | β | sin | θ |) sin | and γ | (4)
In formula, h is unmanned plane height, and l is unmanned plane length；B is unmanned plane width；
W1=bsin | θ |+lsin | β |+hcos | θ | cos | β | (5)
2. calculating the horizontal distance d of unmanned plane and nearest barrier1:
d1=z1 (6)
In formula, z1For the vertical pivot coordinate of nearest barrier；
3. calculating the length Δ U of pixel flight safety frame1With width Delta V1, concrete form are as follows:
In formula, D1Be depth value be d1When image center parallax, B is the distance between two cameras, and f takes the photograph for binocular
The focal length of camera；
In formula, Δ x1For L1In depth value d1When image coordinate system under correspondingly-sized；
In formula, Δ u1For L1In depth value d1When pixel coordinate system under corresponding size, dx be unit pixel horizontal axis object
Manage size；
In formula, Δ v1For W1In depth value d1When pixel coordinate system under corresponding size, dy be unit pixel the longitudinal axis object
Manage size；By Δ u1With Δ v1All expand as original r times, and r > 1；General r takes 1.2, can reduce extraneous factor influence and and
When handle emergency case, then have
ΔU1=r* Δ u1, Δ V1=r* Δ v1 (11)
4. calculating the vertical axis component s in the plummet plane at unmanned plane center and binocular camera centera:
5. calculating saCorrespondingly-sized s under pixel coordinate system1:
6. pixel flight safety frame is offset up s1, traverse pixel flight safety frame: if 1) being not present in pixel flight safety frame
Barrier, unmanned plane fly to pixel flight safety frame direction, after unmanned plane passes through pixel flight safety frame, binocular camera rotation
It goes to consistent with unmanned plane during flying direction；2) if there are barrier in pixel flight safety frame, unmanned plane carries out corresponding selectivity
Avoidance.
4. the unmanned plane selectivity obstacle avoidance system and method according to claim 1 based on binocular vision and three axis holders,
It is characterized in that, the step 3 is specific as follows: based on, there are barrier, unmanned plane carries out 2 kinds in pixel flight safety frame
Selective avoidance mode:
Go slowly 1. unmanned plane reduces speed now, record pixel flight safety frame nearest barrier depth value deep, and with setting
Threshold value set_d compare: deep enters jogging area when being greater than set_d, unmanned plane adjusts pitch angle and roll angle, appropriate to reduce speed
Degree；Deep enters alert zone when being less than set_d, and unmanned plane slows down hovering immediately, and passes through digital transmission module and the transmission of figure transmission module
Information prompts artificial remote control；
2. converting binary picture for disparity map, pixel flight safety frame is traversed, is pushed away by the distribution of the pixel value in analysis block
Survey the position of barrier in frame；If barrier left side, unmanned plane in frame fly to the right relative to pixel flight safety frame；If
Barrier is in frame inner underside, then unmanned plane flies upwards relative to pixel flight safety frame；If barrier is in pixel flight safety
Frame proportion is greater than 80%, then unmanned plane flies 5 meters upwards, if 8 road ultrasonic waves do not detect obstacle within 5 meters
Object, by the airline operation originally planned, binocular camera rotates to consistent with unmanned plane during flying direction unmanned plane；If 5 meters it
Ultrasonic wave at least 1 road in outer 8 road is detected in threshold value lmaxInside there is barrier, unmanned plane slows down hovering immediately, while by binocular
Region where camera rotation to the barrier, and information is sent by digital transmission module and figure transmission module, prompt artificial remote control.
5. the unmanned plane selectivity obstacle avoidance system and method according to claim 1 based on binocular vision and three axis holders,
It is characterized in that, the step 4 is specific as follows: setting [- π, π] for the horizontal plane angle range of unmanned plane, be with head
Reference line is rotated clockwise and is positive, and rotation is negative counterclockwise；Eight road ultrasonic wave places 1 every 45 °, is divided into eight areas
Domain, 60 ° of each region are named as the center line in the 1st region and region by the region of symmetry axis of unmanned plane heading as machine
Head direction, in the direction of the clock demarcate the 2nd, 3,4,5,6,7,8 region, then corresponding center line angle be 0 °, 45 °, 90 °,
135 °, 180 °, -135 °, -90 °, -45 °；Ultrasonic measurement range: 60 ° of horizontal view angle, 60 ° of vertical angle of view, then adjacent two-way is super
There is overlapping region in sound wave, and the centric angle of each lap is 22.5 °, and 67.5 °, 112.5 °, 157.5 °, -157.5 °, -
112..5 °, -67.5 °, -22.5 °；Whether there are obstacles and measurement by eight tunnel ultrasound examination surrounding environments for unmanned plane
At a distance from barrier；If ultrasound examination, which goes out the region, needs avoidance, binocular camera is gone to the center line in the region
Position obtains the ambient image in front of unmanned plane by binocular camera；The ambient image is that two of binocular camera take the photograph
As head first image and second image captured in synchronization difference；To obtain depth information value, and upload to flight
Control module makes corresponding avoidance measure；Further, since the factors such as ambient influence, the depth that binocular vision is read
When angle value fluctuation is larger, unmanned plane mainly takes avoiding obstacles by supersonic wave.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910776823.6A CN110456811A (en) | 2019-08-22 | 2019-08-22 | Unmanned plane selectivity obstacle avoidance system and method based on binocular vision and three axis holders |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910776823.6A CN110456811A (en) | 2019-08-22 | 2019-08-22 | Unmanned plane selectivity obstacle avoidance system and method based on binocular vision and three axis holders |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN110456811A true CN110456811A (en) | 2019-11-15 |
Family
ID=68488469
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201910776823.6A Withdrawn CN110456811A (en) | 2019-08-22 | 2019-08-22 | Unmanned plane selectivity obstacle avoidance system and method based on binocular vision and three axis holders |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN110456811A (en) |
Cited By (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112173103A (en) * | 2020-07-03 | 2021-01-05 | 中建交通建设集团有限公司 | Detection device and method for tunnel working face constructed by drilling and blasting method |
| CN112634317A (en) * | 2020-12-15 | 2021-04-09 | 深圳慧源创新科技有限公司 | Binocular omnidirectional obstacle avoidance method based on mechanical holder and optical flow algorithm |
| CN112801928A (en) * | 2021-03-16 | 2021-05-14 | 昆明理工大学 | Attention mechanism-based millimeter wave radar and visual sensor fusion method |
| WO2021217335A1 (en) * | 2020-04-27 | 2021-11-04 | 深圳市大疆创新科技有限公司 | Movable platform and control method and device therefor |
| CN113900446A (en) * | 2021-10-15 | 2022-01-07 | 贵州师范学院 | Unmanned aerial vehicle hydrological monitoring autonomous obstacle avoidance system and method based on binocular vision |
-
2019
- 2019-08-22 CN CN201910776823.6A patent/CN110456811A/en not_active Withdrawn
Cited By (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2021217335A1 (en) * | 2020-04-27 | 2021-11-04 | 深圳市大疆创新科技有限公司 | Movable platform and control method and device therefor |
| CN113795804A (en) * | 2020-04-27 | 2021-12-14 | 深圳市大疆创新科技有限公司 | Movable platform and control method and device thereof |
| CN112173103A (en) * | 2020-07-03 | 2021-01-05 | 中建交通建设集团有限公司 | Detection device and method for tunnel working face constructed by drilling and blasting method |
| CN112634317A (en) * | 2020-12-15 | 2021-04-09 | 深圳慧源创新科技有限公司 | Binocular omnidirectional obstacle avoidance method based on mechanical holder and optical flow algorithm |
| CN112801928A (en) * | 2021-03-16 | 2021-05-14 | 昆明理工大学 | Attention mechanism-based millimeter wave radar and visual sensor fusion method |
| CN113900446A (en) * | 2021-10-15 | 2022-01-07 | 贵州师范学院 | Unmanned aerial vehicle hydrological monitoring autonomous obstacle avoidance system and method based on binocular vision |
| CN113900446B (en) * | 2021-10-15 | 2022-07-12 | 贵州师范学院 | Unmanned aerial vehicle hydrological monitoring autonomous obstacle avoidance system and method based on binocular vision |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US20220124303A1 (en) | Methods and systems for selective sensor fusion | |
| CN110456811A (en) | Unmanned plane selectivity obstacle avoidance system and method based on binocular vision and three axis holders | |
| US11393350B2 (en) | Systems and methods for vehicle guidance using depth map generation | |
| US20220234733A1 (en) | Aerial Vehicle Smart Landing | |
| CN107223275B (en) | Method and system for fusing multi-channel sensing data | |
| CN109911188B (en) | Bridge detection unmanned aerial vehicle system in non-satellite navigation and positioning environment | |
| US9783320B2 (en) | Airplane collision avoidance | |
| US9367067B2 (en) | Digital tethering for tracking with autonomous aerial robot | |
| CN107703951B (en) | A kind of unmanned plane barrier-avoiding method and system based on binocular vision | |
| Strydom et al. | Visual odometry: autonomous uav navigation using optic flow and stereo | |
| US10878709B2 (en) | System, method, and computer readable medium for autonomous airport runway navigation | |
| CN108306217A (en) | A kind of overhead high-voltage wire intelligent independent is along conducting wire flight cruising inspection system and method | |
| CN106708073B (en) | A kind of quadrotor system of independent navigation power-line patrolling fault detection | |
| CN110427049A (en) | Multi-rotor unmanned aerial vehicle selectivity barrier-avoiding method based on binocular vision and ultrasonic wave | |
| US20220024549A1 (en) | System and method for measuring the distance to an object in water | |
| JP5990453B2 (en) | Autonomous mobile robot | |
| CN109782788A (en) | Unmanned plane low latitude obstacle avoidance system and control method based on binocular vision | |
| WO2017161563A1 (en) | Control method and apparatus for aircraft | |
| CN110427042A (en) | Unmanned plane low latitude barrier-avoiding method based on ultrasonic wave and binocular vision | |
| CN110442145A (en) | Multi-rotor unmanned aerial vehicle holder obstacle avoidance system and method based on binocular vision | |
| CN109857133A (en) | Multi-rotor unmanned aerial vehicle selectivity avoidance obstacle method based on binocular vision | |
| JP2016118995A (en) | Monitoring system | |
| KR20210051030A (en) | Apparatus and method for correcting bias in sensor | |
| EP4159661A1 (en) | Method of guiding the sheave especially of the overhead crane | |
| Del Pizzo et al. | Vessel attitude estimation by camera sensors |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| WW01 | Invention patent application withdrawn after publication |
Application publication date: 20191115 |
|
| WW01 | Invention patent application withdrawn after publication |