WO2019113966A1 - 一种避障方法、装置和无人机 - Google Patents

一种避障方法、装置和无人机 Download PDF

Info

Publication number
WO2019113966A1
WO2019113966A1 PCT/CN2017/116572 CN2017116572W WO2019113966A1 WO 2019113966 A1 WO2019113966 A1 WO 2019113966A1 CN 2017116572 W CN2017116572 W CN 2017116572W WO 2019113966 A1 WO2019113966 A1 WO 2019113966A1
Authority
WO
WIPO (PCT)
Prior art keywords
obstacle
drone
camera
image
focal lengths
Prior art date
Application number
PCT/CN2017/116572
Other languages
English (en)
French (fr)
Inventor
张柯
臧波
Original Assignee
深圳市道通智能航空技术有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 深圳市道通智能航空技术有限公司 filed Critical 深圳市道通智能航空技术有限公司
Priority to EP17832180.8A priority Critical patent/EP3591490B1/en
Priority to PCT/CN2017/116572 priority patent/WO2019113966A1/zh
Priority to CN201780002277.XA priority patent/CN108323190B/zh
Priority to US15/885,041 priority patent/US10860039B2/en
Publication of WO2019113966A1 publication Critical patent/WO2019113966A1/zh

Links

Images

Classifications

    • G—PHYSICS
    • G05—CONTROLLING; REGULATING
    • G05D—SYSTEMS FOR CONTROLLING OR REGULATING NON-ELECTRIC VARIABLES
    • G05D1/00—Control of position, course, altitude or attitude of land, water, air or space vehicles, e.g. using automatic pilots
    • G05D1/10—Simultaneous control of position or course in three dimensions
    • G05D1/101—Simultaneous control of position or course in three dimensions specially adapted for aircraft
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V20/00—Scenes; Scene-specific elements
    • G06V20/10—Terrestrial scenes
    • G06V20/13—Satellite images
    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B64—AIRCRAFT; AVIATION; COSMONAUTICS
    • B64C—AEROPLANES; HELICOPTERS
    • B64C39/00—Aircraft not otherwise provided for
    • B64C39/02—Aircraft not otherwise provided for characterised by special use
    • B64C39/024—Aircraft not otherwise provided for characterised by special use of the remote controlled vehicle type, i.e. RPV
    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B64—AIRCRAFT; AVIATION; COSMONAUTICS
    • B64U—UNMANNED AERIAL VEHICLES [UAV]; EQUIPMENT THEREFOR
    • B64U10/00—Type of UAV
    • B64U10/10—Rotorcrafts
    • B64U10/13—Flying platforms
    • B64U10/14—Flying platforms with four distinct rotor axes, e.g. quadcopters
    • G—PHYSICS
    • G05—CONTROLLING; REGULATING
    • G05D—SYSTEMS FOR CONTROLLING OR REGULATING NON-ELECTRIC VARIABLES
    • G05D1/00—Control of position, course, altitude or attitude of land, water, air or space vehicles, e.g. using automatic pilots
    • G05D1/10—Simultaneous control of position or course in three dimensions
    • G05D1/101—Simultaneous control of position or course in three dimensions specially adapted for aircraft
    • G05D1/102—Simultaneous control of position or course in three dimensions specially adapted for aircraft specially adapted for vertical take-off of aircraft
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/70—Determining position or orientation of objects or cameras
    • G06T7/73—Determining position or orientation of objects or cameras using feature-based methods
    • G06T7/74—Determining position or orientation of objects or cameras using feature-based methods involving reference images or patches
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V20/00—Scenes; Scene-specific elements
    • G06V20/10—Terrestrial scenes
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V20/00—Scenes; Scene-specific elements
    • G06V20/10—Terrestrial scenes
    • G06V20/17—Terrestrial scenes taken from planes or by drones
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N13/00—Stereoscopic video systems; Multi-view video systems; Details thereof
    • H04N13/20—Image signal generators
    • H04N13/204—Image signal generators using stereoscopic image cameras
    • H04N13/239—Image signal generators using stereoscopic image cameras using two 2D image sensors having a relative position equal to or related to the interocular distance
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N13/00—Stereoscopic video systems; Multi-view video systems; Details thereof
    • H04N13/20—Image signal generators
    • H04N13/296—Synchronisation thereof; Control thereof
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
    • H04N23/60—Control of cameras or camera modules
    • H04N23/69—Control of means for changing angle of the field of view, e.g. optical zoom objectives or electronic zooming
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
    • H04N23/80—Camera processing pipelines; Components thereof
    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B64—AIRCRAFT; AVIATION; COSMONAUTICS
    • B64U—UNMANNED AERIAL VEHICLES [UAV]; EQUIPMENT THEREFOR
    • B64U10/00—Type of UAV
    • B64U10/10—Rotorcrafts
    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B64—AIRCRAFT; AVIATION; COSMONAUTICS
    • B64U—UNMANNED AERIAL VEHICLES [UAV]; EQUIPMENT THEREFOR
    • B64U20/00—Constructional aspects of UAVs
    • B64U20/80—Arrangement of on-board electronics, e.g. avionics systems or wiring
    • B64U20/87—Mounting of imaging devices, e.g. mounting of gimbals
    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B64—AIRCRAFT; AVIATION; COSMONAUTICS
    • B64U—UNMANNED AERIAL VEHICLES [UAV]; EQUIPMENT THEREFOR
    • B64U2101/00—UAVs specially adapted for particular uses or applications
    • B64U2101/30—UAVs specially adapted for particular uses or applications for imaging, photography or videography
    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B64—AIRCRAFT; AVIATION; COSMONAUTICS
    • B64U—UNMANNED AERIAL VEHICLES [UAV]; EQUIPMENT THEREFOR
    • B64U2201/00—UAVs characterised by their flight controls
    • B64U2201/10—UAVs characterised by their flight controls autonomous, i.e. by navigating independently from ground or air stations, e.g. by using inertial navigation systems [INS]
    • G—PHYSICS
    • G01—MEASURING; TESTING
    • G01C—MEASURING DISTANCES, LEVELS OR BEARINGS; SURVEYING; NAVIGATION; GYROSCOPIC INSTRUMENTS; PHOTOGRAMMETRY OR VIDEOGRAMMETRY
    • G01C3/00—Measuring distances in line of sight; Optical rangefinders
    • G01C3/32—Measuring distances in line of sight; Optical rangefinders by focusing the object, e.g. on a ground glass screen
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/10—Image acquisition modality
    • G06T2207/10032—Satellite or aerial image; Remote sensing
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/30—Subject of image; Context of image processing
    • G06T2207/30248—Vehicle exterior or interior
    • G06T2207/30252—Vehicle exterior; Vicinity of vehicle
    • G06T2207/30261—Obstacle
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N13/00—Stereoscopic video systems; Multi-view video systems; Details thereof
    • H04N2013/0074—Stereoscopic image analysis
    • H04N2013/0081—Depth or disparity estimation from stereoscopic image signals

Definitions

  • the present application relates to the field of unmanned aerial vehicles, and in particular, to an obstacle avoidance method, device, and drone.
  • the obstacle detection can be roughly divided into: monocular stereo vision remote detection and binocular stereo vision close distance detection.
  • the detection range of binocular stereo vision close-range detection is determined by the baseline length of the left and right eye images, and the detection range is limited. Therefore, when the long-distance obstacle detection is performed, the monocular stereo vision principle is adopted, and the left eye (or right eye, hereinafter collectively referred to as monocular) in the front-view binocular camera is used to shoot different positions, and the scene images of different viewing angles are obtained, and then passed. Matching the pixel position difference of the same point in different perspective images to obtain the depth information of the point.
  • the above detection process requires accurate acquisition of the position and attitude information of the aircraft through the Global Positioning System (GPS) and the airborne inertial device.
  • GPS Global Positioning System
  • the accuracy of the calculation of the depth information is directly affected by the solution data of the above instruments, and the flight is often due to The existence of accumulated error of the airborne inertial device and the lack of positioning accuracy result in unreliable image solving results.
  • the purpose of the embodiments of the present application is to provide an obstacle avoidance method, device and drone, which can avoid measurement interference and positioning error caused by the onboard inertial component during the obstacle detection process, and the calculation precision is high.
  • an embodiment of the present application provides an obstacle avoidance method for an unmanned aerial vehicle, where the drone includes an imaging device, and the imaging device supports optical zooming, and the method includes:
  • Position information of the obstacle is acquired based on the detection image at at least two different focal lengths.
  • the location information includes a distance S between the obstacle and the drone, and a height difference H between the obstacle and the drone;
  • ⁇ is the angle between the flight direction of the drone and the horizontal direction
  • f1 is the distance of the image plane from the optical center when the camera is located at the first focal length
  • h1 is when the camera is located at the first focal length
  • f2 is the distance of the image plane from the optical center when the imaging device is at the second focal length
  • h2 is the obstacle at the second focal length of the imaging device.
  • the imaging height above the center of the image plane 0 ⁇ ⁇ ⁇ 90 degrees.
  • the method further includes:
  • the adjusting the speed direction of the drone according to the position information of the obstacle includes:
  • the method further includes:
  • the camera device is adjusted to face the detection area.
  • the adjusting the camera device to face the detection area includes:
  • an embodiment of the present application provides an obstacle avoidance device for an unmanned aerial vehicle, where the drone includes an imaging device, and the imaging device supports optical zoom, and the device includes:
  • Detecting an image acquisition module configured to acquire a detection image of the camera device at at least two different focal lengths
  • An obstacle confirmation module configured to determine that an obstacle exists in the detection area according to the detection image of the at least two different focal lengths
  • an obstacle position acquiring module configured to acquire position information of the obstacle according to the detection image at at least two different focal lengths.
  • the location information includes a distance S between the obstacle and the drone, and a height difference H between the obstacle and the drone;
  • the obstacle position acquisition module is specifically configured to:
  • ⁇ is the angle between the flight direction of the drone and the horizontal direction
  • f1 is the distance of the image plane from the optical center when the camera is located at the first focal length
  • h1 is when the camera is located at the first focal length
  • f2 is the The distance between the image plane and the optical center when the imaging device is located at the second focal length
  • h2 is the imaging height of the obstacle above the center of the image plane when the imaging device is located at the second focal length, 0 ⁇ 90 degrees.
  • the apparatus further includes:
  • a direction adjustment module configured to adjust a speed direction of the drone according to position information of the obstacle to avoid the obstacle.
  • the direction adjustment module is specifically configured to:
  • the apparatus further includes:
  • the camera adjustment module is configured to adjust the imaging device to face the detection area.
  • the camera adjustment module is specifically configured to:
  • an embodiment of the present application provides a drone, including a body, a arm connected to the body, a power device disposed on the arm, and an image capturing device for acquiring an image.
  • the camera device supports optical zoom, and the drone further includes:
  • the memory stores instructions executable by the processor, the instructions being executed by the processor to enable the processor to perform the method as described above.
  • the camera device is a main camera of the drone.
  • an embodiment of the present application provides a non-transitory computer readable storage medium.
  • the computer readable storage medium stores computer executable instructions that, when executed by a drone, cause the drone to perform the method described above.
  • an embodiment of the present application provides a computer program product, the computer program product comprising a computer program stored on a non-transitory computer readable storage medium, the computer program comprising program instructions, when the program When the command is executed by the drone, the drone is caused to perform the method as described above.
  • the embodiment of the present application acquires detection images of at least two different focal lengths by an image pickup device that supports optical zoom, and then confirms that there is an obstacle in the detection area according to the detection images at at least two different focal lengths, thereby obtaining the position of the obstacle according to the detected image. information. Due to the rapidity of the optical zoom operation of the camera, it is not necessary to accurately acquire the position and attitude information of the aircraft through the GPS and the onboard inertial device, thereby avoiding measurement interference and positioning error caused by the inertial device, and improving the calculation accuracy.
  • FIG. 1 is a schematic structural diagram of a drone provided by an embodiment of the present application.
  • FIG 2 is an application scenario diagram of an obstacle avoidance method and apparatus according to an embodiment of the present application
  • FIG. 3 is a flowchart of an embodiment of an obstacle avoidance method according to an embodiment of the present application.
  • FIG. 4 is a schematic diagram of an image taken by an image pickup apparatus according to an embodiment of the present application at two different focal lengths;
  • FIG. 5 is a view showing an imaging position of an object in an image at two different focal lengths in the embodiment of the present application
  • FIG. 6 is a schematic diagram of imaging of an obstacle under two different focal lengths when the speed direction is a horizontal direction according to an embodiment of the present application
  • FIG. 7 is a schematic diagram of a speed direction adjustment scheme of a drone according to an embodiment of the present application when the speed direction is a horizontal direction;
  • FIG. 8 is a schematic diagram of imaging of an obstacle at two different focal lengths when the speed direction is at an angle with the horizontal direction provided by the embodiment of the present application;
  • FIG. 9 is a schematic diagram of a speed direction adjustment scheme when the UAV has an angle between the speed direction and the horizontal direction in the embodiment of the present application;
  • FIG. 10 is a structural block diagram of an embodiment of an obstacle avoidance device according to an embodiment of the present application.
  • FIG. 11 is a schematic structural diagram of hardware of another embodiment of a drone according to an embodiment of the present application.
  • FIG. 1 is a schematic structural diagram of a drone 10 according to an embodiment of the present application.
  • the drone 10 includes a body 11, an arm 12 connected to the body 11, a power unit 13 disposed at one end of the arm 12, a platform 15 connected to the body 11, and an imaging device 14 connected to the platform 15. And a processor 16 and a memory 17 disposed in the body 11.
  • the number of the arms 12 is four, that is, the aircraft is a quadrotor. In other possible embodiments, the number of the arms 12 may also be 3, 6, 8, 10, and the like.
  • the drone 10 can also be other movable objects such as manned aircraft, model aircraft, unmanned airships, fixed-wing drones, and unmanned hot air balloons.
  • the power unit 13 includes a motor 132 disposed at one end of the arm 12 and a propeller 131 coupled to the rotating shaft of the motor 132.
  • the rotating shaft of the motor 132 rotates to drive the propeller 131 to rotate to provide lift to the drone 10.
  • the pan/tilt 15 serves to alleviate or even eliminate the vibration transmitted from the power unit 13 to the image pickup device 14 to ensure that the image pickup device 14 can capture a stable and clear image or video.
  • the imaging device 14 may be a high-definition camera or a motion camera or the like for performing image capturing. In an embodiment of the present application, the imaging device 14 supports autonomous optical zooming. Camera The device 14 may be directly mounted on the drone 10, or may be mounted on the drone 10 by the pan/tilt head 15 as shown in this embodiment, and the pan/tilt head 15 allows the camera device 14 to be wound around at least one axis with respect to the drone 10 Turn.
  • the processor 16 may include a plurality of functional units, such as a flight control unit for controlling the flight attitude of the aircraft, a target recognition unit for identifying the target, a tracking unit for tracking a specific target, a navigation unit for navigating the aircraft (for example, GPS (Global Positioning System), Beidou, and a data processing unit for processing environmental information acquired by a related airborne device (for example, the imaging device 14).
  • a flight control unit for controlling the flight attitude of the aircraft
  • a target recognition unit for identifying the target
  • a tracking unit for tracking a specific target
  • a navigation unit for navigating the aircraft (for example, GPS (Global Positioning System), Beidou
  • a data processing unit for processing environmental information acquired by a related airborne device (for example, the imaging device 14).
  • FIG. 2 is an application scenario diagram of an obstacle avoidance method and apparatus provided by an embodiment of the present application.
  • the application scenario includes the drone 10 and the obstacle 20 (only one obstacle is shown in FIG. 1, and there may be more obstacles or no obstacles in actual application).
  • the drone 10 needs to identify and avoid obstacles 20 in front of the flight during autonomous return or autonomous flight.
  • the drone 10 uses a method of visual recognition to determine whether there is an obstacle ahead of the flight, which images the image in front of the flight by the imaging device 14, and processes the image by the processor 16 to determine whether an obstacle exists.
  • the drone 10 adjusts the imaging device 14 to at least two different focal lengths by the processor 16, for example, respectively adjusting the imaging device 14 to the first focal length and the second focal length, and then respectively obtaining detection images at different focal lengths, and
  • the detected image is image-decomposed to determine that there is an obstacle in front of the flight.
  • the location information of the obstacle is then obtained to take evasive measures to avoid the obstacle.
  • FIG. 3 is a flowchart of an obstacle avoidance method according to an embodiment of the present disclosure. The method may be performed by the drone 10 in FIG. 1 or FIG. 2, as shown in FIG. 3, the method includes:
  • the drone 10 adjusts the imaging device 14 to face the detection area to capture an image of the detection area.
  • the drone 10 adjusts the line of sight of the imaging device 14 to coincide with the flight direction of the drone 10 such that the imaging device 14 faces the detection area, which is the area facing the flight front.
  • the drone 10 flies to a certain destination, it first adjusts itself to a certain height and maintains horizontal flight.
  • the drone 10 aligns the flight direction with the connection of the current position projection and the destination, and then fixes the position of the camera device 14.
  • the projection of the line of sight axis is parallel to the flight direction, that is, the current position projection is parallel to the line of the destination, the flight direction, and the line of sight projection.
  • S102 Acquire a detection image of the imaging device 14 at at least two different focal lengths.
  • the camera device 14 supports autonomous optical zoom.
  • the image capturing is mostly performed by digital zooming.
  • the image information of the scene is lost.
  • the image capturing device 14 supporting the autonomous optical zoom does not lose the image information of the scene.
  • the optical zoom capability of each camera device is different, and the larger the zoom range of the camera device, the larger the range of detection.
  • At least two preset focal lengths may be selected from the zoom range according to the obstacle avoidance distance of the drone 10, for example, a first preset focal length of 8 mm, a second preset focal length of 20 mm, or a first preset focal length of 10 mm, and a second preset.
  • the focal length is 28mm.
  • images of the detection area are captured at the first preset focal length and the second focal length, respectively, to obtain detection images of the first preset focal length and the second preset focal length.
  • the first preset focal length, the second preset focal length, and the third preset focal length may also be set in advance, and then the detected images at three different focal lengths are respectively obtained.
  • the detected image may be a complete image captured by the imaging device 14, or may be a partial image selected from the complete image. For example, a certain area from the longitudinal center of the image is selected from the complete image after the imaging device 14 is captured as a detected image. Since the objects in the left and right edges of the image taken by the image pickup device 14 do not cause an obstacle to the drone 10, and the detected image is used as a basis for later calculation, the smaller the range, the faster the calculation speed will be. Selecting a partial image as the detected image will increase the calculation speed.
  • S103 Determine that an obstacle exists in the detection area according to the detection images at the at least two different focal lengths.
  • the disparity map is obtained for the detection images of any two different focal lengths, and the pixel position of the corresponding point in the two detected images can be found by the Dense Stereo Matching method, and then the difference between the pixel positions of the two images is calculated. You can get a disparity map.
  • the dense stereo matching method may adopt any one of Sum of absolute differences (SAD), Sum of Squared Differences (SSD) and Census algorithm.
  • the center of the image does not move.
  • the shooting angle of view decreases.
  • the same object has different positions in the two images before and after the zoom.
  • the image planes are at two different positions (two in Fig. 5).
  • the image position difference of the horizontal lines representing the image planes at different positions is gradually reduced in order from left to right.
  • the pixel position difference suddenly becomes large (S2>S1), and the regularity is exactly the opposite.
  • the farther background when the distance is farther, the farther background can be regarded as some points of the same depth, which will exhibit a smaller parallax in the disparity map.
  • the obstacle When an obstacle is present, the obstacle has a depth difference from the farther background, which will exhibit a larger parallax in the disparity map. It is thus possible to determine whether or not there is an obstacle by recognizing whether or not there is a parallax mutation in the parallax map.
  • S104 Acquire location information of the obstacle according to the detection image at at least two different focal lengths.
  • the location information includes a distance S between the obstacle and the drone 10, and a height difference H between the obstacle and the drone 10.
  • the distance S between the obstacle and the drone and the height difference H between the obstacle and the drone can be obtained by the following method:
  • FIG. 6 is a schematic diagram of imaging of obstacles at two different focal lengths, wherein the speed direction of the drone 10 is horizontal, the rectangle on the left represents the image plane, and the rectangle on the right represents the obstacle, horizontal
  • the dotted line is the centerline of the center of the image plane.
  • the distance S is:
  • the height difference is:
  • f1 is the distance of the image plane from the optical center of the imaging device at the first focal length
  • h1 is the imaging height of the obstacle above the center of the image plane
  • f2 is the image plane distance from the optical center of the imaging device at the second focal length.
  • the distance h2 is the imaging height of the obstacle above the centerline of the image plane.
  • the embodiment of the present application sets the imaging device to at least two different focal lengths, and acquires detection images at different focal lengths, and then confirms whether there are obstacles in the detection area according to the detection images under different focal lengths, and obtains the position of the obstacle according to the detected image. information. Due to the rapid zooming operation of the camera, it is not necessary to accurately acquire the position and attitude information of the aircraft through the GPS and the onboard inertial device, thereby avoiding measurement interference and positioning error caused by the inertial device, and improving the calculation accuracy.
  • the embodiment of the present application first detects whether there is an obstacle, and then obtains the position information of the obstacle, and the detection efficiency is high.
  • the drone 10 After obtaining the position information of the obstacle, the drone 10 can directly rise at a certain height to fly at a height higher than the height difference H or horizontally move a certain distance to avoid the obstacle.
  • S105 Adjust a speed direction of the drone 10 according to position information of the obstacle to avoid the obstacle.
  • the speed direction of the drone 10 can be adjusted upward.
  • the height of the drone 10 is higher than the height of the obstacle.
  • H' ⁇ x H
  • ⁇ represents a safety factor.
  • ⁇ 1.1 to 1.3, and the value of ⁇ can be set as needed.
  • the concept of the safety factor is proposed in the present application to ensure that the drone 10 avoids obstacles and ensures that the drone 10 can avoid obstacles when adopting the high-flying method.
  • the drone 10 In the case where the drone 10 is far away from the destination, it is necessary to continuously perform obstacle detection, that is, in some embodiments of the obstacle method, it is necessary to repeatedly perform the operation in FIG. 3 every time interval or interval. step. If the drone 10 performs an obstacle avoidance operation such as the above-described speed direction adjustment, its speed direction will be at an angle ⁇ with the horizontal direction. Or when the drone 10 performs a special task, its speed direction may also be at an angle to the horizontal direction.
  • FIG. 8 shows that the flight direction V' of the drone 10 is at an angle ⁇ with the horizontal direction.
  • the triangle indicated by the bold black solid line is similar to the triangular OCD. Relationship, the sides of the two have the following proportional relationship:
  • the adjustment scheme for the speed direction can refer to FIG. 9.
  • the direction declination A is
  • the embodiment of the present application further provides an obstacle avoidance device, which is disposed inside the drone 10 shown in FIG. 1.
  • the obstacle avoidance device 300 includes:
  • a detection image acquisition module 301 configured to acquire a detection image of the imaging device at at least two different focal lengths
  • An obstacle confirmation module 302 configured to determine that an obstacle exists in the detection area according to the detection image of the at least two different focal lengths
  • the obstacle position obtaining module 303 is configured to acquire position information of the obstacle according to the detection image at at least two different focal lengths.
  • the embodiment of the present application sets the imaging device to at least two different focal lengths, and acquires detection images at different focal lengths, and then confirms whether there are obstacles in the detection area according to the detection images under different focal lengths, and obtains the position of the obstacle according to the detected image. information. Due to the rapid zooming operation of the camera, it is not necessary to accurately acquire the position and attitude information of the aircraft through the GPS and the onboard inertial device, thereby avoiding measurement interference and positioning error caused by the inertial device, and improving the calculation accuracy.
  • the location information includes a distance S between the obstacle and the drone, and a height difference H between the obstacle and the drone;
  • the obstacle position obtaining module 303 is specifically configured to:
  • ⁇ is the angle between the flight direction of the drone and the horizontal direction
  • f1 is the distance of the image plane from the optical center when the camera is located at the first focal length
  • h1 is when the camera is located at the first focal length
  • f2 is the distance of the image plane from the optical center when the imaging device is at the second focal length
  • h2 is the obstacle at the second focal length of the imaging device.
  • the imaging height above the center of the image plane 0 ⁇ ⁇ ⁇ 90 degrees.
  • the obstacle avoidance device 300 further includes:
  • the direction adjustment module 304 is configured to adjust a speed direction of the drone according to position information of the obstacle to avoid the obstacle.
  • the direction adjustment module 304 is specifically configured to:
  • the obstacle avoidance device 300 further includes:
  • the camera adjustment module 305 is configured to adjust the imaging device to face the detection area.
  • the camera adjustment module 305 is specifically configured to:
  • the obstacle avoidance device 300 further includes:
  • the repeating module 306 is configured to repeatedly invoke the focal length setting module, the parallax map obtaining module, the obstacle confirming module, and the obstacle position acquiring module for each preset time or preset distance.
  • the drone 20 includes a body 24, and an arm 25 connected to the body 24.
  • the controller 23 includes at least one processor 21 and a memory 22 built in or externally placed in the drone 20 (in the case of the memory 22 built into the drone 20 in Fig. 11 as an example).
  • the processor 21 and the memory 22 can be connected by a bus or other means.
  • the memory 22 is used as a non-volatile computer readable storage medium, and can be used for storing non-volatile software programs, non-volatile computer-executable programs, and modules, such as program instructions corresponding to the obstacle avoidance method in the embodiment of the present application. / unit (for example, the detected image acquisition module 301, the obstacle confirmation module 302, and the obstacle position acquisition module 303 shown in FIG. 10).
  • the processor 21 executes various functional applications and data processing of the drone 20 by executing non-volatile software programs, instructions, and units stored in the memory 22, that is, implementing the obstacle avoidance method of the above-described method embodiments.
  • the memory 22 may include a storage program area and an storage data area, wherein the storage program area may store an operating system, an application required for at least one function; the storage data area may store data created according to usage of the user terminal device, and the like. Further, the memory 22 may include a high speed random access memory, and may also include a nonvolatile memory such as at least one magnetic disk storage device, flash memory device, or other nonvolatile solid state storage device. In some embodiments, memory 22 may optionally include memory remotely located relative to processor 21, which may be connected to drone 20 via a network.
  • the one or more modules are stored in the memory 22, and when executed by the one or more processors 21, perform an obstacle avoidance method in any of the above method embodiments, for example, performing the above described FIG. Method step S101 to step S105, implementing FIG. 10
  • the above-mentioned UAV 20 can perform the obstacle avoidance method provided by the embodiment of the present application, and has the corresponding functional modules and beneficial effects of the execution method.
  • the obstacle avoidance method provided by the embodiment of the present application.
  • the camera device 26 employs a main camera of the drone 20.
  • the main camera can shoot three-channel images. When stereo matching is required, all three channels must be aligned to determine the same pixel position. Compared with the current single-channel stereo matching method using gray information, the matching accuracy is higher. .
  • Embodiments of the present application provide a non-transitory computer readable storage medium storing computer-executable instructions that are executed by one or more processors, for example, to perform the above
  • the described method steps S101 to S105 in FIG. 3 implement the functions of the modules 301-306 in FIG.
  • the device embodiments described above are merely illustrative, wherein the units described as separate components may or may not be physically separate, and the components displayed as units may or may not be physical units, ie may be located A place, or it can be distributed to multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of the embodiment.
  • the embodiments can be implemented by means of software plus a general hardware platform, and of course, by hardware.
  • a person skilled in the art can understand that all or part of the process of implementing the above embodiments can be completed by a computer program to instruct related hardware, and the program can be stored in a computer readable storage medium. When executed, the flow of an embodiment of the methods as described above may be included.
  • the storage medium may be a magnetic disk, an optical disk, a read-only memory (ROM), or a random access memory (RAM).

Landscapes

  • Engineering & Computer Science (AREA)
  • Multimedia (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Remote Sensing (AREA)
  • Signal Processing (AREA)
  • Theoretical Computer Science (AREA)
  • Aviation & Aerospace Engineering (AREA)
  • Automation & Control Theory (AREA)
  • Radar, Positioning & Navigation (AREA)
  • Mechanical Engineering (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Astronomy & Astrophysics (AREA)
  • Control Of Position, Course, Altitude, Or Attitude Of Moving Bodies (AREA)
  • Studio Devices (AREA)

Abstract

本申请实施例公开了一种避障方法、装置和无人机。所述方法包括：获取所述摄像装置在至少两个不同焦距下的检测图像；根据所述至少两个不同焦距下的检测图像确定检测区域中存在障碍物；根据至少两个不同焦距下的所述检测图像获取所述障碍物的位置信息。本申请实施例通过设置摄像装置至至少两个不同焦距，并获取不同焦距下的检测图像，然后根据不同焦距下的检测图像确认检测区域中是否存在障碍物，以及根据检测图像获得障碍物的位置信息。由于摄像装置变焦操作的快速性，不需要通过GPS和机载惯性器件精准获取飞机的位置和姿态信息，避免了惯性器件造成的测量干扰和定位误差，提高了计算精度。

Description

一种避障方法、装置和无人机 技术领域
本申请涉及无人飞行器技术领域，特别涉及一种避障方法、装置和无人机。
背景技术
在无人机自主返航或者自主飞行过程中，需采用视觉信息进行障碍物检测，障碍物检测大致可分为：单目立体视觉远距离检测和双目立体视觉近距离检测。双目立体视觉近距离检测的检测范围由左、右目图像基线长短决定，检测范围受限制。因此进行远距离障碍物检测时多采用单目立体视觉原理，依据前视双目摄像头中的左目(或者右目，以下统称为单目)进行不同位置的拍摄，获得不同视角的场景图像，进而通过匹配同一点在不同视角图像中的像素位置差获取该点的深度信息。
然而，上述检测过程需要通过全球定位系统(Global Positioning System，GPS)和机载惯性器件精准获取飞机的位置和姿态信息，深度信息的计算精度直接受上述仪器的解算数据影响，飞行中往往由于机载惯性器件累积误差的存在和定位精度的不足导致图像解算结果不可靠。
发明内容
本申请实施例的目的是提供一种避障方法、装置和无人机，可以避免障碍物检测过程中机载惯性器件造成的测量干扰和定位误差，计算精度高。
第一方面，本申请实施例提供了一种避障方法，用于无人机，所述无人机包括摄像装置，所述摄像装置支持光学变焦，所述方法包括：
获取所述摄像装置在至少两个不同焦距下的检测图像；
根据所述至少两个不同焦距下的检测图像确定检测区域中存在障碍物；
根据至少两个不同焦距下的所述检测图像获取所述障碍物的位置信息。
在本申请的一实施例中，所述位置信息包括所述障碍物与所述无人机的距离S，以及所述障碍物与所述无人机的高度差H；
所述根据至少两个不同焦距下的所述检测图像获取所述障碍物的位置信息，包括：
计算所述障碍物与所述无人机的距离S以及所述障碍物与所述无人机的高度差H：
Figure PCTCN2017116572-appb-000001
Figure PCTCN2017116572-appb-000002
其中，θ为所述无人机的飞行方向与水平方向的夹角，f1为所述摄像装置位于第一焦距时像平面距离光心的距离，h1为所述摄像装置位于第一焦距时所述障碍物在所述像平面中心以上的成像高度，f2为所述摄像装置位于第二焦距时像平面距离光心的距离，h2为所述摄像装置位于第二焦距时所述障碍物在所述像平面中心以上的成像高度，0≤θ＜90度。
在本申请的一实施例中，所述方法还包括：
根据所述障碍物的位置信息调整所述无人机的速度方向，以躲避所述障碍物。
在本申请的一实施例中，所述根据所述障碍物的位置信息调整所述无人机的速度方向，包括：
将所述无人机的速度方向向上调整方向偏角A，其中，所述方向偏角A为：
Figure PCTCN2017116572-appb-000003
其中，λ为安全系数。
在本申请的一实施例中，所述方法还包括：
调整所述摄像装置面向检测区域。
在本申请的一实施例中，所述调整所述摄像装置面向检测区域，包括：
调整所述摄像装置的视线轴与所述无人机的飞行方向保持一致。
第二方面，本申请实施例提供了一种避障装置，用于无人机，所述无人机包括摄像装置，所述摄像装置支持光学变焦，所述装置包括：
检测图像获取模块，用于获取所述摄像装置在至少两个不同焦距下的检测图像；
障碍物确认模块，用于根据所述至少两个不同焦距下的检测图像确定检测区域中存在障碍物；
障碍物位置获取模块，用于根据至少两个不同焦距下的所述检测图像获取所述障碍物的位置信息。
在本申请的一实施例中，所述位置信息包括所述障碍物与所述无人机的距离S，以及所述障碍物与所述无人机的高度差H；
所述障碍物位置获取模块，具体用于：
计算所述障碍物与所述无人机的距离S以及所述障碍物与所述无人机的高度差H：
Figure PCTCN2017116572-appb-000004
Figure PCTCN2017116572-appb-000005
其中，θ为所述无人机的飞行方向与水平方向的夹角，f1为所述摄像装置位于第一焦距时像平面距离光心的距离，h1为所述摄像装置位于第一焦距时所述障碍物在所述像平面中心以上的成像高度，f2为所 述摄像装置位于第二焦距时像平面距离光心的距离，h2为所述摄像装置位于第二焦距时所述障碍物在所述像平面中心以上的成像高度，0≤θ＜90度。
在本申请的一实施例中，所述装置还包括：
方向调整模块，用于根据所述障碍物的位置信息调整所述无人机的速度方向，以躲避所述障碍物。
在本申请的一实施例中，所述方向调整模块具体用于：
将所述无人机的速度方向向上调整方向偏角A，其中，所述方向偏角A为：
Figure PCTCN2017116572-appb-000006
其中，λ为安全系数。
在本申请的一实施例中，所述装置还包括：
摄像调整模块，用于调整所述摄像装置面向检测区域。
在本申请的一实施例中，所述摄像调整模块具体用于：
调整所述摄像装置的视线轴与所述无人机的飞行方向保持一致。
第三方面，本申请实施例提供了一种无人机，包括机身、与所述机身相连的机臂、设于所述机臂的动力装置以及用于获取图像的摄像装置，所述摄像装置支持光学变焦，所述无人机还包括：
处理器；以及，
与所述处理器通信连接的存储器；其中，
所述存储器存储有可被所述处理器执行的指令，所述指令被所述处理器执行，以使所述处理器能够执行如上所述的方法。
在本申请的一实施例中，所述摄像装置为所述无人机的主摄像头。
第四方面，本申请实施例提供了一种非易失性计算机可读存储介质， 所述计算机可读存储介质存储有计算机可执行指令，当所述计算机可执行指令被无人机执行时，使所述无人机执行上述的方法。
第五方面，本申请实施例提供了一种计算机程序产品，所述计算机程序产品包括存储在非易失性计算机可读存储介质上的计算机程序，所述计算机程序包括程序指令，当所述程序指令被无人机执行时，使所述无人机执行如上所述的方法。
本申请实施例通过支持光学变焦的摄像装置获取至少两个不同焦距下的检测图像，然后根据至少两个不同焦距下的检测图像确认检测区域中存在障碍物，从而根据检测图像获得障碍物的位置信息。由于摄像装置光学变焦操作的快速性，不需要通过GPS和机载惯性器件精准获取飞机的位置和姿态信息，避免了惯性器件造成的测量干扰和定位误差，提高了计算精度。
附图说明
一个或多个实施例通过与之对应的附图中的图片进行示例性说明，这些示例性说明并不构成对实施例的限定，附图中具有相同参考数字标号的元件表示为类似的元件，除非有特别申明，附图中的图不构成比例限制。
图1是本申请实施例提供的一种无人机的结构示意图；
图2是本申请实施例提供的一种避障方法和装置的应用场景图；
图3是本申请实施例提供的一种避障方法的其中一实施例的流程图；
图4是本申请实施例提供的摄像装置在两个不同焦距下拍摄的图像的示意图；
图5是本申请实施例中物体在两个不同焦距下的图像中的成像位置图；
图6是本申请实施例提供的无人机在速度方向为水平方向时障碍物在两个不同焦距下的成像示意图；
图7是本申请实施例提供的无人机在速度方向为水平方向时速度方向调整方案示意图；
图8是本申请实施例提供的无人机在速度方向与水平方向有夹角时障碍物在两个不同焦距下的成像示意图；
图9是本申请实施例中无人机在速度方向与水平方向有夹角时速度方向调整方案示意图；
图10是本申请实施例提供的一种避障装置的其中一实施例的结构框图；
图11是本申请实施例提供的一种无人机另一实施例的硬件结构示意图。
具体实施方式
为使本申请实施例的目的、技术方案和优点更加清楚，下面将结合本申请实施例中的附图，对本申请实施例中的技术方案进行清楚、完整地描述，显然，所描述的实施例是本申请一部分实施例，而不是全部的实施例。基于本申请中的实施例，本领域普通技术人员在没有作出创造性劳动前提下所获得的所有其他实施例，都属于本申请保护的范围。
图1是本申请实施例提供的一种无人机10的结构示意图。该无人机10包括机身11、与机身11相连的机臂12、设置在机臂12一端的动力装置13、与机身11相连的云台15、与云台15相连的摄像装置14以及设置在机身11内的处理器16和存储器17。
在本实施例中，机臂12的数量为4，即该飞行器为四旋翼飞行器，在其他可能的实施例中，机臂12的数量也可以为3、6、8、10等。无人机10还可以是其他可移动物体，例如载人飞行器、航模、无人飞艇、固定翼无人机和无人热气球等。
动力装置13包括设置在机臂12一端的电机132以及与电机132的转轴相连的螺旋桨131。电机132的转轴转动以带动螺旋桨131旋转从而给无人机10提供升力。
云台15用于减轻甚至消除动力装置13传递给摄像装置14的振动，以保证摄像装置14能够拍摄出稳定清晰的图像或视频。
摄像装置14可以是高清摄像头或者运动相机等，用于完成图像的拍摄，在本申请的一实施例中，摄像装置14支持自主光学变焦。摄像 装置14可以直接搭载在无人机10上，也可以通过如本实施例所示的云台15搭载在无人机10上，云台15允许摄像装置14相对于无人机10绕至少一个轴转动。
处理器16可以包括多个功能性单元，如，用于控制飞行器飞行姿态的飞行控制单元、用于识别目标的目标识别单元、用于跟踪特定目标的跟踪单元、用于导航飞行器的导航单元(例如GPS(Global Positioning System)、北斗)、以及用于处理相关机载设备(如，摄像装置14)所获取的环境信息的数据处理单元等。
图2是本申请实施例提供的避障方法和装置的应用场景图。所述应用场景包括无人机10和障碍物20(图1中只示出了一个障碍物，实际应用中可能会有更多障碍物或者没有障碍物)。
无人机10在自主返航或者自主飞行过程中，需要自己识别并躲避飞行前方的障碍物20。无人机10采用视觉识别的方法来判断飞行前方是否有障碍物，其通过摄像装置14拍摄飞行前方的图像，并通过处理器16对图像进行处理来判断是否有障碍物存在。具体的，无人机10通过处理器16调整摄像装置14至至少两个不同焦距，例如分别调整摄像装置14至第一焦距和第二焦距，然后分别获得在不同焦距下的检测图像，并对该检测图像进行图像解算确定飞行前方存在障碍物。然后获得所述障碍物的位置信息，以采取避让措施躲避所述障碍物。
图3为本申请实施例提供的一种避障方法的流程图，所述方法可以由图1或图2中的无人机10执行，如图3所示，所述方法包括：
S101、调整摄像装14置面向检测区域。
无人机10调整摄像装置14面向检测区域，以拍摄检测区域的图像。换言之，无人机10调整所述摄像装置14的视线轴与所述无人机10的飞行方向保持一致，以使所述摄像装置14面向检测区域，检测区域即飞行前方面向的区域。在实际应用中，当无人机10向某一目的地飞行时，会首先调整自身至一定高度，保持水平方向飞行。无人机10将飞行方向对齐当前位置投影与目的地的连线，然后固定摄像装置14的位 置，使其视线轴投影与飞行方向平行，即当前位置投影与目的地的连线、飞行方向、视线轴投影三线平行。
S102：获取所述摄像装置14在至少两个不同焦距下的检测图像。
在本申请的实施例中，摄像装置14支持自主光学变焦。而当前航拍摄像头拍摄多采用数码变焦的方式，获取局部放大图像时，会损失场景的图像信息，而本申请实施例采用支持自主光学变焦的摄像装置14则不会损失场景的图像信息。通过应用其成像的高质量和自动变焦操作的快速性，可以很大程度降低由机载惯性器件造成的测量干扰，从而可提高计算精度。
每个摄像装置的光学变焦能力不同，摄像装置的变焦范围越大，其检测的范围也越大。可以预先根据无人机10的避障距离从变焦范围中选择至少两个预设焦距，例如第一预设焦距8mm、第二预设焦距20mm，或者第一预设焦距10mm、第二预设焦距28mm。然后分别在第一预设焦距和第二焦距下拍摄检测区域的图像，获得第一预设焦距和第二预设焦距下的检测图像。也可以预先设定第一预设焦距、第二预设焦距和第三预设焦距，然后分别获得在三个不同焦距下的检测图像。
其中，检测图像可以是摄像装置14拍摄的完整图像，也可以是从完整图像中选择的部分图像，例如从摄像装置14拍摄之后的完整图像中选择图像纵向中心左右一定区域作为检测图像。因为摄像装置14拍摄的图像左右边缘内的物体不会造成对无人机10的阻碍，而检测图像作为后面计算的基础，其范围越小计算的速度将越快。选择部分图像作为检测图像，将提高计算速度。
S103:根据所述至少两个不同焦距下的检测图像确定检测区域中存在障碍物。
具体的，根据至少两个不同焦距下的检测图像确定检测区域中存在障碍物，可以先针对不同预设焦距下的检测图像，进行图像解算获得所述检测图像的视差图，然后对所述视差图进行识别以确定所述检测区域中是否存在障碍物。如果步骤S102中获取两幅检测图像，可以获得该两幅检测图像的视差图，如果步骤S102中获得了三幅检测图像，则可 以获得三幅检测图像中的每两幅检测图像的视差图。其中，对任意两个不同焦距下的检测图像求视差图，可以通过稠密立体匹配(Dense Stereo Matching)方法，找到两幅检测图像中对应空间同一点的像素位置，再计算两者像素位置的差便可获取视差图。其中，稠密立体匹配方法可以采用绝对误差和算法(Sum of absolute differences，SAD)、误差平方和算法(Sum of Squared Differences，SSD)和Census算法的任意一种。
本申请实施例中利用视差图来判断是否存在障碍物的原理请参照图4和图5。
如图4所示，图像中心不移动，当焦距增大时，拍摄视角缩小，如果图像分辨率保持不变，则相同物体在变焦前后的两幅图像中位置不同。如图5左图所示，对于连续的具有同样深度(深度可以理解为障碍物距离摄像装置的距离)的物体1、2、3、4，其在两个不同位置像平面(图5中两条横线代表不同位置的像平面)的成像位置差按照从左到右的顺序是逐渐变小的。但是如图5右图所示，同一横向位置4处物体存在深度差时，其像素位置差则会突然变大(S2>S1)，其规律正好截然相反。
利用这样的规律，在距离较远时，其较远的背景可以视为一些深度相同的点，其在视差图中将呈现出较小的视差。当存在障碍物时，该障碍物与较远背景具有深度差，其在视差图中将呈现较大的视差。从而可以通过识别视差图中是否存在视差突变来判断是否存在障碍物。
S104:根据至少两个不同焦距下的所述检测图像获取所述障碍物的位置信息。
在本申请的一实施例中，所述位置信息包括所述障碍物与所述无人机10的距离S，以及所述障碍物与所述无人机10的高度差H。所述障碍物与所述无人机的距离S以及所述障碍物与所述无人机的高度差H可以通过以下方法获得：
请参照图6，图6为障碍物在两个不同焦距下的成像示意图，其中，无人机10的速度方向为水平方向，左侧的长方形代表像平面，右侧的长方形表示障碍物，横向虚线为过像平面中心的中心线。由图6中的几何关系有：
Figure PCTCN2017116572-appb-000007
和
Figure PCTCN2017116572-appb-000008
通过求解式(1)和式(2)可解：
所述距离S为：
Figure PCTCN2017116572-appb-000009
所述高度差为：
Figure PCTCN2017116572-appb-000010
其中，f1为所述摄像装置在第一焦距时像平面距光心的距离，h1为障碍物在像平面中心以上的成像高度，f2为所述摄像装置在第二焦距时像平面距光心的距离，h2为障碍物在像平面中心线以上的成像高度。
本申请实施例通过设置摄像装置至至少两个不同焦距，并获取不同焦距下的检测图像，然后根据不同焦距下的检测图像确认检测区域中是否存在障碍物，以及根据检测图像获得障碍物的位置信息。由于摄像装置变焦操作的快速性，不需要通过GPS和机载惯性器件精准获取飞机的位置和姿态信息，避免了惯性器件造成的测量干扰和定位误差，提高了计算精度。
此外，本申请实施例先检测是否存在障碍物，然后再获得障碍物的位置信息，检测效率高。
获得障碍物的位置信息后，无人机10可以直接上升一定的高度以高于高度差H的高度飞行或者水平移动一定的距离以躲避该障碍物。
S105：根据所述障碍物的位置信息调整所述无人机10的速度方向，以躲避所述障碍物。
在本申请的一实施例中，可以将无人机10的速度方向向上调整方 向偏角A，以使其到达障碍物所在的位置时，无人机10的高度高于障碍物的高度。如图7所示，如果无人机的速度方向由V调整到V’，其到达障碍物所在的位置时高度H’大于障碍物的高度H。其中，H’＝λ×H,λ表示安全系数，在本申请的一实施例中，λ＝1.1～1.3，λ的值可以根据需要设定。本申请提出安全系数的概念是为了保证无人机10避障在采用绕高飞行方法时，保证无人机10一定能够避开障碍物。
由图7中的几何关系可知，方向偏角
Figure PCTCN2017116572-appb-000011
代入式(3)和式(4)可解
Figure PCTCN2017116572-appb-000012
在无人机10距离目的地距离较远的场合，需要不断的进行障碍检测，即在所述障碍方法的某些实施例中，需要每间隔一段时间或者间隔一段距离即重复执行图3中的步骤。如果无人机10执行了一次如上速度方向调整的障碍躲避操作之后，其速度方向将和水平方向呈一夹角θ。或者无人机10在执行特殊任务时，其速度方向也可能和水平方向呈一夹角。
请参照图8,图8示出了无人机10的飞行方向V’与水平方向呈夹角θ，由图8中的几何关系可知，加粗的黑实线标示的三角形与三角形OCD具有相似关系，两者的各边具有如下比例关系：
Figure PCTCN2017116572-appb-000013
Figure PCTCN2017116572-appb-000014
对式(5)和式(6)求解可得：
Figure PCTCN2017116572-appb-000015
Figure PCTCN2017116572-appb-000016
对应的，当无人机10的速度方向与水平方向的夹角为θ时，对速度方向的调整方案可以参照图9，由图9的几何关系可知方向偏角A为
Figure PCTCN2017116572-appb-000017
代入式(7)和式(8)可解：
Figure PCTCN2017116572-appb-000018
本申请实施例还提供了一种避障装置，设置于图1所示的无人机10内部，如图10所示，避障装置300包括：
检测图像获取模块301，用于获取所述摄像装置在至少两个不同焦距下的检测图像；
障碍物确认模块302，用于根据所述至少两个不同焦距下的检测图像确定检测区域中存在障碍物；
障碍物位置获取模块303，用于根据至少两个不同焦距下的所述检测图像获取所述障碍物的位置信息。
本申请实施例通过设置摄像装置至至少两个不同焦距，并获取不同焦距下的检测图像，然后根据不同焦距下的检测图像确认检测区域中是否存在障碍物，以及根据检测图像获得障碍物的位置信息。由于摄像装置变焦操作的快速性，不需要通过GPS和机载惯性器件精准获取飞机的位置和姿态信息，避免了惯性器件造成的测量干扰和定位误差，提高了计算精度。
在本申请的一实施例中，所述位置信息包括所述障碍物与所述无人机的距离S，以及所述障碍物与所述无人机的高度差H；
障碍物位置获取模块303具体用于：
计算所述障碍物与所述无人机的距离S以及所述障碍物与所述无人机的高度差H：
Figure PCTCN2017116572-appb-000019
Figure PCTCN2017116572-appb-000020
其中，θ为所述无人机的飞行方向与水平方向的夹角，f1为所述摄像装置位于第一焦距时像平面距离光心的距离，h1为所述摄像装置位于第一焦距时所述障碍物在所述像平面中心以上的成像高度，f2为所述摄像装置位于第二焦距时像平面距离光心的距离，h2为所述摄像装置位于第二焦距时所述障碍物在所述像平面中心以上的成像高度，0≤θ＜90度。
在本申请的一实施例中，避障装置300还包括：
方向调整模块304，用于根据所述障碍物的位置信息调整所述无人机的速度方向，以躲避所述障碍物。
在本申请的一实施例中，方向调整模块304具体用于：
将所述无人机的速度方向向上调整方向偏角A，其中，所述方向偏角A为：
Figure PCTCN2017116572-appb-000021
其中，λ为安全系数。
在本申请的一实施例中，避障装置300还包括：
摄像调整模块305，用于调整所述摄像装置面向检测区域。
在本申请的一实施例中，摄像调整模块305具体用于：
调整所述摄像装置的视线轴与所述无人机的飞行方向保持一致。
在本申请的一实施例中，避障装置300还包括：
重复模块306，用于每间隔预设时间或者预设距离，重复调用焦距设定模块、视差图获取模块、障碍物确认模块和障碍物位置获取模块。
需要说明的是，上述装置可执行本申请实施例所提供的方法，具备执行方法相应的功能模块和有益效果。未在装置实施例中详尽描述的技术细节，可参见本申请实施例所提供的方法。
图11是本申请实施例提供的一种无人机20的硬件结构示意图，如图11所示，无人机20包括：机身24、与所述机身24相连的机臂25、设于所述机臂的动力装置27、用于获取图像的摄像装置26、设于机身24内的控制器23。控制器23包括至少一个处理器21和内置或者外置于无人机20的存储器22(图11中以存储器22内置于无人机20中为例)。
其中，处理器21和存储器22可以通过总线或者其他方式连接。
存储器22作为一种非易失性计算机可读存储介质，可用于存储非易失性软件程序、非易失性计算机可执行程序以及模块，如本申请实施例中的避障方法对应的程序指令/单元(例如，附图10所示的检测图像获取模块301、障碍物确认模块302和障碍物位置获取模块303)。处理器21通过运行存储在存储器22中的非易失性软件程序、指令以及单元，从而执行无人机20的各种功能应用以及数据处理，即实现上述方法实施例的避障方法。
存储器22可以包括存储程序区和存储数据区，其中，存储程序区可存储操作系统、至少一个功能所需要的应用程序；存储数据区可存储根据用户终端设备使用所创建的数据等。此外，存储器22可以包括高速随机存取存储器，还可以包括非易失性存储器，例如至少一个磁盘存储器件、闪存器件、或其他非易失性固态存储器件。在一些实施例中，存储器22可选包括相对于处理器21远程设置的存储器，这些远程存储器可以通过网络连接至无人机20。
所述一个或者多个模块存储在所述存储器22中，当被所述一个或者多个处理器21执行时，执行上述任意方法实施例中的避障方法，例如，执行以上描述的图3中的方法步骤S101至步骤S105，实现图10中 的模块301-306的功能。
上述无人机20可执行本申请实施例所提供的避障方法，具备执行方法相应的功能模块和有益效果。未在无人机20实施例中详尽描述的技术细节，可参见本申请实施例所提供的避障方法。
可选的，在所述无人机的某些实施例中，所述摄像装置26采用无人机20的主摄像头。主摄像头可以拍摄三通道图像，在立体匹配时需要三个通道都要对准才确定为同一个像素位置，相对于目前的利用灰度信息进行单通道立体匹配方法来说，其匹配精度更高。
本申请实施例提供了一种非易失性计算机可读存储介质，所述计算机可读存储介质存储有计算机可执行指令，该计算机可执行指令被一个或多个处理器执行，例如，执行以上描述的图3中的方法步骤S101至步骤S105，实现图10中的模块301-306的功能。
以上所描述的装置实施例仅仅是示意性的，其中所述作为分离部件说明的单元可以是或者也可以不是物理上分开的，作为单元显示的部件可以是或者也可以不是物理单元，即可以位于一个地方，或者也可以分布到多个网络单元上。可以根据实际的需要选择其中的部分或者全部模块来实现本实施例方案的目的。
通过以上的实施例的描述，本领域普通技术人员可以清楚地了解到各实施例可借助软件加通用硬件平台的方式来实现，当然也可以通过硬件。本领域普通技术人员可以理解实现上述实施例方法中的全部或部分流程是可以通过计算机程序来指令相关的硬件来完成，所述的程序可存储于一计算机可读取存储介质中，该程序在执行时，可包括如上述各方法的实施例的流程。其中，所述的存储介质可为磁碟、光盘、只读存储记忆体(Read-Only Memory,ROM)或随机存储记忆体(RandomAccessMemory,RAM)等。
最后应说明的是：以上实施例仅用以说明本申请的技术方案，而非对其限制；在本申请的思路下，以上实施例或者不同实施例中的技术特征之间也可以进行组合，步骤可以以任意顺序实现，并存在如上所述的本申请的不同方面的许多其它变化，为了简明，它们没有在细节中提供； 尽管参照前述实施例对本申请进行了详细的说明，本领域的普通技术人员应当理解：其依然可以对前述各实施例所记载的技术方案进行修改，或者对其中部分技术特征进行等同替换；而这些修改或者替换，并不使相应技术方案的本质脱离本申请各实施例技术方案的范围。

Claims (16)

  1. 一种避障方法，用于无人机，所述无人机包括摄像装置，所述摄像装置支持光学变焦，其特征在于，所述方法包括：
    获取所述摄像装置在至少两个不同焦距下的检测图像；
    根据所述至少两个不同焦距下的检测图像确定检测区域中存在障碍物；
    根据至少两个不同焦距下的所述检测图像获取所述障碍物的位置信息。
  2. 根据权利要求1所述的方法，其特征在于，所述位置信息包括所述障碍物与所述无人机的距离S，以及所述障碍物与所述无人机的高度差H；
    所述根据至少两个不同焦距下的所述检测图像获取所述障碍物的位置信息，包括：
    计算所述障碍物与所述无人机的距离S以及所述障碍物与所述无人机的高度差H：
    Figure PCTCN2017116572-appb-100001
    Figure PCTCN2017116572-appb-100002
    其中，θ为所述无人机的飞行方向与水平方向的夹角，f1为所述摄像装置位于第一焦距时像平面距离光心的距离，h1为所述摄像装置位于第一焦距时所述障碍物在所述像平面中心以上的成像高度，f2为所述摄像装置位于第二焦距时像平面距离光心的距离，h2为所述摄像装置位于第二焦距时所述障碍物在所述像平面中心以上的成像高度，0≤θ＜90度。
  3. 根据权利要求1或2所述的方法，其特征在于，所述方法还包括：
    根据所述障碍物的位置信息调整所述无人机的速度方向，以躲避所述障碍物。
  4. 根据权利要求3所述的方法，其特征在于，所述根据所述障碍物的位置信息调整所述无人机的速度方向，包括：
    将所述无人机的速度方向向上调整方向偏角A，其中，所述方向偏角A为：
    Figure PCTCN2017116572-appb-100003
    其中，λ为安全系数。
  5. 根据权利要求1-4任意一项所述的方法，其特征在于，所述方法还包括：
    调整所述摄像装置面向检测区域。
  6. 根据权利要求5所述的方法，其特征在于，所述调整所述摄像装置面向检测区域，包括：
    调整所述摄像装置的视线轴与所述无人机的飞行方向保持一致。
  7. 一种避障装置，用于无人机，所述无人机包括摄像装置，所述摄像装置支持光学变焦，其特征在于，所述装置包括：
    检测图像获取模块，用于获取所述摄像装置在至少两个不同焦距下的检测图像；
    障碍物确认模块，用于根据所述至少两个不同焦距下的检测图像确定检测区域中存在障碍物；
    障碍物位置获取模块，用于根据至少两个不同焦距下的所述检测图像获取所述障碍物的位置信息。
  8. 根据权利要求7所述的装置，其特征在于，所述位置信息包括所述障碍物与所述无人机的距离S，以及所述障碍物与所述无人机的高度差H；
    所述障碍物位置获取模块，具体用于：
    计算所述障碍物与所述无人机的距离S以及所述障碍物与所述无人机的高度差H：
    Figure PCTCN2017116572-appb-100004
    Figure PCTCN2017116572-appb-100005
    其中，θ为所述无人机的飞行方向与水平方向的夹角，f1为所述摄像装置 位于第一焦距时像平面距离光心的距离，h1为所述摄像装置位于第一焦距时所述障碍物在所述像平面中心以上的成像高度，f2为所述摄像装置位于第二焦距时像平面距离光心的距离，h2为所述摄像装置位于第二焦距时所述障碍物在所述像平面中心以上的成像高度，0≤θ＜90度。
  9. 根据权利要求7或8所述的装置，其特征在于，所述装置还包括：
    方向调整模块，用于根据所述障碍物的位置信息调整所述无人机的速度方向，以躲避所述障碍物。
  10. 根据权利要求9所述的装置，其特征在于，所述方向调整模块具体用于：
    将所述无人机的速度方向向上调整方向偏角A，其中，所述方向偏角A为：
    Figure PCTCN2017116572-appb-100006
    其中，λ为安全系数。
  11. 根据权利要求7-10任意一项所述的装置，其特征在于，所述装置还包括：
    摄像调整模块，用于调整所述摄像装置面向检测区域。
  12. 根据权利要求11所述的装置，其特征在于，所述摄像调整模块具体用于：
    调整所述摄像装置的视线轴与所述无人机的飞行方向保持一致。
  13. 一种无人机，其特征在于，包括机身、与所述机身相连的机臂、设于所述机臂的动力装置以及用于获取图像的摄像装置，所述摄像装置支持光学变焦，所述无人机还包括：
    处理器；以及，
    与处理器通信连接的存储器；其中，
    所述存储器存储有可被所述处理器执行的指令，所述指令被所述处理器执行，以使所述处理器能够执行如权利要求1-6任一项所述的方法。
  14. 根据权利要求13所述的无人机，其特征在于，所述摄像装置为所述无人机的主摄像头。
  15. 一种非易失性计算机可读存储介质，其特征在于，所述计算机可读存储介质存储有计算机可执行指令，当所述计算机可执行指令被无人机执行时，使所述无人机执行如权利要求1-6任一项所述的方法。
  16. 一种计算机程序产品，其特征在于，所述计算机程序产品包括存储在非易失性计算机可读存储介质上的计算机程序，所述计算机程序包括程序指令，当所述程序指令被无人机执行时，使所述无人机执行如权利要求1-6任一项所述的方法。
PCT/CN2017/116572 2017-12-15 2017-12-15 一种避障方法、装置和无人机 WO2019113966A1 (zh)

Priority Applications (4)

Application Number Priority Date Filing Date Title
EP17832180.8A EP3591490B1 (en) 2017-12-15 2017-12-15 Obstacle avoidance method and device, and unmanned aerial vehicle
PCT/CN2017/116572 WO2019113966A1 (zh) 2017-12-15 2017-12-15 一种避障方法、装置和无人机
CN201780002277.XA CN108323190B (zh) 2017-12-15 2017-12-15 一种避障方法、装置和无人机
US15/885,041 US10860039B2 (en) 2017-12-15 2018-01-31 Obstacle avoidance method and apparatus and unmanned aerial vehicle

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
PCT/CN2017/116572 WO2019113966A1 (zh) 2017-12-15 2017-12-15 一种避障方法、装置和无人机

Related Child Applications (1)

Application Number Title Priority Date Filing Date
US15/885,041 Continuation US10860039B2 (en) 2017-12-15 2018-01-31 Obstacle avoidance method and apparatus and unmanned aerial vehicle

Publications (1)

Publication Number Publication Date
WO2019113966A1 true WO2019113966A1 (zh) 2019-06-20

Family

ID=62894140

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2017/116572 WO2019113966A1 (zh) 2017-12-15 2017-12-15 一种避障方法、装置和无人机

Country Status (4)

Country Link
US (1) US10860039B2 (zh)
EP (1) EP3591490B1 (zh)
CN (1) CN108323190B (zh)
WO (1) WO2019113966A1 (zh)

Families Citing this family (16)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN109143242B (zh) 2018-09-07 2020-04-14 百度在线网络技术（北京）有限公司 障碍物绝对速度估计方法、系统、计算机设备和存储介质
CN109165629B (zh) * 2018-09-13 2019-08-23 百度在线网络技术（北京）有限公司 多焦距视觉障碍物感知方法、装置、设备及存储介质
CN109739256B (zh) * 2018-12-20 2022-04-19 深圳市道通智能航空技术股份有限公司 一种无人机降落避障方法、装置及无人机
CN109828274B (zh) * 2019-01-07 2022-03-04 深圳市道通智能航空技术股份有限公司 调整机载雷达的主探测方向的方法、装置和无人机
CN109703568B (zh) 2019-02-19 2020-08-18 百度在线网络技术（北京）有限公司 自动驾驶车辆行驶策略实时学习的方法、装置和服务器
CN109712421B (zh) 2019-02-22 2021-06-04 百度在线网络技术（北京）有限公司 自动驾驶车辆的速度规划方法、装置和存储介质
CN110333734A (zh) * 2019-05-24 2019-10-15 深圳市道通智能航空技术有限公司 一种无人机及其控制方法、存储介质
CN110667847B (zh) * 2019-10-17 2020-08-18 安徽省徽腾智能交通科技有限公司泗县分公司 无人机智能化飞行高度控制平台
US11946771B2 (en) 2020-04-01 2024-04-02 Industrial Technology Research Institute Aerial vehicle and orientation detection method using same
TWI728770B (zh) * 2020-04-01 2021-05-21 財團法人工業技術研究院 飛行載具及應用其之方向偵測方法
CN111914784B (zh) * 2020-08-10 2022-02-08 北京大成国测科技有限公司 轨旁障碍物入侵实时检测方法、装置及电子设备
CN112364741B (zh) * 2020-11-02 2023-04-18 湖南航天宏图无人机系统有限公司 无人机单目远距离障碍检测方法、装置及无人机
CN112508865B (zh) * 2020-11-23 2024-02-02 深圳供电局有限公司 一种无人机巡检避障方法、装置、计算机设备和存储介质
CN113419555B (zh) * 2021-05-20 2022-07-19 北京航空航天大学 一种基于单目视觉的无人机低功耗自主避障方法及系统
CN115100622B (zh) * 2021-12-29 2023-09-22 中国矿业大学 深部受限空间无人运输设备可行驶域检测和自主避障方法
CN115790420A (zh) * 2022-11-25 2023-03-14 惠州市德赛西威汽车电子股份有限公司 障碍物测距方法、装置、车辆及介质

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20080189036A1 (en) * 2007-02-06 2008-08-07 Honeywell International Inc. Method and system for three-dimensional obstacle mapping for navigation of autonomous vehicles
CN105025219A (zh) * 2014-04-30 2015-11-04 齐发光电股份有限公司 图像获取方法
CN106290246A (zh) * 2016-08-09 2017-01-04 上海禾赛光电科技有限公司 无需gps的无人机的地面定位装置及气体遥测系统
CN107036579A (zh) * 2016-11-17 2017-08-11 上海航天控制技术研究所 一种基于单目液透镜变焦光学系统的目标相对定位方法

Family Cites Families (20)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US5499306A (en) * 1993-03-08 1996-03-12 Nippondenso Co., Ltd. Position-and-attitude recognition method and apparatus by use of image pickup means
US6130705A (en) * 1998-07-10 2000-10-10 Recon/Optical, Inc. Autonomous electro-optical framing camera system with constant ground resolution, unmanned airborne vehicle therefor, and methods of use
US7248968B2 (en) * 2004-10-29 2007-07-24 Deere & Company Obstacle detection using stereo vision
JP5446949B2 (ja) * 2010-02-03 2014-03-19 ソニー株式会社 画像処理装置および画像処理方法、並びにプログラム
CN102349291A (zh) * 2010-03-25 2012-02-08 富士胶片株式会社 图像捕获设备及其控制方法
CN101852609B (zh) * 2010-06-02 2011-10-19 北京理工大学 一种基于机器人双目立体视觉的地面障碍物检测方法
JP2012123296A (ja) * 2010-12-10 2012-06-28 Sanyo Electric Co Ltd 電子機器
US9091762B2 (en) * 2011-10-27 2015-07-28 Gulfstream Aerospace Corporation Methods and systems for avoiding a collision between an aircraft on a ground surface and an obstacle
FR2993509B1 (fr) * 2012-07-17 2015-03-20 Airbus Dispositif electronique de protection contre l'eblouissement d'un pilote ou d'un conducteur.
CN103278139B (zh) * 2013-05-06 2016-11-23 北京航空航天大学 一种可变焦单双目视觉传感装置
CN103411936B (zh) * 2013-07-29 2015-12-02 深圳大学 一种三维纳米分辨定位方法及装置
WO2015105886A1 (en) * 2014-01-10 2015-07-16 Pictometry International Corp. Unmanned aircraft structure evaluation system and method
CN107409205B (zh) * 2015-03-16 2020-03-20 深圳市大疆创新科技有限公司 用于焦距调节和深度图确定的装置和方法
CN106483974B (zh) * 2015-09-02 2019-05-10 中国航空工业第六一八研究所 一种固定翼无人机近距离几何避障方法
CN105627932B (zh) * 2015-12-31 2019-07-30 天津远翥科技有限公司 一种基于双目视觉的测距方法及装置
US11029352B2 (en) * 2016-05-18 2021-06-08 Skydio, Inc. Unmanned aerial vehicle electromagnetic avoidance and utilization system
CN106444837A (zh) * 2016-10-17 2017-02-22 北京理工大学 一种无人机避障方法及系统
CN106708084B (zh) * 2016-11-24 2019-08-02 中国科学院自动化研究所 复杂环境下无人机自动障碍物检测和避障方法
CN106502267B (zh) * 2016-12-06 2019-03-29 上海师范大学 一种无人机避让系统
CN107341454B (zh) * 2017-06-21 2019-12-27 海信集团有限公司 一种场景中障碍物的检测方法及装置、电子设备

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20080189036A1 (en) * 2007-02-06 2008-08-07 Honeywell International Inc. Method and system for three-dimensional obstacle mapping for navigation of autonomous vehicles
CN105025219A (zh) * 2014-04-30 2015-11-04 齐发光电股份有限公司 图像获取方法
CN106290246A (zh) * 2016-08-09 2017-01-04 上海禾赛光电科技有限公司 无需gps的无人机的地面定位装置及气体遥测系统
CN107036579A (zh) * 2016-11-17 2017-08-11 上海航天控制技术研究所 一种基于单目液透镜变焦光学系统的目标相对定位方法

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
See also references of EP3591490A4 *

Also Published As

Publication number Publication date
EP3591490A1 (en) 2020-01-08
US20190187725A1 (en) 2019-06-20
US10860039B2 (en) 2020-12-08
EP3591490A4 (en) 2020-12-09
EP3591490B1 (en) 2021-12-01
CN108323190B (zh) 2022-07-29
CN108323190A (zh) 2018-07-24

Similar Documents

Publication Publication Date Title
WO2019113966A1 (zh) 一种避障方法、装置和无人机
CN107329490B (zh) 无人机避障方法及无人机
CN106529495B (zh) 一种飞行器的障碍物检测方法和装置
WO2020014909A1 (zh) 拍摄方法、装置和无人机
WO2018072657A1 (zh) 图像处理的方法、处理图像的装置、多目摄像装置以及飞行器
WO2018210078A1 (zh) 无人机的距离测量方法以及无人机
CN108235815B (zh) 摄像控制装置、摄像装置、摄像系统、移动体、摄像控制方法及介质
US11057604B2 (en) Image processing method and device
US20220086362A1 (en) Focusing method and apparatus, aerial camera and unmanned aerial vehicle
WO2018120350A1 (zh) 对无人机进行定位的方法及装置
WO2019126930A1 (zh) 测距方法、装置以及无人机
WO2020233682A1 (zh) 一种自主环绕拍摄方法、装置以及无人机
JP6733106B2 (ja) 決定装置、移動体、決定方法、及びプログラム
JP2020012878A (ja) 制御装置、移動体、制御方法、及びプログラム
WO2022193081A1 (zh) 无人机的控制方法、装置及无人机
WO2020019175A1 (zh) 图像处理方法和设备、摄像装置以及无人机
US20210105411A1 (en) Determination device, photographing system, movable body, composite system, determination method, and program
WO2021013143A1 (zh) 装置、摄像装置、移动体、方法以及程序
JP7501535B2 (ja) 情報処理装置、情報処理方法、情報処理プログラム
JP6515423B2 (ja) 制御装置、移動体、制御方法、及びプログラム
US20210112202A1 (en) Control apparatuses, mobile bodies, control methods, and programs
WO2021026754A1 (zh) 一种拍摄装置的对焦控制方法、装置及无人飞行器
JP6641574B1 (ja) 決定装置、移動体、決定方法、及びプログラム
WO2020107487A1 (zh) 图像处理方法和无人机
JP6260803B2 (ja) 撮像システム、移動体、撮像方法、及びプログラム

Legal Events

Date Code Title Description
WWE Wipo information: entry into national phase

Ref document number: 2017832180

Country of ref document: EP

121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 17832180

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE