WO2019015158A1

WO2019015158A1 - Obstacle avoidance method for unmanned aerial vehicle, and unmanned aerial vehicle

Info

Publication number: WO2019015158A1
Application number: PCT/CN2017/108022
Authority: WO
Inventors: 王晓曼
Original assignee: 歌尔科技有限公司
Priority date: 2017-07-21
Filing date: 2017-10-27
Publication date: 2019-01-24
Also published as: CN107329490B; CN107329490A

Abstract

An obstacle avoidance method for an unmanned aerial vehicle, and an unmanned aerial vehicle. The obstacle avoidance method for an unmanned aerial vehicle comprises: acquiring, by means of a binocular camera, a stereoscopic parallax map of a current environment image (101), and clustering pixel points in the stereoscopic parallax map to obtain a plurality of clustering regions (102); based on each clustering region, calculating a flight distance from an environment region corresponding to each clustering region to an unmanned aerial vehicle (103); and determining whether the flight distance from each environment area to the unmanned aerial vehicle is greater than a distance threshold value (104), so that the unmanned aerial vehicle selects any fly-around region from environment regions with a flight distance being greater than the distance threshold value (105), and calculating a rotation angle for the unmanned aerial vehicle to turn to the fly-around region, and controlling the unmanned aerial vehicle so that same flies around according to the rotation angle (106). The autonomous obstacle avoidance and flying around of an unmanned aerial vehicle are realised, thus improving the flight efficiency.

Description

UAV obstacle avoidance method and drone

Technical field

The invention belongs to the field of electronic technology, and in particular to a method for avoiding obstacles of a drone and a drone.

Background technique

With the rapid development of drone technology, drones have been widely used in aerial photography, cargo transportation, resource exploration, and route survey. Since the drone does not have special personnel to operate the drone during flight, the drone needs to fly autonomously according to the planned route, because it is difficult to avoid the obstacles encountered in the actual flight when planning the route.

Unmanned aircraft owners can avoid obstacles encountered in flight based on video autonomous obstacle avoidance methods. The method captures an image of the surrounding environment by the camera carried by the drone, and analyzes the obstacle behavior of the obstacle based on the image taken to analyze whether there is an obstacle in front of the obstacle, so that the drone can be safely flying.

In the prior art, the video-based autonomous obstacle avoidance method is to hover when encountering an obstacle, and cannot continue to fly, and can only transmit the image back to the ground station to recalculate the flight route and then perform obstacle avoidance flight, resulting in the drone's Flight efficiency is low.

Summary of the invention

In view of this, the present invention provides a UAV obstacle avoidance method and a UAV, which solves the technical problem that the UAV has low flight efficiency, and realizes the UAV autonomous obstacle avoidance flying, which can greatly improve Flight efficiency.

In order to solve the above technical problem, the present invention provides a method for obstacle avoidance of a drone, comprising:

Obtaining a stereo disparity map of the current environment image through the binocular camera;

Clustering pixel points in the stereo disparity map to obtain a plurality of cluster regions;

Calculating a flight distance of the environmental area corresponding to each cluster area to the drone based on each cluster area;

Determining whether the flight distance of each environmental area and the drone is greater than a distance threshold;

Selecting any one of the flying areas from an environmental area whose flight distance is greater than the distance threshold;

Calculating a rotation angle of the drone to the any of the flying regions, and controlling the drone to fly around according to the rotation angle.

Preferably, the clustering the pixel points in the stereo disparity map to obtain a plurality of cluster regions includes:

Taking any unmarked pixel as the target pixel, starting from the adjacent pixel of the target pixel Determining whether a pixel difference between each pixel point and the target pixel point is within a preset range;

Pixel points in the adjacent area of the target pixel that are different from the pixel of the target pixel in a preset range are marked as belonging to the same cluster area as the target pixel.

Preferably, the unmarked pixel point is used as a target pixel point, and starting from adjacent pixel points of the target pixel point, sequentially determining whether a pixel difference between each pixel point and the target pixel point is preset The scope includes:

Taking any unmarked pixel as the target pixel, starting from each adjacent pixel of the target pixel, sequentially determining whether the pixel difference between each successive pixel and the target pixel is in advance It is assumed that the pixel difference value up to any one of the pixel points and the target pixel point is not within the preset range.

Preferably, the stereo disparity map for acquiring the current environment image by the binocular camera comprises:

Calibrating the binocular camera to obtain a Q matrix composed of a focal length, a baseline, and an origin offset of the binocular camera;

Performing image correction on the current environment image acquired by the binocular camera based on the Q matrix to obtain a corrected image;

Performing stereo matching calculation on the corrected image to obtain the stereo disparity map;

Calculating, according to each cluster area, a flight distance of the environment area corresponding to each cluster area to the drone includes:

Determining position coordinates of target pixel points in each cluster area;

Calculating, according to the target pixel point of each clustering region, the position coordinates of the environment region corresponding to each of the cluster regions in the three-dimensional coordinate system according to the following coordinate formula;

The position coordinate calculation formula is:

Where (X, Y, Z) is the position coordinate of the environmental region in the three-dimensional coordinate system, and Z represents the flight distance of the environmental region corresponding to each of the cluster regions to the drone, and T _x represents a baseline of the binocular camera, f denotes a focal length of the binocular camera, (x, y) denotes a position coordinate of the target pixel point in any cluster region, and d denotes the one in the any cluster region The parallax corresponding to the target pixel, (c _x , c _y ) is the imaging origin after the binocular camera is corrected.

Preferably, the selecting one of the environment regions from the flight distance greater than the distance threshold comprises:

If the flight area corresponding to the flight direction of the drone and the flight distance of the drone is less than the distance threshold, An environmental area in which the flight distance is greater than the distance threshold and the farthest flight distance is selected as the candidate area;

Determining whether the candidate area satisfies flight conditions;

If yes, the candidate area is used as a flying area;

If not, an environment region whose flight distance is the farthest from the flight distance greater than the distance threshold and not including the candidate region is selected as the candidate region, and the step of determining whether the candidate region satisfies the flight condition is continued.

Preferably, the method further comprises:

If the flight area corresponding to the flight direction of the drone and the flight distance of the drone are greater than the distance threshold, the drone is controlled to fly according to the original route.

Preferably, the method further comprises:

If none of the candidate regions meet the flight conditions, the drone is controlled to hover.

Preferably, the calculating the rotation angle of the drone to the any of the flying regions and controlling the drone to fly according to the rotation angle comprises:

Calculating an angle of rotation of the drone in the flying area according to position coordinates corresponding to the flying area and controlling the drone to fly according to the rotation angle.

The invention also provides an unmanned obstacle avoidance, comprising:

An acquisition module, configured to acquire a stereo disparity map of the current environment image by using a binocular camera;

a clustering module, configured to cluster pixel points in the stereo disparity map to obtain a plurality of cluster regions;

a flight distance calculation module, configured to calculate a flight distance of the environmental region corresponding to each cluster region to the drone based on each cluster region;

a first determining module, configured to determine whether a flight distance of each environmental area and the drone is greater than a distance threshold;

a first selection module, configured to select any one of the surrounding areas from an environmental area whose flight distance is greater than a distance threshold;

The flying module is configured to calculate a rotation angle of the drone to the any of the flying regions, and control the drone to fly around according to the rotation angle.

Preferably, the clustering module comprises:

a second determining unit, configured to use any pixel point that is not marked as a target pixel point, and phase from the target pixel point Starting from adjacent pixels, determining whether the pixel difference between each pixel point and the target pixel point is within a preset range;

And a marking unit, configured to mark a pixel difference between the adjacent pixel of the target pixel and the pixel of the target pixel in a preset range, and mark the same cluster area as the target pixel.

Preferably, the second determining unit is specifically configured to:

Preferably, the first acquiring module is specifically configured to:

The flight distance calculation module is specifically configured to:

Determining position coordinates of target pixel points in each cluster area;

The position coordinate calculation formula is:

Preferably, the first selection module includes: a second selection unit, a third determination unit, a determination unit, and a third selection unit;

The second selecting unit is configured to select a farthest flight distance from an environment region where the flight distance is greater than the distance threshold if the flight distance corresponding to the flight direction of the drone is less than the distance threshold An environment The area is a candidate area;

The third determining unit is configured to determine whether the candidate area meets a flight condition; if yes, trigger the determining unit; if not, trigger the third selecting unit;

The determining unit is configured to use the candidate area as a flying area;

The third selection unit is configured to select, as a candidate region, an environment region whose flight distance is the farthest from the environment region where the flight distance is greater than the distance threshold and does not include the candidate region, and return to determine whether the candidate region meets the flight condition. The steps continue.

Preferably, the method further comprises:

The first control module is configured to control the drone to fly according to the original route if the flight area corresponding to the flight direction of the drone and the flight distance of the drone are greater than a distance threshold.

Preferably, the method further comprises:

And a second control module, configured to control the drone to hover if none of the candidate regions meet the flight condition.

Preferably, the flying module is specifically configured to:

Compared with the prior art, the present invention can obtain the following technical effects:

The invention provides a UAV obstacle avoidance method and a UAV, which acquires a stereo disparity map of a current environment image through a binocular camera and clusters the pixels in the stereo disparity map to obtain a plurality of Cluster area. Based on each cluster area, the flight distance of the environment area corresponding to each cluster area to the drone is calculated. By determining whether the flight distance of each environmental area and the drone is greater than a distance threshold, the drone selects any one of the flying areas from the environmental area whose flight distance is greater than the distance threshold, and calculates the unmanned aerial vehicle Describe the rotation angle of any flying area, and control the drone to fly around according to the rotation angle. By determining the flight distance of the environment area corresponding to each cluster area in the stereo disparity map to the drone, the invention selects the flying area larger than the distance threshold and performs the flying according to the rotation angle of the flying area, thereby realizing The drone autonomously avoids obstacles and greatly improves flight efficiency.

DRAWINGS

1 is a flow chart of an embodiment of a method for obstacle avoidance of a drone according to an embodiment of the present invention;

2 is a flow chart of another embodiment of a method for obstacle avoidance of a drone according to an embodiment of the present invention;

3 is a schematic structural view of an embodiment of a drone according to an embodiment of the present invention;

4 is a schematic structural view of another embodiment of a drone according to an embodiment of the present invention.

Detailed ways

The embodiments of the present invention will be described in detail below with reference to the accompanying drawings and embodiments, in which the present invention can be fully understood and implemented by the technical means of solving the technical problems and achieving the technical effects.

With the rapid development of electronic technology, drones have been widely used in aerial photography, cargo transportation, resource exploration, route survey and other fields. The autonomous flight of the drone requires automatic obstacle avoidance to ensure the flight safety of the drone. The existing automatic obstacle avoidance means may include video-based obstacle avoidance, but currently the video-based autonomous obstacle avoidance method is to hover when encountering an obstacle, unable to continue flying, and can only return the image to the ground station to recalculate the flight. After the route, obstacle avoidance flight, there are many restrictions on the use, resulting in low flight efficiency of the drone.

In order to solve the technical problem that the drone has low flight efficiency, the inventors have proposed a technical solution of the present invention through a series of studies. In the present invention, a stereo disparity map of a current environment image is acquired by a binocular camera and pixels in the stereo disparity map are clustered to obtain a plurality of cluster regions. Based on each cluster area, the flight distance of the environment area corresponding to each cluster area to the drone is calculated. Then, by determining whether the flight distance of each environmental area and the drone is greater than a distance threshold, the drone selects any one of the flying areas from the environmental area whose flight distance is greater than the distance threshold, and controls the drone according to the drone. The rotation angle is made to fly around. By determining the flight distance of the environment area corresponding to each cluster area in the stereo disparity map to the drone, the invention selects the flying area larger than the distance threshold and performs the flying according to the rotation angle of the flying area, thereby realizing The drone autonomously avoids obstacles and greatly improves flight efficiency.

The technical solution of the present invention will be described in detail below with reference to the accompanying drawings.

FIG. 1 is a flowchart of an embodiment of a method for avoiding obstacles of a drone according to an embodiment of the present invention. The method may include:

101: Obtain a stereo disparity map of the current environment image by a binocular camera.

The binocular camera is composed of two left and right cameras at the front end of the drone, and is used to capture the image of the environment around the flight path of the drone. The binocular camera simulates the principle of human eye vision. Two environment images are acquired by the left and right cameras. Since the environment image captured by the binocular image is distorted, it is necessary to correct the collected environment image and use a stereo matching algorithm to calculate Obtain a stereo disparity map. The stereo matching algorithm mainly uses the correspondence between two environmental images acquired by the binocular camera, and obtains a disparity map according to the triangulation principle; after obtaining the disparity information, the depth information of the original environment image can be easily obtained according to the projection model. And three-dimensional information to calculate the stereo disparity map.

102: Cluster the pixels in the stereo disparity map to obtain a plurality of cluster regions.

After obtaining the stereo disparity map of the surrounding environment image, the pixel points may be clustered according to the gray value of each pixel in the stereo disparity map, and the pixel points with the gray value close to each other are divided into one cluster region, thereby Will be in the stereo disparity map The pixels are divided into a plurality of cluster regions.

103: Calculate, according to each cluster area, a flight distance of the environment area corresponding to each cluster area to the drone.

Optionally, after calculating the flight distance of the environment image corresponding to each cluster area to the drone, the environment area may be sorted from large to small according to the distance from each environment area to the drone. Obtain the order of flight distance from the environmental area to the drone.

Optionally, as still another embodiment, the acquiring a stereo disparity map of the current environment image by the binocular camera includes:

Performing image correction on the current environment image acquired by the binocular camera based on the Q matrix to obtain a corrected image; wherein the current environment image acquired by the binocular camera includes two, and thus, two corrected images are obtained correspondingly. .

Performing a stereo matching calculation on the two corrected images to obtain the stereo disparity map; wherein, the formula of the Q matrix may be:

Wherein, the T _x represents a baseline of the binocular camera, and f represents a focal length of the binocular camera, and (c _x , c _y ), (c′ _x , c′ _y ) are respectively corrected by the binocular camera. The latter two imaging origins, and the two corrected imaging origins are the same, ie c _x = c' _x , c _y = c' _y .

The imaging origin refers to the intersection of the optical axis of the binocular camera and the acquired image plane, usually at the center of the image. The imaging origin of the two environmental images acquired by the binocular camera can be obtained by binocular camera calibration. Since the acquired environmental image needs image correction, the same imaging origin after correction of the two environmental images is (c _x , c _y ) Thus, it is possible to obtain an ideal form in which two environmental images are perfectly aligned in parallel.

Optionally, after obtaining the internal and external parameters of the binocular camera by binocular camera calibration, the two environment images acquired by the two cameras may be image corrected by the internal and external parameters of the binocular camera, and the corrected two environment images are obtained. The planes are perfectly parallel aligned to facilitate stereo matching of the two corrected images using a stereo matching algorithm to obtain a stereo disparity map of the current environment image. Optionally, the stereo matching algorithm may calculate the disparity d of each pixel in the corrected image acquired by the binocular camera according to the triangulation principle, so that the parallax image can be obtained, and after obtaining the disparity information, according to the projection model, The depth information and the three-dimensional information of the original environment image are calculated to obtain a stereo disparity map.

Calculating, according to each cluster area, a flight distance corresponding to an environmental area corresponding to each cluster area to the drone From:

Determining position coordinates of target pixel points in each cluster area;

The position coordinate calculation formula can be:

Where (X, Y, Z) is the position coordinate of the environmental region in the three-dimensional coordinate system, and Z may represent the flight distance of the environmental region corresponding to each of the cluster regions to the drone, (x, y) represents the position coordinates of the target pixel point in any of the cluster regions, and d represents the parallax corresponding to the target pixel in any of the cluster regions.

104: Determine whether the flight distance of each environmental area and the drone is greater than a distance threshold.

105: selecting any one of the flying areas from an environmental area whose flight distance is greater than a distance threshold;

Optionally, the distance threshold may be set according to performance parameters of the actual in-flight drone, for example, the drone may hover when it is three meters away from the obstacle in front to ensure that it does not hit the obstacle. It is believed that the drone can set a safe distance of three meters and use the safety distance as a distance threshold to ensure safe flight of the drone.

106: Calculate a rotation angle of the drone to the any of the flying regions, and control the drone to fly around according to the rotation angle.

Optionally, in some embodiments, the calculating the rotation angle of the drone to the any of the flying regions and controlling the drone to fly according to the rotation angle comprises:

After the drone selects the flying area, the drone needs to make a steering flight to the flying area, so it is necessary to calculate the rotation angle of the selected flying area of the drone. The rotation angle can be calculated by the position coordinates (X, Y, Z) in the solid coordinate region corresponding to the environment region corresponding to the flying region, and the drone is controlled to fly around according to the rotation angle.

Optionally, in order to prevent the flight path of the drone from being changed too much, in the case where any of the flying areas can pass through the unmanned aerial vehicle, the flying around the flying area with the smallest rotation angle can be preferentially selected.

In this embodiment, by determining the distance of the environment area corresponding to each cluster area in the stereo disparity map to the drone, the flying area larger than the distance threshold is selected and the flying around the flying area is performed, and The environmental region with the smallest rotation angle and greater than the threshold of the distance zone is preferentially used as the surrounding area to fly around, which not only ensures that the flight path does not change greatly, but also realizes the autonomous autonomous obstacle avoidance of the drone, which greatly improves the flight efficiency.

Optionally, as another embodiment, the pixel points in the stereo disparity map are clustered to obtain multiple aggregations. Class areas include:

Taking any unmarked pixel as the target pixel, starting from the adjacent pixel of the target pixel, sequentially determining whether the pixel difference between each pixel and the target pixel is within a preset range;

Optionally, after clustering any pixel in the stereoscopic disparity map, marking the cluster region where the pixel is located.

For example, it is preferred to select any unmarked pixel point in the stereo disparity map as the target pixel point, and mark the clustering area of the target pixel point as the area 0, and the position coordinate of the target pixel point is ( i, j), the gray value is d ₀ .

Optionally, as another embodiment, the untagged one of the pixel points is used as a target pixel point, and the pixel points and the target pixel point are sequentially determined from adjacent pixel points of the target pixel point. Whether the pixel difference value is within the preset range includes:

Optionally, the determining, from the adjacent pixel points of the target pixel point, sequentially determining whether a pixel difference between each pixel point and the target pixel point is within a preset range, may first select the target pixel point The adjacent position coordinate is the first pixel point of (i, j-1), and the corresponding gray value d _{1 of} the first pixel point is obtained. The preset range of setting the gray value is D, and it is judged whether |d ₀ -d ₁ | is within the preset range, and if so, the first pixel is marked as area 0. Then, it is sequentially determined whether the pixel adjacent to the first pixel belongs to the region 0 until the pixel in the direction that does not belong to the region 0 is found, and the judgment in the direction is stopped. According to the above process, the pixel points in the three directions of the target pixel point, the left and the right direction are sequentially selected to belong to the area 0, and after the clustering of the area 0 is completed, any pixel that is not marked is reselected as the target pixel point is marked as the area 1 and The clustering of the region 1 is completed according to the clustering process of the region 0. By analogy, until the clustering area is marked for each pixel in the stereo disparity map, the clustering is completed, so that a plurality of clustering regions can be obtained.

FIG. 2 is a flowchart of another embodiment of a method for obstacle avoidance of a drone according to an embodiment of the present invention. The method may include:

201: Obtain a stereo disparity map of a current environment image by using a binocular camera;

202: Clustering pixels in the stereo disparity map to obtain a plurality of cluster regions;

203: Calculate, according to each cluster area, a flight distance of the environment area corresponding to each cluster area to the drone.

204: Determine whether a flight distance of each environmental area and the drone is greater than a distance threshold.

Optionally, when determining whether the flight distance of each environment area and the drone is greater than a distance threshold, the flight distances of the drones may be sorted according to the distance from each environment area to obtain an environment according to the environment. The sequence in which the flight distances of the regions are sorted.

Optionally, it may be first determined whether the flight distance of the environment area corresponding to the flight direction of the drone and the drone is greater than a distance threshold.

Optionally, in another embodiment, after determining whether the distance between each of the environmental regions and the drone is greater than a distance threshold, the method may further include:

205: If the flight area corresponding to the flight direction of the UAV and the UAV is less than the distance threshold, select an environment area that is the farthest from the flight area with the flight distance greater than the distance threshold as the candidate area. .

When the flight area corresponding to the flight direction of the drone and the flight distance of the drone is less than the distance threshold, the drone cannot continue to fly according to the initial flight direction, and the environment area larger than the flight distance needs to be selected. Take a fly around. According to the arrangement order of the flight distances corresponding to the respective environmental regions, the environment region with the largest flight distance may be preferentially selected as the candidate region for the flight. Of course, it is also possible to select an environmental area larger than the flight distance closest to the flight direction of the drone as a candidate area to fly around.

206: Determine whether the candidate area meets a flight condition; if yes, perform step 207; if no, perform step 208.

Alternatively, the flight condition may be determined according to the actual size of the candidate area. For example, the size of the drone is 1 meter wide and 0.5 meters high. If the actual size of the candidate area is larger than the size of the drone, the drone can be safely passed, and the candidate area satisfies the flight condition if the candidate area If the actual size is smaller than the size of the drone, the drone cannot fly through the candidate area and therefore does not meet the flight conditions.

207: The candidate area is used as a flying area.

208: Select an environment area whose flight distance is the farthest from the environment area whose flight distance is greater than the distance threshold and does not include the candidate area as the candidate area, and return to step 206 to continue the corresponding operation.

Optionally, if the candidate region does not satisfy the flight condition, the candidate region with the largest flight distance among the environmental regions other than the candidate region is selected as the new candidate region according to the order of the flight distance corresponding to the environment region. It is re-determined whether the candidate area satisfies the flight condition.

209: Calculate a rotation angle of the drone to the any of the flying regions, and control the drone to fly around according to the rotation angle.

Optionally, as still another embodiment, the determining whether the flight distance between each environment area and the drone is large After the distance threshold, it may also include:

Optionally, if none of the candidate regions meet the flight condition, indicating that the drone cannot continue to fly, control the drone to hover and send the surrounding environment image to the ground station to recalculate the flight route, and then wait to receive After the flight route sent by the ground station, follow the flight instructions to avoid obstacles.

The operation of the step 201 - step 203 in the embodiment is the same as the operation of the step 101 - step 103 in the embodiment of FIG. 1 . The operation of the step 209 is the same as the operation of the step 106 in the embodiment of FIG. 1 , and details are not described herein again.

In this embodiment, by first determining whether the environment area corresponding to the flight direction of the drone is greater than the distance threshold, if it is greater than, the flight continues according to the original route; if less, the environment with the flight distance greater than the distance threshold and the farthest flight distance is preferentially selected. The area is used as a candidate area, and it is judged that the candidate area satisfies the flight condition, so that the drone can find an optimal flight path according to the surrounding environment, and realize the autonomous autonomous obstacle avoidance. Flying around, greatly improving flight efficiency.

FIG. 3 is a schematic structural diagram of an embodiment of a drone according to an embodiment of the present invention. The drone may include:

The obtaining module 301 is configured to acquire a stereo disparity map of the current environment image by using the binocular camera.

The binocular camera is composed of two left and right cameras at the front end of the drone, and is used to capture the image of the environment around the flight path of the drone. The binocular camera acquires two environmental images by simulating the human eye vision principle, and the environmental image captured by the binocular image is distorted. Therefore, a stereo matching algorithm is needed to calculate and obtain a stereo disparity map. The stereo matching algorithm mainly uses the correspondence between a pair of environmental images acquired by the binocular camera, and obtains a disparity map according to the triangulation principle; after obtaining the disparity information, the depth information of the original environment image can be easily obtained according to the projection model. And three-dimensional information to calculate the stereo disparity map.

The clustering module 302 is configured to cluster the pixels in the stereo disparity map to obtain a plurality of cluster regions.

After obtaining the stereo disparity map of the surrounding environment image, the pixel points may be clustered according to the gray value of each pixel in the stereo disparity map, and the pixel points with the gray value close to each other are divided into one cluster region, thereby The pixel points in the stereo disparity map are divided into a plurality of cluster regions.

The flight distance calculation module 303 is configured to calculate, according to each cluster area, a flight distance of the environment area corresponding to each cluster area to the drone.

Optionally, as a further embodiment, the acquiring module 301 may be specifically configured to:

Calibrating the binocular camera to obtain a Q composed of the focal length, the baseline, and the origin offset of the binocular camera matrix;

Performing image correction on the current environment image acquired by the binocular camera based on the Q matrix to obtain a corrected image; wherein the current environment image acquired by the binocular camera includes two, and thus, two corrections are obtained after correction image.

Performing a stereo matching calculation on the two corrected images to obtain the stereo disparity map;

Wherein, the formula of the Q matrix can be:

Wherein, the T _x represents a baseline of the binocular camera, and f represents a focal length of the binocular camera, (c _x , c _y ), (c′ _x , c′ _y ) respectively for the binocular camera The two imaging origins corrected by the left and right cameras, and the two corrected imaging origins are the same, ie c _x = c' _x , c _y = c' _y .

Optionally, after obtaining the internal and external parameters of the binocular camera by binocular camera calibration, the two environment images acquired by the two cameras may be image corrected by the internal and external parameters of the binocular camera, and the corrected two environment images are obtained. The planes are perfectly parallel aligned to facilitate stereo matching of the two corrected images using a stereo matching algorithm to obtain a stereo disparity map of the current environment image. Optionally, the stereo matching algorithm may calculate the disparity d of each pixel in the environment image acquired by the binocular camera according to the triangulation principle, so that the parallax image can be obtained. After obtaining the disparity information, the depth information and the three-dimensional information of the original environment image can be obtained according to the projection model, thereby obtaining a stereo disparity map.

The flight distance calculation module 303 can be specifically configured to:

Determining position coordinates of target pixel points in each cluster area;

The position coordinate calculation formula can be:

Where (X, Y, Z) is the position coordinate of the environmental region in the three-dimensional coordinate system, and Z may represent each of the poly (x, y) represents the positional coordinates of the target pixel in any cluster area, and d represents the target in any of the cluster areas. The parallax corresponding to the pixel.

The first determining module 304 is configured to determine whether a flight distance of each environment area and the drone is greater than a distance threshold.

a first selection module 305, configured to select any one of the surrounding areas from an environmental area whose flight distance is greater than a distance threshold;

The flying module 306 is configured to calculate a rotation angle of the drone to the any of the flying regions, and control the drone to fly around according to the rotation angle.

Optionally, in some embodiments, the flying module 306 can be specifically configured to:

In this embodiment, by determining the flight distance of the environment area corresponding to each cluster area in the stereo disparity map to the drone, selecting a flying area larger than the distance threshold and performing a flight according to the rotation angle of the flying area, and The environment area with the smallest rotation angle and greater than the threshold of the distance zone can be preferentially used as the flying area to fly around, which not only ensures that the flight path does not change greatly, but also realizes the autonomous autonomous obstacle avoidance of the drone, which greatly improves the flight efficiency. .

Optionally, in another embodiment, the clustering module may include:

a second determining unit, configured to use any pixel point that is not marked as a target pixel point, and sequentially determine, from the adjacent pixel points of the target pixel point, whether the pixel difference between each pixel point and the target pixel point is Within the preset range;

Optionally, as a further embodiment, the second determining unit may be specifically configured to:

4 is a schematic structural diagram of another embodiment of a drone according to an embodiment of the present invention, and the drone may include:

The obtaining module 401 is configured to acquire a stereo disparity map of the current environment image by using a binocular camera;

The clustering module 402 is configured to cluster the pixels in the stereo disparity map to obtain a plurality of cluster regions.

The flight distance calculation module 403 is configured to calculate, according to each cluster area, a flight distance of the environment area corresponding to each cluster area to the drone.

The first determining module 404 is configured to determine whether a flight distance of each environmental area and the drone is greater than a distance threshold.

Optionally, as a further embodiment, after the first determining module 404, the method may further include:

a first selection module 405, configured to select any one of the surrounding areas from an environmental area whose flight distance is greater than a distance threshold;

The first selection module 405 can include:

The second selecting unit 411 is configured to select, if the flight distance of the environment area corresponding to the flight direction of the drone and the drone is less than the distance threshold, select the farthest flight distance from the environmental area where the flight distance is greater than the distance threshold An environmental area is used as a candidate area.

The third determining unit 412 is configured to determine whether the candidate area meets a flight condition; if yes, trigger the determining unit 413; if not, trigger the third selecting unit 414.

The determining unit 413 is configured to use the candidate area as a flying area.

The third selecting unit 414 is configured to select, as the candidate region, an environment region whose flight distance is the farthest from the environment region whose flight distance is greater than the distance threshold and does not include the candidate region, and returns to step 206 to continue performing the corresponding operation.

The flying module 406 is configured to calculate a rotation angle of the drone to the any of the flying regions, and control the drone to fly around according to the rotation angle.

The acquisition module 401 is the same as the acquisition module 301 in the embodiment of FIG. 3, the clustering module 402 is the same as the clustering module 302 in the embodiment of FIG. 3, and the flight distance calculation module 403 and the flight distance calculation in the embodiment of FIG. The module 303 is the same, and the flying module 406 is the same as the flying module 306 in the embodiment of FIG. 3, and details are not described herein again.

In this embodiment, by determining whether the environment area corresponding to the flight direction of the drone is greater than the distance threshold, if Then, the flight continues according to the original route; if it is smaller, an environment region whose flight distance is greater than the distance threshold and the farthest flight distance is preferentially selected as the candidate region, and it is determined that the candidate region satisfies the flight condition, so that the flight can be performed. According to the surrounding environment, the drone can find the optimal flight path to fly around, and realize the autonomous autonomous obstacle avoidance of the drone, which greatly improves the flight efficiency.

In a typical configuration, a computing device includes one or more processors (CPUs), input/output interfaces, network interfaces, and memory.

The memory may include non-persistent memory, random access memory (RAM), and/or non-volatile memory in a computer readable medium, such as read only memory (ROM) or flash memory. Memory is an example of a computer readable medium.

Computer readable media includes both permanent and non-persistent, removable and non-removable media. Information storage can be implemented by any method or technology. The information can be computer readable instructions, data structures, modules of programs, or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read only memory. (ROM), electrically erasable programmable read only memory (EEPROM), flash memory or other memory technology, compact disk read only memory (CD-ROM), digital versatile disk (DVD) or other optical storage, Magnetic tape cartridges, magnetic tape storage or other magnetic storage devices or any other non-transportable media can be used to store information that can be accessed by a computing device. As defined herein, computer readable media does not include non-transitory computer readable media, such as modulated data signals and carrier waves.

Certain terms are used throughout the description and claims to refer to particular components. Those skilled in the art will appreciate that hardware manufacturers may refer to the same component by different nouns. The present specification and the claims do not use the difference in the name as the means for distinguishing the components, but the difference in function of the components as the criterion for distinguishing. The word "comprising" as used throughout the specification and claims is an open term and should be interpreted as "including but not limited to". "Substantially" means that within the range of acceptable errors, those skilled in the art will be able to solve the technical problems within a certain error range, substantially achieving the technical effects. In addition, the term "coupled" is used herein to include any direct and indirect electrical coupling means. Therefore, if a first device is coupled to a second device, the first device can be directly electrically coupled to the second device, or electrically coupled indirectly through other devices or coupling means. Connected to the second device. The description of the present invention is intended to be illustrative of the preferred embodiments of the invention. The scope of the invention is defined by the appended claims.

It should also be noted that the terms "including", "comprising" or "comprising" or any other variations thereof are intended to encompass a non-exclusive inclusion, such that the item or system comprising a plurality of elements includes not only those elements but also Its His elements, or include elements inherent to such goods or systems. In the absence of more restrictions, elements defined by the phrase "including one..." do not exclude the existence of additional identical elements in the item or system that includes the element.

The above description illustrates and describes several preferred embodiments of the present invention, but as described above, it should be understood that the invention is not limited to the forms disclosed herein, and should not be construed as Other combinations, modifications, and environments are possible and can be modified by the above teachings or related art or knowledge within the scope of the application concept described herein. All changes and modifications made by those skilled in the art are intended to be within the scope of the appended claims.

Claims

A method for obstacle avoidance of a drone, characterized in that it comprises:

Obtaining a stereo disparity map of the current environment image through the binocular camera;

Clustering pixel points in the stereo disparity map to obtain a plurality of cluster regions;

Calculating a flight distance of the environmental area corresponding to each cluster area to the drone based on each cluster area;

Determining whether the flight distance of each environmental area and the drone is greater than a distance threshold;

Selecting any one of the flying areas from an environmental area whose flight distance is greater than the distance threshold;

Calculating a rotation angle of the drone to the any of the flying regions, and controlling the drone to fly around according to the rotation angle.
The method according to claim 1, wherein the clustering the pixel points in the stereo disparity map to obtain a plurality of cluster regions comprises:

Taking any unmarked pixel as the target pixel, starting from the adjacent pixel of the target pixel, sequentially determining whether the pixel difference between each pixel and the target pixel is within a preset range;

Pixel points in the adjacent area of the target pixel that are different from the pixel of the target pixel in a preset range are marked as belonging to the same cluster area as the target pixel.
The method according to claim 2, wherein the unmarked pixel point is used as a target pixel point, and each pixel point and the target are sequentially determined from adjacent pixel points of the target pixel point. Whether the pixel difference of the pixel is within a preset range includes:

Taking any unmarked pixel as the target pixel, starting from each adjacent pixel of the target pixel, sequentially determining whether the pixel difference between each successive pixel and the target pixel is in advance It is assumed that the pixel difference value up to any one of the pixel points and the target pixel point is not within the preset range.
The method according to claim 1, wherein the obtaining a stereo disparity map of the current environment image by the binocular camera comprises:

Calibrating the binocular camera to obtain a Q matrix composed of a focal length, a baseline, and an origin offset of the binocular camera;

Performing image correction on the current environment image acquired by the binocular camera based on the Q matrix to obtain a corrected image;

Performing stereo matching calculation on the corrected image to obtain the stereo disparity map;

Calculating, according to each cluster area, a flight distance of the environment area corresponding to each cluster area to the drone includes:

Determining position coordinates of target pixel points in each cluster area;

Calculating, according to the target pixel point of each clustering region, the position coordinates of the environment region corresponding to each of the cluster regions in the three-dimensional coordinate system according to the following coordinate formula;

The position coordinate calculation formula is:

Where (X, Y, Z) is the position coordinate of the environmental region in the three-dimensional coordinate system, and Z represents the flight distance of the environmental region corresponding to each of the cluster regions to the drone, and T x represents a baseline of the binocular camera, f denotes a focal length of the binocular camera, (x, y) denotes a position coordinate of the target pixel point in any cluster region, and d denotes the one in the any cluster region The parallax corresponding to the target pixel, (c x , c y ) is the imaging origin after the binocular camera is corrected.
The method according to claim 1, wherein the selecting one of the environment regions from the flight distance greater than the distance threshold comprises:

If the flight distance corresponding to the flight direction of the drone and the drone is less than the distance threshold, select an environmental region that is the farthest from the flight region with the flight distance greater than the distance threshold as the candidate region;

Determining whether the candidate area satisfies flight conditions;

If yes, the candidate area is used as a flying area;

If not, an environment region whose flight distance is the farthest from the flight distance greater than the distance threshold and not including the candidate region is selected as the candidate region, and the step of determining whether the candidate region satisfies the flight condition is continued.
The method of claim 5, further comprising:

If the flight area corresponding to the flight direction of the drone and the flight distance of the drone are greater than the distance threshold, the drone is controlled to fly according to the original route.
The method of claim 5, further comprising:

If none of the candidate regions meet the flight conditions, the drone is controlled to hover.
The method according to claim 4, wherein said calculating a rotation angle of said drone to said one of said flying regions, and controlling said drone to fly around said rotation angle, including :

Calculating an angle of rotation of the drone in the flying area according to position coordinates corresponding to the flying area and controlling the drone to fly according to the rotation angle.
A drone, characterized in that it comprises:

An acquisition module, configured to acquire a stereo disparity map of the current environment image by using a binocular camera;

a clustering module, configured to cluster pixel points in the stereo disparity map to obtain a plurality of cluster regions;

a flight distance calculation module, configured to calculate a flight distance of the environmental region corresponding to each cluster region to the drone based on each cluster region;

a first determining module, configured to determine whether a flight distance of each environmental area and the drone is greater than a distance threshold;

a first selection module, configured to select any one of the surrounding areas from an environmental area whose flight distance is greater than a distance threshold;

The flying module is configured to calculate a rotation angle of the drone to the any of the flying regions, and control the drone to fly around according to the rotation angle.
The drone according to claim 9, wherein the clustering module comprises:

a second determining unit, configured to use any pixel point that is not marked as a target pixel point, and sequentially determine, from the adjacent pixel points of the target pixel point, whether the pixel difference between each pixel point and the target pixel point is Within the preset range;

And a marking unit, configured to mark a pixel difference between the adjacent pixel of the target pixel and the pixel of the target pixel in a preset range, and mark the same cluster area as the target pixel.
The UAV according to claim 10, wherein the second determining unit is specifically configured to:

Taking any unmarked pixel as the target pixel, starting from each adjacent pixel of the target pixel, sequentially determining whether the pixel difference between each successive pixel and the target pixel is in advance It is assumed that the pixel difference value up to any one of the pixel points and the target pixel point is not within the preset range.
The UAV according to claim 9, wherein the first acquisition module is specifically configured to:

Calibrating the binocular camera to obtain a Q matrix composed of a focal length, a baseline, and an origin offset of the binocular camera;

Performing image correction on the current environment image acquired by the binocular camera based on the Q matrix to obtain a corrected image;

Performing stereo matching calculation on the corrected image to obtain the stereo disparity map;

The flight distance calculation module is specifically configured to:

Determining position coordinates of target pixel points in each cluster area;

Calculating, according to the target pixel point of each clustering region, the position coordinates of the environment region corresponding to each of the cluster regions in the three-dimensional coordinate system according to the following coordinate formula;

The position coordinate calculation formula is:

Where (X, Y, Z) is the position coordinate of the environmental region in the three-dimensional coordinate system, and Z represents the flight distance of the environmental region corresponding to each of the cluster regions to the drone, and T x represents a baseline of the binocular camera, f denotes a focal length of the binocular camera, (x, y) denotes a position coordinate of the target pixel point in any cluster region, and d denotes the one in the any cluster region The parallax corresponding to the target pixel, (c x , c y ) is the imaging origin after the binocular camera is corrected.
The UAV according to claim 9, wherein the first selection module comprises: a second selection unit, a third determination unit, a determination unit, and a third selection unit;

The second selecting unit is configured to select a farthest flight distance from an environment region where the flight distance is greater than the distance threshold if the flight distance corresponding to the flight direction of the drone is less than the distance threshold An environmental area as a candidate area;

The third determining unit is configured to determine whether the candidate area meets a flight condition; if yes, trigger the determining unit; if not, trigger the third selecting unit;

The determining unit is configured to use the candidate area as a flying area;

The third selection unit is configured to select, as a candidate region, an environment region whose flight distance is the farthest from the environment region where the flight distance is greater than the distance threshold and does not include the candidate region, and return to determine whether the candidate region meets the flight condition. The steps continue.
The drone according to claim 13, further comprising:

The first control module is configured to control the drone to fly according to the original route if the flight area corresponding to the flight direction of the drone and the flight distance of the drone are greater than a distance threshold.
The drone according to claim 13, further comprising:

And a second control module, configured to control the drone to hover if none of the candidate regions meet the flight condition.
The drone according to claim 12, wherein the flying module is specifically configured to:

Calculating an angle of rotation of the drone in the flying area according to position coordinates corresponding to the flying area and controlling the drone to fly according to the rotation angle.