WO2021146969A1

WO2021146969A1 - Distance measurement method, movable platform, device, and storage medium

Info

Publication number: WO2021146969A1
Application number: PCT/CN2020/073656
Authority: WO
Inventors: 刘宝恩; 李鑫超; 王涛
Original assignee: 深圳市大疆创新科技有限公司
Priority date: 2020-01-21
Filing date: 2020-01-21
Publication date: 2021-07-29
Also published as: CN112639881A

Abstract

Disclosed are a distance measurement method, a movable platform, a device, and a storage medium. The method comprises: obtaining a first image of airspace above an unmanned aerial vehicle; after performing semantic recognition on the first image, obtaining a target region affecting the flight of the unmanned aerial vehicle; then controlling the unmanned aerial vehicle to move a preset distance towards a designated direction so as to obtain a second image of the airspace above the unmanned aerial vehicle; then, matching the pixels of the target region in the first image with those in the second image to obtain matched pixels; and finally, determining a distance between the target region and the unmanned aerial vehicle according to the matched pixels, and further controlling a flight state of the unmanned aerial vehicle according to the distance. In distance measurement by means of image recognition, the used image is captured by a camera necessarily provided on the unmanned aerial vehicle, and therefore, distance measurement can also be achieved by the unmanned aerial vehicle on which a depth sensor is not configured.

Description

Distance measurement method, movable platform, equipment and storage medium

Technical field

The present invention relates to the technical field of image recognition, in particular to a distance measurement method, a movable platform, equipment and storage medium.

Background technique

UAV is an unmanned aerial vehicle operated by radio remote control equipment and self-provided program control device. Compared with manned aircraft, it has the characteristics of small size and low cost. It has been widely used in many fields, such as street scene shooting, power inspection, traffic monitoring, post-disaster rescue and so on.

In all stages of UAV flight, obstacles need to be avoided. Especially in the return phase of the drone, during this stage, the drone has an ascending flight. Therefore, it is necessary to detect the distance between the drone and the area above the drone that affects its flight to determine this. Whether the area has an impact on the UAV's ascent flight, where obstacles are usually contained in this area.

In the prior art, the distance measurement is usually realized by using a distance measuring sensor, such as a lidar, which is configured on the UAV. However, when the UAV is not equipped with a ranging sensor, the distance measurement cannot be achieved, so that the UAV has a greater risk of damage during the return process.

Summary of the invention

The invention provides a distance measurement method, a movable platform, equipment and a storage medium, which are used to enable an unmanned aerial vehicle not equipped with a distance measuring sensor to realize distance measurement and ensure measurement accuracy.

The first aspect of the present invention is to provide a distance measurement method, which includes:

Acquiring a first image of the airspace above the drone;

In the first image, determine the target area that affects the flight of the drone according to the semantic category of the pixels in the first image;

Respond to the flight instruction, so that the UAV moves a preset distance in a designated direction to obtain a second image of the airspace above the UAV;

Matching the pixel points of the target area in the first image and the second image to obtain matching pixels;

The distance between the target area and the drone is determined according to the matching pixel points.

The second aspect of the present invention is to provide a movable platform, the movable platform includes: a body, a power system and a control device;

The power system is arranged on the body and used to provide power for the movable platform;

The control device includes a memory and a processor;

The memory is used to store a computer program;

The processor is configured to run a computer program stored in the memory to realize:

Acquiring a first image of the airspace above the drone;

The third aspect of the present invention is to provide a distance measuring device, which includes:

Memory, used to store computer programs;

Acquiring a first image of the airspace above the drone;

The fourth aspect of the present invention is to provide a computer-readable storage medium, the storage medium is a computer-readable storage medium, the computer-readable storage medium stores program instructions, and the program instructions are used in the first aspect. The distance measurement method described.

The distance measurement method, movable platform, equipment, and storage medium provided by the present invention first obtain the first image of the airspace above the drone, and then perform semantic recognition on the first image, so as to determine the influence of no one in the first image. The target area for the aircraft to fly. Then, the drone is controlled to move a preset distance in a designated direction to obtain a second image of the airspace above the drone, that is, the first image and the second image are shot at different locations. Then, the pixel points of the target area in the first image and the second image are matched to obtain matched pixels, and the distance between the target area and the drone is determined according to the matched pixels. Furthermore, it is possible to control the flight status of the drone, namely returning home or hovering in place according to this distance.

According to the above description, compared with the distance measurement method using a depth sensor in the prior art, the present invention provides a method for realizing distance measurement through image recognition. Since the camera is an indispensable device to ensure the normal execution of the task of the UAV, the measurement method provided by the present invention will not affect the size and cost of the UAV while ensuring the accuracy of the measurement. In addition, using the measurement method provided by the present invention, distance measurement can also be achieved for drones that are not equipped with a depth sensor.

Description of the drawings

The drawings described here are used to provide a further understanding of the application and constitute a part of the application. The exemplary embodiments and descriptions of the application are used to explain the application, and do not constitute an improper limitation of the application. In the attached picture:

FIG. 1 is a schematic flowchart of a distance measurement method provided by an embodiment of the present invention;

FIG. 2 is a schematic structural diagram of the UAV according to an embodiment of the present invention when its configured PTZ is in different states;

FIG. 3 is a schematic flowchart of a second image acquisition manner provided by an embodiment of the present invention;

4 is a schematic diagram of an annular field of view corresponding to an image provided by an embodiment of the present invention;

FIG. 5 is a schematic flowchart of a first image acquisition manner according to an embodiment of the present invention;

6 is a schematic flowchart of a method for measuring the distance between a drone and a target area provided by an embodiment of the present invention;

FIG. 7 is a positional relationship of various parameters in a measurement method provided by an embodiment of the present invention as shown in FIG. 6; FIG.

FIG. 8 shows the positional relationship of various parameters in another measurement method provided by the embodiment of the present invention as shown in FIG. 6; FIG.

FIG. 9 is a schematic structural diagram of a distance measuring device provided by an embodiment of the present invention;

FIG. 10 is a schematic structural diagram of a movable platform provided by an embodiment of the present invention;

FIG. 11 is a schematic structural diagram of a distance measuring device provided by an embodiment of the present invention.

Detailed ways

In order to make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the following will clearly and completely describe the technical solutions in the embodiments of the present invention with reference to the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments These are a part of the embodiments of the present invention, but not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative work shall fall within the protection scope of the present invention.

Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by those skilled in the technical field of the present invention. The terms used in the specification of the present invention herein are only for the purpose of describing specific embodiments, and are not intended to limit the present invention.

The distance measurement method provided by the present invention is to measure the distance between the target area above the drone that affects the flight of the drone and the drone, and this distance measurement is particularly important in the automatic return process of the drone.

Specifically, when the drone completes the flight mission or encounters harsh natural environments such as protruding mountains during the flight, or the communication connection with the ground base station is disconnected, in order to ensure the safety of the drone, avoid In the event of a damage accident, the UAV often needs to turn on and return home automatically. And since there is an ascending flight phase during the automatic return of the drone, determining the distance between the drone and the target area becomes an important condition for judging whether the drone can return automatically. At this time, the obstacle distance measurement method provided by each embodiment of the present invention can be used to realize distance measurement.

In the following, some embodiments of the present invention will be described in detail with reference to the accompanying drawings. As long as there is no conflict between the embodiments, the following embodiments and the features in the embodiments can be combined with each other.

FIG. 1 is a schematic flowchart of a distance measurement method provided by an embodiment of the present invention. The main body of the distance measurement method is the measurement equipment. It can be understood that the measurement device can be implemented as software or a combination of software and hardware. The measurement equipment implements the distance measurement method to realize the measurement of the distance between the UAV and the target area that affects the unmanned flight. The measurement equipment in this embodiment and the following embodiments may specifically be movable platforms, such as unmanned aerial vehicles, unmanned vehicles, unmanned ships, and so on. The following embodiments will be described by taking a drone as an execution subject as an example.

Specifically, the method may include:

S101: Obtain a first image of the airspace above the drone.

During the flight of the UAV, its own camera can take the first image at the current location of the UAV. Optionally, the camera configured on the drone may be a monocular camera. This camera can be placed on a platform that can be lifted upwards. When the gimbal is in the raised state, the first image of the airspace above the drone can be taken. The non-raised and raised state of the pan/tilt can be shown in Figure 2. At this time, the angle of view corresponding to the first image may be the same as the angle of view of the monocular camera.

S102. In the first image, determine the target area that affects the flight of the drone according to the semantic category of the pixels in the first image.

After obtaining the first image, the drone can identify the semantic category to which each pixel in the first image belongs, that is, perform pixel-level semantic recognition on the first image. Optionally, this kind of semantic recognition can be accomplished with the aid of a neural network model, and the specific recognition process can be referred to the following related descriptions. Optionally, the above neural network model can realize two classifications, that is, distinguish the sky and obstacles in the first image; it can also realize multi-classification, that is, distinguish the sky, trees, buildings, etc. in the first image. . Furthermore, the target area affecting the flight of the drone is determined in the first image according to the semantic category of each pixel. If the target area contains obstacles, it will affect the normal ascent of the drone.

When the neural network model can realize the two classification, an alternative way is to determine the pixels used to describe the sky as the target pixels, and the target pixels constitute the target area.

It is easy to understand that the location of obstacles adjacent to the sky will also affect the flight of the drone. Another alternative is to use the first image to describe the sky and the obstacles adjacent to the sky. Determine the pixel point as the target pixel point, and the target pixel point constitutes the target area.

In addition, in practical applications, the selection of the target area can also take into account the flying environment of the drone and/or the size of the drone. In another alternative, you can first select the target area. The category to which each pixel in the image belongs determines the candidate area, where the candidate area may be composed of pixels used to describe the sky and the obstacles closest to the sky. Then, according to the size of the drone and/or the distribution of obstacles in the current flight environment of the drone, the range of the candidate area is adjusted. For example, when the size of the drone is small or the obstacles in the flying environment are sparsely distributed, the candidate area can be reduced by a preset multiple to obtain the target area; otherwise, the candidate area can be expanded Preset multiples to get the target area.

S103: Respond to the flight instruction to make the drone move a preset distance in a specified direction to obtain a second image of the airspace above the drone.

Then, in response to the flight control instruction, the drone can fly a preset distance from the current position in the designated direction to another position, and at this other position, a second image of the airspace above the drone can be taken. For the clarity of the subsequent description, the current position of the drone can be called the first position, and the first image is taken at the first position; the other position is called the second position, and the second image is taken at the second position. have to.

Wherein, optionally, the above-mentioned designated direction may be upward, that is, after the first image is taken at the first position, the drone may respond to the received ascending flight control instruction to fly from the first position. To the second position, the monocular camera can take a second image at the second position. In practical applications, the difference between the first position and the second position is usually small, for example, a few centimeters. In addition, the ascent flight control command can be generated autonomously by the drone, or sent to the drone by the pilot through the control device.

S104, matching the pixel points of the target area in the first image and the second image to obtain matching pixels.

After obtaining the second image, the drone can match the pixel points in the target area of the first image with the pixel points in the second image to obtain matching pixels, that is, to obtain at least one matching pixel pair. As described in step 103, since the distance between the first position and the second position is usually small, the objects contained in the first image and the second image are usually the same, but there will be a slight difference in position. At this time, for the two pixels contained in any matched pixel pair, they describe the same object. Optionally, whether the pixels are matched or not can be reflected by the similarity between the pixels.

S105: Determine the distance between the target area and the drone according to the matching pixels.

After the matched pixel points are obtained, optionally, the established ranging model can be preset to realize the ranging, or the triangular ranging principle can be used to realize the ranging. For the specific implementation of the ranging, refer to the following embodiments shown in Figs. 6 to 8.

In the distance measurement method provided in this embodiment, the first image of the airspace above the drone is acquired, and after semantic recognition is performed on it, the target area that affects the flight of the drone can be obtained. Then control the drone to move a preset distance in a designated direction to obtain a second image of the airspace above the drone. Then, the pixel points of the target area in the first image and the second image are matched to obtain matching pixels. Finally, the distance between the target area and the drone is determined according to the matching pixels, and further, the flight status of the drone can be controlled according to this distance. It can be seen that the present invention provides a method for realizing obstacle distance measurement through image recognition. Since the image to be recognized is taken by the necessary camera of the drone itself, it will not affect the size and cost of the drone. At the same time, this method can also be used to measure distances for drones that are not equipped with depth sensors to ensure the accuracy of the measurement.

Regarding step 102 of the embodiment shown in FIG. 1, the above mentioned using the neural network model to perform pixel-level semantic recognition of the first image, and the semantic recognition process can be described in detail below:

The neural network model may specifically be a convolutional neural network (Convolutional Neural Networks, CNN) model. The neural network model can include multiple computing nodes. Each computing node can include a convolution (Conv) layer, batch normalization (BN), and an activation function ReLU. The computing nodes can use skip connection (Skip Connection). ) Way to connect.

The input data of K×H×W can be input into the neural network model, and after the neural network model is processed, the output data of C×H×W can be obtained. Among them, K can represent the number of input channels, and K can be equal to 4, corresponding to the four channels of red (R, red), green (G, green), blue (B, blue) and depth (D, deep) respectively; H can represent the height of the input image (that is, the first image), W can represent the width of the input image, and C can represent the number of categories.

It should be noted that when the input image is too large, an input image can be cut into N sub-images. Correspondingly, the input data can be N×K×H'×W', and the output data can be N×C×H' ×W', where H'can represent the height of the sub-image, and W'can represent the width of the sub-image. Of course, in other embodiments, the feature map may also be obtained in other ways, which is not limited in this application.

Using the above-mentioned pre-trained neural network model to process the environment image to obtain a feature map may specifically include the following steps:

Step 1. Input the environment image into the neural network model to obtain the model output result of the neural network model.

Among them, the model output result of the neural network model may include the confidence feature maps output by multiple output channels, and the multiple output channels can correspond to multiple object categories one-to-one, and the pixel values of the confidence feature maps of a single object category are used To characterize the probability that a pixel is an object category.

Step 2: According to the model output result of the neural network model, a feature map containing semantic information is obtained.

The object category corresponding to the confidence feature map with the largest pixel value at the same pixel location in the multiple confidence feature maps one-to-one corresponding to the multiple output channels may be used as the object category of the pixel location to obtain the feature map.

Suppose that the number of output channels of the neural network model is 4, and the output result of each channel is a confidence feature map, that is, the 4 confidence feature maps are the confidence feature map 1 to the confidence feature map 4, and the confidence The degree characteristic map 1 corresponds to the sky, the confidence characteristic map 2 corresponds to buildings, the confidence characteristic map 3 corresponds to trees, and the confidence characteristic map 4 corresponds to "other". In these categories, except for the sky, the rest can be regarded as obstacles.

For example, when the pixel value at the pixel location (100, 100) in the confidence feature map 1 is 70, the pixel value at the pixel location (100, 100) in the confidence feature map 2 is 50, and the pixel at the pixel location (100, 100) in the confidence feature map 3 When the value is 20, and the pixel value of the pixel position (100, 100) in the confidence feature map 4 is 20, it can be determined that the pixel position (100, 100) is the sky.

For another example, when the pixel value at the pixel location (100, 80) in the confidence feature map 1 is 20, the pixel value at the pixel location (100, 80) in the confidence feature map 2 is 30, and the pixel location in the confidence feature map 3 When the pixel value of (100,80) is 20, and the pixel value of pixel position (100,80) in the confidence feature figure 4 is 70, it can be determined that the pixel position (100,80) is other, that is, it is not trees, buildings, and Any of the trees.

At the same time, for step 104 of the embodiment shown in FIG. 1, the above has also provided a pixel matching method, that is, first calculate the similarity between each pixel in the target area and each pixel in the second image, and then At least one pair of matching pixels is obtained according to the similarity. However, since the target area and the second image each contain a large number of pixels, this results in a large amount of calculation in the matching process, which will occupy more computing resources of the drone, and also make the matching efficiency inefficient. high.

Therefore, in order to avoid the above-mentioned problems, optionally, the drone can use a feature point detection algorithm pre-configured in itself to extract the target area and feature pixel points in the second image respectively. Among them, the detected characteristic pixel points are usually corner points in the image. Optionally, the aforementioned feature point detection algorithm may be a Scale-invariant Feature Transform (SIFT) algorithm, a Speeded-Up Robust Features (SURF) algorithm, or a binary robust independent feature. (Binary Robust Independent Elementary Features, BRIEF for short) algorithm and so on. In practical applications, while obtaining a characteristic pixel according to the above algorithm, a descriptor used to describe the attribute of the characteristic pixel can also be obtained, and this descriptor can be expressed in a vector form.

Then, the drone can perform matching processing on the target area and the characteristic pixel points in the second image to obtain at least one matching pixel pair. For any matched pixel pair, the two characteristic pixels in it also describe the same object.

Optionally, whether the characteristic pixels are matched can also be reflected by the similarity between the characteristic pixels. Specifically, the similarity between pixels may be the similarity between the descriptors of characteristic pixels. If the similarity is greater than or equal to the preset threshold, it is determined that the two characteristic pixel points match, so as to form a pair of matched pixels from the two.

For the similarity between the descriptors, optionally, the feature pixel points obtained by using different feature point detection algorithms may use different similarity calculation methods. For example, for the feature pixels obtained by the BRIEF algorithm, the similarity can be obtained by calculating the Hamming distance between the feature pixels; for the descriptor obtained by the SIFT algorithm or the SURF algorithm, the description corresponding to the feature pixel can be calculated The Euclidean distance between the children to get the similarity.

In summary, after extracting the feature pixels of the target area and the second image, the number of pixels for matching can be greatly reduced, which also greatly reduces the amount of calculation in the matching process to ensure matching efficiency.

In addition, for the method of acquiring the first image, a method has been provided in step 101 of the embodiment shown in FIG. 1. At this time, the field of view corresponding to the first image is the same as the field of view of the monocular camera, and both are smaller. of. It is easy to understand that the larger the field of view corresponding to the first image, the more comprehensive the description of the airspace above the drone. The first image with this large field of view can more accurately calculate the target area and the unmanned area. The distance between the drones can more accurately control the automatic return of the drone.

Based on this, as shown in FIG. 3, another optional first image acquisition manner, that is, step 101, an optional implementation manner may be:

S1011: Respond to the first flight control instruction to make the UAV rotate and fly one circle in situ at the first position.

S1012: Obtain the first image taken by the monocular camera of the drone during the rotating flight of the drone.

When the drone is hovering in the first position, it responds to the first flight control command, so that the drone rotates and flies once in the first position. During this rotating flight, the monocular camera on the drone can take the first image corresponding to the circular field of view. This annular field of view can be shown in Figure 4.

For the acquisition method of the second image, a method has been provided in step 103 of the embodiment shown in FIG. 1. However, in this acquisition method, the drone has an ascending flight process, and this ascending process can easily cause the drone to fall. Enter the target area and collide with obstacles in the target area, or even cause the drone to be damaged. Therefore, in order to avoid the above situation, as shown in FIG. 5, another optional second image acquisition manner, that is, step 103, an optional implementation manner may be:

S1031: Respond to the second flight control instruction to cause the drone to descend and fly a preset distance from the first position to the second position.

S1032: When the drone is in the second position, acquire a second image taken by the monocular camera.

When the drone is hovering in the first position, the monocular camera on the drone can take an image of the airspace above the drone, which is the first image. In response to the received second flight control command, the drone will descend and fly from the current first position to the second position. At this time, the monocular camera configured on the drone will again take an image of the airspace above the drone, which is the second image. Through the above method, the drone has obtained the first image and the second image. Through this method of descending flight, not only can the images taken at different heights be obtained, but it can also prevent the drone from falling into the target area above itself. At this time, the field of view corresponding to the second image may be the same as the field of view of the monocular camera.

The field of view of the monocular camera is usually small. Similar to the description in the embodiment shown in Figure 3, the larger the field of view corresponding to the second image, the more comprehensive the description of the airspace above the drone by the second image. The second image with this large field of view can also more accurately calculate the distance between the target area and the drone, and can more accurately control the drone to return home.

Therefore, after the drone responds to the second flight control command to fly down to the second position, it can also respond to the third flight control command, so that the drone can rotate and fly in the second position for one full circle, and obtain the The second image taken by the monocular camera during the rotating flight of a person on the plane. After rotating and flying, the second image obtained corresponds to the circular field of view in the airspace above the drone. This annular field of view can also be shown in Figure 4.

Combining the embodiments shown in Figures 3 to 5, the drone can obtain the first image and the second image with a circular field of view, so as to more accurately calculate the relationship between the target area and the drone based on the image of the large field of view. The distance between the two can control the drone to return home more accurately.

According to the embodiments shown in FIGS. 3 to 5 above, it can be seen that the drone can take a first image at the first position, and after a descending flight, it can obtain a second image taken at the second position. Based on this method of obtaining the first image and the second image through descending flight, after the UAV carries out pixel matching, as shown in Figure 6, an option is to determine the difference between the UAV and the target area based on the matched pixel pair. The distance between them, that is, an optional implementation of step 105 can be:

S1051: Determine a first distance between the first pixel point and the image center of the first image.

S1052: Determine a second distance between the second pixel point and the image center of the second image.

S1053: Determine the distance between the target area and the drone according to the preset distance, the camera parameters of the monocular camera, the first distance, and the second distance.

After step 104, the drone can already obtain at least one matched pixel pair, where any matched pixel pair may consist of a first pixel in the first image and a second pixel in the second image. The drone can use any matching pixel pair to determine the distance between the drone and the target area.

Assuming that any matching pixel pair A includes a first pixel A ₁ and a second pixel A ₂ , the image center of the first image including the first pixel A ₁ _{is O 1} , and the first pixel including the second pixel A ₂ The image center of the second image is O ₂ . At this time, the distance between the target area and the drone can be calculated according to the following formula:

After finishing:

Among them, the specific positional relationship of the parameters in the formula can be as shown in Figure 7. Specifically, A ₀ is the target area, x is the horizontal distance between the drone and the target area, that is, the line segment O ₀ A ₀ , and z is the vertical distance between the drone and the target area, that is, the line segment O ₀ P ₁ , O ₀ is the optical center of the monocular camera configured on the UAV. d ₀ is the distance between the _{first position P 1} where the first image is captured and the second position P _{2 where the second image is captured.} d ₁ is the distance between the first pixel A ₁ and the image center O _{1 of the first image.} d ₂ is the distance between the second pixel A ₂ and the image center O _{2 of the second image.} f is the focal length of the monocular camera, that is, the line segment O ₁ P _{1 in the} figure, that is, the line segment O ₂ P ₂ .

Among them, the drone can first obtain _{the pixel coordinates of the first pixel A 1} and the image center O _{1 of} the first image in the first image. Then, the first distance d ₁ _{between the first pixel point A 1} and the image center O ₁ is determined according to the pixel coordinates of the two. The calculation method of the second distance d ₂ is also similar, and will not be repeated here.

According to the above description, the distance between the target area and the drone can be determined by using any matching pixel pair. However, in order to ensure the accuracy of distance measurement, optionally, the above calculation can be performed on multiple matched pixel pairs to obtain multiple distances, and the distance between the drone and the target area can be determined based on the multiple distances. . For example, the average or median of multiple distances is determined as the distance between the drone and the target area.

In addition, as shown in the implementation method provided in Figure 1, the drone can take a first image at the first position, and then after an ascending flight, obtain a second image taken at the second position. For this method of obtaining the first image and the second image by ascending flight, the above-mentioned method can also be used to realize distance measurement. However, the above-mentioned first position P ₁ , the image center O ₁ of the first image, the first pixel A ₁ and the second position P ₂ , the image center O ₂ of the second image, and the second pixel A _{2 are} between The positional relationship becomes as shown in Figure 8.

At this time, the following formula can be used to calculate the distance between the target area and the drone:

After finishing:

Among them, A ₀ is the target area, x is the horizontal distance between the drone and the target area, that is, the line segment O ₀ A ₀ , and z is the vertical distance between the drone and the target area, that is, the line segment O ₀ P ₁ , O ₀ It is the optical center of the monocular camera configured on the UAV. d ₀ is the distance between the _{first position P 1} where the first image is captured and the second position P _{2 where the second image is captured.} d ₁ is the distance between the first characteristic pixel A ₁ and the image center O _{1 of the first image.} d ₂ is the distance between the second characteristic pixel A ₂ and the image center O _{2 of the second image.} f is the focal length of the monocular camera, that is, the line segment O ₁ P _{1 in the} figure, that is, the line segment O ₂ P ₂ .

For content not described in detail here, reference may be made to related descriptions in the embodiment shown in FIG. 7, and details are not described herein again.

It should be noted that, according to the above description, pixel point matching can also be performed on the characteristic pixel points in the first image and the second image, and any matching pixel pair obtained at this time contains the first image in the first image. A characteristic pixel and a second characteristic pixel in the second image. Then the first distance between the first characteristic pixel point and the image center of the first image can be further determined, and the second distance between the second characteristic pixel point and the image center of the first image can be determined, and then use Figure 7 or Figure 7 The method shown in 8 calculates the distance between the target area and the drone.

Since the number of characteristic pixels in an image is much smaller than the number of all pixels, using at least one matching pixel pair containing characteristic pixels to achieve distance measurement requires less calculation and higher calculation efficiency.

After the distance between the drone and the target area is obtained by using the distance measurement methods provided in the foregoing embodiments, the movement of the drone can be controlled according to the distance. Specifically, if this distance meets the preset conditions, it indicates that the target area above the drone is far, and the ascending flight phase of the drone during the return home process will not be affected. At this time, the drone can control the return home Respond to instructions to control the drone to return home automatically. Otherwise, control the drone to continue hovering.

In addition, on the basis of the foregoing embodiment, when it is determined that the distance between the drone and the target area meets the preset conditions, the unmanned return can be controlled. It is easy to understand that any flight process of a drone requires battery power. Therefore, before controlling the drone to return to home, you can also determine the power required during the return process. If the current remaining power is more than the return point When power is needed, the drone will be controlled to return home.

As for the determination of the power required during the return home process, an alternative way is to first estimate the wind speed information from the current position to the destination of the return home based on the historical wind speed information. Then determine the ground speed information for landing from the current position to the return destination, so as to determine the power required for the drone's return process based on the wind speed information and ground speed information.

At the same time, after obtaining the distance between the drone and the target area, the point cloud data corresponding to the target area can be further obtained. This point cloud data can describe the flight environment in which the drone is located. This point cloud data can be used to plan the return path for the drone to ensure that the drone can return to home automatically and safely according to this path.

FIG. 9 is a schematic structural diagram of a distance measuring device provided by an embodiment of the present invention; referring to FIG. 9, this embodiment provides a distance measuring device, which can execute the above-mentioned distance measuring method; specifically, Distance measuring devices include:

The acquiring module 11 is configured to acquire a first image of the airspace above the drone.

The area determining module 12 is configured to determine, in the first image, a target area that affects the flight of the drone according to the semantic category of pixels in the first image.

The response module 13 is configured to respond to flight instructions, so that the UAV moves a predetermined distance in a designated direction, and obtains a second image of the airspace above the UAV.

The matching module 14 is configured to match the pixel points of the target area in the first image and the second image to obtain matching pixels;

The distance determining module 15 is configured to determine the distance between the target area and the UAV according to the matching pixel points.

The device shown in FIG. 9 can also execute the methods of the embodiments shown in FIG. 1 to FIG. 8. For parts that are not described in detail in this embodiment, please refer to the related description of the embodiments shown in FIG. 1 to FIG. 8. For the implementation process and technical effects of this technical solution, please refer to the description in the embodiment shown in FIG. 1 to FIG. 8, which will not be repeated here.

FIG. 10 is a schematic structural diagram of a movable platform provided by an embodiment of the present invention; referring to FIG. 10, an embodiment of the present invention provides a movable platform, and the movable platform is at least one of the following: Aircraft, unmanned ships, unmanned vehicles; specifically, the movable platform includes: a body 21, a power system 22, and a control device 23.

The power system 22 is arranged on the body 21 and is used to provide power for the movable platform.

The control device 23 includes a memory 231 and a processor 232.

The memory is used to store a computer program;

Acquiring a first image of the airspace above the drone;

Further, the movable platform also includes a monocular camera 24, which is arranged on the body 21;

The processor 232 is further configured to: respond to the first flight control instruction, so that the UAV rotates and flies once at the first position;

During the rotating flight of the drone, the first image captured by the monocular camera configured on the drone is acquired.

Further, the processor 232 is further configured to: respond to a second flight control instruction to cause the drone to descend and fly a preset distance from the first position to a second position;

When the drone is located at the second position, the second image captured by the monocular camera is acquired.

Further, the monocular camera 24 is placed on a pan-tilt that can swing upward, so that the monocular camera 24 can capture the first image and the second image;

The processor 232 is further configured to: respond to a third flight control instruction, so that the UAV rotates and flies in situ at the second position for one circle;

Acquire the second image taken by the monocular camera during the unmanned aircraft rotating flight, wherein the first image and the second image correspond to the ring shape in the airspace above the drone Field of view.

Further, the processor 232 is further configured to: identify the semantic category of pixels in the first image;

According to the semantic category of the pixel point, the target area in the first image is determined.

Further, the semantic category of the pixel includes sky and obstacles, and the processor 232 is further configured to: according to the semantic category of the pixel, determine the description of the sky and the obstacle closest to the sky in the first image. Target pixels, the target area is formed by the target pixels.

Further, the processor 232 is further configured to: determine a candidate area in the first image according to the semantic category of the pixel;

According to the volume of the unmanned aerial vehicle and/or the distribution of obstacles in the flying environment of the unmanned aerial vehicle, the candidate area is adjusted to obtain the target area.

Further, the processor 232 is further configured to: calculate the similarity between the pixels in the target area and the pixels in the second image;

According to the similarity between the pixel points, a pixel point that matches the pixel point in the target area is determined in the second image.

Further, the processor 232 is further configured to: if the distance between the drone and the target area meets a preset condition, respond to a return home control instruction to make the drone return home automatically.

Further, the processor 232 is further configured to determine the point cloud data corresponding to the target area according to the distance between the target area and the drone.

The movable platform shown in FIG. 10 can execute the methods of the embodiments shown in FIGS. 1 to 8. For parts that are not described in detail in this embodiment, please refer to the related descriptions of the embodiments shown in FIGS. 1 to 8. For the implementation process and technical effects of this technical solution, please refer to the description in the embodiment shown in FIG. 1 to FIG. 8, which will not be repeated here.

In a possible design, the structure of the distance measuring device shown in FIG. 11 can be implemented as an electronic device, which can be a drone. As shown in FIG. 11, the electronic device may include: one or more processors 31 and one or more memories 32. Wherein, the memory 32 is used to store a program that supports the electronic device to execute the distance measurement method provided in the embodiments shown in FIGS. 1 to 8 above. The processor 31 is configured to execute a program stored in the memory 32.

Specifically, the program includes one or more computer instructions, and the following steps can be implemented when one or more computer instructions are executed by the processor 31:

Acquiring a first image of the airspace above the drone;

Wherein, the structure of the distance measuring device may further include a communication interface 33 for the electronic device to communicate with other devices or a communication network.

Further, the device also includes a monocular camera;

The processor 31 is further configured to: respond to the first flight control instruction to make the UAV rotate and fly one circle in situ at the first position;

Further, the processor 31 is further configured to: respond to a second flight control instruction to cause the drone to descend and fly a preset distance from the first position to a second position;

Further, the monocular camera is placed on a pan-tilt that can swing upward, so that the monocular camera 24 can capture the first image and the second image;

The processor 31 is further configured to: respond to a third flight control instruction, so that the UAV rotates and flies in situ at the second position for one circle;

Further, the processor 31 is further configured to: identify the semantic category of pixels in the first image;

Further, the semantic category of the pixel includes sky and obstacle, and the processor 31 is further configured to: according to the semantic category of the pixel, determine the description of the sky and the obstacle closest to the sky in the first image. Target pixels, the target area is formed by the target pixels.

The matching pixel pair is determined according to the similarity between the characteristic pixel points.

Further, the processor 31 is further configured to: determine a candidate area in the first image according to the semantic category of the pixel;

Further, the processor 31 is further configured to: calculate the similarity between the pixels in the target area and the pixels in the second image;

Further, the processor 31 is further configured to: if the distance between the drone and the target area meets a preset condition, respond to a return home control instruction to make the drone return home automatically.

Further, the processor 31 is further configured to: determine the point cloud data corresponding to the target area according to the distance between the target area and the drone.

The device shown in FIG. 11 can execute the methods of the embodiments shown in FIG. 1 to FIG. 8. For parts that are not described in detail in this embodiment, reference may be made to the related description of the embodiments shown in FIG. 1 to FIG. 8. For the implementation process and technical effects of this technical solution, please refer to the description in the embodiment shown in FIG. 1 to FIG. 8, which will not be repeated here.

In addition, an embodiment of the present invention provides a computer-readable storage medium. The storage medium is a computer-readable storage medium. The computer-readable storage medium stores program instructions. The distance measurement method.

The technical solutions and technical features in each of the above embodiments can be singly or combined in case of conflict with the present invention, as long as they do not exceed the cognitive scope of those skilled in the art, they all belong to the equivalent embodiments within the protection scope of this application. .

In the several embodiments provided by the present invention, it should be understood that the related detection device (for example: IMU) and method disclosed may be implemented in other ways. For example, the embodiments of the remote control device described above are merely illustrative. For example, the division of the modules or units is only a logical function division. In actual implementation, there may be other division methods, such as multiple units or components. It can be combined or integrated into another system, or some features can be ignored or not implemented. In addition, the displayed or discussed mutual coupling or direct coupling or communication connection may be indirect coupling or communication connection through some interfaces, remote control devices or units, and may be in electrical, mechanical or other forms.

The units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the objectives of the solutions of the embodiments.

In addition, the functional units in the various embodiments of the present invention may be integrated into one processing unit, or each unit may exist alone physically, or two or more units may be integrated into one unit. The above-mentioned integrated unit can be implemented in the form of hardware or software functional unit.

If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer readable storage medium. Based on this understanding, the technical solution of the present invention essentially or the part that contributes to the existing technology or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium , Including several instructions to make a computer processor (processor) execute all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage media include: U disk, mobile hard disk, Read-Only Memory (ROM), Random Access Memory (RAM, Random Access Memory), magnetic disks or optical disks and other media that can store program codes.

The above are only the embodiments of the present invention, which do not limit the scope of the present invention. Any equivalent structure or equivalent process transformation made by using the content of the description and drawings of the present invention, or directly or indirectly applied to other related technologies In the same way, all fields are included in the scope of patent protection of the present invention.

Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that: The technical solutions recorded in the foregoing embodiments can still be modified, or some or all of the technical features can be equivalently replaced; and these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the technical solutions of the embodiments of the present invention. scope.

Claims

A distance measurement method for drones, characterized in that the method includes:

Acquiring a first image of the airspace above the drone;

In the first image, determine the target area that affects the flight of the drone according to the semantic category of the pixels in the first image;

Respond to the flight instruction, so that the UAV moves a preset distance in a designated direction to obtain a second image of the airspace above the UAV;

Matching the pixel points of the target area in the first image and the second image to obtain matching pixels;

The distance between the target area and the drone is determined according to the matching pixel points.
The method according to claim 1, wherein the acquiring a first image of the airspace above the drone comprises:

Respond to the first flight control instruction to make the UAV rotate and fly one circle in the first position;

During the rotating flight of the drone, the first image captured by the monocular camera configured on the drone is acquired.
The method according to claim 2, wherein the responding to a flight instruction so that the drone moves a predetermined distance in a specified direction to obtain a second image of the airspace above the drone, comprising :

Respond to the second flight control instruction to cause the drone to descend and fly a preset distance from the first position to a second position;

When the drone is located at the second position, the second image captured by the monocular camera is acquired.
The method according to claim 2, wherein the acquiring the second image taken by the monocular camera when the drone is at the second position comprises:

Respond to the third flight control instruction to make the UAV rotate and fly one circle in situ at the second position;

Acquiring the second image taken by the monocular camera during the unmanned on-board rotating flight. 5. The method according to claim 2 or 4, wherein the first image and the second image correspond to a circular field of view in the airspace above the drone.
The method according to claim 2 or 4, wherein the monocular camera is placed on a pan-tilt that can swing upwards, so that the monocular camera can capture the first image and the second image. image.
The method according to claim 1, wherein, in the first image, determining the target area that affects the flight of the drone according to the semantic category of the pixels in the first image comprises:

Identifying the semantic category of pixels in the first image;

According to the semantic category of the pixel point, the target area in the first image is determined.
The method according to claim 7, wherein the semantic categories of pixels include sky and obstacles;

The determining the target area in the first image according to the semantic category of the pixel includes:

According to the semantic category of the pixel, the target pixel used to describe the sky and the obstacle closest to the sky in the first image is determined, so that the target pixel constitutes the target area.
The method according to claim 7, wherein the determining the target area in the first image according to the semantic category of the pixel point comprises:

Determining a candidate area in the first image according to the semantic category of the pixel;

According to the volume of the unmanned aerial vehicle and/or the distribution of obstacles in the flying environment of the unmanned aerial vehicle, the candidate area is adjusted to obtain the target area.
The method according to any one of claims 7 to 9, wherein the matching the pixel points of the target area in the first image and the second image to obtain matching pixels includes :

Calculating the similarity between the pixels in the target area and the pixels in the second image;

According to the similarity between the pixel points, a pixel point that matches the pixel point in the target area is determined in the second image.
The method according to claim 1, wherein the method further comprises:

If the distance between the drone and the target area satisfies a preset condition, respond to the return home control instruction to make the drone return home automatically.
The method according to claim 1, wherein the method further comprises:

According to the distance between the target area and the drone, the point cloud data corresponding to the target area is determined.
A movable platform, characterized in that it at least includes a body, a power system and a control device;

The power system is arranged on the body and used to provide power for the movable platform;

The control device includes a memory and a processor;

The memory is used to store a computer program;

The processor is configured to run a computer program stored in the memory to realize:

Acquiring a first image of the airspace above the drone;

In the first image, determine the target area that affects the flight of the drone according to the semantic category of the pixels in the first image;

Respond to the flight instruction, so that the UAV moves a preset distance in a designated direction to obtain a second image of the airspace above the UAV;

Matching the pixel points of the target area in the first image and the second image to obtain matching pixels;

The distance between the target area and the drone is determined according to the matching pixel points.
The platform according to claim 13, wherein the platform further comprises a monocular camera arranged on the body;

The processor is also used for:

Respond to the first flight control instruction to make the UAV rotate and fly one circle in the first position;

During the rotating flight of the drone, the first image captured by the monocular camera configured on the drone is acquired.
The platform according to claim 14, wherein the processor is further configured to:

Respond to the second flight control instruction to cause the drone to descend and fly a preset distance from the first position to a second position;

When the drone is located at the second position, the second image captured by the monocular camera is acquired.
The platform according to claim 14, wherein the monocular camera is placed on a pan-tilt that can swing upward, so that the monocular camera can capture the first image and the second image;

The processor is also used for:

Respond to the third flight control instruction to make the UAV rotate and fly one circle in situ at the second position;

Acquire the second image taken by the monocular camera during the unmanned aircraft rotating flight, wherein the first image and the second image correspond to the ring shape in the airspace above the drone Field of view.
The platform according to claim 13, wherein the processor is further configured to:

Identifying the semantic category of pixels in the first image;

According to the semantic category of the pixel point, the target area in the first image is determined.
The platform according to claim 17, wherein the semantic categories of pixels include sky and obstacles;

The processor is also used for:

The determining the target area in the first image according to the semantic category of the pixel includes:

According to the semantic category of the pixel, the target pixel used to describe the sky and the obstacle closest to the sky in the first image is determined, so that the target pixel constitutes the target area.
The platform according to claim 17, wherein the processor is further configured to:

Determining a candidate area in the first image according to the semantic category of the pixel;

According to the volume of the unmanned aerial vehicle and/or the distribution of obstacles in the flying environment of the unmanned aerial vehicle, the candidate area is adjusted to obtain the target area.
The platform according to any one of claims 17 to 19, wherein the processor is further configured to:

Calculating the similarity between the pixels in the target area and the pixels in the second image;

According to the similarity between the pixel points, a pixel point that matches the pixel point in the target area is determined in the second image.
The platform according to claim 13, wherein the processor is further configured to:

If the distance between the drone and the target area satisfies a preset condition, respond to the return home control instruction to make the drone return home automatically.
The platform according to claim 13, wherein the processor is further configured to:

According to the distance between the target area and the drone, the point cloud data corresponding to the target area is determined.
A distance measuring device, characterized in that the measuring device includes:

Memory, used to store computer programs;

The processor is configured to run a computer program stored in the memory to realize:

Acquiring a first image of the airspace above the drone;

In the first image, determine the target area that affects the flight of the drone according to the semantic category of the pixels in the first image;

Respond to the flight instruction, so that the UAV moves a preset distance in a designated direction to obtain a second image of the airspace above the UAV;

Matching the pixel points of the target area in the first image and the second image to obtain matching pixels;

The distance between the target area and the drone is determined according to the matching pixel points.
The device according to claim 23, wherein the device further comprises: a monocular camera;

The processor is also used for:

Respond to the first flight control instruction to make the UAV rotate and fly one circle in the first position;

During the rotating flight of the drone, the first image captured by the monocular camera configured on the drone is acquired.
The device according to claim 24, wherein the processor is further configured to:

Respond to the second flight control instruction to cause the drone to descend and fly a preset distance from the first position to a second position;

When the drone is located at the second position, the second image captured by the monocular camera is acquired.
The device according to claim 24, wherein the device further comprises a pan/tilt, and the monocular camera is placed on the pan/tilt capable of swinging upward, so that the monocular camera can take pictures of the The first image and the second image;

The processor is also used for:

Respond to the third flight control instruction to make the UAV rotate and fly one circle in situ at the second position;

Acquire the second image taken by the monocular camera during the unmanned aircraft rotating flight, wherein the first image and the second image correspond to the ring shape in the airspace above the drone Field of view.
The device according to claim 23, wherein the device further comprises:

Identifying the semantic category of pixels in the first image;

According to the semantic category of the pixel point, the target area in the first image is determined.
The device according to claim 27, wherein the semantic categories of pixels include sky and obstacles;

The device also includes:

According to the semantic category of the pixel, the target pixel used to describe the sky and the obstacle closest to the sky in the first image is determined, so that the target pixel constitutes the target area.
The device according to claim 27, wherein the device further comprises:

Determining a candidate area in the first image according to the semantic category of the pixel;

According to the volume of the unmanned aerial vehicle and/or the distribution of obstacles in the flying environment of the unmanned aerial vehicle, the candidate area is adjusted to obtain the target area.
The device according to any one of claims 27 to 29, wherein the device further comprises:

Calculating the similarity between the pixels in the target area and the pixels in the second image;

According to the similarity between the pixel points, a pixel point that matches the pixel point in the target area is determined in the second image.
The device according to claim 23, wherein the device further comprises:

If the distance between the drone and the target area satisfies a preset condition, respond to the return home control instruction to make the drone return home automatically.
The device according to claim 23, wherein the device further comprises:

According to the distance between the target area and the drone, the point cloud data corresponding to the target area is determined.
A computer-readable storage medium, wherein the storage medium is a computer-readable storage medium, and program instructions are stored in the computer-readable storage medium, and the program instructions are used to implement any one of claims 1 to 12 The distance measurement method described in the item.