WO2021146972A1

WO2021146972A1 - Airspace detection method, movable platform, device, and storage medium

Info

Publication number: WO2021146972A1
Application number: PCT/CN2020/073659
Authority: WO
Inventors: 刘宝恩; 王涛; 李鑫超
Original assignee: 深圳市大疆创新科技有限公司
Priority date: 2020-01-21
Filing date: 2020-01-21
Publication date: 2021-07-29
Also published as: CN112567308A

Abstract

An airspace detection method, a movable platform, a device, and a storage medium. The method comprises: responding to a flight control instruction so that an unmanned aerial vehicle flies according to a preset flight mode (S101); in a flight process, obtaining an environment image in a preset field of view above the unmanned aerial vehicle (S102); and if it is identified that there is no object of an obstacle type in the environment image, determining that airspace above the unmanned aerial vehicle is flyable (S104). On one hand, compared with methods of regarding airspace above an unmanned aerial vehicle to be flyable by default, whether the airspace above the unmanned aerial vehicle is flyable is determined by means of detection. When the airspace above is flyable, the unmanned aerial vehicle can fly upwards, and automatic return is further implemented to avoid damage to the unmanned aerial vehicle. On the other hand, the detection method is not intended to detect the entire field of view above the unmanned aerial vehicle, but to detect the preset field of view obtained by flight according to the preset flight mode, so that the detection range is narrowed, the amount of calculation is reduced, and the detection efficiency is improved.

Description

Airspace detection method, movable platform, equipment and storage medium

Technical field

The present invention relates to the technical field of image processing, in particular to an airspace detection method, a movable platform, equipment and storage medium.

Background technique

UAV is an unmanned aerial vehicle operated by radio remote control equipment and self-provided program control device. Compared with manned aircraft, it has the characteristics of small size and low cost. It has been widely used in many fields, such as street scene shooting, power inspection, traffic monitoring, post-disaster rescue and so on.

In all stages of UAV flight, obstacles need to be avoided. Take the drone's return as an example. During the return process, the drone has an upward flight. At this time, it is necessary to distinguish whether the drone is in the flyable airspace, that is, whether there are obstacles. In the prior art, it is usually assumed that the airspace above the drone is flyable, but this is obviously unreasonable and brings a greater risk of damage to the drone during the return process.

Summary of the invention

The invention provides an airspace detection method, a movable platform, equipment and a storage medium, which are used for realizing accurate detection of the flightable airspace above the drone.

The first aspect of the present invention is to provide an airspace detection method, the method includes:

Respond to flight control commands to make the UAV fly in a preset flight mode;

In the process of flying in the preset flight mode, acquiring an environment image corresponding to the preset field of view above the drone;

Identifying the category to which the object in the environmental image belongs;

If there is no object belonging to the obstacle category in the environment image, it is determined that the airspace above the drone is a flyable airspace.

The second aspect of the present invention is to provide a movable platform, the movable platform includes: a body, a power system and a control device;

The power system is arranged on the body and used to provide power for the movable platform;

The control device includes a memory and a processor;

The memory is used to store a computer program;

The processor is configured to run a computer program stored in the memory to realize: responding to flight control instructions to make the UAV fly in a preset flight mode;

The third aspect of the present invention is to provide an airspace detection device, which includes:

Memory, used to store computer programs;

The fourth aspect of the present invention is to provide a computer-readable storage medium, the storage medium is a computer-readable storage medium, the computer-readable storage medium stores program instructions, and the program instructions are used in the first aspect. The airspace detection method described.

The airspace detection method, movable platform, equipment and storage medium provided by the present invention respond to flight control instructions, and make the UAV fly according to the preset flight mode according to the response result. During the flight, the environment image in the preset field of view above the drone is acquired and the environment image is recognized. If it is recognized that there is no object belonging to the obstacle category in the environmental image, it can be considered that there is no obstacle above the drone, and the airspace above the drone is determined to be flyable. On the one hand, compared to the way in the prior art that the airspace above the drone is a flyable airspace, the present invention provides an airspace detection method to determine whether the airspace above the drone is a flyable airspace through detection. When it is detected that there are no obstacles in the airspace above, the drone can fly upwards and further return home to avoid damage to the drone. On the other hand, the airspace detection method provided by the present invention does not detect the entire field of view above the drone, but detects the preset field of view obtained by flying according to the preset flight mode, so that the detection range in the detection process is Reduce, reduce the amount of calculation, and improve the detection efficiency.

Description of the drawings

The drawings described here are used to provide a further understanding of the application and constitute a part of the application. The exemplary embodiments and descriptions of the application are used to explain the application, and do not constitute an improper limitation of the application. In the attached picture:

FIG. 1 is a schematic flowchart of an airspace detection method provided by an embodiment of the present invention;

FIG. 2 is a schematic structural diagram of a pan/tilt configured for drones in different states according to an embodiment of the present invention;

3 is a schematic diagram of a circular field of view corresponding to an environmental image provided by an embodiment of the present invention;

4 is a schematic flowchart of another airspace detection method provided by an embodiment of the present invention;

FIG. 5a is a schematic flowchart of yet another airspace detection method according to an embodiment of the present invention;

FIG. 5b is a schematic flowchart of yet another airspace detection method according to an embodiment of the present invention;

FIG. 5c is a schematic flowchart of another airspace detection method according to an embodiment of the present invention;

6 is a schematic diagram of the relationship between the first distance and the second distance provided by an embodiment of the present invention;

FIG. 7 is a schematic structural diagram of an airspace detection device provided by an embodiment of the present invention;

FIG. 8 is a schematic structural diagram of a movable platform provided by an embodiment of the present invention;

FIG. 9 is a schematic structural diagram of an airspace detection device provided by an embodiment of the present invention.

Detailed ways

In order to make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the following will clearly and completely describe the technical solutions in the embodiments of the present invention with reference to the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments These are a part of the embodiments of the present invention, but not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative work shall fall within the protection scope of the present invention.

Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by those skilled in the technical field of the present invention. The terms used in the specification of the present invention herein are only for the purpose of describing specific embodiments, and are not intended to limit the present invention.

Before detailed introduction to the airspace detection method provided by the embodiment of the present invention, the automatic return mechanism of the UAV will be briefly introduced.

During the flight, the UAV is often in the range of beyond the visual range. When the UAV completes the flight mission or encounters the harsh natural environment during the flight, such as the protruding mountain peak, or the ground base station. When the communication connection is disconnected, in order to ensure the safety of the drone and avoid damage accidents, the drone often needs to return home automatically. Since there is an ascending flight stage during the automatic return of the UAV, whether there are obstacles above the UAV becomes an important factor affecting the automatic return of the UAV. At this time, the airspace detection method provided by the following embodiments can be used to determine whether there is an obstacle above the drone, that is, to determine whether the drone can automatically return home.

In the following, some embodiments of the present invention will be described in detail with reference to the accompanying drawings. As long as there is no conflict between the embodiments, the following embodiments and the features in the embodiments can be combined with each other.

FIG. 1 is a schematic flowchart of an airspace detection method provided by an embodiment of the present invention. The main body of the airspace detection method is airspace detection equipment. It is understandable that the airspace detection device can be implemented as software or a combination of software and hardware. The airspace detection device executes the airspace detection method to detect whether the airspace above the drone is a flightable airspace. The airspace detection equipment in this embodiment and the following embodiments may specifically be a movable platform, such as a drone. Specifically, the method may include:

S101: Respond to the flight control command to make the UAV fly in a preset flight mode.

When the drone needs to return home automatically, it will generate a flight control command. The drone responds to this command to make itself fly according to the preset flight mode, and the preset flight mode has a one-to-one correspondence with the flight control command. For example, the flight control command can be rotating and flying in place at the first position, or flying from the first position to a second position beyond the preset distance, where the first position can be that the drone generates flight control. The position at the time of the command can be any position during the entire flight of the UAV.

S102: Acquire an environment image corresponding to the preset field of view above the drone during the flight in the preset flight mode.

In the process of flying in the preset flight mode, the upward-viewing camera configured by the UAV itself can continuously take pictures of the top of the UAV to obtain environmental images. The particularity of the flying mode of the UAV directly leads to the particularity of the field of view corresponding to the obtained environmental image, that is, within a certain field of view. In other words, in response to flight control instructions to fly in a preset flight mode, its direct function is to obtain an image of the environment in the preset field of view.

Optionally, the top-viewing camera configured on the UAV can be a monocular camera, and the top-viewing function of the monocular camera can be realized with the help of a pan-tilt, that is, the monocular camera is placed on a pan-tilt that can be lifted upwards. Up, the monocular camera can take images corresponding to the environment above the drone by lifting up the pan/tilt. Among them, the lifted and non-lifted states of the PTZ can be shown in Figure 2. For the flight control command, an alternative way may be to control the UAV to rotate and fly one circle at the current position. Since the up-view camera on the UAV itself has a certain field of view angle, after rotating and flying, the field of view corresponding to the obtained environment image is a circular field of view, as shown in Figure 3, and the circular field of view The height is determined by the field of view of the monocular camera.

S103: Identify the category to which the object in the environmental image belongs.

Then, the drone will recognize the acquired environment image to determine the category of each object in the environment image. This recognition is actually at the pixel level, that is, to identify the category to which each pixel in the environmental image belongs. For the pixel-level recognition process, optionally, it can be done by a neural network model.

Specifically, the neural network model may be a convolutional neural network (Convolutional Neural Networks, CNN) model. The neural network model can include multiple computing nodes. Each computing node can include a convolution (Conv) layer, batch normalization (BN), and an activation function ReLU. The computing nodes can use skip connection (Skip Connection). ) Way to connect.

The input data of K×H×W can be input into the neural network model, and after the neural network model is processed, the output data of C×H×W can be obtained. Among them, K can represent the number of input channels, and K can be equal to 4, corresponding to the four channels of red (R, red), green (G, green), blue (B, blue) and depth (D, deep) respectively; H can represent the height of the input image (ie, environmental image), W can represent the width of the input image, and C can represent the number of categories.

It should be noted that when the input image is too large, an input image can be cut into N sub-images. Correspondingly, the input data can be N×K×H'×W', and the output data can be N×C×H' ×W', where H'can represent the height of the sub-image, and W'can represent the width of the sub-image. Of course, in other embodiments, the feature map may also be obtained in other ways, which is not limited in this application.

Using the above-mentioned pre-trained neural network model to process the environment image to obtain a feature map may specifically include the following steps:

Step 1. Input the environment image into the neural network model to obtain the model output result of the neural network model.

Among them, the model output result of the neural network model may include the confidence feature maps output by multiple output channels, and the multiple output channels can correspond to multiple object categories one-to-one, and the pixel values of the confidence feature maps of a single object category are used To characterize the probability that a pixel is an object category.

Step 2: According to the model output result of the neural network model, a feature map containing semantic information is obtained.

The object category corresponding to the confidence feature map with the largest pixel value at the same pixel location in the multiple confidence feature maps one-to-one corresponding to the multiple output channels may be used as the object category of the pixel location to obtain the feature map.

Suppose that the number of output channels of the neural network model is 4, and the output result of each channel is a confidence feature map, that is, the 4 confidence feature maps are the confidence feature map 1 to the confidence feature map 4, and the confidence The degree characteristic map 1 corresponds to the sky, the confidence characteristic map 2 corresponds to buildings, the confidence characteristic map 3 corresponds to trees, and the confidence characteristic map 4 corresponds to "other". In these categories, except for the sky, the rest can be regarded as obstacles.

For example, when the pixel value at the pixel location (100, 100) in the confidence feature map 1 is 70, the pixel value at the pixel location (100, 100) in the confidence feature map 2 is 50, and the pixel at the pixel location (100, 100) in the confidence feature map 3 When the value is 20, and the pixel value of the pixel position (100, 100) in the confidence feature map 4 is 20, it can be determined that the pixel position (100, 100) is the sky.

For another example, when the pixel value at the pixel location (100, 80) in the confidence feature map 1 is 20, the pixel value at the pixel location (100, 80) in the confidence feature map 2 is 30, and the pixel location in the confidence feature map 3 When the pixel value of (100,80) is 20, and the pixel value of pixel position (100,80) in the confidence feature figure 4 is 70, it can be determined that the pixel position (100,80) is other, that is, it is not trees, buildings, and Any of the trees.

S104: If there is no object belonging to the obstacle category in the environment image, it is determined that the airspace above the drone is a flyable airspace.

Based on the recognition result of the environment image, in one case, if it is recognized that there is no object belonging to the obstacle category in the environment image, indicating that there is no obstacle in the airspace above the drone, it is determined that the airspace above is a flightable airspace. At this point, the UAV can start to fly and return home automatically.

In another case, if it is recognized that there are objects belonging to the obstacle category in the environmental image, indicating that there are obstacles above the drone, the airspace above the drone is determined to be non-flyable airspace. At this time, the drone needs to continue hovering at the current position.

The airspace detection method provided in this embodiment responds to flight control instructions, and makes the UAV fly in a preset flight mode according to the response result. During the flight, the environment image in the preset field of view above the drone is acquired and the environment image is recognized. If it is recognized that there are no objects belonging to the obstacle category in the environmental image, it can be considered that there is no obstacle above the drone, and the airspace above the drone is determined to be flyable. On the one hand, compared to the prior art method in which the airspace above the drone is assumed to be flyable airspace, the present invention provides an airspace detection method to determine whether the airspace above the drone is flyable airspace through detection. When it is detected that there are no obstacles in the airspace above, the drone can fly upwards and further realize automatic return to avoid damage to the drone. On the other hand, the airspace detection method provided by the present invention does not detect the entire field of view above the drone, but detects the preset field of view obtained by flying according to the preset flight mode, so that the detection range in the detection process is Reduce, reduce the amount of calculation, and improve the detection efficiency.

In addition, in this embodiment, only the image taken by the upward-viewable camera can be used to realize airspace detection, that is, a brand-new airspace detection method is provided. Compared with the method of using binocular cameras or depth sensors for airspace detection, this embodiment is more suitable for drones that are not equipped with binocular cameras or depth sensors.

According to the description in the foregoing embodiment, it can be seen that the drone is equipped with an upward-looking camera, which usually has a larger field of view angle, such as 45 degrees. However, in practical applications, the angle of the field of view of the camera can also be used as the upper limit, and the angle of the field of view can be adjusted in a targeted manner to obtain the preset field of view corresponding to the environmental image.

Based on this, FIG. 4 is a schematic flowchart of another airspace detection method provided by an embodiment of the present invention. As shown in FIG. 4, after step 101, the airspace detection method may further include the following steps:

S201: Determine the field of view angle of the preset annular field of view according to the field of view angle of the upward-looking camera configured on the drone.

For brevity in the subsequent description, the angle of view of the top-view camera itself is referred to as the first angle, and the angle of the preset circular field of view corresponding to the preset flying mode is referred to as the second angle. The second angle is less than or equal to the first angle.

By default, the second angle can be set equal to the first angle. However, in practical applications, a second angle smaller than the first angle can also be determined. After the angle of the field of view is adjusted to the second angle, the ring width of the annular field of view also becomes smaller.

However, in practical applications, in addition to determining the second angle based on the first angle, it can be more refined, optionally, based on the volume of the drone and the flight environment of the drone. To determine the second angle.

For example, when the size of the drone is small or the flying environment is relatively open, the second angle can be determined as an angle slightly smaller than the first angle. Since the original images captured by the up-view camera correspond to the first angle, after the second angle is determined, the original image will be intercepted, and the intercepted image is the environment image. After the interception process, the size of the environment image will be smaller than the size of the original image, that is, the field of view of the spatial detection is reduced, thereby reducing the amount of calculation and improving the detection efficiency.

When the size of the drone is large or the flying environment is relatively narrow, the second angle can be determined to be equal to the first angle to ensure that the airspace above the drone is detected within the maximum field of view angle, and as far as possible to avoid the occurrence of errors. Damaged man and machine.

In this embodiment, during actual flight, the first angle of the up-view camera itself can also be adjusted according to parameters such as the flight environment and the volume of the drone to obtain the second angle that meets the actual flight. By adjusting the angle, the field of view of the airspace detection can be reduced, the calculation amount is reduced, and the detection efficiency can be improved.

According to the description in each of the foregoing embodiments, it can be seen that there is a strong correlation between the flight control command, the preset angle of view corresponding to the environment image, and the detection of the flightable airspace. Based on the foregoing embodiments, when the flight control instruction is specifically the first flight control instruction, FIG. 5a is a schematic flowchart of another airspace detection method provided by an embodiment of the present invention. As shown in Figure 5a, the method may include the following steps:

S301: Respond to the first flight control instruction to make the UAV rotate and fly one circle in situ at the first position.

The first flight control instruction may specifically be a control instruction for controlling the drone to rotate and fly one circle in situ at the first position. The drone responds to this instruction and starts to fly one circle in the first position.

S302. In the process of rotating and flying at the first position, acquire multiple images taken at intervals of a preset angle, and the multiple images acquired at the first position correspond to the first preset annular field of view above the drone.

S303: Perform merging processing on multiple images to obtain an environmental image.

During the rotating flight of the drone, its own up-view camera will take multiple images at preset angle intervals, and then merge the multiple images to obtain environmental images. The field of view corresponding to this environment image is the first preset annular field of view, and the scene included in the environment image is the scene in the first preset annular field of view above the drone, and the first preset annular field of view corresponds to Above the fuselage of the drone.

S304: Identify the category to which the object in the environmental image belongs.

S305: If there is no object belonging to the obstacle category in the environment image, it is determined that the airspace above the drone is a flyable airspace.

The execution process of the foregoing step 304 and step 305 is similar to the corresponding steps of the foregoing embodiment, and reference may be made to the related description in the embodiment shown in FIG. 1, which will not be repeated here.

In this embodiment, in response to the first flight control instruction, the UAV rotates and flies once in the first position. It is precisely because of this special flight mode that the acquired environment image corresponds to a special field of view, that is, the first preset annular field of view above the drone. Finally, through the recognition of environmental images, it can be accurately determined whether there are obstacles in the airspace above the drone, that is, whether the drone can return home automatically.

In the embodiment shown in FIG. 5a, it is detected whether there is an obstacle above the drone body, and the detection range is actually a minimum detection range. In order to achieve automatic return to home and avoid damage to the drone, on the one hand, obstacle detection is often required on the wing of the drone. And on the other hand, in practical applications, the UAV is affected by strong winds or other environmental factors during the ascending flight, and there may be shaking during the ascending flight. Based on the above two considerations, the detection range of obstacles can also be appropriately expanded.

Fig. 5b is a schematic flowchart of yet another airspace detection method according to an embodiment of the present invention. As shown in Figure 5b, the method may include the following steps:

S401: Respond to the first flight control instruction to make the UAV rotate and fly one circle in situ at the first position.

S402: During the rotating flight at the first position, acquire multiple images taken at intervals of a preset angle, and the multiple images acquired at the first location correspond to the first preset annular field of view above the drone.

The execution process of the foregoing step 401 and step 402 is similar to the corresponding steps of the foregoing embodiment, and reference may be made to the related description in the embodiment shown in FIG. 5a, which will not be repeated here.

S403: Respond to the second flight control instruction to make the drone fly to a second position that is a preset distance from the first position.

S404: Acquire an image taken at the second position, where the image obtained at the second position corresponds to a preset angle of view above the drone.

S405: Generate an environment image based on the multiple images acquired at the first location and the images acquired at the second location.

After responding to the first flight control instruction, it may continue to respond to the second flight control instruction, so that the drone flies from the current first position to the second position. Wherein, the first position and the second position are separated by a preset distance. The up-view camera on the drone can continue to take images in this second position. At this time, the number of images may be one, and the field of view of this image is the preset field of view angle above the drone when it is at the second position. At this time, the preset field of view is composed of the first preset annular field of view and the preset angle of field of view.

For the preset distance, optionally, the preset distance should not be less than the difference between the first distance L1 and the second distance L2. Among them, the first distance L1 is the distance between the center of the drone's fuselage and the top-viewing camera, and the second distance L2 is the maximum distance between the center of the fuselage and the edge of the drone's wing. For the position and size relationship between the first distance and the second distance, please refer to the label in FIG. 6.

Then, the multiple images taken by the rotating flight at the first position are merged, and the merged result and the image taken at the second position are jointly determined as the environment image.

S406: Identify the category to which the object in the environmental image belongs.

S407: If there is no object belonging to the obstacle category in the environment image, it is determined that the airspace above the drone is a flyable airspace.

The execution process of the foregoing step 406 and step 407 is similar to the corresponding steps of the foregoing embodiment, and reference may be made to the related description in the embodiment shown in FIG. 1, which will not be repeated here.

When it is determined that the airspace above the drone is a flyable airspace, the drone will return to the first position again and return home at the first position.

In this embodiment, the drone will first rotate and fly in the first position and then fly to the second position, thereby completing the preset flight mode. During the flight in this special flight mode, multiple images will be taken at the first position, and images will also be taken at the second position, thereby generating environmental images based on all the captured images. This environment image corresponds to the first preset annular field of view above the drone at the first position, and also corresponds to the preset angle of view above the drone at the second position. Finally, through the recognition of environmental images, it can be determined whether there are obstacles in the airspace above the drone. Compared with the embodiment shown in Figure 5a, the detection range of obstacles in this embodiment is expanded from the first position to the second position. This expansion of the range can determine whether there are obstacles above the UAV fuselage and wings at the same time. In this way, it is possible to more accurately determine whether the UAV can return home automatically.

Although the detection range has been expanded from the first position to the second position in the embodiment shown in Fig. 5b, the embodiment shown in Fig. 5b only performs obstacle detection based on an image taken at the second position, which is obviously It is not possible to fully detect whether there are obstacles in the airspace above the drone.

On this basis, Fig. 5c is a schematic flowchart of another airspace detection method provided by an embodiment of the present invention. As shown in Figure 5c, the method may include the following steps:

S501: Respond to the first flight control instruction to make the UAV rotate and fly one circle in situ at the first position.

S502: During the rotating flight at the first position, acquire multiple images taken at intervals of a preset angle, and the multiple images acquired at the first location correspond to the first preset annular field of view above the drone.

The execution process of the foregoing steps 501 to 502 is similar to the corresponding steps of the foregoing embodiment, and reference may be made to the related description in the embodiment shown in FIG. 1, which will not be repeated here.

S503: Respond to the third flight control instruction to make the UAV rotate and fly one circle in situ at a second position that is a preset distance from the first position.

S504: During the rotating flight at the second position, acquire multiple images taken at intervals of a preset angle, and the multiple images acquired at the second location correspond to the second preset annular field of view above the drone.

After responding to the first flight control, it is also possible to respond to the third flight control command. In order to make the UAV fly from the first position to the second position, and rotate and fly one circle in the second position. After the drone rotates and flies in the second position, the upward-viewing camera can also take multiple images in the second position at intervals of a preset angle. At this time, the corresponding field of view angles of the multiple images taken at the second position will collectively form a second preset annular field of view above the drone at the second position. Step 503 to step 504 are actually basically similar to the execution process of step 501 to step 502, and the specific content can refer to the corresponding content in the foregoing embodiment.

S505: Perform merging processing on the multiple images respectively obtained at the first position and the second position respectively, to obtain an environment image.

After the above steps, multiple images taken at the first position and multiple images taken at the second position have been acquired. At this time, the images taken at different positions are merged, so as to obtain the environment image corresponding to the first position and the environment image corresponding to the second position, so that the two environmental images can be subsequently identified.

S506: Identify the category to which the object in the environmental image belongs.

S507: If there is no object belonging to the obstacle category in the environment image, it is determined that the airspace above the drone is a flyable airspace.

The execution process of the foregoing step 506 and step 507 is similar to the corresponding steps of the foregoing embodiment, and reference may be made to the related description in the embodiment shown in FIG. 1, which will not be repeated here.

In this embodiment, the UAV will rotate and fly one circle in the first position and the second position respectively, and recognize based on multiple images taken at the two positions. Through the special flying method of rotating flight, it is possible to comprehensively determine whether there are obstacles in the airspace above the drone's fuselage, and to comprehensively determine whether there are obstacles in the airspace above the drone's wing. The embodiment shown in Figure 5b can more comprehensively and accurately determine whether the UAV can return home automatically.

In addition, on the basis of the above embodiments, when the airspace above the drone is a flyable airspace, the unmanned return can be controlled. It is easy to understand that any flight process of the drone requires battery power. Therefore, before controlling the drone to return home, you can also determine the power required during the return process. If the current remaining power is more than the return required When the battery is charged, the drone will be controlled to return home.

As for the determination of the power required during the return home process, an alternative way is to first estimate the wind speed information from the current position to the destination of the return home based on the historical wind speed information. Then determine the ground speed information for landing from the current position to the return destination, so as to determine the power required for the drone's return process based on the wind speed information and ground speed information.

FIG. 7 is a schematic structural diagram of an airspace detection device provided by an embodiment of the present invention; referring to FIG. 5, this embodiment provides an airspace detection device, which can execute the above-mentioned airspace detection method; specifically , The airspace detection device includes:

The response module 11 is used to respond to flight control commands to make the UAV fly in a preset flight mode.

The acquiring module 12 is configured to acquire an environment image corresponding to the preset field of view above the drone during the flight in the preset flight mode.

The recognition module 13 is used to recognize the category to which the object in the environmental image belongs.

The determining module 14 is configured to determine that the airspace above the drone is a flyable airspace if there is no object belonging to the obstacle category in the environment image.

The device shown in FIG. 7 can also execute the methods of the embodiments shown in FIGS. 1 to 6. For parts that are not described in detail in this embodiment, reference may be made to the related descriptions of the embodiments shown in FIGS. 1 to 6. For the implementation process and technical effects of this technical solution, please refer to the description in the embodiment shown in FIG. 1 to FIG. 6, which will not be repeated here.

FIG. 8 is a schematic structural diagram of a movable platform provided by an embodiment of the present invention; referring to FIG. 8, an embodiment of the present invention provides a movable platform, which is the following unmanned aerial vehicle; , The movable platform includes: a body 21, a power system 22, and a control device 23.

The power system 22 is arranged on the body 21 and is used to provide power for the movable platform.

The control device 23 includes a memory 231 and a processor 232.

The memory is used to store a computer program;

Further, the processor 232 is further configured to: if an object belonging to an obstacle category is recognized in the environment image, determine that the airspace above the drone is a non-flyable airspace.

Further, the body 21 is provided with a camera 24 capable of looking upwards.

The processor responds to the flight control instruction to make the drone fly in a preset flight mode, so that the processor obtains the environment image in the preset circular field of view above the drone;

The processor 232 is further configured to determine the field of view angle of the preset annular field of view according to the field of view angle of the up-view camera.

Further, the processor 232 is further configured to: determine the field of view of the preset circular field of view according to the angle of the field of view of the up-view camera, the volume of the drone, and the flying environment of the drone angle.

Further, the processor 232 is further configured to: respond to the first flight control instruction, so that the UAV rotates and flies once at the first position.

Further, the processor 232 is further configured to: during the rotating flight at the first position, obtain a plurality of images taken at intervals of a preset angle, and the plurality of images obtained at the first position correspond to the The first preset annular field of view above the drone;

Merging the multiple images is performed to obtain the environmental image.

Further, the processor 232 is also used for:

Respond to the second flight control instruction to cause the drone to fly to a second position that is a preset distance from the first position, wherein the preset distance is not less than the difference between the first distance and the second distance , The first distance is the distance between the center of the drone's fuselage and the up-view camera, and the second distance is the distance between the center of the fuselage and the edge of the drone's wing The maximum distance.

Further, the processor 232 is further configured to: acquire an image taken at the second position, where the image acquired at the second position corresponds to a preset angle of view above the drone;

The environment image is generated based on a plurality of images acquired at the first location and an image acquired at the second location.

Further, the processor 232 is further configured to: respond to a third flight control instruction, so that the UAV rotates and flies in situ at a second position that is a preset distance from the first position, wherein the The preset distance is not less than the difference between the first distance and the second distance, the first distance is the distance between the center of the drone body and the up-view camera, and the second distance is the The maximum distance between the center of the fuselage and the edge of the UAV wing.

Further, the processor 232 is further configured to: during the rotating flight at the second position, obtain a plurality of images taken at intervals of a preset angle, and the plurality of images obtained at the second position correspond to the The second preset annular field of view above the drone;

The multiple images respectively acquired at the first position and the second position are merged to obtain the environment image.

The movable platform shown in FIG. 8 can execute the methods of the embodiments shown in FIGS. 1 to 6. For parts that are not described in detail in this embodiment, please refer to the related descriptions of the embodiments shown in FIGS. 1 to 6. For the implementation process and technical effects of this technical solution, please refer to the description in the embodiment shown in FIG. 1 to FIG. 6, which will not be repeated here.

In a possible design, the structure of the airspace detection device shown in FIG. 9 can be implemented as an electronic device, and the electronic device can be a drone. As shown in FIG. 9, the electronic device may include: one or more processors 31 and one or more memories 32. Among them, the memory 32 is used to store a program that supports the electronic device to execute the airspace detection method provided in the above-mentioned embodiments shown in FIGS. 1 to 6. The processor 31 is configured to execute a program stored in the memory 32.

Specifically, the program includes one or more computer instructions, and the following steps can be implemented when one or more computer instructions are executed by the processor 31:

Respond to flight control commands to make the UAV fly in a preset flight mode;

During the flight in the preset flight mode, obtain the environment image corresponding to the preset field of view above the drone;

Identify the category to which objects in the environment image belong;

If there are no objects belonging to the obstacle category in the environment image, the airspace above the drone is determined to be flyable airspace.

The structure of the airspace detection device may also include a communication interface 33 for the electronic device to communicate with other devices or a communication network.

Further, the processor 31 is further configured to: if it is recognized that there is an object belonging to the obstacle category in the environment image, determine that the airspace above the drone is a non-flyable airspace.

Further, the processor responds to the flight control instruction to make the drone fly in a preset flight mode, so that the processor obtains the environment image in the preset circular field of view above the drone;

Before the step of acquiring the environment image, the processor 31 is further configured to determine the field of view angle of the preset annular field of view according to the field of view angle of the up-view camera configured on the drone.

Further, before the step of acquiring environmental images, the processor 31 is further configured to: determine according to the field of view angle of the up-view camera, the volume of the drone, and the flight environment of the drone The field of view angle of the preset annular field of view.

Further, the processor 31 is further configured to: respond to the first flight control instruction to make the UAV rotate and fly once in the first position.

Further, the processor 31 is further configured to: during the rotating flight at the first position, obtain a plurality of images taken at intervals of a preset angle, and the plurality of images obtained at the first position correspond to the The first preset annular field of view above the drone;

Merging the multiple images is performed to obtain the environmental image.

Further, the processor 31 is further configured to: respond to a second flight control instruction to cause the drone to fly to a second position that is a preset distance from the first position, wherein the preset distance is not Is less than the difference between a first distance and a second distance, the first distance being the distance between the center of the drone's body and the up-view camera, and the second distance being the distance between the center of the body and the The maximum distance between the edges of the UAV's wings.

Further, the processor 31 is further configured to: acquire an image taken at the second position, where the image acquired at the second position corresponds to a preset angle of view above the drone;

Further, the processor 31 is further configured to: respond to a third flight control instruction, so that the UAV rotates and flies in situ at a second position that is a preset distance from the first position, wherein the The preset distance is not less than the difference between the first distance and the second distance, the first distance is the distance between the center of the drone body and the up-view camera, and the second distance is the The maximum distance between the center of the fuselage and the edge of the UAV wing.

Further, the processor 31 is further configured to: during the rotating flight at the second position, obtain a plurality of images taken at intervals of a preset angle, and the plurality of images obtained at the second position correspond to the The second preset annular field of view above the drone;

The device shown in FIG. 9 can execute the methods of the embodiments shown in FIGS. 1 to 6. For parts that are not described in detail in this embodiment, reference may be made to the related descriptions of the embodiments shown in FIGS. 1 to 6. For the implementation process and technical effects of this technical solution, please refer to the description in the embodiment shown in FIG. 1 to FIG. 6, which will not be repeated here.

In addition, an embodiment of the present invention provides a computer-readable storage medium, the storage medium is a computer-readable storage medium, the computer-readable storage medium stores program instructions, and the program instructions are used to implement the above-mentioned airspace in FIGS. 1 to 6 Detection method.

The technical solutions and technical features in each of the above embodiments can be singly or combined in case of conflict with the present invention, as long as they do not exceed the cognitive scope of those skilled in the art, they all belong to the equivalent embodiments within the protection scope of this application. .

In the several embodiments provided by the present invention, it should be understood that the related detection device (for example: IMU) and method disclosed may be implemented in other ways. For example, the embodiments of the remote control device described above are merely illustrative. For example, the division of the modules or units is only a logical function division. In actual implementation, there may be other division methods, such as multiple units or components. It can be combined or integrated into another system, or some features can be ignored or not implemented. In addition, the displayed or discussed mutual coupling or direct coupling or communication connection may be indirect coupling or communication connection through some interfaces, remote control devices or units, and may be in electrical, mechanical or other forms.

The units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the objectives of the solutions of the embodiments.

In addition, the functional units in the various embodiments of the present invention may be integrated into one processing unit, or each unit may exist alone physically, or two or more units may be integrated into one unit. The above-mentioned integrated unit can be implemented in the form of hardware or software functional unit.

If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer readable storage medium. Based on this understanding, the technical solution of the present invention essentially or the part that contributes to the existing technology or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium , Including several instructions to make a computer processor (processor) execute all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage media include: U disk, mobile hard disk, Read-Only Memory (ROM), Random Access Memory (RAM, Random Access Memory), magnetic disks or optical disks and other media that can store program codes.

The above are only the embodiments of the present invention, which do not limit the scope of the present invention. Any equivalent structure or equivalent process transformation made by using the content of the description and drawings of the present invention, or directly or indirectly applied to other related technologies In the same way, all fields are included in the scope of patent protection of the present invention.

Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that: The technical solutions recorded in the foregoing embodiments can still be modified, or some or all of the technical features can be equivalently replaced; and these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the technical solutions of the embodiments of the present invention. scope.

Claims

An airspace detection method, characterized in that the method includes:

Respond to flight control commands to make the UAV fly in a preset flight mode;

In the process of flying in the preset flight mode, acquiring an environment image corresponding to the preset field of view above the drone;

Identifying the category to which the object in the environmental image belongs;

If there is no object belonging to the obstacle category in the environment image, it is determined that the airspace above the drone is a flyable airspace.
The method according to claim 1, wherein the method further comprises:

If it is recognized that there is an object belonging to the obstacle category in the environment image, it is determined that the airspace above the drone is a non-flyable airspace.
The method according to claim 1, wherein the responding to the flight control instruction to make the drone fly in a preset flight mode is used to obtain the environment in the preset circular field of view above the drone image.
The method according to claim 3, characterized in that, before the acquiring an environment image corresponding to a preset field of view above the drone, the method further comprises:

The field of view angle of the preset annular field of view is determined according to the field of view angle of the up-view camera configured on the drone.
The method according to claim 4, wherein the determining the field of view angle of the preset annular field of view according to the field of view angle of the up-view camera configured on the drone comprises:

According to the field of view angle of the up-view camera, the volume of the drone, and the flying environment of the drone, the field of view angle of the preset annular field of view is determined.
The method according to claim 4, wherein the responding to the flight control control to make the UAV fly in a preset flight mode comprises:

In response to the first flight control instruction, the UAV rotates and flies one circle in the first position.
The method according to claim 6, wherein, in the process of flying in the preset flight mode, acquiring an environment image corresponding to the preset field of view above the drone during the flight ,include:

During the rotating flight at the first position, multiple images taken at intervals of a preset angle are acquired, and the multiple images acquired at the first position correspond to the first preset ring above the drone Field of view

Merging the multiple images is performed to obtain the environmental image.
The method according to claim 7, wherein the responding to the control of the aircraft to make the UAV fly in a preset flight mode further comprises:

In response to the second flight control command, the drone is caused to fly to a second position that is a preset distance from the first position.
The method according to claim 8, characterized in that, in the process of flying in the preset flight mode, acquiring an environment image corresponding to the preset field of view above the drone during the flight, and include:

Acquiring an image taken at the second position, where the image acquired at the second position corresponds to a preset angle of view above the drone;

The environment image is generated based on a plurality of images acquired at the first location and an image acquired at the second location.
The method according to claim 7, wherein the responding to the flight control instruction to make the UAV fly in a preset flight mode, further comprises:

In response to the third flight control instruction, the UAV is made to rotate and fly one circle in situ at a second position that is a preset distance from the first position.
The method according to claim 10, characterized in that, in the process of flying in the preset flight mode, acquiring an environment image corresponding to the preset field of view above the drone during the flight, and include:

During the rotating flight at the second position, multiple images taken at intervals of a preset angle are acquired, and the multiple images acquired at the second position correspond to the second preset ring above the drone Field of view

The multiple images respectively acquired at the first position and the second position are merged to obtain the environment image.
The method according to claim 8 or 10, wherein the preset distance is not less than the difference between the first distance and the second distance, wherein the first distance is the center of the drone and the The distance between the up-view cameras, and the second distance is the maximum distance between the center of the fuselage and the wing edge of the drone.
A movable platform, characterized in that, the mobile platform includes: a body, a power system, and a control device;

The power system is arranged on the body and used to provide power for the movable platform;

The control device includes a memory and a processor;

The memory is used to store a computer program;

The processor is configured to run a computer program stored in the memory to realize: responding to flight control instructions to make the UAV fly in a preset flight mode;

In the process of flying in the preset flight mode, acquiring an environment image corresponding to the preset field of view above the drone;

Identifying the category to which the object in the environmental image belongs;

If there is no object belonging to the obstacle category in the environment image, it is determined that the airspace above the drone is a flyable airspace.
The platform according to claim 13, wherein the processor is further configured to:

If it is recognized that there is an object belonging to the obstacle category in the environment image, it is determined that the airspace above the drone is a non-flyable airspace.
The platform according to claim 13, wherein the body includes a camera capable of looking upwards;

The processor responds to the flight control instruction to make the drone fly in a preset flight mode, so that the processor obtains the environment image in the preset circular field of view above the drone;

The processor is also used for:

The field of view angle of the preset annular field of view is determined according to the field of view angle of the up-view camera.
The platform according to claim 15, wherein the processor is further configured to:

According to the field of view angle of the up-view camera, the volume of the drone, and the flying environment of the drone, the field of view angle of the preset annular field of view is determined.
The platform according to claim 15, wherein the processor is further configured to:

In response to the first flight control instruction, the UAV rotates and flies one circle in the first position.
The platform according to claim 17, wherein the processor is further configured to:

During the rotating flight at the first position, multiple images taken at intervals of a preset angle are acquired, and the multiple images acquired at the first position correspond to the first preset ring above the drone Field of view

Merging the multiple images is performed to obtain the environmental image.
The platform according to claim 18, wherein the processor is further configured to:

Respond to the second flight control instruction to cause the drone to fly to a second position that is a preset distance from the first position, wherein the preset distance is not less than the difference between the first distance and the second distance , Wherein the first distance is the distance between the center of the drone's fuselage and the up-view camera, and the second distance is the distance between the center of the fuselage and the edge of the drone's wing The maximum distance between.
The platform according to claim 19, wherein the processor is further configured to:

Acquiring an image taken at the second position, where the image acquired at the second position corresponds to a preset angle of view above the drone;

The environment image is generated based on a plurality of images acquired at the first location and an image acquired at the second location.
The platform according to claim 18, wherein the processor is further configured to:

In response to the third flight control command, the UAV makes a full-rotation flight in a second position that is a preset distance from the first position, wherein the preset distance is not less than the first distance and the first position. The difference between the two distances, the first distance is the distance between the center of the drone and the up-view camera, and the second distance is the center of the drone and the drone. The maximum distance between the edges of the wings.
The platform according to claim 21, wherein the processor is further configured to:

During the rotating flight at the second position, multiple images taken at intervals of a preset angle are acquired, and the multiple images acquired at the second position correspond to the second preset ring above the drone Field of view

The multiple images respectively acquired at the first position and the second position are merged to obtain the environment image.
An airspace detection equipment, characterized in that the equipment includes:

Memory, used to store computer programs;

The processor is configured to run a computer program stored in the memory to realize:

Respond to flight control commands to make the UAV fly in a preset flight mode;

In the process of flying in the preset flight mode, acquiring an environment image corresponding to the preset field of view above the drone;

Identifying the category to which the object in the environmental image belongs;

If there is no object belonging to the obstacle category in the environment image, it is determined that the airspace above the drone is a flyable airspace.
The device according to claim 23, wherein the processor is further configured to:

If it is recognized that there is an object belonging to the obstacle category in the environment image, it is determined that the airspace above the drone is a non-flyable airspace.
The device according to claim 23, wherein the processor responds to flight control instructions, so that the UAV will fly in a preset flight mode, so that the processor can obtain information above the UAV. Preset the environment image in the annular field of view;

The processor is also used for:

The field of view angle of the preset annular field of view is determined according to the field of view angle of the up-view camera configured on the drone.
The device according to claim 25, wherein the processor is further configured to:

According to the field of view angle of the up-view camera, the volume of the drone, and the flying environment of the drone, the field of view angle of the preset annular field of view is determined.
The device according to claim 25, wherein the processor is further configured to:

In response to the first flight control instruction, the UAV rotates and flies one circle in the first position.
The device according to claim 27, wherein the processor is further configured to:

During the rotating flight at the first position, multiple images taken at intervals of a preset angle are acquired, and the multiple images acquired at the first position correspond to the first preset ring above the drone Field of view

Merging the multiple images is performed to obtain the environmental image.
The device according to claim 28, wherein the processor is further configured to:

Respond to the second flight control instruction to cause the drone to fly to a second position that is a preset distance from the first position, wherein the preset distance is not less than the difference between the first distance and the second distance , The first distance is the distance between the center of the drone's fuselage and the up-view camera, and the second distance is the distance between the center of the fuselage and the edge of the drone's wing The maximum distance.
The device according to claim 29, wherein the processor is further configured to:

Acquiring an image taken at the second position, where the image acquired at the second position corresponds to a preset angle of view above the drone;

The environment image is generated based on a plurality of images acquired at the first location and an image acquired at the second location.
The device according to claim 28, wherein the processor is further configured to:

In response to the third flight control command, the UAV makes a full-rotation flight in a second position that is a preset distance from the first position, wherein the preset distance is not less than the first distance and the first position. The difference between the two distances, the first distance is the distance between the center of the drone and the up-view camera, and the second distance is the center of the drone and the drone. The maximum distance between the edges of the wings.
The device according to claim 31, wherein the processor is further configured to:

During the rotating flight at the second position, multiple images taken at intervals of a preset angle are acquired, and the multiple images acquired at the second position correspond to the second preset ring above the drone Field of view

The multiple images respectively acquired at the first position and the second position are merged to obtain the environment image.
A computer-readable storage medium, wherein the storage medium is a computer-readable storage medium, and program instructions are stored in the computer-readable storage medium, and the program instructions are used to implement any one of claims 1 to 12 The airspace detection method described in item.