WO2022036500A1

WO2022036500A1 - Flight assisting method for unmanned aerial vehicle, device, chip, system, and medium

Info

Publication number: WO2022036500A1
Application number: PCT/CN2020/109515
Authority: WO
Inventors: 吕熙敏; 李翔; 段武阳
Original assignee: 深圳市大疆创新科技有限公司
Priority date: 2020-08-17
Filing date: 2020-08-17
Publication date: 2022-02-24
Also published as: CN113795803B; CN113795803A

Abstract

A flight assisting method for an unmanned aerial vehicle (110, 600, 901), a device, a chip (800), a system (100, 900), and a medium. The flight assisting method comprises: obtaining flight status information of the unmanned aerial vehicle (110, 600, 901), the flight status information of the unmanned aerial vehicle (110, 600, 901) comprising a flight velocity vector of the unmanned aerial vehicle (110, 600, 901) (S201); obtaining obstacle information of a flight scene of the unmanned aerial vehicle (110, 600, 901) (S202); and sending the flight status information and the obstacle information to a display device (130, 700, 902) for display, the display of the flight status information and the obstacle information being used for assisting a user in performing the flight of the unmanned aerial vehicle (110, 600, 901) (S203).

Description

Flight assistance method, device, chip, system and medium for unmanned aerial vehicle

technical field

The present application relates to the technical field of unmanned aerial vehicles, and in particular, to a flight assistance method, device, chip, system and medium for unmanned aerial vehicles.

Background technique

A photographing device is mounted on the unmanned aerial vehicle, and the photographing device may be, for example, a wide-angle lens. By observing the field of view image captured by the wide-angle lens on the display device of the UAV, the user can fly the UAV from a first-person perspective.

When flying the UAV from the first-person perspective, users usually need to rely on their own sense and proficiency to control the UAV to avoid obstacles. However, for users without flying experience, it is difficult to control the UAV to avoid obstacles by relying on their own feeling and proficiency.

SUMMARY OF THE INVENTION

Embodiments of the present application provide a flight assistance method, device, chip, system and medium for an unmanned aerial vehicle, which are used to assist a user in flying the unmanned aerial vehicle and reduce the difficulty of user operation of the unmanned aerial vehicle.

In a first aspect, an embodiment of the present application provides a flight assistance method for an unmanned aerial vehicle, which is applied to an unmanned aerial vehicle, and the method includes:

acquiring flight status information of the unmanned aerial vehicle, where the flight status information includes a flight velocity vector, and the flight velocity vector is used to indicate the magnitude and direction of the flying velocity of the unmanned aerial vehicle;

Obtain the obstacle information of the flight scene where the unmanned aerial vehicle is located;

Send the flight status information and the obstacle information to a display device, and the display device can display the flight status information and the obstacle information, and the display of the flight status information and the obstacle information It is used to assist the user in the flight of the unmanned aerial vehicle.

In a second aspect, an embodiment of the present application provides a flight assistance method for an unmanned aerial vehicle, which is applied to a display device of an unmanned aerial vehicle, and the method includes:

The flight status information and the obstacle information are displayed, and the display of the flight status information and the obstacle information is used to assist the user to fly the UAV.

In a third aspect, an embodiment of the present application provides an unmanned aerial vehicle, including: a processor and a memory, where the memory is used to store instructions, and the processor invokes the instructions stored in the memory to perform the following operations:

Send the flight status information and the obstacle information to a display device, and the display device can display the flight status information and the obstacle information, and the display of the flight status information and the obstacle information It is used for assisting the user to fly the unmanned aerial vehicle.

In a fourth aspect, an embodiment of the present application provides a display device, including: a processor and a display device;

The processor is configured to obtain flight status information of the unmanned aerial vehicle, and obtain obstacle information of the flight scene where the unmanned aerial vehicle is located, the flight status information includes a flight speed vector, and the flight speed vector is used to indicate the The magnitude and direction of the flight speed of the UAV;

The display device is configured to display the flight status information and the obstacle information, and the display of the flight status information and the obstacle information is used to assist a user in flying the unmanned aerial vehicle.

In a fifth aspect, an embodiment of the present application provides a chip, including: a transceiver, a memory, and a processor;

the transceiver, used for data transmission and reception;

the memory for storing program instructions;

The processor is configured to invoke the program instructions in the memory and execute the flight assistance method for an unmanned aerial vehicle described in the embodiments of the present application according to the first aspect or the second aspect according to the data received by the transceiver The flight assistance method for the unmanned aerial vehicle described in the embodiments of the present application.

In a sixth aspect, an embodiment of the present application provides an unmanned aerial system, including the unmanned aerial vehicle described in the embodiment of the present application in the third aspect, the display device and the control terminal described in the embodiment of the present application in the fourth aspect above , the control terminal is used to control the flight of the unmanned aerial vehicle.

In a seventh aspect, an embodiment of the present application provides a computer-readable storage medium, including instructions, when the instructions are run on a computer, the computer is made to execute the flight assistance method for an unmanned aerial vehicle as described in the first aspect above Or the flight assistance method for an unmanned aerial vehicle as described in the second aspect above.

In an eighth aspect, an embodiment of the present application provides a computer program product containing instructions, which, when the instructions are run on a computer, cause the computer to execute the flight assistance method for an unmanned aerial vehicle as described in the first aspect above or as described in the first aspect. The flight assistance method of the unmanned aerial vehicle according to the above second aspect.

The flight assistance method, device, chip, system and medium of the unmanned aerial vehicle provided by the embodiments of the present application, on the unmanned aerial vehicle, obtain the flight status information of the unmanned aerial vehicle and the obstacle information of the flight scene where the unmanned aerial vehicle is located, and then fly the unmanned aerial vehicle. The state information and obstacle information are sent to the display device, and the display device can display the flight state information and obstacle information, wherein the flight state information includes a flight speed vector, and the flight speed vector is used to indicate the size and magnitude of the flight speed of the unmanned aerial vehicle. The display of direction, flight status information and obstacle information is used to assist the user to fly the UAV. Therefore, with the assistance of the flight status information and obstacle information of the UAV displayed by the display device, the user can more accurately grasp the flight status of the UAV, especially the flight speed of the UAV, and at the same time, know the flight status of the UAV. In the case of obstacles in the flight scene where the UAV is located, the UAV is operated to avoid obstacles and reduce the difficulty of user operation of the UAV.

Description of drawings

1 is a schematic diagram of an application scenario provided by an embodiment of the present application;

2 is a flowchart of a method for assisting flight of an unmanned aerial vehicle according to an embodiment of the present application;

3a is a schematic diagram of sub-regional display of a scene image, flight status information, obstacle information, and a virtual model of an obstacle in a flight assistance method for an unmanned aerial vehicle provided by an embodiment of the application;

3b is a schematic diagram of sub-regional display of a scene image, flight status information, obstacle information, and a virtual model of an obstacle in a flight assistance method for an unmanned aerial vehicle provided by another embodiment of the present application;

FIG. 4 is a flowchart of a flight assistance method for an unmanned aerial vehicle provided by another embodiment of the present application;

5a is a schematic diagram of displaying a flight velocity vector through a vector ball and projection in a flight assistance method for an unmanned aerial vehicle provided by another embodiment of the present application;

5b is a schematic diagram of displaying a flight velocity vector through a vector ball and projection in a flight assistance method for an unmanned aerial vehicle provided by another embodiment of the present application;

5c is a schematic diagram of displaying a flight velocity vector through a vector ball and projection in a flight assistance method for an unmanned aerial vehicle provided by another embodiment of the present application;

6 is a schematic structural diagram of an unmanned aerial vehicle provided by an embodiment of the present application;

FIG. 7 is a schematic structural diagram of a display device provided by an embodiment of the present application;

FIG. 8 is a schematic structural diagram of a chip provided by an embodiment of the present application;

FIG. 9 is a schematic structural diagram of an unmanned aerial system according to an embodiment of the present application.

detailed description

The technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present application.

It should be noted that when a component is referred to as being "fixed to" another component, it can be directly on the other component or there may also be a centered component. When a component is said to be "connected" to another component, it can be directly connected to the other component or there may be a centered component at the same time.

Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the technical field to which this application belongs. The terms used herein in the specification of the application are for the purpose of describing specific embodiments only, and are not intended to limit the application. As used herein, the term "and/or" includes any and all combinations of one or more of the associated listed items.

The flight assistance method, device, chip, system and medium for the unmanned aerial vehicle provided in the embodiments of the present application can be applied to the flight process of the unmanned aerial vehicle, and the unmanned aerial vehicle can be configured to fly by external force or fly by its own power, For example, it can be a traversing plane. FIG. 1 is a schematic diagram of an application scenario provided by an embodiment of the present application. As shown in FIG. 1 , FIG. 1 shows an unmanned aerial system 100 . The unmanned aerial system 100 includes an unmanned aerial vehicle 110 , a display device 130 and a control terminal 140 . Among them, the UAV 110 can wirelessly communicate with the display device 130 and the control terminal 140 .

The unmanned aerial vehicle 110 includes a power system 150, a flight control system 160, a frame, and a gimbal 120 carried on the frame.

The power system 150 may include one or more electronic governors (referred to as ESCs for short) 151, one or more propellers 153, and one or more electric motors 152 corresponding to the one or more propellers 153, wherein the electric motors 152 are connected to the Between the electronic governor 151 and the propeller 153, the motor 152 and the propeller 153 are arranged on the arm of the unmanned aerial vehicle 110; the electronic governor 151 is used to receive the driving signal generated by the flight control system 160, and provide driving according to the driving signal Electric current is supplied to the motor 152 to control the rotational speed of the motor 152 . The motor 152 is used to drive the propeller to rotate, thereby providing power for the flight of the UAV 110, and the power enables the UAV 110 to achieve one or more degrees of freedom movement. In certain embodiments, UAV 110 may rotate about one or more axes of rotation. For example, the above-mentioned rotation axis may include a roll axis (Roll), a yaw axis (Yaw), and a pitch axis (pitch). It should be understood that the motor 152 may be a DC motor or an AC motor. In addition, the motor 152 may be a brushless motor or a brushed motor.

The flight control system 160 may include a flight controller 161 and a sensing system 162. The sensing system 162 is used to measure the attitude information of the UAV 110, that is, the position information and state information of the UAV 110 in space, for example, three-dimensional position, 3D angle, 3D velocity, 3D accelerator and 3D angular velocity, etc. The sensing system 162 may include, for example, at least one of sensors such as a gyroscope, an ultrasonic sensor, an electronic compass, an inertial measurement unit (IMU), a vision sensor, a global navigation satellite system, and a barometer. For example, the global navigation satellite system may be the Global Positioning System (GPS). The flight controller 161 is used to control the flight of the UAV 110 , for example, the flight of the UAV 110 can be controlled according to the attitude information measured by the sensing system 162 . It should be understood that the flight controller 161 can control the UAV 110 according to pre-numbered program instructions, and can also control the UAV 110 by responding to one or more remote sensing signals from the control terminal 140 .

The pan/tilt head 120 may include a motor 122 . The pan/tilt 120 is used to carry a payload, and the payload may be, for example, a photographing device 123 . The flight controller 161 can control the movement of the gimbal 120 through the motor 122 . Optionally, as another embodiment, the pan/tilt 120 may further include a controller for controlling the movement of the pan/tilt 120 by controlling the motor 122 . It should be understood that the gimbal 120 may be independent of the UAV 110 , or may be a part of the UAV 110 . It should be understood that the motor 122 may be a DC motor or an AC motor. In addition, the motor 122 may be a brushless motor or a brushed motor. It should also be understood that the gimbal 120 may be located on the top of the UAV 110 or on the bottom of the UAV 110 .

The photographing device 123 may be, for example, a device for capturing images such as a camera or a video camera, and the photographing device 123 may communicate with the flight controller 161 and take pictures under the control of the flight controller 161 . The photographing device 123 in this embodiment at least includes a photosensitive element, such as a complementary metal oxide semiconductor (Complementary Metal Oxide Semiconductor, CMOS) sensor or a charge-coupled device (Charge-coupled Device, CCD) sensor. It can be understood that the photographing device 123 can also be directly fixed on the unmanned aerial vehicle 110, so that the gimbal 120 can be omitted.

The display device 130 is located at the ground end of the unmanned aerial vehicle 100 , can communicate with the unmanned aerial vehicle 110 in a wireless manner, and can be used to display the attitude information of the unmanned aerial vehicle 110 . In addition, the image captured by the capturing device 123 may also be displayed on the display device 130 . It should be understood that the display device 130 may be an independent device (eg, a head-mounted display device), or may be integrated in the control terminal 140 .

The control terminal 140 is located at the ground end of the unmanned aerial vehicle system 100 , and can communicate with the unmanned aerial vehicle 110 in a wireless manner, so as to remotely control the unmanned aerial vehicle 110 .

It should be understood that the above naming of the components of the unmanned aerial system is only for the purpose of identification, and should not be construed as a limitation on the embodiments of the present application.

During the flight of the unmanned aerial vehicle, the unmanned aerial vehicle flies according to the received control command of the control terminal, and at the same time sends the image captured by the photographing device to the display device. The display device outputs the scene image from the first-person perspective with the UAV as the observer. The user controls the UAV to avoid obstacles to fly based on the scene image from the first-person perspective and his own experience. The scene image is usually where the UAV is located. 3D live-action image of the flight scene. Unmanned aerial vehicle users, especially those who have no experience in flying through drones, rely only on the scene images from the first-person perspective with the unmanned aerial vehicle as the observer to fly the unmanned aerial vehicle, which is difficult to get started and operate.

The flight assistance method, device, chip, system, and medium for an unmanned aerial vehicle provided by the embodiments of the present application acquire the flight status information of the unmanned aerial vehicle and the obstacle information of the flight scene where the unmanned aerial vehicle is located, and the display device comprehends the flight status information and the obstacle information of the unmanned aerial vehicle. The obstacle information is displayed, wherein the display of the flight status information and the obstacle information is used to assist the user to fly the unmanned aerial vehicle, and the flight status information includes the flight speed vector. Therefore, through the display of the flight status information and obstacle information, it helps the user to grasp the flight status of the UAV and the obstacles in the flight scene, and the user can control more accurately based on the displayed flight status information and obstacle information. The UAV avoids obstacles to fly (including flying through obstacles), which effectively reduces the difficulty of getting started and operating the UAV, and at the same time does not make the UAV flight unchallenging.

Some embodiments of the present application will be described in detail below with reference to the accompanying drawings. The following embodiments and features of the embodiments may be combined with each other without conflict.

FIG. 2 is a flowchart of a method for assisting the flight of an unmanned aerial vehicle provided by an embodiment of the present application. As shown in FIG. 2 , the method of this embodiment can be applied to an unmanned aerial vehicle, and the method of this embodiment may include:

S201. Acquire flight status information of the unmanned aerial vehicle, where the flight status information of the unmanned aerial vehicle includes a flight speed vector of the unmanned aerial vehicle.

In this embodiment, when it is detected that the unmanned aerial vehicle is in a flying state, or when a user's flight assistance request is received, or when it is detected that the preset flight assistance mode of the unmanned aerial vehicle is turned on, for example, the unmanned aerial vehicle can The sensor system (for example, the sensor system 162 in FIG. 1 ) on the device acquires the flight status information of the UAV. It should be noted that the flight status information is real-time flight status information of the unmanned aerial vehicle.

The flight state information of the UAV includes the flight speed vector of the UAV, and the flight speed vector is used to indicate the magnitude and direction of the flight speed of the UAV.

Optionally, the flying speed of the unmanned aerial vehicle may include the linear velocity of the flying of the unmanned aerial vehicle.

Optionally, in addition to the linear speed of the flight of the UAV, the flight speed of the UAV may also include the angular velocity and/or acceleration of the flight of the UAV.

Optionally, in addition to the flight speed vector of the UAV, the flight status information of the UAV may also include the flight altitude, real-time position, and the like of the UAV.

The flight assist mode may be manually turned on or off by the user. For example, the user may choose to turn on or turn off the flight assist mode according to the difficulty of the UAV flying in the current flight scenario. The flight assist mode can also be turned on or off by the flight control system (such as the flight control system 160 in FIG. 1 ) of the UAV, for example, the flight control system controls the flight according to the user’s flight record information (such as flight duration, flight score) When the assist mode is turned on or off, when the flight duration is less than the preset flight duration threshold or the flight score is less than the preset flight score threshold, the flight control system controls the flight assist mode to turn on, and vice versa, controls the flight assist mode to turn off.

S202. Obtain obstacle information of the flight scene where the unmanned aerial vehicle is located.

In this embodiment, when it is detected that the unmanned aerial vehicle is in a flying state, or a flight assistance request from a user is received, or it is detected that the flight assistance mode of the unmanned aerial vehicle is turned on, the sensor on the unmanned aerial vehicle can be used The system obtains the obstacle information of the flight scene where the UAV is located. For example, the radar device in the sensing system can measure whether there is an obstacle near the UAV, and if there is, the orientation of the obstacle relative to the UAV can be further measured. For another example, a scene image of the flight scene can also be photographed by a photographing device (eg, photographing device 123 in FIG. 1 ), and whether there is an obstacle near the UAV is determined by performing obstacle recognition on the scene image. Exemplarily, the flight scene where the unmanned aerial vehicle is located may refer to a three-dimensional space range of a preset size centered on the unmanned aerial vehicle, for example, a three-dimensional space range centered on the unmanned aerial vehicle and a predetermined distance as a spherical radius. Exemplarily, the flight scene where the unmanned aerial vehicle is located may refer to the spatial range in the direction of the current speed of the unmanned aerial vehicle. In other words, the obstacle information of the flight scene where the UAV is located may be the obstacle information in the direction of the current speed of the UAV. Obstacles can be dynamic objects or static objects, such as mountains, bridges, and flying birds.

It should be pointed out that the execution order of the above operations S201 and S202 can be interchanged.

S203. Send the flight status information and obstacle information to a display device, where the display device can display the flight status information and obstacle information, and the display of the flight status information and obstacle information is used to assist the user in flying the UAV.

In this embodiment, the acquired flight state information and obstacle information may be sent to a display device (eg, the display device 130 in FIG. 1 ), and the display device may, after receiving the flight state information and obstacle information, update the flight state information and obstacle information. to display. Accordingly, the user can view the flight status information and obstacle information through the display device, and with the aid of the displayed flight status information and obstacle information, adjust the flight attitude of the UAV (such as flight direction, flight speed, flight altitude, etc.). etc.), and more confidently carry out the obstacle avoidance flight of the UAV. In order to ensure the flight assistance effect, the flight status information and obstacle information can be obtained synchronously, and the synchronously obtained flight status information and obstacle information can be sent to the display device for display.

In this embodiment, the display device displays the flight status information, including the display device displaying the flight speed vector of the unmanned aerial vehicle. Therefore, the user can intuitively and accurately grasp the size and direction of the flight speed of the UAV based on the displayed flight speed vector, instead of judging the flight speed of the UAV only based on their own visual perception, and then can judge the flight speed of the UAV according to the displayed flight speed. Vector and obstacle information, adjust the size and direction of the flight speed of the UAV, and realize the obstacle avoidance flight of the UAV.

The flight assistance method for an unmanned aerial vehicle provided by this embodiment, by acquiring the flight state information of the unmanned aerial vehicle and the obstacle information of the flight scene where the unmanned aerial vehicle is located, and sending the flight state information and the obstacle information to the display device, the display device can The flight status information and obstacle information are displayed, so as to effectively assist the user to fly the UAV through the displayed flight status information (especially the flight speed vector) and obstacle information. With the aid of the displayed flight status information and obstacle information, users without flight experience can become familiar with the flight of UAVs more quickly. With the aid of the displayed flight status information and obstacle information, users with flight experience can also It can improve its ability to fly unmanned aerial vehicles more quickly.

In some embodiments, the obstacle information in the flight scene where the UAV is located includes the relative positions between the obstacles in the flight scene and the UAV.

Optionally, the relative position between the obstacle and the UAV may be the distance between the obstacle and the origin of the body coordinate system of the UAV.

Optionally, the relative position between the obstacle and the unmanned aerial vehicle may be the distance between the obstacle and one or more preset key positions on the unmanned aerial vehicle, for example, one or more key positions on the unmanned aerial vehicle may be Including one or more of the nose, propeller, and tail of the unmanned aerial vehicle.

Optionally, a cuboid model corresponding to the unmanned aerial vehicle may be pre-built, and the relative position between the obstacle and the unmanned aerial vehicle may include the distance between the obstacle and each edge or face of the cuboid model.

Optionally, the relative position between the obstacle and the UAV may include the distance between the three-axis directions of the UAV (ie, the roll axis, the yaw axis, and the pitch axis) and the obstacle.

When the obstacle information includes the relative position between the obstacle and the unmanned aerial vehicle, a possible implementation of the above S203 is: sending the flight speed vector of the unmanned aerial vehicle and the relative position between the obstacle and the unmanned aerial vehicle For the display device, the display device can display the flight speed vector of the UAV and the relative position of the obstacle and the UAV in the flight scene where the UAV is located. Therefore, this embodiment provides users with more intuitive flight status information and obstacle information by displaying the flight speed vector of the UAV and the relative position between the obstacle and the UAV, and improves the flight assistance of the UAV. Effect.

In some embodiments, scene images of the flight scene may also be acquired. Wherein, the scene image of the flight scene can be acquired by the photographing device on the unmanned aerial vehicle. The scene image includes a scene image from a first-person perspective with the unmanned aerial vehicle as the observer, and the range of the scene image is related to the viewing angle of the photographing device, which is not limited herein.

After acquiring the scene image of the flight scene, a possible implementation manner of the above S203 is: sending the flight status information of the unmanned aerial vehicle, the obstacle information of the flight scene where the unmanned aerial vehicle is located, and the scene image of the flight scene to the display device, The display device can display the scene image, and display the flight status information and obstacle information on the displayed scene image, and the display of the flight status information and obstacle information on the scene image is used to show the flight of the UAV to the user Status (especially the flight speed) and the relative position between the obstacles in the flight scene and the UAV to assist the user in the obstacle avoidance flight of the UAV.

In this embodiment, after the display device receives the flight status information of the unmanned aerial vehicle, the obstacle information of the flight scene where the unmanned aerial vehicle is located, and the scene image of the flight scene, on the displayed scene image, the flight status information and the obstacle and the obstacle are displayed. Relative position between UAVs. The display device can display the flight status information at a designated position on the scene image (for example, the center of the scene image, so that the user can notice the flight status information), and can display the relative position between the obstacle and the UAV on the scene image. near the corresponding obstacle. For example, when the distance between obstacle A and the front of the UAV is 10 meters, the distance can be displayed on obstacle A on the scene image. For another example, when the UAV traverses the traversable channel of obstacle B (for example, obstacle B is a bridge body, and the traversable channel of obstacle B is a bridge hole), the traversable channel of obstacle B can be displayed on the scene image. On each surface of the , display the distance between each surface and the UAV (for example, display the distance between the two sides and the top surface of the bridge opening and the UAV). Therefore, the flight status information and obstacle information of the unmanned aerial vehicle can be displayed to the user more vividly and intuitively, and the flight assistance effect of the unmanned aerial vehicle can be improved.

In this embodiment, when the flight state information includes the flight speed vector, the user can adjust the flight speed vector displayed on the display device by adjusting the flight speed of the UAV until the flight speed vector displayed on the display device satisfies the user Require. For example, when flying through, the user can adjust the flight speed of the UAV so that the direction of the flight speed vector displayed on the display device is aligned with the traversable channel of the obstacle in the scene image, and the direction of the flight speed vector is aligned with the obstacle The traversable channel of , indicates that the UAV is flying towards the traversable channel of the obstacle. As the distance between the UAV and the traversable channel is closer, the alignment operation is more accurate, enabling the UAV to pass through the obstacle. The traversable channel effectively reduces the operational difficulty of traversing flight.

In some embodiments, in addition to the relative position between the obstacle and the UAV, the obstacle information of the flight scene where the UAV is located also includes at least one of the type, shape, and size of the obstacle in the flight scene.

When the obstacle information of the flight scene where the UAV is located also includes at least one of the type, shape and size of the obstacle in the flight scene, the obstacle can be established according to at least one of the type, shape and size of the obstacle A possible implementation of the above S203 is: sending the flight state information, obstacle information and the virtual model of the obstacle to the display device, and the display device can display the flight state information, obstacle information and the virtual model of the obstacle. Model display, wherein, the display of flight status information, obstacle information and the virtual model of the obstacle is used to show the user the flight status (especially the flight speed) of the UAV, the obstacles, and the relationship between the obstacles and the UAV. The relative position between the two to assist the user in the obstacle avoidance flight of the unmanned aerial vehicle.

In this embodiment, a virtual model of the obstacle may be established according to at least one of the type, shape, and size of the obstacle in the obstacle information. Specifically, if the obstacle information only includes the type of the obstacle in addition to the relative position between the obstacle and the UAV, a virtual model that conforms to the type of the obstacle can be constructed, for example, when the type of the obstacle is a bird , a virtual model of the obstacle can be built based on the usual shape and size of the bird. If the obstacle information only includes the shape or size of the obstacle in addition to the relative position between the obstacle and the UAV, a virtual model that conforms to the shape or size of the obstacle can be constructed. The specific construction method of the virtual model is here. No restrictions.

In this embodiment, after the virtual model of the obstacle is constructed, the flight state information, the relative position between the obstacle and the UAV, and the virtual model of the obstacle can be sent to the display device, and the display device receives the flight state information, After the relative position between the obstacle and the UAV and the virtual model of the obstacle, the flight status information, the relative position between the obstacle and the UAV and the virtual model of the obstacle are displayed, and the flight status is displayed to the user. Based on the information and the relative position between the obstacle and the UAV, the virtual model of the obstacle is displayed to the user. According to the virtual model of the obstacle, the user can grasp more information about the obstacle and can control the unmanned aerial vehicle more accurately. The aircraft avoids or traverses obstacles.

On the basis of obtaining the scene image of the flight scene and the virtual model of the obstacle, a possible implementation of the above S203 is: sending the flight status information, the obstacle information, the virtual model of the obstacle and the scene image to the display device, The display device can display the scene image in the first preset display area, and display the flight status information and obstacle information on the displayed scene image, and the scene image, flight status information and obstacle information in the first preset display area The display of object information is used to show the user the flight scene, the flight status of the UAV in the flight scene, the obstacles and the relative positions between the obstacles and the UAV, and the display device can be displayed in the second preset display area. The virtual model of the obstacle is displayed, and the display of the virtual model in the second preset area is used to show the obstacle in the flight scene to the user.

In this embodiment, the display area of the display device can be divided into two parts, one part is the first preset display area, the other part is the second preset display area, and the first preset display area is used for displaying scene images and flight status information and the relative position between the obstacle and the UAV in the obstacle information, and the second preset display area is used to display the virtual model of the obstacle in the flight scene. For example, as shown in FIG. 3a, the first preset display area and the second preset display area occupy different display areas of the display device respectively, and the user can view scene images, flight status information and obstacles by viewing different display areas. While viewing the relative positions of the obstacles and the UAV in the information, the virtual model of the obstacles in the flight scene can be viewed, which avoids the excessive stacking and complexity of the content displayed on the display device, and makes the display content more cool.

Optionally, the area of the first preset display area is larger than the area of the second preset display area. Therefore, the user can view the scene image, flight status information and the relative position between the obstacle and the UAV in the first preset display area with a larger area, and can view it in the second preset display area with a smaller display area To the virtual model of the obstacle to ensure the display effect of the scene image, and then to ensure the user's first-person view of the UAV flight experience.

In another possible implementation manner, the second preset display area is a partial display area in the first preset display area. For example, as shown in FIG. 3b, the first preset display area may be the entire display area of the display device, and the second preset display area may be, for example, a part of the display area in the lower right corner, which is equivalent to the virtual model of the obstacle superimposed on the lower right corner of the scene image. part of the scene, the virtual model of the obstacle may occlude part of the scene image.

Optionally, in addition to displaying the virtual model of the obstacle in the second preset display area, a real three-dimensional image of the obstacle may also be displayed in the second preset display area, so that the user can view the real situation of the obstacle at various angles. For example, the current geographic location of the UAV can be obtained, images related to the current geographic location can be obtained from the Internet, obstacles can be identified in these images, and three-dimensional images of obstacles can be obtained; Shooting devices are arranged at a plurality of fixed positions (eg, preset positions on a building), and three-dimensional images of obstacles are collected by these shooting devices, or three-dimensional images of obstacles are obtained by other unmanned aerial vehicles flying together.

In some embodiments, on the basis that the flight state information of the unmanned aerial vehicle includes the flight speed vector of the unmanned aerial vehicle, a predicted flight trajectory can be generated according to the flight velocity vector of the unmanned aerial vehicle, and the predicted flight trajectory can be sent to the display device , the display device can display the predicted flight trajectory, wherein the display of the predicted flight trajectory is used to assist the user to fly the unmanned aerial vehicle.

In this embodiment, it can be assumed that the unmanned aerial vehicle flies at a constant speed within the preset time period, and according to the flight speed vector of the unmanned aerial vehicle, the predicted flight trajectory of the unmanned aerial vehicle within the preset time period can be calculated. Send the predicted flight trajectory to the display device, and the display device will display the predicted flight trajectory after receiving the predicted flight trajectory, so that the user can grasp the predicted flight trajectory of the UAV and adjust the flight direction of the UAV based on the predicted flight trajectory . After viewing the flight prediction trajectory on the display device, the user can predict the direction and area that the UAV will fly to in advance. If the user does not want the UAV to fly according to the flight prediction trajectory, the flight of the UAV can be adjusted. .

Further, according to the generated flight prediction trajectory, it can be determined whether the unmanned aerial vehicle exists in the risk of collision with obstacles in the flight scene, and if so, the control terminal of the display device and/or the unmanned aerial vehicle (that is, in FIG. The control terminal 140) sends a collision reminder message to remind the user to adjust the flight speed of the UAV to avoid collision between the UAV and the obstacle. Wherein, whether the UAV has a risk of colliding with an obstacle can be determined according to whether the predicted flight trajectory intersects with the obstacle or whether the shortest distance between the predicted flight trajectory and the obstacle is less than a preset distance threshold.

Further, if there is a risk of collision between the UAV and the obstacle, the flight obstacle avoidance trajectory is generated according to the relative position between the UAV and the obstacle, and the flight obstacle avoidance trajectory is sent to the display device. The flight obstacle avoidance trajectory is displayed. The display of the flight obstacle avoidance trajectory is used to assist the user in the obstacle avoidance flight of the unmanned aerial vehicle. Therefore, when there is a risk of collision between the UAV and the obstacle, the flight obstacle avoidance trajectory is provided to the user, so that the user can control the UAV to fly according to the prompts of the flight obstacle avoidance trajectory to avoid the occurrence of the UAV and the obstacle. collision. The flight obstacle avoidance trajectory can be generated according to the flight speed vector of the UAV and the relative position between the UAV and the obstacle, and the specific process is not limited.

Further, after generating the flight obstacle avoidance trajectory, the flight controller in the unmanned aerial vehicle (for example, the flight controller 161 in FIG. 1 ) can control the unmanned aerial vehicle to fly according to the flight obstacle avoidance trajectory, thereby realizing the automatic avoidance of the unmanned aerial vehicle. obstacle flight.

Further, after the flight obstacle avoidance trajectory is generated, the target stick amount can be determined according to the flight obstacle avoidance trajectory, the current stick amount of the control terminal is obtained, the stick amount increment is determined according to the current stick amount and the target stick amount, and the stick amount is incremented. It is sent to the display device, and the display device can display the increment of the stick amount, wherein the display of the increment of the stick amount is used to assist the user to control the unmanned aerial vehicle to fly according to the flight obstacle avoidance trajectory.

In this embodiment, the flight speed vector of the unmanned aerial vehicle corresponding to the flight obstacle avoidance trajectory can be determined, and then according to the preset conversion relationship between the flight speed vector and the stick amount, the target stick amount is determined, and the difference between the flight speed vector and the stick amount is determined. The conversion relationship between them is not limited here. The current rod amount can be obtained from the control command received from the control terminal. The stroke increment is obtained based on the difference between the current stroke and the target stroke. The display device displays the rod increment after receiving the rod increment. Therefore, the user can perform corresponding control operations at the control terminal according to the displayed stick increment, so that the UAV flies according to the flight obstacle avoidance trajectory.

Further, if the unmanned aerial vehicle has the risk of colliding with the obstacle, obtain the estimated flight time of the unmanned aerial vehicle to the position where it collides with the obstacle, and if the estimated flying time is less than or equal to the preset time threshold, control the The unmanned aerial vehicle stops, so that when it is too late to control the unmanned aerial vehicle to fly according to the flight obstacle avoidance trajectory, the unmanned aerial vehicle can be stopped in time to avoid the collision between the unmanned aerial vehicle and the obstacle. Among them, the position where the predicted flight trajectory of the UAV intersects the obstacle is the position where the UAV collides with the obstacle. The estimated flight time for the human aircraft to reach this position.

In some embodiments, during the flight of the unmanned aerial vehicle, if it is detected that the unmanned aerial vehicle passes through an obstacle in the flight scene, the shooting on the gimbal of the unmanned aerial vehicle (ie the gimbal 120 in FIG. 1 ) is performed. A device that captures obstacle-passing images and/or obstacle-passing videos of UAVs, and stores the obstacle-passing images and/or obstacle-passing videos locally in the UAV or sends them to the server for storage or to the control of the UAV Terminal storage for user viewing. Among them, the sensing system on the unmanned aerial vehicle can be used to detect whether the unmanned aerial vehicle is passing through obstacles in the flight scene. For example, when the unmanned aerial vehicle passes through the bridge hole, it can detect whether the unmanned aerial vehicle is about to enter the bridge hole through the radar device, light sensing device, etc. on the unmanned aerial vehicle. Obstacle crossing images and/or obstacle crossing videos.

In some embodiments, on the basis that the flight state information of the UAV includes the flight speed vector of the UAV, if it is detected that the direction of the flight speed vector of the UAV changes, control the movement on the gimbal of the UAV. The shooting direction of the shooting device is turned to the direction of the flight speed vector, and the display device is controlled to display the scene image shot by the shooting device. The image center of the scene image displayed by the display device is the direction of the flight velocity vector of the unmanned aerial vehicle. Therefore, the image center can be used to assist the user in determining the direction of the flying velocity vector of the unmanned aerial vehicle.

In this embodiment, the gimbal of the unmanned aerial vehicle can be rotated, so that the shooting direction of the shooting device on the gimbal is consistent with the direction of the flight speed vector of the unmanned aerial vehicle. Since the scene image captured by the shooting device is a three-dimensional image, When the shooting direction of the shooting device is consistent with the direction of the flight speed vector of the unmanned flight speed, the image center of the scene image captured by the shooting device points to the direction of the flight speed vector of the unmanned aerial vehicle. For example, affected by the wind speed, the flight speed vector of the UAV is shifted to the right, and the gimbal rotates to the right, so that the shooting direction of the shooting device on the gimbal is consistent with the direction of the flight speed vector, and the user will clearly feel The scene image displayed by the display device (or the display screen where the scene image is located) moves. Therefore, when the user feels the movement of the scene image displayed by the display device, the user can determine the flight of the UAV through the image center of the scene image. The direction of the velocity vector. For example, when crossing an obstacle, the user can adjust the flight speed vector of the UAV on the remote control terminal, so that the center of the image of the scene image displayed in the display image is aligned with the traversable channel of the obstacle. The closer the distance of the traversable channel, the more accurate the adjustment process of the user, and finally the unmanned aerial vehicle can fly through the traversable channel.

FIG. 4 is a flowchart of a flight assistance method for an unmanned aerial vehicle provided by another embodiment of the present application. As shown in Figure 4, the method is applied to the display device of the unmanned aerial vehicle, and the method may include:

S401. Acquire flight status information of the unmanned aerial vehicle, where the flight status information of the unmanned aerial vehicle includes a flight speed vector of the unmanned aerial vehicle.

Among them, the flight speed vector is used to indicate the magnitude and direction of the flight speed of the UAV.

In this embodiment, flight status information from the unmanned aerial vehicle can be received. For example, the UAV can send its own flight status information directly to the display device, or can send its own flight status information to the display device via a signal forwarding device (for example, a repeater). For details of the flight status information, reference may be made to the relevant description of S201, which will not be repeated.

S402. Obtain obstacle information of the flight scene where the unmanned aerial vehicle is located.

In this embodiment, the obstacle information of the flight scene where the unmanned aerial vehicle is located can be received from the unmanned aerial vehicle. For example, the UAV can directly send obstacle information of its own flight scene to the display device, or can send the obstacle information of its own flight scene to the display device via a signal forwarding device (eg, a repeater). For details of the obstacle information, reference may be made to the relevant description of S202, and details are not repeated here.

It should be pointed out that the execution order of the above operations S401 and S402 can be interchanged.

S403 , displaying the flight status information and the obstacle information, and the display of the flight status information and the obstacle information is used to assist the user to fly the unmanned aerial vehicle.

In this embodiment, after receiving the flight status information and obstacle information of the UAV, the display device displays the flight status information and obstacle information. Accordingly, the user can view the flight status information and obstacle information through the display device. With the aid of the displayed flight status information and obstacle information, the flight attitude of the unmanned aerial vehicle can be adjusted, and the obstacle avoidance flight of the unmanned aerial vehicle can be carried out with more certainty. In order to ensure the flight assistance effect, the flight status information and obstacle information obtained at the same time can be displayed synchronously.

The flight assistance method for the unmanned aerial vehicle provided by this embodiment effectively assists the user in the flight of the unmanned aerial vehicle through the displayed flight status information and obstacle information. With the aid of the displayed flight status information (especially the flight speed vector) and obstacle information, users without flight experience can quickly become familiar with the flight of UAVs, and users with flight experience can be familiar with the flight status information and obstacles displayed in the displayed flight status information and obstacles. With the assistance of object information, it can also improve its ability to fly unmanned aerial vehicles more quickly.

In some embodiments, the obstacle information in the flight scene where the UAV is located includes the relative positions between the obstacles in the flight scene and the UAV. Optionally, the relative position between the obstacle and the UAV may be the distance between the obstacle and the origin of the body coordinate system of the UAV; or, the relative position between the obstacle and the UAV may be the obstacle The distance between the object and one or more preset key locations on the UAV, for example, the one or more key locations on the UAV may include one or more of the nose, propeller, and tail of the UAV ; Or, the cuboid model corresponding to the unmanned aerial vehicle can be pre-built, and the relative position between the obstacle and the unmanned aerial vehicle can include the distance between the obstacle and each edge or face of the cuboid model; or, the distance between the obstacle and the unmanned aerial vehicle The relative position between the two can include the distance between the three-axis directions of the UAV (ie, the roll axis, the yaw axis, and the pitch axis) and the obstacle.

In some embodiments, a scene image of the flight scene where the UAV is located is obtained. Among them, the scene image of the flight scene from the unmanned aerial vehicle may be received.

On the basis of obtaining the scene image, the flight state information and the obstacle information, a possible implementation of the above S403 is: displaying the scene image, and displaying the flight state information and the obstacle information on the displayed scene image , the display of the flight status information and obstacle information on the scene image is used to show the user the flight status of the UAV and the relative position between the obstacles and the UAV in the flight scene, so as to assist the user to conduct the UAV Obstacle avoidance flight.

In some embodiments, virtual models of obstacles in the flight scene where the UAV is located are obtained. Among them, the virtual model of the obstacle sent by the unmanned aerial vehicle can be received.

On the basis of obtaining the virtual model of the obstacle, the flight status information and the obstacle information, a possible implementation of the above S403 is: displaying the flight status information, the obstacle information and the virtual model of the obstacle, wherein the flight status The display of status information, obstacle information and the virtual model of the obstacle is used to show the user the flight status of the UAV, obstacles, and the relative positions of the obstacles and the UAV, so as to assist the user to conduct the UAV. Obstacle avoidance flight.

Optionally, the acquired obstacle information may further include at least one of the type, shape, and size of the obstacle in the flight scene where the unmanned aerial vehicle is located. Therefore, the display device can also construct a virtual model of the obstacle according to at least one of the type, shape, and size of the obstacle.

On the basis of obtaining the scene image of the flight scene, the virtual model of the obstacle, the flight state information and the obstacle information, a possible implementation of the above S303 is: displaying the scene image in the first preset display area, And display the flight status information and obstacle information on the displayed scene image. The display of the scene image, flight status information and obstacle information in the first preset display area is used to show the user the flight scene and the information in the flight scene. The flight status of the UAV, the obstacles and the relative positions between the obstacles and the UAV; the virtual model of the obstacle is displayed in the second preset display area, and the display of the virtual model in the second preset area uses It is used to show the user the obstacles in the flight scene. Wherein, for the first preset display area and the second preset display area, reference may also be made to the content related to the first preset display area and the second preset area in the above-mentioned corresponding embodiments, which will not be repeated.

In some embodiments, the flight speed vector is the size and direction of the flight speed of the UAV in the three-dimensional space. In order to improve the display effect of the flight speed vector on the display device, the flight speed vector can be projected onto a two-dimensional coordinate plane, The projection of the flight velocity vector on the coordinate plane is obtained, and the projection is displayed. The display of the projection is used to assist the user in determining the direction of the flight speed vector. For example, when the projection points to the upper right of the coordinate plane, it means that the UAV is flying forward and to the right. The change of the length of the projection reflects the change of the size of the flight speed vector, and the change of the direction of the projection reflects the change of the direction of the flight speed vector.

Optionally, the coordinate plane includes the YOZ plane of the body coordinate system of the unmanned aerial vehicle, or the YOZ plane of the geodetic coordinate system, so as to fit the user's habit of judging directions.

In a possible implementation manner, in the process of displaying the projection, the projection of the flight speed vector on the coordinate plane can be displayed through a preset vector mark, wherein the projection is the center of the vector mark and the coordinate plane The connection between the upper coordinate origins, the center of the vector marker is used to assist the user to determine the direction of the flight speed vector.

In this embodiment, the vector mark is a preset graphic, and when the projection changes with the flight speed vector of the UAV, including the change of the length of the projection and the change of the direction of the projection, the position of the vector mark on the coordinate plane also changes. The user can grasp the change of the flight speed vector according to the position change of the vector mark or the position change of the center of the vector mark. Therefore, the change of the flight speed vector is made more obvious by the vector mark, which makes the display of the flight speed vector more intuitive.

Optionally, since the relative speed of the flight speed vector changes greatly when the flight speed vector is small (for example, the acceleration is large), the flight speed vector is relatively unstable, and the vector mark changes frequently on the coordinate plane, which affects the user's visual experience. Utilize auxiliary users to fly unmanned aerial vehicles. Therefore, in the case that the flight speed vector is less than or equal to the preset first speed threshold, the vector mark can remain unchanged at the coordinate origin.

Optionally, the size of the vector marker decreases as the flight speed vector increases, so that the user can clearly experience the change in the flight speed of the UAV according to the change in the size of the displayed vector marker. Further, when the flight speed vector is less than or equal to the preset second speed threshold, the size of the vector marker is the preset maximum size, and when the flight speed vector is greater than or equal to the preset third speed threshold, the size of the vector marker is It is the preset minimum size, so as to prevent the vector mark from being too large to occupy a large display area, and to prevent the vector mark from being too small to be easily observed by the user.

Optionally, the vector marker is a vector sphere or a vector triangle, wherein the vector sphere refers to a vector marker in the shape of a sphere, and the vector triangle refers to a vector marker in the shape of a triangle.

In addition, the vector ball can also be in other shapes, such as the circle shown in Fig. 5a, Fig. 5b and Fig. 5c, and the vector mark can be referred to as a vector circle. In Figure 5a, Figure 5b and Figure 5c, the two dotted lines represent the X coordinate axis and the Y coordinate axis of the coordinate plane respectively, and the line connecting the center of the vector circle and the origin of the coordinate axis is the projection of the flight speed vector of the UAV . When the flight speed vector is less than or equal to the first speed threshold, as shown in Figure 5a, the vector circle is located at the origin of the coordinates. When the flight speed vector is greater than the first speed threshold, as shown in Figures 5b and 5c, the vector circle follows the flight speed The longer the projection, the smaller the vector circle, and the shorter the projection, the larger the vector circle, that is, the larger the flight speed vector, the smaller the vector circle, and the smaller the flight speed vector, the larger the vector circle.

In this embodiment, when the user is flying the unmanned aerial vehicle, he can adjust the flying speed of the unmanned aerial vehicle at the control terminal, so that the center of the vector mark is aligned with the traversable passage of the obstacle. And the center of the vector circle in Fig. 5c is aligned with the bridge hole, and the alignment operation is more accurate as the distance between the UAV and the traversable channel is closer. When aligned, the direction of the flight velocity vector indicating the UAV points to the traversable channel, which in turn indicates that the UAV is flying toward the traversable channel. Therefore, based on the displayed flight speed vector and obstacle information, the operation difficulty of the UAV crossing flight is reduced, and the user is effectively assisted in the UAV flight.

In some embodiments, the predicted flight trajectory of the unmanned aerial vehicle may also be acquired and displayed, wherein the display of the predicted flight trajectory of the unmanned aerial vehicle is used to assist the user in flying the unmanned aerial vehicle.

For example, the display device can display the flight prediction trajectory in the form of one or more curves or straight lines; or, by darkening the color of the picture or adding a dotted line, the corresponding area on the screen (such as a rectangular area, an arc area, etc.) is framed and selected. The area selected by the box represents the flight prediction trajectory; or, the flight prediction trajectory is displayed through multiple nodes (multiple nodes are connected to form the flight prediction trajectory). Therefore, the user can observe the predicted flight trajectory displayed by the display device, and if the user does not want the UAV to fly according to the predicted flight trajectory, the flight of the UAV can be adjusted.

In some embodiments, the flight obstacle avoidance trajectory of the unmanned aerial vehicle may also be acquired and displayed, wherein the flight obstacle avoidance trajectory of the unmanned aerial vehicle is used to assist the user in the obstacle avoidance flight of the unmanned aerial vehicle. The display method of the flight obstacle avoidance trajectory may refer to the display method of the flight prediction trajectory, which will not be repeated here.

In some embodiments, the pole increment corresponding to the flight obstacle avoidance trajectory may also be acquired, and the acquired pole increment may be displayed. Among them, the display of the stick increment is used to assist the user to control the UAV to fly according to the flight obstacle avoidance trajectory. For example, the display device may display the increment of the rod amount in the form of text or images. In the specific implementation process, for example, the increment of the stick amount and the direction of the stick can be displayed on the image to assist the user to control the unmanned aerial vehicle to fly.

For details of the embodiment of the present application provided in FIG. 4 , reference may be made to the content related to the display device in FIG. 2 , which will not be repeated here.

It should be noted that, any of the above embodiments may be implemented independently, or may be implemented by any combination of at least two of the above embodiments, which is not limited thereto.

The embodiments of the present application also provide a computer-readable storage medium, where instructions are stored in the readable storage medium, when the readable storage medium runs on the computer, the computer is made to execute the unmanned aerial vehicle in any of the corresponding embodiments above. Some or all of the steps of the flight assistance method.

The embodiments of the present application also provide a computer program product containing instructions, when the instructions are executed on a computer, the computer is made to execute part of the control method of the wireless image transmission device according to any one of the above corresponding embodiments or all steps.

FIG. 6 is a schematic structural diagram of an unmanned aerial vehicle according to an embodiment of the present application. As shown in FIG. 6 , the unmanned aerial vehicle 600 of this embodiment may include: a processor 601 and a memory 602 . Optionally, the UAV 600 further includes a photographing device 603 . The processor 601, the memory 602, and the photographing device 603 are connected, for example, through a communication bus.

The memory 602 is used for storing instructions, and the processor 601 invokes the instructions stored in the memory 602 to perform the following operations: acquiring the flight status information of the unmanned aerial vehicle, the flight status information includes a flight speed vector, and the flight speed vector is used to indicate the flight of the unmanned aerial vehicle The size and direction of the speed; obtain the obstacle information of the flight scene where the UAV is located; send the flight status information and obstacle information to the display device, and the display device can display the flight status information and obstacle information, flight status information and obstacle information The display of object information is used to assist the user to fly the UAV.

In some embodiments, the obstacle information includes relative positions between the obstacles in the flight scene and the UAV.

In some embodiments, the relative position between the obstacle in the flight scene and the UAV includes the distance between the three-axis direction of the UAV and the obstacle.

In some embodiments, the obstacle information further includes at least one of types, shapes, and sizes of obstacles in the flight scene.

In some embodiments, the photographing device 603 is used to:

Get the scene image of the flight scene;

The processor 601 is specifically used for:

Send the flight status information, obstacle information and scene image to the display device, the display device can display the scene image, and display the flight status information and obstacle information, flight status information and obstacle information on the displayed scene image The display on the scene image is used to show the user the flight status of the UAV and the relative positions between the obstacles in the flight scene and the UAV, so as to assist the user in the obstacle avoidance flight of the UAV.

In some embodiments, before sending the flight state information and the obstacle information to the display device, the processor 601 is further configured to:

According to the obstacle information, build a virtual model of obstacles in the flight scene;

When the processor 601 sends the flight status information and the obstacle information to the display device, it is specifically used for:

Send the flight status information, obstacle information and the virtual model of the obstacle to the display device, and the display device can display the flight status information, obstacle information and the virtual model of the obstacle. The display of the virtual model is used to show the flight state of the UAV, obstacles, and the relative positions between the obstacles and the UAV to the user, so as to assist the user in the obstacle avoidance flight of the UAV.

In some embodiments, the photographing device 603 is used to:

Get the scene image of the flight scene;

The processor 601 is specifically used for:

Send the flight status information, obstacle information, the virtual model of the obstacle and the scene image to the display device, and the display device can display the scene image in the first preset display area, and display the flight status information on the displayed scene image. Display with obstacle information. The display of scene images, flight status information and obstacle information in the first preset display area is used to show the user the flight scene, as well as the flight status, obstacles and obstacles of the UAV in the flight scene. The relative position between the object and the UAV, the display device can display the virtual model of the obstacle in the second preset display area, and the display of the virtual model in the second preset display area is used to show the user in the flight scene obstacles.

In some embodiments, the processor 601 is further configured to:

According to the flight speed vector of the UAV, the flight prediction trajectory is generated; the flight prediction trajectory is sent to the display device, and the display device can display the flight prediction trajectory, and the display of the flight prediction trajectory is used to assist the user to fly the UAV.

In some embodiments, the processor 601 is further configured to:

According to the flight prediction trajectory, determine whether the UAV exists in the risk of collision with obstacles in the flight scene, and if so, send a collision reminder message to the display device and/or the control terminal of the UAV.

In some embodiments, the processor 601 is further configured to:

If there is a risk of collision between the UAV and the obstacle, the flight obstacle avoidance trajectory is generated according to the relative position between the UAV and the obstacle; the flight obstacle avoidance trajectory is sent to the display device, and the display device can avoid the flight obstacle. The trajectory is displayed, and the display of the flight obstacle avoidance trajectory is used to assist the user in the obstacle avoidance flight of the UAV.

In some embodiments, the processor 601 is further configured to:

Control the UAV to fly according to the flight obstacle avoidance trajectory.

In some embodiments, the processor 601 is further configured to:

Determine the target stick amount according to the flight obstacle avoidance trajectory; obtain the current stick amount of the control terminal; determine the stick amount increment according to the current stick amount and the target stick amount; send the stick amount increment to the display device, and the display device can measure the stick amount Incremental display, and the display of stick increments is used to assist the user in the obstacle avoidance flight of the UAV.

In some embodiments, the processor 601 is further configured to:

If the UAV exists at the risk of collision with an obstacle, obtain the estimated flight time of the UAV to the position where it collides with the obstacle; if the estimated flight time is less than or equal to the preset time threshold, control the UAV to brake stop.

In some embodiments, the processor 601 is further configured to:

If it is detected that the unmanned aerial vehicle passes through the obstacle in the flight scene, send a shooting instruction to the shooting device 603;

The photographing device 603 is further used for:

In response to the photographing instruction of the processor 601 , the obstacle crossing image and/or the obstacle crossing video of the unmanned aerial vehicle is photographed.

Wherein, the obstacle crossing image and/or the obstacle crossing video may be stored locally, or stored in the server, or stored in the control terminal of the unmanned aerial vehicle.

In some embodiments, the processor 601 is further configured to:

If it is detected that the direction of the flight speed vector of the unmanned aerial vehicle changes, control the shooting direction of the shooting device 603 on the gimbal of the unmanned aerial vehicle to turn to the direction of the flight speed vector; send the scene image captured by the shooting device 603 to the display device , the display device can display the scene image captured by the shooting device 603, the image center of the scene image displayed by the display device is the direction of the flight speed vector of the UAV, and the image center is used to assist the user to determine the flight speed vector of the UAV direction.

Optionally, the unmanned aerial vehicle 600 further includes a communication device, and the communication device is used for communicating with the display device and the control terminal, for example: sending flight status information, obstacle information, obstacle model, pole increment, etc. to the display device, Receive control commands from the control terminal.

Optionally, the unmanned aerial vehicle 600 further includes a sensing device, and the sensing device is used to acquire flight status information of the unmanned aerial vehicle, for example, the sensing device is the sensing system 162 in FIG. 1 .

The apparatus of this embodiment can be used to execute the technical solution of the method embodiment shown in FIG. 2 , and the implementation principle and technical effect thereof are similar, and are not repeated here.

FIG. 7 is a schematic structural diagram of a display device 700 according to an embodiment of the present application. As shown in FIG. 7 , the display device 700 in this embodiment includes: a processor 701 and a display device 702 . The processor 701 and the display device 702 are connected, for example, through a communication bus.

The processor 701 is used to obtain the flight status information of the unmanned aerial vehicle, and obtain the obstacle information of the flight scene where the unmanned aerial vehicle is located. The flight status information includes a flight speed vector, and the flight speed vector is used to indicate the magnitude and direction;

The display device 702 is used to display the flight status information and obstacle information, and the display of the flight status information and the obstacle information is used to assist the user to fly the unmanned aerial vehicle.

In some embodiments, the processor 701 is further configured to:

Get the scene image of the flight scene;

The display device 702 is specifically used for:

Display the scene image, and display the flight status information and obstacle information on the displayed scene image, and the display of the flight status information and obstacle information on the scene image is used to show the user the flight status of the UAV, and The relative position between the obstacle in the flight scene and the UAV to assist the user in the obstacle avoidance flight of the UAV.

In some embodiments, the processor 701 is further configured to:

Obtain virtual models of obstacles in the flight scene;

The display device 702 is specifically used for:

Display the flight status information, obstacle information and the virtual model of the obstacle. The display of the flight status information, obstacle information and the virtual model of the obstacle is used to show the flight status, obstacles, and obstacles of the UAV to the user The relative position between the UAV and the UAV to assist the user in the obstacle avoidance flight of the UAV.

In some embodiments, the processor 701 is further configured to:

The display device 702 is specifically used for:

In some embodiments, the processor 701 is further configured to:

Get the scene image of the flight scene;

The display device 702 is specifically used for:

The scene image is displayed in the first preset display area, and the flight status information and obstacle information are displayed on the displayed scene image, and the scene image, flight status information and obstacle information in the first preset display area are displayed. Display, which is used to show the user the flight scene, the flight status of the UAV in the flight scene, the obstacles and the relative positions between the obstacles and the UAV;

The virtual model of the obstacle is displayed in the second preset display area, and the display of the virtual model in the second preset display area is used to show the obstacle in the flight scene to the user.

In some embodiments, the processor 701 is further configured to:

Obtain the projection of the flight velocity vector on the preset coordinate plane;

The display device 702 is specifically used for:

The projection is displayed, and the projection display is used to assist the user in determining the direction of the flight velocity vector.

In some embodiments, the coordinate plane includes the YOZ plane of the body coordinate system of the UAV, or the YOZ plane of the geodetic coordinate system.

In some embodiments, the display device 702 is specifically configured to:

The projection of the flight speed vector on the coordinate plane is displayed through the preset vector marker, and the projection is the line between the center of the vector marker and the coordinate origin on the coordinate plane. The center of the vector marker is used to assist the user to determine the flight speed. The direction of the vector.

In some embodiments, when the flight speed vector is less than or equal to a preset first speed threshold, the vector marker is located at the coordinate origin.

In some embodiments, the size of the vector markers decreases as the flight speed vector increases.

In some embodiments, when the flight speed vector is less than or equal to a preset second speed threshold, the size of the vector mark is a preset maximum size; when the flight speed vector is greater than or equal to a preset third speed threshold , the size of the vector marker is the preset minimum size.

In some embodiments, the vector markers are vector spheres or vector triangles.

In some embodiments, the processor 701 is further configured to:

Obtain the flight prediction trajectory of the UAV;

The display device 702 is further used for:

The flight prediction trajectory is displayed, and the display of the flight prediction trajectory is used to assist the user to fly the unmanned aerial vehicle.

In some embodiments, the processor 701 is further configured to:

Obtain the flight obstacle avoidance trajectory of the UAV;

The display device 702 is further used for:

The flight obstacle avoidance trajectory is displayed, and the flight obstacle avoidance trajectory is used to assist the user in the obstacle avoidance flight of the UAV.

In some embodiments, the processor 701 is further configured to:

Obtain the rod increment corresponding to the flight obstacle avoidance trajectory;

The display device 702 is further used for:

The stick increment is displayed, and the display of stick increment is used to assist the user to control the UAV to fly according to the flight obstacle avoidance trajectory.

Optionally, the display device 700 further includes a communication device, and the communication device is used to communicate with the UAV, for example, used to receive flight status information, obstacle information, obstacle model, pole increment, etc. sent by the UAV.

Optionally, the display device 700 further includes a memory for storing a computer program. When the processor 701 invokes the computer program from the memory, some or all of the steps involved in the display device in the foregoing embodiments are implemented.

The apparatus in this embodiment can be used to execute the technical solution of the method embodiment shown in FIG. 4 , and the implementation principle and technical effect thereof are similar, and are not repeated here.

FIG. 8 is a schematic structural diagram of a chip 800 provided by an embodiment of the application. As shown in FIG. 8 , the chip 800 may include: a transceiver 801, a memory 802, and a processor 803, wherein the transceiver 801, the memory 802, and the processor 803 is connected via a bus, for example.

Among them, the transceiver 801 is used for data transmission and reception. Memory 802 for storing program instructions. The processor 803 is used to call the program instructions in the memory 802 and according to the received data of the transceiver 802, execute part or all of the steps of the flight assistance method of the unmanned aerial vehicle in any of the above-mentioned corresponding embodiments, and its realization principle and technical effect similar, and will not be repeated here.

FIG. 9 is a schematic structural diagram of an unmanned aerial system 900 according to an embodiment of the present application. As shown in FIG. 9 , the unmanned aerial system 900 includes an unmanned aerial vehicle 901 , a display device 902 and a control terminal 903 . The unmanned aerial vehicle 901 is respectively connected to the display device 902 and the control terminal 903 , and the display device 902 may be an independent device or may be integrated on the control terminal 903 . Wherein, the unmanned aerial vehicle 901 can perform the relevant operations of the unmanned aerial vehicle in the apparatus embodiment shown in FIG. 6 , and the display device 902 can perform the relevant operations of the display apparatus in the apparatus embodiment shown in FIG. 7 , and the implementation principles and technical effects thereof are similar, It will not be repeated here.

In the above-mentioned embodiments, it may be implemented in whole or in part by software, hardware, firmware or any combination thereof. When implemented in software, it can be implemented in whole or in part in the form of a computer program product. A computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the procedures or functions according to the embodiments of the present application are generated in whole or in part. The computer may be a general purpose computer, a special purpose computer, a computer network, or other programmable device. Computer instructions may be stored in or transmitted over a computer-readable storage medium. Computer instructions may be sent from one website site, computer, server, or data center to another website site, computer, via wire (eg, coaxial cable, fiber optic, digital subscriber line (DSL)) or wireless (eg, infrared, wireless, microwave, etc.) , server or data center for transmission. A computer-readable storage medium can be any available medium that can be accessed by a computer or a data storage device such as a server, data center, or the like that contains one or more of the available mediums integrated. Useful media may be magnetic media (eg, floppy disks, hard disks, magnetic tapes), optical media (eg, DVDs), or semiconductor media (eg, solid state disks (SSDs)), and the like.

Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present application, but not to limit them; although the present application has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that: The technical solutions described in the foregoing embodiments can still be modified, or some or all of the technical features thereof can be equivalently replaced; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the technical solutions of the embodiments of the present application. Scope.

Claims

A flight assistance method for an unmanned aerial vehicle, applied to the unmanned aerial vehicle, wherein the method comprises:

acquiring flight status information of the unmanned aerial vehicle, where the flight status information includes a flight velocity vector, and the flight velocity vector is used to indicate the magnitude and direction of the flying velocity of the unmanned aerial vehicle;

Obtain the obstacle information of the flight scene where the unmanned aerial vehicle is located;

Send the flight status information and the obstacle information to a display device, and the display device can display the flight status information and the obstacle information, and the display of the flight status information and the obstacle information It is used for assisting the user to fly the unmanned aerial vehicle.
The method according to claim 1, wherein the obstacle information includes a relative position between the obstacle in the flight scene and the UAV.
The method according to claim 2, wherein the relative position between the obstacle in the flight scene and the unmanned aerial vehicle comprises a three-axis direction of the unmanned aerial vehicle and the obstacle. distance.
The method according to claim 2 or 3, wherein the obstacle information further includes at least one of types, shapes, and sizes of obstacles in the flight scene.
The method according to any one of claims 1-4, wherein the method further comprises:

acquiring a scene image of the flight scene;

The sending the flight status information and the obstacle information to the display device includes:

Send the flight status information, the obstacle information and the scene image to the display device, and the display device can display the scene image and display the flight status on the displayed scene image. The status information and the obstacle information are displayed, and the display of the flight status information and the obstacle information on the scene image is used to show the user the flight status of the UAV and the flight status The relative position between the obstacle in the scene and the UAV to assist the user in the obstacle avoidance flight of the UAV.
The method according to any one of claims 1-4, wherein after acquiring the obstacle information of the flight scene where the UAV is located, the method further comprises:

According to the obstacle information, establish a virtual model of the obstacle in the flight scene;

The sending the flight status information and the obstacle information to the display device includes:

Send the flight state information, the obstacle information, and the virtual model of the obstacle to the display device, and the display device can compare the flight state information, the obstacle information, and the obstacle's The virtual model is displayed, and the display of the flight status information, the obstacle information and the virtual model of the obstacle is used to show the user the flight status of the UAV, the obstacle, and the The relative position between the obstacle and the unmanned aerial vehicle to assist the user in the obstacle avoidance flight of the unmanned aerial vehicle.
The method according to claim 6, wherein the method further comprises:

acquiring a scene image of the flight scene;

The sending of the flight status information, the obstacle information and the virtual model of the obstacle to the display device includes:

Send the flight status information, the obstacle information, the virtual model of the obstacle and the scene image to the display device, and the display device can display the scene image in the first preset display area display, and display the flight status information and the obstacle information on the displayed scene image, the scene image, the flight status information and the obstacle in the first preset display area The display of object information is used to show the user the flight scene, the flight status of the UAV in the flight scene, the obstacle and the relationship between the obstacle and the UAV. relative position, the display device can display the virtual model of the obstacle in the second preset display area, and the display of the virtual model in the second preset display area is used to show the user all the Describe the obstacles in the flight scene.
The method according to any one of claims 1-7, wherein the method further comprises:

generating a predicted flight trajectory according to the flight speed vector of the unmanned aerial vehicle;

Sending the predicted flight trajectory to the display device, where the display device can display the predicted flight trajectory, and the display of the predicted flight trajectory is used to assist the user in flying the UAV.
The method according to claim 8, wherein the method further comprises:

According to the flight prediction trajectory, determine whether the UAV has a risk of colliding with an obstacle in the flight scene, and if so, send a message to the display device and/or the control terminal of the UAV Collision reminder message.
The method according to claim 9, wherein the method further comprises:

If the unmanned aerial vehicle has a risk of colliding with the obstacle, generating a flight obstacle avoidance trajectory according to the relative position between the unmanned aerial vehicle and the obstacle;

Send the flight obstacle avoidance trajectory to the display device, and the display device can display the flight obstacle avoidance trajectory, and the display of the flight obstacle avoidance trajectory is used to assist the user in performing the unmanned aerial vehicle. Obstacle avoidance flight.
The method of claim 10, wherein the method further comprises:

The unmanned aerial vehicle is controlled to fly according to the flight obstacle avoidance trajectory.
The method of claim 10, wherein the method further comprises:

Determine the target stick amount according to the flight obstacle avoidance trajectory;

obtain the current rod amount of the control terminal;

Determine the stroke increment according to the current stroke and the target stroke;

Send the stick amount increment to the display device, and the display device can display the stick amount increment, and the display of the stick amount increment is used to assist the user to control the unmanned flight according to the The flight obstacle avoidance trajectory flies.
The method according to any one of claims 9-12, wherein the method further comprises:

If the unmanned aerial vehicle has a risk of colliding with the obstacle, obtaining the estimated flight time of the unmanned aerial vehicle to the position where the collision collides with the obstacle;

If the estimated flight time is less than or equal to a preset time threshold, the UAV is controlled to stop.
The method according to any one of claims 1-13, wherein the method further comprises:

If it is detected that the unmanned aerial vehicle passes through the obstacle in the flight scene, the obstacle crossing image and/or the obstacle crossing of the unmanned aerial vehicle is photographed by the photographing device on the gimbal of the unmanned aerial vehicle video;

Store the obstacle traversing image and/or the obstacle traversing video locally or send it to a server or to the control terminal of the unmanned aerial vehicle, and the server and the control terminal are capable of traversing the obstacle. Images and/or video of said obstacle crossing are stored.
The method according to any one of claims 1-14, wherein the method further comprises:

If it is detected that the direction of the flight speed vector of the unmanned aerial vehicle changes, controlling the shooting direction of the shooting device on the gimbal of the unmanned aerial vehicle to turn to the direction of the flight speed vector;

Sending the scene image captured by the shooting device to the display device, the display device can display the scene image captured by the shooting device, and the image center of the scene image displayed by the display device is the unmanned person The direction of the flight speed vector of the aircraft, and the image center is used to assist the user to determine the direction of the flight speed vector of the unmanned aerial vehicle.
A flight assistance method for an unmanned aerial vehicle, which is applied to a display device of an unmanned aerial vehicle, wherein the method comprises:

acquiring flight status information of the unmanned aerial vehicle, where the flight status information includes a flight velocity vector, and the flight velocity vector is used to indicate the magnitude and direction of the flying velocity of the unmanned aerial vehicle;

Obtain the obstacle information of the flight scene where the unmanned aerial vehicle is located;

The flight status information and the obstacle information are displayed, and the display of the flight status information and the obstacle information is used to assist the user to fly the UAV.
The method according to claim 16, wherein the obstacle information includes a relative position between the obstacle in the flight scene and the UAV.
The method according to claim 17, wherein the relative position between the obstacle in the flight scene and the unmanned aerial vehicle comprises a three-axis direction of the unmanned aerial vehicle and the obstacle. distance.
The method according to claim 17 or 18, wherein the obstacle information further includes at least one of types, shapes, and sizes of obstacles in the flight scene.
The method according to any one of claims 16-19, wherein the method further comprises:

acquiring a scene image of the flight scene;

The displaying of the flight status information and the obstacle information includes:

The scene image is displayed, and the flight state information and the obstacle information are displayed on the displayed scene image, and the display of the flight state information and the obstacle information on the scene image is used for To show the user the flight status of the UAV and the relative position between the obstacles in the flight scene and the UAV, so as to assist the user in avoiding obstacles of the UAV flight.
The method according to any one of claims 16-19, wherein after acquiring the obstacle information of the flight scene where the UAV is located, the method further comprises:

obtaining virtual models of obstacles in the flight scene;

The displaying of the flight status information and the obstacle information includes:

displaying the flight state information, the obstacle information and the virtual model of the obstacle, and the display of the flight state information, the obstacle information and the virtual model of the obstacle is used to show the user The flight status of the UAV, the obstacle, and the relative position between the obstacle and the UAV are displayed, so as to assist the user in the obstacle avoidance flight of the UAV.
The method according to any one of claims 16-19, wherein after acquiring the obstacle information of the flight scene where the UAV is located, the method further comprises:

According to the obstacle information, establish a virtual model of the obstacle in the flight scene;

The displaying of the flight status information and the obstacle information includes:

displaying the flight state information, the obstacle information and the virtual model of the obstacle, and the display of the flight state information, the obstacle information and the virtual model of the obstacle is used to show the user The flight status of the UAV, the obstacle, and the relative position between the obstacle and the UAV are displayed, so as to assist the user in the obstacle avoidance flight of the UAV.
The method according to claim 21 or 22, wherein the method further comprises:

acquiring a scene image of the flight scene;

The displaying of the flight status information, the obstacle information and the virtual model includes:

The scene image is displayed in a first preset display area, and the flight status information and the obstacle information are displayed on the displayed scene image, and all information in the first preset display area is displayed. The display of the scene image, the flight status information and the obstacle information is used to show the user the flight scene, and the flight status of the UAV, the obstacles and the UAV in the flight scene to the user. the relative position between the obstacle and the UAV;

The virtual model of the obstacle is displayed in the second preset display area, and the display of the virtual model in the second preset display area is used to show the obstacle in the flight scene to the user.
The method according to any one of claims 16-23, wherein displaying the flight status information comprises:

obtaining the projection of the flight speed vector on the preset coordinate plane;

The projection is displayed, and the display of the projection is used to assist the user in determining the direction of the flight velocity vector.
The method according to claim 24, wherein the coordinate plane comprises the YOZ plane of the body coordinate system of the UAV or the YOZ plane of the geodetic coordinate system.
The method according to claim 24 or 25, wherein the displaying the projection comprises:

The projection of the flight speed vector on the coordinate plane is displayed through a preset vector mark, where the projection is a line connecting the center of the vector mark and the coordinate origin on the coordinate plane, and the The center of the vector marker is used to assist the user in determining the direction of the flight velocity vector.
The method according to claim 26, wherein, when the flight speed vector is less than or equal to a preset first speed threshold, the vector marker is located at the coordinate origin.
The method of claim 26 or 27, wherein the size of the vector marker decreases as the flight speed vector increases.
The method according to any one of claims 26-28, characterized in that, when the flight speed vector is less than or equal to a preset second speed threshold, the size of the vector marker is a preset maximum size; When the flight speed vector is greater than or equal to a preset third speed threshold, the size of the vector marker is a preset minimum size.
The method according to any one of claims 26-29, wherein the vector label is a vector sphere or a vector triangle.
The method according to any one of claims 16-30, wherein the method further comprises:

The predicted flight trajectory of the unmanned aerial vehicle is acquired and displayed, and the display of the predicted flight trajectory is used to assist the user in flying the unmanned aerial vehicle.
The method according to any one of claims 16-31, wherein the method further comprises:

Obtain and display the flight obstacle avoidance trajectory of the unmanned aerial vehicle, where the flight obstacle avoidance trajectory is used to assist the user in the obstacle avoidance flight of the unmanned aerial vehicle.
The method of claim 32, wherein the method further comprises:

Obtain the pole increment corresponding to the flight obstacle avoidance trajectory;

The stick increment is displayed, and the display of the stick increment is used to assist the user to control the UAV to fly according to the flight obstacle avoidance trajectory.
An unmanned aerial vehicle, comprising: a processor and a memory, wherein the memory is used to store instructions, and the processor invokes the instructions stored in the memory to perform the following operations:

acquiring flight status information of the unmanned aerial vehicle, where the flight status information includes a flight velocity vector, and the flight velocity vector is used to indicate the magnitude and direction of the flying velocity of the unmanned aerial vehicle;

Obtain the obstacle information of the flight scene where the unmanned aerial vehicle is located;

Send the flight status information and the obstacle information to a display device, and the display device can display the flight status information and the obstacle information, and the display of the flight status information and the obstacle information It is used for assisting the user to fly the unmanned aerial vehicle.
The unmanned aerial vehicle of claim 34, wherein the obstacle information includes a relative position between an obstacle in the flight scene and the unmanned aerial vehicle.
The unmanned aerial vehicle according to claim 35, wherein the relative position between the obstacle in the flight scene and the unmanned aerial vehicle comprises the distance between the three-axis direction of the unmanned aerial vehicle and the obstacle. distance between.
The unmanned aerial vehicle according to claim 35 or 36, wherein the obstacle information further includes at least one of types, shapes, and sizes of obstacles in the flight scene.
The unmanned aerial vehicle according to any one of claims 34-37, wherein the aerial vehicle further comprises a photographing device; the photographing device is used for:

acquiring a scene image of the flight scene;

The processor is specifically used for:

Send the flight status information, the obstacle information and the scene image to the display device, and the display device can display the scene image and display the flight status on the displayed scene image. The status information and the obstacle information are displayed, and the display of the flight status information and the obstacle information on the scene image is used to show the user the flight status of the UAV and the flight status The relative position between the obstacle in the scene and the UAV to assist the user in the obstacle avoidance flight of the UAV.
The unmanned aerial vehicle according to any one of claims 34-37, wherein before the processor sends the flight state information and the obstacle information to a display device, the processor is further configured to:

According to the obstacle information, establish a virtual model of the obstacle in the flight scene;

When sending the flight state information and the obstacle information to the display device, the processor is specifically used for:

Send the flight state information, the obstacle information, and the virtual model of the obstacle to the display device, and the display device can compare the flight state information, the obstacle information, and the obstacle's The virtual model is displayed, and the display of the flight status information, the obstacle information and the virtual model of the obstacle is used to show the user the flight status of the UAV, the obstacle, and the The relative position between the obstacle and the unmanned aerial vehicle to assist the user in the obstacle avoidance flight of the unmanned aerial vehicle.
The unmanned aerial vehicle according to claim 39, wherein the unmanned aerial vehicle further comprises a photographing device; the photographing device is used for:

acquiring a scene image of the flight scene;

The processor is specifically used for:

Send the flight status information, the obstacle information, the virtual model of the obstacle and the scene image to the display device, and the display device can display the scene image in the first preset display area display, and display the flight status information and the obstacle information on the displayed scene image, the scene image, the flight status information and the obstacle in the first preset display area The display of object information is used to show the user the flight scene, the flight status of the UAV in the flight scene, the obstacle and the relationship between the obstacle and the UAV. relative position, the display device can display the virtual model of the obstacle in the second preset display area, and the display of the virtual model in the second preset display area is used to show the user all the Describe the obstacles in the flight scene.
The unmanned aerial vehicle according to any one of claims 34-40, wherein the processor is further configured to:

generating a predicted flight trajectory according to the flight speed vector of the unmanned aerial vehicle;

Sending the predicted flight trajectory to the display device, where the display device can display the predicted flight trajectory, and the display of the predicted flight trajectory is used to assist the user in flying the UAV.
The unmanned aerial vehicle of claim 41, wherein the processor is further configured to:

According to the flight prediction trajectory, determine whether the UAV exists in the risk of collision with the obstacles in the flight scene, and if so, send a message to the display device and/or the control terminal of the UAV Collision reminder message.
The unmanned aerial vehicle of claim 42, wherein the processor is further configured to:

If the unmanned aerial vehicle has a risk of colliding with the obstacle, generating a flight obstacle avoidance trajectory according to the relative position between the unmanned aerial vehicle and the obstacle;

Send the flight obstacle avoidance trajectory to the display device, and the display device can display the flight obstacle avoidance trajectory, and the display of the flight obstacle avoidance trajectory is used to assist the user in performing the unmanned aerial vehicle. Obstacle avoidance flight.
The unmanned aerial vehicle of claim 43, wherein the processor is further configured to:

The unmanned aerial vehicle is controlled to fly according to the flight obstacle avoidance trajectory.
The unmanned aerial vehicle of claim 43, wherein the processor is further configured to:

Determine the target stick amount according to the flight obstacle avoidance trajectory;

obtain the current rod amount of the control terminal;

Determine the stroke increment according to the current stroke and the target stroke;

Sending the stick amount increment to the display device, and the display device can display the stick amount increment, and the display of the stick amount increment is used to assist the user in performing the unmanned aerial vehicle. Obstacle avoidance flight.
The unmanned aerial vehicle according to any one of claims 42-45, wherein the processor is further configured to:

If the unmanned aerial vehicle is at risk of colliding with the obstacle, obtaining the estimated flight time of the unmanned aerial vehicle to the position where it collides with the obstacle;

If the estimated flight time is less than or equal to a preset time threshold, the UAV is controlled to stop.
The unmanned aerial vehicle according to any one of claims 34-46, wherein the unmanned aerial vehicle further comprises a photographing device; the processor is further configured to:

If it is detected that the unmanned aerial vehicle passes through an obstacle in the flight scene, sending a shooting instruction to the shooting device;

The photographing device is used for:

In response to the photographing instruction of the processor, photographing an obstacle-passing image and/or obstacle-passing video of the UAV; wherein the obstacle-passing image and/or the obstacle-passing video are stored locally or Stored in the server or stored in the control terminal of the unmanned aerial vehicle.
The unmanned aerial vehicle according to any one of claims 34-47, wherein the processor is further configured to:

If it is detected that the direction of the flight speed vector of the unmanned aerial vehicle changes, controlling the shooting direction of the shooting device on the gimbal of the unmanned aerial vehicle to turn to the direction of the flight speed vector;

Sending the scene image captured by the shooting device to the display device, the display device can display the scene image captured by the shooting device, and the image center of the scene image displayed by the display device is the unmanned person The direction of the flight speed vector of the aircraft, and the image center is used to assist the user to determine the direction of the flight speed vector of the unmanned aerial vehicle.
A display device, comprising: a processor and a display device;

The processor is configured to acquire flight status information of the unmanned aerial vehicle, and obtain obstacle information of the flight scene where the unmanned aerial vehicle is located, the flight status information includes a flight speed vector, and the flight speed vector is used to indicate the The magnitude and direction of the flight speed of the UAV;

The display device is configured to display the flight status information and the obstacle information, and the display of the flight status information and the obstacle information is used to assist a user in flying the unmanned aerial vehicle.
The device of claim 49, wherein the obstacle information includes a relative position between the obstacle in the flight scene and the UAV.
The device according to claim 50, wherein the relative position between the obstacle in the flight scene and the unmanned aerial vehicle comprises a three-axis direction of the unmanned aerial vehicle and the obstacle. distance.
The device according to claim 50 or 51, wherein the obstacle information further includes at least one of types, shapes, and sizes of obstacles in the flight scene.
The device according to any one of claims 49-52, wherein the processor is further configured to:

acquiring a scene image of the flight scene;

The display device is specifically used for:

The scene image is displayed, and the flight state information and the obstacle information are displayed on the displayed scene image, and the display of the flight state information and the obstacle information on the scene image is used for To show the user the flight status of the UAV and the relative position between the obstacles in the flight scene and the UAV, so as to assist the user in avoiding obstacles of the UAV flight.
The device according to any one of claims 49-52, wherein the processor is further configured to:

obtaining virtual models of obstacles in the flight scene;

The display device is specifically used for:

Displaying the flight state information, the obstacle information and the virtual model of the obstacle, and the display of the flight state information, the obstacle information and the virtual model of the obstacle is used to show the user The flight status of the UAV, the obstacle, and the relative position between the obstacle and the UAV are displayed, so as to assist the user in the obstacle avoidance flight of the UAV.
The device according to any one of claims 49-52, wherein the processor is further configured to:

According to the obstacle information, establish a virtual model of the obstacle in the flight scene;

The display device is specifically used for:

displaying the flight state information, the obstacle information and the virtual model of the obstacle, and the display of the flight state information, the obstacle information and the virtual model of the obstacle is used to show the user The flight status of the UAV, the obstacle, and the relative position between the obstacle and the UAV are displayed, so as to assist the user in the obstacle avoidance flight of the UAV.
The device according to claim 54 or 55, wherein the processor is further configured to:

acquiring a scene image of the flight scene;

The display device is specifically used for:

The scene image is displayed in a first preset display area, and the flight status information and the obstacle information are displayed on the displayed scene image, and all information in the first preset display area is displayed. The display of the scene image, the flight status information and the obstacle information is used to show the user the flight scene, and the flight status of the UAV, the obstacles and the UAV in the flight scene to the user. the relative position between the obstacle and the UAV;

The virtual model of the obstacle is displayed in the second preset display area, and the display of the virtual model in the second preset display area is used to show the obstacle in the flight scene to the user.
The device according to any one of claims 49-56, wherein the processor is further configured to:

obtaining the projection of the flight speed vector on the preset coordinate plane;

The display device is specifically used for:

The projection is displayed, and the display of the projection is used to assist the user in determining the direction of the flight velocity vector.
The device according to claim 57, wherein the coordinate plane comprises a YOZ plane of the body coordinate system of the unmanned aerial vehicle, or a YOZ plane of the geodetic coordinate system.
The device according to claim 57 or 58, wherein the display device is specifically used for:

The projection of the flight speed vector on the coordinate plane is displayed through a preset vector mark, where the projection is a line connecting the center of the vector mark and the coordinate origin on the coordinate plane, and the The center of the vector marker is used to assist the user in determining the direction of the flight velocity vector.
The device according to claim 59, wherein the vector marker is located at the origin of the coordinates when the flight speed vector is less than or equal to a preset first speed threshold.
The apparatus of claim 59 or 60, wherein the size of the vector marker decreases as the flight speed vector increases.
The device according to any one of claims 59-61, characterized in that, when the flight speed vector is less than or equal to a preset second speed threshold, the size of the vector marker is a preset maximum size; When the flight speed vector is greater than or equal to a preset third speed threshold, the size of the vector marker is a preset minimum size.
The device according to any one of claims 59-62, wherein the vector label is a vector sphere or a vector triangle.
The device according to any one of claims 49-63, wherein the processor is further configured to:

obtaining the predicted flight trajectory of the unmanned aerial vehicle;

The display device is also used for:

The predicted flight trajectory is displayed, and the display of the predicted flight trajectory is used to assist the user in flying the unmanned aerial vehicle.
The device according to any one of claims 49-64, wherein the processor is further configured to:

obtaining the flight obstacle avoidance trajectory of the unmanned aerial vehicle;

The display device is also used for:

The flight obstacle avoidance trajectory is displayed, and the flight obstacle avoidance trajectory is used to assist the user in the obstacle avoidance flight of the unmanned aerial vehicle.
The device of claim 65, wherein the processor is further configured to:

Obtain the pole increment corresponding to the flight obstacle avoidance trajectory;

The display device is also used for

The stick amount increment is displayed, and the display of the stick amount increment is used to assist the user to control the UAV to fly according to the flight obstacle avoidance trajectory.
A chip, characterized by comprising: a transceiver, a memory and a processor;

the transceiver, used for data transmission and reception;

the memory for storing program instructions;

The processor is configured to invoke the program instructions in the memory and execute the method according to any one of claims 1-15 or any one of claims 16-33 according to the received data of the transceiver.
An unmanned aerial system, comprising: the unmanned aerial vehicle according to any one of claims 34-48, the display device according to any one of claims 49-66, and a control terminal, the control terminal for controlling the flight of the unmanned aerial vehicle.
A computer-readable storage medium, characterized by comprising instructions that, when executed on a computer, cause the computer to perform the method of any one of claims 1-15 or any one of claims 16-33 .