WO2020000386A1

WO2020000386A1 - Flight control method, device and system, and storage medium

Info

Publication number: WO2020000386A1
Application number: PCT/CN2018/093726
Authority: WO
Inventors: 周游; 李进吉; 唐克坦
Original assignee: 深圳市大疆创新科技有限公司
Priority date: 2018-06-29
Filing date: 2018-06-29
Publication date: 2020-01-02
Also published as: CN110325940A

Abstract

Provided in an embodiment of the present invention are a flight control method, device and system. The method comprises: receiving a command to pass through or bypass one or more objects, the command comprising an identifier of each of the one or more objects; determining, according to pose information of the objects corresponding to the respective identifiers, flight paths along which the objects are to be passed through or bypassed; and controlling an unmanned aerial vehicle to pass through or bypass the respective objects on the basis of the flight paths. The invention enhances convenience of passing through or bypassing obstacles.

Description

Flight control method, equipment, system and storage medium

Technical field

The present invention relates to the field of control technology, and in particular, to a flight control method, device, system, and storage medium.

Background technique

With the development of flight technology, drones have become a relatively popular research topic, and are widely used in plant protection, aerial photography, forest fire monitoring and other fields, bringing many conveniences to people's lives and work. Because the drone is restricted by flying height and other factors, obstacles are usually encountered during the flight. In practical applications, when the drone encounters obstacles during the flight, the terminal can only be manually operated by the user. To adjust the flight trajectory of the drone to avoid obstacles, it can be seen that this way of avoiding obstacles is cumbersome to operate.

Summary of the invention

Embodiments of the present invention provide a flight control method, equipment, system, and storage medium, which can plan the flight trajectory of a drone based on the pose of an obstacle to automatically avoid the obstacle.

In a first aspect, an embodiment of the present invention provides a flight control method. The method includes:

Receiving an instruction to traverse at least one object, the instruction including an identity of each object in the at least one object;

Determining a flight trajectory through each object according to pose information of the object corresponding to each identity;

Controlling the drone to traverse each of the objects based on the flight trajectory.

In a second aspect, an embodiment of the present invention provides another flight control method, which includes:

Obtaining a reference identifier;

Acquiring pose information of an object associated with the reference identifier;

Determining a flight trajectory according to the pose information of the object;

Controlling the drone to fly based on the flight trajectory.

According to a third aspect, an embodiment of the present invention provides a control device, where the control device includes a memory and a processor;

The memory is configured to store at least one program instruction, and the at least one program instruction includes a first program instruction and a second program instruction.

The processor executes the first program instruction stored in the memory, and when the first program instruction is executed, the processor is configured to perform the following steps:

The processor executes the second program instruction stored in the memory, and when the second program instruction is executed, the processor is configured to perform the following steps:

Obtaining a reference identifier;

Controlling the drone to fly based on the flight trajectory.

In a fourth aspect, an embodiment of the present invention provides an unmanned aerial vehicle system, including an unmanned aerial vehicle and a control device;

The control device is configured to execute the flight control method according to the first aspect and the second aspect to plan a flight trajectory for the drone.

The drone is configured to fly according to the flight trajectory to pass through at least one object.

According to a fifth aspect, an embodiment of the present invention provides a computer-readable storage medium, wherein the computer storage medium stores a computer program, where the computer program includes program instructions, and the program instructions are executed by a processor The processor is caused to execute the steps of the flight control method according to the first aspect and the second aspect.

According to a sixth aspect, an embodiment of the present invention provides a computer program product, wherein the computer program product includes a non-transitory computer-readable storage medium storing a computer program, and the computer program is operable to enable a computer to implement Steps of the flight control method according to the first aspect and the second aspect.

In the embodiment of the present invention, the control device receives an instruction to traverse at least one object (that is, an obstacle), and the instruction includes the identity of each object in the at least one object, and determines the traverse according to the pose information of the object associated with each identity. Flight trajectories of all the objects in the at least one object, and controlling the drone to traverse each of the objects based on the flight trajectories. It can be seen that this embodiment can automatically plan the flight trajectory of the drone based on the pose information of the object, without the need for the user to manually adjust the flight trajectory of the drone, so as to satisfy the user's automation and intelligence of the drone crossing obstacles Changing needs.

BRIEF DESCRIPTION OF THE DRAWINGS

In order to more clearly explain the embodiments of the present invention or the technical solutions in the prior art, the drawings used in the embodiments will be briefly introduced below. Obviously, the drawings in the following description are only some of the present invention. For those of ordinary skill in the art, other embodiments may be obtained based on these drawings without paying creative effort.

FIG. 1 is a schematic structural diagram of an unmanned aerial vehicle system according to an embodiment of the present invention;

2 is a schematic flowchart of a flight control method according to an embodiment of the present invention;

3 is a schematic flowchart of another flight control method according to an embodiment of the present invention;

4 is a schematic diagram of an adjusted target image according to an embodiment of the present invention;

5 is a schematic diagram of a binarized target image according to an embodiment of the present invention;

6 is a schematic diagram of a target image according to an embodiment of the present invention;

7 is a schematic diagram of a target image after perspective transformation according to an embodiment of the present invention;

8 is a schematic diagram of a rasterized target image according to an embodiment of the present invention;

9 is a schematic diagram of a target image after pixel extraction according to an embodiment of the present invention;

10 is a schematic diagram of a two-dimensional code image provided by an embodiment of the present invention;

11 is a schematic diagram of a position relationship between an object and a reference identifier according to an embodiment of the present invention;

12 is a schematic flowchart of another flight control method according to an embodiment of the present invention;

FIG. 13 is a schematic structural diagram of a control device according to an embodiment of the present invention.

detailed description

Hereinafter, some embodiments of the present invention will be described in detail with reference to the drawings. In the case of no conflict, it is obvious that the described embodiments are only a part of the embodiments of the present invention, but not all the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by a person of ordinary skill in the art without creative efforts shall fall within the protection scope of the present invention. In addition, without conflict, the following embodiments and features in the embodiments can be combined with each other.

The terminology used herein is for the purpose of describing particular embodiments only and is not intended to limit the invention. The singular forms "a," "the," and "the" as used in this invention and in the claims are intended to include the plural forms as well, unless the context clearly indicates otherwise. It should be understood that the term "and / or" as used herein refers to any or all possible combinations that include one or more of the associated listed items.

The flight control method provided by the embodiment of the present invention may be executed by a control device, and the control device may be provided on a drone, or may be provided on a ground terminal (such as a remote controller). The flight control method can be applied to drone-based flight control. In other embodiments, it can also be applied to flight control of a mobile device such as a robot capable of autonomous movement. The following is performed using a flight control method applied to a drone. for example.

In practice, when an unmanned aerial vehicle encounters an obstacle during flight, it can only be avoided by the user's manual operation. Therefore, the way to avoid the obstacle is cumbersome. In order to solve the problem, an embodiment of the present invention provides a flight control method. In the flight control method, a control device receives an instruction to cross at least one object (that is, an obstacle), and the instruction includes the identity of each object in the at least one object. Identification, determining flight trajectories of passing through all the objects in the at least one object according to pose information of the objects associated with each identity, and controlling the drone to traverse each of the objects based on the flight trajectories. It can be seen that this embodiment can automatically plan the flight trajectory of the drone based on the pose information of the object, without the need for the user to manually adjust the flight trajectory of the drone, so as to satisfy the user's automation and intelligence of the drone crossing obstacles. Changing needs.

In addition, the user can also accurately adjust the flight trajectory of the drone through the object through simple programming. For example, if the user programs and inputs the instruction that includes the identity identifiers 1, 2, and 4 of the passing object, the control device can execute the instructions in the embodiment of this application. The content described above is used to automatically plan the flight trajectory, and based on the flight trajectory to fly through the objects whose identity is 1, 2, 4 to stimulate the user's programming interest.

Another implementation manner of the embodiment of the present invention includes: the control device acquires at least one reference identifier, and for each reference identifier, determines the flight trajectory of the drone according to the pose information of the object associated with each identity identifier, and controls the drone The aircraft flies based on the flight trajectory. It can be seen that this embodiment can automatically plan the drone's flight trajectory based on the pose information of the object, without the need for the user to manually adjust the drone's flight trajectory, improve the convenience of crossing obstacles, and meet the user's requirements for unmanned The need for automation and intelligence of aircraft crossing obstacles.

In addition, the flight trajectory can be automatically planned according to the pose information of the object, and the designated position can be automatically reached based on the flight trajectory.

In order to facilitate understanding of the flight control method described in this application, an embodiment of the present application also provides a drone system, that is, the drone system shown in FIG. 1 is used for explanation.

Please refer to FIG. 1, which is a schematic structural diagram of an unmanned aerial vehicle system according to an embodiment of the present invention. The system includes a control device 51 and a drone 52. The control device 51 may be a control terminal of a drone, and may specifically be one or more of a remote controller, a smart phone, a tablet computer, a laptop computer, a ground station, and a wearable device (watch, bracelet). For example, the drone 52 may be a rotary wing drone, such as a quad-rotor drone, a six-rotor drone, an eight-rotor drone, or a fixed-wing drone. The drone includes a power system 521, which is used to provide flying power for the drone. The power system 521 includes one or more of a propeller, a motor, and an ESC. The drone may also include a gimbal 522 and The imaging device 523 is mounted on the main body of the drone through the gimbal 522. The shooting device is used to take images or videos during the flight of the drone, including but not limited to multispectral imagers, hyperspectral imagers, visible light cameras, and infrared cameras. The gimbal is a multi-axis transmission and stabilization system. The gimbal motor compensates for the shooting angle of the imaging device by adjusting the rotation angle of the rotating shaft, and prevents or reduces the shake of the imaging device by setting an appropriate buffer mechanism.

In one embodiment, the control device 51 may be configured with an interaction device for interacting with a user. The interaction device may be one or more of a touch display screen, a keyboard, a button, a joystick, and a pulsator. Provide a user interface. The user can enter an instruction to traverse at least one object on the user interface of the control device 51, such as entering the identity of the at least one object. After the control device 51 receives the instruction, the control device 51 can The identification of each object obtains the pose information of each object, and determines the flight trajectory of the drone crossing all the objects in at least one object according to the pose information of each object, and controls the drone 52 to traverse each based on the flight trajectory. Object.

Please refer to FIG. 2, which is a schematic flowchart of a flight control method according to an embodiment of the present invention. The method may be executed by a control device, where the specific explanation of the control device is as described above. As shown in FIG. 2, the flight control method may include the following steps.

S101. Receive an instruction to traverse at least one object, where the instruction includes an identity of each object in the at least one object.

In the embodiment of the present invention, the control device may provide a user interaction interface, and the user may input an instruction to traverse one or more objects on the user interaction interface according to requirements. For example, a user may click an interface to input an instruction to traverse at least one object, the instruction includes an identity of each object in the at least one object, and the control device receives the user's input instruction to traverse at least one object, and according to the object included in the instruction, Identity, identifying the objects the drone needs to traverse.

The identity of an object is an identity that can uniquely identify the object, and the identity may be composed of at least one of a number, a character, a character, or a letter.

S102. Determine a flight trajectory that passes through each object according to pose information of the object corresponding to each identity.

In the embodiment of the present invention, after determining the objects to be traversed, the control device may obtain the pose information of each object from a database according to the identity of each object, and the pose information of multiple objects is stored in the database in advance, The flight trajectory passing through each object is determined according to the pose information of the object corresponding to each identity.

Wherein, at least one object may be one or more objects. When at least one object includes only one object, the control device may determine a flight trajectory passing through the object according to the pose information of the object corresponding to the identity; when the at least one object includes multiple objects, the control device may The pose information of the objects determines the relative pose information between each two objects, and the flight trajectory of all the objects passing through at least one of the objects is determined according to the relative pose information.

Among them, the pose information includes translation information and rotation information. The relaxation information is in the world coordinate system or the camera coordinate system. The translation information may be coordinate information. The rotation information includes rotation angles of multiple axes, for example, a pitch axis, a skew One or more of a roll axis and a roll axis. In one embodiment, the instruction further carries a sequence of traversing each object, and step S202 includes: determining a flight trajectory through each object according to pose information of the object corresponding to each identity and the sequence of traversing each object. .

Wherein, the control device determines the flight trajectory passing through each object according to the pose information of the object corresponding to each identity and the order through each object, including: obtaining the order through each object from the instruction; according to each The pose information of the object corresponding to the identity is used to determine the relative pose information of each two objects; and the flight trajectory of all the objects passing through the at least one object is determined according to the relative pose information and the order of crossing each object.

For example, the objects that need to pass include Object 1, Object 2, and Object 3. The order of crossing each object is to pass through Object 1, then Object 2, and then Object 3; according to the pose information of Object 1 and Object 2 The relative pose information of the object 2 and the object 1 is determined by the pose information, and the relative pose information of the object 2 and the object 3 is determined according to the pose information of the object 2 and the pose information of the object 3; The relative pose information determines that object 2 is located to the left of object 1, and according to the relative pose information of object 2 and object 3 to determine that object 3 is behind object 2, the flight trajectory is after crossing object 1, turning to the left and crossing the object 2, then turn around and pass through Object 3.

Among them, in one embodiment, the order of traversing each object may be set by the user programmatically. For example, in addition to carrying the identity of each object to be traversed in the instruction entered by the user, the user may also traverse each object. Order.

In one embodiment, the order of traversing each object may default to the order in which the user enters the identity of the object. For example, an object with an identity of 1 is recorded as object 1, an object with an identity of 2 is recorded as object 2, and an object with an identity of 3 is recorded as object 3. If the user enters 1 in sequence on the user interaction interface of the control device , 2, 3, then the order of crossing each object is determined to be through object 1, then through object 2, and finally through object 3.

In one embodiment, the order of crossing each object is additionally set by the user, or is set by the control device according to the distance between the various objects. For example, the order of traversing each object is set by the control device according to the distance between the various objects. Assume that the objects to be traversed are Object 1, Object 2, Object 3, Object 1 is closest to the current drone, and the object is The distance between 1 and object 2 is the shortest. The order of crossing objects can be through object 1, then object 2, and then object 3.

S103. Control the drone to traverse each object based on the flight trajectory.

In the embodiment of the present invention, after the flight trajectory is obtained, the control device may control the drone to traverse each object based on the flight trajectory.

Among them, the object refers to an obstacle, and the shape of the obstacle can be a ring or a solid object. When the obstacle is a ring, passing through the object can mean passing through the center of the ring; When it is a solid object, passing through the object can mean flying away from the solid object. For example, passing through the object can mean passing from the top, bottom, left, or right of the solid object.

In the embodiment of the present invention, the control device receives an instruction to traverse at least one object (that is, an obstacle), and the instruction includes the identity of each object in the at least one object, and determines the traverse according to the pose information of the object associated with each identity. Flight trajectories of all the objects in the at least one object, and controlling the drone to traverse each of the objects based on the flight trajectories. It can be seen that this embodiment can automatically plan the drone's flight trajectory based on the pose information of the object, without the need for the user to manually adjust the drone's flight trajectory, improving the convenience of crossing obstacles, and satisfying users to the The need for automation and intelligence of aircraft crossing obstacles.

In addition, the user can also adjust the flight trajectory of the drone through the object through simple programming. For example, if the user programs and inputs the instruction that includes the identity identifiers 1, 2, and 4 of the crossing object, the control device can execute the description in the embodiment of the present application. Content to automatically plan the flight trajectory, and based on the flight trajectory to traverse objects with identity 1,2, 4 during the flight, which stimulates the user's programming interest.

Please refer to FIG. 3. FIG. 3 is a schematic flowchart of a flight control method according to an embodiment of the present invention. The method may be executed by a control device. The specific explanation of the control device is as described above. The method includes:

S201. Receive an instruction to traverse at least one object, where the instruction includes an identity of each object in the at least one object.

S202. For each object, according to the reference identifier associated with the object, obtain relative pose information corresponding to the reference identifier that is stored in advance, and the relative pose information is between the object and the reference identifier associated with the object. Relative pose information.

In order to pass through each object accurately, the control device may control the drone to pass through the object according to the pose information of the object. Specifically, for each object in at least one object, the control device may obtain a reference identifier associated with the object. , According to the reference identifier associated with the object, obtaining the pre-stored relative pose information corresponding to the reference identifier.

The pose information includes at least one of relative coordinate information, global coordinate information, world coordinate information, rotation, and displacement vectors.

Before step S202, the method further includes: obtaining a reference identifier associated with each object through the shooting device based on the flight trajectory, and the shooting device is set on the drone.

During the flight of the drone based on the flight trajectory, the shooting device may be controlled to obtain a reference identifier associated with each object by shooting, or the shooting device may be controlled to obtain a reference identifier associated with each object by means of infrared scanning. The reference mark here may be a two-dimensional code, a random dot matrix, or other markers.

When the reference identifier associated with the object is a two-dimensional code, controlling the shooting device to obtain a reference identifier associated with each object through shooting includes steps S11 to S19:

S11. Control the shooting device to shoot the reference mark associated with the object to obtain a target image.

S12 adjusts the size of the target image.

In order to remove the noise in the target image and reduce the amount of subsequent calculations, the control device can adjust the size of the target image. Specifically, the control device can use the downsampling method to adjust the size of the target image, or adjust the target image by cropping. The size, that is, the edge area of the target image is cropped, and the adjusted target image is shown in FIG. 4.

S13. Perform filtering processing on the adjusted target image.

In order to further filter out noises to make the image smooth and the edges of the QR code in the target image sharper, the control device can use a filtering algorithm to filter the adjusted target image. The filtering algorithm here includes bilateral filtering or Peak filtering and more.

S14. Binarize the target image after the filtering process.

In order to extract key information (such as a QR code) in the target image, the control device can perform grayscale processing on the filtered target image, and use dynamic thresholds to binarize the target image after grayscale processing. The target image of is shown in Figure 5. Here, binarization refers to setting the gray value of the pixels on the image to 0 or 255, which means that the entire image presents obvious visual effects of only black and white.

S15. Obtain a closed region from the target image after the binarization process.

Find closed edge contours on the binarized target image. Usually, one image includes multiple closed edge contours. The edge points on these edge contours are further fitted into multiple closed regions. Specifically, the control device Polygonal fitting can be performed on each edge contour using a fitting algorithm to obtain multiple closed regions. The fitting algorithm here includes the Ramer-Douglas-Pucker algorithm and so on.

S16. Obtain an effective closed area from the multiple closed areas.

Because there are invalid closed areas in multiple closed areas, multiple closed areas need to be screened to filter out the invalid closed areas to obtain valid closed areas. As shown in Figure 6, the areas in the white box in the figure are invalid closed areas. . Specifically, according to the shape of the two-dimensional code, it can be known that the area where the two-dimensional code is located is close to a square (such as a trapezoid or a rhombus) and has a convex edge. Therefore, a plurality of closed areas are not close to a square, and non-convex edges are closed. The area is determined to be an invalid area, and the invalid area is excluded. Further, an area having an area larger than a first area threshold or an area smaller than a second preset area threshold among a plurality of closed areas may be excluded to obtain a valid closed area.

The first preset area threshold is larger than the second preset area threshold, and the first preset area threshold and the second preset area threshold can be set according to the size of the two-dimensional code.

S17. Perform perspective transformation on the effective closed area.

Generally, the shape of the two-dimensional code is a standard square. Due to the shooting angle of the two-dimensional code, the shape of the two-dimensional code obtained by the image is distorted, so that the shape of the two-dimensional code in the target image is not a standard square. The device can perform a perspective transformation on the effective closed area to correct the two-dimensional code. Further, a Dali algorithm or the like is used to perform a binarization process on the corrected closed area to obtain a perspective-transformed image as shown in FIG. 7.

S18. Determine the validity of the two-dimensional code.

In order to easily identify the two-dimensional code, the edge of the two-dimensional code is usually set as a black frame. Therefore, the control device can determine whether the edge of the closed region after the perspective transformation is a black frame. If not, determine the closed region after the perspective transformation. The included two-dimensional code is invalid two-dimensional code; otherwise, the two-dimensional code included in the closed region after the perspective transformation is determined as a valid two-dimensional code, and step S18 is performed.

S19. Extract the information of the two-dimensional code.

When it is determined that the two-dimensional code included in the closed region after the perspective transformation is a valid two-dimensional code, the closed region after the perspective transformation is rasterized to obtain an image as shown in FIG. 8. According to FIG. 8, there are some white grids. It contains a small part of the black area, and some black grids include a small part of the white area, that is, these grids are unstable, and impurities are present. Therefore, the control device can ignore the pixels around the small grid, and only extract the pixels in the middle of each small grid as shown in FIG. 9. The closed area after extracting the pixels is shown in FIG. 10, and the image in FIG. 10 is a two-dimensional code. Further, the control device can recognize the image shown in FIG. 10 and read the information in the image. Among them, the black box in FIG. 10 indicates 1 and the white box indicates 0. The acquisition steps can be S11-S19, or some of them.

Step S202 includes: obtaining pre-stored reference identification information that matches the reference identification, and obtaining relative pose information of the reference identification and the object according to the reference identification information.

The control device may obtain the relative pose information corresponding to the reference mark according to the reference mark. Specifically, the control device may obtain pre-stored reference mark information that matches the reference mark, and obtain the pre-stored relative position corresponding to the reference mark according to the reference mark information. Posture information.

For example, if the reference identifier is a two-dimensional code, the control device may compare the two-dimensional code obtained by the photographing device with a pre-stored two-dimensional code in the database (that is, the pre-stored reference identification information). If the two-dimensional code matches the pre-stored two-dimensional code, the relative pose information of the reference identifier and the object is obtained according to the pre-stored two-dimensional code.

The matching of the two-dimensional code obtained by the photographing device with the pre-stored two-dimensional code may mean that the value of the two-dimensional code obtained by shooting is the same as or approximately the same as the value of the two-dimensional code stored in advance.

The process of obtaining relative pose information is a cyclic process, that is, after passing through the current object, the control device can obtain reference identification information that matches the reference identification of the next object, and matches according to the reference identification of the next object. The reference position information corresponding to the reference position information, the relative position information is the relative position information between the next object and the reference identifier associated with the next object, until all objects are passed through, and the acquisition of the relative position ends. Steps for Posture Information.

When the reference identifier is a two-dimensional code, step S203 includes: scanning the two-dimensional code, and obtaining relative pose information corresponding to the pre-stored reference identifier from the two-dimensional code.

In order to reduce the pressure on the control device to store information, some information can be stored in the device connected to the QR code. For example, the relative pose information corresponding to the pre-stored reference identifier is stored in the device. Therefore, the control device can scan The dimension code obtains the relative pose information corresponding to the reference identifier that is stored in advance. Specifically, the control device scans the two-dimensional code and obtains the relative pose information corresponding to the reference identifier that is stored in advance from the device connected to the two-dimensional code.

In one embodiment, the object is a ring-shaped object, and the shape of the ring-shaped object may be circular, square, or the like. The reference identifier is set on the associated object. Specifically, the reference identifier may be set on the bottom, upper, left, or right of the associated object.

For example, assuming that the object is a ring (that is, a circular ring-shaped object), the reference identifier is a two-dimensional code, and the reference identifier is set at the bottom of the object, the posture relationship between the object and the reference identifier is shown in FIG. 11.

S203. Determine pose information of the object according to the relative pose information corresponding to the reference identifier.

In the embodiment of the present invention, after acquiring the relative pose information corresponding to the reference identifier, the control device may determine the pose information of the object according to the relative pose information corresponding to the reference identifier.

The pose information of the object includes at least one of rotation information and displacement information. Rotation may refer to a rotation angle of the object relative to the photographing device, and displacement information refers to a translation distance of the object relative to the photographing device.

In one embodiment, the pose information of the reference identifier is determined according to the two-dimensional pixel coordinate points of the reference identifier in the pixel coordinate system, the three-dimensional coordinate points of the camera in the world coordinate system, and internal parameters of the camera. , Determining the pose information of the object according to the pose information of the reference identifier and the relative pose information corresponding to the reference identifier.

The control device may determine the pose information of the object based on the pose information of the reference mark and the relative pose information corresponding to the reference mark. Specifically, the control device may use a pose estimation algorithm and two reference marks in the pixel coordinate system. Dimensional pixel coordinate points, three-dimensional coordinate points of the photographing device in the world coordinate system, and internal parameters of the photographing device determine the pose information of the reference mark, and according to the pose information of the reference mark and the relative pose corresponding to the reference mark The information determines the pose information of the object.

Among them, the pose estimation algorithm may include an N-point perspective algorithm (Perspective-n-Point, PnP). The two-dimensional pixel coordinate points are the characteristic points of the reference mark in the image coordinate system. The world coordinate system is the global coordinate system. The global coordinate system refers to a coordinate system that has been defined in advance.

For example, the two-dimensional pixel coordinate point of the reference identifier in the pixel coordinate system is [uv 1] ^T , and the three-dimensional coordinate point of the shooting device in the world coordinate system is [x _c y _c z _c 1] ^T. The internal parameters of the shooting device include: the focal length f, the number of pixels m _x and m _y per unit distance in the x-axis and y-axis directions, the optical center positions μ ₀ and v ₀ , and the distortion parameter γ between the x-axis and y-axis, The relationship between the two-dimensional pixel coordinate point of the reference identifier in the pixel coordinate system, the three-dimensional coordinate point of the camera in the world coordinate system, and the internal parameters of the camera can be expressed as the following formula 1-3:

Among them, k in formula 3 is a constant, [x _w y _w z _w 1] in formulas 2 and 3 ^T is a point in the world coordinate system of the reference mark, R is a rotation vector of the reference mark, and T is a reference mark. Displacement vector, according to the above formulas 1-3, the pose information of the reference mark can be solved, that is, the reference mark is the world coordinate information, rotation and displacement vector, and then according to the pose information of the reference mark and the corresponding reference object The relative pose information determines the pose information of the object.

It should be noted that, generally, the pose of the reference mark is constant, and the drone is in a moving state. Then the control device can calculate the world coordinate information of the same reference mark in different poses. If the calculated world coordinate information is Are all the same, it is determined that the calculated world coordinate information is correct, otherwise it is determined that the calculated world coordinate information is wrong.

S204. Generate a flight trajectory through each object according to the pose information of the object corresponding to each identity.

S205. Control the drone to traverse each object based on the flight trajectory.

In steps S204 to S205, in order to achieve automatic obstacle crossing, the control device may generate a flight trajectory through each object according to the pose information of the object corresponding to each identity, and control the drone to traverse based on the flight trajectory. The each object.

In the embodiment of the present invention, the control device receives an instruction to traverse at least one object (that is, an obstacle), and the instruction includes an identity identifier of each object in the at least one object; and obtains the reference stored in advance according to the reference identifier associated with the object. Identify the relative pose information corresponding to the identifier, determine the pose information of the object according to the relative pose information corresponding to the reference identifier, and control the drone to pass through the object based on the pose information of the object. It can be seen that this embodiment can automatically plan the flight trajectory of the drone based on the pose information of the object, without the need for a user to manually adjust the flight trajectory of the drone, and more accurately pass through obstacles based on the attitude information of the object. Improve the convenience of crossing obstacles and meet users' needs for automation and intelligence of drones crossing obstacles.

Based on the foregoing description of a flight control method and a drone system, an embodiment of the present invention provides another flight control method. Please refer to FIG. 12, the method may be executed by a control device, where the specific explanation of the control device is as follows Previously mentioned. The method includes:

S301. Obtain a reference identifier.

During the flight of the drone based on the flight trajectory, the control device may control the shooting device to obtain a reference identifier associated with each object by shooting, or control the shooting device to acquire the reference identifier associated with each object by means of infrared scanning.

The reference mark may be a two-dimensional code, a random dot matrix, or other markers. The reference identifier can be set on an object, and one or more reference identifiers can be set on an object.

In one embodiment, a reference identifier is set on an object, for example, only a reference identifier associated with the object is set on each object. When the reference identifier is set on the object associated with the reference identifier, the control device can obtain a reference identifier each time, cross or reach the current object, and then obtain the reference identifier associated with the next object until it passes through or reaches all objects To end obtaining the reference identifier.

In one embodiment, multiple reference identifiers are set on an object. For example, multiple reference identifiers are set on the first object that traverses, and the control device can obtain multiple reference identifiers each time.

S302. Acquire posture information of an object associated with the reference identifier.

The database of the control device includes posture information of an object associated with multiple reference identifiers. The control device may obtain a pre-stored reference identifier matching the reference identifier from the database, and obtain an association with the reference identifier according to the pre-stored reference identifier. Pose information of the object.

Among them, the pose information includes translation information and rotation information. The relaxation information is in the world coordinate system or the camera coordinate system. The translation information may be coordinate information. The rotation information includes rotation angles of multiple axes, for example, a pitch axis, a skew One or more of a roll axis and a roll axis. S303. Determine a flight trajectory according to the pose information of the object.

In the embodiment of the present invention, the reference identifier has a one-to-one correspondence with the object. If at least one reference identifier is obtained, it is determined that the drone needs to cross or reach at least one object, and at least one object may be one or more objects. When only at least one object is included in the at least one object, the control device may determine the flight trajectory of the drone according to the pose information of the object corresponding to the identity; when the at least one object includes multiple objects, the control device may The pose information of the objects determines the relative pose information between each two objects, and the flight trajectory of the drone is determined based on the relative pose information.

S304. Control the drone to fly based on the flight trajectory.

In the embodiment of the present invention, after the flight trajectory is obtained, the control device may control the drone to fly based on the flight trajectory.

The object may refer to a location where the drone needs to reach, for example, the location may refer to a certain building.

In one embodiment, if the object can refer to a location where the drone needs to arrive, step S304 includes: controlling the drone to fly based on the flight trajectory and reach the location where the object is located.

The object may refer to an obstacle, and the shape of the obstacle may be a ring or a solid object.

Step S304 includes controlling the drone to pass through the object based on the flight trajectory, or controlling the drone to fly around each object based on the flight trajectory.

In one embodiment, when the obstacle is a ring-shaped object, controlling the drone to fly based on the flight trajectory includes controlling the drone to pass through the ring-shaped object based on the flight trajectory, such as through the ring-shaped object. The center of the object.

In one embodiment, when the obstacle is a solid object, controlling the drone to fly based on the flight trajectory includes controlling the drone to fly around each object based on the flight trajectory, such as from the upper part of the solid object, Bottom, left, or right bypass flight.

In the embodiment of the present invention, the control device obtains at least one reference identifier, and for each reference identifier, determines the flight trajectory of the drone according to the pose information of the object associated with each identity, and controls the drone based on the flight trajectory flight. It can be seen that this embodiment can automatically plan the drone's flight trajectory based on the pose information of the object, without the need for the user to manually adjust the drone's flight trajectory, improve the convenience of crossing obstacles, and meet the user's requirements for unmanned The need for automation and intelligence of aircraft crossing obstacles. In addition, the flight trajectory can be automatically planned according to the pose information of the object, and the designated position can be automatically reached based on the flight trajectory.

Based on the above description of a flight control method and a drone system, an embodiment of the present invention provides a control device. Referring to FIG. 13, the control device includes: at least one processor 401, such as a CPU; and at least one memory 402. The communication device 403, the sensor 404, and the controller 405, the processor 401, the memory 402, and the communication device 403, the sensor 404, and the controller 405 are connected through a bus 406.

The communication device 403 may be used to receive flight status information of the drone, and image data and the like captured by the shooting device during the drone flight.

The sensor 404 is configured to obtain a reference identifier and the like.

The memory 402 is configured to store at least one program instruction, where the at least one program instruction includes a first program instruction and a second program instruction. The processor 401 may call a first program instruction stored in the memory 402 to perform the following steps:

Optionally, the processor 401 may call a first program instruction stored in the memory 402 to perform the following steps:

For the object corresponding to each identity, according to the reference identifier associated with the object, obtain pre-stored relative pose information corresponding to the reference identifier, where the relative pose information is that the object is associated with the object Relative pose information between the reference identifiers;

Determining the pose information of the object according to the relative pose information corresponding to the reference identifier;

A flight trajectory of the drone crossing the object is generated based on the pose information of the object.

Based on the flight trajectory, a reference identifier associated with each object is obtained by the photographing device.

Determining pose information of the reference mark according to a two-dimensional pixel coordinate point of the reference mark in a pixel coordinate system, a three-dimensional coordinate point of the shooting device in a world coordinate system, and internal parameters of the shooting device;

Determining the pose information of the object according to the pose information of the reference identifier and relative pose information of the reference identifier and the object.

Obtaining pre-stored reference identification information that matches the reference identification;

Obtaining the relative pose information corresponding to the reference identifier stored in advance according to the reference identifier information.

Scan the two-dimensional code to obtain the relative pose information corresponding to the reference identifier stored in advance.

A flight trajectory passing through each object is determined according to pose information of the object corresponding to each identity and the sequence of passing through each object.

The reference identifier is set on an associated object; the object is a ring-shaped object.

In one embodiment, the processor 401 may call a second program instruction stored in the memory 402 to perform the following steps:

Obtaining a reference identifier;

Controlling the drone to fly based on the flight trajectory.

Optionally, the processor 401 may call a second program instruction stored in the memory 402 to perform the following steps:

Controlling the drone to pass through the object based on the flight trajectory, or controlling the drone to fly around the each object based on the flight trajectory.

In the embodiment of the present invention, a computer-readable storage medium is also provided. The computer-readable storage medium stores a computer program, and the computer program is implemented by a processor to implement the present invention, FIG. 2, FIG. 3, and FIG. 12. The flight control method described in the corresponding embodiment can also implement the control device of the corresponding embodiment of the present invention described in FIG. 13, and details are not described herein again.

The computer-readable storage medium may be an internal storage unit of the device according to any one of the foregoing embodiments, such as a hard disk or a memory of the device. The computer-readable storage medium may also be an external storage device of the device, such as a plug-in hard disk, a Smart Media Card (SMC), and a Secure Digital (SD) card equipped on the device. , Flash card (Flash card) and so on. Further, the computer-readable storage medium may further include both an internal storage unit of the device and an external storage device. The computer-readable storage medium is used to store the computer program and other programs and data required by the control device. The computer-readable storage medium may also be used to temporarily store data that has been or will be output.

Those of ordinary skill in the art can understand that all or part of the processes in the method of the foregoing embodiment can be implemented by using a computer program to instruct related hardware. The program can be stored in a computer-readable storage medium. When executed, the processes of the embodiments of the methods described above may be included. The storage medium may be a magnetic disk, an optical disk, a read-only memory (Read-Only Memory, ROM), or a random access memory (Random, Access Memory, RAM).

What has been disclosed above are only the preferred embodiments of the present invention, and of course, the scope of the rights of the present invention cannot be limited by this. Therefore, equivalent changes made according to the claims of the present invention still fall within the scope of the present invention.

Claims

A flight control method is applied to a control device for controlling the flight of an unmanned aerial vehicle, and is characterized in that it includes:

Receiving an instruction to traverse at least one object, the instruction including an identity of each object in the at least one object;

Determining a flight trajectory through each object according to pose information of the object corresponding to each identity;

Controlling the drone to traverse each of the objects based on the flight trajectory.
The method according to claim 1, wherein determining the flight trajectory through each object according to the pose information of the object corresponding to each identity identifier comprises:

For the object corresponding to each identity, according to the reference identifier associated with the object, the relative pose information of the reference identifier that is stored in advance is obtained, and the relative pose information is the object associated with the object. Relative pose information between the reference identifiers;

Determining the pose information of the object according to the relative pose information corresponding to the reference identifier;

A flight trajectory of the drone crossing the object is generated based on the pose information of the object.
The method according to claim 2, wherein the drone comprises a photographing device, and the method further comprises:

Based on the flight trajectory, a reference identifier associated with each object is obtained by the photographing device.
The method according to claim 3, further comprising:

Determining pose information of the reference mark according to a two-dimensional pixel coordinate point of the reference mark in a pixel coordinate system, a three-dimensional coordinate point of the shooting device in a world coordinate system, and internal parameters of the shooting device;

The determining the pose information of the object according to the relative pose information corresponding to the reference identifier includes:

Determining the pose information of the object according to the pose information of the reference identifier and relative pose information between the reference identifier and an object associated with the reference identifier.
The method according to claim 2, wherein the acquiring the relative pose information corresponding to the reference identifier stored in advance according to the reference identifier associated with the object comprises:

Obtaining pre-stored reference identification information that matches the reference identification;

Obtaining the relative pose information corresponding to the reference identifier stored in advance according to the reference identifier information.
The method according to claim 2, wherein the reference identifier is a two-dimensional code, and the obtaining the relative pose information corresponding to the reference identifier stored in advance according to the reference identifier associated with the object comprises:

Scan the two-dimensional code to obtain the relative pose information corresponding to the reference identifier stored in advance.
The method according to any one of claims 1-6, wherein the instruction further carries a sequence of traversing each object, and the traversing is determined according to pose information of the object corresponding to each identity identifier. The trajectory of each object, including:

A flight trajectory passing through each object is determined according to pose information of the object corresponding to each identity and the sequence of passing through each object.
The method according to claim 7, wherein the reference identifier is set on an associated object; the object is a ring-shaped object.
A flight control method is applied to a control device for controlling the flight of an unmanned aerial vehicle, and is characterized in that it includes:

Obtaining a reference identifier;

Acquiring pose information of an object associated with the reference identifier;

Determining a flight trajectory according to the pose information of the object;

Controlling the drone to fly based on the flight trajectory.
The method according to claim 9, wherein the controlling the drone to fly based on the flight trajectory comprises:

Controlling the drone to pass through the object based on the flight trajectory, or controlling the drone to fly around the object based on the flight trajectory.
A control device used for controlling the flight of a drone, which is characterized by including a memory and a processor;

The memory is used to store program instructions;

The processor executes program instructions stored in the memory. When the program instructions are executed, the processor is configured to perform the following steps:

Receiving an instruction to traverse at least one object, the instruction including an identity of each object in the at least one object;

Determining a flight trajectory through each object according to pose information of the object corresponding to each identity;

Controlling the drone to traverse each of the objects based on the flight trajectory.
The device according to claim 11, wherein:

The processor is configured to obtain, for each object, the relative pose information corresponding to the reference identifier stored in advance according to the reference identifier associated with the object, where the relative pose information is the object and all The relative pose information between the reference identifiers associated with the object; and determining the pose information of the object according to the relative pose information corresponding to the reference identifier.
The device according to claim 12, wherein the drone includes a photographing device,

The processor is configured to obtain a reference identifier associated with each object through the photographing device based on the flight trajectory.
The device according to claim 13, characterized in that:

The processor is further configured to determine the reference according to a two-dimensional pixel coordinate point of the reference identifier in a pixel coordinate system, a three-dimensional coordinate point of the photographing device in a world coordinate system, and an internal parameter of the photographing device. The pose information of the identifier; determining the pose information of the object according to the pose information of the reference identifier and relative pose information of the reference identifier and the object.
The device according to claim 12, characterized in that:

The processor is configured to obtain pre-stored reference identifier information that matches the reference identifier; and acquire, according to the reference identifier information, relative pose information corresponding to the pre-stored reference identifier.
The device according to claim 12, wherein the reference identifier is a two-dimensional code,

The processor is configured to scan the two-dimensional code to obtain the relative pose information corresponding to the reference identifier stored in advance.
The device according to any one of claims 11-16, wherein:

The processor is configured to determine a flight trajectory that passes through each object according to pose information of the object corresponding to each identity and the sequence that passes through each object.
The device according to claim 17, wherein the reference identifier is set on an associated object; the object is a ring-shaped object.
A control device for controlling the flight of a drone, characterized in that it includes a memory and a processor;

The memory is used to store program instructions;

The processor executes program instructions stored in the memory. When the program instructions are executed, the processor is configured to perform the following steps:

Obtaining a reference identifier;

Acquiring pose information of an object associated with the reference identifier;

Determining a flight trajectory according to the pose information of the object;

Controlling the drone to fly based on the flight trajectory.
The device according to claim 19, wherein:

The processor is configured to control the drone to pass through each object based on the flight trajectory, or control the drone to fly around each object based on the flight trajectory.
A drone system, which is characterized by comprising a drone and a control device;

The control device is configured to execute the flight control method according to any one of claims 1 to 10 to plan a flight trajectory for the drone;

The drone is configured to fly according to the flight trajectory to pass through at least one object.
A computer-readable storage medium, characterized in that the computer storage medium stores a computer program, wherein the computer program includes program instructions, and the program instructions, when executed by a processor, cause the processor to execute the program according to claim 1 The steps of the flight control method according to any one of 10 to 10.
A computer program product, characterized in that the computer program product includes a non-transitory computer-readable storage medium storing a computer program, and the computer program is operable to cause a computer to implement any one of claims 1 to 10 The steps of the flight control method are described.